Your IP : 216.73.217.13
U
�Ad�e � @ s: d dl Z d dlZd dlZd dlZd dlZd dlmZ d dlmZm Z m
Z
m Z m Z d dl
mZ d dlmZ e�e�Zddd� d fd
e ji fd d d� d fd
�ZdZdZdZdZdZdZdZ dZ dZ dZ eeeee e e e fZ dZ!G dd� de"�Z#G dd � d �Z$G d d � d e j%d �Z&G d d!� d!e&�Z'G d"d#� d#e&�Z(d&d$d%�Z)dS )'� N)�log)�net�sources�subp�
url_helper�util)�BrokenMetadata)�ec2�network_configc C s | S �N� ��xr r �E/usr/lib/python3/dist-packages/cloudinit/sources/helpers/openstack.py�<lambda> � r � �meta_js�authorized_keysc C s | S r r r
r r r r r )zetc/network/interfaceszmeta.jszroot/.ssh/authorized_keys))zlocal-hostnameZhostnameF)� instance-idZuuidT�latestz
2012-08-10z
2013-04-04z
2013-10-17z
2015-10-15z
2016-06-30z
2016-10-06z
2017-02-22z
2018-08-27)
NZbgpovsZbridgeZ cascadingZdvsZethernetZhw_vebZhypervZovsZphyZtapZ vhostuserZvifc @ s e Zd ZdS )� NonReadableN)�__name__�
__module__� __qualname__r r r r r Q s r c @ s4 e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd S ) � SourceMixinc C s> | j s
d S | j �di �}|�� D ]\}}||kr | S q d S )Nzblock-device-mapping)Z ec2_metadata�get�items)�self�nameZbdmZent_name�devicer r r �_ec2_name_to_deviceV s
z SourceMixin._ec2_name_to_devicec C s$ d}| j dkrd}t�| j�|��S )NZ public_keys� � public-keys)�versionr Znormalize_pubkey_data�metadatar )r r r r r �get_public_ssh_keys_ s
z SourceMixin.get_public_ssh_keysc C sR d }z2d| }|dkr d| }t �|�}|r4|d }W n tjk
rL Y nX |S )NzLABEL=%sZswapzTYPE=%sr )r Zfind_devs_withr ZProcessExecutionError)r r r ZcriteriaZ dev_entriesr r r �_os_name_to_devicee s
z SourceMixin._os_name_to_devicec C sV |sd S |� d�sd| }tj�|�r*|S | �tj�|��}|rRt�d||� |S d S )N�/z/dev/%sz Remapped device name %s => %s)�
startswith�os�path�existsZ
_remap_device�basename�LOG�debug)r r Zremappedr r r �_validate_device_namer s
z!SourceMixin._validate_device_namec C s� |sd S |g}|dkr"|� dd� |dkr4|�d� d }t�d|� |D ] }| �|�}| �|�}|rH qjqH|s�t�d|� |D ] }| �|�}| �|�}|r~ q�q~|s�d S t�d||� |S d S )N�rootr Zamiz(Using ec2 style lookup to find device %sz.Using openstack style lookup to find device %szMapped %s to device %s)�insert�appendr. r/ r! r0 r' )r r �namesr �nr r r �device_name_to_device� s2
z!SourceMixin.device_name_to_deviceN)r r r r! r& r' r0 r6 r r r r r U s
r c @ sh e Zd Zdd� Zejdd� �Zejddd��Zejdd � �Zejd
d � �Z d d
� Z
ddd�Z dd� Z dS )�
BaseReaderc C s
|| _ d S r )� base_path�r r8 r r r �__init__� s zBaseReader.__init__c G s d S r r �r �base�add_onsr r r �
_path_join� s zBaseReader._path_joinFc C s d S r r �r r+ �decoder r r �
_path_read� s zBaseReader._path_readc C s d S r r �r r r r �_fetch_available_versions� s z$BaseReader._fetch_available_versionsc C s d S r r rB r r r �_read_ec2_metadata� s z BaseReader._read_ec2_metadatac
C s� z | � � }W n6 tk
rB } zt�d| j|� g }W 5 d }~X Y nX dd� ttt��D �}t}|D ]}||krpqb|} qzqbt�d||� |S )Nz4Unable to read openstack versions from %s due to: %sc S s g | ]}|�qS r r )�.0�vr r r �
<listcomp>� s z4BaseReader._find_working_version.<locals>.<listcomp>z Selected version '%s' from %s) rC � Exceptionr. r/ r8 �reversed�list� OS_VERSIONS� OS_LATEST)r Zversions_available�eZ supportedZselected_versionZpotential_versionr r r �_find_working_version� s, ��z BaseReader._find_working_versionc C s\ |� dd��d�}|�d�}dd� |D �}|s:td| ��| j| jdf|�� }| j||d�S ) NZ content_pathr r( c S s g | ]}t |�r|�qS r )�len)rE �pr r r rG � s z1BaseReader._read_content_path.<locals>.<listcomp>z!Item %s has no valid content path� openstack�r@ )r �lstrip�splitr r> r8 rA )r �itemr@ r+ Z path_piecesZ valid_piecesr r r �_read_content_path� s
z BaseReader._read_content_pathc s� t jtjtttfd�� � �fdd�}ddd�}|��� �}|�� D ]�\}\}}}�� �j
|�}d}d}z�� |�}W nD t k
r� } z&|s�t
�d || � nt
�d
|| � W 5 d} ~ X Y nX d }|r�|s�td | ��|�r"|�r"z ||�}W n6 tk
�r } ztd
|| f �| �W 5 d} ~ X Y nX |rB|||< qB|d }
d|
k�r�|
d } zt�| �|
d< W n6 ttfk
�r� } ztd| �| �W 5 d} ~ X Y nX i } |
�dg �}
|
D ]f}d|k�r��q�|d }z��|�| |< W n6 tk
�r
} ztd|| f �| �W 5 d} ~ X Y nX �q�| |d< |
�dd�}|�rxz�j|d d�}||d< W n2 t k
�rv } ztd| �| �W 5 d} ~ X Y nX z|
d d |d< W n tk
�r� Y nX ��� |d< tD ]@\}}}|�r�||
k�r�td| ��||
k�r�|
�|�|
|< �q�|S )z�Reads a version 2 formatted location.
Return a dict with metadata, userdata, ec2-metadata, dsmode,
network_config, files and version (2).
If not a valid location, raise a NonReadable exception.
)Z
root_typesc s� i }�� d| d�dtjf|d< �� d| d�ddd� f|d < �� d| d
�d� f|d < �� d| d �d� f|d
< �� d| d�d� f|d< |S )NrQ zmeta_data.jsonTr% Z user_dataFc S s | S r r r
r r r r � r z7BaseReader.read_v2.<locals>.datafiles.<locals>.<lambda>�userdatazvendor_data.jsonZ
vendordatazvendor_data2.jsonZ vendordata2znetwork_data.jsonZ networkdata)r> r � load_json)r$ �files�Zload_json_anytyper r r � datafiles� s, � � � � �z%BaseReader.read_v2.<locals>.datafilesr � )rW r$ NFz*Failed reading optional path %s due to: %sz+Failed reading mandatory path %s due to: %sTzMissing mandatory path: %s� Failed to process path %s: %sr% � random_seedz.Badly formatted metadata random_seed entry: %srY r+ z#Failed to read provided file %s: %sr
rR z(Failed to read network configuration: %s�meta�dsmodez ec2-metadatazNo '%s' entry in metadata)� functools�partialr rX �dictrJ �strrN r r> r8 rA �IOErrorr. r/ r rH r �base64Z b64decode�
ValueError� TypeErrorr rV �KeyErrorrD �
KEY_COPIES)r r[ �results�datar r+ Zrequired�
translator�foundrM r% r^ rY Zmetadata_filesrU Znet_item�contentZ
target_keyZ
source_keyZ is_requiredr rZ r �read_v2� s� �!� � �
��
��
�� ��
zBaseReader.read_v2N)F)F)
r r r r: �abc�abstractmethodr> rA rC rD rN rV rp r r r r r7 � s
r7 )� metaclassc sF e Zd Z� fdd�Zdd� Zddd�Zdd � Zd
d � Zd d
� Z� Z S )�ConfigDriveReaderc s t t| ��|� d | _d S r )�superrt r: � _versionsr9 �� __class__r r r: f s zConfigDriveReader.__init__c G s |gt |� }tjj|� S r )rJ r* r+ �join)r r< r= Z
componentsr r r r> j s z ConfigDriveReader._path_joinFc C s t j||d�S )NrR )r Z load_filer? r r r rA n s z ConfigDriveReader._path_readc s@ | j d kr:| �| jd�� � fdd�t�� �D �}t|�| _ | j S )NrQ c s$ g | ] }t j�t j�� ��r|�qS r )r* r+ �isdirry �rE �d�r+ r r rG t s �z?ConfigDriveReader._fetch_available_versions.<locals>.<listcomp>)rv r> r8 r* �listdir�sorted)r rn r r} r rC q s
�
z+ConfigDriveReader._fetch_available_versionsc
C sn | � | jddd�}tj�|�s"i S zt�| �|��W S tk
rh } zt d||f �|�W 5 d }~X Y nX d S )Nr r zmeta-data.jsonr] )
r> r8 r* r+ r, r rX rA rH r )r r+ rM r r r rD | s �
��z$ConfigDriveReader._read_ec2_metadatac C s� i }t �� D ]&}| �| j|�}tj�|�r |||< q t|�dkrNtd| j ��i }t � � D ]�\}\}}}||k�r|| }z| �
|�}W n0 t k
r� } zt d| �| �W 5 d} ~ X Y nX z||�||< W n4 t
k
r� } zt d|| f �| �W 5 d} ~ X Y nX qZt�|�||< qZ|d }
|d } | �d|
�}
|
�rN|
�� } d d
� | D �|d< d | k�rd| d |d < d |d
�}
d| k�r�| d |
d< | �dd�|
d< i |
d< |
S )z�Reads a version 1 formatted location.
Return a dict with metadata, userdata, dsmode, files and version (1).
If not a valid path, raise a NonReadable exception.
r z%s: no files foundzFailed to read: %sNr] r r r# c S s" g | ]}t |�r|�d �s|�qS )�#)rO r) )rE �liner r r rG � s
�z-ConfigDriveReader.read_v1.<locals>.<listcomp>r r" )r$ r% r` z user-datar rW rY )�FILES_V1�keysr> r8 r* r+ r, rO r r rA re r rH �copy�deepcopyr �
splitlines)r rn r r+ Zmd�keyrm �default�contentsrM Zkeydatar �linesrk r r r �read_v1� sT
�� �
�
zConfigDriveReader.read_v1)F)
r r r r: r> rA rC rD r� �
__classcell__r r rw r rt e s
rt c s@ e Zd Zd� fdd� Zdd� Zddd �Zd
d � Zd d
� Z� ZS )�MetadataReaderN� c s4 t t| ��|� || _t|�| _t|�| _d | _d S r ) ru r� r: � ssl_details�float�timeout�int�retriesrv )r Zbase_urlr� r� r� rw r r r: � s
zMetadataReader.__init__c C sb | j d k r| j S g }| �| jd�}| j|dd�}|�� D ] }|�� }|sJq8|�|� q8|| _ | j S )NrQ TrR )rv r> r8 rA r� �stripr3 )r rn Z version_pathro r� r r r rC � s
z(MetadataReader._fetch_available_versionsFc C s: dd� }t j|| j| j| j|d�}|r0|j�� S |jS d S )Nc S s: z t |j�}|dkrW dS W n ttfk
r4 Y nX dS )Ni� FT)r� �coderh rg )Z
_request_args�causer� r r r �should_retry_cb� s
z2MetadataReader._path_read.<locals>.should_retry_cb)r� r� r� Z exception_cb)r Zreadurlr� r� r� r� r@ )r r+ r@ r� Zresponser r r rA � s
�
zMetadataReader._path_readc G s t j|f|�� S r )r Z combine_urlr; r r r r> s zMetadataReader._path_joinc C s t j| j| j| jd�S )N)r� r� r� )r Zget_instance_metadatar� r� r� rB r r r rD s
�z!MetadataReader._read_ec2_metadata)Nr� r� )F) r r r r: rC rA r> rD r� r r rw r r� � s
r� c s* | dkr dS ddddddgddd d
d d d
ddddg d��| � dg �}| � dg �}| � dg �}g }i �d}d}g }|D �]4�g } t�fdd���� D ��}
d�kr��d |
d< d} �� d�rԈ� d��� } | ��d < |
� d�| �d �d d �} �fd d �|D �D �](}
t�fd d�|
�� D ��}|
d d k�r@|�dd!i� n�|
d d"k�r^|�dd#i� nz|
d d$k�r�|�d|
d i� nX|
d d%k�r�|�d&|
� d'�d(�� n2|
d d)k�r�|
�d*d+i� |�d,|
� d'�d(�� |
d d-k�r�|
�d*d.i� |
d d/k�r
d.|d/< |
d d0k�r d.|d0< | �|� �q|
�d| i� �d d1k�r�i }| �r\| |d< ��� D ]4\}}|d2k�r|�qdn|�d3��rd|�||i� �qd|| }|d47 }|�|
d5d6t��d2 �f� |
�||d7�� || d< n��d d8k�rfd9�d: �d; f }|
�|�d; �d< d=�� |�|
d:d6�d: f� |�|
dd>�d; �d: f� | ��d< |d?�� n.�d t k�r�t
� d@�d � |
�dA| dB�� |�|
� | �| d < qtdCd � |D �}|�s�|�r�|dk�r�t �
� }��� D ]:\}}|� d��r��q�|� dD�|k�r�||dD |d< �q�|D ]D}|� d�}|�s@tdE| ��||k�rVtdF| ��|| |d< �q |D ]L\}
}� }t|ttf��r�� �fdGd �|D �|
|< n� �| d |
|< �qjt �� }|�r�|D ],}
|
d |k�r�||
d |
d< dH|
d< �q�|D ]"} | }
|
�ddIi� |�|
� �q�d4|dJ�S )Ka) Return a dictionary of network_config by parsing provided
OpenStack ConfigDrive NetworkData json format
OpenStack network_data.json provides a 3 element dictionary
- "links" (links are network devices, physical or virtual)
- "networks" (networks are ip network configurations for one or more
links)
- services (non-ip services, like dns)
networks and links are combined via network items referencing specific
links via a 'link_id' which maps to a links 'id' field.
To convert this format to network_config yaml, we first iterate over the
links and then walk the network list to determine if any of the networks
utilize the current link; if so we generate a subnet entry for the device
We also need to map network_data.json fields to network_config fields. For
example, the network_data links 'id' field is equivalent to network_config
'name' field for devices. We apply more of this mapping to the various
link types that we encounter.
There are additional fields that are populated in the network_data.json
from OpenStack that are not relevant to network_config yaml, so we
enumerate a dictionary of valid keys for network_yaml and apply filtering
to drop these superflous keys from the network_config yaml.
Nr �type� mac_address�subnets�paramsZmtu�addressZnetmaskZ broadcastZmetricZgatewayZ pointopointZscopeZdns_nameserversZ
dns_searchZroutes)�physical�subnet�links�networks�serviceszbond%dr c 3 s& | ] \}}|� d kr||fV qdS )r� Nr �rE �krF ��
valid_keysr r � <genexpr>P s z#convert_net_json.<locals>.<genexpr>Zethernet_mac_address�id)r �macr� r� c s g | ]}|d � d kr|�qS )�linkr� r )rE r5 )r� r r rG f s z$convert_net_json.<locals>.<listcomp>c 3 s& | ] \}}|� d kr||fV qdS )r� Nr r� r� r r r� g s Z ipv4_dhcpZdhcp4� ipv6_dhcpZdhcp6)Z
ipv6_slaaczipv6_dhcpv6-stateless�ipv6_dhcpv6-stateful)�ipv4�staticr� Z
ip_address)r� r� )�ipv6�static6z accept-raFr� )r� r� Tr� r� )�bondZ
bond_linksr� r" Zbond_interfacesz%s)r� r )Zvlanz%s.%sZ vlan_link�vlan_idZvlan_mac_address)r r� r� z%%s.%s)r� r z9Unknown network_data link type (%s); treating as physicalr� )r� r� c S s& g | ] }|� d �dkrd|kr|�qS )r� r� r )r r{ r r r rG � s r� z#No mac_address or name entry for %sz"Unable to find a system nic for %sc s g | ]}� �| d �qS )r r )rE �target)�fmt� link_id_infor r rG � s Z
infinibandZ
nameserver)r$ �config)r rc r �lower�updater3 r) r� r� �KNOWN_PHYSICAL_TYPESr. Zwarningr Zget_interfaces_by_macrg �
isinstancerJ �tupleZget_ib_hwaddrs_by_interface) Z network_jsonZ
known_macsr� r� r� Z link_updatesZ
bond_name_fmtZ bond_numberr� r� ZcfgZ
link_mac_addrZcurinfoZnetworkr� r� r� rF Z link_namer Z
need_namesZ_link_id�infor| r� r� ZtargetsZib_known_hwaddrs�servicer )r� r� r� r� r �convert_net_json s2 � ��
�
� �����
��
����
�
� r� )NN)*rq rf r� ra r* Z cloudinitr Zloggingr r r r r Zcloudinit.sourcesr Zcloudinit.sources.helpersr Z getLoggerr r. rX r� rj rL Z OS_FOLSOMZ
OS_GRIZZLYZ OS_HAVANAZ
OS_LIBERTYZ
OS_NEWTON_ONEZ
OS_NEWTON_TWOZOS_OCATAZOS_ROCKYrK r� re r r �ABCMetar7 rt r� r� r r r r �<module> sP
�� L Eo: