Your IP : 216.73.217.13
U
���b�( � @ s8 d dl Z d dlZd dlZd dlZd dlZd dlmZ d dlZd dlZd dl m
Z
m Z m Z m
Z
mZmZmZ d dlmZ d dlZd dlmZmZ d dlZd dlmZ d dlmZ d dlm Z d dl Zd d l m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z( d d
l)m*Z* ej+ej,fZ-dd
d�Z.dd� Z/dddgZ0G dd� de �Z1dS )� N)�datetime)�ImporterHelper�SoSTimeoutError�bold�sos_get_command_output�TIMEOUT_DEFAULT�listdir�
is_executable)�rmtree)�ThreadPoolExecutor� TimeoutError)�_sos)� __version__)� SoSComponent)
�Report�Section�Command�
CopiedFile� CreatedFile�Alert�Note�PlainTextReport�
JSONReport�
HTMLReport)�
SoSCleanerF�, c C s� g }| }|rt | �d }nd}|D ]:}t |�t |� t |� dkrP|�|� |}|| | }q"|t |� d � |kr�|d t |� � }|�|� |S )N� � �H )�len�append)Z
first_line�items�indent�sep�lines�line�newline�item� r( �5/usr/lib/python3/dist-packages/sos/report/__init__.py� _format_list) s
r* c C s t �d�| �d�S )z� This function will format --since arg to append 0s if enduser
didn't. It's used in the _get_parser.
This will also be a good place to add human readable and relative
date parsing (like '2 days ago') in the future z{:<014s}z %Y%m%d%H%M%S)r �strptime�format)�dater( r( r) �
_format_since; s r. �auto�always�neverc 5 s� e Zd ZdZdZdZdddddddddg g dddg g dg dddddd g g g ddddddg deeg dddg ddddddddddd
�4Z� fd d �Ze d
d� �Z
e dd� �Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd�d#d$�Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Zd/d0� Zd1d2� Z e d3d4� �Z d�d5d6�Z d7d8� Z d9d:� Z!d;d<� Z"d=d>� Z#d?d@� Z$dAdB� Z%d�dDdE�Z&dFdG� Z'dHdI� Z(dJdK� Z)dLdM� Z*dNdO� Z+dPdQ� Z,dRdS� Z-dTdU� Z.dVdW� Z/dXdY� Z0dZd[� Z1d�d\d]�Z2d^d_� Z3d`da� Z4dbdc� Z5ddde� Z6dfdg� Z7dhdi� Z8djdk� Z9dldm� Z:dndo� Z;dpdq� Z<drds� Z=dtdu� Z>dvdw� Z?dxdy� Z@dzd{� ZAd|d}� ZBd~d� ZCd�d�� ZDd�d�� ZEd�d�� ZF� ZGS )�� SoSReportzaRun a set of commands and file collections and save them to a report for
future analysis
z.Collect files and command output in an archiveTFr r/ N� � /etc/sos/cleaner/default_mapping)4�
alloptions�all_logs�build�case_id�chroot�clean�container_runtime�keep_binary_files�desc�domains�disable_parsersZdry_run�
estimate_only� experimental�enable_plugins�keywords� keyword_file�plugopts�label� list_plugins� list_presets�
list_profiles�log_size�map_file�
skip_commands�
skip_files� skip_plugins�
namespaces� no_report� no_env_vars� no_postproc� no_update�note� only_plugins�preset�plugin_timeout� cmd_timeout�profiles�since�verify�allow_system_changes� usernames�upload�
upload_urlZupload_directoryZ upload_userZ upload_passZ
upload_methodZupload_no_ssl_verifyZupload_protocol�
add_preset�
del_presetc s� t t| ��|||� g | _g | _g | _t� | _d | _|| _ d| _
d | _ i | _ | �
� | �� | j�� | _| jj�d�| _| jj | jd kr�| j�| jj �| _ | j s�tj�d| jj � | j� � | _ d| j_ | j s�| j� � | _ | j� | j j� | � | j�| _t | j jd��r*| j jj!dk�r*t"d� | �#� | �$� d }| jj
| _
| jj
�rPd
}n | j�%� �rn| j
t&j'k�rnd }| j(�)d | j
|f � | jj*t+k�r�| j(�,d
| jj* � t-�.� | j/�0� | �1d� | �2� | �3� | �4� d S )N�/�reportrV zUnknown preset: '%s'
T� verbosityr z~
WARNING: It is not recommended to set verbosity via the preset as it might have
unforseen consequences for your report logs.
�default�cmdline�policyzset sysroot to '%s' (%s)zinvalid chroot mode: %s� )5�superr2 �__init__�loaded_plugins�skipped_plugins� all_options�set�env_vars�archiveZ_args�sysrootrV �estimated_plugsizes� print_header�
_set_debugrg Zis_rootZ_is_root�manifestZ
components� add_section� report_md�opts� arg_defaults� find_preset�sys�stderr�writeZ probe_presetrH �mergeZapply_options_from_cmdline�hasattrrd �printZ_setup_logging�_set_directoriesZ in_container�osr# �soslog�debugr9 � chroot_modes�error�logging�shutdown�
tempfile_utilr: �_exit�_check_container_runtime�_get_namespaces�_get_hardware_devices)�self�parser�argsrf �msg�� __class__r( r) rj � sZ �
zSoSReport.__init__c
C s, |� dd�}|jddddddd � |jd
dd dd d � |jd
ddd tdd� |jdddddd � |jddddd� |jddddd d d � |jd d d d � |jd!d"tdd#d$d%� |jd&dd'd(� |jd)dd*d(� |jd+dd,dd-d � |jd.d/d0d1td2g d3� |jd4d5d6d0d7td8g d3� |jd9d:dd;d<d� |jd=d>dd?dd@d � |jdAddBd(� |jdCddDddEd � |jdFddGtdHdIdJ� |jdKd dLd � |jdMdNd0dOtdPg d3� |jdQddRddSd � |jdTddUddVd � |jdWddXddYdZ� |jd[tdd#d\d%� |jd]d^d0d_td`g d3� |jdadtdbd dc� |jddd ded � |jdfd dgd � |jdhdidjd0dktg dldJ� |jdmg d0dndodp� |jdqg d0drdsdp� |jdtdduddvd � |jdwddxddyd � |jdzddd{d|� |jd}d d~d � |jdd d�d � |jd�d d�d � |jd�d d�d � |jd�d d d�d�gd�d�� |jd�ddd�d�� |jd�d d d�d�d�gd�d�� |�� }|jd�tdd�d�� |jd�tdd�d�� |� d�d��}|jd�d�d�d�ddd�d�� |jd�d�g d0d�d�� |jd�d0g d�d�d�� |jd�d0g d�d�d�� |jd�d d�d�d�� |jd�ddd�d�d�� |jd�d�d�d�d�� |jd�ddd�d�dp� |jd�d�g d0d�d�� d S )�NzReport Optionsz.These options control how report collects dataz-az --alloptions�
store_truer5 Fz%enable all options for loaded plugins)�action�destre �helpz
--all-logsr6 z-collect all available logs regardless of sizez--sinceZstorerZ zbEscapes archived files older than date. This will also affect --all-logs. Format: YYYYMMDD[HHMMSS])r� r� re �typer� z--buildr7 z;preserve the temporary directory and do not package resultsz --case-idr8 zspecify case identifier)r� r� r� z-cz--chrootr9 r/ zHchroot executed commands to SYSROOT [auto, always, never] (default=auto)z--container-runtimezLDefault container runtime to use for collections. 'auto' for policy control.)re r� z--descz
--descriptionr z Description for a new preset)r� r� re r� z --dry-runz#Run plugins but do not collect data)r� r� z--estimate-onlyzzApproximate disk space requirements for a real sos run; disables --clean and --collect, sets --threads=1 and --no-postprocz--experimentalrA zenable experimental pluginsz-ez--enable-plugins�extendrB zenable these plugins)r� r� r� r� re z-kz--plugin-optionz
--plugoptsrE z7plugin options in plugname.option=value format (see -l)z--labelz--namerF z"specify an additional report labelz-lz--list-pluginsrG z)list plugins and available plugin optionsz--list-presetsz#display a list of available presetsz--list-profilesrI zBdisplay a list of available profiles and plugins that they includez
--log-sizerJ r3 z)limit the size of collected logs (in MiB))r� r� r� re r� z --namespaceszDlimit number of namespaces to collect output for - 0 means unlimitedz-nz--skip-pluginsrN zdisable these pluginsz --no-reportrP z disable plaintext/HTML reportingz
--no-env-varsrQ z$Do not collect environment variablesz
--no-postprocrR zDisable all post-processing)re r� r� r� z--notez Behaviour notes for new presetz-oz--only-pluginsrU zenable these plugins onlyz--presetzA preset identifier)r� r� r� re z--plugin-timeoutz set a timeout for all pluginsz
--cmd-timeoutz%set a command timeout for all pluginsz-pz --profilez
--profilesrY z)enable plugins used by the given profilesz--skip-commandsrL z do not execute these commands)re r� r� r� z --skip-filesrM zdo not collect these filesz--verifyr[ z+perform data verification during collectionz--allow-system-changesr\ zJRun commands even if they can change the system (e.g. load kernel modules)z--uploadz+Upload archive to a policy-default location)r� re r� z --upload-urlz&Upload the archive to specified serverz--upload-directoryz$Specify upload directory for archivez
--upload-userz'Username to authenticate to server withz
--upload-passz'Password to authenticate to server withz--upload-methodZputZpostz HTTP method to use for uploading)re �choicesr� z--upload-no-ssl-verifyz'Disable SSL verification for upload url)re r� r� z--upload-protocolZhttpsZftpZsftpz$Manually specify the upload protocol� --add-presetz#Add a new named command line preset)r� r� r� z --del-presetz$Delete the named command line presetzCleaner/Masking Optionsz7These options control how data obfuscation is performedz--cleanz --cleanerz--maskr: z Obfuscate sensitive information)r� re r� r� z --domainsr> z$Additional domain names to obfuscatez--disable-parsersr? zCDisable specific parsers, so that those elements are not obfuscated)r� re r� r� z
--keywordsrC z List of keywords to obfuscatez--keyword-filerD z&Provide a file a keywords to obfuscate)re r� r� z --no-updaterS z%Do not update the default cleaner mapz
--map-filerK r4 z;Provide a previously generated mapping file for obfuscation)r� re r� z--keep-binary-filesr< zGKeep unprocessable binary files in the archive instead of removing themz --usernamesr] z List of usernames to obfuscate)Zadd_argument_group� add_argumentr. �str�intZ add_mutually_exclusive_group)�clsr� Z
report_grpZ
preset_grpZ cleaner_grpr( r( r) �add_parser_options� s� �
� � � �
�
��
��� �
�
�
�
�� �
��
� � � � �
�
���
�
�
� � �
�������
�
�
��
�
� �
�� ���
�z SoSReport.add_parser_optionsc C s� |� d� |�d� |�d� |�d� |jdd�}|�d� |�d� |jd d�}|�d
td �td �f � |�d
td� � dddd�}d}|D ] }|jd�||| �dd� q�|�|� d S )NzSoS Report Detailed Helpz�The report command is the most common use case for SoS, and aims to collect relevant diagnostic and troubleshooting data to assist with issue analysis without actively performing that analysis on the system while it is in use.z�Additionally, sos report archives can be used for ongoing inspection for pre-emptive issue monitoring, such as that done by the Insights project.z�The typical result of an execution of 'sos report' is a tarball that contains troubleshooting command output, copies of config files, and copies of relevant sections of the host filesystem. Root privileges are required for collections.z How Collections Are Determined)�titlez�SoS report performs it's collections by way of 'plugins' that individually specify what files to copy and what commands to run. Plugins typically map to specific components or software packages.aE Plugins may specify different collections on different distributions, and some plugins may only be for specific distributions. Distributions are represented within SoS by 'policies' and may influence how other SoS commands or options function. For examplepolicies can alter where the --upload option defaults to or functions.zSee Alsoz>For information on available options for report, see %s and %szsos report --helpzman sos-reportz.The following %s sections may be of interest:
zsos helpz'Information on the plugin design of sosz!Information on a specific $pluginz+How sos operates on different distributions)zreport.pluginszreport.plugins.$pluginZpoliciesr z {:<36}{}F)r& )Z set_titleZadd_textrv r r, )r� �sectionZpsecZssecZ
help_linesZhelplnZlnr( r( r) � display_helpy sH
��� �� �����zSoSReport.display_helpc C s t dtdtf � � d S )Nz
%s
zsosreport (version %s))r� �_r �r� r( r( r) rs � s zSoSReport.print_headerc C s( | � � | �� d�| �� | �� d�| _d S )N)�blockZfibre)Zstorage�networkZnamespaced_network)�_get_block_devs�_get_fibre_devs�_get_network_devs� _get_network_namespace_devices�devicesr� r( r( r) r� � s ��z SoSReport._get_hardware_devicesc C s� | j jdkr�| j j�� }|dkr6i | j_| j�d� n�| jjsTd| }| j�|� nl|| jj�� kr�d� dd� | jj�� D ��}t
d||f ��n.| jj| | jjd < | j�d
| jjd j � d S ) a[ Check the loaded container runtimes, and the policy default runtime
(if set), against any requested --container-runtime value. This can be
useful for systems that have multiple runtimes, such as RHCOS, but do
not have a clearly defined 'default' (or one that is determined based
entirely on configuration).
r/ )Znone�offZdiabledz0Disabled all container runtimes per user option.zZWARNING: No container runtimes are active, ignoring option to set default runtime to '%s'
r c s s | ]}|d kr|V qdS )re Nr( ��.0�pr( r( r) � <genexpr>� s �z5SoSReport._check_container_runtime.<locals>.<genexpr>zSCannot use container runtime '%s': no such runtime detected. Available runtimes: %sre z%Set default container runtime to '%s'N) rx r; �lowerrg Zruntimesr� �info�warn�keys�join� Exception�name)r� Zcrunr� Zvalidr( r( r) r� � s. �� � ��z"SoSReport._check_container_runtimec
C s� zDg }ddddg}|D ](}t j�d| �r|�t�d| �� q|W S tk
r~ } z | j�d| � g W Y �S d}~X Y nX dS ) z�Enumerate a list of fibrechannel devices on this system so that
plugins can iterate over them
These devices are used by add_device_cmd() in the Plugin class.
Zfc_hostZ fc_transportZfc_remote_portsZ fc_vportsz
/sys/class/%sz/sys/class/%s/*z#Could not get fibre device list: %sN)r� �path�isdirr� �globr� r� r� )r� ZdevsZdevdirsZdevdir�errr( r( r) r� � s �zSoSReport._get_fibre_devsc
s� z�dd� t �d�D �}td�}g � |d dkr^|d �� D ]"}|�� d �dd �}� �|� q:� fd
d�|D �}tt|�t|� �}|W S t k
r� } z | j
� d | � g W Y �S d }~X Y nX d S )
z�Enumerate a list of block devices on this system so that plugins
can iterate over them
These devices are used by add_device_cmd() in the Plugin class.
c S s g | ] }d | �qS )z/dev/%sr( )r� �dr( r( r) �
<listcomp>� s z-SoSReport._get_block_devs.<locals>.<listcomp>z
/sys/blockzlosetup --all --noheadings�statusr �output�:r c s" g | ]}|� d �r|� kr|�qS )Zloop��
startswith�r� �dev�Zreal_loop_devicesr( r) r� s
�z#Could not get block device list: %sN) r� r r �
splitlines�split�replacer �listrn r� r� r� )r� Z device_listZ loop_devicesZloop_devZ loop_deviceZghost_loop_devsZdev_listr� r( r� r) r� � s zSoSReport._get_block_devsc C s d| � � i| _d S )Nr� )�_get_network_namespacesrO r� r( r( r) r�
s �zSoSReport._get_namespacesc
C s� g g g g d�}zPt d| jjd�r.|�| �� � |d s\| j�d� |�| �� � | �� |d< W n2 t k
r� } z| j�
d| � W 5 d}~X Y nX |S ) a Helper to encapsulate network devices probed by sos. Rather than
probing lists of distinct device types like we do for storage, we can
do some introspection of device enumeration where a single interface
may have multiple device types. E.G an 'ethernet' device could also be
a bond, and that is pertinent information for device iteration.
:returns: A collection of enumerated devices sorted by device type
:rtype: ``dict`` with keys being device types
)�ethernet�bridgeZteam�bondZnmcli)rq r� zZNetwork devices not enumerated by nmcli. Will attempt to manually compile list of devices.r� z'Could not enumerate network devices: %sN) r rx rq �update�_get_nmcli_devsr� r� �
_get_eth_devs�_get_bridge_devsr� r� )r� �_devsr� r( r( r) r� s" ��"zSoSReport._get_network_devsc C s* i }| j d D ]}d| �|�i||< q|S )zqEnumerate the network devices that exist within each network
namespace that exists on the system
r� r� )rO r� )r� Z_nmdevsZnmspr( r( r) r� . s �
z(SoSReport._get_network_namespace_devicesc
C s� i }z�t d�}|d dkrp|d �� dd� D ]@}|�� \}}||krR|g||< n|| �|� |d �|� q.tt|d ��|d< W n2 tk
r� } z| j�d| � W 5 d}~X Y nX |S ) z�Use nmcli, if available, to enumerate network devices. From this
output, manually grok together lists of devices.
z nmcli --fields DEVICE,TYPE devr� r r� rh Nr� z!Could not parse nmcli devices: %s) r r� r� r r� rn r� r� r� )r� r� Z_ndevsr� ZdnameZdtyper� r( r( r) r� 9 s "zSoSReport._get_nmcli_devsc
s� dg� g }|s,� fdd�t d| jj�D �}n|zDd| }t|�}|d dkrn|d �� D ]}|� krV|�|� qVW n6 tk
r� } z| j�d ||f � W 5 d
}~X Y nX |d d� |D �d d� |D �d
�S )a8 Enumerate a list of ethernet network devices so that plugins can
reliably iterate over the same set of devices without doing piecemeal
discovery.
These devices are used by `add_device_cmd()` when `devices` includes
"ethernet" or "network".
:param namespace: Inspect this existing network namespace, if
provided
:type namespace: ``str``
:returns: All valid ethernet devices found, potentially within a
given namespace
:rtype: ``list``
Zbonding_mastersc s g | ]}|� kr|�qS r( r( r� �Z filt_devsr( r) r� ` s �z+SoSReport._get_eth_devs.<locals>.<listcomp>z/sys/class/netz"ip netns exec %s ls /sys/class/netr� r r� z6Could not determine network namespace '%s' devices: %sNc S s g | ]}|� d �r|�qS )r� r� )r� Zbdr( r( r) r� s s
c S s g | ]}|� d �r|�qS )�tunr� )r� Ztdr( r( r) r� t s
)r� r� r� ) r rx rq r r� r r� r� r� )r� � namespaceZ _eth_devsZ_nscmdZ_nsoutZ_nsethr� r( r� r) r� M s.
� �� �zSoSReport._get_eth_devsc C s� g }zt ddd�}W n2 tk
rF } z| j�d| � W 5 d}~X Y nX |d dkr�|d �� d d� D ]R}z|�|�� d � W qh tk
r� } z| j�d
||f � W 5 d}~X Y qhX qh|S ) z�Enumerate a list of bridge devices so that plugins can reliably
iterate over the same set of bridges.
These devices are used by `add_device_cmd()` when `devices` includes
"bridge" or "network".
z
brctl show� ��timeoutz&Unable to enumerate bridge devices: %sNr� r r� rh z)Could not parse device from line '%s': %s)r r� r� r� r� r r� r� )r� Z_bridgesZ_boutr� Z_bliner( r( r) r� w s " ��zSoSReport._get_bridge_devsc C s\ g }t d�}|d dkrX|d �� D ]2}|�� s$|dd� �� rBq$|�|�d�d � q$|S )z�Enumerate a list of network namespaces on this system so that
plugins can iterate over them
Note that stderr is not collected, so no handling of error lines.
zip netnsr� r r� Nrh r )r r� �isspacer � partition)r� Zout_nsZip_netnsr% r( r( r) r� � s z!SoSReport._get_network_namespacesc C s4 | j | j| j| j| j| j| j| jj| j| j | j
d� S )N) �cmddir�logdir�rptdir�tmpdirr� rg rq rd Z cmdlineoptsr� rO ) r� r� r� r� r� rg rq rx rd r� rO r� r( r( r) � get_commons� s �zSoSReport.get_commonsc C s
| j �� S �N)r� �newr� r( r( r) �
get_temp_file� s zSoSReport.get_temp_filec C s4 | j �| jd� | j �| jd� | j �| jd� d S )Ni� )rp �makedirsr� r� r� r� r( r( r) �_make_archive_paths� s z SoSReport._make_archive_pathsc C s d| _ d| _d| _d S )NZ sos_commands�sos_logs� sos_reports)r� r� r� r� r( r( r) r� � s zSoSReport._set_directoriesc C s" | j jr| jt_d| _nd| _d S )NTF)rx r� �
_exceptionr{ �
excepthook�
raise_pluginsr� r( r( r) rt � s zSoSReport._set_debugc C sL t td�stj�� s$t�| ||� n$tj| ||dtjd� t� t �
� dS )z* Wrap exception in debugger if not in tty Zps1� )�limit�fileN) r r{ r| �isatty�__excepthook__� traceback�print_exception�stdoutr� �pdbZpm)�etypeZeval_Zetracer( r( r) r� � s �zSoSReport._exceptionc C sV | j s | jr>t�� \}}}tj|||tjd� t� t� |� |rR|rR| �
||� d S )N)r� ) r� Z exit_processr{ �exc_infor� r� r� r� r� Z post_mortem�_log_plugin_exception)r� �plugname�funcr� �val�tbr( r( r) �handle_exception� s
zSoSReport.handle_exceptionc C sT t | dd �r(| jj| jtj�dd�d� t | dd �rP| jj| jtj�dd�d� d S )N� sos_log_filer� �sos.log�r� �sos_ui_log_file�ui.log)�getattrrp �add_filer� r� r� r� r� r� r( r( r) �
_add_sos_logs� s
�
�zSoSReport._add_sos_logsc sT � j j}t� j j�sdS t|d�s&dS |r<� �|�� �s<dS t� fdd�|jD ��S )NTrY Fc s g | ]}|� j jk�qS r( )rx rY r� r� r( r) r� � s z,SoSReport._is_in_profile.<locals>.<listcomp>)rx rU r rY r �_is_not_specifiedr� �any)r� � plugin_classrU r( r� r) �_is_in_profile� s
zSoSReport._is_in_profilec C s || j jkS r� )rx rN �r� � plugin_namer( r( r) � _is_skipped� s zSoSReport._is_skippedc C s* || � � ��� o(|| jjko(|| jjkS r� )r� Z
check_enabledrx rB rU �r� r Z pluginClassr( r( r) � _is_inactive� s
�
�zSoSReport._is_inactivec C s* || � � ��� o(|| jjko(|| jjkS r� )r� Zdefault_enabledrx rB rU r r( r( r) �_is_not_default� s
�
�zSoSReport._is_not_defaultc C s | j jo|| j jkS r� )rx rU r r( r( r) r � s
�zSoSReport._is_not_specified�unknownc C s" | j �|�� || �� �|f� d S r� )rl r r� r� )r� r �reasonr( r( r) �_skip s
�zSoSReport._skipc C s | j �|�� || �� �f� d S r� )rk r r� r� )r� r r( r( r) �_load
s
�zSoSReport._loadc C s� t jjj}tt jj�}|�� }g | _t� | _t | j
j�}| j j }g }| j
j
r\|�t jjj� t|| �}| j j}t| j
j�} |D �]�}
tj�|
�\} } �zz|| |�}
t |
�s�W q�| j �|
�}||| j
j
d��s| j�td�|
� | j
jdk�r| �|td�� W q�| j�| � | � |�}|�s4| �|td�� W q�| � | ��rT| �|td�� W q�| � | |��rv| �|td�� W q�| � | |��r�| �|td�� W q�t |d ��r�| j�!|j� | �o�|}| �"| ��r�|�r�| �|td
�� W q�|jD ]}|| k�r�| �#|� �q�| �$|� W q� t%k
�rV } z$| j�td �|
|f � | �&� W 5 d }~X Y q�X q�t | �dk�r�| j�'td �d
�(| � � | �)� | �*d� d S )N)rA z%plugin %s does not validate, skippingr zdoes not validateZexcludedZskippedZinactiveZoptionalrY z
not specifiedz(plugin %s does not install, skipping: %sz+Unknown or inactive profile(s) provided: %sr rh )+�sosrc �plugins�
import_pluginr Z get_modules� plugin_namesrn rY r rx rg Zvalid_subclassesrA r ZExperimentalPlugin�tuple�validate_pluginr� r� r� �splitextZ match_pluginr� �warningr� rd r r r
r r
r r� r �remover r� r� r� r� rI r� )r� r �helperr Zusing_profilesZpolicy_classesZ
extra_classesZvalid_plugin_classesr Zremaining_profiles�plugZplugbaseZextZplugin_classesr Z
in_profileZdefault_profile�i�er( r( r) � load_plugins s�
�
�
� �zSoSReport.load_pluginsc C s< | j jr8| jD ](\}}|j�� D ]}t|jkr d|_q qd S )NT)rx r5 rk �options�values�boolZval_type�value)r� r� r �optr( r( r) �_set_all_options_ s
zSoSReport._set_all_optionsc C s | j j�ri }| j jD ]�}z|�d�\}}W n tk
rD d}Y nX t|t�r�|�� }|dkrfd}n0|dkrtd}n"z t|�}W n tk
r� Y nX z|�d�\}}W n tk
r� |}d}Y nX z || W n tk
r� i ||< Y nX ||| |< q| j D ]�\}}||k�r|| D ]�}||j
k�rL| j � d||f � | �
d� z6|j
| �|| | � | j �d ||j
| f � W n: tk
�r� } z| j � |� | �
d� W 5 d }~X Y nX �q ||= �q|�� D ]}| j � d
| � �q�|�� �r| j � d � d S ) N�=T)�on�enable�enabled�trueZyes)r� �disableZdisabledZfalseZnoF�.z#no such option "%s" for plugin (%s)rh zSet %s plugin option to %szGWARNING: unable to set option for disabled or non-existing plugin (%s).r )rx rE r� �
ValueError�
isinstancer� r� r� �KeyErrorrk r r� r� r� Z set_valuer� r� r� )r� rx r$ r� r r� r� r( r( r) �
_set_tunablesf sh
�
��
�
zSoSReport._set_tunablesc C s� dd l }|�| jj| jj�D ]6}|�d�d }|| jkr | j�d| � | � d� q | jj
D ] }|| jkr\| j� d| � q\d S )Nr r, z=a non-existing plugin (%s) was specified in the command line.rh z+Requested to skip non-existing plugin '%s'.) � itertools�chainrx rU rB r� r r� �fatalr� rN r )r� r1 �pluginr r( r( r) �_check_for_unknown_plugins� s
�
�
�z$SoSReport._check_for_unknown_pluginsc C s2 | j D ]&\}}|jD ]}| j�|j| � qqd S r� )rk r rm r )r� r r4 r$ r( r( r) �_set_plugin_options� s
z SoSReport._set_plugin_optionsc C s� d}g }| j jdkr.|d| j j g7 }d| j _| j jsH|dg7 }d| j _| j jsb|dg7 }d| j _| j jr||dg7 }d| j _| j jr�|d g7 }d| j _|r�|d
d �|� 7 }n|d 7 }|d
7 }|S )Nz
Estimate-only mode enabledrh z--threads=%s overriden to 1z--build enabledTz--no-postproc enabledz--clean disabledFz--upload* options disabledz", which overrides some options:
z
r, z
)rx �threadsr7 rR r: r^ r� )r� r� Zext_msgr( r( r) �_set_estimate_only� s,
z SoSReport._set_estimate_onlyc C s^ | j �d� t| j�r8| j �dt| j�t| j�f � n| j �dt| j� � | j �d� d S )Nr z %d profiles, %d pluginsz %d profiles)�ui_logr� r rk rY r� r( r( r) � _report_profiles_and_plugins� s
�z&SoSReport._report_profiles_and_pluginsc
C s� | j s | js | j�td�� d S | j rl| j�td�� | j�d� | j D ] \}}| j�d||�� f � qHn| j�td�� | j�d� | jr�| j�td�� | j�d� | jD ]$\}}}| j�d|||�� f � q�| j�d� | j�rB| j�td�� | j d d
� � }|D ]�}|| }|j
}|j
d k�rf|d k�rH| j j �pDt
}n |d
k�rb| j j�p^t
}nt
}|jdk�r|| j j }| j�d|j||jf � �q| j�d� | j�td�� | jD ]�}|jdk�r�|j
|jk�rސq�t|j
t��r|j
dk�r�d} nd} n|j
} | d k�rd } | j�d|jd |j | |jf � �q�n| j�td�� | j�d� t| j�}
|
�� td|
dd�} | D ]} | j�d| � �q�| �� d S )Nzno valid plugins foundz,The following plugins are currently enabled:r z %-20s %szNo plugin enabled.z-The following plugins are currently disabled:z %-20s %-14s %sz4The following options are available for ALL plugins:r rh ���r� � cmd-timeout�postprocz %-25s %-15s %sz+The following plugin options are available:)r� r= r< Tr' r� r, z No plugin options available.z
Profiles: �r" � %s)rk rl r� r3 r� r9 r� Zget_descriptionrm Zget_default_plugin_optsr# rx rW r rX r� rR r= re r. r" r4 r� rY �sortr* r: )
r� r� r Z plugclassr Z _defaultsZ_optr$ r� ZtmpoptrY r$ r% r( r( r) rG � s~
� �
�
zSoSReport.list_pluginsc C s� | j s| j�td�� d S | j�td�� | j�d� dd� }t| j �}|�� |D ]`}g }| jD ]$\}}||�rb||j krb|� |� qbt
d| |dd�}|D ]}| j�d | � q�qT| � � d S )
Nzno valid profiles foundz%The following profiles are available:r c S s
t | d�S )NrY )r )�cr( r( r) � _has_prof, s z*SoSReport.list_profiles.<locals>._has_profz%-15s Tr> r? ) rY r� r3 r� r9 r� r� r@ rk r r* r: ) r� rB rY Zprofiler r� r4 r$ r% r( r( r) rI % s"
zSoSReport.list_profilesc C s� | j js | j�td�� d S | j�td�� | j�d� | j j�� D ]�}|sNqD| j �|�}| j�dd|j f � | j�dd|j
f � |j r�| j�dd|j f � | j j
dkr�|j �� }d d
}t||d d d
�}|D ]}| j�|� q�| j�d� qDd S )Nzno valid presets foundz$The following presets are available:r z%14s %szname:z description:znote:r z%14s zoptions:Tr )r" r# )rg Zpresetsr� r3 r� r9 r� r� rz r� r= rT rx rd �to_argsr* )r� rV r� Z options_strr$ r% r( r( r) rH ; s(
zSoSReport.list_presetsc
C s� | j }|�|�r$| j�d| � dS |p.| jj}|p:| jj}z|j|||| jd� W n8 tk
r� } z| j�d| � W Y �dS d}~X Y nX | j �
d�}| j d|� | j |d d� }| j� d |d
� |�f � d S ) z�Add a new command line preset for the current options with the
specified name.
:param name: the name of the new preset
:returns: True on success or False otherwise
z"A preset named '%s' already existsF)r� r= rT rx zCould not add preset: %sNr� r r� z"Added preset '%s' with options %s
r T)
rg rz r9 r� rx r= rT r` r� rf �indexr� r� )r� r� r= rT rg r Z arg_indexr� r( r( r) r` S s"
�zSoSReport.add_presetc
C s� | j }|�|�s$| j�d| � dS z|j|d� W n< tk
rp } z | j�t|�d � W Y �dS d}~X Y nX | j�d| � dS )z�Delete a named command line preset.
:param name: the name of the preset to delete
:returns: True on success or False otherwise
zPreset '%s' not foundF�r� �
NzDeleted preset '%s'
T)rg rz r9 r� ra r� r� r� )r� r� rg r r( r( r) ra p s
zSoSReport.del_presetc
C s� | j �� }| jjr || �� 7 }| jjr4| j�|� np|td�7 }z t |� W nV t
k
rv | j� d� | � d� Y n. t
k
r� } z| � d|� W 5 d }~X Y nX d S )Nz,Press ENTER to continue, or CTRL-C to quit.
zExiting on user cancel� rh )rg Zget_msgrx r@ r8 �batchr9 r� r� �input�KeyboardInterruptr� r� r� )r� r� r r( r( r) rH � s
zSoSReport.batchc C sd t �� }d}d| }tj�| j|�}| j�d|||f � | j�d| � | jj d| |dd� d S )Nz!caught exception in plugin methodz%s-plugin-errors.txtz %s "%s.%s()"zwriting traceback to %s�%s
�a)�mode)
r� �
format_excr� r� r� r� r� r� rp �
add_string)r� r4 �methodZtracer� Zplugin_err_logZlogpathr( r( r) r� � s z SoSReport._log_plugin_exceptionc
C s� | j �� z&| j�td�� | �� | �� W d S ttfk
r� } z<|j t
krpt d� t d|j � t d� nt d| � � W 5 d }~X Y nN t
k
r� } z0| j�d� | j�d� t�� | j�|� W 5 d }~X Y nX | �d� d S )Nz Setting up archive ...r z %s while setting up archivez Error setting up archive: %sz) Unexpected exception setting up archive:rh )rg Zpre_workr9 r� r� Z
setup_archiver� �OSError�IOError�errno�fatal_fs_errorsr� �strerrorr� r� r� � print_excr� �r� r r( r( r) �prework� s&
zSoSReport.preworkc C sj | j �td�� | jD �]L\}}z�| jj�|� |�t| jj|�� t �
� }|j � d|� | j
|_
|�� |�� | j�|j� | jjr�|�� t �
� }|j � d|� |j � d|| � W q tk
r� � Y q ttfk
�rB } zP|jtk�r&| j �d� | j �d|j � | j �d� | � d� | � |d� W 5 d }~X Y q t k
�rb | � |d� Y qX qd S ) Nz Setting up plugins ...Z setup_startZ setup_endZ
setup_timer z %s while setting up pluginsrh �setup) r9 r� r� rk rw r rv Zset_plugin_manifestr r �nowru � add_fieldrp Zadd_default_collectionsrY ro r� Z _env_varsrx r[ Z setup_verifyrJ rQ rR rS rT r� rU r� r� r� )r� r� r �start�endr r( r( r) rY � s> � �
zSoSReport.setupc C s, g }|� dt � | jjd�|�dd� dS )zWFetch version information from all plugins and store in the report
version filez
sosreport: %srF z version.txt)�contentr� N)r r rp rO r� )r� Zversionsr( r( r) �version� s
�zSoSReport.versionc C s� | j �td�� | j �d� d}g | _g | _| jD ] }|d7 }| j�||d f� q2z^g }t| jj ��}|�
| j t | j��}W 5 Q R X |D ]}|s�| j
�d| � q�| j �d� W n* tk
r� | j �d� t�d� Y nX d S )Nz! Running plugins. Please wait ...r r rh z!Unexpected plugin task result: %sz
Exiting on user cancel
)r9 r� r� �pluglist�
running_plugsrk r r rx r7 �map�_collect_pluginr� r� r� rJ r� r� r� )r� Z plugruncountr ZresultsZexecutor�resr( r( r) �collect� s.
�� zSoSReport.collectc C s� t d���}z|| j|d d d }|�| j|�}|jp:d}t�� }|j�d|� |j |d� t�� }|j�d|� |j�d|| � W n~ t
k
�r d|d }| j � d | � | j
�|� | j�|d � | j|d d d �� |jd
d � |j�� Y nX W 5 Q R X | jj�r�dd lm} | | j�� �}
td
d� |
� d�D ��| j |d < t � | j�� �D ]J} t j �!| j�� | �} t j �"| ��r�t j �#| ��s�t$| � n
t �%| � �qhd
S )zaWraps the collect_plugin() method so we can apply a timeout
against the plugin as a wholerh r NZ
start_timer� Zend_timeZrun_timezPlugin %s timed outz
%s
T)�wait��Pathc S s g | ]}|� � j�qS r( ��lstat�st_size�r� �fr( r( r) r� s z-SoSReport._collect_plugin.<locals>.<listcomp>�**/*)&r rk Zsubmit�collect_pluginr� r rZ ru r[ �resultr r9 r� r� r� ra r Zset_timeout_hitr� Z_threads�clearrx r@ �pathlibrh rp Z get_tmp_dir�sumr� rr r� r r� r� r� �islinkr
�unlink) r� r4 ZpoolZ_plug�tr� r\ r] r� rh � tmpdir_pathrm r( r( r) rc � s>
�
zSoSReport._collect_pluginc
C s� z*|\}}| j |d d }| j�|� W n tk
r@ Y dS X t| j �}dd||f |dd�dd� | jD �� f }| �|� z�|�� z| j� |� W n t
k
r� Y nX z| j� |� W n t
k
r� Y nX d }t| j�t | j j
�k�r | j�r d
ddd�d d� | jD �� f }| j�s2| j�s2d }|�rB| �|� W n� tk
�rZ Y n� ttfk
�r� } zJ|jtk�r�| j�d
|j � | j�d| j � t�d� | �|d� W 5 d }~X Y n" tk
�r� | �|d� Y nX d S )Nrh Fz Starting %-5s %-15s %sz%d/%dz
[Running: %s]r c s s | ]
}|V qd S r� r( r� r( r( r) r� / s z+SoSReport.collect_plugin.<locals>.<genexpr>r z Finishing plugins %-12s %sc s s | ]
}|V qd S r� r( r� r( r( r) r� D s z
Finished running pluginsz!
%s while collecting plugin dataz& Data collected still available at %s
re )rk ra r r� r r� � ui_progressre r` r r- r� rx r7 r rQ rR rS rT r9 r� rU r� r� r� r� ) r� r4 �countr� r Znumplugs� status_liner� r r( r( r) ro $ sb
�
�� ��
zSoSReport.collect_pluginc C sN | j jdkr$| j js$d|�d� }nd| }| j jsJtj�|� tj�� d S )Nr z
%s�Z rK ) rx rd rH �ljust�quietr{ r� r} �flush)r� rz r( r( r) rx Y s zSoSReport.ui_progressc C s@ | j s
d S d�dd� dd� | j D �D ��d }| j�|d� d S )NrF c S s g | ]\}}d ||f �qS )z%s=%sr( )r� r� r� r( r( r) r� e s z.SoSReport.collect_env_vars.<locals>.<listcomp>c S s0 g | ](}t j�|�d k r|dt j�|� f�qS )Nz%s)r� �environ�get)r� r� r( r( r) r� g s �Z environment)ro r� rp rO )r� �envr( r( r) �collect_env_varsb s
��zSoSReport.collect_env_varsc
C s� t � }| jD ]�\}}t|d�}|jD ]}|�t|�� q$|jrN|�t|j�� |jD ]"}|�t |d d|d d�� qT|j
D ]4}|�t |d dt j
�d| �� d |d �d
�� q~|jD ]&\}}} |�t|t j
�d|�d�� q�|�|� q td d ftd
dftddff}
|
D ]�\} } }
zT| �� }| |��� }|�dd��� }|�|� |�� | jj |t j
�d| �d� W nl t t fk
�r� } zH|j t k�r�| j!�"d� | j!�"d|j#|
f � | j!�"d� | �$d� W 5 d }~X Y nX �qd S )NrE Zsrcpathz..Zdstpath)r� �href�cmdr r� r� )r� Z return_coder� zsos.txt�textzsos.jsonZJSONzsos.htmlZHTMLzutf-8r� r� r� r z %s while writing %s reportrh )%r rk r Zalerts�addr Z custom_textr Z copied_filesr Zexecuted_commandsr r� r� r� r� Z copy_stringsr r r r r� Zunicode�encode�decoder} r~ rp r rQ rR rS rT r9 r� rU r� )r� rc r� r r� Zalertrm r� r^ ZtagsZ
formatlistZclass_�filenameZtype_�fdr� r r( r( r) �generate_reportsl s\
�
��
�
�
� � zSoSReport.generate_reportsc C s� | j D ]�\}}z(|�d�r$|�� n| j�d| � W q ttfk
r� } zN|jtkr�| j �
d� | j �
d|j � | j �
d� | � d� | �
|d� W 5 d }~X Y q tk
r� | �
|d� Y qX qd S )Nr= z Skipping postproc for plugin %sr z% %s while post-processing plugin datarh )rk Z
get_optionr= r� r� rQ rR rS rT r9 r� rU r� r� r� )r� r� r r r( r( r) r= � s$
�
�
zSoSReport.postprocc C sn |sdS z@d}t |d�}t�|�}|�|�}|s2q>|�|� q"|�� W n tk
rd | �� Y nX |�� S )NFi �rb) �open�hashlibr� �readr� �closer� r� Z hexdigest)r� rp � hash_nameZ hash_sizeZ
archive_fpZdigestZhashdatar( r( r) �_create_checksum� s
zSoSReport._create_checksumc C s0 t |d | d�}|r$|�|d � |�� d S )Nr, �wrF )r� r} r� )r� rp r� �checksum�fpr( r( r) �_write_checksum� s zSoSReport._write_checksumc
s d }d }d }d}| j jr�zJ| j| j| j| j | jd�}td|d�}|�| j� � � |�
� \}}d}W n2 t k
r� } zt t
d| �� W 5 d }~X Y nX | �� | jd k r�| j�| j j� |�r tj�| j| jj�} |jtj�| dd�dd� |jtj�| dd �d d� |jtj�| d
d �d d� | j j�rd d
lm� d dlm}
|
| j�} tdd� | � d�D ��| j d< � t| j � � ��} | j � d| � t!| j �"� dd� dd�d d� }
d�� fdd�|
D ��}| j � d| � | j � d � | j � d � | j � d � | j � d � | j j#�s t�$d �}| j j%�s.t t
d �� z�z&|�rD| j�&|� | j�'| j j�}W n� t(t)fk
�r� } zFt d � t t
d!|j*| j� � f �� t d � |j+t,k�r�| �-d"� W 5 d }~X Y n. t k
�r� | j j.�rނ n
Y W �dS Y nX W 5 t�$|� X n�| j j/�s| j j0�r | j �1d#� | j� � }tj�2|�}z4tj�| j|�}|�rX|�3|�}t�4||� |}W n, t(t)fk
�r� t t
d$| �� Y dS X d }| j j#�s|�s�t d%� �nNz| j�5� }| �6||�}W n$ t k
�r� t t
d&�� Y dS X z| �7|||� W n* t(t)fk
�r4 t t
d'| �� Y nX tj�| jtj�2|��}|�rf|�3|�8d(d)��}t�9|�}|d* | }|d* | }zt�4||� |}W n, t(t)fk
�r� t t
d+| �� Y dS X zt�4||� W n* t(t)fk
�r t t
d,| �� Y nX | j j#�s(| jj:|||||d-� n| jj:||||d-� | j j;�sP| j j<�r�| j j#�s�z | j�=|� | j � t
d.�� W n4 t k
�r� } z| j �>d/| � W 5 d }~X Y nX nd0}| j �>|� t?�@� | jA�r�| jA�� | j�r�tj�B| j��r�tC| j� dS )1NF)rg r� �sys_tmpr ru T)Zin_place� hook_commonsz%ERROR: Unable to obfuscate report: %sr� r� )Z
short_namer r� z
manifest.jsonr ��get_human_readablerg c S s g | ]}|� � j�qS r( ri rl r( r( r) r� s z(SoSReport.final_work.<locals>.<listcomp>rn Zsos_logs_reportszPEstimated disk space requirement for whole uncompressed sos report directory: %sc S s | d S )Nrh r( )�xr( r( r) �<lambda> � z&SoSReport.final_work.<locals>.<lambda>)�key�reverse� z, c 3 s& | ] \}}d |� |dd�f V qdS )z%s: %sr )Z precisionNr( )r� r� �vr� r( r) r� s ��z'SoSReport.final_work.<locals>.<genexpr>zFive biggest plugins: %sr z>Please note the estimation is relevant to the current options.z�Be aware that the real disk space requirements might be different. A rule of thumb is to reserve at least double the estimation.�? z Creating compressed archive...z %s while finalizing archive %srh zQ
Unable to encrypt when using --build. Encryption is only available for archives.zError moving directory: %sz Creating archive tarball failed.z:Error generating archive checksum after archive creation.
z#Error writing checksum for file: %sz.tarz-obfuscated.tarr, z Error moving archive file: %sz Error moving checksum file: %s)rK z Uploaded archive successfullyzUpload attempt failed: %szEUnable to upload archive when using --build as no archive is created.)Drx r: rg r� r� ru r Zset_target_pathrp Zget_archive_path�executer� r� r� r Zadd_final_manifest_dataZcompression_typer� r� r� �_nameZobfuscate_filer@ �
sos.utilitiesr� rr rh rs r� rr r! r9 r� �sortedr! r7 �umaskr} Zrename_archive_root�finalizerQ rR rU rS rT r� r� Z encrypt_passZ encrypt_keyr� �basenameZobfuscate_string�renameZget_preferred_hash_namer� r� r� �statZdisplay_resultsr^ r_ Zupload_archiver� r� r� r� r� r
)r� rp Z directoryrK Zdo_cleanr� ZcleanerZ_pathsr� Z_dirrh rw Z_sumZ
bigpluginsZbp_outZ old_umaskr Zdir_nameZ final_dirr� r� �
final_nameZ archivestatZ archive_hashZ
final_hashr� r( r� r) �
final_work� s. � "
���
�
�
���
��
�
�
� �
$
zSoSReport.final_workc C s | j s| j�td�� dS dS )Nz no valid plugins were enabledFT)rk r� r� r� r� r( r( r) �verify_plugins� s zSoSReport.verify_pluginsc C s� | j �d| j� | j �d| jr&| jjnd� | j �d| jj� | j �d� | j �
� D ]\}}| j j �||� qT| j �d| jj � | j �d| jj � | j �d� d S )
zgAdd 'global' data to the manifest, that is any information that is
not plugin-specific
rq rV ZunsetrY r� Zenabled_pluginsZdisabled_pluginsr N)
rw r[ rq rV r� Zadd_listrx rY rv r� r! rB rN )r� r� r# r( r( r) �add_manifest_data� s � zSoSReport.add_manifest_datac C sz d}| j �|tdd�| j�f � | jj�� }dtd| jjd�|�f }| j �|� | j �dtdd�| j�� �f � d S )Nz[%s:%s] executing 'sos %s'rY r z'[%s:%s] using '%s' preset defaults (%s)z![%s:%s] effective options now: %s) r� r� �__name__r� rf rV rx rC r� )r� r� Z preset_argsr( r( r) �_merge_preset_options� s � �z SoSReport._merge_preset_optionsc
C s� �z6| j �| �� � | �� | �� | �� | �� | �� | �� | j j
rX| �
� t �| j j rl| � � t �| j j
r�| �
� t �| j jr�| �| j j�W S | j jr�| �| j j�W S | �� s�W dS | �� | �� | �� | �� | �� | j js�| �� | j j�s| �� | j j�s | �� n | j � d� | � � | � � W S t k
�rt | j j!�rV� t"�#dd �dk�sp| �$� Y n| t%k
�r� | j �&d� | �$� | �'d� Y nJ t k
�r� } z*t"�#dd �dk�s�| �$� t(�)|j*� W 5 d }~X Y nX | �'d� d S )NFz)Skipping postprocessing of collected dataZ
SOS_TEST_LOGSZkeepz
Exiting on user cancelrG rh )+rg Z set_commonsr� r r% r� r0 r5 r6 rx rG �
SystemExitrI rH r` ra r� r� rH rX rY re rQ r� rP r� rR r= r9 r� r_ r� rQ r� r� �getenvZcleanuprJ r� r� r{ �exit�coderW r( r( r) r� � sf
zSoSReport.execute)N)NN)r )r r )Hr� �
__module__� __qualname__�__doc__r= Z
root_requiredr ry rj � classmethodr� r� rs r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� rt � staticmethodr� r� r r r
r r
r r r r r% r0 r5 r6 r8 r: rG rI rH r` ra rH r� rX rY r_ re rc ro rx r� r� r= r� r� r� r� r� r� r� �
__classcell__r( r( r� r) r2 G s� �7 ?
4
7
*
O;
K
,5
7 Gr2 )Fr )2r{ r� r� rS r� r r� Zsos.report.pluginsr r� r r r r r r r Zshutilr
r� Zconcurrent.futuresr r r� r
r� r Z
sos.componentr Z sos.policiesZsos.report.reportingr r r r r r r r r r Z sos.cleanerr ZENOSPCZEROFSrT r* r. r� r2 r( r( r( r) �<module> s. $ 0