Sindbad~EG File Manager
3
���]� � @ s� d Z ddlZddlZddlZddljZddlm Z ddl
m Z ddl m
Z
mZmZmZmZmZmZmZmZmZmZmZmZ dZG d d
� d
ejj �Z dd d
�Z
dS )z>html2text: Turn HTML into equivalent Markdown-structured text.� N)�wrap)�config)
�dumb_css_parser�
element_style� escape_md�escape_md_section�google_fixed_width_font�google_has_height�google_list_style�google_text_emphasis�hn�list_numbering_start�pad_tables_in_text�skipwrap� unifiable_n�� � � c s� e Zd Zddejf� fdd� Z� fdd�Zdd� Zd d
� Z� fd d �Z d
d� Z
dd� Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd.d"d#�Zd/d$d%�Zd&d'� Zd(d)� Zd*d+� Zd,d-� Z� ZS )0� HTML2TextN� c s
t � jdd� d| _d| _d| _tj| _tj| _ tj
| _ || _ tj
| _tj| _tj| _tj| _tj| _tj| _tj| _tj| _ tj | _ tj | _ tj!| _"tj#| _$d| _%d| _&d| _'d| _(tj)| _*tj+| _,d| _-tj.| _/tj0| _1tj2| _3tj4| _5tj6| _7d| _8tj9| _:tj;| _<|dk�r | j=| _>n|| _>g | _?d| _@d| _Ad| _Bd| _Cd| _Dg | _Eg | _Fd| _Gd| _HtIjJd �| _Kd| _Lg | _Md| _Nd| _Od| _Pd| _Qd| _Rd
| _Sd| _Td| _Ud| _Vi | _Wg | _Xd| _Yd| _Zd| _[d| _\d| _]i | _^|| __d| _`d| _ad| _bd| _cd tjdd < dS )
z�
Input parameters:
out: possible custom replacement for self.outtextf (which
appends lines of text).
baseurl: base URL of the document we process
F)Zconvert_charrefsr �*�_z**NTz^[a-zA-Z+]+://r z _place_holder;�nbsp)e�super�__init__�
split_next_td�td_count� table_startr Z UNICODE_SNOB� unicode_snobZ ESCAPE_SNOB� escape_snobZLINKS_EACH_PARAGRAPH�links_each_paragraph�
body_widthZSKIP_INTERNAL_LINKS�skip_internal_linksZ INLINE_LINKS� inline_linksZ
PROTECT_LINKS�
protect_linksZGOOGLE_LIST_INDENT�google_list_indentZIGNORE_ANCHORS� ignore_linksZ
IGNORE_IMAGES�
ignore_imagesZIMAGES_AS_HTML�images_as_htmlZ
IMAGES_TO_ALT�
images_to_altZIMAGES_WITH_SIZE�images_with_sizeZIGNORE_EMPHASIS�ignore_emphasisZ
BYPASS_TABLES�
bypass_tablesZ
IGNORE_TABLES�
ignore_tables�
google_doc� ul_item_mark�
emphasis_mark� strong_markZSINGLE_LINE_BREAK�single_line_breakZUSE_AUTOMATIC_LINKS�use_automatic_links�hide_strikethroughZ MARK_CODE� mark_codeZWRAP_LIST_ITEMS�wrap_list_itemsZ
WRAP_LINKS�
wrap_linksZ
PAD_TABLES�
pad_tablesZDEFAULT_IMAGE_ALT�default_image_alt� tag_callbackZ
OPEN_QUOTE�
open_quoteZ CLOSE_QUOTE� close_quote�outtextf�out� outtextlist�quiet�p_p�outcount�start�space�a�astack�maybe_automatic_link�
empty_link�re�compile�absolute_url_matcher�acount�list�
blockquote�pre�startpre�code�quote� br_toggle� lastWasNL� lastWasList�style� style_def� tag_stack�emphasis�drop_white_space�inheader�
abbr_title� abbr_data� abbr_list�baseurl�stressed�preceding_stressed�preceding_data� current_tag� UNIFIABLE)�selfr> r_ � bodywidth)� __class__� � /usr/lib/python3.6/__init__.pyr " s�
zHTML2Text.__init__c s |j dd�}t� j|� d S )Nz</' + 'script>z </ignore>)�replacer �feed)re �data)rg rh ri rk | s zHTML2Text.feedc C s8 | j |� | j d� | j| j� �}| jr0t|�S |S d S )Nr )rk �optwrap�closer8 r )re rl Zmarkdownrh rh ri �handle� s
zHTML2Text.handlec C s" | j j|� |r |d dk| _d S )N� �
���)r? �appendrT )re �srh rh ri r= � s zHTML2Text.outtextfc sZ t � j� | j� | jddd� dj| j�}| jr@tjj d }nd}|j
d|�}g | _|S )Nr �end)�forceznbsp;� z _place_holder;) r rn �pbr�o�joinr? r �html�entities�html5rj )re Zouttextr )rg rh ri rn � s
zHTML2Text.closec C s | j | j|�d� d S )NT)� handle_data�charref)re �crh rh ri �handle_charref� s zHTML2Text.handle_charrefc C s | j |�}|r| j|d� d S )NT)� entityrefr~ )re r� �refrh rh ri �handle_entityref� s
zHTML2Text.handle_entityrefc C s | j ||dd� d S )NT)rC )�
handle_tag)re �tag�attrsrh rh ri �handle_starttag� s zHTML2Text.handle_starttagc C s | j |d dd� d S )NF)rC )r� )re r� rh rh ri �
handle_endtag� s zHTML2Text.handle_endtagc C s� d|kr dS d}xpt | j�D ]b\}}d|krv|d |d krvd|ksLd|krrd|krvd|krv|d |d krvd}nd}|r |S q W dS )z�
:type attrs: dict
:returns: The index of certain set of attributes (of a link) in the
self.a list. If the set of attributes is not found, returns None
:rtype: int
�hrefNF�titleT)� enumeraterE )re r� �match�irE rh rh ri �
previousIndex� s zHTML2Text.previousIndexc C s� t |�}t |�}d|ko | j}d}x$tjD ]}||ko<||k}|r*P q*W d|koVd|k} t|�opt|� op| j }
|�r|s�| s�|
r�| jd7 _|r�| jd7 _| r�| j| j � | j
d7 _
|r�| j| j � | j
d7 _
|
�r�| jd� | j
d7 _
d| _ n�|�s| �s|
�r,| jd8 _d| _
|
�rZ| j
�rJ| j
d8 _
n
| jd� d| _ |�r�| j
�rx| j
d8 _
n | j| j � | �r�| j
�r�| j
d8 _
n | j| j � |�s�| �r�| j �r�| jd� |�r�| jd8 _dS ) z/
Handles various text emphases
z line-throughF�italicrp �`Trw N)r r4 r ZBOLD_TEXT_STYLE_VALUESr rO rY r@ ry r0 rZ r1 rQ rD ) re rC � tag_style� parent_styleZ tag_emphasisZparent_emphasisZ
strikethroughZboldZ bold_markerr� Zfixedrh rh ri �handle_emphasis� s^
zHTML2Text.handle_emphasisc C s� || _ |d kri }nt|�}| jd k r>| j| |||�dkr>d S |rx| jd k rx|d^krx|dksb| jrx| jd� d | _d | _| jr�i }|r�| jr�| jd_ d }t || j
|�}| jj |||f� n4| jr�| jj � nd i i f\}}}| jr�| jd` d }t
|��r0| j� |�r&d| _| jt
|�d d
� n
d | _d S |dak�r�| j�rf|�r\t|��r\| j� n| j� n | j�rz|dk�rzn| j� |dk�r�|�r�| jdk�r�| jd� n
| jd� |dk�r�|�r�| j� | jd� | j� |dbk�r |�r�| jd
7 _n| jd
8 _|dk�r:|�r,| jd
7 _n| jd
8 _|dck�rJd| _|dk�r�|�r�| j� | jddd� d| _| jd
7 _n| jd
8 _| j� dd� }|ddk�r�| j �r�|�r�|| ��r�d
| j }n| j}| j|� |�r�d| _|dek�r<| j �r<|�r || ��r d
| j } n| j} | j| � |�r<d| _|dfk�rv|�r\|| ��r\d$}
nd%}
| j|
� |�rvd| _| j�r�| j�s�| j|||� |dgk�r�| j �r�| jd)� | j | _ |d*k�r|�r�d | _ d+| _ d,|k�r|d, | _ n&| j d k �r| j | j | j < d | _ d | _ |d-k�rJ| j!�s4| j| j"� n | j| j#� | j! | _!dhd.d/�} |d0k�r| j$ �r|�r�d1|k�r�|d1 d k �r�| j%�o�|d1 j&d � �r�| jj |� |d1 | _d| _| j'�r�d2|d1 d3 |d1< n | jj d � �n | j�r| jj � } | j�r| j �rd | _n�| �r| j�r:| jd� d | _d | _| j(�r�y"| d, �rV| d, nd+}
t)|
�}
W n& t*k
�r� | | | d1 d+� Y nX | | | d1 |
� nh| j+| �}|d k �r�| j,| } n.| j-d
7 _-| j-| d4< | j.| d5< | j,j | � | jd6t/| d4 � d7 � |dk�r,|�r,| j �r,d8|k�r,| j0�s>|d8 |d1< |j1d9��pN| j2}| j3�st| j4�r�d:|k�std;|k�r�| jd<|d8 d= � d:|k�r�| jd>|d: d= � d;|k�r�| jd?|d; d= � |�r�| jd@| d= � | jdA� d S | jd k �r\| j}| j0�rFt)|�|k�rF| j5j6|��rF| jd2t)|� d3 � d | _d S | jd� d | _d | _| j0�rt| jt)|�� n�| jdBt)|� d7 � | j(�r�|j1d1��p�d+}| jdCt)t7j8| j9|�� dD � nh| j+|�}|d k �r�| j,| }n.| j-d
7 _-| j-|d4< | j.|d5< | j,j |� | jdt/|d4 � d7 � |dk�rD|�rD| j� |dk�r^| �r^| j:� |dEk�rx|�rx| jdF� |dEk�r�| �r�| j:� |dik� r(| j; �r�| j< �r�| j� |�r�| j�r�t=|�}n|}t>|�}| j;j ||dI�� n0| j;� r | j;j � | j � r | j; � r | jdJ� d| _<nd | _<|dKk� r�| j:� |� r�| j;� rZ| j;dj }n
dHddI�}| j� rx| j?|�}n
t@| j;�}| jdL| � |dM dHk� r�| j| jAd
� n4|dM dGk� r�|dN d
7 < | jt/|dN �dO � d| _|dkk� r�| jB�
r |dQk� r�|�
rn| j� n �n�| jC�
r�|�
r2| j� |dlk�
rf|�
rT| jdTjD|�� n| jdUjD|�� n(|�
r~| jdVjD|�� n| jdWjD|�� �n |dPk�
r�|�
r�d| _E| jF�
r�| jd2tGjH d3 � | jd� n&| jF�
r�| jdXtGjH d3 � | jd� |dmk� r |� r | jI� r| jdY� d| _I|dQk� r4|� r4d| _J|dQk� rT| � rTd | _I| j� |dQk� r�| � r�| jE� r�| jdZjKd[g| jJ �� | j� d | _E|dnk� r�|� r�| jJd
7 _J|d\k� r�|� r�d| _Ld| _ nd | _ | jM� r�| jNd]� | j� d S )oNT�p�divrV �dl�dtZimg�[Frp � �#rw �brr z
> z
Zhrz* * *�head�script�bodyrN z> )rv c S s | j otjd| j d �S )Nz[^\s]rp rr )rb rI r� )re rh rh ri �no_preceding_space| s z0HTML2Text.handle_tag.<locals>.no_preceding_space�emr� �u�strong�b�del�strikert z ~~z~~�kbdrQ �ttr� �abbrr r� �qc S s@ t j| j|�}|j� r dj|�nd}| jdjt|�|d�� d S )Nz "{}"r z]({url}{title}))�urlr� )�urlparse�urljoinr_ �strip�formatry r )re �linkr� r� rh rh ri �link_url� s z&HTML2Text.handle_tag.<locals>.link_urlrE r� �<�>�countrB z][�]�src�alt�widthZheightz
<img src='z' zwidth='zheight='zalt='z/>z![�(�)Zddz �ol�ul)�name�numrq �liz r� r� z. �table�tr�td�thz<{}>
z
</{}>z<{}>z</{}>z</z| �|z---rO z
[/code])r� r� rV r� r� rr rr )r� r� )r� rV r� )r� )r� r� r� )r� r� )r� r� rt )r� rQ r� )r )r� r� rr )r� r� r� r� )r� r� )r� r� )r� r� )Orc �dictr: rG r' ry rH r. rX r rW rs �popr r� r[ r �soft_brrF rN r@ rV rC r+ r0 r` r1 r� rO rQ r\ r] r^ rR r; r< r& r" �
startswithr$ r# r �KeyErrorr� rE rL rB �strr) �getr9 r( r* rK r� r� r� r_ rx rM rU r
r
�google_nest_count�lenr/ r- r, r� r r8 r ZTABLE_MARKER_FOR_PADr r rz rP r5 r> )re r� r� rC r� r� Zdummyr� rY r� r� r� rE r� r� r� r� Z
list_styleZnumbering_startr� �
nest_countrh rh ri r� s$
zHTML2Text.handle_tagc C s | j dkrd| _ dS )z Pretty print has a line breakr rp N)rA )re rh rh ri rx � s
z
HTML2Text.pbrc C s | j r
dnd| _dS )z Set pretty print to 1 or 2 linesrp r� N)r2 rA )re rh rh ri r� � s z HTML2Text.pc C s | j � d| _dS )z Soft breaksz N)rx rS )re rh rh ri r� � s zHTML2Text.soft_brFc C sF | j dk r| j |7 _ | j�sB| jrT|j� }| jrF| jp>| j rF|}|dkrTd| _|r�| j r�tjdd|�}|r�|d dkr�d| _ |dd� }| r�| r�dS | j
r�|j d� r�|j d � r�d| }| j r�| j
d
� d| _d | j }|o�|o�|d d k �r| j�r|d7 }| j�rX| j�s&|d 7 }x tt| j��D ]}|d 7 }�q6W |jdd| �}| j
�rxd
| _
| j�rx|jd�}| j�r�d
| _ d| _d
| _|dk�r�d| _| j
d� d
| _ | j�r�| j
| jd | | j � d
| _ d| _| j �r | j�s�| j
d� d
| _ | j�r�| jdk�r | j�s&|dk�r�|dk�r:| j
d� g }x�| jD ]x}| j|d k�r�| j
dt|d � d tj | j |d � � d|k�r�| j
d|d d � | j
d� n
|j |� �qFW | j|k�r�| j
d� || _| j �r$|dk�r$x2| j j � D ]$\} }
| j
d| d |
d � �q�W d| _| j
|� | jd7 _dS )z6
Deal with indentation and whitespace
Nr r z\s+rw Trp rq z
z
[code]r� z Fru r� rB z [r� z]: r� r� z (r� z *[)!r] r@ r. �lstriprZ rO rQ rI �subrD rP r� r5 r> rA rN rM �ranger� rj rC rS rT rE r rB r� r� r� r_ rs r^ �items) re rl �puredatarv Zlstripped_dataZbqr� Znewar� r� Z
definitionrh rh ri ry � s�
(
z HTML2Text.oc C s |sd S | j r$|j� }d| _ d| _n:| jr^tjd|d �rXt| j� rX| jdkrXd| }d| _| jrt| jj t
|�� | j d k r�| j }||kr�| j j|�r�| j
r�| jd | d
� d| _d S | jd � d | _ d| _| j r�| j r�| r�t|| jd �}|| _| j|dd
� d S )NFTz[^\s.!?]r rE rQ rO rw r� r� r� )Zsnob)r� )rE rQ rO )r` r� ra rI r� r rc rV rW �updater rG rK r3 ry rH rQ rO r r rb )re rl Z entity_charr� rh rh ri r~ s:
zHTML2Text.handle_datac C sb |d dkr t |dd � d�}nt |�}| j r@|tkr@t| S yt|�S tk
r\ dS X d S )Nr �x�Xrp � r )r� r� )�intr r �chr�
ValueError)re r� r� rh rh ri r = s zHTML2Text.charrefc C sd | j r |tjkr tj| S ytjj|d }W n tk
rL d| d S X |dkr`tj| S |S )N�;�&r )r r rd r{ r| r} r� )re r� Zchrh rh ri r� K s
zHTML2Text.entityrefc C s* d}d|kr&t |d dd� �| j }|S )zq
Calculate the nesting count of google doc lists
:type style: dict
:rtype: int
r z margin-leftNr� ���)r� r% )re rV r� rh rh ri r� T s zHTML2Text.google_nest_countc C s | j s
|S d}d}| js d| _x�|jd�D ]�}t|�dkr�t|| j| j�s�d}|jd| j �rdd}n|jd�rrd}t || j d|d�}|dj
|�7 }|j d�r�|d 7 }d
}q�|r�|d7 }d
}q�|d 7 }d }n t j
j|�s�||d 7 }d
}q*|d k r*|d7 }|d
7 }q*W |S )
zi
Wrap all paragraphs in the provided text.
:type text: str
:rtype: str
r r Frq z z z> )Zbreak_long_wordsZsubsequent_indentz
rp z
r� )r! r7 r# �splitr� r r6 r� r/ r rz �endswithr ZRE_SPACEr� )re �text�result�newlinesZpara�indent�wrappedrh rh ri rm b sF
zHTML2Text.optwrap)FF)F)�__name__�
__module__� __qualname__r �
BODY_WIDTHr rk ro r= rn r� r� r� r� r� r� r� rx r� r� ry r~ r r� r� rm �
__classcell__rh rh )rg ri r ! s. Z G }
o
+ r r c C s$ |d krt j}t||d�}|j| �S )N)r_ rf )r r� r ro )r{ r_ rf �hrh rh ri � html2text� s r� )r r r )r N) �__doc__Z
html.entitiesr{ Z html.parserrI Z urllib.parse�parser� �textwrapr r� r Zhtml2text.utilsr r r r r r r
r r r
r r r � __version__�parserZ
HTMLParserr rh rh rh ri �<module> s"
<
Sindbad File Manager Version 1.0, Coded By Sindbad EG ~ The Terrorists