o ?"g@s`dZddlmZmZmZerddlZeeZddZ ddZ d d Z dd d Z dddZ dS)z2HQQ (Half-Quadratic Quantization) integration file)is_hqq_availableis_torch_availableloggingNcCs|D]\}}||_qdSN) named_modulesname)modelrmoduler _/mnt/skqttb/ctump_chatbot/chatbot/lib/python3.10/site-packages/transformers/integrations/hqq.pyautoname_modulessr cCsddd|dDS)N.cSs g|] }|dvr|s|qS))r layers) isnumeric).0nr r r !s z&name_to_linear_tag..)joinsplit)rr r r name_to_linear_tag srcCsRtr ddlm}t}|D]\}}t|tjj|fr$| t |qt |S)Nr) HQQLinear) rhqq.core.quantizersetr isinstancetorchnnLinearaddrlist)r r linear_tagsrr r r r get_linear_tags%s r!c Cs|D]b\}}|durg}||t|tjjrNt|j}||vrA||durA|||j|_ t ||j|_ |j| dd}dD]}t ||dqEtt|dkrat|||d\}}|dq||fS)NFT)W_qmetar patch_paramshas_been_replaced)named_childrenappendrrrrrr_modules quant_configtype source_clsrequires_grad_setattrlenrchildren_prepare_for_hqq_linearpop) r r%r&current_key_namerr linear_tagatt_r r r r20s,    r2Fcs|durgn|}t|t||j}|jttt|t|tfddDr>ddD}|n fddD}t |||d\}}|j |d|j _ |s`t d |S) a Prepares nn.Linear layers for HQQ quantization. Since each layer type can have separate quantization parameters, we need to do the following: 1- tag each module with its neme via autoname_modules() 2- Extract linear_tags (e.g. ['self_attn.q_proj', ...]) 3- Map quantization parameters as a dictionary linear_tag -> quant_params as HQQLinear exepects it, this is referred to as patch_params Nc3s|]}|vVqdSrr rkey)r r r ksz)prepare_for_hqq_linear..cSsi|]}|dqSrr r8r r r mz*prepare_for_hqq_linear..csi|]}|qSr r )rk)r+r r r;qr<r$)r+ quant_method skip_moduleszconfigquantization_configloggerwarning)r rDmodules_to_not_convertr&r?r%r )r r+r prepare_for_hqq_linearUs(     rHr)NNF)__doc__utilsrrrr get_logger__name__rEr rr!r2rHr r r r s   %