+ QV-j9^RIHt^RIHtHt]!4'd^RIt^RIHt^RIHuH t ]P!] 4t ^tRRlt]P RRl4t!RR ]P$4t!R R ]P(P*4t!R R ]P(P*4t!RR]P04tRRRllt!RR4tR#))should_convert_module)is_torch_availableloggingNcXV^8dQhR\PR\P/#)rquantized_weightsreturn)torchTensor)formats"q/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/transformers/integrations/bitnet.py __annotate__r s"##ELL#U\\#c\VPpV^,\,^, \,p\V4^8XdV3pM V.VR,O5pV^, p\P!W0P \P R7pVP\P 4p\\V^,V,^,4p\V4FLpWr,p\W,V^,4p VRW, ;;;WXV ^V,,,uuu%KN V#)a Packs a tensor of quantized weights into a compact format using 2 bits per value. Parameters: ----------- quantized_weights : torch.Tensor A tensor containing ternary quantized weights with values in {-1, 0, 1}. These values are adjusted to {0, 1, 2} before being packed. Returns: -------- torch.Tensor A packed tensor where each element stores 4 quantized values (each using 2 bits) in an 8-bit format. NNdevicedtypeN) shapeVALUES_PER_ITEMlenr zerosruint8tominrange) roriginal_shaperow_dimpacked_tensor_shapepackedunpackeditistartends & r pack_weightsr&s ',,Na ?2Q6?JG >a&j&<);< [[,5M5MUZU`U` aF ##EKK0H _~a0G;q@ AB 2Y %/>!#45#+8##6!a%#?? MrcxV^8dQhR\PR\PR\P/#)rr rr)r r r)r s"r r r 8s2A"A"5<<A" A" A"rcVPp\V4^8XdV^,\,pV3pM!V^,\,pV.VR,O5p\P!W@P \P R7p\\4FIpWb^,,pWr^,,p^^V,,p W ,^V,, WWV%KK VPV4^, #)u Unpacks a tensor of quantized weights that were stored in a packed format using 2 bits per value. Parameters: ----------- packed : torch.Tensor A tensor containing packed weights where each element represents 4 quantized values (using 2 bits per value). dtype : torch.dtype The dtype of the returned Tensor Returns: -------- torch.Tensor A tensor of unpacked weights, where each value is converted from its packed 2-bit representation. Example: -------- packed = torch.tensor([[0b10100001, 0b00011000], [0b10010000, 0b00001010]], dtype=torch.uint8) # Unpack the values unpacked = unpack_weights(packed) # Resulting unpacked tensor print(unpacked) # Output: tensor([[ 0, -1], [-1, 1], [-1, 1], [-1, 1], [ 1, 0], [ 0, -1], [ 1, -1], [ 1, -1]]) Explanation of the example: --------------------------- Let's take the first value for example 0b10100001, we will only focus on the first column, because every element is unpacked across the first dimension - First 2 bits: `01` → 0 at [0][0] - Second 2 bits: `00` → -1 at [0][2] - Third 2 bits: `10` → 1 at [0][4] - Fourth 2 bits: `10` → 1 at [0][6] the second value of the same row (0b10010000) will give the values for [0][1], [0][3], [0][5], [0][7] We subtract 1 because during the packing process, it's easier to work with values like 0, 1, and 2. To make this possible, we add 1 to the original ternary weights (which are typically -1, 0, and 1) when packing them. When unpacking, we reverse this by subtracting 1 to restore the original ternary values. rr) rrrr rrrrr) r r packed_shapeoriginal_row_dimunpacked_shaper!r#r$r%masks && r unpack_weightsr-7sb<\"-=>{{>--u{{SH ? #O#1o%QU|%}!a%8s $ ;;u  !!rcaa]tRt^|toRV3RlV3Rlllt]P RRl4t]P R4tRt Rt Vt V;t #) BitLinearc 8<V^8dQhRS[RS[RS[RS[RS[/#)r in_features out_featuresbias use_rms_norm rms_norm_epsintboolfloat)r __classdict__s"r r BitLinear.__annotate__}sD(H(H(H(H (H(H(Hrc <\S V`4WPnWnW nVP R\ P!V\,V3\ PVR74VP R\ P!^VVR74V'd*VP R\ P!W%VR74MRVn RVn V'd^RI HpV!WR7Vn R#R#)weightrr weight_scaler3N LlamaRMSNormeps)super__init__rr1r2register_bufferr rrronesr3rms_normmodels.llama.modeling_llamarA) selfr1r2r3rrr4r5rA __class__s &&&&&&&& r rEBitLinear.__init__}s  &(   KK0+>kk     JJ     lY_)` aDI  B(GDM rc `^V^, ,)p^V^, ,^, pWAP4PRRR7PPRR7, pW,P 4PW44pVP \ P4V3#)a Activation function : Performs symmetric, per-token quantization on the input activations. Parameters: ----------- input : torch.Tensor Input activations to be quantized. num_bits : int, optional (default=8) Number of bits to use for quantization, determining the quantization range. Returns: -------- result : torch.Tensor Quantized activation tensor, with values mapped to an `int8` range. scale : torch.Tensor The per-channel scaling factors used to quantize the tensor. Tdimkeepdimh㈵>r)absmaxvaluesclamproundrr int8)rJinputnum_bitsQnQpscaleresults&&& r activation_quantBitLinear.activation_quants$X\" # 8a< 1 $YY[__T_:AAGGDGQQ-&&(..r6yy$e++rc$WV,, pV#N)rJrZ input_scaler?outs&&&& r post_quant_processBitLinear.post_quant_processs\12 rcVPeVPV4pVPp\W PR7pVP V4wrE\ P !VPVP4V4pVPW`PV4pVPe2W`PP^R4PV4, pV#)NrrS) rHr=r-rr`Flinearrrgr?r3view expand_as)rJrZww_quant input_quantreys&& r forwardBitLinear.forwards == $MM%(E KK **5#'#8#8#? HH[^^DJJ/ 9  # #A'8'8+ F 99 2&003 3Ar)r3rr1r2rH)NNFư>)) __name__ __module__ __qualname____firstlineno__rEr compiler`rgrs__static_attributes____classdictcell__ __classcell__rKr:s@@r r/r/|sL(H(HT ]],,. ]]  rr/cba]tRt^toRt]]PR44t]R4t Rt Vt R#) WeightQuantz Implements a custom autograd function for weight quantization. This performs ternary quantization (-1, 0, 1) based on scaling by the mean absolute value of the weights. It uses the Straight-Through Estimator (STE) for the backward pass. c VPpVP4pRVP4P4P RR7, pW,P 4P R^4V, pVPV4#)g?rQrRrS)rr9rTmeanclamp_rXrWr)ctxr=rr^s&& r rsWeightQuant.forwardsp fjjl'')00T0::.'')//A6>yyrc&VP4pV#rccloner grad_output grad_inputs&& r backwardWeightQuant.backward &&( rrdN rwrxryrz__doc__ staticmethodr r{rsrr|r}r:s@r rrs> ]]  rrcba]tRt^toRt]]PR44t]R4t Rt Vt R#)ActQuanta$ Implements a custom autograd function for activation quantization. This performs symmetric 8-bit quantization (to the range [-128, 127]) based on the maximum absolute value along the last dimension (per-token/row scaling). It uses the Straight-Through Estimator (STE) for the backward pass. c:VPpVP4p^VP4PRRR7PP RR7, pW,P 4PR^4V, pVPV4#)TrNrQrRrSi) rr9rTrUrVrrXrWr)r activationrr^s&& r rsActQuant.forwards  %%' jnn&**r4*@GGNNSWNXX (//177cBUJ }}U##rc&VP4pV#rcrrs&& r rActQuant.backwardrrrdNrrs@r rrs> ]]$$rrcNaa]tRtRtoRV3RlV3RllltRtRtRtVtV;t #) AutoBitLinearic ><V^8dQhRS[RS[RS[RS[RS[RS[/#)rr1r2r3 online_quantr4r5r6)r r:s"r r AutoBitLinear.__annotate__sODDDD DDDDrc <\S V`WV4W`nRVnV'd^RIHp V !WR7VnV'gGVP R\P!^VVR74VPVP4R#R#)Nr@rBr?r>) rDrErrHrIrArFr rG"_register_load_state_dict_pre_hook load_hook) rJr1r2r3rrrr4r5rArKs &&&&&&&&& r rEAutoBitLinear.__init__su D9(  B(GDM   !   3 3DNN CrcVR,V9djWR,,PVPP8wd8\WR,,VPPR7WR,&V#)r=rj)rr=r-)rJ state_dictprefixargskwargss&&&*,r rAutoBitLinear.load_hook sb X * ,XN7rcRpVP4EFwrE\WA4'gK\P!R4;_uu_4\ V\ P 4'Ed_V'dVPR8Xd\VPVPVPRJVPPVPPVPR8HVPVP R7pVPR8XdVP#R4M\%VPVPVPRJVPPVPPV'd VPMRV'd VP MRR 7pVP#R4VP'WF4R pRRR4EK V'g\(P+R 4V# +'giEK;i) a9 Public method that replaces the linear layers of the given model with bitnet quantized layers. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. modules_to_not_convert (`list[str]`, *optional*, defaults to `None`): A list of nn.Linear weights to not convert. If a parameter path is in the list (e.g. `lm_head.weight`), the corresponding module will not be converted. quantization_config (`BitNetConfig`): The quantization config object that contains the quantization parameters. Fmeta autobitlinearNonline)r1r2r3rrrr4r5offlineru)r1r2r3rrr4r5TzYou are loading your model using bitnet but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.) named_modulesrr r isinstancennLinear linear_classrr1r2r3r=rquantization_moder4r5requires_grad_r/ set_submoduleloggerwarning)modelrquantization_confighas_been_replaced module_namemodule new_modules&&& r replace_with_bitnet_linearr;s$224 $[II  \\& ! !&")),,&+>+K+K+^!.$*$6$6%+%8%8#[[4%}}33$mm11&9&K&Kx&W%8%E%E%8%E%E "J+<< I"11%8!*$*$6$6%+%8%8#[[4%}}33$mm11I\%8%E%EbgI\%8%E%Ebf"J--e4##K<$(!7" ! 5@   LI" ! !s )G64DG67G66 H c<a]tRtRtoRtRV3RlRlltRtVtR#)BitNetDeserializeiucWnR#rc hf_quantizer)rJrs&&r rEBitNetDeserialize.__init__vs(rNc <V^8dQhRS[S[S[S[P,3,RS[P P R,RS[R,RS[S[S[P3,/#)r input_dictrNfull_layer_namer)dictrrr r rModule)r r:s"r r BitNetDeserialize.__annotate__yse$$d5<<001$xx%$t $ c5<< $rc VP4F(wrV\V\4'gKV^,W&K* RpVPV4p^RIHp Rp VP p VefVebV !W#4wr\V R4'dF\V R4'd4V PpVP^,pV\,V8XdRp V 'd-VP\P4p\VV R7pWx/#)r=)get_module_from_nameFr2r1Trj)itemsrrpopquantizers.quantizers_utilsrrhasattrr2rrrr rr-)rJrrrrkeyvalue key_weightr=rneeds_unpacking target_dtyper_ expected_out actual_out weight_uint8s&&&&, r convertBitNetDeserialize.convertys%**,JC%&&"'( -  +F||  !<,UDIFv~..76=3Q3Q &22 #\\!_ /<?&*O !99U[[1L#L EF##rrNN)rwrxryrzrErr|r}rs@r rrus)$$$rrr)rrutilsrrr torch.nnrtorch.nn.functional functionalrk get_loggerrwrrr&r{r-rr/autogradFunctionrrrrrrrdrr rs?/##   H %#LA"A"HT Tn%..)).u~~&&.7BII7t7t $ $r