+ QV-jt9^RIt^RIHt^RIHtHt^RIHtHtH t H t H t ] !4'd^RI t ] !4'd^RIt^RIHt^RIHt]!4'd ^RIt^RIHtHt] P.!]4t!RR]4t!R R ]4t!R R ]4t!R R]4tRRRlltRRRlltRt RRlt!RRlt"R#)N) ConversionOps)get_module_from_nameshould_convert_module)get_available_devicesis_accelerate_availableis_bitsandbytes_availableis_torch_availablelogging)Conv1D)add_hook_to_moduleremove_hook_from_modulec<a]tRt^toRtRV3RlRlltRtVtR#)Bnb4bitQuantizecWnR#N hf_quantizerselfrs&&w/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/transformers/integrations/bitsandbytes.py__init__Bnb4bitQuantize.__init__(Nc <V^8dQhRS[S[S[S[P,3,RS[R,RS[P P R,RS[S[S[P3,/#) input_dictfull_layer_nameNmodelreturndictstrlisttorchTensornnModule)format __classdict__s"r __annotate__Bnb4bitQuantize.__annotate__"sed5<<001txx%  c5<< rc /pVP4Fwrg\V\4'd V^,p\W64wr\ VP \ 4'd VPpVPV4p \PP!V3RR/V PBPVP4p RVnWV&K V#)a* we need to store some parameters to create the quantized weight. For example, bnb requires 6 values that are stored in the checkpoint to recover the quantized weight. So we store them in a dict that it stored in hf_quantizer for now as we can't save it in the op since we create an op per tensor. requires_gradFT)items isinstancer$r issubclass source_clsr Tget_parameter_or_bufferbnbr' Params4bit__dict__todevice_is_hf_initialized) rrrrkwargsresult param_namevaluemodule_ old_value new_values &&&&, rconvertBnb4bitQuantize.convert"s!+!1!1!3 J%&&a-U?IF&++V4455jAI))%[u[ HZHZ[^^_d_k_klI(,F %!*: "4  rrNN__name__ __module__ __qualname____firstlineno__rrC__static_attributes____classdictcell__r*s@rrrs)rrc<a]tRt^@toRtRV3RlRlltRtVtR#)Bnb4bitDeserializecWnR#rrrs&&rrBnb4bitDeserialize.__init__ArrNc <V^8dQhRS[S[S[S[P,3,RS[P P R,RS[R,RS[S[S[P3,/#rrrNrr r!)r)r*s"rr+Bnb4bitDeserialize.__annotate__Dse''d5<<001'xx%'t ' c5<< 'rc b\V4^8XdV#VP4F(wrV\V\4'gKV^,W&K* RpVP V4p\ W#4wr\ PPPVVRVPV R7p RV n W{/#)zO Deserialization of bnb keys. We need 6 keys to recreate the quantized weights weightF)dataquantized_statsr.r9r?T) lenr/r0r$poprr5r'r6from_prequantizedr9r:) rrrrr;keyr> key_weightrVr?r@rBs &&&&, rrCBnb4bitDeserialize.convertDs z?a  $**,JC%&&"'( -  +(@ FF%%77&== 8 %)!&&rrrErFrMs@rrOrO@s)'''rrOc<a]tRt^ctoRtRV3RlRlltRtVtR#)Bnb8bitQuantizecWnR#rrrs&&rrBnb8bitQuantize.__init__drrNc <V^8dQhRS[S[S[S[P,3,RS[P P R,RS[R,RS[S[S[P3,/#rSr!)r)r*s"rr+Bnb8bitQuantize.__annotate__gsed5<<001xx%t  c5<< rc /pVP4Fwrg\V\4'd V^,MTp\W&4wr\ VP \ 4'd VPpVPp VPV4Pp V PRR4\PP!VPR43RR/V BPV 4p WV&K V#)rSCBNcpur.F)r/r0r$rr1r2r r3r9r4r7rZr5r' Int8Paramsr8) rrrrr;r<r=r>r?r@ value_device params_kwargsrBs &&&&, rrCBnb8bitQuantize.convertgs!+!1!1!3 J *5$ 7 7E!HUE,U?IF&++V44 <r?r@r]rVrBs &&&&, rrCBnb8bitDeserialize.converts z?a  $**,JC%&&"'( -)@  '..?HH"5)u FF%%fLELVLOOPVP]P]^ $(!&&rrrErFrMs@rrmrms)'''rrmc|V^8dQhR\PPR\\,R,/#)rrmodules_to_not_convertN)r%r'r(r$r#)r)s"rr+r+s1JJ 88??J I,Jrc RpVP4EF;wrV\WQ4'gKRp\P!R4;_uu_4\ V\ 4'g\ V4\PJEd\ V\ 4'dVPPwrMVPpVPp VP4R8Xd\PPVV VP RJVP"VP$R7pV'dDVPP&P)\P*R7VPnM\PP-VV VP RJVP.VP0VP2VP4R7pV'd@VPP&P)VP4R7VPnVe5\ V4VnVP9R4VP;WW4RpRRR4EK> V'g\<P?R 4V# +'giEKq;i) a A helper function to replace all `torch.nn.Linear` modules by bnb modules from the `bitsandbytes` library. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. modules_to_not_convert (`list[str]`, defaults to `None`): A list of nn.Linear weights to not convert. If a parameter path is in the list (e.g. `lm_head.weight`), the corresponding module will not be converted. quantization_config (`BitsAndBytesConfig`): The quantization config object that contains the quantization parameters. pre_quantized (`book`, defaults to `False`): Whether the model is pre-quantized or not FNmetallm_int8)has_fp16_weights threshold)dtype)compress_statistics quant_type quant_storageTzYou are loading your model using eetq but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.) named_modulesrr%r9r0r typer'LinearrVshape in_features out_featuresquantization_methodr5 Linear8bitLtbiasllm_int8_has_fp16_weightllm_int8_thresholdrWr8int8 Linear4bitbnb_4bit_compute_dtypebnb_4bit_use_double_quantbnb_4bit_quant_typebnb_4bit_quant_storager2requires_grad_ set_submoduleloggerwarning) rruquantization_config pre_quantizedhas_been_replaced module_namer? new_modulerrs &&&& rreplace_with_bnb_linearrs ($224 $[II  \\& ! !&&))T&\RYY-Fff--06 0C0C-K"("4"4K#)#6#6L&::< J!$!4!4#$ 4/)<)U)U"5"H"H "5"J%2<1B1B1G1G1J1JQVQ[Q[1J1\ )).!$!2!2#$ 4/+BB,?,Y,Y#6#J#J&9&P&P"3"J%1;1B1B1G1G1J1J"5"L"L2K2 )).),0LJ)--e4'' @(,%Q" ! 5\   La" ! !s CI2)B%I2A7I22 J cV^8dQhRR/#)rrVztorch.nn.Parameter)r)s"rr+r+s"6rc\V\PP4'g\ R\ V4 R24hVP PpVR9dV#VR8Xd7\PPVPVP4pV#VPfVPVn \\PR4'd7\PPVPVP4pV#VPVPP!R^4,R,pV#)z Helper function to dequantize 4bit or 8bit bnb weights. If the weight is not a bnb quantized weight, it will be returned as is. z1Input weight should be of type nn.Parameter, got z insteadr6int8_vectorwise_dequantg@ ?)r6rh)r0r%r' Parameter TypeErrorr __class__rGr5 functionaldequantize_4bitrW quant_staterfhasattrrview)rVstatecls_name output_tensor dequantizeds&& rdequantize_bnb_weightrs fehh00 1 1KDQWL>Yabcc((H33 <66v{{FDVDVW  yyJJ s~~899nn<FLLDE % 2 23HX]3^ _J *6==%@F}##5fll^DOP##&Efll^Sbchbiij$kl5) % 2 26 :J "&vz**!??6x@'/":8 MM&--.. /    8 $ 3 56  m  L;&%%s AI,, I= c\\R\44p\\44pVP V4'g>V'dRT;'gR RV R2p\ V4h\ PR4R#R#) zk Validates if the available devices are supported by bitsandbytes, optionally raising an exception if not. supported_torch_devicesz3None of the available devices `available_devices = NzY` are supported by the bitsandbytes version you have installed: `bnb_supported_devices = z`. Please check the docs to see if the backend you intend to use is available and how to install it: https://huggingface.co/docs/bitsandbytes/main/en/installationz+No supported devices found for bitsandbytesFT)rr5setr intersection RuntimeErrorrr)raise_exceptionbnb_supported_devicesavailable_deviceserr_msgs& r!validate_bnb_backend_availabilityrLs$C)BCEJ134  ) )*? @ @ EFWF_F_[_E`az{PzQQrr w' 'DE r)NNFrrE)F)#rcore_model_loadingrquantizers.quantizers_utilsrrutilsrrrr r bitsandbytesr5r%torch.nnr' pytorch_utilsr raccelerate.hooksr r get_loggerrGrrrOr`rmrrrrrrrrrs.U&L   H %mD ' 'Fm:''DJ\<"*Zr