+ TV-jz^RIt^RIHt^RIHtHtHt^RIHt ^RI H t ^RI H t ^RIHtHtHtHtHtRRRllt.ROt.ROtRRR lltR R ltR t]R 8Xd]!R4]!4R#R#)N)Path)CallableOptionalUnion)tree_map_with_path)dequantize_modelloadquantize_modelsave upload_to_hubc V^8dQhR\R\PR\R\\\P\ .\ \\ 3,3,/#)recipemodel group_sizereturn)strnnModuleintrdictrbool)formats"_/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx_lm/convert.py __annotate__rsN9!9! 9! 9!/29! sBIIt$eD$J&7789!caaaa a a Ro ^oVR8Xd^o M.VR8Xd^o ^oM"VR8Xd^o MVR8Xd^o M\RV 24hVP4UUu.uFwr4RV9gKVNK ppp\V4^8Xd \R4h\V^,P R44FwopVP 4'gKM \VP 4o R VVVV V V 3R llpV#uuppi) affine mixed_2_6 mixed_3_4 mixed_3_6 mixed_4_6zInvalid quant recipe down_projz2Model does not have expected keys for mixed quant..crV^8dQhR\R\PR\\\ 3,/#)rpathmoduler)rrrrrr)rs"rr3mixed_quant_predicate_builder..__annotate__0s8JJJ J tTz Jrc<\VPR44S8d"\VPR4S,4M^pVS ^,8;'g6V^S ,^,8;'gVS ^,, ^,^8HpRV9gRV9gRV9dV'd RSRSRS/#RV9dV'd RSRSRS/#R V9d RSRSRS/#RSRSRS/#) a%Implements mixed quantization predicates with similar choices to, for example, llama.cpp's Q4_K_M. Ref: https://github.com/ggerganov/llama.cpp/blob/917786f43d0f29b7c77a0c56767c0fa4df68b1c5/src/llama.cpp#L5265 By Alex Barron: https://gist.github.com/barronalex/84addb8078be21969f1690c1454855f3 r$v_projv_a_projv_b_projrbitsmoder#lm_head)lensplitr) r&r'index use_more_bitsr high_bitslayer_locationlow_bitsr. num_layerss && rmixed_quant_predicate.mixed_quant_predicate0s4::c?#n4  3/ 0  J!O # 2 2J!++ 2 2 a'1,1    d 2jD6H *fiN N $ = *fiN N   *fiN Nj&(FDIIr) ValueError named_modulesr0 enumerater1isdigitlayers) rrrk_ down_keysr8r4r5r6r.r7s &&f @@@@@rmixed_quant_predicate_builderrBs DI  ;  ;  ; 09::$224I4tq q8H4II 9~MNN'y|'9'9#'>? 99;; @U\\"JJJ: ! OJs  C2&C2cV^8dQhR\R\R\R\\,R\\,R\R\\,R\R \\,R \R \\\ \\ P\.\\\3,3,\3,,R \/ #) rhf_pathmlx_pathquantize q_group_sizeq_bitsq_modedtype upload_reporevision dequantizequant_predicatetrust_remote_code) rrrrrrrrr)rs"rrrUsZ-Z- Z-Z-Z-3- Z- SM Z-  Z- C=Z-Z-smZ-Z- hRYY-uT4Z/@@A3FGZ-Z-rc aa\V\4'd \V4pVP4'd\ RV R24h\ R4\ VVRRV /RR7wrp\V \4'd VR8wd \ R4h\V V V4p SfVPR R4oSf-VPR R4;p'dVPR R4oS\9d[\ R S4\\S4o\V R R4oVV3RlpV P\VV P444V'dV 'd \ R4hV'd\ R4\V VVVVV R7wrV 'd;\ R4VP!RR4VP!RR4\#V 4p \%VVV V V4Ve\'W4R#R#)zCannot save to the path zY as it already exists. Please delete the file/directory or specify a new path to save to.z[INFO] LoadingTrO)rL return_configtokenizer_configlazyrz4Quant predicates only support 'affine' quantization.N torch_dtype text_configrJz[INFO] Using dtype:cast_predicatecR#)T)r@s&rconvert..sDrc<S!V4'dG\P!VP\P4'dVP S4#V#)N)mx issubdtyperJfloatingastype)r?vrVrJs&&r set_dtypeconvert..set_dtypes9a  R]]177BKK%H%Hxx&rz/Choose either quantize or dequantize, not both.z[INFO] Quantizing)r.rNz[INFO] Dequantizing quantizationquantization_config) isinstancerrexistsr:printr rBgetMODEL_CONVERSION_DTYPESgetattrr\updater parametersr poprr r )rDrErFrGrHrIrJrKrLrMrNrOr tokenizerconfigrUrarVs&&&&&&f&&&&& @rconvertrpUs"(C  >&xj1R R   #-/@A   Ef/3'' X SU U7      } =$/ }M4)HH+H. '' #U+E" (8.I   ' 53C3C3EFGJJKK !"&    +   #$ >4( ($/ '  h,rc8V^8dQhR\P/#)rr)argparseArgumentParser)rs"rrrsKK(11Krc^\P!RR7pVPRR\RR7VPR\RR R 7VPR R R RR7VPRR\RR7VPRR\RR7VPRR\R.R)OR7VPRR\ \RR7VPRR\\ RR 7VPR!R"\RR7VPR#R$R%RRR&7VPR'R(RRR&7V#)*z Configures and returns the argument parser for the script. Returns: argparse.ArgumentParser: Configured argument parser. z(Convert Hugging Face model to MLX format) descriptionz --hf-pathz--modelzSPath to the model. This can be a local path or a Hugging Face Hub model identifier.)typehelpz --mlx-path mlx_modelzPath to save the MLX model.)rvdefaultrwz-qz --quantizezGenerate a quantized model. store_true)rwactionz--q-group-sizezGroup size for quantization.N)rwrvryz--q-bitsz!Bits per weight for quantization.z--q-modezThe quantization mode.r)rwrvrychoicesz--quant-predicatezMixed-bit quantization recipe.F)rwr|rvrequiredz--dtypezvType to save the non-quantized parameters. Defaults to config.json's `torch_dtype` or the current model weights dtype.)rwrvr|ryz --upload-repoz-The Hugging Face repo to upload the model to.z-dz --dequantizezDequantize a quantized model.)rwr{ryz--trust-remote-codez)Trust remote code when loading tokenizer.)rmxfp4nvfp4mxfp8)rrrs add_argumentrr QUANT_RECIPESri)parsers rconfigure_parserrs $ $>F   b   3 :W  l!>|  +     0     % 5  -   F '   <      ,   8  Mrcb\4pVP4p\R/\V4BR#)NrX)r parse_argsrpvars)rargss rmainrs'  F    D d4jr__main__zCalling `python -m mlx_lm.convert ...` directly is deprecated. Use `mlx_lm.convert ...` or `python -m mlx_lm convert ...` instead.)@)rr r!r")float16bfloat16float32) rxFNNrNNNFNF)rrpathlibrtypingrrrmlx.corecorer\mlx.nnr mlx.utilsrutilsrr r r r rBrrirprr__name__rgrXrrrsq,,(9!xE <Z-zK\  z  O F r