+ NV-j9^RIt^RIHtHtHt^RIHt^RIH t ^RI H t Rt RRRRR R R/R R lllt !R R] 4t!RR] 4t!RR] 4tR#)N)CallableOptionalUnion)Module)tree_map_with_pathcZRRRRRRRR/pW0,wrET;'gTT;'gT3#)affinemxfp4nvfp4mxfp8)@) r)r)r)mode group_sizebits mode_defaultsdefault_group_size default_bitss&&& h/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx/nn/layers/quantized.py_defaults_for_moder sG' M (5':$  + ++T-A-A\ AArr quantize_inputFclass_predicatecV^8dQhR\R\R\R\R\R\\ \\.\ \\3,3,,/#)modelrrrrr)rintstrboolrrrdict)formats"r __annotate__r&shI!I! I!I! I!  I!  I!hV}eD$J6G'GHII!rcaaaaaS;'gRoVVVVV3RlpVP4p\Wg\PR7pVP V4R#)a[Quantize the sub-modules of a module according to a predicate. By default all layers that define a ``to_quantized()`` method will be quantized. Both :obj:`Linear` and :obj:`Embedding` layers will be quantized. The module is updated in-place. Note: ``quantize_input=True`` is only supported for ``"nvfp4"`` and ``"mxfp8"`` modes and :obj:`Linear` layers. Args: model (mlx.nn.Module): The model whose leaf modules may be quantized. group_size (Optional[int]): The quantization group size (see :func:`mlx.core.quantize`). Default: ``None``. bits (Optional[int]): The number of bits per parameter (see :func:`mlx.core.quantize`). Default: ``None``. mode (str): The quantization method to use (see :func:`mlx.core.quantize`). Default: ``"affine"``. quantize_input (bool): Whether to quantize activations. Default: ``False``. class_predicate (Optional[Callable]): A callable which receives the :obj:`Module` path and :obj:`Module` itself and returns ``True`` or a dict of params for ``to_quantized`` if it should be quantized and ``False`` otherwise. If ``None``, then all layers that define a ``to_quantized()`` method are quantized. Default: ``None``. Example: Weight only quantization for all layers that define a ``to_quantized()`` method: >>> import mlx.nn as nn >>> nn.quantize(model, group_size=64, bits=4, mode="affine") Weight and input quantization for all linear layers: >>> predicate = lambda p, m: isinstance(m, nn.Linear) >>> nn.quantize(model, mode="nvfp4", quantize_input=True, class_predicate=predicate) c\VR4#) to_quantized)hasattr)_ms&&rquantize..Cs wq.7Qrc<S!W4;p'd\VR4'd\V\4'd(RSRSRS/pS'dSVR&VP!R/VB#\V\4'd:RV9d!VR,'gVP R4VP!R/VB#\ R4h\ R\V4 24hV#)r)rrrrzZ``class_predicate`` must return a bool or a dict of parameters to pass to ``to_quantized``z!Unable to quantize model of type r)r* isinstancer#r)r$pop ValueErrortype) pathr,bool_or_paramskwargsrrrrrs && r_maybe_quantize!quantize.._maybe_quantizeEs,T5 5> 5q.))nd33*JfdSF%3A/0>>3F3355(N:N(EE'**+;<>>;N;;$O !#DT!WI!NOOHr)is_leafN) leaf_modulesrr is_moduleupdate_modules)r rrrrrr7leavess&ffddd rquantizer>sNZ&RR*QO0   !F AQAQ RF  rcxaa]tRt^btoRtR V3RlV3RllltRtRtRt] R V3RlRll4t R t Vt V;t #) QuantizedEmbeddingaThe same as :obj:`Embedding` but with a quantized weight matrix. :obj:`QuantizedEmbedding` also provides a :meth:`from_embedding` classmethod to convert embedding layers to :obj:`QuantizedEmbedding` layers. Args: num_embeddings (int): How many possible discrete tokens can we embed. Usually called the vocabulary size. dims (int): The dimensionality of the embeddings. group_size (Optional[int]): The group size to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. bits (Optional[int]): The bit width to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. mode (str): The quantization method to use (see :func:`mlx.core.quantize`). Default: ``"affine"``. c 8<V^8dQhRS[RS[RS[RS[RS[/#)rnum_embeddingsdimsrrrr!r")r% __classdict__s"rr&QuantizedEmbedding.__annotate__us=     rc<\S V`4\WSV4wVnVnWPn\ P!^V, 4p\PPW3VR7p\P!WsWER7vVn Vn pV'd V^,MRVnWnW nVP#4R#))shapescalerN)super__init__rrrrmathsqrtmxrandomnormalr>weightscalesbiasesrBrCfreeze) selfrBrCrrrrJrSrU __class__s &&&&&& rrMQuantizedEmbedding.__init__us &8$%O"  !d(#!!(>e!L,.KK - ) T[6$*fQit ,  rc VPR4p\P!VR,V,VR,V,Ve W!,MRVPVPVP R7#)rUrSrTN)rTrUrrr)getrP dequantizerrr)rWxrUs&& r__call__QuantizedEmbedding.__call__sZ(#}} N1 >!$ & 269   rc  \P!VVR,VR,VPR4RVPVPVP R7#)z Call the quantized embedding layer as a quantized linear layer. Use this for example when input embedding and output projection weights are tied. rSrTrUTrTrU transposerrrrPquantized_matmulr[rrrrWr]s&&r as_linearQuantizedEmbedding.as_linearsL"" N>88H%  rc VP RVP RVP RVP RVP 2 #)z, , group_size=, bits=, mode=)rBrCrrrrWs&r _extra_reprQuantizedEmbedding._extra_reprsF""#2dii[1//*'$))GDII; P rc2<V^8dQhRS[RS[RS[RS[/#)rembedding_layerrrrrr!r")r%rEs"rr&rFs3   rc VPPwrVV!WVW#VR7p\P!VPVVVR7vVnVnpV'dV^,VnV#RVnV#)zHCreate a :obj:`QuantizedEmbedding` layer from an :obj:`Embedding` layer.rKN)rSrIrPr>rTrU) clsrprrrembedding_dimsrCqlrUs &&&&& rfrom_embedding!QuantizedEmbedding.from_embeddingss /55;; zd C(*  " "   ) % 29v "(F1I  .2  r)rUrrCrrrBrTrSNNr )__name__ __module__ __qualname____firstlineno____doc__rMr^rfrm classmethodrv__static_attributes____classdictcell__ __classcell__rXrEs@@rr@r@bsA$6   $ rr@craa]tRt^toRtR V3RlV3RllltRtRt]R V3RlRll4t Rt Vt V;t #) QuantizedLinearaApplies an affine transformation to the input using a quantized weight matrix. It is the quantized equivalent of :class:`mlx.nn.Linear`. For now its parameters are frozen and will not be included in any gradient computation but this will probably change in the future. :obj:`QuantizedLinear` also provides a classmethod :meth:`from_linear` to convert linear layers to :obj:`QuantizedLinear` layers. Args: input_dims (int): The dimensionality of the input features. output_dims (int): The dimensionality of the output features. bias (bool, optional): If set to ``False`` then the layer will not use a bias. Default: ``True``. group_size (Optional[int]): The group size to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. bits (Optional[int]): The bit width to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. mode (str): The quantization method to use (see :func:`mlx.core.quantize`). Default: ``"affine"``. c ><V^8dQhRS[RS[RS[RS[RS[RS[/#)r input_dims output_dimsbiasrrr)r!r#r")r%rEs"rr&QuantizedLinear.__annotate__sG          rc<\S V`4\WdV4wVnVnW`n\ P!^V, 4p\PPV)VW!3R7p\P!WWVR7vVn Vn p V 'd V ^,MRVnV'd\P!V34VnVP#4R#)rHlowhighrIrKN)rLrMrrrrrNrOrPrQuniformr>rSrTrUzerosrrV) rWrrrrrrrJrSrUrXs &&&&&&& rrMQuantizedLinear.__init__s &8$%O"  !j.)""+#  -/KK - ) T[6$*fQit  +0DI rc VPPwrV^ ,VP,pRV RV RRV9 RVP RVP RVP 2 #)r input_dims=, output_dims=z, bias=rrirjrk)rSrIrrrrWout_dimsin_dimss& rrmQuantizedLinear._extra_reprsm KK--R88H%  T>L Arc2<V^8dQhRS[RS[RS[RS[/#r linear_layerrrrrq)r%rEs"rr&rs3   rc  VPPwrVV!WeRW#VR7p\P!VPVVVR7vVnVnpV'd V^,MRVnRV9dVP VnV#)zACreate a :obj:`QuantizedLinear` layer from a :obj:`Linear` layer.FrKNr)rSrIrPr>rTrUr) rsrrrrrrrurUs &&&&& r from_linearQuantizedLinear.from_linears~#/"5"5";"; % M(*      ) % 29v "(F1IT \ !"''BG r)rrUrrrrTrS)TNNr rx) ryrzr{r|r}rMrmr^r~rrrrrs@@rrrs=,  D  rrcaa]tRtRtoRtRV3RlV3RllltRtRtRtV3RlV3R llt R t ] RV3R lR ll4t R t VtV;t#)QQLineari1aQuantizes the input and applies an affine transformation using quantized weights. Two use cases are supported: 1) **Eval**: The weights are frozen and stored in quantized form together with their scales (``self.weight`` is quantized and ``self.scales`` is provided). 2) **Train**: The weights are stored in higher precision and are quantized on the fly during computation so that gradients with respect to the weights can be computed. To switch between the two cases, use ``layer.eval()`` and ``layer.train()`` respectively. Compared to the :class:`mlx.nn.QuantizedLinear` layer, this layer quantizes the input as well and includes weights in gradient computations. :obj:`QQLinear` also provides the class method :meth:`from_linear` to convert :class:`mlx.nn.Linear` layers to :obj:`QQLinear` layers. Note: This layer does not support a bias term yet. Args: input_dims (int): The dimensionality of the input features. output_dims (int): The dimensionality of the output features. group_size (Optional[int]): The group size to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. bits (Optional[int]): The bit width to use for the quantized weight. See :func:`~mlx.core.quantize`. Default: ``None``. mode (Optional[str]): The quantization method to use (see :func:`mlx.core.quantize`). Currently, only ``"nvfp4"`` and ``"mxfp8"`` are supported. Default: ``"nvfp4"``. c 8<V^8dQhRS[RS[RS[RS[RS[/#)rrrrrrrD)r%rEs"rr&QQLinear.__annotate__Rs=          rc<\SV`4\WSV4wVnVnWPn\ P!^V, 4p\PPV)VW!3R7Vn RVn R#)rHrFN) rLrMrrrrrNrOrPrQrrS _quantized)rWrrrrrrJrXs&&&&&& rrMQQLinear.__init__Rsp &8$%O"  !j.)ii''+(   rc VPPwrVPP\P8XdV^ ,VP ,pRV RV RVP RVP RVP 2 #)rrrrirjrk)rSrIdtyperPuint32rrrrs& rrmQQLinear._extra_reprhsw KK-- ;;   )| 1G'. ;//*'$))GDII; P rcVP'gY\P!VPVPVP VP R7wVnVnRVnR#R#)rKTN)rrPr>rSrrrrTrls&rr>QQLinear.quantizeqsL'){{  YY ( $DK #DOrcVP'dm\P!VPVPVP VP VPR7VnVPR4RVnR#R#)rTrrrrTFN) rrPr\rSrTrrr __delattr__rls&rr\QQLinear.dequantize{sZ ???-- {{??YYYY DK   X &#DO rc <V^8dQhRS[/#)rr)r#)r%rEs"rr&rstrc<\SV`V4VP'dVP4R#VP 4R#)N)rL_set_training_mode _trainingr\r>)rWrrXs&&rrQQLinear._set_training_modes, "4( >>> OO  MMOrc \P!VVR,VPR4VPVPVP R7pV#)rSrTr)rPqqmmr[rrrres&&rr^QQLinear.__call__sB GG N88H%  rc2<V^8dQhRS[RS[RS[RS[/#rrq)r%rEs"rr&rs3   rc VPPwrVVPR4e \R4hV!WeW#VR7pVPVnVP VP 4V#)z:Create a :obj:`QQLinear` layer from a :obj:`Linear` layer.rz#QQLinear does not support bias yet.rK)rSrIr[NotImplementedErrortraintraining)rsrrrrrrrus&&&&& rrQQLinear.from_linearsg#/"5"5";";   F # /%&KL L * F ''  &&' r)rrrrrTrS)NNr )ryrzr{r|r}rMrmr>r\rr^r~rrrrrs@@rrr1sQ@  , # $ rr)NN)rNtypingrrrmlx.corecorerPmlx.nn.layers.baser mlx.utilsrrr>r@rrrrrrs{ ,,%(BI!  I! ! I!MQI!I!XccLfffRyvyr