+ NV-j?P ^RIt^RIHt^RIHt^RIHt^RIH t Rt ]!]PRR7R4t ]!]PRR7R 4t ]!]PRR7R 4t]!]PRR7R 4t]!]PRR7R[R l4t]!]PRR7R\R l4t]!]PRR7R]Rl4t]!]PRR7R\Rl4t]!]PRR7R4t]!]PRR7R4t]!]PRR7R^RRll4t]!]PRR7R]Rl4t]!]PRR7R4t]!]PRR7R4t]!]PRR7RRl4t]!]PRR7R4t]!]PRR7R4tR\RRllt]!]PRR7R_RRll4t]!]PRR7R4t]!]PRR7R R!l4t ]!]PRR7R"R#l4t!]!]PRR7R$4t"]!]PRR7R`R%l4t#]!]PRR7R^R&l4t$]!]PRR7R\R'l4t%R(t&!R)R*] 4t'] !] 4!R+R,] 44t(] !]!4!R-R.] 44t)] !] 4!R/R0] 44t*] !]4!R1R2] 44t+] !]4!R3R4] 44t,!R5R6] 4t-!R7R8] 4t.] !]4!R9R:] 44t/] !]4!R;R<] 44t0] !]4!R=R>] 44t1!R?R@] 4t2!RARB] 4t3] !]4!RCRD] 44t4] !]4!RERF] 44t5] !]4!RGRH] 44t6!RIRJ] 4t7!RKRL] 4t8] !]&4!RMRN] 44t9] !]"4!RORP] 44t:!RQRR] 4t;] !]4!RSRT] 44t<] !]#4!RURV] 44t=] !]$4!RWRX] 44t>] !]%4!RYRZ] 44t?R#)aN)partialAny)ModulecaV3RlpV#)c<V3RlVnV#)c<S!V4#N)_xfs&&j/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx/nn/layers/activations.py<_make_activation_module..decorator.. s ad)__call__)klassrs&r decorator*_make_activation_module..decorator s* rr )rrsf r_make_activation_moduler s rT) shapelessc.\P!V4#)zdApplies the sigmoid function. .. math:: \text{Sigmoid}(x) = \sigma(x) = \frac{1}{1 + \exp(-x)} mxsigmoidr s&rrrs ::a=rc0\P!V^4#)zAApplies the Rectified Linear Unit. Simply ``mx.maximum(x, 0)``. rmaximumrs&rrelur!s ::a rcX\P!\P!V^44#)uTApplies the ReLU² activation function. Applies :math:`\max(0, x)^2` element wise. )rsquarer rs&rrelu2r$&s 99RZZ1% &&rcZ\P!\P!V^4R4#)zXApplies the Rectified Linear Unit 6. Applies :math:`\min(\max(x, 0), 6)` element wise. g@rminimumr rs&rrelu6r(/s ::bjjA& ,,rc<\P!W,V4#)zXApplies the Leaky Rectified Linear Unit. Simply ``mx.maximum(negative_slope * x, x)``. r)r negative_slopes&&r leaky_relur+8s ::n(! ,,rc@V\P!WRR7, #)zYApplies the Log Softmax function. Applies :math:`x + \log \sum_i e^{x_i}` element wise. T)axiskeepdims)r logsumexpr r-s&&r log_softmaxr1As r||A48 88rcz\P!V^8W\P!V4^, ,4#)z^Applies the Exponential Linear Unit. Simply ``mx.where(x > 0, x, alpha * (mx.exp(x) - 1))``. )rwhereexpr alphas&&relur7Js* 88AE1rvvay1}5 66rc0\P!WR7#)z\Applies the Softmax function. Applies :math:`\frac{e^{x_i}}{\sum_j e^{x_j}}` element wise. r-rsoftmaxr0s&&rr;r;Ss ::a ##rc0\P!V^4#)zPApplies the Softplus function. Applies :math:`\log(1 + \exp(x))` element wise. )r logaddexprs&rsoftplusr>\s <<1 rcf\P!V^\P!V4,4#)zPApplies the Softsign function. Applies :math:`\frac{x}{1 + |x|}` element wise. )rdivideabsrs&rsoftsignrBes! 99QBFF1I &&rc$V^8dQhR\/#)lambdfloat)formats"r __annotate__rIos B B Brc\P!\P!V4V8V\P!V4V,, ^4#)zApplies the Softshrink activation function. .. math:: \text{softshrink}(x) = \begin{cases} x - \lambda & \text{if } x > \lambda \\ x + \lambda & \text{if } x < -\lambda \\ 0 & \text{otherwise} \end{cases} )rr3rAsignr rEs&&r softshrinkrMns6 88BFF1I%q2771:+='=q AArc\P!VR4V\P!\P!VR4V, 4^, ,,#)zApplies the Continuously Differentiable Exponential Linear Unit. Applies :math:`\max(0, x) + \min(0, \alpha * (\exp(x / \alpha) - 1))` element wise. )rr r4r'r5s&&rcelurP|s= ::a  1c0BU0J)Ka)O P PPrc<V\P!V4,#)zApplies the Sigmoid Linear Unit. Also known as Swish. Applies :math:`x \sigma(x)` element wise, where :math:`\sigma(\cdot)` is the logistic sigmoid. rrs&rsilurRs rzz!} rc\V)4)#)zeApplies the Log Sigmoid function. Applies :math:`\log(\sigma(x)) = -\log(1 + e^{-x})` element wise. )r>rs&r log_sigmoidrTs aRL=rc8V^8dQhR\P/#rDreturnrarray)rHs"rrIrIs 2 2rxx 2rcV^\P!V\P!^4, 4,,^, #)zApplies the Gaussian Error Linear Units function. .. math:: \textrm{GELU}(x) = x * \Phi(x) where :math:`\Phi(x)` is the Gaussian CDF. See also :func:`gelu_approx` and :func:`gelu_fast_approx` for faster approximations. )rerfmathsqrtrs&rgelur^s/ BFF1tyy|+,, - 11rc RV,^\P!\P!^\P, 4VRV^,,,,4,,#)a\An approximation to Gaussian Error Linear Unit. See :func:`gelu` for the exact computation. This function approximates ``gelu`` with a maximum absolute error :math:`< 0.0005` in the range :math:`[-6, 6]` using the following .. math:: x = 0.5 * x * \left(1 + \text{Tanh}\left((\sqrt{2 / \pi} * \left(x + 0.044715 * x^3\right)\right)\right) ?gHm?)rtanhr\r]pirs&r gelu_approxrcsD 7a"''$))AK"8A1a4 0 \\ \lambda \alpha (\exp(x) - 1) & \text{if } x \leq 0 \end{cases} where :math:`\lambda = 1.0507` and :math:`\alpha = 1.67326`. See also :func:`elu`. gGG?g䃞ͪ?)r7rs&rselurus q'?V ##rcxV^8dQhR\PR\PR\P/#)rDr r6rWrX)rHs"rrIrIs-77RXX7bhh72887rcv\P!^V4V\P!^V4,,#)zApplies the element-wise parametric ReLU. .. math:: \text{PReLU}(x) = \max(0,x) + a * \min(0,x) where :math:`a` is an array. rr r'r5s&&rprelurys* ::a ebjjA&66 66rcXV^8dQhR\PR\P/#)rDr rWrX)rHs"rrIrIs" $ $BHH $ $rcNV\P!\V44,#)zApplies the Mish function, element-wise. Mish: A Self Regularized Non-Monotonic Neural Activation Function. Reference: https://arxiv.org/abs/1908.08681 .. math:: \text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x)) )rrar>rs&rmishr|s rwwx{# ##rc\P!V^,^4pV\P!V^4,^, #)zsApplies the hardswish function, element-wise. .. math:: \text{Hardswish}(x) = x * \min(\max(x + 3, 0), 6) / 6 rx)r max_x_3s& r hardswishrs3jjQ"G rzz'1% % ))rcX\P!\P!W4V4#)zrApplies the HardTanh function. Applies :math:`\max(\min(x, \mathrm{max\_val}), \mathrm{min\_val})` element-wise. r&)r min_valmax_vals&&&r hard_tanhr*s ::bjj,g 66rc`\P!\P!V4V8V^4#)zApplies the HardShrink activation function. .. math:: \text{hardshrink}(x) = \begin{cases} x & \text{if } x > \lambda \\ x & \text{if } x < -\lambda \\ 0 & \text{otherwise} \end{cases} )rr3rArLs&&r hard_shrinkr3s$ 88BFF1I%q! ,,rc4\P!V)VR7#)z^Applies the Softmin function. Applies :math:`\frac{e^{-x_i}}{\sum_j e^{-x_j}}` element-wise. r9r:r0s&&rsoftminrAs ::qbt $$rc.\P!V4#)zAApplies the hyperbolic tangent function. Simply ``mx.tanh(x)``. )rrars&rraraJs 771:rcXaa]tRtRtoRtRV3RlV3RllltV3RlRltRtVtV;t #) GLUiRric <V^8dQhRS[/#)rDr-)rg)rH __classdict__s"rrIGLU.__annotate___sSrc0<\SV`4WnR#r )super__init__r-)selfr- __class__s&&rr GLU.__init___s  rc <V^8dQhRS[/#rVr)rHrs"rrIrcs((S(rc.\WPR7#)r0)rnr-rr s&&rr GLU.__call__csQYY''rr9 __name__ __module__ __qualname____firstlineno____doc__rr__static_attributes____classdictcell__ __classcell__rrs@@rrrRs# (((rrc]tRtRtRtRtR#)SigmoidigzrApplies the sigmoid function, element-wise. .. math:: \text{Sigmoid}(x) = \sigma(x) = \frac{1}{1 + \exp(-x)} r Nrrrrrrr rrrrgrrc]tRtRtRtRtR#)MishipzApplies the Mish function, element-wise. Reference: https://arxiv.org/abs/1908.08681 .. math:: \text{Mish}(x) = x * \text{Tanh}(\text{Softplus}(x)) r Nrr rrrrpsrrc]tRtRtRtRtR#)ReLUi|zuApplies the Rectified Linear Unit. Simply ``mx.maximum(x, 0)``. See :func:`relu` for the functional equivalent. r Nrr rrrr|rrrc]tRtRtRtRtR#)ReLU2iuZApplies the ReLU² activation function. See :func:`relu2` for the functional equivalent. r Nrr rrrrrrc]tRtRtRtRtR#)ReLU6izWApplies the Rectified Linear Unit 6. See :func:`relu6` for the functional equivalent. r Nrr rrrrrrrc@aa]tRtRtoRtRV3RlltRtRtVtV;t #) LeakyReLUizApplies the Leaky Rectified Linear Unit. Simply ``mx.maximum(negative_slope * x, x)``. Args: negative_slope: Controls the angle of the negative slope. Default: ``1e-2`` c0<\SV`4WnR#r )rr_negative_slope)rr*rs&&rrLeakyReLU.__init__s -rc,\WP4#r )r+rrs&&rrLeakyReLU.__call__s!1122r)rg{Gz?rrs@@rrrs.33rrc@aa]tRtRtoRtRV3RlltRtRtVtV;t #)ELUizApplies the Exponential Linear Unit. Simply ``mx.where(x > 0, x, alpha * (mx.exp(x) - 1))``. See :func:`elu` for the functional equivalent. Args: alpha: the :math:`\alpha` value for the ELU formulation. Default: ``1.0`` c0<\SV`4WnR#r rr_alpharr6rs&&rr ELU.__init__  rc,\WP4#r )r7rrs&&rr ELU.__call__s1kk""rr?rrs@@rrrs##rrc]tRtRtRtRtR#)SoftmaxizRApplies the Softmax function. See :func:`softmax` for the functional equivalent. r Nrr rrrrrrrc]tRtRtRtRtR#)SoftplusizTApplies the Softplus function. See :func:`softplus` for the functional equivalent. r Nrr rrrrrrrc]tRtRtRtRtR#)SoftsignizTApplies the Softsign function. See :func:`softsign` for the functional equivalent. r Nrr rrrrrrrc@aa]tRtRtoRtRV3RlltRtRtVtV;t #) SoftshrinkizApplies the Softshrink function. See :func:`softshrink` for the functional equivalent. Args: lambd: the :math:`\lambda` value for Softshrink. Default: ``0.5`` c0<\SV`4WnR#r )rrrE)rrErs&&rrSoftshrink.__init__s  rc,\WP4#r )rMrErs&&rrSoftshrink.__call__s!ZZ((r)rEr`rrs@@rrrs))rrc@aa]tRtRtoRtRV3RlltRtRtVtV;t #)CELUia$Applies the Continuously Differentiable Exponential Linear Unit. Applies :math:`\max(0, x) + \min(0, \alpha * (\exp(x / \alpha) - 1))` element wise. See :func:`celu` for the functional equivalent. Args: alpha: the :math:`\alpha` value for the CELU formulation. Default: ``1.0`` c0<\SV`4WnR#r rrs&&rr CELU.__init__rrc,\WP4#r )rPrrs&&rr CELU.__call__sA{{##rrrrrs@@rrrs$$rrc]tRtRtRtRtR#)SiLUizgApplies the Sigmoid Linear Unit. Also known as Swish. See :func:`silu` for the functional equivalent. r Nrr rrrrrrrc]tRtRtRtRtR#) LogSoftmaxizZApplies the Log Softmax function. See :func:`log_softmax` for the functional equivalent. r Nrr rrrrrrrc]tRtRtRtRtR#) LogSigmoidizZApplies the Log Sigmoid function. See :func:`log_sigmoid` for the functional equivalent. r Nrr rrrrrrrcLaa]tRtRtoRtRV3RlltV3RlRltRtVtV;t #)PReLUi a?Applies the element-wise parametric ReLU. Applies :math:`\max(0, x) + a * \min(0, x)` element wise, where :math:`a` is an array. See :func:`prelu` for the functional equivalent. Args: num_parameters: number of :math:`a` to learn. Default: ``1`` init: the initial value of :math:`a`. Default: ``0.25`` c^<\SV`4\P!V.V4VnR#r )rrrfullweight)rnum_parametersinitrs&&&rrPReLU.__init__s# gg~.5 rc4<V^8dQhRS[P/#rDr rX)rHrs"rrIPReLU.__annotate__s%%"((%rc,\WP4#r )ryrrs&&rrPReLU.__call__sQ $$r)r)g?rrs@@rrr s 6%%%rrc@aa]tRtRtoRtRV3RlltRtRtVtV;t #)GELUi aApplies the Gaussian Error Linear Units. .. math:: \textrm{GELU}(x) = x * \Phi(x) where :math:`\Phi(x)` is the Gaussian CDF. However, if ``approx`` is set to 'precise' or 'fast' it applies .. math:: \textrm{GELUApprox}(x) &= 0.5 * x * \left(1 + \text{Tanh}\left((\sqrt{2 / \pi} * \left(x + 0.044715 * x^3\right)\right)\right) \\ \textrm{GELUFast}(x) &= x * \sigma\left(1.702 * x\right) respectively. .. note:: For compatibility with the PyTorch API, 'tanh' can be used as an alias for 'precise'. See :func:`gelu`, :func:`gelu_approx` and :func:`gelu_fast_approx` for the functional equivalents and information regarding error bounds. Args: approx ('none' | 'precise' | 'fast'): Which approximation to gelu to use if any. ch<\SV`4Wn.ROpW9d\RV RV R24hR#)nonezThe approximation should be in z but 'z ' was givenN)rpreciserafast)rr_approx ValueError)rapproxallowedrs&& rr GELU.__init__<sB  5  1'& T  !rcVPR8Xd \V4#VPR9d \V4#\V4#)r)rra)rr^rcrers&&rr GELU.__call__Es9 <<6 !7N \\0 0q> !""r)r)rrrs@@rrr s6##rrc]tRtRtRtRtR#)TanhiMzZApplies the hyperbolic tangent function. See :func:`tanh` for the functional equivalent. r Nrr rrrrMrrrc]tRtRtRtRtR#) HardswishiUzdApplies the hardswish function, element-wise. See :func:`hardswish` for the functional equivalent. r Nrr rrrrUrrrcXaa]tRtRtoRtRV3RlV3RllltV3RlRltRtVtV;t #) Stepi]rrc <V^8dQhRS[/#)rDrprF)rHrs"rrIStep.__annotate__ms##%#rc0<\SV`4WnR#r )rrrp)rrprs&&rr Step.__init__ms "rc4<V^8dQhRS[P/#rrX)rHrs"rrIrqs''"(('rc,\WP4#r )rsrprs&&rr Step.__call__qsA~~&&r)rprOrrs@@rrr]s# ##'''rrc]tRtRtRtRtR#)SELUiuz]Applies the Scaled Exponential Linear Unit. See :func:`selu` for the functional equivalent. r Nrr rrr r urrr c]tRtRtRtRtR#)HardTanhi}zUApplies the HardTanh function. See :func:`hard_tanh` for the functional equivalent. r Nrr rrr r }rrr c]tRtRtRtRtR#) HardShrinkizApplies the HardShrink function. See :func:`hard_shrink` for the functional equivalent. Args: lambd: the :math:`\lambda` value for Hardshrink. Default: ``0.5`` r Nrr rrr r srr c]tRtRtRtRtR#)SoftminizRApplies the Softmin function. See :func:`softmin` for the functional equivalent. r Nrr rrrrrrrrrrrr)gr)@r\ functoolsrtypingrmlx.corecorermlx.nn.layers.baserrcompilerr!r$r(r+r1r7r;r>rBrMrPrRrTr^rcrernrsruryr|rrrrrarrrrrrrrrrrrrrrrrrrrrr r r rr rrrs % t$% t$% t$'%' t$-%- t$-%- t$9%9 t$7%7 t$$%$ t$% t$'%' t$ B% B t$Q%Q t$% t$% t$ 2% 2 t$ S% S  t$%%%*   t$)%)& t$ $% $  t$7%7 t$ $% $ t$*%* t$7%7 t$ -% - t$%%%(&(*!f"66F F 33"#&#$!f""v#"v#))"$6$&6%&%&%F%(*#6*#Z6#$'6'06#v$%&!f"r