+ TV-j|<^RIHt^RIHt^RIHt^RIHt^RI H t H t H t ]!RR] 44t !RR]P4t!R R ]P4t!R R ]P4t!R R]P4t!RR]P4tR#)) dataclassN)swiglu) BaseModelArgscreate_attention_maskscaled_dot_product_attentioncZa]tRt^ toRt^t^t^tRtRt Rt Rt Rt Rt RtV3R ltR tVtR#) ModelArgsii gư>i+TiQNcHVPfVPVnR#R#N)num_key_value_headsnum_attention_headsselfs&c/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx_lm/models/qwen.py __post_init__ModelArgs.__post_init__s!  # # +'+'?'?D $ ,c<V^8dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R &S[;R &#)  model_type hidden_sizer num_hidden_layers kv_channelsmax_position_embeddingslayer_norm_epsilonintermediate_sizeno_bias vocab_size)strintfloatbool)format __classdict__s"r __annotate__ModelArgs.__annotate__ sO!     !'$"r)r )__name__ __module__ __qualname____firstlineno__rr rrrrrrrr r__annotate_func____static_attributes____classdictcell__)r$s@rr r sLK!K#' $"GJ@rr cHaa]tRt^toV3RlV3RlltRRltRtVtV;t#) Attentionc <V^8dQhRS[/#rargsr )r#r$s"rr%Attention.__annotate__ s::Y:rc<\SV`4VPpVPVnW P,p\P !VRR7VnVPVP,p\P!W$^,RR7Vn \P!W$VP'*R7Vn VR,Vn R#)F) traditionalTbiasNg) super__init__rr nnRoPE rotary_embrLinearc_attnrc_projscale)rr2rhidden_size_per_attention_head proj_size __class__s&& rr:Attention.__init__ s && #'#;#; )48P8P)P&''"@eT$$t'?'?? ii ]F ii =MN 3T9 rc VPV4p\P!V^RR7wrVpVPwrp VP WVP R4P ^^^^4p VP WVP R4P ^^^^4p VP WVP R4P ^^^^4p VeMVPWPR7p VPWPR7p VPW4wrM"VPV 4p VPV 4p \WWVPVR7pVP ^^^^4P WR4pVPV4#))axis)offset)cacherAmask) r?mxsplitshapereshaper transposer=rIupdate_and_fetchrrAr@)rxrKrJqkvqkvBL_querieskeysvaluesoutputs&&&& r__call__Attention.__call__1sZkk!n((3+a''a))A$":":B?II!QPQSTUyyt77<FFq!QPQR1!9!92>HHAqRST  ooglloCG??4 ?=D 11$?LD&oog.G??4(D- 6djjt !!!Q1-55aB?{{6""r)r?r@r r=rANN r'r(r)r*r:r_r,r- __classcell__rDr$s@@rr/r/s::"##rr/cDaa]tRt^LtoV3RlV3RlltRtRtVtV;t#)MLPc <V^8dQhRS[/#r1r3)r#r$s"rr%MLP.__annotate__Ms   Y  rc<\SV`4\P!VPVP ^,VP '*R7Vn\P!VPVP ^,VP '*R7Vn\P!VP ^,VPVP '*R7Vn R#)rr7N) r9r:r;r>rrrw1w2r@rr2rDs&&rr: MLP.__init__Ms ))   d449DLL@P ))   d449DLL@P ii  " "a ')9)9DLL@P  rczVPV4pVPV4pVP\W244#r )rjrkr@r)rrSa1a2s&& rr_ MLP.__call__Zs/ WWQZ WWQZ{{6">**r)r@rjrkrbrds@@rrfrfLs   ++rrfcHaa]tRt^`toV3RlV3RlltRRltRtVtV;t#)TransformerBlockc <V^8dQhRS[/#r1r3)r#r$s"rr%TransformerBlock.__annotate__asYrc(<\SV`4\P!VPVP R7Vn\V4Vn\P!VPVP R7Vn \V4Vn R#)epsN) r9r:r;RMSNormrrln_1r/attnln_2rfmlprls&&rr:TransformerBlock.__init__as` JJt//T5L5LM dO JJt//T5L5LM t9rcTpVPV4pVPWVR7pW,pVPV4pVPV4pW,pV#))rKrJ)rzr{r|r})rrSrKrJresiduals&&&& rr_TransformerBlock.__call__isT IIaL IIa%I 0< IIh  HHQK Lr)r{rzr|r}rarbrds@@rrsrs`s  rrscHaa]tRt^utoV3RlV3RlltRRltRtVtV;t#) QwenModelc <V^8dQhRS[/#r1r3)r#r$s"rr%QwenModel.__annotate__vsNNYNrcV<\SV`4\P!VPVP 4Vn\VP4Uu.uFp\V4NK upVn \P!VP VPR7Vn R#uupirw)r9r:r; Embeddingrrwterangerrshryrln_f)rr2rZrDs&& rr:QwenModel.__init__vsw <<1A1AB278N8N2OP2OQ"4(2OPJJt//T5L5LM QsB&cVPV4pVfR.\VP4,p\W2^,4p\ VPV4FwrVV!W4V4pK VP V4#r )rlenrrzipr)rinputsrJrSrKlayercs&&& rr_QwenModel.__call__|sj HHV  =FS[(E$Qa1DFFE*HEaq!A+yy|r)rrrr rbrds@@rrrusNN  rrcdaa]tRt^toV3RlV3RlltRV3RlRllt]R4tRtVt V;t #)Modelc <V^8dQhRS[/#)rconfigr3)r#r$s"rr%Model.__annotate__syrc<\SV`4VPVn\V4Vn\ P !VPVPVP'*R7Vn Wn R#)r7N) r9r:rr transformerr;r>rrrlm_headr2)rrrDs&&rr:Model.__init__sY  ++$V,yy    1 1FNN8J   rcN<V^8dQhRS[PRS[P/#)rrSreturn)rMarray)r#r$s"rr%rs' 88  rcFVPW4pVPV4#r )rr)rrSrJys&&& rr_Model.__call__s"   Q &||Arc.VPP#r )rrrs&rlayers Model.layerss!!!r)r2rrrr ) r'r(r)r*r:r_propertyrr,r-rcrds@@rrrs2"""rr) dataclassesrmlx.corecorerMmlx.nnr; activationsrbaserrrr Moduler/rfrsrrrrrs"TT @ @ @$*# *#Z+"))+(ryy* ("BII"r