+ QV-j/^RIt^RIt^RIHt^RIHt^RItR]PPPR]PPPR]PPPR]PPPR]PPPR ]PPPR ]PPPR ]PPP R ]PPP"R ]PPP$R]PPP&R]PPP(R]PPP*R]PPP,/tR7RRllt R7RRllt RRlt RRlt RRlt RRltR8RRlltR9R R!lltR9R"R#lltR:R$R%lltR:R&R'lltR;R(R)lltRR0ltR1tR2tR?t]R34t]R44t]R54t]R64t R#)@N) defaultdict)contextmanageruniform_normal_ constant_ones_zeros_eye_dirac_xavier_uniform_xavier_normal_kaiming_uniform_kaiming_normal_ trunc_normal_ orthogonal_sparse_c V^8dQhR\PR\R\R\PR,R\P/#)tensorab generatorNreturntorchTensorfloat Generator)formats"l/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/transformers/initialization.py __annotate__r!*sF LL"-2EJ__W[E[ \\cV\VRR4'g\R,!WW#R7#V#)_is_hf_initializedFr)rrrgetattrTORCH_INIT_FUNCTIONS)rrrrs&&&&r rr*s+ 6/ 7 7#J/qVV Mr"c V^8dQhR\PR\R\R\PR,R\P/#)rrmeanstdrNrr)rs"r r!r!2sF LL %27JO//\`J` \\r"cV\VRR4'g\R,!WW#R7#V#)r$Fr)r)r*rr%)rr)r*rs&&&&r rr2s+ 6/ 7 7#I.vc__ Mr"cdV^8dQhR\PR\R\P/#)rrvalr)rrr)rs"r r!r!:s)ell5<<r"cT\VRR4'g\R,!WR7#V#)r$Fr)r-r%)rr-s&&r rr:s' 6/ 7 7#K0AA Mr"cXV^8dQhR\PR\P/#rrrrr)rs"r r!r!@s"%,,5<<r"cR\VRR4'g\R,!V4#V#)r$Frr%rs&r rr@s' 6/ 7 7#G,V44 Mr"cXV^8dQhR\PR\P/#r0r1)rs"r r!r!Fs"5<<ELLr"cR\VRR4'g\R,!V4#V#)r$Fr r%r3s&r r r Fs' 6/ 7 7#H-f55 Mr"cXV^8dQhR\PR\P/#r0r1)rs"r r!r!Ls"%,,r"cR\VRR4'g\R,!V4#V#)r$Fr r%r3s&r r r Ls' 6/ 7 7#F+F33 Mr"cdV^8dQhR\PR\R\P/#)rrgroupsr)rrint)rs"r r!r!Rs)5<<U\\r"cT\VRR4'g\R,!WR7#V#)r$Fr )r9r%)rr9s&&r r r Rs' 6/ 7 7#H-fDD Mr"cV^8dQhR\PR\R\PR,R\P/#rrgainrNrr)rs"r r!r!Xs=ELLZ^H^jojvjvr"cV\VRR4'g\R,!WVR7#V#)r$Fr r>rr%rr>rs&&&r r r Xs+ 6/ 7 7#$56vT]^^ Mr"cV^8dQhR\PR\R\PR,R\P/#r=r)rs"r r!r!^s=5<<uuY]G]iniuiur"cV\VRR4'g\R,!WVR7#V#)r$Fr r@r%rAs&&&r r r ^s+ 6/ 7 7#$45fS\]] Mr"c V^8dQhR\PR\R\R\R\PR,R\P/#rrrmode nonlinearityrNrrrrstrr)rs"r r!r!dU   LL       %   \\ r"cX\VRR4'g\R,!WW#VR7#V#)r$FrrrFrGrr%rrrFrGrs&&&&&r rrds5 6/ 7 7#$67 d   Mr"c V^8dQhR\PR\R\R\R\PR,R\P/#rErH)rs"r r!r!rrJr"cX\VRR4'g\R,!WW#VR7#V#)r$FrrLr%rMs&&&&&r rrrs5 6/ 7 7#$56 d   Mr"cV^8dQhR\PR\R\R\R\R\PR,R\P/#) rrr)r*rrrNrr)rs"r r!r!s_   LL         %  \\ r"c X\VRR4'g\R,!WW#WER7#V#)r$Fr)r)r*rrrr%)rr)r*rrrs&&&&&&r rrs. 6/ 7 7#O4VCXYoo Mr"cV^8dQhR\PR\R\PR,R\P/#r=r)rs"r r!r!sA LL % \\ r"cV\VRR4'g\R,!WVR7#V#)r$Frr@r%rAs&&&r rrs, 6/ 7 7#M26PYZZ Mr"c V^8dQhR\PR\R\R\PR,R\P/#)rrsparsityr*rNrr)rs"r r!r!sF LL$)05IN[_I_ \\r"cV\VRR4'g\R,!WW#R7#V#)r$Fr)rUr*rr%)rrUr*rs&&&&r rrs+ 6/ 7 7#I.vcgg Mr"cxV^8dQhR\PR\PR\P/#)rrotherrr1)rs"r r!r!s-%,,u|| r"c\VRR4'g:\P!4;_uu_4VPV4uuRRR4#V# +'giT#;i)r$FN)r&rno_gradcopy_)rrXs&&r r[r[sB 6/ 7 7 ]]__<<&_ M_ Ms A A c\PPPV4wr4VR8XdTpM VR8XdTpMVR8XdW4,^, pRX, pVR8Xd+\ V\ P !V4R, R7R #VR8Xd$\V\ P !V4R7R #VR8Xd-\ P !^V,4p\W)V4R #\R V 24h) fan_infan_outfan_avg?truncated_normalg۶%?)r*normaluniformzinvalid distribution N) rnninit_calculate_fan_in_and_fan_outrmathsqrtrr ValueError)rrF distributionr]r^denomvariancebounds&&& r _variance_scalingrnshhmmAA&IOF x    !Q&U{H))f$))H"58K"KL  !DIIh/0  " !h,''0?@@r"cH\VRR4'g\VRRR7V#)r$Fr]rarFrjr&rnr3s&r lecun_normal_rrs$ 6/ 7 7&x>PQ Mr"cH\VRR4'g\VRRR7V#)r$Fr]rbrprqr3s&r default_flax_embed_init_rts# 6/ 7 7&xhG Mr"c #h"\\4p\FpV\P9gK\PV,p\ P 4FEp\W#4'gK\W#4W,V&\W#\4V,4KG K RxVP4F*wr$VP4Fwr5\W#V4K K, R# TP4F*wr$TP4Fwr5\Y#T4K K, i;i5i)av Guard the `torch.nn.init` primitive functions to behave exactly like the functions in this file, i.e. be protected against the `_is_hf_initialized` flag to avoid re-init if the param was already loaded. Usually, all models are using the init from `transformers` which are already guarded, but just to make extra sure and for remote code, we also use this context manager. N) rdictTORCH_MODULES_TO_PATCHsysmodulesr'keyshasattrr&setattrglobalsitems) originals module_namemodule func_name functionsfuncs r guard_torch_init_functionsrsD!I11Kckk)[1!5!:!:!v7Q ))479Y3GH"=2 "+!2 F#,??#4 40$5"3!2 F#,??#4 40$5"3s)D2C/.empty_func r"N)modeling_utilsrrrvrwrxryr'rzr{r&r| init_weightsr~) rrrrrroriginal_init_weightsrrs r no_init_weightsrs0 D!I=1Kckk)[1!5!:!:!v7Q ))4:>"=2!0 < <'1$ "+!2 F#,??#4 40$5"3(=$ "+!2 F#,??#4 40$5"3(=$s)E DD;AE AEE c#r"^RIHpRpVPpWnRxW nR# XTni;i5i)a Disable weight tying during loading with `from_pretrained`. This is needed as we want to have access to ALL weights in the state_dict during `from_pretrained`, and otherwise tying them would remove them from it, as it's called in `post_init` when instantiating. rcR#rrrs*,r r"no_tie_weights..empty_func$rr"N)rr tie_weights)rroriginal_tie_weightss r no_tie_weightsrs:0 ;.::&0# ';#&:#s 7+7 47c#a"\PoV3RlpV\nRxS\nR# S\ni;i5i)av During meta-device model initialisation, ``torch.linspace`` produces meta tensors that have no data. Custom models loaded from the Hub (remote code) often call ``.item()`` on these tensors to compute scalar hyperparameters (e.g. stochastic-depth / drop-path schedules). Native transformers models already pass ``device="cpu"`` explicitly for such calls (see e.g. ``modeling_swin.py``, ``modeling_pvt_v2.py``), but remote-code models written before v5 do not. This context manager patches ``torch.linspace`` to default to ``device="cpu"`` when no explicit device is requested, matching the best practice already used throughout transformers. Calls that supply an explicit ``device`` argument (e.g. ``device=self.logits.device``) are left untouched. ``torch.arange`` is intentionally NOT patched because it is used in RoPE computations where the device must match model parameters. c8<VPRR4S!V/VB#)devicecpu) setdefault)rroriginal_linspaces*,r _safe_linspace5meta_device_safe_creation_ops.._safe_linspaceEs#(E* $1&11r"N)rlinspace)rrs @r meta_device_safe_creation_opsr1s7$2$EN+ **s"A7 A AA)r`N))r`N)rr] leaky_reluN)rr`gg@N)rN)g{Gz?N)r]rb) z torch.nn.initztorch.nn.modules.activationztorch.nn.modules.transformerztorch.nn.modules.linearztorch.nn.modules.lossztorch.nn.modules.batchnormztorch.nn.modules.convztorch.nn.modules.normalizationztorch.nn.modules.rnnztorch.nn.modules.sparse)!rgrx collectionsr contextlibrrrdrerrrrr r r r r rrrrrr'r[rnrrrtrwrrrrrr"r rs #%  && uxx}}$$(( UXX]]  ehhmm"" EHHMM   ehhmm""uxx}}44ehhmm22 66uxx}}44UXX]]00588==,, uxx}}$$$         A,  114!=!=H;;*++r"