+ QV-jiLaRktS0t^RIt^RIt^RIt^RIt^RIHt^RIH t ^RI H t ^RI H t ^RIHt^RIHt^RIHtHt^R IHt^R IHtHt^R IHtHt^R IHt]'d^R I H!t!^RI"H#t#^RI$H%t%]!4'd ^RI&t&^RI'H(t(]PR!]*4t+^RI,H-t-H.t.H/t/H0t0H1t1H2t2^RI,H3t4^RI,H5t6^RI,H7t7^RI,H9t:Rt;]PxP{RR4P}4t?Rt@]?]9tARRlt5RRlt9RR].!RRR 7/R!R].!R"R!R 7/R#R]0P].!R$R%R 7/R&]0P].!R$R%R 7/R']0P].!R(R#R 7/R)]0P].!R*R#R 7/R+]0P].!R$R%R 7//R,R].!R-R.R 7/R/R]0P].!R0R/R 7]0P].!R0R/R 7/R&]0P].!R1R/R 7/R']0P].!R0R/R 7/R2]0P].!R0R3R 7//R4R]0P]0P,].!R5R4^R67//R7R]0P]0P,].!R5R7^R67//R8R]0P]0P,].!R5R8^R67//R9R]0P]0P,].!R5R:^R67//R;R]0P]0P,].!R5R<^R67//R=R]0P]0P,].!R5R=^R67/// tE]^k]7eFR']0P]7!R>R?R@7/R]0P]7!R>R?R@7]0P]7!R>R?R@7//]ERA&RBtFRlRCltGRMRNRORP^/RQRNRRRP^/RSRNRRRP^/RTRNRURP^/RVRNRWRP^/RXRNRYRZR[//tH]^k/tI]^kR\R]ltJRmR^R_lltK]I3R`RalltLRnRbRcllt3RdReltMRsN] Rf4tORgRhltPRiRjltQ.RoOtRR# ]8dRt7ELi;i ]8dRt;ELi;i ]8d+Rt@RtARDt5REt9!RFRG4t.!RHRI4t/RJt2RKt1RLtGLi;i)pN)Callable)contextmanager)Path) ModuleType) TYPE_CHECKING)version)!get_checkpoint_conversion_mapping®ister_checkpoint_conversion_mapping)register_patch_mapping)ENV_VARS_TRUE_VALUESlogging)is_kernels_availableis_torch_available)flash_attention_forward)PretrainedConfig)PreTrainedModel) KernelConfig)DeviceLayerRepositoryLocalLayerRepositoryModeregister_kernel_mappingreplace_kernel_forward_from_hub) get_kernel)use_kernel_forward_from_hub)FuncRepository)use_kernel_func_from_hubTFUSE_HUB_KERNELSYESc$V^8dQhR\/#) layer_namestr)formats"v/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/transformers/integrations/hub_kernels.py __annotate__r'Os###cn\'d \V4#\PR\ 24R#)Fkernels hub usage is disabled through the environment USE_HUB_KERNELS=cV#Nclss&r&-use_kernel_forward_from_hub..Vssr()_kernels_enabled$_kernels_use_kernel_forward_from_hublogger warning_once_TRANSFORMERS_USE_HUB_KERNELS)r"s&r&rrOs5  7 C C   XYvXwx # "r(c$V^8dQhR\/#)r! func_namer#)r%s"r&r'r'Xs % %C %r(c\'d\'d \V4#\'g\P R4R#\P R\ 24R#)zwuse_kernel_func_from_hub is not available in the installed kernels version. Please upgrade kernels to use this feature.r*cV#r,r-funcs&r&r0*use_kernel_func_from_hub..esr()r2_has_use_kernel_func_from_hub!_kernels_use_kernel_func_from_hubr4r5r6)r8s&r&rrXs_   = =4Y? ?00##B% $##\]z\{|% $r(MultiScaleDeformableAttentioncudaz!kernels-community/deformable-detrrepo_idr" Llama4TextMoezkernels-community/moeRMSNormzkernels-community/liger_kernels LigerRMSNormrocmxpuzkernels-community/rmsnormmpszkernels-community/mlx_rmsnormnpuMLPzmedmekk/triton-llama-mlpTritonLlamaMLPMegaBlocksMoeMLPzkernels-community/megablockszahadnagy/megablockscpuCPUMegaBlocksMoeMLPFastGELUzkernels-community/activation)rCr"r QuickGELUNewGELUSiLUSiluGeLUGeluGeluTanhzkernels-community/rotaryapply_rotary_transformers)rCr8rotary_pos_embcaSV9;'gW\;QJd-V3RlVP44F 'gK R# R#!V3RlVP444#)c3l<"TF)p\V\4;'d \VS4xK+ R#5ir,) isinstancedicthas_key).0vkeys& r& has_key..s+[PZ1z!T2FFwq#FPZs44TF)anyvalues)dras&fr&r^r^sLax[[33[PQPXPXPZ[33[3[3[PQPXPXPZ[[[r(cVf\p\VR4'd\RR7'g \R4h\ V4R#)NrHz0.10.2) MIN_VERSIONzfkernels uses an incompatible version. Please install the latest version with `pip install -U kernels`.)_KERNEL_MAPPINGr^r ImportErrorr)mappings&r&$register_kernel_mapping_transformersrls> ?%G 7E " "+?H+U+Ux  (r(c RpV#)cV#r,r-r.s&r& decorator.use_kernel_forward_from_hub..decoratorsJr(r-argskwargsros*, r&rrs r(c RpV#)cV#r,r-r;s&r&ro+use_kernel_func_from_hub..decoratorsKr(r-rqs*, r&rrs r(c&a]tRtRtoRtRtVtR#)ric\R4h)zNLayerRepository requires `kernels` to be installed. Run `pip install kernels`. RuntimeErrorselfrrrss&*,r&__init__LayerRepository.__init__sop pr(r-N__name__ __module__ __qualname____firstlineno__r}__static_attributes____classdictcell__ __classdict__s@r&rrs q qr(rc&a]tRtRtoRtRtVtR#)ri c\R4h)zSLocalLayerRepository requires `kernels` to be installed. Run `pip install kernels`.ryr{s&*,r&r}LocalLayerRepository.__init__!stu ur(r-Nrrs@r&rr s v vr(rc\R4h)z^replace_kernel_forward_from_hub requires `kernels` to be installed. Run `pip install kernels`.ryrrrss*,r&rr$s l  r(c\R4h)zVregister_kernel_mapping requires `kernels` to be installed. Run `pip install kernels`.ryrs*,r&rr)ssttr(c\R4h)zcregister_kernel_mapping_transformers requires `kernels` to be installed. Run `pip install kernels`.ryrs*,r&rlrl,s q  r(z causal-conv1drCzkernels-community/causal-conv1drz mamba-ssmzkernels-community/mamba-ssmzfalcon_mamba-ssmzfinegrained-fp8z!kernels-community/finegrained-fp8z deep-gemmzkernels-community/deep-gemmz sonic-moezkernels-community/sonic-moerevisionz ep-supportc>V^8dQhR\R,R\/#)r!attn_implementationNreturn)r$bool)r%s"r&r'r'>s3:$r(cLVRJ;'d\P!RV4RJ#)zJCheck whether `attn_implementation` matches a kernel pattern from the hub.Nz'^[^/:]+/[^/:]+(?:@[^/:]+)?(?::[^/:]+)?$)research)rs&r& is_kernelr>s3 4' c c II@BU V^b br(c dV^8dQhR\R\R,R\R\R,/#)r!rattention_wrapperNallow_all_kernelsr)r$rrr)r%s"r&r'r'Fs;DDD19DD\`D$Dr(c^RIHp^RIHpRV9dVP R4^,MTp\ V4'gR#\ 'g \R4hRV9d%VP R4wrgVP4pMTpRpVP4pVPR4wrhp VP4pV 'dV P4MRp \WiVR7p R p \T R 4'dTf\pTp M2\T R4'd^RIHpTeTMTp Rp MTe \#Y4p TP%TX 4TP%YT ,4T # \dp \R T R T R 24hRp ? ii;i)a Load and register the kernel associated to `attn_implementation`. Args: attn_implementation: A string, usually a kernel repo like "kernels-community/flash-mla". attn_wrapper: a callable for the wrapper around the attention implementation. In `transformers` we have a wrapper around the `flash_attn_var_len` call, and the same goes for `sdpa` and `eager`. They just prepare the arguments properly. This is mostly used for continious batching, where we want the `paged` wrapper, which calls the paged cache. allow_all_kernels (`bool`, optional): Whether to load kernels from unverified hub repos, if it is a custom kernel outside of the `kernels-community` hub repository. )ALL_MASK_ATTENTION_FUNCTIONS)ALL_ATTENTION_FUNCTIONS|N`kernels` is either not installed or uses an incompatible version. Please install the latest version with `pip install -U kernels`.:@)rrz-An error occurred while trying to load from 'z': .flash_attention_2flash_attn_varlen_funcsparse_atten_func)msa_attention_forwardsdpa) masking_utilsrmodeling_utilsrsplitr_kernels_availablerjstrip partitionr Exception ValueErrorhasattrr msa_attentionrgetattrregister)rrrrractual_attn_namerC kernel_name_revkernelemask_implementationkernel_functionrs&&& r&load_and_register_attn_kernelrFs =8->x-HHkgeekgNNr(cPV^8dQhR\\,\,/#)r! module_names)listr)r%s"r&r'r's$$d8nx&?$r(cFa\S\4'dS.oV3RlpV#)a3 This decorator attaches the target function within the module as a plain attribute (not as a submodule). Keep in mind that this registration is only meant for `kernelize` to recognize its target modules (i.e. function exchanged for a weightless `nn.Module` with the same forward) to then exchange to the kernel variation (in-place) if the conditions are met. We cache each of these function-based registrations: After proper registration and exchange it is removed from the module's `_modules` dict as it does not really act as `nn.Module` but a base function. c<<aVPoVV3RlpWnV#)c<S!V.VO5/VBVPPR/4pSFQp\VRR4;'g#\VRR4;'g\VRR4pVf\RV: 24hWCV&KS R#)_hidden_kernelsrNkernel_layer_namer8z)Could not infer kernel function name for )__dict__ setdefaultrr)r|rrrshidden_kernelsfnnamer orig_inits&*, r&new_init8use_kernelized_func..decorator..new_inits d ,T ,V ,"]]556GLN"B D166r#6=66r;5 <$'PQSPV%WXX(*t$#r()r})r/rrrs& @r&ro&use_kernelized_func..decoratorsLL  *"  r()r\r)rrosf r&use_kernelized_funcrs&,))$~ . r(c#."RsRxRsR# Rsi;i5i)z Context manager used to adjust the value of the global `ALLOW_HUB_KERNELS`. This is needed, as this argument cannot be forwarded directly to the `__init__` of the models, where we set the attention implementation. TNF)rr-r(r&allow_all_hub_kernelsrs!"  "Esc^V^8dQhR\R\\,R\R\/#)r! parent_cls child_names kernel_clsr)typerr$)r%s"r&r'r's4c r(caaaVPoVVV3Rlp\RVP 2V3RV/4pRVP 2VnV#)a" Create a new class that inherits from `parent_cls` and fuses the child modules specified in `child_names with the provided `kernel_cls`. The first child in `child_names` will be replaced with the `kernel_cls`, and the rest will be replaced with `nn.Identity()` to keep the same interface. c<S!V.VO5/VBSUu.uFp\W4NK ppS!V!p\VS^,V4SR,F"p\W\P!44K$ R#uupi)r:NNN)rsetattrnnIdentity) r|rrrsrchildrenkernel_instancerr original_inits &*, r& patched_init9make_parent_class_for_kernel_fusion..patched_init#snd,T,V,4?@KDGD'K@$h/k!no6OOD D .$AsA3Fusedr})r}rrr)rrrr patched_clsrs&ff @r&#make_parent_class_for_kernel_fusionrsW''M/z2234zmjR^E_`K!&z'>'>&?@K r(c(V^8dQhRRRRRRRR/#) r!r/ztype[PreTrainedModel]configr kernel_configrrNr-)r%s"r&r'r'0s6y/y/ y/ y/"y/ y/r(c  a\VR4'd\VPR4'g\RVP R24hVPPp/p/pRpVP P 4EF_wrx\V\4'dP\VP44^8wd\RV 24h\\VP444p M,\V\4'dTp M\RV: RV: 24hV PR4wrp V 'd V 'g\R V : RV: 24hVP'd@V P!R 4P#R 4R",p \%\'V 4V V R 7pM \)WR 7pVP+4pVf\R V: 24h\,P.P1VP24pV'd\5VVP R2R4MRp\V\4'd1VfWV&EKVPVnVWG&WVP&EM\V\84'EdVf(\RVP: RVP R24hVPVnVUu.uF pV^,NK ppVUu.uFpVP;R^4^,NK pp\\=V44^8wd\RV 24hV^,P?RR4pVUu.uFpVP;R^4^,NK ppVf0\@PB!R4;_uu_4V!V4pRRR4RpVPE4Fwpo\FPH!VV4'gK%\J;QJdV3RlV4F 'dK RM RM!V3RlV44'g!\RV: RV RVP: 24hRp\MS4p\OVVV4VVP&K V'g\RV: RVP: R24h\QVRR 7\VR!4'd<\SV4p\UVPV4pVe VV,p\YVVRR 7WVP&EKb WRnR#uupiuupiuupi +'giEL};i)# config_class model_typezModel z# has no config_class or model_type.NzLExpected exactly one kernel repo regardless of device/mode specificity, got zInvalid hub repo z for layer rzInvalid kernel repo string r) repo_path package_namer"rBz*Could not load kernel class from hub_repo=Layoutz Fused kernel z* requires a companion layout class named 'zLayout' in the same module.rzGAll patterns for a fused kernel must share the same parent module, got *z\w+metaFc3<<"TFp\SV4xK R#5ir,)r)r_childrs& r&rb;register_kernel_replacements_and_fusions..sK{e76511{sTzModule z* does not have the expected child modules z required for the fused kernel zNo module matched pattern z for fused kernel z3. Provide the full dotted path from the model root.) overwriteconversion_mapping)-rr rrr kernel_mappingitemsr\r]lenrenextiterr$ruse_local_kernelrstriprrrrloadsysmodulesrrrrtuplersplitsetrtorchdevice named_modulesr fullmatchallrrr r rrr )r/r r r  patch_mapping new_mapping meta_modelr"hub_reporepo_strrCrlayer_name_in_reporrepor kernel_mod layout_clsitem glob_patternspparent_patternsparent_patternr matched_anyr module_clsexisting transformsrs&&& @r&(register_kernel_replacements_and_fusionsr=0s 3 ' 'ws7G7G/V/V6#,,/RSTT!!,,J%'MKJ - < < B B D h % %8??$%* bckblmD!234H # & &H0 K ~VW W)1););C)@&&0:8,kR\Q_`a a  ) ) )">>#.44S9"=L'w-)-D #7RDYY[  I(VW W[[__Z%:%:; R\WZJ,?,?+@)GNbf  j# & &!*2J',6+>+>J ((2M %08 ++ , E * *! #J$7$7#:;(1122MO ,6+>+>J (1;<T!WWM<environrupperr6rr2 INFERENCETRAINING TORCH_COMPILErir^rlrr?rrrrrrrr=__all__r')r@s@r&rWsT $% ,j41I4602   H %B  * .Y(,%%'JJNN3De$L$R$R$T!48LL# % ( O;:*  O/*  =-! =-! 7(! ;(! =-!5  B  O2+   :1 :1!  11! :1! :4!/ <  !3!33_:)6   !3!33_:*6   !3!33_:(6   !3!33_:vWX6   !3!33_:vWX6   !3!33_:z[\6 wBeOBJ! 6B]!  ~6B] 6B]!  - () \)fi!BIqQ):IqI$A9aP #F STU):IqI):J U 28:9DNPf- `O:$P " " 4y/x Y  .(-%.D%   qqvv u E% sU#O2OO"JO2 OO2OO2" O/+O2.O//O22.P#"P#