+ QV-jxx!aEREt0t^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RI t ^RI t ^RI t ^RI t ^RI t ^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIt^RIHtHt^RIHtHtHtHtH t ^RIH!t!^RI"H#t#H$t$^RIH%t%H&t&^RI'H(t(^RI)H*t*^R I+H,t,H-t-^R IH.t.^R I/H0t0^RI1t1^R I2H3t3H4t4^R I5H6t6^RI7H t8],'d^RI9H:t:M]-t:^RI;Ht>H?t?H@t@HAtAHBtB^RICHDtD^RIEHFtFHGtGHHtHHItIHJtJHKtKHLtLHMtMHNtNHOtOHPtPHQtQHRtRHStSHTtTHUtUHVtVHWtWHXtXHYtYHZtZH[t[H\t\H]t]H^t^H_t_H`t`HataHbtbHctcHdtdHeteHftfHgtgHhthHitiHjtjHktkHltlHmtmHntnHotoHptpHqtqHrtrHstsHtttHutuHvtvHwtwHxtxHytyHztzH{t{H|t|H}t}H~t~HtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHtHt]K!4'd^RIHtHt^RIHt]!4'd&^RIHtHtHtHtHtHtHt^RIHt^RIHt^RIHtM]t]tRtRtRtRtRtRtRR R!R"R#R$R%R&R'R(/tRR R)R*R+R,R-R./t/R/^ bR0^bR1R2bR3R2bR4R2bR5^cbR6^ bR7^bR8^bR9^bR:^ bR;RbR?^bR@^bRA^bRB^bRC^RD^RE^$RFR2RG^RH^RI^/Ct]!4'dh^RIt^RJIHt^RKIHtHt]PlEPRJt]PlEPRJt]!]PlRLR4RJt]!]RMR4RJtMRNtRNtRNtRNt]8EP!]4tERROltERRPlt]!RQRNRR7t]!RSR2RR7t]!RTRNRR7t]!RURNRR7t]!RVR2RR7t]!RWRNRR7t]!RXR2RR7t]!RYR2RR7tRZtR[tR\tR]tR^tR_tR`tRatRbtRctRdtRetRftRgtRhtRitRjtRktRltRmt]F3RnRollt]I3RpRqllt]G3RrRslltERRtRulltRvtRwtRxEtRyEtRzEtR{EtR|EtR}EtR~RlEtRRlEtREtREt REt REt REt REt REtREtREtREtREtREtREtREtREtREtREtREtREtREtREtREtREtRRlEtREt REt!REt"REt#REt$REt%REt&REt'REt(REt)REt*REt+REt,REt-]!4'Ed^RItR] EP\9d,] EP\R,Et/]EP`!E]/4Et1R] EP\9d] EP\R,Et4E]4R8Xd1]EPEPk4'gE]6!RE]4 R24hE]4RL8Xd]!4'gE]6!RE]4 R24hE]4RM8Xd]!4'gE]6!RE]4 R24hE]4R8Xd]!4'gE]6!RE]4 R24hE]4R8Xd]!4'gE]6!RE]4 R24h]EPn!E]44Et1Mq]EPEPk4'dREt4MK]!4'dRMEt4M:]!4'dREt4M)]!4'dREt4M]!4'dRLEt4MREt4MREt4REt9REt:REt;REtERRRllEt?ERR^/RRlllEt@REtAREtBREtCREtDREtEREtFREtGREtHREtIREtJREtKREtLREtMREtNREtOREtPREtQREtRREtSREtTREtUREtVREtWREtXREtYREtZREt[REt\REt]REt^REt_REt`REtaREtbREtcREtdREteREtfREtgREthREtiREtjREtkREtlREtmREtnREtoREtpREtqREtrREtsREttREtuREtvREtwREtxREtyERRlEtzRRlEt{ERERlEt|EREt}EREt~EREtEREt!ERER4Et!ERER E]4Et!ER ER E]4Et!ER ER 4Et]PBER4Et!ERER4Et]PBERERl4Et!ERER]EP4EtEREt]PBER4Et/EtEREtEREt^REIEt!ERER4EtEREtERERERllEtERERERllEtER EtER!EtERER"lEtER#EtER$Et!ER%ER&E]4EtERER'ER(llEt!ER)ER*4EtERER+ER,llEtERER-ER.llEtER/EtERER0lEtER1EtER2Et!ER3ER4]EPF4Et!ER5ER6]4EtER7ER8lEt]!4'Ed1R]EPEPNR]EPNER9]EPN/EtR]EPEPRRRER9R/EtR]EPEPVRER:ER9ER;/EtR]EPEPZRRER9R/EtR]EPEP^R^ER9^/EtR]EPEPbRRER9R/EtR]EPEPfR^ER9^/EtR]EPEPjRRER9R/EtR]EPRRER9R/EtM8ER9R/EtER9R/EtER9ER</EtER9R/EtER9R/EtER9^/EtER9^/EtER9R/EtER9R/Et]!4'dJ]EPpEPNE]R&]EPpEPVE]R&]EPpE]R&]!4'df]EPrEPRE]R&]EPrEPNE]R&]EPrEPVE]R&]EPrE]R&]!4'df]EPtEPRE]RM&]EPtEPNE]RM&]EPtEPVE]RM&]EPtE]RM&]!4'd]EPvEPRE]RL&]EPvEPNE]RL&]EPvEPVE]RL&]EPvEPbE]RL&]EPvEPbE]RL&]EPvEP^E]RL&]EPvEPfE]RL&]EPvEPjE]RL&]EPvE]RL&]!4'dX]EPEPRE]ER=&]EPEPNE]ER=&]EPEPVE]ER=&ER>ER?lEtER@ERAlEtERBERClEtERDERElEtERFERGlEtERHERIlEtERJERKlEtERLERMlEtERNEROlEt]!4'EdcERP] EP\9EdO] EP\ERP,Et]*!E]4EP4'gE]6!ERQE] 24h] EPEP] EPEPE]44wEtEt1]EPEPE]4E]RE]EPERR4Et]EP`!E]4EtE]EPEtR] EP\9d/E]4E]8wd&ERUE]4 ERVE] ERW2EtE]ERX, EtE]6!E]4hE]Et4ERYERZlEtE]!E]ER[4E]!E]ER\4E]!E]ER]4ER^EtE]ERER_ER`ll4EtE]E]R,E]R,E]R,3,EtE]E]R,RE],E]E]E]3,,3,Et]%ERaERbl4EtERERcERdllEt!EReERf]E]]-3,4EtERgEtERhEtERiEtERjEtERkEtERlEtERmEtERnEtERERoERpllEtERERqlEtERrEtERERslEtERtEtERuEt!ERvERw4Et!ERxERy] EP4EtE]!4Et]^kERzER{lEtER|ER}lEt]EP!ER~.ERO4Et!ERER4EtERERERllEtERERlEt]!ER4EtR# E]2dEt3E]2!RE]/ RE]3 24E]3hREt3EA3ii;i E]8dEt3E]8!RE]4 24E]3hREt3EA3ii;i E]6dEt3E]6!ERSE] 24E]3hREt3EA3ii;i E]dEt3E]!ERT4E]3hREt3EA3ii;i(N)UserDict defaultdict)Callable GeneratorIterableIteratorMapping)contextmanager)MISSINGfields)cachewraps)StringIO)Path) TYPE_CHECKINGAny)mock)patch) create_repo delete_repo)version)logging)Trainer)is_clearml_availableis_optuna_availableis_ray_availableis_swanlab_availableis_tensorboard_availableis_trackio_availableis_wandb_available)is_deepspeed_available)iACCELERATE_MIN_VERSIONGGUF_MIN_VERSIONSAFE_WEIGHTS_INDEX_NAMETRITON_MIN_VERSIONWEIGHTS_INDEX_NAMEis_accelerate_availableis_apex_availableis_apollo_torch_availableis_aqlm_availableis_auto_round_availableis_av_availableis_bitsandbytes_availableis_bs4_availableis_causal_conv1d_availableis_compressed_tensors_availableis_cv2_availableis_cython_availableis_decord_availableis_detectron2_availableis_essentia_availableis_faiss_availableis_fbgemm_gpu_availableis_flash_attn_2_availableis_flash_attn_3_availableis_flash_attn_4_available#is_flash_linear_attention_availableis_flute_availableis_fouroversix_availableis_fp_quant_availableis_fsdp_availableis_g2p_en_availableis_galore_torch_availableis_gguf_availableis_gptqmodel_availableis_grokadamw_availableis_hadamard_availableis_hqq_available#is_huggingface_hub_greater_or_equalis_ipython_availableis_jinja_availableis_jmespath_availableis_jumanpp_availableis_kernels_availableis_levenshtein_availableis_librosa_availableis_liger_kernel_availableis_lomo_availableis_mistral_common_availableis_multipart_availableis_natten_availableis_nltk_availableis_numba_availableis_onnx_availableis_openai_availableis_optimum_availableis_optimum_quanto_availableis_pandas_availableis_peft_availableis_phonemizer_availableis_pretty_midi_availableis_psutil_availableis_pyctcdecode_availableis_pytesseract_availableis_pytest_availableis_pytest_order_available!is_pytorch_quantization_availableis_quark_availableis_qutlass_availableis_rjieba_availableis_sacremoses_availableis_schedulefree_availableis_scipy_availableis_sentencepiece_availableis_seqio_availableis_serve_availableis_spacy_availableis_speech_availableis_spqr_availableis_sudachi_availableis_sudachi_projection_availableis_tiktoken_availableis_timm_availableis_tokenizers_availableis_torch_available!is_torch_bf16_available_on_device!is_torch_fp16_available_on_deviceis_torch_greater_or_equalis_torch_hpu_availableis_torch_mlu_availableis_torch_neuroncore_availableis_torch_npu_availableis_torch_optimi_availableis_torch_tensorrt_fx_availableis_torch_tf32_availableis_torch_tpu_availableis_torch_xla_availableis_torch_xpu_availableis_torchao_availableis_torchaudio_availableis_torchcodec_availableis_torchvision_availableis_triton_availableis_vision_availableis_vptq_available strtobool)AcceleratorState PartialState)is_fp8_available)Module _get_checker_get_continue_on_failure _get_runner _is_mocked_patch_unwrap_mock_awareget_optionflags)skip) import_path) DoctestItemzjulien-c/bert-xsmall-dummyzjulien-c/dummy-unknownzjulien-c/dummy-diff-tokenizer__DUMMY_TRANSFORMERS_USER__zhttps://hub-ci.huggingface.co%hf_94wBhPGp6KrrTH3KDchhKpRxZwd6dmHWLL config_classConfigcausal_lm_class ForCausalLMquestion_answering_classForQuestionAnsweringsequence_classification_classForSequenceClassificationtoken_classification_classForTokenClassificationtext_config_class TextConfigvision_config_class VisionConfigconditional_generation_classForConditionalGeneration batch_size seq_length is_trainingTuse_input_mask use_labels vocab_size hidden_sizenum_hidden_layersnum_attention_headsnum_key_value_headsintermediate_size hidden_actgelumax_position_embeddingsi pad_token_id bos_token_id eos_token_idexpert_intervalmoe_layer_start_indexmoe_intermediate_sizeshared_expert_intermediate_sizeshared_expert_gatemoe_num_shared_expertsnum_experts_per_tok num_experts) load_file)FLASH_ATTN_KERNEL_FALLBACKPreTrainedModelxpunpuFc\PV,p\V4pV# \d\RT R24hi;i \dTpT#i;i)If set, z must be yes or no.)osenvironr ValueErrorKeyErrorkeydefaultvalue_values&& k/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/transformers/testing_utils.pyparse_flag_from_envrsp B 3  Bu%F M Bxu,?@A A B  MA 'A AAc\PV,p\V4pV# \d\RT R24hi;i \dTpT#i;i)rz must be a int.)rrintrrrs&& rparse_int_from_envr*sj> 3 >ZF M >xuO<= = >  MrRUN_SLOWr RUN_FLAKYRUN_CUSTOM_TOKENIZERSHUGGINGFACE_CO_STAGINGRUN_PIPELINE_TESTSRUN_AGENT_TESTSRUN_TRAINING_TESTSRUN_TENSOR_PARALLEL_TESTSc\'g\P!RR7!V4#^RIpVPP 4!V4# \ dTu#i;i)z Decorator marking a test as a staging test. Those tests will run using the staging environment of huggingface.co instead of the real model hub. ztest is staging testreasonN) _run_stagingunittestrpytestmarkis_staging_test ImportError test_casers& rrrAsW <}}$:;IFF < ;;..0; ;   A AAc\'g\P!RR7!V4#^RIpVPP 4!V4# \ dTu#i;i)zz Decorator marking a test as a pipeline test. If RUN_PIPELINE_TESTS is set to a falsy value, those tests will be skipped. ztest is pipeline testrN)_run_pipeline_testsrrrris_pipeline_testrrs& rrrRX  }}$;>rcFa\P!S4V3Rl4pV#)cn<S!V/VB# \dp\P!RT 24hRp?ii;i)z)Test skipped due to NotImplementedError: N)NotImplementedErrorrSkipTest)argskwargse test_funcs*, rwrapper(skip_if_not_implemented..wrappersE Ud-f- -" U##&OPQs$ST T Us 4/4) functoolsr)r r sf rskip_if_not_implementedrs'__YU U Nrc\V4FOpVPR4'gK\W4p\V4'gK:\ W\ V44KQ V#)zH Class decorator to apply @skip_if_not_implemented to all test methods. test_)dir startswithgetattrcallablesetattrr)cls attr_nameattrs& rapply_skip_if_not_implementedrsMX    ( (3*D~~(?(EF  JrcD\P!\R4!V4#)z Decorator marking a test for a custom tokenizer. Custom tokenizers require additional dependencies, and are skipped by default. Set the RUN_CUSTOM_TOKENIZERS environment variable to a truthy value to run them. ztest of custom tokenizers)rr_run_custom_tokenizersrs&rcustom_tokenizersrs   57R ST] ^^rcL\P!\4R4!V4#)zu Decorator marking a test that requires BeautifulSoup4. These tests are skipped when BeautifulSoup4 isn't installed. ztest requires BeautifulSoup4)rrr.rs&r require_bs4rs"   /13Q RS\ ]]rcL\P!\4R4!V4#)z Decorator marking a test that requires GaLore. These tests are skipped when GaLore isn't installed. https://github.com/jiaweizzhao/GaLore ztest requires GaLore)rrrArs&rrequire_galore_torchr "   8:= )rrr'rr9s&&rrequire_accelerater@s3    ,0TU`Ta.b  rc$V^8dQhR\/#r7r:)r<s"rr=r=$s   rcaV3RlpV#)ze Decorator marking a test that requires triton. These tests are skipped when triton isn't installed. cV<\P!\S4RS 24!V4#)z test requires triton version >= )rrrr?s&r decorator!require_triton..decorator)s0""#6{#CGghsgtEuv   r)r9rDsf rrequire_tritonrG$s  rc$V^8dQhR\/#r7r:)r<s"rr=r=1rcT\P!\V4RV 24!V4#)zb Decorator marking a test that requires ggguf. These tests are skipped when gguf isn't installed. ztest requires gguf version >= )rrrBr?s&&r require_ggufrK1s1   0=A_`k_l?m n rc$V^8dQhR\/#r7r:)r<s"rr=r=:rIrcT\P!\V4RV 24!V4#)za Decorator marking a test that requires fsdp. These tests are skipped when fsdp isn't installed. ztest requires torch version >= )rrr?r?s&&r require_fsdprN:s1   0=A`al`m?n o rcL\P!\4R4!V4#)zl Decorator marking a test that requires g2p_en. These tests are skipped when SentencePiece isn't installed. ztest requires g2p_en)rrr@rs&rrequire_g2p_enrPC!   246L Mi XXrcL\P!\4R4!V4#)ze Decorator marking a test that requires rjieba. These tests are skipped when rjieba isn't installed. ztest requires rjieba)rrrfrs&rrequire_rjiebarSJrQrcL\P!\4R4!V4#)zc Decorator marking a test that requires jinja. These tests are skipped when jinja isn't installed. ztest requires jinja)rrrIrs&r require_jinjarUQ!   135J KI VVrcL\P!\4R4!V4#)zi Decorator marking a test that requires jmespath. These tests are skipped when jmespath isn't installed. ztest requires jmespath)rrrJrs&rrequire_jmespathrXX"   468P QR[ \\rcL\P!\4R4!V4#)ztest requires ONNX)rrrVrs&r require_onnxr[_s   024H I) TTrcL\P!\4R4!V4#)zc Decorator marking a test that requires Timm. These tests are skipped when Timm isn't installed. ztest requires Timm)rrrsrs&r require_timmr]cr5rcL\P!\4R4!V4#)zg Decorator marking a test that requires NATTEN. These tests are skipped when NATTEN isn't installed. ztest requires natten)rrrSrs&rrequire_nattenr_m!   246L Mi XXrcL\P!\4R4!V4#)zi Decorator marking a test that requires PyTorch. These tests are skipped when PyTorch isn't installed. test requires PyTorch)rrrurs&r require_torchrcws!   135L Mi XXrc$V^8dQhR\/#r8rr:)r<s"rr=r=s  C rcaV3RlpV#)z Decorator marking a test that requires PyTorch version >= `version`. These tests are skipped when PyTorch version is less than `version`. cV<\P!\S4RS 24!V4#)z!test requires PyTorch version >= )rrrxrrs&rrD1require_torch_greater_or_equal..decorators0""#= `version`. These tests are skipped when huggingface_hub version is less than `version`. cV<\P!\S4RS 24!V4#)z)test requires huggingface_hub version >= )rrrGrhs&rrD;require_huggingface_hub_greater_or_equal..decorators2"" / 8    "!"sA.. B9BBcL\P!\4R4!V4#)z Decorator marking a test that requires Flash Linear Attention. These tests are skipped when Flash Linear Attention isn't installed. z&test requires `flash-linear-attention`)rrr;rs&rrequire_flash_linear_attentionrs*   +-0  rcL\P!\4R4!V4#)zt Decorator marking a test that requires causal-conv1d. These tests are skipped when causal-conv1d isn't installed. ztest requires `causal-conv1d`)rrr/rs&rrequire_causal_conv1drs*   "$'  rcL\P!\4R4!V4#)zc Decorator marking a test that requires PEFT. These tests are skipped when PEFT isn't installed. ztest requires PEFT)rrr[rs&r require_peftrr5rcL\P!\4R4!V4#)zq Decorator marking a test that requires Torchvision. These tests are skipped when Torchvision isn't installed. ztest requires Torchvision)rrrrs&rrequire_torchvisionrr2rcL\P!\4R4!V4#)zo Decorator marking a test that requires Torchcodec. These tests are skipped when Torchcodec isn't installed. ztest requires Torchcodec)rrrrs&rrequire_torchcodecrs"   68:T UV_ ``rcL\P!\4R4!V4#)zm Decorator marking a test that requires torchaudio. These tests are skipped when torchaudio isn't installed. test requires torchaudio)rrrrs&rrequire_torchaudior"   68:T UV_ ``rcL\P!\4R4!V4#)s Decorator marking a test that requires SentencePiece. These tests are skipped when SentencePiece isn't installed. ztest requires SentencePiece)rrrjrs&rrequire_sentencepiecer#s"   9;=Z [\e ffrcL\P!\4R4!V4#)zm Decorator marking a test that requires Sacremoses. These tests are skipped when Sacremoses isn't installed. ztest requires Sacremoses)rrrgrs&rrequire_sacremosesr*rrcL\P!\4R4!V4#)rztest requires Seqio)rrrkrs&r require_seqior1rVrcL\P!\4R4!V4#)zk Decorator marking a test that requires Scipy. These tests are skipped when SentencePiece isn't installed. ztest requires Scipy)rrrirs&r require_scipyr8rVrcL\P!\4R4!V4#)uw Decorator marking a test that requires 🤗 Tokenizers. These tests are skipped when 🤗 Tokenizers isn't installed. ztest requires tokenizers)rrrtrs&rrequire_tokenizersr?rrcL\P!\4R4!V4#)ze Decorator marking a test that requires pandas. These tests are skipped when pandas isn't installed. ztest requires pandas)rrrZrs&rrequire_pandasrFrQrcL\P!\4R4!V4#)zo Decorator marking a test that requires PyTesseract. These tests are skipped when PyTesseract isn't installed. ztest requires PyTesseract)rrr`rs&rrequire_pytesseractrM"   79;V WXa bbrcL\P!\4R4!V4#)z Decorator marking a test that requires PyTorch Quantization Toolkit. These tests are skipped when PyTorch Quantization Toolkit isn't installed. z*test requires PyTorch Quantization Toolkit)rrrcrs&rrequire_pytorch_quantizationrTs&   @BDp q rcL\P!\4R4!V4#)zz Decorator marking a test that requires the vision dependencies. These tests are skipped when torchaudio isn't installed. ztest requires vision)rrrrs&rrequire_visionr^s!   246L Mi XXrcL\P!\4R4!V4#)zc Decorator marking a test that requires SpaCy. These tests are skipped when SpaCy isn't installed. ztest requires spacy)rrrmrs&r require_spacyrfrVrc\4'g\P!RR7!V4#^RIp\P!VP P 4^8R4!V4#)a Decorator marking a test that requires a multi-GPU CUDA setup (in PyTorch). These tests are skipped on a machine without multiple CUDA GPUs. To run *only* the multi_gpu tests, assuming all test names contain multi_gpu: $ pytest -sv ./tests -k "multi_gpu" rbrNz test requires multiple CUDA GPUsrurrtorchrcuda device_countrrs& rrequire_torch_multi_gpurmsQ   }}$;` abk llrc\4'g\P!RR7!V4#\P!\ \ 4^8R4!V4#)a Decorator marking a test that requires a multi-accelerator (in PyTorch). These tests are skipped on a machine without multiple accelerators. To run *only* the multi_accelerator tests, assuming all test names contain multi_accelerator: $ pytest -sv ./tests -k "multi_accelerator" rbrz#test requires multiple acceleratorsrurrrbackend_device_count torch_devicers&rrequire_torch_multi_acceleratorr|sK   }}$;= z acceleratorsr)rrs&rrD/require_torch_n_accelerators..decoratorsV!##==(?@K K""#7 #E#JN_`a_bboLpq   rrF)rrDsf rrequire_torch_n_acceleratorsrs  rc\4'g\P!RR7!V4#^RIp\P!VP P 4^8R4!V4#)zG Decorator marking a test that requires 0 or 1 GPU setup (in PyTorch). rbrNztest requires 0 or 1 GPUrrs& rrequire_torch_non_multi_gpursQ   }}$;X YZc ddrc\4'g\P!RR7!V4#\P!\ \ 4^8R4!V4#)zO Decorator marking a test that requires 0 or 1 accelerator setup (in PyTorch). rbrz test requires 0 or 1 acceleratorrrs&r#require_torch_non_multi_acceleratorrsG   }}$;^ _`i jjrc\4'g\P!RR7!V4#\P!\ \ 4^8R4!V4#)zT Decorator marking a test that requires 0 or 1 or 2 accelerator setup (in PyTorch). rbrz&test requires 0 or 1 or 2 acceleratorsrrs&r"require_torch_up_to_2_acceleratorsrsK   }}$;.decoratorcs"6"8#,#,W]] MM),,44Y? @ M M> ]]? +>,"" #'[\k[llm%n  rrF)rrDsf r(require_torchao_version_greater_or_equalrbs rcL\P!\4R4!V4#)z8Decorator marking a test that requires Torch-TensorRT FXztest requires Torch-TensorRT FX)rrr~rs&rrequire_torch_tensorrt_fxrns   =?Ab cdm nnrcJ\P!\R8HR4!V4#)8Decorator marking a test that requires CUDA and PyTorch.rztest requires CUDArrs&rrequire_torch_gpurss   |v57K LY WWrcJ\P!\R8HR4!V4#)rmpsztest requires MPSrrs&rrequire_torch_mpsrxs   |u46I J9 UUrc$V^8dQhR\/#r8memoryfloat)r<s"rr=r=}s  U rc\4'gV#^RIp\P!VP 4P R, V8RV R24!V4#)zWDecorator marking a test that requires a CPU RAM with more than `memory` GiB of memory.Nz'test requires a machine with more than z GiB of CPU RAM memory@)r^psutilrrvirtual_memorytotal)rrrs&& rrequire_large_cpu_ramr}sZ     %%/&8 1&9OP  rc$V^8dQhR\/#rr)r<s"rr=r=surc\R8wd"\P!RV R2R7!V4#\P!\P P ^4PR, V8RV R24!V4#)zXDecorator marking a test that requires a CUDA GPU with more than `memory` GiB of memory.rz(test requires a CUDA GPU with more than  GiB of memoryrz#test requires a GPU with more than r)rrrrrrget_device_properties total_memory)rrs&&rrequire_torch_large_gpur suv}}&NvhVd$efgpqq    ((+887BVK -fX^D  rrc$V^8dQhR\/#rr)r<s"rr=r=s R Ru Rrc,aV3RlpVfV#V!V4#)z\Decorator marking a test that requires an accelerator with more than `memory` GiB of memory.c <\R9d!\P!RS R24!V4#\\\4p\P !VP ^4PR, S8RS R24!V4#)rz*test requires a GPU or XPU with more than r)rrr)rrrrrrrr )tc torch_accelrs& rmemory_decorator9require_torch_large_accelerator..memory_decorators~  .==#MfXUc!defhi ie\2 ""  - -a 0 = = G& P8 O   rrF)rrrs&d rrequire_torch_large_acceleratorrs! )0 Q6Fy6QQrcj\P!\RJ;'d \R8gR4!V4#)zMDecorator marking a test that requires an accessible accelerator and PyTorch.Nrztest requires acceleratorrrs&rrequire_torch_acceleratorrs2   |47QQLE2require_cuda_capability_at_least..srzRequires CUDA capability >= r)rr is_availableget_device_capabilityrskipIf)majorminorr capabilitys&& r require_cuda_capability_at_leastr%s]  :: " " $ $**113J ??:6:VW\V]]^_d^e8f ggrcV\P!\\4R4!V4#)zBDecorator marking a test that requires a device that supports bf16z&test requires device with bf16 support)rrrvrrs&rrequire_torch_bf16r'rrc0a\S4V3Rl4pV#)c<\4'dK\P!4p\P!R4S!V/VB\P!V4#S!V/VB# \P!T4i;iT)rr$are_deterministic_algorithms_enableduse_deterministic_algorithms)rroriginal_staters*, rr .require_deterministic_for_xpu..wrappersk ! # #"GGIN C2248 $1&122>Bd-f- -22>Bs A$$A<)r)rr sf rrequire_deterministic_for_xpur/s! 9 . . NrcL\P!\4R4!V4#)z[Decorator marking a test that requires Ampere or a newer GPU arch, cuda>=11 and torch>=1.7.zAtest requires Ampere or a newer GPU arch, cuda>=11 and torch>=1.7)rrrrs&rrequire_torch_tf32r1s&   !#f  rcL\P!\4R4!V4#)z2Decorator marking a test that requires detectron2.ztest requires `detectron2`)rrr4rs&rrequire_detectron2r3s   68:V WXa bbrcL\P!\4R4!V4#)z-Decorator marking a test that requires faiss.ztest requires `faiss`)rrr6rs&r require_faissr5s   135L Mi XXrcL\P!\4R4!V4#)zeDecorator marking a test that requires IPython. These tests are skipped when IPython isn't installed.ztest requires `IPython`)rrrHrs&rrequire_ipythonr7s   357P QR[ \\rcL\P!\4R4!V4#)zg Decorator marking a test that requires optuna. These tests are skipped when optuna isn't installed. ztest requires optuna)rrrrs&rrequire_optunar9r`rcL\P!\4R4!V4#)zk Decorator marking a test that requires Ray/tune. These tests are skipped when Ray/tune isn't installed. ztest requires Ray/tune)rrrrs&r require_rayr;s!   /13K LY WWrcL\P!\4R4!V4#)zi Decorator marking a test that requires swanlab. These tests are skipped when swanlab isn't installed. ztest requires swanlab)rrrrs&rrequire_swanlabr="   357N OPY ZZrcL\P!\4R4!V4#)zi Decorator marking a test that requires trackio. These tests are skipped when trackio isn't installed. ztest requires trackio)rrrrs&rrequire_trackior@r>rcL\P!\4R4!V4#)ze Decorator marking a test that requires wandb. These tests are skipped when wandb isn't installed. ztest requires wandb)rrr rs&r require_wandbrBs!   135J KI VVrcL\P!\4R4!V4#)zd Decorator marking a test requires clearml. These tests are skipped when clearml isn't installed. ztest requires clearml)rrrrs&rrequire_clearmlrD&r>rcL\P!\4R4!V4#)z2 Decorator marking a test that requires deepspeed ztest requires deepspeed)rrr!rs&rrequire_deepspeedrF0r+rcL\P!\4R4!V4#)z- Decorator marking a test that requires apex ztest requires apex)rrr(rs&r require_apexrH7!   024H I) TTrcL\P!\4R4!V4#)z- Decorator marking a test that requires aqlm ztest requires aqlm)rrr*rs&r require_aqlmrK>rIrcL\P!\4R4!V4#)z- Decorator marking a test that requires vptq ztest requires vptq)rrrrs&r require_vptqrMErIrcL\P!\4R4!V4#)z- Decorator marking a test that requires spqr ztest requires spqr)rrrors&r require_spqrrOLrIrcL\P!\4R4!V4#)z+ Decorator marking a test that requires av ztest requires av)rrr,rs&r require_avrQSs    02D Ei PPrcL\P!\4R4!V4#)z/ Decorator marking a test that requires decord ztest requires decord)rrr3rs&rrequire_decordrSZrQrcL\P!\4R4!V4#)z Decorator marking a test that requires the bitsandbytes library. Will be skipped when the library or its hard dependency torch is not installed. ztest requires bitsandbytes)rrr-rs&rrequire_bitsandbytesrUa"   8:\ ]^g hhrcL\P!\4R4!V4#)z- Decorator for compressed_tensors dependency z test requires compressed_tensors)rrr0rs&rrequire_compressed_tensorsrfs"   >@Bd efo pprcL\P!\4R4!V4#)z% Decorator for fbgemm_gpu dependency ztest requires fbgemm-gpu)rrr7rs&rrequire_fbgemm_gpurhrrcL\P!\4R4!V4#)z Decorator for quark dependency ztest requires quark)rrrdrs&r require_quarkrjrVrcp\P!\4;'d \4R4!V4#)z; Decorator marking a test that requires higgs and hadamard z/test requires flute and fast_hadamard_transform)rrr<rErs&rrequire_flute_hadamardrls4   88!6!8:k  rcL\P!\4R4!V4#)z4 Decorator marking a test that requires fouroversix ztest requires fouroversix)rrr=rs&rrequire_fouroversixrnrrcL\P!\4R4!V4#)z= Decorator marking a test that requires fp_quant and qutlass ztest requires fp_quant)rrr>rs&rrequire_fp_quantrprYrcL\P!\4R4!V4#)z0 Decorator marking a test that requires qutlass ztest requires qutlass)rrrers&rrequire_qutlassrrrYrcL\P!\4R4!V4#)z3 Decorator marking a test that requires phonemizer ztest requires phonemizer)rrr\rs&rrequire_phonemizerrtrrcL\P!\4R4!V4#)z4 Decorator marking a test that requires pyctcdecode ztest requires pyctcdecode)rrr_rs&rrequire_pyctcdecodervrrcL\P!\4R4!V4#)z. Decorator marking a test that requires numba ztest requires numba)rrrUrs&r require_numbarxrVrcL\P!\4R4!V4#)z0 Decorator marking a test that requires librosa ztest requires librosa)rrrNrs&rrequire_librosarzrYrcL\P!\4R4!V4#)z9 Decorator marking a test that requires python-multipart ztest requires python-multipart)rrrRrs&rrequire_multipartr|s"   579Y Z[d eercL\P!\4R4!V4#)z5 Decorator marking a test that requires liger_kernel ztest requires liger_kernel)rrrOrs&rrequire_liger_kernelr~rVrcL\P!\4R4!V4#)z1 Decorator marking a test that requires essentia ztest requires essentia)rrr5rs&rrequire_essentiarrYrcL\P!\4R4!V4#)z4 Decorator marking a test that requires pretty_midi ztest requires pretty_midi)rrr]rs&rrequire_pretty_midirrrc2\P!V4RJ#r)shutilwhich)cmds&r cmd_existsrs << D ((rcN\P!\R4R4!V4#)z8 Decorator marking a test that requires `/usr/bin/time` /usr/bin/timeztest requires /usr/bin/time)rrrrs&rrequire_usr_bin_timers#   z/:@Bs t rcL\P!\4R4!V4#)0 Decorator marking a test that requires jumanpp ztest requires jumanpp)rrrKrs&rrequire_jumanpprrYrcL\P!\4R4!V4#)rztest requires cython)rrr2rs&rrequire_cythonrrQrcL\P!\4R4!V4#)zi Decorator marking a test that requires TikToken. These tests are skipped when TikToken isn't installed. ztest requires TikToken)rrrrrs&rrequire_tiktokenr&rYrcL\P!\4R4!V4#)ze Decorator marking a test that requires speech. These tests are skipped when speech isn't available. r)rrrnrs&rrequire_speechr-s"   246P QR[ \\rcL\P!\4R4!V4#)z/ Decorator marking a test that requires openai ztest requires openai)rrrWrs&rrequire_openair4rQrcL\P!\4R4!V4#)zg Decorator marking a test that requires the serving dependencies (fastapi, uvicorn, pydantic, openai). z"test requires serving dependencies)rrrlrs&r require_server;s"   135Y Z[d eercL\P!\4R4!V4#)zu Decorator marking a test that requires mistral-common. These tests are skipped when mistral-common isn't available. ztest requires mistral-common)rrrQrs&rrequire_mistral_commonrBrdrcb\4'd^RIpVPP4#^#)z% Return the number of available gpus N)rurrr)rs r get_gpu_countrIs&zz&&((rc\P!4^,^,p\PP \PP V44pVP R4'g"\PP V4pK8V'd \PPW 4#V#)z Args: append_path: optional path to append to the tests dir path Return: The full path to the `tests` dir, so that the tests can be invoked from anywhere. Optionally `append_path` is joined after the `tests` dir the former is provided. tests)inspectstackrpathabspathdirnameendswithjoin) append_pathcaller__file__ tests_dirs& r get_tests_dirrUsw]]_Q'*N ?@I  ))GGOOI. ww||I33rc0V^8dQhR\R\/#)r8trainerreturn)rr)r<s"rr=r=lsSrctVPpVP4pVPWR7pV^,pV#))r dataloader)rget_train_dataloaderset_initial_training_values)r training_argstrain_dataloaderinitial_training_valuessteps_per_epochs& rget_steps_per_epochrls>LLM335%AA}Ar-a0O rcV^8dQhR\\\\3,,R\\\\3,RR3,/#)r8side_effect_valuesrN)listdictr;rr)r<s"rr=r=vs? % %T#u*-. %tCJt+, %rc#>"TRjxL VR,xKL5i)z Function that returns side effects for the _evaluate method. Used when we're unsure of exactly how many times _evaluate will be called. NrF)rs&revaluate_side_effect_factoryrvs#"!!  $$"s cR\P!RRV^\P4#)z^.*\r)resub MULTILINE)bufs&rapply_print_resetsrs 66(BQ 55rc\V4P4pVPVP44pVR8wgQRV RV 24hR#)zexpecting to find z in output: fNr)rlowerfind)outwhatout_pr match_strs&& rassert_screenoutrsH  $ * * ,F DJJL)I ?L0mF8LL?rc.ROpVFp\WR4K .ROpVFDp\W4'gK\W4fK%VFp\\W4VR4K KF R#) rms_norm_eps?N)rlayer_norm_epsnorm_eps norm_epsilonlayer_norm_epsilonbatch_norm_eps) text_config vision_config audio_config text_encoder audio_encoderdecoder)rhasattrr)config target_attrs target_attrattrsrs& rset_config_for_less_flaky_testrsaL$ S)$ iE 6 WV%:%F+ -{C@ ,rcdRp.ROp\4'd\V\PP4'dnVP 4FWp\ V4PPV4'gK/VF"p\W44'gK\W4R4K$ KY R#R#R#) LayerNormrN)r GroupNorm BatchNormRMSNorm BatchNorm2d BatchNorm1dBitGroupNormActivationWeightStandardizedConv2d)epsepsilonvariance_epsilon) ru isinstancernnrmodulestype__name__rrr)model target_namesrmodulers& rset_model_for_less_flaky_testrs L:L 5%((// B BmmoFF|$$--l;;(Dv,,c2)&!Crc@a]tRtRtoRtR RltRtRtRtRt Vt R#) CaptureStdia Context manager to capture: - stdout: replay it, clean it up and make it available via `obj.out` - stderr: replay it and make it available via `obj.err` Args: out (`bool`, *optional*, defaults to `True`): Whether to capture stdout or not. err (`bool`, *optional*, defaults to `True`): Whether to capture stderr or not. replay (`bool`, *optional*, defaults to `True`): Whether to replay or not. By default each captured stream gets replayed back on context's exit, so that one can see what the test was doing. If this is a not wanted behavior and the captured data shouldn't be replayed, pass `replay=False` to disable this feature. Examples: ```python # to capture stdout only with auto-replay with CaptureStdout() as cs: print("Secret message") assert "message" in cs.out # to capture stderr only with auto-replay import sys with CaptureStderr() as cs: print("Warning: ", file=sys.stderr) assert "Warning" in cs.err # to capture both streams with auto-replay with CaptureStd() as cs: print("Secret message") print("Warning: ", file=sys.stderr) assert "message" in cs.out assert "Warning" in cs.err # to capture just one of the streams, and not the other, with auto-replay with CaptureStd(err=False) as cs: print("Secret message") assert "message" in cs.out # but best use the stream-specific subclasses # to capture without auto-replay with CaptureStd(replay=False) as cs: print("Secret message") assert "message" in cs.out ```cW0nV'd\4VnRVnMRVnRVnV'd\4VnRVnR#RVnRVnR#)z=error: CaptureStd context is unfinished yet, called too earlyNznot capturing stdoutznot capturing stderr)replayrout_bufrerr_buferr)selfrrrs&&&&r__init__CaptureStd.__init__sI #:DLVDHDL-DH #:DLVDHDL-DHrcVP'd+\PVnVP\nVP'd+\P VnVP\nV#r)rsysstdoutout_oldrstderrerr_oldrs&r __enter__CaptureStd.__enter__sE <<<::DLCJ <<<::DLCJ rcVP'dqVP\nVPP 4pVP 'd \PP V4\V4VnVP'diVP\n VPP 4pVP 'd \PP V4W n R#R#r) rrrrgetvaluerwriterrrrrr)rexccaptureds&* r__exit__CaptureStd.__exit__s <<<CJ||,,.H{{{   *)(3DH <<<CJ||,,.H{{{   *H rcRpVP'dVRVP R2, pVP'dVRVP R2, pV#)rzstdout:  stderr: )rrrr)rmsgs& r__repr__CaptureStd.__repr__"sL <<< XdhhZr* *C <<< XdhhZr* *C r)rrrrrrrN)TTT r __module__ __qualname____firstlineno____doc__rr rr__static_attributes____classdictcell__ __classdict__s@rrrs$. `."   rrc:aa]tRtRtoRtRV3RlltRtVtV;t#) CaptureStdouti1z+Same as CaptureStd but captures only stdoutc*<\SV`RVR7R#)F)rrNsuperrrr __class__s&&rrCaptureStdout.__init__4 U62rrFr* rrrrrrrr __classcell__r(r!s@@rr#r#1533rr#c:aa]tRtRtoRtRV3RlltRtVtV;t#) CaptureStderri8z+Same as CaptureStd but captures only stderrc*<\SV`RVR7R#)F)rrNr%r's&&rrCaptureStderr.__init__;r*rrFr*r+r-s@@rr0r08r.rr0c<a]tRtRtoRtRtRtRtRtRt Vt R#) CaptureLoggeri?a: Context manager to capture `logging` streams Args: logger: 'logging` logger object Returns: The captured output is available via `self.out` Example: ```python >>> from transformers import logging >>> from transformers.testing_utils import CaptureLogger >>> msg = "Testing 1, 2, 3" >>> logging.set_verbosity_info() >>> logger = logging.get_logger("transformers.models.bart.tokenization_bart") >>> with CaptureLogger(logger) as cl: ... logger.info(msg) >>> assert cl.out, msg + " " ``` cWn\4Vn\P!VP4VnRVnR#)rN)loggerrior StreamHandlershr)rr6s&&rrCaptureLogger.__init__Xs- *''0rcPVPPVP4V#r)r6 addHandlerr9r s&rr CaptureLogger.__enter__^s tww' rcVPPVP4VPP 4VnR#r)r6 removeHandlerr9r7r r)rrs&*rrCaptureLogger.__exit__bs, !!$''*77##%rc"RVP R2#)z captured: r)rr s&rrCaptureLogger.__repr__fsDHH:R((r)r7r6rr9Nrr s@rr4r4?s#0 &))rr4c#"\P!4p\P!V4Rx\P!V4R# \P!T4i;i5i)aE This is a context manager to temporarily change transformers modules logging level to the desired value and have it restored to the original setting at the end of the scope. Example: ```python with LoggingLevel(logging.INFO): AutoModel.from_pretrained("openai-community/gpt2") # calls logger.info() several times ``` N)transformers_logging get_verbosity set_verbosity)level orig_levels& r LoggingLevelrIjsH&335J7**51 **:6**:6sA&A A& A##A&cFa]tRtRtoRtR V3RlRlltRtRtRtVt R#) TemporaryHubRepoiaCreate a temporary Hub repository and return its `RepoUrl` object. This is similar to `tempfile.TemporaryDirectory` and can be used as a context manager. For example: with TemporaryHubRepo(token=self._token) as temp_repo: ... Upon exiting the context, the repository and everything contained in it are removed. Example: ```python with TemporaryHubRepo(token=self._token) as temp_repo: model.push_to_hub(tmp_repo.repo_id, token=self._token) ``` NcF<V^8dQhRS[R,RS[R,RR/#)r8 namespaceNtokenrr:)r<r!s"rr=TemporaryHubRepo.__annotate__s0CC#*CC$JCRVCrcW n\P!4;_uu_4p\V4PpVeV RV 2p\ W@PR7VnRRR4R# +'giR#;i)N/)rN)rNtempfileTemporaryDirectoryrnamerrepo_url)rrMrNtmp_dirrepo_ids&&& rrTemporaryHubRepo.__init__s\  ( ( * *g7m((G$&Kq 2'zzBDM + * * *s D(()113"ll1oBll2&G%''))w/@.H.H.J.J ") J4K_K_J`ab b--7!00:=++e3 rcVP#r)ror s&rtest_file_pathTestCasePlus.test_file_paths###rc,\VP4#r)r;ror s&rtest_file_path_strTestCasePlus.test_file_path_strs4''((rcVP#r)rrr s&r test_file_dirTestCasePlus.test_file_dir """rc,\VP4#r)r;rrr s&rtest_file_dir_strTestCasePlus.test_file_dir_str$4&&''rcVP#r)rur s&rrTestCasePlus.tests_dir(s rc,\VP4#r)r;rur s&r tests_dir_strTestCasePlus.tests_dir_str,s4??##rcVP#r)rvr s&r examples_dirTestCasePlus.examples_dir0s!!!rc,\VP4#r)r;rvr s&rexamples_dir_strTestCasePlus.examples_dir_str4s4%%&&rcVP#r)rtr s&r repo_root_dirTestCasePlus.repo_root_dir8rrc,\VP4#r)r;rtr s&rrepo_root_dir_strTestCasePlus.repo_root_dir_str<rrcVP#r)rwr s&rsrc_dirTestCasePlus.src_dir@s }}rc,\VP4#r)r;rwr s&r src_dir_strTestCasePlus.src_dir_strDs4==!!rc 4\PP4pVPVP.pRVP 9dVP VP4VP VPRR44RPV4VR&V#)a Return a copy of the `os.environ` object that sets up `PYTHONPATH` correctly, depending on the test suite it's invoked from. This is useful for invoking external programs from the test suite - e.g. distributed training. It always inserts `./src` first, then `./tests` or `./examples` depending on the test suite type and finally the preset `PYTHONPATH` if any (all full resolved paths). z /examples PYTHONPATHr:) rrcopyrrrappendrgetr)renvpathss& rget_envTestCasePlus.get_envHsxjjoo'')9)9: $00 0 LL.. / SWW\2./HHUOL rNc VeVfRpVfRp\V4P4pVPR4'g\RV R24hVRJd/VP 4'd\ P !VRR7VPRRR7M!VfRpVfRp\P!4pVRJdVPPV4V'd\V4P4#T#)a Args: tmp_dir (`string`, *optional*): if `None`: - a unique temporary path will be created - sets `before=True` if `before` is `None` - sets `after=True` if `after` is `None` else: - `tmp_dir` will be created - sets `before=True` if `before` is `None` - sets `after=False` if `after` is `None` before (`bool`, *optional*): If `True` and the `tmp_dir` already exists, make sure to empty it right away if `False` and the `tmp_dir` already exists, any existing files will remain there. after (`bool`, *optional*): If `True`, delete the `tmp_dir` at the end of the test if `False`, leave the `tmp_dir` and its contents intact at the end of the test. return_pathlib_obj (`bool`, *optional*): If `True` will return a pathlib.Path object Returns: tmp_dir(`string`): either the same value as passed via *tmp_dir* or the path to the auto-selected tmp dir TFz./zI`tmp_dir` can only be a relative path, i.e. `./some/path`, but received `` ignore_errorsrqexist_ok) rrprrexistsrrmtreemkdirrRmkdtemprmr)rrVbeforeafterreturn_pathlib_objrs&&&&& rget_auto_remove_tmp_dir$TestCasePlus.get_auto_remove_tmp_dirZs4   ~}=((*D%%d++ _`g_hhij ~$++-- gT: JJtdJ 3~}&&(G D=  " " ) )' 2*"mockenv_context..s?13,QQs N)rrsetr frozensetupdatepop)rerstompedr update_after remove_afterrs*, @rmockenv_contextrs  **C \\rF \\rF6;;=!CK/3sxxz?BG'./w!s1vIwL/???L+ 6#)*6aD 6*  < )*\\*0 + + < )*\\**s_%EAE7D  E!D6DDE,D EDEE/E  EEc`RpV\9d!VPVRRRR7^\V&R#R#)z This function is to be called from `conftest.py` via `pytest_addoption` wrapper that has to be defined there. It allows loading both `conftest.py` files at once without causing a failure due to adding the same `pytest` option. z--make-reportsstoreFzSgenerate report files. The value of this option is used as a prefix to report names)actionrhelpN)pytest_opt_registered addoption)parseroptions& rpytest_addoption_sharedr sCF ** f  )*f%+rc ^RIHp\V4'gRpVPpVP 4pVP P pVPpRV 2p\V4PRRR7R"Uu/uF pW RV R2bK p p.p VPP4F1p V F(p \V R4'gKV PV 4K* K3 V 'dV PRRR7\V R,R4;_uu_4p RpV P!R4\#V 4FywrV P$V8d*V P!\V 4V, RV R24M>V P!V P$R RV P&R RV P( R24K{ RRR4RpRVP n\V R,R4;_uu_4p V!W=4VnVP-4RRR4\V R ,R4;_uu_4p V!W=4VnV!V4RRR4R VP n\V R ,R4;_uu_4p V!W=4VnVP-4RRR4\V R,R4;_uu_4p V!W=4VnVP/4RRR4\V R ,R4;_uu_4p V!W=4VnVP14VP14RRR4R!Vn\V R ,R4;_uu_4p V!W=4VnVP34RRR4\V R ,R4;_uu_4p V!W=4VnVP54RRR4W@nW`nWSP nR#uupi +'giEL/;i +'giEL;i +'giEL;i +'giEL;i +'giEL^;i +'giEL!;i +'giL;i +'giL;i)#ai Generate multiple reports at the end of test suite run - each report goes into a dedicated file in the current directory. The report files are prefixed with the test suite name. This function emulates --duration and -rA pytest arguments. This function is to be called from `conftest.py` via `pytest_terminal_summary` wrapper that has to be defined there. Args: - tr: `terminalreporter` passed from `conftest.py` - id: unique id like `tests` or `examples` that will be incorporated into the final reports filenames - this is needed as some jobs have multiple runs of pytest, so we can't have them overwrite each other. NB: this functions taps into a private _pytest API and while unlikely, it could break should pytest do internal changes - also it calls default internal methods of terminalreporter which can be hijacked by various `pytest-` plugins and interfere. )create_terminal_writerrzreports/Tr durationserrors failures_longfailures_short failures_linestats summary_shortwarningsrQz.txtdurationcVP#r)rxs&rr.pytest_terminal_summary_main..L sr)rreversewg?zslowest durations z durations < z secs were omittedz02.2fzs z<8 rNc VPR4pV'gR#VPRR4VFpVPV4pVPRVRRR7\P!RRVP ^\P \P,4pVPPV4K R#) failedN=zFAILURES SHORT STACKrzT)redboldz.*_ _ _ (_ ){10,}_ _ r) getreports write_sep_getfailureheadlinerr longreprtextrDOTALL_twline)trreportsreprlongreprs& rsummary_failures_short.summary_failures_shortV s--)  S01C((-C LLct$L 7vv6CQQ% O+ . O?  P  P' * P; > Q  Q" % Q5 c&a]tRtRtoRtRtVtR#) _RunOutputi c*WnW nW0nR#r) returncoderr)rr*rrs&&&&rr_RunOutput.__init__ s$  r)r*rrN)rrrrrrrr s@rr(r( srr(ch"VP4GRjxL pV'd V!V4K+R#L5iTN)readline)streamcallbackrs&& r _read_streamr1 s) __&&  TN  's20 2 2c$V^8dQhR\/#r8rr()r<s"rr=r= s&0&0bl&0rc aaaa "V'd\RRPV44\P!V^,.VR,O5RVR\PP R\PP RV/GRjxL p.o.oR V3Rllo \P !\P!\VPVV 3R l44\P!\VPVV 3R l44.VR 7GRjxL \VP 4GRjxL SS4#LL)L5i) z Running: rrNNstdinrrrNc<VPR4P4pVPV4S'g\W0VR7R#R#)utf-8)fileN)decoderstriprprint)rsinkpipelabelquiets&&&&rtee_stream_subprocess..tee s8{{7#**, D %D )rc8<S!VS\PRR7#)zstdout:r@)rr)lrrBs&rr$_stream_subprocess.. QSZZ_hAirc8<S!VS\PRR7#)zstderr:rE)rr)rFrrBs&rrrG rHrtimeout)r) r=rasynciocreate_subprocess_exec subprocessPIPEwait create_taskr1rrr() rrr7rKrAechoprrrBs &&&&f& @@@r_stream_subprocessrT s  mSXXc]+,, A R!!&&  !!&&    A" C C* ,,    QXX7i j k    QXX7i j k    AFFHnc3 //E 6%s7BE D; BED=E0D?1 E=E?Ec$V^8dQhR\/#r3r4)r<s"rr=r= s`jrc \P!4pVP\WW#WER74pRP V4pVP ^8d:RP VP 4p \RV RVP RV 24hVP'g"VP 'g\RV R24hV#))rr7rKrArRrrrz' failed with returncode z, The combined stderr from workers follows: z' produced no output.) rLget_event_looprun_until_completerTrr*r RuntimeErrorr) rrr7rKrArRloopresultcmd_strrs &&&&&& rrr s  ! ! #D  $ $3uU^FhhsmG 16==)y1&2C2C1DE::@ C   ===Qwi'<=>> Mrc\PPRR4p\P!RRV^\P 4p\ V4#)z Returns an int value of worker's numerical id under `pytest-xdist`'s concurrent workers `pytest -n N` regime, or 0 if `-n 1` or `pytest-xdist` isn't being used. PYTEST_XDIST_WORKERgw0z^gwr)rrrrrrr)workers rpytest_xdist_worker_idra s= ZZ^^15 9F VVFB2<< 8F v;rc^RIpVPVPVP4;_uu_4pVPR4VP 4^,uuRRR4# +'giR#;i)a  Returns a free port number that can be fed to `torch.distributed.launch`'s `--master_port` argument. Binds to port 0 to let the OS assign an available port, avoiding collisions from hardcoded ports and TCP TIME_WAIT issues between sequential subprocess launches. N)rr)socketAF_INET SOCK_STREAMbind getsockname)rcss rget_torch_dist_unique_portri sL v~~v'9'9 : :a w}}q! ; : : :s (A&& A7 c a^RIp\V\4'dVUu.uFp\VS4NK up#\V\4'd3\;QJd.V3RlV4FNK 5#!V3RlV44#\WP 4'd\VP 44#\V\4'd:VP4UUu/uFwrE\VS4\VS4bK upp#\V\\VP34'gVfV#\4'd;\V\P4'd\VP 4S4#\V\4'd \!VS4#\WP"VP$VP&34'd\VP)4S4#\+R\-V4 24huupiuuppi)z Simplifies an object by rounding float numbers, and downcasting tensors/numpy arrays to get simple equality test within tests. Nc3<<"TFp\VS4xK R#5ir)nested_simplify)ritemdecimalss& rr"nested_simplify.. sE_T844szNot supported: )numpyrrrltuplendarraytolistr itemsr;rint64rurTensorrroundint32float32float16rmrvr)objrnnprmrvs&f rrlrl s| #t>#s* **a !%(mc%(//2C.DD "a'''$9??$+ ++ sB%C*>"C** C; c`\V\PP4'dV#W3#r)r collectionsabcrrs&r to_2tupler2 s%![__--.. 6Mrc]tRtRtRtR#)SubprocessCallExceptioni9 rFN)rrrrrrFrrrr9 srrc:V^8dQhR\\,/#)r8command)rr;)r<s"rr=r== scrcZ\P!V\PR7pV'd&\VR4'dVP R4pV#R# \P d?p\ RRPT4 RTPP 4 24ThRp?ii;i)z Runs `command` with `subprocess.check_output` and will potentially return the `stdout`. Will also properly capture if an error occurred while running `command` )rr;r9z Command `rz$` failed with the following error: N) rN check_outputSTDOUTrr;CalledProcessErrorrroutput)r return_stdoutrr s&& r run_commandr= s ((9J9JK vx((w/M   ( (%)**PQRQYQYQ`Q`QbPc d  s-A$AB*,9B%%B*cja]tRtRtoRtRtV3RlRltV3RlRlt]V3RlR l4t R t Vt R #) RequestCounteriN an Helper class that will count all requests made online. Might not be robust if urllib3 changes its logging format but should be good enough for us. Usage: ```py with RequestCounter() as counter: _ = AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert") assert counter["GET"] == 0 assert counter["HEAD"] == 1 assert counter.total_calls == 1 ``` cpa\\4Sn\P!4Sn.SnV3Rlp^RIp\P!VPPRV!VPPP4R7Sn SPP4SnS#)c<aVV3RlpV#)cp<SPP\P!44S!V/VB#r) _extra_infor threading get_ident)rrfuncrs*,rwrapHRequestCounter.__enter__..patched_with_thread_info..wrapd s0  '' (;(;(=>T,V,,rrF)rrrsf rpatched_with_thread_info:RequestCounter.__enter__..patched_with_thread_infoc s -KrNdebug) side_effect)rr_counterrr _thread_idrurllib3robjectconnectionpoollogrpatcherstartr)rrrsf rr RequestCounter.__enter__^ s#C( #--/  ||  " " & &=UV]VlVlVpVpVvVv=w  LL&&(  rc<V^8dQhRR/#r8rNrF)r<r!s"rr=RequestCounter.__annotate__r s  4 rc"\VPP4\VP48XgQh\ VPVPP4Fwr4W0P 8wdKVP R,R8XdK1VP ^,VP R,,pRF+pWe9gK VPV;;,^, uu&K K VPP4R#)r8i3r6Nr) HEADGETPOSTPUTDELETECONNECTOPTIONSTRACEPATCH) r rcall_args_listrziprrrrstop)rrr thread_idcallrmethods&*, rrRequestCounter.__exit__r s499++,D4D4D0EEEE"4#3#3TYY5M5MNOIOO+yy}#))A,2.Cj=MM&)Q.)k O rc&<V^8dQhRS[RS[/#)r8rrr;r)r<r!s"rr=r s""s"s"rc(VPV,#r)r)rrs&&r __getitem__RequestCounter.__getitem__ s}}S!!rc <V^8dQhRS[/#r3r)r<r!s"rr=r s++S+rcH\VPP44#r)sumrrr s&r total_callsRequestCounter.total_calls s4=='')**r)rrrrrN) rrrrrr rrrrrrr s@rrrN s7 (  ""++rrcXV^8dQhR\R\R,R\R,/#)r8 max_attemptswait_before_retryN description)rrr;)r<s"rr=r= s,&&3&ut|&Y\_cYc&rcaaVV3RlpV#)aw To decorate flaky tests. They will be retried on failures. Please note that our push tests use `pytest-rerunfailures`, which prompts the CI to rerun certain types of failed tests. More specifically, if the test exception contains any substring in `FLAKY_TEST_FAILURE_PATTERNS` (in `.circleci/create_circleci_config.py`), it will be rerun. If you find a recurrent pattern of failures, expand `FLAKY_TEST_FAILURE_PATTERNS` in our CI configuration instead of using `is_flaky`. Args: max_attempts (`int`, *optional*, defaults to 5): The maximum number of attempts to retry the flaky test. wait_before_retry (`float`, *optional*): If provided, will wait that number of seconds before retrying the test. description (`str`, *optional*): A string to describe the situation (what / where / why is flaky, link to GH issue/PR comments, errors, etc.) c<a\P!S4VVV3Rl4p\P!\R4!V4#)c <^pVS8d S!V/VB#S!V/VB# \dNp\PRT RT RS R24Se\P!S4T^, pRp?KkRp?ii;i)rzTest failed with z at try rQrN)rvr6errortimesleep)rr retry_countrr test_func_refrs*, rr ,is_flaky..decorator..wrapper sK ,%($9&99!$1&1 1 !%LL#4SE+aP\~]^!_`(4 #451$K %s A4AA//A4z test is flaky)r rrr_run_flaky_tests)rr rrsf rrDis_flaky..decorator s9  ' 2 ( 2""#3_EgNNrrF)rrrrDsff& ris_flakyr s&O& rc>V^8dQhR\R\R,/#)r8rrNrr)r<s"rr=r= sC rc\^RIHpT!TT;'g^VRJ\P3R7#)a  To decorate tests that download from the Hub. They can fail due to a variety of network issues such as timeouts, connection resets, etc. Uses exponential backoff starting from `wait_before_retry`. Args: max_attempts (`int`, *optional*, defaults to 5): The maximum number of attempts to retry the flaky test. wait_before_retry (`float`, *optional*, defaults to 2): If provided, the initial delay in seconds before the first retry. Subsequent retries use exponential backoff with jitter. )retryN) max_retries initial_delayjitter exceptions) utils.genericrhttpx HTTPError)rrrs&& r hub_retryr s4%  ',,1 ,OO%  rcp\4'd&^RIpVPP^4!V4#V#)a Decorator marking a test with order(1). When pytest-order plugin is installed, tests marked with this decorator are guaranteed to run first. This is especially useful in some test settings like on a Gaudi instance where a Gaudi device can only be used by a single process at a time. So we make sure all tests that run in a subprocess are launched first, to avoid device allocation conflicts. N)rbrrorderrs& r run_firstr s0!""{{  #I..rchVf*\\PPRR44pRp\P !V4pVP ^4pVP^4pVPW#R7VPWWs3R7pVP4VPVR7p VP4VPVR7X R,eVPV R, 4R#R# \d,p TP4TPT 4Rp ? LkRp ? ii;i)a To run a test in a subprocess. In particular, this can avoid (GPU) memory issue. Args: test_case (`unittest.TestCase`): The test that will run `target_func`. target_func (`Callable`): The function implementing the actual testing logic. inputs (`dict`, *optional*, defaults to `None`): The inputs that will be passed to `target_func` through an (input) queue. timeout (`int`, *optional*, defaults to `None`): The timeout (in seconds) that will be passed to the input and output queues. If not specified, the env. variable `PYTEST_TIMEOUT` will be checked. If still `None`, its value will be set to `600`. NPYTEST_TIMEOUT600spawnrJ)targetrr)rrrrmultiprocessing get_contextQueue JoinableQueueputProcessr task_donerv terminatefailr) r target_funcinputsrK start_methohdctx input_queue output_queueprocessresultsr s &&&& rrun_test_in_subprocessr sbjjnn%5u=>M  % %m 4C))A,K$$Q'LOOFO,kk3WkXG MMO""7"3  LLL!w#''*+-$ qs"C;; D1!D,,D1cRaa^RIo\P!S4VV3Rl4pV#)z To decorate a test to run in a subprocess using the `subprocess` module. This could avoid potential GPU memory issues (GPU OOM or a test that causes many subsequential failing with `CUDA error: device-side assert triggered`). Nc<\P!RR4R8Xd S !V/VBR#RP\PP R4P R4RR4p\ P!\P4pRVR&RVR&RV9d\VR,PP4p\V4FcwrVWaR,P9gKVP R4R ,pRPS PR , .V,4WE&Ke \P R R .V,pVUu.uF qfR 8wgK VNK ppM\P R R V .p\P !WCRRR7R#uupi \P"Ed+pTP$P'4pTP R4p RT ^,9dRp T R ,Fp T P)R4'd8T \+R4Rp RPT P R4R ,4p KQT P)R4'd!T P-R4'd RT 9dM"\+T 4^8gKT RT 2, p K RT ,p T .T ,p RPT 4pS P/TRR7hRp?ii;i)_INSIDE_SUB_PROCESSN1rPYTEST_CURRENT_TESTtrueCI pytestconfig::r6__file__z-mrz --no-summaryT)rcheckcapture_outputrz= test session starts =rzFAILED  - rz failed in z (subprocess) F)pytracer)rgetenvrrrrrdeepcopyrinvocation_paramsrr __globals__r executablerNrunrrr;rr rr)rrtestrridxrr exception_messagertextrrrs*, rr *run_test_using_subprocess..wrapper s~ 99*D 1S 8 $ !& !88BJJNN+@AGGLSbQRD( DmmBJJ/-0)*#D "V+"6.#9#K#K#P#PQG"+G"4~ 6 ; ;;#'::d#3B#7D+/999I9I*9U8V5X[_5_+`GL#5#&..!1D(CgMG*1I'Q.5Hqq'GIG#&..!1D(tfNGwtDQ J 00 D$%HHOO$5!)//5-a8D %b ??955#'I(8#9D#%774::e+>> )(.*?```))flagsFz load_dataset(z# doctest: +IGNORE_RESULTz(>>> .*load_dataset\(.*)z\1 # doctest: +IGNORE_RESULTz>>>z...zcuda|to\(0\)|device=0Tr)rrrrrsearchr)stringskip_cuda_testscodeblock_pattern codeblocks is_cuda_foundr% codeblockmodified_strings&& rpreprocess_stringr(K sB+299EJM!*-  i ',Gy,XFF#>@_ajkJM i 5I#5 2I>> M .O ''*- rcaa]tRtRtoRt]P !R]P]P,4t ] PPRR4R8Ht R V3RlltV3RltR tVtV;t#) HfDocTestParserih a Overwrites the DocTestParser from doctest to properly parse the codeblocks that are formatted with black. This means that there are no extra lines at the end of our snippets. The `# doctest: +IGNORE_RESULT` marker is also added anywhere a `load_dataset` call is made as a print would otherwise fail the corresponding line. Tests involving cuda are skipped base on a naive pattern that should be updated if it is not enough. a # Source consists of a PS1 line followed by zero or more PS2 lines. (?P (?:^(?P [ ]*) >>> .*) # PS1 line (?:\n [ ]* \.\.\. .*)*) # PS2 lines \n? # Want consists of any non-blank lines that do not start with PS1. (?P (?:(?![ ]*$) # Not a blank line (?![ ]*>>>) # Not a line starting with PS1 # !!!!!!!!!!! HF Specific !!!!!!!!!!! (?:(?!```).)* # Match any character except '`' until a '```' is found (this is specific to HF because black removes the last line) # !!!!!!!!!!! HF Specific !!!!!!!!!!! (?:\n|$) # Match a new line or end of string )*) SKIP_CUDA_DOCTEST0rc L<\WP4p\SV` W4#)z Overwrites the `parse` method to incorporate a skip for CUDA tests, and remove logs and dataset prints before calling `super().parse` )r(r"r&r)rr!rTr(s&&&rrHfDocTestParser.parse s$ #6+?+?@w}V**rc&<V^8dQh/S[;R&#)r8r")bool)r<r!s"rr=HfDocTestParser.__annotate__h sDKErrF)z)rrrrrrcompilerVERBOSE _EXAMPLE_RErrrr"r__annotate_func__rrr,r-s@@rr*r*h sZ** \\BJJ &K&JJNN+>DKO+Krr*c6a]tRtRtoRtV3RlRltRtVtR#)HfDoctestModulei zw Overwrites the `DoctestModule` of the pytest package to make sure the HFDocTestParser is used when discovering tests. c0<V^8dQhRS[S[,/#r3)rr)r<r!s"rr=HfDoctestModule.__annotate__ sC_C_+.C_rc#"!RR\P4pVPPR8XdaVPP P VPVPPR4VPPR7pMF\VPVPPVPPR4R7pV!\4R7p\V4p\R V\!4\#VP4R 7pVP%XVP&4F;pVP('gK\*P,!WPWVR 7xK= R # \dBTPPR4'd\RTP,4Lhi;i5i) cNaa]tRtRtoRtV3RltV3RlV3RlltRtVtV;t #)7HfDoctestModule.collect..MockAwareDocTestFinderi zA hackish doctest finder that overrides stdlib internals to fix a stdlib bug. https://github.com/pytest-dev/pytest/issues/3456 https://bugs.python.org/issue25532 c <\V\4'd\VRV4p\VR4'd\P !V4p\ SV`VV4#)zDoctest code does not take into account `@property`, this is a hackish way to fix it. https://bugs.python.org/issue17446 Wrapped Doctests will need to be unwrapped so the correct line number is returned. This will be reported upstream. #8796 fget __wrapped__)rrrrrunwrapr& _find_lineno)rr{ source_linesr(s&&&rrADHfDoctestModule.collect..MockAwareDocTestFinder._find_lineno sUc8,,!#vs3C3 ..!..-Cw+ rc<V^8dQhRR/#rrF)r<r!s"rr=DHfDoctestModule.collect..MockAwareDocTestFinder.__annotate__ s  TX rc <\V4'dR#\4;_uu_4\SV` WW4WVV4RRR4R# +'giR#;ir)rrr&_find) rrr{rTrrBglobsseenr(s &&&&&&&&rrG=HfDoctestModule.collect..MockAwareDocTestFinder._find s<c??-//GMD,t0///s A A rF) rrrrrrArGrrr,r-s@@rMockAwareDocTestFinderr< s   (   rrKz conftest.py importmode)rootpath)rootmodedoctest_ignore_import_errorszunable to import module %r)rF)verbose optionflagscheckercontinue_on_failure)rTrunnerdtestN)doctest DocTestFinderrrTr pluginmanager_importconftest getoptionrMrrr rr*rrrrrrrkr from_parent)rrKrfinderrRrUrs& rcollectHfDoctestModule.collect sT! W%:%:! F 99>>] *[[..>>  %%l3--?F $II--..|<(/@A%d+ # N 8 E   KK8D}}}!--d6^^9! ;;''(FGG5 AB  s8BGAFA4G*G+G ,G G G  GrFN)rrrrrr^rrr s@rr7r7 s C_C_rr7cRV^8dQhR\R\\\3,/#)r8devicedispatch_table)r;rr)r<s"rr=r= s#  c 4X ;N rcW9d1\VR,4'g VR,#VR,!V/VB#W,p\V4'gV#V!V/VB#r)r)rarbrrfns&&*, r_device_agnostic_dispatchre s` #y122!), ,i($9&99  B B<< t v rrc^#rrFrFrrrr sqrc^#)rrFrFrrrr s1rc^#rgrFrFrrrr% sqrxlac0V^8dQhR\R\/#)r8raseedr)r<s"rr=r=Q sHHH3Hrc$\V\V4#r)reBACKEND_MANUAL_SEED)rarls&&rbackend_manual_seedroQ s $V-@$ GGrc$V^8dQhR\/#r8rar:)r<s"rr=r=U BBBrc"\V\4#r)reBACKEND_EMPTY_CACHEras&rbackend_empty_cachervU  $V-@ AArc$V^8dQhR\/#rqr:)r<s"rr=r=Y sCCCrc"\V\4#r)reBACKEND_DEVICE_COUNTrus&rrrY s $V-A BBrc$V^8dQhR\/#rqr:)r<s"rr=r=] sQQsQrc"\V\4#r)re"BACKEND_RESET_MAX_MEMORY_ALLOCATEDrus&r"backend_reset_max_memory_allocatedr~] s $V-O PPrc$V^8dQhR\/#rqr:)r<s"rr=r=a sNNCNrc"\V\4#r)reBACKEND_RESET_PEAK_MEMORY_STATSrus&rbackend_reset_peak_memory_statsra s $V-L MMrc$V^8dQhR\/#rqr:)r<s"rr=r=e sKKKrc"\V\4#r)reBACKEND_MAX_MEMORY_ALLOCATEDrus&rbackend_max_memory_allocatedre s $V-I JJrc$V^8dQhR\/#rqr:)r<s"rr=r=i sGGSGrc"\V\4#r)reBACKEND_MEMORY_ALLOCATEDrus&rbackend_memory_allocatedri s $V-E FFrc$V^8dQhR\/#rqr:)r<s"rr=r=m rrrc"\V\4#r)reBACKEND_SYNCHRONIZErus&rbackend_synchronizerm rwrc$V^8dQhR\/#rqr:)r<s"rr=r=q sOOSOrc"\V\4#r)re BACKEND_TORCH_ACCELERATOR_MODULErus&r backend_torch_accelerator_modulerq s $V-M NNrTRANSFORMERS_TEST_DEVICE_SPECzISpecified path to device spec file is not a file or not found. Received '.pyz;Provided device spec file was not a Python file! Received 'z.Device spec file did not contain `DEVICE_NAME`zBMismatch between environment variable `TRANSFORMERS_TEST_DEVICE` 'z' and device found in spec 'z' zNEither unset `TRANSFORMERS_TEST_DEVICE` or ensure it matches device spec name.cRV^8dQhR\\\3,R\/#)r8device_fn_dictattribute_name)rr;r)r<s"rr=r= s$  T#x-5H Z] rc\\V4pW \&R# \d*pRT9d\RT R\ R24ThRp?R#Rp?ii;i)rrz` not found in 'z)' and no default fallback function found.N)rdevice_spec_modulerAttributeErrordevice_spec_path)rrspec_fnr s&& rupdate_mapping_from_specr s` !"4nE/6|,! N2(N++;V^8dQhR\R,R\/#)r8 propertiesNr)PackedDevicePropertiesr)r<s"rr=r= s!%%&-%%rczVf \4#VwrVfRRrCM\V\4'dTRrCMVwr4WV3#)z Unpack a `PackedDeviceProperties` tuple into consistently formatted `DeviceProperties` tuple. If properties is None, it is fetched. N)rrr)r device_type major_minorr"r#s& runpack_device_propertiesr sN $&&)KTu K % %"Du"  u $$rca]tRtRtoV3RlRltV3RlRlt]V3RlRl4t]V3RlR l4tRV3R lR llt R t Rt Vt R #) Expectationsi c <V^8dQhRS[/#r3)r)r<r!s"rr=Expectations.__annotate__ s>>>rc 4VP\44#)a% Find best matching expectation based on environment device properties. We look at device_type, major and minor versions of the drivers. Expectations are stored as a dictionary with keys of the form (device_type, (major, minor)). If the major and minor versions are not provided, we use None. )find_expectationrr s&rget_expectationExpectations.get_expectation s $$%:%<==rcF<V^8dQhRS[S[S[S[3,,/#r3)rrqrr)r<r!s"rr=r s'PP$u%5s%:;<PrcVPP4UUu.uFwr\V4V3NK upp#uuppir)datartr)rrr}s& runpackedExpectations.unpacked s4=AYY__=NO=NTQ)!,a0=NOOOs:c&<V^8dQhRS[RS[/#)r8expectation_keyr)rr0)r<r!s"rr=r s77$:7t7rc j\;QJdRV4F 'dK R# R#!RV44#)a This function returns True if the expectation_key is the Default expectation (None, None). When an Expectation dict contains a Default value, it is generally because the test existed before Expectations. When we modify a test to use Expectations for a specific hardware, we don't want to affect the tests on other hardwares. Thus we set the previous value as the Default expectation with key (None, None) and add a value for the specific hardware with key (hardware_type, (major, minor)). c3("TFqRJxK R#5irrF)rrSs& rr*Expectations.is_default.. s6o9osFT)r)rs&r is_defaultExpectations.is_default s-s6o6ss6s6s6o666rc,<V^8dQhRS[RS[RS[/#)r8rotherr)rr)r<r!s"rr=r s$*3Crc Vwr#pVwrVp^pVe7W%8Xd1V^, pVe#W68XdV^, pVeWG8Xd V^, pMVR9d VR9dRp\PV4'dRpV#)a Returns score indicating how similar two instances of the `Properties` tuple are. Rules are as follows: * Matching `type` adds one point, semi-matching `type` adds 0.1 point (e.g. cuda and rocm). * If types match, matching `major` adds another point, and then matching `minor` adds another. * The Default expectation (None, None) is worth 0.5 point, which is better than semi-matching. More on this in the `is_default` function. g?g?)rr)rr) rrrr"r#other_device_type other_major other_minorscores && rrExpectations.score s%/! E6;3   "{'G QJE U%9 $)=QJE , ,1BFV1VE  " "5 ) )E rNc&<V^8dQhRS[RS[/#)r8rr)rr)r<r!s"rr=r7 s+;UXrc a\VP4V3RlR7wr#\PSV4^8Xd\ RS 24hV#)z Find best matching expectation based on provided device properties. We score each expectation, and to distinguish between expectations with the same score, we use the major and minor version numbers, prioritizing most recent versions. c<\PSV^,4V^,^,eV^,^,MRV^,^,eV^,^,3#R3#)rr)rr)rrs&rr/Expectations.find_expectation..? sX"":qt4Q47.!QBQ47.!Q57r)rz"No matching expectation found for )maxrrrr)rr result_keyr[s&f rrExpectations.find_expectation7 sO # MMO    j* 5 :A*NO O rcVP #r)rr s&rrExpectations.__repr__K s))rrF))NNN) rrrrrr staticmethodrrrrrrr s@rrr sW>>PP77<(rrca\PPRR4pVP4R9pV'd^RIpVP oV3RlpW!nR#R#)av Patch `torch.compile` to always use `fullgraph=True`. This is useful when some `torch.compile` tests are running with `fullgraph=False` and we want to be able to run them with `fullgraph=True` in some occasion (without introducing new tests) to make sure there is no graph break. After PR #40137, `CompileConfig.fullgraph` is `False` by default, this patch is necessary. TORCH_COMPILE_FORCE_FULLGRAPHrNc<RVR&S!V/VB#)T fullgraphrF)rr orig_methods*,rpatched0patch_torch_compile_force_graph..patcheda s"&F; // /r)yesr ontyr)rrrrrr2)force_fullgraphrrrs @rpatch_torch_compile_force_graphrO sNjjnn%DbIO%++-1UUOmm  0  rc 8\PPRR4PR4^,pVPR4wrp\P !4pYreRRRrp\ V4Fp VPV P4'gK&RV PP9gKC\V PPR,V4'gKrT pV PPR,pV Pp\W4p M Ve VPp Rp .p Rp\ V4Fp VPV P4'dLRV PP9d1\V PPR,V4'dRpM+V PPP R 8XdT p RpMV'gKV P#V 4K Rp\ V 4FPp\$P&!VVPVPP(VPP*4pTpKR XpV PPR ,p\,P.!4pRp\ V4F"p VP0V P29gK T pK$ \P4P7VP4pVPpX p^R IHp\=V4;_uu_4pVP?4pV!V4pR PAVPCV^, 4PD4pRRR4\=V4;_uu_4pVP?4pV!V4pR PAVPCV^, 4PD4pRRR4R V RR RV RV RX 2 pV RR RV RV RX 2 pVVVVV VVVVVVV3 # +'giL;i +'giLR;i)zt Collect some information about the current test. For example, test full name, line number, stack, traceback, etc. rrrr NrFTrr)Sourcerztest:  z test context: rz caller context: P--------------------------------------------------------------------------------)#rrrrrrreversedrfunctionframef_localsrfilenamerlinenof_codeco_namertypes TracebackTypef_lastif_lineno traceback extract_stackrrrrelpath_pytest._code.sourcerrreadr getstatementr) full_test_name test_file test_class test_namestack_from_inspectactual_test_file_actual_test_class test_frametest_obj test_methodr line_numberframe_of_patched_objcaptured_frames to_capturetb_next frame_infor]test_tracebackorigin_method_being_patchedr caller_frame caller_path caller_lineno test_linenorfprhsourcetest_code_contextcaller_code_context test_infos r_get_test_infor)i sZZ^^$92>DDSI!LN'5';';D'A$I9!,5((,dD+J,-   0 0%++... ,,V4i@@J{{++F3H %~~ !(6K %.( ''  OJ,-   0 0%++... ,,V4i@@J [[   ' '9 4#( J  :  " "5 ).G/  **:*:J.get_argument_name sk dCHH % %77N cmm , ,;;t$ $ cll + + # #;;t$ $rrreval)rOpositional_args keyword_argsstarargsNrzError parsing: )r rrr,rrbodyCallrrkeywordsargr SyntaxErrorrr=) code_contextr3indenttree call_noder[r<arg_namekeywordr s & r_get_call_argumentsrD sN % \%8%8%:!; ;F''f b9L!%yyF3$))SXX....II !2D$ F!~~,S1()00:& %--;;&'8'GF8$ 1?H:B>*7;;7.M/ 8   (% s#$$ %s AEBEE*E%%E*c V RV 2p\PP\PP RR4R4p\ VR4;_uu_4pVP V RR R24RRR4V# +'giT#;i)zgCombine the information about the test and the call information to a patched function/method within it.r#_PATCHED_TESTING_METHODS_OUTPUT_DIRrcaptured_info.txtaNzx========================================================================================================================)rrrrrrr)r(inforSr$s&& r_prepare_debugging_inforJsz[TF #D  RZZ^^$I2NPcdA a D6i[-.  K  Ks A<< B c.pRp\VPR4'dVPPpMb\VPR4'dG\VPPR4'd!VPPPp\VR4'dV'dVF:pVwpwrxp VP R\ V4 R2RVP RR R ^/4K< \VPR4'd&VPPP4Mp\VPR4'dU\VPPR4'd/VPPPP4\Pp V \V4n V !V.VO5/VB\WP4p V PP\!V 4,p V ^,R ,p V ^,R ,pV Uu.uF qR ,NK ppRR R2P#V4pV'd\ V 4 RR R\%V4 RR RV RR R\%V4 RR RRP#RV44 RR RR 2p\V 4!V4pV P&VnV P(VnTp V P=V P+V4huupi)a Used to report a test that has failures captured and handled by patched functions/methods (without re-raise). The patched functions/methods refer to the `patched` defined in `_patch_with_call_info`, which is applied to `torch.testing.assert_close` and `unittest.case.TestCase.assertEqual`. The objective is to avoid a failure being silence after being processed. If there is any failure that is not handled by the patched functions/methods, we add custom error message for them along with the usual pytest failure report. Nrr[_outcomemessagerrr:ztest_modeling_vit.pyr exceptionrrIz Handled Failures: (z handled): z Unhandled Failures: (z unhandled): , c3p"TF,qR, RVR, VR, RVR, 2xK. R#5i)r: rMr:rrNrF)rrIs& rr$_patched_tearDown..ss7 oYnQU6l^2d9o.tF|nAd6l^ LYns46z Note: This failure occurred after other failures analyzed by the patched assertion methods. To see the full details, temporarily disable assertion patching. zP================================================================================r)rrLrr[rr;rclear_patched_tearDown orig_tearDownrrr_testMethodName__func__captured_failuresrrr  __cause__ __context__with_traceback)rrrregular_failures_infor error_entry test_instanceexc_typeexc_objexc_tbrUrrXcaptured_exceptionscaptured_tracebackrcapturued_infocapturued_info_strenhanced_messageenhanced_exceptions&*, rrTrT&s& Ft}}h''%%  ) )gdmm6J6JH.U.U%%,,tZ  V!K9D 6M6Hv ! ( (#g,t4H--2C  " 4==( + + MM & & ( T]]H - -'$--:N:NPX2Y2Y MM ' ' - - /&33M'DJ$((($ 4 45K#,,>>r+O,A.{;*1-k:):;):Aii):N;z.33NC #$7 89:  '()    /012   oYn oopq   $""567GH':'D'D$)<)H)H&1 .  , ,-? @@C.patchedszz~~3R88// /; <  ( (9 22ODD $X 7 7 2?!/*;X''Em9 N\st(--00-AQ_vw(--00,@P^hi(--002CEUcmn(--00/CSakl(--002FHXfpqrc VV^8dQhR\R\R\R\R,/#)r8scriptnproc_per_node is_torchrunrN)r;rr0r)r<s"rr=r=s3MMSM#MDMdUYkMrc  \P!RRR7;_uu_4pVPV4VP4VP ^4V'd-RV R\ 4 RVP 2P4pMRVP .p\P!VRVRRR 7pR R R 4R # \Pdp\R TP 24hR p?ii;i +'giR #;i) zmRun the `script` using `torchrun` command for multi-processing in a subprocess. Captures errors as necessary.zw+r)rOsuffixztorchrun --nproc_per_node z --master_port rpython3T)rrrrz"The following error was captured: N) rRNamedTemporaryFilerflushseekrirTrrNrrrvr)rrrrtmprrzr s&&&& rtorchrunrs  $ $$u = = &   ,^, =,, M@ KL L M > = =s*A5C2B==C/C**C//C22 D caa\V\P4'g\P!V4pRpVP^8Xd\P!V.4pRpVP^8:g\ VP ^R4^08XdVP4PR4p\V4pVPRR4PRR4pRV9dVPRR4pKVPR R 4pR V9dVPR R 4pKV'dV^RpRS,V,pV#\V4pSfR V9;'gRV9o\P!SR7RS,R,pVRPVV3RlV44, pVRRS,,R,, p\P!R R7V#)uJFormat torch's tensor in a pretty way to be shown 👀 in the test report.FTrztensor(r)z[ [rrz Nze+ze-)sci_modez[ , c3N<"TFp\VS^,SR7xK R#5i)r) indent_levelrN)_format_tensor)rrrrs& rr!_format_tensor..os&kijdeN1r;rz\nrrqrz[]z()rzone-line)r?rOr rQ,r)r?rOr prefixrrOTFrc3,"TF pRV9xK R#5i)rNrFrrs& rr!_format_py_obj..#s%M_d!m_sc<\V4\\\39Ed8.p\V4\Jd(VP RVP 444M2\V4\\39dVP RV44\ ;QJdRV4F 'gK RM RM !RV44'd6\V4^8dR#\V4\V^,4JdR#S#\\V44^8dR#V^,\\39dS#V^,\Jd\V4^8XdS#S#R#)c38"TFp\V4xK R#5irrrs& rr<_format_py_obj..use_one_line_repr..:s,KlT!WWlc38"TFp\V4xK R#5irrrs& rrr<s,BcT!WWcrc3H"TFq\\\39xK R#5irrrqrrs& rrr?sK]ud 33]s "TF) rrrqrextendranyr rrrr;)r{ element_typescould_use_one_lineno_new_line_in_elementss& ruse_one_line_repr)_format_py_obj..use_one_line_repr4s9ud 33$&MCyD(%,,,Kcjjl,KKctUm3%,,,Bc,BBsK]KsssK]KKKs8a<#( 9DQL8#( 763}-.2$%Q'C<766&q)S0s8q=#::$65rrrrr)rr;r(rrrrrrrrtr rxrrrrrrrrqr)$r{r?rOr r quoted_stringr parenthesisp1p2elements_without_indentrrr} last_elementokovrogroupsr x_expandednot_last_elementshould_finalize_xshould_finalize_bufbuf_not_fit_into_one_line prev_type current_type type_changed orig_buf_lennot_fit_into_one_lineelement_strings one_line_formrrrs$&&&&& @@rrxrxs } sGV6 *e 3"S'689 9 3==55c""   5 (%c* %--dE:    #3 3S   #< < D T D ]]334"$ c4 (5 Va"c#hl2 #AqjzQVW#!A:;G299;-3R (.."))+b /NO 6$C.Q"c#hl2 "fqjtT`fh(..qxxz: ) 78FC JJqMRJ"c*A&BQ&FF  *!,!,cSA!TRUR\R\R^M_`cMdHeL3x!|26s)42GD1W.TRVWZWaWaWcRdehklelRmMn +y8 *.' Y#%5,#:M&+#s"$)!s +*/'"""3x !#,0)!C'03sa6A:>N7O6P0RUXY]YbYbcfgjhjckYlUm0mpw0w-,!$SbA"MM1#."* c#2h/bc(C$$MM#&CG9L s8a<$( !s"(+sa6A:6F/G.H(JSQUQZQZ[^Q_M`(`cf(f%$AMM1#& c"f$%bT,U[\U[ccQ&1*%567499S>IIU[\%**_--Bw'(--#&#%M_%M###%M_%M"M4  II?&K?aqxxz?&KLM!d=/"6Mz! "%&(8'9!;c&k!ICP]L^!^be!e ( V!%%#f,-m_=-3E2c7F& )* MM]'Ls 7].]3c\VR4;_uu_4pVPV4RRR4R# +'giR#;i)rN)rr)r:contentr#s&& r write_filergs' dCA  s 1 A c\VR4;_uu_4p\P!V4uuRRR4# +'giR#;i)rN)rjsonload)r:fhs& rread_json_filerls& dCByy} s 6 A cV]tRtRtRtRtRtRtRtRt Rt R t R t R t R tR tRtRtRtRtRtR#)Colorsiwz0ANSI color codes for terminal output formatting.zzzzzzzzzzzzzzzrFN)rrrrrRESETBOLDDIMREDGREENYELLOWBLUEMAGENTACYANWHITE BRIGHT_RED BRIGHT_GREEN BRIGHT_YELLOW BRIGHT_BLUE BRIGHT_CYANrrFrrrrwsT: E D C C E F DG D EJLMKKrrc aa]tRtRtoRt]P ]P]P,]P]P]P]P]P]P]P ]P"]P,/t0RmtR V3RlV3RllltV3RlRltRtVtV;t#) ColoredFormatteriz5Custom formatter that adds colors based on log level.cB<V^8dQhRS[R,RS[R,/#)r8fmtNdatefmtr:)r<r!s"rr=ColoredFormatter.__annotate__s#''C$J'd 'rc&<\SV`W4R#rr%)rr rr(s&&&rrColoredFormatter.__init__s &rc:<V^8dQhRS[PRS[/#)r8recordr)r LogRecordr;)r<r!s"rr=rs%TTW..T3Trc VPVP9pV'dnVPWP4pVP 4p\ P V RVP RVPR RV \ P 2 #VPPVP\ P4pVPpV VR \ P 2p\ P VPWP4 \ P 2p\ P VP \ P 2p VP 4pV RV RV RV 2#)r8) rTDIMMED_LOGGERS formatTimer getMessagerr levelnamer LEVEL_COLORSrlevelnor) rr is_dimmed timestamprMcolorrcolored_levelname colored_time colored_names && rr<ColoredFormatter.formats<KK4#6#66  =I'')Gjj\)C }C@P@PQR?SSVW^V_`f`l`l_mn n!!%%fnnfllC$$ $gi]6<<.A!**doofll&K%LV\\N[ !++v{{mFLL>B ##%s<.4E3Fc'SSrrF>rrhttpcorerequestsr_)rrrrrrDEBUGrrrINFOrWARNINGrERRORrCRITICALrrrrr<rrr,r-s@@rr r s?  vzzFKK/ fll-- v((&++(9(99 LBN''TTTrr c8V^8dQhR\P/#r3)rLogger)r<s"rr=r=s'..rc\P!R4pVP\P4VP'g\P !\ P4pVP\P4\ PP4'd\RR7pM\P!RRR7pVPV4VPV4RVn V#)aInitialize a test-specific logger with colored stderr handler and INFO level for tests. Uses a named logger instead of root logger to avoid conflicts with pytest-xdist parallel execution. Uses stderr instead of stdout to avoid deadlocks with pytest-xdist output capture. ztransformers.training_testz%Y-%m-%d %H:%M:%S)rz4%(asctime)s - %(name)s - %(levelname)s - %(message)sF)r getLoggersetLevelr'handlersr8rrisattyr  Formatter setFormatterr< propagate)r6ch formatters rinit_test_loggerr7s   ; s. ##$c"$r MemoryStatsca]tRtRtoRtRtV3RlRltV3RlRltV3RlR ltV3R lR lt V3R lR lt Rt Vt R#)CPUMemoryMonitoriz1Monitor CPU memory usage for the current process.cLRVn^VnRVn^Vn^Vn\ 4'dq^RIpVP\P!44VnVP4pVPVnVPVP4VnR#R#)CPUN) device_name _peak_rss_processr total_memory_gibr^rrrgetpidrr_to_gib)rrmem_infos& rrCPUMemoryMonitor.__init__s   !  "NN299;7DM,,.H (D $(LL1B1B$CD ! !rc&<V^8dQhRS[RS[/#r8memory_in_bytesrr)r<r!s"rr=CPUMemoryMonitor.__annotate__s66s6u6rc VR, #)zConvert bytes to GiB.rrFrrNs&&rrICPUMemoryMonitor._to_gibs"455rc&<V^8dQhRS[RS[/#rMr)r<r!s"rr=rOs;;s;u;rc \VP^8XdR#RV,VP, #)z,Convert bytes to percentage of total memory.ggY@)r rQs&&r_to_pctCPUMemoryMonitor._to_pcts)    !&):):::rc<V^8dQhRR/#rrF)r<r!s"rr=rOs>>d>rc VPeBVPP4Pp\VPV4VnR#R#)zUpdate peak memory tracking.N)rF memory_inforssrrE)r current_rsss& r _update_peakCPUMemoryMonitor._update_peaks; == $--33599K =DN %rc <V^8dQhRS[/#r3)r?)r<r!s"rr=rOs  ; rc  B\4'g\^^^^^^^4#^RIpVP4VPP 4pVP 4p\VPVP4VPVP4VPVP4VPVP4VPVP4VPVP4VPVP4R7#)zGet current memory statistics.Nrss_gibrss_pctvms_gib peak_rss_gib peak_rss_pct available_gib total_gib)r^r?rr\rFrYrrIrZrUvmsrE availabler)rrrJsys_mems& r get_statsCPUMemoryMonitor.get_statss"$$q!Q1a3 3 ==,,.'')LL.LL.LL.dnn5dnn5,,w'8'89ll7==1  rc<V^8dQhRR/#rrF)r<r!s"rr=rO4s==$=rc xVPe,VPP4PVnR#R#)zReset peak memory tracking.N)rFrYrZrEr s&rreset_peak_stats!CPUMemoryMonitor.reset_peak_stats4s+ == $!]]668<>   ,==rrAcRV^8dQhR\PR,R\/#)r8r9Nr)rr,rA)r<s"rr=r=:s$gnnt.CO_rc\4pVeD\4'd#VPRVPR R24V#VP R4V#)zBuild and initialize a CPU memory monitor. Args: logger_instance: Optional logger to log initialization info. If None, no logging is done. Returns: CPUMemoryMonitor instance. z CPU memory monitor initialized: z.2fz GiB totalz0psutil not available, memory monitoring disabled)rAr^rIrGr<)r9monitors& rbuild_cpu_memory_monitorrt:s[ G"   #CGD\D\]`Caak!l m N  # #$V W Nrc$V^8dQhR\/#)r8folderr:)r<s"rr=r=Ls55C5rc \P!V4EFp\PPW4pVP R4'd\VP RR4P RR4p\ V4p\P!V4\P!WC4KV\8XgK\PPV\4p\V4;_uu_4p\P!VP44pRRR4\P!V4RXP!49dIVR,p/pVP#4F)wrV P RR4P RR4W&K+ XVR&\VR4;_uu_4pVP%\P&!V^R74RRR4EK R# +'giL;i +'giEK;i) zConvert all safetensors files into torch bin files, to mimic saving with torch (since we still support loading bin files, but not saving them anymore)z .safetensorsz.binr pytorch_modelN weight_mapr)r?)rlistdirrrrrrrersaver$r&rrloadsr rrtrdumps) rvr:rnew_path state_dictr#indexrynew_weight_maprr}s & rconvert_all_safetensors_to_binsrLsb 6"ww||F) == ( (||NF;CCG_]H"4J IIdO JJz , , ,ww||F,>?Hdq 1668, IIdOuzz|+"<0 !#&,,.DA() .&(I(Q(QRY[j(kN%/"0E, h$$ 534%$'#%$$s0%G )(G3 G0 3 H c#a"\PoV3RlpV\nRxS\nR# S\ni;i5i)zSince we don't support saving with torch `.bin` files anymore, but still support loading them, we use this context to easily create the bin files and try to load them backc6<S!W.VO5/VB\V4R#r)r)rsave_directoryrr original_saves&&*,rnew_save2force_serialization_as_bin_files..new_savens $ @ @ @ +N ;rN)rsave_pretrained)rrs @r force_serialization_as_bin_filesrfs8 8'77  <+3' *7'-'sA%7 A AAcVV^8dQh/^\9d\\,;R&#)r8r;)__conditional_annotations__rr;)r<s"rr=r=s \E$#3s8#]E r)Fr)z1.12.0)P))NNNFF)NNFT)rl)NN)rr8r_r-)rN)rrNrr`(rr,r contextlibrrWr rrrrrrrrrrrNrrRrrrrrrrcollections.abcrrrrr r dataclassesr r r rr7rpathlibrtypingrrr unittest.mockrrhuggingface_hubrr packagingr transformersrDrr integrationsrrrrrrr integrations.deepspeedr!utilsr"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQrRrSrTrUrVrWrXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirjrkrlrmrnrorprqrrrsrtrurvrwrxryrzr{r|r}r~rrrrrrrrrrrraccelerate.staterraccelerate.utils.importsr_pytest.doctestrrrrrrr_pytest.outcomesr_pytest.pathlibrrrrSMALL_MODEL_IDENTIFIERDUMMY_UNKNOWN_IDENTIFIERDUMMY_DIFF_TOKENIZER_IDENTIFIERUSERENDPOINT_STAGINGTOKEN_COMMON_MODEL_NAMES_MAP_VLM_COMMON_MODEL_NAMES_MAP_TEXT_MODEL_TESTER_DEFAULTSrsafetensors.torchrmodeling_utilsrrhiprrrrrr get_loggerrr6rrrrrrrrrrrrrrrrrrrrrr r#r%r(r*r-r/r1r4r@rGrKrNrPrSrUrXr[r]r_rcrkrpr{r}rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrbackend import_modulerzModuleNotFoundErrorr rrrrarYrrrrrrr rrrrr%r'r/r1r3r5r7r9r;r=r@rBrDrFrHrKrMrOrQrSrUrXr[r]r_rarcrfrhrjrlrnrprrrtrvrxrzr|r~rrrrrrrrrrrrrrrrrrrrrrr#r0r4rIrKrfrrhrrrrr&rLr(r1rTrrarirlrrrvrrrrrrrrr( DocTestParserr*r7re manual_seedrn empty_cachertrrzreset_max_memory_allocatedr}max_memory_allocatedrreset_peak_memory_statsrmemory_allocatedr synchronizerrrrrrrorvrr~rrrrrris_filerrrealpathdevice_spec_dirrr import_namer DEVICE_NAMErDrrrrrrqr;rrrrrrrr)rDrJrTrrrrrrrrxrrrr2r rr;r7r> namedtupler?rArtrrr=)rs@rrsW        -LL%'"% 48 G;jjjjjjjjjjjjjjjjjjjjjjjjjjjZ?9&+" FK63"A%2 0 H} 6#%@ ": H>"$> "!4d  $  " 211&sAA A!"q#$QR%r$a1118+K]]&&d2N]]''t3NEMM5$7tCME5$/t;MNNMM  ( ( 2  &j%@&{DA,-DeT"#;UK )*>M&'8%H)*>M01LVZ[<" = : = D K? _^__eU_eVcU6L'9 0@YYW]UUYY  u&gjj,  UcaagaWWaYcYW m   ev k ^aa g\`\ g g"bjj0**89 ''0A"RZZ/zz"<= 6 !%***A*A*C*C+L>:BC  5 )?)A)A+L>:AB  5 )?)A)A+L>:AB  5 )?)A)A+L>:AB  5 )?)A)A+L>:AB   \*A  " "  ! !  ! !  ! !  ! !  L[ o X V   Rr R R  h  c Y ] YX[[W[_UUUUQYe[X_S`iqaWc][acW[fe]c)f[[Y]]Yfi . % %86M A*3*bbV3J33J3()()V 77(VV@ (X&8$$X&v/ ++B*&B)P&0&0R, "76," i "8+8+v&&R0$(.V7t:++g++++\I_fI_X     && u  5$$  && t4  '' y9  55 t4*&  // q1$  22 t4'#  ++ q1   && t4   t4($ %d+$d+%y1*3T):&'0$&7#$-q> )1~$d+(14'8$!&!6!6"'))"8"8.3ii$U+!&!6!6!&!6!6"'))"8"8.3ii$U+!&!6!6!&!6!6"'))"8"8.3ii$U+!&!6!6!&!6!6"'))"8"805 0Q0Q&u--2YY-N-N#E**/))*H*H '&+ii&@&@U#!&!6!6.3ii$U+!&!7!7!&!7!7"'**"9"9HHBBCCQQNNKKGGBBOO'"**4::&EF$%--//[\l[mn   WW]]277+;+;