+ QV-jaR6tQ0tRt^RIt^RIt^RIt^RIt^RIt^RIHt^RI H t ^RI H t ^RI Ht^RIt^RIHtHtHtHtHtHtHtHtHtHtHt^RIHt^RIHtH t H!t!H"t"H#t#H$t$H%t%H&t&H't'H(t(^R I)H*t*H+t+^R I,H-t-H.t.H/t/H0t0R t1R t2R t3]+Ph!]54t6Rs7]^kRRlt8!RR] RR7t9]Pt!R]PvPy]PzR44t>Rt?]!4PtARtBRtCR7RRlltDRRRRRR/RRlltER tFR8R!R"lltGR#R$ltHR%R&ltIR9R'R(lltJR7R)RRRR*R/R+R,llltK!R-R.4tLR/R0ltMR:R1ltNR;R2R3lltO!R4R54tPR#)environjsonsplit Exceptionloadsgetenv) instance_datadlc_container_useddlc_tagsagemaker_paramsruns_distributed_trainingtraining_job_arn account_idsagemaker_objects r'define_sagemaker_informationrs) "**-I"JKPPR *73(..s3A6 zz")),A4"HI LP` `yy!343C3O!'',Q/UYJ  "7>RYY|T2=#6=#6!#<$&8('  ' !sA!D//EEcTV^8dQhR\\,R,R\/#)r$ user_agentNr%)rAr=)r&s"r'r(r(s#s T 1Sr)cR\ R\PP4^, R\ 2p\ 4'dVR\ 4 2, p\P'd VR,#\4'd=VRRPR\4P444,, p\PPRR 4P!4\"9d VR , p\%V\&4'd7VRRPR VP444,, pV#\%V\(4'dVRV,, pV#) z> Formats a user-agent string with basic info about a request. z transformers/z ; python/z ; session_id/z; torch/z; telemetry/offz; c34"TFwrV RV 2xK R#5ir_Nr/.0kvs& r' "http_user_agent..s]6\da!AaSz6\TRANSFORMERS_IS_CIz ; is_ci/truec34"TFwrV RV 2xK R#5irr/rs& r'rrsI6Hda!AaSz6Hr)rsysversionr SESSION_IDr rr HF_HUB_DISABLE_TELEMETRYr!joinritemsr>rrupperr isinstancerAr=)ruas& r'http_user_agentrs&  Ys{{/@/@/B1/E.FmT^S_ `B *,-..)))%%%#%% dTYY]6R6T6Z6Z6\]]]] zz~~*B/557;OO n*d## dTYYIj6F6F6HIIII I J $ $ dZ Ir)cfV^8dQhR\R,R\R,R\R,/#)r$rWNr;r%)r=)r&s"r'r(r(s3 I IsTz Id IsUYz Ir)cVeVeV#\\V4P44p\P!RV4pVfR#VP 4^,p\ P!V4'dV#R#)zH Extracts the commit hash from a resolved filename toward a cache file. Nzsnapshots/([^/]+)/)r=ras_posixresearchgroupsrmatch)rWr;rs&& r'extract_commit_hashrsq 7]+4467M YY,m >;HDHr)ctV^8dQhR\\P,R\R\R,/#)r$rRfilenamer%Nr=r>r?)r&s"r'r(r(s3992;;&99 4Z 9r)c H\RRVRV./VBpVe V^,pV#TpV#)a? Tries to locate a file in a local folder and repo, downloads and cache it if necessary. Args: path_or_repo_id (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a model repo on huggingface.co. - a path to a *directory* potentially containing the file. filename (`str`): The name of the file to locate in `path_or_repo`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. repo_type (`str`, *optional*): Specify the repo type (useful when downloading from a space for instance). Passing `token=True` is required when you want to use a private model. Returns: `Optional[str]`: Returns the resolved file (to the cache folder if downloaded from a repo). Examples: ```python # Download a model weight from the Hub and cache it. model_weights_file = cached_file("google-bert/bert-base-uncased", "pytorch_model.bin") ``` rR filenamesr/) cached_files)rRrkwargsfiles&&, r' cached_filers>n  X XH: XQW XD&47D K-1D Kr)c#RV^8dQhR\\P,R\\,R\\P,R,R\R\ \\3,R,R\\,R,R\R,R \R \R \R,R \\ \\3,,R,R \R\R\R\R,R\ R,R\\,R,/#)r$rRrr4Nr5r6r8r9r7r:rTr _raise_exceptions_for_gated_repo%_raise_exceptions_for_missing_entries'_raise_exceptions_for_connection_errors _commit_hashr<r%)r=r>r?r[r@rArB)r&s"r'r(r(*s!2;;&CyR[[ 4'  #s(^d "  #:  DjTzd38n$t+'+,0.2* t !$ #Y%r)c < \4'd V'g\PR4RpVfRpVUu.uF#p\PP VV4NK% pp\ V4p.pVFp\PPV4'gK*\PP VV4p\PPV4'gOV 'dEV\PP VR48wd VfRMTp\V RV RV R V R 24hKVPV4K \PPV4'd V'dV#R#Vf\Pp\V\4'd \ V4p.p^pVepV'ghVFap\VVW.V R 7pVfKV\ JdV^, pVPV4K>V 'g V^, pKQ\R V R V R24h V\#V48Xd\#V4^8dV#R#\%V 4p \#V4^8Xd2\'TV^,\#V4^8XdRMTV VVV VVVVVR7 M\)VVV VVV VVVVVR7 VUu.uFp\9VVW&4NK pp\D;QJdRV4F 'gK RM RM !RV44'dV 'd\GVV4UUu.uFwppVeK VNK ppp\#V4^8Xd/V^,\PP VR48XdR#VfRMTp\#V4^8Xd R V^, 2MR!.VO5 2p\V R"V RV R V R 24hVUu.uF pVfK VNK pp\#V4^8XdRpV#TpV#uupi \*Edp\T\,4'd&\T\.4'g\T R24Th\T\04'd\T RT R24Th\T\24'd\RTP4 RT R24Th\T\64'd\T 4ThTUu.uFp\9TTY&T 4NK Muupipp\:;QJdRT4F 'dK RM RM !RT44'dTuRp?#\T\.4'd*T 'gRp?R#\RT R\ T4 24Th\T\<4'd;T 'gRp?R#T 'd\R\P> R24ThRp?EL\T\@4'd7\T\B4'g!T 'gRp?R#\RT RT 24Th\T\B4'gThRp?EL[Rp?ii;iuupiuuppiuupi)#a Tries to locate several files in a local folder and repo, downloads and cache them if necessary. Args: path_or_repo_id (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a model repo on huggingface.co. - a path to a *directory* potentially containing the file. filenames (`list[str]`): The name of all the files to locate in `path_or_repo`. cache_dir (`str` or `os.PathLike`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force to (re-)download the configuration files and override the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `hf auth login` (stored in `~/.huggingface`). revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. local_files_only (`bool`, *optional*, defaults to `False`): If `True`, will only try to load the tokenizer configuration from local files. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. repo_type (`str`, *optional*): Specify the repo type (useful when downloading from a space for instance). Private args: _raise_exceptions_for_gated_repo (`bool`): if False, do not raise an exception for gated repo error but return None. _raise_exceptions_for_missing_entries (`bool`): if False, do not raise an exception for missing entries but return None. _raise_exceptions_for_connection_errors (`bool`): if False, do not raise an exception for connection errors but return None. _commit_hash (`str`, *optional*): passed when we are chaining several calls to various files (e.g. when loading a tokenizer or a pipeline). If files are cached for this commit hash, avoid calls to head and get from the cache. Passing `token=True` is required when you want to use a private model. Returns: `Optional[str]`: Returns the resolved file (to the cache folder if downloaded from a repo). Examples: ```python # Download a model weight from the Hub and cache it. model_weights_file = cached_file("google-bert/bert-base-uncased", "pytorch_model.bin") ``` z+Offline mode: forcing local_files_only=TrueTNrz config.jsonmainz& does not appear to have a file named z#. Checkout 'https://huggingface.co/z/tree/z' for available files.rVzCould not locate z inside .) r:rTr9r4rr5r6r8r7r<) allow_patternsrTr9r4rr5r6r8r7r<a  is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `hf auth login` or by passing `token=` is not a valid git identifier (branch name, tag name or commit id) that exists for this model name. Check the model page at 'https://huggingface.co/' for available revisions.zPermissionError at z when downloading z. Check cache directory permissions. Common causes: 1) another user is downloading the same model (please wait); 2) a previous download was canceled and the lock file needs manual removal.c3("TFqRJxK R#5iNr/rrs& r'rcached_files..s;ND4NFz`You are trying to access a gated repo. Make sure to have access to it at https://huggingface.co/z. zWe couldn't connect to 'z' to load the files, and couldn't find them in the cached files. Check your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.z:There was a specific connection error when trying to load z: c3("TFqRJxK R#5irr/rs& r'rrs 3ND4rbrr=isdirisfileOSErrorappendr HF_HUB_CACHErrrrlenrr rrrrrPermissionErrorr ValueErrorrXallrENDPOINTrranyzip)rRrr4r5r6r8r9r7r:rTrrrrrr<deprecated_kwargsrfull_filenamesexisting_filesrrW revision_ file_countereresolved_filesoriginalresolvedmissing_entriesmsgs&&&&&&&&&&&&&&&&, r'rr*s=`!1 AB AJJ bggll9d3 NJ/*ON" 77== ) )GGLL(CM77>>-008XV_anIo=o*2*:I!*++QRZQ[\33B2C6)Tjl   ! !- 0# ww}}_%%!/~9T9** )T"" N NL&H2YajM( (88 A%L"))-8> A%L!$5hZxGXXY"Z[['s>**!$^!4q!8~BdB ,J] ~ ! # ! "%i.A"5$9#!#%-!1%  -#!#%-!1% \cqbpV^!/8YQbp s 3N 3sss 3N 3338]>A.R`>av>a(:(em88>av ~ ! #(:bggll9Vc>d(d&.FH 474HA4MmOA./ 0UabudsbuavSw 8=`ap`qqwyBxCC$ $  (6J~tdd~NJ 0A5TN  * *:aAS3T3T:VWfVggjkljmnouv vA122G3y=~ wKs)OAO$O$/X X&X4X?X$ X 0B=X-S$X+XXX  X18X0X8X3X*XX r7rTc6V^8dQhR\\P,R\R\R,R\\\3,R,R\\,R,R\R\\ ,R,R \R,/#) r$ path_or_reporr9Nr6r8r7r4rT)r=r>r?rAr@r)r&s"r'r(r(,sR!R! #R!R!DjR!#s(^d " R! #:  R!R!TzD R!TzR!r)c \PPV4'd=\PP\PP W44#\ VVVVVR7p \ V \4p V'dV #\4P\WW'R7\V\4R7R^ R7p \%T 4R# \Pdh\P\P \"3dT u#i;i \&d-p \(P+T 4\-T RT R24T hR p ? i\.d*p \(P+T 4\-T R 24T hR p ? i\0d-p \(P+T 4\-T R T R 24T hR p ? i\2dR#\4dT u#i;i) a Checks if a repo contains a given file without downloading it. Works for remote repos and local folders. If offline mode is enabled, checks if the file exists in the cache. This function will raise an error if the repository `path_or_repo` is not valid or if `revision` does not exist for this repo, but will return False for regular connection errors. )rZrr9rTr4)rr9rT)r8rF)headersfollow_redirectstimeoutTzN is a gated repository. Make sure to request access at https://huggingface.co/z~ and pass a token having permission to this repo either by logging in with `hf auth login` or by passing `token=`.NzE is not a local folder or a valid repository name on 'https://hf.co'.rr)r>rbrrrrrr=rheadr rrre ProxyError ConnectErrorTimeoutExceptionrrrrerrorrrrrr) rrr9r6r8r7r4rTr cached_pathhas_file_in_cacheresponsers &&&&&$$$, r'has_filer,s2 ww}}\""ww~~bggll<BCC ) K#;4   !=%% | _$5_=NO" & !H%       6 68L M!  !  Qn&&2^4R R    #u Q&klmstt  QjKKW.Xr t    !  !sl ;C D=DDD G('E GG$F GG'F66 GGGGGca]tRtRtoRtV3RlRltRV3RlRlltR tR RR RR RR RRRRRRRRR/V3RlRlltRt Vt R#)PushToHubMixinizO A Mixin containing the functionality to push a model or tokenizer to the hub. cD<V^8dQhRS[S[P,/#)r$ working_dirr)r&rCs"r'r(PushToHubMixin.__annotate__s ddr{{1Bdr)c  \P!V4Uu/uF?q"\PP\PP W44bKA up#uupi)zC Returns the list of files with their last modification timestamp. )r>listdirrbgetmtimer)selfrfs&& r'_get_files_timestamps$PushToHubMixin._get_files_timestampssILN::VaKbcKba277##BGGLL$@AAKbcccsAA!NFc<V^8dQhRS[S[P,RS[RS[S[S[3,RS[R,RS[S[,R,RS[RS[R,R S[R,/#) r$rrZfiles_timestampscommit_messageNr8 create_prr9commit_description)r=r>r?rAfloatr@)r&rCs"r'r(rsM M 2;;&M M sEz* M d M czD M M *M  $JM r)c  DVfRVPP9dRpMRVPP9dRpMsRVPP9dRpMURVPP9dR pM7R VPP9dR pMR VPP 2p\P!V4U u.uFTp W9gJ\PP \PP W44W9,8gKRV NKV p p V U u.uFp \PP\PP W44'gE\PP\PP W44'gKV NK p p .p V EFp \PP\PP W44'd\P!\PP W44FZp V P\\PP WV 4\PP W4R 74K\ KV P\\PP W4V R 74EK Ve5VPR4'g\4PW'VRR7\"P%RV RRP V 4 24\4P'VV VVVVVR7#uup iuup i \d/p T PP R8XdT'dRp ? LhRp ? ii;i)zX Uploads all modified files in `working_dir` to `repo_id`, based on `files_timestamps`. NModelz Upload modelConfigz Upload config TokenizerzUpload tokenizerFeatureExtractorzUpload feature extractor ProcessorzUpload processorzUpload )path_or_fileobjr]zrefs/prT)rZbranchr8exist_okiz!Uploading the following files to z: ,)rZ operationsrrr8rr9) __class__rEr>rrbrrrrrr startswithr0 create_branchrr status_coderr create_commit)rrrZrrr8rr9rrmodified_filesrrrs&&&&&&&&& r'_upload_modified_files%PushToHubMixin._upload_modified_filess  !$..111!/T^^444!0 7 77!3#t~~'>'>>!; 7 77!3#*4>>+B+B*C!DZZ , ,(BGG,<,rbrr) rrZrrr!r8r9rr"r$ model_cardtmp_dirs &&$$$$$$$$ r' push_to_hubPushToHubMixin.push_to_hubst(&&wuW[&\dd/wEJ  ( ( * *g   H OOBGGLL+> ?..!#-#!#5/ + * * *s AB66 C r/)NNFNN) rErFrGrH__doc__rrrr0rJrKrLs@r'rrsdd M M ^wP &* P *. P $P$(P $P P,2P"&PPPr)rc:V^8dQhR\\,/#)r$size)r&r=)r&s"r'r(r(2skk39kr)c\V\4'dV#VP4PR4'd\VRR 4R ,#VP4PR4'd\VRR 4R ,#VP4PR4'd\VRR 4R ,#VP4PR4'd8\VRR4R,pVPR4'd V^,#T#VP4PR4'd8\VRR4R,pVPR4'd V^,#T#VP4PR4'd8\VRR4R,pVPR4'd V^,#T#\ R 4h)a Converts a size expressed as a string with digits an unit (like `"5MB"`) to an integer (in bytes). Args: size (`int` or `str`): The size to convert. Will be directly returned if an `int`. Example: ```py >>> convert_file_size_to_int("1MiB") 1048576 ``` GIBNMIBKIBGBbMBKBzR`size` is not in a valid format. Use an integer followed by the unit, e.g., '5GB'.i@iiiʚ;i@Bi)rr&rrcr)r4int_sizes& r'convert_file_size_to_intr@2s~$  zz|U##49~'' zz|U##49~'' zz|U##49~'' zz|T""tCRy>U+ $ c 2 2x1}@@ zz|T""tCRy>U+ $ c 2 2x1}@@ zz|T""tCRy>U+ $ c 2 2x1}@@ i jjr)c  \PPV4'g\RV RV R24h\ V4;_uu_4p \ P !V P44pRRR4\\XR,P444pVR,p\VR,P44VR&VR,P4VR&\PPV4'd4VU u.uF#p \PPW V 4NK% pp VV3#\!VVVVVVVVVV V V R7 pVV3# +'giL;iuup i) a For a given model: - download and cache all the shards of a sharded checkpoint if `pretrained_model_name_or_path` is a model ID on the Hub - returns the list of paths to all the shards, as well as some metadata. For the description of each arg, see [`PreTrainedModel.from_pretrained`]. `index_filename` is the full path to the index (downloaded and cached if `pretrained_model_name_or_path` is a model ID on the Hub). zCan't find a checkpoint index (z) in rN weight_mapmetadataall_checkpoint_keys) r4r5r6r7r8rr9r:rr<)r>rbrropenrrreadsortedsetvaluesr[keyscopyrrr)pretrained_model_name_or_pathindex_filenamer4r5r6r7r8rr9r:rr<rrindexshard_filenamessharded_metadatacached_filenamess&&&&&&&&&&&&, r'get_checkpoint_shard_filesrRSsU2 77>>. ) ):>:J%PmOnnopqq n   1668$ S|!4!;!;!=>?OZ(.253F3K3K3M.N*+%*<%8%=%=%?\" ww}}233^mn^mYZ277<<(ERST^mn 000$%%)!  - --=  os %E)E- E* czV^8dQhR\R\\,R,R\R,R\/#)r$rZr$Nr8r%)r=r[r )r&s"r'r(r(s9s$s)d2BRUX\R\hqr)c\P!WR7pVetVP P f.VP nVFDpWcP P 9gKVP P PV4KF V# \d0Rp\Tf.MTRR7p\P!YTR7pLi;i)aq Creates or loads an existing model card and tags it. Args: repo_id (`str`): The repo_id where to look for the model card. tags (`list[str]`, *optional*): The list of tags to add in the model card token (`str`, *optional*): Authentication token, obtained with `huggingface_hub.HfApi.login` method. Will default to the stored token. r(uThis is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.r+)r$r,)model_description)r loadrr from_templatedatar$r)rZr$r8r.rU card_data model_tags&&& r'r*r*s]^^G9   ??   '#%JOO I 4 44$$++I6  ]e!T\rtR`a ,,Y\ ]sB7C  C cXa]tRtRtoRtR V3RlRlltRtRtV3RlR ltR t Vt R#) PushInProgressizb Internal class to keep track of a push in progress (which might contain multiple `Future` jobs). NcF<V^8dQhRS[PR,RR/#)r$jobsNr%)rFuture)r&rCs"r'r(PushInProgress.__annotate__s$11W^^d21d1r)c.Vf .VnR#TVnR#rr^)rr^s&&r'__init__PushInProgress.__init__s,B D r)c\;QJd&RVP4F 'dK R# R#!RVP44#)c3@"TFqP4xK R#5ir)done)rjobs& r'r)PushInProgress.is_done..s3#88::sFT)rr^rs&r'is_donePushInProgress.is_dones3s33ss3s3s3333r)cF\P!VP4R#r)rwaitr^rjs&r'wait_until_donePushInProgress.wait_until_dones TYYr)c<V^8dQhRR/#)r$r%Nr/)r&rCs"r'r(r`s   r)cVPUu.uF5pVP4'dKVP4'dK3VNK7 upVnR#uupir)r^cancelrg)rrhs& r'rsPushInProgress.cancelsGyy  JJL %(HHJ C    sAAArbr) rErFrGrHr2rcrkrorsrJrKrLs@r'r\r\s(114   r)r\cNV^8dQh/^\9d\R,;R&#)r$Nr.)__conditional_annotations__r)r&s"r'r(r(s$vwr))NNNr)NFNNNFrNNTTTNN) NFNFNNNrNN)NN)Rrvr2rr>rrr+ concurrentrpathlibrtypingruuidrrehuggingface_hubrrrr r r r r rrrhuggingface_hub.file_downloadrhuggingface_hub.utilsrrrrrrrrrrrrr import_utilsrrr r!#LEGACY_PROCESSOR_CHAT_TEMPLATE_FILECHAT_TEMPLATE_FILEra get_loggerrErr.r0r2rrbrHF_HOMErN TRANSFORMERS_DYNAMIC_MODULE_NAMEhexrS3_BUCKET_PREFIXCLOUDFRONT_DISTRIB_PREFIXrXrorrrrrrrr@rRr*r\r()rvs@r'rs      <   #';#*/   H %  Ye  99/i>O>OQZ1[\#9 W[[ H8  *r *r ! *r $ *rZ6* I9xDR!#R!$(R!!R!R!jnnbkB:.z@  r)