+ LV-jx^RIt^RIt^RIt^RIt^RIt^RIHt^RIHt^RIt^RI t ^RI H t H t H t Ht^RIHt^RIHt^RIHt^RIHtHtHtHtHt^R IHt]P:!R 4t]P:!R 4t]P@!R 4t!R t"!RR] 4t#!RR]4t$]P:!R4t%Rt&!RR]4t'!RR] 4t(RRlt)RRlt*RRlt+]!]+4t,R#)N)copy)urlparse)AbstractAsyncStreamedFileAsyncFileSystemsync sync_wrapper)DEFAULT_CALLBACK)FSTimeoutError)AbstractBufferedFile)DEFAULT_BLOCK_SIZEglob_translate isfilelike nullcontexttokenize)AllBytesz6<(a|A)\s+(?:[^>]*?\s+)?(href|HREF)=["'](?P[^"']+)z-(?Phttp[s]?://[-a-zA-Z0-9@:%_+.~#?&/=]+)z fsspec.httpc6"\P!R/VB#5i)N)aiohttp ClientSession)kwargss,l/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/fsspec/implementations/http.py get_clientr s  *6 **sc Haa]tRt^$toRtR tRtRRRRRRRRR]R3 V3Rllt] R4t R t ] R 4t R t]R 4t]V3R l4tR!RltR!Rlt]!]4tRtR"RltR#]3RltR#]RR3RltR$RltRtR%RltR&RltRtRt R'Rlt!Rt"R(Rlt#Rt$Vt%V;t&#))HTTPFileSystema Simple File-System for fetching data via HTTP(S) ``ls()`` is implemented by loading the parent page and doing a regex match on the result. If simple_link=True, anything of the form "http(s)://server.com/stuff?thing=other"; otherwise only links within HTML href tags will be used. URLs are passed unfiltered to aiohttp, so all addresses are accessible. Where URLs are supplied by a user, the calling application may wish to filter to prevent scanning. /TNbytesFc  <\SV`!V3RVRV/V BVeTM\VnWnW0nWPnW`nT ;'g/VnWn Wn Wn RVn \V 4p V PRR4VnV PRR4V PRR4V PRR4Wn R#) a  NB: if this is called async, you must await set_client Parameters ---------- block_size: int Blocks to read bytes; if 0, will default to raw requests file-like objects instead of HTTPFile instances simple_links: bool If True, will consider both HTML tags and anything that looks like a URL; if False, will consider only the former. same_scheme: True When doing ls/glob, if this is True, only consider paths that have http/https matching the input URLs. size_policy: this argument is deprecated client_kwargs: dict Passed to aiohttp.ClientSession, see https://docs.aiohttp.org/en/stable/client_reference.html For example, ``{'auth': aiohttp.BasicAuth('user', 'pass')}`` get_client: Callable[..., aiohttp.ClientSession] A callable, which takes keyword arguments and constructs an aiohttp.ClientSession. Its state will be managed by the HTTPFileSystem class. storage_options: key-value Any other parameters passed on to requests cache_type, cache_options: defaults used in open() asynchronousloopNuse_listings_cacheFlistings_expiry_time max_pathsskip_instance_cache)super__init__r block_size simple_links same_schema cache_type cache_options client_kwargsrencodedr_sessionrpopr )selfr'r& same_scheme size_policyr)r*rrr+rr,storage_optionsrequest_options __class__s&&&&&&&&&&&&, rr%HTTPFileSystem.__init__4sT WLWtWW(2(>*DV(&$**00b$ %  /"1"5"56JE"R2D9K.148% cR#)httprr/s&rfsidHTTPFileSystem.fsidusr6cD\P!WPR7#))r,)yarlURLr,r/urls&&r encode_urlHTTPFileSystem.encode_urlysxx\\22r6cVe1VP4'd\WPRR7R#\ VRR4pVeVP4R#R# \\\ 3dL=i;i)Ng?)timeout _connector) is_runningrclose TimeoutErrorr NotImplementedErrorgetattr_close)rsession connectors&& r close_sessionHTTPFileSystem.close_session|sm   1 1 T==#6G\48      !!.2EF  sAA32A3c<"VPf~VP!RRVP/VPBGRjxL VnVP'g7\ P !WPVPVP4VP#L]5i)Nrr)r-rrr+rweakreffinalizerNr9s&r set_sessionHTTPFileSystem.set_sessionsp == "&//"Wtyy"WDDVDV"WWDM$$$  '9'9499dmmT}}Xs:BBBABc V#)z-For HTTP, we always want to keep the full URLr)clspaths&&r_strip_protocolHTTPFileSystem._strip_protocols  r6cJ<\SV`V4p\V4^8dV#R#))r$_parentlen)rVrWparr4s&& rr]HTTPFileSystem._parents&god# s8a<Jr6c ("VPP4pVPV4\P V4VP 4GRjxL pVP !VPV43/VPB;_uu_4GRjxL pVPWa4RVP9d+VPR,PR4^,pMRpVR9dVPRR7GRjxL pVP'dG\PV4\PV4U u.uF q^,NK up ,p M-\PV4U u.uF q^,NK p p M.p RRR4GRjxL \#4p \%V4p X EF~p \'V \(4'd V ^,p V P+R4'd.\-V 4^8dV P. RV P0 V 2p V P+R4'dVP2'dAV P+VP5R4R,4'dV P7V 4KV P9RR4P+VP9RR4P5R4R,4'dV P7V 4EK1EK4V R9gEK>V P7RP;VP5R4V P=R4.44EK V 'gBVP?R4'd+VPAVP5R4R R 7GRjxL p V'd2V U u.uF$p R T R RR V P?R4'dRMR/NK& up #\CV 4#ELnEL4ELuup iuup i \ d.p ELZi;iELS +GRjxL 'giELk;iLuup i5i)N Content-Type;ignore)errorsrz://r8httpsF)detailnamesizetype directoryfile)z text/htmlN)z..z../)"rrupdateloggerdebugrSgetrA_raise_not_found_for_statusheaders partitiontextr'ex2findallexUnicodeDecodeErrorsetr isinstancetuple startswithr^schemenetlocr(rstripaddreplacejoinlstripendswith_ls_realsorted)r/r@rgrkwrLrmimetypertulinksoutpartsls&&&, rrHTTPFileSystem._ls_reals( [[    & S((**;;ts3Ct{{CCCq  , ,Q 4*99^4>>sCAF..!"x!88D((( # D 12::dCS4TCSaqTTCS4T T/1zz$/? @/?!1/? @%DC(e A!U##aD||C  SVaZ||nC ~aS9||F##### SZZ_s5J(K(KGGAJYYw/::KK077 *D-+ D> 6D> 8 Ec"VPP4pVPV4\P V4VP 4GRjxL pVP !VPV43/VB;_uu_4GRjxL p\VPR,4p VPV 4VPW4\V4'dTp M \VR4p Rp V 'dQVP P#V4GRjxL p V P%V 4VP'\)V 44KX\V4'gV P+4RRR4GRjxL R#EL&L \\3dRp Li;iL \T4'gT P+4ii;iLM +GRjxL 'giR#;i5i)Nzcontent-lengthwbT)rrrmrnrorSrprAintrr ValueErrorKeyErrorset_sizerqropencontentrwriterelative_updater^rG) r/rpathlpath chunk_sizecallbackrrrLrrioutfilechunks &&&&&, r _get_fileHTTPFileSystem._get_filesp[[    & U((**;;tu5<<<< 199%567   d #  , ,Q 6%  ud+ $"#)).."<.gen_chunks(s%++%e, ud+A%%affQl3FF1I%%ga&>?z*K,,SZ8FF:.Es%AD  A0C8;2C8. D 8 D  D Nz.method has to be either 'post' or 'put', not: data)rput) rIrrrmrSlowerrrJrArq) r/rrrrmethodmoderrrrLmethresps &f&ff&&, r _put_fileHTTPFileSystem._put_files ; %&78 8 /.[[    &((**  (@ K w'.HZ\HRHHHD  , ,T 9IHH+IHHHsaAD C'A D ?C)D C- D  C+!D )D +D -D 3C64 D ?D  D cX"VPP4pVPV4\P V4VP 4GRjxL pVP !VPV43/VBGRjxL pT;_uu_4GRjxL V'dVPWa4VPR8uuRRR4GRjxL #L{LSLBL +GRjxL 'giR#;i \dR#\PdT'dhR#i;i5i)NiF) rrrmrnrorSrprArqrrr ClientError)r/rWstrictrrrLrs&&&, r_existsHTTPFileSystem._existsMs [[    &  LL  ,,..Gkk$//$"7>2>>Aqq44Q=xx#~qq/>qqq! ""  s,D*(C8C)C8CC8CC8(C C8 CC8D*C8C8C8C8C5 !C$" C5 -C5 /C83D*5C88 D'D*D'D'"D'#D*&D''D*cD"VP!V3/VBGRjxL #L5iN)r)r/rWrs&&,r_isfileHTTPFileSystem._isfile_s\\$1&1111s   c VR8wd\hVeTM VPpVPP4p VPV R&V P V4/p T;'g4V P VP !V3/VB4;'g V R,p\VPVP4p V'dnV'dfV PRR4'dN\VV3RT RTRTRTR T;'g VPR T;'g VPR VP/V B#\VV3RVR VPRV /V B#) aSMake a file-like object Parameters ---------- path: str Full URL with protocol mode: string must be "rb" block_size: int or None Bytes to download in one request; use instance value if None. If zero, will return a streaming Requests file-like instance. kwargs: key-value Any other parameters, passed to requests calls rrripartialTrLr&rr)r*r)rIr&rrrrminforrrSrpHTTPFiler)r*HTTPStreamFile) r/rWrr& autocommitr)r*rirrrrLs &&&&&&&&, r_openHTTPFileSystem._openbsq2 4<% %#-#9Zt [[   !..> &MMt{{499T#c$i&*iin&9DIIcNs4y h*He,\\$1&111 6M 4(B6(B"BCCII D'N "8Gn++C0G 'A+&D1'--c2Q6ED1'--c2Q6E 4<##'99T? %)*;%<%B%B3%G!%K"2X=    +/ 8< @F  !R!HI**W%"(.."23  44}}QXXc]+.LK/JJsOO   4   J9 a2#C,   suAK<B:K<K. K< K<K<.K1/ K<K6K<1K<4K<6K<c"\VPV4GRjxL 4#L \\3dR#i;i5i)NF)boolrrr)r/rWs&&r_isdirHTTPFileSystem._isdirs9 dhhtn,- -,!:.  s,A(&(A(=A=Ac "VPV4pVPR/4p\\V44VR&VP 4GRjxL pVP !VP V43RVRV/VB;_uu_4GRjxL pVP4RRR4GRjxL R#L]L(L +GRjxL 'giR#;i5i)a( Write bytes to a remote file over HTTP. Parameters ---------- path : str Target URL where the data should be written value : bytes Data to be written mode : str How to write to the file - 'overwrite' or 'append' **kwargs : dict Additional parameters to pass to the HTTP request rrContent-LengthNr)rXr.strr^rSrrAr) r/rWvaluerrr@rrrLrs &&&&, r _pipe_fileHTTPFileSystem._pipe_file!s""4(**Y+$'E O !((**;; OOC  ', 6= AG         +    s`ACB.6CB0C B4 C'B2(C0C2C4C :B=; C C  C) r-r&r*r)r+r,rrr(r'r )r8rf)T)NNiP)F)rNNNNN)rNr)r)'__name__ __module__ __qualname____firstlineno____doc__rseprr%propertyr:rA staticmethodrNrS classmethodrXr]rrrlsrqrr rrrrrrrrrr r__static_attributes____classdictcell__ __classcell__r4 __classdict__s@@rrr$s !H C ?&B3  :x c B$&(1;K$D! 0:d$27r 9 CDGR!!r6rc|aa]tRtRtoRtR V3RlltR V3RlltRt]!]4t Rt Rt ]!] 4t Rt VtV;t#) ri<aC A file-like object pointing to a remote HTTP(S) resource Supports only reading, with read-ahead of a predetermined block-size. In the case that the server does not supply the filesize, only reading of the complete file in one go is supported. Parameters ---------- url: str Full URL of the remote resource, including the protocol session: aiohttp.ClientSession or None All calls will be made within this session, to avoid restarting connections where the server allows this block_size: int or None The amount of read-ahead to do, in bytes. Default is 5MB, or the value configured for the FileSystem creating this file size: None or int If given, this is the size of the file in bytes, and we don't attempt to call the server to find the value. kwargs: all other key-values are passed to requests calls. c  <VR8wd \R4hWnWnW nW0nRVRVRR/Vn\ S V`!R RVRVRVR VR VR V/V BR #)rzFile mode not supportedrhrirjrlfsrWrr&r)r*Nr)rIrrr@rLdetailsr$r%) r/r%r@rLr&rr)r*rirrrr4s &&&&&&&&&&&,rr%HTTPFile.__init__Us 4<%&?@ @(  VT66B      "  "  (   r6c ^<V^8dKVP^8Xd:VPeVPVP8:gVP4VPfV^8dVP4M'\ VPVP, V4p\ SV`V4#)zRead bytes from file Parameters ---------- length: int Read up to this many bytes. If negative, read all content to end of file. If the server has not supplied the filesize, attempting to read only part of the data will raise a ValueError. )locri blocksize _fetch_allrr$r)r/lengthr4s&&rr HTTPFile.readts}aZDHHMYY*tyyDNN/J OO  99 z!TXX-v6Fw|F##r6c `"\PRV 24\VP\4'gVP P !VPPVP43/VPBGRjxL pT;_uu_4GRjxL VP4VP4GRjxL p\ \V4RRVR7Vn\V4VnRRR4GRjxL R#R#LLoLIL +GRjxL 'giR#;i5i)zRead whole file in one shot, without caching This is only called when position is still at zero, and read() is called without a byte-count. zFetch all for N)rifetcherr*r)rnrorzcacherrLrpr%rAr@rrrr^ri)r/rrs& rasync_fetch_allHTTPFile.async_fetch_alls  ~dV,-$**h//ll&&tww'9'9$(('CSt{{SSAqq""$FFHn%S44c  H qq0S$qqqsrBD.D  D.D D.$DD 1D5 D.D D. D. DD.D+ D D+ #D+ % D.c HVPRR4p\P!RV4pV'gR#V^,R8XdR;rEM3V^,PR4Uu.uFp\ V4NK upwrEV^,R8XdRM\ V^,4pWEV3#uupi)zParse the Content-Range header Content-Ranger\zbytes (\d+-\d+|\*)/(\d+|\*)Nr-)NNN)rprrsplitr)r/rrsmrrxtotals&& r_parse_content_rangeHTTPFile._parse_content_ranges KK , HH3Q 7# # Q43; EC*+A$**S/:/Q#a&/:JE! QqT5  ;s Bc *"\PRV RV RV 24VPP4pVP R/4P4pRV RV^, 2VR&\PVP RVR, 24VP P!VPPVP 43RV/VBGRjxL pT;_uu_4GRjxL VPR 8XdRRR4GRjxL R #VP4VP^8H;'gcVPVP4^,V8H;'g6\VPPR V^,44W!, 8*pV'dVP4GRjxL pMV^8d \!R 4h^p.pVP"PR 4GRjxL p V 'd4VP%V 4V\'V 4, pWV, 8dMK_R P)V4RW!, pTuuRRR4GRjxL #ELELzEL[LLyL +GRjxL 'giR#;i5i)a Download a block of data The expectation is that the server returns only the requested bytes, with HTTP code 206. If this is not the case, we first check the headers, and then stream the output - if the data size is bigger than we requested, an exception is raised. zFetch range for z: r5rrbytes=rz : Nir6rzThe HTTP server doesn't appear to support range requests. Only reading this file from the beginning is supported. Open with block_size=0 for a streaming file interface.i)rnrorrr.r@rLrpr%rArrr;rrrrrrappendr^r) r/rrrrrrresponse_is_rangerclrs &&& rasync_fetch_rangeHTTPFile.async_fetch_ranges*  'vRwau=>!!#**Y+002#E7!C!G95 zWW%5$678,,"" GG  txx ( 29 =C  11xx311    CPP,,QYY7:eCPPqyy}}%5sQw?@CKO  !FFHn M"#)).."77E 5)c%j(e +!,hhsmMck2S11 $%8?111sCJI'J0I*1J4I6 JI-J%I6?,I6,=I6*I6=I0>:I68I29 I6AI6 J!I4"J*J-J0I62I64J6J <I?= J J J)rr0r&rrLrir@)NNrrNNNF)rrrrrr%rr1rr+r;rB _fetch_rangerrr r!s@@rrr<s?0 >$.%"o.J !9v 12LLr6rz([*[])c4\PV4pVRJ#r) magic_checksearch)r7rs& rrrs   q !E  r6claa]tRtRtoRV3RlltR RltR Rlt]!]4tRt V3Rlt Rt Vt V;t #) ric .<aaaSPRR4SnSSnVSnVSnVR8wd\ hRSRR/Sn\SS` !R RVRSRVR R /SBVVV3R lp\SPV4Sn VPSnR#) rFrrhriNr%rWrr)nonec<"SPP!SPPS43/SBP 4GRjxL pSPP VS4V#L"5ir)rLrpr%rA __aenter__rq)rrr/r@s rcor$HTTPStreamFile.__init__..corsYll&&tww'9'9#'>I&ITTVVA GG / /3 7HWsAA0 A. #A0r) r.rr@rrLrr&r$r%rr) r/r%r@rrrLrrOr4s f&f&&&l rr%HTTPStreamFile.__init__s"JJ~u=  4< VT2  QBQSQtQQ&Q  dii%GG r6cjV^8Xd V^8XdR#WP8Xd V^8XdR#\R4h)rNzCannot seek streaming HTTP file)r)r)r/r)whences&&&rrHTTPStreamFile.seeks0 !8!  ((?v{ :;;r6c"VPPPV4GRjxL pV;P\ V4, unV#L(5ir)rrrr)r^)r/numrs&& r_readHTTPStreamFile._read s<FFNN'',, CH -s)AA)AcB"VPP4R#5ir)rrGr9s&rrKHTTPStreamFile._closes  sc<\P!VP4VP4\SV`4R#r)asynciorun_coroutine_threadsaferKrr$rGr/r4s&rrGHTTPStreamFile.closes&(( B  r6)rr&rrrLr@)rNN)rrD)rrrrr%rrWrrrKrGrrr r!s@@rrrs/$<  Dr6rcLaa]tRtRtoRV3RlltRRltV3RltRtVtV;t #)ric <W nWPnRVnVR8wd\hRVRR/VnWpn\ SV`WVRR7W`nR#)NrrhrirL)r%rWrr)) r@rLrrr&rr$r%ri) r/r%r@rrrLrirr4s &&&&&&&,rr%AsyncStreamFile.__init__sQ  4< VT2   BtG r6c"VPfVPP!VPP VP 43/VP BP4GRjxL pVPPW P 4W nVPPPV4GRjxL pV;P\V4, un V#LL*5ir) rrLrpr%rAr@rrNrqrrr)r^)r/rVrrs&& rrAsyncStreamFile.read)s 66>ll&&""488,04 jlA GG / /88 <FFFNN'',, CH  -s%A*C1,C--AC1C/)C1/C1c<"VPe"VPP4RVn\SV`4GRjxL R#L5ir)rrGr$r^s&rrGAsyncStreamFile.close4s3 66  FFLLNDFgmosAA A A )r&rrrLrir@)rNNNrD) rrrrr%rrGrrr r!s@@rrrs  r6rcd"VP4pVPR/4P4pRV RV^, 2VR&VP!V3RV/VBGRjxL pVP4T;_uu_4GRjxL VP 4GRjxL pRRR4GRjxL V'dB\ VR4;_uu_4p V P V4V PX4RRR4R#X#LLyLcLU +GRjxL 'giLl;i +'giR#;i5i)rrr>r5rNzr+b)rr.rprrrrr) rLr@rrrlrrrrrrs &&&&&, r get_rangerh;s [[]FjjB',,.Gwaay1GGkk#9w9&99AqqFFHnq $  ! FF5M GGCL   :qqq  sAD0C8"D0C:D0DC<D D0(C>) D05D0 #D, D0:D0<D>D0D D  D D D0 D- ' D0c"\PRV4VP4pVPRR4pVP R/4P4pRVR&WSR&/pVR8XdVP !V3RV/VBGRjxL pM3VR 8XdVP!V3RV/VBGRjxL pM\ R V 24hT;_uu_4GRjxL VP4R VP9dJR VP9gVPR ,R9d \VPR ,4VR &MFRVP9d6\VPR,PR4^,4VR &RVP9d-VPR,PR4^,VR&VPP R4R8XdRVR&\VP4VR&RF;pVPP V4'gK&VPV,Wh&K= RRR4GRjxL V#ELELELL +GRjxL 'giT#;i5i)zCall HEAD on the server to get details about the file (size/checksum etc.) Default operation is to explicitly allow redirects and use encoding 'identity' (no compression) to get the true size of the target. zRetrieve file size for %sallow_redirectsTrridentityzAccept-EncodingrNrpz)size_policy must be "head" or "get", got rzContent-Encodingrir4rrbrcrz Accept-RangesrLFrr@)rkr\)ETagz Content-MD5Digestz Last-Modified)rnrorr.rpr TypeErrorrrrrr6rsrr@) r@rLr1rarrrrchecksum_fields &&&, rrrLs  LL,c2 [[]F %t ,B ::i $ ) ) +D(D 9 Df,,sABA&A A  ++c@2@@ @CK=QRRqq  qyy ("2aii@R6SX7 #199-=#>?V  )qyy9??DQGHDL QYY & yy8BB3GJD  99== )V 3$DO!%%jU PNyy}}^,,'(yy'@$Q1q8 KC B @qqq8 KsyBI6I$I6*I+!I6 I I6EI#I< I6II6I6I6I6I3 I" I3 +I3 - I6c"Vf\4GRjxL p\V.VO5RV/VBGRjxL pVPR4#L.L5i)NrLri)rrrp)r@rLargsrrs&&*, r _file_sizerssG" $CB4BB6B BD 88F % BsAAAAAAr)r)-r\rloggingrrQr urllib.parserrr= fsspec.asynrrrrfsspec.callbacksr fsspec.exceptionsr fsspec.specr fsspec.utilsr r rrrcachingrrrwru getLoggerrnrrrrHrrrrhrrs file_sizerr6rr~s  ! VV-,,ZZMNjjEF   = )+U!_U!pk3#k3\jj"  &)&R/@"0f  $ r6