iHZddlZddlZddlmZddlmZmZmZmZm Z m Z m Z ddl m Z ddlZddlmZmZmZmZdej(defdZdej(defd Zdej(defd Zd ee ed eeej(fdee efd Zd eeej(fdee efdZdej(defdZdddd eeej(fdeeedeeedeeeeffdZ d2dej>j@dedeeeefdefdZ! d3dej>j@de eejDfdede eefde eeeeff dZ# d4d eeej(fdeeeefde$fdZ% d4d eeej(fde eejDfdeeeeffdZ& d5de eejDfde eefdeeej(ffdZ'de$deeej(ffd Z(e)ed!dZ*e)ed"dZ+e)ed#dZ,e)ed$dZ-ej\d%ej^d&ej`d&ejbd'ejdd'ejfd'ejhd(ejjd(ej8d(ejld%ejnd%e*d(e+d(e,d(e-d(iZ8e ejre d)k\r5e8juejvd%ejxd&ejzd'iejlej^ejdejbej\ej`ejfejjejhej8e*e+ejnd* Z>e ejre d)k\r3e>juejvejxejzd+d,edej~fd-Z@deeej(ffd.ZAdej(d/ede$fd0ZBd eeej(fdeeeeefffd1ZCy)6N) defaultdict)AnyDictListOptionalSetTupleUnion)Version) deserialize safe_open serializeserialize_filetensorreturnc |jjS#t$r2 |jjcYS#t$rYYywxYwwxYw)Nr)untyped_storagedata_ptr ExceptionstorageNotImplementedErrorrs f/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/safetensors/torch.py storage_ptrr s[%%'0022  >>#,,. ."   s, AA A AAAAc|jr8|jddjt|jz}|S|j}|S)N)nelementviewr_SIZEdtype)rstops r_end_ptrr"sO {{2r"++-fll0CC K  Kc. |jjS#t$rk |jj t |j zcYS#t$r)|jt |j zcYcYSwxYwwxYwN) rnbytesAttributeErrorrsizerr rrrs r storage_sizer) s ;%%'..00 ; ;>>#((*U6<<-@@ @" ;??$uV\\':: : ; ;s, B1AB-B BBBtensors state_dictcg}|D]}t|dkr|j|#g}|D]2}||}|j|jt||f4|j |d\}}} |j| h|ddD]4\} } }| |k\r|j|hn|dj || }6|S)Nrr)lenappendrr"sortadd) r*r+filtered_tensorssharedareasnamer_ last_stop last_namestartr!s r_filter_shared_not_sharedr;-s v;?  # #F + D%F LL&//+Xf-=tD E  "'(9i ,!&qr E4 ! ''/ $((.I "+( r#ctt}|jD]y\}}|jt jdk7s)t |dk7s8t |dk7sG||jt |t |fj|{tt|j}t||}|S)Nmetar) rsetitemsdevicetorchrr)r2listsortedvaluesr;)r+r*kvs r_find_shared_tensorsrGHs#G  "1 HH V, ,A!#Q1$ QXX{1~|A? @ D DQ G#6'..*+,G'rGrrBrI RuntimeErrorrC difference intersectionr0) r+rJrKshareds to_remover4r6complete_names keep_name preferreds r_remove_duplicate_namesrU]s8 /*O  &M":.GD!I$ Gfd Z5E(FTf G ((.x0 4/03 #--m< tI/2I '44^DI"4 ?3A6 6NDy )$++D1#7< 9 Hs D D modelfilenamemetadataforce_contiguousc|j}t|}|jD]\}}|D]}|i}||vr|||<||=|r1|jD cic]\} } | | j}} } t |||ycc} } w#t $r } t | } | dz } t | d} ~ wwxYw)a Saves a given torch model to specified filename. This method exists specifically to avoid tensor sharing issues which are not allowed in `safetensors`. [More information on tensor sharing](../torch_shared_tensors) Args: model (`torch.nn.Module`): The model to save on disk. filename (`str`): The filename location to save the file metadata (`Dict[str, str]`, *optional*): Extra information to save along with the file. Some metadata will be added for each dropped tensors. This information will not be enough to recover the entire shared structure but might help understanding things force_contiguous (`boolean`, *optional*, defaults to True): Forcing the state_dict to be saved as contiguous tensors. This has no effect on the correctness of the model, but it could potentially change performance if the layout of the tensor was chosen specifically for that reason. NrXzT Or use save_model(..., force_contiguous=True), read the docs for potential caveats.)r+rUr? contiguous save_file ValueErrorstr) rVrWrXrYr+ to_removes kept_nameto_remove_grouprQrErFemsgs r save_modelres6!!#J(4J&0&6&6&8" ?(I(&/#9%)'94>4D4D4FG4FDAqa'4F G*h:H !f eeos B?B B=B88B=strictr@ct||}|j}t||j}|j |d\}}t |}|j D]0} | D])} | |vr|j| |j| +2|r|s|rdjt|D cgc]} d| d c} } djt|D cgc]} d| d c} } d|jjd}|r|d | z }|r|d | z }t|||fScc} wcc} w) aA Loads a given filename onto a torch model. This method exists specifically to avoid tensor sharing issues which are not allowed in `safetensors`. [More information on tensor sharing](../torch_shared_tensors) Args: model (`torch.nn.Module`): The model to load onto. filename (`str`, or `os.PathLike`): The filename location to load the file from. strict (`bool`, *optional*, defaults to True): Whether to fail if you're missing keys or having unexpected ones. When false, the function simply returns missing and unexpected names. device (`Union[str, int]`, *optional*, defaults to `cpu`): The device where the tensors need to be located after load. available options are all regular torch device locations. Returns: `(missing, unexpected): (List[str], List[str])` `missing` are names in the model which were not modified during loading `unexpected` are names that are on the file, but weren't used during the load. )r@)rJF)rfz, "z#Error(s) in loading state_dict for :z# Missing key(s) in state_dict: z& Unexpected key(s) in state_dict: ) load_filer+rUkeysload_state_dictr>rDr0removejoinrC __class____name__rM)rVrWrfr@r+model_state_dictr`missing unexpectedrbrQrE missing_keysunexpected_keyserrors r load_modelrwsZ:8F3J'')(*//*;J // 5/IGZ'lG%,,.(I'!!),y) )/ 7jyyF7O!DOqAaS(O!DE ))vj7I$J7I!q1X7I$JK5eoo6N6N5OqQ  ;L>J JE  >>OP PE5!! J "E$Js ; D8' D=cHtt||}t|}|S)a$ Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensors (`Dict[str, torch.Tensor]`): The incoming tensors. Tensors need to be contiguous and dense. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `bytes`: The raw bytes representing the format Example: ```python from safetensors.torch import save import torch tensors = {"embedding": torch.zeros((512, 1024)), "attention": torch.zeros((256, 256))} byte_data = save(tensors) ``` r[)r_flattenbytes)r*rX serializedresults rsaver}s$68G,x@J : F Mr#c2tt|||y)ah Saves a dictionary of tensors into raw bytes in safetensors format. Args: tensors (`Dict[str, torch.Tensor]`): The incoming tensors. Tensors need to be contiguous and dense. filename (`str`, or `os.PathLike`)): The filename we're saving into. metadata (`Dict[str, str]`, *optional*, defaults to `None`): Optional text only metadata you might want to save in your header. For instance it can be useful to specify more about the underlying tensors. This is purely informative and does not affect tensor loading. Returns: `None` Example: ```python from safetensors.torch import save_file import torch tensors = {"embedding": torch.zeros((512, 1024)), "attention": torch.zeros((256, 256))} save_file(tensors, "model.safetensors") ``` r[N)rry)r*rWrXs rr]r]s>8G$hBr#ci}t|d|5}|jD]}|j|||< ddd|S#1swY|SxYw)a Loads a safetensors file into torch format. Args: filename (`str`, or `os.PathLike`): The name of the file which contains the tensors device (`Union[str, int]`, *optional*, defaults to `cpu`): The device where the tensors need to be located after load. available options are all regular torch device locations. Returns: `Dict[str, torch.Tensor]`: dictionary that contains name as key, value as `torch.Tensor` Example: ```python from safetensors.torch import load_file file_path = "./my_folder/bert.safetensors" loaded = load_file(file_path) ``` pt) frameworkr@N)r offset_keys get_tensor)rWr@r|frEs rrjrj6sQ2F 8tF ;qA QF1I! < M < Ms *AAdatac.t|}t|S)a Loads a safetensors file into torch format from pure bytes. Args: data (`bytes`): The content of a safetensors file Returns: `Dict[str, torch.Tensor]`: dictionary that contains name as key, value as `torch.Tensor` on cpu Example: ```python from safetensors.torch import load file_path = "./my_folder/bert.safetensors" with open(file_path, "rb") as f: data = f.read() loaded = load(data) ``` )r _view2torch)rflats rloadrVs. t D t r# float8_e4m3fn float8_e5m2float8_e8m0fnufloat4_e2m1fn_x2r-r.z2.3.0) F64F32F16BF16I64I32I16I8U8BOOLF8_E4M3F8_E5M2C64)U64U32U16 dtype_strct|Sr%)_TYPES)rs r _getdtypers ) r#ci}|D]\}}t|d}t|ddk(r2td|dDsJtj|d|}n,tj |d|j |d}tjdk(r3tj|jjd }|||<|S) Nr rrc3&K|] }|dk( yw)rN).0xs r z_view2torch..s2z!qAvzsshape)r bigFinplace) rr/anyrAempty frombufferreshapesys byteorder from_numpynumpybyteswap)safeviewr|rErFr arrs rrrs F1!G*% qy>Q 2qz22 22++aj6C""1V9E:BB1W:NC ==E !""399;#7#7#7#FGCq  Mr#r6ch|jtjk7rtd|d|j std|d|j j dk7r|jd}ddl}ddl }t|j|jj}t|j}||z}|j!}|dk(ry|j#||j%|j&}|j(j+||f} t,j.dk(rjtj0|j0tj2|j2tj4|j4tj6|j8tj8|j8tj:|j:tj<|j<tj>|j>tj@t@tjB|jBtD|j<tF|j<tjH|jHi } | |j} | jK| jMd } | jOS) Nz)You are trying to save a sparse tensor: `` which this library does not support. You can make it a dense tensor before saving with `.to_dense()` but be aware this might make a much larger file than needed.z1You are trying to save a non contiguous tensor: `a` which is not allowed. It either means you are trying to save tensors which are reference of each other in which case it's recommended to save only the full tensors, and reslice at load time, or simply call `.contiguous()` on your tensor to pack it before saving.cpurr#rFr)(layoutrAstridedr^ is_contiguousr@typetoctypesrintprodritemrr rcastPOINTERc_ubyte ctypeslibas_arrayrrint64float32int32bfloat16float16int16uint8int8boolfloat64_float8_e4m3fn _float8_e5m2 complex64rrtobytes) rr6rnplengthbytes_per_item total_bytesptrnewptrrNPDTYPESnpdtypes r_tobytesrs }} %7v>4 4     !?vF& &  }}U"5!&++- .F6<<(N>)K // C ax [[fnnV^^< =F << + 8D }} KK MM2:: KK NNBJJ MM2:: KK KK JJ JJ MM2:: BHH "(( OOR\\ "6<<(yy!**5*9 <<>r#c t|tstdt|g}|j D]h\}}t|t j std|dt||jt jk7sX|j|j|rtd|dt|}g}|D]"}t|dkDs|j|$|rtd|d|j Dcic]E\}}|t|jjd d |j t#||d Gc}}Scc}}w) Nz4Expected a dict of [str, torch.Tensor] but received zKey `z1` is invalid, expected torch.Tensor but received z*You are trying to save a sparse tensors: `rr.z Some tensors share memory, this will lead to duplicate memory on disk and potential differences when loading them again: z. A potential way to correctly save your model is to use `save_model`. More information at https://huggingface.co/docs/safetensors/torch_shared_tensors .r)r rr) isinstancedictr^rr?rATensorrrr0rGr/rMr_r splitrr)r*invalid_tensorsrErFshared_pointersfailingnamess rryrys gt $B4=/ R  O 1!U\\*sKDQRG9U  88u}} $  " "1 % 88IJ4 4  +73OG  u:> NN5 !!FGNFOO    MMO   $DAq \'',R0WWQN  $   sA E)NT)Trr%)r)Dosr collectionsrtypingrrrrrr r packaging.versionr rA safetensorsr r rrrrrr"r)r_r;rGrrIrUnnModulerePathLikerwrzr}r]rjrgetattrrr _float8_e8m0_float4_e2m1_x2rrrrrrrrrrr __version__updateuint64uint32uint16rr rrrryrr#rrs #???% II   U\\c ; ;# ; #c(^)-c5<<.?)@ #c(^6 T#u||*;%< c#h $,0)- -S%,,&'-d3i(-DI& -  #tCy. -f*.! . 88??..tCH~&. .h# 3 88??3C$%3 3 #s(O 3  49d3i  3nLP #u||# $08c3h0H F*.C #u||# $CC$%CtCH~&CFBGC$%/4S#X #u|| @uc5<</086umT2 u.5 %!3T: KK MM1 KK NNA MM1 KK KK JJ JJ MM1 OOQA!!Q " 5  !11 LL LL! LL! LL!  == == == NN ;; ;; ;; ** ++ JJ ??  5  !11 MM<<<<<< T#u||"34"8U\\888v,d3 ,-,$sDcN7J2K,r#