(titddlZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z ddlmZddlmZddlmZmZmZmZddlZddlZddlZddlmZmZmZddlmZdd lmZm Z dd l!m"Z"d d l#m$Z$d d l%m&Z&m'Z'd dl(m)Z)d dl*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;d dlm?Z?m@Z@mAZAd dlBmCZCddlDmEZEmFZFmGZGddlHmIZImJZJmKZKmLZLmMZMmNZNmOZOmPZPGddZQe;jeSZTejdZVe jje jje jje jje jje jje jje jje jje jje jje jje jje jjdZfe:ddrdZgndZge6rddlhZhddlhmiZiddljmkZkmlZld ej@jd!ejfd"Zod ej@jd!ejfd#Zqe d$ZrGd%d&ej@je?ZsGd'd(esZty))N) OrderedDict) ExitStackcontextmanager)wraps)Path)AnyCallableContextManagerType) DDUFEntry create_repo"split_torch_state_dict_into_shards)validate_hf_hub_args)Tensornn)Self) __version__)DiffusersAutoQuantizerDiffusersQuantizer)QuantizationMethod) CONFIG_NAMEFLAX_WEIGHTS_NAMEHF_ENABLE_PARALLEL_LOADINGSAFE_WEIGHTS_INDEX_NAMESAFETENSORS_WEIGHTS_NAMEWEIGHTS_INDEX_NAME WEIGHTS_NAME _add_variant_get_checkpoint_shard_files_get_model_file deprecateis_accelerate_availableis_bitsandbytes_availableis_bitsandbytes_versionis_peft_availableis_torch_versionlogging)is_torch_dist_rank_zero)PushToHubMixinload_or_create_model_cardpopulate_model_card)empty_device_cache)ContextParallelConfigContextParallelModelPlanParallelConfig)_caching_allocator_warmup_determine_device_map_expand_device_map_fetch_index_file_fetch_index_file_legacy_load_shard_file!_load_shard_files_with_threadpoolload_state_dictc.eZdZdZdeefdZdZdZy)ContextManagersz Wrapper for `contextlib.ExitStack` which enters a collection of context managers. Adaptation of `ContextManagers` in the `fastcore` library. context_managersc0||_t|_yN)r<rstack)selfr<s n/Volumes/fast/ai/experiments/MLX_z-image/.venv/lib/python3.12/site-packages/diffusers/models/modeling_utils.py__init__zContextManagers.__init__Ts 0[ c\|jD]}|jj|yr>)r<r? enter_context)r@context_managers rA __enter__zContextManagers.__enter__Xs$#44O JJ $ $_ 5 5rCc<|jj|i|yr>)r?__exit__)r@argskwargss rArIzContextManagers.__exit__\s T,V,rCN) __name__ __module__ __qualname____doc__listr rBrGrIrCrAr;r;Ns" !n)=!6-rCr;z(.*?)-\d{5}-of-\d{5})uniform_normal_ trunc_normal_ constant_xavier_uniform_xavier_normal_kaiming_uniform_kaiming_normal_uniformnormalxavier_uniform xavier_normalkaiming_uniformkaiming_normal>=1.9.0TF)dispatch_model)load_offloaded_weightssave_offload_index parameterreturncddlm} ||S#t$rYnwxYw tj|j |j }t|jS#t$rgdtjjdtttt ffd}|j#|}t|}|djcYSwxYw)Nr)_get_group_onload_devicemodulerfc|jjDcgc]\}}tj|s||f!}}}|Scc}}wr>__dict__itemstorch is_tensorrikvtupless rAfind_tensor_attributesz4get_parameter_device..find_tensor_attributessD)/)>)>)@W)@AEOOTUDVq!f)@FWMX AAget_members_fnr.)hooks.group_offloadingrh ValueError itertoolschain parametersbuffersnextdevice StopIterationrnrModulerPtuplestrr_named_members)rerhparameters_and_buffersrtgen first_tuples rAget_parameter_devicersA ' 22    %"+1E1E1GIZIZI\!]*+222  % 588?? tE#v+DV?W &&6L&M3i 1~$$$ %s AA''A-CCcR t|tjrR|jD]?\ }t |ds|j }|j d}|3|jcSd}|jD]f\ }|j}t |dr+|jrt fd|jDrI|jsZ|jcS|jD],}|j}|js |jcS||Sdtjdttt ffd}|j#| } d} | D](}|} |d js|d jcS| | d jSy) zz Returns the first found floating dtype in parameters if there is one, otherwise returns the last dtype it found. _diffusers_hooklayerwise_castingN_keep_in_fp32_modulesc3&K|]}|v ywr>rQ).0mnames rA z&get_parameter_dtype..sG'F!AI'Fsrirfc|jjDcgc]\}}tj|s||f!}}}|Scc}}wr>rkrps rArtz3get_parameter_dtype..find_tensor_attributessD%+__%:%:%<S%> ! !}"""rCc#Kd}tjD]*\}}ttjj ||, dtjD]*\}}ttjj ||,y#tjD]*\}}ttjj ||,wxYww)z Context manager to globally disable weight initialization to speed up loading large models. To do that, all the torch.nn.init function are all replaced with skip. cyr>rQ)rJrKs rA _skip_initz#no_init_weights.._skip_inits rCN)TORCH_INIT_FUNCTIONSrmsetattrrnrinit)rr init_funcs rAno_init_weightsrs 0557i tZ084  499;OD) EHHMM4 3 <399;OD) EHHMM4 3 Z<fd?Z=d@Z>ddAdBe?e@zdCeAeeBfdzfdDZCe5 dddEeDdFeEed:ee1jdzdGeEedHedIedJeFdzdKedLee$jLzdzdMeEedzdNee-ze$jXzeAeee-ze$jXzfzdOedzdPee1jdzdzdQeAeeGfdzdRedzdSef dTZHe5dUZIdNefdVZJe5dLe$jLde$jLfdWZKede$jXfdXZ,ede$jLfdYZ&dedZed[ede-fd\ZLdfd]ZMde$jjjfd^ed eddfd_ZPdEeDddfd`ZQxZRS)g ModelMixina$ Base class for all models. [`ModelMixin`] takes care of storing the model configuration and provides methods for loading, downloading and saving models. - **config_name** ([`str`]) -- Filename to save a model to when calling [`~models.ModelMixin.save_pretrained`]. )_diffusers_version _class_name _name_or_pathFNTc0t|d|_yr>)superrB_gradient_checkpointing_func)r@ __class__s rArBzModelMixin.__init__s ,0)rCrrfc Fd|jvxrt|jd|}||jv}|rY|sWd|dt|jd|dt|jd|d }t dd |d d |j |St ||S) a~The only reason we overwrite `getattr` here is to gracefully deprecate accessing config attributes directly. See https://github.com/huggingface/diffusers/pull/3129 We need to overwrite __getattr__ here in addition so that we don't trigger `torch.nn.Module`'s __getattr__': https://pytorch.org/docs/stable/_modules/torch/nn/modules/module.html#Module _internal_dictzAccessing config attribute `z` directly via 'z1' object attribute is deprecated. Please access 'z' over 'z,'s config object instead, e.g. 'unet.config.z'.zdirect config name accessz1.0.0F) standard_warn stacklevel)rlrtyperLr"rr __getattr__)r@r is_in_config is_attributedeprecation_messagers rArzModelMixin.__getattr__s(4==8kWT]]ScEdfj=k t}},  $@FVW[\`WaWjWjVkl]^b]ccklpqulvllk@@lmqlrrt#u  17.s*m^lYZ7167TA._gradient_checkpointing_func-sO:J4QY:Z6`b {{--88OO"rCT)enabler) _supports_gradient_checkpointingryrrL_set_gradient_checkpointing)r@rrs rAenable_gradient_checkpointingz(ModelMixin.enable_gradient_checkpointings`44>>**+,XY  ' . +G ' ((Rm(nrCcB|jr|jdyy)z Deactivates gradient checkpointing for the current model (may be referred to as *activation checkpointing* or *checkpoint activations* in other frameworks). F)rN)rrrs rAdisable_gradient_checkpointingz)ModelMixin.disable_gradient_checkpointing9s#  0 0  , ,E , : 1rCvalidcdtjjffd |jD]/}t |tjjs(|1y)z= Set the switch for the npu flash attention. riczt|dr|j|jD] }| y)Nset_use_npu_flash_attention)rrchildren)richild$fn_recursive_set_npu_flash_attentionrs rArzTModelMixin.set_use_npu_flash_attention..fn_recursive_set_npu_flash_attentionFs5v<=2259*4U;+rCNrnrrrr)r@rrirs ` @rArz&ModelMixin.set_use_npu_flash_attentionAsC  < <mmoF&%((//24V<&rCc&|jdy)z< Enable npu flash attention from torch_npu TNrrs rAenable_npu_flash_attentionz%ModelMixin.enable_npu_flash_attentionQs ((.rCc&|jdy)z= disable npu flash attention from torch_npu FNrrs rAdisable_npu_flash_attentionz&ModelMixin.disable_npu_flash_attentionXs ((/rCuse_xla_flash_attentionpartition_specc dtjjffd |jD]/}t |tjjs(|1y)Nrict|dr|jfi|jD] }| y)Nset_use_xla_flash_attention)rrr)rir fn_recursive_set_flash_attentionrKrrs rArzPModelMixin.set_use_xla_flash_attention..fn_recursive_set_flash_attentiones@v<=2223JNe^de*07+rCr)r@rrrKrirs ``` @rArz&ModelMixin.set_use_xla_flash_attention_sH  8UXX__ 8 8mmoF&%((//208&rCc ,|jd|fi|y)zJ Enable the flash attention pallals kernel for torch_xla. TNr)r@rrKs rAenable_xla_flash_attentionz%ModelMixin.enable_xla_flash_attentionps )((~HHrCc&|jdy)zK Disable the flash attention pallals kernel for torch_xla. FNrrs rAdisable_xla_flash_attentionz&ModelMixin.disable_xla_flash_attentionvs ((/rC attention_opcdtjjffd |jD]/}t |tjjs(|1y)Nric|t|dr|j|jD] }| y)N+set_use_memory_efficient_attention_xformers)rrr)rirrfn_recursive_set_mem_effrs rArzXModelMixin.set_use_memory_efficient_attention_xformers..fn_recursive_set_mem_effs7vLMBB5,W*(/+rCr)r@rrrirs `` @rArz6ModelMixin.set_use_memory_efficient_attention_xformers|sC 0UXX__ 0mmoF&%((//2(0&rCc(|jd|y)u( Enable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). When this option is enabled, you should observe lower GPU memory usage and a potential speed up during inference. Speed up during training is not guaranteed. > [!WARNING] > ⚠️ When memory efficient attention and sliced attention are both enabled, memory efficient attention takes > precedent. Parameters: attention_op (`Callable`, *optional*): Override the default `None` operator for use as `op` argument to the [`memory_efficient_attention()`](https://facebookresearch.github.io/xformers/components/ops.html#xformers.ops.memory_efficient_attention) function of xFormers. Examples: ```py >>> import torch >>> from diffusers import UNet2DConditionModel >>> from xformers.ops import MemoryEfficientAttentionFlashAttentionOp >>> model = UNet2DConditionModel.from_pretrained( ... "stabilityai/stable-diffusion-2-1", subfolder="unet", torch_dtype=torch.float16 ... ) >>> model = model.to("cuda") >>> model.enable_xformers_memory_efficient_attention(attention_op=MemoryEfficientAttentionFlashAttentionOp) ``` TNr)r@rs rA*enable_xformers_memory_efficient_attentionz5ModelMixin.enable_xformers_memory_efficient_attentions< 88|LrCc&|jdy)zs Disable memory efficient attention from [xFormers](https://facebookresearch.github.io/xformers/). FNrrs rA+disable_xformers_memory_efficient_attentionz6ModelMixin.disable_xformers_memory_efficient_attentions 88?rC storage_dtyperskip_modules_pattern.skip_modules_classes non_blockingcddlm}d}| ddlm}|}d}|j|t |jz }|j |t |j z }t t|}tr6|s4ddl m } dd l m } dd l m} | | | fD]} |t | jz }|!t j#d |j$}|||||||y) a Activates layerwise casting for the current model. Layerwise casting is a technique that casts the model weights to a lower precision dtype for storage but upcasts them on-the-fly to a higher precision dtype for computation. This process can significantly reduce the memory footprint from model weights, but may lead to some quality degradation in the outputs. Most degradations are negligible, mostly stemming from weight casting in normalization and modulation layers. By default, most models in diffusers set the `_skip_layerwise_casting_patterns` attribute to ignore patch embedding, positional embedding and normalization layers. This is because these layers are most likely precision-critical for quality. If you wish to change this behavior, you can set the `_skip_layerwise_casting_patterns` attribute to `None`, or call [`~hooks.layerwise_casting.apply_layerwise_casting`] with custom arguments. Example: Using [`~models.ModelMixin.enable_layerwise_casting`]: ```python >>> from diffusers import CogVideoXTransformer3DModel >>> transformer = CogVideoXTransformer3DModel.from_pretrained( ... "THUDM/CogVideoX-5b", subfolder="transformer", torch_dtype=torch.bfloat16 ... ) >>> # Enable layerwise casting via the model, which ignores certain modules by default >>> transformer.enable_layerwise_casting(storage_dtype=torch.float8_e4m3fn, compute_dtype=torch.bfloat16) ``` Args: storage_dtype (`torch.dtype`): The dtype to which the model should be cast for storage. compute_dtype (`torch.dtype`): The dtype to which the model weights should be cast during the forward pass. skip_modules_pattern (`tuple[str, ...]`, *optional*): A list of patterns to match the names of the modules to skip during the layerwise casting process. If set to `None`, default skip patterns are used to ignore certain internal layers of modules and PEFT layers. skip_modules_classes (`tuple[Type[torch.nn.Module], ...]`, *optional*): A list of module classes to skip during the layerwise casting process. non_blocking (`bool`, *optional*, defaults to `False`): If `True`, the weight casting operations are non-blocking. r)apply_layerwise_castingTN)DEFAULT_SKIP_MODULES_PATTERNFr) LoHaLayer) LoKrLayer) LoraLayerzW`compute_dtype` not provided when enabling layerwise casting. Using dtype of the model.)hooksrhooks.layerwise_castingrrr _skip_layerwise_casting_patternssetr&peft.tuners.loha.layerrpeft.tuners.lokr.layerrpeft.tuners.lora.layerradapter_layer_namesloggerinfor) r@rrrrrruser_provided_patternsrrrrlayers rAenable_layerwise_castingz#ModelMixin.enable_layerwise_castingsd 4!%  ' N#? %* "  % % 1 E$*D*D$E E  0 0 < E$*O*O$P P $S)=%>?  '= 9 8 8#Y :$e.G.G(HH$;  KKq r JJM -0DFZ\h rCcpu block_level onload_deviceoffload_device offload_typenum_blocks_per_group use_stream record_streamoffload_to_disk_path block_modulesexclude_kwargsc ddlm} t|dd&t|ddr|rd} tj | |j s"t |jjd| |||||||||| | | y) a Activates group offloading for the current model. See [`~hooks.group_offloading.apply_group_offloading`] for more information. Example: ```python >>> from diffusers import CogVideoXTransformer3DModel >>> transformer = CogVideoXTransformer3DModel.from_pretrained( ... "THUDM/CogVideoX-5b", subfolder="transformer", torch_dtype=torch.bfloat16 ... ) >>> transformer.enable_group_offload( ... onload_device=torch.device("cuda"), ... offload_device=torch.device("cpu"), ... offload_type="leaf_level", ... use_stream=True, ... ) ``` r)apply_group_offloading enable_tilingN use_tilingFa/Applying group offloading on autoencoders, with CUDA streams, may not work as expected if the first forward pass is executed with tiling enabled. Please make sure to either: 1. Run a forward pass with small input shapes. 2. Or, run a forward pass with tiling disabled (can still use small dummy inputs).z does not support group offloading. Please make sure to set the boolean attribute `_supports_group_offloading` to `True` in the class definition. If you believe this is a mistake, please open an issue at https://github.com/huggingface/diffusers/issues.) rirrrrrrrlow_cpu_mem_usagerrr) rrgetattrrwarning_supports_group_offloadingryrrL)r@rrrrrrrrrrrrmsgs rAenable_group_offloadzModelMixin.enable_group_offloadsH 3 4$ / ;l\a@bgqe  NN3 ..>>**+,TU  ')%!5%!'/!5') rCbackendc&ddlm}ddlm}m}m}m}ddlm}m }tjd|||f} d} |jD],} t| | s| j} t| dd*d } n|j!}|j"j%D chc]} | j&}} ||vr!t)d |d d j+|z||}| rB|j-|s1t/|j0}t)d |j&d|d|||||jD]1} t| | s| j} | t3| ds+|| _3|j7|ycc} w)a Set the attention backend for the model. Args: backend (`str`): The name of the backend to set. Must be one of the available backends defined in `AttentionBackendName`. Available backends can be found in `diffusers.attention_dispatch.AttentionBackendName`. Defaults to torch native scaled dot product attention as backend. r.AttentionModuleMixin)AttentionBackendName_AttentionBackendRegistry%_check_attention_backend_requirements"_maybe_download_kernel_for_backend AttentionMochiAttentionTAttention backends are an experimental feature and the API may be subject to change.F_parallel_configNTz `backend=z ` must be one of the following: , z>Context parallelism is enabled but current attention backend 'zS' does not support context parallelism. Please set a compatible attention backend: z' using `model.set_attention_backend()`._attention_backend) attentionr%attention_dispatchr&r'r(r)attention_processorr+r,rrrr processorrlower __members__valuesvalueryjoin_is_context_parallel_availablesorted_supports_context_parallelrr0set_active_backend)r@r"r%r&r'r(r)r+r,attention_classesparallel_config_setrir4xavailable_backendscompatible_backendss rAset_attention_backendz ModelMixin.set_attention_backendJs 4   Cmn&8LM#llnFf&78((Iy"4d;G&*# %--//C/O/O/V/V/XY/X!agg/XY , ,z *JKdiiXjNkkl l&w/ '@'_'_`g'h"()B)])]"^ PQXQ^Q^P_`>>Q=RRy{  .g6*73llnFf&78((I  ;O(P+2I ( % "44W=3Zs"Fcddlm}ddlm}m}t j d|||f}|jD]1}t||s|j}| t|ds+d|_ 3y)z Resets the attention backend for the model. Following calls to `forward` will use the environment default, if set, or the torch native scaled dot product attention. r.r$r*r-Nr0) r1r%r3r+r,rrrrr4rr0)r@r%r+r,r>rir4s rAreset_attention_backendz"ModelMixin.reset_attention_backendsi 4Bmn&8LMllnFf&78((I  ;O(P+/I ( %rCsave_directoryis_main_process save_functionsafe_serializationvariantmax_shard_size push_to_hubc tjj|rtj d|dyt |dd} | I| duxrt | txr | j} | s#td| jjd|rtnt} t| |} | jddjd d } tj |d |r|j#d d} |j#dd}|j#dd}|j#dd}|j#d|j%tjj&d}t)|d ||j*}|}|r|j-||j/}t1||| }|r tj2|D]}||j4j7vr tjj9||}tjj|s`| jddjd d}|jdd}|jddjd d}|j;|st<j?|tj@||j4jCD]\}}|Dcic]}|||jE}}tjj9||}|r%tFjHjK||ddintIjL|||jNr|jP|jRd}|rtTntV}tjj9|t||}tY|dd5}t[j\|dd d!z} |j_| dddtjad"|d#tc|j4d$|d%n8tjj9|| }!tjad&|!|rXte'}"tg|"}"|"jMti|d(jk|jm||| )yycc}w#1swYxYw)*a Save a model and its configuration file to a directory so that it can be reloaded using the [`~models.ModelMixin.from_pretrained`] class method. Arguments: save_directory (`str` or `os.PathLike`): Directory to save a model and its configuration file to. Will be created if it doesn't exist. is_main_process (`bool`, *optional*, defaults to `True`): Whether the process calling this is the main process or not. Useful during distributed training and you need to call this function on all processes. In this case, set `is_main_process=True` only on the main process to avoid race conditions. save_function (`Callable`): The function to use to save the state dictionary. Useful during distributed training when you need to replace `torch.save` with another method. Can be configured with the environment variable `DIFFUSERS_SAVE_MODE`. safe_serialization (`bool`, *optional*, defaults to `True`): Whether to save the model using `safetensors` or the traditional PyTorch way with `pickle`. variant (`str`, *optional*): If specified, weights are saved in the format `pytorch_model..bin`. max_shard_size (`int` or `str`, defaults to `"10GB"`): The maximum size for a checkpoint before being sharded. Checkpoints shard will then be each of size lower than this size. If expressed as a string, needs to be digits followed by a unit (like `"5GB"`). If expressed as an integer, the unit is bytes. Note that this limit will be decreased after a certain period of time (starting from Oct 2024) to allow users to upgrade to the latest version of `diffusers`. This is to establish a common default size for this argument across different libraries in the Hugging Face ecosystem (`transformers`, and `accelerate`, for example). push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face Hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. zProvided path (z#) should be a directory, not a fileN hf_quantizerzThe model is quantized with z and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.z.binz {suffix}.binz .safetensorsz{suffix}.safetensorsTexist_okcommit_messageprivate create_prFtokenrepo_id)rPrRrT)rKfilename_patternz{suffix}formatpt)metadata)r[ weight_mapwzutf-8)encodingr)indent sort_keys z:The model is bigger than the maximum size per checkpoint (z) and is going to be split in z^ checkpoint shards. You can find where each parameters has been saved in the index located at .zModel weights saved in )rTz README.md)rTrQrS)7ospathisfilererrorrrris_serializableryquantization_config quant_methodrrrreplacemakedirspopsplitsepr rU save_config state_dictrlistdirfilename_to_tensorskeysr9 startswith _REGEX_SHARD fullmatchremoverm contiguous safetensorsrn save_filesave is_shardedr[tensor_to_filenamerropenjsondumpswriter lenr+r,ras_posix_upload_folder)#r@rFrGrHrIrJrKrLrKrNquantization_serializable weights_nameweights_name_patternrQrRrSrTrU model_to_saverpstate_dict_splitfilename full_filenameweights_without_extfilename_without_exttensorstensorshardfilepathindexsave_index_filefcontentpath_to_weights model_cards# rAsave_pretrainedzModelMixin.save_pretrainedszX 77>>. ) LL?>*::]^ _ t^T:  #D(1|-?@1 00 & - 2<3S3S3`3`2abyy 4F/< #L': +33FNKSS 2  NT2 #ZZ(8$?NjjD1G ;6IJJw-EjjN,@,@,Mb,QRG!'D'QVW__G    % %n 5#--/ > ~H\  JJ~6/CCHHJJ " ^X F ww~~m4&:&B&B62&N&V&VWegi&j#&9&A&A*b&Q#'/'7'7'C'K'KN\^'_$''(;<$../CDPIIm,7 "2!E!E!K!K!M HgKRS7VZ/::<<7ESww||NH=H!!!++E8xQUFV+W 5(+"N  & &,55.AAE:L5QcO ggll><Y`;abOosW=**U1EL > KKL^L\] 0 D DEFG$$3#4A7  !ggll><HO KK1/1BC D 27%HJ,Z8J OOD=FFH I   -#   ;T">=s(S -SScXt|dd}| td|j|S)z Potentially dequantize the model in case it has been quantized by a quantization method that support dequantization. rNNz?You need to first quantize your model in order to dequantize it)rry dequantize)r@rNs rArzModelMixin.dequantize6s5 t^T:  ^_ _&&t,,rCpretrained_model_name_or_pathc &|jdd}|jdd}|jdd}|jdd}|jdd}|jdd}|jd d} |jd d} |jd d} |jd d} |jd d} |jdd}|jdd}|jdd}|jdd}|jdt}|jdd}|jdd}|jdd}|jdd}|jdd}|jdd}t}|r |s td| Ct | t j s)t j} tjd| dd}|d}d}|r!tsd}tjd|ts td|tdd s td!|durtdd s td"|dur|td#|d$t |t jrd%|i}nQt |tr|d&vr d%t j|i}n$t |t r|d)kr td*d%|i}||d}n |s td+|r|tdd,s td-t"d.d/d0}i}|}|j$|f|dd||| | | | ||d1 |\}}}t'j(|}d|vxr|ddu} | s|?| rt+j,|d||d<n||d<t+j.|d| 2}!nd}!|!|!j1| ||3|!j3| } |!j5|}|!j6j8j:|d4<|d}tj=d5n |s td6|j>duxr|!duxs tA|!d7d}"|"rF|j>}#t |#tBs|#g}#|d}tj=d8n|s td9g}#d}$d}%d}&d}'tDjFjI|}(|(|| xsd%|||||| | | |||d:})tKdPi|)}'|,|'tDjFjM|'s tOdPi|)}'|'|s|'jQrd}$|$r |r td;|r=tS|tT|||| | | | ||< }%|j.|fi|}*d=d>l+m,}+|+|*|%}*np|$rt[||'||| | || | xsd%|? \}%}&n(|r& tS|t]t^||||| | | | |||@ }%|%'|$s%tS|t]td||||| | | | |||@ }%t |%tBs|%g}%d}-| Y| tAt dDdk(sEt | t j st| dEtg| d(|ji| }-tkg}.|r#|.jmtojpts|.5|j.|fi|}*ddd|-t jt|-d}/|$s"tw|%d)||F}/*jy|/|$r|&dG}0ntC|/j{}0|!|!j}*||#Ht*||| |#|!}|!|!j1|I|j|*|/|%||0|||||| |!|#|||J\}*}1}2}3}4}5|1|2|3|5dK}6||||4dL}7t|*fi|7|!|!j|*|!|*_C| )| tAt dDdk(r|!|"s|*j| }*|!|*j|| Mn|*j|N|*j||*j|O|r|*|6fS|*S#t$rtd'|d(wxYw#t`$r>},tjcdA|dB|,|stjdCYd},~,d},~,wwxYw#1swYxYw)QuH! Instantiate a pretrained PyTorch model from a pretrained model configuration. The model is set in evaluation mode - `model.eval()` - by default, and dropout modules are deactivated. To train the model, set it back in training mode with `model.train()`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*): Can be either: - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the model weights saved with [`~ModelMixin.save_pretrained`]. cache_dir (`str | os.PathLike`, *optional*): Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used. torch_dtype (`torch.dtype`, *optional*): Override the default `torch.dtype` and load the model with another dtype. force_download (`bool`, *optional*, defaults to `False`): Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. proxies (`dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. output_loading_info (`bool`, *optional*, defaults to `False`): Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub. token (`str` or *bool*, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used. revision (`str`, *optional*, defaults to `"main"`): The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git. from_flax (`bool`, *optional*, defaults to `False`): Load the model weights from a Flax checkpoint save file. subfolder (`str`, *optional*, defaults to `""`): The subfolder location of a model file within a larger model repository on the Hub or locally. mirror (`str`, *optional*): Mirror source to resolve accessibility issues if you're downloading a model in China. We do not guarantee the timeliness or safety of the source, and you should refer to the mirror site for more information. device_map (`int | str | torch.device` or `dict[str, int | str | torch.device]`, *optional*): A map that specifies where each submodule should go. It doesn't need to be defined for each parameter/buffer name; once a given module name is inside, every submodule of it will be sent to the same device. Defaults to `None`, meaning that the model will be loaded on CPU. Examples: ```py >>> from diffusers import AutoModel >>> import torch >>> # This works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map="cuda" ... ) >>> # This also works (integer accelerator device ID). >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map=0 ... ) >>> # Specifying a supported offloading strategy like "auto" also works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", subfolder="unet", device_map="auto" ... ) >>> # Specifying a dictionary as `device_map` also works. >>> model = AutoModel.from_pretrained( ... "stabilityai/stable-diffusion-xl-base-1.0", ... subfolder="unet", ... device_map={"": torch.device("cuda")}, ... ) ``` Set `device_map="auto"` to have 🤗 Accelerate automatically compute the most optimized `device_map`. For more information about each option see [designing a device map](https://huggingface.co/docs/accelerate/en/concept_guides/big_model_inference#the-devicemap). You can also refer to the [Diffusers-specific documentation](https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#model-sharding) for more concrete examples. max_memory (`Dict`, *optional*): A dictionary device identifier for the maximum memory. Will default to the maximum memory available for each GPU and the available CPU RAM if unset. offload_folder (`str` or `os.PathLike`, *optional*): The path to offload weights if `device_map` contains the value `"disk"`. offload_state_dict (`bool`, *optional*): If `True`, temporarily offloads the CPU state dict to the hard drive to avoid running out of CPU RAM if the weight of the CPU state dict + the biggest shard of the checkpoint does not fit. Defaults to `True` when there is some disk offload. low_cpu_mem_usage (`bool`, *optional*, defaults to `True` if torch version >= 1.9.0 else `False`): Speed up model loading only loading the pretrained weights and not initializing the weights. This also tries to not use more than 1x model size in CPU memory (including peak memory) while loading the model. Only supported for PyTorch >= 1.9.0. If you are using an older version of PyTorch, setting this argument to `True` will raise an error. variant (`str`, *optional*): Load weights from a specified `variant` filename such as `"fp16"` or `"ema"`. This is ignored when loading `from_flax`. use_safetensors (`bool`, *optional*, defaults to `None`): If set to `None`, the `safetensors` weights are downloaded if they're available **and** if the `safetensors` library is installed. If set to `True`, the model is forcibly loaded from `safetensors` weights. If set to `False`, `safetensors` weights are not loaded. disable_mmap ('bool', *optional*, defaults to 'False'): Whether to disable mmap when loading a Safetensors model. This option can perform better when the model is on a network mount or hard drive, which may not handle the seeky-ness of mmap very well. > [!TIP] > To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in with `hf > auth login`. You can also activate the special > ["offline-mode"](https://huggingface.co/diffusers/installation.html#offline-mode) to use this method in a > firewalled environment. Example: ```py from diffusers import UNet2DConditionModel unet = UNet2DConditionModel.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", subfolder="unet") ``` If you get the error message below, you need to finetune the weights for your downstream task: ```bash Some weights of UNet2DConditionModel were not initialized from the model checkpoint at stable-diffusion-v1-5/stable-diffusion-v1-5 and are newly initialized because the shapes did not match: - conv_in.weight: found shape torch.Size([320, 4, 3, 3]) in the checkpoint and torch.Size([320, 9, 3, 3]) in the model instantiated You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. ``` cache_dirNignore_mismatched_sizesFforce_download from_flaxproxiesoutput_loading_infolocal_files_onlyrTrevision torch_dtype subfolder device_map max_memoryoffload_folderoffload_state_dictrrJuse_safetensorsrh dduf_entries disable_mmapparallel_configzEParallel loading is not supported when not using `low_cpu_mem_usage`.zPassed `torch_dtype` z7 is not a `torch.dtype`. Defaulting to `torch.float32`.Ta,Cannot initialize model with low cpu memory usage because `accelerate` was not found in the environment. Defaulting to `low_cpu_mem_usage=False`. It is strongly recommended to install `accelerate` for faster and less memory-intense model loading. You can do so with: ``` pip install accelerate ``` .zLoading and dispatching requires `accelerate`. Please make sure to install accelerate or set `device_map=None`. You can install accelerate with `pip install accelerate`.r`raztLoading and dispatching requires torch >= 1.9.0. Please either update your PyTorch version or set `device_map=None`.z~Low memory initialization requires torch >= 1.9.0. Please either update your PyTorch version or set `low_cpu_mem_usage=False`.zEYou cannot set `low_cpu_mem_usage` to `False` while using device_map=zO for loading and dispatching. Please make sure to set `low_cpu_mem_usage=True`.rX)autobalancedbalanced_low_0 sequentialzWhen passing device_map as a string, the value needs to be a device name (e.g. cpu, cuda:0) or 'auto', 'balanced', 'balanced_low_0', 'sequential' but found rbrznYou can't pass device_map as a negative int. If you want to put the model on the cpu, pass device_map = 'cpu' z>Passing along a `device_map` requires `low_cpu_mem_usage=True`z1.10z=`low_cpu_mem_usage` and `device_map` require PyTorch >= 1.10.modelpytorch diffusers file_type framework) rreturn_unused_kwargsreturn_commit_hashrrrrTrr user_agentr) pre_quantized)rrrquantz>Set `low_cpu_mem_usage` to True as `hf_quantizer` is not None.zD`low_cpu_mem_usage` cannot be False or None when using quantization.use_keep_in_fp32_moduleszGSet `low_cpu_mem_usage` to True as `_keep_in_fp32_modules` is not None.zH`low_cpu_mem_usage` cannot be False when `keep_in_fp32_modules` is True.)is_localrrrrrJrrrrTrr commit_hashrzFLoading of sharded checkpoints is not supported when `from_flax=True`.) rrrrrrTrrrrr.)%load_flax_checkpoint_in_pytorch_model)rrrrTrrrr) rrrrrrTrrrrrz(An error occurred while trying to fetch z: zXDefaulting to unsafe serialization. Pass `allow_pickle=False` to raise an error instead. float8_e4m3fnzA needs to be of type `torch.dtype`, e.g. `torch.float16`, but is )rrall_checkpoint_keys)rrkeep_in_fp32_modules)r) rrrrrrrNrris_parallel_loading_enabledr) missing_keysunexpected_keysmismatched_keys error_msgs)r offload_dir offload_index)r_pre_quantization_dtype)r)configrQ)Hrl_LOW_CPU_MEM_USAGE_DEFAULTrNotImplementedErrorrrnrfloat32rrr#r'ryrr RuntimeErrorintr load_configcopydeepcopyrmerge_quantization_configs from_configvalidate_environmentupdate_torch_dtypeupdate_device_maprhrir8r rrrPrcrdisdirr5existsr6is_filer!rmodeling_pytorch_flax_utilsrr rrIOErrorrfrr_set_default_torch_dtyperappend accelerateinit_empty_weightsr;set_default_dtyper9_fix_state_dict_keys_on_loadrspreprocess_modelr3_load_pretrained_modelrbpostprocess_modelrNtoregister_to_configevalenable_parallelism)8clsrrKrrrrrrrrTrrrrrrrrrJrrhrrrr allow_pickler unused_kwargs config_pathrrrrNrrr|resolved_model_filesharded_metadata index_filerindex_file_kwargsrre dtype_orig init_contextsrp loaded_keysrrrrr loading_infodevice_map_kwargss8 rAfrom_pretrainedzModelMixin.from_pretrainedBsp FJJ{D1 "(**-F"N$4e<JJ{E2 **Y-$jj)>F!::&8$? 7D)::j$/jj5 JJ{D1 ZZ d3 ZZ d3 $4d;#ZZ(E4:JJ~t4T zz.%8 IOTegkIl&@# &/@%&mn n  ":k5;;+O--K NN' }4kl   ""OL %<%> %  NN.   !*A*C%`   !*:4*I%&   $-=dG-L%.   %**@WXbWcdRR  j%,, /j)J  C (Z?s-s  %,,z":;  C (A~ E!*-  ! ($(!& !abb %.>tV.L !`aa% "  4 .=S__ . !%#)-!%. . * { v&.7eFCX? -J"b.", 0 $&(  ';):;   J$6bggnnZ>X1F4EFJ  !|z7I7I7KJ )ef f "1-.#-!1!#%' # $COOF!>!@ A ] +#COOF>bbcddlm}t|ddtjk(rdt|ddr%t ddrt d|jd t|d dr%t dd rt d |jd ||r/tjd|jjd|St|4|i|S)Nr_is_group_offload_enabledquantization_methodis_loaded_in_8bitF<0.48.0zCalling `cuda()` is not supported for `8-bit` quantized models with the installed version of bitsandbytes. The current device is `L`. If you intended to move the model, please install bitsandbytes >= 0.48.0.is_loaded_in_4bit0.43.2zCalling `cuda()` is not supported for `4-bit` quantized models with the installed version of bitsandbytes. The current device is `L`. If you intended to move the model, please install bitsandbytes >= 0.43.2. The module 'zD' is group offloaded and moving it using `.cuda()` is not supported.)rxrrrBITS_AND_BYTESr%ryrrrrrLrcuda)r@rJrKrrs rAr zModelMixin.cudaasF 4. 59K9Z9Z Zt0%8=TUXZb=c ..2kk];GH2E:?VWZ\d?e ..2kk];GH %T * NNt~~6677{| Kw|T,V,,rCcddlm}td|Dxsd|v}d|v}|D]+}t|ts t j |d}-|s%|D] }t|t jsd}nt|ddr |r td t|d dtjk(rdt|d dr%td d rtd|j dt|ddr%td drtd|j d||r1|r/tjd|j j"d|St%|L|i|S#t$rYEwxYw)Nrrc3PK|]}t|tj ywr>)rrnr)rargs rArz ModelMixin.to..s)XSWC*S%,,*GSWrrrT is_quantizedFzCasting a quantized model to a new `dtype` is unsupported. To set the dtype of unquantized layers, please use the `torch_dtype` argument when loading the model using `from_pretrained` or `from_single_file`rrrrzCalling `to()` is not supported for `8-bit` quantized models with the installed version of bitsandbytes. The current device is `rrrzCalling `to()` is not supported for `4-bit` quantized models with the installed version of bitsandbytes. The current device is `rrzB' is group offloaded and moving it using `.to()` is not supported.)rxrrrrrnrrrrryrrr%rrrrLrr)r@rJrKrdevice_arg_or_kwarg_presentdtype_present_in_argsr rs rArz ModelMixin.to|sF&))XSW)X&X&n\dhn\n# '6 1Cc3'  S!.2+ %c5;;/,0) 4 /$ z 4. 59K9Z9Z Zt0%8=TUXZb=c ..2kk];GH2E:?VWZ\d?e ..2kk];GH %T */J NNt~~6677yz Kwz4*6**C   sE E,+E,cLt|ddr tdt| |S)Nr Fz`.half()` is not supported for quantized model. Please use the model as it is, since the model has already been cast to the correct `dtype`.)rryrhalfr@rJrs rArzModelMixin.halfs3 4 /G  7<& &rCcLt|ddr tdt| |S)Nr Fz`.float()` is not supported for quantized model. Please use the model as it is, since the model has already been cast to the correct `dtype`.)rryrfloatrs rArzModelMixin.floats3 4 /G  7=$' 'rCct|dd}|s#td|jjdd}|j D]/}|jj|vs|j |i|d}1|std|dy) u Compiles *only* the frequently repeated sub-modules of a model (e.g. the Transformer layers) instead of compiling the entire model. This technique—often called **regional compilation** (see the PyTorch recipe https://docs.pytorch.org/tutorials/recipes/regional_compilation.html) can reduce end-to-end compile time substantially, while preserving the runtime speed-ups you would expect from a full `torch.compile`. The set of sub-modules to compile is discovered by the presence of **`_repeated_blocks`** attribute in the model definition. Define this attribute on your model subclass as a list/tuple of class names (strings). Every module whose class name matches will be compiled. Once discovered, each matching sub-module is compiled by calling `submodule.compile(*args, **kwargs)`. Any positional or keyword arguments you supply to `compile_repeated_blocks` are forwarded verbatim to `torch.compile`. _repeated_blocksNzM`_repeated_blocks` attribute is empty. Set `_repeated_blocks` for the class `z&` to benefit from faster compilation. FTz$Regional compilation failed because z% classes are not found in the model. )rryrrLrcompile)r@rJrKrepeated_blockshas_compiled_regionsubmods rAcompile_repeated_blocksz"ModelMixin.compile_repeated_blockss"$(:DA99=9P9P8QQwy $llnF((O;//&*#% #66GGlm #rC)cp_planrrc tjdtjj s)tjj s t dddlm}ddl m }ddl m }m }ddlm}m}t#|t$r t'| }tjj)} tjj+} tj,j/j0} tj2| } tj4| | | j7z} |||f}|j8|j;D]}t#||s|j<}| t?|d s+|j@}||jC\}}n||}|jE|sHtG|jH}tKd |jLjNd |jPd |dnd}|j8L|j8}tjjRjU| |jV|jX}|j[| | | |||_.|j;D]1}t#||s|j<}| t?|ds+||_.3|j8>||j^ tKd||n |j^}|||j8|yy)Nz`enable_parallelism` is an experimental feature. The API may change in the future and breaking changes may be introduced at any time without warning.zXtorch.distributed must be available and initialized before calling `enable_parallelism`.r)apply_context_parallelr.r$)r&r'r*)context_parallel_configr0zrir4attention_backend_rBr# cp_configs rArzModelMixin.enable_parallelisms  d   --/8I8I8X8X8Zj  D3WB f3 4#FCF  ))+&&557 hh//166 // < k4-2L2L2N+NO&8LM  ) ) 5,,.!&*;<",, $GI?S,T$-$@$@!$,+D+W+W+Y(%q(<=N(O%0OOPab*01J1e1e*f'$VW`WjWjWsWsVtu-->-D-D,EFFFYEZ[@A3)6  ) ) 566I$$00AA'$//(77BD  T:vD 9 &llnFf&78((I  ;M(N)/I & %  ) ) 54==#8 r")!4g$--G "4)G)G Q 6rCrprrrassign_to_params_buffersrNrrrrrrrrrc|j}t|j}tt|t|z }||j ||d}tt|t|z }|j 7|j D](}|Dcgc]}t j|||}}*g}g}| :d| jvr(| tdtj|d| d} | t| |}t||| || d| jvrind}d\}}| rtj}i}||g}t!j"|rt$nt&fid|d |d | d | d |d | d|d|d|d|d|d|d|d|d| d|}|r||\}}}} || z }||z }nW|}!t)|dkDr)ddi}"t+sd|"d<t-j.|fi|"}!|!D]}#||#\}}}} || z }||z }t1|@t)|dkDr2t3||d}| r"t5|||t7j8|t)|dkDr?dj;|}$d|$vr|$dz }$t=d |j>j@d!|$t)|dkDr9tBjEd"|d#|j@d$d%j;|gn-tBjGd&|j>j@d't)|dkDr4tBjEd(|j>j@d)|d*|d+nUt)|dk(rGtBjGd,|j>j@d-|d.|j>j@d/t)|dkDrdd0j;|D%&'cgc]\}%}&}'d1|%d2|&d3|'d4c}'}&}%}(tBjEd(|j>j@d)|d5|(d+||||||fScc}wcc}'}&}%w)6NrX)prefixdiskzThe current `device_map` had weights offloaded to the disk. Please provide an `offload_folder` for them. Alternatively, make sure you have `safetensors` installed if the model you are using offers the weights in this format.TrO)NNrmodel_state_dictrrrNrrrrrrstate_dict_indexstate_dict_folderrrrr.desczLoading checkpoint shardsdisablerz z size mismatchz_ You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.z#Error(s) in loading state_dict for z: z(Some weights of the model checkpoint at z! were not used when initializing z: r/z9All model checkpoint weights were used when initializing z. zSome weights of z3 were not initialized from the model checkpoint at z and are newly initialized: zo You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.zAll the weights of z/ were initialized from the model checkpoint at zf. If your task is similar to the task the model of the checkpoint was trained on, you can already use z* for predictions without further training.raz- z: found shape z in the checkpoint and z in the model instantiatedz= and are newly initialized because the shapes did not match: )$rprPrsrupdate_missing_keys"_keys_to_ignore_on_load_unexpectedresearchr7ryrcrkr4r2tempfilemkdtemp functoolspartialr8r7rr)r(tqdmr-rdrcshutilrmtreer9rrrLrrr ))rrrprrrrr9rNrrrrrrrrrr= expected_keysrrpatrqrrexpanded_device_maprr?r>load_fn_mismatched_keys _error_msgs shard_filesshard_tqdm_kwargs shard_file error_msgkeyshape1shape2mismatched_warnings) rArz!ModelMixin._load_pretrained_model;s*!++--2245 C .[1AAB  #';;E   !f 0A0A0C&C% :  NT:!)%)"  !"4Z"O  %e-@% V(4:CTCTCV9V\` .8++  ( 0 0 2 !   !$., ##1L -Rb  . "    &  "6 & $ , ( * . 0 %< 0! "&# ( 'MTUhMi JM+-={ + %J / /O-K&'!+%+-H$I!.037%i0%ll+>TBST ) QXYcQdN /1A;k) #33*   $]);a)? }n = M!&u.>@QR /0 z?Q  J/I)+w !DU__E]E]D^^cdmcnop p  ! # NN:;X:YYz{~|H|H{IINPTPYPYZiPjOkNlm  KKSTYTcTcTlTlSmmpq r | q NN"5??#;#;"<=122N|n]nn   !Q & KK%eoo&>&>%?@123CCH??C[C[B\]--   ! #!%0?/>+VV^F83J6(Rlm/>"  NN"5??#;#;"<=123./0AA lO_mU___C#\jsP8;P8P= ctj|jj}|j Dcic]&\}}|j tj k(s$||(}}}t|j Dchc]%\}}|j tj k7s$|'c}}}t|jdhz }||fScc}}wcc}}w)Nr@) inspect signaturerBr|rmdefault_emptyrrs)robjr|rqrrrequired_parametersoptional_parametersexpected_moduless rA_get_signature_keyszModelMixin._get_signature_keyss&&s||4?? 0:0@0@0Bb0B1aiiSZSaSaFaq!t0Bb!1A1A1C"c1CAqyyT[TbTbGb11C"cd2779:fXE!444 c"cs%C#C%C )C ct}|g}t|dkDr|jd}|jj|vrut |t rI|j%t|jjd|d|t|jz}|t|jz }t|dkDrt|S)a Get the modules of the model that should not be split when using device_map. We iterate through the modules to get the underlying `_no_split_modules`. Args: device_map (`str`): The device map value. Options are ["auto", "balanced", "balanced_low_0", "sequential"] Returns: `list[str]`: list of modules that should not be split rrVz does not support `device_map='z_'`. To implement support, the model class needs to implement the `_no_split_modules` attribute.) rrrlrrLrr_no_split_modulesryrPr)r@rrfmodules_to_checkris rA_get_no_split_modulesz ModelMixin._get_no_split_moduless E 6"#a'%))"-F((0AAfj1//7(%//8899XYcXdeZZ ->FD\D\@],]) D):$;; "#a'%&&rCc|jstd|jd|dtj d|jd|dt j }t j||S)a Change the default dtype and return the previous one. This is needed when wanting to instantiate the model under specific dtype. Args: dtype (`torch.dtype`): a floating dtype to set to. Returns: `torch.dtype`: the original `dtype` that can be used to restore `torch.set_default_dtype(dtype)` if it was modified. If it wasn't, returns `None`. Note `set_default_dtype` currently only works with floating-point types and asserts if for example, `torch.int64` is passed. So if a non-float `dtype` is passed this functions will throw an exception. zCan't instantiate z model under dtype=z' since it is not a floating point dtypezInstantiating z model under default dtype rb)rryrLrr rnget_default_dtyper)rrrs rArz#ModelMixin._set_default_torch_dtypesw"&&$S\\N2EeWLst   nS\\N2MeWTUVW,,.  &rCct|S)z `torch.device`: The device on which the module is (assuming that all the module parameters are on the same device). )rrs rArzModelMixin.devices $D))rCct|S)zw `torch.dtype`: The dtype of the module (assuming that all the module parameters have the same dtype). )rrs rArzModelMixin.dtype's #4((rConly_trainableexclude_embeddingsc t|dd}|rtrddl}n td|rh|j Dcgc]%\}}t |t js!|d'}}}|jDcgc] \}}||vs |} }}nt|j} g} | D]} | js|r|rt | j jrht| dr| j} n%t| dr| jj } nd } | j#| j%d z| z| j#| j%t'| Scc}}wcc}}w) a Get number of (trainable or non-embedding) parameters in the module. Args: only_trainable (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of trainable parameters. exclude_embeddings (`bool`, *optional*, defaults to `False`): Whether or not to return only the number of non-embedding parameters. Returns: `int`: The number of parameters. Example: ```py from diffusers import UNet2DConditionModel model_id = "stable-diffusion-v1-5/stable-diffusion-v1-5" unet = UNet2DConditionModel.from_pretrained(model_id, subfolder="unet") unet.num_parameters(only_trainable=True) 859520964 ``` rFrNzbitsandbytes is not installed but it seems that the model has been loaded in 4bit precision, something went wrong make sure to install bitsandbytes with `pip install bitsandbytes`. You also need a GPU. z.weight element_size quant_storager.r)rr$ bitsandbytesryrrr EmbeddingrrPr| requires_grad Params4bitrrprqitemsizernumelsum) r@rmrnrbnbr module_typeembedding_param_namesretotal_parameters total_numelr num_bytess rAnum_parameterszModelMixin.num_parameters.sw0$D*=uE (** p :>:L:L:N%:N%6T;R\]hjljvjvRw4& :N "%261F1F1H 1HodIDXmLm 1H   $DOO$56  %E"".%E366;L;L)Mun5$)$6$6$8  8$)$7$7$@$@ $% &&u{{}q'89'DE&&u{{}5&;5% s"E9"E9? E? E?c@t|jDcgc]#}|j|jz%c}}|rKt|j Dcgc]#}|j|jz%c}}||z}|Scc}wcc}w)a Get the memory footprint of a model. This will return the memory footprint of the current model in bytes. Useful to benchmark the memory footprint of the current model and design some tests. Solution inspired from the PyTorch discussions: https://discuss.pytorch.org/t/gpu-memory-that-model-uses/56822/2 Arguments: return_buffers (`bool`, *optional*, defaults to `True`): Whether to return the size of the buffer tensors in the computation of the memory footprint. Buffers are tensors that do not require gradients and not registered as parameters. E.g. mean and std in batch norm layers. Please see: https://discuss.pytorch.org/t/what-pytorch-means-by-buffers/120266/2 )rxr|nelementrpr})r@return_buffersrmembufmem_bufss rAget_memory_footprintzModelMixin.get_memory_footprintnsHYZHYu5>>#e&8&8&::HYZ[ 4<<>Z>CCLLNS-=-=-??>Z[H.C [Zs (B (Brcd}|jD]>\}}t|dstjd|d|d||_||_d}@|s#t d|jjdy) NFrz Setting `gradient_checkpointing=z` for ''Tz The module z does not support gradient checkpointing. Please make sure to use a module that supports gradient checkpointing by creating a boolean attribute `gradient_checkpointing`.) rrrdebugrrryrrL)r@rris_gradient_checkpointing_setrris rArz&ModelMixin._set_gradient_checkpointings).% ..0LD&v78 ?xwtfTUVW6Q306-04- 1-dnn5567~ -rCcPgfdd|D] }|d|vr|j|d||d<|d|vr|j|d||d<|d|vr|j|d||d<|d |vr|j|d ||d <|d |vr|j|d ||d <|d |vr|j|d ||d<|d|vr|j|d||d<|d|vs|j|d||d<|S)aK This function fix the state dict of the model to take into account some changes that were made in the model architecture: - deprecated attention blocks (happened before we introduced sharded checkpoint, so this is why we apply this method only when loading non sharded checkpoints for now) ct|dr|jrj||jD]\}}|dk(r|n|d|}||y)N_from_deprecated_attn_blockrXrb)rrrnamed_children)rrisub_name sub_module deprecated_attention_block_pathsrecursive_find_attn_blocks rArzJModelMixin._fix_state_dict_keys_on_load..recursive_find_attn_blocks_v<=&BdBd077=(.(=(=(?$*'+rz8$q 7K)(J?)@rCrXz .query.weightz .to_q.weightz .query.biasz .to_q.biasz .key.weightz .to_k.weightz .key.biasz .to_k.biasz .value.weightz .to_v.weightz .value.biasz .to_v.biasz.proj_attn.weightz.to_out.0.weightz.proj_attn.biasz.to_out.0.bias)rl)r@rprdrrs @@rArz'ModelMixin._fix_state_dict_keys_on_loads,.( @ ""d+ 5D}%34>NNdV=CY4Z dV<01{#z12<..D6AU2V dV:./{#z14>NNdV;CW4X dV<01y!Z/2<..D6AS2T dV:./}%34>NNdV=CY4Z dV<01{#z12<..D6AU2V dV:./()Z78B$O`Ga8b dV#345':56@nnv_E]6^ dV>23354rCr>)rfN)TNTN10GBF) FFNTNNNNNNFF)FF)T)SrLrMrNrOr config_name_automatically_saved_argsrrCrfrrrrr.r2 _skip_keysrBrrrpropertyboolrr rrrrrrrrrrrrnrrrr rrr rrr!rCrErcPathLikerr classmethodrrrrr rrrrr1r/dictr0rrrPrr rrdrhrrrrrrr __classcell__rs@rArrsxK V',$)-& '+$!%HJ1 )))$n4nn oTXodho<;==$= /0PT9'+9=E_9 9"IDI 0 1 1U]`dUd 1pt 1MxRVMbfM@@&+%8%8,07;IM" S {{S {{T)S $CHo4 S $D$93$>?$F S  S  S p(4u||E':)+/" #+/$(%)B ||B  B  B "Dj B  B B B "DjB TzB d B  B H<>S<>T<>|0,!%)-#'"$*!Ybkk)YY $ Y ! Y t Yc YYv -ZC"++*T1!_`"&*D[#_`$%_`_`B55'':U[[U[[4* **)u{{)) > T> t> `c> @&"5;;KaKaKlKl@H $1{1t1rCrc\eZdZdZeedeejzdzffd Z xZ S)LegacyModelMixinz A subclass of `ModelMixin` to resolve class mapping from legacy classes (like `Transformer2DModel`) to more pipeline-specific classes (like `DiTTransformer2DModel`). rNc ddlm}|j}|jdd}|jdd}|jdd}|jdd}|jdd} |jd d} |jd d} |} td d d } |j | f|dd|||| | | | d |\}}}|||}||urt t|"|fi|S|j|fi|S)Nr.)_fetch_remapped_cls_from_configrrFrrrTrrrrrT) rrrrrrrTrrr) model_loading_utilsrrrlrrrrr)rrrKr kwargs_copyrrrrrTrrrrrr7remapped_classrs rArz LegacyModelMixin.from_pretraineds< Ikkm JJ{D1 $4e<**Y-!::&8$? 7D)::j$/JJ{D1 4 % " 's   !%#)-!    19E S )>J-1< 2>112O_S^_ _rC) rLrMrNrOrrrrcrrrrs@rArrs= /`C"++rs"  #066 VV6"C?&>\\2__   --$   H %rzz12   wwWW**""ww..gg,,00ww..wwggnngg,,WW**ww..gg,,"D'"!%!&)K%EHHOO% %44#588??4#u{{4#n44&].]@77`z7`rC