(ti0 ddlZddlmZmZddlZddlZddlmZm Z ddl m Z m Z ddl mZddlmZmZmZe rddlZ dd ed ed ed d ej.fdZGddeeZy)N)CallableLiteral) ConfigMixinregister_to_config) deprecateis_scipy_available) randn_tensor)KarrasDiffusionSchedulersSchedulerMixinSchedulerOutputnum_diffusion_timestepsmax_betaalpha_transform_type)cosineexplaplacereturnc 6|dk(rd}n |dk(rd}n|dk(rd}ntd|g}t|D]<}||z }|dz|z }|jtd||||z z |>t j |tj S) a> Create a beta schedule that discretizes the given alpha_t_bar function, which defines the cumulative product of (1-beta) over time from t = [0,1]. Contains a function alpha_bar that takes an argument t and transforms it to the cumulative product of (1-beta) up to that part of the diffusion process. Args: num_diffusion_timesteps (`int`): The number of betas to produce. max_beta (`float`, defaults to `0.999`): The maximum beta to use; use values lower than 1 to avoid numerical instability. alpha_transform_type (`str`, defaults to `"cosine"`): The type of noise schedule for `alpha_bar`. Choose from `cosine`, `exp`, or `laplace`. Returns: `torch.Tensor`: The betas used by the scheduler to step the model outputs. rcftj|dzdz tjzdz dzS)NgMb?gT㥛 ?r)mathcospits w/Volumes/fast/ai/experiments/MLX_z-image/.venv/lib/python3.12/site-packages/diffusers/schedulers/scheduling_sasolver.py alpha_bar_fnz)betas_for_alpha_bar..alpha_bar_fn=s-88QY%/$''9A=>!C Crc dtjdd|z ztjddtjd|z zz dzz}tj|}tj |d|zz S)Ngr ?rgư>)rcopysignlogfabsrsqrt)rlmbsnrs rrz)betas_for_alpha_bar..alpha_bar_fnBsmq#'22TXXa!diiPSVWPWFXBX>X[_>_5``C((3-C99SAG_- -rrc2tj|dzS)Ng()rrrs rrz)betas_for_alpha_bar..alpha_bar_fnIs88AI& &rz"Unsupported alpha_transform_type: r dtype) ValueErrorrangeappendmintorchtensorfloat32)rrrrbetasit1t2s rbetas_for_alpha_barr6#s0x' D  * .  & '=>R=STUU E * + ( (!e. . S\"- R0@@@(KL, <<U]] 33rc0feZdZdZeDcgc]}|j c}}ZdZedddddddd dd d d d dd d d d d e d dddfde de de de de jee zdzde de de dedzdede de de ded ed!ed"ed#ed$e d%e d&e dzd'e d(e f.d)Zed*Zed+ZdWd,e fd-ZdXd.e d/e ej.zfd0Zd1ej2d2ej2fd3Zd4Zd5Zd6ej2d2ej2fd7Zd6ej2d.e d2ej2fd8Z dYd6ej2d.e d9e d:e d2ej2f d;Zdd<d=ej2d1ej2d2ej2fd>Z d?Z!d@Z"dAZ#dBZ$d=ej2d1ej2dCej2dDe dEej2d2ej2f dFZ%dGej2dHej2dIej2dJej2dDe dEej2d2ej2fdKZ& dZdLe ej2zdMej2dzd2e fdNZ'dOZ( d[d=ej2dLe d1ej2dPed2e)e*zf dQZ+d1ej2d2ej2fdRZ,dSej2dCej2dTejZd2ej2fdUZ.dVZ/ycc}}w)\SASolverScheduleru `SASolverScheduler` is a fast dedicated high-order solver for diffusion SDEs. This model inherits from [`SchedulerMixin`] and [`ConfigMixin`]. Check the superclass documentation for the generic methods the library implements for all schedulers such as loading and saving. Args: num_train_timesteps (`int`, defaults to 1000): The number of diffusion steps to train the model. beta_start (`float`, defaults to 0.0001): The starting `beta` value of inference. beta_end (`float`, defaults to 0.02): The final `beta` value. beta_schedule (`str`, defaults to `"linear"`): The beta schedule, a mapping from a beta range to a sequence of betas for stepping the model. Choose from `linear`, `scaled_linear`, or `squaredcos_cap_v2`. trained_betas (`np.ndarray`, *optional*): Pass an array of betas directly to the constructor to bypass `beta_start` and `beta_end`. predictor_order (`int`, defaults to 2): The predictor order which can be `1` or `2` or `3` or '4'. It is recommended to use `predictor_order=2` for guided sampling, and `predictor_order=3` for unconditional sampling. corrector_order (`int`, defaults to 2): The corrector order which can be `1` or `2` or `3` or '4'. It is recommended to use `corrector_order=2` for guided sampling, and `corrector_order=3` for unconditional sampling. prediction_type (`str`, defaults to `epsilon`, *optional*): Prediction type of the scheduler function; can be `epsilon` (predicts the noise of the diffusion process), `sample` (directly predicts the noisy sample`) or `v_prediction` (see section 2.4 of [Imagen Video](https://huggingface.co/papers/2210.02303) paper). tau_func (`Callable`, *optional*): Stochasticity during the sampling. Default in init is `lambda t: 1 if t >= 200 and t <= 800 else 0`. SA-Solver will sample from vanilla diffusion ODE if tau_func is set to `lambda t: 0`. SA-Solver will sample from vanilla diffusion SDE if tau_func is set to `lambda t: 1`. For more details, please check https://huggingface.co/papers/2309.05019 thresholding (`bool`, defaults to `False`): Whether to use the "dynamic thresholding" method. This is unsuitable for latent-space diffusion models such as Stable Diffusion. dynamic_thresholding_ratio (`float`, defaults to 0.995): The ratio for the dynamic thresholding method. Valid only when `thresholding=True`. sample_max_value (`float`, defaults to 1.0): The threshold value for dynamic thresholding. Valid only when `thresholding=True` and `algorithm_type="dpmsolver++"`. algorithm_type (`str`, defaults to `data_prediction`): Algorithm type for the solver; can be `data_prediction` or `noise_prediction`. It is recommended to use `data_prediction` with `solver_order=2` for guided sampling like in Stable Diffusion. lower_order_final (`bool`, defaults to `True`): Whether to use lower-order solvers in the final steps. Default = True. use_karras_sigmas (`bool`, *optional*, defaults to `False`): Whether to use Karras sigmas for step sizes in the noise schedule during the sampling process. If `True`, the sigmas are determined according to a sequence of noise levels {σi}. use_exponential_sigmas (`bool`, *optional*, defaults to `False`): Whether to use exponential sigmas for step sizes in the noise schedule during the sampling process. use_beta_sigmas (`bool`, *optional*, defaults to `False`): Whether to use beta sigmas for step sizes in the noise schedule during the sampling process. Refer to [Beta Sampling is All You Need](https://huggingface.co/papers/2407.12173) for more information. lambda_min_clipped (`float`, defaults to `-inf`): Clipping threshold for the minimum value of `lambda(t)` for numerical stability. This is critical for the cosine (`squaredcos_cap_v2`) noise schedule. variance_type (`str`, *optional*): Set to "learned" or "learned_range" for diffusion models that predict variance. If set, the model's output contains the predicted Gaussian variance. timestep_spacing (`str`, defaults to `"linspace"`): The way the timesteps should be scaled. Refer to Table 2 of the [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://huggingface.co/papers/2305.08891) for more information. steps_offset (`int`, defaults to 0): An offset added to the inference steps, as required by some model families. r ig-C6?g{Gz?linearNrepsilonFgףp= ??data_predictionTinflinspacernum_train_timesteps beta_startbeta_end beta_schedule trained_betaspredictor_ordercorrector_orderprediction_typetau_func thresholdingdynamic_thresholding_ratiosample_max_valuealgorithm_typelower_order_finaluse_karras_sigmasuse_exponential_sigmasuse_beta_sigmasuse_flow_sigmas flow_shiftlambda_min_clipped variance_typetimestep_spacing steps_offsetc|jjrts tdt |jj|jj |jj gdkDr td|+tj|tj|_ n|dk(r-tj|||tj|_ nk|dk(r6tj|dz|dz|tjdz|_ n0|d k(rt||_ nt|d |jd |jz |_tj"|j d |_tj&|j$|_tj&d|j$z |_tj,|j(tj,|j*z |_d|j$z |j$z dz|_d |_| dvrt| d |jd|_t7jd |dz |t6jdddj9}tj:||_dgt?||dz z|_ dgt?||dz z|_!| d|_"n| |_"| dk(|_#d |_$d|_%d|_&d|_'|j0jQd|_y)Nz:Make sure to install scipy if you want to use beta sigmas.r znOnly one of `config.use_beta_sigmas`, `config.use_exponential_sigmas`, `config.use_karras_sigmas` can be used.r)r9 scaled_linearr!rsquaredcos_cap_v2z is not implemented for r;rdim)r<noise_predictionc|dk\r|dkrdSdS)Ni r rrs rz,SASolverScheduler.__init__..s18Sa&Ga&Grr<cpu))configrOr ImportErrorsumrNrMr+r/r0r1r2r>r6NotImplementedError __class__alphascumprodalphas_cumprodr%alpha_tsigma_tr#lambda_tsigmasinit_noise_sigmanum_inference_stepsnpcopy from_numpy timestepsmax timestep_list model_outputsrG predict_x0lower_order_nums last_sample _step_index _begin_indexto)selfr?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQrRrSrTrUrss r__init__zSASolverScheduler.__init__s6 ;; & &/A/CZ[ [  ++T[[-O-OQUQ\Q\QnQno pst tA   $m5==IDJ h & H>QY^YfYfgDJ o -OcM'--    J1 1,-@ADJ%7OPTP^P^O_&`a aDJJ& #mmDKKQ?zz$"5"56 zz!d&9&9"9:  $,,/%))DLL2II D///43F3FF3N !$ !H H%(88PQUQ_Q_P`&ab b$( KK#6#:rtjdd|jj z |dz} d | z }tj|jj@|zd|jj@dz |zzz ddj}||jj zj}tj2||ddgjtj4}ntjB|tj dtE||}d|j(dz |j(dz d z} tj2|| ggjtj4}tjF||_$tjF|jK|tj |_&tE||_'dgtQ|jjR|jjTdz z|_+d|_,d|_-d|_.d|_/|jHjKd |_$ycc} wcc} wcc} w)a Sets the discrete timesteps used for the diffusion chain (to be run before inference). Args: num_inference_steps (`int`): The number of diffusion steps used when generating samples with a pre-trained model. device (`str` or `torch.device`, *optional*): The device to which the timesteps should be moved to. If `None`, the timesteps are not moved. rr>r Nr\leadingtrailingzY is not supported. Please make sure to choose one of 'linspace', 'leading' or 'trailing'.r!) in_sigmasror;)rr*ra)0r/ searchsortedfliprlrbrRr?numpyitemrTrpr>roundrqastypeint64arangerUr+arrayrir#rM_convert_to_karras _sigma_to_t concatenater1rN_convert_to_exponentialrO_convert_to_betarPrQinterplenrrrmr|rsrortrDrErvrxryrzr{) r}ror clipped_idx last_timesteprs step_ratiorm log_sigmassigmarg sigma_lasts r set_timestepszSASolverScheduler.set_timesteps s((DMMA3)GIgIgh ++99KGNNPVVX  ;; ' ': 5 A}q02E2IJPPRSWUWSWXY\Z\]bbdkklnltltu [[ ) )Y 6&+>+BCJ1&9A&=>KRRTUYWYUYZ[^\^_ddfmmnpnvnvwI 11 1I [[ ) )Z 788;NNJ -ZK@FFHMMOVVWYW_W_`I NI;;//01JK A 3 33t7J7JJsRSVVF^ ;; ( (WWV_))+F,,vSf,gFSY!ZSY%$"2"25*"ESY!Z[aacI^^VVBC[$9:AA"**MF [[ / /WWV_))+F11FXk1lFSY!ZSY%$"2"25*"ESY!Z[I^^VVBC[$9:AA"**MF [[ ( (WWV_))+F**VQd*eFSY!ZSY%$"2"25*"ESY!Z[I^^VVBC[$9:AA"**MF [[ ( ([[A (G(G$GI\_`I`aF6\FWWT[[33f<T[[E[E[^_E_ciDi@ijklomopuuwF$++"A"AAGGII^^VVBC[$9:AA"**MFYYy"))As6{*CVLFt221559L9LQ9OOTWWJ^^Vj\$:;BB2::NF&&v. )))477vU[[7Y#&y>   ++T[[-H-H1-L MN!"  kknnU+ I"[ "[ "[s#\0)\5!\:samplercb|j}|j^}}}|tjtjfvr|j }|j ||tj|z}|j}tj||jjd}tj|d|jj}|jd}tj|| ||z }|j ||g|}|j!|}|S)az Apply dynamic thresholding to the predicted sample. "Dynamic thresholding: At each sampling step we set s to a certain percentile absolute pixel value in xt0 (the prediction of x_0 at timestep t), and if s > 1, then we threshold xt0 to the range [-s, s] and then divide by s. Dynamic thresholding pushes saturated pixels (those near -1 and 1) inwards, thereby actively preventing pixels from saturation at each step. We find that dynamic thresholding results in significantly better photorealism as well as better image-text alignment, especially when using very large guidance weights." https://huggingface.co/papers/2205.11487 Args: sample (`torch.Tensor`): The predicted sample to be thresholded. Returns: `torch.Tensor`: The thresholded sample. r rY)r.rt)r*shaper/r1float64floatreshaperpprodabsquantilerbrIclamprJ unsqueezer|)r}rr* batch_sizechannelsremaining_dims abs_sampless r_threshold_samplez#SASolverScheduler._threshold_sample_s( 06 - H~  6 6\\^F Hrww~7N,NOZZ\ NN:t{{'M'MST U KK 1$++66  KKNVaR+a/ HF~F5! rctjtj|d}||ddtjfz }tj|dk\dj dj |jddz }|dz}||}||}||z ||z z } tj | dd} d| z |z| |zz} | j|j} | S)a Convert sigma values to corresponding timestep values through interpolation. Args: sigma (`np.ndarray`): The sigma value(s) to convert to timestep(s). log_sigmas (`np.ndarray`): The logarithm of the sigma schedule used for interpolation. Returns: `np.ndarray`: The interpolated timestep value(s) corresponding to the input sigma(s). g|=Nr)axisr)rtr ) rpr#maximumnewaxiscumsumargmaxcliprr) r}rr log_sigmadistslow_idxhigh_idxlowhighwrs rrzSASolverScheduler._sigma_to_tsFF2::eU34 Jq"**}55))UaZq188a8@EE*JZJZ[\J]`aJaEbQ;!(#9_t , GGAq! Ug H , IIekk "rcr|jjr d|z }|}||fSd|dzdzdzz }||z}||fS)a( Convert sigma values to alpha_t and sigma_t values. Args: sigma (`torch.Tensor`): The sigma value(s) to convert. Returns: `tuple[torch.Tensor, torch.Tensor]`: A tuple containing (alpha_t, sigma_t) values. r rr!)rbrP)r}rrjrks r_sigma_to_alpha_sigma_tz)SASolverScheduler._sigma_to_alpha_sigma_tsY ;; & &%iGG E1HqLS01GgoGrrct|jdr|jj}nd}t|jdr|jj}nd}||n|dj }||n|dj }d}t j dd|}|d|z z}|d|z z}||||z zz|z} | S)a Construct the noise schedule as proposed in [Elucidating the Design Space of Diffusion-Based Generative Models](https://huggingface.co/papers/2206.00364). Args: in_sigmas (`torch.Tensor`): The input sigma values to be converted. num_inference_steps (`int`): The number of inference steps to generate the noise schedule for. Returns: `torch.Tensor`: The converted sigma values following the Karras noise schedule. sigma_minN sigma_maxr\rg@r )hasattrrbrrrrpr>) r}rrorrrhoramp min_inv_rho max_inv_rhorms rrz$SASolverScheduler._convert_to_karrass$ 4;; , --II 4;; , --II!*!6IIbMrr#)r}rrorrrms rrz)SASolverScheduler._convert_to_exponentials" 4;; , --II 4;; , --II!*!6IIbMscipystatsrppf) r}rrorrrrtimesteprrms rrz"SASolverScheduler._convert_to_betas 0 4;; , --II 4;; , --II!*!6IIbM [!TIP] > The algorithm and model type are decoupled. You can use either data_prediction or noise_prediction for both > noise prediction and data prediction models. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. Returns: `torch.Tensor`: The converted model output. rrNr /missing `sample` as a required keyword argumentrs1.0.0zPassing `timesteps` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`)r<r:)learned learned_ranger v_predictionflow_predictionzprediction_type given as zd must be one of `epsilon`, `sample`, `v_prediction`, or `flow_prediction` for the SASolverScheduler.)r[zQ must be one of `epsilon`, `sample`, or `v_prediction` for the SASolverScheduler.)rpopr+rrmrrrbrKrFrSrHrrjrk) r}rrargskwargsrrrjrkx0_predr:s rconvert_model_outputz&SASolverScheduler.convert_model_output?s2"$i!m47J1M >4y1}a !RSS   Z   DOO,77> ;; % %)< <{{**i7;;,,0LL#/2A2#6L!Gl$::gE,,8&,,>!F*W|-CC,,0AA++doo6 7\#99 / 0K0K/LMVV {{''009N[[ ' '+? ?{{**i7;;,,0LL*1bqb51G*G,,8!Gl$::gE,,>!L07V3CC / 0K0K/LMAA {{''#'<<#94<<;Q!Gg$55@009!Gg$55@N/@rct|dvsJd|dk(r2tj| tj||z dz zS|dk(r;tj| |dztj||z z|dzz zS|dk(rMtj| |dzd|zzdztj||z z|dzd|zzdzz zS|dk(r_tj| |dzd|dzzzd|zzdztj||z z|dzd|dzzzd|zzdzz zSy) zd Calculate the integral of exp(-x) * x^order dx from interval_start to interval_end rr rr)order is only supported for 0, 1, 2 and 3rr rrNr/r)r}orderinterval_start interval_ends r%get_coefficients_exponential_negativez7SASolverScheduler.get_coefficients_exponential_negatives $Q&QQ$ A:99l]+uyy9V/WZ[/[\ \ aZ99l]+!#uyy1N'OOS_bcScd aZ99l]+"Q%77!;uyyXfIf?gg?Q%559; aZ99l]+"Q):%::Q=OORSS))L>9:;?Qq%881|;KKaOQ rc|dvsJdd|dzz|z}d|dzz|z}|dk(r;tj|dtj||z z zd|dzzz S|dk(rGtj||dz |dz tj||z zz zd|dzzdzz S|dk(rYtj||dzd|zz dz|dzd|zz dztj||z zz zd|dzzdzz S|dk(rktj||dzd|dzzz d|zzdz |dzd|dzzz d|zzdz tj||z zz zd|dzzdzz Sy ) zl Calculate the integral of exp(x(1+tau^2)) * x^order dx from interval_start to interval_end rrr rrrrNr)r}rrrtauinterval_end_covinterval_start_covs r%get_coefficients_exponential_positivez7SASolverScheduler.get_coefficients_exponential_positives& $Q&QQ$QJ,6#q&jN: A: *+q599?ORd?d=e3f/fgklortuoukuv aZ *+%))A-=MPb=b;c1dde QJ1$ & aZ *+%q(1/?+??!C)1,q3E/EEIii"25G"G HIJJ QJ1$ & aZ *+%q(1/?/B+BBQIYEYY\]])1,q3Eq3H/HH1OaKaadeeii"25G"G HIJJ QJ1$ & rc |dvsJ|t|dz k(sJ|dk(rdggS|dk(r@d|d|dz z |d |d|dz z gd|d|dz z |d |d|dz z ggS|dk(r|d|dz |d|dz z}|d|dz |d|dz z}|d|dz |d|dz z}d|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z gd|z |d |dz |z |d|dz|z ggS|dk(r|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}|d|dz |d|dz z|d|dz z}d|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z gd|z |d |dz |dz |z |d|dz|d|dzz|d|dzz|z |d |dz|dz|z ggSy)zB Calculate the coefficient of lagrange polynomial rr rrrN)r)r}r lambda_list denominator1 denominator2 denominator3 denominator4s rlagrange_polynomial_coefficientz1SASolverScheduler.lagrange_polynomial_coefficientsa  $$$K(1,,,, A:C5L aZQ+a.89 ^O{1~ A'FG Q+a.89 ^O{1~ A'FG  aZ'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L'N[^; AQ\]^Q_@_`L $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB  $!!n_{1~5EN[^3lB "aZQ+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  Q+a.0q>KN24q>KN24  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V  $!!n_{1~5 AF,V#AQ7%a.;q>9:%a.;q>9:# # "!n_{1~5 AF,V E-- +rc |dvsJ|t|k(sJdg}|j|dz |}t|D]}}d} t|D]Z} |jr'| ||| |j |dz | z |||zz } 6| ||| |j |dz | z ||zz } \|j | t||k(sJd|S)N)r rrrz4the length of lambda list must be equal to the orderr rz3the length of coefficients does not match the order)rrr,rwrrr-) r}rrrrr coefficientslagrange_coefficientr3 coefficientjs rget_coefficients_fnz%SASolverScheduler.get_coefficients_fnKs $$$K((`*``( #CCEAI{[uAK5\??#7#:1#=@j@j A ~|SA$K #7#:1#=@j@j A ~|A$K "    ,< E)`+``)rnoiserrc  t|dkDr|dn|jdd}|t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td | tdd d |j} |j |j dz|j |j } } |j| \} } |j| \} } tj| tj| z }tj| tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p|j|||||}|}|jrM|dk(rG|j |j dz }|j|\}}tj|tj|z }|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z z cc<|dxxdtjd|dzz|zz|dzdz |d|dzzzdz tjd|dzz| zzd|dzzdzz z z||z z zcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z| |dz zz }K|d|dzz | z||z| |dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| | z |z|z|z}|j#|j$}|S)ag One step for the SA-Predictor. Args: model_output (`torch.Tensor`): The direct output from the learned diffusion model at the current timestep. prev_timestep (`int`): The previous discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. order (`int`): The order of SA-Predictor at this timestep. Returns: `torch.Tensor`: The sample tensor at the previous timestep. r prev_timestepNr rrz.missing `noise` as a required keyword argumentr.missing `order` as a required keyword argumentr,missing `tau` as a required keyword argumentrzPassing `prev_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r;rrr+rrvrmrrr/r# zeros_liker,r-rrwrr%r|r*) r}rrrrrrrrmodel_output_listrksigma_s0rjalpha_s0rl lambda_s0 gradient_parthrr3sialpha_sisigma_si lambda_sigradient_coefficientsx temp_sigma temp_alpha_s temp_sigma_s temp_lambda_s noise_partx_ts r!stochastic_adams_bashforth_updatez3SASolverScheduler.stochastic_adams_bashforth_update_s/6$'t9q=QfjjRV6W >4y1}a !RSS =4y1}Q !QRR =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK!+ , KK ( 77@!99(C(99W% '(::IIh'%))H*== ((0 y  uA1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  !% 8 8 8U`be f  ?? "[[1)<= -1-I-I*-U* l % , 7%))L:Q Q %a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02( &a(iiS!Vx 789!tax1CF #3a#7%))QaZUVTVDW:X#X^_beghbh^hmn]n"ooq!=02(uAaZii#q& H 456,A./(!a%1 2 1sAv:!8;PQR;S!SVgjknojohpVq!qq  ?? 5::a%))BaK!O2L.L#MMPUUJwEIIa!e4Dq4H)IIEQJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo rthis_model_outputry last_noise this_samplec . t|dkDr|dn|jdd} |t|dkDr|d}n td|t|dkDr|d}n td|t|dkDr|d}n td |t|d kDr|d }n td |t|d kDr|d }n td | tddd|j} |j |j |j |j dz } } |j| \} } |j| \}} tj| tj| z }tj|tj| z }tj|}||z }g}t|D]n}|j |z }|j|j |\}}tj|tj|z }|j|p| |gz}|j|||||}|}|jr|dk(r|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zz cc<|dxxdtjd|dzz|zz|dz |d|dzzzdz tjd|dzz| zzd|dzzdz|zz z zzcc<t|D]i}|jr<|d|dzz| ztj|dz |zz||z||dz zz }K|d|dzz | z||z||dz zz }k|jr;| tj dtjd|dzz|zz z|z}n7|| ztj tjd|zdz z|z}|jr,tj|dz |z| | z z|z|z|z}n| |z |z|z|z}|j#|j$}|S)a One step for the SA-Corrector. Args: this_model_output (`torch.Tensor`): The model outputs at `x_t`. this_timestep (`int`): The current timestep `t`. last_sample (`torch.Tensor`): The generated sample before the last predictor `x_{t-1}`. this_sample (`torch.Tensor`): The generated sample after the last predictor `x_{t}`. order (`int`): The order of SA-Corrector at this step. Returns: `torch.Tensor`: The corrected sample tensor at the current timestep. r this_timestepNr z4missing `last_sample` as a required keyword argumentrz3missing `last_noise` as a required keyword argumentrz4missing `this_sample` as a required keyword argumentrrrrzPassing `this_timestep` is deprecated and has no effect as model output conversion is now handled via an internal counter `self.step_index`r;rr)r}rryrrrrrrrrrkrrjr rlr r r rr3r rrrmodel_prev_listrrrrs rstochastic_adams_moulton_updatez1SASolverScheduler.stochastic_adams_moulton_updates>$'t9q=QfjjRV6W  4y1}"1g  !WXX  4y1}!!W  !VWW  4y1}"1g  !WXX =4y1}Q !QRR ;4y1}1g !OPP  $ ^  !.. KK ( KK!+ , 77@!99(C(99W% '(::IIh'%))H*== ((5 y  uA1$B!%!=!=dkk"o!N Hh (+eii.AAI   y )  ,/@.AA $ 8 8 8U`be f  ?? &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( &a(iiS!Vx 7891uQaZ 01 4uyy!c1f*RSQSAT7U U[\_bde_e[ejkZknoZoppr( uAaZii#q& H 456,A./&Ah/ 0 1sAv:!8;PQR;S!SVehilmhmfnVo!oo  ?? 5::a%))BaK!O2L.L#MMPZZJwEIIa!e4Dq4H)IIJVJ ??))c1fIM*g.@AAE UXbbCX%*]:ZGCffQWWo rrschedule_timestepsc| |j}||k(j}t|dk(rt|jdz }|St|dkDr|dj}|S|dj}|S)a Find the index for a given timestep in the schedule. Args: timestep (`int` or `torch.Tensor`): The timestep for which to find the index. schedule_timesteps (`torch.Tensor`, *optional*): The timestep schedule to search in. If `None`, uses `self.timesteps`. Returns: `int`: The index of the timestep in the schedule. rr )rsnonzerorr)r}rr"index_candidatesrs rindex_for_timestepz$SASolverScheduler.index_for_timestepds$  %!% .(:CCE  A %T^^,q0J ! "Q &)!,113J*!,113Jrc|jVt|tjr%|j |j j }|j||_y|j|_y)z Initialize the step_index counter for the scheduler. Args: timestep (`int` or `torch.Tensor`): The current timestep for which to initialize the step index. N) r isinstancer/Tensorr|rsrr&rzr{)r}rs r_init_step_indexz"SASolverScheduler._init_step_indexsW    #(ELL1#;;t~~'<'<=#66x@D #00D r return_dictc|j td|j|j||jdkDxr|jdu}|j ||}|rS|j |jd}|j||j|j||j|}tt|jj|jjdz dz D]@} |j | dz|j | <|j| dz|j| <B||j d<||jd<t#|j$||j&|j(} |jj*rt-|jjt/|j0|jz } t-|jjt/|j0|jz dz} n,|jj} |jj} t-| |j2dz|_t-| |j2d z|_ |j4dkDsJ|jdkDsJ||_| |_ |j |jd}|j7||| |j4| } |j2t|jj|jjdz kr|xj2dz c_|xj8dz c_|s| fSt;| S) a Predict the sample from the previous timestep by reversing the SDE. This function propagates the sample with the SA-Solver. Args: model_output (`torch.Tensor`): The direct output from learned diffusion model. timestep (`int`): The current discrete timestep in the diffusion chain. sample (`torch.Tensor`): A current instance of a sample created by the diffusion process. generator (`torch.Generator`, *optional*): A random number generator. return_dict (`bool`): Whether or not to return a [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`. Returns: [`~schedulers.scheduling_utils.SchedulerOutput`] or `tuple`: If return_dict is `True`, [`~schedulers.scheduling_utils.SchedulerOutput`] is returned, otherwise a tuple is returned where the first element is the sample tensor. NzaNumber of inference steps is 'None', you need to run 'set_timesteps' after creating the schedulerrrr\)rryrrrrr ) generatorrr*r)rrrrr) prev_sample)ror+rr*ryrrGrur!rthis_corrector_orderr,rtrbrDrErvr rrr*rLr.rrsrxthis_predictor_orderrrzr)r}rrrr-r+ use_correctormodel_output_convert current_taur3rr0r/r.s rstepzSASolverScheduler.steps,<  # # +s  ?? "  ! !( +!+L0@0@0L #88f8U --(:(:2(>?K99"6 ,,??"// :Fs4;;66 8S8SVW8WX[\\]A$($6$6q1u$=D  q !$($6$6q1u$=D  q !^"62!)2   &&$$   ;; ( (#&t{{'B'BCDWZ^ZiZiDi#j #&t{{'B'BCDWZ^ZiZiDilmDm#n #';;#>#> #';;#>#> $'(S>SVW>W$X!$'(S>SVW>W$X!((1,,,((1,,,!mmD$6$6r$:; <<-++ =   3t{{'B'BDKKD_D_bcDc#d d  ! !Q & ! A> !;77rc|S)a? Ensures interchangeability with schedulers that need to scale the denoising model input depending on the current timestep. Args: sample (`torch.Tensor`): The input sample. Returns: `torch.Tensor`: A scaled input sample. r_)r}rrrs rscale_model_inputz#SASolverScheduler.scale_model_inputs  roriginal_samplesrsc|jj|j|_|jj|j}|j|j}||dz}|j }t |j t |j kr=|jd}t |j t |j kr=d||z dz}|j }t |j t |j kr=|jd}t |j t |j kr=||z||zz}|S)a2 Add noise to the original samples according to the noise magnitude at each timestep (this is the forward diffusion process). Args: original_samples (`torch.Tensor`): The original samples to which noise will be added. noise (`torch.Tensor`): The noise to add to the samples. timesteps (`torch.IntTensor`): The timesteps indicating the noise level for each sample. Returns: `torch.Tensor`: The noisy samples. )rr)r!r\r )rir|rr*flattenrrr)r}r7rrsrisqrt_alpha_prodsqrt_one_minus_alpha_prod noisy_sampless r add_noisezSASolverScheduler.add_noisesa2#1144>$=qAFF$=>L E$("%9=  ($(",1"%/"&"'', % %%*5\M$( *1S, S,S, S,  S, zzDK/$6 S,S,S,S,T/S,S,%*S, S,S, S,  !S,"!%#S,$%S,&'S,()S,*"+S,,Tz-S,./S,01S,S,j  !!(3(O,O,S5<rXsy$ $ A1,XX @H14 1414""<=14 \\ 14hc/ c/r