ib (ddlZddlZddlmZmZmZmZddlZddlmZddl m Z GddejjZ GddejjZGd d ejjZGd d ejjZGd dejjZGddejjZGddejZGddejZdej*dededej*fdZdZd'dej*dedededej*f dZd(d ej*ded!ededej*f d"Zd#eedefd$Zd#eedefd%Zd#eedefd&Zy))N)AnyDictListOptional)nn) functionalc eZdZdZd dedededeffd Zede jfdZ d e jde jfd Z xZ S) _ScaledEmbeddingaFMake continuous embeddings and boost learning rate Args: num_embeddings (int): number of embeddings embedding_dim (int): embedding dimensions scale (float, optional): amount to scale learning rate (Default: 10.0) smooth (bool, optional): choose to apply smoothing (Default: ``False``) num_embeddings embedding_dimscalesmoothct|tj|||_|rt j |jjjd}|t jd|dzjdddfz }||jjjdd|jjxj|zc_||_ y)Nrdim) super__init__r Embedding embeddingtorchcumsumweightdataarangesqrtr )selfr r r rr __class__s o/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/torchaudio/models/_hdemucs.pyrz_ScaledEmbedding.__init__-s nmD \\$.."7"7"<"z%_HEncLayer.__init__..m BKKMr group_normc0tj|Sr#r GroupNormrEr9s rrFz%_HEncLayer.__init__..o [! r?norm_fnpad_valklassrs ` rrz_HEncLayer.__init__\s   H)  $**3*3X,,x /E,QVQ]Q],r r4ceZdZdZ ddededededededed ed ed ed eeee fd effd Z de jdee jfdZ xZS) _HDecLayeraDecoder layer. This used both by the time and the frequency branches. Args: chin (int): number of input channels. chout (int): number of output channels. last (bool, optional): whether current layer is final layer (Default: ``False``) kernel_size (int, optional): Kernel size for encoder (Default: 8) stride (int): Stride for encoder layer (Default: 4) norm_groups (int, optional): number of groups for group norm. (Default: 1) empty (bool, optional): used to make a layer with just the first conv. this is used before merging the time and freq. branches. (Default: ``False``) freq (bool, optional): boolean for whether conv layer is for frequency (Default: ``True``) norm_type (str, optional): Norm type, either ``group_norm `` or ``none`` (Default: ``group_norm``) context (int, optional): context size for the 1x1 conv. (Default: 1) dconv_kw (Dict[str, Any] or None, optional): dictionary of kwargs for the DConv class. (Default: ``None``) pad (bool, optional): true to pad the input. Padding is done so that the output size is always the input size / stride. (Default: ``True``) r5r6lastr7r8r9r:r;r<r=r>r?c t|| i} d} | dk(rfd} | r||z dzdk7r td||z dz} nd} | |_||_||_||_||_||_||_ tj}tj}|r(|dg}|dg}tj}tj}||||||_| ||_|jr3tj"|_tj"|_y||d|zdd| zzd| |_| d|z|_y)Nc*tjSr#rBrDs rrFz%_HDecLayer.__init__..rGr rHc0tj|Sr#rJrLs rrFz%_HDecLayer.__init__..rMr rOrz#Kernel size and stride do not alignr)rrrbr?rsr;r5r:r8r7rrPConvTranspose1drQConvTranspose2dconv_trrUrCrTrS)rr5r6rsr7r8r9r:r;r<r=r>r?rXrZklass_trrs ` rrz_HDecLayer.__init__s@   H)  $*0+0+d..Xell-C.r rrc+eZdZdZ d$deededededededed ed ed ed ed edededededededededef*fd Z dZ d%dZ d&de jdedededef dZd Zd!Zd"e jfd#ZxZS)'HDemucsa# Hybrid Demucs model from *Hybrid Spectrogram and Waveform Source Separation* :cite:`defossez2021hybrid`. See Also: * :class:`torchaudio.pipelines.SourceSeparationBundle`: Source separation pipeline with pre-trained models. Args: sources (List[str]): list of source names. List can contain the following source options: [``"bass"``, ``"drums"``, ``"other"``, ``"mixture"``, ``"vocals"``]. audio_channels (int, optional): input/output audio channels. (Default: 2) channels (int, optional): initial number of hidden channels. (Default: 48) growth (int, optional): increase the number of hidden channels by this factor at each layer. (Default: 2) nfft (int, optional): number of fft bins. Note that changing this requires careful computation of various shape parameters and will not work out of the box for hybrid models. (Default: 4096) depth (int, optional): number of layers in encoder and decoder (Default: 6) freq_emb (float, optional): add frequency embedding after the first frequency layer if > 0, the actual value controls the weight of the embedding. (Default: 0.2) emb_scale (int, optional): equivalent to scaling the embedding learning rate (Default: 10) emb_smooth (bool, optional): initialize the embedding with a smooth one (with respect to frequencies). (Default: ``True``) kernel_size (int, optional): kernel_size for encoder and decoder layers. (Default: 8) time_stride (int, optional): stride for the final time layer, after the merge. (Default: 2) stride (int, optional): stride for encoder and decoder layers. (Default: 4) context (int, optional): context for 1x1 conv in the decoder. (Default: 4) context_enc (int, optional): context for 1x1 conv in the encoder. (Default: 0) norm_starts (int, optional): layer at which group norm starts being used. decoder layers are numbered in reverse order. (Default: 4) norm_groups (int, optional): number of groups for group norm. (Default: 4) dconv_depth (int, optional): depth of residual DConv branch. (Default: 2) dconv_comp (int, optional): compression of DConv branch. (Default: 4) dconv_attn (int, optional): adds attention layers in DConv branch starting at this layer. (Default: 4) dconv_lstm (int, optional): adds a LSTM layer in DConv branch starting at this layer. (Default: 4) dconv_init (float, optional): initial scale for the DConv branch LayerScale. (Default: 1e-4) sourcesaudio_channelschannelsgrowthnfftdepthfreq_emb emb_scale emb_smoothr7 time_strider8r= context_enc norm_startsr9 dconv_depth dconv_comp dconv_attn dconv_lstm dconv_initc t+|||_||_||_||_| |_| |_| |_||_ |jdz|_ d|_ tj|_tj|_tj|_tj|_|}|dz}|}|}|jdz}t%|jD]}||k\}||k\}||k\rdnd}|dkD}| } | }!|s|dk7r t'd| dz}!| } d}"d}#|r || kr|}!d}"d}#|!| ||"|||||||d d }$t)|$}%d |%d <| |%d <| |%d<d|%d<t)|$}&|#rt+||}|}t-||fd|i|$}'|r>|#dur|dk(r d|%d<d|%d <t-||f||#d|%}(|j j/|(|jj/|'|d k(r'|jt1|j z}|dz}t3||f|d k(| d|&})|r0t3||f|#|d k(| d|%}*|j"j5d |*|jj5d |)|}|}t7||z}t7||z}|r || krd}n|| z}|d k(s|st9||| ||_ ||_t=|y)NrNrOrHnonerz$When freq is false, freqs must be 1.TF)lstmattnrcompressinit)r7r8r;r?r<r9r>rr;r7r8r?r=)r=r:)rsr=)r:rsr=)rr )rrrrrrr7r=r8r hop_lengthrr ModuleList freq_encoder freq_decoder time_encoder time_decoderrangerbdictmaxr4appendlenrrinsertr,r freq_emb_scale_rescale_module),rrrrrrrrrrr7rr8r=rrr9rrrrrr5chin_zr6chout_zfreqsindexrrr<r;strikerr? last_freqkwkwtkw_decenctencdectdecrs, rrzHDemucs.__init__QsI0   , &    ))q. MMOMMOMMOMMO Q4::&EJ&DJ&D(-(< &I19DDCA:$%KLL!Ao"CI,  #&*  ( *& Br(CCK!,C "CMCJ"XFeW-VWHkHRHC$$%CM)*C &!$[{)[WZ[!!((.    $ $S )z**S->>WfY5A:wYRXYC!%hYUaZY`hdgh!!((D1    $ $Q ,DF'E&7*+GK'Ef$Ezh 0zYb c &.#W'Z r c|j}|j}|}||dzk7r tdtt j |j d|z }|dzdz}|j|||||zz|j dz d}t|||dddddf}|j d|dzk7r td |ddd|zf}|S) NrNzHop length must be nfft // 4r]rOr^reflect)mode.zESpectrogram's last dimension must be 4 + input size divided by stride) rrrbr,mathceilr__pad1d_spectro)rr$hlrx0rkr?rms r_specz HDemucs._specs __yy  ?;< < 1772;+, -Agk KK3b2g  ;)K L Qb !#ssA+ . 772;"q& de e c1q2v:o r c|j}tj|gd}tj|ddg}|dzdz}|tt j ||z zd|zz}t |||}|d|||zf}|S)N)rrrrrOr^)r}.)rrar?r,rr _ispectro)rrmr}rr?rkr$s r_ispeczHDemucs._ispecs __ EE!\ " EE!aV Agk #dii ,- -C 7 aB ' c3v%% &r r$ padding_left padding_rightrvaluec|jd}|dk(r/t||}||krtj|d||z dzf}tj|||f||S)zWrapper around F.pad, in order for reflect padding when num_frames is shorter than max_pad. Add extra zero padding around in order for padding to not break.r]rrr)r_rrar?)rr$rrrrr}max_pads rrzHDemucs._pad1dsf 9 , 6G EE!a6!1A!567uuQ}5tUCCr c|j\}}}}tj|jddddd}|j ||dz||}|S)NrrrNrOr^)r_r view_as_realrdre)rrmrgrhrirjms r _magnitudezHDemucs._magnitudesSgg 1b!   q ! ) )!Q1a 8 IIaQA &r c|j\}}}}}|j||dd||jdddddd}tj|j }|S)Nr]rOrrrNr^)r_r`rdrview_as_complex contiguous)rrrgSrhrirjr&s r_maskz HDemucs._masks^1aQffQ2q"a(00Aq!QB##CNN$45 r inputc|jdk7rtd|j|jd|jk7rtd|jdd|}|jd}|j |}|j |}|}|j\}}}} |j dd } |jdd } || z d | zz }|} | j d d } | jd d }| | z d |zz } g}g}g}g}t|jD]7\}}|j|jdd }|t|jkrU|j| jd|j|}|| } |js|j| n| }|||}|d k(r|jytj |jd|j"}|j|j%d d d d d d fj'|}||j(|zz}|j|:tj*|}tj*|} t|j,D]\}}|j/d}||||j/d\}}|j0t|j2z }||k\s[|j2||z }|j/d}|jrD|jddk7rtd|j|d d d d d f}||d |\} }|j/d}|| ||\} }t|d k7r t5dt|d k7r t5dt|d k7r t5dt|j6} |j9|| d|| }|| d d d fz| d d d fz}|j;|}!|j=|!|}| j9|| d|} | |d d d fz| d d d fz} | |z}|S)aHDemucs forward call Args: input (torch.Tensor): input mixed tensor of shape `(batch_size, channel, num_frames)` Returns: Tensor output tensor split into sources of shape `(batch_size, num_sources, channel, num_frames)` r^zDExpected 3D tensor with dimensions (batch, channel, frames). Found: rzZThe channel dimension of input Tensor must match `audio_channels` of HDemucs model. Found:.r])rrOr^T)rkeepdimgh㈵>)rrONr)devicerOz0If tdec empty is True, pre shape does not match zsaved is not emptyzlengths_t is not emptyzsaved_t is not empty)ndimrbr_rrrmeanstd enumeraterrrrr:rrrrt expand_asr zeros_likerpoprrAssertionErrorrr`rr)"rrr$r}rmmagrgrhFqrjrrxtmeantstdtsavedsaved_tlengths lengths_tidxencoder[rfrsembdecoder{preoffsetrlength_t_rzouts" rr'zHDemucs.forward sR ::?cdidodocpqr r ;;q>T00 0Q(+   JJu ooa  gg 1b!vv)Tv2ee 4e0 X$* %FD1vv&$v/5jTD[ )! $T%6%67KC NN1772; 'FS**++  ".((-"XzzNN2& Fq&!AaxDMM5ll1772;qxx@mmC(**,T1a-=>HHK++c11 LLO/82   Q    a %T%6%67KC99R=DAtW[[_5FAsZZ#d&7&7"88Ff}((v6$==,::yy|q((+[\_\e\e[f)ghhaAg,C dH5EB";;r?D T84EB!8$ u:? !56 6 y>Q  !9: : w<1  !78 8   FF1aR # AtG tAtG} ,zz!} KKf % WWQ2v & $q$w- %4. 0 Fr )rO0rOg? TrnrOrNrrrNrNrOrNrNrN-C6?r#)zerog)r(r)r*r+rrpr,r-r.rrrrr0rrrr'r1r2s@rrr-s!L  -~c~~ ~  ~  ~~~~~~~~~~ !~"#~$%~&'~()~*+~,-~@0D DCDDSVDhmDoU\\or rcfeZdZdZ ddededededededed ed ed effd Zd Z xZ S)rWa New residual branches in each encoder layer. This alternates dilated convolutions, potentially with LSTMs and attention. Also before entering each residual branch, dimension is projected on a smaller subspace, e.g. of dim `channels // compress`. Args: channels (int): input/output channels for residual branch. compress (float, optional): amount of channel compression inside the branch. (default: 4) depth (int, optional): number of layers in the residual branch. Each layer has its own projection, and potentially LSTM and attention.(default: 2) init (float, optional): initial scale for LayerNorm. (default: 1e-4) norm_type (bool, optional): Norm type, either ``group_norm `` or ``none`` (Default: ``group_norm``) attn (bool, optional): use LocalAttention. (Default: ``False``) heads (int, optional): number of heads for the LocalAttention. (default: 4) ndecay (int, optional): number of decay controls in the LocalAttention. (default: 4) lstm (bool, optional): use LSTM. (Default: ``False``) kernel_size (int, optional): kernel size for the (dilated) convolutions. (default: 3) rrrrr<rheadsndecayrr7c &t|| dzdk(r td||_||_t ||_|dkD} d} |dk(rd} t||z } tj}tjg|_ t|j D]}| r td|nd}|| dzz}tj|| | ||| | |tj| d|zd| d|ztjdt!||g}|r|j#d t%| || | r|j#d t'| dd tj(|}|jj+|y) NrOrz(Kernel size should not be divisible by 2c*tjSr#rBrDs rrFz!_DConv.__init__..rGr rHc.tjd|S)NrrJrDs rrFz!_DConv.__init__..s Q 2r r)dilationpaddingr^)rrT)layersr{)rrrbrrabsrr,rGELUrrrpowrPGLU _LayerScaler _LocalState_BLSTM Sequentialr)rrrrrr<rrrrr7dilaterXhiddenactrErrmodslayerrs rrz_DConv.__init__sj  ?a GH H    Z *  $2GX()ggmmB' tzz"A$*s1ayH+"23G (FK(T[\ &!h,2H %q Hd+D A{6vNO AvfQTBCMM4(E KK  u %##r c>|jD] }|||z}|S)zDConv forward call Args: x (torch.Tensor): input tensor for convolution Returns: Tensor Output after being run through layers. )r)rr$r s rr'z_DConv.forwards$[[EE!H A!r ) rNrOrrHFrNrNFr^) r(r)r*r+r,r-rpr.rr'r1r2s@rrWrW}s.%1&1&1& 1&  1&  1&1&1&1&1&1&f r rWcfeZdZdZddedeffd ZdejdejfdZ xZ S) rae BiLSTM with same hidden units as input dim. If `max_steps` is not None, input will be splitting in overlapping chunks and the LSTM applied separately on each chunk. Args: dim (int): dimensions at LSTM layer. layers (int, optional): number of LSTM layers. (default: 1) skip (bool, optional): (default: ``False``) rr{ct|d|_tjd||||_tj d|z||_||_y)NT) bidirectional num_layers hidden_size input_sizerO) rr max_stepsrLSTMrLinearlinearr{)rrrr{rs rrz_BLSTM.__init__sI GG$6s_bc iiC-  r r$r!c B|j\}}}|}d}d}d}d} |jc||jkDrT|j}|dz}t|||} | jd} d}| jddddj d||}|jddd}|j |d}|j |}|jddd}|rg} |j |d||} |dz} t| D]m} | dk(r | j| dd| ddd| f(| | dz k(r| j| dd| dd| dfO| j| dd| dd| | fotj| d} | d d|f} | }|jr||z}|S) aBLSTM forward call Args: x (torch.Tensor): input tensor for BLSTM shape is `(batch_size, dim, time_steps)` Returns: Tensor Output after being run through bidirectional LSTM. Shape is `(batch_size, dim, time_steps)` FrNrOTrr^r].) r_r_unfoldrdrerrrrrcatr{)rr$rgrhrjrlframedwidthr8nframesframesr&limitks rr'z_BLSTM.forwards''1a  >> %!dnn*<NNEaZFQv.Fll1oGFq!Q*222q%@A IIaA  IIaLO KKN IIaA  CYYq"a/FaKE7^6JJvaAww&678'A+%JJvaAuvo67JJvaAueV|&;<= $))C$Cc2A2g,CA 99AAr )rF) r(r)r*r+r,r.rrr0r'r1r2s@rrrs6C4..%,,.r rcjeZdZdZd dededeffd ZdejdejfdZxZ S) raLocal state allows to have attention based only on data (no positional embedding), but while setting a constraint on the time window (e.g. decaying penalty term). Also a failed experiments with trying to provide some frequency based attention. rrrcttt| ||zdk7r td||_||_t j||d|_t j||d|_ t j||d|_ t j|||zd|_ |rm|jjxjdzc_ |jj tdd|jjjddt j||dzz|d|_y)z Args: channels (int): Size of Conv1d layers. heads (int, optional): (default: 4) ndecay (int, optional): (default: 4) rz$Channels must be divisible by heads.rg{Gz?Nzbias must not be None.r)rrrrbrrrrPcontentquerykey query_decayrrbiasproj)rrrrrs rrz_LocalState.__init__s k4)+ e q CD D  yy8Q7 YYx15 99Xx399Xuv~qA     # # ( (D 0 ($$, !9::,.D   ! ! & &q )IIh2Ha@ r r$r!c|j\}}}|j}tj||j|j }|dddf|dddfz }|j |j||d|}|j|j||d|} tjd| |} | tj| jdz} |jrtjd|jdz|j|j } |j|j||d|} tj| dz } | jddd |jztj|jz } | tjd| | z } | j!tj"|| jtj$dtj&| d }|j)|j||d|}tjd ||}|j+|d|}||j-|zS) zLocalState forward call Args: x (torch.Tensor): input tensor for LocalState Returns: Tensor Output after being run through LocalState layer. )rdtypeNr]zbhct,bhcs->bhtsrOrzfts,bhfs->bhtsirzbhts,bhct->bhcs)r_rrrrr,r&r`r'einsumrrrr(sigmoidr masked_fill_eyer.softmaxr%rer*)rr$rgrhrjrindexesdeltaquerieskeysdotsdecaysdecay_q decay_kernelweightsr%results rr'z_LocalState.forward6s''1a ,,qA4 747#33**Q-$$Qr15xx{5"a0||-tW=  $**Q-(( ;;\\!T[[1_QXXQWWUF&&q)..q%Q?GmmG,q0G"KKAq11EIIK?$))DKKBXXL ELL!1<I ID %))AdkkLdS--!,,,q/&&q%Q7/'B2q)499V$$$r )rNrN) r(r)r*r+r,rrr0r'r1r2s@rrrsA AASAcA2#%#%%,,#%r rcfeZdZdZddedeffd ZdejdejfdZ xZ S) rzLayer scale from [Touvron et al 2021] (https://arxiv.org/pdf/2103.17239.pdf). This rescales diagonally residual outputs close to 0 initially, then learnt. rrct|tjt j |d|_||j jddy)z Args: channels (int): Size of rescaling init (float, optional): Scale to default to (default: 0) T) requires_gradN)rrr Parameterrzerosr r)rrrrs rrz_LayerScale.__init__as= \\%++hd"KL ! r r$r!c.|jdddf|zS)zLayerScale forward call Args: x (torch.Tensor): input tensor for LayerScale Returns: Tensor Output after rescaling tensor. N)r )rr$s rr'z_LayerScale.forwardkszz!T'"Q&&r )r) r(r)r*r+r,r-rrr0r'r1r2s@rrr\s6""E" ' '%,, 'r rar7r8r!ct|jdd}t|jd}tj||z }|dz |z|z}t j |d||z g}t|jDcgc]}|j|}}|ddk7r td|dd|dgz}|j||j||j||Scc}w)zGiven input of size [*OT, T], output Tensor of size [*OT, F, K] with K the kernel size, by extracting frames with the given stride. This will pad the input so that `F = ceil(T / K)`. see https://github.com/pytorch/pytorch/issues/60466 Nr]rr)rr?zData should be contiguous.) listr_r,rrrar?rrr8rbr as_strided) rBr7r8r_r}n_frames tgt_lengthrstridess rrrxs " E  Fyy&)HQ,&(;6J AAzF234A(-aeeg7qxx}G7r{a566crlfa[(G LL LL <<w '' 8s C6c|jD]}t|tjtjtj tj fsL|jjj}|dz dz}|jxj|zc_ |j|jxj|zc_ y)zI Rescales initial weight scale for all models within the module. g?g?N) modules isinstancerrPrwrQrxrrdetachrr))modulesubrr s rrrs~~ cBIIr'9'9299bFXFXY Z**.."))+C3Y3&E JJOOu $Oxx# &  r r$n_fftrr?c zt|jdd}t|jd}|jd|}t j ||d|zz|t j |j||dddd }|j\}}} |j|| g|j|S)Nr]rTr)window win_length normalizedcenterreturn_complexpad_mode) rDr_r,rerstft hann_windowtoextendr`) r$rOrr?otherr}rmrrframes rrrs " E  F "fA   S  '**1- AggOAue LL% 66%=r rmr}c t|jdd}t|jd}t|jd}d|zdz }|jd||}|d|zz}t j |||t j |j|j|d|d} | j\} }|j|| j|S)Nrr]rOrT)rQrRrSr}rT) rDr_r,r`ristftrXrYrealr) rmrr}r?r[rr rOrRr$rs rrrs " E  E  F IME r5&!A1s7#J     ,//7 AIAv LL 66%=r rct|ddS)zBuilds low nfft (1024) version of :class:`HDemucs`, suitable for sample rates around 8 kHz. Args: sources (List[str]): See :py:func:`HDemucs`. Returns: HDemucs: HDemucs model. irrrrrrs r hdemucs_lowrd 7Q 77r ct|ddS)aBuilds medium nfft (2048) version of :class:`HDemucs`, suitable for sample rates of 16-32 kHz. .. note:: Medium HDemucs has not been tested against the original Hybrid Demucs as this nfft and depth configuration is not compatible with the original implementation in https://github.com/facebookresearch/demucs Args: sources (List[str]): See :py:func:`HDemucs`. Returns: HDemucs: HDemucs model. rrrarbrcs rhdemucs_mediumrgs 7Q 77r ct|ddS)zBuilds medium nfft (4096) version of :class:`HDemucs`, suitable for sample rates of 44.1-48 kHz. Args: sources (List[str]): See :py:func:`HDemucs`. Returns: HDemucs: HDemucs model. rrrarbrcs r hdemucs_highrirer )irr)rrr)rtypingtprrrrrrtorch.nnrraModuler r4rrrrWrrrr0r,rrrrrprdrgrir rros4 ,, $#uxx#Lkk\sslMehhooM` TUXX__Tn@UXX__@FB%"))B%J'"))'8(u||(#(s(u||(( ' SC#V[VbVb(3C#V[VbVb. 8c 8w 88DI8'8& 8$s) 8 8r