iZDddlZddlmZmZmZmZddlZddlmZmZddl m Z dgZ d"de de ded ed ejj f d Z d#d e d e de de deeee ee fde ded ed ejj$fdZded efdZGddej*ZGddej*ZGddej*ZGddej*ZGddej*ZGdd ej*ZGd!dej*Zy)$N)ListOptionalTupleUnion)nnTensor) functional Tacotron2in_dimout_dimbias w_init_gainreturnctjj|||}tjjj |j tjjj ||S)aLinear layer with xavier uniform initialization. Args: in_dim (int): Size of each input sample. out_dim (int): Size of each output sample. bias (bool, optional): If set to ``False``, the layer will not learn an additive bias. (Default: ``True``) w_init_gain (str, optional): Parameter passed to ``torch.nn.init.calculate_gain`` for setting the gain parameter of ``xavier_uniform_``. (Default: ``linear``) Returns: (torch.nn.Linear): The corresponding linear layer. r gain)torchrLinearinitxavier_uniform_weightcalculate_gain)r r r rlinears p/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/torchaudio/models/tacotron2.py_get_linear_layerr)sTXX__VW4_ 8F HHMM!!&--ehhmm6R6RS^6_!` M in_channels out_channels kernel_sizestridepaddingdilationc \|'|dzdk7r tdt||dz zdz }tjj |||||||}tjj j |jtjj j||S)al1D convolution with xavier uniform initialization. Args: in_channels (int): Number of channels in the input image. out_channels (int): Number of channels produced by the convolution. kernel_size (int, optional): Number of channels in the input image. (Default: ``1``) stride (int, optional): Number of channels in the input image. (Default: ``1``) padding (str, int or tuple, optional): Padding added to both sides of the input. (Default: dilation * (kernel_size - 1) / 2) dilation (int, optional): Number of channels in the input image. (Default: ``1``) w_init_gain (str, optional): Parameter passed to ``torch.nn.init.calculate_gain`` for setting the gain parameter of ``xavier_uniform_``. (Default: ``linear``) Returns: (torch.nn.Conv1d): The corresponding Conv1D layer. zkernel_size must be odd)r r!r"r#r r) ValueErrorintrrConv1drrrr) rrr r!r"r#r rconv1ds r_get_conv1d_layerr+;s4 ?a 67 7h+/2Q67 XX__ F HHMM!!&--ehhmm6R6RS^6_!` Mrlengthsctj|j}tjd||j|j }||j dkj}tj|d}|S)alReturns a binary mask based on ``lengths``. The ``i``-th row and ``j``-th column of the mask is ``1`` if ``j`` is smaller than ``i``-th element of ``lengths. Args: lengths (Tensor): The length of each element in the batch, with shape (n_batch, ). Returns: mask (Tensor): The binary mask, with shape (n_batch, max of ``lengths``). r)devicedtyper&) rmaxitemaranger.r/ unsqueezebytele)r,max_lenidsmasks r_get_mask_from_lengthsr9isjii %%'G ,,q''.. NC '##A& & , , .D 88D! D Krc@eZdZdZdededeffd ZdedefdZxZS) _LocationLayeraLocation layer used in the Attention model. Args: attention_n_filter (int): Number of filters for attention model. attention_kernel_size (int): Kernel size for attention model. attention_hidden_dim (int): Dimension of attention hidden representation. attention_n_filterattention_kernel_sizeattention_hidden_dimc t|t|dz dz }td|||ddd|_t ||dd|_y)Nr&r%F)r r"r r!r#tanhr r)super__init__r(r+ location_convrlocation_dense)selfr<r=r>r" __class__s rrCz_LocationLayer.__init__s` ,q0A56. - 0  45f rattention_weights_catrcn|j|}|jdd}|j|}|S)aLocation layer used in the Attention model. Args: attention_weights_cat (Tensor): Cumulative and previous attention weights with shape (n_batch, 2, max of ``text_lengths``). Returns: processed_attention (Tensor): Cumulative and previous attention weights with shape (n_batch, ``attention_hidden_dim``). r&r%)rD transposerE)rFrHprocessed_attentions rforwardz_LocationLayer.forwardsA#001FG1;;AqA"112EF""r __name__ __module__ __qualname____doc__r(rCrrL __classcell__rGs@rr;r;zs<   # "  *#V##rr;c~eZdZdZdedededededdf fd Zd ed ed edefd Zdeded ed ededeeeff dZ xZ S) _AttentionaLocally sensitive attention model. Args: attention_rnn_dim (int): Number of hidden units for RNN. encoder_embedding_dim (int): Number of embedding dimensions in the Encoder. attention_hidden_dim (int): Dimension of attention hidden representation. attention_location_n_filter (int): Number of filters for Attention model. attention_location_kernel_size (int): Kernel size for Attention model. attention_rnn_dimencoder_embedding_dimr>attention_location_n_filterattention_location_kernel_sizerNct|t||dd|_t||dd|_t|dd|_t ||||_td |_ y)NFr@rAr&rinf) rBrCr query_layer memory_layervr;location_layerfloatscore_mask_value)rFrVrWr>rXrYrGs rrCz_Attention.__init__sx ,->@T[`ntu- !#7eQW ##7G, ' *   "'u rqueryprocessed_memoryrHc|j|jd}|j|}|jt j ||z|z}|j d}|S)a=Get the alignment vector. Args: query (Tensor): Decoder output with shape (n_batch, n_mels * n_frames_per_step). processed_memory (Tensor): Processed Encoder outputs with shape (n_batch, max of ``text_lengths``, attention_hidden_dim). attention_weights_cat (Tensor): Cumulative and previous attention weights with shape (n_batch, 2, max of ``text_lengths``). Returns: alignment (Tensor): attention weights, it is a tensor with shape (batch, max of ``text_lengths``). r&r%)r\r3r_r^rr@squeeze)rFrbrcrHprocessed_queryprocessed_attention_weightsenergies alignments r_get_alignment_energiesz"_Attention._get_alignment_energiessh**5??1+=>&*&9&9:O&P#66%**_7R%RUe%efg$$Q' rattention_hidden_statememoryr8c|j|||}|j||j}tj|d}t j |jd|}|jd}||fS)aPass the input through the Attention model. Args: attention_hidden_state (Tensor): Attention rnn last output with shape (n_batch, ``attention_rnn_dim``). memory (Tensor): Encoder outputs with shape (n_batch, max of ``text_lengths``, ``encoder_embedding_dim``). processed_memory (Tensor): Processed Encoder outputs with shape (n_batch, max of ``text_lengths``, ``attention_hidden_dim``). attention_weights_cat (Tensor): Previous and cumulative attention weights with shape (n_batch, current_num_frames * 2, max of ``text_lengths``). mask (Tensor): Binary mask for padded data with shape (n_batch, current_num_frames). Returns: attention_context (Tensor): Context vector with shape (n_batch, ``encoder_embedding_dim``). attention_weights (Tensor): Attention weights with shape (n_batch, max of ``text_lengths``). r&dim) rj masked_fillraFsoftmaxrbmmr3re) rFrkrlrcrHr8riattention_weightsattention_contexts rrLz_Attention.forwards~.001GIY[pq ))$0E0EF IIiQ7!II&7&A&A!&DfM-55a8 "333r) rNrOrPrQr(rCrrjrrLrRrSs@rrUrUs.. #." . &) . ), . .*Vvflqw*4 &44! 4 & 4  4 vv~ 4rrUcFeZdZdZdedeeddffd ZdedefdZxZ S) _PrenetzPrenet Module. It is consists of ``len(output_size)`` linear layers. Args: in_dim (int): The size of each input sample. output_sizes (list): The output dimension of each linear layers. r out_sizesrNc t||g|ddz}tjt ||Dcgc]\}}t ||dc}}|_ycc}}w)NFr)rBrCr ModuleListziprlayers)rFr rxin_sizesin_sizeout_sizerGs rrCz_Prenet.__init__ s_ 8in,mmY\]egpYq rYqBU7H wu =Yq r  rsA xc|jD]3}tjtj||dd}5|S)zPass the input through Prenet. Args: x (Tensor): The input sequence to Prenet with shape (n_batch, in_dim). Return: x (Tensor): Tensor with shape (n_batch, sizes[-1]) ?T)ptraining)r}rqdropoutrelu)rFrrs rrLz_Prenet.forwards6kkF !&&+sTBA"r) rNrOrPrQr(rrCrrLrRrSs@rrwrws9 s tCy T   F rrwcDeZdZdZdedededeffd Zdedefd ZxZS) _PostnetaPostnet Module. Args: n_mels (int): Number of mel bins. postnet_embedding_dim (int): Postnet embedding dimension. postnet_kernel_size (int): Postnet kernel size. postnet_n_convolution (int): Number of postnet convolutions. n_melspostnet_embedding_dimpostnet_kernel_sizepostnet_n_convolutionct |tj|_t |D]}|dk(r|n|}||dz k(r|n|}||dz k(rdnd}||dz k(r|n|} |jj tjt|||dt|dz dz d|tj| t|j|_ y)Nrr&rr@r%r r!r"r#r) rBrCrr{ convolutionsrangeappend Sequentialr+r( BatchNorm1dlenn_convs) rFrrrrirr init_gain num_featuresrGs rrCz_Postnet.__init__*s MMO,-A$%F&0EK%&+@1+D%E6K`L$%*?!*C$D&I%&+@1+D%E6K`L    $ $ %#$$7 #%81%<$A B!"$-NN<0   .(4,,- rrrc,t|jD]{\}}||jdz kr>> encoder = _Encoder(3, 512, 5) >>> input = torch.rand(10, 20, 30) >>> output = encoder(input) # shape: (10, 30, 512) rWencoder_n_convolutionencoder_kernel_sizerNc t|tj|_t |D]e}tj t|||dt|dz dz ddtj|}|jj|gtj|t|dz ddd|_ |jjy)Nr&r%rrT) batch_first bidirectional)rBrCrr{rrrr+r(rrLSTMlstmflatten_parameters)rFrWrr_ conv_layerrGs rrCz_Encoder.__init__ks MMO,-A!)) 3!4q!8A => &45 J    $ $Z 0.GG ! %) *     $$&rr input_lengthsc|jD]<}tjtj||d|j}>|j dd}|j }tjjj||d}|j|\}}tjjj|d\}}|S)a_Pass the input through the Encoder. Args: x (Tensor): The input sequences with shape (n_batch, encoder_embedding_dim, n_seq). input_lengths (Tensor): The length of each input sequence with shape (n_batch, ). Return: x (Tensor): A tensor with shape (n_batch, n_seq, encoder_embedding_dim). rr&r%T)r) rrqrrrrJcpurutilsrnnpack_padded_sequencerpad_packed_sequence)rFrrroutputsrs rrLz_Encoder.forwards%%D !&&a/3 >A& KK1 %))+ HHLL - -aD - QYYq\ XX\\55g45P rrMrSs@rrr]sN '"' #'! '  'B6rrc!eZdZdZdededededededed ed ed ed ed edededdffd ZdedefdZ dede eeeeeeeeffdZ dedefdZ dededede eeeffdZ dedededed ed!ed"ed#eded$ed%ede eeeeeeeeef fd&Zded'ed(ede eeeffd)Zdedefd*Zej$j&ded(ede eeeeffd+ZxZS),_Decodera,Decoder with Attention model. Args: n_mels (int): number of mel bins n_frames_per_step (int): number of frames processed per step, only 1 is supported encoder_embedding_dim (int): the number of embedding dimensions in the encoder. decoder_rnn_dim (int): number of units in decoder LSTM decoder_max_step (int): maximum number of output mel spectrograms decoder_dropout (float): dropout probability for decoder LSTM decoder_early_stopping (bool): stop decoding when all samples are finished attention_rnn_dim (int): number of units in attention LSTM attention_hidden_dim (int): dimension of attention hidden representation attention_location_n_filter (int): number of filters for attention model attention_location_kernel_size (int): kernel size for attention model attention_dropout (float): dropout probability for attention LSTM prenet_dim (int): number of ReLU units in prenet layers gate_threshold (float): probability threshold for stop token rn_frames_per_steprWdecoder_rnn_dimdecoder_max_stepdecoder_dropoutdecoder_early_stoppingrVr>rXrYattention_dropout prenet_dimgate_thresholdrNct|||_||_||_||_||_| |_||_||_ | |_ ||_ ||_ t||z| | g|_tj | |z||_t%||| | | |_tj ||z|d|_t+||z||z|_t+||zddd|_y)NTr&sigmoidrA)rBrCrrrWrVrrrrrrrrwprenetrLSTMCell attention_rnnrUattention_layer decoder_rnnrlinear_projection gate_layer)rFrrrWrrrrrVr>rXrYrrrrGs rrCz_Decoder.__init__s $  !2%:"!2.$ 0,!2.&<#f'88:z:RS [[6K)KM^_)  ! ' *  ;;'8;P'PRacgh!2?EZ3Z\bev\v!w+ 3 3QTy rrlc|jd}|j}|j}tj||j |j z||}|S)amGets all zeros frames to use as the first decoder input. Args: memory (Tensor): Encoder outputs with shape (n_batch, max of ``text_lengths``, ``encoder_embedding_dim``). Returns: decoder_input (Tensor): all zeros frames with shape (n_batch, max of ``text_lengths``, ``n_mels * n_frames_per_step``). rr/r.sizer/r.rzerosrrrFrln_batchr/r. decoder_inputs r_get_initial_framez_Decoder._get_initial_frameN++a.  GT[[4;Q;Q-QY^gmn rc|jd}|jd}|j}|j}tj||j ||}tj||j ||}tj||j ||}tj||j ||} tj||||} tj||||} tj||j||} |jj|} |||| | | | | fS)aInitializes attention rnn states, decoder rnn states, attention weights, attention cumulative weights, attention context, stores memory and stores processed memory. Args: memory (Tensor): Encoder outputs with shape (n_batch, max of ``text_lengths``, ``encoder_embedding_dim``). Returns: attention_hidden (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). attention_cell (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). decoder_hidden (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). decoder_cell (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). attention_weights (Tensor): Attention weights with shape (n_batch, max of ``text_lengths``). attention_weights_cum (Tensor): Cumulated attention weights with shape (n_batch, max of ``text_lengths``). attention_context (Tensor): Context vector with shape (n_batch, ``encoder_embedding_dim``). processed_memory (Tensor): Processed encoder outputs with shape (n_batch, max of ``text_lengths``, ``attention_hidden_dim``). rr&r) rr/r.rrrVrrWrr])rFrlrmax_timer/r.attention_hiddenattention_celldecoder_hidden decoder_cellrtattention_weights_cumrurcs r_initialize_decoder_statesz#_Decoder._initialize_decoder_statess *++a.;;q>  ;;w0F0Fe\bcWd.D.DEZ`aWd.B.B%X^_{{7D,@,@V\] !KKvV % GXUSY Z!KK1K1KSXagh//<>@ #--a3>>@ ##A&DKK8(|((%0 #--a3 \:55rrrrrrrtrrurcr8c tj||fd} |j| ||f\}}tj||j |j }tj|jd|jdfd} |j|| | | | \}}||z }tj||fd}|j|||f\}}tj||j|j }tj||fd}|j|}|j|}|||||||||f S)a& Decoder step using stored states, attention and memory Args: decoder_input (Tensor): Output of the Prenet with shape (n_batch, ``prenet_dim``). attention_hidden (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). attention_cell (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). decoder_hidden (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). decoder_cell (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). attention_weights (Tensor): Attention weights with shape (n_batch, max of ``text_lengths``). attention_weights_cum (Tensor): Cumulated attention weights with shape (n_batch, max of ``text_lengths``). attention_context (Tensor): Context vector with shape (n_batch, ``encoder_embedding_dim``). memory (Tensor): Encoder output with shape (n_batch, max of ``text_lengths``, ``encoder_embedding_dim``). processed_memory (Tensor): Processed Encoder outputs with shape (n_batch, max of ``text_lengths``, ``attention_hidden_dim``). mask (Tensor): Binary mask for padded data with shape (n_batch, current_num_frames). Returns: decoder_output: Predicted mel spectrogram for the current frame with shape (n_batch, ``n_mels``). gate_prediction (Tensor): Prediction of the stop token with shape (n_batch, ``1``). attention_hidden (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). attention_cell (Tensor): Hidden state of the attention LSTM with shape (n_batch, ``attention_rnn_dim``). decoder_hidden (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). decoder_cell (Tensor): Hidden state of the decoder LSTM with shape (n_batch, ``decoder_rnn_dim``). attention_weights (Tensor): Attention weights with shape (n_batch, max of ``text_lengths``). attention_weights_cum (Tensor): Cumulated attention weights with shape (n_batch, max of ``text_lengths``). attention_context (Tensor): Context vector with shape (n_batch, ``encoder_embedding_dim``). rzr&rn) rcatrrqrrrr3rrrrr)rFrrrrrrtrrurlrcr8 cell_inputrH decoder_hidden_attention_contextdecoder_outputgate_predictions rdecodez_Decoder.decodecsiRYY /@A2F +/+=+=jK[]kJl+m(.99%5t7M7Mt}}] % +<+F+Fq+IK`KjKjklKm*ntu v/3/C/C f&68Mt0 ,, !22 #35F"GL '+'7'7 XdGe'f$ >43G3GW+099nFW5X^_+`(//0PQ//*JK        !   rmel_specgram_truthmemory_lengthsc |j|jd}|j|}tj||fd}|j |}t |}|j|\}}} } } } } }ggg}}}t||jddz kr|t|}|j|||| | | | | ||| \ }}}}} } } } } ||jdgz }||jdgz }|| gz }t||jddz kr|jtj|tj|tj|\}}}|||fS)aDecoder forward pass for training. Args: memory (Tensor): Encoder outputs with shape (n_batch, max of ``text_lengths``, ``encoder_embedding_dim``). mel_specgram_truth (Tensor): Decoder ground-truth mel-specs for teacher forcing with shape (n_batch, ``n_mels``, max of ``mel_specgram_lengths``). memory_lengths (Tensor): Encoder output lengths for attention masking (the same as ``text_lengths``) with shape (n_batch, ). Returns: mel_specgram (Tensor): Predicted mel spectrogram with shape (n_batch, ``n_mels``, max of ``mel_specgram_lengths``). gate_outputs (Tensor): Predicted stop token for each timestep with shape (n_batch, max of ``mel_specgram_lengths``). alignments (Tensor): Sequence of attention weights from the decoder with shape (n_batch, max of ``mel_specgram_lengths``, max of ``text_lengths``). rrnr&)rr3rrrrr9rrrrrerstack)rFrlrrrrr8rrrrrtrrurc mel_outputsrr mel_output gate_outputrs rrLz_Decoder.forwards,//7AA!D 334FGM>#BJ^4%n5  + +F 3      !  13B:\ +!4!4Q!7!!;;*3{+;`_. See Also: * :class:`torchaudio.pipelines.Tacotron2TTSBundle`: TTS pipeline with pretrained model. Args: mask_padding (bool, optional): Use mask padding (Default: ``False``). n_mels (int, optional): Number of mel bins (Default: ``80``). n_symbol (int, optional): Number of symbols for the input text (Default: ``148``). n_frames_per_step (int, optional): Number of frames processed per step, only 1 is supported (Default: ``1``). symbol_embedding_dim (int, optional): Input embedding dimension (Default: ``512``). encoder_n_convolution (int, optional): Number of encoder convolutions (Default: ``3``). encoder_kernel_size (int, optional): Encoder kernel size (Default: ``5``). encoder_embedding_dim (int, optional): Encoder embedding dimension (Default: ``512``). decoder_rnn_dim (int, optional): Number of units in decoder LSTM (Default: ``1024``). decoder_max_step (int, optional): Maximum number of output mel spectrograms (Default: ``2000``). decoder_dropout (float, optional): Dropout probability for decoder LSTM (Default: ``0.1``). decoder_early_stopping (bool, optional): Continue decoding after all samples are finished (Default: ``True``). attention_rnn_dim (int, optional): Number of units in attention LSTM (Default: ``1024``). attention_hidden_dim (int, optional): Dimension of attention hidden representation (Default: ``128``). attention_location_n_filter (int, optional): Number of filters for attention model (Default: ``32``). attention_location_kernel_size (int, optional): Kernel size for attention model (Default: ``31``). attention_dropout (float, optional): Dropout probability for attention LSTM (Default: ``0.1``). prenet_dim (int, optional): Number of ReLU units in prenet layers (Default: ``256``). postnet_n_convolution (int, optional): Number of postnet convolutions (Default: ``5``). postnet_kernel_size (int, optional): Postnet kernel size (Default: ``5``). postnet_embedding_dim (int, optional): Postnet embedding dimension (Default: ``512``). gate_threshold (float, optional): Probability threshold for stop token (Default: ``0.5``). mask_paddingrn_symbolrsymbol_embedding_dimrWrrrrrrrVr>rXrYrrrrrrrNct|||_||_||_t j |||_tj jj|jjt||||_ t|||| | | | | |||||||_t!|||||_yN)rBrCrrrr Embedding embeddingrrrrrencoderrdecoderrpostnet)rFrrrrrrWrrrrrrrVr>rXrYrrrrrrrGs rrCzTacotron2.__init__s2 ( !2h0DE  %%dnn&;&;< 57LNab    !    "  ' *      (=?RTij rtokens token_lengthsrrc|j|jdd}|j||}|j|||\}}}|j |} || z} |j rt |} | j|j| jd| jd} | jddd} |j| d| j| d|j| dddddfd|| ||fS)aPass the input through the Tacotron2 model. This is in teacher forcing mode, which is generally used for training. The input ``tokens`` should be padded with zeros to length max of ``token_lengths``. The input ``mel_specgram`` should be padded with zeros to length max of ``mel_specgram_lengths``. Args: tokens (Tensor): The input tokens to Tacotron2 with shape `(n_batch, max of token_lengths)`. token_lengths (Tensor): The valid length of each sample in ``tokens`` with shape `(n_batch, )`. mel_specgram (Tensor): The target mel spectrogram with shape `(n_batch, n_mels, max of mel_specgram_lengths)`. mel_specgram_lengths (Tensor): The length of each mel spectrogram with shape `(n_batch, )`. Returns: [Tensor, Tensor, Tensor, Tensor]: Tensor Mel spectrogram before Postnet with shape `(n_batch, n_mels, max of mel_specgram_lengths)`. Tensor Mel spectrogram after Postnet with shape `(n_batch, n_mels, max of mel_specgram_lengths)`. Tensor The output for stop token at each time step with shape `(n_batch, max of mel_specgram_lengths)`. Tensor Sequence of attention weights from the decoder with shape `(n_batch, max of mel_specgram_lengths, max of token_lengths)`. r&r%)rrgNg@@) rrJrrrrr9expandrrpermute masked_fill_) rFr r rrembedded_inputsencoder_outputsrrmel_specgram_postnetr8s rrLzTacotron2.forwardsB..0::1a@,, F15 \-2>2 . lJ $||L9+.BB   )*>?D;;t{{DIIaL$))A,GD<<1a(D  % %dC 0 - -dC 8  % %d1a7mS 91<KKrr,c|j\}}|Itj|gj|j |j |j }|J|j|jdd}|j||}|jj||\}}} } |j|} || z} | jd||jdd} | || fS)aUsing Tacotron2 for inference. The input is a batch of encoded sentences (``tokens``) and its corresponding lengths (``lengths``). The output is the generated mel spectrograms, its corresponding lengths, and the attention weights from the decoder. The input `tokens` should be padded with zeros to length max of ``lengths``. Args: tokens (Tensor): The input tokens to Tacotron2 with shape `(n_batch, max of lengths)`. lengths (Tensor or None, optional): The valid length of each sample in ``tokens`` with shape `(n_batch, )`. If ``None``, it is assumed that the all the tokens are valid. Default: ``None`` Returns: (Tensor, Tensor, Tensor): Tensor The predicted mel spectrogram with shape `(n_batch, n_mels, max of mel_specgram_lengths)`. Tensor The length of the predicted mel spectrogram with shape `(n_batch, )`. Tensor Sequence of attention weights from the decoder with shape `(n_batch, max of mel_specgram_lengths, max of lengths)`. r&r%r)rrtensorr tor.r/rrJrrrrunfold) rFr r,r max_lengthrrrrrrmel_outputs_postnets rrzTacotron2.infers2%ll ?llJ<077@CCFMMSYS_S_`G"""..0::1a@,,@<@LLr)s|8// $  cCtQTdidldldsds*59++++  + eCeCj01 2 +  + ++ XX__+\Fv".#RYY.#bT4T4nbii<:ryy:zEryyEP}Mryy}M@qE qEr