i ^dZddlZddlmZmZddlmZmZddlZ ddl m Z m Z ddl mZmZmZmZmZmZddlmZddlmZmZdd lmZmZdd lmZdd lmZm Z m!Z!gd Z"d Z# ddZ$dZ%ddZ&dZ'GddeeeeeeZ(Gdde(Z)Gdde(Z*Gdde(Z+GddeeeZ,y)zG The :mod:`sklearn.pls` module implements Partial Least Squares (PLS). N)ABCMetaabstractmethod)IntegralReal)pinvsvd) BaseEstimatorClassNamePrefixFeaturesOutMixinMultiOutputMixinRegressorMixinTransformerMixin _fit_context)ConvergenceWarning) check_arraycheck_consistent_length)Interval StrOptions)svd_flip) FLOAT_DTYPEScheck_is_fitted validate_data)PLSSVD PLSCanonical PLSRegressionc t|dd\}}}|jjj}ddd}t j |||zt j |jz}t j||kD}|ddd|f}||d|z}t jt jt j||d|S)NF) full_matrices check_finiteg@@g.A)fd) rdtypecharlowernpmaxfinfoepssum transpose conjugatedot)ausvhtfactorcondranks u/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/sklearn/cross_decomposition/_pls.py _pinv2_oldr4 s 1E>HAq"  AS !F 66!9vay 288A;?? 2D 66!d( D !UdU( A5DMA << RVVAr%4y%9: ;;ctj|jj t fd|j D}d}|dk(rt|t|} } t|D]} |dk(rtj |} n7tj|j |tj||z } | tjtj| | zz} tj|| } |dk(rtj | }nAtj|j | tj| j | z }|r/|tjtj||zz}tj||tj||zz }| |z }tj|||ks|jddk(rn| } dz}||k(rtjdt |fS#t $r}t d|d}~wwxYw)a?Return the first left and right singular vectors of X'y. Provides an alternative to the svd(X'y) and uses the power method instead. With norm_y_weights to True and in mode A, this corresponds to the algorithm section 11.3 of the Wegelin's review, except this starts at the "update saliences" part. c3zK|]2}tjtj|kDs/|4ywN)r#anyabs).0colr&s r3 z;_get_first_singular_vectors_power_method..?s*GcsRVVBFF3K#4E-Fscs0;;y residual is constantNdBz$Maximum number of iterations reached)r#r%r r&nextT StopIterationr4ranger*sqrtshapewarningswarnr)Xymodemax_itertolnorm_y_weightsy_scoree x_weights_oldX_pinvy_pinvi x_weightsx_score y_weightsx_weights_diffn_iterr&s @r3(_get_first_singular_vectors_power_methodr[2s ((177   C=GaccGGM s{$A 1  8_ 3;vw/IqssG,rvvgw/GGIRWWRVVIy9:S@@ &&I& 3;vw/IqssG,rvvgii/III   9!=>D DI&&I&"&&I*F*LM"]2 66.. 1C 71771:? ! -0UF  <>PQ i ''U =451<=sH,, I5 IIctj|j|}t|d\}}}|dddf|dddffS)zbReturn the first left and right singular vectors of X'y. Here the whole SVD is computed. FrNr)r#r*rCr)rJrKCU_Vts r3_get_first_singular_vectors_svdrbmsD qssAA1E*HAq" QT7Bq!tH r5c||jd}||z}|jd}||z}|rA|jdd}d||dk(<||z}|jdd}d||dk(<||z}nDtj|jd}tj|jd}||||||fS)z{Center X, y and scale if the scale parameter==True Returns ------- X, y, x_mean, y_mean, x_std, y_std raxisrA)reddofg?)meanstdr#onesrG)rJrKscalex_meany_meanx_stdy_stds r3_center_scale_xyrpwsVVV^FKA VVV^FKA 11%!esl U 11%!esl U  # # a --r5ctjtj|}tj||}||z}||z}y)z7Same as svd_flip but works on 1d arrays, and is inplaceN)r#argmaxr:sign)r,vbiggest_abs_val_idxrss r3 _svd_flip_1drvsA))BFF1I. 771() *DIAIAr5c $eZdZUdZeedddgdgeddhged d hged d hgeedddgeed ddgdgdZe e d<e dddd d dddddZ e ddZddZddZddZddZfdZxZS) _PLSaPartial Least Squares (PLS) This class implements the generic PLS algorithm. Main ref: Wegelin, a survey of Partial Least Squares (PLS) methods, with emphasis on the two-block case https://stat.uw.edu/sites/default/files/files/reports/2000/tr371.pdf rANleftclosedboolean regression canonicalAr@rnipalsr n_componentsrkdeflation_moderL algorithmrMrNcopy_parameter_constraintsTư>)rkrrLrrMrNrct||_||_||_||_||_||_||_||_yr8)rrrLrkrrMrNr) selfrrkrrLrrMrNrs r3__init__z _PLS.__init__s>),  "   r5prefer_skip_nested_validationc  t||t||tjd|jd}t |dtjd|jd}|j dk(rd|_|jdd}nd|_|jd }|jd}|jd}|j}|jd k(r t||n t|||}||kDrtd |d |d |jdk(|_|j}t|||j \} } |_|_|_|_tj*||f|_tj*||f|_tj*||f|_tj*||f|_tj*||f|_tj*||f|_g|_tj:| j<j>} tA|D]q} |jBdk(rtjDtjF| d| zkd } d| dd| f< tI| | |jJ|jL|jN|\}}}|j8jY|n|jBdk(rt[| | \}}t]tj^| |}|rd}ntj^||}tj^| ||z }tj^|| tj^||z }| tj`||z} |jdk(rFtj^|| tj^||z }| tj`||z} |jd k(rFtj^|| tj^||z }| tj`||z} ||j,dd| f<||j.dd| f<||j0dd| f<||j2dd| f<||j4dd| f<|j6dd| f<ttj^|j,tctj^|j4jd|j,d|_3tj^|j.tctj^|j6jd|j.d|_4tj^|jf|j6jd|_5|jj|j(zjd|j&z |_5|j$|_6|jfjd|_7|S#tP$r3}tS|dk7rtUjVd| Yd}~d}~wwxYw)Fit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples,) or (n_samples, n_targets) Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : object Fitted model. Tr force_writeablerensure_min_samplesrKF input_namer rr ensure_2drArr}`n_components` upper bound is . Got instead. Reduce `n_components`.r~r rdrgN)rLrMrNrOr>z$y residual is constant at iteration r)r)8rrr#float64rrndim _predict_1dreshaperGrrmin ValueError_norm_y_weightsrprk_x_mean_y_mean_x_std_y_stdzeros x_weights_ y_weights_ _x_scores _y_scores x_loadings_ y_loadings_n_iter_r%r r&rErallr:r[rLrMrNrDstrrHrIappendrbrvr*outerrrC x_rotations_ y_rotations_coef_ intercept__n_features_out)rrJrKnpqrrank_upper_boundrOXkyky_epskyk_maskrVrXrrQx_scoresy_ssy_scores x_loadings y_loadingss r3fitz_PLS.fitsa& 1%   **     **    66Q;#D  "a A$D  GGAJ GGAJ GGAJ(( ,, ))At'7'7'9'9:4;;&4<<' =A#\BA ii4+;+;+=+=>O t{{ *O t|| +O"O3 3r5ct|t|||td}||jz}||jj z|j z}|jr|jS|S)aLPredict targets of given samples. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples. copy : bool, default=True Whether to copy `X` or perform in-place normalization. Returns ------- y_pred : ndarray of shape (n_samples,) or (n_samples, n_targets) Returns predicted values. Notes ----- This call requires the estimation of a matrix of shape `(n_features, n_targets)`, which may be an issue in high dimensional space. Fr) rrrrrrCrrravel)rrJry_preds r3predictz _PLS.predicts`,  $L N T\\TZZ\\!DOO3!%!1!1v||~=v=r5cF|j||j||S)aLearn and apply the dimension reduction on the train data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of predictors. y : array-like of shape (n_samples, n_targets), default=None Target vectors, where `n_samples` is the number of samples and `n_targets` is the number of response variables. Returns ------- self : ndarray of shape (n_samples, n_components) Return `x_scores` if `y` is not given, `(x_scores, y_scores)` otherwise. rrrrJrKs r3 fit_transformz_PLS.fit_transform!$xx1~''1--r5cht|}d|j_d|j_|S)NTF)super__sklearn_tags__regressor_tags poor_score target_tagsrequired)rtags __class__s r3rz_PLS.__sklearn_tags__s1w'))-&$)! r5r)NTr8T)__name__ __module__ __qualname____doc__rrrrrdict__annotations__rrrrrrrrr __classcell__rs@r3rxrxs"(AtFCD%|[&ABCS#J'( %!234h4?@q$v67  $D #   *5`6`D%N*X>:.(r5rx) metaclassceZdZUdZiej Zeed<dD]Zeje d dddddfd Z fd Z xZ S) ra PLS regression. PLSRegression is also known as PLS2 or PLS1, depending on the number of targets. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, n_features]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in :term:`fit` before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_scores_ : ndarray of shape (n_samples, n_components) The transformed training samples. y_scores_ : ndarray of shape (n_samples, n_components) The transformed training targets. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_target, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. Examples -------- >>> from sklearn.cross_decomposition import PLSRegression >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> pls2 = PLSRegression(n_components=2) >>> pls2.fit(X, y) PLSRegression() >>> y_pred = pls2.predict(X) For a comparison between PLS Regression and :class:`~sklearn.decomposition.PCA`, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_pcr_vs_pls.py`. rrrLrTrrrkrMrNrc 4t|||ddd|||y)Nr}rrrrrrrrkrMrNrrs r3rzPLSRegression.__init__ms/ %'  r5clt||||j|_|j|_|S)r)rrr x_scores_r y_scores_)rrJrKrs r3rzPLSRegression.fit{s.$  Aq r5r) rrrrrxrrrparampoprrrrs@r3rrs]eN$Cd&A&A#BDB8""5)9  '+cu4  r5rceZdZUdZiej Zeed<dD]Zeje d ddddddfd Z xZ S) ra^ Partial Least Squares transformer and regressor. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. algorithm : {'nipals', 'svd'}, default='nipals' The algorithm used to estimate the first singular vectors of the cross-covariance matrix. 'nipals' uses the power method while 'svd' will compute the whole SVD. max_iter : int, default=500 The maximum number of iterations of the power method when `algorithm='nipals'`. Ignored otherwise. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. Empty if `algorithm='svd'`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- CCA : Canonical Correlation Analysis. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import PLSCanonical >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [2.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> plsca = PLSCanonical(n_components=2) >>> plsca.fit(X, y) PLSCanonical() >>> X_c, y_c = plsca.transform(X, y) r)rrLTrrr)rkrrMrNrc 4t|||dd||||y)Nr~rrr)rrrkrrMrNrrs r3rzPLSCanonical.__init__s/ %&  r5r rrrrrxrrrrrrrrs@r3rrs``D$Cd&A&A#BDB+""5),     r5rceZdZUdZiej Zeed<dD]Zeje d dddddfd Z xZ S) CCAa Canonical Correlation Analysis, also known as "Mode B" PLS. For a comparison between other cross decomposition algorithms, see :ref:`sphx_glr_auto_examples_cross_decomposition_plot_compare_cross_decomposition.py`. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, default=2 Number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. max_iter : int, default=500 The maximum number of iterations of the power method. tol : float, default=1e-06 The tolerance used as convergence criteria in the power method: the algorithm stops whenever the squared norm of `u_i - u_{i-1}` is less than `tol`, where `u` corresponds to the left singular vector. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If False, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the cross-covariance matrices of each iteration. y_weights_ : ndarray of shape (n_targets, n_components) The right singular vectors of the cross-covariance matrices of each iteration. x_loadings_ : ndarray of shape (n_features, n_components) The loadings of `X`. y_loadings_ : ndarray of shape (n_targets, n_components) The loadings of `y`. x_rotations_ : ndarray of shape (n_features, n_components) The projection matrix used to transform `X`. y_rotations_ : ndarray of shape (n_targets, n_components) The projection matrix used to transform `y`. coef_ : ndarray of shape (n_targets, n_features) The coefficients of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. intercept_ : ndarray of shape (n_targets,) The intercepts of the linear model such that `y` is approximated as `y = X @ coef_.T + intercept_`. .. versionadded:: 1.1 n_iter_ : list of shape (n_components,) Number of iterations of the power method, for each component. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. PLSSVD : Partial Least Square SVD. Examples -------- >>> from sklearn.cross_decomposition import CCA >>> X = [[0., 0., 1.], [1.,0.,0.], [2.,2.,2.], [3.,5.,4.]] >>> y = [[0.1, -0.2], [0.9, 1.1], [6.2, 5.9], [11.9, 12.3]] >>> cca = CCA(n_components=1) >>> cca.fit(X, y) CCA(n_components=1) >>> X_c, y_c = cca.transform(X, y) rrTrrrc 4t|||ddd|||y)Nr~r@rrrrs r3rz CCA.__init__xs/ %&  r5rrrs@r3rrsXXt$Cd&A&A#BDB8""5)9  '+cu4   r5rceZdZUdZeedddgdgdgdZeed<dd d d d Z e d d Z ddZ ddZ y)raPartial Least Square SVD. This transformer simply performs an SVD on the cross-covariance matrix `X'y`. It is able to project both the training data `X` and the targets `y`. The training data `X` is projected on the left singular vectors, while the targets are projected on the right singular vectors. Read more in the :ref:`User Guide `. .. versionadded:: 0.8 Parameters ---------- n_components : int, default=2 The number of components to keep. Should be in `[1, min(n_samples, n_features, n_targets)]`. scale : bool, default=True Whether to scale `X` and `y`. copy : bool, default=True Whether to copy `X` and `y` in fit before applying centering, and potentially scaling. If `False`, these operations will be done inplace, modifying both arrays. Attributes ---------- x_weights_ : ndarray of shape (n_features, n_components) The left singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. y_weights_ : ndarray of (n_targets, n_components) The right singular vectors of the SVD of the cross-covariance matrix. Used to project `X` in :meth:`transform`. n_features_in_ : int Number of features seen during :term:`fit`. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Defined only when `X` has feature names that are all strings. .. versionadded:: 1.0 See Also -------- PLSCanonical : Partial Least Squares transformer and regressor. CCA : Canonical Correlation Analysis. Examples -------- >>> import numpy as np >>> from sklearn.cross_decomposition import PLSSVD >>> X = np.array([[0., 0., 1.], ... [1., 0., 0.], ... [2., 2., 2.], ... [2., 5., 4.]]) >>> y = np.array([[0.1, -0.2], ... [0.9, 1.1], ... [6.2, 5.9], ... [11.9, 12.3]]) >>> pls = PLSSVD(n_components=2).fit(X, y) >>> X_c, y_c = pls.transform(X, y) >>> X_c.shape, y_c.shape ((4, 2), (4, 2)) rANryrzr|rrkrrT)rkrc.||_||_||_yr8r)rrrkrs r3rzPLSSVD.__init__s(  r5rc&t||t||tjd|jd}t |dtjd|jd}|j dk(r|jdd}|j}t|jd |jd|jd}||kDrtd |d |d t|||j\}}|_|_|_|_tj$|j&|}t)|d \}}}|ddd|f}|d|}t+||\}}|j&} ||_| |_|j,jd|_|S)aJFit model to data. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets) Targets. Returns ------- self : object Fitted estimator. TrrrKFrrArrrrrr]N)rrr#rrrrrrrrGrrprkrrrrr*rCrrrrr) rrJrKrrr^r_r-raVs r3rz PLSSVD.fits" 1%   **     **    66Q; "a A (( qwwqz1771:qwwqzB * *01A0BC#n$DF  FV q$**F B1dlDL$+t{ FF133Nq.1b a,    B2 DD#44Q7 r5ct|t||tjd}||jz |j z }tj ||j}|~t|ddtj}|jdk(r|jdd}||jz |jz }tj ||j}||fS|S)a  Apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Samples to be transformed. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- x_scores : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. F)r rrK)rrr rAr)rrr#rrrr*rrrrrrr)rrJrKXrryrrs r3rzPLSSVD.transforms&  $5 A$,,$++ -66"doo. =A#bjjQAvv{IIb!$dll"dkk1Bvvb$//2HX% %r5cF|j||j||S)aLearn and apply the dimensionality reduction. Parameters ---------- X : array-like of shape (n_samples, n_features) Training samples. y : array-like of shape (n_samples,) or (n_samples, n_targets), default=None Targets. Returns ------- out : array-like or tuple of array-like The transformed data `X_transformed` if `y is not None`, `(X_transformed, y_transformed)` otherwise. rrs r3rzPLSSVD.fit_transform7rr5rr8)rrrrrrrrrrrrrrr5r3rrshAH"(AtFCD $D 4 5>6>@@.r5r)rrrFr)-rrHabcrrnumbersrrnumpyr# scipy.linalgrr sklearn.baser r r r r rsklearn.exceptionsr sklearn.utilsrrsklearn.utils._param_validationrrsklearn.utils.extmathrsklearn.utils.validationrrr__all__r4r[rbrprvrxrrrrrr5r3rs'""2>@*QQ 5<&=B8(v.4]# ]@ VDVrB 4B Jk $k \B. ,.> B.r5