+ LV-jPjRt^RIHt^RIHt^RIt.ROt!RR]4t Rt RRlt RRlt ] t RR ltRR ltR tR tR tRRltRRltRRltRRltRRltR#)z Functions --------- .. autosummary:: :toctree: generated/ line_search_armijo line_search_wolfe1 line_search_wolfe2 scalar_search_wolfe1 scalar_search_wolfe2 )warn)DCSRCHNLineSearchWarningc]tRt^tRtR#)rN)__name__ __module__ __qualname____firstlineno____static_attributes__rk/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/scipy/optimize/_linesearch.pyrrsr cN^Tu;8dTu;8d^8gM\R4hR#)z.'c1' and 'c2' do not satisfy'0 < c1 < c2 < 1'.N) ValueError)c1c2s&&r _check_c1_c2rs( ORO!O./ / r c aaaaaaaaVf S!S.SO5!pV.o^.o^.oVVVVV3Rlp VVVVVV3Rlp\P!VS4p\WWVVWWV R7 wpppVS^,S^,VVS^,3#)a As `scalar_search_wolfe1` but do a line search to direction `pk` Parameters ---------- f : callable Function `f(x)` fprime : callable Gradient of `f` xk : array_like Current point pk : array_like Search direction gfk : array_like, optional Gradient of `f` at point `xk` old_fval : float, optional Value of `f` at point `xk` old_old_fval : float, optional Value of `f` at point preceding `xk` The rest of the parameters are the same as for `scalar_search_wolfe1`. Returns ------- stp, f_count, g_count, fval, old_fval As in `line_search_wolfe1` gval : array Gradient of `f` at the final point Notes ----- Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1``. c^<S^;;,^, uu&S!SVS,,.SO5!#rr)sargsffcpkxks&r philine_search_wolfe1..phiRs( 1 ad"T""r c<S!SVS,,.SO5!S^&S^;;,^, uu&\P!S^,S4#rnpdot)rrfprimegcgvalrrs&r derphi"line_search_wolfe1..derphiVs@ad*T*Q 1 vvd1gr""r )rramaxaminxtol)r!r"scalar_search_wolfe1)rr#rrgfkold_fval old_old_fvalrrrr(r)r*rr&derphi0stpfvalrr$r%sffff&&&f&&&&& @@@r line_search_wolfe1r2%sL {R$ 5D B B#### ffS"oG. tT;Cx 1r!udHd1g 55r c \WV4Vf V!R4pVf V!R4pVe2V^8wd+\RRW#, ,V, 4p V ^8dRp MRp ^dp \WWVWV4p V !WWKR7wrr/WV3#)a Scalar function search for alpha that satisfies strong Wolfe conditions alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Function at point `alpha` derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0 old_phi0 : float, optional Value of phi at previous point derphi0 : float, optional Value derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax, amin : float, optional Maximum and minimum step size xtol : float, optional Relative tolerance for an acceptable step. Returns ------- alpha : float Step size, or None if no suitable step was found phi : float Value of `phi` at the new point `alpha` phi0 : float Value of `phi` at `alpha=0` Notes ----- Uses routine DCSRCH from MINPACK. Parameters `c1` and `c2` must satisfy ``0 < c1 < c2 < 1`` as described in [1]_. References ---------- .. [1] Nocedal, J., & Wright, S. J. (2006). Numerical optimization. In Springer Series in Operations Research and Financial Engineering. (Springer Series in Operations Research and Financial Engineering). Springer Nature. ?)phi0r/maxiter)\(@)rminr)rr&r6old_phi0r/rrr(r)r*alpha1r7dcsrchr0phi1tasks&&&&&&&&&& r r+r+dsj |2w*1 S&$/27:; A:FG CT :F"7Ct d?r c faaaaa aaaaaa^.o^.oR.oR.oVVVVV3Rlp VoVVVVVVV3RloVf S!S.SO5!p\P!VS4pS e VV VVVV3RlpMRp\V SWVWWWR7 wppppVf\R\^R7M S^,pVS^,S^,VVV3#)a Find alpha that satisfies strong Wolfe conditions. Parameters ---------- f : callable f(x,*args) Objective function. myfprime : callable f'(x,*args) Objective function gradient. xk : ndarray Starting point. pk : ndarray Search direction. The search direction must be a descent direction for the algorithm to converge. gfk : ndarray, optional Gradient value for x=xk (xk being the current parameter estimate). Will be recomputed if omitted. old_fval : float, optional Function value for x=xk. Will be recomputed if omitted. old_old_fval : float, optional Function value for the point preceding x=xk. args : tuple, optional Additional arguments passed to objective function. c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size extra_condition : callable, optional A callable of the form ``extra_condition(alpha, x, f, g)`` returning a boolean. Arguments are the proposed step ``alpha`` and the corresponding ``x``, ``f`` and ``g`` values. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha : float or None Alpha for which ``x_new = x0 + alpha * pk``, or None if the line search algorithm did not converge. fc : int Number of function evaluations made. gc : int Number of gradient evaluations made. new_fval : float or None New function value ``f(x_new)=f(x0+alpha*pk)``, or None if the line search algorithm did not converge. old_fval : float Old function value ``f(x0)``. new_slope : float or None The local slope along the search direction at the new value ````, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. The search direction `pk` must be a descent direction (e.g. ``-myfprime(xk)``) to find a step length that satisfies the strong Wolfe conditions. If the search direction is not a descent direction (e.g. ``myfprime(xk)``), then `alpha`, `new_fval`, and `new_slope` will be None. Examples -------- >>> import numpy as np >>> from scipy.optimize import line_search An objective function and its gradient are defined. >>> def obj_func(x): ... return (x[0])**2+(x[1])**2 >>> def obj_grad(x): ... return [2*x[0], 2*x[1]] We can find alpha that satisfies strong Wolfe conditions. >>> start_point = np.array([1.8, 1.7]) >>> search_gradient = np.array([-1.0, -1.0]) >>> line_search(obj_func, obj_grad, start_point, search_gradient) (1.0, 2, 1, 1.1300000000000001, 6.13, [1.6, 1.4]) Nc^<S^;;,^, uu&S!SVS,,.SO5!#rr)alpharrrrrs&r rline_search_wolfe2..phis( 1 ebj(4((r c<S^;;,^, uu&S!SVS,,.SO5!S^&VS^&\P!S^,S4#rr )rArr#r$r% gval_alpharrs&r r&"line_search_wolfe2..derphi#sI 1 ebj040Q 1 vvd1gr""r cr<S^,V8wd S!V4SVS,,pS!WVS^,4#rr) rArxr&extra_conditionr%rDrrs && r extra_condition2,line_search_wolfe2..extra_condition20s6!}%u URZA"5S$q': :r )r7*The line search algorithm did not converge stacklevel)r!r"scalar_search_wolfe2rr)rmyfprimerrr,r-r.rrrr(rHr7rr/rI alpha_starphi_star derphi_starr&rr#r$r%rDsf&ff&&&f&&&f& @@@@@@r line_search_wolfe2rSs| B B 6DJ))F##  {R$ffS"oG" ; ;  2F b 3//J(K 9 1 .1g r!ubeXx DDr c l\WV4Vf V!R4pVf V!R4p^p Ve)V^8wd"\RR W#, ,V, 4p MRp V ^8dRp Ve \W4p V!V 4p Tp TpVfRp\V 4EF pV ^8Xg Ve9W8d3RpTpTpRpV ^8XdRpM RRV 2,p\V\^R7MV^8pWW[,V,,8gW8d V'd\ WV WWW$WVV4 wpppMV!V 4p\ V4V)V,8:dV!W4'd T pT pTpMeV^8d\ WV V VWW$WVV4 wpppMF^V ,pVe \VV4pT p Tp T p V!V 4p TpEK T pT pRp\R \^R7VVVV3#) aFind alpha that satisfies strong Wolfe conditions. alpha > 0 is assumed to be a descent direction. Parameters ---------- phi : callable phi(alpha) Objective scalar function. derphi : callable phi'(alpha) Objective function derivative. Returns a scalar. phi0 : float, optional Value of phi at 0. old_phi0 : float, optional Value of phi at previous point. derphi0 : float, optional Value of derphi at 0 c1 : float, optional Parameter for Armijo condition rule. c2 : float, optional Parameter for curvature condition rule. amax : float, optional Maximum step size. extra_condition : callable, optional A callable of the form ``extra_condition(alpha, phi_value)`` returning a boolean. The line search accepts the value of ``alpha`` only if this callable returns ``True``. If the callable returns ``False`` for the step length, the algorithm will continue with new iterates. The callable is only called for iterates satisfying the strong Wolfe conditions. maxiter : int, optional Maximum number of iterations to perform. Returns ------- alpha_star : float or None Best alpha, or None if the line search algorithm did not converge. phi_star : float phi at alpha_star. phi0 : float phi at 0. derphi_star : float or None derphi at alpha_star, or None if the line search algorithm did not converge. Notes ----- Uses the line search algorithm to enforce strong Wolfe conditions. See Wright and Nocedal, 'Numerical Optimization', 1999, pp. 59-61. Nr4r5cR#)Tr)rArs&&r rH-scalar_search_wolfe2..extra_conditionsr z7Rounding errors prevent the line search from convergingz4The line search algorithm could not find a solution zless than or equal to amax: rLrKr8)rr9rangerr_zoomabs)rr&r6r:r/rrr(rHr7alpha0r;phi_a1phi_a0 derphi_a0irPrQrRmsgnot_first_iteration derphi_a1alpha2s&&&&&&&&&& r rNrNIsp |2w* F1 S&$/27:; z V" [FFI 7^ Q;4+ JHDK{OL4TF;< 'A 6 !e BK'11 1  #6ff$"R_F .J+ 6N  Nrc'k )v..# !'  Nff$i"R_F .J+ V  &FV cj   9 1 . x{ 22r c d\P!RRRR7;_uu_4TpW0, pWP, p W,^,W, ,p \P!R4p V ^,V R&V^,)V R&V ^,)V R&V^,V R&\P!V \P!WA, Wx,, Wa, Wy,, .4P 44wrW,p W,p W,^V ,V,, pW )\P !V4,^V ,, ,pRRR4\P!X4'gR#T# \d RRR4R#i;i +'giLJ;i) z Finds the minimizer for a cubic polynomial that goes through the points (a,fa), (b,fb), and (c,fc) with derivative at a of fpa. If no minimizer can be found, return None. raisedivideoverinvalidN)ri)rr)r)rjr)rjrj) r!errstateemptyr"asarrayflattensqrtArithmeticErrorisfinite)afafpabfbcrCdbdcdenomd1ABradicalxmins&&&&&&& r _cubicminrsQ G'7 C C ABBWNbg.E&!BQwBtHaxBtHaxBtHQwBtHVVB BGaf,<,.Gaf,<,>!??FwyJFQ JA JAea!eai'GRWWW--!a%88D! D& ;;t   K  % D C" # D Cs)FD8F FFFF F/ c\P!RRRR7;_uu_4TpTpW0R,, pWE, Wg,, Ww,, pWRV,, , p RRR4\P!X 4'gR#T # \d RRR4R#i;i +'giLJ;i)zz Finds the minimizer for a quadratic polynomial that goes through the points (a,fa), (b,fb) with derivative at a of fpa. rdrer5@N)r!rkrprq) rrrsrtrurvDrxryr~rs &&&&& r _quadminrs G'7 C C AAWB!&RW-AC!G}$D D ;;t   K   D C  D Cs)B2A B B/#B2.B//B22 C c ^ p ^p RpRpTp^pW, pV^8dYppMYppV ^8dVV,p\WWAVVV4pV ^8Xg!XeVVX, 8gVVV,8dGVV,p\WWAV4pVeVVV, 8gVVV,8dVRV,,pV!V4pVWyV,V,,8gVV8d TpTpTpTpM]V!V4p\V4V )V,8:dV !VV4'dTpTpTpM@VW, ,^8d TpTpTpTpMTpTpTpTpTpV ^, p W8gEKERpRpRpVVV3#)zZoom stage of approximate linesearch satisfying strong Wolfe conditions. Part of the optimization algorithm in `scalar_search_wolfe2`. Notes ----- Implements Algorithm 3.6 (zoom) in Wright and Nocedal, 'Numerical Optimization', 1999, pp. 61. g?皙?N?)rrrY)a_loa_hiphi_lophi_hi derphi_lorr&r6r/rrrHr7r^delta1delta2phi_reca_recdalpharrrucchka_jqchkphi_aj derphi_aja_starval_star valprime_stars&&&&&&&&&&&& r rXrXsG A F FG E  A:qAqqA EF?DD)6!7,C F q4xS1t8^F?D4&AC qv34<SZ'S TsF7N* *&0@GEDFs I9~"W,f1M1M! ) $+&!+  DF!I Q KFH M  8] **r caaaaa \P!S4o^.o VVV VV3RlpVf V!R4p MTp \P!VS4p \WWVR7wrV S ^,V 3#)aMinimize over alpha, the function ``f(xk+alpha pk)``. Parameters ---------- f : callable Function to be minimized. xk : array_like Current point. pk : array_like Search direction. gfk : array_like Gradient of `f` at point `xk`. old_fval : float Value of `f` at point `xk`. args : tuple, optional Optional arguments. c1 : float, optional Value to control stopping criterion. alpha0 : scalar, optional Value of `alpha` at start of the optimization. Returns ------- alpha f_count f_val_at_alpha Notes ----- Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 c^<S^;;,^, uu&S!SVS,,.SO5!#rr)r;rrrrrs&r rline_search_armijo..phis( 1 fRi'$''r r4)rrZ)r! atleast_1dr"scalar_search_armijo)rrrr,r-rrrZrr6r/rAr=rsfff&&f&& @r line_search_armijorosnD r B B((2wffS"oG&s'.46KE "Q% r c X\WW#WEVVR7pV^,V^,^V^,3#)z0 Compatibility wrapper for `line_search_armijo` )rrrZ)r) rrrr,r-rrrZrs &&&&&&&& r line_search_BFGSrs4 1"82"( *A Q41q!A$ r c V!V4pWaW4,V,,8:dWF3#V)V^,,R, Wa, W$,, , pV!V4pWW7,V,,8:dWx3#Wu8Ed~V^,V^,,Wt, ,p V^,W, W',, ,V^,Wa, W$,, ,, p W, p V^,)W, W',, ,V^,Wa, W$,, ,,p W, p V )\P!\V ^,^V ,V,, 44,RV ,, p V!V 4p WW<,V,,8:dW3#W|, VR, 8g^W, , R8d VR, p TpT pTpT pEKRV3#)aMinimize over alpha, the function ``phi(alpha)``. Uses the interpolation algorithm (Armijo backtracking) as suggested by Wright and Nocedal in 'Numerical Optimization', 1999, pp. 56-57 alpha > 0 is assumed to be a descent direction. Returns ------- alpha phi1 rg@gQ?N)r!rorY)rr6r/rrZr)r\r;r[factorrrrurbphi_a2s&&&&&& r rrs[F  ')))~Z&!) #c )V]W=M-M NF [F7***~ -VQY&&-8 AI7 8 AI7 8 9 J QYJ&-'.8 9 AI7 8 9 J"rwws1a4!a%'/#9:;;AFV RYw.. .> ! Ov| +FM0AT/Ic\F <r cVR,p\V4p ^p ^p ^p WV,,p V!V 4wrWV,WZ^,,V,, 8:dT p MV ^,V,V^V ,^, V,,, pWV,, p V!V 4wrWV,W[^,,V,, 8:dV )p M~V ^,V,V^V ,^, V,,, p\P!VWj,Wz,4p \P!VWk,W{,4p EK;WW3#)a Nonmonotone backtracking line search as described in [1]_ Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. prev_fs : float List of previous merit function values. Should have ``len(prev_fs) <= M`` where ``M`` is the nonmonotonicity window parameter. eta : float Allowed merit function increase, see [1]_ gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position References ---------- [1] "Spectral residual method without gradient information for solving large-scale nonlinear systems of equations." W. La Cruz, J.M. Martinez, M. Raydan. Math. Comp. **75**, 1429 (2006). )maxr!clip)rx_kdprev_fsetagammatau_mintau_maxf_kf_baralpha_palpha_mrAxpfpFpalpha_tpalpha_tms&&&&&&&& r _nonmonotone_line_search_cruzrs$P "+C LEGG E  Q; 2 uz1C77 7E A:#rQwY]C,?'?@ Q; 2 uz1C77 7HE A:#rQwY]C,?'?@''(G$5w7HI''(G$5w7HI b r c ^p ^p ^p WV,,pV!V4wppWV,W{^,,V,, 8:dT p MV ^,V,V^V ,^, V,,, pWV,, pV!V4wppWV,W|^,,V,, 8:dV )p M~V ^,V,V^V ,^, V,,, p\P!VW,W,4p \P!VW,W,4p EK=W,^,pW,WF,,V,V, pTpWVVWE3#)a Nonmonotone line search from [1] Parameters ---------- f : callable Function returning a tuple ``(f, F)`` where ``f`` is the value of a merit function and ``F`` the residual. x_k : ndarray Initial position. d : ndarray Search direction. f_k : float Initial merit function value. C, Q : float Control parameters. On the first iteration, give values Q=1.0, C=f_k eta : float Allowed merit function increase, see [1]_ nu, gamma, tau_min, tau_max : float, optional Search parameters, see [1]_ Returns ------- alpha : float Step length xp : ndarray Next position fp : float Merit function value at next position Fp : ndarray Residual at next position C : float New value for the control parameter C Q : float New value for the control parameter Q References ---------- .. [1] W. Cheng & D.-H. Li, ''A derivative-free nonmonotone line search and its application to the spectral residual method'', IMA J. Numer. Anal. 29, 814 (2009). )r!r)rrrrrxQrrrrnurrrArrrrrQ_nexts&&&&&&&&&&& r _nonmonotone_line_search_chengr2sD^GG E  Q; 2B S5A:-33 3E A:#rQwY]C,?'?@ Q; 2B S5A:-33 3HE A:#rQwY]C,?'?@''(G$5w7HI''(G$5w7HIVaZF 17 b F*AA b"a ""r )rr2rSr+rNr) NNNr-C6??2:0yE>+=)NNNrrrrr) NNNrrrNN )NNNrrNNr)rrrj)rrjr)rrr)rrrg333333?)__doc__warningsr_dcsrchrnumpyr!__all__RuntimeWarningrrr2r+ line_searchrSrNrrrXrrrrrrr r rs  !  /<6~JZ! LE^Q3hD*T+v1h7|EPN#r