+ TV-jX2X^RIt^RIHt^RIHtHtHtHt^RIH t RRR^^RR.3RRllt RRRllt ]!] P] PP] PPR 7R R l4t]!] P] PP] PPR 7RR R ll4t]!] P] PP] PPR 7RRl4t]!] P] PP] PPR 7RRl4t]!] P] PP] PPR 7R4tRRRlltRRRlltRRRlltR#)N)partial)CallableDictListOptionalcV^8dQhR\R\R\R\R\R\R\R\\,R \\P .\P 3,/ #) temptop_pmin_pmin_tokens_to_keeptop_kxtc_probability xtc_thresholdxtc_special_tokensreturn)floatintrrmxarray)formats"d/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx_lm/sample_utils.py __annotate__r sz;; ; ; ; ;  ;  ;;S ;rxxj"(("#;c2aaaaaaaaa S^8XdR#.o S^8dSR8dS PV3Rl4SR8wdS PVV3Rl4SR8dS PVVV3Rl4S^8dS PV3Rl4V V3RlpV#) a Make a sampler function for use with ``generate_step``. Args: temp (float): The temperature for sampling, if 0 the argmax is used. Default: ``0``. top_p (float, optional): Nulceus sampling, higher means model considers more less likely words. min_p (float, optional): The minimum value (scaled by the top token's probability) that a token probability must have to be considered. min_tokens_to_keep (int, optional): Minimum number of tokens that cannot be filtered by min_p sampling. top_k (int, optional): The top k tokens ranked by probability to constrain the sampling to. xtc_probability (float, optional): The probability of applying XTC sampling. xtc_threshold (float, optional): The threshold the probs need to reach for being sampled. xtc_special_tokens (list(int), optional): List of special tokens IDs to be excluded from XTC sampling. Returns: Callable[mx.array, mx.array]: A sampler which takes log-probabilities and returns tokens. c2\P!VRR7#)axis)rargmax)xs&rmake_sampler../s12.r?c<\VS4#N) apply_top_p)r#r s&rr$r%4 +a*?rrc<\VSS4#r() apply_min_p)r#r rs&rr$r%6s+a@R*Src <\VSSS4#r() apply_xtc)r#rrrs&rr$r%9si?MCUVrc<\VS4#r() apply_top_k)r#rs&rr$r%<r*rc><SF pV!V4pK \VS4#r()categorical_sampling)logprobsmethodsampling_methodsr s& rsamplermake_sampler..sampler?s&&Fh'H'$Hd33r)append) r r r rrrrrr6r5s ffffffff @r make_samplerr9 sH qy.. qyUS[ ?@ | ST V  qy ?@4 Nrc(V^8dQhR\\\\3,,R\\,R\\,R\\,R\\,R\\,R\\,/#)r logit_biasrepetition_penaltyrepetition_context_sizepresence_penaltypresence_context_sizefrequency_penaltyfrequency_context_size)rrrr)rs"rrrHst66c5j)*6 6&c]6uo 6 $C= 6   6%SM6rca a.pV'ds\P!\VP444o \P!\VP 444oV V3RlpVP V4\ W3\W43\WV3.p V F,wrp V fK V ^8wgKVP V !W44K. V#)aQ Make logits processors for use with ``generate_step``. Args: repetition_penalty (float, optional): A (sign-aware) multiplicative penalty for repeating tokens. repetition_context_size (int, optional): The number of tokens to consider for repetition penalty. Default: ``20``. presence_penalty (float, optional): An additive penalty to reduce repeating tokens. presence_context_size (int, optional): The number of tokens to consider for the presence penalty. Default: ``20``. frequency_penalty (float, optional): An additive penalty to reduce repeating tokens. The tokens are penalized proportionally to their frequency. frequency_context_size (int, optional): The number of tokens to consider for the frequency penalty. Default: ``20``. logit_bias (dictionary, optional): Additive logit bias. Returns: List[Callable[[mx.array, mx.array], mx.array]]: A list of logits processors. Each processor in the list is a callable which takes an array of tokens and an array of logits and returns the updated logits. cL<VPRS3,PS4#)NNN)atadd)_logitsindicesvaluess&&rlogit_bias_processor4make_logits_processors..logit_bias_processoros!99QZ(,,V4 4r) rrlistkeysrJr8make_repetition_penaltymake_presence_penaltymake_frequency_penalty)r;r<r=r>r?r@rAlogits_processorsrKrepetition_penalties make_penaltypenalty context_sizerIrJs&&&&&&& @@rmake_logits_processorsrWHsD((4  123$z00234 5   !56 !"4N  0H !2K 0D+ |  7a<  $ $\'%H I0D r)inputsoutputscdV^8dQhR\PR\R\P/#)r r3rr)rrr)rs"rrrs.hh XXrc zVPR ,p\V\4'd^Tu;8dV8gM\RV RV R24h\P !V)V^, R R7RVR13,p\P !W\P!\R4)VP4R R7pV#) z Sample from only the top K tokens ranked by probability. Args: logprobs: A vector of log probabilities. top_k (int): Top k tokens to sample from. z(`top_k` has to be an integer in the (0, z] interval, but is .kthr .Ninfrr!) shape isinstancer ValueErrorr argpartitionput_along_axisrrdtype)r3r vocab_sizemask_idxmasked_logprobss&& rr0r0s#J eS ! !!e*@j*@6zlCgQ   yeaibA#uv+NH''BHHeEl]HNNC"O rcpV^8dQhR\PR\R\R\P/#)r r3r rr)rrrr)rs"rrrs8.?.?hh.? .?.?XX .?rc^Tu;8:dR8:gM\RV 24h\V\4'dV^8d\RV 24h\P!VR RR7pV\ P !V4,pW8pV^8dB\P!W)R R7pVRV)R13,p\P!VVR R R 7p\P!V\R 4)V4#) a7 Apply min-p sampling to the logprobs. Min-p keeps all tokens that are above a minimum probability, scaled by the probability of the most likely token. As a result, the filter is more aggressive given a very high-probability token. Args: logprobs: A vector of log probabilities. min_p (float): Minimum token probability. Typical values are in the 0.01-0.2 range, comparably selective as setting `top_p` in the 0.99-0.8 range. min_tokens_to_keep (int, optional): Minimum number of tokens that cannot be filtered. Default: ``1``. r&z9`min_p` has to be a float in the [0, 1] interval, but is z:`min_tokens_to_keep` has to be a positive integer, but is T)r keepdimsr].NFrr_r!) rbrarrmaxmathlogrcrdwherer)r3r r top_logprobs scaled_min_ptokens_to_remove top_indicess&&& rr,r,s,  # Gw O   (# . .3E3IHI[H\ ]  66(d;L$((5/1L.Aooh4GbQ !#(:':';";< ,,      88$uU|mX >>rcdV^8dQhR\PR\R\P/#)r r3r r)rrr)rs"rrrs)  "(( 5 RXX rc \P!V4p\P!VRR7p\P!W#RR7p\P!VRR7p\P !\P !V4V\P!VPR,VPR7RR7p\P!WVRR7p\P!V^V, 8V\R4)4#)z Apply top-p (nucleus) sampling to logits. Args: logprobs: A vector of log probabilities. top_p: The cumulative probability threshold for top-p filtering. Returns: token selected based on the top-p criterion. r)rer_r!) rexpargsorttake_along_axiscumsumrd zeros_likearanger`reror)r3r probssorted_indices sorted_probscumulative_probsinverse_indicess&& rr)r)s FF8 EZZr2N%%e"ELyyB7'' n% .&&r*.2F2FG  O ))*:RTU 881u9$ u  rc V^8dQhR\PR\R\R\\,R\P/#)r rHrrrr)rrrrr)rs"rrrsF!! HH!!!S ! XX !rc ^Tu;8:dR8:gM\RV 24h^Tu;8:dR8:gM\RV 24h\P!VR4pV\P!WB8V\P4P 48pV'dRVRV3&\P!\P P^^4V8V\P!V\P)V44#)aa Apply XTC sampling to the logits. Args: logits: The logits from the model's output. xtc_probability (float): Probability of XTC sampling to happen for each token xtc_threshold (float): The threshold the probs need to reach for being sampled. special_tokens_ids (list(int)): List of special tokens IDs to be excluded from XTC sampling. g?z?`threshold` has to be a float in the [0, 0.5] interval, but is r&z?`probability` has to be a float in the [0, 1] interval, but is F.r!)rbrsoftmaxror_minrandomuniform)rHrrrr|masks&&&& rr.r.s  %# %Mm_ ]    'C 'MoM^ _   JJvr "E 288E15"&&AEEG GD(-S$ $% 88 !Q/1 w' rc\\PPV^V, ,4#r)rr categorical)rHr s&&rr2r2s 99 1t8!4 55rc0V^8dQhR\R\/#r rUrVrr)rs"rrrs((U(#(rczaaS^8g\S\\34'g\RS 24hVV3RlpV#)aP Make repetition penalty processor. Paper: https://arxiv.org/abs/1909.05858 Args: penalty (float): The repetition penalty factor to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array]: The repetition penalty processor. z*penalty must be a non-negative float, got c<\V4^8dAVS)RpVRV3,p\P!V^8VS,VS, 4pW!RV3&V#rNrD)lenrro)tokensrHselected_logitsrVrUs&& rrepetition_penalty_processor=make_repetition_penalty..repetition_penalty_processor,sc v;?\MN+F$QY/O hh!#')')O !01f9  r)rarrrb)rUrVrsff rrOrOs<{*WsEl;;EgYOPP  ('rc0V^8dQhR\R\/#rr)rs"rrr;s&&5&&rcaaVV3RlpV#)a Make a presence penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit if the token has occured at least once in the ``context_size`` previous tokens. Args: penalty (float): The presence penalty to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array] cb<\V4^8dVS)RpVRV3;;,S,uu&V#r)rrrHrVrUs&&rpresence_penalty_processor9make_presence_penalty..presence_penalty_processorLs5 v;?\MN+F 1f9  (  r)rUrVrsff rrPrP;s" &%rc0V^8dQhR\R\/#rr)rs"rrrUs''E''rcaaVV3RlpV#)a Make a frequency penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit for every time that the token has occured in the ``context_size`` previous tokens. The difference with the presence penalty is that the more often a token occurs the more it will be penalized. Args: penalty (float): The frequency penalty to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array] c|<\V4^8d+VS)RpVPRV3,PS4pV#r)rrEsubtractrs&&rfrequency_penalty_processor;make_frequency_penalty..frequency_penalty_processoris> v;?\MN+FYYq&y)227;F rr)rUrVrsff rrQrQUs( '&r)NNNrNrr)r)rm functoolsrtypingrrrrmlx.corecorerr9rWcompilerstater0r,r)r.r2rOrPrQrrrrsN 11 $&;|6r BIIOORYY__EF0 BIIOORYY__E.?F.?b BIIOORYY__E F F BIIOORYY__E!F!H BIIOORYY__E6F6(B&4''r