+ UV-j*^RIt^RIt^RIt^RIt^RIt^RIt^RIHt^RIH t ^RI H t H t H t HtHtHtHt^RIHt^RIHt^RIHt^RIHt^RIHt^RIHt ^R I!H"t"^R I#H$t$^R I%H&t&^R I'H(t(^R I)H*t*H+t+^RI,H-t-H.t.H/t/H0t0^RI1H2t2H3t3H4t4H5t5H6t6]Pn!R4t8Rt9Rt:Rt;RtRt?Rt@^tA^tBRtC^tD^@tERtF^ tG^tHRtIRtJRtKRtL^@tM^0tNRtORRltP]P!]P!44tS]F3RRllt]PR5RR ll4tU] !R!R"44tV!R#R$4tW^R%IXHVtVHStSHUtU^R&IYHMtMHZtZH[t[H\t\H]t]H^t^R'R(lt_R)R*lt`R+R,lta^R-IbHctcR6R.R/lltdR7R0R1llteR2tf]gR38Xd]h!R44]f!4R#R#)8N)Sequence) dataclass)AnyDict GeneratorListOptionalTupleUnion) tree_reduce)maybe_quantize_kv_cache)PreTrainedTokenizer)apc)cache)apply_chat_template)format_speculative_stats)make_streaming_detokenizer)TurboQuantKVCacheturboquant_enabled)StoppingCriteriaThinkingBudgetCriterialoadprepare_inputs)DEFAULT_IMAGE_GUIDANCEDEFAULT_IMAGE_SIZEDEFAULT_IMAGE_STEPSDEFAULT_IMAGE_TASKrun_image_generation_clizmlx_vlm.generatez mlx-community/nanoLLaVA-1.5-8bitzWhat are these?i?uniformzzic \P!RR7pVPR\\RR7VPR\RRRR7VPR \R R R7VPR \R\ R R7VPR\R R\ R2R7VPR\\RR7VPR\R RR7VPR\\RR7VPR\R RR7VPR\R RR7VPR\R\RR7VPR\R\R R7VPR!\R\R"R7VPR#\R$R%R7VPR&\RR R'R7VPR(\R\R)R7VPR*\R R+R7VPR,\\R-R7VPR.\R R/\ R02R7VPR1\R R2R7VPR3\R R4R7VPR5\R R6R7VPR7\R R8R7VPR9\R R:R7VPR;\R RR?R@7VPRA\R RB\" RC2R7VPRD\R RER7VPRFRGRH.RGRIRJ7VPRK\R RLR7VPRM\R RNR7VPRO\\$RPR7VPRQ\R RRR7VPRS\\&RTR7VPRU\R RVR7VPRW\\&RXR7VPRY\R RZR7VPR[\\&R\R7VPR]R>R^R@7VPR_\P(R`RaRb7VPRc\RR RdR7VPRe\R RfR7VPRg\R RhR7VPRi\R\*RjR7VPRk\\,RlR7VPRm\\.RnR7VPRoR>RpR@7VPRqR>RrR@7VPRs\RtRuR7VPRvR>RwR@7VPRxRyR>RzR@7VPR{\0P2/R|R7VPR}\0P2/R~R7VPR\\4RR7VPR\R RR7VPR\R .RORR7VPR\R RR7VPRR>RR@7VPR\R RR7VPR\\6RR7VPR\\8RR7VP;4#)z*Generate text from an image using a model.) descriptionz--modelz;The path to the local model directory or Hugging Face repo.)typedefaulthelpz--output-modalitytextzKGenerate text with a VLM or generate an image with a supported image model.)r$choicesr%r&z--outputNz!Output path for image generation.z--taskz;Image task to run when --output-modality image is selected.z--sizez3Image size as WIDTHxHEIGHT. Generation defaults to z5; editing defaults to the first reference image size.z--stepsz Number of image inference steps.z--seedzyPRNG seed for reproducible sampling and diffusion canvas init. Image generation/editing defaults to a random 32-bit seed.z --guidancez6Classifier-free guidance for image generation/editing.z--prompt-expansion-modelzfText model path or Hugging Face repo used to expand plain image prompts into Ideogram 4 JSON captions.z--adapter-pathz The path to the adapter weights.z--image+z$URL or path of the image to process.)r$nargsr%r&z--audioz$URL or path of the audio to process.z--videoz$URL or path of the video to process.z--fpsg@z)Frames-per-second to sample from --video.z--resize-shapezResize shape for the image.z--promptz%Message to be processed by the model.z--systemzSystem message for the model.z --max-tokensz%Maximum number of tokens to generate.z--max-denoising-stepsziMaximum denoising steps for diffusion generation. Default: the checkpoint's generation config (typically zZ). Adaptive stopping usually converges canvases earlier; set lower to hard-cap throughput.z--block-lengthz+Block length for diffusion text generation.z--num-to-transferzHTarget number of masked tokens to transfer per diffusion denoising step.z--max-transfer-per-stepz?Maximum confident masked tokens to transfer per denoising step.z--editing-thresholdz9Confidence threshold for diffusion post-fill token edits.z--max-post-stepsz4Maximum diffusion post-fill editing steps per block.z--stability-stepsz?Stop post-fill refinement after this many stable no-edit steps.z--diffusion-full-canvas store_truezjUse the checkpoint canvas length for diffusion generation even when --max-tokens requests a partial block.)actionr&z--diffusion-min-canvas-lengthzDMinimum active canvas length for diffusion partial blocks. Default: .z--diffusion-max-canvas-lengthzMaximum active canvas length for diffusion generation. Default: the checkpoint canvas length; set lower to trade quality for throughput.z--diffusion-sampler entropy-boundzconfidence-thresholdz/Canvas update sampler for diffusion generation.)r(r%r&z --thresholdzToken probability threshold for diffusion confidence transfer. Default: 0.9 for confidence-threshold sampling; masked-diffusion models use their checkpoint reference defaults.z--min-thresholdzALowest token probability threshold for masked diffusion transfer.z --temperaturezTemperature for sampling.z--repetition-penaltyz/Penalty factor for previously generated tokens.z--repetition-context-sizez>Number of recent generated tokens used for repetition penalty.z--presence-penaltyz2Additive penalty for tokens that already appeared.z--presence-context-sizez     >     P    # (     >   # / M     A   ! / K     :   " / L   )normalize_resize_shape..s7uU s"s FTz)resize_shape must contain 1 or 2 integers)rV) isinstancerr:byteslenall ValueErrortuple)rWs&rRnormalize_resize_shaperis~68$$6C<00 K6 ! C77CCC77 7 7DEE%([A%5F1Ivay !H5=HrTc$V^8dQhR\/#)rVkv_quant_scheme)r:)rYs"rRrZrZs11  1rTcaaaSfR#\SV4'dUVVV3Rlo\V4^8d\V4^, MRp\V4FwrgWe8XdK S!V4W&K R#\VSV\ S4R7R#)Nc<\V\4'dV#\V\P4'dV#\V\P4'dIVP ^8Xd \SR7#VP S8dV#\P !VSR7#\V\P4'd*VPUu.uF pS!V4NK upVnV#\V\4'd!\V4Fwr!S!V4W&K V#\V\4'd3\;QJd.V3RlV4FNK 5#!V3RlV44#V#uupi)r)bitsc34<"TF pS!V4xK R#5ir])r^ sub_entryquantize_entrys& rRr`Bmaybe_quantize_kv_cache..quantize_entry.. sN9^I66s) rcrrRotatingKVCacheKVCacheoffset from_cache CacheListcacheslist enumeraterh)entryrqikv_bitsrrquantized_kv_starts& rRrr/maybe_quantize_kv_cache..quantize_entrys%!233 %!6!677 %//<<1$,'::<<"44 L(33EHH%11KP<<Xwired_limit..+s"Arxx)@)@sXX~IcIrT max_recommended_working_set_sizeg?z0[WARNING] Generating with a model that requires z6 MB which is close to the maximum recommended size of z MB. This can be slow. See the documentation for possible work-arounds: https://github.com/ml-explore/mlx-lm/tree/main#large-modelsi)rmetal is_availabler device_infoprintset_wired_limit synchronize)rr model_bytes max_rec_sizemodel_mb max_rec_mb old_limitss&& rR wired_limitrs 88 " " I5RSK>>#$FGLS<''%'!U*  >xjIAAK MJ J ""<0I&  q! NN  9%  q! NN  9%s BE  C5$AE 5AEE cXa]tRtRtoRtRtRt^t^t^t Rt Rt Rt ^t RtV3RltRtVtR#)GenerationResultiCNrc<V^8dQh/S[;R&S[S[,;R&S[S[S[,,;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R &S[;R &S[S[,;R &#) rVr'tokenlogprobs prompt_tokensgeneration_tokens total_tokens prompt_tpsgeneration_tps peak_memory cached_tokens finish_reason)r:r r<rr=)rY __classdict__s"rRrZGenerationResult.__annotate__Cs N C=tE{#*     C='rTrp)__name__ __module__ __qualname____firstlineno__r'rrrrrrrrrr__annotate_func____static_attributes____classdictcell__rs@rRrrCsFDE&*HMLJNKM$(MrTrcNa]tRtRtoRtRtV3RlRltV3RlRltRtVt R #) PromptCacheStateiTaHolds KV cache and token history across conversation turns. Pass this to stream_generate via the ``prompt_cache_state`` kwarg to reuse the KV cache from previous turns. Only the new tokens (after the common prefix) are processed, avoiding redundant prefill. c"RVnRVnR#r]r token_ids)selfs&rR__init__PromptCacheState.__init__\s*. .2rTc&<V^8dQhRS[RS[/#)rVnew_idsrX)rzr<)rYrs"rRrZPromptCacheState.__annotate__`s$3rTc VPf^#\\VP4\V44p\V4F%pVPV,W,8wgK#Vu# V#)z>Return the number of leading tokens that match the cached ids.)rminrerange)rrmax_lenr}s&& rRfind_prefix_length#PromptCacheState.find_prefix_length`sU >> !c$..)3w<8wA~~a GJ. rTc&<V^8dQhRS[RS[/#)rVrkv_cache)rz)rYrs"rRrZrjsrTc 2\V4VnW nR#)z9Store the full token sequence and corresponding KV cache.N)rzrr)rrrs&&&rRupdatePromptCacheState.updatejsi rTrN) rrrr__doc__rrrrrrs@rRrrTs#3rTr)rgeneration_streamr)rDDiffusionOutputHandlerdiffusion_kwargs_from_argsis_diffusion_modelis_masked_diffusion_model%stream_diffusion_generate_from_kwargscDV^8dQhR\PR\/#)rVrrX)rrbool)rYs"rRrZrZ{s,,")),,rTc\V4#r])r)rs&rRis_masked_diffusion_text_modelr{s $U ++rTcrV^8dQhR\PR\\\3,R\ /#)rVrkwargsrX)rrrr:rr)rYs"rRrZrZs.  299 d38n QU rTc\V4'gR#\VRR4p\VRR4R8wdR#VPR4pVeVR8g#R#)FconfigNdefault_generation_modearTgeneration_mode)rgetattrget)rrrrs&& rR_use_masked_diffusion_text_pathrsV )% 0 0 UHd +Fv0$74?jj!23O"$&& rTc V^8dQhR\PR\PR\\P,R\ \ \3,R\/#)rVrfull_input_idsmaskrrX) rrrrr rr:rr)rYs"rRrZrZsS%% 99%HH% 288 % cN %  %rTc\VRR4p\VRR4p\V4'gR#\VR4'g\VR4'gR#V!VVPRR4VPRR4V4wrg\TR4'dYdn\TR4'dYtnYsR &R# \d"p\ P R T4Rp?R #Rp?ii;i) abPrime Qwen-style mRoPE metadata before a cached-prefix trim. Qwen VL language models keep ``_rope_deltas`` on the model object and use it when continuing from a non-empty KV cache. If APC trims the prompt to only the uncached suffix, the suffix alone is not enough to recompute the original prompt's RoPE delta, so derive it from the full prompt first. language_modelNget_rope_indexT _rope_deltas _position_idsimage_grid_thwvideo_grid_thwzJCould not prime cached-prefix RoPE state; falling back to cold prefill: %sF rope_deltas) rcallablehasattrr Exceptionloggerwarningrr) rrrrlmr position_idsres &&&& rR_prime_cached_prefix_rope_staters ($ /BR!148N N # # B ' '72+G+G $2  JJ' . JJ' .  % ! r?##'r>""%'=   X   s-B88 C$CC$) generate_stepcfV^8dQhR\PR\R\R\\\ \,3,R\\\ \,3,R\\\ \,3,R\\\ \RR3,3,/#) rVr processorpromptr0audiovideorXN)rrrr:r rr)rYs"rRrZrZsvv 99v"v v d3i v d3i v d3i v 3 #tT/* *+vrTc+<#aXaYaZa["\VR4'd VPMTpVPRR4pVPRR4p VPR\4p VPR\4p VPRR4p VPR R4p V 'd(\VR 4'd\ VP 4M.pVPPRY9d\VR R4RJMR p\VPR R44p\VPRR4pVPRR4pVPRR4pVPRR4oYVPRR4pT;'gRpT;'gRpT;'gRpVPRR4e7VPR4pVPRR4pVPRR4pM\V3RVRVRVRVRVR VRV/VBpVPRR4pVPRR4pVPRR4pVP4UUu/uFwppVRZ9gKVVbK pppVPV4\W4'EdVf VfVe \!R4hVPR\"4pVPR\$4pVPR\&4pVPR \(4pVPR!4p V f+\VR"R4p!VPR#\V!R$^ 44p \VR"R4p!/p"R[F3wp#p$VPV#4p%V%f\V!V$R4p%V%fK.V%V"V#&K5 /p&0R\mp'VP4U#U%u/uFwp#p%V#V'9gKV#V%bK p(p#p%\*P,!4p)VP.P0!V3RTR%VPR%^ 4R#T R&TRVeVR'8dRMTR VeV^8:dRMTR(R R)VRVR V R*V&/ V"BV(Bp*\2P4!V*4\*P,!4V), p+V&PR+R,4p,V,^8dVP6V,, MR,p-\9V+V,, R-4p.V*^,P;4p/VP=V/V R.7p0\?T0V/'d V/R],MRRVP6\AV/4VP6\AV/4,T-\AV/4V., \2PB!4R/, V/'d!VPEV/R],4'dR0MR1\GV&PR244R37 xR#VetVepVelVPV4p1V1eV1VR4&MP\VR54'd?VPIV4p2\2P4!V24VPKVV24V2VR4&^p3VPM4P;4oZ.p4^oXRp5\NPP!VP4o[\NPR!SZS[4p6\9^V6^, 4p7R6VZV[3R7llp8R8VZV[3R9llp9\UV4'd\WVVVVVVVV4RjxL R#SYe'\NPX!VP.4p5V5fRoYSYe0\NPZ!VVR:7p:\NP\!VV:4oXVEeBVP^Ee3VPaSZ4p;V;^8EdV;VPb^,8EdV8!V;4'd\eVVVV4'dT;p3VR;V;R13,pRpVPR4R4VP^pV>V;8gKMV=PfR;R;RV;1R;3,V=n3V=PhR;R;RV;1R;3,V=n4\V=R=4'gKV;V=n5K V&SYEeFV3^8XEd>V5R?8XdSYPmSZSXV7R@7wp?p@V?ejX@^8dcX@VPb^,8dKV8!X@4'd=\eVVVV4'd)X@p3VR;V@R13,pRpVPR4R4V?VR>&EMSYPoSZSXRA7wpAp;V;^8d$V9!V;4'dSYPqXA4.pA^p;Rp?^p@V;VPb^,8d"SYPmSZSX\9V;V74R@7wp?p@RpB^pC\9V;X@4VPb^,8d1SYPsSZSX\9V;X@V74\9V;V@4^8RB7wpBpCXC\9V;X@48d~XCVPb^,8dfXA'dSYPqXA4V8!XC4'd=\eVVVV4'd)XCp3VR;VCR13,pRpVPR4R4XBVR>&EMOX@V;8d}X@VPb^,8deXA'dSYPqXA4V8!X@4'd=\eVVVV4'd)X@p3VR;V@R13,pRpVPR4R4V?VR>&MV;^8dV;VPb^,8dV8!V;4'du\eVVVV4'daXAp4T;p3VR;V;R13,pRpVPR4R4\NPt!VAV;VPb^,,^,RC7VR>&M+SYPqXA4MXA'dSYPqXA4V ekVPwV RRD7R],pDT ;'d"XDVPM4P;49p \yVV V V V RE7Vn=VPzVRF&MRVn=R>V9d6\^P|!VP.VPRGR4RH7VR>&VR>,pEV3VP6,pF\V\.4;_uu_4\V4pG\VRFR4pHRpIRpJSYegV5R?8Xd`V3^8XdY\NP!SZ\ASZ4SYP, S[\ASZ4^, RI7pIVI^8:dRpIRJVXVYVZ3RKllpJ\VVVV3RLXJRMXIRV/VBpK\*P,!4p).p/RpL\VK4EF5wpMwpNpOVM^8Xdb\*P,!4V), p,XFV,, p-\*P,!4p)SYe$V5R?8XdV3^8XdSYPSZXESXRA7V/PXN4XHe XH!XN4VPEXN4'dR0pLMXGPXNVRO7\?VGPVNXOXFXM^,VFVM,^,X-VM^,\*P,!4V), , \2PB!4R/, V3RP7 xEK8 R1pLV/'gl\*P,!4V), p,V,^8d XFV,, MR,p-\?RQRRXF^VFV-R,\2PB!4R/, V3R1RR7 xRRR4R#XGP4\?VGPXNXOXFXM^,VFVM,^,X-VM^,\*P,!4V), , \2PB!4R/, V3XLRR7 xRpQVeNSZV/URu.uF'pR\VRRS4'dXRP4MXRNK) upR,pQVPVQXE4SYEeV5RT8XEdXQf<SZV/URu.uF'pR\VRRS4'dXRP4MXRNK) upR,pQ.pS.pTR pUXEFtp=\V=RXSPVRVRXV1R;3,4XTPVRVRVV1R;3,4Kv XU'd8XS'd0SYPXQXSXTSXV3RW7pWSYPqV4VW,4MSYPqV44\2P!4RRR4R#uuppiuup%p#iE Ln \d"pP\PRNTP4RpP?PELRpP?Pii;iuupRiuupRi \d2pP\PRXTP4SYPqT44RpP?PLRpP?Pii;i +'giR#;i5i)^aJ A generator producing text based on the given prompt from the model. Args: model (nn.Module): The model to use for generation. processor (PreTrainedTokenizer): The tokenizer/processor. prompt (str): The input prompt text. image (Union[str, List[str]], optional): Image path(s) or URL(s). audio (Union[str, List[str]], optional): Audio file path(s). prefill_step_size (int, optional): Number of tokens to process per prefill step. When set, enables chunked prefill which processes long prompts in smaller chunks to reduce peak memory usage. kwargs: Additional options passed to :func:`generate_step`. See :func:`generate_step` for more details. Yields: Generator[GenerationResult]: A generator producing GenerationResult objects containing the generated text, tokens, and statistics. tokenizerverboseFthinking_budgetNthinking_end_tokenthinking_start_tokenenable_thinkingskip_special_tokensall_special_ids chat_templateT resize_shapeimage_token_index vision_cacheprompt_cache_state apc_manager apc_tenant input_ids pixel_valuesrimagesrvideospromptsadd_special_tokensattention_maskz/Diffusion text generation models are text-only. max_tokens temperaturetop_ptop_kmax_denoising_stepsrstepsdefault_diffusion_steps block_length gen_lengthr eos_early_stop visualizestats prompt_timerg& .>)r eAstoplengthtext_already_printed) r'rrrrrrrrrr)cached_image_features encode_imagec0V^8dQhR\R\/#rV prefix_lenrXr<r)rYs"rRrZ%stream_generate..__annotate__s  S T rTc4<\P!SVS4#r])_apcprefix_leaves_text_only_suffixr.full_input_ids_listmultimodal_token_idss&rR_apc_suffix_is_text_only1stream_generate.._apc_suffix_is_text_onlys22     rTc0V^8dQhR\R\/#r-r/)rYs"rRrZr0s    rTc4<\P!SVS4#r])r2prefix_contains_media_tokensr4s&rR_apc_prefix_has_media_tokens5stream_generate.._apc_prefix_has_media_tokenss00     rT)r image_ref:NNNkeysrvrexact) extra_hashmin_prefix_tokensrA)rArBallow_memory_overlap)min_capacity_tokens)r)rrrrrthinking_budget_criteria max_kv_size)rG)max_prefix_tokenscJV^8dQhR\R\\,RR/#)rVr.rrXN)r<rr)rYs"rRrZr0\s%  S S  d rTc6<SPSRVVSR7R#)NrC)store_exact_cache)r.rapc_extra_hashrr5s&&rRexact_checkpoint)stream_generate..exact_checkpoint\s&--' 4 -.rTprompt_cache_checkpointprompt_cache_checkpoint_lenz APC exact-cache store failed: %s)skip_special_token_ids) r'rrrrrrrrrr) r'rrrrrrrrrritemblockrW.)rAskip_first_n_tokenszAPC store failed: %s)gemma3gemma3ngemma4gemma4_unified)rrr)) thresholddefault_diffusion_threshold) min_thresholddefault_diffusion_min_threshold)editing_threshold#default_diffusion_editing_threshold)num_to_transfer!default_diffusion_num_to_transfer)max_transfer_per_step'default_diffusion_max_transfer_per_step)max_post_steps default_diffusion_max_post_steps)stability_steps!default_diffusion_stability_steps>rrrrYrrr r[rcr_rer]rrar)QrrpoprOrNsetr r model_typerrirritemsrrrgrBrE DEFAULT_TOP_P DEFAULT_TOP_Ktime perf_counterrr1revalsizemaxtolistdecoderreget_peak_memorystopping_criteriarr+putflattenr2 multimodal_token_ids_from_configmedia_safe_prefix_minrrmodel_apc_modehash_image_payloadtenant_scoped_hashrrshaperr?rWrvlookup_exact_cache lookup_prefixreleaselookup_prefix_disk_cachemake_warm_kv_cacheencoderrFmake_prompt_cacherrr%adjust_prefix_to_text_suffix_boundaryexact_cache_guard_tokensrr{rKrrrappend add_token last_segmentfinalizerRstore_kv_blocks clear_cache)\rrrr0rrrrrrrrrr rQrr r rrrrrrinputskv data_kwargsrrrrrr tuned_kwargskey config_attrr_generation_statshandled_generation_kwargsmodel_generate_kwargstic generated total_timer%rgeneration_timegenerated_tokensr'cachedfeaturesreused_prefix_lenapc_blocks_in_useapc_modeapc_safe_prefix_minapc_safe_prefix_lookup_minr7r< image_hashr.rc cached_lenexact_prompt_cacheexact_prefix_lenmatched_blocksdisk_prompt_cachedisk_prefix_lenthinking_start_token_id tracked_cachetotal_prompt_tokens detokenizerthinking_criteriaexact_checkpoint_lenrMgenrnrrrall_idst layer_keys layer_valuesokoff new_blocksrLrr5r6s\&&&&&&, @@@@rRstream_generaters38(/y+'F'F ##IIjjE*Gjj!2D9O$8:TU!:: <jj!2E:O!**%:EB 796G#H#H I % %&  << " "&W W  ?D1T9  *&**^T*JKL .A4H::nd3L$8$?-3ZZ t-LK & < >J MMTE MMTE MMTE zz+t$0JJ{+ zz.$7 zz&$'           0  &   2    JJ{D1 zz.$7 zz*D1  &1GG AqD&   k"&u55   1U5FNO OZZ .@A jj0CD  7M2 7M2$jj)>?  &UHd3F"(**)BBG# $/  ! CJJsOE} T: $) S!! % !$%lln! , U33 CJ, ! !((11  #  NB7 &  "   -5C<$U   -5A:$5      !4 #  $  &&(3. &**=#> 5@1_Y^^k1# j;6=$Q<..0 2E  *:"2&#..!"23"#.>*??!/0?B**,s2$//0@0DEE!%&6&:&:;Q&R!S!  $ E$5,:R!!%(  .4F* + UN + +)),7H GGH    UH -.6F* +#++-446-/N"H@@N44"%Q(;a(?!@    %  8       "    &&u';';<  K,,,RWX 00ZH%*<*B*B*N'::;NO >j9??1+==' 338Wy$99%/!%an5 #  2D9-33!Aq&))aff.@%&VV\\!_ % 2%&VVAq+:+q,@%AAF'(xx1kzk10D'EAH&q(33+5"*2~&#4#9 w 3>3Q3Q#)"<4R4 0  0 #.$q($yq'99,-=>>3E9dFSS$4!%a)9):&:; #  2D9);~&)4)B)B#*C* &NJA~">z"J"J##N3!# !%  IOOA..7B7U7U'-&)*6P&Q8V84"$4 !% O:/09??13EE88+#1*-&,6+ .1=M-NQR-R9 3!? #j2B"CC#iooa&88!''7+#5eYfUU(7% )!_-=*= >I#'LJJ6=->F>* :-2BY__UVEW2W!''7+$5eYfUU(8% )!-=->*> ?I#'LJJ6=-?F>*aJ1C$C+5eYfUU(6%(2% )!Z[. 9I#'LJJ6=-1-D-D&,69K,Ka,O.F>*  ''7##N3""+"2"2 U#3# #*  #y'8'8':'A'A'C C .D+1!5+ . *.7-O-O)*-1 *V#!&!8!8   =$7" ~>*M+inn< U./ 0 00; #I/I4P#  "x7':?PTU?U#'#M#M#'(;+O+OO$"%&9":Q"> $ $q('+$        %5  )=      !'+ $-cN A xAv"//1C7 0;> '')+ G+)Q.N#55/)'56  # #E *!,!%(**511 &   ! !%@V ! W# --!1"#a%014q8% !A$*;*;*=*CD..036/  C%3^%M++-3K>IAo,{:SVJ"1"#0%"..036/&   K 1 0N ))-!e,q014!Ed&7&7&9C&?@**,s2+'  (,  ))?O-?O!GAv..A5?O-G  % %g} =  "x7': 7?1GW5GW!GAv$6$6A=GW5G.0 /1 &A640A8T2A!!Xt4CyAI"%%aTcT1 o6 ''#ttQ,8'*!,!R R24\-R)/RV bRV P@bRV PBbRV P6bRV P:bRV PDbRV P8bRV P<bRV P>bR V PFbR!V PHbR"V PJbR#V PLbR$V PNbR%V PPbR&V PRb#)*a Generate text from the model. Args: model (nn.Module): The language model. tokenizer (PreTrainedTokenizer): The tokenizer. prompt (str): The string prompt. temperature (float): The temperature for sampling (default 0). max_tokens (int): The maximum number of tokens (default 100). verbose (bool): If ``True``, print tokens and timing information (default ``False``). formatter (Optional[Callable]): A function which takes a token and a probability and displays it. repetition_penalty (float, optional): The penalty factor for repeating tokens. repetition_context_size (int, optional): The number of tokens to consider for repetition penalty. NzFiles:  zPrompt:r eos_tokensrurzGstopping_criteria must be an instance of StoppingCriteria or a callablerT)endflushr&)r'rz!No text generated for this promptzPrompt: z tokens, z.3fz tokens-per-secz Generation: z Peak memory: z GBr'rrrrrrrrrrdiffusion_canvas_tokensdiffusion_denoising_stepsdiffusion_work_tokensdiffusion_canvas_tpsdiffusion_work_tpsz ==========z ==========rp)*rextendrcrzrrrrruadd_eos_token_idsrrrgresetr eos_token_idris_draft handle_draftr) handle_textr'rrrtfinishrerrrrrrrrrrrrrrr)rrrr0rrrrfilesr' last_responserrurdiffusion_outputresponses&&&&&&&, rRr1r1s6 h   LL    LL    LL*UD"9"9w G w& i DML$/J #6=(/y+'F'F ##II-eWE##55jA  & ')9 : :h ? ? +< 'Y  ##))%,,*C*CD# &?FJP      ) )( 3$M  111$00?? (--Rt 4  "Tr7I7I7Kc7QRR% o t9> 5 6 }2239'',O =  =::;9++C0 A   m77&VP|eVP|VR?&V'EdVP~eVP~VR&VPeVPVR&VPeVPVR&VPeVPVR&VPeVPVR &VPeVPVR &VPeVPVR&VPeVPVR&VPeVPVR&VPW\W44\VVR@4p\VVV VPVPVP3/VBFupVP'dVPV4K(VVP, pVPVP4'dK^\3VPR2R47Kw VPMR!RAR#V/4VPV4\34EKR#/RVPbRBVPbR%VPbR&VPBbR6VPlbR5VPjbR7VPnbR8VPpbR9VPrbR:VPtbR;VPvbR&VP|eVP|VR?&\gV4'EdVP~eVP~VR&VPeVPVR&VPeVPVR&VPeVPVR&VPeVPVR &VPeVPVR &VPeVPVR&VPeVPVR&VPeVPVR&VPW\W44Ve2VVRI&VP4VRJ&VPeVPVRK&\VVV 3/VBpVP'g\3VP4Ve\V4pVe\3V4R#R#R# \6d#p \3R T 24RpRTnRp ? EL~Rp ? ii;i \N\P3dTPMT4EKi;i)Loutput_modalityr'r0Nseedrdiffusion_full_canvasFdiffusion_min_canvas_lengthdiffusion_max_canvas_lengthdiffusion_samplerr.rYr[r r_rar]rcre gen_kwargs)revisiontrust_remote_codequantize_activations) load_draftervalidate_drafter_compatibilityzLoading drafter (autoz): )kindu! → auto-detected --draft-kind=r-u$ → drafter requires --draft-kind=z; using z instead of zfSpeculative drafter is incompatible with the target model; falling back to autoregressive generation. rolesystemcontentrrfps num_images num_audiosunicode_escaperr rrr)VisionFeatureCachezUser:userrz Assistant:)rrrrepetition_penaltyrepetition_context_sizepresence_penaltypresence_context_sizefrequency_penaltyfrequency_context_sizerr prefill_step_sizeT assistantrrrGr~rrkr draft_model draft_kinddraft_block_size)YrSrrrrandomrrjrsetattrrcr0r:rrrr adapter_pathrrrrrspeculative.draftersrrrrrgrrrzrerrrrcodecsrsrUnicodeDecodeError UnicodeErrorr processor_kwargsrrrrrchatrrrinputrrrrrrrrr rrr r_rarYr[r]rcrerrrrrr'rrrrGr~rrHrrr1r) argsdiffusion_arg_defaultsnamer%rrrrrr resolved_kindrrrrchat_template_kwargsrrr decoded_tokenrris_masked_text_diffusionrrr stream_kwargsrchunkrresultr$s rRr/r/ms  Dt&/7: &tVT". tyy! t%t%t_T4T$4b 0557 t"" D (8$**c""jj\ $**c""jj\ $**c""jj\   00!66 E\\FK #W !$//";";V!'>#$  $ $ 0-1-F-FF) * yyy5)+ #A%#H  ;;; KK9dkkB CGn$d$ KKD9 :(&  &  ' FH ,B 'doot//$d&=&=)4+G+G"D$9$9')C)C#T%;%;($*E*E   M  ,040A0A n-%%1595K5K 12''++7;?;S;SM"78$$0484E4EM.1''37;7K7KM"34--9=A=W=WM"9:>>-15M+.%%1595G5GM/2))59=9O9OM"56&&26:6I6IM"23''37;7K7KM"34  !;D!I J5e]DQ (    >>>$11%8EJJ&'33EJJ??%**"- KKiB C  # #H - GI%N TZZ TZZ  TZZ  488  4++  $//   !$"9"9  &t'C'C   5 5  $T%?%?  !7!7  %d&A&A  t||  4++  t|| T//! " w')@ # ( !$"9"9) *+ .    ()-):):J~ &  ! ! -.2.D.DJ* + )% 0 0''3484L4L 01  ,-1->-> >*##/040D0D ,-))56:6P6P 23~~)*... ;'!!-.2.@.@ ?+%%1262H2H ./""./3/B/B +,##/040D0D ,-4TBC  "(3J} %'+J| $$$0151F1F -.       ||| &++   ",[9E e ! #Y # >>?SB K"DOO  #P' 5 )!!%(( )s*k;(k6 k3k..k36"ll__main__zCalling `python -m mlx_vlm.generate ...` directly is deprecated. Use `mlx_vlm generate` or `python -m mlx_vlm generate` instead.r])NNN)NNNF)ir7r contextlibrKloggingrmcollections.abcr dataclassesrtypingrrrrr r r mlx.corecorermlx.nnr mlx.utilsr mlx_lm.generater r transformersrrrr2modelsr prompt_utilsrspeculative.utilsrtokenizer_utilsrr3rrutilsrrrrr0rrrrr getLoggerrr;r>r?r@rArBrErk DEFAULT_SEEDrl DEFAULT_MIN_PrFrIrHDEFAULT_COMPLETION_BATCH_SIZEDEFAULT_PREFILL_BATCH_SIZErNrOrJrMrDrCrSrinew_thread_local_streamdefault_devicercontextmanagerrrrcommon diffusionrrrrrrrrrrrr1r/rrrprTrRr$s   $!EEE!R,.88>RR   - .7   "    "$# "('! &(#(*%Yx I ..r/@/@/BC3 1h "&"&J  ( (  ( 8ED, %PvrxvOd z  K F rT