+ TV-j/+|^RIt^RIt^RIt^RIt^RIt^RIt^RIHt^RIH t ^RI H t ^RI Ht^RIt^RIt^RIHt^RIHtHt^RIHtHtHtHt^RIHtHtH t H!t!^RI"H#t#H$t$H%t%]PLt']'PQR]PR!R ]PT4]+!R 44/R R bR RbRRbRRbRR/R/R/R/R//bRRbR^bR^bR^bRRbR^bR R!bR"^ bR#^bR$RbR%R&bR'^dbR(RR)R*R+R,R-RR.RR/^R0^R1RR2R3^R4R5R6R7/R8RR9RR:R/ Ct,R;t-RER<R=llt.R>R?lt/RER@RAllt0RBt1]2RC8Xd]3!RD4]1!4R#R#)FN)Path)get_reporting_callbacks) CacheDataset load_dataset) TrainingArgsTrainingCallbackevaluatetrain)build_schedulelinear_to_lora_layers load_adaptersprint_trainable_parameters) _parse_sizeload save_configztag:yaml.org,2002:floatz^(?: [-+]?(?:[0-9][0-9_]*)\.[0-9_]*(?:[eE][-+]?[0-9]+)? |[-+]?(?:[0-9][0-9_]*)(?:[eE][-+]?[0-9]+) |\.[0-9_]+(?:[eE][-+][0-9]+)? |[-+]?[0-9][0-9_]*(?::[0-5]?[0-9])+\.[0-9_]* |[-+]?\.(?:inf|Inf|INF) |\.(?:nan|NaN|NAN))$z -+0123456789.modelzQwen/Qwen3-0.6br Ffine_tune_typelora optimizeradamoptimizer_configadamwmuonsgd adafactordatazmlx-community/WikiSQLseed num_layers batch_sizeitersi val_batches learning_rategh㈵>steps_per_reportsteps_per_evalresume_adapter_file adapter_pathadapters save_everytest test_batchesimax_seq_lengthiconfiggrad_checkpointgrad_accumulation_stepsclear_cache_threshold lr_schedulelora_parametersrankdropoutgscaleg4@ mask_prompt report_to project_namec\P!RR7pVPR\RR7VPRRRRR 7VPR \R R7VPR \.R>OR R7VPR\.R?ORRR7VPRRRRR 7VPR\RR7VPR\RR7VPR\RR7VPR\RR7VPR\ RR7VPR\RR7VPR \R!R7VPR"\R#R7VPR$\R%R7VPR&\R'R7VPR(\R)R7VPR*RR+RR 7VPR,\R-R7VPR.\R/R7VPR0R1\R2R7VPR3RR4RR 7VPR5\ ^R6R77VPR8\RR9R77VPR:\RR;R77VPR<\R=R7V#)@zLoRA or QLoRA finetuning.) descriptionz--modelz;The path to the local model directory or Hugging Face repo.)typehelpz--train store_truez Do trainingN)actionr;defaultz--datazuDirectory with {train, valid, test}.jsonl files or the name of a Hugging Face dataset (e.g., 'mlx-community/wikisql')z--fine-tune-typez4Type of fine-tuning to perform: lora, dora, or full.)r:choicesr;z --optimizerz>Optimizer to use for training: adam, adamw, sgd, or adafactor.)r:r?r>r;z --mask-promptz)Mask the prompt in the loss when trainingz --num-layersz=Number of layers to fine-tune. Default is 16, use -1 for all.z --batch-sizezMinibatch size.z--iterszIterations to train for.z --val-batchesz@Number of validation batches, -1 uses the entire validation set.z--learning-ratezAdam learning rate.z--steps-per-reportz0Number of training steps between loss reporting.z--steps-per-evalz-Number of training steps between validations.z--grad-accumulation-stepsz;Number of steps to accumulate before each optimizer update.z--resume-adapter-filez?Load path to resume training from the given fine-tuned weights.z--adapter-pathz*Save/load path for the fine-tuned weights.z --save-everyz"Save the model every N iterations.z--testz'Evaluate on the test set after trainingz--test-batchesz8Number of test set batches, -1 uses the entire test set.z--max-seq-lengthzMaximum sequence length.z-cz--configz3A YAML configuration file with the training optionsz--grad-checkpointz0Use gradient checkpointing to reduce memory use.z--clear-cache-thresholdz>Clear the allocator cache between steps if it grows too large.)r:r>r;z --report-tozDServices to report logs to ('wandb', 'swanlab', or 'wandb,swanlab').z--project-namezEProject name for logging. Defaults to the name of the root directory.z--seedz The PRNG seed)rdorafull)rrrrr)argparseArgumentParser add_argumentstrintfloatr)parsers \/Volumes/fast/ai/experiments/ui-tars-smoke/.venv/lib/python3.14/site-packages/mlx_lm/lora.py build_parserrJQs(  $ $1L MF   J       H   ( C    = M   8     L  S7HI  2LM   O  )R7 pVP@'d\CVP@4M VPDp VPFPI4p VPJPMV /4p V R8Xd\NPPp MnV R8Xd\NPRp MVV R8Xd\NPTp M>V R8Xd\NPVp M&V R8Xd\NPXp M\RV 24hV !RRV /V Bp \[VVV \]V4\]V4VR7R#)zRequested to train z layers but the model only has z layers.rANr@)use_doraz Received unknown fine-tune-type z Loading fine-tuned weights from F)strictT)parentsexist_okzadapters.safetensorszadapter_config.json) rr r!r#r$steps_per_save adapter_filer+r-r.rrrrrzUnsupported optimizer: r")rargsr train_dataset val_datasetrN)rr@)/mxrandomrfreezerlenlayers ValueErrorrmaxunfreezer1r r%print load_weightsrrr&mkdirrvarsrrr r!r#r$r(r+r-r.r0r r"rlowerrgetoptimAdamAdamWMuonSGD Adafactorr r)rZr train_set valid_setrNlr&rY training_argslroptimizer_namer opt_classopts&&&&& rI train_modelrzsIINN499 LLN U\\**!$//!23&&)%,,&7%8 B   f$s4??A6689A JJL: $   0 0  OO  ))V3  ;D2BCDD  9" 9(8 9C  "9- ++ rKc8V^8dQhR\P/#)rMr)rOrP)rQs"rIrRrR1s B B BrKc\V\V4VPVPVPR7p\ P !V4p\RVR RVR R24R#))rdatasetr num_batchesr+z Test loss z.3fz , Test ppl .N)r rrr*r+mathexprf)rZrtest_set test_losstest_ppls&&& rIevaluate_modelr1s[X&??%%** Ixx "H Jyo[#a @ArKc$V^8dQhR\/#)rMrN)r)rQs"rIrRrR?s..!1.rKc\PPVP4\VPVP VP \V4R7p\R4\VPRR/R7wr#\R4\W4wrEpVP'd:VP'g(VP R8wd\W P 4M6VP'd\R4\WWEV4M \!R4hVP'd\R 4\#WV4R #R #) )r7log_dirr,zLoading pretrained modeltrust_remote_codeT)tokenizer_configzLoading datasetsTrainingz.Must provide at least one of --train or --testTestingN)npr_rrr6r7r&rirfrrrr)r r rzrcr)rZrNr tokenizerrrrsrs&& rIrunr?sIINN499/ &&!!Dz  $%DJJ:Mt9TUE %1$%B"I( yyy    " %!2!2 3  jD7HIIJJ yyy itH-rKc^R\PR&\4pVP4pVPp\ V4pV'dx\ RV4\VR4;_uu_4p\P!V\4pRRR4VP4FwrEVPVR4eKWQV&K! \P4FwrEVPVR4eKWQV&K! \\P !R/VB4R# +'giL;i)trueTOKENIZERS_PARALLELISMzLoading configuration filerNr])osenvironrJ parse_argsr,rirfopenyamlr yaml_loaderitemsrkCONFIG_DEFAULTSrtypesSimpleNamespace)rHrZr,filekvs rImainr^s+1BJJ'( ^F    D [[F :D  *F3 &#  $YYt[1FLLNDAxx4 (Q#  %%' 88At  $G(%%& s -D D, __main__zwCalling `python -m mlx_lm.lora...` directly is deprecated. Use `mlx_lm.lora...` or `python -m mlx_lm lora ...` instead.)N)4rBrrrerwarningspathlibrmlx.corecorer^mlx.nnrOmlx.optimizers optimizersrlnumpyrrtuner.callbacksrtuner.datasetsrr tuner.trainerrrr r tuner.utilsr r r rutilsrrr SafeLoaderradd_implicit_resolvercompileXlistrrJrzrrr__name__rfr]rKrIrs  46JJ 21oo !!JJ     $ $ U$f$ $  rR  $ #$ A$"$!$  T!$"2#$$T%$&'$(c)$*4+$,J-$.#/$0 ECd duqQ49c7DA5DG$NDNVr B.>', z  H F rK