+ G-j3^RIt^RIt^RIt^RIt^RIHtRRltRtRtRt Rt RRlt R t R t R t]R 8Xd]!4t]!R ]4^RIHt]!]P(4]P*'g#]P,'gQR4h]!]4tM ]P*t^RIHt]!]]4t]F t]!]4K R#R#)N) TensorProtoc `\P!4pVPRRR\RR7VPRRR\RR7VPRR R\^R R 7VPR R R\^ RR 7VPRR\^RR 7VPRR\^RR 7VPRR\RRR 7VPRR\ RRR 7VPRR\R9RR 7VPRR\RRR 7VPRR\RRR 7VPR R\RR!R 7VPR"RR#.R:OR$R%7VPR&R'RR(R)R*7VP RR+7VPR,R\R-R.R 7VPR/RR(R0R*7VP RR17VPR2RR(R3R*7VP RR47VPR5R6RR(R77VP RR87VPV4#);z-iz--inputFz2Set the input file for reading the profile results)requiredtypehelpz-mz--modelzIonnx model path to run profiling. Required when --input is not specified.z-bz --batch_sizezbatch size of input)rrdefaultrz-sz--sequence_lengthzsequence length of inputz--past_sequence_lengthzpast sequence length for gpt2z--global_lengthz&number of global tokens for longformerz --samplesiz\number of samples to test. Set it large enough to reduce the variance of performance result.z --thresholdg{Gz?zfThreshold of run time ratio among all nodes. Nodes with larger ratio will show in top expensive nodes.z --thread_numznumber of threads to usez--input_ids_nameNz"input name for input IDs, for bertz--segment_ids_namez$input name for segment IDs, for bertz--input_mask_namez'input name for attention mask, for bertz--dummy_inputsrzEType of model inputs. The default will create dummy inputs with ones.)rrchoicesrz-gz --use_gpu store_truezuse GPU)ractionr)use_gpuz --providercudazExecution provider to usez--basic_optimizationz_Enable only basic graph optimizations. By default, all optimizations are enabled in OnnxRuntime)basic_optimizationz--kernel_time_onlyz.Only include the kernel time and no fence time)kernel_time_onlyz-vz --verbose)rr )verbose)bertgpt2 longformerr)argparseArgumentParser add_argumentstrintfloat set_defaults parse_args)argvparsers& r/Volumes/fast/ai/experiments/nudenet-smoke/.venv/lib/python3.14/site-packages/onnxruntime/transformers/profiler.pyparse_argumentsr s  $ $ &F    A     X     "     '     ,    5    k    u    '    1    3    6  9 T  kE,U^_ &   (   n   51  =   / kE,O &   T ""c ^RIHpV!VVVV'*VRR7pVFpVPRV4p K VP4p V #)r)create_onnxruntime_sessionT)enable_all_optimization num_threadsenable_profilingN)benchmark_helperr#run end_profiling) onnx_model_pathr providerr thread_num all_inputsr#sessioninputs_ profile_files &&&&&& r run_profiler2sV;($6 6 G KKf %((*L r!c\VPR44\8Xd\WPR44#R#)valueN)r WhichOneofrgetattr)dims&rget_dim_from_type_protor8s2489P4QUX4X73w/ 0b^bbr!c~VPPPUu.uFp\V4NK up#uupiN) tensor_typeshaper7r8) type_protods& rget_shape_from_type_protor?s40:0F0F0L0L0P0P Q0P1 #A &0P QQ Qs:c\/pVP4EFvp\VP4p.p\V4F.wr\ V \ 4'gKVP V4K0 \V4^8dR#\V4^8d WV^,&\V4^8d W&V^,&VPPPp V \P\P\P39gQhV \P8Xd\PM4V \P8Xd\P M\P"p \P$!WkR7p WVP&&EKy \)V4U u.uFqNK pp V#uup i)zCreate dummy inputs for ONNX model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples Returns: List[Dict]: list of inputs Ndtype)'get_graph_inputs_excluding_initializersr?r enumerate isinstancerappendlenr; elem_typerFLOATINT32INT64numpyfloat32int64int32onesnamerange) onnx_model batch_sizesequence_lengthsamples dummy_inputs graph_inputr< symbol_dimsir7rH data_typedatar0r-s&&&& rcreate_dummy_inputsr]s[L!IIK )+*:*:; &FA#s##""1%' { a  { a $.+a. ! { a $3+a. !$$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%1)-[%%&/L2).g71,J7 8s F)c P^RIHpHpV!WWV4wrp V!VVV^{RV V V RR7 p V #)aCreate dummy inputs for BERT model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length samples (int): number of samples input_ids_name (str, optional): Name of graph input for input IDs. Defaults to None. segment_ids_name (str, optional): Name of graph input for segment IDs. Defaults to None. input_mask_name (str, optional): Name of graph input for attention mask. Defaults to None. Returns: List[Dict]: list of inputs )find_bert_inputsgenerate_test_dataF) test_casesseedr input_ids segment_ids input_maskrandom_mask_length)bert_test_datar_r`) rSrTrUrVinput_ids_namesegment_ids_nameinput_mask_namer_r`rcrdrer-s &&&&&&& rcreate_bert_inputsrksG.D)9*Vf)x&IJ#   J r!c RVRVRVRW#,/p/pVP4EF8p\VP4p\V4F;wr\ V \ 4'gKW9d\ RV 24hWZ,W&K= VPPPp V \P\P\P39gQhV \P8Xd\PM4V \P8Xd\PM\P p \P"!WR7p WVP$&EK; \'V4Uu.uFqNK ppV#uupi)aCreate dummy inputs for GPT-2 model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length past_sequence_length (int): past sequence length samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs rTseq_len past_seq_len total_seq_lensymbol is not supported: rA)rCr?rrDrEr RuntimeErrorr;rHrrIrJrKrLrMrNrOrPrQrR)rSrTrUpast_sequence_lengthrVsymbolsrWrXr<rZr7rHr[r\r0r-s&&&&& rcreate_gpt2_inputsrt sI$ j?,? GL!IIK )+*:*:;&FA#s##%&)B3%'HII&|EH ' $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  zz%1)-[%%&#L&).g71,J7 8s- E;cPRVRV/p/pVP4EFjp\VP4p\V4F;wr\ V \ 4'gKW9d\ RV 24hWZ,W&K= VPPPp V \P\P\P39gQhV \P8Xd\PM4V \P8Xd\PM\P p RVP"9d"\P$!WR7p ^V RRV13&M\P&!WR7p WVP"&EKm \)V4Uu.uFqNK ppV#uupi)aCreate dummy inputs for Longformer model. Args: onnx_model (OnnxModel): ONNX model batch_size (int): batch size sequence_length (int): sequence length global_length (int): number of global tokens samples (int): number of samples Raises: RuntimeError: symbolic is not supported. Use the tool convert_longformer_to_onnx.py to export ONNX model instead. Returns: List[Dict]: list of inputs rTrUrpglobalrA:NNNN)rCr?rrDrErrqr;rHrrIrJrKrLrMrNrOrQzerosrPrR)rSrTrU global_lengthrVrsrWrXr<rZr7rHr[r\r0r-s&&&&& rcreate_longformer_inputsry;sf Z):OLGL!IIK )+*:*:;&FA#s##%&)B3%'HII&|EH ' $$00:: [.. 0A0A;CTCTUUUUK--- MM!*k.?.?!?%++U[[  {'' ';;u6D&'DN]N" #::e5D)-[%%&-L0).g71,J7 8s F#c VP^8d VPM\P!RR7pR\P9d\ V4\PR&^RIHp^RIH pV!V!VP44pRpVPR8XdO\VVPVPVPVP VP"VP$4pMVPR8Xd9\'VVPVPVP(VP4pMtVPR 8Xd9\+VVPVPVP,VP4pM+\/W@PVPVP4p\1VPVP2VP4VP6VPV4pV#) rF)logicalOMP_NUM_THREADS)load) OnnxModelNrrr)r,psutil cpu_countosenvironronnxr}rSr~modelrWrkrTrUrVrhrirjrtrrryrxr]r2r r+r)argsr%r}r~rSr-r1s& rr(r(js%)__q%8$//f>N>NW\>]K *(+K(8 $%$4 +,JJ F"'  OO  LL     ! !      f $'  OO   % % LL     l *-  OO     LL  )__dFZFZ\`\h\hi       L r!__main__ Arguments) setup_loggerzMrequires either --model to run profiling or --input to read profiling results)process_resultsr:)NNN)rrrLrrrr r2r8r?r]rkrtryr(__name__ argumentsprintr'rrinputrr1profile_result_processorrresultsliner!rrs I#X&cR'T&R-`,^3l z!I +y!-""# ???o oo9~   8lI6G d #r!