+ G-j:^RIt^RIt^RIt^RIt^RIt^RIt^RIHt^RIHt^RI H t ^RI t ^RI t ^RI HtHtHt^RIHt^RIt]P(!]4tRtRt!R R ]4tR tR tR t]R8Xd ]!4R#R#)N)deque)datetime)Path) ModelProto TensorProto numpy_helper) OnnxModelconstant_shape_opt__reshape_input_shape__caa]tRt^!toRtV3RltRtRtRtRt Rt RRlt RR lt R t R tV3R lR ltV3RlRltRV3RlRlltRtVtV;t#)BertOnnxModelShapeOptimizerz This optimizer will replace Shape output or the shape input of Reshape node by initializer. Currently, it requires model inputs to have static shape. c:<\SV`VP4R#)N)super__init__model)self onnx_model __class__s&&y/Volumes/fast/ai/experiments/nudenet-smoke/.venv/lib/python3.14/site-packages/onnxruntime/transformers/shape_optimizer.pyr$BertOnnxModelShapeOptimizer.__init__'s ))*c \P!V\PR7pVPR\4p\ P PV\PVPVR7pVPV4V#)z( Add an initializer for constant shape. dtypeConstant)name data_typedimsvals) npasarrayint64create_node_nameCONSTANT_SHAPE_NAME_PREFIXonnxhelper make_tensorrINT64shapeadd_initializer)rr) shape_valueconstant_shape_nametensors&& radd_shape_initializer1BertOnnxModelShapeOptimizer.add_shape_initializer*srjjbhh7 "33J@Z[(($!''"" )  V$ rc VP4p.pVPPPFRpVPR8XgKVP ^,V9gK0VP VP ^,4KT V#)z4 Returns a list of output names of all Shape nodes. Shape)input_name_to_nodesrgraphnodeop_typeoutputappend)rr2outputsr4s& rget_shape_outputs-BertOnnxModelShapeOptimizer.get_shape_outputs9sj#668JJ$$))D||w&;;q>%88NN4;;q>2* rc VP4.pVPPPF8pVPR8XgKVP VP ^,4K: V#)z7 Returns a list of shape input names of Reshape nodes. Reshape)output_name_to_noderr3r4r5r7input)r shape_inputsr4s& rget_reshape_shape_inputs4BertOnnxModelShapeOptimizer.get_reshape_shape_inputsGsY   " JJ$$))D||y(##DJJqM2*rc .p.pVPPPFpVPR8XgKVP^,pVP R\ 4p\PPRV.V.R7pVPV4VPV4K VPV4V#)zp For each Reshape node, create a Shape node for its first input. Returns the output names of these Shape nodes. r< Reshape_Inputr1)inputsr8) rr3r4r5r>r#RESHAPE_INPUT_SHAPE_PREFIXr%r& make_noder7 add_nodes)r output_names nodes_to_addr4r> output_name shape_nodes& radd_shape_for_reshape_input7BertOnnxModelShapeOptimizer.add_shape_for_reshape_inputTs   JJ$$))D||y( 1 "33OE_` ![[227E7U`Ta2b ##J/##K0 * |$rc .pVPPPUu.uFq3PNK ppVFpVP V4eKVP V4WT9gK1\ PP4pWVnVPPPPV.4VP V4K V#uupi)z- Add a list of output names to graph output. ) rr3r6rget_initializerr7r%r&ValueInfoProtoextend)r extra_outputsnames_to_evaluater6rHr output_infos&& radd_extra_graph_output2BertOnnxModelShapeOptimizer.add_extra_graph_outputfs26**2B2B2I2IJ2I 2I J!D##D)5  $ $T *'"kk88: #'    ''.. }=##D)"! KsCc  4VPPPFpVPV9gKVPP P P^,pW%nVPP P P^,pVPR4'd W5nKVPR4'gKVPV8wgK\RV R^ RVP RVP R2 4h R#) zO Update the model to use static axes instead of dynamic axes for graph inputs. dim_param dim_valuez!Unable to set dimension value to z for axis z of z'. Contradicts existing dimension value .N) rr3r>rtype tensor_typer)dimrYHasField ValueError)rrD batch_size max_seq_lenr> dim_protos&&&& ruse_static_input,BertOnnxModelShapeOptimizer.use_static_inputzsZZ%%++EzzV#!JJ2288<jj)Gjj)Gjj)G !^hhw'Ghhw'Ghhw'Gj;P rc lW#V.Vn.p V'd V PVP44V 'dCVP4p VP 4p V PV 4V PV 4\ V 4^8XdR#VP V 4pVPVPWg4\VR4;_uu_4pVPVPP44RRR4\P!4p\PPVn\P"!VVRR.R7p^pVPP$P&F6pVP(V8XgKVP*P,P.pK8 VP1W#WFVV4pVP3VV4p/p\5V4FwppVV,VV&K \6P9RV 24V 'd3\5X 4F#wppX V,pVP;VVVV 4K% VP=4F3wppVP?V4pVPAVVP(4K5 VPCV4R# +'giEL;i)rNwbCUDAExecutionProviderCPUExecutionProvider) providerszshapes=)" bert_inputsrQr9r@rLlenrUrcopenwriterSerializeToString onnxruntimeSessionOptionsGraphOptimizationLevelORT_DISABLE_ALLgraph_optimization_levelInferenceSessionr3r>rr[r\rtryrun enumerateloggerdebugupdate_target_shapeitemsr.replace_input_of_all_nodes prune_graph)rtemp_model_pathrprqrrrHr`rsenable_shape_optenable_reshape_optverboserRreshape_shape_inputsreshape_input_shapesrSout sess_optionssessionrtr>rDr8shapesir shape_input input_shaper)r-s&&&&&&&&&&& rshape_optimization.BertOnnxModelShapeOptimizer.shape_optimizationsK&J?   !7!7!9 : #'#@#@#B #'#C#C#E  !5 6  !5 6 }  "  77 F d.. L /4 ( (C IIdjj224 5)"113 0;0R0R0b0b -..  .0FG   ZZ%%++EzzY&!JJ22<< ,)))*Ziktu++/8 !23GAt"1:F4L4  wvh'( "+,@"A;215 ((k7S#B"<<>KD%//6F  + +D&++ >* &C) ( (s *J"" J3 c W!9d W,pM-VPV4pVfQh\P!V4pW19d W,pM-VPV4pVfQh\P!V4p.p\V4FFwrV \ V48d!Wy,V 8XdVP ^4K5VP V 4KH WV&\ PRV RV RV 24R#)z Update the target shape to use 0 to represent that dimension value does not change. For example, shape of source data is (2, 5, 8) and target shape is (2, 5, 4, 2), the target shape will be updated to (0, 0, 4, 2). Nz source_shape=z, target_shape=z, new_target_shape=)rOrto_arrayrrr7rr) rrrrr target_shape initializer source_shapenew_target_shaperrYs &&&&& rr/BertOnnxModelShapeOptimizer.update_target_shapes  !.L..{;K* **'00=L  !.L..{;K* **'00=L%l3LA3|$$I)E ''* '' 2 4 /{ }\N/,Obcsbtuvrc <V^8dQhRS[/#)r>)str)format __classdict__s"r __annotate__(BertOnnxModelShapeOptimizer.__annotate__ s``C`rcVPV4'gJVPPPUu.uFqPNK pp\ RX RV 24hR#uupi)zInput z% does not exist in the graph inputs: N)find_graph_inputrr3r>r Exception)rr> valid_namess&& rvalidate_input*BertOnnxModelShapeOptimizer.validate_input s^$$U++37::3C3C3I3IJ3I%::3IKJfUG+PQ\P]^_ _,JsA#c0<V^8dQhRS[S[,/#)rrH)listr)rrs"rrrseeT#YercVPPPUu.uFq"PNK ppVFpWC9gK \ RV RV 24h R#uupi)zOutput z& does not exist in the graph outputs: N)rr3r6rr)rrHr6rrs&& rvalidate_outputs,BertOnnxModelShapeOptimizer.validate_outputss[151A1A1H1HI1Hv{{1H I D&'$/UVaUb cdd!JsAcb<V^8dQhRS[RS[RS[RS[RS[RS[RS[S[,R,/#) r output_pathrprqrrrrrHN)rboolr)rrs"rrrsZ9:9:9:9: 9:  9:  9:!9:3i$&9:rc VPPPF@p V PP \ 4'gK*\ PR4R# VPV4VPV4VPV4Ve#VPV4VPV4VPPPU u.uFqPNK p p V'g V'Ed\VP44^8wd\ PR4R#\P!4;_uu_4pRP!\"P$!4P'R44pV 'dRMTp\(P*P-VV4pVP/VVVVV VV VVV 4 RRR4\ P1RX 24\ P3RV RV 24VeI\5VR 4;_uu_4pVP7VPP944RRR4R#R#uup i +'giL;i +'giR#;i) z5Skip shape optimization since it has been done beforeNz9Skip shape optimization since graph input number is not 3z temp_{}.onnxz%m_%d-%H_%M_%SrZz$Temp model with additional outputs: zZShape optimization is done. The optimized model might only work for input with batch_size=z sequence_length=r|)rr3rr startswithr$rinforrrr6r'get_graph_inputs_excluding_initializerstempfileTemporaryDirectoryrrnowstrftimeospathjoinrrwarningrrr)rrrprqrrrrrHr`rsrr-r6remaining_outputstemp_dirtemp_file_namedir temp_filers&&&&&&&&&&& roptimize$BertOnnxModelShapeOptimizer.optimizes jj&&22F{{%%&@AA ST3 I& K( J'  #  ! !, /   \ *7;zz7G7G7N7NO7NV[[7NO 114??ABaG WX,,..(!/!6!6x||~7N7NO_7`!a$c(GGLLn= ''%#$&  / LL? {K L NNlmwlxyJKZJ[\   "k4((C $**6689)( #9P/.,)((s IA:I!&*I4! I1 4 J )r)rg))NrgrF)__name__ __module__ __qualname____firstlineno____doc__rr.r9r@rLrUrcryrrrrr__static_attributes____classdictcell__ __classcell__)rrs@@rr r !sg +   $!( BB'Hw:`` ee 9:9:9:rr c\P!4pVPRR\R7VPRR\R7VPRR\R7VPRR\R7VPRR\R7VPRR\R R 7VPR R\^R 7VPR R\^R 7VPR RRR7VP RR7VPRRRR7VP RR7VPRRRR7VP RR7VP 4pV#)z--inputT)requiredr[z--outputz --input_idsz --segment_idsz --input_maskz--output_namesFN)rr[defaultz --batch_sizez--sequence_lengthz--enable_shape_opt store_true)raction)rz--enable_reshape_opt)rz --verbose)r)argparseArgumentParser add_argumentrint set_defaults parse_args)parserargss rparse_argumentsrSsG  $ $ &F  Ds;  T<  3? $SA C@ (5sDQ S!L +e#sS ,u\R / .|T 51  eLI &    D Krc\P!\P4pV'd7VP \P !R44\P pM5VP \P !R44\PpVPV4\PV4\PV4R#)z8[%(filename)s:%(lineno)s - %(funcName)20s()] %(message)sz%(filename)20s: %(message)sN) logging StreamHandlersysstdout setFormatter FormatterDEBUGINFOsetLevelr addHandler)r log_handler logging_levels& r setup_loggingrgs'' 3K  !2!23m!no   !2!23P!QR ' k" OOM"rc t\4p\VP4VPfRMVPP R4p\ 4p\ VPR4;_uu_4pVPVP44RRR4\V4p\V4pVPVPVPVPVP VP"VP$VVP&VP(VP4 R# +'giL;i)N;rb)rrrrHsplitrrr>ParseFromStringreadr r rr6rprqrrrrr`rs)rrHr input_filer optimizers rmainrts  D$,,,,44$:K:K:Q:QRU:VL LE djj$  : joo/0 5!J+J7I            s 3 D'' D7 __main__)rrrrerr collectionsrrpathlibrnumpyr r%rrrrr r getLoggerrrr$rEr rrrrrrs  66    8 $34o:)o:d ( #6 zFr