+ G-j ^RIHt^RIt^RIt^RIt^RIt^RIHt^RIH t ^RI t ^RI t ^RI H t HtHt^RI HtHtHt^RI Ht^RIHtHtHtHt^R IHt^R IHtHtHt^R I H!t!R t#R t$Rt%Rt&Rt'Rt(Rt)Rt*Rt+Rt,/t-].!]4Uu/uF%p]/!]0!]V4]14'gK]0!]V4VbK' upt2!RR]4t3!RR]4t4!RR]4t5!RR]4t6]P"Pn] Pp!R4]P"Pr] Pp!R4]P"Pt] Pp!R 4]P"Pv] Pp!R!4]P"Px] ]P"Pz]]P"P|]/t?]P"Pr] P!^] PR"7] P!^] PR"73]P"Pn] P!Rh] PR"7] P!^] PR"73]P"Pv] P!^] PR"7] P!R#] PR"73]P"Pt] P!Ri] PR"7] P!R$] PR"73]P"P|] P!^]R"7] P!^]R"73]P"Pz] P!Rj]R"7] P!^]R"73/tE]P"Pn] P!Rk] PR"7] P!^] PR"73]P"Pt] P!Rl] PR"7] P!R$] PR"73/tF]P"Pr] P!^] PR"7] P!^] PR"73]P"Pn] P!Rm] PR"7] P!^@] PR"73]P"Pv] P!^] PR"7] P!R$] PR"73]P"Pt] P!Rn] PR"7] P!R%] PR"73]P"P|] P!^]R"7] P!^]R"73]P"Pz] P!Ro]R"7] P!^]R"73/tGR&Rp/R'ltHRqR(ltIRrR)ltJR*tKRsR+R,lltLRsR-R.lltMRqR/R0lltNRtR1ltORtR2ltPR3R4ltQR5R6ltR!R7R84tS!R9R:4tT!R;R<4tUR=tVR>tWR?tXR@tYRARBltZRCt[RuRDlt\RvRElt]RFRGlt^RHRIlt_RJRKlt`RLRMltaRNROltbRPRQltcRRRSltdRTRUlteRVRWltfRXRYltgRZR[lthR\R]ltiR^R_ltjR`RaltkRbRcltlRdReltmRfRgltnR# ]"dRt!EL_i;iuupi)w) annotationsN)Enum)Path) float8_e4m3fnint4uint4) ModelProto TensorProtoexternal_data_helper)onnx_pb) make_graph make_model make_nodemake_tensor_value_info)ReferenceEvaluator)GraphOptimizationLevelInferenceSessionSessionOptions)to_array_extendedz onnx.quantizez0.1.0ai.onnxz com.microsoftQuantizeLinear_QuantizeLinear_InputDequantizeLinear_DequantizeLinear_Output _quantizedlc4]tRt^4t^t^tRt]R4tRt R#)QuantizationModec VP#Nnameselfs&u/Volumes/fast/ai/experiments/nudenet-smoke/.venv/lib/python3.14/site-packages/onnxruntime/quantization/quant_utils.py__str__QuantizationMode.__str__8 yyc P\V,# \d \4hi;ir)rKeyError ValueError)modes&r$ from_stringQuantizationMode.from_string;s) #D) ) ,   %N) __name__ __module__ __qualname____firstlineno__ IntegerOps QLinearOpsr% staticmethodr-__static_attributes__r0r(r$rr4s%JJr(rc4]tRt^Ct^t^tRt]R4tRt R#)QuantizedValueTypec VP#rr r"s&r$r%QuantizedValueType.__str__Gr'r(c P\V,# \d \4hi;ir)r:r*r+)vs&r$r-QuantizedValueType.from_stringJs) %a( ( ,  r/r0N) r1r2r3r4Input Initializerr%r7r-r8r0r(r$r:r:Cs% EKr(r:cX]tRt^Rt^t^t^t^t^t^t ^t Rt ] R4t ]R4tRtR#) QuantTypec VP#rr r"s&r$r%QuantType.__str__[r'r(c P\V,# \d \4hi;ir)rCr*r+)ts&r$r-QuantType.from_string^s( Q<  ,  r/c (V\P8Xd\P#V\P8Xd\P #V\P 8Xd\P#V\P8Xd\P#V\P8Xd\P#V\P8Xd\P#V\P8Xd\P#\!RV: R24h)zUnexpected value qtype=.)rCQInt8r INT8QUInt8UINT8QUInt16UINT16QInt16INT16 QFLOAT8E4M3FN FLOAT8E4M3FNQUInt4UINT4QInt4INT4r+r"s&r$ tensor_typeQuantType.tensor_typees 9?? "## # 9## #$$ $ 9$$ $%% % 9## #$$ $ 9** *++ + 9## #$$ $ 9?? "## #24(!<==r(r0N)r1r2r3r4rKrMrSrQrOrWrUr%r7r-propertyrYr8r0r(r$rCrCRsR E FM FG E F >>r(rCc4]tRt^xt^t^tRt]R4tRt R#) QuantFormatc VP#rr r"s&r$r%QuantFormat.__str__|r'r(c P\V,# \d \4hi;ir)r]r*r+)formats&r$r-QuantFormat.from_strings) v& & ,  r/r0N) r1r2r3r4 QOperatorQDQr%r7r-r8r0r(r$r]r]xs%I Cr(r]int8uint8int16uint16dtypeiii@zero_point_indexc.p\V4EF wr4\P!\V4\P4'd'VP \P !V44MC\V\P4'dVP V4M\RV RV 24hW08XgKVR,pVP\P8Xg"VP\P8XgK\RVP 24h \V4^8d \V4#V^,#)zarg z is not an array: zzero_point cannot be ) enumeratenumpy issubdtypetypenumberappendarray isinstancendarray TypeErrorrjfloat32float16lentuple)rkargsnew_argsiar>s$* r$ _check_typersH$   DGU\\ 2 2 OOEKKN + 5== ) ) OOA d1#%7s;< <  Aww%--'177emm+C"7y ABB "(ma/5?@Xa[@r(cV\9g QRV R24hV\PP\PP\PP \PP 39Ed<V^8wd\RV: R24hVP\P8Xd\PpMIVP\P8Xd\PpM\RVP R24h\\!\#R.R.\$P&P)RV.^.4R7\#R.ROR .4.R \+R VR 4\+R VR 4.\+R VR 4.44p\-V4p\/VP1R R VR V/4^,4#\V,p \3VRRR7wrVe \5W4MT p Ve \7W4MT p \P8!VP;\P4V, P=4V,4p\P>!WWR7\/VP;V 44#)Unexpected data type > requested. Only INT8, UINT8, INT16, and UINT16 are supported.z2zero_point is expected to be null for float 8 not rJzUnexpected dtype Constant zero_point)valuerXscaleYquNF) reduce_range symmetric)out)rrr) ONNX_TYPE_TO_NP_TYPE onnx_protor rTFLOAT8E4M3FNUZ FLOAT8E5M2FLOAT8E5M2FNUZNotImplementedErrorrjrorxFLOATryFLOAT16r+rr ronnxhelper make_tensorrrrrunget_qmin_qmax_for_qTypemaxminasarrayastyperoundclip)qTypearrrrlowhigh onnx_type onnx_modelrefrjqminqmaxcliplowcliphigharr_fp32s&&&&&& r$quantize_nparrayrs' ( ( w&de ( ++--))--  ?%(Z[eZhhi&jk k 99 %#))I YY%-- '#++I0 1=> > "Bdkk>U>UVbdikmpqor>s.0LseT  *3 4@*7ItD(UD9:  !,3774#sGU)CDQGHH%U+,URWX $'O#d.&*&63t?D==#**U]]";e"C!J!J!Lz!YZ 8h=8??5122r(c "V^8gV^8d\RV RV 24h\P!V\P!^VPR74p\P !V\P!^VPR74pVe2\ W\P!WPPR7,4pV'dF\P !\P!V4\P!V44pV)pV5pW#8:gQRV RV 24h\P!W, \PR7p\P!V\PR7\P!V\PR7, p\P!Wx, 4p V ^8gQR4hV \P!VP4P8dH\P!RVPR7p \P!^VPR7p W.#V'di\P!\P!W#,\P!R\PR7, 4VPR7p MC\P!\P!W V , , 4VPR7p V PVP4p W.#) aCalculate the scale s and zero point z for the quantization relation r = s(q-z), where r are the original values and q are the corresponding quantized values. r and z are calculated such that every value within [rmin,rmax] has an approximate representation within [qmin,qmax]. In addition, qmin <= z <= qmax is enforced. If the symmetric flag is set to True, the interval [rmin,rmax] is symmetrized to [-absmax, +absmax], where absmax = max(abs(rmin), abs(rmax)). :parameter rmin: minimum value of r :parameter rmax: maximum value of r :parameter qmin: minimum value representable by the target quantization data type :parameter qmax: maximum value representable by the target quantization data type :parameter symmetric: True if the floating-point range should be made symmetric. Defaults to False. :parameter min_real_range: Minimum floating-point range (i.e., rmax - rmin) to enforce. Defaults to None. :return: zero and scale [z, s] Bqmin and qmax must meet requirement: qmin <= 0 <= qmax while qmin:, qmmax:rizqmin=z > qmax=z scale issue?g@)r+rominimumrtrjmaximumrrabsfloat64finfotinyrr) rminrmaxrrrmin_real_rangeabsmaxdrdqrrs &&&&&& r$compute_scale_zprs( ax4!8]^b]ccklpkqrss ==u{{1DJJ? @D ==u{{1DJJ? @D!4 nJJ OOPuyy $@ww <55htf55< T[ 6B T /%++d%--2X XB KK E A:$}$: u{{4::&+++ Ctzz2[[$**5      T[EKK5==,QQRZ^ZdZdJU[[u 1D%ETZZXJ TZZ(  r(cRpV\9dV\P8Xd^RIHpTp\ R4Uu.uFp\ V4NK pp\P!VUu.uFAp\P!V4'dK!\P!V4'dK?VNKC up\PR7pM\RV R24hV\V&MV\P8Xd ^RIHpTpVf\RV R24h\P!\V,4p\P!^VR7p \P!W, VPR7p W.#uupiuupi) aZCalculate the scale s for a float8 type (E4M3FN). The function assumes the coefficient distribution and the float 8 distribution are similar to two gaussian laws. :return: zero and scale [z, s] More details in notebook `quantization_fp8.ipynb `_. N)rrizQuantization to element_type=z not implemented.zUnexpected element_type rJ)FLOAT8_DISTRIBUTIONSr rT ml_dtypesrrangefloatrortisnanisinfrxr+rwstdrj) element_typerzp_dtyperr~ all_valuesfvaluesstd_f8zerors && r$compute_scale_zp_float8r,s$H// ;33 3 /$H,1#J7Jq%(JJ7[[&TJqekk!nU[[QR^JT\a\i\iF<\NJ[\] ]-3\* 11 1+ 2<.BCC YY+L9 :F ;;q )D KK CII 6E =#8TsE."E3E3#E3c8V^8dQhRRRRRRRRRR R R R R R R /#)data numpy.ndarray quant_typeonnx.TensorProto.DataTyperboolrrz float | None rmin_override rmax_overridereturnz#tuple[numpy.ndarray, numpy.ndarray]r0)ras"r$ __annotate__rPsb;G;G ;G);G;G ;G ! ;G  ;G ;G);Gr(c0\V\P4'g\R\ V4 R24hVeTpM#\ V4'dVP 4MRpVeTpM#\ V4'dVP4MRp\P!WpPR7p\P!WPR7p\P!RVPR7p V\P8XdDV'd \R4h\P!V4p \W4wr\W^R7#V\P \P"\P$\P&\P(\P*39d`\-WVR7wr\ V4'd\/WxWW$4wrM"\P!^V PR7p \W^R7#\1R V R24h) a Returns the zero_point and scale for the given data. :param data: The data for which to compute quantization parameters. :param quant_type: The quantization data type. :param symmetric: whether symmetric quantization is used or not. :parameter reduce_range: True if the quantization range should be reduced. Defaults to False. :parameter min_real_range: Minimum floating-point range (i.e., rmax - rmin) to enforce. Defaults to None. :parameter rmin_override: The value of rmin to use if not None. Otherwise, uses min(data). :parameter rmax_override: The value of rmax to use if not None. Otherwise, uses max(data). :return: zero point and scale z%Weight must be given as an array not rJgrirz1Unsupported option reduce_range=True for float 8.)rkrz Unexpected value for quant_type=)rurorvrwrqrzrrrtrjr rT RuntimeErrorrrrrLrNrRrPrXrVrrr+)rrrrrrrrrrrrrrs&&&&&&& r$compute_data_quant_paramsrPs* dEMM * *?T |1MNN  YYtxxzC  YYtxxzC ;;t:: .D ;;t:: .D KK4:: .E[--- RS Siio3JD :qAA -ZQZ[  t99 0T c JQdjj9J:qAA 7 |1E FFr(cV^8dQhRR/#)rrz2tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]r0)ras"r$rrs:=:=7:=r(c \VVVVVVV4wrxV\P8Xd\WW4p \ V P \ P4P4^,^8H4'dg\ P!V4p \RV P4 RV P4 RV P4 RV P4 R2 4hWxV 3#V\P\P\P\P \P"\P$39d\WW4p WxV 3#\'RV R24h)a  :param data: data to quantize :param qType: data type to quantize to. :param symmetric: whether symmetric quantization is used or not. :parameter reduce_range: True if the quantization range should be reduced. Defaults to False. :parameter min_real_range: Minimum floating-point range (i.e., rmax - rmin) to enforce. Defaults to None. :parameter rmin_override: The value of rmin to use if not None. Otherwise, uses min(data). :parameter rmax_override: The value of rmax to use if not None. Otherwise, uses max(data). :return: minimum, maximum, zero point, scale, and quantized weights To pack weights, we compute a linear transformation - when data `type == uint8` mode, from `[rmin, rmax]` -> :math:`[0, 2^{b-1}]` and - when data `type == int8`, from `[-m , m]` -> :math:`[-(2^{b-1}-1), 2^{b-1}-1]` where `m = max(abs(rmin), abs(rmax))` and add necessary intermediate nodes to transform quantized weight to full weight using the equation :math:`r = S(q-z)`, where - *r*: real original value - *q*: quantized value - *S*: scale - *z*: zero point z+One of the quantized value is NaN data in [z, z], quantized_data in [z].zUnexpected value for qType=rJ)rr rTranyviewrorfravelrrrrrLrNrRrPrXrVr+) rrrrrrrrrquantized_datanp_datas &&&&&&& r$ quantize_datarsO82  J ((()%uI ##EKK06683>3F G GmmD)G=gkkm_Bw{{}o^&&4&8&8&:%;2n>P>P>R=SSUW .00  *%uI.00 25'; <39d\3R V R 24h\A\CVP-444p\PDPGVWP$VR R 7pV#\PDPIV4p\P!VVR7PVP$4p\PJPMVV4pV#)a Returns a quantized version of the given ONNX initializer. :param weight: The ONNX initializer to quantize. :param quant_type: The final quantized data type. :param zero_point: The zero-point value to use for quantization. :param scale: The scale value to use for quantization. :param axis: The quantization axis if quantizing per-channel. Defaults to None. :param quant_weight_name: The name of the quantized initializer. If not specified, the quantized name is generated. :return: The quantized ONNX initializer. NzThe initializer of shape z! could not be created, expecting :N Nz, got z and shape=z raw=:NNrJzQuantized weights for z. must be 8-bit before packing as 4-bit values.T)rawri)'tensor_proto_to_arrayrrshapelistrtakersrorreshape concatenater!TENSOR_NAME_QUANT_SUFFIXrr rT data_typedimsextendflattencopytobytesraw_datarrstrrXrVrjrrbytespack_bytes_to_4bitrrtensor_dtype_to_np_dtype numpy_helper from_array)rrrrrr weight_data q_weight_data channel_count channel_dimsquantized_channel_data_listr~ channel_data channel_scalechannel_zero_pointquantized_channel_data q_weight_nameq_weight_initializercheck packed_dataquant_np_dtypes&&&&&& r$quantize_onnx_initializerrs((/K*.M |(5F5F5H%\ #))$/ K--.  &(#}%A&++A4L!HM!+ %5..0-& " ( . .u}}=S/T/\/\]i/j k&))*EL ):%6;;-PhOi@jMT%%222#//1)3&!!((5$1!(5(=(=(?(D(D(F(N(N(P%  (&&:;E{{k///5==?mF[F[F]3]"/ 0A0A/BBc$,,.s34F5==?3;O:PP[\b\h\h[iS!56t<=Q@(  ((--t/?/?/E/EF F   tUm 3!7 Ftuv v.}/D/D/FGH  ${{66}jR]R]_jpt6u  ==jI m>JRRSYS^S^_ #00;;M=Y r(cV\PP8Xd \R4hRpV'd\P V4pM6V'dV\ 9d\ V,pM\P V4pV'g\RV R24hVwrEV^8gV^8d(\RV RV RVP RV R V R V 2 4hV#) z Return qmin and qmax, the minimum and maximum value representable by the given qType :parameter qType: onnx.onnx_pb.TensorProto.UINT8 or onnx.onnx_pb.TensorProto.UINT8 :return: qmin, qmax z;This function is not implemented for float 8 as not needed.Nrrrrz, dtype=z, reduce_range=z , symmetric=z, qType=) rr rTrONNX_INT_TYPE_REDUCED_RANGEgetONNX_INT_TYPE_SYMMETRIC_RANGEONNX_INT_TYPE_RANGEr+rj)rrrqrangerrs&&& r$rrs  &&333!"_`` F,007 u ==.u5$((/ 07uvwwJD ax4!86$x |?<.Y" 8E7 4  Mr(c0\WVR7wr4WC, #)z Helper function to get the quantization range for a type. parameter qType: quantization type. return: quantization range. r)r)rrrrrs&&& r$get_qrange_for_qTyper:s ) RJD ;r(c$V^8dQhRRRRRR/#)rrintrankrztuple[bool, int]r0)ras"r$rrDs"   C ,< r(cLV^8d W,MTpV^8;'dW!8pW23#)z Helper function that tries to return a normalized axis in the range [0, rank - 1]. :parameter axis: The axis to normalize. :parameter rank: The tensor rank (number of dimensions). :return (is_valid, axis_norm) r0)rr axis_normis_valids&& r$normalize_axisr Ds0 $ax TIA~22)"2H  r(c V^8dQhRRRR/#)rsrc_8bitrr bytearrayr0)ras"r$rrPs9r(cR\V4pV^8Xd \4#V^,^,p\V4p^p^pWA^, 8dHW^,,^,^,W,^,,W5&V^, pV^, pKTWA8dW,^,W5&V#)a. Copies a source array of 8-bit values into a destination bytearray of packed 4-bit values. Assumes that the source values are already in the appropriate int4 range. :parameter src_8bit: The 8-bit element values to pack. :return A bytearray with every two 8-bit src elements packed into a single byte. )rzr#)r" num_elemsdst_sizedstsrc_idst_is& r$rrPsH IA~{A !#H H CEE a-  *S0Q68?S;PQ      _s* Jr(c*]tRtRtRt..R3RltRtR#)QuantizedInitializerinzB Represents a linearly quantized weight input from ONNX operators Nc  rWnW nW0nW@nWPnW`nWpnWnWnR#r) r! initializerrminsrmaxs zero_pointsscalesrrr) r#r!r-r.r/r0r1rrrs &&&&&&&&&&r$__init__QuantizedInitializer.__init__ss4 &  &  , r() rrr-r!rr/r.r1r0r1r2r3r4__doc__r2r8r0r(r$r+r+ns r(r+c$]tRtRtRtRRltRtR#)QuantizedValueizA Represents a linearly quantized value (input\output\intializer) Nc  rWnW nW0nW@nWPnW`nWpnWnWnR#r) original_nameq_name scale_namezp_name value_typer node_type node_qtype scale_type) r#r!new_quantized_namer;zero_point_namequantized_value_typerr>r?r@s &&&&&&&&&&r$r2QuantizedValue.__init__s2"( $& . "$$r() rr?r>r9r:r;r@r=r<)NNNNr4r0r(r$r7r7s%%r(r7c ]tRtRtRtRtRtR#)BiasToQuantizeiz# Represents a bias to be quantized c *WnW nW0nR#r bias_name input_name weight_name)r#rIrJrKs&&&&r$r2BiasToQuantize.__init__s"$&r(rHNr4r0r(r$rFrFs 'r(rFcVP^8Xd\RVP R24hVP^8XdVPpEM5VP^8XdVPpEMVP^8XdVP pMVP^8XdVP pMVP^8XdVPpMVP^8XdVPpMVP^8XdVPpMVP^8XdVPpMbVP^ 8XdVPpMDVP^ 8XdVPpM&\RVP RVP R24hVPV/#)z Convert attribute to kwarg format for use with onnx.helper.make_node. :parameter attribute: attribute in AttributeProto format. :return: attribute in {key: value} format. z attribute z does not have type specified.z has unsupported type rJ) rqr+r!rr~srGgfloatsintsstringstensorsgraphs) attributers& r$attribute_to_kwargrVs; ~~:inn%55STUU~~  1   1   1   1   1    1  1 !! 1 !! 2   :inn%55KINNK[[\]^^ NNE ""r(cVUu.uFq"PV8XgKVNK pp\V4^8d V^,#R#uupi)z Helper function to find item by name in a list. parameter item_name: name of the item. parameter item_list: list of items. return: item if found. None otherwise. N)r!rz) item_name item_listitemitemss&& r$ find_by_namer\sB( Bid99 +ATTiE B5zA~58/4/ Cs??cbRp\\V44FpW,V8XgKTpK V#)z; Helper function to return index of an item in a node list rm)rrz) elem_name elem_listelem_idxr~s&& r$get_elem_indexras2H 3y> " <9 $H# Or(cF\PPRW.V4#)z Helper function to create a Mul node. parameter inputs: list of input names. parameter output: output name. parameter name: name of the node. return: Mul node in NodeProto format. Mul)rrr)inputsoutputr!s&&&r$ get_mul_noderfs ;; $ ??r(c$V^8dQhRRRRRR/#)rfilenamer identifierrrr0)ras"r$rrs&RR4RSRTRr(c|VPPVPV,VP,4#)zh Helper function to generate a identifiable filepath by concatenating the given identifier as a suffix. )parentjoinpathstemsuffix)rhris&&r$generate_identified_filenameros+ ?? # #HMMJ$>$P QQr(c`^RIp^RIHp^RIpVP!VP R7\ R4\ V4\ R4\ V4VPWRR7VPR4VPR4VPR 4VP4R#) rN) thresholdz Histogram:zHistogram Edges:T)fillz Tensor valueCountszTensor value V.S. Counts) sysmatplotlib.pyplotpyplotroset_printoptionsmaxsizeprintstairsxlabelylabeltitleshow)hist hist_edgesrtpltros&& r$ apply_plotrs{# S[[1 , $K  *JJtdJ+JJ~JJxII()HHJr(c  aaaaa^RIo^RIp^RIo^RIHuHuHp^RIHuHuHp^RI H oH oH o\P!RV 24!VVVVV3RlRSP4pSP!WR7p\#\$P&P)VR4R4;_uu_4pVP+V4RRR4SP-^4pVP/R 4p .p \1VP344Fp W ,p V P54p \7V P9R V4P;44\7V P9R V4P;44.p\=\?V44pV PAV 4pV PAV4pVPCV 4VPEV V4VPGV V4VPIV 4pV PKV4K VPMV \OV 44V FpV PQV4K V PS4pVPUV 4VPWV V4VPYV 4pV P[V4V P]4p\#\$P&P)VR 4R 4;_uu_4pVP+V4RRR4\$P^P9RR4R9dVPPaV^4pVPc4p\eV4F\pVPgV4p\P!VPi44\P!VPk44K^ \#\$P&P)VR4R4;_uu_4p\1VP344Fp W ,p V P54p \7V P9R V4P;44\7V P9R V4P;44.pV R,\=\?V44,pVP+V4VP+R4K RRR4R# +'giEL;i +'giEL;i +'giR#;i)z6 Helper function to write calibration table to files. N)CalibrationMethod TensorData TensorsDatazcalibration cache: c,<]tRtRtVVVVV3RltRtR#)*write_calibration_table..MyEncoderi#c v<\VSS34'dVP4#\VSP4'd*RVP4R\ VP 4RR/#\VS4'd$RVP PR\ V4/#SPPW4#)rrjCLSz numpy.arrayr) ruto_dictrvtolistrrj __class__r1 JSONEncoderdefault)r#objrrrjsonnps&&r$r2write_calibration_table..MyEncoder.default$s# K899{{}$#rzz** gs399~um\\#011s}}55wCII##++D6 6r(r0N)r1r2r3r4rr8)rrrrrsr$ MyEncoderr#s  7 7r(r)clszcalibration.jsonwihighestlowestzcalibration.flatbufferswbQUANTIZATION_DEBUG0zcalibration.cache  )1)6r flatbuffersro5onnxruntime.quantization.CalTableFlatBuffers.KeyValue quantizationCalTableFlatBuffersKeyValue5onnxruntime.quantization.CalTableFlatBuffers.TrtTableTrtTable"onnxruntime.quantization.calibraterrrlogginginfordumpsopenospathjoinwritertBuildersortedkeysrrrrZrr CreateString KeyValueStartKeyValueAddKeyKeyValueAddValue KeyValueEndrsTrtTableStartDictVectorrzPrependUOffsetTRelative EndVector TrtTableStartTrtTableAddDict TrtTableEndFinishOutputenvironGetRootAsTrtTable DictLengthrDictKeyValue)calibration_cachedirrrrr json_datafilerbuilderkey_value_listkeyrd_valuesrPrflat_key flat_value key_value main_dict cal_tablebufdict_lenr~rrrrrs&& @@@@@r$write_calibration_tablers LLLL]] LL&'8&9:;77D$$7 , ># (,,y$/446 7 (,,x.335 6 CK '',))%0 w'2!!':6((1 i(#0& $$Wc..AB# '' 2$!!#I 7# Wi0$$W-I NN9 .. C bggll3 9:D A AT 3 B zz~~*C0H<%%77Q? '')xA!q)I LL ) LL* +! bggll3 34c : :d+0023C&+F~~'Hhll9d388:;hll8T2779:F#ICK 00E JJu  JJt 4 ; :g : 9 9L B A A ; : :s%-R/SCS/ S  S  S( cV^8HP\P4pV^8gP\P4pVP4pVPV, pV'gR#V\ V4,\ V4, pVR8gQRV RV RV 24hVP\P4pWqV,V)V,,, pV^8*P4^8XgQhV#)ajGiven a discrete distribution (may have not been normalized to 1), smooth it by replacing zeros with eps multiplied by a scaling factor and taking the corresponding amount off the non-zero values. Ref: http://web.engr.illinois.edu/~hanj/cs412/bk3/KL-divergence.pdf https://github.com//apache/incubator-mxnet/blob/master/python/mxnet/contrib/quantization.py Nrzn_zeros=z , n_nonzeros=z, eps1=)rrorxsumsizer)pepsis_zeros is_nonzerosn_zeros n_nonzeroseps1rs&& r$smooth_distributionrosQu}}-H6//%--0KllnG'!J  w % "3 3D #:Q'- |74&QQ: 88EMM "D(Nte{2 22D AI??  !! ! Kr(cV^8dQhRR/#)r model_pathrr0)ras"r$rrsnnnr(c\P!VP4RR7p\;QJd0RVPP 4F 'gK R# R#!RVPP 44#)F)load_external_datac3N"TFp\P!V4xK R#5ir)r uses_external_data).0 intializers& r$ *model_has_external_data..s!mUlz#66zBBUls#%T)rloadas_posixrgraphr-)rmodels& r$model_has_external_datars^ IIj))+ FE 3mUZU`U`UlUlm33m3m3mUZU`U`UlUlm mmr(c V^8dQhRRRR/#)rrropt_model_pathr0)ras"r$rrs k kt kT kr(c\4pVP4Vn\PVn/pR.VR&\ VP4V3RR./VBpR#)z Generate model that applies graph optimization (constant folding, etc.) parameter model_path: path to the original onnx model parameter opt_model_path: path to the optimized onnx model :return: optimized onnx model ConstantSharingdisabled_optimizers providersCPUExecutionProviderN)rroptimized_model_filepathrORT_ENABLE_BASICgraph_optimization_levelr)rr sess_optionkwargs_s&& r$optimize_modelrsb!"K+9+B+B+DK(+A+R+RK( F%6$7F !,,. jH^G_jcijAr(cV^8dQhRR/#rrr r0)ras"r$rrs77J7r(cRR/pVP'd;VPF*pVPVPVP/4K, \P P W4R#)z>Tag the model that it went through quantization pre-processingonnx.quant.pre_processonnxruntime.quantNmetadata_propsupdaterrrrset_model_props)rrprops& r$add_pre_process_metadatarsV.0CDN ((D  ! !488TZZ"8 9)KK6r(c V^8dQhRRRR/#rrr rrr0)ras"r$rrs*r(cVP'd;VPF*pVPR8XgKVPR8XgK)R# R#)zCCheck the model whether it went through quantization pre-processingrrTFrrr)rrs& r$model_has_pre_process_metadatar s? ((Dxx33 FY8Y) r(cV^8dQhRR/#rr0)ras"r$rrs77j7r(cRR/pVP'd;VPF*pVPVPVP/4K, \P P W4R#) onnx.inferrNr)rrrs& r$add_infer_metadatarsV"$78N %%A  ! !155!''"2 3&KK6r(c V^8dQhRRRR/#r r0)ras"r$rrsJ4r(cVP'd;VPF*pVPR8XgKVPR8XgK)R# R#)rrTFr )rrs& r$model_has_infer_metadatars> %%Auu $4G)G& r(c V^8dQhRRRR/#)rrr rrr0)ras"r$rrsZCr(cVPUu.uF)qP'dVPR8XgK'VNK+ pp\V4^8wd \R4hV^,PpV#uupi)rz$Failed to find proper ai.onnx domain) opset_importdomainrzr+version)ropsetai_onnx_domain opset_versions& r$get_opset_versionrsj).););m);<<a?@@"1%--M  ns $A0A0c$V^8dQhRRRRRR/#)rrr weight_typerCrr0)ras"r$rrs!!! !!z!r(c\V4pTp\VRV4pV^8d=V\PP8Xd\ P !RV R24^pMEV^ 8Xd\ P !RV R24M#V^ 8d\ P !RV R24^ pW28wd+\PPW4p\V4pV#)rYz$The original model opset version is z, which does not support quantization to float 8. Please update the model to opset >= 19. Automatically update the model to opset 19. Please verify the quantized model.ze, which does not support node fusions. Please update the model to opset >= 11 for better performance.z, which does not support quantization. Please update the model to opset >= 11. Automatically update the model to opset 11. Please verify the quantized model.) rgetattrrr rTrwarningversion_converterconvert_version&save_and_reload_model_with_shape_infer)rrrtarget_opset_versionweight_quant_types&& r$update_opset_versionr(s%e,M( ]KHr/43C3C3P3PP2=/B1 1 " " 2=/BM M  2=/B1 1 ",&&66uS7u= Lr(c V^8dQhRRRR/#)rrrrr r0)ras"r$rrsDZr(c\VR4p\PP\ V4\ V44\P !VP 44p\V4VP4V#)z -inferred) rorshape_inferenceinfer_shapes_pathrrrrunlink)rinferred_model_pathrs& r$load_model_with_shape_inferr/s`6z;O**3z?C@Srks #   00>>&QQ-PP7      ,$2'!47 4Dq4Dq SZ[fhiSjloHp* Q'*4Dq  t   #>#>L $  V!4  %++g"6  %++g"6!!5;;x#8''  %  5;;q #DekkRU]b]h]hFi"j%++d%**"Eu{{SV^c^h^hGi!j!!EKK$F TYafamamHn#o  5;;vU[[#I5;;W\didodoKp"q  5;;q#> BV[@\"]%++b"=u{{1TX?Y!Z %++d%**"Eu{{SV^c^h^hGi!j  5;;vU[[#I5;;W\didodoKp"q!   5;;q #DekkRU]b]h]hFi"j%++c"DekkRT\a\f\fFg!h!!EKK$F TYafamamHn#o  5;;vU[[#I5;;W\didodoKp"q  5;;q#> AUZ@["\%++b"=u{{1TX?Y!Z  A A 13h<~!H;G|:=zL ^@ <>%%8''"#J0@R$Yx2n k 77!H7+,2-3/$rs"Z;[ %[ ; [[