+ G-j{^RIt^RIt^RIt^RIt^RIt^RIHt^RIHt^RI H t ^RI t ^RI t ^RI HtHtHtHt^RIt^RIHtHtHtRRltR(R R llt!R R 4t!R R4t!RR]4t!RR]P8R7t!RR4t!RR]4t!RR]4t !RR] 4t!!RR] 4t"!RR] 4t#!R R!]P8R7t$!R"R#]$4t%RR$]PLR%R/3R&R'llt'R#))N)Sequence)Enum)Path) ModelProto TensorProtohelper numpy_helper) apply_plotload_model_with_shape_infersmooth_distributioncxV^8dQhR\PR\PR\P/#)pkqkreturn)npndarray)formats"s/Volumes/fast/ai/experiments/nudenet-smoke/.venv/lib/python3.14/site-packages/onnxruntime/quantization/calibrate.py __annotate__rs-     c@\P!VPVPR7pVR,\P!VR,VR,, 4,VR&V^8HV^8,p^W#&V^8V^8,p\P W$(&V#)z See https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.rel_entr.html#scipy.special.rel_entr. Python implementation. dtype:NNN)remptyshaperloginf)rrresc2c1s&& rrel_entrr"s ((288288 ,C URVVBqEBqEM* *CF 'bAg BCG q&R!V BvvCH Jrc V^8dQhR\PR\PR\R,R\R\P/#)rrrbaseNaxisr)rrfloatint)rs"rrr'sJ   $,   ZZ rcVeV^8gQR4hVfQR4h\P!V4P\P4pRV,\P!WRR7, p\P!V4P\P4p\P !W4wrRV,\P!WRR7, p\ W4p\P!WCR7pVeV\P!V4,pVPVP4#)z Simplifeied version of entropy. Source: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html. This avoids taking a dependency on scipy just for this function. z0base={base} must be a positive number or `None`.z qk is None?T)r%keepdimsr%) rasarrayastypefloat32sumbroadcast_arraysr"rr)rrr$r%vecss&&&& rentropyr3's <4!8W%WW # >'<'> B  rzz *B rBFF248 8B B  rzz *B   (FB rBFF248 8B 2 C sA  RVVD\ 88BHH rcta]tRt^Cto]!.RO4t]!.RO4tRt]R4t ]R4t Rt Rt Vt R#) TensorDatac \VP44VnVP4Fwr#V\P 9d"\ RV: R\P R24hV\P9dy\VR4'g\ R\V4 RV: 24hVP\P\P39d\ RVP RV: 24h\WV4K R#)zUnexpected value z not in .rUnexpected type z for k=zUnexpected dtype N)listkeys_attrsitemsr5_allowed ValueError_floatshasattrtyperrfloat16r.setattr)selfkwargskvs&, r__init__TensorData.__init__Gs6;;=) LLNDA +++ #4QE*BUBUAVVW!XYYJ&&&q'**$'7Qyu%MNN772::rzz"::$'8 %NOO DQ #rc\VR4'd\VR4'g\R\V4 R24hVPVP3#)lowesthighestz0Attributes 'lowest' and/or 'highest' missing in r7)r@AttributeErrordirrKrLrDs&r range_valueTensorData.range_valueSsLtX&&gdI.F.F #STWX\T]S^^_!`a a T\\**rc\VR4'd\VR4'g\R\V4 R24hVPVP3#)avgstdz)Attributes 'avg' and/or 'std' missing in r7)r@rMrNrSrTrOs&ravg_stdTensorData.avg_stdYsItU##74+?+? #LSQUYKWX!YZ Z$((##rcVPUu/uFq\W4bK ppVPPVR&V#uupi)CLS)r;getattr __class____name__)rDrFdatas& rto_dictTensorData.to_dict_s?-1[[9[74##[9nn--U  :sA)r;N)rSrTrKrLhist hist_edgesbins)rSrTrKrLr`)r[ __module__ __qualname____firstlineno__ frozensetr=r?rHpropertyrPrUr]__static_attributes____classdictcell__ __classdict__s@rr5r5CsRZ[HIJG  ++ $$ rr5cba]tRt^ftoV3RlRltRtRtRtRtRt Rt R t R t R t VtR #) TensorsDatacF<V^8dQhRS[S[S[S[,3,/#)rr\)dictstrr5tuple)rrjs"rrTensorsData.__annotate__gs#c:;M6M1Nrc Wn/VnVP4EF>wr4\V\4'g\ R\ V4 R24h\V\4'dV\P8Xd;\V4^8Xd+\V^,V^,R7VPV&K\V4^8Xd;\V^,V^,V^,V^,R7VPV&K\ RVR R\V4 RV R24h\V\4'g\ R\ V4 R24hW@PV&EKA R #) zKeys must be strings not r7rKrL)rKrLr_razUnexpected tuple for rz , it has z elements: zValues must be TensorData not N) calibration_methodr\r< isinstancero TypeErrorrArpCalibrationMethodMinMaxlenr5)rDrur\rFrGs&&& rrHTensorsData.__init__gs""4 JJLDAa%%";DG9A FGG!U##%):)A)AAc!fPQk#-QqT1Q4#HDIIaLq6Q;#-QqT1Q4aPQdYZ[\Y]#^DIIaL"7!uIc!fX[YZX[[\ ]^^a,,"@a  KLLIIaL!rc#:"VPRjxL R#L5iNr\rOs&r__iter__TensorsData.__iter__ys99s cWP9#r}r~rDkeys&&r __contains__TensorsData.__contains__|siirc(VPV,#r}r~rs&&r __getitem__TensorsData.__getitem__syy~rcbWP9d\RV: R24hW PV&R#)z)Only an existing tensor can be modified, z is not.N)r\ RuntimeError)rDrvalues&&&r __setitem__TensorsData.__setitem__s- ii !J3'QYZ[ [ #rc6VPP4#r})r\r:rOs&rr:TensorsData.keyssyy~~rc6VPP4#r})r\valuesrOs&rrTensorsData.valuessyy!!rc6VPP4#r})r\r<rOs&rr<TensorsData.itemssyy  rcfRVPPRVPRVP/pV#)rXr\ru)rZr[r\ru)rDr\s& rr]TensorsData.to_dicts5 4>>** DII $"9"9   r)rur\N)r[rbrcrdrHrrrrr:rr<r]rgrhris@rrlrlfs<$   "!rrlc&]tRt^t^t^t^t^tRtR#)rxN) r[rbrcrdryEntropy Percentile Distributionrgrrrrxrxs FGJLrrxca]tRt^to]R4t]PV3RlRl4tRt Rt Rt V3RlRlt R t VtR #) CalibrationDataReadercp\VR4;'d\VP4;'g\#)get_next)r@callablerNotImplemented)clssubclasss&&r__subclasshook__&CalibrationDataReader.__subclasshook__s+*-MM(8;L;L2M``R``rc <V^8dQhRS[/#rr)rn)rrjs"rr"CalibrationDataReader.__annotate__s""$"rc \h)z9generate the input data dict for ONNXinferenceSession runNotImplementedErrorrOs&rrCalibrationDataReader.get_nexts "!rcV#r}rrOs&rrCalibrationDataReader.__iter__s rc:VP4pVf\hV#r})r StopIteration)rDresults& r__next__CalibrationDataReader.__next__s >  rc\hr}rrOs&r__len__CalibrationDataReader.__len__!!rc&<V^8dQhRS[RS[/#)r start_index end_index)r')rrjs"rrrs""S"S"rc\hr}r)rDrrs&&&r set_rangeCalibrationDataReader.set_rangerrrN)r[rbrcrd classmethodrabcabstractmethodrrrrrrgrhris@rrrsLaa "" """rr) metaclassca]tRt^toRV3RlRlltR.3RltRtV3RlRltR tR t V3R lR lt V3R lRlt Rt Vt R#)CalibraterBaseNcT<V^8dQhRS[S[,RS[S[,R,/#r model_pathop_types_to_calibrateNrorr)rrjs"rrCalibraterBase.__annotate__s, < <$J < (}t3 = 2Gb. :param per_channel: whether to compute ranges per each channel. z model_path should be model path.NCPUExecutionProvider)rvror rmodelr>raugmented_model_path symmetricuse_external_data_format per_channel augment_model infer_sessionexecution_providers)rDrrrrrrs&&&&&&&rrHCalibraterBase.__init__s" j# & &4T*5EFDJ  D ) )4Z@DJ?@ @%:"$8!"(@%&!!$:#; rrc 2WnVP4R#)zj reset the execution providers to execute the collect_data. It triggers to re-creating inference session. N)rcreate_inference_session)rDrs&&rset_execution_providers&CalibraterBase.set_execution_providerss$7  %%'rc \P!4p\PPVn\P !VP VVPR7VnR#)z) create an OnnxRuntime InferenceSession. ) sess_options providersN) onnxruntimeSessionOptionsGraphOptimizationLevelORT_DISABLE_ALLgraph_optimization_levelInferenceSessionrrr)rDrs& rr'CalibraterBase.create_inference_sessionsN#113 0;0R0R0b0b -(99  % %%.. rc <V^8dQhRS[/#)rr)r)rrjs"rrrs111rc VPPUu/uFq"PVbK ppTPVPPUu/uFqDPVbK up4TPVPP Uu/uFqUPVbK up4VPP Uu0uFqfPkK pp\4p\P\P0p VPPFp VP'dV PVP9gK2\P!V P V P4Fvp W9gK W;,pVP P#R4'gK6VP P$P&V 9gK]W9gKeVP)V 4Kx K W3#uupiuupiuupiuupi)z select input/output tensors of candidate nodes to calibrate. returns: tensors (set): set of tensor name. value_infos (dict): tensor name to value info. tensor_type)graph value_infonameupdateoutputinput initializersetrFLOATFLOAT16noderop_type itertoolschainrAHasFieldr elem_typeadd) rDrvi value_infosotitinitrtensors_to_calibratetensor_type_to_calibrater tensor_names && rselect_tensors_to_calibrate*CalibraterBase.select_tensors_to_calibrates.3[[-C-CD-Crww{-C D%++2D2DE2DBGGRK2DEF%++2C2CD2CBGGRK2CDE-2[[-D-DE-DTyy-D E"u$/$5$5{7J7J#K KK$$D---A[A[1[#,??4::t{{#KK"1(5GG,,];;!#!4!4!>!>BZ!Z!,!?044[A$L%$00)EEDEsG:G?HH c VP#)z@ return: augmented onnx model. Call after calling augment_graph )rrOs&rget_augment_model CalibraterBase.get_augment_modelszzrc \h)z abstract method: augment the input model to prepare for collecting data. It will: 1. augment the model to be able to collect desired statistics data 2. save augmented model to augmented_model_paths rrOs&r augment_graphCalibraterBase.augment_graphs "!rc <V^8dQhRS[/#r data_readerr)rrjs"rrrs""(="rc \h)zp abstract method: collect the tensors that will be used for range computation. It can be called multiple times. r)rDr s&&r collect_dataCalibraterBase.collect_data "!rc <V^8dQhRS[/#rrl)rrjs"rrr"s""k"rc \h)zU abstract method: compute data based on the calibration method stored in TensorsData rrOs&r compute_dataCalibraterBase.compute_data"rr) rrrrrrrrr)Naugmented_model.onnxFFF)r[rbrcrdrHrrrrrr rrgrhris@rrrsK < = 2Gb :param moving_average: compute the moving average of the minimum and maximum values instead of the global minimum and maximum. :param averaging_constant: constant smoothing factor to use when computing the moving average. :param max_intermediate_outputs: maximum number of intermediate outputs before an intermediate range is computed. :param per_channel: whether to compute ranges per each channel. )rrrrrNz;Invalid averaging constant, which should not be < 0 or > 1.)superrHintermediate_outputscalibrate_tensors_rangerzrrrnum_model_outputsrmodel_original_outputsmoving_averager>averaging_constantmax_intermediate_outputs) rDrrrrrrr r!rrrZs &&&&&&&&&& rrHMinMaxCalibrater.__init__*s.  "7!5%=#  %'!'+$!$TZZ%5%5%<%.get_op_version^sO % 2 2 99==*=*=>>'///!3!GyPRST TrcT<a\V3Rl\SPPP44\ SPPP44pVF;pSPPPP W#4V^, pK= R#)c3R<"TFwrSVP9gKVxK R#5ir})r).0ixrs& r GMinMaxCalibrater.augment_graph..insert_nodes..fs%Z?tq;RSRYRYCY?s' 'N)next enumeraterrrrzinsert)r new_nodesindexrrDsf& r insert_nodes4MinMaxCalibrater.augment_graph..insert_nodesdsxZy)9)9)>)>?Z\_`d`j`j`p`p`u`u\vE"   %%,,U9 "rc,<^pVR,V,pVR,p\PPW.V.W#R7p\PPRVS.V.VR7pSPPP Uu/uFqwP VbK ppTPSPPPU u/uFqP V bK up 4TPSPPPU u/uFqP V bK up 4W9d(W,PPPp M\RV: R24hSP'Ed5\W,PPP P"4p ^.\%^V 4Op S!VSP4^8d2VP&P)\P*!RV 44M\-\.P0!44p\2P4!\6P8!V \6P:R 7V4pVPP)V4SPPP<P)V4S!WV.4SPPPP)\P>!W;R .44R #uupiuup iuup i) __Reshape)r*rReshape)inputsoutputsrz'Unable to guess tensor type for tensor zE, running shape inference before quantization may resolve this issue.axesrN) r&r make_noderrrrrrrrArrr>rrzrdimrange attributeappendmake_attributerouuiduuid4r from_arrayrarrayint64rmake_tensor_value_info)rreduce_op_namer* reduce_outputintermediate_output reduce_node reshape_noderror0 onnx_type tensor_rank reduced_axesreduce_axes_name reduce_axesr+r9reshape_shape_namerDs&& radd_reduce_min_max:MinMaxCalibrater.augment_graph..add_reduce_min_maxlsH(#->M"/*"< ++// 0C/Dx0K ;;00+-?@&( 1L261A1A1L1LM1L277B;1LKM   4::3C3C3J3JK3Ja 3JK L   4::3C3C3I3IJ3Ia 3IJ K)'499EEOO  =k_MZZ !+":"?"?"K"K"Q"Q"U"UV !:E![$9: !.$**=B))001F1Fv|1\]'*4::<'8$"."9"9"((JJ$$0077 D L&A B JJ   # # * *6+H+Hdhci+j k3NKJs L L L ReduceMin ReduceMaxsave_as_external_dataN)rrrorIrJr rKrrLrMrrrGr&saverr) rDtensorsr= reshape_shaper[tensorr+r9rZs f @@@rrMinMaxCalibrater.augment_graphSs 55djjA  .$//"RXX0NPbc  $$++M: U  , l, l\F v{ 3 v{ 3 JJ  % %"&"?"? rc.VnR#r}rrOs&rclear_collected_data%MinMaxCalibrater.clear_collected_data $&!rc <V^8dQhRS[/#rr )rrjs"rrrs$$(=$rc VP4pV'gMVPP\VPP 4VPP RV4RR7UUu.uF$wr4VPVP9dTMRNK& upp4VPfK\VP4VP8XgKVP4K\VP4^8XdVPf \R4hVP4p\V\ 4'g\#R\%V4 R24hVP4R#uuppi)TNFstrictNo data is collected.z+compute_data must return a TensorsData not r7)rrrGzipr get_outputsrunrrr!rzrirr>rrvrlrwrA)rDr r@sess_orts&& rr MinMaxCalibrater.collect_datas; ))+F  % % , ,*-**668$:L:L:P:PQUW]:^gl** $[[0K0KKEQUU* --9112d6S6SS))+ t(( )Q .43O3O3W45 5    ![))I$q'RSTU U !!#'s3*E0 cV'gV#VP4EFOwr4\V\4'd(VP^,pVP^,pMVwrV\W#,\4'd4W#,P^,pW#,P^,pM W#,wrxVP'd@WPP Wu, ,,p W`P W, ,,p M\ WW4p \Wh4p \V\4'g\W#,\4'd\WR7W#&EKJW3W#&EKR V#)rrs)r<rvr5rPrr minmax) rD old_range new_rangerrold_minold_maxnew_minnew_max min_value max_values &&& r merge_rangeMinMaxCalibrater.merge_ranges #//+JC%,,++A.++A.#( ).*55#.44Q7#.44Q7#,> """#&=&=AR&SS #&=&=AR&SS 1 1 %,, 9>:0V0V!+9!P "+!7 3,6rc <V^8dQhRS[/#rr)rrjs"rrrs3,3,k3,rc  \VP4^8Xd VP#\\VP^,44Uu.uF-qPP 4V,P NK/ ppVPUu.uFp\\W#RR74NK pp/pVF=pVP4F&wrxVPV.4PV4K( K? W PRp \^\V 4^4Uu.uF!qV,PR4^,NK# p pVUu/uFqVP9gKWV,bK p p.p \^\V 4^4EF6pVP'dS\ P"!WV,,^R7p \ P"!WV^,,,^R7pMQ\ P$!WV,,^R7p \ P&!WV^,,,^R7pVP('dZ\ P&!\ P*!V 4\ P*!V4.^R7pV PV)V34EK$V PW34EK9 \-\.P0\\WRR744pVP'd.VP3VPV4VnVP#VVnVP#uupiuupiuupiuupi)zt Compute the min-max range of tensor :return: dictionary mapping: {added node names: (ReduceMin, ReduceMax) pairs } FrnNr=r+)rzrrrErrrrrnrqr< setdefaultrGr rpartitionrrrnanmeannanminnanmaxrabsrlrxryr)rDr0 output_namesrQoutput_dicts_listmerged_output_dictdrFrGadded_output_namescalibrate_tensor_namesmerged_added_output_dictpairsmin_value_arraymax_value_arraymax_absolute_valuenew_calibrate_tensors_ranges& rrMinMaxCalibrater.compute_datas t(( )Q .// /JOPSTXTmTmnoTpPqJrsJrQ**668;@@Jr s(,'@'@ '@# \uE F'@   "A "--a4;;A>"#**@*@*BC>CAsK]G^`a>b" >bq ! , ,S 1! 4 4>b " /A$ .@TMhMhDh $A!$ $.@ !$ q#0115A""""$**-EYZF[-\cd"e"$**-EYZ]^Y^F_-`gh"i"$)),DXYEZ,[bc"d"$)),DXY\]X]E^,_fg"h~~~%'YY0GP_I`/ahi%j" 113EFG o?@6'2  $ $d3/EUZ+[&\' #  ' ' '+/+;+;D= 2Gb :param method: A string. One of ['entropy', 'percentile']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param num_quantized_bins: number of quantized bins. Default 128. :param percentile: A float number between [0, 100]. Default 99.99. :param scenario: see :class:`DistributionCalibrater` )rrrrN)rrHrrrzrrrrrr collectormethodnum_binsnum_quantized_bins percentilerscenario) rDrrrrrrrrrrrrZs &&&&&&&&&&& rrHHistogramCalibrater.__init__s2  "7!5%=  %'!'+$!$TZZ%5%5%<%c nVPVP4wVnpVPFJpW P9gKVPPP P W,4KL \P!VPVPVPR7R#)zk make all quantization_candidates op type nodes as part of the graph output. :return: augmented ONNX model r_N) rrrrrrrGr&rbrr)rDrres& rr!HistogramCalibrater.augment_graphFs 261Q1QRVR\R\1].!;//F888   ''..{/BC0 JJ  % %"&"?"? rc.VnR#r}rhrOs&rri(HistogramCalibrater.clear_collected_dataVrkrc <V^8dQhRS[/#rr )rrjs"rrrYs2$2$(=2$rc  VPP4Uu0uFq"PkK ppVPP4Uu.uFq"PNK ppVP 4pV'gMVPP RV4p.p\ V4FJwrWH,V9d(VP\P!V 44K9VPV 4KL VPPV4K\VP4^8Xd \R4hVPU u.uFp \\WJRR74NK p p /p V F=p V P4F&wrV PV.4PV4K( K? V Uu/uF pVVP 9gKVV V,bK" ppVP"'gS\%VP&VP(VP*VP,VP.VP0R7VnVP"P3V4VP54R#uupiuupiuup iuupi)zi Entropy Calibrator collects operators' tensors as well as generates tensor histogram for each operator. NrpFrn)rrrrrr)r get_inputsrrrrrsr5rGcopyrrzr>rnrqr<rrrHistogramCollectorrrrrrrcollectri)rDr node_arginput_names_setrr@rA fixed_outputs output_indexrrQr merged_dictrrFrGr0clean_merged_dicts&& rr  HistogramCalibrater.collect_dataYs:>9K9K9V9V9XY9XX==9XY6:6H6H6T6T6VW6V( 6V W ))+F((,,T6:GM(1'(:$ -@!((6):;!((0 );  % % , ,] ; t(( )Q .45 5(,'@'@ '@# \uE F'@   "A &&q"-44Q7"#9Df 1qDLeLeGe.Q A. f~~~/{{..#'#:#:?? DN 01 !!#]ZW, gsI)I.I30I8I8c <V^8dQhRS[/#rr)rrjs"rrrsLLkLrc VP'g \R4h\V\4'd\P pMf\V\ 4'd\PpM?\V\4'd\PpM\R\V4 R24h\WPP44#)zh Compute the min-max range of tensor :return: dictionary mapping: {tensor name: (min value, max value)} z9No collector created and can't generate calibration data.zUnknown calibrater z". This method must be overwritten.)rr>rvEntropyCalibraterrxrPercentileCalibraterrDistributionCalibraterrrwrArlcompute_collection_result)rDcals& rr HistogramCalibrater.compute_datas ~~~XY Y d- . .#++C 2 3 3#..C 4 5 5#00C1$t*=_`a a3 H H JKKr) rrrrrrrrrrr) NrFrF-X@same) r[rbrcrdrHrrir rrgrhrrs@@rrrs7*!*!X '2$2$hLLLrrcBaa]tRtRtoRV3RlV3RllltRtVtV;t#)ricT<V^8dQhRS[S[,RS[S[,R,/#rr)rrjs"rrEntropyCalibrater.__annotate__,  $J  (}t3 rc  6<\S V`VVVVVVVVR7R#)a| :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param num_quantized_bins: number of quantized bins. Default 128. )rrrrNrrH) rDrrrrrrrrrZs &&&&&&&&&rrHEntropyCalibrater.__init__s/*   ! $1  rr)NrFr3Frrr[rbrcrdrHrgrhrrs@@rrr   rrcBaa]tRtRtoRV3RlV3RllltRtVtV;t#)ricT<V^8dQhRS[S[,RS[S[,R,/#rr)rrjs"rr!PercentileCalibrater.__annotate__rrc  6<\S V`VVVVVVVVR7R#)ag :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_quantized_bins: number of quantized bins. Default 128. :param percentile: A float number between [0, 100]. Default 99.99. )rrrrNr) rDrrrrrrrrrZs &&&&&&&&&rrHPercentileCalibrater.__init__s/*   ! $!  rr)NrFrFrrrrs@@rrrrrrcBaa]tRtRtoRV3RlV3RllltRtVtV;t#)ricT<V^8dQhRS[S[,RS[S[,R,/#rr)rrjs"rr#DistributionCalibrater.__annotate__s,  $J  (}t3 rc  4<\SV`VVVVVVVR7R#)a, :param model_path: ONNX model to calibrate. It is a model path :param op_types_to_calibrate: operator types to calibrate. By default, calibrate all the float32/float16 tensors. :param augmented_model_path: save augmented model to this path. :param use_external_data_format: use external data format to store model which size is >= 2Gb :param method: A string. One of ['entropy', 'percentile', 'distribution']. :param symmetric: make range of tensor symmetric (central point is 0). :param num_bins: number of bins to create a new histogram for collecting tensor values. :param scenario: for float 8 only, if `scenario="same"`, the algorithm weights and float 8 follow the same distribution, if `scenario="p3"`, it assumes the weights follow a gaussian law and float 8 ~ X^3 where X is a gaussian law )rrrNr) rDrrrrrrrrZs &&&&&&&&rrHDistributionCalibrater.__init__s,.   ! $  rr)NrF distributionrrrrs@@rrrs   rrcla]tRtRtoRt]P R4t]P R4tRt Vt R#)CalibrationDataCollectorizD Base class for collecting data for calibration-based quantization. c \h)zk Generate informative data based on given data. name_to_arr : dict tensor name to NDArray data rrD name_to_arrs&&rr CalibrationDataCollector.collect s "!rc \h)z/ Get the optimal result among collection data. rrOs&rr2CalibrationDataCollector.compute_collection_results "!rrN) r[rbrcrd__doc__rrrrrgrhris@rrrs> "" ""rrcza]tRtRtoRtRtRtRtRtRt Rt R t R t R t ]RR l4tR tRtRtVtR#)riaL Collecting histogram for each tensor. Percentile and Entropy method are supported. ref: https://github.com//apache/incubator-mxnet/blob/master/python/mxnet/contrib/quantization.py ref: https://docs.nvidia.com/deeplearning/tensorrt/pytorch-quantization-toolkit/docs/_modules/ pytorch_quantization/calib/histogram.html c\/VnWnW nW0nW@nWPnW`nR#r})histogram_dictrrrrrr)rDrrrrrrs&&&&&&&rrHHistogramCollector.__init__&s)  " "4$ rcVP#r})rrOs&rget_histogram_dict%HistogramCollector.get_histogram_dict/s"""rc\R4VPR9dVPV4#VPR8Xd5VP'dVP V4#VPV4#\ R4h)z/Collecting tensor data and making histogram ...rDOnly 'entropy', 'percentile' or 'distribution' methods are supported>r3r)printr collect_valuercollect_absolute_valuer>rs&&rrHistogramCollector.collect2sn ?@ ;;5 5%%k2 2 [[L (~~~22;??))+66cd drc VP4EFwr#\V\4'dVF:p\V\P4'dK%QR\ V4 RV: 24h VUu0uFqUP kK pp\V4^8XgQRV RV: 24h\P!V4pM=\V\P4'g\R\ V4 RV: 24hTpVP4pVP^8d.\P!V4p\P!V4p MD\P!^VP R7p\P!^VP R7p \P!V4pW P 9dy\P"!WpP$R7wrV P'VP 4p VP \P(8wgQR4hWW3VP V&EKVP V,p V ^,p V ^,p\+V R4'gQR\ V 4 24h\+VR4'gQR\ V4 24hV ^,pV ^,p\P!V4pVVR ,8d]V^,V^,, p\P,!VR ,V,VV,V4p\P.!VV34p\P"!VVR7wrV P'VP 4p V R \V4;;;V, uuu%VP \P(8wgQR4hW\1W4\3W43VP V&EK R #uupi) z% Collect histogram on absolute value r8z for tensor=z6The calibration expects only one element type but got r)razMonly float32 or float16 is supported, every constant must be explicitly typedrz'old_min should be a numpy array but is Nra)r<rvr9rrrArrzr,r>flattensizerrrLabsoluter histogramrr-float64r@arangehstackrxry)rDrredata_arrarradtypes data_arr_nprrr_r` old_histogramr|r}old_histold_hist_edges temp_amaxwidth new_bin_edgess&& rr)HistogramCollector.collect_absolute_valueAsE!, 1 1 3 F(D))#C%c2::66l:J4PS9+Uabhak8ll6$+348a''846{a'LVHT`ag`jk'!jj2 "**55 #3DN3Crrcompute_entropycompute_percentilecompute_distributionrOs&rr,HistogramCollector.compute_collection_results"""c$*=*=&>!&C\] ] @~^_ ;;) #'') ) [[L (**, , [[N *,,. .cd drcZVP^8gVP^d8d \R4hVPpVPp/p\R\ V4 24\RVP 24\RRV, RV R24VP 4EFwrEV^,pV^,pVP4p\P!Wh, 4p VP'dr\P!WR, 4p \P!Wz,VPR7)\P!Wz,VPR73W4&MRV, R , p \P!V R V , 4p \P!W4p \P!W|,VPR7\P!Wz,VPR73W4&V^,p V^,pW4,^,V 8dWV,^,3W4&W4,^,V8dW4,^,V3W4&.W4,OVR ,O5W4&\PP!R R 4R9gEK\#Wg4EK V#)rzrrrzrr<r/rcumsumr searchsortedrLrosenvirongetr )rDrrthresholds_dictrerr_r`totalcdf idx_rightpercent_to_cut_one_sideidx_leftrrs& rr%HistogramCollector.compute_percentiles. ??Q $//C"7[\ \,,__  $S%8$9:; +DMM?;< uz12!JrcVPpVPp/p\R\V4 24\RVP R24\RVP 24VP 4FkwrEVP WR4pWcV&.VOVR,O5W4&\PPRR4R9gKQ\V^,V^,4Km V#)rrz: (The number may increase depends on the data it collects)zNumber of quantized bins : rrrr) rrrrzrr<get_entropy_thresholdr"r#r$r )rDrrr%reroptimal_thresholds& rr"HistogramCollector.compute_entropys,,!44 $S%8$9:; +DMM?:tuv +D,C,C+DEF!/!5!5!7 F $ : :9 Y &7F #&J(9&JIbM&JO #zz~~2C8HD9Q<16"8rcV^8:d\RV R24hVRRVR,,R,pV^8XdW,P4VP4, pW^,,P4VP4, V^,, R,p\P!WAPR7\P!WQPR73#\ V4V8Xd\ V4^,^8XdWV,,P4VP4, pWV,V, ^,,P4VP4, R,p\P!WAPR7\P!WQPR73#\P !V4V, p^V\P!V4&^V\P!V4&\P !V4V,V,pW,P4VP4, pW^,,P4VP4, V^,, R,p\P!WAPR7\P!WQPR73#)rzpower=z <= 0 is invalid.N:r<NNg?rra) r>r/rrLrr'risnanisinf)r_r`powerrrSrTfacts&&& r_avg_stdHistogramCollector._avg_std s A:veW,=>? ?Sb/JrN2c9 A:=%%'$((*4C19$))+dhhj836AcIC88C'7'78"((3N^N^:__ _ u: 3u:>Q#6%-',,.;CEMC/A55::rrrzrr<rrrr5r5rxryr"r#r$r ) rDrr%rerr_r`avg_coefstd_coefs & rr'HistogramCollector.compute_distribution"s ==3 RS S,, $S%8$9:; +DMM?;< DMM,A./!/!5!5!7 FQ>RZZ/ //>>RZZ/ //##rzz1 11&0%!~~'"( 'O #zz~~2C8HD4,3"86rc  V^,pV^,pVPpV^,pV^,pV^,Pp\P!Wg, ^,4p \ V P4U u.uF3p \P !^VR7\P !^VR73NK5 p p \ Wv^,^4EFRp Wj, p \ Wj,^,V4p WL,WM,3WV, &\P!W9N>)rrrrrrrN)r<)r[rbrcrdrrHrrrrrrrr staticmethodr5rr-rgrhris@rrrsf!# e8st@@ e,\*\\*&PX!X!rrrFcjV^8dQhR\\,R\\,R,/#)rrrNr)rs"rrrs1NKNK :NK#C=4/NKrc RpV\P8XdqVPRR4pVPRR4p VPRR4p VPRR4p VPRR4p \VVVVVV V V V R7 pEMV\P8XdJVPR ^4p VPR ^4pVPRR4p\ VVVVVV VR 7pMV\P 8XdJVPR R 4p VPR R4pVPRR4p\VVVVVV VR7pMJV\P8Xd6VPR R 4p VPRR4p\VVVVV VR7pV'd1VP4V'dWWn VP4V#\RV 24h)NrFrr rr!r)rrrr r!rrr)rrrrrrrT)rrrrrr)rrrzUnsupported calibration method )rxryr$rrrrrrrrrrr>)rrrcalibrate_methodrr extra_options calibratorrrr r!rrrrrs&&&&&&& rcreate_calibratorrXsJ,333!%%k59 &**+;UC*../CTJ#0#4#45OQU#V #'' u= %  ! %=)1%=#   .66 6 $$Z5*../CSI!%%k59 &  ! %=1  .99 9 $$Z6"&&|V< !%%k48 )  ! %=!  .;; ; $$Z6 $$Z8+  ! %=    " -6 *++- 67G6HI JJr)Nr)(rrrr"rIcollections.abcrenumrpathlibrnumpyrr&rrrr r quant_utilsr r r r"r3r5rlrxABCMetarrrrrrrrrryrXrrrr_s  $ >>UU 8  F11h"ckk"4k"k"\m,~m,`DL.DLN + D . D  0  F"",E!1E!T 37/&--"NKNKr