io4ddlmZmZmZmZmZddlZddlZddl Z ddl Z ddl mZmZddlmZmZGddZdZdZd ZGd d eZd Zd Zd ZGddeZdded edefdZy))ListUnionOptionalSetTupleN) AnnotationTimeline)DetailsMetricComponentsc LeZdZdZedefdZedefdZfdZ dZ dZ e dZ dd eeefd eeefd ed eefd Zddedej*fdZdZdZdedeeeffdZdZd eeefd eeefdefdZdefdZddedeeeeefffdZ xZ!S) BaseMetricz :class:`BaseMetric` is the base class for most pyannote evaluation metrics. Attributes ---------- name : str Human-readable name of the metric (eg. 'diarization error rate') returnc2t|jdz)Nz\ is missing a 'metric_name' class method. It should return the name of the metric as string.NotImplementedError__name__clss j/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/pyannote/metrics/base.py metric_namezBaseMetric.metric_name1s"! LLP P  c2t|jdz)Nzh is missing a 'metric_components' class method. It should return the list of names of metric components.rrs rmetric_componentszBaseMetric.metric_components8s"! LLV V  rc tt| |jj |_t |jj|_|jyN) superr __init__ __class__r metric_name_setr components_reset)selfkwargsrs rrzBaseMetric.__init__?sH j$(* NN668%()I)I)K%L rcB|jDcic]}|dc}Scc}w)N)r!r#values rinit_componentszBaseMetric.init_componentsEs&(,(8(89(8us (8999s ct|_t|_|jD]}d|j|<y)z.Reset accumulated components and metric valuesr&N)dict accumulated_listresults_r!r's rr"zBaseMetric.resetHs5%)V"f %%E'*D  e $&rc"|jS)z Metric name.)rr#s rnamezBaseMetric.nameOs!!r reference hypothesisdetaileduric >|j||fi|}|j|||j<|xs t|dd}|jj ||f|j D]}|j|xx||z cc<|r|S||jS)aCompute metric value and accumulate components Parameters ---------- reference : type depends on the metric Manual `reference` hypothesis : type depends on the metric Evaluated `hypothesis` uri : optional Override uri. detailed : bool, optional By default (False), return metric value only. Set `detailed` to True to return dictionary where keys are components names and values are component values Returns ------- value : float (if `detailed` is False) Metric value components : dict (if `detailed` is True) `components` updated with metric value r5NA)compute_componentscompute_metricrgetattrr.appendr!r,)r#r2r3r4r5r$ componentsr1s r__call__zBaseMetric.__call__Ws6-T,,Y MfM )-(;(;J(G 4$$%4WYt4 c:./$$D   d #z$'7 7 #%  $++,,rdisplaycRg}g}d|jv}|jD]\}}i}|r|d}|jD]^\} } | |jk(r d| z|| df< | dk(r| || df<-| || df<|s7dkDrd| z|z || df<Jtj || df<`|j ||j |i}|j}|r|d}|jD]^\} } | |jk(r d| z|| df< | dk(r| || df<-| || df<|s7dkDrd| z|z || df<Jtj || df<`dt|z||jdf<|j ||j dtj|} || d<| jd} tjj| j| _| |jg|jz} |rt| j!dd d d | S) akEvaluation report Parameters ---------- display : bool, optional Set to True to print the report to stdout. Returns ------- report : pandas.DataFrame Dataframe with one column per metric component, one row per evaluated item, and one final row for accumulated results. totald%rTOTALitemTFrightc$dj|SNz{0:.2f}formatfs rz#BaseMetric.report..s9+;+;A+>r)indexsparsifyjustify float_format)rr.itemsr1npnanr;r,abspd DataFrame set_index MultiIndex from_tuplescolumnsprint to_string) r#r>reporturispercentr5r<rowr@keyr(dfs rr^zBaseMetric.reportsJT3355#}}OCC"7+(..0 U$))#$'%KCSMG^#(CRL#(CRL 19,/%K%,?CSM,.FFCSM1 MM#  KK % -(&& w'E$**,JCdii #e CH $CG $CG qy(+e e(;CH (*CH -"CIoDIIsN c G \\& !6 \\& !]]..rzz:  d4466 7   "#!>   rcN|jd}|jddS)NF)r>c$dj|SrHrIrKs rrMz$BaseMetric.__str__..s93C3CA3Fr)rOrQ)r^r])r#r^s r__str__zBaseMetric.__str__s0U+)F   rc8|j|jS)z0Compute metric value from accumulated components)r9r,r0s r__abs__zBaseMetric.__abs__s""4#4#455r componentcj|tdddk(rt|jS|j|S)aGet value of accumulated `component`. Parameters ---------- component : str Name of `component` Returns ------- value : type depends on the metric Value of accumulated `component` N)slicer+r,)r#ris r __getitem__zBaseMetric.__getitem__s7 dD$/ /))* *$$Y/ /rc#@K|jD] \}}||f yw)z*Iterator over the accumulated (uri, value)N)r.)r#r5ris r__iter__zBaseMetric.__iter__s""mmNCy. ,sc Ft|jjdz)aCompute metric components Parameters ---------- reference : type depends on the metric Manual `reference` hypothesis : same as `reference` Evaluated `hypothesis` Returns ------- components : dict Dictionary where keys are component names and values are component values z is missing a 'compute_components' method.It should return a dictionary where keys are component names and values are component values.rrr)r#r2r3r$s rr8zBaseMetric.compute_componentss*(" NN # #'I I  rr<cFt|jjdz)aACompute metric value from computed `components` Parameters ---------- components : dict Dictionary where keys are components names and values are component values Returns ------- value : type depends on the metric Metric value z is missing a 'compute_metric' method. It should return the actual value of the metric based on the precomputed component dictionary given as input.rp)r#r<s rr9zBaseMetric.compute_metrics*" NN # #'` `  ralphacB|jDcgc]\}}||j}}}t|dk(r tdt|dk(r$t j d|dx}x}}|||ffSt jj||dScc}}w)aPCompute confidence interval on accumulated metric values Parameters ---------- alpha : float, optional Probability that the returned confidence interval contains the true metric value. Returns ------- (center, (lower, upper)) with center the mean of the conditional pdf of the metric value and (lower, upper) is a confidence interval centered on the median, containing the estimate to a probability alpha. See Also: --------- scipy.stats.bayes_mvs rzFPlease evaluate a bunch of files before computing confidence interval.zCCannot compute a reliable confidence interval out of just one file.)rr) r.rlen ValueErrorwarningswarnscipystats bayes_mvs)r#rr_rvaluescenterloweruppers rconfidence_intervalzBaseMetric.confidence_interval(s.48==A=41a!D%%&=A v;! ef f [A  MM_ `%+AY .F .UUE5>) );;((u(=a@ @BsB)FN)F)g?)"r __module__ __qualname____doc__ classmethodstrrr rrr)r"propertyr1rr rboolrr=rVrWr^rfrhfloatr rlrnr8r9rr __classcell__)rs@rr r 'sT C   "2   :+""?C+-%*(<"=+-"8Z#78+-+-.6sm+-ZRdRr||Rh 60S0U5'>-B0&!  &+Hj,@&A ',Xz-A'B )0 4  ("A"AUE%,//0"Arr precisionz # retrievedz# relevant retrievedcFeZdZdZedZedefdZdede fdZ y) PrecisionaU :class:`Precision` is a base class for precision-like evaluation metrics. It defines two components '# retrieved' and '# relevant retrieved' and the compute_metric() method to compute the actual precision: Precision = # retrieved / # relevant retrieved Inheriting classes must implement compute_components(). ctSr)PRECISION_NAMErs rrzPrecision.metric_name^srrcttgSr)PRECISION_RETRIEVEDPRECISION_RELEVANT_RETRIEVEDrs rrzPrecision.metric_componentsbs#%ABBrr<c\|t}|t}|dk(r|dk(rytd||z S)z#Compute precision from `components`r&r?rC)rrrvr#r< numerator denominators rr9zPrecision.compute_metricfs?;<  !45 # A~ n${* *rN rrrrrrr rr rr9rrrrRsM C"2CC + +U +rrrecallz # relevantcFeZdZdZedZedefdZdede fdZ y)RecallaG :class:`Recall` is a base class for recall-like evaluation metrics. It defines two components '# relevant' and '# relevant retrieved' and the compute_metric() method to compute the actual recall: Recall = # relevant retrieved / # relevant Inheriting classes must implement compute_components(). ctSr) RECALL_NAMErs rrzRecall.metric_namesrrcttgSr)RECALL_RELEVANTRECALL_RELEVANT_RETRIEVEDrs rrzRecall.metric_componentss!:;;rr<c\|t}|t}|dk(r|dk(rytd||z S)z Compute recall from `components`r&rrrC)rrrvrs rr9zRecall.compute_metrics>89  1 # A~ n${* *rNrrrrrrxsJ <"2<< + +U +rrrcH||zdk(ryd||zz|z|z||z|z|zz S)uCompute f-measure f-measure is defined as follows: F(P, R, b) = (1+b²).P.R / (b².P + R) where P is `precision`, R is `recall` and b is `beta` r&rrtr)rrbetas r f_measurers?6S  t Oy (6 1TD[95Lv5U VVr)r)typingrrrrrrwnumpyrSpandasrV scipy.statsry pyannote.corerr pyannote.metrics.typesr r r rrrrrrrrrrrrrrs854.<cAcAL #5+ +B 2+Z+B W W WE Wr