i5ddlmZddlmZmZmZddlmZddlm Z ddl m Z ddl m Z mZddlmZmZmZddlmZdd lmZdd lmZdd lmZmZesd d gZGdde ZGddeZGdde Zy))Sequence)AnyOptionalUnion)Tensor)Literal)_ClassificationTaskWrapper)BinaryConfusionMatrixMulticlassConfusionMatrix)"_binary_cohen_kappa_arg_validation_cohen_kappa_reduce&_multiclass_cohen_kappa_arg_validation)Metric)ClassificationTaskNoMultilabel)_MATPLOTLIB_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPEzBinaryCohenKappa.plotzMulticlassCohenKappa.plotc eZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed < dd e d e e de eddededd f fd ZdefdZ dde eeeefde edefdZxZS)BinaryCohenKappaa Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for binary tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): A int or float tensor of shape ``(N, ...)``. If preds is a floating point tensor with values outside [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Additionally, we convert to int tensor with thresholding using the value in ``threshold``. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. tip:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``bc_kappa`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: threshold: Threshold for transforming probability to binary (0,1) predictions ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (preds is int tensor): >>> from torch import tensor >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Example (preds is float tensor): >>> from torchmetrics.classification import BinaryCohenKappa >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0.35, 0.85, 0.48, 0.01]) >>> metric = BinaryCohenKappa() >>> metric(preds, target) tensor(0.5000) Fis_differentiableThigher_is_betterfull_state_updateplot_lower_bound?plot_upper_boundN threshold ignore_indexweightslinear quadraticnone validate_argskwargsreturnc ht|||fddd||r t|||||_||_yNF) normalizer$)super__init__r rr$)selfrrrr$r% __class__s |/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/torchmetrics/classification/cohen_kappa.pyr+zBinaryCohenKappa.__init__ds@ L`DPU`Y_`  .y, P *cBt|j|jSzCompute metric.r confmatrr,s r.computezBinaryCohenKappa.computer"4<<>>r/valaxc&|j||S)a9Plot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting a single value >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> metric.update(rand(10), randint(2,(10,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import rand, randint >>> # Example plotting multiple values >>> from torchmetrics.classification import BinaryCohenKappa >>> metric = BinaryCohenKappa() >>> values = [ ] >>> for _ in range(10): ... values.append(metric(rand(10), randint(2,(10,)))) >>> fig_, ax_ = metric.plot(values) _plotr,r7r8s r.plotzBinaryCohenKappa.plotvPzz#r""r/)?NNTNN)__name__ __module__ __qualname____doc__rbool__annotations__rrrfloatrrintrrr+rr5rrrrr= __classcell__r-s@r.rr$s7r$t#!d!#t#!e!!e!&*DH" + +sm +'"?@A +  +  +  +?? _c(#E&(6*:":;<(#IQRZI[(# (#r/rc eZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <d Z e ed < ddedeedeeddededd f fd ZdefdZ ddeeeeefdeedefdZxZS)MulticlassCohenKappaa" Calculate `Cohen's kappa score`_ that measures inter-annotator agreement for multiclass tasks. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. As input to ``forward`` and ``update`` the metric accepts the following input: - ``preds`` (:class:`~torch.Tensor`): Either an int tensor of shape ``(N, ...)` or float tensor of shape ``(N, C, ..)``. If preds is a floating point we apply ``torch.argmax`` along the ``C`` dimension to automatically convert probabilities/logits into an int tensor. - ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. .. tip:: Additional dimension ``...`` will be flattened into the batch dimension. As output to ``forward`` and ``compute`` the metric returns the following output: - ``mcck`` (:class:`~torch.Tensor`): A tensor containing cohen kappa score Args: num_classes: Integer specifying the number of classes ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation weights: Weighting type to calculate the score. Choose from: - ``None`` or ``'none'``: no weighting - ``'linear'``: linear weighting - ``'quadratic'``: quadratic weighting validate_args: bool indicating if input arguments and tensors should be validated for correctness. Set to ``False`` for faster computations. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Example (pred is integer tensor): >>> from torch import tensor >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([2, 1, 0, 1]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) Example (pred is float tensor): >>> from torchmetrics.classification import MulticlassCohenKappa >>> target = tensor([2, 1, 0, 0]) >>> preds = tensor([[0.16, 0.26, 0.58], ... [0.22, 0.61, 0.17], ... [0.71, 0.09, 0.20], ... [0.05, 0.82, 0.13]]) >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric(preds, target) tensor(0.6364) FrTrrrrrrClassplot_legend_nameN num_classesrrr r$r%r&c ht|||fddd||r t|||||_||_yr()r*r+rrr$)r,rOrrr$r%r-s r.r+zMulticlassCohenKappa.__init__s@ lbdRWb[ab  2; g V *r/cBt|j|jSr1r2r4s r.r5zMulticlassCohenKappa.computer6r/r7r8c&|j||S)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure object and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a single value >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> metric.update(randn(20,3).softmax(dim=-1), randint(3, (20,))) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> from torch import randn, randint >>> # Example plotting a multiple values >>> from torchmetrics.classification import MulticlassCohenKappa >>> metric = MulticlassCohenKappa(num_classes=3) >>> values = [] >>> for _ in range(20): ... values.append(metric(randn(20,3).softmax(dim=-1), randint(3, (20,)))) >>> fig_, ax_ = metric.plot(values) r:r<s r.r=zMulticlassCohenKappa.plotr>r/)NNTr@)rArBrCrDrrErFrrrrGrrNstrrHrrrr+rr5rrrrr=rIrJs@r.rLrLs:x$t#!d!#t#!e!!e!#c# '+DH" + +sm +'"?@A +  +  +  +?? _c(#E&(6*:":;<(#IQRZI[(# (#r/rLcleZdZdZ ddeddeddedeedeed d eed e d e d e fdZ y) CohenKappaa0Calculate `Cohen's kappa score`_ that measures inter-annotator agreement. .. math:: \kappa = (p_o - p_e) / (1 - p_e) where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is the expected agreement when both annotators assign labels randomly. Note that :math:`p_e` is estimated using a per-annotator empirical prior over the class labels. This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the ``task`` argument to either ``'binary'`` or ``'multiclass'``. See the documentation of :class:`~torchmetrics.classification.BinaryCohenKappa` and :class:`~torchmetrics.classification.MulticlassCohenKappa` for the specific details of each argument influence and examples. Legacy Example: >>> from torch import tensor >>> target = tensor([1, 1, 0, 0]) >>> preds = tensor([0, 1, 0, 0]) >>> cohenkappa = CohenKappa(task="multiclass", num_classes=2) >>> cohenkappa(preds, target) tensor(0.5000) Nclstask)binary multiclassrrOrr rr$r%r&c @tj|}|j|||d|tjk(r t |fi|S|tj k(r4t |tstdt|dt|fi|Std|d)zInitialize task metric.)rrr$z+`num_classes` is expected to be `int` but `z was passed.`zTask z not supported!) rfrom_strupdateBINARYr MULTICLASS isinstancerH ValueErrortyperL)rVrWrrOrrr$r%s r.__new__zCohenKappa.__new__=s.66t< 'v> >5o677r/)r?NNNT) rArBrCrDrarrGrrHrErrrbr/r.rUrU"s:%)DH&*"8 , 8,-88c] 8 '"?@A 8 sm 888 8r/rUN) collections.abcrtypingrrrtorchrtyping_extensionsr torchmetrics.classification.baser ,torchmetrics.classification.confusion_matrixr r 2torchmetrics.functional.classification.cohen_kappar r rtorchmetrics.metricrtorchmetrics.utilities.enumsrtorchmetrics.utilities.importsrtorchmetrics.utilities.plotrr__doctest_skip__rrLrUrcr/r.rpsp%''%Gi 'G@@/1LMz#,z#z~#4~#B.8+.8r/