i !ddlmZddlmZmZmZmZddlmZddl m Z ddl m Z m Z mZmZddlmZddlmZddlmZmZdd lmZmZdd lmZmZesd gZerdd Zerees dd gZndd gZGddeZy ))Sequence)AnyClassVarOptionalUnion)Tensor)Literal)_LPIPS_lpips_compute _lpips_update _NoTrainLpips)Metric) dim_zero_cat)_SKIP_SLOW_DOCTEST_try_proceed_with_timeout)_MATPLOTLIB_AVAILABLE_TORCHVISION_AVAILABLE)_AX_TYPE_PLOT_OUT_TYPEz*LearnedPerceptualImagePatchSimilarity.plotNctddy)NTvgg) pretrainednet)r l/Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/torchmetrics/image/lpip.py_download_lpipsr s $E*r%LearnedPerceptualImagePatchSimilarityc (eZdZUdZdZeed<dZeed<dZeed<dZ e ed<d Z e ed <e e ed <d Zeed <d gZee eed< ddeddeeddededdf fd Zde de ddfdZde fdZ d deee ee fdeedefdZxZS)!ra The Learned Perceptual Image Patch Similarity (`LPIPS_`) calculates perceptual similarity between two images. LPIPS essentially computes the similarity between the activations of two image patches for some pre-defined network. This measure has been shown to match human perception well. A low LPIPS score means that image patches are perceptual similar. Both input image patches are expected to have shape ``(N, 3, H, W)``. The minimum size of `H, W` depends on the chosen backbone (see `net_type` arg). .. hint:: Using this metrics requires you to have ``torchvision`` package installed. Either install as ``pip install torchmetrics[image]`` or ``pip install torchvision``. As input to ``forward`` and ``update`` the metric accepts the following input - ``img1`` (:class:`~torch.Tensor`): tensor with images of shape ``(N, 3, H, W)`` - ``img2`` (:class:`~torch.Tensor`): tensor with images of shape ``(N, 3, H, W)`` As output of `forward` and `compute` the metric returns the following output - ``lpips`` (:class:`~torch.Tensor`): returns float scalar tensor with average LPIPS value over samples Args: net_type: str indicating backbone network type to use. Choose between `'alex'`, `'vgg'` or `'squeeze'` reduction: str indicating how to reduce over the batch dimension. Choose between `'sum'`, `'mean'`,`'none'` or `None`. normalize: by default this is ``False`` meaning that the input is expected to be in the [-1,1] range. If set to ``True`` will instead expect input to be in the ``[0,1]`` range. kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info. Raises: ModuleNotFoundError: If ``torchvision`` package is not installed ValueError: If ``net_type`` is not one of ``"vgg"``, ``"alex"`` or ``"squeeze"`` ValueError: If ``reduction`` is not one of ``"mean"`` or ``"sum"`` Example: >>> from torch import rand >>> from torchmetrics.image.lpip import LearnedPerceptualImagePatchSimilarity >>> lpips = LearnedPerceptualImagePatchSimilarity(net_type='squeeze') >>> # LPIPS needs the images to be in the [-1, 1] range. >>> img1 = (rand(10, 3, 100, 100) * 2) - 1 >>> img2 = (rand(10, 3, 100, 100) * 2) - 1 >>> lpips(img1, img2) tensor(0.1024) >>> from torch import rand, Generator >>> from torchmetrics.image.lpip import LearnedPerceptualImagePatchSimilarity >>> gen = Generator().manual_seed(42) >>> lpips = LearnedPerceptualImagePatchSimilarity(net_type='squeeze', reduction='none') >>> # LPIPS needs the images to be in the [-1, 1] range. >>> img1 = (rand(2, 3, 100, 100, generator=gen) * 2) - 1 >>> img2 = (rand(2, 3, 100, 100, generator=gen) * 2) - 1 >>> lpips(img1, img2) tensor([0.1024, 0.0938]) Tis_differentiableFhigher_is_betterfull_state_updategplot_lower_boundg?plot_upper_bound all_scoresrfeature_network__jit_ignored_attributes__net_typeralexsqueeze reduction)summeannone normalizekwargsreturnNc Ft|d i|ts tdd}||vrt d|d|dt ||_d}||vrt d|d|||_t|tst d |||_ |jd gd y) NzLPIPS metric requires that torchvision is installed. Either install as `pip install torchmetrics[image]` or `pip install torchvision`.r)z#Argument `net_type` must be one of z , but got .)r)r.r-r/Nz$Argument `reduction` must be one of z/Argument `normalize` should be an bool but got r%)defaultdist_reduce_fxr) super__init__rModuleNotFoundError ValueErrorr rr, isinstanceboolr0 add_state)selfr(r,r0r1valid_net_typevalid_reduction __class__s rr8z.LearnedPerceptualImagePatchSimilarity.__init__rs "6"%%e  4 > )B>BRR\]e\ffghi i X.7 O +COCTT^_h^ijk k")T*NykZ[ [" |RErimg1img2ct|||j|j}|jj |y)z(Update internal states with lpips score.)rr0N)r rr0r%append)r>rBrClosss rupdatez,LearnedPerceptualImagePatchSimilarity.updates,T4TXXP t$rcZt|j}t||jS)z+Compute final perceptual similarity metric.)r,)rr%r r,)r>scoress rcomputez-LearnedPerceptualImagePatchSimilarity.computes!doo.f??rvalaxc&|j||S)aPlot a single or multiple values from the metric. Args: val: Either a single result from calling `metric.forward` or `metric.compute` or a list of these results. If no value is provided, will automatically call `metric.compute` and plot that result. ax: An matplotlib axis object. If provided will add plot to that axis Returns: Figure and Axes object Raises: ModuleNotFoundError: If `matplotlib` is not installed .. plot:: :scale: 75 >>> # Example plotting a single value >>> import torch >>> from torchmetrics.image.lpip import LearnedPerceptualImagePatchSimilarity >>> metric = LearnedPerceptualImagePatchSimilarity(net_type='squeeze') >>> metric.update(torch.rand(10, 3, 100, 100), torch.rand(10, 3, 100, 100)) >>> fig_, ax_ = metric.plot() .. plot:: :scale: 75 >>> # Example plotting multiple values >>> import torch >>> from torchmetrics.image.lpip import LearnedPerceptualImagePatchSimilarity >>> metric = LearnedPerceptualImagePatchSimilarity(net_type='squeeze') >>> values = [ ] >>> for _ in range(3): ... values.append(metric(torch.rand(10, 3, 100, 100), torch.rand(10, 3, 100, 100))) >>> fig_, ax_ = metric.plot(values) )_plot)r>rKrLs rplotz*LearnedPerceptualImagePatchSimilarity.plotsPzz#r""r)r*r.F)NN)__name__ __module__ __qualname____doc__r r<__annotations__r!r"r#floatr$listrr&strr'rr rrr8rGrJrrrrrO __classcell__)rAs@rrr)s+:x#t""d"#t#!e!!e!V  OS 8=gc 3=7=>D F23FG$9:;F F  F  F>%6%%D% @@ _c(#E&(6*:":;<(#IQRZI[(# (#r)r2N) collections.abcrtypingrrrrtorchrtyping_extensionsr #torchmetrics.functional.image.lpipsr r r r torchmetrics.metricrtorchmetrics.utilitiesrtorchmetrics.utilities.checksrrtorchmetrics.utilities.importsrrtorchmetrics.utilities.plotrr__doctest_skip__rrrrrrdsr%11%dd&/WX@DE+";O"LCEqr?AmnZ#FZ#r