iq=ddlZddlZddlmZmZmZmZmZmZm Z ddl Z ddl Z ddl mcmZddlmZmZmZddlmZddlmZmZddlmZddlmZddlmZdd l m!Z!Gd d eZ"y) N)DictListOptionalSequenceTextTupleUnion)Problem ResolutionSpecifications)SegmentationTask)SegmentSlidingWindowFeature)Protocol)SegmentationProtocol)BaseWaveformTransform)Metricc*eZdZdZ ddedeeedfdeeede dee e e e ffdee e d ee d e d ee d ed eedeee eeeeffffd ZdefdZdfd Zde de de fdZde fdZde fdZedZxZS)MultiLabelSegmentationa Generic multi-label segmentation Multi-label segmentation is the process of detecting temporal intervals when a specific audio class is active. Example use cases include speaker tracking, gender (male/female) classification, or audio event detection. Parameters ---------- protocol : Protocol cache : str, optional As (meta-)data preparation might take a very long time for large datasets, it can be cached to disk for later (and faster!) re-use. When `cache` does not exist, `Task.prepare_data()` generates training and validation metadata from `protocol` and save them to disk. When `cache` exists, `Task.prepare_data()` is skipped and (meta)-data are loaded from disk. Defaults to a temporary path. classes : List[str], optional List of classes. Defaults to the list of classes available in the training set. duration : float, optional Chunks duration. Defaults to 2s. warm_up : float or (float, float), optional Use that many seconds on the left- and rightmost parts of each chunk to warm up the model. While the model does process those left- and right-most parts, only the remaining central part of each chunk is used for computing the loss during training, and for aggregating scores during inference. Defaults to 0. (i.e. no warm-up). balance: Sequence[Text], optional When provided, training samples are sampled uniformly with respect to these keys. For instance, setting `balance` to ["database","subset"] will make sure that each database & subset combination will be equally represented in the training samples. weight: str, optional When provided, use this key to as frame-wise weight in loss function. batch_size : int, optional Number of training samples per batch. Defaults to 32. num_workers : int, optional Number of workers used for generating training samples. Defaults to multiprocessing.cpu_count() // 2. pin_memory : bool, optional If True, data loaders will copy tensors into CUDA pinned memory before returning them. See pytorch documentation for more details. Defaults to False. augmentation : BaseWaveformTransform, optional torch_audiomentations waveform transform, used by dataloader during training. metric : optional Validation metric(s). Can be anything supported by torchmetrics.MetricCollection. Defaults to AUROC (area under the ROC curve). Nprotocolcacheclassesdurationwarm_upbalanceweight batch_size num_workers pin_memory augmentationmetricc t|tstdt|dt |||||| | | | | ||_||_||_y)NzHMultiLabelSegmentation task expects a SegmentationProtocol but you gave z. )rrrrrr r!r) isinstancer ValueErrortypesuper__init__rrr)selfrrrrrrrrrrr r! __class__s /Volumes/fast/ai/experiments/voice-extract-mac/.venv/lib/python3.12/site-packages/pyannote/audio/tasks/segmentation/multilabel.pyr'zMultiLabelSegmentation.__init__\sx($89Z[_`h[iZjjlm   !#!%     prepared_datac r|j!|jstjd}|jrGt j |jj|jj}n|jj}|jt}t}|D]}|jdd}|s-tjd|dd|dd}t||D]}||vs|j||j|Dcgc]}|j|c}tj |tj"|d <||_n=t}|D]}|jdd}|s-tjd|dd|dd}t|t%|t%|jz } | r?tjd|dd|dd d j'| d }t)||jt%|t%|jzDcgc]}|jj|c}tj |jtj"|d <tj*t-|t-|jftj.} t1|D] \} }d | | |f<| |d<|j3ycc}wcc}w)NaE Could not infer list of classes. Either provide a list of classes when instantiating the task, or make sure that the training protocol provides a 'classes' entry. See https://github.com/pyannote/pyannote-database#segmentation for more details. rz File "uriz" (from databaseaW database) does not provide a 'classes' entry. Please make sure the corresponding training protocol provides a 'classes' entry for all files. See https://github.com/pyannote/pyannote-database#segmentation for more details. dtype classes-listz; database) provides extra classes (z, z,) that are ignored. Tclasses-annotated)r has_classestextwrapdedenthas_validation itertoolschainrtrain developmentlistgetr$appendindexnparraystr_setjoinprintzeroslenbool_ enumerateclear) r(r,msg files_iterrannotated_classesfile file_classesklass extra_classesannotated_classes_arrayfile_ids r*post_prepare_dataz(MultiLabelSegmentation.post_prepare_datas  << (8(8//C   " ##%t}}'@'@'BJ,,.J << fG $ "#xx 48 #"//#E{m8D4D3EFC%S/))EG+u-*"((7CD|eW]]5)|D%#,-/HHWBGG,LM. )"DL!% "#xx 48 #"//#E{m8D4D3EFC%S/) #L 1C 4E E  "//#E{m8D4D3EF((, -(@'ABC #J!((&)%6T\\9J%J%JE **51%J3#@-/HHT\\,QM. )#%(( " #S%6 7rxx# !**; < GW8< #GW$4 5!=-D )*!mEDs -L/ "L4 ct||t|jdtj t j|j|j|j|_ y)Nr2)rproblem resolutionr min_durationr) r&setupr r,r MULTI_LABEL_CLASSIFICATIONr FRAMErrXrspecifications)r(stager)s r*rYzMultiLabelSegmentation.setupsS  e,&&~666!'']]**LL  r+rS start_timec|j|}t|||z}t}|jjj ||\|d<}|j d|j dd|k(}||d|jk|d|jkDz} |jjj} d|jjjz} tj| d|j|jz | z } tjdtj| | z jt } tj"| d|j|jz | z }tj|| z jt }|jj%t||jj&j(z}tj*|t-|j dftj.  }d|d d |j d |f<t1| || d D]\} }}d || |d z|f<t3||jj|j4|d<|j d|}|j6j8Dcic]}||| c}|d<||dd<|Scc}w)aPrepare chunk for multi-label segmentation Parameters ---------- file_id : int File index start_time : float Chunk start time duration : float Chunk duration. Returns ------- sample : dict Dictionary containing the chunk data with the following keys: - `X`: waveform - `y`: target (see Notes below) - `meta`: - `database`: database index - `file`: file index Notes ----- y is a trinary matrix with shape (num_frames, num_classes): - 0: class is inactive - 1: class is active - -1: we have no idea Xzannotations-segmentsrSstartendg?rr2r0Nr3global_label_idx)labelsyzaudio-metadatametarN)get_filerdictmodelaudiocropr,rbrareceptive_fieldsteprr@maximumroundastypeintminimum num_frameshparams sample_rateonesrGint8ziprrr1names)r(rSr^rrNchunksample_ annotationschunk_annotationsrnhalfra start_idxrbend_idxrtrflabelmetadatakeys r* prepare_chunkz$MultiLabelSegmentation.prepare_chunks>}}W% J$9:))..tU;s Q(()?@   5 6y AW L (  !EII -+e2Du{{2R S  zz))..TZZ//888 ,W5u{{CekkQTXXJJq"((54<"89@@E jj*51599= KdR((3:&--c2ZZ** (TZZ//;;; < WWD&&~67 ''   BC!T   3 4W = =>!$ w 12D E" E3)*AecAgou$ %" + tzz))$,, s %%&67@8@8L8LM8L#x},8LMv!(vv Ns K. batch_idxch|d}|j|}|d}|j|jk(sJ|dk7}||}||}tj||j t j }t j|ry|jjd|ddddd|iS) Nr`rfz loss/trainFTon_stepon_epochprog_barloggerloss) rjshapeFbinary_cross_entropyr%torchfloatisnanlogr(batchrr`y_predy_truemaskrs r* training_stepz$MultiLabelSegmentation.training_stepKs #JAs||v||+++ $r\%%ffkk%++.FG ;;t      ~r+c<|d}|j|}|d}|j|jk(sJ|dk7}||}||}tj||j t j }|jjd|ddddd|iS) Nr`rfrloss/valFTrr)rjrrrr%rrrrs r*validation_stepz&MultiLabelSegmentation.validation_stephs #JAs||v||+++$r\%%ffkk%++.FG     ~r+cy)aQuantity (and direction) to monitor Useful for model checkpointing or early stopping. Returns ------- monitor : str Name of quantity to monitor. mode : {'min', 'max} Minimize See also -------- lightning.pytorch.callbacks.ModelCheckpoint lightning.pytorch.callbacks.EarlyStopping )rmin)r(s r* val_monitorz"MultiLabelSegmentation.val_monitors&!r+) NNg@gNN NFNN)N)__name__ __module__ __qualname____doc__rrr strrrrrrrrboolrrrr'rTrYrrrpropertyr __classcell__)r)s@r*rr(sb1l-1'+58,0!%%) 8rsI0EEE HHE7&;Qn!-n!r+