ix@NdZddlZddlZddlZddlmZddlmZddlmZm Z ddl m Z m Z m Z mZddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZddlmZddl m!Z!ddl"m#Z#ddl$m%Z%e rddl&m'Z'ejPe)Z*edZ+de,fdZ-GddeZ.y)zzFabric/PyTorch Lightning logger that enables remote experiment tracking, logging, and artifact management on lightning.ai.N) Namespace)Mapping)datetimetimezone) TYPE_CHECKINGAnyOptionalUnion)RequirementCache)Tensor)Module)override)Loggerrank_zero_experiment)get_filesystem) _add_prefix)rank_zero_only)_PATH)ModelCheckpoint)_scan_checkpoints) Experimentzlitlogger>=0.1.0returncddlm}|S)zceZdZdZdZ d0deedeedeedeeeefde d e d e d eed dfd Z e e d efdZ e e d eefdZe e d efdZe e d efdZe d efdZe d eefdZe ed edfdZe ed efdZe ed1deeefdeed dfdZe e d1deeeefefdeeeefd dfdZe ed1dedee d dfdZ!e ed2dZ"e ed1d eed dfd!Z#edeeeefefd dfd"Z$e d3ded#eed$e d%eedeeeefd df d&Z%e d4d'ed$e d%eed dfd(Z&ed5d'ed$e d efd)Z'ed6d#eed$e d%eed efd*Z(ed4d'ed$e d%eed efd+Z)ed'ed dfd,Z*d-e+d dfd.Z,d-e+d dfd/Z-y)7 LitLoggerzaLogger that enables remote experiment tracking, logging, and artifact management on lightning.ai.-Nroot_dirname teamspacemetadata store_step log_model save_logscheckpoint_namerc Jtj|xsd|_|xs t|_d|_||_d|_d|_t|j|_ d|_ |xsi|_ d|_ ||_||_d|_i|_||_y)aInitialize the LightningLogger. Args: root_dir: Folder where logs and metadata are stored (default: ./lightning_logs). name: Name of your experiment (defaults to a generated name). teamspace: Teamspace name where charts and artifacts will appear. metadata: Extra metadata to associate with the experiment as tags. log_model: If True, automatically log model checkpoints as artifacts. save_logs: If True, capture and upload terminal logs. checkpoint_name: Override the base name for logged checkpoints. Example:: from lightning.pytorch import Trainer from lightning.pytorch.demos.boring_classes import BoringModel, BoringDataModule from lightning.pytorch.loggers.litlogger import LitLogger class LoggingModel(BoringModel): def training_step(self, batch, batch_idx: int): loss = self.step(batch) # logging the computed loss self.log("train_loss", loss) return {"loss": loss} trainer = Trainer( max_epochs=10, enable_model_summary=False, logger=LitLogger("./lightning_logs", name="boring_model") ) model = BoringModel() data_module = BoringDataModule() trainer.fit(model, data_module) trainer.test(model, data_module) z./lightning_logsNF)osfspath _root_dirr_name_version _teamspace_sub_dir_prefixr_fs_step _metadata _is_ready _log_model _save_logs_checkpoint_callback_logged_model_time_checkpoint_name) selfr#r$r%r&r'r(r)r*s r__init__zLitLogger.__init__:s\8#A/AB646 '+ #  !$..1 !R##?C!46 /rc|jS)z Gets the name of the experiment.)r1r?s rr$zLitLogger.name|szzrc|jS)z2Get the experiment version - its time of creation.)r2rBs rversionzLitLogger.versions}}rc|jS)zBGets the save directory where the litlogger experiments are saved.)r0rBs rr#zLitLogger.root_dirs~~rcntjj|j|j}t |j tr*tjj||j }tjj|}tjj|S)zThe directory for this run's tensorboard checkpoint. By default, it is named ``'version_${self.version}'`` but it can be overridden by passing a string value for the constructor's version parameter instead of ``None`` or an int. ) r.pathjoinr#r$ isinstancesub_dirstr expandvars expanduser)r?log_dirs rrNzLitLogger.log_dirso'',,t}}dii8 dllC (ggll7DLL9G''$$W-ww!!'**rc|jSN)rNrBs rsave_dirzLitLogger.save_dirs ||rc|jS)zCGets the sub directory where the TensorBoard experiments are saved.)r4rBs rrJzLitLogger.sub_dirs}}rrc ddl}|j |jS|jsd|_tjdk(sJd|j r'|j j|j d|jJtjtjjd}|jdd |_|j!|j"|j$|j&|j(j+Dcic]\}}|t-|c}}dd|j. |jScc}}w) z3Returns the underlying litlogger Experiment object.rNTz+tried to init log dirs in non global_rank=0)exist_ok milliseconds)timespecz+00:00Z)r$r#r%r&r'store_created_atr)) litlogger experimentr9rrankr#r6makedirsrDrnowrutc isoformatreplacer2initr1r0r3r8itemsrKr;)r?rY timestampkvs rrZzLitLogger.experiments     +'' '~~!DN""a'V)VV' == HH  dmmd  ; <<  X\\2<?[gDJJ7\s7-Cparamsct|tr |j}|j|jxsi||_yzLog hyperparams.NrIr__dict__updater8)r?rprhs rlog_hyperparamszLitLogger.log_hyperparamss3 fi (__F dnn*+rmodel input_arrayc<tjdtdy)Nz&LitLogger does not support `log_graph`) stacklevel)warningswarn UserWarning)r?rwrxs r log_graphzLitLogger.log_graphs  > XYZrcyrPrBs rsavezLitLogger.saves rstatuscddl}|j9|jr|j|j|j |yy)Nr)rYrZr<_scan_and_log_checkpointsfinalize)r?rrYs rrzLitLogger.finalizesC     +((..t/H/HI   v & ,rct|tr |j}|j|jxsi||_yrrrs)r?rps r log_metadatazLitLogger.log_metadatas3 fi (__F dnn*+r staging_dirverboserDcNddl}|j}|j|||||y)aSave and upload a model object to cloud storage. Args: model: The model object to save and upload (e.g., torch.nn.Module). staging_dir: Optional local directory for staging the model before upload. verbose: Whether to show progress bar during upload. version: Optional version string for the model. metadata: Optional metadata dictionary to store with the model. rN)rYrZr()r?rwrrrDr&rYros rr(zLitLogger.log_models&&  OOE;(KrrGcJddl}|j}|j|||y)aVUpload a model file or directory to cloud storage using litmodels. Args: path: Path to the local model file or directory to upload. verbose: Whether to show progress bar during upload. Defaults to False. version: Optional version string for the model. Defaults to the experiment version. rN)rYrZlog_model_artifactr?rGrrDrYros rrzLitLogger.log_model_artifact)s"  OO$$T7G ?(;D %rcZt||j}|D]\}}}}|js|jj|_|dt t jtjj}|j|d|||j|<y)zGFind new checkpoints from the callback and log them as model artifacts.r"T)rrDN) rr=r>rZr$intrr]rr^rcr)r?r checkpointsrc path_ckpt_scoretag unique_tags rrz#LitLogger._scan_and_log_checkpointss'(;T=T=TU 1< -Iy&#(((,(<(<%5#hll8<<&@&J&J&L"M!NOJ  # #ItZ # P1:D # #I .2=r)NNNNTFTNrP)rN)NFNN)FN)T)NFN).__name__ __module__ __qualname____doc__rlr rrKdictboolr@propertyrr$rDr#rNrQrJrrZrrgrfloatrrnr rrrvr r rrrrr(rrrrrrrrrrrr!r!5sk%)"#'-1)-<05/<0sm<0C= <0 4S>* <0  <0<0<0"#<0 <0D c # #  + + +##$H\2$$>#S## 873:#6 8hsm 8W[ 8 8-1  d38ni/0  $sCx.)       [v[HV4D[PT[[  'x}''' d38ni/0    &*!%-1 LLc]L L # L 4S>* L LL.!% ===# =  ==&9S949399 BXc]BDB[cdg[hBtwBB"DsDTDHUXMDehDD"!S!T!!. < rs #'66=&H>9?27A$g!'(:;q;q;r