Why ml_logger¶
People use different tools for logging experimental results - Tensorboard, Wandb etc to name a few. Working with different collaborators, I will have to switch my logging tool with each new project. So I made this simple tool that provides a common interface to logging results to different loggers.
Installation¶
pip install "mllogger[all]"
If you want to use only the filesystem logger, use
pip install "mllogger"
Install from source
git clone git@github.com:shagunsodhani/ml-logger.git
cd ml-logger
pip install ".[all]"
Alternatively,
pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger[all]"
If you want to use only the filesystem logger, use pip install .
or
pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger"
.
Use¶
Make a
logbook_config
:from ml_logger import logbook as ml_logbook logbook_config = ml_logbook.make_config( logger_dir = <path to write logs>, wandb_config = <wandb config or None>, tensorboard_config = <tensorboard config or None>, mlflow_config = <mlflow config or None>)
The API for
make_config
can be accessed here.Make a
LogBook
instance:logbook = ml_logbook.LogBook(config = logbook_config)
Use the
logbook
instance:log = { "epoch": 1, "loss": 0.1, "accuracy": 0.2 } logbook.write_metric(log)
The API for
write_metric
can be accessed here.
Note¶
If you are writing to wandb, the
log
must have a key calledstep
. If yourlog
already captures thestep
but as a different key (sayepoch
), you can pass thewandb_key_map
argument (set as{epoch: step}
). For more details, refer the documentation here.If you are writing to mlflow, the
log
must have a key calledstep
. If yourlog
already captures thestep
but as a different key (sayepoch
), you can pass themlflow_key_map
argument (set as{epoch: step}
). For more details, refer the documentation here.If you are writing to tensorboard, the
log
must have a key calledmain_tag
ortag
which acts as the data Identifier and another key calledglobal_step
. These keys are described here. If yourlog
already captures these values but as different key (saymode
formain_tag
andepoch
forglobal_step
), you can pass thetensorboard_key_map
argument (set as{mode: main_tag, epoch: global_step}
). For more details, refer the documentation here.
Dev Setup¶
pip install -e ".[dev]"
Install pre-commit hooks
pre-commit install
The code is linted using:
black
flake8
mypy
isort
Tests can be run locally using
nox