torchtraining.callbacks.comet module¶

Integrate torchtraining with comet.ml experiment management tool.

Note

IMPORTANT: This module is experimental and may not be working correctly. Use at your own risk and report any issues you find.

Note

IMPORTANT: This module needs comet-ml Python package to be available. You can install it with pip install -U torchtraining[neptune]

Usage is similar to torchtraining.callbacks.Tensorboard, except creating Experiment instead of torch.utils.tensorboard.SummaryWriter.

Example:

import torchtraining as tt
import torchtraining.callbacks.comet as comet


class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        images, labels = sample
        ...
        return loss, images


project = comet.Experiment()
step = TrainStep(criterion, device)

# You have to split `tensor` as only single image can be logged
step ** tt.Select(1) ** tt.OnSplittedTensor(comet.Image(experiment))
step ** tt.Select(0) ** tt.cast.Item() ** comet.Scalar(experiment)

class torchtraining.callbacks.comet.Asset(experiment, name=None, overwrite=False, copy_to_tmp=True, step=None, metadata=None)[source]¶

Bases: torchtraining._base.Operation

Logs the Asset passed during call.

Parameters

experiment (Experiment) – Object representing single experiment
name (str, optional) – A custom file name to be displayed. If not provided the filename from the file_data argument will be used.
overwrite (bool, optional) – If True will overwrite all existing assets with the same name. Default: False
copy_to_tmp (bool, optional) – If file_data is a file-like object, then this flag determines if the file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud. Default: True
step (int, optional) – Used to associate the asset to a specific step. Default: None
metadata (typing.Dict, optional) – Optional. Some additional data to attach to the the asset data. Must be a JSON-encodable dict. Default: None

Returns

Data passed initially to the operation.

Return type

str | File-like

forward(data)[source]¶

Parameters: data (str | File-like) – Either the file path of the file you want to log, or a file-like asset.

class torchtraining.callbacks.comet.AssetData(experiment, name=None, overwrite=False, step=None, metadata=None)[source]¶

Bases: torchtraining._base.Operation

Log given data (str, binary or JSON).

Parameters

experiment (Experiment) – Object representing single experiment
name (str, optional) – A custom file name to be displayed. If not provided the filename from the file_data argument will be used.
overwrite (bool, optional) – If True will overwrite all existing assets with the same name. Default: False
step (int, optional) – Used to associate the asset to a specific step. Default: None
metadata (typing.Dict, optional) – Optional. Some additional data to attach to the the asset data. Must be a JSON-encodable dict. Default: None

Returns

Data passed initially to the operation.

Return type

str | File-like

forward(data)[source]¶

Parameters: data (str | JSON) – Data to log

class torchtraining.callbacks.comet.AssetFolder(experiment, step=None, log_file_name=False, recursive=False)[source]¶

Bases: torchtraining._base.Operation

Logs all the files located in the given folder as assets.

Parameters

experiment (Experiment) – Object representing single experiment
step (int, optional) – Used to associate the asset to a specific step. Default: None
log_file_name (bool, optional) – If True, log the file path with each file. Default: False
recursive (bool, optional) – If true recurse folder and save file names. Default: False

Returns

folder – Data passed initially to the operation.

Return type

str

forward(data)[source]¶

Parameters: folder (str) – Path to the folder to be logged.

class torchtraining.callbacks.comet.Audio(experiment, sample_rate=None, name=None, overwrite=False, copy_to_tmp=True, step=None, metadata=None)[source]¶

Bases: torchtraining._base.Operation

Logs the audio Asset determined by audio data.

Parameters

experiment (Experiment) – Object representing single experiment
sample_rate (int, optional) – The sampling rate given to scipy.io.wavfile.write for creating the wav file.
name (str, optional) – A custom file name to be displayed. If not provided the filename from the file_data argument will be used.
overwrite (bool, optional) – If True will overwrite all existing assets with the same name. Default: False
copy_to_tmp (bool, optional) – If file_data is a file-like object, then this flag determines if the file is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud. Default: True
step (int, optional) – Used to associate the asset to a specific step. Default: None
metadata (typing.Dict, optional) – Optional. Some additional data to attach to the the asset data. Must be a JSON-encodable dict. Default: None

Returns

data – Data passed initially to the operation.

Return type

str | np.array

forward(data)[source]¶

Parameters: data (str | np.array) – Either the file path of the file you want to log, or a numpy array given to scipy.io.wavfile.write for wav conversion.

class torchtraining.callbacks.comet.Clean(experiment)[source]¶

Bases: torchtraining._base.Operation

Clean experiment loggers

Parameters: experiment (Experiment) – Object representing single experiment
Returns: Data passed initially to the operation.
Return type: Any

forward(data)[source]¶

Parameters: data (Any) – Anything, will be forwarded

class torchtraining.callbacks.comet.ConfusionMatrix(experiment, title='Confusion Matrix', row_label='Actual Category', column_label='Predicted Category', max_examples_per_cell=25, max_categories=25, winner_function=None, index_to_example_function=None, cache=True, file_name='confusion-matrix.json', overwrite=False, step=None, **kwargs)[source]¶

Bases: torchtraining._base.Operation

Logs confusion matrix.

Parameters

experiment (Experiment) – Object representing single experiment
title (str, optional) – A custom name to be displayed. By default, it is “Confusion Matrix”.
row_label (str, optional) – Label for rows. By default, it is “Actual Category”.
column_label (str, optional) – Label for columns. By default, it is “Predicted Category”.
max_example_per_cell (int, optional) – Maximum number of examples per cell. By default, it is 25.
max_categories (int, optional) – Max number of columns and rows to use. By default, it is 25.
winner_function (Callable(List) -> List, optional) – A function that takes in an entire list of rows of patterns, and returns the winning category for each row. By default, it is argmax.
index_to_example_function (Callable, optional) – A function that takes an index and returns either a number, a string, a URL, or a {“sample”: str, “assetId”: str} dictionary. See below for more info. By default, the function returns a number representing the index of the example.
cache (bool, optional) – Should the results of index_to_example_function be cached and reused? By default, cache is True.
selected (List, optional) – None, or list of selected category indices. These are the rows/columns that will be shown. By default, select is None. If the number of categories is greater than max_categories, and selected is not provided, then selected will be computed automatically by selecting the most confused categories.
kwargs (optional) – any extra keywords and their values will be passed onto the index_to_example_function.

Returns

data – Data passed initially to the operation.

Return type

List[List[double]]

forward(data)[source]¶

Parameters: data (List[List[double]]) – Matrix-like list contianing confusion matrix.

class torchtraining.callbacks.comet.Curve(experiment, name=None, overwrite=False, step=None)[source]¶

Bases: torchtraining._base.Operation

Log timeseries data.

Parameters

experiment (Experiment) – Object representing single experiment
name (str) – Name of data
overwrite (bool, optional) – If True overwrite previous log. Default: False
step (int, optional) – Step value. Default: None

Returns

data – Data passed initially to operation

Return type

Tuple(List[Number], List[Number])

forward(data)[source]¶

Parameters: data (Tuple(List[Number], List[Number])) – Either the file path of the file you want to log, or a numpy array given to scipy.io.wavfile.write for wav conversion.

class torchtraining.callbacks.comet.Embedding(experiment, image_data=None, image_size=None, image_preprocess_function=None, image_transparent_color=None, image_background_color_function=None, title='Comet Embedding', template_filename='template_projector_config.json', group=None)[source]¶

Bases: torchtraining._base.Operation

Log a multi-dimensional dataset and metadata for viewing with Comet’s Embedding Projector.

This feature is currently deemed experimental.

Parameters

experiment (Experiment) – Object representing single experiment
image_data (List[Array] | Images, optional) – List of arrays or Images
image_size (int, optional (required if image_data is given)) – The size of each image
image_preprocess_function (Callable, optional) – If image_data is an array, apply this function to each element first
image_transparent_color (Tuple, optional) – (red, green, blue) tuple
image_background_color_function (Callable(int) -> Tuple(red, green, blue), optional) – A function that takes an index, and returns a (red, green, blue) color tuple
title (str, optional) – Name of tensor
template_filename (str, optional) – name of template JSON file

Returns

data – Tensors to visualize in 3D and labels for each tensor.

Return type

Tuple(torch.Tensor, torch.Tensor)

forward(data)[source]¶

Parameters: data (Tuple(torch.Tensor, torch.Tensor)) – Tensors to visualize in 3D and labels for each tensor.

class torchtraining.callbacks.comet.Figure(experiment, figure_name=None, overwrite=False, step=None)[source]¶

Bases: torchtraining._base.Operation

Logs the global Pyplot figure or the passed one and upload its svg version to the backend.

Parameters

experiment (Experiment) – Object representing single experiment
figure_name (str, optional) – Name of the figure
overwrite (bool, optional) – If another figure with the same name exists, it will be overwritten if overwrite is set to True. Default: False
step (int, optional) – Used to associate figure to a specific step.

Returns

data – Data passed initially to operation.

Return type

figure, optional.

forward(data)[source]¶

Parameters: data (figure, optional.) – The figure you want to log. If None passed, the global pyplot figure will be logged and uploaded

class torchtraining.callbacks.comet.Histogram3d(experiment, name=None, step=None, **kwargs)[source]¶

Bases: torchtraining._base.Operation

Logs a histogram of values for a 3D chart as an asset for this experiment.

Calling this method multiple times with the same name and incremented steps will add additional histograms to the 3D chart on Comet.ml.

Parameters

experiment (Experiment) – Object representing single experiment
name (str, optional) – Name of summary
step (int, optional.) – Used as the Z axis when plotting on Comet.ml.
**kwargs – Additional keyword arguments for histogram.

Returns

data – Summarization of histogram (passed to forward).

Return type

List | Tuple | Array | Histogram object

forward(data)[source]¶

Parameters: data (List | Tuple | Array | Histogram object) – Summarization of histogram

class torchtraining.callbacks.comet.Image(experiment, name=None, overwrite=False, image_format='png', image_scale=1.0, image_shape=None, image_colormap=None, image_minmax=None, image_channels='last', copy_to_tmp=True, step=None)[source]¶

Bases: torchtraining._base.Operation

Logs the image. Images are displayed on the Graphics tab on Comet.ml.

Parameters

experiment (Experiment) – Object representing single experiment
name (str, optional) – A custom name to be displayed on the dashboard. If not provided the filename from the image_data argument will be used if it is a path.
overwrite (bool, optional) – If another image with the same name exists, it will be overwritten if overwrite is set to True.
image_format (str, optional) – Default: ‘png’. If the image_data is actually something that can be turned into an image, this is the format used. Typical values include ‘png’ and ‘jpg’.
image_scale (float, optional) – Default: 1.0. If the image_data is actually something that can be turned into an image, this will be the new scale of the image.
image_shape (Tuple, optional) – Default: None. If the image_data is actually something that can be turned into an image, this is the new shape of the array. Dimensions are (width, height).
image_colormap (str, optional) – If the image_data is actually something that can be turned into an image, this is the colormap used to colorize the matrix.
image_minmax ((Number, Number), optional) – If the image_data is actually something that can be turned into an image, this is the (min, max) used to scale the values. Otherwise, the image is autoscaled between (array.min, array.max).
image_channels (str, optional. Default 'last'.) – If the image_data is actually something that can be turned into an image, this is the setting that indicates where the color information is in the format of the 2D data. ‘last’ indicates that the data is in (rows, columns, channels) where ‘first’ indicates (channels, rows, columns).
copy_to_tmp (bool, optional) – If image_data is not a file path, then this flag determines if the image is first copied to a temporary file before upload. If copy_to_tmp is False, then it is sent directly to the cloud. Default: True
step (int, optional) – Used to associate the audio asset to a specific step. Default: None

Returns

data – See forward for possibilities

Return type

Multiple objects

forward(data)[source]¶

Parameters

data (Multiple objects) –

One of:

a path (string) to an image
a file-like object containing an image
a numpy matrix
a TensorFlow tensor
a PyTorch tensor
a list or tuple of values
a PIL Image

class torchtraining.callbacks.comet.Notification(experiment, title, status=None, additional_data=None)[source]¶

Bases: torchtraining._base.Operation

Send yourself a notification through email when an experiment ends.

Parameters

experiment (Experiment) – Object representing single experiment
title (str) – Subject of the email
status (str) – Final status of the experiment. Typically, something like “finished”, “completed” or “aborted”.
additional_data (dict) – Dictionary of key/values to notify.

Returns

data – Anything passed to forward

Return type

Any

forward(data)[source]¶

Parameters: data (Any) – Anything as it’s not send to the function, just passes through

class torchtraining.callbacks.comet.Other(experiment)[source]¶

Bases: torchtraining._base.Operation

Reports a key and value to the Other tab on Comet.ml.

Useful for reporting datasets attributes, datasets path, unique identifiers etc.

Parameters: experiment (Experiment) – Object representing single experiment
Returns: data – Tuple with key and value
Return type: Tuple[Any, Any]

forward(data)[source]¶

Parameters: data (Tuple[Any, Any]) – Tuple with key and value

class torchtraining.callbacks.comet.Others(experiment)[source]¶

Bases: torchtraining._base.Operation

Reports dictionary of key/values to the Other tab on Comet.ml.

Useful for reporting datasets attributes, datasets path, unique identifiers etc.

Parameters: experiment (Experiment) – Object representing single experiment
Returns: data – Dict with any keys and values
Return type: Dict[Any, Any]

forward(data)[source]¶

Parameters: data (Dict[Any, Any]) – Dict with any keys and values

class torchtraining.callbacks.comet.Scalar(experiment, name, step=None, epoch=None, include_context=True)[source]¶

Bases: torchtraining._base.Operation

Logs scalar value under specified name.

Usually used to log metric values (like accuracy)

Parameters

experiment (Experiment) – Object representing single experiment
name (str) – Name of scalar / metric.
step (int, optional) – Used as the X axis when plotting on comet.ml
epoch (int, optional) – Used as the X axis when plotting on comet.ml
include_context (bool, optional) – If set to True (the default), the current context will be logged along the metric.

Returns

data – Value passed to forward

Return type

Number

forward(data)[source]¶

Parameters: data (Number) – Value to log

class torchtraining.callbacks.comet.Scalars(experiment, prefix=None, step=None, epoch=None)[source]¶

Bases: torchtraining._base.Operation

Logs dictionary of scalars / metrics.

Usually used to log metric values (like accuracy)

Parameters

experiment (Experiment) – Object representing single experiment
prefix (str, optional) – Name of prefix used when logging into comet.ml
step (int, optional) – Used as the X axis when plotting on comet.ml
epoch (int, optional) – Used as the X axis when plotting on comet.ml

Returns

data – Dictionary passed to forward

Return type

Dict

forward(data)[source]¶

Parameters: data (Dict) – Dictionary with values to log

class torchtraining.callbacks.comet.Table(experiment, filename: str, headers: Union[bool, List] = False)[source]¶

Bases: torchtraining._base.Operation

Logs tabular data.

These strings appear on the Text Tab in the Comet UI.

Parameters

experiment (Experiment) – Object representing single experiment
filename (str) – Filename ending in “.csv”, or “.tsv”
headers (bool | List) – If True, will add column headers automatically if tabular_data is given; if False, no headers will be added; if list then it will be used as headers.

Returns

data – Data received in forward

Return type

tensor | List[List]

forward(data)[source]¶

Parameters: data (tensor | List[List]) – Data that can be interpreted as 2D tabular data.

class torchtraining.callbacks.comet.Text(experiment, step: int = None, metadata=None)[source]¶

Bases: torchtraining._base.Operation

Logs the text.

These strings appear on the Text Tab in the Comet UI.

Parameters

experiment (Experiment) – Object representing single experiment
step (int, optional) – Used to associate text to a specific step
metadata (JSON-like, optional) – Additional data attached to text.

Returns

data – Received text

Return type

str

forward(data)[source]¶

Parameters: data (str) – Text to be stored