torchtraining

Classes

Root module of torchtraining containing common operations.

Note

IMPORTANT: This module is one of the most important and is used in almost any DL task so be sure to understand it!

Operations in this module can be used to:

• control pipeline flow

• select output from `step`s

• send data to multiple operations

See below for more info.

class torchtraining.Flatten(types: Tuple = (<class 'list'>, <class 'tuple'>))

Flatten arbitrarily nested data.

Example:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        module1, module2 = module
        ...
        return ((logits1, targets1), (logits2, targets2), module1, module2)


step = TrainStep(criterion, device)

# Tuple (logits1, targets1, logits2, targets2, module1, module2)
step ** tt.Flatten()

Parameters

types (Tuple[type], optional) – Types to be considered non-flat. Those will be recursively flattened. Default: (list, tuple)

Returns

Single tuple with all elements (not being tuple or list).

Return type

Tuple[samples]

forward(data) → List[Any]

Parameters

data (Iterable[Iterable ... Iterable[Any]]) – Arbitrarily nested data being one of type provided in types.

class torchtraining.If(condition: Union[bool, Callable[Any, bool]], operation: Callable[Any, Any])

Run operation only If condition is True.

condition can also be a single argument callable, in this case it can be dependent on data, see below:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        ...
        return loss


step = TrainStep(criterion, device)
step ** tt.If(lambda loss: loss ** 10, tt.callbacks.Logger("VERY HIGH LOSS!!!"))

Parameters

• condition (bool | Callable(Any) -> bool) – If boolean value and if True, run underlying Operation (or other Callable). If Callable, should take data as argument and return decision based on that as single bool.

• operation (torchtraining.Operation | Callable) – Operation to run if True

Returns

If true, returns value from operation, otherwise passes original data

Return type

Any

forward(data: Any) → Any

Parameters

data (Any) – Anything you want (usually torch.Tensor like stuff).

class torchtraining.IfElse(condition: bool, operation1: Callable[Any, Any], operation2: Callable[Any, Any])

Run operation1 only if condition is True, otherwise run operation2.

condition can also be a single argument callable, in this case it can be dependent on data, see below:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        ...
        return loss


step = TrainStep(criterion, device)

step ** tt.IfElse(
    lambda loss: loss ** 10,
    tt.callbacks.Logger("VERY HIGH LOSS!!!"),
    tt.callbacks.Logger("LOSS IS NOT THAT HIGH..."),
)

Parameters

• condition (bool) – Boolean value. If true, run underlying Op (or other Callable).

• operation1 (torchtraining.Op | Callable) – Operation or callable getting single argument (data) and returning anything.

• operation2 (torchtraining.Op | Callable) – Operation or callable getting single argument (data) and returning anything.

Returns

If true, returns value from operation1, otherwise from operation2.

Return type

Any

forward(data: Any) → Any

Parameters

data (Any) – Anything you want (usually torch.Tensor like stuff).

class torchtraining.Lambda(operation: Callable[Any, Any], name: str = 'torchtraining.Lambda')

Run user specified operation on data.

Example:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        ...
        return accuracy


step = TrainStep(criterion, device)

# If you want to get that SOTA badly, we got ya covered
step ** tt.Lambda(lambda accuracy: accuracy * 2)

Parameters

• operation (Callable(Any) -> Any) – Single argument callable getting data and returning some value.

• name (str, optional) – string representation of this operation (if any). Default: torchtraining.metrics.Lambda

Returns

Value returned from operation

Return type

Any

forward(data: Any) → Any

Parameters

data (Any) – Anything you want (usually torch.Tensor like stuff).

class torchtraining.OnSplittedTensor(operation: torchtraining._base.Operation, dim: int = 0)

Split tensor along dimension and apply operation on each element.

By default, torch.Tensor will be splitted along batch (dim=0)

Note

IMPORTANT: After splitting first dimension is squeezed via torch.squeeze.

Example:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        # Dummy step
        images, labels
        return images


step = TrainStep(criterion, device)

# Assume summary_writer is instance of torch.utils.tensorboard.SummaryWriter()
step ** tt.OnSplittedTensor(tt.callbacks.tensorboard.Image(summary_writer))
# Each image will be saved separately

Parameters

• operation (tt.Operation | Callable(data) -> Any) – Operation which will be applied to each element of torch.Tensor.

• dim (int, optional) – Dimension along which data torch.Tensor will be splitted. Default: 0

Returns

Splitted data along dim (unmodified by operation).

Return type

Tuple[torch.Tensor]

forward(data)

Parameters

data (torch.Tensor) – Tensor to split and apply operation on.

class torchtraining.Select(**output_selection: int)

Select output item(s) returned from step or iteration objects.

Allows users to focus on specific part of output and pipe specified values to other operations (like metrics, loggers etc.).

Note

IMPORTANT: This operation is run in almost any case so be sure to understand how it works.

Example:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        # Generate loss and other necessary items
        ...
        return loss, predictions, targets


step = TrainStep(criterion, device)
# Select `loss` and perform backpropagation
# Only single value will be forward to backward from
# (loss, predictions, targets) tuple
step ** tt.Select(loss=0) ** tt.pytorch.Backward()

Note

Name of keyword argument can be arbitrary but you are really encouraged to name it like the variable returned from step (or at least make it understandable to others).

Parameters

output_selection (**output_selection) – name: output_index mapping selecting which element from step returned tuple to choose. name can be arbitrary, but should be named like the variable returned from step. See example above.

Returns

If single int is passed output_selection return single element from Iterable. Otherwise returns chosen elements as list

Return type

Any | List[Any]

forward(data: Iterable[Any]) → Any

Parameters

data (Iterable[Any]) – Iterable (e.g. tuple, list) with any elements.

class torchtraining.Split(*operations: Callable[Any, Any], return_modified: bool = False)

Split operation with data to multiple components.

Note

IMPORTANT: This operation is run in almost any case so be sure to understand how it works.

Useful when users wish to use calculated result in multiple places. Example calculating metrics and logging them in multiple places:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        # Generate loss and other necessary items
        ...
        # Assume binary classification
        return loss, logits, targets


step = TrainStep(criterion, device)

# Push (logits, targets) to Precision and Recall
# and log those values after calculating metrics
step ** tt.Select(logits=1, targets=2) ** tt.Split(
    tt.metrics.classification.binary.Precision() ** tt.callbacks.Logger("Precision"),
    tt.metrics.classification.binary.Recall() ** tt.callbacks.Logger("Recall"),
)

Parameters

• *operations (Callable(data) -> Any) – Operations to which results will be passed.

• return_modified (bool, optional) – Return outputs from operations as a list if True. If False, returns original data passed into Split. Default: False

Returns

Returns data passed originally or list containing modified data returned from operations.

Return type

data | List[modified data]

forward(data: Any) → Union[Any, List[Any]]

Parameters

data (Any) – Data which will be passed to provided operations.

class torchtraining.ToAll(operation: Callable)

Apply operation to each element of sample.**

Note

If you want to apply operation to all nested elements (e.g. in nested tuple), please use torchtraining.Flatten object first.

Example:

class TrainStep(tt.steps.Train):
    def forward(self, module, sample):
        ...
        return loss


step = TrainStep(criterion, device)

step ** tt.If(
    lambda loss: loss ** 10,
    tt.callbacks.Logger("VERY HIGH LOSS!!!"),
    tt.callbacks.Logger("LOSS IS NOT THAT HIGH..."),
)

Parameters

operation (Callable) – Pipe to apply to each element of sample.

Returns

Tuple consisting of subsamples with operation applied.

Return type

Tuple[operation(subsample)]

forward(sample)

Parameters

data (Any) – Anything you want (usually torch.Tensor like stuff).