Shortcuts

torchfunc.performance.technology

Analyse technological aspects (e.g. compatibility with Tensor Cores) of your module.

Using functionalities below you can check whether your architecture can use technology dependent speed improvements.

class torchfunc.performance.technology.TensorCores(linear_types=(<class 'torch.nn.modules.linear.Linear'>, <class 'torch.nn.modules.linear.Bilinear'>), convolution_types=(<class 'torch.nn.modules.conv.Conv1d'>, <class 'torch.nn.modules.conv.Conv2d'>, <class 'torch.nn.modules.conv.Conv3d'>), linear_inputs=None, linear_outputs=None, convolution_inputs=None, convolution_outputs=None, float_types=(torch.float16, ), integer_types=(torch.int16, ))[source]

Perform Tensor Cores compatibility tests for given module and it’s submodules/children.

Interpretation of data returned from this function may pose some problems to users unfamiliar with ideas standing behind Tensor Cores.

Is is advised to use method tips to get user friendly information your torch.nn.Module’s compatitilibty with Tensor Cores.

Example:

model = torch.nn.Sequential(
    torch.nn.Linear(128, 100).half(), # Half precision is compatible
    torch.nn.ReLU(),
    torch.nn.Linear(100, 50),
    torch.nn.ReLU(),
    torch.nn.Linear(50, 10).half(),
)

analysis = torchscripts.peformance.technology.TensorCores().children(model)
# Should return dictionary indicating problems with second Linear (wrong shape and type)
# And last Linear (wrong shape)
linear_types

Tuple of types to be considered linear and which should run with tensor cores kernels.

Default: (torch.nn.Linear, torch.nn.Bilinear)

Type

Tuple[torch.nn.Module], optional

convolution_types

Tuple of types to be considered convolutional and which should run with tensor cores kernels.

Default: (torch.nn.Conv1d, torch.nn.Conv2d, torch.nn.Conv3d)

Type

Tuple[torch.nn.Module], optional

linear_inputs

Dict-like where key is the type of module (e.g. torch.nn.Linear) and values are tuples of attribute names specifying names of input attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like in_features for torch.nn.Linear. More than one input can be specified, as is the case for torch.nn.Bilinear.

Default: {default_type: ("in_features",), torch.nn.Bilinear: ("in_features1", "in_features2")}

Type

Dict[torch.nn.Module, Tuple[str]], optional

linear_outputs

Dict-like where key is the type of module (e.g. torch.nn.Linear) and values are tuples of attribute names specifying names of output attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like out_features for torch.nn.Linear. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("out_features",)}

Type

Dict[torch.nn.Module, Tuple[str]], optional

convolution_inputs

Dict-like where key is the type of module (e.g. torch.nn.Conv2d) and values are tuples of attribute names specifying names of input channels attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like in_channels for all torch’s convolutions. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("in_channels",)}

Type

Dict[torch.nn.Module, Tuple[str]], optional

convolution_outputs

Dict-like where key is the type of module (e.g. torch.nn.Conv2d) and values are tuples of attribute names specifying names of output channels attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like out_channels for all torch’s convolutions. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("out_channels",)}

Type

Dict[torch.nn.Module, Tuple[str]], optional

float_types

Floating point types compatible with TensorCores.

Default: (torch.half, )

Type

typing.Tuple[types], optional

integer_types

Interger types compatible with TensorCores.

Default: (torch.short, )

Type

typing.Tuple[types], optional

children(module: torch.nn.modules.module.Module)[source]

Check Tensor Cores compatibility using children() method (shallow scanning).

Parameters

module (torch.nn.Module) – Module to be scanned for Tensor Cores compatibility

Returns

Multilevel dictionary describing modules incompatible with tensor cores. First level consists of two fields:

  • type: incompatible type with TensorCores

  • shape: incompatible types with TensorCores

Second level for type:

  • float: module is floating point type but it’s type is incompatible.

Contains list of submodule’s indices posing this problem. - integer: module is integer type but it’s type is incompatible Contains list of submodule’s indices posing this problem.

Second level for shape:

  • float: module is floating point type and has incorrect shape

  • integer: module is integer type and has incorrect shape

Third level for shape’s float and integer:

  • input: module’s input shape is incompatible with Tensor Cores

Contains list of submodule’s indices posing this problem. - output: module’s output shape is incompatible with Tensor Cores Contains list of submodule’s indices posing this problem.

As it’s hard to parse, it is suggested to use tips for readable output.

Return type

Nested dictionary

modules(module: torch.nn.modules.module.Module)[source]

Check Tensor Cores compatibility using modules() method (recursive scanning).

Parameters

module (torch.nn.Module) – Module to be scanned for Tensor Cores compatibility

Returns

Multilevel dictionary describing modules incompatible with tensor cores. First level consists of two fields:

  • type: incompatible type with TensorCores

  • shape: incompatible types with TensorCores

Second level for type:

  • float: module is floating point type but it’s type is incompatible.

Contains list of submodule’s indices posing this problem. - integer: module is integer type but it’s type is incompatible Contains list of submodule’s indices posing this problem.

Second level for shape:

  • float: module is floating point type and has incorrect shape

  • integer: module is integer type and has incorrect shape

Third level for shape’s float and integer:

  • input: module’s input shape is incompatible with Tensor Cores

Contains list of submodule’s indices posing this problem. - output: module’s output shape is incompatible with Tensor Cores Contains list of submodule’s indices posing this problem.

As it’s hard to parse, it is suggested to use tips for readable output.

Return type

Nested dictionary

tips(module: torch.nn.modules.module.Module) → str[source]

Return str representation of modules() method.

It is advised to use this function to get tips in order to easily fix possible performance issues related to Tensor Cores.

Parameters

module (torch.nn.Module) – Module to be scanned

Returns

String representing tips related to Tensor Cores.

Return type

str