torchfunc.performance.technology¶

Analyse technological aspects (e.g. compatibility with Tensor Cores) of your module.

Using functionalities below you can check whether your architecture can use technology dependent speed improvements.

class torchfunc.performance.technology.TensorCores(linear_types=(<class 'torch.nn.modules.linear.Linear'>, <class 'torch.nn.modules.linear.Bilinear'>), convolution_types=(<class 'torch.nn.modules.conv.Conv1d'>, <class 'torch.nn.modules.conv.Conv2d'>, <class 'torch.nn.modules.conv.Conv3d'>), linear_inputs=None, linear_outputs=None, convolution_inputs=None, convolution_outputs=None, float_types=(torch.float16, ), integer_types=(torch.int16, ))[source]¶

Perform Tensor Cores compatibility tests for given module and it’s submodules/children.

Interpretation of data returned from this function may pose some problems to users unfamiliar with ideas standing behind Tensor Cores.

Is is advised to use method tips to get user friendly information your torch.nn.Module’s compatitilibty with Tensor Cores.

Example:

model = torch.nn.Sequential(
    torch.nn.Linear(128, 100).half(), # Half precision is compatible
    torch.nn.ReLU(),
    torch.nn.Linear(100, 50),
    torch.nn.ReLU(),
    torch.nn.Linear(50, 10).half(),
)

analysis = torchscripts.peformance.technology.TensorCores().children(model)
# Should return dictionary indicating problems with second Linear (wrong shape and type)
# And last Linear (wrong shape)

linear_types¶

Tuple of types to be considered linear and which should run with tensor cores kernels.

Default: (torch.nn.Linear, torch.nn.Bilinear)

Type: Tuple[torch.nn.Module], optional

convolution_types¶

Tuple of types to be considered convolutional and which should run with tensor cores kernels.

Default: (torch.nn.Conv1d, torch.nn.Conv2d, torch.nn.Conv3d)

Type: Tuple[torch.nn.Module], optional

linear_inputs¶

Dict-like where key is the type of module (e.g. torch.nn.Linear) and values are tuples of attribute names specifying names of input attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like in_features for torch.nn.Linear. More than one input can be specified, as is the case for torch.nn.Bilinear.

Default: {default_type: ("in_features",), torch.nn.Bilinear: ("in_features1", "in_features2")}

Type: Dict[torch.nn.Module, Tuple[str]], optional

linear_outputs¶

Dict-like where key is the type of module (e.g. torch.nn.Linear) and values are tuples of attribute names specifying names of output attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like out_features for torch.nn.Linear. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("out_features",)}

Type: Dict[torch.nn.Module, Tuple[str]], optional

convolution_inputs¶

Dict-like where key is the type of module (e.g. torch.nn.Conv2d) and values are tuples of attribute names specifying names of input channels attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like in_channels for all torch’s convolutions. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("in_channels",)}

Type: Dict[torch.nn.Module, Tuple[str]], optional

convolution_outputs¶

Dict-like where key is the type of module (e.g. torch.nn.Conv2d) and values are tuples of attribute names specifying names of output channels attributes of this type of layer. You could use collections.defaultdict for easier specification of prevailing attribute names like out_channels for all torch’s convolutions. More than one output can be specified, same as linear_inputs.

Default: {default_type: ("out_channels",)}

Type: Dict[torch.nn.Module, Tuple[str]], optional

float_types¶

Floating point types compatible with TensorCores.

Default: (torch.half, )

Type: typing.Tuple[types], optional

integer_types¶

Interger types compatible with TensorCores.

Default: (torch.short, )

Type: typing.Tuple[types], optional

children(module: torch.nn.modules.module.Module)[source]¶

Check Tensor Cores compatibility using children() method (shallow scanning).

Parameters

module (torch.nn.Module) – Module to be scanned for Tensor Cores compatibility

Returns

Multilevel dictionary describing modules incompatible with tensor cores. First level consists of two fields:

type: incompatible type with TensorCores
shape: incompatible types with TensorCores

Second level for type:

float: module is floating point type but it’s type is incompatible.

Contains list of submodule’s indices posing this problem. - integer: module is integer type but it’s type is incompatible Contains list of submodule’s indices posing this problem.

Second level for shape:

float: module is floating point type and has incorrect shape
integer: module is integer type and has incorrect shape

Third level for shape’s float and integer:

input: module’s input shape is incompatible with Tensor Cores

Contains list of submodule’s indices posing this problem. - output: module’s output shape is incompatible with Tensor Cores Contains list of submodule’s indices posing this problem.

As it’s hard to parse, it is suggested to use tips for readable output.

Return type

Nested dictionary

modules(module: torch.nn.modules.module.Module)[source]¶

Check Tensor Cores compatibility using modules() method (recursive scanning).