Deep networks

Contents

• Deep networks
  • Deep network implementations
  • Noisy convolution

Deep network implementations

mermaid.deep_networks.DimNoisyConv(dim)

mermaid.deep_networks.DimConv(dim)

mermaid.deep_networks.DimBatchNorm(dim)

mermaid.deep_networks.DimInstanceNorm(dim)

mermaid.deep_networks.DimNormalization(dim, normalization_type, nr_channels, im_sz)

mermaid.deep_networks.DimConvTranspose(dim)

mermaid.deep_networks.DimNoisyConvTranspose(dim)

mermaid.deep_networks.DimMaxPool(dim)

class mermaid.deep_networks.conv_norm_in_rel(dim, in_channels, out_channels, kernel_size, im_sz, stride=1, active_unit='relu', same_padding=False, normalization_type='layer', reverse=False, group=1, dilation=1, use_noisy_convolution=False, noisy_convolution_std=0.25, noisy_convolution_optimize_over_std=False, use_noise_layer=False, noise_layer_std=0.25, start_reducing_from_iter=0)

forward(x, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.encoder_block_2d(input_feature, output_feature, im_sz, use_dropout, normalization_type, dim)

apply_dropout(input)

forward_with_normalization(x)

forward_without_normalization(x)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.decoder_block_2d(input_feature, output_feature, im_sz, pooling_filter, use_dropout, normalization_type, dim, last_block=False)

apply_dropout(input)

forward_with_normalization(x)

forward_without_normalization(x)

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.DeepNetwork(dim, n_in_channel, n_out_channel, im_sz, params)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

initialize_network_weights()

class mermaid.deep_networks.Unet(dim, n_in_channel, n_out_channel, im_sz, params)

unet include 4 down path (1/16) and 4 up path (16)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

forward(x, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.Simple_Unet(dim, n_in_channel, n_out_channel, im_sz, params)

unet include 4 down path (1/16) and 4 up path (16)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

forward(x, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.Encoder_decoder(dim, n_in_channel, n_out_channel, im_sz, params)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.Simple_consistent(dim, n_in_channel, n_out_channel, im_sz, params)

apply_dropout(input)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

forward(x, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.Unet_no_skip(dim, n_in_channel, n_out_channel, im_sz, params)

unet include 4 down path (1/16) and 4 up path (16)

get_last_kernel_size()

Returns the size of the kernel along one direction (needs to be taken to the power of the dimension) for the last convolutional layer. This allows for example to scale numerical algorithms with respect to it. :return:

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.WeightRangeLoss(dim, decay_factor, weight_type)

forward(x, spacing, weights)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

cal_weights_for_weightrange(epoch)

class mermaid.deep_networks.WeightInputRangeLoss

forward(x, spacing, use_weighted_linear_softmax=False, weights=None, min_weight=0.0, max_weight=1.0, dim=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.HLoss

forward(x, spacing)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.GlobalHLoss

forward(x, spacing)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.OMTLoss(spacing, desired_power, use_log_transform, params, img_sz)

OMT Loss function

compute_omt_penalty(weights, multi_gaussian_stds)

cal_weights_for_omt(epoch)

forward(weights, gaussian_stds)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.TotalVariationLoss(dim, im_sz, spacing, use_omt_weighting=False, gaussian_stds=None, omt_power=1.0, omt_use_log_transformed_std=True, params=None)

Loss function to penalize total variation

params = None

ParameterDict() parameters

compute_local_weighted_tv_norm(I, weights, spacing, nr_of_gaussians, use_color_tv, pnorm=2)

forward(input_images, label_probabilities, use_color_tv=False)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.deep_networks.ClusteringLoss(dim, params)

Loss function for image clustering (this is here a relaxation of normalized cuts)

params = None

ParameterDict() parameters

get_last_kernel_size()

forward(input_images, spacing, label_probabilities)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Noisy convolution

class mermaid.noisy_convolution.NoisyLinear(in_features, out_features, bias=True, factorised=True, std_init=None)

Applies a noisy linear transformation to the incoming data: \(y = (mu_w + sigma_w \cdot epsilon_w)x + mu_b + sigma_b \cdot epsilon_b\) More details can be found in the paper ZZ _ . :param in_features: size of each input sample :param out_features: size of each output sample :param bias: If set to False, the layer will not learn an additive bias. Default: True :param factorised: whether or not to use factorised noise. Default: True :param std_init: initialization constant for standard deviation component of weights. If None, defaults to 0.017 for independent and 0.4 for factorised. Default: None

Shape:

• Input: (N, in_features)
• Output:(N, out_features)

weight

the learnable weights of the module of shape (out_features x in_features)

bias

the learnable bias of the module of shape (out_features)

Examples::

>>> m = nn.NoisyLinear(20, 30)
>>> input = autograd.Variable(torch.randn(128, 20))
>>> output = m(input)
>>> print(output.size())

reset_parameters(bias)

scale_noise(size)

forward(input)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyLayer(std_init=None, start_reducing_from_iter=25)

forward(input, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConv1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 1D noisy convolution over an input signal composed of several input planes.

In the simplest case, the output value of the layer with input size \((N, C_{in}, L)\) and output \((N, C_{out}, L_{out})\) can be precisely described as:

\[\begin{equation*} \text{out}(N_i, C_{out_j}) = \text{bias}(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{out_j}, k) \star \text{input}(N_i, k) \end{equation*},\]

where \(\star\) is the valid cross-correlation operator, \(N\) is a batch size, \(C\) denotes a number of channels, \(L\) is a length of signal sequence.

• stride controls the stride for the cross-correlation, a single number or a one-element tuple.

• padding controls the amount of implicit zero-paddings on both sides for padding number of points.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor \frac{\text{out_channels}}{\text{in_channels}} \right\rfloor\)).

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The configuration when groups == in_channels and out_channels == K * in_channels where K is a positive integer is termed in literature as depthwise convolution.

In other words, for an input of size \((N, C_{in}, L_{in})\), if you want a depthwise convolution with a depthwise multiplier K, then you use the constructor arguments \((\text{in_channels}=C_{in}, \text{out_channels}=C_{in} * K, ..., \text{groups}=C_{in})\)

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – Zero-padding added to both sides of the input. Default: 0
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

Shape:

• Input: \((N, C_{in}, L_{in})\)

• Output: \((N, C_{out}, L_{out})\) where

  \[L_{out} = \left\lfloor\frac{L_{in} + 2 \times \text{padding} - \text{dilation} \times (\text{kernel_size} - 1) - 1}{\text{stride}} + 1\right\rfloor\]

weight

the learnable weights of the module of shape (out_channels, in_channels, kernel_size)

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

Examples:

>>> m = nn.NoisyConv1d(16, 33, 3, stride=2)
>>> input = torch.randn(20, 16, 50)
>>> output = m(input)

forward(input, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 2D noisy convolution over an input signal composed of several input planes.

In the simplest case, the output value of the layer with input size \((N, C_{in}, H, W)\) and output \((N, C_{out}, H_{out}, W_{out})\) can be precisely described as:

\[\begin{equation*} \text{out}(N_i, C_{out_j}) = \text{bias}(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{out_j}, k) \star \text{input}(N_i, k) \end{equation*},\]

where \(\star\) is the valid 2D cross-correlation operator, \(N\) is a batch size, \(C\) denotes a number of channels, \(H\) is a height of input planes in pixels, and \(W\) is width in pixels.

• stride controls the stride for the cross-correlation, a single number or a tuple.

• padding controls the amount of implicit zero-paddings on both sides for padding number of points for each dimension.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, dilation can either be:

• a single int – in which case the same value is used for the height and width dimension
• a tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The configuration when groups == in_channels and out_channels == K * in_channels where K is a positive integer is termed in literature as depthwise convolution.

In other words, for an input of size \((N, C_{in}, H_{in}, W_{in})\), if you want a depthwise convolution with a depthwise multiplier K, then you use the constructor arguments \((\text{in_channels}=C_{in}, \text{out_channels}=C_{in} * K, ..., \text{groups}=C_{in})\)

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – Zero-padding added to both sides of the input. Default: 0
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

Shape:

• Input: \((N, C_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\\W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\end{aligned}\end{align} \]

weight

the learnable weights of the module of shape (out_channels, in_channels, kernel_size[0], kernel_size[1])

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

Examples:

>>> # With square kernels and equal stride
>>> m = nn.NoisyConv2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.NoisyConv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> # non-square kernels and unequal stride and with padding and dilation
>>> m = nn.NoisyConv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)

forward(input, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConv3d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 3D noisy convolution over an input signal composed of several input planes.

In the simplest case, the output value of the layer with input size \((N, C_{in}, D, H, W)\) and output \((N, C_{out}, D_{out}, H_{out}, W_{out})\) can be precisely described as:

\[\begin{equation*} \text{out}(N_i, C_{out_j}) = \text{bias}(C_{out_j}) + \sum_{k = 0}^{C_{in} - 1} \text{weight}(C_{out_j}, k) \star \text{input}(N_i, k) \end{equation*},\]

where \(\star\) is the valid 3D cross-correlation operator

• stride controls the stride for the cross-correlation.

• padding controls the amount of implicit zero-paddings on both sides for padding number of points for each dimension.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, dilation can either be:

• a single int – in which case the same value is used for the depth, height and width dimension
• a tuple of three ints – in which case, the first int is used for the depth dimension, the second int for the height dimension and the third int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The configuration when groups == in_channels and out_channels == K * in_channels where K is a positive integer is termed in literature as depthwise convolution.

In other words, for an input of size \((N, C_{in}, D_{in}, H_{in}, W_{in})\), if you want a depthwise convolution with a depthwise multiplier K, then you use the constructor arguments \((\text{in_channels}=C_{in}, \text{out_channels}=C_{in} * K, ..., \text{groups}=C_{in})\)

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – Zero-padding added to all three sides of the input. Default: 0
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

Shape:

• Input: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, D_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor\\H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor\\W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - \text{dilation}[2] \times (\text{kernel_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor\end{aligned}\end{align} \]

weight

the learnable weights of the module of shape (out_channels, in_channels, kernel_size[0], kernel_size[1], kernel_size[2])

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

Examples:

>>> # With square kernels and equal stride
>>> m = nn.NoisyConv3d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.NoisyConv3d(16, 33, (3, 5, 2), stride=(2, 1, 1), padding=(4, 2, 0))
>>> input = torch.randn(20, 16, 10, 50, 100)
>>> output = m(input)

forward(input)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConvTranspose1d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 1D noisy transposed convolution operator over an input image composed of several input planes.

This module can be seen as the gradient of Conv1d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation).

• stride controls the stride for the cross-correlation.

• padding controls the amount of implicit zero-paddings on both sides for kernel_size - 1 - padding number of points. See note below for details.

• output_padding controls the additional size added to one side of the output shape. See note below for details.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The padding argument effectively adds kernel_size - 1 - padding amount of zero padding to both sizes of the input. This is set so that when a Conv1d and a ConvTranspose1d are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when stride > 1, Conv1d maps multiple input shapes to the same output shape. output_padding is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that output_padding is only used to find output shape, but does not actually add zero-padding to output.

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – kernel_size - 1 - padding zero-padding will be added to both sides of the input. Default: 0
• output_padding (int or tuple, optional) – Additional size added to one side of the output shape. Default: 0
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

Shape:

• Input: \((N, C_{in}, L_{in})\)

• Output: \((N, C_{out}, L_{out})\) where

  \[L_{out} = (L_{in} - 1) \times \text{stride} - 2 \times \text{padding} + \text{kernel_size} + \text{output_padding}\]

weight

the learnable weights of the module of shape (in_channels, out_channels, kernel_size[0], kernel_size[1])

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

forward(input, output_size=None, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConvTranspose2d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 2D noisy transposed convolution operator over an input image composed of several input planes.

This module can be seen as the gradient of Conv2d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation).

• stride controls the stride for the cross-correlation.

• padding controls the amount of implicit zero-paddings on both sides for kernel_size - 1 - padding number of points. See note below for details.

• output_padding controls the additional size added to one side of the output shape. See note below for details.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, output_padding can either be:

• a single int – in which case the same value is used for the height and width dimensions
• a tuple of two ints – in which case, the first int is used for the height dimension, and the second int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The padding argument effectively adds kernel_size - 1 - padding amount of zero padding to both sizes of the input. This is set so that when a Conv2d and a ConvTranspose2d are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when stride > 1, Conv2d maps multiple input shapes to the same output shape. output_padding is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that output_padding is only used to find output shape, but does not actually add zero-padding to output.

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – kernel_size - 1 - padding zero-padding will be added to both sides of each dimension in the input. Default: 0
• output_padding (int or tuple, optional) – Additional size added to one side of each dimension in the output shape. Default: 0
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

Shape:

• Input: \((N, C_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}H_{out} = (H_{in} - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{kernel_size}[0] + \text{output_padding}[0]\\W_{out} = (W_{in} - 1) \times \text{stride}[1] - 2 \times \text{padding}[1] + \text{kernel_size}[1] + \text{output_padding}[1]\end{aligned}\end{align} \]

weight

the learnable weights of the module of shape (in_channels, out_channels, kernel_size[0], kernel_size[1])

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

Examples:

>>> # With square kernels and equal stride
>>> m = nn.NoisyConvTranspose2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.NoisyConvTranspose2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)
>>> # exact output size can be also specified as an argument
>>> input = torch.randn(1, 16, 12, 12)
>>> downsample = nn.NoisyConv2d(16, 16, 3, stride=2, padding=1)
>>> upsample = nn.NoisyConvTranspose2d(16, 16, 3, stride=2, padding=1)
>>> h = downsample(input)
>>> h.size()
torch.Size([1, 16, 6, 6])
>>> output = upsample(h, output_size=input.size())
>>> output.size()
torch.Size([1, 16, 12, 12])

forward(input, output_size=None, iter=0)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class mermaid.noisy_convolution.NoisyConvTranspose3d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, scalar_sigmas=True, optimize_sigmas=False, std_init=None, start_reducing_from_iter=25)

Applies a 3D noisy transposed convolution operator over an input image composed of several input planes. The transposed convolution operator multiplies each input value element-wise by a learnable kernel, and sums over the outputs from all input feature planes.

This module can be seen as the gradient of Conv3d with respect to its input. It is also known as a fractionally-strided convolution or a deconvolution (although it is not an actual deconvolution operation).

• stride controls the stride for the cross-correlation.

• padding controls the amount of implicit zero-paddings on both sides for kernel_size - 1 - padding number of points. See note below for details.

• output_padding controls the additional size added to one side of the output shape. See note below for details.

• dilation controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.

• groups controls the connections between inputs and outputs. in_channels and out_channels must both be divisible by groups. For example,

  • At groups=1, all inputs are convolved to all outputs.
  • At groups=2, the operation becomes equivalent to having two conv layers side by side, each seeing half the input channels, and producing half the output channels, and both subsequently concatenated.
  • At groups= in_channels, each input channel is convolved with its own set of filters (of size \(\left\lfloor\frac{\text{out_channels}}{\text{in_channels}}\right\rfloor\)).

The parameters kernel_size, stride, padding, output_padding can either be:

• a single int – in which case the same value is used for the depth, height and width dimensions
• a tuple of three ints – in which case, the first int is used for the depth dimension, the second int for the height dimension and the third int for the width dimension

Note

Depending of the size of your kernel, several (of the last) columns of the input might be lost, because it is a valid cross-correlation, and not a full cross-correlation. It is up to the user to add proper padding.

Note

The padding argument effectively adds kernel_size - 1 - padding amount of zero padding to both sizes of the input. This is set so that when a Conv3d and a ConvTranspose3d are initialized with same parameters, they are inverses of each other in regard to the input and output shapes. However, when stride > 1, Conv3d maps multiple input shapes to the same output shape. output_padding is provided to resolve this ambiguity by effectively increasing the calculated output shape on one side. Note that output_padding is only used to find output shape, but does not actually add zero-padding to output.

Parameters:

• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. Default: 1
• padding (int or tuple, optional) – kernel_size - 1 - padding zero-padding will be added to both sides of each dimension in the input. Default: 0
• output_padding (int or tuple, optional) – Additional size added to one side of each dimension in the output shape. Default: 0
• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

Shape:

• Input: \((N, C_{in}, D_{in}, H_{in}, W_{in})\)

• Output: \((N, C_{out}, D_{out}, H_{out}, W_{out})\) where

  \[ \begin{align}\begin{aligned}D_{out} = (D_{in} - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{kernel_size}[0] + \text{output_padding}[0]\\H_{out} = (H_{in} - 1) \times \text{stride}[1] - 2 \times \text{padding}[1] + \text{kernel_size}[1] + \text{output_padding}[1]\\W_{out} = (W_{in} - 1) \times \text{stride}[2] - 2 \times \text{padding}[2] + \text{kernel_size}[2] + \text{output_padding}[2]\end{aligned}\end{align} \]

weight

the learnable weights of the module of shape (in_channels, out_channels, kernel_size[0], kernel_size[1], kernel_size[2])

Type:Tensor

bias

the learnable bias of the module of shape (out_channels)

Type:Tensor

Examples:

>>> # With square kernels and equal stride
>>> m = nn.NoisyConvTranspose3d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.NoisyConv3d(16, 33, (3, 5, 2), stride=(2, 1, 1), padding=(0, 4, 2))
>>> input = torch.randn(20, 16, 10, 50, 100)
>>> output = m(input)

forward(input, output_size=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.