elektronn2.neuromancer.neural module¶

class elektronn2.neuromancer.neural.Perceptron(**kwargs)[source]¶

Bases: elektronn2.neuromancer.neural.NeuralLayer

Perceptron Layer.

Parameters:

parent (Node or list of Node) – The input node(s).
n_f (int) – Number of filters (nodes) in layer.
activation_func (str) – Activation function name.
flatten (bool) –
batch_normalisation (str or None) – Batch normalisation mode. Can be False (inactive), “train” or “fadeout”.
dropout_rate (float) – Dropout rate (probability that a node drops out in a training step).
name (str) – Perceptron name.
print_repr (bool) – Whether to print the node representation upon initialisation.
w (np.ndarray or T.TensorVariable) – Weight matrix. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
b (np.ndarray or T.TensorVariable) – Bias vector. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
gamma – (For batch normalisation) Initialises gamma parameter.
mean – (For batch normalisation) Initialises mean parameter.
std – (For batch normalisation) Initialises std parameter.
gradnet_mode –

make_dual(parent, share_w=False, **kwargs)[source]¶

Create the inverse of this Perceptron.

Most options are the same as for the layer itself. If kwargs are not specified, the values of the primal layers are re-used and new parameters are created.

Parameters:	parent (Node) – The input node. share_w (bool) – If the weights (`w`) should be shared from the primal layer. kwargs (dict) – kwargs that are passed through to the constructor of the inverted Perceptron (see signature of `Perceptron`). `n_f` is copied from the existing node on which `make_dual` is called. Every other parameter can be changed from the original `Perceptron`’s defaults by specifying it in `kwargs`.
Returns:	The inverted perceptron layer.
Return type:	Perceptron

class elektronn2.neuromancer.neural.Conv(**kwargs)[source]¶

Bases: elektronn2.neuromancer.neural.Perceptron

Convolutional layer with subsequent pooling.

Examples

Examples for constructing convolutional neural networks can be found in examples/neuro3d.py and examples/mnist.py.

Parameters:

parent (Node) – The input node.
n_f (int) – Number of features.
filter_shape (tuple) – Shape of the convolution filter kernels.
pool_shape (tuple) – Shape of max-pooling to be applied after the convolution. None (default) disables pooling along all axes.
conv_mode (str) –
Possible values: * “valid”: Only apply filter to complete patches of the image.

Generates output of shape: image_shape -filter_shape + 1.
- ”full”: Zero-pads image to multiple of filter shape to generate output of shape: image_shape + filter_shape - 1.
- ”same”: Zero-pads input image so that the output shape is equal to the input shape (Only supported for odd filter sizes).
activation_func (str) – Activation function name.
mfp (bool) – Whether to apply Max-Fragment-Pooling in this Layer.
batch_normalisation (str or False) – Batch normalisation mode. Can be False (inactive), “train” or “fadeout”.
dropout_rate (float) – Dropout rate (probability that a node drops out in a training step).
name (str) – Layer name.
print_repr (bool) – Whether to print the node representation upon initialisation.
w (np.ndarray or T.TensorVariable) – Weight matrix. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
b (np.ndarray or T.TensorVariable) – Bias vector. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
gamma – (For batch normalisation) Initialises gamma parameter.
mean – (For batch normalisation) Initialises mean parameter.
std – (For batch normalisation) Initialises std parameter.
gradnet_mode –
invalidate_fov (bool) – Overrides the computed fov with an invalid value to force later recalculation (experimental).

make_dual(parent, share_w=False, mfp=False, **kwargs)[source]¶

Create the inverse (UpConv) of this Conv node.

Most options are the same as for the layer itself. If kwargs are not specified, the values of the primal layers are re-used and new parameters are created.

Parameters:	parent (Node) – The input node. share_w (bool) – If the weights (`w`) should be shared from the primal layer. mfp (bool) – If max-fragment-pooling is used. kwargs (dict) – kwargs that are passed through to the new `UpConv` node (see signature of `UpConv`). `n_f` and `pool_shape` are copied from the existing node on which `make_dual` is called. Every other parameter can be changed from the original `Conv`’s defaults by specifying it in `kwargs`.
Returns:	The inverted conv layer (as an `UpConv` node).
Return type:	UpConv

class elektronn2.neuromancer.neural.UpConv(**kwargs)[source]¶

Bases: elektronn2.neuromancer.neural.Conv

Upconvolution layer. Also known as transposed convolution.

See http://deeplearning.net/software/theano/tutorial/conv_arithmetic.html#transposed-convolution-arithmetic

E.g. pooling + upconv with pool_shape = 3:

  x x x x x x x x x    before pooling (not in this layer)
   \|/   \|/   \|/     pooling (not in this layer)
    x     x     x      input to this layer
0 0 x 0 0 x 0 0 x 0 0  unpooling + padding (done in this layer)
   /|\   /|\   /|\     conv on unpooled (done in this layer)
  y y y y y y y y y    result of this layer

Parameters:

parent (Node) – The input node.
n_f (int) – Number of filters (nodes) in layer.
pool_shape (tuple) – Size of the UpConvolution.
activation_func (str) – Activation function name.
identity_init (bool) – Initialise weights to result in pixel repetition upsampling
batch_normalisation (str or False) – Batch normalisation mode. Can be False (inactive), “train” or “fadeout”.
dropout_rate (float) – Dropout rate (probability that a node drops out in a training step).
name (str) – Layer name.
print_repr (bool) – Whether to print the node representation upon initialisation.
w (np.ndarray or T.TensorVariable) – Weight matrix. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
b (np.ndarray or T.TensorVariable) – Bias vector. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
gamma – (For batch normalisation) Initialises gamma parameter.
mean – (For batch normalisation) Initialises mean parameter.
std – (For batch normalisation) Initialises std parameter.
gradnet_mode –

make_dual(*args, **kwargs)[source]¶

Create the inverse (UpConv) of this Conv node.

Most options are the same as for the layer itself. If kwargs are not specified, the values of the primal layers are re-used and new parameters are created.

Parameters:

parent (Node) – The input node.
share_w (bool) – If the weights (w) should be shared from the primal layer.
mfp (bool) – If max-fragment-pooling is used.
kwargs (dict) – kwargs that are passed through to the new UpConv node (see signature of UpConv). n_f and pool_shape are copied from the existing node on which make_dual is called. Every other parameter can be changed from the original Conv’s defaults by specifying it in kwargs.

Returns:

The inverted conv layer (as an UpConv node).: NOTE: docstring was inherited

Return type:

UpConv

class elektronn2.neuromancer.neural.Crop(**kwargs)[source]¶

Bases: elektronn2.neuromancer.node_basic.Node

This node type crops the output of its parent.

Parameters:	parent (Node) – The input node whose output should be cropped. crop (tuple or list of ints) – Crop each spatial axis from either side by this number. name (str) – Node name. print_repr (bool) – Whether to print the node representation upon initialisation.

class elektronn2.neuromancer.neural.LSTM(**kwargs)[source]¶

Bases: elektronn2.neuromancer.neural.NeuralLayer

Long short term memory layer.

Using an implementation without peepholes in f, i, o, i.e. weights cell state is not taken into account for weights. See http://colah.github.io/posts/2015-08-Understanding-LSTMs/.

Parameters:

parent (Node) – The input node.
memory_states (Node) – Concatenated (initial) feed-back and cell state (one Node!).
n_f (int) – Number of features.
activation_func (str) – Activation function name.
flatten –
batch_normalisation (str or None) – Batch normalisation mode. Can be False (inactive), “train” or “fadeout”.
dropout_rate (float) – Dropout rate (probability that a node drops out in a training step).
name (str) – Layer name.
print_repr (bool) – Whether to print the node representation upon initialisation.
w (np.ndarray or T.TensorVariable) – Weight matrix. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
b (np.ndarray or T.TensorVariable) – Bias vector. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
gamma – (For batch normalisation) Initialises gamma parameter.
mean – (For batch normalisation) Initialises mean parameter.
std – (For batch normalisation) Initialises std parameter.
gradnet_mode –

class elektronn2.neuromancer.neural.FragmentsToDense(**kwargs)[source]¶: Bases: elektronn2.neuromancer.node_basic.Node

class elektronn2.neuromancer.neural.Pool(**kwargs)[source]¶

Bases: elektronn2.neuromancer.node_basic.Node

Pooling layer.

Reduces the count of training parameters by reducing the spatial size of its input by the factors given in pool_shape.

Pooling modes other than max-pooling can only be selected if cuDNN is available.

Parameters:

parent (Node) – The input node.
pool_shape (tuple) – Tuple of pooling factors (per dimension) by which the input is downsampled.
stride (tuple) – Stride sizes (per dimension).
mfp (bool) – If max-fragment-pooling should be used.
mode (str) –
(only if cuDNN is available) Mode can be any of the modes supported by Theano’s dnn_pool(): (‘max’, ‘average_inc_pad’, ‘average_exc_pad’, ‘sum’).

’max’ (default): max-pooling ‘average’ or ‘average_inc_pad’: average-pooling ‘sum’: sum-pooling
name (str) – Name of the pooling layer.
print_repr (bool) – Whether to print the node representation upon initialisation.

elektronn2.neuromancer.neural.Dot¶: alias of elektronn2.neuromancer.neural.Perceptron

class elektronn2.neuromancer.neural.FaithlessMerge(**kwargs)[source]¶

Bases: elektronn2.neuromancer.node_basic.Node

FaithlessMerge node.

Parameters:	hard_features (Node) – easy_features (Node) – axis – failing_prob (float) – The higher the more often merge is unreliable hardeasy_ratio (float) – The higher the more often the harder features fail instead of the easy ones name (str) – Name of the pooling layer. print_repr: bool Whether to print the node representation upon initialisation.

class elektronn2.neuromancer.neural.GRU(**kwargs)[source]¶

Bases: elektronn2.neuromancer.neural.NeuralLayer

Gated Recurrent Unit Layer.

Parameters:

parent (Node) – The input node.
memory_state (Node) – Memory node.
n_f (int) – Number of features.
activation_func (str) – Activation function name.
flatten (bool) – (Unsupported).
batch_normalisation (str or None) – Batch normalisation mode. Can be False (inactive), “train” or “fadeout”.
dropout_rate (float) – Dropout rate (probability that a node drops out in a training step).
name (str) – Layer name.
print_repr (bool) – Whether to print the node representation upon initialisation.
w (np.ndarray or T.TensorVariable) – (Unsupported). Weight matrix. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
b (np.ndarray or T.TensorVariable) – (Unsupported). Bias vector. If this is a np.ndarray, its values are used to initialise a shared variable for this layer. If it is a T.TensorVariable, it is directly used (weight sharing with the layer which this variable comes from).
gamma – (For batch normalisation) Initialises gamma parameter.
mean – (For batch normalisation) Initialises mean parameter.
std – (For batch normalisation) Initialises std parameter.
gradnet_mode –

class elektronn2.neuromancer.neural.LRN(**kwargs)[source]¶

Bases: elektronn2.neuromancer.node_basic.Node

LRN (Local Response Normalization) layer.

Parameters:	parent (Node) – The input node. filter_shape (tuple) – mode (str) – Can be “spatial” or “channel”. alpha (float) – k (float) – beta (float) – name (str) – Node name. print_repr (bool) – Whether to print the node representation upon initialisation.

elektronn2.neuromancer.neural.AutoMerge(parent1, parent2, upconv_n_f=None, merge_mode='concat', disable_upconv=False, upconv_kwargs=None, name='merge', print_repr=True)[source]¶

Merges two network branches by automatic cropping and upconvolutions.

(Wrapper for UpConv, Crop, Concat and Add.)

Tries to automatically align and merge a high-res and a low-res (convolution) output of two branches of a CNN by applying UpConv and Crop to make their shapes and strides compatible. UpConv is used if the low-res parent’s strides are at least twice as large as the strides of the high-res parent in any dimension.

The parents are automatically identified as high-res and low-res by their strides. If both parents have the same strides, the concept of high-res and low-res is ignored and this function just crops the larger parent’s output until the parents’ spatial shapes match and then merges them.

This function can be used to simplify creation of e.g. architectures similar to U-Net (see https://arxiv.org/abs/1505.04597) or skip-connections.

If a ValueError that the shapes cannot be aligned is thrown, you can try changing the filter shapes and pooling factors of the (grand-)parent nodes or add/remove Convolutions and Crops in the preceding branches until the error disappears (of course you should try to keep those changes as minimal as possible).

(This function is an alias for UpConvMerge.)

Parameters:	parent1 (Node) – First parent to be merged. parent2 (Node) – Second parent to be merged. upconv_n_f (int) – Number of filters for the aligning `UpConv` for the low-res parent. merge_mode (str) – How the merging should be performed. Available options: ‘concat’ (default): Merge with a `Concat` node. ‘add’: Merge with an `Add` node. disable_upconv (bool) – If `True`, no automatic upconvolutions are performed to match strides. upconv_kwargs (dict) – Additional keyword arguments that are passed to the `UpConv` constructor if upconvolution is applied. name (str) – Name of the final merge node. print_repr (bool) – Whether to print the node representation upon initialisation.
Returns:	`Concat` or `Add` node (depending on `merge_mode`) that merges the aligned high-res and low-res outputs.
Return type:	Concat or Add

elektronn2.neuromancer.neural.UpConvMerge(parent1, parent2, upconv_n_f=None, merge_mode='concat', disable_upconv=False, upconv_kwargs=None, name='merge', print_repr=True)¶

Merges two network branches by automatic cropping and upconvolutions.

(Wrapper for UpConv, Crop, Concat and Add.)

Tries to automatically align and merge a high-res and a low-res (convolution) output of two branches of a CNN by applying UpConv and Crop to make their shapes and strides compatible. UpConv is used if the low-res parent’s strides are at least twice as large as the strides of the high-res parent in any dimension.

The parents are automatically identified as high-res and low-res by their strides. If both parents have the same strides, the concept of high-res and low-res is ignored and this function just crops the larger parent’s output until the parents’ spatial shapes match and then merges them.

This function can be used to simplify creation of e.g. architectures similar to U-Net (see https://arxiv.org/abs/1505.04597) or skip-connections.

If a ValueError that the shapes cannot be aligned is thrown, you can try changing the filter shapes and pooling factors of the (grand-)parent nodes or add/remove Convolutions and Crops in the preceding branches until the error disappears (of course you should try to keep those changes as minimal as possible).

(This function is an alias for UpConvMerge.)

Parameters:	parent1 (Node) – First parent to be merged. parent2 (Node) – Second parent to be merged. upconv_n_f (int) – Number of filters for the aligning `UpConv` for the low-res parent. merge_mode (str) – How the merging should be performed. Available options: ‘concat’ (default): Merge with a `Concat` node. ‘add’: Merge with an `Add` node. disable_upconv (bool) – If `True`, no automatic upconvolutions are performed to match strides. upconv_kwargs (dict) – Additional keyword arguments that are passed to the `UpConv` constructor if upconvolution is applied. name (str) – Name of the final merge node. print_repr (bool) – Whether to print the node representation upon initialisation.
Returns:	`Concat` or `Add` node (depending on `merge_mode`) that merges the aligned high-res and low-res outputs.
Return type:	Concat or Add

class elektronn2.neuromancer.neural.Pad(**kwargs)[source]¶

Bases: elektronn2.neuromancer.node_basic.Node

Pads the spatial axes of its parent’s output.

Parameters:	parent (Node) – The input node whose output should be padded. pad (tuple or list of ints) – The padding length from either side for each spatial axis value (float) – Value of the padding elements (default: 0.0) name (str) – Node name. print_repr (bool) – Whether to print the node representation upon initialisation.