e2cnn.kernels

This subpackage implements the complete analytical solutions of the equivariance constraints on the kernel space as explained in General E(2)-Equivariant Steerable CNNs. The bases of equivariant kernels should be built through factory functions.

Typically, the user does not need to interact directly with this subpackage. Instead, we suggest to use the interface provided in e2cnn.gspaces.

This subpackage depends only on e2cnn.group.

Abstract Classes

class Basis(dim, shape)[source]

Abstract class for implementing the basis of a space of maps between feature spaces (either kernels or PDOs).

See also

e2cnn.kernels.KernelBasis and e2cnn.diffops.DiffopBasis

Parameters

  • dim (int) – the dimensionality of the basis (number of elements)

  • shape (tuple) – a tuple containing \(c_\text{out}\) and \(c_\text{in}\)

Variables

  • ~.dim (int) – the dimensionality of the basis (number of elements)

  • ~.shape (tuple) – a tuple containing \(c_\text{out}\) and \(c_\text{in}\)

class KernelBasis(dim, shape)[source]

Abstract class for implementing the basis of a kernel space. A kernel space is the space of functions in the form:

\[\mathcal{K} := \{ \kappa: X \to \mathbb{R}^{c_\text{out} \times c_\text{in}} \}\]

where \(X\) is the base space on which the kernel is defined. For instance, for planar images \(X = \R^2\).

Parameters

  • dim (int) – the dimensionality of the basis \(|\mathcal{K}|\) (number of elements)

  • shape (tuple) – a tuple containing \(c_\text{out}\) and \(c_\text{in}\)

Variables

  • ~.dim (int) – the dimensionality of the basis \(|\mathcal{K}|\) (number of elements)

  • ~.shape (tuple) – a tuple containing \(c_\text{out}\) and \(c_\text{in}\)

abstract sample(points, out=None)[source]

Sample the continuous basis elements on discrete points in points. Optionally, store the resulting multidimentional array in out.

points must be an array of shape (D, N), where D is the dimensionality of the (parametrization of the) base space while N is the number of points.

Parameters

  • points (ndarray) – points where to evaluate the basis elements

  • out (ndarray, optional) – pre-existing array to use to store the output

Returns

the sampled basis

sample_masked(points, mask, out=None)[source]

Mask the basis sampled through sample() to drop some basis elements using the input `mask`.

`mask` must be a `np.uint8` array of shape `(len(self), )`.

Parameters

  • points (ndarray) – points where to evaluate the basis elements

  • mask (ndarray) – binary array to mask the basis

  • out (ndarray, optional) – pre-existing array to use to store the output

Returns

the sampled basis

class EmptyBasisException[source]

Exception raised when a Basis with no elements is built.

Kernel Bases

Tensor Product of radial and angular bases

class PolarBasis(radial, angular)[source]

Bases: e2cnn.kernels.basis.KernelBasis

Build the tensor product basis of a radial profile basis and an angular profile basis for kernels over the plane. Given two bases \(A = \{a_i\}_i\) and \(B = \{b_j\}_j\), this basis is defined as

\[C = A \otimes B = \left\{ c_{i,j}(x, y) := a_i(r) b_j(\phi) \right\}_{i,j}\]

where \((r, \phi)\) is the polar coordinates of the points \((x, y)\) on the plane.

Parameters
Variables
sample(points, out=None)[source]

Sample the continuous basis elements on a discrete set of points on the plane. Optionally, store the resulting multidimensional array in out.

points must be an array of shape (2, N) containing N points on the plane. Note that the points are specified in cartesian coordinates \((x, y)\).

Parameters

  • points (ndarray) – points on the plane where to evaluate the basis elements

  • out (ndarray, optional) – pre-existing array to use to store the output

Returns

the sampled basis

Radial Profile

class GaussianRadialProfile(radii, sigma)[source]

Bases: e2cnn.kernels.basis.KernelBasis

Basis for kernels defined over a radius in \(\R^+_0\).

Each basis element is defined as a Gaussian function. Different basis elements are centered at different radii (rings) and can possibly be associated with different widths (sigma).

More precisely, the following basis is implemented:

\[\mathcal{B} = \left\{ b_i (r) := \exp \left( \frac{ \left( r - r_i \right)^2}{2 \sigma_i^2} \right) \right\}_i\]

In order to build a complete basis of kernels, you should combine this basis with a basis which defines the angular profile through PolarBasis.

Parameters

  • radii (list) – centers of each basis element. They should be different and spread to cover all domain of interest

  • sigma (list or float) – widths of each element. Can potentially be different.

sample(radii, out=None)[source]

Sample the continuous basis elements on the discrete set of radii in radii. Optionally, store the resulting multidimentional array in out.

radii must be an array of shape (1, N), where N is the number of points.

Parameters

  • radii (ndarray) – radii where to evaluate the basis elements

  • out (ndarray, optional) – pre-existing array to use to store the output

Returns

the sampled basis

General Steerable Basis for equivariant kernels

class SteerableKernelBasis(irreps_basis, in_repr, out_repr, **kwargs)[source]

Bases: e2cnn.kernels.basis.KernelBasis

Implements a general basis for the vector space of equivariant kernels. A \(G\)-equivariant kernel \(\kappa\), mapping between an input field, transforming under \(\rho_\text{in}\) (in_repr), and an output field, transforming under \(\rho_\text{out}\) (out_repr), satisfies the following constraint:

\[\kappa(gx) = \rho_\text{out}(g) \kappa(x) \rho_\text{in}(g)^{-1} \qquad \forall g \in G, \forall x \in X\]

As the kernel constraint is a linear constraint, the space of equivariant kernels is a vector subspace of the space of all convolutional kernels. It follows that any equivariant kernel can be expressed in terms of a basis of this space.

This class solves the kernel constraint for two arbitrary representations by combining the solutions of the kernel constraints associated to their IrreducibleRepresentation s. In order to do so, it relies on irreps_basis which solves individual irreps constraints. irreps_basis must be a class (subclass of IrrepsBasis) which builds a basis for equivariant kernels associated with irreducible representations when instantiated.

The groups \(G\) which are currently implemented are origin-preserving isometries (what are called structure groups, or sometimes gauge groups, in the language of Gauge Equivariant CNNs ). The origin-preserving isometries of \(\R^d\) are subgroups of \(O(d)\), i.e. reflections and rotations. Therefore, equivariance does not enforce any constraint on the radial component of the kernels. Hence, this class only implements a basis for the angular part of the kernels.

In order to build a complete basis of kernels, you should combine this basis with a basis which defines the radial profile (such as GaussianRadialProfile) through PolarBasis.

\[\mathcal{B} = \left\{ b_i (r) := \exp \left( \frac{ \left( r - r_i \right)^2}{2 \sigma_i^2} \right) \right\}_i\]

Warning

Typically, the user does not need to manually instantiate this class. Instead, we suggest to use the interface provided in e2cnn.gspaces.

Parameters

  • irreps_basis (class) – class defining the irreps basis. This class is instantiated for each pair of irreps to solve all irreps constraints.

  • in_repr (Representation) – Representation associated with the input feature field

  • out_repr (Representation) – Representation associated with the output feature field

  • **kwargs – additional arguments used when instantiating irreps_basis

sample(angles, out=None)[source]

Sample the continuous basis elements on the discrete set of angles in angles. Optionally, store the resulting multidimentional array in out.

A value of nan is interpreted as the angle of a point placed on the origin of the axes.

angles must be an array of shape (1, N), where N is the number of points.

Parameters

  • angles (ndarray) – angles where to evaluate the basis elements

  • out (ndarray, optional) – pre-existing array to use to store the output

Returns

the sampled basis

Bases for Group Actions on the Plane

The following factory functions provide an interface to build the bases for kernels equivariant to groups acting on the two dimensional plane. The names of the functions follow this convention kernels_[G]_act_R[d], where \(G\) is the origin-preserving isometry group while \(\R^d\) is the space on which it acts, interpreted as the domain of the kernel \(\kappa: \R^d \to \R^{c_\text{out} \times c_\text{in}}\). Here, \(d=2\) as we only consider the planar setting. In the language of Gauge Equivariant CNNs , the origin-preserving isometry \(G\) is called structure group (or, sometimes, gauge group).

Reflections and Continuous Rotations

kernels_O2_act_R2(in_repr, out_repr, radii, sigma, axis=1.5707963267948966)[source]

Builds a basis for convolutional kernels equivariant to reflections and continuous rotations, modeled by the group \(O(2)\). in_repr and out_repr need to be Representation s of O2.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

Because \(O(2)\) contains all rotations, the reflection element of the group can be associated to any reflection axis. Reflections along other axes can be obtained by composition with rotations. However, a choice of this axis is required to align the basis with respect to the action of the group.

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

  • axis (float, optional) – angle of the axis of the reflection element

Continuous Rotations

kernels_SO2_act_R2(in_repr, out_repr, radii, sigma)[source]

Builds a basis for convolutional kernels equivariant to continuous rotations, modeled by the group \(SO(2)\). in_repr and out_repr need to be Representation s of SO2.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

Reflections and Discrete Rotations

kernels_DN_act_R2(in_repr, out_repr, radii, sigma, axis=1.5707963267948966, max_frequency=None, max_offset=None)[source]

Builds a basis for convolutional kernels equivariant to reflections and \(N\) discrete rotations, modeled by the group \(D_N\). in_repr and out_repr need to be Representation s of DihedralGroup.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

The parameter axis is the angle in radians (with respect to the horizontal axis, rotating counter-clockwise) which defines the reflection axis for the reflection element of the group.

Frequencies also appear in a basis with a period of \(N\), i.e. if the basis contains an element with frequency \(k\), then it also contains an element with frequency \(k + N\). In the analytical solutions shown in Table 12 here, each solution has a parameter \(t\) or \(\hat{t}\). max_offset defines the maximum absolute value of these two numbers.

Either max_frequency or max_offset must be specified.

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

  • max_frequency (int) – maximum frequency of the basis

  • max_offset (int) – maximum offset in the frequencies of the basis

  • axis (float) – angle defining the reflection axis

Discrete Rotations

kernels_CN_act_R2(in_repr, out_repr, radii, sigma, max_frequency=None, max_offset=None)[source]

Builds a basis for convolutional kernels equivariant to \(N\) discrete rotations, modeled by the group \(C_N\). in_repr and out_repr need to be Representation s of CyclicGroup.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

The analytical angular solutions of kernel constraints belong to an infinite dimensional space and so can be expressed in terms of infinitely many basis elements, each associated with one unique frequency. Because the kernels are then sampled on a finite number of points (e.g. the cells of a grid), only low-frequency solutions needs to be considered. This enables us to build a finite dimensional basis containing only a finite subset of all analytical solutions. max_frequency is an integer controlling the highest frequency sampled in the basis.

Frequencies also appear in a basis with a period of \(N\), i.e. if the basis contains an element with frequency \(k\), then it also contains an element with frequency \(k + N\). In the analytical solutions shown in Table 11 here, each solution has a parameter \(t\) or \(\hat{t}\). max_offset defines the maximum absolute value of these two numbers.

Either max_frequency or max_offset must be specified.

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

  • max_frequency (int) – maximum frequency of the basis

  • max_offset (int) – maximum offset in the frequencies of the basis

Reflections

kernels_Flip_act_R2(in_repr, out_repr, radii, sigma, axis=1.5707963267948966, max_frequency=None, max_offset=None)[source]

Builds a basis for convolutional kernels equivariant to reflections. in_repr and out_repr need to be Representation s of CyclicGroup with N=2.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

The parameter axis is the angle in radians (with respect to the horizontal axis, rotating counter-clockwise) which defines the reflection axis.

The analytical angular solutions of kernel constraints belong to an infinite dimensional space and so can be expressed in terms of infinitely many basis elements. Only a finite subset can however be implemented. max_frequency and max_offset defines two ways to do so and therefore it is necessary to specify one of them. See kernels_CN_act_R2() for more details.

Todo

remove max_offset as it is equivalent to max_frequency here

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

  • axis (float) – angle defining the reflection axis

  • max_frequency (int) – maximum frequency of the basis

  • max_offset (int) – maximum offset in the frequencies of the basis

Trivial Action

kernels_Trivial_act_R2(in_repr, out_repr, radii, sigma, max_frequency=None, max_offset=None)[source]

Builds a basis for unconstrained convolutional kernels.

This is equivalent to use kernels_CN_act_R2() with an instance of CyclicGroup with N=1 (the trivial group \(C_1\)).

in_repr and out_repr need to be associated with an instance of CyclicGroup with N=1.

Because the equivariance constraints allow any choice of radial profile, we use a GaussianRadialProfile. radii specifies the radial distances at which the rings are centered while sigma contains the width of each of these rings (see GaussianRadialProfile).

The analytical angular solutions of kernel constraints belong to an infinite dimensional space and so can be expressed in terms of infinitely many basis elements. Only a finite subset can however be implemented. max_frequency and max_offset defines two ways to do so and therefore it is necessary to specify one of them. See kernels_CN_act_R2() for more details.

Todo

remove max_offset as it is equivalent to max_frequency here

Parameters

  • in_repr (Representation) – the representation specifying the transformation of the input feature field

  • out_repr (Representation) – the representation specifying the transformation of the output feature field

  • radii (list) – radii of the rings defining the basis for the radial profile

  • sigma (list or float) – widths of the rings defining the basis for the radial profile

  • axis (float) – angle defining the reflection axis

  • max_frequency (int) – maximum frequency of the basis

  • max_offset (int) – maximum offset in the frequencies of the basis