GitHub

Tensors in GeometricMachineLearning

We typically store training data as tensors with three axes in GeometricMachineLearning. This allows for a parallel computation of matrix products, also for the special arrays such as StrictlyLowerTriangular, StrictlyUpperTriangular, SymmetricMatrix and SkewSymMatrix and objects of Manifold type such as the StiefelManifold.

Library Functions

GeometricMachineLearning._mul — Function
_mul(W::AbstractMatrix, x::AbstractArray)
_mul(x::AbstractArray{<:Number, 3}, W::AbstractMatrix)
_mul(x::AbstractArray{<:Number, 3}, y::AbstractArray{<:Number, 3})

Multiply a matrix and a tensor, or two tensors, slice by slice along the third axis.

  • The left product multiplies W onto x from the left. For a vector or a matrix x it is W * x. For a tensor it is one GEMM: x is reshaped into a matrix with size(x, 1) rows, so that _mul(W, x)[:, :, k] == W * x[:, :, k].
  • The right product multiplies W onto every slice of the tensor x from the right, _mul(x, W)[:, :, k] == x[:, :, k] * W. It is one GEMM between two permutedims.
  • The tensor product multiplies the slices of two tensors pairwise, _mul(x, y)[:, :, k] == x[:, :, k] * y[:, :, k], with the tensor_tensor_mul kernel.

A SymmetricMatrix, SkewSymMatrix, StrictlyLowerTriangular or StrictlyUpperTriangular weight multiplies a vector or a matrix through its own *. Before a GEMM with a tensor, and as the right argument of a left product, it is made dense once per call, on its own backend, and its gradient reaches the storage through ProjectTo. Element types are promoted as * promotes them.

Examples

A left product:

using GeometricMachineLearning: _mul

B = [1 1 1; 1 1 1; 1 1 1;;; 2 2 2; 2 2 2; 2 2 2]
A = [3 0 0; 0 2 0; 0 0 1]

_mul(A, B)

# output

3×3×2 Array{Int64, 3}:
[:, :, 1] =
 3  3  3
 2  2  2
 1  1  1

[:, :, 2] =
 6  6  6
 4  4  4
 2  2  2

A right product:

using GeometricMachineLearning: _mul

A = [1 1 1; 1 1 1; 1 1 1;;; 2 2 2; 2 2 2; 2 2 2]
B = [3 0 0; 0 2 0; 0 0 1]

_mul(A, B)

# output

3×3×2 Array{Int64, 3}:
[:, :, 1] =
 3  2  1
 3  2  1
 3  2  1

[:, :, 2] =
 6  4  2
 6  4  2
 6  4  2
source
GeometricMachineLearning.assign_q_and_p — Function
assign_q_and_p(x::AbstractArray, N::Integer)

Split x along its first axis, and return the (q, p) NamedTuple of the rows at the indices 1:N and at the indices N + 1:2N, as indexing x returns them: copies for an Array, and ranges for a range. The indices are these whatever the first axis of x is, so an offset axis is split at index 1, not at its first row. Rows at other indices are ignored, and the other axes are kept. For N ≤ 0 both halves are empty.

Throws a BoundsError when the first axis of x does not hold the indices 1:2N, and an ArgumentError when x has no axis.

Slicing works on every backend, and Zygote differentiates it, so the split needs neither a kernel nor an rrule.

source