Symmetric Skew-Symmetric and Triangular Matrices.
Among the special arrays implemented in GeometricMachineLearning SymmetricMatrix, SkewSymMatrix, UpperTriangular and LowerTriangular are the most common ones and these can also be found in other libraries; LinearAlgebra.jl has an implementation of a symmetric matrix called Symmetric for example. The versions of these matrices in GeometricMachineLearning are however more memory efficient as they only store as many parameters as are necessary, i.e. $n(n+1)/2$ for the symmetric matrix and $n(n-1)/2$ for the other three. In addition operations such as matrix and tensor multiplication are implemented for these matrices to work in parallel on GPU.
We now show the various matrices. First UpperTriangular:
\[U = \begin{pmatrix} 0 & a_{12} & \cdots & a_{1n} \\ 0 & \ddots & & a_{2n} \\ \vdots & \ddots & \ddots & \vdots \\ 0 & \cdots & 0 & 0 \end{pmatrix}.\]
The matrix LowerTriangular:
\[L = \begin{pmatrix} 0 & 0 & \cdots & 0 \\ a_{21} & \ddots & & \vdots \\ \vdots & \ddots & \ddots & \vdots \\ a_{n1} & \cdots & a_{n(n-1)} & 0 \end{pmatrix}.\]
An instance of SkewSymMatrix can be written as $A = L - L^T$ or $A = U - U^T$:
\[A = \begin{pmatrix} 0 & - a_{21} & \cdots & - a_{n1} \\ a_{21} & \ddots & & \vdots \\ \vdots & \ddots & \ddots & \vdots \\ a_{n1} & \cdots & a_{n(n-1)} & 0 \end{pmatrix}.\]
And lastly a SymmetricMatrix:
\[L = \begin{pmatrix} a_{11} & a_{21} & \cdots & a_{n1} \\ a_{21} & \ddots & & \vdots \\ \vdots & \ddots & \ddots & \vdots \\ a_{n1} & \cdots & a_{n(n-1)} & a_{nn} \end{pmatrix}.\]
Note that any matrix $M\in\mathbb{R}^{n\times{}n}$ can be written
\[M = \frac{1}{2}(M - M^T) + \frac{1}{2}(M + M^T),\]
where the first part of this matrix is skew-symmetric and the second part is symmetric. This is also how the constructors for SkewSymMatrix and SymmetricMatrix are designed:
using GeometricMachineLearning
M = rand(3, 3)3×3 Matrix{Float64}:
0.0290565 0.741593 0.576903
0.942328 0.119505 0.534915
0.670688 0.775377 0.0592862A = SkewSymMatrix(M)3×3 SkewSymMatrix{Float64, Vector{Float64}}:
0.0 -0.100367 -0.0468928
0.100367 0.0 -0.120231
0.0468928 0.120231 0.0B = SymmetricMatrix(M)3×3 SymmetricMatrix{Float64, Vector{Float64}}:
0.0290565 0.841961 0.623796
0.841961 0.119505 0.655146
0.623796 0.655146 0.0592862M ≈ A + BtrueHow are Special Matrices Stored?
The following image demonstrates how special matrices are stored in GeometricMachineLearning:
So what is stored internally is a vector of size $n(n-1)/2$ for the skew-symmetric matrix and the triangular matrices and a vector of size $n(n+1)/2$ for the symmetric matrix. We can sample a random skew-symmetric matrix:
using GeometricMachineLearning
import Random
Random.seed!(123)
A = rand(SkewSymMatrix, 5)5×5 SkewSymMatrix{Float64, Vector{Float64}}:
0.0 -0.9063 -0.443494 -0.512083 -0.427328
0.9063 0.0 -0.745673 -0.253849 -0.867547
0.443494 0.745673 0.0 -0.334152 -0.581912
0.512083 0.253849 0.334152 0.0 -0.311448
0.427328 0.867547 0.581912 0.311448 0.0and then access the vector:
A.S10-element Vector{Float64}:
0.906299638797481
0.44349373245960455
0.7456733811393941
0.5120830400366143
0.2538490889415096
0.33415153638191886
0.4273278808735992
0.867547200255958
0.5819123423876457
0.3114475007050529This is equivalent to sampling a vector and then assigning a matrix:
using GeometricMachineLearning
import Random
Random.seed!(123)
S = rand(5 * (5 - 1) ÷ 2)
SkewSymMatrix(S, 5)5×5 SkewSymMatrix{Float64, Vector{Float64}}:
0.0 -0.9063 -0.443494 -0.512083 -0.427328
0.9063 0.0 -0.745673 -0.253849 -0.867547
0.443494 0.745673 0.0 -0.334152 -0.581912
0.512083 0.253849 0.334152 0.0 -0.311448
0.427328 0.867547 0.581912 0.311448 0.0These special matrices are important for SympNets, volume-preserving transformers and linear symplectic transformers.
Parallel Computation
The functions GeometricMachineLearning.mat_tensor_mul and GeometricMachineLearning.tensor_mat_mul are also implemented for these matrices for efficient parallel computations. This is elaborated on when we introduce pullbacks.
Library Functions
GeometricMachineLearning.UpperTriangular — TypeLowerTriangular(S::AbstractVector, n::Int)Build a lower-triangular matrix from a vector.
A lower-triangular matrix is an $n\times{}n$ matrix that has ones on the diagonal and zeros on the upper triangular.
The data are stored in a vector $S$ similarly to other matrices. See LowerTriangular, SkewSymMatrix and SymmetricMatrix.
The struct two fields: S and n. The first stores all the entries of the matrix in a sparse fashion (in a vector) and the second is the dimension $n$ for $A\in\mathbb{R}^{n\times{}n}$.
Examples
using GeometricMachineLearning
S = [1, 2, 3, 4, 5, 6]
UpperTriangular(S, 4)
# output
4×4 UpperTriangular{Int64, Vector{Int64}}:
0 1 2 4
0 0 3 5
0 0 0 6
0 0 0 0GeometricMachineLearning.UpperTriangular — MethodUpperTriangular(A::AbstractMatrix)Build a lower-triangular matrix from a matrix.
This is done by taking the lower left of that matrix.
Examples
using GeometricMachineLearning
M = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
UpperTriangular(M)
# output
4×4 UpperTriangular{Int64, Vector{Int64}}:
0 2 3 4
0 0 7 8
0 0 0 12
0 0 0 0GeometricMachineLearning.LowerTriangular — TypeLowerTriangular(S::AbstractVector, n::Int)Build a lower-triangular matrix from a vector.
A lower-triangular matrix is an $n\times{}n$ matrix that has ones on the diagonal and zeros on the upper triangular.
The data are stored in a vector $S$ similarly to other matrices. See UpperTriangular, SkewSymMatrix and SymmetricMatrix.
The struct two fields: S and n. The first stores all the entries of the matrix in a sparse fashion (in a vector) and the second is the dimension $n$ for $A\in\mathbb{R}^{n\times{}n}$.
Examples
using GeometricMachineLearning
S = [1, 2, 3, 4, 5, 6]
LowerTriangular(S, 4)
# output
4×4 LowerTriangular{Int64, Vector{Int64}}:
0 0 0 0
1 0 0 0
2 3 0 0
4 5 6 0GeometricMachineLearning.LowerTriangular — MethodLowerTriangular(A::AbstractMatrix)Build a lower-triangular matrix from a matrix.
This is done by taking the lower left of that matrix.
Examples
using GeometricMachineLearning
M = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
LowerTriangular(M)
# output
4×4 LowerTriangular{Int64, Vector{Int64}}:
0 0 0 0
5 0 0 0
9 10 0 0
13 14 15 0GeometricMachineLearning.SymmetricMatrix — TypeSymmetricMatrix(S::AbstractVector, n::Integer)Instantiate a symmetric matrix with information stored in vector S.
A SymmetricMatrix $A$ is a matrix $A^T = A$.
Internally the struct saves a vector $S$ of size $n(n+1)\div2$. The conversion is done the following way:
\[[A]_{ij} = \begin{cases} S[( (i-1) i ) \div 2 + j] & \text{if $i\geq{}j$}\\ S[( (j-1) j ) \div 2 + i] & \text{else}. \end{cases}\]
So $S$ stores a string of vectors taken from $A$: $S = [\tilde{a}_1, \tilde{a}_2, \ldots, \tilde{a}_n]$ with $\tilde{a}_i = [[A]_{i1},[A]_{i2},\ldots,[A]_{ii}]$.
Also see SkewSymMatrix, LowerTriangular and UpperTriangular.
Examples
using GeometricMachineLearning
S = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
SymmetricMatrix(S, 4)
# output
4×4 SymmetricMatrix{Int64, Vector{Int64}}:
1 2 4 7
2 3 5 8
4 5 6 9
7 8 9 10GeometricMachineLearning.SymmetricMatrix — MethodSymmetricMatrix(A::AbstractMatrix)Perform 0.5 * (A + A') and store the matrix in an efficient way (as a vector with $n(n+1)/2$ entries).
If the constructor is called with a matrix as input it returns a symmetric matrix via the projection:
\[A \mapsto \frac{1}{2}(A + A^T).\]
Examples
using GeometricMachineLearning
M = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
SymmetricMatrix(M)
# output
4×4 SymmetricMatrix{Float64, Vector{Float64}}:
1.0 3.5 6.0 8.5
3.5 6.0 8.5 11.0
6.0 8.5 11.0 13.5
8.5 11.0 13.5 16.0Extend help
Note that the constructor is designed in such a way that it always returns matrices of type SymmetricMatrix{<:AbstractFloat} when called with a matrix, even if this matrix is of type AbstractMatrix{<:Integer}.
If the user wishes to allocate a matrix SymmetricMatrix{<:Integer} the constructor SymmetricMatrix(::AbstractVector, n::Integer) has to be called.
GeometricMachineLearning.SkewSymMatrix — TypeSkewSymMatrix(S::AbstractVector, n::Integer)Instantiate a skew-symmetric matrix with information stored in vector S.
A skew-symmetric matrix $A$ is a matrix $A^T = -A$.
Internally the struct saves a vector $S$ of size $n(n-1)\div2$. The conversion is done the following way:
\[[A]_{ij} = \begin{cases} 0 & \text{if $i=j$} \\ S[( (i-2) (i-1) ) \div 2 + j] & \text{if $i>j$}\\ S[( (j-2) (j-1) ) \div 2 + i] & \text{else}. \end{cases}\]
Also see SymmetricMatrix, LowerTriangular and UpperTriangular.
Examples
using GeometricMachineLearning
S = [1, 2, 3, 4, 5, 6]
SkewSymMatrix(S, 4)
# output
4×4 SkewSymMatrix{Int64, Vector{Int64}}:
0 -1 -2 -4
1 0 -3 -5
2 3 0 -6
4 5 6 0GeometricMachineLearning.SkewSymMatrix — MethodSkewSymMatrix(A::AbstractMatrix)Perform 0.5 * (A - A') and store the matrix in an efficient way (as a vector with $n(n-1)/2$ entries).
If the constructor is called with a matrix as input it returns a skew-symmetric matrix via the projection:
\[A \mapsto \frac{1}{2}(A - A^T).\]
Examples
using GeometricMachineLearning
M = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
SkewSymMatrix(M)
# output
4×4 SkewSymMatrix{Float64, Vector{Float64}}:
0.0 -1.5 -3.0 -4.5
1.5 0.0 -1.5 -3.0
3.0 1.5 0.0 -1.5
4.5 3.0 1.5 0.0Extend help
Note that the constructor is designed in such a way that it always returns matrices of type SkewSymMatrix{<:AbstractFloat} when called with a matrix, even if this matrix is of type AbstractMatrix{<:Integer}.
If the user wishes to allocate a matrix SkewSymMatrix{<:Integer} the constructor SkewSymMatrix(::AbstractVector, n::Integer) has to be called.