PSD-Like Layers and Analytic Layer Inversion
Cotangent-lift PSDLayer and TransposePSDLayer layers change the dimension of a canonical symplectic vector space using a single block matrix $\Phi \in \mathrm{St}(n, N)$, while inverse_layer constructs the exact analytic inverse (or symplectic transpose) of GradientLayerQ, GradientLayerP, SymplecticAttentionQ, SymplecticAttentionP, and PSDLayer sharing the same parameter NamedTuple.
Library Functions
GeometricMachineLearning.TransposePSDLayer — Type
TransposePSDLayer(input_dim, output_dim)Make an instance of TransposePSDLayer.
This layer is the analytic transpose (and left inverse on the image of PSDLayer) of PSDLayer(output_dim, input_dim), sharing the same parameter tuple ps = (weight = Φ,) with $\Phi \in \mathrm{St}(n, N)$:
\[A^T = \begin{bmatrix} \Phi^T & \mathbb{O} \\ \mathbb{O} & \Phi^T \end{bmatrix}.\]
GeometricMachineLearning.inverse_layer — Function
inverse_layer(l)Return the analytic symplectic inverse (or Stiefel transpose for PSDLayer) of a layer l or layer type, sharing the same parameter NamedTuple ps without duplicating or mutating weights:
GradientLayerQ$\leftrightarrow$InverseGradientLayerQGradientLayerP$\leftrightarrow$InverseGradientLayerPSymplecticAttentionQ$\leftrightarrow$InverseSymplecticAttentionQSymplecticAttentionP$\leftrightarrow$InverseSymplecticAttentionPPSDLayer{2n, 2N}$\leftrightarrow$TransposePSDLayer{2N, 2n}
GeometricMachineLearning.InverseGradientLayer — Type
InverseGradientLayer{M, N, TA, C}(second_dim, activation)Analytic symplectic inverse of GradientLayer.
Given a GradientLayer that shifts one canonical coordinate by $\nabla V$ evaluated at the conjugate coordinate, InverseGradientLayer subtracts the exact same gradient update using the identical parameter NamedTuple ps.
GeometricMachineLearning.InverseGradientLayerQ — Type
InverseGradientLayerQ(n, upscaling_dimension, activation = identity)Make an instance of InverseGradientLayer that subtracts the gradient shift from the $q$ coordinate (analytic inverse of GradientLayerQ).
GeometricMachineLearning.InverseGradientLayerP — Type
InverseGradientLayerP(n, upscaling_dimension, activation = identity)Make an instance of InverseGradientLayer that subtracts the gradient shift from the $p$ coordinate (analytic inverse of GradientLayerP).
GeometricMachineLearning.InverseSymplecticAttention — Type
InverseSymplecticAttention{M, N, LayerType, Symmetric, AT}(activation)Analytic symplectic inverse of SymplecticAttention.
Subtracts the attention gradient update using the identical parameter NamedTuple ps = (A = ...,).
GeometricMachineLearning.InverseSymplecticAttentionQ — Type
InverseSymplecticAttentionQ(M; symmetric = sa_symmetric_default, activation = sa_activation_default)Make an instance of InverseSymplecticAttention that subtracts the attention gradient shift from the $q$ coordinate (analytic inverse of SymplecticAttentionQ).
GeometricMachineLearning.InverseSymplecticAttentionP — Type
InverseSymplecticAttentionP(M; symmetric = sa_symmetric_default, activation = sa_activation_default)Make an instance of InverseSymplecticAttention that subtracts the attention gradient shift from the $p$ coordinate (analytic inverse of SymplecticAttentionP).