GitHub

Linear Symplectic Transformer

The linear symplectic transformer consists of a combination of linear symplectic attention and gradient layers and is visualized below:

Library Functions

GeometricMachineLearning.LinearSymplecticTransformer — Type

Realizes the linear Symplectic Transformer.

Constructor:

The constructor is called with the following arguments

  1. dim::Int: System dimension
  2. seq_length::Int: Number of time steps that the transformer considers.

Optional keyword arguments:

  • n_sympnet::Int=2: The number of sympnet layers in the transformer.
  • upscaling_dimension::Int=2*dim: The upscaling that is done by the gradient layer.
  • L::Int=1: The number of transformer units.
  • activation=tanh: The activation function for the SympNet layers.
  • init_upper::Bool=true: Specifies if the first layer is a $Q$-type layer (init_upper=true) or if it is a $P$-type layer (init_upper=false).
source