References

S. Lipschutz. General Topology (McGraw-Hill Book Company, New York City, New York, 1965). ↩¹ ↩²

S. Lang. Fundamentals of differential geometry. Vol. 191 (Springer Science & Business Media, 2012). ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

S. I. Richard L. Bishop. Tensor Analysis on Manifolds (Dover Publications, Mineola, New York, 1980). ↩¹ ↩² ↩³ ↩⁴

S. Lang. Real and functional analysis. Vol. 142 (Springer Science & Business Media, 2012). ↩¹ ↩²

M. P. Do Carmo and J. Flaherty Francis. Riemannian geometry. Vol. 2 (Springer, 1992). ↩¹ ↩²

P.-A. Absil, R. Mahony and R. Sepulchre. Riemannian geometry of Grassmann manifolds with a view on algorithmic computation. Acta Applicandae Mathematica 80, 199–220 (2004). ↩¹ ↩² ↩³ ↩⁴

D. D. Holm, T. Schmah and C. Stoica. Geometric mechanics and symmetry: from finite to infinite dimensions. Vol. 12 (Oxford University Press, Oxford, UK, 2009). ↩¹

P.-A. Absil, R. Mahony and R. Sepulchre. Optimization algorithms on matrix manifolds (Princeton University Press, Princeton, New Jersey, 2008). ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

T. Bendokat, R. Zimmermann and P.-A. Absil. A Grassmann manifold handbook: Basic geometry and computational aspects, arXiv preprint arXiv:2011.13699 (2020). ↩¹ ↩² ↩³ ↩⁴ ↩⁵

B. Brantner. Generalizing Adam To Manifolds For Efficiently Training Transformers, arXiv preprint arXiv:2305.16901 (2023). ↩¹ ↩² ↩³

E. Hairer, C. Lubich and G. Wanner. Geometric Numerical integration: structure-preserving algorithms for ordinary differential equations (Springer, Heidelberg, 2006). ↩¹ ↩² ↩³

T. Bendokat and R. Zimmermann. The real symplectic Stiefel and Grassmann manifolds: metrics, geodesics and applications, arXiv preprint arXiv:2108.12447 (2021). ↩¹ ↩² ↩³

B. Gao, N. T. Son, P.-A. Absil and T. Stykel. Riemannian optimization on the symplectic Stiefel manifold. SIAM Journal on Optimization 31, 1546–1575 (2021). ↩¹

B. O'neill. Semi-Riemannian geometry with applications to relativity (Academic press, New York City, New York, 1983). ↩¹ ↩²

E. Celledoni and A. Iserles. Approximating the exponential from a Lie algebra to a Lie group. Mathematics of Computation 69, 1457–1480 (2000). ↩¹

C. Fraikin, K. Hüper and P. V. Dooren. Optimization over the Stiefel manifold. In: PAMM: Proceedings in Applied Mathematics and Mechanics, Vol. 7 no. 1 (Wiley Online Library, 2007); pp. 1062205–1062206. ↩¹

A. Edelman, T. A. Arias and S. T. Smith. The geometry of algorithms with orthogonality constraints. SIAM Journal on Matrix Analysis and Applications 20, 303–353 (1998). ↩¹

C. Moler and C. Van Loan. Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review 45, 3–49 (2003). ↩¹

N. J. Higham. The scaling and squaring method for the matrix exponential revisited. SIAM Journal on Matrix Analysis and Applications 26, 1179–1193 (2005). ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

N. J. Higham. Functions of matrices: theory and computation (SIAM, Philadelphia, PA, 2008). ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

A. H. Al-Mohy and N. J. Higham. A new scaling and squaring algorithm for the matrix exponential. SIAM Journal on Matrix Analysis and Applications 31, 970–989 (2010). ↩¹ ↩² ↩³

R. B. Sidje. Expokit: A software package for computing matrix exponentials. ACM Transactions on Mathematical Software 24, 130–156 (1998). ↩¹

M. Schlarb. Covariant Derivatives on Homogeneous Spaces: Horizontal Lifts and Parallel Transport. The Journal of Geometric Analysis 34, 1–43 (2024). ↩¹

I. Goodfellow, Y. Bengio and A. Courville. Deep learning (MIT press, Cambridge, MA, 2016). ↩¹ ↩²

L. Kong, Y. Wang and M. Tao. Momentum stiefel optimizer, with applications to suitably-orthogonal attention, and optimal transport, arXiv preprint arXiv:2205.14173v3 (2023). ↩¹

J. Li, F. Li and S. Todorovic. Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform. In: International Conference on Learning Representations (2020), arXiv:2002.01113. ↩¹ ↩² ↩³ ↩⁴

J. Nocedal and S. J. Wright. Numerical optimization (Springer, New York, NY, 2006). Second Edition. ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸

I. Loshchilov and F. Hutter. Decoupled Weight Decay Regularization. In: International Conference on Learning Representations (2019). ↩¹ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

Y. Gu and Z. Xie. Mano: Restriking Manifold Optimization for LLM Training, arXiv preprint arXiv:2601.23000 (2026). ↩¹

F. Mezzadri. How to generate random matrices from the classical compact groups, arXiv preprint math-ph/0609050 (2006). ↩¹

M. J. Kochenderfer and T. A. Wheeler. Algorithms for optimization (Mit Press, 2019). ↩¹ ↩²

T. Frankel. The geometry of physics: an introduction (Cambridge university press, Cambridge, UK, 2011).

B. Gao, N. T. Son and T. Stykel. Optimization on the symplectic Stiefel manifold: SR decomposition-based retraction and applications. Linear Algebra and its Applications 682, 50–85 (2024).