Accelerated Optimization with Orthogonality Constraints

Accelerated Optimization with Orthogonality Constraints

Year:    2021

Author:    Jonathan W. Siegel

Journal of Computational Mathematics, Vol. 39 (2021), Iss. 2 : pp. 207–226

Abstract

We develop a generalization of Nesterov's accelerated gradient descent method which is designed to deal with orthogonality constraints. To demonstrate the effectiveness of our method, we perform numerical experiments which demonstrate that the number of iterations scales with the square root of the condition number, and also compare with existing state-of-the-art quasi-Newton methods on the Stiefel manifold. Our experiments show that our method outperforms existing state-of-the-art quasi-Newton methods on some large, ill-conditioned problems.

You do not have full access to this article.

Already a Subscriber? Sign in as an individual or via your institution

Journal Article Details

Publisher Name:    Global Science Press

Language:    English

DOI:    https://doi.org/10.4208/jcm.1911-m2018-0242

Journal of Computational Mathematics, Vol. 39 (2021), Iss. 2 : pp. 207–226

Published online:    2021-01

AMS Subject Headings:   

Copyright:    COPYRIGHT: © Global Science Press

Pages:    20

Keywords:    Riemannian optimization Stiefel manifold Accelerated gradient descent Eigenvector problems Electronic structure calculations.

Author Details

Jonathan W. Siegel

  1. Adaptive Localized Cayley Parametrization for Optimization Over Stiefel Manifold and Its Convergence Rate Analysis

    Kume, Keita | Yamada, Isao

    IEEE Access, Vol. 12 (2024), Iss. P.31312

    https://doi.org/10.1109/ACCESS.2024.3368631 [Citations: 0]
  2. Machine learning universal bosonic functionals

    Schmidt, Jonathan | Fadel, Matteo | Benavides-Riveros, Carlos L.

    Physical Review Research, Vol. 3 (2021), Iss. 3

    https://doi.org/10.1103/PhysRevResearch.3.L032063 [Citations: 15]
  3. Generalized left-localized Cayley parametrization for optimization with orthogonality constraints

    Kume, Keita | Yamada, Isao

    Optimization, Vol. 73 (2024), Iss. 4 P.1113

    https://doi.org/10.1080/02331934.2022.2142471 [Citations: 1]
  4. A Dynamic Smoothing Technique for a Class of Nonsmooth Optimization Problems on Manifolds

    Beck, Amir | Rosset, Israel

    SIAM Journal on Optimization, Vol. 33 (2023), Iss. 3 P.1473

    https://doi.org/10.1137/22M1489447 [Citations: 1]
  5. Distributed Pose-Graph Optimization With Multi-Level Partitioning for Multi-Robot SLAM

    Li, Cunhao | Guo, Guanghui | Yi, Peng | Hong, Yiguang

    IEEE Robotics and Automation Letters, Vol. 9 (2024), Iss. 6 P.4926

    https://doi.org/10.1109/LRA.2024.3382531 [Citations: 1]