Gradient Descent Ascent for Minimax Problems on Riemannian Manifolds.
basic_science · Level V
Where this comes from
- Record sourced from PubMed, PMID 37018266.
- Also identified by DOI 10.1109/TPAMI.2023.3234160.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
In the paper, we study a class of useful minimax problems on Riemanian manifolds and propose a class of effective Riemanian gradient-based methods to solve these minimax problems. Specifically, we propose an effective Riemannian gradient descent ascent (RGDA) algorithm for the deterministic minimax optimization. Moreover, we prove that our RGDA has a sample complexity of O(κ<sup>2</sup>ϵ<sup>-2</sup>) for finding an ϵ-stationary solution of the Geodesically-Nonconvex Strongly-Concave (GNSC) minimax problems, where κ denotes the condition number. At the same time, we present an effective Riemannian stochastic gradient descent ascent (RSGDA) algorithm for the stochastic minimax optimization, which has a sample complexity of O(κ<sup>4</sup>ϵ<sup>-4</sup>) for finding an ϵ-stationary solution. To further reduce the sample complexity, we propose an accelerated Riemannian stochastic gradient descent ascent (Acc-RSGDA) algorithm based on the momentum-based variance-reduced technique. We prove that our Acc-RSGDA algorithm achieves a lower sample complexity of ~O(κ<sup>4</sup>ϵ<sup>-3</sup>) in searching for an ϵ-stationary solution of the GNSC minimax problems. Extensive experimental results on the robust distributional optimization and robust Deep Neural Networks (DNNs) training over Stiefel manifold demonstrate efficiency of our algorithms.