Year: 2023
Author: Jonathan W. Siegel, Jinchao Xu
Journal of Computational Mathematics, Vol. 41 (2023), Iss. 3 : pp. 525–541
Abstract
We introduce a new algorithm, extended regularized dual averaging (XRDA), for solving regularized stochastic optimization problems, which generalizes the regularized dual averaging (RDA) method. The main novelty of the method is that it allows a flexible control of the backward step size. For instance, the backward step size used in RDA grows without bound, while for XRDA the backward step size can be kept bounded. We demonstrate experimentally that additional control over the backward step size can speed up the convergence of the algorithm while preserving desired properties of the iterates, such as sparsity. Theoretically, we show that the XRDA method achieves the same convergence rate as RDA for general convex objectives.
You do not have full access to this article.
Already a Subscriber? Sign in as an individual or via your institution
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/jcm.2210-m2021-0106
Journal of Computational Mathematics, Vol. 41 (2023), Iss. 3 : pp. 525–541
Published online: 2023-01
AMS Subject Headings:
Copyright: COPYRIGHT: © Global Science Press
Pages: 17
Keywords: Convex Optimization Subgradient Methods Structured Optimization Non-smooth Optimization.