Year: 2019
Author: Zorana Lužanin, Irena Stojkovska, Milena Kresoja
Journal of Computational Mathematics, Vol. 37 (2019), Iss. 1 : pp. 76–94
Abstract
A stochastic approximation (SA) algorithm with new adaptive step sizes for solving unconstrained minimization problems in noisy environment is proposed. New adaptive step size scheme uses ordered statistics of fixed number of previous noisy function values as a criterion for accepting good and rejecting bad steps. The scheme allows the algorithm to move in bigger steps and avoid steps proportional to $1/k$ when it is expected that larger steps will improve the performance. An algorithm with the new adaptive scheme is defined for a general descent direction. The almost sure convergence is established. The performance of new algorithm is tested on a set of standard test problems and compared with relevant algorithms. Numerical results support theoretical expectations and verify efficiency of the algorithm regardless of chosen search direction and noise level. Numerical results on problems arising in machine learning are also presented. Linear regression problem is considered using real data set. The results suggest that the proposed algorithm shows promise.
You do not have full access to this article.
Already a Subscriber? Sign in as an individual or via your institution
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/jcm.1710-m2017-0021
Journal of Computational Mathematics, Vol. 37 (2019), Iss. 1 : pp. 76–94
Published online: 2019-01
AMS Subject Headings:
Copyright: COPYRIGHT: © Global Science Press
Pages: 19
Keywords: Unconstrained optimization Stochastic optimization Stochastic approximation Noisy function Adaptive step size Descent direction Linear regression model.
Author Details
-
Optimal Homotopy Asymptotic Method-Least Square for Solving Nonlinear Fractional-Order Gradient-Based Dynamic System from an Optimization Problem
Okundalaye, Oluwaseun Olumide
Othman, Wan Ainun Mior
Kumaresan, Nallasamy
Advances in Mathematical Physics, Vol. 2020 (2020), Iss. P.1
https://doi.org/10.1155/2020/8049397 [Citations: 2]