On the Adaptive Deterministic Block Coordinate Descent Method with Momentum for Solving Large Linear Least-Squares Problems

Abstract

Inspired by Polyak’s heavy-ball method, this paper proposes an adaptive deterministic block coordinate descent method with momentum (mADBCD) for efficiently solving large-scale linear least-squares problems. The proposed method introduces a novel column selection criterion based on the Euclidean norm of the residual vector of the normal equation. In contrast to classical block coordinate descent methods, mADBCD does not require a fixed pre-partitioning of the column indices of the coefficient matrix and avoids the expensive computation of Moore Penrose pseudoinverses of submatrices at each iteration. The method adaptively updates the block index set at each step, thereby improving both flexibility and scalability. When the coefficient matrix is of full column rank, we prove that mADBCD converges linearly to the unique solution of the least-squares problem. Numerical experiments are conducted to show that mADBCD outperforms several recent block coordinate descent methods in terms of iteration count and CPU time. In particular, when solving extremely sparse least-squares problems, mADBCD is the first block coordinate descent method reported to achieve CPU time nearly comparable to that of the classical least squares QR (LSQR) method [Paige and Saunders, ACM Trans. Math. Softw., 8 (1982)].