Year: 2022
Author: Lingfeng Li, Xue-Cheng Tai, Jiang Yang
Communications in Computational Physics, Vol. 32 (2022), Iss. 4 : pp. 1007–1038
Abstract
In this work, we study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Adding the regularization term to the training loss is equivalent to using neural networks to solve some variational problems, mostly in high dimensions in practical applications. We introduce a general framework to analyze the error between neural network solutions and true solutions to variational problems. The error consists of three parts: the approximation errors of neural networks, the quadrature errors of numerical integration, and the optimization error. We also apply the proposed framework to two-layer networks to derive a priori error estimate when the true solution belongs to the so-called Barron space. Moreover, we conduct some numerical experiments to show that neural networks can solve corresponding variational problems sufficiently well. The networks with gradient-based regularization are much more robust in image applications.
You do not have full access to this article.
Already a Subscriber? Sign in as an individual or via your institution
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/cicp.OA-2021-0211
Communications in Computational Physics, Vol. 32 (2022), Iss. 4 : pp. 1007–1038
Published online: 2022-01
AMS Subject Headings: Global Science Press
Copyright: COPYRIGHT: © Global Science Press
Pages: 32
Keywords: Machine learning regularization generalization error image classification.