Year: 2023
Author: Chenglong Bao, Qianxiao Li, Zuowei Shen, Cheng Tai, Lei Wu, Xueshuang Xiang
East Asian Journal on Applied Mathematics, Vol. 13 (2023), Iss. 3 : pp. 524–549
Abstract
In its simplest form, convolution neural networks (CNNs) consist of a fully connected two-layer network $g$ composed with a sequence of convolution layers $T.$ Although $g$ is known to have the universal approximation property, it is not known if CNNs, which have the form $g◦T$ inherit this property, especially when the kernel size in $T$ is small. In this paper, we show that under suitable conditions, CNNs do inherit the universal approximation property and its sample complexity can be characterized. In addition, we discuss concretely how the nonlinearity of $T$ can improve the approximation power. Finally, we show that when the target function class has a certain compositional form, convolutional networks are far more advantageous compared with fully connected networks, in terms of the number of parameters needed to achieve the desired accuracy.
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/eajam.2022-270.070123
East Asian Journal on Applied Mathematics, Vol. 13 (2023), Iss. 3 : pp. 524–549
Published online: 2023-01
AMS Subject Headings:
Copyright: COPYRIGHT: © Global Science Press
Pages: 26
Keywords: Convolutional networks approximation scaling analysis compositional functions.