Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't (CSIAM Transactions on Applied Mathematics, 2020)
The purpose of this article is to review the achievements made in the lastfew years towards the understanding of the reasons behind the success and subtletiesof neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to theinsight we have gained from careful numerical experiments as well as the analysis ofsimplified models. Along the way, we also list the open problems which we believe tobe the most important topics for further study. This is not a complete overview overthis quickly moving field, but we hope to provide a perspective which may be helpfulespecially to new researchers in the area.
Keywords: Neural networks; machine learning; supervised learning; regression problems; approximation; optimization; estimation; a priori estimates; Barron space; multi-layer space; flow-induced function space.