Close Search Advanced
Journals
Resources
About Us
Open Access

Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't

Towards a Mathematical Understanding of Neural Network-Based Machine Learning: What We Know and What We Don't
Preview
Add to basket

Year:    2020

Author:    Weinan E, Chao Ma, Lei Wu, Stephan Wojtowytsch

CSIAM Transactions on Applied Mathematics, Vol. 1 (2020), Iss. 4 : pp. 561–615

Abstract

The purpose of this article is to review the achievements made in the last few years towards the understanding of the reasons behind the success and subtleties of neural network-based machine learning. In the tradition of good old applied mathematics, we will not only give attention to rigorous mathematical results, but also pay attention to the insight we have gained from careful numerical experiments as well as the analysis of simplified models. Along the way, we also list the open problems which we believe to be the most important topics for further study. This is not a complete overview over this quickly moving field, but we hope to provide a perspective which may be helpful especially to new researchers in the area.

Submit Article

You do not have full access to this article.

Already a Subscriber? Sign in as an individual or via your institution

Journal Article Details

Publisher Name:    Global Science Press

Language:    English

DOI:    https://doi.org/10.4208/csiam-am.SO-2020-0002

CSIAM Transactions on Applied Mathematics, Vol. 1 (2020), Iss. 4 : pp. 561–615

Published online:    2020-01

AMS Subject Headings:    Global Science Press

Copyright:    COPYRIGHT: © Global Science Press

Pages:    55

Keywords:    Neural networks machine learning supervised learning regression problems approximation optimization estimation a priori estimates Barron space multi-layer space flow-induced function space.

Author Details

Weinan E

Chao Ma

Lei Wu

Stephan Wojtowytsch

  1. Generalization error of random feature and kernel methods: Hypercontractivity and kernel matrix concentration

    Mei, Song | Misiakiewicz, Theodor | Montanari, Andrea

    Applied and Computational Harmonic Analysis, Vol. 59 (2022), Iss. P.3

    https://doi.org/10.1016/j.acha.2021.12.003 [Citations: 25]

  2. Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks

    Li, Yuqing | Luo, Tao | Ma, Chao

    SIAM Journal on Mathematics of Data Science, Vol. 4 (2022), Iss. 2 P.694

    https://doi.org/10.1137/21M140955X [Citations: 0]

  3. Crystal morphing: Structural interpolation including crystal invariances

    Oba, Junpei | Kajita, Seiji

    Physical Review Materials, Vol. 6 (2022), Iss. 2

    https://doi.org/10.1103/PhysRevMaterials.6.023801 [Citations: 5]

  4. A deep-learning assisted bioluminescence tomography method to enable radiation targeting in rat glioblastoma

    Rezaeifar, Behzad | Wolfs, Cecile J A | Lieuwes, Natasja G | Biemans, Rianne | Reniers, Brigitte | Dubois, Ludwig J | Verhaegen, Frank

    Physics in Medicine & Biology, Vol. 68 (2023), Iss. 15 P.155013

    https://doi.org/10.1088/1361-6560/ace308 [Citations: 3]

  5. Energetic Variational Neural Network Discretizations of Gradient Flows

    Hu, Ziqing | Liu, Chun | Wang, Yiwei | Xu, Zhiliang

    SIAM Journal on Scientific Computing, Vol. 46 (2024), Iss. 4 P.A2528

    https://doi.org/10.1137/22M1529427 [Citations: 0]

  6. A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems

    Lu, Jianfeng | Lu, Yulong

    Communications of the American Mathematical Society, Vol. 2 (2022), Iss. 1 P.1

    https://doi.org/10.1090/cams/5 [Citations: 9]

  7. Energetic Variational Neural Network Discretizations to Gradient Flows

    Hu, Ziqing | Liu, Chun | Wang, Yiwei | Xu, Zhiliang

    SSRN Electronic Journal, Vol. (2022), Iss.

    https://doi.org/10.2139/ssrn.4159429 [Citations: 0]

  8. Computing Offloading With Fairness Guarantee: A Deep Reinforcement Learning Method

    Hao, Hao | Xu, Changqiao | Zhang, Wei | Yang, Shujie | Muntean, Gabriel-Miro

    IEEE Transactions on Circuits and Systems for Video Technology, Vol. 33 (2023), Iss. 10 P.6117

    https://doi.org/10.1109/TCSVT.2023.3255229 [Citations: 8]

  9. Underload city conceptual approach extending ghost city studies

    Zhang, Xiuyuan | Du, Shihong | Taubenböck, Hannes | Wang, Yi-Chen | Du, Shouhang | Liu, Bo | Feng, Yuning

    npj Urban Sustainability, Vol. 2 (2022), Iss. 1

    https://doi.org/10.1038/s42949-022-00057-x [Citations: 4]

  10. Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation

    Jentzen, Arnulf | Welti, Timo

    Applied Mathematics and Computation, Vol. 455 (2023), Iss. P.127907

    https://doi.org/10.1016/j.amc.2023.127907 [Citations: 4]

  11. Artificial intelligence - enabled soft sensor and internet of things for sustainable agriculture using ensemble deep learning architecture

    Wongchai, Anupong | Shukla, Surendra Kumar | Ahmed, Mohammed Altaf | Sakthi, Ulaganathan | Jagdish, Mukta | kumar, Ravi

    Computers and Electrical Engineering, Vol. 102 (2022), Iss. P.108128

    https://doi.org/10.1016/j.compeleceng.2022.108128 [Citations: 68]

  12. Infinite‐width limit of deep linear neural networks

    Chizat, Lénaïc | Colombo, Maria | Fernández‐Real, Xavier | Figalli, Alessio

    Communications on Pure and Applied Mathematics, Vol. 77 (2024), Iss. 10 P.3958

    https://doi.org/10.1002/cpa.22200 [Citations: 0]

  13. CNN- Based Demodulation of Color Shift Keying in Screen Camera Communications

    Gordillo, Alex Cartagena

    2022 First International Conference on Computer Communications and Intelligent Systems (I3CIS), (2022), P.117

    https://doi.org/10.1109/I3CIS56626.2022.10076150 [Citations: 0]

  14. A Review on Thermal Imaging-Based Breast Cancer Detection Using Deep Learning

    Tsietso, Dennies | Yahya, Abid | Samikannu, Ravi | Khattak, Hasan Ali

    Mobile Information Systems, Vol. 2022 (2022), Iss. P.1

    https://doi.org/10.1155/2022/8952849 [Citations: 5]

  15. Control of Partial Differential Equations via Physics-Informed Neural Networks

    García-Cervera, Carlos J. | Kessler, Mathieu | Periago, Francisco

    Journal of Optimization Theory and Applications, Vol. 196 (2023), Iss. 2 P.391

    https://doi.org/10.1007/s10957-022-02100-4 [Citations: 4]

  16. Asymptotic-Preserving Neural Networks for multiscale hyperbolic models of epidemic spread

    Bertaglia, Giulia | Lu, Chuan | Pareschi, Lorenzo | Zhu, Xueyu

    Mathematical Models and Methods in Applied Sciences, Vol. 32 (2022), Iss. 10 P.1949

    https://doi.org/10.1142/S0218202522500452 [Citations: 8]

  17. SRMD: Sparse Random Mode Decomposition

    Richardson, Nicholas | Schaeffer, Hayden | Tran, Giang

    Communications on Applied Mathematics and Computation, Vol. 6 (2024), Iss. 2 P.879

    https://doi.org/10.1007/s42967-023-00273-x [Citations: 3]
  18. Advances in Numerical Methods for Hyperbolic Balance Laws and Related Problems

    Asymptotic-Preserving Neural Networks for Hyperbolic Systems with Diffusive Scaling

    Bertaglia, Giulia

    2023

    https://doi.org/10.1007/978-3-031-29875-2_2 [Citations: 2]

  19. Stationary Density Estimation of Itô Diffusions Using Deep Learning

    Gu, Yiqi | Harlim, John | Liang, Senwei | Yang, Haizhao

    SIAM Journal on Numerical Analysis, Vol. 61 (2023), Iss. 1 P.45

    https://doi.org/10.1137/21M1445363 [Citations: 2]

  20. Approximation results for Gradient Flow Trained Shallow Neural Networks in 1d

    Gentile, Russell | Welper, Gerrit

    Constructive Approximation, Vol. 60 (2024), Iss. 3 P.547

    https://doi.org/10.1007/s00365-024-09694-0 [Citations: 0]

  21. Generalization error of GAN from the discriminator’s perspective

    Yang, Hongkang | E, Weinan

    Research in the Mathematical Sciences, Vol. 9 (2022), Iss. 1

    https://doi.org/10.1007/s40687-021-00306-y [Citations: 5]