Close Search Advanced
Journals
Resources
About Us
Open Access

The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks

The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks
Read Now
Download (PDF)

Year:    2022

Author:    Maarten V. de Hoop, Daniel Zhengyu Huang, Elizabeth Qian, Andrew M. Stuart

Journal of Machine Learning, Vol. 1 (2022), Iss. 3 : pp. 299–341

Abstract

The term ‘surrogate modeling’ in computational science and engineering refers to the development of computationally efficient approximations for expensive simulations, such as those arising from numerical solution of partial differential equations (PDEs). Surrogate modeling is an enabling methodology for many-query computations in science and engineering, which include iterative methods in optimization and sampling methods in uncertainty quantification. Over the last few years, several approaches to surrogate modeling for PDEs using neural networks have emerged, motivated by successes in using neural networks to approximate nonlinear maps in other areas. In principle, the relative merits of these different approaches can be evaluated by understanding, for each one, the cost required to achieve a given level of accuracy. However, the absence of a complete theory of approximation error for these approaches makes it difficult to assess this cost-accuracy trade-off. The purpose of the paper is to provide a careful numerical study of this issue, comparing a variety of different neural network architectures for operator approximation across a range of problems arising from PDE models in continuum mechanics.

Submit Article

Journal Article Details

Publisher Name:    Global Science Press

Language:    English

DOI:    https://doi.org/10.4208/jml.220509

Journal of Machine Learning, Vol. 1 (2022), Iss. 3 : pp. 299–341

Published online:    2022-01

AMS Subject Headings:   

Copyright:    COPYRIGHT: © Global Science Press

Pages:    43

Keywords:    Computational partial differential equations Surrogate modeling Operator approximation Neural networks Computational complexity.

Author Details

Maarten V. de Hoop

Daniel Zhengyu Huang

Elizabeth Qian

Andrew M. Stuart

  1. Numerical Analysis Meets Machine Learning

    Operator learning

    Kovachki, Nikola B. | Lanthaler, Samuel | Stuart, Andrew M.

    2024

    https://doi.org/10.1016/bs.hna.2024.05.009 [Citations: 3]

  2. Plasma surrogate modelling using Fourier neural operators

    Gopakumar, Vignesh | Pamela, Stanislas | Zanisi, Lorenzo | Li, Zongyi | Gray, Ander | Brennand, Daniel | Bhatia, Nitesh | Stathopoulos, Gregory | Kusner, Matt | Peter Deisenroth, Marc | Anandkumar, Anima

    Nuclear Fusion, Vol. 64 (2024), Iss. 5 P.056025

    https://doi.org/10.1088/1741-4326/ad313a [Citations: 5]

  3. Long-term predictions of turbulence by implicit U-Net enhanced Fourier neural operator

    Li, Zhijie | Peng, Wenhui | Yuan, Zelong | Wang, Jianchun

    Physics of Fluids, Vol. 35 (2023), Iss. 7

    https://doi.org/10.1063/5.0158830 [Citations: 18]

  4. Learning Homogenization for Elliptic Operators

    Bhattacharya, Kaushik | Kovachki, Nikola B. | Rajan, Aakila | Stuart, Andrew M. | Trautner, Margaret

    SIAM Journal on Numerical Analysis, Vol. 62 (2024), Iss. 4 P.1844

    https://doi.org/10.1137/23M1585015 [Citations: 1]

  5. Mitigating spectral bias for the multiscale operator learning

    Liu, Xinliang | Xu, Bo | Cao, Shuhao | Zhang, Lei

    Journal of Computational Physics, Vol. 506 (2024), Iss. P.112944

    https://doi.org/10.1016/j.jcp.2024.112944 [Citations: 3]

  6. Physics-Informed Neural Operator for Learning Partial Differential Equations

    Li, Zongyi | Zheng, Hongkai | Kovachki, Nikola | Jin, David | Chen, Haoxuan | Liu, Burigede | Azizzadenesheli, Kamyar | Anandkumar, Anima

    ACM / IMS Journal of Data Science, Vol. 1 (2024), Iss. 3 P.1

    https://doi.org/10.1145/3648506 [Citations: 25]

  7. Improved generalization with deep neural operators for engineering systems: Path towards digital twin

    Kobayashi, Kazuma | Daniell, James | Alam, Syed Bahauddin

    Engineering Applications of Artificial Intelligence, Vol. 131 (2024), Iss. P.107844

    https://doi.org/10.1016/j.engappai.2024.107844 [Citations: 10]

  8. Deep learning methods for blood flow reconstruction in a vessel with contrast enhanced x‐ray computed tomography

    Shusong, Huang | Monica, Sigovan | Bruno, Sixou

    International Journal for Numerical Methods in Biomedical Engineering, Vol. 40 (2024), Iss. 1

    https://doi.org/10.1002/cnm.3785 [Citations: 2]

  9. AI-assisted modeling of capillary-driven droplet dynamics

    Demou, Andreas D. | Savva, Nikos

    Data-Centric Engineering, Vol. 4 (2023), Iss.

    https://doi.org/10.1017/dce.2023.19 [Citations: 1]

  10. Solving multiphysics-based inverse problems with learned surrogates and constraints

    Yin, Ziyi | Orozco, Rafael | Louboutin, Mathias | Herrmann, Felix J.

    Advanced Modeling and Simulation in Engineering Sciences, Vol. 10 (2023), Iss. 1

    https://doi.org/10.1186/s40323-023-00252-0 [Citations: 4]

  11. Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks

    Adcock, Ben | Brugiapaglia, Simone | Dexter, Nick | Moraga, Sebastian

    Neural Networks, Vol. 181 (2025), Iss. P.106761

    https://doi.org/10.1016/j.neunet.2024.106761 [Citations: 0]

  12. Operator learning for homogenizing hyperelastic materials, without PDE data

    Zhang, Hao | Guilleminot, Johann

    Mechanics Research Communications, Vol. 138 (2024), Iss. P.104281

    https://doi.org/10.1016/j.mechrescom.2024.104281 [Citations: 0]

  13. Convergence Rates for Learning Linear Operators from Noisy Data

    de Hoop, Maarten V. | Kovachki, Nikola B. | Nelsen, Nicholas H. | Stuart, Andrew M.

    SIAM/ASA Journal on Uncertainty Quantification, Vol. 11 (2023), Iss. 2 P.480

    https://doi.org/10.1137/21M1442942 [Citations: 9]

  14. CAS4DL: Christoffel adaptive sampling for function approximation via deep learning

    Adcock, Ben | Cardenas, Juan M. | Dexter, Nick

    Sampling Theory, Signal Processing, and Data Analysis, Vol. 20 (2022), Iss. 2

    https://doi.org/10.1007/s43670-022-00040-8 [Citations: 1]

  15. A Born Fourier Neural Operator for Solving Poisson’s Equation With Limited Data and Arbitrary Domain Deformation

    Zong, Zheng | Wang, Yusong | He, Siyuan | Wei, Zhun

    IEEE Transactions on Antennas and Propagation, Vol. 72 (2024), Iss. 2 P.1827

    https://doi.org/10.1109/TAP.2023.3338770 [Citations: 1]
  16. Machine Learning in Modeling and Simulation

    Physics-Informed Deep Neural Operator Networks

    Goswami, Somdatta | Bora, Aniruddha | Yu, Yue | Karniadakis, George Em

    2023

    https://doi.org/10.1007/978-3-031-36644-4_6 [Citations: 24]

  17. Parameter identification by deep learning of a material model for granular media

    Tanyu, Derick Nganyu | Michel, Isabel | Rademacher, Andreas | Kuhnert, Jörg | Maass, Peter

    GEM - International Journal on Geomathematics, Vol. 15 (2024), Iss. 1

    https://doi.org/10.1007/s13137-024-00253-0 [Citations: 0]

  18. Solving Poisson’s Equation in Electromagnetics with Limited Data and Arbitrary Domain Deformation Using Physics-enhanced Neural Operator

    Zong, Zheng | Wei, Zhun

    2024 Photonics & Electromagnetics Research Symposium (PIERS), (2024), P.1

    https://doi.org/10.1109/PIERS62282.2024.10618026 [Citations: 0]