Less Emphasis on Hard Regions: Curriculum Learning of PINNs for Singularly Perturbed Convection-Diffusion-Reaction Problems
Year: 2024
Author: Yufeng Wang, Cong Xu, Min Yang, Jin Zhang
East Asian Journal on Applied Mathematics, Vol. 14 (2024), Iss. 1 : pp. 104–123
Abstract
Although physics-informed neural networks (PINNs) have been successfully applied in a wide variety of science and engineering fields, they can fail to accurately predict the underlying solution in slightly challenging convection-diffusion-reaction problems. In this paper, we investigate the reason of this failure from a domain distribution perspective, and identify that learning multi-scale fields simultaneously makes the network unable to advance its training and easily get stuck in poor local minima. We show that the widespread experience of sampling more collocation points in high-loss regions hardly help optimize and may even worsen the results. These findings motivate the development of a novel curriculum learning method that encourages neural networks to prioritize learning on easier non-layer regions while downplaying learning on harder regions. The proposed method helps PINNs automatically adjust the learning emphasis and thereby facilitates the optimization procedure. Numerical results on typical benchmark equations show that the proposed curriculum learning approach mitigates the failure modes of PINNs and can produce accurate results for very sharp boundary and interior layers. Our work reveals that for equations whose solutions have large scale differences, paying less attention to high-loss regions can be an effective strategy for learning them accurately.
You do not have full access to this article.
Already a Subscriber? Sign in as an individual or via your institution
Journal Article Details
Publisher Name: Global Science Press
Language: English
DOI: https://doi.org/10.4208/eajam.2023-062.170523
East Asian Journal on Applied Mathematics, Vol. 14 (2024), Iss. 1 : pp. 104–123
Published online: 2024-01
AMS Subject Headings:
Copyright: COPYRIGHT: © Global Science Press
Pages: 20
Keywords: Physics-informed neural network convection-diffusion-reaction boundary layer interior layer curriculum learning.
Author Details
-
On Loss Functionals for Physics-Informed Neural Networks for Steady-State Convection-Dominated Convection-Diffusion Problems
Frerichs-Mihov, Derk | Henning, Linus | John, VolkerCommunications on Applied Mathematics and Computation, Vol. (2024), Iss.
https://doi.org/10.1007/s42967-024-00433-7 [Citations: 0] -
Physics-Informed neural network for level set method in vapor condensation
Tang, Minghai | Xin, Zhiqiang | Wang, LeiInternational Journal of Heat and Fluid Flow, Vol. 110 (2024), Iss. P.109651
https://doi.org/10.1016/j.ijheatfluidflow.2024.109651 [Citations: 0]