Loss-driven dynamic weight and residual transformation in physics-informed neural network.

Abstract

Physics-Informed Neural Networks (PINNs) have attracted substantial interest as a powerful approach for addressing both forward and inverse problems associated with partial differential equations (PDEs). In this study, we reveal a persistent phenomenon of imbalance within the empirical risk loss function. Building upon this observation, we introduce a loss-driven dynamic weight PINN, along with a theoretical analysis grounded in the neural tangent kernel. To assess the efficacy of our proposed method, we conduct comprehensive evaluations across various complex, time-dependent physical phenomena, employing different learning rate decay strategies. A series of experiments are carried out to scrutinize the influence of dynamic weight update frequencies and allocation principles on both the accuracy and computational efficiency. Furthermore, to shed light on the underlying causes of PINN failures in solving inverse problem, we offer a theoretical explanation that involves multi-parameter discovery via gradient descent for high-dimensional nonlinear PDEs. Leveraging the concept of limiting equivalence, we propose an innovative algorithm, termed Residual Transformation PINN (ResTranPINN), which not only accelerates convergence but also diminishes computational time. These studies provide new insights and a comprehensive framework for understanding PINN optimization and diagnosing possible failure modes.