Multi-particle neural operator transformer for solving partial differential equations.

Abstract

In recent years, propelled by significant advancements in computing hardware, deep neural networks have demonstrated remarkable potential in addressing mathematical problems, notably solving partial differential equations. Among them, the Transformer architecture has gained significant traction in operator learning. However, most existing approaches exploit its approximation capabilities without delving into the design or refinement of its internal mechanisms. To bridge this gap, we propose the Multi-particle Neural Operator Transformer (MPNOT), which establishes a theoretical connection between multi-particle dynamical systems and attention mechanisms. In contrast to classical operators like Fourier Neural Operator (FNO) and Deep Operator Network (DeepONet), as well as more recent Transformer-based operator architectures, MPNOT achieves enhanced modeling of local spatial variations through the introduction of the novel multi-particle attention layer. Furthermore, MPNOT incorporates the theory of multi-particle reaction-diffusion dynamical systems, enhancing the interpretability and generalization of the Transformer-based operator architecture. In this study, we demonstrate the effectiveness of the proposed MPNOT across several benchmark problems, including Burgers' equation, Reaction-Diffusion equation, Navier-Stokes equation, Allen-Cahn equation, and others. Experimental results demonstrate that MPNOT is a promising and effective approach for learning operators that map between infinite-dimensional function spaces.