A team of researchers from Pennsylvania State University, USA, and King Abdullah University of Science and Technology, Saudi Arabia, have proposed a novel method for resolving nonlinear partial differential equations (PDEs) with multiple solutions. Their method, called the Newton Informed Neural Operator (NINO), utilises neural network techniques and is based on operator learning. This approach helps to identify a number of solutions in a single training process, overcoming the issues encountered by function learning methods in neural networks.
Current research on PDEs commonly focuses on those with a single solution; however PDEs offering multiple solutions present a significant challenge. A range of neural network methods have been designed to tackle PDEs (such as PINN, the Deep Ritz method and DeepONet), but they typically only learn one solution per training process. In contrast, NINO is able to learn multiple solutions more efficiently within a singular learning process, even when working with small data points.
The NINO technique also incorporates classical Newton methods, improving the network architecture and enabling a better structuring of the problems in operator learning. The researchers introduced two training methods alongside NINO. The first of these uses supervised data and applies the Mean Squared Error Loss (MSEL) as the main optimization requirement. The alternative method combines supervised and unsupervised learning, utilising a hybrid function loss. This is integrated with MSEL for a small amount of data with the ground truth and Newton’s loss for a large amount of data without the ground truth.
The team benchmarked the Newton solver method against the Neural operator method, evaluating the performance based on total execution time. This included aspects like the setup of matrices and vectors, GPU computation, and CUDA stream synchronization. The Newton solver method utilized 10 streams and CuPy with CUDA to maximize GPU parallel processing, while the Neural operator was naturally parallelized, making complete use of the GPU architecture without multiple streams.
In summary, NINO offers a potentially ground-breaking solution to the problem of non-linear PDEs with multiple solutions, overcoming the issues faced by the function learning techniques in neural networks. It offers efficient learning of the Newton operator, minimizes the amount of required supervised data and can resolve the problem in less time than traditional Newton methods.
