Reduced Basis Approximations of Parameterized Dynamic PDEs via Neural Networks
Explore the source record for details and available documents.
Cyr, Eric Christopher↗
SEARCH · Search NASA
Results for “Parameterized PDEs”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
ECLEIRS: Exact conservation law embedded identification of reduced states for parameterized nonlinear conservation laws from sparse and noisy data
Multi-query applications such as parameter estimation, uncertainty quantification and design optimization for parameterized partial differential equation (PDE) systems are expensive. While reduced/latent state dynamics approaches for parameterized PDEs offer a viable alternative, these approaches rely on high-quality data and struggle with highly sparse spatiotemporal noisy measurements typically obtained from experiments. Furthermore, there is no guarantee that these models satisfy governing physical conservation laws. In this article, we propose a reduced state dynamics approach, referred to as ECLEIRS, that embeds exact conservation in the solution and flux representation by utilizing a space-time divergence-free neural network formulation. We compare ECLEIRS with other reduced state dynamics approaches, those that do not enforce any physical constraints and those with physics-informed loss functions, for three shock-propagation problems: 1-D advection, 1-D Burgers and 2-D Euler equations. In conclusion, the numerical experiments conducted in this study demonstrate that ECLEIRS provides the most accurate prediction of dynamics for unseen parameters even in the presence of highly sparse and noisy data.
97 MATHEMATICS AND COMPUTING↗
Learning high-dimensional parametric maps via reduced basis adaptive residual networks
We propose a scalable framework for the learning of high-dimensional parametric maps via adaptively constructed residual network (ResNet) maps between reduced bases of the inputs and outputs. When just few training data are available, it is beneficial to have a compact parametrization in order to ameliorate the ill-posedness of the neural network training problem. By linearly restricting high-dimensional maps to informed reduced bases of the inputs, one can compress high-dimensional maps in a constructive way that can be used to detect appropriate basis ranks, equipped with rigorous error estimates. A scalable neural network learning framework is thus to learn the nonlinear compressed reduced basis mapping. Unlike the reduced basis construction, however, neural network constructions are not guaranteed to reduce errors by adding representation power, making it difficult to achieve good practical performance. Inspired by recent approximation theory that connects ResNets to sequential minimizing flows, we present an adaptive ResNet construction algorithm. This algorithm allows for depth-wise enrichment of the neural network approximation, in a manner that can achieve good practical performance by first training a shallow network and then adapting. We prove universal approximation of the associated neural network class for $L^2_v$ functions on compact sets. Our overall framework allows for constructive means to detect appropriate breadth and depth, and related compact parametrizations of neural networks, significantly reducing the need for architectural hyperparameter tuning. Numerical experiments for parametric PDE problems and a 3D CFD wing design optimization parametric map demonstrate that the proposed methodology can achieve remarkably high accuracy for limited training data, and outperformed other neural network strategies we compared against.
42 ENGINEERING↗
Learning Only on Boundaries: A Physics-Informed Neural Operator for Solving Parametric Partial Differential Equations in Complex Geometries
Recently, deep learning surrogates and neural operators have shown promise in solving partial differential equations (PDEs). However, they often require a large amount of training data and are limited to bounded domains. In this work, we present a novel physics-informed neural operator method to solve parameterized boundary value problems without labeled data. By reformulating the PDEs into boundary integral equations (BIEs), we can train the operator network solely on the boundary of the domain. This approach reduces the number of required sample points from $O(N^d)$ to $O(N^{d-1}$), where $d$ is the domain’s dimension, leading to a significant acceleration of the training process. Additionally, our method can handle unbounded problems, which are unattainable for existing physics-informed neural networks (PINNs) and neural operators. Finally, our numerical experiments show the effectiveness of parameterized complex geometries and unbounded problems.
97 MATHEMATICS AND COMPUTING↗
Neural Network-Based Variational Methods for Solving Quadratic Porous Medium Equations in High Dimensions
Here, in this paper, we propose and study neural network-based methods for solutions of high-dimensional quadratic porous medium equation (QPME). Three variational formulations of this nonlinear PDE are presented: a strong formulation and two weak formulations. For the strong formulation, the solution is directly parameterized with a neural network and optimized by minimizing the PDE residual. It can be proved that the convergence of the optimization problem guarantees the convergence of the approximate solution in the $L^1$ sense. The weak formulations are derived following (Brenier in Examples of hidden convexity in nonlinear PDEs, 2020) which characterizes the very weak solutions of QPME. Specifically speaking, the solutions are represented with intermediate functions who are parameterized with neural networks and are trained to optimize the weak formulations. Extensive numerical tests are further carried out to investigate the pros and cons of each formulation in low and high dimensions. This is an initial exploration made along the line of solving high-dimensional nonlinear PDEs with neural network-based methods, which we hope can provide some useful experience for future investigations.
97 MATHEMATICS AND COMPUTING↗
Higher-order LaSDI: Reduced order modeling with multiple time derivatives
Solving complex partial differential equations (PDEs) is essential across scientific disciplines but often requires numerical models that can be prohibitively expensive in time-sensitive applications. Reduced-order models (ROMs) address this challenge by exploiting low-dimensional structure to create fast approximations. The Latent Space Dynamics Identification (LaSDI) framework has demonstrated success in learning ROMs for parameterized PDE families, but remains limited to first-order systems. Here, in this paper, we propose Higher-Order LaSDI (HLaSDI), which extends the LaSDI framework to PDEs with arbitrary order of time derivatives. This generalization significantly expands the applicability of LaSDI-based methods to systems previously outside their scope, including hyperbolic PDEs. We demonstrate HLaSDI’s accuracy and efficiency on several linear and nonlinear benchmark problems.
97 MATHEMATICS AND COMPUTING↗
Separable physics-informed DeepONet: Breaking the curse of dimensionality in physics-informed machine learning
The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. Here, to address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers’ equation, Biot’s consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning.
Neural operator↗
A framework for data-driven solution and parameter estimation of PDEs using conditional generative adversarial networks
We employ and adapt the image-to-image translation concept based on conditional generative adversarial networks (cGAN) for learning a forward and an inverse solution operator of partial differential equations (PDEs). We focus on steady-state solutions of coupled hydromechanical processes in heterogeneous porous media and present the parameterization of the spatially heterogeneous coefficients, which is exceedingly difficult using standard reduced-order modeling techniques. We show that our framework provides a speed-up of at least 2,000 times compared to a finite-element solver and achieves a relative root-mean-square error (r.m.s.e.) of less than 2% for forward modeling. For inverse modeling, the framework estimates the heterogeneous coefficients, given an input of pressure and/or displacement fields, with a relative r.m.s.e. of less than 7%, even for cases where the input data are incomplete and contaminated by noise. The framework also provides a speed-up of 120,000 times compared to a Gaussian prior-based inverse modeling approach while also delivering more accurate results.
97 MATHEMATICS AND COMPUTING↗
Synergistic learning with multi-task DeepONet for efficient PDE problem solving
Multi-task learning (MTL) is an inductive transfer mechanism designed to leverage useful information from multiple tasks to improve generalization performance compared to single-task learning. It has been extensively explored in traditional machine learning to address issues such as data sparsity and overfitting in neural networks. In this work, we apply MTL to problems in science and engineering governed by partial differential equations (PDEs). However, implementing MTL in this context is complex, as it requires task-specific modifications to accommodate various scenarios representing different physical processes. To this end, we present a multi-task deep operator network (MT-DeepONet) to learn solutions across various functional forms of source terms in a PDE and multiple geometries in a single concurrent training session. We introduce modifications in the branch network of the vanilla DeepONet to account for various functional forms of a parameterized coefficient in a PDE. Additionally, we handle parameterized geometries by introducing a binary mask in the branch network and incorporating it into the loss term to improve convergence and generalization to new geometry tasks. Our approach is demonstrated on three benchmark problems: (1) learning different functional forms of the source term in the Fisher equation; (2) learning multiple geometries in a 2D Darcy Flow problem and showcasing better transfer learning capabilities to new geometries; and (3) learning 3D parameterized geometries for a heat transfer problem and demonstrate the ability to predict on new but similar geometries. Finally, our MT-DeepONet framework offers a novel approach to solving PDE problems in engineering and science under a unified umbrella based on synergistic learning that reduces the overall training cost for neural operators.
42 ENGINEERING↗
Learning subgrid-scale models with neural ordinary differential equations
We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.
97 MATHEMATICS AND COMPUTING↗
Non-intrusive nonlinear model reduction via machine learning approximations to low-dimensional operators
Abstract Although projection-based reduced-order models (ROMs) for parameterized nonlinear dynamical systems have demonstrated exciting results across a range of applications, their broad adoption has been limited by their intrusivity: implementing such a reduced-order model typically requires significant modifications to the underlying simulation code. To address this, we propose a method that enables traditionally intrusive reduced-order models to be accurately approximated in a non-intrusive manner. Specifically, the approach approximates the low-dimensional operators associated with projection-based reduced-order models (ROMs) using modern machine-learning regression techniques. The only requirement of the simulation code is the ability to export the velocity given the state and parameters; this functionality is used to train the approximated low-dimensional operators. In addition to enabling nonintrusivity, we demonstrate that the approach also leads to very low computational complexity, achieving up to $$10^3{\times }$$ 10 3 × in run time. We demonstrate the effectiveness of the proposed technique on two types of PDEs. The domain of applications include both parabolic and hyperbolic PDEs, regardless of the dimension of full-order models (FOMs).
42 ENGINEERING↗
Reduced order models for thermal radiative transfer problems based on moment equations and data-driven approximations of the Eddington tensor
Here a new group of structure and asymptotic preserving reduced-order models (ROMs) for multidimensional nonlinear thermal radiative transfer (TRT) problems is presented. They are formulated by means of the nonlinear projective approach and data compression techniques. The nonlinear projection is applied to the Boltzmann transport equation (BTE) to derive a hierarchy of low-order moment equations. Approximation of the Eddington tensor that provides exact closure for the system of moment equations is found with projection-based data-driven methodologies. These include the (i) proper orthogonal decomposition (POD), (ii) dynamic mode decomposition (DMD) and (iii) a variant of the DMD. A parameterization is derived for this ROM for the temperature of radiation incoming to the problem domain (the radiation drive temperature). This parameterization is informed from results of a dimensionless study of the TRT problem. Analysis of the ROMs is performed on the classical Fleck-Cummings TRT multigroup test problem in 2D geometry with a radiation-driven Marshak wave. Numerical results are presented to demonstrate the performance of these ROMs for the simulation of evolving radiation and heat waves. Results show these models to be sufficiently accurate for practical computations with rather low-rank representations of the Eddington tensor. As the rank of the approximation is increased, the errors of solutions generated by the ROMs gradually decreases.
42 ENGINEERING↗
Differentiable Multiphysics Codes: A Breakthrough Technology for Simulation and Computing
This document summarizes the findings of a strategic planning exercise commissioned by the Weapons Simulation and Computing, Computational Physics (WSC/CP) program at the Lawrence Livermore National Laboratory (LLNL) in FY24. During the year, the committee met with multiple stakeholder communities to gather input, opinions, suggestions and concerns which have been incorporated throughout this document. The key findings from this exercise are summarized: • The development of multiphysics modelling and simulation (mod/sim) codes and software technologies, their deployment on exascale compute platforms, and their broad adoption across the NNSA is a major success of the Advanced Simulation and Computing (ASC) program and the Exascale Computing Project (ECP). Sustained investment in these core technologies is essential. • Today’s state of the art involves running ensembles of O(100K) simulations to perform uncertainty quantification (UQ) and design studies using multiple statistical methods such as Bayesian optimization to understand sensitivities of our models and explore parameterized design spaces. Even with exascale computing, we are practically limited to O(10) parameters in these studies since the number of simulations required to sample the space scales exponentially with the number of design parameters. • The data from these simulation ensembles is increasingly being used to train machine learned (ML) surrogates (or reduced order models, ROMs) which can then be used for optimization or real time design exploration. However, the trained surrogates are still limited in the number of parameters they can represent due to the sampling limitations previously noted. • Augmenting our suite of integrated multiphysics simulation codes, both current and emerging, with the ability to compute gradients (solution derivatives) of arbitrary simulation outputs with respect to (some or all) simulation inputs would be a breakthrough technology, opening the door to a new era of efficient and automated inverse design based on verified and validated mod/sim capabilities. • This capability, which we refer to as differentiable multiphysics codes (DMCs), would revolutionize both UQ and optimization studies by breaking the curse of dimensionality that presently limits our “gradient-free” ensemble based computing approach. A similar breakthrough occurred in the AI/ML community once the ability to compute gradients of arbitrary loss functions using back-propagation became commonplace. Gradient information from the multiphysics codes can also be used to dramatically improve the efficiency and scale of training of ML/ROM surrogates for rapid assessments. • Achieving this in our suite of codes will be a grand challenge, similar to the amount of effort that was required to transition from CPU to GPU computing. It will require buy-in from the entire WSC/CP program and beyond, including all integrated codes, physics and engineering models, third-party library dependencies and performance portability abstractions. It will also require investment in research and development of numerical methods for computing adjoints of coupled physics across multiple adaptively refined moving meshes and of stochastic (Monte Carlo) and mesh free (SPH) methods. • New software and numerical techniques, largely pioneered by the AI/ML community, make this feasible. Chief among these is automatic differentiation (AD), the ability to employ AD at point-wise locations in a physics calculation (instead of traditional black-box approaches) and the ability to perform “back-propagation in time” (or reverse mode AD) for non-linear partial differential equations (PDEs). Fundamentally, the conclusion of this strategic planning exercise is that the time is right to undertake a large scale effort in WSC, centered on the existing integrated codes, to continue the natural evolution of mod/sim in the age of AI/ML. Instead of attempting to replace mod/sim with purely data driven AI/ML models, we believe the key to success is to integrate AI/ML by building on top of the decades of hard-won knowledge and the verified/validated multiphysics modelling capability that is the hallmark of the ASC program.
97 MATHEMATICS AND COMPUTING↗