Search NASA⌕ Search

SEARCH · Search NASA

Results for “nonlinear partial differential equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification

Full-waveform inversion (FWI) is an accurate imaging approach for modeling the velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. However, the strong nonlinearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We have adopted a neural-network-based full-waveform inversion (NNFWI) method that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, which suppresses noise in the gradients and mitigates local minima. Furthermore, the velocity model generated by neural networks is input to the same partial differential equation (PDE) solvers used in conventional FWI. The gradients of the neural networks and PDEs are calculated using automatic differentiation, which back propagates gradients through the acoustic PDEs and neural network layers to update the weights of the generative neural network. Experiments on 1D velocity models, the Marmousi model, and the 2004 BP model determine that NNFWI can mitigate local minima, especially for imaging high-contrast features such as salt bodies, and it significantly improves the inversion in the presence of noise. Adding dropout layers to the neural network model also allows analyzing the uncertainty of the inversion results through Monte Carlo dropout. NNFWI opens a new pathway to combine deep learning and FWI for exploiting the characteristics of deep neural networks and the high accuracy of PDE solvers. Because NNFWI does not require extra training data and optimization loops, it provides an attractive and straightforward alternative to conventional FWI.

58 GEOSCIENCES↗
DOE OSTI2024

Learning physics-based reduced-order models from data using nonlinear manifolds

Here we present a novel method for learning reduced-order models of dynamical systems using nonlinear manifolds. First, we learn the manifold by identifying nonlinear structure in the data through a general representation learning problem. The proposed approach is driven by embeddings of low-order polynomial form. A projection onto the nonlinear manifold reveals the algebraic structure of the reduced-space system that governs the problem of interest. The matrix operators of the reduced-order model are then inferred from the data using operator inference. Numerical experiments on a number of nonlinear problems demonstrate the generalizability of the methodology and the increase in accuracy that can be obtained over reduced-order modeling methods that employ a linear subspace approximation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗
DOE OSTI2022

Machine Learning with Gradient-Based Optimization of Nuclear Waste Vitrification with Uncertainties and Constraints

Gekko is an optimization suite in Python that solves optimization problems involving mixed-integer, nonlinear, and differential equations. The purpose of this study is to integrate common Machine Learning (ML) algorithms such as Gaussian Process Regression (GPR), support vector regression (SVR), and artificial neural network (ANN) models into Gekko to solve data based optimization problems. Uncertainty quantification (UQ) is used alongside ML for better decision making. These methods include ensemble methods, model-specific methods, conformal predictions, and the delta method. An optimization problem involving nuclear waste vitrification is presented to demonstrate the benefit of ML in this field. ML models are compared against the current partial quadratic mixture (PQM) model in an optimization problem in Gekko. GPR with conformal uncertainty was chosen as the best substitute model as it had a lower mean squared error of 0.0025 compared to 0.018 and more confidently predicted a higher waste loading of 37.5 wt% compared to 34 wt%. The example problem shows that these tools can be used in similar industry settings where easier use and better performance is needed over classical approaches. Future works with these tools include expanding them with other regression models and UQ methods, and exploration into other optimization problems or dynamic control.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Assimilating partial observation to enhance feedback control of stochastic dynamical systems

Here, in this paper, we present a novel methodology to tackle feedback optimal control problems in scenarios where the exact state of the controlled process is unknown. It integrates data assimilation techniques and optimal control solvers to manage partial observation of the state process, a common occurrence in practical scenarios. Traditional stochastic optimal control methods assume full state observation, which is often not feasible in real-world fluid dynamics control problems. Our approach underscores the significance of utilizing observational data to inform control policy design. Specifically, we introduce a kernel learning backward stochastic differential equation (SDE) filter to enhance data assimilation efficiency and propose a sample-wise stochastic optimization method within the stochastic maximum principle framework. We demonstrate the efficacy and accuracy of our method in the control of advection-diffusion-reaction flow problem and the Dubins airplane maneuvering problem with model uncertainty.

data driven↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Degradation of performance in ICF implosions due to Rayleigh–Taylor instabilities: A Hamiltonian perspective

The Rayleigh–Taylor instability (RTI) is an ubiquitous phenomenon that occurs in inertial-confinement-fusion (ICF) implosions and is recognized as an important limiting factor of ICF performance. To analytically understand the RTI dynamics and its impact on ICF capsule implosions, we develop a first-principle variational theory that describes an imploding spherical shell undergoing RTI. The model is based on a thin-shell approximation and includes the dynamical coupling between the imploding spherical shell and an adiabatically compressed fluid within its interior. Using a quasilinear analysis, we study the degradation trends of key ICF performance metrics (e.g., stagnation pressure, residual kinetic energy, and areal density) as functions of initial RTI parameters (e.g., the initial amplitude and Legendre mode), as well as the 1D implosion characteristics (e.g., the convergence ratio). We compare analytical results from the theory against nonlinear results obtained by numerically integrating the governing equations of this reduced model. Our findings emphasize the need to incorporate polar flows in the calculation of residual kinetic energy and demonstrate that higher convergence ratios in ICF implosions lead to significantly greater degradation of key performance metrics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Physics-informed State-space Neural Networks for transport phenomena

This work introduces Physics -informed State -space neural network Models (PSMs), a novel solution to achieving real-time optimization, flexibility, and fault tolerance in autonomous systems, particularly in transportdominated systems such as chemical, biomedical, and power plants. Traditional data -driven methods fall short due to a lack of physical constraints like mass conservation; PSMs address this issue by training deep neural networks with sensor data and physics -informing using components' Partial Differential Equations (PDEs), resulting in a physics -constrained, end -to -end differentiable forward dynamics model. Further, through two in silico experiments - a heated channel and a cooling system loop - we demonstrate that PSMs offer a more accurate approach than a purely data -driven model. In the former experiment, PSMs demonstrated significantly lower average root -mean -square errors across test datasets compared to a purely data -driven neural network, with reductions of 44 %, 48 %, and 94 % in predicting pressure, velocity, and temperature, respectively. Beyond accuracy, PSMs demonstrate a compelling multitask capability, making them highly versatile. In this work, we showcase two: supervisory control of a nonlinear system through a sequentially updated state -space representation and the proposal of a diagnostic algorithm using residuals from each of the PDEs. The former demonstrates PSMs' ability to handle constant and time -dependent constraints, while the latter illustrates their value in system diagnostics and fault detection.

42 ENGINEERING↗
DOE OSTI2023

Applications of physics informed neural operators

Abstract We present a critical analysis of physics-informed neural operators (PINOs) to solve partial differential equations (PDEs) that are ubiquitous in the study and modeling of physics phenomena using carefully curated datasets. Further, we provide a benchmarking suite which can be used to evaluate PINOs in solving such problems. We first demonstrate that our methods reproduce the accuracy and performance of other neural operators published elsewhere in the literature to learn the 1D wave equation and the 1D Burgers equation. Thereafter, we apply our PINOs to learn new types of equations, including the 2D Burgers equation in the scalar, inviscid and vector types. Finally, we show that our approach is also applicable to learn the physics of the 2D linear and nonlinear shallow water equations, which involve three coupled PDEs. We release our artificial intelligence surrogates and scientific software to produce initial data and boundary conditions to study a broad range of physically motivated scenarios. We provide the source code , an interactive website to visualize the predictions of our PINOs, and a tutorial for their use at the Data and Learning Hub for Science .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Non-autoregressive time-series methods for stable parametric reduced-order models

Advection-dominated dynamical systems, characterized by partial differential equations, are found in applications ranging from weather forecasting to engineering design where accuracy and robustness are crucial. There has been significant interest in the use of techniques borrowed from machine learning to reduce the computational expense and/or improve the accuracy of predictions for these systems. These rely on the identification of a basis that reduces the dimensionality of the problem and the subsequent use of time series and sequential learning methods to forecast the evolution of the reduced state. Often, however, machine-learned predictions after reduced-basis projection are plagued by issues of stability stemming from incomplete capture of multiscale processes as well as due to error growth for long forecast durations. To address these issues, we have developed a non-autoregressive time series approach for predicting linear reduced-basis time histories of forward models. In particular, we demonstrate that non-autoregressive counterparts of sequential learning methods such as long short-term memory (LSTM) considerably improve the stability of machine-learned reduced-order models. Further, we evaluate our approach on the inviscid shallow water equations and show that a non-autoregressive variant of the standard LSTM approach that is bidirectional in the principal component directions obtains the best accuracy for recreating the nonlinear dynamics of partial observations. Moreover-and critical for many applications of these surrogates-inference times are reduced by three orders of magnitude using our approach, compared with both the equation-based Galerkin projection method and the standard LSTM approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Thermomechanics coupling to Monte Carlo particle transport on unstructured mesh geometries using Cardinal

Geometry deformation due to thermal expansion influences neutron transport in many systems. Studying this phenomenon involves coupling models for neutronics, thermal hydraulics, and solid mechanics. To enable high fidelity modeling of these coupled physics, new capabilities were introduced in Cardinal, coupling OpenMC Monte Carlo particle transport models with MOOSE thermomechanical physics on unstructured moving-mesh geometries. In this work, we present a fully open-source capability leveraging on-the-fly mesh skinning to automatically regenerate OpenMC geometry, which allows multiphysics feedback from temperature, density, and geometry changes. The new capability is verified using an analytic benchmark slab problem, which couples S 2 neutron transport with thermal conduction, convective boundary conditions, Doppler-broadened cross sections, and nonlinear thermal expansion effects along the heated slab. Cardinal reproduces the analytic solutions for the neutron flux, heating, k eff , and temperature with demonstrated convergence in various error terms including mesh resolution and cross section temperature library spacing. For the nominal benchmark conditions and with a fine mesh, maximum relative errors for neutron flux, temperature, and heating are lower than 1%, while errors in integral quantities such as k eff and slab length are within 1 pcm and 48 µm, respectively. This work (i) presents a new numerical approach to thermomechanics coupling with OpenMC models, (ii) is the first (to our knowledge) to utilize a mechanical partial differential equation (PDE) solution to solve the (Griesheimer and Kooreman, 2022) analytic benchmark, and (iii) develops this verified capability within an open-source package.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Adaptive Uncertainty Quantification for Stochastic Hyperbolic Conservation Laws

Here, we propose a predictor-corrector adaptive method for the study of hyperbolic partial differential equations (PDEs) under uncertainty. Constructed around the framework of stochastic finite volume (SFV) methods, our approach circumvents sampling schemes or simulation ensembles while also preserving fundamental properties, in particular hyperbolicity of the resulting systems and conservation of the discrete solutions. Furthermore, we augment the existing SFV theory with a priori convergence results for statistical quantities, in particular push-forward densities, which we demonstrate through numerical experiments. By linking refinement indicators to regions of the physical and stochastic spaces, we drive anisotropic refinements of the discretizations, introducing new degrees of freedom where deemed profitable. To illustrate our proposed method, we consider a series of numerical examples for nonlinear hyperbolic PDEs based on Burgers’ and Euler’s equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

A model for discrete fracture-clay rock interaction incorporating electrostatic effects on transport

Abstract A model based on the code CrunchClay is presented for a fracture-clay matrix system that takes electrostatic effects on transport into account. The electrostatic effects on transport include those associated with the development of a diffusion potential as captured by the Nernst-Planck equation, and the formation of a diffuse layer bordering negatively charged clay particles within which partial anion exclusion occurs. The model is based on a dual continuum formulation that accounts for diffuse layer and bulk water pore space, providing a more flexible framework than is found in the classical mean electrostatic potential models. The diffuse layer model is obtained by volume averaging ion concentrations in the Poisson-Boltzmann equation, but also includes the treatment of longitudinal transport within this continuum. The calculation of transport within the bulk and diffuse layer porosity is based on a new formulation for the Nernst-Planck equation that considers averaging of diffusion coefficients and accumulation factors at grid cell interfaces. Equations for function residuals and the associated Jacobian matrix are presented such that the system of nonlinear differential-algebraic equations can be solved with Newton’s method. As an example, we consider a 2D system with a single discrete fracture within which flow and advective transport occurs that is coupled to diffusion in the clay-rich matrix. The simulation results demonstrate the lack of retardation for anions (e.g., 36 Cl − ) of the contaminant plume within the fracture flow system because they are largely excluded from the charged clay rock, while the migration of cations (e.g., 90 Sr ++ ) is more strongly attenuated. The diffusive loss of divalent cations in particular from the fracture is accentuated by their accumulation in the diffuse layer within the clay-rich matrix.

58 GEOSCIENCES↗
DOE OSTI2023

Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles

Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models, require a large number of observations and lack operational guarantees for safety-critical systems. Data-driven models leveraging available partially characterized dynamics have potential to provide reliable systems models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work we explore this middle-ground between expert-modeled and pure data-driven modeling. We present control-oriented parametric models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations to construct data-driven blackbox and graybox representations of the AUV dynamics. In addition, we explore a hybrid formulation that explicitly models the residual error related to imperfect graybox models. We compare the prediction performance of the learned models for different distributions of initial conditions and control inputs to assess their suitability for control.

Shaw Cortez, Wenceslao E.↗
DOE OSTI2023

Modelling and simulation of brinicle formation

Below the Arctic sea ice, under the right conditions, a flux of icy brine flows down into the sea. The icy brine has a much lower fusion point and is denser than normal seawater. As a result, it sinks while freezing everything around it, forming an ice channel called a brinicle (also known as ice stalactite). In this paper, we develop a mathematical model for this phenomenon, assuming cylindrical symmetry. The fluid is considered to be viscous and quasi-stationary. The heat and salt transport are weakly coupled to the fluid motion and are modelled with the corresponding conservation equations, accounting for diffusive and convective effects. Finite-element discretization is employed to solve the coupled system of partial differential equations. We find that the model can capture the general behaviour of the physical system and generate brinicle-like structures while also recovering dendrite composition, which is a physically expected feature aligned with previous experimental results. This represents, to our knowledge, the first complete model proposed that captures the global structure of the physical phenomenon even though it has some discrepancies, such as brine accumulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Absolute stimulated Raman side scatter in direct-drive laser-produced plasmas

Production of suprathermal electrons by stimulated Raman scattering (SRS) is a principal concern for contemporary direct-drive inertial confinement fusion experiments at the National Ignition Facility and similar systems since such electrons penetrate and preheat the target core, preventing efficient implosion. The higher temperatures and longer scale lengths in these experiments favor SRS over two-plasmon decay, which predominated in earlier experiments. In particular, current experiments are expected to exceed the threshold for absolute Raman side scatter, which would then dominate the interaction since it grows temporally until saturated by nonlinear mechanisms such as hot-electron production. Until recently, analyses of SRS side scatter have treated the case of a single laser beam incident on a plasma, but the direct-drive approach to laser fusion employs a multitude of beams to drive the implosion. Here in this paper, we present an analysis that can be applied to an arbitrary number of beams with varied angles of incidence and polarizations. In the case of a single beam, it allows a physically motivated derivation and verification of an analytic threshold formula. In the general case of multiple beams and arbitrary orientation and polarizations, the threshold is found by numerical integration of a set of first-order linear partial differential equations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Higher-order LaSDI: Reduced order modeling with multiple time derivatives

Solving complex partial differential equations (PDEs) is essential across scientific disciplines but often requires numerical models that can be prohibitively expensive in time-sensitive applications. Reduced-order models (ROMs) address this challenge by exploiting low-dimensional structure to create fast approximations. The Latent Space Dynamics Identification (LaSDI) framework has demonstrated success in learning ROMs for parameterized PDE families, but remains limited to first-order systems. Here, in this paper, we propose Higher-Order LaSDI (HLaSDI), which extends the LaSDI framework to PDEs with arbitrary order of time derivatives. This generalization significantly expands the applicability of LaSDI-based methods to systems previously outside their scope, including hyperbolic PDEs. We demonstrate HLaSDI’s accuracy and efficiency on several linear and nonlinear benchmark problems.

97 MATHEMATICS AND COMPUTING↗