SEARCH · Search NASA

Results for “nonlinear partial differential equations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

All sources NASA NTRS DOE OSTI

NASA NTRS1996

Low-Dimensional Dynamical Models of Thermal Convection

A low-dimensional dynamic model for transitional buoyancy-driven flow in a differentially heated tall enclosure is presented. The full governing partial differential equations with the associated boundary conditions are solved by a spectral element method for a cavity of aspect ratio A=20. Proper orthogonal decomposition is applied to the oscillatory solution at Prandtl number Pr=P tau (omega) = 0.71 and Grashof number G tau (omega) = 3.2 x 10 (exp 4) to construct empirical eigenfunctions. Using the four most energetic empirical eigenfunctions for the velocity and temperature as basis functions and applying Galerkin's method, a reduced model consisting of eight nonlinear ordinary differential equations is obtained. Close to the 'design' conditions (P tau(omega) G tau(omega)), the low-order model (LOM) predictions are in excellent agreement with the predictions of the full model. In particular, the critical Grashof number at the onset of the first temporal flow instability (Hopf bifurcation) was well as the frequency and amplitude of oscillations at supercritical conditions are in excellent agreement with the predictions of the full model. Far from the 'design' conditions, the LOM predicts the existence of multiple stable steady solutions at large values of G tau, and a unique stable steady solution at small values of G tau, and exhibits hysteretic behavior that is qualitatively similar to that observed in direct numerical simulations based on the full model.

Liakopoulos, Anthony↗

NASA NTRS1981

Multigrid solvers on parallel computers

Massively parallel computers, as considered in this investigation, are not yet available. However, a large-scale parallel computer cannot usefully be designed before the hypothetical algorithms which will employ it are studied. Most of the studies of parallel partial differential equations (PDE) solvers are based on solution techniques much slower (on sequential machines) than multigrid methods. Multigrid methods are highly parallelizable. Each of their processes can simultaneously be performed at all grid points. The present investigation is concerned with a preliminary exploration of the potential of multigrid, or, more generally, Multi-Level Adaptive Techniques (MLAT) on computers with many processors. Basic processes are considered, taking into account coarse-grid approximation, relaxation, coarse-grid corrections, full multigrid algorithms, nonlinear problems and eigenvalue problems, fine-to-coarse correction, and chains of problems. Details of parallel multigrid processing are also examined.

Brandt, A.↗

NASA NTRSDate not supplied

Fluid System Stability Analysis Techniques

Fluid systems, or networks, consist of multiple components that work together to achieve some desired thermofluid state. For any generic application, this fluid state can be a combination of the fluid pressure, flow rate, enthalpy, or species concentration. Fluid system components, such as pumps and valves, are often governed by nonlinear differential equations, resulting in complex component-to-component interactions. System-level fluid network stability occurs when the flow through the system can maintain a steady-state solution in the presence of small perturbations, which depends on these component interactions. System instability, however, can go undetected until issues arise during integrated system testing. This presentation explores a method for system designers to think of the fluid network as an assembly of components, each with their own thermofluid surfaces of partial stability, called nullclines. The intersections of all nullclines yields system-level solutions, called equilibrium points. When designers define operating points, they are tuning system parameters so that these equilibrium points move to the desired location in the thermofluid state plane. However, linearization theory shows us that the dynamic behavior around these points can be unstable. The local stability of these equilibrium points can be assessed analytically with eigen-analysis, or numerically by propagating state-plane samples to construct a phase portrait. Investigating a phase portrait can help designers gain a qualitative understanding of a system’s dynamic performance. This understanding can then help inform requirement definitions, component selection, and operational procedures. This presentation includes an example of the phase portrait technique on a system featuring a centrifugal pump and a back-pressure regulator (BPR). Numerical modeling of this system suggests that equilibrium points on the left-hand side of the pump curve are dynamically unstable.

Fluid System↗

NASA NTRSDate not supplied

Fluid System Stability Analysis Techniques

Fluid System↗

DOE OSTI2024

Bayesian reduced-order deep learning surrogate model for dynamic systems described by partial differential equations

We propose a reduced-order deep-learning surrogate model for dynamic systems described by time-dependent partial differential equations. This method employs space–time Karhunen–Loève expansions (KLEs) of the state variables and space-dependent KLEs of space-varying parameters to identify the reduced (latent) dimensions. Subsequently, a deep neural network (DNN) is used to map the parameter latent space to the state variable latent space. An approximate Bayesian method is developed for uncertainty quantification (UQ) in the proposed KL-DNN surrogate model. The KL-DNN method is tested for the linear advection–diffusion and nonlinear diffusion equations, and the Bayesian approach for UQ is compared with the deep ensembling (DE) approach, commonly used for quantifying uncertainty in DNN models. It was found that the approximate Bayesian method provides a more informative distribution of the PDE solutions in terms of the coverage of the reference PDE solutions (the percentage of nodes where the reference solution is within the confidence interval predicted by the UQ methods) and log predictive probability. The DE method is found to underestimate uncertainty and introduce bias. For the nonlinear diffusion equation, we compare the KL-DNN method with the Fourier Neural Operator (FNO) method and find that KL-DNN is 10% more accurate and needs less training time than the FNO method.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Integrating deep neural networks with full-waveform inversion: Reparameterization, regularization, and uncertainty quantification

Full-waveform inversion (FWI) is an accurate imaging approach for modeling the velocity structure by minimizing the misfit between recorded and predicted seismic waveforms. However, the strong nonlinearity of FWI resulting from fitting oscillatory waveforms can trap the optimization in local minima. We have adopted a neural-network-based full-waveform inversion (NNFWI) method that integrates deep neural networks with FWI by representing the velocity model with a generative neural network. Neural networks can naturally introduce spatial correlations as regularization to the generated velocity model, which suppresses noise in the gradients and mitigates local minima. Furthermore, the velocity model generated by neural networks is input to the same partial differential equation (PDE) solvers used in conventional FWI. The gradients of the neural networks and PDEs are calculated using automatic differentiation, which back propagates gradients through the acoustic PDEs and neural network layers to update the weights of the generative neural network. Experiments on 1D velocity models, the Marmousi model, and the 2004 BP model determine that NNFWI can mitigate local minima, especially for imaging high-contrast features such as salt bodies, and it significantly improves the inversion in the presence of noise. Adding dropout layers to the neural network model also allows analyzing the uncertainty of the inversion results through Monte Carlo dropout. NNFWI opens a new pathway to combine deep learning and FWI for exploiting the characteristics of deep neural networks and the high accuracy of PDE solvers. Because NNFWI does not require extra training data and optimization loops, it provides an attractive and straightforward alternative to conventional FWI.

58 GEOSCIENCES↗

NASA NTRS1974

Fast direct numerical solution of the nonhomogeneous Cauchy-Riemann equations

A fast direct (noniterative) 'Cauchy-Riemann Solver' is developed for solving the finite-difference equations representing systems of first-order elliptic partial differential equations in the form of the nonhomogeneous Cauchy-Riemann equations. The method is second-order accurate and requires approximately the same computer time as a fast cyclic-reduction Poisson solver. The accuracy and efficiency of the direct solver are demonstrated in an application to solving an example problem in aerodynamics: subsonic inviscid flow over a biconvex airfoil. The analytical small-perturbation solution contains singularities, which are captured well by the computational technique. The algorithm is expected to be useful in nonlinear subsonic and transonic aerodynamics.

Lomax, H.↗

NASA NTRS1980

Towards sub-optimal stochastic control of partially observable stochastic systems

A class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design is examined. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in dimensions is proven.

Ruzicka, G. J.↗

DOE OSTI2024

Learning physics-based reduced-order models from data using nonlinear manifolds

Here we present a novel method for learning reduced-order models of dynamical systems using nonlinear manifolds. First, we learn the manifold by identifying nonlinear structure in the data through a general representation learning problem. The proposed approach is driven by embeddings of low-order polynomial form. A projection onto the nonlinear manifold reveals the algebraic structure of the reduced-space system that governs the problem of interest. The matrix operators of the reduced-order model are then inferred from the data using operator inference. Numerical experiments on a number of nonlinear problems demonstrate the generalizability of the methodology and the increase in accuracy that can be obtained over reduced-order modeling methods that employ a linear subspace approximation.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1984

Variational objective analyses for cyclone studies

The basic analysis equations, i.e., the two horizontal momentum equations, the hydrostatic equation, and the integrated continuity equation were derived for the nonlinear vertical coordinate, nondimensionalized, and expressed in finite differences on a staggered grid. Special care was taken to transform the hydrostatic equation and the pressure gradient terms of the horizontal momentum equations to nearly eliminate truncation error over steeply sloping terrain. This formulation also eliminated explicit reference to orographically induced variations in the thermodynamic variables so that the variational adjustments are on the scale of the meteorological perturbations. The analysis equations were subjected to the Euler-Lagrange operations as expressed for finite differences and an additional set of five partial differential equations was derived, bringing to nine the number of equations in Model I. Higher order terms, terms containing observed quantities, and terms containing none of the variables to be adjusted were grouped into forcing functions and the equations were solved for the zero order terms. Zero order variables were eliminated between these equations and there resulted two diagnostic equations which take the form of general linear second order partial differential equations with nonconstant coefficients.

Achtemeier, G. L.↗

NASA NTRS1980

Towards sub-optimal stochastic control of partially observable stochastic systems

The paper deals with a class of multidimensional stochastic control problems with noisy data and bounded controls encountered in aerospace design. The emphasis is on suboptimal design, the optimality being taken in quadratic mean sense. To that effect the problem is viewed as a stochastic version of the Lurie problem known from nonlinear control theory. The main result is a separation theorem (involving a nonlinear Kalman-like filter) suitable for Lurie-type approximations. The theorem allows for discontinuous characteristics. As a byproduct the existence of strong solutions to a class of non-Lipschitzian stochastic differential equations in n dimensions is proved.

Ruzicka, G. J.↗

DOE OSTI2025

Optimizing the optimizer for physics-informed neural networks and Kolmogorov-Arnold networks

Physics-Informed Neural Networks (PINNs) have revolutionized the computation of PDE solutions by integrating partial differential equations (PDEs) into the neural network’s training process as soft constraints, becoming an important component of the scientific machine learning (SciML) ecosystem. More recently, physics-informed Kolmogorv-Arnold networks (PIKANs) have also shown to be effective and comparable in accuracy with PINNs. In their current implementation, both PINNs and PIKANs are mainly optimized using first-order methods like Adam, as well as quasi-Newton methods such as BFGS and its low-memory variant, L-BFGS. However, these optimizers often struggle with highly nonlinear and non-convex loss landscapes, leading to challenges such as slow convergence, local minima entrapment, and (non)degenerate saddle points. In this study, we investigate the performance of Self- Scaled BFGS (SSBFGS), Self-Scaled Broyden (SSBroyden) methods and other advanced quasi-Newton schemes, including BFGS and L-BFGS with different line search strategies. These methods dynamically rescale updates based on historical gradient information, thus enhancing training efficiency and accuracy. We systematically compare these optimizers – using both PINNs and PIKANs – on key challenging PDEs, including the Burgers, Allen-Cahn, Kuramoto-Sivashinsky, Ginzburg-Landau, and Stokes equations. Additionally, we evaluate the performance of SSBFGS and SSBroyden for Deep Operator Network (DeepONet) architectures, demonstrating their effectiveness for data-driven operator learning. Our findings provide state-of-the-art results with orders-of-magnitude accuracy improvements without the use of adaptive weights or any other enhancements typically employed in PINNs. More broadly, our work reveal insights into the effectiveness of quasi-Newton optimization strategies in significantly improving the convergence and accurate generalization of PINNs and PIKANs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗

DOE OSTI2022

Machine Learning with Gradient-Based Optimization of Nuclear Waste Vitrification with Uncertainties and Constraints

Gekko is an optimization suite in Python that solves optimization problems involving mixed-integer, nonlinear, and differential equations. The purpose of this study is to integrate common Machine Learning (ML) algorithms such as Gaussian Process Regression (GPR), support vector regression (SVR), and artificial neural network (ANN) models into Gekko to solve data based optimization problems. Uncertainty quantification (UQ) is used alongside ML for better decision making. These methods include ensemble methods, model-specific methods, conformal predictions, and the delta method. An optimization problem involving nuclear waste vitrification is presented to demonstrate the benefit of ML in this field. ML models are compared against the current partial quadratic mixture (PQM) model in an optimization problem in Gekko. GPR with conformal uncertainty was chosen as the best substitute model as it had a lower mean squared error of 0.0025 compared to 0.018 and more confidently predicted a higher waste loading of 37.5 wt% compared to 34 wt%. The example problem shows that these tools can be used in similar industry settings where easier use and better performance is needed over classical approaches. Future works with these tools include expanding them with other regression models and UQ methods, and exploration into other optimization problems or dynamic control.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

Assimilating partial observation to enhance feedback control of stochastic dynamical systems

Here, in this paper, we present a novel methodology to tackle feedback optimal control problems in scenarios where the exact state of the controlled process is unknown. It integrates data assimilation techniques and optimal control solvers to manage partial observation of the state process, a common occurrence in practical scenarios. Traditional stochastic optimal control methods assume full state observation, which is often not feasible in real-world fluid dynamics control problems. Our approach underscores the significance of utilizing observational data to inform control policy design. Specifically, we introduce a kernel learning backward stochastic differential equation (SDE) filter to enhance data assimilation efficiency and propose a sample-wise stochastic optimization method within the stochastic maximum principle framework. We demonstrate the efficacy and accuracy of our method in the control of advection-diffusion-reaction flow problem and the Dubins airplane maneuvering problem with model uncertainty.

data driven↗

DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1994

Spectral Methods for Computational Fluid Dynamics

As a tool for large-scale computations in fluid dynamics, spectral methods were prophesized in 1944, born in 1954, virtually buried in the mid-1960's, resurrected in 1969, evangalized in the 1970's, and catholicized in the 1980's. The use of spectral methods for meteorological problems was proposed by Blinova in 1944 and the first numerical computations were conducted by Silberman (1954). By the early 1960's computers had achieved sufficient power to permit calculations with hundreds of degrees of freedom. For problems of this size the traditional way of computing the nonlinear terms in spectral methods was expensive compared with finite-difference methods. Consequently, spectral methods fell out of favor. The expense of computing nonlinear terms remained a severe drawback until Orszag (1969) and Eliasen, Machenauer, and Rasmussen (1970) developed the transform methods that still form the backbone of many large-scale spectral computations. The original proselytes of spectral methods were meteorologists involved in global weather modeling and fluid dynamicists investigating isotropic turbulence. The converts who were inspired by the successes of these pioneers remained, for the most part, confined to these and closely related fields throughout the 1970's. During that decade spectral methods appeared to be well-suited only for problems governed by ordinary diSerential eqllations or by partial differential equations with periodic boundary conditions. And, of course, the solution itself needed to be smooth. Some of the obstacles to wider application of spectral methods were: (1) poor resolution of discontinuous solutions; (2) inefficient implementation of implicit methods; and (3) drastic geometric constraints. All of these barriers have undergone some erosion during the 1980's, particularly the latter two. As a result, the applicability and appeal of spectral methods for computational fluid dynamics has broadened considerably. The motivation for the use of spectral methods in numerical calculations stems from the attractive approximation properties of orthogonal polynomial expansions.

Zang, T. A.↗

NASA NTRS1997

[Development of New Mathematical Methodology in Air Traffic Control for the Analysis of Hybrid Systems]

The aim of this research is to develop new mathematical methodology for the analysis of hybrid systems of the type involved in Air Traffic Control (ATC) problems. Two directions of investigation were initiated. The first used the methodology of nonlinear generalized functions, whose mathematical foundations were initiated by Colombeau and developed further by Oberguggenberger; it has been extended to apply to ordinary differential. Systems of the type encountered in control in joint work with the PI and M. Oberguggenberger. This involved a 'mixture' of 'continuous' and 'discrete' methodology. ATC clearly involves mixtures of two sorts of mathematical problems: (1) The 'continuous' dynamics of a standard control type described by ordinary differential equations (ODE) of the form: {dx/dt = f(x, u)} and (2) the discrete lattice dynamics involved of cellular automata. Most of the CA literature involves a discretization of a partial differential equation system of the type encountered in physics problems (e.g. fluid and gas problems). Both of these directions requires much thinking and new development of mathematical fundamentals before they may be utilized in the ATC work. Rather than consider CA as 'discretization' of PDE systems, I believe that the ATC applications will require a completely different and new mathematical methodology, a sort of discrete analogue of jet bundles and/or the sheaf-theoretic techniques to topologists. Here too, I have begun work on virtually 'virgin' mathematical ground (at least from an 'applied' point of view) which will require considerable preliminary work.

Hermann, Robert↗