Search NASA⌕ Search

SEARCH · Search NASA

Results for “NONLINEAR MAPPING APPROXIMATOR”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

51 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Mean Flow from Phase Averages in the 2D Boussinesq Equations

The atmosphere and ocean are described by highly oscillatory PDEs that challenge both our understanding of their dynamics and their numerical approximation. This paper presents a preliminary numerical study of one type of phase averaging applied to mean flows in the 2D Boussinesq equations that also has application to numerical methods. The phase averaging technique, well-known in dynamical systems theory, relies on a mapping using the exponential operator, and then an averaging over the phase. The exponential operator has connections to the Craya–Herring basis pioneered by Jack Herring to study the fluid dynamics of oscillatory, nonlinear fluid dynamics. In this paper, we perform numerical experiments to study the effect of this averaging technique on the time evolution of the solution. We explore its potential as a definition for mean flows. We also show that, as expected from theory, the phase-averaging method can reduce the magnitude of the time rate of change in the PDEs, making them potentially suitable for time stepping methods.

Wingate, Beth A.↗
DOE OSTI2022

A comprehensive and fair comparison of two neural operators (with practical extensions) based on $\mathrm{FAIR}$ data

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

42 ENGINEERING↗
DOE OSTI2023

Spatially quasi-periodic water waves of finite depth

We present a numerical study of spatially quasi-periodic gravity-capillary waves of finite depth in both the initial value problem and travelling wave settings. We adopt a quasi-periodic conformal mapping formulation of the Euler equations, where one-dimensional quasi-periodic functions are represented by periodic functions on a higher-dimensional torus. We compute the time evolution of free surface waves in the presence of a background flow and a quasi-periodic bottom boundary and observe the formation of quasi-periodic patterns on the free surface. Two types of quasi-periodic travelling waves are computed: small-amplitude waves bifurcating from the zero-amplitude solution and larger-amplitude waves bifurcating from finite-amplitude periodic travelling waves. We derive weakly nonlinear approximations of the first type and investigate the associated small-divisor problem. We find that waves of the second type exhibit striking nonlinear behaviour, e.g. the peaks and troughs are shifted non-periodically from the corresponding periodic waves due to the activation of quasi-periodic modes.

Science & Technology - Other Topics↗
DOE OSTI2022

The nonlinear rheology of complex yield stress foods

Many foods have a yield stress that allows them to retain a desired shape at rest, but transition into a viscous fluid when being served or consumed. The determination of the yield stress of the food dictates how the foods are formed and packaged, how they are served, and how they are perceived when being eaten. Oscillatory shearing provides an ideal test protocol to map the rheology across a range of time and flow strength scales. We couple oscillatory shearing and an iterative recovery procedure to show that the yielding process is a continuous transition for two common yield stress foods. We show that unrecoverable processes from oscillatory tests are equivalent to the steady shear flow behavior. We show that this yielding behavior can be well approximated by a recently published model that treats yield stress materials as continuous viscoelastic fluids with a rate-dependent relaxation time and has parameters that can be obtained from the linear viscoelastic oscillatory frequency sweep and the steady shear flow curve.

Griebler, James J.↗
DOE OSTI2021

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, nonconvexity, data sparsity, and nontrivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss--Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to nonquadratic objective functions, most notably cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification, GNvpro solves the optimization problem more efficiently than commonly used stochastic gradient descent (SGD) schemes. Finally, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Skyrmions and hopfions in three-dimensional frustrated magnets

Here, a model of an inversion-symmetric frustrated spin system is introduced which hosts three-dimensional extensions of magnetic skyrmions. In the continuum approximation, this model reduces to a nonlinear sigma model on a squashed sphere that has a natural interpolating parameter. At one limit of the parameter, the model reduces to a frustrated magnetic system earlier considered by Sutcliffe as a host to hopfions, and in the other limit, it becomes very similar to the 3D Skyrme model. To better understand the relation between hopfions and 3D skyrmions, a model interpolating between the Faddeev-Niemi and the Skyrme models is reconsidered and it is shown that energies of the solitons obey a linear Bogomol'nyi-Prasad-Sommerfeld bound. The 3D skyrmions in the frustrated magnetic model are found and compared to the rational map ansatz.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

AEOLUS: Advances in Experimental Design, Optimal Control, and Learning for Uncertain Complex Systems

Sustained advances in the mathematics of modeling and simulation have resulted in the capability today for routine simulation of a number of large scale complex DOE-relevant systems. As remarkable as this capability for solving the so-called forward problem is, it is typically only the first step-an inner loop within an outer loop that explores the simulation model's parameter space and decision space to characterize uncertainty in the model's predictions, learn unknown model parameters from data, design the most informative experiments, determine optimal control strategies, and create optimal designs. Broadly, what unifies all of these outer loop problems is that they are, in one form or another, optimization problems over parameter/control/design space that are constrained by complex uncertain models. To fully realize the power of scientific simulation as a basis for scientific discovery, technological innovation, and rational decision-making, it is imperative to move beyond simulation to tackle the outer loop of optimization for learning from data, experimental design, and control with complex uncertain models. When the models under consideration are large-scale and complex, and when the optimization variable and uncertain parameter spaces are high (or infinite) dimensional, this constitutes a grand challenge of the highest order, and is intractable with conventional methods. To overcome these challenges, the AEOLUS Center was established to develop a unified mathematical, computational, and statistical framework for (1) Learning predictive models from complex data via Bayesian inference and optimization, and (2) Optimizing experiments, processes, and designs using the resulting uncertain models. These problems are intractable with conventional methods, for several reasons: (1) The simulation problems that govern the inner loops of the optimization problems are expensive to execute (due to severe nonlinearity, heterogeneity, multiphysics/multiscale coupling); (2) The optimization variable and uncertain parameter spaces are high dimensional, often stemming from discretizations of infinite dimensional fields such as initial conditions, sources, or material properties. We argue that the key to overcoming these challenges is to develop new mathematical, computational, and statistical methods that exploit the structure of the Bayesian inference and optimization problems mediated by their underlying complex uncertain models. This structure includes the regularity, sparsity, geometry, low intrinsic dimensionality, and multifidelity nature of the maps from uncertain parameter/optimization variable spaces to the specific objectives targeted: Bayesian inference, optimal experimental design, and optimal control design. Black box methods developed as generic tools are incapable of exploiting this structure. To be successful, we must create, integrate, and cross-fertilize ideas across multiple areas of applied math--including approximation theory, Bayesian inference, data science, experimental design, information theory, machine learning, model reduction, optimal control theory, parallel algorithms, PDE-constrained optimization, randomized algorithms, stochastic optimization, and uncertainty quantification--all while exploiting the structure of the problems at hand. With this goal in mind, we have marshaled a team of leading authorities in these areas. While the methods we develop will be broadly applicable across a wide spectrum of DOE problems in which experiments inform models and the systems those models describe must be optimized under uncertainty, we have chosen a specific area, advanced manufacturing and materials, to drive our work. AMM is characterized by complex models across multiple scales, and is a rich source of challenging problems in inference, experimental design, and optimal control, requiring multifaceted and integrated advances in applied mathematics. As such, AMM serves as an excellent vehicle to motivate and demonstrate the advances in applied mathematics developed by our center.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

GenMod: A generative modeling approach for spectral representation of PDEs with random inputs

Here, we propose a method for quantifying uncertainty in high-dimensional PDE systems with random parameters, where the number of solution evaluations is small. Parametric PDE solutions are often approximated using a spectral decomposition based on polynomial chaos expansions. For the class of systems we consider (i.e., high dimensional with limited solution evaluations) the coefficients are given by an underdetermined linear system in a regression formulation. This implies additional assumptions, such as sparsity of the coefficient vector, are needed to approximate the solution. Here, we present an approach where we assume the coefficients are close to the range of a generative model that maps from a low to a high dimensional space of coefficients. Our approach is inspired be recent work examining how generative models can be used for compressed sensing in systems with random Gaussian measurement matrices. Using results from PDE theory on coefficient decay rates, we construct an explicit generative model that predicts the polynomial chaos coefficient magnitudes. The algorithm we developed to find the coefficients, which we call GenMod, is composed of two main steps. First, we predict the coefficient signs using Orthogonal Matching Pursuit. Then, we assume the coefficients are within a sparse deviation from the range of a sign-adjusted generative model. This allows us to find the coefficients by solving a nonconvex optimization problem, over the input space of the generative model and the space of sparse vectors. We obtain theoretical recovery results for a Lipschitz continuous generative model and for a more specific generative model, based on coefficient decay rate bounds. We examine three high-dimensional problems and show that, for all three examples, the generative model approach outperforms sparsity promoting methods at small sample sizes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Linear response of molecular polaritons

In this article, we show that the collective light–matter strong coupling regime where N molecular emitters couple to the photon mode of an optical cavity can be mapped to a quantum impurity model where the photon is the impurity that is coupled to a bath of anharmonic transitions. In the thermodynamic limit where N ≫ 1, we argue that the bath can be replaced with an effective harmonic bath, leading to a dramatic simplification of the problem into one of the coupled harmonic oscillators. We derive simple analytical expressions for linear optical spectra (transmission, reflection, and absorption) where the only molecular input required is the molecular linear susceptibility. This formalism is applied to a series of illustrative examples, showing the role of temperature, disorder, vibronic coupling, and optical saturation of the molecular ensemble, explaining that it is useful even when describing an important class of nonlinear optical experiments. For completeness, we provide Appendixes A–C that include a self-contained derivation of the relevant spectroscopic observables for arbitrary anharmonic systems (for both large and small N) within the rotating-wave approximation. While some of the presented results herein have already been reported in the literature, we provide a unified presentation of the results as well as new interpretations that connect powerful concepts in open quantum systems and linear response theory with molecular polaritonics.

Chemistry↗
DOE OSTI2026

Analytic Chromaticity Formulas for the Muon g -2 experiment at fermilab

The Muon g-2 Experiment (E989) at Fermilab measured the muon anomalous magnetic moment aμ with unprecedented precision of 127 ppb using a storage ring. The experiment pushed the limits in terms of accuracy and systematic errors. Achieving the 78-ppb total systematic uncertainty required precise beam dynamics modeling and corrections for effects on the measured muon precession frequency. Here, we derived the first analytic aberration formulas up to the second order for the muon g-2 storage ring’s combined-function electrostatic quadrupoles (superimposed magnetic dipole and electric quadrupole fields) using an order-by-order perturbation method. From these, we obtained the exact chromaticity formulas for three ring models of different granularity and validated them against numerical calculations using COSY INFINITY, achieving analytic-numerical agreement to ⁠$\mathscr{O}(10^{-10})$. This work resolved discrepancies between previous approximate derivations and provided essential beam dynamics results for Runs 4-6 analyses. We also calculated nonlinear chromaticities up to ninth order. The experiment completed its final Run 6 in July 2023, collecting 21 times more data than the previous muon g-2 experiment at Brookhaven National Laboratory, with the final result announced in June 2025.

aberrations↗
DOE OSTI2022

Operator inference for non-intrusive model reduction with quadratic manifolds

This paper proposes a novel approach for learning a data-driven quadratic manifold from high-dimensional data, then employing this quadratic manifold to derive efficient physics-based reduced-order models. The key ingredient of the approach is a polynomial mapping between high-dimensional states and a low-dimensional embedding. This mapping consists of two parts: a representation in a linear subspace (computed in this work using the proper orthogonal decomposition) and a quadratic component. The approach can be viewed as a form of data-driven closure modeling, since the quadratic component introduces directions into the approximation that lie in the orthogonal complement of the linear subspace, but without introducing any additional degrees of freedom to the low-dimensional representation. Combining the quadratic manifold approximation with the operator inference method for projection-based model reduction leads to a scalable non-intrusive approach for learning reduced-order models of dynamical systems. Further, applying the new approach to transport-dominated systems of partial differential equations illustrates the gains in efficiency that can be achieved over approximation in a linear subspace.

42 ENGINEERING↗
DOE OSTI2022

Manifold Learning-Based Polynomial Chaos Expansions for High-Dimensional Surrogate Models

In this work we introduce a manifold learning-based method for uncertainty quantification (UQ) in systems describing complex spatiotemporal processes. Our first objective is to identify the embedding of a set of high-dimensional data representing quantities of interest of the computational or analytical model. For this purpose, we employ Grassmannian diffusion maps, a two-step nonlinear dimension reduction technique which allows us to reduce the dimensionality of the data and identify meaningful geometric descriptions in a parsimonious and inexpensive manner. Polynomial chaos expansion is then used to construct a mapping between the stochastic input parameters and the diffusion coordinates of the reduced space. An adaptive clustering technique is proposed to identify an optimal number of clusters of points in the latent space. The similarity of points allows us to construct a number of geometric harmonic emulators which are finally utilized as a set of inexpensive pretrained models to perform an inverse map of realizations of latent features to the ambient space and thus perform accurate out-of-sample predictions. Thus, the proposed method acts as an encoder-decoder system which is able to automatically handle very high-dimensional data while simultaneously operating successfully in the small-data regime. The method is demonstrated on two benchmark problems and on a system of advection-diffusion-reaction equations which model a first-order chemical reaction between two species. In all test cases, the proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks.

42 ENGINEERING↗
DOE OSTI2021

Multi-fidelity Bayesian neural networks: Algorithms and applications

Here we propose a new class of Bayesian neural networks (BNNs) that can be trained using noisy data of variable fidelity, and we apply them to learn function approximations as well as to solve inverse problems based on partial differential equations (PDEs). These multi-fidelity BNNs consist of three neural networks: The first is a fully connected neural network, which is trained following the maximum a posteriori probability (MAP) method to fit the low-fidelity data; the second is a Bayesian neural network employed to capture the cross-correlation with uncertainty quantification between the low- and high-fidelity data; and the last one is the physics-informed neural network, which encodes the physical laws described by PDEs. For the training of the last two neural networks, we first employ the mean-field variational inference (VI) to maximize the evidence lower bound (ELBO) to obtain informative prior distributions for the hyperparameters in the BNNs, and subsequently we use the Hamiltonian Monte Carlo (HMC) method to estimate accurately the posterior distributions for the corresponding hyperparameters. We demonstrate the accuracy of the present method using synthetic data as well as real measurements. Specifically, we first approximate a one- and four-dimensional function, and then infer the reaction rates in one- and two-dimensional diffusion-reaction systems. Moreover, we infer the sea surface temperature (SST) in the Massachusetts and Cape Cod Bays using satellite images and in-situ measurements. Taken together, our results demonstrate that the present method can capture both linear and nonlinear correlation between the low- and high-fidelity data adaptively, identify unknown parameters in PDEs, and quantify uncertainties in predictions, given a few scattered noisy high-fidelity data. Finally, we demonstrate that we can effectively and efficiently reduce the uncertainties and hence enhance the prediction accuracy with an active learning approach, using as examples a specific one-dimensional function approximation and an inverse PDE problem.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Symbolic pregression: Discovering physical laws from distorted video

In this work, we present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled synthetic video (or, more generally, for discovering and modeling predictable features in time-series data). We first train an autoencoder that maps each video frame into a low-dimensional latent space where the laws of motion are as simple as possible, by minimizing a combination of nonlinearity, acceleration, and prediction error. Differential equations describing the motion are then discovered using Pareto-optimal symbolic regression. We find that our pre-regression (“pregression”) step is able to rediscover Cartesian coordinates of unlabeled moving objects even when the video is distorted by a generalized lens. Using intuition from multidimensional knot theory, we find that the pregression step is facilitated by first adding extra latent space dimensions to avoid topological problems during training and then removing these extra dimensions via principal component analysis. An inertial frame is autodiscovered by minimizing the combined equation complexity for multiple experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Taylor approximation variance reduction for approximation errors in PDE-constrained Bayesian inverse problems

In numerous applications, surrogate models are used as a replacement for accurate parameter-to-observable mappings when solving large-scale inverse problems governed by partial differential equations (PDEs). The surrogate model may be a computationally cheaper alternative to the accurate parameter-to-observable mappings and/or may ignore additional unknowns or sources of uncertainty. The Bayesian approximation error (BAE) approach provides a means to account for the induced uncertainties and approximation errors, i.e. the errors between the accurate parameter-to-observable mapping and the surrogate. The statistics of these errors are, however, in general unknown a priori, and are thus calculated using Monte Carlo sampling. Although the sampling is typically carried out offline, i.e. before considering the data, the process can still represent a computational bottleneck. In this work, we develop a scalable computational approach for reducing the costs associated with the sampling stage of the BAE approach. Specifically, we consider the Taylor expansion of the accurate and surrogate forward models with respect to the uncertain parameter fields either as a control variate for variance reduction or as a means to directly and efficiently approximate the mean and covariance of the approximation errors. We propose efficient methods for evaluating the expressions for the mean and covariance of the Taylor approximations based on linear(-ized) PDE solves. Furthermore, the proposed approach is independent of the dimension of the uncertain parameter, depending instead on the intrinsic dimension of the data, ensuring scalability to high-dimensional problems. The potential benefits of the proposed approach are demonstrated for two high-dimensional inverse problems governed by PDE examples, namely for the estimation of a distributed Robin boundary coefficient in a linear diffusion problem, and for a coefficient estimation problem governed by a nonlinear diffusion problem.

Bayesian approximation error↗