SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Guidelines for the design of multilayer neural networks for the identification of nonlinear mappings are considered. Since nonlinear mappings can be approximated by a one-hidden-layer neural network, an approach to determine the sufficient number of hidden layer nodes to achieve a global minima of the identification error function is considered.
Analysis of n-dimensional nonlinear mappings representative of P and S functions
This report describes a nonlinear mapping technique where the unknown static or dynamic system is approximated by a sum of dimensionally increasing functions (one-dimensional curves, two-dimensional surfaces, etc.). These lower dimensional functions are synthesized from a set of multi-resolution basis functions, where the resolutions specify the level of details at which the nonlinear system is approximated. The basis functions also cause the parameter estimation step to become linear. This feature is taken advantage of to derive a systematic procedure to determine and eliminate basis functions that are less significant for the particular system under identification. The number of unknown parameters that must be estimated is thus reduced and compact models obtained. The lower dimensional functions (identified curves and surfaces) permit a kind of "visualization" into the complexity of the nonlinearity itself.
This report describes a nonlinear mapping technique where the unknown static or dynamic system is approximated by a sum of dimensionally increasing functions (one-dimensional curves, two-dimensional surfaces, etc.). These lower dimensional functions are synthesized from a set of multi-resolution basis functions, where the resolutions specify the level of details at which the nonlinear system is approximated. The basis functions also cause the parameter estimation step to become linear. This feature is taken advantage of to derive a systematic procedure to determine and eliminate basis functions that are less significant for the particular system under identification. The number of unknown parameters that must be estimated is thus reduced and compact models obtained. The lower dimensional functions (identified curves and surfaces) permit a kind of "visualization" into the complexity of the nonlinearity itself.
Function classes of n-dimensional nonlinear mappings generalizing matrices, including convergence for Jacobi and Gauss-Seidel process
Classes of n dimensional functions nonlinear matrix mapping, emphasizing block-Jacobi and block-Gauss-Seidel process convergence for continuous surjective M functions
This paper addresses the issue of applying a globally convergent optimization algorithm to the training of multilayer perceptrons, a class of Artificial Neural Networks. The multilayer perceptrons are trained towards the solution of two highly nonlinear problems: (1) signal detection in a multi-user communication network, and (2) solving the inverse kinematics for a robotic manipulator. The research is motivated by the fact that a multilayer perceptron is theoretically capable of approximating any nonlinear function to within a specified accuracy. The algorithm that has been employed in this study combines the merits of two well known optimization algorithms, the Conjugate Gradients and the Trust Regions Algorithms. The performance is compared to a widely used algorithm, the Backpropagation Algorithm, that is basically a gradient-based algorithm, and hence, slow in converging. The performances of the two algorithms are compared with the convergence rate. Furthermore, in the case of the signal detection problem, performances are also benchmarked by the decision boundaries drawn as well as the probability of error obtained in either case.
Topological mappings for use in design of optimal nonlinear feedback control systems
In this article we consider two dynamical systems: the McMillan sextupole and octupole integrable mappings originally introduced by Edwin McMillan; the second one is also known as canonical McMillan map. Both of them are simplest symmetric McMillan maps with only one intrinsic parameter, the trace of the Jacobian at the fixed point. While these dynamical systems have numerous of applications and are used in many areas of math and physics, some of their dynamical properties have not been described yet. We fulfill the gap and provide complete description of all stable trajectories including parametrization of invariant curves, Pioncaré rotation numbers and canonical action-angle variables. In the second part we relate these maps with general chaotic map in McMillan-Turaev form. We show that McMillan sextupole and octupole mappings are first order approximations of dynamics around the fixed point, in a similar way as linear map and quadratic invariant (Courant-Snyder invariant in accelerator physics) is the zeroth order approximation (known as linearization). Finally we suggest the new formalism of nonlinear Twiss parameters which incorporate dependence of rotation number as a function of amplitude, in contrast to e.g. betatron phase advance used in accelerator physics which is independent of amplitude. Specifically in application to accelerator physics this new formalism is capable of predicting dynamical aperture around 1-st, 2-nd, 3-rd and 4-th order resonances for flat beams, which is critical for beam injection/extraction.
Reduced-order manifold combustion models are commonly used to lower the cost of reacting Large Eddy Simulations (LES) and may be derived either from physical principles as in flamelet models or from data-driven methods like Principal Component Analysis (PCA). In either case, neural networks are increasingly used as part of these models to provide a nonlinear mapping between a small set of pre-defined variables that parameterize the manifold and outputs of interest, such as reaction rates. In this work, we propose a new manifold-based modeling approach that combines the definition of the manifold-parameterizing variables (linear combinations of species), the nonlinear mapping to the outputs, and closure of filtered quantities for LES into the structure of a single neural network. This allows the process used to train the neural network to simultaneously optimize both the functional form of the model and the identities of the inputs to the model. The new approach can flexibly incorporate thermochemical data from any combustion system; if trained on data from 1D flames it can be interpreted as an optimized flamelet model, but it can also be used to learn models from data from more complex configurations. This work presents a priori evaluations of the new approach in both contexts. Evaluation using data from 1D premixed flames demonstrates the physical interpretability of the manifold variables generated by the new approach. Evaluation using data from direct numerical simulations of turbulent flames shows improved predictions relative to either flamelet or PCA-based models in a more complex configuration.
A theorem and its corollary are presented in which conditions are stated for a global transformation of a nonlinear system to a linear system. An explicit method for constructing such a transformation is given, and the important noncharacteristic matrix is introduced. Examples illustrating the technique are presented.
Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.
The applicability of ERS-1 wind and wave data for wave models was studied using the WAM third generation wave model and SEASAT altimeter, scatterometer and SAR data. A series of global wave hindcasts is made for the surface stress and surface wind fields by assimilation of scatterometer data for the full 96-day SEASAT and also for two wind field analyses for shorter periods by assimilation with the higher resolution ECMWF T63 model and by subjective analysis methods. It is found that wave models respond very sensitively to inconsistencies in wind field analyses and therefore provide a valuable data validation tool. Comparisons between SEASAT SAR image spectra and theoretical SAR spectra derived from the hindcast wave spectra by Monte Carlo simulations yield good overall agreement for 32 cases representing a wide variety of wave conditions. It is concluded that SAR wave imaging is sufficiently well understood to apply SAR image spectra with confidence for wave studies if supported by realistic wave models and theoretical computations of the strongly nonlinear mapping of the wave spectrum into the SAR image spectrum. A closed nonlinear integral expression for this spectral mapping relation is derived which avoids the inherent statistical errors of Monte Carlo computations and may prove to be more efficient numerically.
Here, this paper focuses on the development of multifidelity modeling approaches using neural network surrogates, where training data arising from multiple model forms and resolutions are integrated to predict high-fidelity response quantities of interest at lower cost. We focus on the context of quantum chemistry and the integration of information from multiple levels of theory. Important foundations include the use of symmetry function-based atomic energy vector constructions as feature vectors for representing structures across families of molecules and single-fidelity neural network training capabilities that learn the relationships needed to map feature vectors to potential energy predictions. These foundations are embedded within several multifidelity topologies that decompose the high-fidelity mapping into model-based components, including sequential formulations that admit a general nonlinear mapping across fidelities and discrepancy-based formulations that presume an additive decomposition. Methodologies are first explored and demonstrated on a pair of simple analytical test problems and then deployed for potential energy prediction for C 5 H 5 using B2PLYP-D3/6-311++G(d,p) for high-fidelity simulation data and Hartree–Fock 6-31G for low-fidelity data. For the common case of limited access to high-fidelity data, our computational results demonstrate that multifidelity neural network potential energy surface constructions achieve roughly an order of magnitude improvement, either in terms of test error reduction for equivalent total simulation cost or reduction in total cost for equivalent error.
We explore using neural operators, or neural network representations of nonlinear maps between function spaces, to accelerate infinite-dimensional Bayesian inverse problems (BIPs) with models governed by nonlinear parametric partial differential equations (PDEs). Neural operators have gained significant attention in recent years for their ability to approximate the parameter-to-solution maps defined by PDEs using as training data solutions of PDEs at a limited number of parameter samples. The computational cost of BIPs can be drastically reduced if the large number of PDE solves required for posterior characterization are replaced with evaluations of trained neural operators. However, reducing error in the resulting BIP solutions via reducing the approximation error of the neural operators in training can be challenging and unreliable. We provide an a priori error bound result that implies certain BIPs can be ill-conditioned to the approximation error of neural operators, thus leading to inaccessible accuracy requirements in training. To reliably deploy neural operators in BIPs, we consider a strategy for enhancing the performance of neural operators: correcting the prediction of a trained neural operator by solving a linear variational problem based on the PDE residual. We show that a trained neural operator with error correction can achieve a quadratic reduction of its approximation error, all while retaining substantial computational speedups of posterior sampling when models are governed by highly nonlinear PDEs. The strategy is applied to two numerical examples of BIPs based on a nonlinear reaction–diffusion problem and deformation of hyperelastic materials. We demonstrate that posterior representations of the two BIPs produced using trained neural operators are greatly and consistently enhanced by error correction.
This article extends the study of the dynamical properties of the symmetric McMillan map, emphasizing its utility in understanding and modeling complex nonlinear systems. Although the map features six parameters, we demonstrate that only two are irreducible: the linearized rotation number at the fixed point and a nonlinear parameter representing the ratio of terms in the biquadratic invariant. Through a detailed analysis, we classify regimes of stable motion, provide exact solutions to the mapping equations, and derive a canonical set of action-angle variables, offering analytical expressions for the rotation number and nonlinear tune shift. We further establish connections between general standard-form mappings and the symmetric McMillan map, using the area-preserving Hénon map and accelerator lattices with thin sextupole magnet as representative case studies. Our results show that, despite being a second-order approximation, the symmetric McMillan map provides a highly accurate depiction of dynamics across a wide range of system parameters, demonstrating its practical relevance in both theoretical and applied contexts.
Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. Here, we demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.
Cyber-Physical Systems (CPSs) like the power grid are critically important but also increasingly vulnerable; ensuring reliable system operation in the face of disruptions is becoming more and more challenging. Multi-Level Optimization (MLO) is a powerful way to model adversarial interactions, which naturally makes it applicable to studying CPS security. However, MLO typically does not address underlying system dynamics, and incorporating nonlinear dynamics is generally infeasible. In this paper, we show how to combine MLO with the Koopman Operator (KO) to remedy this. The KO maps nonlinear dynamics to a lifted space in which those dynamics are linear, thus making it ideal for use with MLO. Moreover, the structure of the KO also provides convenient ways to incorporate domain knowledge into the data-driven process of learning the KO representation of a given system. Here we then demonstrate the use of MLO-KO on a small example problem taken from the power grid domain, discuss the scalability and computational cost of MLO-KO, and identify future research directions for this work.