SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
How do we accelerate scientific progress in the Nuclear Data (ND) field? By selecting via Machine Learning (ML) an optimum combination of differential and integral experiments to reduce ND uncertainties.
Abstract Data-driven turbulence modeling is experiencing a surge in interest following algorithmic and hardware developments in the data sciences. We discuss an approach using the differentiable physics paradigm that combines known physics with machine learning to develop closure models for Burgers’ turbulence. We consider the one-dimensional Burgers system as a prototypical test problem for modeling the unresolved terms in advection-dominated turbulence problems. We train a series of models that incorporate varying degrees of physical assumptions on an a posteriori loss function to test the efficacy of models across a range of system parameters, including viscosity, time, and grid resolution. We find that constraining models with inductive biases in the form of partial differential equations that contain known physics or existing closure approaches produces highly data-efficient, accurate, and generalizable models, outperforming state-of-the-art baselines. Addition of structure in the form of physics information also brings a level of interpretability to the models, potentially offering a stepping stone to the future of closure modeling.
The reaction distribution in macro or device-scale has been studied for redox flow batteries. The reaction distribution on electrode pore-scale structure however is not well understood, lacking especially on how the reaction distribution on the pore-scale may impact the overall performance of a flow battery. This study introduces for the first time a framework of a multi-scale model that provides understanding of the relationship between the pore-scale electrode structure reaction and the device-scale electrochemical reaction uniformity within the flow battery. A reduced order model is constructed based on 128 pore-scale simulations, which provide a quantitative relationship between the battery operation conditions (inlet velocity, current density, inlet concentration) and the surface reaction uniformity for the pore-scale sample. The multi-scale framework upscales this pore-scale surface reaction uniformity to device-scale combined uniformity. Based on the multi-scale model, a time-varying optimization of the inlet velocity is established, leading to significant reduction on pump power consumption with targeted surface reaction uniformity. The multi-scale model establishes the critical link between the micro-structure of a flow battery component and its performance at the macro-scale, therefore providing rationale for further operational or material optimization.
This paper presents a sparsified Fourier neural operator for coupled time-dependent partial differential equations (ST-FNO) as an efficient machine learning surrogate for fluid and particle-based fusion codes such as NIMROD (Non-Ideal Magnetohydrodynamics with Rotation - Open Discussion) and GTC (Gyrokinetic Toroidal Code). ST-FNO leverages the structures in the governing equations and utilizes neural operators to represent Green's function-like numerical operators in the corresponding numerical solvers. Once trained, ST-FNO can rapidly and accurately predict dynamics in fusion devices compared with first-principle numerical algorithms. In general, ST-FNO represents an efficient and accurate machine learning surrogate for numerical simulators for multi-variable nonlinear time-dependent partial differential equations, with the proposed architectures and loss functions. The efficacy of ST-FNO has been demonstrated using quiescent H-mode simulation data from NIMROD and kink-mode simulation data from GTC. The ST-FNO H-mode results show orders of magnitude reduction in memory and central processing unit usage in comparison with the numerical solvers in NIMROD when computing fields over a selected poloidal plane. The ST-FNO kink-mode results achieve a factor of 2 reduction in the number of parameters compared to baseline FNO models without accuracy loss.
Abstract The geometry of 4-string contact interaction of closed string field theory is characterized using machine learning. We obtain Strebel quadratic differentials on 4-punctured spheres as a neural network by performing unsupervised learning with a custom-built loss function. This allows us to solve for local coordinates and compute their associated mapping radii numerically. We also train a neural network distinguishing vertex from Feynman region. As a check, 4-tachyon contact term in the tachyon potential is computed and a good agreement with the results in the literature is observed. We argue that our algorithm is manifestly independent of number of punctures and scaling it to characterize the geometry ofn-string contact interaction is feasible.
Machine learning enables unbinned, highly-differential cross section measurements. A recent idea uses generative models to morph a starting simulation into the unfolded data. We show how to extend two morphing techniques, Schrödinger Bridges and Direct Diffusion, in order to ensure that the models learn the correct conditional probabilities. This brings distribution mapping (DM) to a similar level of accuracy as the state-of-the-art conditional generative unfolding methods. Numerical results are presented with a standard benchmark dataset of single jet substructure as well as for a new dataset describing a 22-dimensional phase space of Z+2 -jets.
One of the major objectives of the experimental programs at the LHC is the discovery of new physics. This requires the identification of rare signals in immense backgrounds. Using machine learning algorithms greatly enhances our ability to achieve this objective. With the progress of quantum technologies, quantum machine learning could become a powerful tool for data analysis in high energy physics. In this study, using IBM gate-model quantum computing systems, we employ the quantum variational classifier method and the quantum kernel estimator method in two recent LHC flagship physics analyses: $t\bar{t}H$ (Higgs boson production in association with a top quark pair) and $H\rightarrow\mu\mu$ (Higgs boson decays to two muons). We have obtained early results with 10 qubits on the IBM quantum simulator and the IBM quantum hardware. On the quantum simulator, the quantum machine learning methods perform similarly to classical algorithms such as SVM (support vector machine) and BDT (boosted decision tree), which are often employed in LHC physics analyses. On the quantum hardware, the quantum machine learning methods have shown promising discrimination power, comparable to that on the quantum simulator. This study demonstrates that quantum machine learning has the ability to differentiate between signal and background in realistic physics datasets.
Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.
Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.
Protective vaccines and reliable diagnostics are essential tools for controlling viral diseases. However, the efficacy of these tools can be diminished by mutations in viral genomes. The delay between the emergence of new viral strains and the redesign of vaccines and diagnostics allows for continued viral transmission. Is it possible to address this challenge by computationally predicting viral genome sequence evolution? Can we “future-proof” vaccines and diagnostics by targeting both current and anticipated future sequence variants? While predicting viral evolution is still an unsolved, “grand challenge” problem in biology, the large, and rapidly growing, number of SARS-CoV-2 genome sequences provide an opportunity to quantify the ability of machine learning to predict viral genome sequence evolution. Towards this end, we have developed a simple computational model for predicting viral evolution at the level of individual nucleotides. The key metric for quantifying the per-base, prediction accuracy for viral evolution is the Mann-Whitney U statistic (or, equivalently, the area under the receiver operator curve). Since the Mann-Whitney U statistic is not a differentiable function, existing deep leaning packages (like Pytorch and Keras/TensorFlow) are not useful, as they require that the accuracy metric/objective function be analytically differentiable with respect to the model parameters. To overcome this challenge, we have implemented custom software, “FutureTense”, that can train a machine learning model by maximizing the non-differentiable Mann-Whitney U statistic. This software trains a machine learning model by exploring along the direction of the discrete gradient of the Mann-Whitney U statistic in the model parameter space. Parallel computing and genome sequence-specific optimizations are used to accelerate model training. The resulting machine learning model learns the observed high C->U mutation rates in the SARS-CoV-2 genome (which are potentially induced by host defenses) and provides prediction accuracies that are significantly better than one would expect from random chance. While predicting viral evolution is still quite far from a solved problem, the surprising performance of this simple model gives hope that the accuracy of predicting viral genome evolution can be further increased by more sophisticated approaches.
To convert lightning indices generated by numerical weather prediction experiments into binary lightning hazard, a machine-learning tool was developed. This tool, consisting of parallel multilayer perceptron classifiers, was trained on an ensemble of planetary boundary layer schemes and microphysics parameterizations that generated four different lightning indices over 1 week. In a subsequent week, the multi-physics ensemble was applied and the machine-learning tool was used to evaluate the accuracy. Unintuitively, the machine-learning tool performed better on the testing dataset than the training dataset. Much of the error may be attributed to mischaracterizing the convection. The combination of the machine learning model and simulations could not differentiate between cloud-to-cloud lightning and cloud-to-ground lightning, despite being trained on cloud-to-ground lightning. It was found that the simulation most representative of the local operational model was the most accurate simulation tested.
Solving fluid dynamics equations often requires the use of closure relations that account for missing microphysics. For example, when solving equations related to fluid dynamics for systems with a large Reynolds number, sub-grid effects become important and a turbulence closure is required, and in systems with a large Knudsen number, kinetic effects become important and a kinetic closure is required. By adding an equation governing the growth and transport of the quantity requiring the closure relation, it becomes possible to capture microphysics through the introduction of ‘hidden variables’ that are non-local in space and time. The behavior of the ‘hidden variables’ in response to the fluid conditions can be learned from a higher fidelity or ab-initio model that contains all the microphysics. In our study, a partial differential equation simulator that is end-to-end differentiable is used to train judiciously placed neural networks against ground-truth simulations. We show that this method enables an Euler equation based approach to reproduce non-linear, large Knudsen number plasma physics that can otherwise only be modeled using Boltzmann-like equation simulators such as Vlasov or particle-in-cell modeling.
In this work, we present Lift & Learn, a physics-informed method for learning low-dimensional models for large-scale dynamical systems. The method exploits knowledge of a system’s governing equations to identify a coordinate transformation in which the system dynamics have quadratic structure. This transformation is called a lifting map because it often adds auxiliary variables to the system state. The lifting map is applied to data obtained by evaluating a model for the original nonlinear system. This lifted data is projected onto its leading principal components, and low-dimensional linear and quadratic matrix operators are fit to the lifted reduced data using a least-squares operator inference procedure. Analysis of our method shows that the Lift & Learn models are able to capture the system physics in the lifted coordinates at least as accurately as traditional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numerical experiments on the FitzHugh–Nagumo neuron activation model and the compressible Euler equations demonstrate the generalizability of our model.
Operator learning techniques have recently emerged as a powerful tool for learning maps between infinite-dimensional Banach spaces. Trained under appropriate constraints, they can also be effective in learning the solution operator of partial differential equations (PDEs) in an entirely self-supervised manner. In this work we analyze the training dynamics of deep operator networks (DeepONets) through the lens of Neural Tangent Kernel theory, and reveal a bias that favors the approximation of functions with larger magnitudes. To correct this bias we propose to adaptively re-weight the importance of each training example, and demonstrate how this procedure can effectively balance the magnitude of back-propagated gradients during training via gradient descent. We also propose a novel network architecture that is more resilient to vanishing gradient pathologies. Taken together, our developments provide new insights into the training of DeepONets and consistently improve their predictive accuracy by a factor of 10-50x, demonstrated in the challenging setting of learning PDE solution operators in the absence of paired input-output observations.
Multigrid methods are one of the most efficient techniques for solving large sparse linear systems arising from partial differential equations (PDEs) and graph Laplacians from machine learning applications. One of the key components of multigrid is smoothing, which aims at reducing high-frequency errors on each grid level. However, finding optimal smoothing algorithms is problem-dependent and can impose challenges for many problems. In this paper, we propose an efficient adaptive framework for learning optimized smoothers from operator stencils in the form of convolutional neural networks (CNNs). Here, the CNNs are trained on small-scale problems from a given type of PDEs based on a supervised loss function derived from multigrid convergence theories and can be applied to large-scale problems of the same class of PDEs. Numerical results on anisotropic rotated Laplacian problems and variable coefficient diffusion problems demonstrate improved convergence rates and solution time compared with classical hand-crafted relaxation methods.
As part of the Cyclotron Road program, Dauntless.io, Inc. extended its existing ultrafast (0.00001 – 0.001 second) real-time, adaptive machine learning platform for modeling & control of complex, high-value physical systems. Objectives included ability to scale ubiquitously & remain differentiated on a 5+ year horizon in the highly competitive, AI / machine learning market — a rapidly growing market that (overall) is estimated to be worth $191 B by 2024 at a CAGR of 37%.
Deep neural networks (DNN) have been used to model nonlinear relations between physical quantities. Those DNNs are embedded in physical systems described by partial differential equations (PDE) and trained by minimizing a loss function that measures the discrepancy between predictions and observations in some chosen norm. This loss function often includes the PDE constraints as a penalty term when only sparse observations are available. As a result, the PDE is only satisfied approximately by the solution. However, the penalty term typically slows down the convergence of the optimizer for stiff problems. We present a new approach that trains the embedded DNNs while numerically satisfying the PDE constraints. We develop an algorithm that enables differentiating both explicit and implicit numerical solvers in reverse-mode automatic differentiation. This allows the gradients of the DNNs and the PDE solvers to be computed in a unified framework. We demonstrate that our approach enjoys faster convergence and better stability in relatively stiff problems compared to the penalty method. Furthermore, our approach allows for the potential to solve and accelerate a wide range of data-driven inverse modeling, where the physical constraints are described by PDEs and need to be satisfied accurately.
Gekko is an optimization suite in Python that solves optimization problems involving mixed-integer, nonlinear, and differential equations. The purpose of this study is to integrate common Machine Learning (ML) algorithms such as Gaussian Process Regression (GPR), support vector regression (SVR), and artificial neural network (ANN) models into Gekko to solve data based optimization problems. Uncertainty quantification (UQ) is used alongside ML for better decision making. These methods include ensemble methods, model-specific methods, conformal predictions, and the delta method. An optimization problem involving nuclear waste vitrification is presented to demonstrate the benefit of ML in this field. ML models are compared against the current partial quadratic mixture (PQM) model in an optimization problem in Gekko. GPR with conformal uncertainty was chosen as the best substitute model as it had a lower mean squared error of 0.0025 compared to 0.018 and more confidently predicted a higher waste loading of 37.5 wt% compared to 34 wt%. The example problem shows that these tools can be used in similar industry settings where easier use and better performance is needed over classical approaches. Future works with these tools include expanding them with other regression models and UQ methods, and exploration into other optimization problems or dynamic control.