Search NASA⌕ Search

SEARCH · Search NASA

Results for “ML Surrogate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Machine Learned Empirical Numerical Integrator from Simulated Data

Recently, a number of state-of-the-art surrogate machine learning (ML) models have been designed for global weather and climate prediction, which have been trained using reanalysis data products. Reanalysis data products are constructed using numerical model simulations that combine numerical integration of partial differential equations and parameterization schemes. These products are typically only archived and made available using coarsened spatial and temporal resolutions. This study explores the impact of the numerical generation methods used to produce the training datasets and the temporal resolution of those datasets on machine learning surrogate models. Using the nonlinear vector autoregression (NVAR) machine as an explainable ML technique, simple dynamical systems are emulated with ML models trained on data produced by three classical numerical integration schemes. NVAR is validated as a skillful ML method, capable of producing accurate predictions and, more importantly, reconstructing both the underlying dynamics and the numerical integration scheme used to generate the training data. However, the machine fails to generalize predictions on unseen test data generated by different numerical integration schemes, despite the underlying dynamical system being the same. This result provides a word of caution for the growing field of machine learning emulation of weather and climate dynamics. Furthermore, we illustrate using NVAR that training on temporally coarsened data may increase the required complexity of ML models and potentially introduce new numerical challenges. Finally, we discover that empirical integration schemes with arbitrary time-stepping sizes can be constructed directly from the data, which implies a potential for the development of empirical numerical integration schemes.

54 ENVIRONMENTAL SCIENCES↗

Optimization of the generator coordinate method with machine-learning techniques for nuclear spectra and neutrinoless double- β decay: Ridge regression for nuclei with axial deformation

The generator coordinate method (GCM) is an important tool of choice for modeling large-amplitude collective motion in atomic nuclei. The computational complexity of the GCM increases rapidly with the number of collective coordinates. It imposes a strong restriction on the applicability of the method. In this work, we propose a subspace-reduction algorithm that employs optimal statistical ML models as surrogates for exact quantum-number projection calculations for norm and Hamiltonian kernels. The model space of the original GCM is reduced to a subspace relevant for nuclear low energy spectra and the NME of ground state to ground state 0νββ decay based on the orthogonality condition (OC) and the energy-transition-orthogonality procedure (ENTROP), respectively. For simplicity, the polynomial ridge regression (RR) algorithm is used to learn the norm and Hamiltonian kernels of axially deformed configurations. The efficiency and accuracy of this algorithm are illustrated for 76 Ge and 76 Se by comparing results obtained using the optimal RR models to direct GCM calculations. The low-lying energy spectra of 76 Ge and 76 Se, as well as the 0νββ-decay NME between their ground states, are computed. Furthermore, the results show that the performance of the GCM+OC/ENTROP+RR is more robust than that of the GCM+RR alone, and the former can reproduce the results of the original GCM calculation accurately with a significantly reduced computational cost.

59 ≤ A ≤ 89↗

Surrogate Modelling of 3rd Integer Resonant Extraction at Fermilab Delivery Ring

We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This effort is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment s production target, wherein the goal is to extract a uniform slice of the circulating $1e12$ protons in the DR over 25,000 turns (43~ms). The DR contains 3 harmonic sextupoles which excite a third-integer resonance as well as three fast, tune-ramping quadrupole magnets which drive the horizontal tune towards the $29/3$ resonance. In our initial work the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the environment in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The control agent will be hosted on a dedicated Arria 10 FPGA, introducing its own requirements on control agent architecture. In this work we report the accuracy and fidelity of surrogate models in comparison to the response dynamics of the physics simulator.

Narayanan, Aakaash [Fermilab] (ORCID:0000000157944↗

A Fortran–Python interface for integrating machine learning parameterization into earth system models

Abstract. Parameterizations in earth system models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation, and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran–Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and scikit-learn. We demonstrate the interface's modularity and reusability through two cases: an ML trigger function for convection parameterization and an ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES↗

A Fortran-Python Interface for Integrating Machine Learning Parameterization into Earth System Models

Parameterizations in Earth System Models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran-Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and Scikit-learn. We demonstrate the interface's modularity and reusability through two cases: a ML trigger function for convection parameterization and a ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES↗

Application of an automated machine learning-genetic algorithm (AutoML-GA) coupled with computational fluid dynamics simulations for rapid engine design optimization

In recent years, the use of machine learning-based surrogate models for computational fluid dynamics (CFD) simulations has emerged as a promising technique for reducing the computational cost associated with engine design optimization. However, such methods still suffer from drawbacks. One main disadvantage is that the default machine learning (ML) hyperparameters are often severely suboptimal for a given problem. This has often been addressed by manually trying out different hyperparameter settings, but this solution is ineffective in case of a high-dimensional hyperparameter space. Besides this problem, the amount of data needed for training is also not known a priori. In response to these issues that need to be addressed, the present work describes and validates an automated active learning approach, AutoML-GA, for surrogate-based optimization of internal combustion engines. In this approach, a Bayesian optimization technique is used to find the best machine learning hyperparameters based on an initial dataset obtained from a small number of CFD simulations. Subsequently, a genetic algorithm is employed to locate the design optimum on the ML surrogate surface. In the vicinity of the design optimum, the solution is refined by repeatedly running CFD simulations at the projected optima and adding the newly obtained data to the training dataset. It is demonstrated that AutoML-GA leads to a better optimum with a lower number of CFD simulations, compared to the use of default hyperparameters. The proposed framework offers the advantage of being a more hands-off approach that can be readily utilized by researchers and engineers in industry who do not have extensive machine learning expertise.

Owoyele, Opeoluwa↗

Optimization of artificial viscosity in production codes based on Gaussian Regression surrogate models

To accurately model flows with shock waves using staggered-grid Lagrangian hydrodynamics, artificial viscosity has to be introduced to convert kinetic energy into internal energy, thereby increasing the entropy across shocks. Determining the appropriate strength of the artificial viscosity is an art and strongly depends on the particular problem and experience of the researcher. The objective of this study is to pose the problem of finding the appropriate strength of artificial viscosity as an optimization problem and solve this problem using machine learning (ML) tools, specifically using surrogate models based on Gaussian Process regression and Bayesian analysis. We describe the optimization method and discuss various practical details of its implementation. The shock-containing problems for which we apply this method all have been implemented in the LANL code FLAG. First, we apply ML to find optimal values to isolated shock problems of different strengths. Second, we apply ML to optimize viscosity for a 1D propagating detonation problem based on Zel’dovich-von Neumann-Doring (ZND) detonation theory using a reactive burn model. We compare results for default (currently used values in FLAG) and optimized values of artificial viscosity for these problems demonstrating the potential for significant improvement in the accuracy of computations.

42 ENGINEERING↗

A Robust Machine Learning Schema for Developing, Maintaining, and Disseminating Machine Learning Models

Recent advances in the development of machine learning (ML) algorithms have enabled the creation of predictive models that can improve decision making, decrease computational cost, and improve efficiency in a variety of fields. As an organization begins to develop and implement such models, the data used in the training, validation, and testing of ML models, the model parameters, and the use cases or limitations of the models must be properly stored to ensure models are both fully traceable and used correctly. In the context of predicting material behavior, advances in computationally intense, physics-based modeling of material behavior at various length scales and the emergence of Integrated Computational Materials Engineering (ICME) have driven the need for developing data-driven surrogate models of the physics-based simulation tools using ML techniques. Surrogate model development allows for accurate material behavior prediction at a fraction of the cost of its physics-based counterpart, allowing for multiscale simulations of real-world applications, further enabling the ability to design fit-for-purpose materials for a reasonable computational investment. However, training such models requires extensive data, and thus, effective data management is necessary to reach the full potential that ML can offer to material design and ICME. This paper proposes a generalized, robust schema that allows organizations to store both real (experimental) and virtual (simulation) data used to train ML models and the defining model parameters and architectures within the Granta MI Platform. The developed schema allows for various types of data inputs and outputs, including single point values, time-series data, and images that can be used in the prediction of material behavior, while following outlined best practices for effective data management. An effective schema for ML data and models can help prevent the recreation of virtual/real training data and surrogate models, help reduce the time to create new models similar to existing ones by offering a starting point in the hyperparameter determination stages, minimize resources devoted to verification and validation (V&V) and certification of models, and ensure that data and surrogate models are not misused due to full traceability of both the data and ML model. It also allows organizations access to models that have already been developed, such that they can be used in the design of new materials, enabling the overall goals of ICME.

Brandon L. Hearley↗

A combined first principles study of the structural, magnetic, and phonon properties of monolayer CrI3

The first magnetic 2D material discovered, monolayer (ML) CrI3, is particularly fascinating due to its ground state ferromagnetism. However, because ML materials are difficult to probe experimentally, much remains unresolved about ML CrI3’s structural, electronic, and magnetic properties. Here, we leverage Density Functional Theory (DFT) and high-accuracy Diffusion Monte Carlo (DMC) simulations to predict lattice parameters, magnetic moments, and spin–phonon and spin–lattice coupling of ML CrI3. We exploit a recently developed surrogate Hessian DMC line search technique to determine CrI3’s ML geometry with DMC accuracy, yielding lattice parameters in good agreement with recently published STM measurements—an accomplishment given the ~10% variability in previous DFT-derived estimates depending upon the functional. Strikingly, we find that previous DFT predictions of ML CrI3’s magnetic spin moments are correct on average across a unit cell but miss critical local spatial fluctuations in the spin density revealed by more accurate DMC. DMC predicts that magnetic moments in ML CrI3 are 3.62 μB per chromium and -0.145 μB per iodine, both larger than previous DFT predictions. The large disparate moments together with the large spin–orbit coupling of CrI3’s I-p orbital suggest a ligand superexchange-dominated magnetic anisotropy in ML CrI3, corroborating recent observations of magnons in its 2D limit. We also find that ML CrI3 exhibits a substantial spin–phonon coupling of ~3.32 cm-1. Our work, thus, establishes many of ML CrI3’s key properties, while also continuing to demonstrate the pivotal role that DMC can assume in the study of magnetic and other 2D materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Predicting Fiber Failure of Plain Weave Fabric with Recursive Multiscale Micromechanics

Recent advances in the development of machine learning (ML) algorithms have enabled the creation of predictive models that can improve decision making, decrease computational cost, and improve efficiency in a variety of fields. As an organization begins to develop and implement such models, the data used in the training, validation, and testing of machine learning models, the model parameters, and the use cases or limitations of the models must be properly stored to ensure models are both fully traceable and used correctly. In the context of predicting material behavior, advances in computationally intense, physics-based, modeling of material behavior at various length scales, and the emergence of Integrated Computational Materials Engineering (ICME) have driven the need for developing data-driven surrogate models of the physics-based simulation tools using machine learning (ML) techniques. Surrogate model development allows for accurate material behavior prediction at a fraction of the cost of its physics-based counterpart, allowing for multiscale simulations of real-world applications, further enabling the ability to design fit-for-purpose materials for a reasonable computational investment. However, training such models requires extensive data, and thus effective data management is necessary to reach the full potential that ML can offer to material design and ICME. This paper proposes a generalized, robust schema that allows organizations to store both real (experimental) and virtual (simulation) data used to train machine learning models and the defining model parameters and architectures. The developed schema allows for various types of data inputs and outputs, including single point values, time-series data, and images that can be used in for various types of machine learning models while following outlined best practices for effective data management. An effective schema for machine learning data and models can help prevent the recreation of virtual/real training data and surrogate models, can help reduce the time to create new models similar to existing ones by offering a starting point in the hyperparameter determination stages, minimize resources devoted to verification and validation (V&V) and certification of models, and ensure that data and surrogate models are not misused due to full traceability of both the data and ML model. It also allows organizations access to models that have already been developed, such that they can be used in the design of new materials, enabling the overall goals of ICME.

Failure↗

Development and assessment of a reactor system prognosis model with physics-guided machine learning

Autonomous control systems provide recommendations to help operators in decision-making during plant operations ranging from normal operation to accident management. An important step of autonomous control is prognosis. In nuclear engineering domain, prognosis is the process of predicting future conditions of a system or equipment based on present signs and symptoms of a fault. The prognosis model allows predicting future reactor states for possible candidate control strategies so that the outcomes can be evaluated to determine the best control strategy. The prognosis model requires representing direct relationships between the symptoms and the predictions. In nuclear engineering, computational simulations are approximate representations of the operation of the real system. However, prognosis with computational simulations requires high computation power and time due to possible large number of scenarios. Necessary computation resources can be reduced with machine learning (ML) approach for fast predictions by building a surrogate function using the simulation data. A critical issue is, ML models are ignorant of physical knowledge, and these models approximate statistical relationships between the system variables. This ignorance can produce results that are inconsistent with physical laws, even if an optimal result is achieved from a mathematical point of view. Physics-guided machine learning (PGML) is an approach to tackle this issue. Here, this work formulates and illustrates a framework to guide development and assessment of the ML-based prognosis model for autonomous control systems. The development of the prognosis model considers the training of a ML model which consists of optimizing many aspects of the ML approach. The assessment of the prognosis model considers training data limitations and uncertainties of the ML approach. Prognosis models with standalone ML and PGML are developed and assessed on the loss-of-flow scenario of Experimental Breeder Reactor II. The results indicate that PGML based prognosis model has the best performance compared to other prognosis models.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Model-based, in-situ, non-destructive qualification and certification of parts made by autonomous additive manufacturing

To address the significant productivity challenges associated with the qualification and certification (Q&C) tasks of additively manufactured (AM) parts, which have traditionally relied on rigorous post‐build inspection and testing, we propose an integrated framework that combines model‐based qualification and certification (MBQ&C) with autonomous additive manufacturing (AAM). MBQ&C employs high‐fidelity predictive models, developed within the Integrated Computational Materials Engineering (ICME) paradigm, to simulate process–structure–property–performance relationships for assessing a part’s fitness for use. Since predictive models are commonly machine learning (ML)-based or reduced-order surrogates of validated physics models, they run efficiently, enabling timely inference. In parallel, the self-driving AAM utilises ML-based adaptive, closed‐loop control strategies to avoid, mitigate, or repair defects and anomalies during fabrication, thereby increasing the likelihood of producing acceptable parts. A key feature of the combined AAM-MBQ&C framework is that predictive models explicitly incorporate defects or anomalies that persist after the build, using instance-specific data captured via in-situ sensing. This customisation enables a build‐specific assessment of fitness for use, rather than relying on nominal or generic parameters. Such individualised evaluation provides a robust basis for Q&C-related acceptance decisions relating to each build. Additionally, the rapid solution capabilities of ML or reduced-order models enable the determination of a part’s suitability for service shortly after build completion. As the framework matures, it has the potential to substantially reduce reliance on conventional point‐design approaches—such as time‐consuming post‐build computed tomography scanning and costly destructive testing. Thus, the AAM-MBQ&C framework represents a transformative, scalable strategy for quality assurance of AM components, as parts produced within a stable, validated, and certified envelope can be certified with reduced testing. Key benefits include: (1) significant gains in Q&C productivity through efficient, model-centric assessment; (2) performance-based classification of defects into critical and non-critical categories; (3) the ability to predict potential deviations in the performance of parts affected by real-time, adaptive process control interventions relative to those produced under a certified process, and (4) the enabling of virtual Q&C for service environments that are difficult, hazardous, or impractical to access or reproduce experimentally. Collectively, these capabilities strengthen the business case for AM, particularly for high‐consequence and mission‐critical applications. Finally, although this work focuses on powder-based AM, the proposed techniques could be extended to AM processes employing alternative feedstock forms.

Gunasegaram, Dayalan↗

Explainability and extrapolation of machine learning models for predicting the glass transition temperature of polymers

Abstract Machine learning (ML) offers promising tools to develop surrogate models for polymers' structure–property relations. Surrogate models can be built upon existing polymer data and are useful for rapidly predicting the properties of unknown polymers. The accuracy of such ML models appears to depend on the feature space representation of polymers, the range of training data, and learning algorithms. Here, we establish connections between these factors for predicting the glass transition temperature (T g ) of polymers. Our analysis suggests linear models with fewer fitting parameters are as accurate as nonlinear models with many hidden and unexplainable parameters. Also, the performance of a monomer topology‐based ML model is found to be qualitatively identical to that of a physicochemical descriptor‐based ML model. We find that the ML models's performance in the extrapolative region is enhanced as the property range of the training data increases. Moreover, we establish newT g – polymer chemistry correlations via ML. Our work illustrates how ML can advance the fundamental understanding of polymer structure–property correlations and its efficacy for extrapolation problems.

Polymer Science↗

Uncertainty propagation and sensitivity analysis for constrained optimization of nuclear waste vitrification

Abstract The vitrification of high‐level waste (HLW) by heating a mixture of glass‐forming chemicals (GFCs) with the waste can be improved using a constrained optimization problem. This study explores how different uncertainty propagation (UP) methods implemented with the optimization process can affect the glass formulation of nuclear waste glasses. UP is the effort of propagating uncertain inputs through a system to understand and quantify output distributions. Uncertainty intervals are crafted from output distributions to inform the optimization algorithm. UP is often implemented with Monte Carlo (MC) sampling for large nonlinear systems, which can be difficult to implement within a constrained optimization algorithm that requires derivative information. Other UP methods often used for optimization under uncertainty (OUU) can be designed to work within an established constrained optimization framework. Methods of UP are evaluated in this study including iterative sampling approaches, first‐order approximations, and surrogate modeling with machine learning (ML). A method of dimensional reduction based on global sensitivity analysis is introduced to support the UP methods for the large dimensionality of the problem. Analytical UP methods able to achieve similar optimums 10 times faster than the baseline MC approach, and produce 93.9% similar output distributions are reported.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

MULTI-LEADER: MULTI-source LEarning-Accelerated Design of high-Efficiency multi-stage compRessor (Final Technical Report)

The objective of MULTI-LEADER is to cut design costs by 80% while generating more energy-efficient designs of multi-stage compressors by developing and implementing novel machine learning (ML) techniques, which enable faster and fewer design iterations, improved solver performance, and concurrent multi-disciplinary design. Current industrial practices for the design of multi-stage compressors involve simulation-based design optimization with successive levels of model fidelity, iteratively evaluated between distinct disciplines, one stage at a time to tackle the high dimensional design variations. This project addresses these key design challenges: (1) concurrent optimization of multiple stages under many non-linear constraints; (2) multitude of evaluation of high-fidelity and expensive solvers and their gradients during optimization convergence in high-dimensional design; (3) multi-disciplinary design to maximize aerodynamic performance while guaranteeing structural integrity and additive manufacturability; (4) utilization of multiple fidelity of solvers with disparate parameterization and modeling assumptions. MULTI-LEADER achieved more than 5x speed up in detailed design of more energy-efficient compressors via these machine learning (ML) innovations: (i) rapid design surrogates by multi-source learning from diverse fidelities across multiple disciplines, (ii) physics-constrained data-augmented modeling for improved empiricism, (iii) generative manifold embedding for high dimensional concurrent design without gradient information; (iv) budget-constrained fidelity-adaptive sampling towards fewer design iterations.

33 ADVANCED PROPULSION SYSTEMS↗

Data-driven learning of nonlocal physics from high-fidelity synthetic data

A key challenge to nonlocal models is the analytical complexity of deriving them from first principles, and frequently their use is justified a posteriori. Here, we extract nonlocal models from data, circumventing these challenges and providing data-driven justification for the resulting model form. Extracting data-driven surrogates is a major challenge for machine learning (ML) approaches, due to nonlinearities and lack of convexity — it is particularly challenging to extract surrogates which are provably well-posed and numerically stable. Our scheme not only yields a convex optimization problem, but also allows extraction of nonlocal models whose kernels may be partially negative while maintaining well-posedness even in small-data regimes. To achieve this, based on established nonlocal theory, we embed in our algorithm sufficient conditions on the non-positive part of the kernel that guarantee well-posedness of the learnt operator. These conditions are imposed as inequality constraints to meet the requisite conditions of the nonlocal theory. We demonstrate this workflow for a range of applications, including reproduction of manufactured nonlocal kernels; numerical homogenization of Darcy flow associated with a heterogeneous periodic microstructure; nonlocal approximation to high-order local transport phenomena; and approximation of globally supported fractional diffusion operators by truncated kernels.

42 ENGINEERING↗

ML-based Dimension Reduction Strategies

Deep learning (DL)--based surrogate models have achieved success in various applications in carbon capture and storage (CCS). However, the model training on high-dimensional spaces is computationally expensive and impractical for large-scale and complex geological models, because the models usually contain hundreds of thousands to millions of grid cells, each with a set of parameters. Furthermore, the high cost of generating training data with sufficient variation is another limitation of model training on high-dimensional spaces, which may result in overfitting and reduce the model efficiency and prediction performance. We proposed the workflow incorporating dimension reduction methods and deep learning models, which aim to extract the latent variables of input parameters and output state variables, and then build the mapping function at the latent spaces. The proposed workflow can significantly reduce the computational complexity in solving both forward and inverse problems compared to models trained on high-dimensional spaces. Dimensionality reduction models showed great potential in workflows for fast reservoir simulation, history matching, prior model generation, visualization, and more, ultimately enhancing DL model performance in related SMART Work Packages.

Hosseini, Seyyed↗

MFiX Development Updates

This presentation discusses recent developments of the Multiphase Flow with Interphase eXchanges (MFiX) software. A brief review of all modeling approaches along with their cost versus accuracy is provided to guide users when selecting a model for a given application. The main new features of the past five official releases of MFiX are described. Improvement in chemistry management allow for easier simulation setup, and faster simulation speed. Progress in the implementation of a thin-wall boundary condition is presented. The major new model development over the past year is the release of the Glued Sphere Particle model (GSP), where component spheres are combined together to represent non-spherical particles. The integration of Machine Learning (ML) workflow in the CFD process is discussed with two applications: a surrogate model for stiff chemistry and the development of a PIC stress model using ML.

Dietiker, Jeff↗