Leveraging Approximation Theory for Efficient Scientific Machine Learning
Explore the source record for details and available documents.
Actor, Jonas [Sandia National Laboratories (SNL-NM↗
SEARCH · Search NASA
Results for “machine learning efficiency”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Leveraging Numerical Methods for Efficient Scientific Machine Learning
Explore the source record for details and available documents.
Actor, Jonas [Sandia National Laboratories (SNL-NM↗
A Single-Feature Machine Learning Method for Detecting Multiple Types of Events from PMU Data
This paper describes simple and efficient machine learning (ML) methods for efficiently detecting multiple types of power system events captured by PMUs scarcely placed in a large power grid. It uses a single feature from each PMU based on a rectangle area enclosing the event in a given data window. This single feature is sufficient to enable commonly used ML models to detect different types of events quickly and accurately. The feature is used by five ML models on four different data-window sizes. The results indicated a tradeoff between the execution speed and detection accuracy in variety of data-window size choices. The proposed method is insensitive to most data quality issues typical for data from field PMUs, and thus it does not require major data cleansing efforts prior to feature extraction.
Dokic, Tatjana↗
Machine Learning Using a Simple Feature for Detecting Multiple Types of Events From PMU Data
This paper describes simple and efficient machine learning (ML) methods for efficiently detecting multiple types of power system events captured by PMUs scarcely placed in a large power grid. It uses a single feature from each PMU based on a rectangle area enclosing the event in a given data window. This single feature is sufficient to enable commonly used ML models to detect different types of events quickly and accurately. The feature is used by five ML models on four different data-window sizes. The results indicated a tradeoff between the execution speed and detection accuracy in variety of data-window size choices. Here, the proposed method is insensitive to most data quality issues typical for data from field PMUs, and thus it does not require major data cleansing efforts prior to feature extraction.
Big data↗
Sparsified time-dependent Fourier neural operators for fusion simulations
This paper presents a sparsified Fourier neural operator for coupled time-dependent partial differential equations (ST-FNO) as an efficient machine learning surrogate for fluid and particle-based fusion codes such as NIMROD (Non-Ideal Magnetohydrodynamics with Rotation - Open Discussion) and GTC (Gyrokinetic Toroidal Code). ST-FNO leverages the structures in the governing equations and utilizes neural operators to represent Green's function-like numerical operators in the corresponding numerical solvers. Once trained, ST-FNO can rapidly and accurately predict dynamics in fusion devices compared with first-principle numerical algorithms. In general, ST-FNO represents an efficient and accurate machine learning surrogate for numerical simulators for multi-variable nonlinear time-dependent partial differential equations, with the proposed architectures and loss functions. The efficacy of ST-FNO has been demonstrated using quiescent H-mode simulation data from NIMROD and kink-mode simulation data from GTC. The ST-FNO H-mode results show orders of magnitude reduction in memory and central processing unit usage in comparison with the numerical solvers in NIMROD when computing fields over a selected poloidal plane. The ST-FNO kink-mode results achieve a factor of 2 reduction in the number of parameters compared to baseline FNO models without accuracy loss.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Indirect Tool Condition Monitoring Using Ensemble Machine Learning Techniques
Abstract Tool condition monitoring (TCM) has become a research area of interest due to its potential to significantly reduce manufacturing costs while increasing process visibility and efficiency. Machine learning (ML) is one analysis technique which has demonstrated advantages for TCM applications. However, the commonly studied individual ML models lack generalizability to new machining and environmental conditions, as well as robustness to the unbalanced datasets which are common in TCM. Ensemble ML models have demonstrated superior performance in other fields, but have only begun to be evaluated for TCM. As a result, it is not well understood how their TCM performance compares to that of individual models, or how homogeneous and heterogeneous ensemble models’ performances compare to one another. To fill in these research gaps, milling experiments were conducted using various cutting conditions, and the model groups were compared across several performance metrics. Statistical t-tests were also used to evaluate the significance of model performance differences. Through the analysis of four individual ML models and five ensemble models, all based on the processes’ sound, spindle power, and axial load signals, it was found that on average, the ensemble models performed better than the individual models, and that the homogeneous ensembles outperformed the heterogeneous ensembles.
Engineering↗
Perspectives on Polyolefin Catalysis in Microfluidics for High-Throughput Screening: A Minireview
Polyolefins are the largest produced plastics in the world which traditionally employ continuous stirred tank reactors and fluidized bed reactors for commercial production. The operating condition, reaction kinetics, and molecular interactions inside the reactor strongly affect the polyolefin properties, which require stringent process control in conventional procedures. Understanding the catalytic pathway, behavior of polymer particles and effect of reactor conditions are essential for designing specific polymer properties, namely the molecular weight, chain length, polydispersity, etc. Microfluidics can play a significant role in designing polymers tailored to the user needs. Smaller channel dimensions help obtain uniform reaction conditions over the length of the microfluidic reactor in a controlled environment. With real-time monitoring techniques in microfluidics, even single particle growth of polymer can be studied to understand the parameters affecting the polymer properties. High throughput microfluidics can help catalyst screening in a short duration with less consumption of reagents generating less waste. When supplemented with efficient machine learning algorithms, automated high throughput microfluidics has the potential to rapidly optimize the process and develop new knowledge even with a limited data set. When trained on data sets generated using microfluidic experiments that are designed efficiently with working knowledge of the process, machine learning algorithms can provide the relationship between the multivariable parameters space and polymer properties, which is not possible with the traditional statistical methods and interpolation techniques. Here, the rise in the utilization of microfluidics, with the advancement of machine learning algorithms, for polyolefin catalysis, highlights the importance of microfluidics for catalyst discovery, parameter optimization, and understanding reaction pathway for producing polymers with specific properties for specialized applications.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Machine learning predictions of diffusion in bulk and confined ionic liquids using simple descriptors
Ionic liquids have many intriguing properties and widespread applications such as separations and energy storage. However, ionic liquids are complex fluids and predicting their behavior is difficult, particularly in confined environments. We introduce fast and computationally efficient machine learning (ML) models that can predict diffusion coefficients and ionic conductivity of bulk and nanoconfined ionic liquids over a wide temperature range (350–500 K). The ML models are trained on molecular dynamics simulation data for 29 unique ionic liquids as bulk fluids and confined in graphite slit pores. This model is based on simple physical descriptors of the cations and anions such as molecular weight and surface area. Here, we also demonstrate that accurate results can be obtained using only descriptors derived from SMILES (simplified molecular-input line-entry system) codes for the ions with minimal computational effort. This offers a fast and efficient method for estimating diffusion and conductivity of nanoconfined ionic liquids at various temperatures without the need for expensive molecular dynamics simulations.
74 ATOMIC AND MOLECULAR PHYSICS↗
Machine Learning for Fairness-Aware Load Shedding: A Real-Time Solution via Identifying Binding Constraints: Preprint
Timely and effective load shedding in power systems is critical for maintaining supply-demand balance and preventing cascading blackouts. To eliminate load shedding bias against specific regions in the system, optimization-based methods are uniquely positioned to help balance between economic and fairness considerations. However, the resulting optimization problem involves complex constraints, which can be time-consuming to solve and thus cannot meet the real-time requirements of load shedding. To tackle this challenge, in this paper we present an efficient machine learning algorithm to enable millisecond-level computation for the optimization-based load shedding problem. Numerical studies on both a 3-bus toy example and a realistic RTS-GMLC system have demonstrated the validity and efficiency of the proposed algorithm for delivering fairness-aware and real-time load shedding decisions.
97 MATHEMATICS AND COMPUTING↗
Physics-Guided Continual Learning for Predicting Emerging Aqueous Organic Redox Flow Battery Material Performance
Aqueous organic redox flow batteries (AORFBs) have gained popularity in renewable energy storage due to their low cost, environmental friendliness and scalability. The rapid discovery of aqueous soluble organic (ASO) redox-active materials necessitates efficient machine learning surrogates for predicting battery performance. The physics-guided continual learning (PGCL) method proposed in this study can incrementally learn data from new ASO electrolytes while addressing catastrophic forgetting issues in conventional machine learning. Using a AORFB database with a thousand potential materials generated by a 780 $\text{cm}^2$ interdigitated cell model, PGCL incorporates AORFB physics to optimize the continual learning task formation and training strategies to retain previously learned battery material knowledge. Finally, the trained PGCL demonstrates its capability in assessing emerging ASO materials within the established parameter space when evaluated with the dihydroxyphenazine isomers.
25 ENERGY STORAGE↗
Micromechanical Surrogate Machine Learning Model for Creep Deformation Modeling
Process variability during the manufacture of gas turbine engine hot section components can significantly affect the material’s resulting microstructure. In casting, for instance, geometric variation within a component (thin sections versus thick sections, radial location) influences cooling rates and the resulting grain size. The high temperature creep response is known to be sensitive to grain size owing to a diffusional creep mechanism which occurs more readily along grain boundaries. Microstructural variation correspondingly drives mechanical behavior which propagates into component scale performance uncertainty. These factors are essential when planning inspection, maintenance, and repair strategies within a reliability framework. These benefits provide opportunities to increase overall energy efficiency through refined margins. Critically, there is an opportunity to bolster existing data-driven reliability models using physics-driven process-structure-property relations. Here we present recent work establishing a framework for evaluating the probabilistic creep performance of high-temperature materials. A novel microstructure-sensitive crystal plasticity finite element model is established that captures both grain boundary and crystallographic deformation effects. The computationally expensive physics model is calibrated using a statistical approach and this high-fidelity model is subsequently used to train a computationally efficient machine learning surrogate model. The surrogate model is essential for sampling a large ensemble of simulated structure-property pair results. The ensemble data are then mined to extract salient trends to be incorporated into a microstructure-sensitive reliability model. The proposed approach represents a novel way to capture microstructure-sensitive trends from physics-based models within a modern reliability framework.
Fernandez-Zelaia, Patxi [ORNL]↗
Estimating the Impacts of Increasing Temperatures and the Efficacy of Climate Adaptation Strategies in Urban Microclimates with Deep Learning
As urbanization and climate change progress, understanding and addressing urban heat becomes a priority for climate adaptation efforts. High temperatures concentrated in the urban core can drive increased risk of heat-related death and illness as well as increased energy demand for cooling. However, modeling the urban microclimate is an ongoing field of research typically burdened by an imprecise description of the built environment, incomplete observational records, significant computational cost, and a lack of high-resolution estimates of the impacts of increasing temperatures. Here, we present computationally efficient machine learning methods that can improve the accuracy of urban temperature estimates when compared to historical reanalysis data. These models are applied to a neighborhood in Los Angeles, and we compare the energy benefits of heat mitigation strategies to the impacts of climate change. We find that cooling demand is likely to increase substantially through midcentury, but engineered high-albedo surfaces could lessen this increase by more than 50 %. The corresponding increase in winter gas heating offsets the summer cooling benefit in the current climate, but total annual energy use from combined heating and cooling with electric heat pumps benefits from the engineered heat mitigation strategies under both current and future climates.
54 ENVIRONMENTAL SCIENCES↗
Ps and Qs: Quantization-Aware Pruning for Efficient Low Latency Neural Network Inference
Efficient machine learning implementations optimized for inference in hardware have wide-ranging benefits, depending on the application, from lower inference latency to higher data throughput and reduced energy consumption. Two popular techniques for reducing computation in neural networks are pruning, removing insignificant synapses, and quantization, reducing the precision of the calculations. In this work, we explore the interplay between pruning and quantization during the training of neural networks for ultra low latency applications targeting high energy physics use cases. Techniques developed for this study have potential applications across many other domains. We study various configurations of pruning during quantization-aware training, which we term quantization-aware pruning, and the effect of techniques like regularization, batch normalization, and different pruning schemes on performance, computational complexity, and information content metrics. We find that quantization-aware pruning yields more computationally efficient models than either pruning or quantization alone for our task. Further, quantization-aware pruning typically performs similar to or better in terms of computational efficiency compared to other neural architecture search techniques like Bayesian optimization. Surprisingly, while networks with different training configurations can have similar performance for the benchmark application, the information content in the network can vary significantly, affecting its generalizability.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Uniform-in-phase-space data selection with iterative normalizing flows
Improvements in computational and experimental capabilities are rapidly increasing the amount of scientific data that are routinely generated. In applications that are constrained by memory and computational intensity, excessively large datasets may hinder scientific discovery, making data reduction a critical component of data-driven methods. Datasets are growing in two directions: the number of data points and their dimensionality. Whereas dimension reduction typically aims at describing each data sample on lower-dimensional space, the focus here is on reducing the number of data points. A strategy is proposed to select data points such that they uniformly span the phase-space of the data. The algorithm proposed relies on estimating the probability map of the data and using it to construct an acceptance probability. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. Instead of binning the phase-space to estimate the probability map, its functional form is approximated with a normalizing flow. Therefore, the method naturally extends to high-dimensional datasets. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data are available.
97 MATHEMATICS AND COMPUTING↗
Understanding Adsorption and Reactions at Aqueous Oxide Interfaces with Neural Network Potential Molecular Dynamics
Chemical processes at metal oxide−water interfaces are of central importance in geochemistry, biology, and energy technologies. A better understanding of these processes would allow us to make a significant step toward optimizing and controlling them, which could in turn lead to broader impacts. Computational modeling is indispensable to accomplishing this task because complexity and disorder often make it difficult to extract atomistic information from experiments. Balancing computational cost and accuracy, simulation schemes based on efficient machine learning representations of the potential energy surface (PES) predicted by ab initio calculations have become increasingly popular over the past decade. In particular, several studies have demonstrated the ability of machine learning models to accurately reproduce the complex ab initio PESs of aqueous oxide interfaces, allowing simulations of systems and processes that are not accessible with ab initio methods. In this Account, we review our recent efforts to understand adsorption processes and reactions at aqueous oxide interfaces using deep potential molecular dynamics (DPMD), a simulation scheme employing deep neural networks (DNNs), which has proven to be quite successful in accurately describing many different systems in the condensed phase. After summarizing the DPMD methodology, we first review our work on the acid−base chemistry of oxide surfaces in contact with water, a fundamental characteristic that controls proton transfer and surface charge at the interface. We focus on the aqueous interface of rutile IrO 2 , an oxide material thus far considered the best catalyst for the oxygen evolution reaction (OER). We show that this interface is characterized by a large fraction of dissociated water and a strong Brønsted acidity of the surface sites, in good agreement with the experimentally measured value of the point of zero proton charge. In our second example, we investigate how the adsorption of organic species from ambient air or water affects the structure and wettability of the aqueous interfaces of TiO 2 , a prototypical photocatalytic material. This is a question that is relevant to understanding the UV-induced hydrophilicity of TiO 2 surfaces, a property at the basis of self-cleaning windows and related applications. Specifically focusing on formic and acetic acids, the two most common atmospheric organic acids, our simulations reveal that these acids control the wettability of TiO 2 largely through acid−base chemistry at the interface rather than chemisorption on the oxide surface, a finding that could help improve the design of self-cleaning surfaces and photocatalytic devices. Finally, we review our recent study of methanol at TiO 2 −water interfaces, a system whose interest is largely motivated by the role of methanol in enhancing photocatalytic hydrogen evolution on TiO 2 . Our simulations provide mechanistic insights into the coupled roles of the organic adsorbate and water at the TiO 2 interface, with implications for how methanol enhances the activity of H 2 evolution.
adsorption↗
Anthropogenic influence on extreme precipitation over global land areas seen in multiple observational datasets
The intensification of extreme precipitation under anthropogenic forcing is robustly projected by global climate models, but highly challenging to detect in the observational record. Large internal variability distorts this anthropogenic signal. Models produce diverse magnitudes of precipitation response to anthropogenic forcing, largely due to differing schemes for parameterizing subgrid-scale processes. Meanwhile, multiple global observational datasets of daily precipitation exist, developed using varying techniques and inhomogeneously sampled data in space and time. Previous attempts to detect human influence on extreme precipitation have not incorporated model uncertainty, and have been limited to specific regions and observational datasets. Using machine learning methods that can account for these uncertainties and capable of identifying the time evolution of the spatial patterns, we find a physically interpretable anthropogenic signal that is detectable in all global observational datasets. Machine learning efficiently generates multiple lines of evidence supporting detection of an anthropogenic signal in global extreme precipitation.
54 ENVIRONMENTAL SCIENCES↗
sup3ruhi (Super Resolution for Renewable Resource Data and Urban Heat Islands) [SWR-25-05]
Urban heat is a growing concern, particularly in dense metropolitan areas where high temperatures increase the risk of heat-related illness and drive energy expenses for cooling. Estimating the effects of urban heat remains a challenge due to limitations in describing the built environment, computational constraints, and the need for high-resolution data. This software presents open-source, computationally efficient machine learning methods that enhance the accuracy of urban temperature estimates compared to historical reanalysis data. Models trained using this software have been applied to urban microclimates in Los Angeles and Seattle showing greater accuracy and less bias when compared to low-resolution reanalysis datasets like ERA5 and even when compared to high-resolution mesoscale numerical weather models like WRF with an urban canopy model. Initial findings highlight how machine learning can support urban heat resilience planning by enabling improved assessments of local heat islands, mitigation strategies, and their energy implications. This software is an extension of (sup3r). This software supports the following publication: Buster, Grant, et al. Tackling Extreme Urban Heat: A Machine Learning Approach to Assess the Impacts of Climate Change and the Efficacy of Climate Adaptation Strategies in Urban Microclimates. arXiv:2411.05952, arXiv, 8 Nov. 2024. arXiv.org, https://doi.org/10.48550/arXiv.2411.05952. And has related public data records available at: Buster, Grant, Cox, Jordan, Benton, Brandon, and King, Ryan. Super-Resolution for Renewable Resource Data and Urban Heat Islands (Sup3rUHI). United States: N.p., 16 Oct, 2024. Web. https://data.openei.org/submissions/6220.
Buster, Grant [National Renewable Energy Laborator↗
MATSE: Multi-fidelity assisted time-series emulation
I am going to present my work on multifidelity timeseries models at MS&T in Pittsburgh. We develop efficient machine learning methodologies to accelerate time-series predictions from a hierarchy of complex physics-based models.
Katona, Ryan Michael↗