Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Normalizing flows for domain adaptation when identifying Λ hyperon events

Here this study focuses on the application of a normalizing flow as a method of domain adaptation when classifying physics data. Normalizing flows offer a way to transform data points between two different distributions. The present study investigates a novel method of transforming latent representations of physics data to a normal distribution and then to a physics distribution again. The final distribution models a simulated distribution. After being transformed, the data can be classified by a neural network trained on labeled simulation data. The present study succeeds in training two normalizing flows that can transform between data (or simulation) and a Gaussian distribution.

47 OTHER INSTRUMENTATION↗

EV Charging Infrastructure Energization An Overview of Approaches for Simplifying and Accelerating Timelines to Processing EV Charging Load Service Requests

The United States has seen significant growth in electric vehicle (EV) adoption, leading to increased demand for EV charging infrastructure. Over the past decade, EV charging infrastructure site developers, site hosts, and electric distribution utilities have navigated the process to integrate chargers onto the electric grid. Site developers and site hosts have raised the alarm that the integration process for high-powered EV charging projects does not meet the needs of the EV market for timeliness or cost. High-powered charging stations typically require a load service request or an agreement with the local utility to connect to the grid. The process of energizing a new high-powered charging site can be complex and time-consuming, often taking up to 2 years. This timeline is the result of current utility energization processes having been designed for construction projects that take longer to build (i.e., buildings). The specific challenges stem from various factors, including compartmentalization in application processes, the integration of EV charging process approvals with other distributed energy resources (DERs), and the need to ensure grid reliability. The energization process needs to evolve to meet the growing demand for high-powered EV charging. This white paper compiles information gathered through various conversations with key stakeholders, including utilities, utility regulators, EV charging operators, site developers, and authorities having jurisdiction (AHJ) as well as through an extensive literature review. This document identifies the challenges and provides potential solutions to streamline the process of connecting EV charging infrastructure to the power grid in the United States, serving as a starting point for future conversations around these solutions. The solutions noted in this white paper require collaborative efforts among utilities, regulators, and EV charging infrastructure developers to streamline the grid connection process for EV charging infrastructure. They are broadly organized into four areas: 1. Increase data access and transparency: Develop automated load service request tools, integrate hosting capacity and load service request analyses, incorporate EV adoption forecasts, and provide transparency on the processing queue. 2. Improve energization processes and timing: Create fast-track options based on prescreening criteria, provide flexibility or phased approvals in the load service request/interconnection process, build internal knowledge within utilities about EV charging technologies, and provide standardized workforce training. 3. Promote economic efficiency: Right size distribution components to accurately reflect the load requirements of EV charging infrastructure, make proactive investments in grid infrastructure based on EV adoption forecasts and growth projections, and consider energy equity and environmental justice factors such as equitable access to EV charging when planning infrastructure. 4. Improve grid reliability and resilience: Use load management/power control systems (PCS) at EV charging stations, adopt and implement harmonized standards for communication protocols and information models between the EV charging and grid control infrastructure, and address cybersecurity considerations by implementing robust security measures and standards for EV charging infrastructure—with particular emphasis on clarifying the security requirements for the interface to the grid. The objective of the solutions proposed in this white paper is to accelerate the timeline and decrease costs associated with connecting EV charging infrastructure to the grid. Electric utilities, utility regulators, EV charging infrastructure developers, and site hosts will first need to understand which solutions are available in their service territory, and if warranted, which combination of solutions would support their specific needs. Through the successful implementations of solutions at scale detailed here, industry will demonstrate a new and innovative ecosystem where timely deployment and energization of EV charging infrastructure with greater grid resiliency and reliability is a reality.

24 POWER TRANSMISSION AND DISTRIBUTION↗

On the Generalizability of Time-of-Flight Convolutional Neural Networks for Noninvasive Acoustic Measurements

Bulk wave acoustic time-of-flight (ToF) measurements in pipes and closed containers can be hindered by guided waves with similar arrival times propagating in the container wall, especially when a low excitation frequency is used to mitigate sound attenuation from the material. Convolutional neural networks (CNNs) have emerged as a new paradigm for obtaining accurate ToF in non-destructive evaluation (NDE) and have been demonstrated for such complicated conditions. However, the generalizability of ToF-CNNs has not been investigated. In this work, we analyze the generalizability of the ToF-CNN for broader applications, given limited training data. We first investigate the CNN performance with respect to training dataset size and different training data and test data parameters (container dimensions and material properties). Furthermore, we perform a series of tests to understand the distribution of data parameters that need to be incorporated in training for enhanced model generalizability. This is investigated by training the model on a set of small- and large-container datasets regardless of the test data. We observe that the quantity of data partitioned for training must be of a good representation of the entire sets and sufficient to span through the input space. The result of the network also shows that the learning model with the training data on small containers delivers a sufficiently stable result on different feature interactions compared to the learning model with the training data on large containers. To check the robustness of the model, we tested the trained model to predict the ToF of different sound speed mediums, which shows excellent accuracy. Furthermore, to mimic real experimental scenarios, data are augmented by adding noise. We envision that the proposed approach will extend the applications of CNNs for ToF prediction in a broader range.

47 OTHER INSTRUMENTATION↗

SDYN-GANs: Adversarial learning methods for multistep generative models for general order stochastic dynamics

We introduce adversarial learning methods for data-driven generative modeling of dynamics of nth-order stochastic systems. Our approach builds on Generative Adversarial Networks (GANs) with generative model classes based on stable m-step stochastic numerical integrators. From observations of trajectory samples, we introduce methods for learning long-time predictors and stable representations of the dynamics. Our approaches use discriminators based on Maximum Mean Discrepancy (MMD), training protocols using both conditional and marginal distributions, and methods for learning dynamic responses over different time-scales. We show how our approaches can be used for modeling physical systems to learn force-laws, damping coefficients, and noise-related parameters. Our adversarial learning approaches provide methods for obtaining stable generative models for dynamic tasks including long-time prediction and developing simulations for stochastic systems.

• Artificial intelligence (AI) / machine learning ↗

Multi-head physics-informed neural networks for learning functional priors and uncertainty quantification

In numerous applications, the integration of prior knowledge and historical information is essential, particularly for tasks requiring the solution of ordinary or partial differential equations (ODEs/PDEs) in data-sparse or noisy environments. For instance, achieving accurate solutions to time-dependent PDEs with limited initial condition measurements necessitates an effective strategy for embedding prior knowledge. Hard-parameter sharing architectures in neural networks (NNs) have demonstrated success in both traditional and scientific machine learning domains, facilitating the learning of informative representations. Here, in this study, we introduce a novel, yet efficient, method to enhance physics-informed neural networks (PINNs) by incorporating a multi-head structure that enables the learning of functional priors from both empirical data and governing physical laws. This prior information can then be used to address data sparsity and high-level noise in solving ODE/PDE problems with uncertainty quantification (UQ). The approach, termed Multi-Head PINN (MH-PINN), consists of a shared body NN and multiple head NNs, each corresponding to an individual PINN instance. Our framework for functional prior learning is carried out in two stages: (1) training the MH-PINNs to develop a shared body NN alongside multiple head NNs, and (2) employing these trained head NNs to estimate a prior distribution through a normalizing flow-based density estimator. The learned functional prior can then be applied as a regularization mechanism in deterministic contexts or as an informative prior within a Bayesian inference framework, aiding in the resolution of subsequent ODE/PDE tasks. We evaluate the efficacy of MH-PINNs across five benchmark problems, including a high-dimensional parametric PDE, all characterized by data sparsity or substantial noise levels. Our findings reveal that MH-PINNs deliver accurate solutions and robust UQ, demonstrating adaptability across a range of complex and challenging scenarios.

Bayesian inference↗

Optimisation of the Kaplan hydropower system via PID 2 and digital twin

Here, this paper proposes a proportional–integral-double–derivative (PID 2 ) optimisation method for the Kaplan hydropower system by building a digital twin. The study first uses one multilayer perceptron (MLP) to model the hydroturbine dynamic and then adopts three connected MLPs to model the generator dynamic, both in an open-loop fashion. Inspired by stochastic distribution control (SDC) theory, we regard the training of the turbine's neural network model as a process control problem, and we propose minimising entropy loss to update the network parameters. The next step is to build the digital twin by connecting the neural network models with a PID 2 controller and a lead-lag exciter and run the whole model in a closed-loop fashion. After that, a binary search approach is applied to optimise the PID 2 parameters based on the obtained digital twin model. The simulation results show that the proposed method can reduce the mean square tracking error by more than 90%. Furthermore, the method is extended to jointly optimise the PID 2 controller and excitation system gains through multiobjective optimisation, leveraging Pareto frontier analysis to balance active power and voltage tracking performance. Simulation results confirm the effectiveness of the proposed method, achieving a 83.46% reduction in relative mean square error of active power, a 47.13% reduction in terminal voltage tracking error, and an 82.78% improvement in the overall scalarized objective.

Hydropower system↗

Dynamical Sketching for Enhanced Communication Efficiency in Federated Learning

Federated learning (FL) has revolutionized distributed machine learning by enabling collaborative model training without sharing local data. However, communication efficiency and privacy guarantees remain significant challenges. This paper introduces a dynamic sketching mechanism in FL, optimizing the trade-off between communication efficiency and model accuracy. By dynamically selecting the sketch matrix size, our approach adapts to the evolving characteristics of the data and the model, ensuring optimal performance across diverse scenarios. We leverage Bayesian optimization to systematically tune the sketch parameters, achieving an effective balance between resource efficiency and model performance. Experimental results on the MNIST dataset using a convolutional neural network (CNN) architecture validate the proposed method's efficiency and scalability. Our dynamic sketching approach significantly outperforms fixed-size sketching techniques, achieving higher compression ratios (up to 62x) and providing better privacy guarantees while maintaining high model accuracy. These findings highlight the robustness and versatility of our approach and make it a valuable solution for privacy-preserving, communication-efficient federated learning.

Afrose, Sharmin [ORNL]↗

A Risk-Informed Approach to Trustworthiness Assessment in Digital Twins-Based Autonomous Control

In autonomous control systems, digital twins (DTs) are used to perform diagnostic and prognostic functions. The trustworthiness of these DTs is dependent on quality and coverage of the training data, model accuracy and integrity of sensor data. This work introduces a methodology to determine the trustworthiness of a DT system given faulty sensor data using a risk informed approach. Bayesian Belief Networks (BBNs) are used to propagate uncertainties and determine the probability of trustable recommendations. The decision to trust the control action provided by the DT is based on the DT output, expert opinion, and severity of problems. The performance of DTs is reliant on the data they are trained on. When they encounter out of distribution data, the trustworthiness of the recommendations decreases. To address this issue, we include an expert component that provides input on sensor degradation. For this, we utilize a generative artificial intelligence (AI) model, such as Generative Pretrained Transformer (GPT). The GPT functions as an expert with broad knowledge. The GPT is fine-tuned to understand and discriminate sensor degradation scenarios using manufactured data. This methodology is demonstrated through a case study on a Nearly Autonomous Management and Control System (NAMAC) during a steady state scenario. Various sensor degradation types with different severity levels are considered. Degraded sensor data is processed by the DT system and the fine-tuned GPT. Finally, using the BBN, we combine the GPT information and the DT output with its sources of uncertainty. This provides an output regarding the trustworthiness of the DT recommendation.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Temperature Field Reconstruction of Surfaces Heated Through Radiative Heat Transfer Using Convolutional Neural Networks

Microreactors could play a crucial role in decarbonizing our energy portfolio. However, their development and implementation come with specific challenges, particularly regarding cost. Due to their compact size and the harsh operational environment, collecting real-time data on reactor operation can be challenging. Many probe designs are unable to withstand extreme conditions (e.g., temperature, radiation) in the reactor. In this context, using convolutional neural networks (CNNs) can pave the way for developing a nonintrusive approach that relies solely on ex-core sensors. A well-trained physics-informed CNN can reconstruct the distribution of a given physical quantity over a domain using only a few sensors, allowing us to reconstruct the desired field distribution even in a limited space or complex geometries where a large array of sensors is impractical. In this work, we present the initial steps toward developing a real-time tool for monitoring the thermal behavior of nuclear reactor pressure vessels. Based on an experimental setup, a computational model using the Multiphysics Object-Oriented Simulation Environment (moose) framework was built, where the Ray Tracing and Heat Conduction modules were used to evaluate the temperature distribution over a convex metal surface heated through radiative heat transfer. This metal surface represents a section of a heated nuclear reactor vessel wall. The model also accounts for solid mechanics physics through the moose Solid Mechanics module. In situ experimental data, acquired from a Texas A&M facility, were used to validate the computational model. Part of the data generated by the moose model was used to train the convolutional neural network to reconstruct the vessel wall's outer surface temperature. The CNN generalization was then compared against the experimental and computational data.

Aldeia Machado, Luiz Carlos↗

Transferable predictions of energetic and structural properties for refractory solid solution alloys across chemical compositions

We present a data-efficient approach to train graph neural networks (GNNs) on density functional theory (DFT) data for accurate and transferable predictions of energetic and structural properties of refractory solid solution alloys in the niobium-tantalum-vanadium (Nb-Ta-V) chemical space. We start by training the GNN model only on DFT data that describes refractory binary alloys niobium-tantalum (Nb-Ta), niobium-vanadium (Nb-V), and tantalum-vanadium (Ta-V) to predict formation enthalpy and root mean squared displacement. Once trained, the GNN predictions are tested on DFT data describing refractory ternary alloys Nb-Ta-V. While, unsurprisingly, direct transferability from binary to ternary is not sufficiently accurate, augmenting the training with only 1% of the available ternary data (uniformly distributed across the entire range of chemical compositions) improves significantly the quality of the GNN predictions. For comparison, we assess the transferability in the opposite direction by training GNN models on ternary Nb-Ta-V data and making predictions on binaries Nb-Ta, Nb-V, and Ta-V, which exhibits notably higher predictive errors. The proposed methodology, which favors transferability from lower-component to higher-component alloys, offers an efficient path towards avoiding the curse of dimensionality incurred when collecting DFT data for discovery and design of multi-component disordered alloys.

Density functional theory calculations↗

Exploring the energy landscape of RBMs: reciprocal space insights into bosons, hierarchical learning and symmetry breaking

Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. In this work, we address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), a class of generative models known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation for RBMs, we reveal a connection between these models, diffusion processes, and systems of coupled bosons. Our analysis shows that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values of the weight matrix, whose distribution follows the Marc̆enko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau’s theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings not only bridge the gap between disparate generative frameworks but also shed light on the fundamental processes underpinning learning in deep generative models.

97 MATHEMATICS AND COMPUTING↗

Image-Based Fracture Surface Defect Characterization Methods for Additively Manufactured Ti-6Al-4V Tested in Fatigue

Abstract Fatigue initiation in additively manufactured samples/parts often occurs at processed-induced defects such as lack-of-fusion (LoF), keyhole, or other morphological/microstructural defects that have unique characteristics and measurable qualities. Attempts at identifying and minimizing such defects have utilized optimized processing conditions along with in situ and ex situ characterization that includes metallography and/or X-ray computed tomography (XCT). This paper highlights the benefits of using fracture surface analyses to detect and quantify defects that may not be detected by metallography/XCT due to sectioning and resolution limits. In addition to using manual quantification of fatigue initiating LoF and keyhole defects on fracture surfaces, image-based machine learning using convolutional neural networks such as U-Net were also used to automate the process. Statistical analyses were used to identify the extreme cases of defects that initiated and accelerated fatigue and to model the distribution of defect size and shape characteristics to distinguish the type of defect. Initial results show agreement between trained machine learning models and ground truth data in defect segmentation, and the distributions of defect characteristics are distinguishable to particular process-induced defect types.

Materials Science↗

Uncertainty quantification for neural network potential foundation models

Abstract For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

36 MATERIALS SCIENCE↗

O'Hare Airport roadway traffic prediction via data fusion and Gaussian process regression

This study proposes an approach of leveraging information gathered from multiple traffic data sources at different resolutions to obtain approximate inference on the traffic distribution of Chicago's O'Hare Airport area. Specifically, it proposes the ingestion of traffic datasets at different resolutions to build spatiotemporal models for predicting the distribution of traffic volume on the road network. Due to its good adaptability and flexibility for spatiotemporal data, the Gaussian process (GP) regression was employed to provide short-term forecasts using data collected by loop detectors (sensors) and supplemented by telematics data. The GP regression is used to make predictions of the distribution of the proportion of sensor data traffic volume represented by the telematics data for each location of the sensors. Consequently, the fitted GP model can be used to determine the approximate traffic distribution for a testing location outside of the training points. Policymakers in the transportation sector can find the results of this work helpful for making informed decisions relating to current and future transportation conditions in the area.

42 ENGINEERING↗

High-Fidelity Multiphysics Modeling of a Heat Pipe Microreactor Using BlueCrab

Researchers who are actively developing nuclear microreactors are planning to employ innovative designs and features using traditional commercial modeling tools that may be inadequate for their design and licensing activities. The codes developed under the U.S. Department of Energy Office of Nuclear Energy Advanced Modeling and Simulation (NEAMS) program provide flexibility in terms of geometry modeling and multiphysics coupling and are particularly well suited for modeling novel microreactor concepts. To test the maturity of these codes, this paper introduces a conceptual heat pipe microreactor (HP-MR) designed to gather various technologies of interest to microreactor developers such as control drums, heat pipes, and hydride moderators. Here, the objective of this effort is to demonstrate NEAMS tools capability to perform high-fidelity multiphysics simulations, using coupled neutronics (via the Griffin code), heat conduction (via the BISON code), heat pipe modeling (via the Sockeye code), and hydrogen redistribution in hydride metal moderator (via the SWIFT code). Codes are coupled in-memory through the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, which permits flexible multiphysics data transfer schemes. The analysis confirmed two key aspects of the HP-MR concept: (1) its ability to follow the power load requested from the heat pipe and (2) its ability to avoid heat pipe cascading failure unless designed with high power close to operating failure limits of its heat pipes. The developed computational model was distributed publicly on the Virtual Test Bed for training purposes to accelerate adoption by industry and to provide a high-fidelity multiphysics solution for benchmarking against other tools. Additional multiphysics analyses including other transients and coupled physics were identified as necessary future work, together with a focus on validating multiphysics behavior against experiments.

Microreactor↗

Beam loss modeling and mitigation due to intra-beam stripping

Intra-Beam Stripping (IBS) is a critical beam loss mechanism in high-intensity H- linacs and presents a significant limitation to increasing beam power. This work presents a computational framework to evaluate and mitigate IBS-induced beam loss along the Spallation Neutron Source (SNS) LINAC. Our calculation is based on an analytic theory and involves evaluation of a 9D integral using the Monte-Carlo technique. We first benchmarked our calculations against simplified, analytically solvable cases. We then applied our algorithm to Gaussian bunches with a known probability density function (PDF). We next expanded our algorithm to arbitrary bunch distributions using the Neural Spline Flow (NSF) models trained on PyORBIT tracking data. In the future, we plan to validate our algorithm experimentally and apply it to design IBS mitigation strategies.

Nln, Shivam [ORNL]↗

Grid-Ready Energy Analytics Training with Data (“GREAT with Data”)

GridEd is a collaborative educational initiative consisting of the Electric Power Research Institute (EPRI), 5 Partner Universities (Stony Brook University, The University of Texas at Austin, University of California – Riverside, Virginia Tech, Washington State University), and participating industry sponsors. This educational initiative focuses on developing and training the next generation of power engineers so they can help shape the electric grid of the future by anticipating and fulfilling the needs of changing electric industry requirements. GridEd is leveraging electric industry research to educate a future electric grid workforce by empowering new and continuing education students, not only to become competent and well-informed engineers, but also to participate and influence major technological, social, and policy decisions that address critical global challenges. GridEd’s activities are centered around four core pillars: Enhancement of university power systems engineering curricula; Professional development and training for a diverse electric industry workforce; Stimulating students to join the movement for the next generation of power engineers, and; Improve workforce development efforts in the electric utility industry. Major accomplishments over the course of the project were: Over 50 unique professional short courses were delivered by more than 30 instructors across the GridEd network; Over 3,600 unique learners, many who took multiple courses, received more than 27,000 professional development hours (PDH) and over 1,000 certificates of completion; Approximately 2,900 unique learners undertook a course that was offered LIVE online or in-person; Approximately 700 unique learners undertook a course that was offered as computer-based training (CBT); Over 35 unique university courses were delivered by more than 30 instructors to 1,500 university students across the GridEd network; One-hundred-and-fifty-seven (157) students were funded to completed 43 student projects in topics of power systems and data science, and; Six (6) Historically Black Colleges and Universities (HBCUs) recruited as Affiliate Universities via Utility Partners. The professional training initiative and workforce development activities launched by this project will be sustained through EPRI’s collaborative business model with industry. Stimulating students to join the power engineering workforce of the future and the enhancement of tertiary training may continue to need government support.

14 SOLAR ENERGY↗

Unsupervised domain adaptation for radioisotope identification in gamma spectroscopy

Training machine learning models for radioisotope identification using gamma spectroscopy remains an elusive challenge for many practical applications, largely stemming from the difficulty of acquiring and labeling large, diverse experimental datasets. Simulations can mitigate this challenge, but the accuracy of models trained on simulated data can deteriorate substantially when deployed to an out-of-distribution operational environment. In this study, we demonstrate that unsupervised domain adaptation (UDA) can improve the ability of a model trained on synthetic data to generalize to a new testing domain, provided unlabeled data from the target domain are available. Conventional supervised techniques are unable to utilize this data because the absence of isotope labels precludes defining a supervised classification loss. Instead, we first pretrain a spectral classifier using labeled synthetic data and subsequently leverage unlabeled target data to align the learned feature representations between the source and target domains. We compare a range of different UDA techniques, finding that minimizing the maximum mean discrepancy (MMD) between source and target feature vectors yields the most consistent improvement to testing scores. For instance, using a custom transformer-based neural network, we achieved a testing accuracy of $0.904 \pm 0.022$ on an experimental LaBr test set after performing unsupervised feature alignment via MMD minimization, compared to $0.754 \pm 0.014$ before alignment. Overall, our results highlight the potential of using UDA to adapt a radioisotope classifier trained on synthetic data for real-world deployment.

Lalor, Peter W.↗