Search NASA⌕ Search

SEARCH · Search NASA

Results for “successive training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Offshore Wind Workforce Safety Standards and Training Resource

The standardization of safety training requirements for people working at sea to build and operate offshore wind energy projects, adoption of those standards, and alignment with offshore safety training curriculum development was identified as a high-priority workforce gap in the U.S. Offshore Wind Workforce Assessment (Stefek, 2022). Safety is a top priority for the offshore wind industry; however, there has been uncertainty surrounding what unique safety standards and training will be needed and how various offshore wind safety stakeholders will be involved. The scope of safety training for this resource includes workers who build and operate offshore wind energy projects at sea and defines the stakeholders in the complex ecosystem. This fact sheet aims to help reduce these uncertainties by: 1) Delineating jurisdictional lines, while clarifying roles and responsibilities amongst stakeholders; 2) Identifying key actions or partnerships that can encourage further success with safety training and workforce development.

offshore wind↗

Industrial Assessment Center

Established in 1990, San Diego State University’s (SDSU) Industrial Assessment Center (IAC) is proud of its years of service. During this period, it has served over 620 small and medium-sized manufacturing plants in Southern California. SDSU/IAC’s efforts to transfer state-of-the-art technologies to industry have increased revenues, cultivated creativity, improved energy efficiencies, and benefited the environment. The Center has contributed to the region's economic growth and stability by assisting small and medium size companies to better compete in the global market. It has helped mitigate climate change by reducing greenhouse gas emissions. IAC activities have fostered productive relationships between the University and local industry, assisted industrial sectors to improve their energy efficiency and enhance their manufacturing productivity, in turn impacting the material and working conditions of their employees. In addition to financial savings and environmental benefits, we have trained tens of students who became energy specialists in various companies. Thus, a substantial benefit of the IAC has been the ongoing training of engineering faculty and students. All IAC graduates were offered jobs before or within weeks of their graduation. The activities of the SDSU/IAC have expanded the institutional expertise of the College and improved the knowledge base of the faculties involved leading to several related publications, master’s theses, and senior student projects. Significant number of peer-reviewed publications of the IAC director at SDSU have greatly benefitted from the experience of the Center. As a result of this extensive exposure to manufacturing processes, the SDSU/IAC has grown to be an integral component of SDSU’s engineering research and training. We have successfully built upon these established achievements and academic excellence. IAC service to industry is particularly vital in Southern California, a region with one of the highest manufacturing concentrations in the country. SDSU/IAC has understood and implemented the overall objectives of DOE’s IAC program and guidelines except for the pandemic years when the country’s manufacturing sector was put in dire stress. In addition to student training and service to industry, IAC’s contribution to state and local governments as well as utility companies to assess energy policies and design rebate and incentive strategies cannot be undermined.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Online evolutionary neural architecture search for multivariate non-stationary time series forecasting

Time series forecasting (TSF) is one of the most important tasks in data science. TSF models are usually pre-trained with historical data and then applied on future unseen datapoints. However, real-world time series data is usually non-stationary and models trained offline usually face problems from data drift. Models trained and designed in an offline fashion can not quickly adapt to changes quickly or be deployed in real-time. To address these issues, this work presents the Online NeuroEvolution-based Neural Architecture Search (ONE-NAS) algorithm, which is a novel neural architecture search method capable of automatically designing and dynamically training recurrent neural networks (RNNs) for online forecasting tasks. Without any pre-training, ONE-NAS utilizes populations of RNNs that are continuously updated with new network structures and weights in response to new multivariate input data. ONE-NAS is tested on real-world, large-scale multivariate wind turbine data as well as the univariate Dow Jones Industrial Average (DJIA) dataset. These results demonstrate that ONE-NAS outperforms traditional statistical time series forecasting methods, including online linear regression, fixed long short-term memory (LSTM) and gated recurrent unit (GRU) models trained online, as well as state-of-the-art, online ARIMA strategies. Additionally, results show that utilizing multiple populations of RNNs which are periodically repopulated provide significant performance improvements, allowing this online neural network architecture design and training to be successful.

97 MATHEMATICS AND COMPUTING↗

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗

A New Era of Discovery: The 2023 Long-Range Plan for Nuclear Science (V.1.2)

Nuclear science is the investigation of how protons and neutrons are formed from elementary particles and how the forces between those particles produce both nuclei and the vast variety of nuclear phenomena that occur in the universe. It has evolved into a broad field that addresses profound scientific questions: Where does the mass of visible matter come from? How do stars ignite, live, and die? How do nuclei illuminate the search for new laws of nature? This science points the way to using nuclei to build new technologies that benefit society. The 2015 Nobel Prize in physics was shared by nuclear physicists Art McDonald and Takaaki Kajita for the discovery of neutrino oscillations, which confirmed that neutrinos have mass. Our progress on big questions like this one since 2015 has been remarkable owing to new experimental tools, theoretical breakthroughs, powerful computational techniques, and the talented people who make these innovations possible. Focusing on these new tools, the Facility for Rare Isotope Beams (FRIB) at Michigan State University is already producing exciting results on decays of never-before-produced isotopes a year after it was completed on time and on budget. The energy upgrade of the Continuous Electron Beam Accelerator Facility (CEBAF) at the Thomas Jefferson National Accelerator Facility (Jefferson Lab) was also completed on schedule and on budget—new data from this facility are revealing the spectrum, structure, and dynamics of protons, neutrons, nuclei, and mesons. On the theory front, we can now calculate the distribution of quarks inside the proton from first principles. The implementation of artificial intelligence (AI) and machine learning (ML) techniques has led to improved data analysis and increased efficiency in running experiments and theoretical calculations. The impact of nuclear science goes beyond expanding the frontiers of knowledge about matter in the universe. We simultaneously develop a STEM work force that advances the security, technology, health, and wealth of our nation. Some connections are obvious. Expert scientists trained to work with radioactive nuclei are in demand in nuclear security arenas and are highly sought after by various government agencies and private industries. Graduate students and postdoctoral fellows (postdocs) obtain extensive computational, modeling, and data science skills that are similarly in high demand. Less obvious but equally important is the connection between these trained scientists and success in other professions, including medicine, energy, and entrepreneurial pursuits. The workforce that enables discovery in nuclear science also makes breakthroughs in technologies with tremendous impact on the nation’s economic advancement.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Anomaly Detection In DUNE Using AI/ML

We designed and built an AI/ML model to detect anomalies in the DUNE far detector data. The model has been trained on simulated radiological background (rbkg) data, which is the major background for supernova burst neutrinos. The trained model was evaluated on both new samples of radiological backgrounds and supernova burst neutrino events in the elastic scattering and charged current interaction channels. We found that the trained model can successfully identify supernova burst neutrino events as anomalies while identifying radiological backgrounds as nominal events.

Novello, Eric [Unlisted, US]↗

Neural Network Prediction of Strong Lensing Systems with Domain Adaptation and Uncertainty Quantification

Modeling strong gravitational lenses is computationally expensive for the complex data from modern and next-generation cosmic surveys. Deep learning has emerged as a promising approach for finding lenses and predicting lensing parameters, such as the Einstein radius. Mean-variance Estimators (MVEs) are a common approach for obtaining aleatoric (data) uncertainties from a neural network prediction. However, neural networks have not been demonstrated to perform well on out-of-domain target data successfully - e.g., when trained on simulated data and applied to real, observational data. In this work, we perform the first study of the efficacy of MVEs in combination with unsupervised domain adaptation (UDA) on strong lensing data. The source domain data is noiseless, and the target domain data has noise mimicking modern cosmology surveys. We find that adding UDA to MVE increases the accuracy on the target data by a factor of about two over an MVE model without UDA. Including UDA also permits much more well-calibrated aleatoric uncertainty predictions. Advancements in this approach may enable future applications of MVE models to real observational data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Finding Real Uncertainties From Physical Simulations

Modeling strong gravitational lenses is computationally expensive for the complex data from modern and next-generation cosmic surveys. Deep learning has emerged as a promising approach for finding lenses and predicting lensing parameters, such as the Einstein radius. Mean-variance Estimators (MVEs) are a common approach for obtaining aleatoric (data) uncertainties from a neural network prediction. However, neural networks have not been demonstrated to perform well on out-of-domain target data successfully - e.g., when trained on simulated data and applied to real, observational data. In this work, we perform the first study of the efficacy of MVEs in combination with unsupervised domain adaptation (UDA) on strong lensing data. The source domain data is noiseless, and the target domain data has noise mimicking modern cosmology surveys. We find that adding UDA to MVE increases the accuracy on the target data by a factor of about two over an MVE model without UDA. Including UDA also permits much more well-calibrated aleatoric uncertainty predictions. Advancements in this approach may enable future applications of MVE models to real observational data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Anomaly Detection in DUNE FD LArTPC Readouts

We designed and built an AI/ML model to detect anomalies in the DUNE far detector data. The model has been trained on simulated radiological background (rbkg) data, which is the major background for supernova burst neutrinos that we want to detect as anomaly in thios work. The trained model was evaluated on both new samples of radiological backgrounds and supernova burst neutrino events in the elastic scattering and charged current interaction channels. We found that the trained model can successfully identify supernova burst neutrino events as anomalies while identifying radiological backgrounds as nominal events.

Novello, Eric [Fermilab]↗

Quantum Computing Strategy 2026

Quantum computing (QC) is a rapidly maturing technology with the potential for revolutionary impacts on stockpile stewardship science and national security. Recent developments in fault-tolerant architectures have compressed vendor roadmaps, and predictions of a production-ready quantum computer by the mid-2030s are becoming increasingly credible. This strategy provides a roadmap for integrating QC into the Advanced Simulation and Computing (ASC) program by investing in four strategic focus areas: 1. Develop Capabilities in Mission-Relevant Quantum Applications: ASC will prioritize developing quantum-ready applications in mission areas that have shown significant promise for quantum advantage, including simulations of materials in extreme environments, nuclear dynamics, solving linear and nonlinear partial differential equations, and uncertainty quantification. These applications directly support stockpile stewardship science and modernization objectives. 2. Conduct R&D in Algorithms, Software, and Hardware: Sustained research into quantum algorithms, robust software tools, and quantum hardware is essential. ASC will develop efficient quantum algorithms; invest in quantum compilers, debuggers, and performance tools; and explore specialized quantum hardware tailored to NNSA’s unique requirements. 3. Engage with Vendors and Partners: Early and active collaboration with commercial quantum hardware vendors and academic partners is critical. Through testbeds, co-design agreements, and quantum demonstration facilities, ASC will influence hardware design, gain early access to emerging technologies, and ensure that quantum platforms evolve to meet mission needs. 4. Build Knowledge, Experience, and Workforce: Expanding and upskilling the quantum-trained workforce is essential to long-term success. This includes hiring, internal training, university outreach, and postdoctoral support to ensure ASC maintains the expertise required to operate, program, and integrate quantum systems as they become available. While quantum computing will never replace classical computing, it has the potential to solve certain problems with speed and accuracy that would be unachievable using any conceivable classical high-performance computing (HPC) system. By investing strategically in QC, ASC will help propel the emergent QC industry, maintain U.S. technological leadership, ensure mission readiness, and position itself to rapidly adopt quantum technologies as they mature.

97 MATHEMATICS AND COMPUTING↗

Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training

Distributed training increases the number of batches processed per iteration either by scaling-out (adding more nodes) or scaling-up (increasing the batch-size). However, the largest configuration does not necessarily yield the best performance. Horizontal scaling introduces additional communication overhead, while vertical scaling is constrained by computation cost and device memory limits. Thus, simply increasing the batch-size leads to diminishing returns: training time and cost decrease initially but eventually plateaus, creating a knee-point in the time/cost vs. batch-size pareto curve. The optimal batch-size therefore depends on the underlying model, data and available compute resources. Large batches also suffer from worse model quality due to the well-known “generalization gap”. In this paper, we present Tula, an online service that automatically optimizes time, cost, and convergence quality for large-batch training of convolutional models. It combines parallel-systems modeling with statistical performance prediction to identify the optimal batchsize. Tula predicts training time and cost within 7.5−14% error across multiple models, and achieves up to 20× overall speedup and improves test accuracy by ≈9% on average over standard large-batch training on various vision tasks, thus successfully mitigating the generalization gap and accelerating training at the same time.

Tyagi, Sahil [ORNL] (ORCID:0009000783144745)↗

The Importance of Being Adaptable: An Exploration of the Power and Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy Clusters

The application of deep machine learning methods in astronomy has exploded in the last decade, with new models showing remarkably improved performance on benchmark tasks. Not nearly enough attention is given to understanding the models' robustness, especially when the test data are systematically different from the training data, or "out of domain." Domain shift poses a significant challenge for simulation-based inference, where models are trained on simulated data but applied to real observational data. In this paper, we explore domain shift and test domain adaptation methods for a specific scientific case: simulation-based inference for estimating galaxy cluster masses from X-ray profiles. We build datasets to mimic simulation-based inference: a training set from the Magneticum simulation, a scatter-augmented training set to capture uncertainties in scaling relations, and a test set derived from the IllustrisTNG simulation. We demonstrate that the Test Set is out of domain in subtle ways that would be difficult to detect without careful analysis. We apply three deep learning methods: a standard neural network (NN), a neural network trained on the scatter-augmented input catalogs, and a Deep Reconstruction-Regression Network (DRRN), a semi-supervised deep model engineered to address domain shift. Although the NN improves results by 17% in the Training Data, it performs 40% worse on the out-of-domain Test Set. Surprisingly, the Scatter-Augmented Neural Network (SANN) performs similarly. While the DRRN is successful in mapping the training and Test Data onto the same latent space, it consistently underperforms compared to a straightforward Yx scaling relation. These results serve as a warning that simulation-based inference must be handled with extreme care, as subtle differences between training simulations and observational data can lead to unforeseen biases creeping into the results.

Ntampaka, Michelle [Baltimore, Space Telescope Sci↗

Throughput-Oriented and Accuracy-Aware DNN Training with BFloat16 on GPU

Deep Neural Networks (DNNs) have transformed the field of artificial intelligence and achieved extraordinary success in many areas. The training of DNNs is commonly compute and memory-intensive, which has resulted in several optimizations in the training phase. Among them, reduced precision is a typical and widely used technique to accelerate DNN training and reduce memory requirements. However, applying a widely adopted reduced precision format such as Float16 to all involved operations in DNN training is not optimal as the use of Float16 in some operations can hurt model accuracy. Meanwhile, additional optimizations including loss scaling and autocast techniques can mitigate the accuracy loss but lead to inherent overhead and inadequate use of reduced precision. In this work, we leverage another reduced precision format, BFloat16, and introduce a throughput-oriented and accuracy-aware approach to maximize the performance potential of DNN training. Since the high throughput provided by BFloat16 format is accompanied by low precision of the floating-point representation, this approach achieves high throughput by using BFloat16 on all DNN operations and avoids the accuracy loss through a customized accuracy-aware normalization. Results show that our approach outperforms the state-of-the-art mixed-precision training by 1.21x on an NVIDIA A100 GPU.

Xie, Zhen↗

Fine-tuning machine-learned particle-flow reconstruction for new detector geometries in future colliders

We demonstrate transfer learning capabilities in a machine-learned algorithm trained for particle-flow reconstruction in high energy particle colliders. This paper presents a cross-detector fine-tuning study, where we initially pretrain the model on a large full simulation dataset from one detector design, and subsequently fine-tune the model on a sample with a different collider and detector design. Specifically, we use the Compact Linear Collider detector (CLICdet) model for the initial training set and demonstrate successful knowledge transfer to the CLIC-like detector (CLD) proposed for the Future Circular Collider in electron-positron mode. We show that with an order of magnitude less samples from the second dataset, we can achieve the same performance as a costly training from scratch, across particle-level and event-level performance metrics, including jet and missing transverse momentum resolution. Furthermore, we find that the fine-tuned model achieves comparable performance to the traditional rule-based particle-flow approach on event-level metrics after training on 100,000 CLD events, whereas a model trained from scratch requires at least 1 million CLD events to achieve similar reconstruction performance. To our knowledge, this represents the first full-simulation cross-detector transfer learning study for particle-flow reconstruction. These findings offer valuable insights towards building large foundation models that can be fine-tuned across different detector designs and geometries, helping to accelerate the development cycle for new detectors and opening the door to rapid detector design and optimization using machine learning.

43 PARTICLE ACCELERATORS↗

Multilabel proportion prediction and out-of-distribution detection on gamma spectra of short-lived fission products

In the machine learning problem of multilabel classification, the objective is to determine for each test instance which classes the instance belongs to. In this work, we consider an extension of multilabel classification, called multilabel proportion prediction, in the context of radioisotope identification (RIID) using gamma spectra data. We aim to not only predict radioisotope proportions, but also identify out-of-distribution (OOD) spectra. We achieve this goal by viewing gamma spectra as discrete probability distributions, and based on this perspective, we develop a custom semi-supervised loss function that combines a traditional supervised loss with an unsupervised reconstruction error function. Our approach was motivated by its application to the analysis of short-lived fission products from spent nuclear fuel. In particular, we demonstrate that a neural network model trained with our loss function can successfully predict the relative proportions of 37 radioisotopes simultaneously. The model trained with synthetic data was then applied to measurements taken by Pacific Northwest National Laboratory (PNNL) to conduct analysis typically done by subject-matter experts. Here, we also extend our approach to successfully identify when measurements are OOD, and thus should not be trusted, whether due to the presence of a novel source or novel proportions.

Anomaly detection↗

Efficient online quantum circuit learning with no upfront training

Optimization is a promising candidate for studying the utility of variational quantum algorithms (VQAs). However, evaluating cost functions using quantum hardware introduces runtime overheads that limit exploration. Surrogate-based methods can reduce calls to a quantum computer, yet existing approaches require hyperparameter pre-training and have been tested only on small problems. Here, we show that surrogate-based methods can enable successful optimization at scale, without pre-training, by using radial basis function interpolation (RBF) to construct an adaptive, hyperparameter-free surrogate. Using the surrogate as an acquisition function drives hardware queries to the vicinity of the true optima. For 16-qubit random 3-regular Max-Cut instances with the Quantum Approximate Optimization Algorithm (QAOA), our method outperforms state-of-the-art approaches, without considering their upfront training costs. Furthermore, we successfully optimize QAOA circuits for 127-qubit random Ising models on an IBM processor using 10 4 −10 5 measurements. Strong empirical performance demonstrates the promise of automated surrogate-based learning for large-scale VQA applications.

97 MATHEMATICS AND COMPUTING↗

Catalytic Resonance Theory: Forecasting the Flow of Programmable Catalytic Loops

Chemical transformations on catalyst surfaces occur through series and parallel reaction pathways. These complex networks and their behavior can be most simply evaluated through a three-species surface reaction loop (A* to B* to C* to A*) that is internal to the overall chemical reaction. Application of an oscillating dynamic catalyst to this reactive loop has been shown to exhibit one of three types of behavior: (1) a positive net flux of molecules about the loop in the clockwise direction, (2) a negative net flux of molecules about the loop in the counterclockwise direction, or (3) negligible flux of molecules about the loop at the limit cycle of reaction. Three-species surface loops were simulated with microkinetic modeling to assess the reaction loop behavior resulting from a catalytic surface oscillating between two or more catalyst surface energy states. Selected input parameters for the simulations spanned an 11-dimensional parameter space using 127 688 different parameter combinations. Their converged limit cycle solutions were analyzed for their loop turnover frequencies, the majority of which were found to be approximately zero. Classification and regression machine learning models were trained to predict the sign and magnitude of the loop turnover frequency and successfully performed above accessible baselines. Notably, the classification models exhibited a baseline weighted F1 score of 0.49, whereas trained models achieved weighted F1 scores of 0.94 and 0.96 when trained on the parameters used to define the simulations and derived rate constants, respectively. The trained models successfully predicted catalytic loop behavior, and interpretation of these models revealed all input parameters to be important for the prediction and performance of each model.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Demonstration of μCHP in Light Commercial Hot Water Applications

Internal combustion engine driven combined heat and power (CHP) systems produce power while the waste heat is recovered and used for another purpose. For µCHP systems (less than 50kWe output) overall efficiencies can reach over 90% for products currently on the market throughout the world. On a fuel basis this offers an advantage over using grid-based power and another fuel for heat. Additionally, the cost of natural gas on an energy unit basis is significantly less expensive than electric power. This difference can lead to substantial savings for the end user of a CHP system. In some parts of the world, particularly Germany and Japan, µCHP has become a common part of the heat and distributed generation equipment mix. Market analysis conducted in 2012 showed that in North America there is potential of a 5,300 units per year market for µCHP, resulting in an annual energy savings of approximately 13.5 TBtu and more than $84 million in annual customer savings. However, µCHP still has not seen much market penetration in North America. This study helps to identify some of the reasons why, address roadblocks that still need to be overcome and provide experience to established best practices. Nine commercial sites were initially identified (fitness club, surgical center, senior community, resort and spa, commercial laundry, two multi-family housing, restaurant and manufacturing facility). Systems from two different manufacturers were installed at five sites total, but due to unforeseen circumstances, only three of those sites were finally commissioned: test data were collected by Oak Ridge National Laboratory. While not all systems were installed as intended, the project resulted in the following key findings: Micro combined heat and power is an unfamiliar technology for building owners and plumbing and electrical contractors that serve the buildings most suited for µCHP. Manufacturers will need to be aggressive in providing training and providing broad support to successfully bring the technology to market.; The proper integration of µCHP into other building systems is the most critical part of a successful installation. Manufacturers will need to be able to provide the expertise to train, advise and support the mechanical contractors during the design and installation of the heat integration system.; Regulations governing µCHP varies across the country. There are a few UL standards (UL 2200 and UL 1741) that apply nationally. Emissions are governed by EPA except in California where the California Air Resources Board (CARB) has established their own standard. Requirements that dictate installation and interconnection are dependent on local building codes and local utility processes and requirements. Some out of date building codes do not offer clear guidance for these requirements, leading to confusion. This makes developing training and determining broadly applicable best practices difficult.; Installation costs of µCHP are a key factor in making an installation successful. It is critical that these costs and all national, state and local regulations be understood and accounted for by all parties involved in the installation before a project is undertaken.; A robust customer service organization is important to meet the needs of the customers. These systems require annual maintenance and service and local capability is needed to keep costs inline.

03 NATURAL GAS↗