Search NASA⌕ Search

SEARCH · Search NASA

Results for “Training Principles”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing

Energy efficiency of training and inferencing with large neural network models is a critical challenge facing the future of sustainable large-scale machine learning workloads. This paper introduces an alternative strategy, called phantom parallelism, to minimize the net energy consumption of traditional tensor (model) parallelism, the most energy-inefficient component of large neural network training. The approach is presented in the context of feed-forward network architectures as a preliminary, but comprehensive, proof-of-principle study of the proposed methodology. We derive new forward and backward propagation operators for phantom parallelism, implement them as custom autograd operations within an end-to-end phantom parallel training pipeline and compare its parallel performance and energy-efficiency against those of conventional tensor parallel training pipelines. Formal analyses that predict lower bandwidth and FLOP counts are presented with supporting empirical results on up to 256 GPUs that corroborate these gains. Experiments are shown to deliver ∼50% reduction in the energy consumed to train FFNs using the proposed phantom parallel approach when compared with conventional tensor parallel methods. Additionally, the proposed approach is shown to train smaller phantom models to the same model loss on smaller GPU counts as larger tensor parallel models on larger GPU counts offering the possibility for even greater energy savings.

Seal, Sudip [ORNL] (ORCID:0000000332330656)↗
DOE OSTI2022

Training neural networks using Metropolis Monte Carlo and an adaptive variant

Abstract We examine the zero-temperature Metropolis Monte Carlo (MC) algorithm as a tool for training a neural network by minimizing a loss function. We find that, as expected on theoretical grounds and shown empirically by other authors, Metropolis MC can train a neural net with an accuracy comparable to that of gradient descent (GD), if not necessarily as quickly. The Metropolis algorithm does not fail automatically when the number of parameters of a neural network is large. It can fail when a neural network’s structure or neuron activations are strongly heterogenous, and we introduce an adaptive Monte Carlo algorithm (aMC) to overcome these limitations. The intrinsic stochasticity and numerical stability of the MC method allow aMC to train deep neural networks and recurrent neural networks in which the gradient is too small or too large to allow training by GD. MC methods offer a complement to gradient-based methods for training neural networks, allowing access to a distinct set of network architectures and principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Structural Health Monitoring of Epoxy Bond between Dissimilar Materials

Structures composed of adhesively bonded materials face many challenges when subjected to thermal loading, as adherend material(s) behave differently and can induce stresses in the glue bond. Current methods to test bond strength are limited to destructive tests, which increase costs to produce adhesively bonded structures. Strain gages are proposed to monitor the expansion profile of adhesive bonds as structures undergo thermal loading. Because resulting signals are DC in nature, no spectral quantities can be observed. Principle component analysis (PCA) is proposed to differentiate between damaged and undamaged structures. Training data are taken for various damage states and the minimum Mahalanobis distance is calculated to categorize testing data into damage state. The results indicate the potential for a strain-gage based monitoring system with PCA based decision-making, with further research necessary prior to implementation of the system.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Cluster-Graph Fingerprinting: A Framework for Quantitative Analysis of Machine-Learned Interatomic Model Training and Simulation Data

Machine-learned interatomic models represent a significant advancement in simulation methods, extending the predictive ability of first-principles methods to previously inaccessible length and time scales. However, the data-driven nature of these models can lead to difficult-to-detect errors that can compromise prediction accuracy. To address this challenge, we introduce a novel fingerprinting approach based on the Chebyshev Interaction Model for Efficient Simulation (ChIMES) ML-IAM graph-based descriptor. Our strategy enables efficient and statistically rigorous analysis of system configurations used in ML-IAM training and those generated by their application, e.g., in molecular dynamics simulations. We demonstrate that these fingerprints can effectively assess novelty of a configuration relative to an existing data set and determine dissimilarity among individual configurations, which are two key tasks in workflows for active learning-based ML-IAM training, data set curation, and on-the-fly uncertainty quantification.

36 MATERIALS SCIENCE↗
NASA NTRS2011

Preflight and In-Flight Exercise Conditions for Astronauts on the International Space Station

The physiological demands of spaceflight require astronauts to have certain physical abilities. They must be able to perform routine and off-nominal physical work during flight and upon re-entry into a gravity environment to ensure mission success, such as an Extra Vehicular Activity (EVA) or emergency egress. To prepare the astronauts for their mission, a Wyle Astronaut Strength Conditioning and Rehabilitation specialist (ASCR) works individually with the astronauts to prescribe preflight strength and conditioning programs and in-flight exercise, utilizing Countermeasure Systems (CMS) exercise hardware. PURPOSE: To describe the preflight and in-flight exercise programs for ISS crewmembers. METHODS: Approximately 2 years before a scheduled launch, an ASCR is assigned to each astronaut and physical training (PT) is routinely scheduled. Preflight PT of astronauts consists of carrying out strength, aerobic and general conditioning, employing the principles of periodization. Exercise programs are prescribed to the astronauts to account for their individual fitness levels, planned mission-specific tasks, areas of concern, and travel schedules. Additionally, astronauts receive instruction on how to operate CMS exercise hardware and receive training for microgravity-specific conditions. For example, astronauts are scheduled training sessions for the International Space Station (ISS) treadmill (TVIS) and cycle ergometer (CEVIS), as well as the Advanced Resistive Exercise Device (ARED). In-flight programs are designed to maintain or even improve the astronauts pre-flight levels of fitness, bone health, muscle strength, power and aerobic capacity. In-flight countermeasure sessions are scheduled in 2.5 h blocks, six days a week, which includes 1.5 h for resistive training and 1 h for aerobic exercise. CONCLUSIONS: Crewmembers reported the need for more scheduled time for preflight training. During flight, crewmembers have indicated that the in-flight exercise is sufficient, but would like more reliable and capable hardware.

Guilliams, Mark E.↗
NASA NTRSDate not supplied

TPSAS-NF1676L-13606-DND

For over two decades, researchers at NASA Langley Research Center have been combining principles of engineering and psychology to explore the cognitive experiences of system operators. Conceptualizing the application of this interdisciplinary combination in the context of operators? extracurricular activities has resulted in the creation of numerous mental training spin-off technologies. Methods derived from feedback control concepts have been employed to assess the brains responses when managing automation and principles of conditioning have been embodied in devices designed to reinforce effective modes of psychophysiological activity. These methods and devices will be discussed and how they are being applied to ameliorate attention and stress disorders in clinical practice and impediments to optimal performance in sports will be demonstrated.

Alan T Pope↗
DOE OSTI2023

Fitting a deep generative hadronization model

Hadronization is a critical step in the simulation of high-energy particle and nuclear physics experiments. As there is no first principles understanding of this process, physically-inspired hadronization models have a large number of parameters that are fit to data. Deep generative models are a natural replacement for classical techniques, since they are more flexible and may be able to improve the overall precision. Proof of principle studies have shown how to use neural networks to emulate specific hadronization when trained using the inputs and outputs of classical methods. However, these approaches will not work with data, where we do not have a matching between observed hadrons and partons. In this paper, we develop a protocol for fitting a deep generative hadronization model in a realistic setting, where we only have access to a set of hadrons in data. Our approach uses a variation of a Generative Adversarial Network with a permutation invariant discriminator. We find that this setup is able to match the hadronization model in Herwig with multiple sets of parameters. This work represents a significant step forward in a longer term program to develop, train, and integrate machine learning-based hadronization models into parton shower Monte Carlo programs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Deposition Height Prediction in Directed Energy Deposition

Using 316L stainless steel as a model material, reduced-order models are developed to predict capture efficiency, deposition height, and site-specific hardness in directed energy deposition. Capture efficiency is predicted over a 15 to 55 pct range using a dimensionless number derived from processing conditions and thermophysical properties. Deposition height is predicted over a 0.3 to 1.3 mm range without in situ sensing or prior training data, using two models based on the same mass and energy-balance principles. Predictions are compared with machine learning approaches. A quantitative relationship links deposition height, primary dendrite arm spacing (PDAS), and hardness: heights of 0.3 to 1.1 mm correspond to PDAS values of 2.7 to 5.1 µm and Vickers hardness (HV) of 160 to 219. Thinner layers cool more rapidly, producing finer microstructures and higher hardness. Samples fabricated with in situ variations in deposition height exhibited up to 55 HV differences between thick and thin regions, demonstrating that local control of deposition height enables predictive, site-specific hardness within a single build. These results establish deposition height prediction as a pathway for a priori process design and property control in directed energy deposition for 316L stainless steel.

Kunkel, William [Univ. of Wisconsin, Madison, WI (↗
DOE OSTI2022

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

An emerging use-case of machine learning (ML) is to train a model on a high-performance system and deploy the trained model on energy-constrained embedded systems. Neuromorphic hardware platforms, which operate on principles of the biological brain, can significantly lower the energy overhead of a machine learning inference task, making these platforms an attractive solution for embedded ML systems. In this paper, we present a design-technology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non-Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay introduced by parasitic components on its current paths, and 2) the varying delay to sense different resistance states of its NVM cells. Based on these two observations, we make the following three contributions. First, on the technology front, we propose an optimization scheme where the NVM resistance state that takes the longest time to sense is set on current paths having the least delay, and vice versa, reducing the average PE latency, which improves the QoS. Second, on the architecture front, we introduce isolation transistors within each PE to partition it into regions that can be individually power-gated, reducing both latency and energy. Finally, on the system-software front, we propose a mechanism to leverage the proposed technological and architectural enhancements when implementing a machine-learning inference task on neuromorphic PEs of the hardware. Evaluations with a recent neuromorphic hardware architecture show that our proposed design-technology co-optimization approach improves both performance and energy efficiency of machine-learning inference tasks without incurring high cost-per-bit.

42 ENGINEERING↗
DOE OSTI2020

Toward fusion plasma scenario planning for NSTX-U using machine-learning-accelerated models

One of the most promising devices for realizing power production through nuclear fusion is the tokamak. To maximize performance, it is preferable that tokamak reactors achieve advanced operating scenarios characterized by good plasma confinement, improved magnetohydrodynamic (MHD) stability, and a largely non-inductively driven plasma current. Such scenarios could enable steady-state reactor operation with high \emph{fusion gain} --- the ratio of produced fusion power to the external power provided through the plasma boundary. Precise and robust control of the evolution of the plasma boundary shape as well as the spatial distribution of the plasma current, density, temperature, and rotation will be essential to achieving and maintaining such scenarios. The complexity of the evolution of tokamak plasmas, arising due to nonlinearities and coupling between various parameters, motivates the use of model-based control algorithms that can account for the system dynamics. In this work, a learning-based accelerated model trained on data from the National Spherical Torus Experiment Upgrade (NSTX-U) is employed to develop planning and control strategies for regulating the density and temperature profile evolution around desired trajectories. The proposed model combines empirical scaling laws developed across multiple devices with neural networks trained on empirical data from NSTX-U and a database of first-principles-based computationally intensive simulations. The reduced execution time of the accelerated model will enable practical application of optimization algorithms and reinforcement learning approaches for scenario planning and control development. An initial demonstration of applying optimization approaches to the learning-based model is presented, including a strategy for mitigating the effect of leaving the finite validity range of the accelerated model. The approach shows promise for actuator planning between experiments and in real-time.

machine learning↗
NASA NTRS2020

Designing and Training for Appropriate Trust in Increasingly Autonomous Advanced Air Mobility Operations: A Mental Model Approach: Version 1

To enable effective human-autonomy teaming (HAT) in Advanced Air Mobility (AAM) operations, the current paper presents a theoretical framework to design and train for appropriate trust in automation. The novel contribution of this work resides in connecting the construct of trust to mental models and showing how this method could be used to enable emerging HAT concepts such as Adaptive Trust Calibration. To contextualize this framework, in section 2 we discuss simplified vehicle operations (SVO) and remote vehicle operations (RVO), which are leading operational concepts within AAM. In section 3 we describe our perspective on automation and increasingly autonomous systems and present a brief discussion on human-automation interaction and human-autonomy teaming. In section 4 we provide a detailed discussion on the construct of trust in automation. In section 5 we present a framework that associates mental models with trust through principles of transparent design. Finally, in section 6 we present three descriptive models for designing and training for appropriate trust in increasingly autonomous systems.

Human-Autonomy Teaming↗
DOE OSTI2021

Materials representation and transfer learning for multi-property prediction

The adoption of machine learning in materials science has rapidly transformed materials property prediction. Hurdles limiting full capitalization of recent advancements in machine learning include the limited development of methods to learn the underlying interactions of multiple elements as well as the relationships among multiple properties to facilitate property prediction in new composition spaces. To address these issues, we introduce the Hierarchical Correlation Learning for Multi-property Prediction (H-CLMP) framework that seamlessly integrates: (i) prediction using only a material's composition, (ii) learning and exploitation of correlations among target properties in multi-target regression, and (iii) leveraging training data from tangential domains via generative transfer learning. The model is demonstrated for prediction of spectral optical absorption of complex metal oxides spanning 69 three-cation metal oxide composition spaces. H-CLMP accurately predicts non-linear composition-property relationships in composition spaces for which no training data are available, which broadens the purview of machine learning to the discovery of materials with exceptional properties. This achievement results from the principled integration of latent embedding learning, property correlation learning, generative transfer learning, and attention models. The best performance is obtained using H-CLMP with transfer learning [H-CLMP(T)] wherein a generative adversarial network is trained on computational density of states data and deployed in the target domain to augment prediction of optical absorption from composition. H-CLMP(T) aggregates multiple knowledge sources with a framework that is well suited for multi-target regression across the physical sciences.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A nearsighted force-training approach to systematically generate training data for the machine learning of large atomic structures

A challenge of atomistic machine-learning (ML) methods is ensuring that the training data are suitable for the system being simulated, which is particularly challenging for systems with large numbers of atoms. Most atomistic ML approaches rely on the nearsightedness principle (“all chemistry is local”), using information about the position of an atom’s neighbors to predict a per-atom energy. Here, in this work, we develop a framework that exploits the nearsighted nature of ML models to systematically produce an appropriate training set for large structures. We use a per-atom uncertainty estimate to identify the most uncertain atoms and extract chunks centered around these atoms. It is crucial that these small chunks are both large enough to satisfy the ML’s nearsighted principle (that is, filling the cutoff radius) and are large enough to be converged with respect to the electronic structure calculation. We present data indicating when the electronic structure calculations are converged with respect to the structure size, which fundamentally limits the accuracy of any nearsighted ML calculator. These new atomic chunks are calculated in electronic structures, and crucially, only a single force—that of the central atom—is added to the growing training set, preventing the noisy and irrelevant information from the piece’s boundary from interfering with ML training. The resulting ML potentials are robust, despite requiring single-point calculations on only small reference structures and never seeing large training structures. We demonstrated our approach via structure optimization of a 260-atom structure and extended the approach to clusters with up to 1415 atoms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Deep Generative Modeling of Periodic Variable Stars Using Physical Parameters

The ability to generate physically plausible ensembles of variable sources is critical to the optimization of time domain survey cadences and the training of classification models on data sets with few to no labels. Traditional data augmentation techniques expand training sets by reenvisioning observed exemplars, seeking to simulate observations of specific training sources under different (exogenous) conditions. Unlike fully theory-driven models, these approaches do not typically allow principled interpolation nor extrapolation. Moreover, the principal drawback of theory-driven models lies in the prohibitive computational cost of simulating source observables from ab initio parameters. In this work, we propose a computationally tractable machine learning approach to generate realistic light curves of periodic variables capable of integrating physical parameters and variability classes as inputs. Our deep generative model, inspired by the transparent latent space generative adversarial networks, uses a variational autoencoder (VAE) architecture with temporal convolutional network layers, trained using the OGLE-III optical light curves and physical characteristics (e.g., effective temperature and absolute magnitude) from Gaia DR2. A test using the temperature–shape relationship of RR Lyrae demonstrates the efficacy of our generative “physics-enhanced latent space VAE” (PELS-VAE) model. Such deep generative models, serving as nonlinear nonparametric emulators, present a novel tool for astronomers to create synthetic time series over arbitrary cadences.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

FAIR AI models in high energy physics

Abstract The findable, accessible, interoperable, and reusable (FAIR) data principles provide a framework for examining, evaluating, and improving how data is shared to facilitate scientific discovery. Generalizing these principles to research software and other digital products is an active area of research. Machine learning models—algorithms that have been trained on data without being explicitly programmed—and more generally, artificial intelligence (AI) models, are an important target for this because of the ever-increasing pace with which AI is transforming scientific domains, such as experimental high energy physics (HEP). In this paper, we propose a practical definition of FAIR principles for AI models in HEP and describe a template for the application of these principles. We demonstrate the template’s use with an example AI model applied to HEP, in which a graph neural network is used to identify Higgs bosons decaying to two bottom quarks. We report on the robustness of this FAIR AI model, its portability across hardware architectures and software frameworks, and its interpretability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

A deep learning interatomic potential developed for atomistic simulation of carbon materials

Interatomic potentials based on neural-network machine learning method have attracted considerable attention in recent years owing to their outstanding ability to balance the accuracy and efficiency in atomistic simulations. In this work, a neural-network potential (NNP) for carbon is generated to simulate the structural properties of various carbon structures. The potential is trained using a database consisting of crystalline and liquid structures obtained by the first-principles density functional theory (DFT) calculations. The developed potential accurately predicts the energies and forces in crystalline and liquid carbon structures, the energetic stability of defected graphene, and the structures of amorphous carbon as the function of density. As a result, the excellent accuracy and transferability of the NNP provide a promising tool for accurate atomistic simulations of various carbon materials with faster speed and much lower cost.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Solving the master equation without kinetic Monte Carlo: Tensor train approximations for a CO oxidation model

In multiscale modeling of heterogeneous catalytic processes, one crucial point is the solution of a Markovian master equation describing the stochastic reaction kinetics. Usually, this is too high-dimensional to be solved with standard numerical techniques and one has to rely on sampling approaches based on the kinetic Monte Carlo method. In this study we break the curse of dimensionality for the direct solution of the Markovian master equation by exploiting the Tensor Train Format for this purpose. The performance of the approach is demonstrated on a first principles based, reduced model for the CO oxidation on the RuO{sub 2}(110) surface. We investigate the complexity for increasing system size and for various reaction conditions. The advantage over the stochastic simulation approach is illustrated by a problem with increased stiffness.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2016

TH-CD-202-07: A Methodology for Generating Numerical Phantoms for Radiation Therapy Using Geometric Attribute Distribution Models

Purpose: A persistent challenge for the quality assessment of radiation therapy treatments (e.g. contouring accuracy) is the absence of the known, ground truth for patient data. Moreover, assessment results are often patient-dependent. Computer simulation studies utilizing numerical phantoms can be performed for quality assessment with a known ground truth. However, previously reported numerical phantoms do not include the statistical properties of inter-patient variations, as their models are based on only one patient. In addition, these models do not incorporate tumor data. In this study, a methodology was developed for generating numerical phantoms which encapsulate the statistical variations of patients within radiation therapy, including tumors. Methods: Based on previous work in contouring assessment, geometric attribute distribution (GAD) models were employed to model both the deterministic and stochastic properties of individual organs via principle component analysis. Using pre-existing radiation therapy contour data, the GAD models are trained to model the shape and centroid distributions of each organ. Then, organs with different shapes and positions can be generated by assigning statistically sound weights to the GAD model parameters. Organ contour data from 20 retrospective prostate patient cases were manually extracted and utilized to train the GAD models. As a demonstration, computer-simulated CT images of generated numerical phantoms were calculated and assessed subjectively and objectively for realism. Results: A cohort of numerical phantoms of the male human pelvis was generated. CT images were deemed realistic both subjectively and objectively in terms of image noise power spectrum. Conclusion: A methodology has been developed to generate realistic numerical anthropomorphic phantoms using pre-existing radiation therapy data. The GAD models guarantee that generated organs span the statistical distribution of observed radiation therapy patients, according to the training dataset. The methodology enables radiation therapy treatment assessment with multi-modality imaging and a known ground truth, and without patient-dependent bias.

60 APPLIED LIFE SCIENCES↗