Search NASA⌕ Search

SEARCH · Search NASA

Results for “training effectiveness”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Trainability of ReLU networks and Data-dependent Initialization

Here, in this paper, we study the trainability of rectified linear unit (ReLU) networks at initialization. A ReLU neuron is said to be dead if it only outputs a constant for any input. Two death states of neurons are introduced–tentative and permanent death. A network is then said to be trainable if the number of permanently dead neurons is sufficiently small for a learning task. We refer to the probability of a randomly initialized network being trainable as trainability. We show that a network being trainable is a necessary condition for successful training, and the trainability serves as an upper bound of training success rates. In order to quantify the trainability, we study the probability distribution of the number of active neurons at initialization. In many applications, overspecified or overparameterized neural networks are successfully employed and shown to be trained effectively. With the notion of trainability, we show that overparameterization is both a necessary and a sufficient condition for achieving a zero training loss. Furthermore, we propose a data-dependent initialization method in an overparameterized setting. Numerical examples are provided to demonstrate the effectiveness of the method and our theoretical findings.

97 MATHEMATICS AND COMPUTING↗

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

Li, Chaojian [ORNL] (ORCID:0000000340309777)↗

The Umbra Simulation and Integration Framework Applied to Emergency Response Training

The Mine Emergency Response Interactive Training Simulation (MERITS) is intended to prepare personnel to manage an emergency in an underground coal mine. The creation of an effective training environment required realistic emergent behavior in response to simulation events and trainee interventions, exploratory modification of miner behavior rules, realistic physics, and incorporation of legacy code. It also required the ability to add rich media to the simulation without conflicting with normal desktop security settings. Our Umbra Simulation and Integration Framework facilitated agent-based modeling of miners and rescuers and made it possible to work with subject matter experts to quickly adjust behavior through script editing, rather than through lengthy programming and recompilation. Integration of Umbra code with the WebKit browser engine allowed the use of JavaScript-enabled local web pages for media support. This project greatly extended the capabilities of Umbra in support of training simulations and has implications for simulations that combine human behavior, physics, and rich media.

Hamilton, Paul Lawrence↗

Summary of Technical Interchange Meetings (TIMs) Designed to Enable Earth Independent Medical Operations (EIMO)

The Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program hosted a series of TIMs in 2023-2024 designed to stimulate discussion around specific topics with the goal of enabling EIMO. In context of the thematic constituent elements of EIMO, namely pre-mission planning, acute/emergent/prolonged medical decision making, supply/resource management and task load management, subject matter experts from industry, academia and government (NASA and other Agencies) provided valuable and actionable guidance and recommendations. Earth-based medical experts will remain indispensable for pre-mission planning, however, management of acute/emergent medical contingencies will require a gradual transition of medical care and decision making from terrestrial to space-based assets to enable support of astronaut health and performance and reduce overall mission risk. Key to achieving these enhancements is providing an integrated data system platform capable of utilizing multiple data streams in concert with a variety of on-board databases and passive monitoring of video and wearable sensors to enable a multi-modal, agentic AI-based clinical decision support system (CDSS) to support crew medical officer (CMO) medical decision-making. The EIMO series of TIMs (I-V) have proven to be instructive and portend a significant paradigm shift will be necessary to maintain crew health and performance on exploration class missions. Importantly, since the expected paradigm shift will be significantly different from the methods of operation that have been employed for the majority of missions from the inception of human spaceflight to date, any proposed methods must be deployed in the setting of ongoing operations early and be “tested, reviewed and practiced” while reliable back-up is available to facilitate an Enterprise-wide level of comfort and acceptance. Serious constraints on data transmission coupled with a large and expanding universe of on-board medical informatics data streams will necessitate implementation of a CDSS to supplant the current reliance on support provided by ground-based SMEs. Establishment of trust in the system by CMO/crew and the ground-based medical support team will be essential. Co-development of a CDSS with industry partners will assure that state of the art tools can be employed, and industry efficiencies can be leveraged. Training regimens, materials and tools must evolve to be responsive (just-in-time training) and facilitate autonomous execution of procedures. Proficiency metrics should be established and be based on validated competencies or milestones as opposed to a prescribed number of training hours. Training should be prioritized for broad, translatable skills that have universal application across a variety of medical conditions. Repetition was deemed to be the key to achieving proficiency and emphasis should lie in procedural training which is known to extinguish more rapidly than diagnostic skills. Advanced tools, e.g., extended reality, can provide more realistic and effective training. Use of advanced probabilistic risk assessment tools will be essential to optimize the medical system capability while carefully balancing risk relative to mass/power/volume limitations. Importance of factoring use-life of medical supplies and maintaining awareness of redundancy and opportunity to re-purpose under off nominal situations was emphasized. Consideration of adopting optimized performance standards vs. “good-enough” performance thresholds is warranted. The use of legacy systems as opposed to creating new systems may be preferable. Managing task load and associated cognitive load will be essential to maintain operational safety and behavioral health. ExMC aspires to create a shared EIMO paradigm and strategic vision for advancing medical system design through novel technologies, training, protocols, and support capabilities, built upon the spirit of successful strategies and innovations over the past six decades of space medicine operations.

Jay Lemery↗

Uncontrolled Learning: Codesign of Neuromorphic Hardware Topology for Neuromorphic Algorithms

Neuromorphic computing has the potential to revolutionize future technologies and our understanding of intelligence, yet it remains challenging to realize in practice. The learning-from-mistakes algorithm, inspired by the brain's simple learning rules of inhibition and pruning, is one of the few brain-like training methods. This algorithm is implemented in neuromorphic memristive hardware through a codesign process that evaluates essential hardware trade-offs. While the algorithm effectively trains small networks as binary classifiers and perceptrons, performance declines significantly with increasing network size unless the hardware is tailored to the algorithm. This work investigates the trade-offs between depth, controllability, and capacity—the number of learnable patterns—in neuromorphic hardware. This highlights the importance of topology and governing equations, providing theoretical tools to evaluate a device's computational capacity based on its measurements and circuit structure. The findings show that breaking neural network symmetry enhances both controllability and capacity. Additionally, by pruning the circuit, neuromorphic algorithms in all-memristive circuits can utilize stochastic resources to create local contrasts in network weights. Through combined experimental and simulation efforts, the parameters are identified that enable networks to exhibit emergent intelligence from simple rules, advancing the potential of neuromorphic computing.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Development and evaluation of a prototype in-flight instrument flight rules (IFR) procedures trainer

An in-flight instrument flight rules (IFR) procedures trainer capable of providing simulated indications of instrument flight in a typical general aviation aircraft independent of ground based navigation aids was developed. The IFR navaid related instruments and circuits from an ATC 610J table top simulator were installed in a Cessna 172 aircraft and connected to its electrical power and pitot static systems. The benefits expected from this hybridization concept include increased safety by reducing the number of general aviation aircraft conducting IFR training flights in congested terminal areas, and reduced fuel use and instruction costs by lessening the need to fly to and from navaid equipped airports and by increased efficiency of the required in-flight training. Technical feasibility was demonstrated and the operational feasibility of the concept was evaluated. Results indicated that the in-flight simulator is an effective training device for teaching IFR procedural skills.

Aaron, J. B., Jr.↗

The relationship between workload and training - An introduction

This paper reviews the relationships among workload, performance, and training. Its goal is to introduce the concepts of workload and training and to suggest how they may be related. It suggests some of the practical and theoretical benefits to be derived from their joint consideration. Training effectiveness can be improved by monitoring trainee workload and the reliability of workload predictions, and measures can be improved by identifying and controlling the training levels of experimental subjects.

Hart, Sandra G.↗

Evidence Report: Risk of Performance Errors Due to Training Deficiencies

Substantial evidence supports the claim that inadequate training leads to performance errors. Barshi and Loukopoulos (2012) demonstrate that even a task as carefully developed and refined over many years as operating an aircraft can be significantly improved by a systematic analysis, followed by improved procedures and improved training (see also Loukopoulos, Dismukes, & Barshi, 2009a). Unfortunately, such a systematic analysis of training needs rarely occurs during the preliminary design phase, when modifications are most feasible. Training is often seen as a way to compensate for deficiencies in task and system design, which in turn increases the training load. As a result, task performance often suffers, and with it, the operators suffer and so does the mission. On the other hand, effective training can indeed compensate for such design deficiencies, and can even go beyond to compensate for failures of our imagination to anticipate all that might be needed when we send our crew members to go where no one else has gone before. Much of the research literature on training is motivated by current training practices aimed at current training needs. Although there is some experience with operations in extreme environments on Earth, there is no experience with long-duration space missions where crews must practice semi-autonomous operations, where ground support must accommodate significant communication delays, and where so little is known about the environment. Thus, we must develop robust methodologies and tools to prepare our crews for the unknown. The research necessary to support such an endeavor does not currently exist, but existing research does reveal general challenges that are relevant to long-duration, high-autonomy missions. The evidence presented here describes issues related to the risk of performance errors due to training deficiencies. Contributing factors regarding training deficiencies may pertain to organizational process and training programs for spaceflight, such as when training programs are inadequate or unavailable. Furthermore, failure to match between tasks on the one hand, and learning and memory abilities on the other hand is a contributing factor, especially when individuals' relative efficiency with which new information is acquired, and adjustments made in behavior or thinking, are inconsistent with mission demands. Thus, if training deficiencies are present, the likelihood of errors or of the inability to successfully complete a task increases. What's more, the overall risk to the crew, the vehicle, and the mission increases.

Barshi, Immanuel↗

Adversarial uncertainty quantification in physics-informed neural networks

Here, we introduce a deep learning framework for quantifying and propagating uncertainty in systems governed by non-linear differential equations using physics-informed neural networks. Specifically, we employ latent variable models to construct probabilistic representations for the system states, and put forth an adversarial inference procedure for training them on data, while constraining their predictions to satisfy given physical laws expressed by partial differential equations. Such physics-informed constraints provide a regularization mechanism for effectively training deep generative models as surrogates of physical systems in which the cost of data acquisition is high, and training data-sets are typically small. This provides a flexible framework for characterizing uncertainty in the outputs of physical systems due to randomness in their inputs or noise in their observations that entirely bypasses the need for repeatedly sampling expensive experiments or numerical simulators. Moreover, we demonstrate the effectiveness of our approach through a series of examples involving uncertainty propagation in non-linear conservation laws, and the discovery of constitutive laws for flow through porous media directly from noisy data.

97 MATHEMATICS AND COMPUTING↗

Data Augmentation for Neutron Spectrum Unfolding with Neural Networks

Neural networks require a large quantity of training spectra and detector responses in order to learn to solve the inverse problem of neutron spectrum unfolding. In addition, due to the under-determined nature of unfolding, non-physical spectra which would not be encountered in usage should not be included in the training set. While physically realistic training spectra are commonly determined experimentally or generated through Monte Carlo simulation, this can become prohibitively expensive when considering the quantity of spectra needed to effectively train an unfolding network. In this paper, we present three algorithms for the generation of large quantities of realistic and physically motivated neutron energy spectra. Using an IAEA compendium of 251 spectra, we compare the unfolding performance of neural networks trained on spectra from these algorithms, when unfolding real-world spectra, to two baselines. We also investigate general methods for evaluating the performance of and optimizing feature engineering algorithms.

McGreivy, James (ORCID:0000000321723411)↗

Artificial Intelligence and Computer Vision for Cost-Effective Benthic Habitat Characterizations

Machine learning applied to computer vision and pattern recognition is a type of artificial intelligence that has advanced rapidly in the last 10 to 15 years, spurred forward by breakthroughs in deep convolutional neural networks. These state-of-the-art methods are poised to become widely used in environmental monitoring applications as a result of the increasing abundance of data available from different imaging platforms (e.g., fixed-point cameras, drone surveys, high-resolution satellite data) that can be analyzed to observe, model, and understand environmental conditions. Computer vision and pattern recognition (CVPR) tools advance our ability to use imagery and camera-based tools in cost-effective ways for environmental monitoring. Although these techniques offer great potential, some challenges remain, such as the need for large sets of labeled images for model training and validation and optimized hardware and software to ensure that the models can be trained effectively and in a reasonable amount of time. Here, we have overcome many of these rate-limiting challenges by using a diverse image library built across multiple projects coupled with staff expertise and onsite computing resources. We are working toward a fully automated SPI processing system and also are moving to develop CVPR analytical tools for other imaging platforms and data sets.

02 PETROLEUM↗

Development of Training Programs to Optimize Planetary Ambulation

Astronauts experience disturbances in functional mobility following their return to Earth due to adaptive responses that occur during exposure to the microgravity conditions of space flight. Despite significant time spent performing in-flight exercise routines, these training programs have not been able to mitigate postflight alterations in postural and locomotor function. Therefore, the goal of our two inter-related projects (NSBRI-ground based and ISS flight study, "Mobility") is to develop and test gait training programs that will serve to optimize functional mobility during the adaptation period immediately following space flight, thereby improving the safety and efficiency of planetary ambulation. The gait training program entails manipulating the sensory conditions of treadmill exercise to systematically challenge the balance and gait control system. This enhances the overall adaptability of locomotor function enabling rapid reorganization of gait control to respond to ambulation in different gravitational environments. To develop the training program, we are conducting a series of ground-based studies evaluating the training efficacy associated with variation in visual flow, body loading, and support surface stability during treadmill walking. We will also determine the optimal method to present training stimuli within and across training sessions to maximize both the efficacy and efficiency of the training procedure. Results indicate that variations in both visual flow and body unloading during treadmill walking leads to modification in locomotor control and can be used as effective training modalities. Additionally, the composition and timing of sensory challenges experienced during each training session has significant impact on the ability to rapidly reorganize locomotor function when exposed to a novel sensory environment. We have developed the capability of producing support surface variation during gait training by mounting a treadmill on a six-degree-of-freedom motion device. This hardware development will allow us to evaluate the efficacy of this type of training in conjunction with variation in visual flow and body unloading.

Bloomberg, J. J.↗

Transfer learning for metamaterial design and simulation

Abstract We demonstrate transfer learning as a tool to improve the efficacy of training deep learning models based on residual neural networks (ResNets). Specifically, we examine its use for study of multi-scale electrically large metasurface arrays under open boundary conditions in electromagnetic metamaterials. Our aim is to assess the efficiency of transfer learning across a range of problem domains that vary in their resemblance to the original base problem for which the ResNet model was initially trained. We use a quasi-analytical discrete dipole approximation (DDA) method to simulate electrically large metasurface arrays to obtain ground truth data for training and testing of our deep neural network. Our approach can save significant time for examining novel metasurface designs by harnessing the power of transfer learning, as it effectively mitigates the pervasive data bottleneck issue commonly encountered in deep learning. We demonstrate that for the best case when the transfer task is sufficiently similar to the target task, a new task can be effectively trained using only a few data points yet still achieve a test mean absolute relative error of 3 % with a pre-trained neural network, realizing data reduction by a factor of 1000.

Peng, Rixi↗

Stimulus control in pigeons after extended discriminative training

The effects of amount of training on conditioned inhibition and on the degree of stimulus control were studied using pigeons. The ability of an S- associated with non-reinforcement of suppress positive reinforced behavior was acquired very rapidly during discriminative training. Increased S+, S- training appeared to weaken this conditioned inhibitory effect while at the same time more S+ training apparently increased the amount of external inhibition (non-conditioned inhibition) of positively reinforced behavior by a novel stimulus. Behavioral contrast and incremental generalization gradients along the S- dimension (inhibitory dimensional control) were absent at all stages of training. Behavioral contrast and inhibitory dimensional control are therefore not necessary concomitants of conditioned inhibition by an S-. A new method of assessing the suppressive effects of stimuli during generalization tests was described.

Yarczower, M.↗

Joule Heating Effect on Field-Free Magnetization Switching by Spin-Orbit Torque in Exchange-Biased Systems

Switching of magnetization via spin-orbit torque provides an efficient alternative for nonvolatile memory and logic devices. However, to achieve deterministic switching of perpendicular magnetization, an external magnetic field collinear with the current is usually required, which makes these devices inappropriate for practical applications. As such, in this work, we examine the current-induced magnetization switching in a perpendicularly magnetized exchange-biased Pt/CoFe/IrMn system. A magnetic field annealing technique is used to introduce in-plane exchange biases, which are quantitatively characterized. Under proper conditions, field-free current-driven switching is achieved.We study the Joule heating effect, and we show how it can decrease the in-plane exchange bias and degrade the field-free switching. Furthermore, we discuss that the exchange-bias training effect can have similar effects.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Study on the occurrence of spontaneously established perpendicular exchange bias in Co{sub 49}Pt{sub 51}/IrMn bilayers

In this study, perpendicular exchange bias (PEB) effect in the as-grown Co{sub 49}Pt{sub 51}/IrMn bilayers was demonstrated at room temperature using single-layered Co{sub 49}Pt{sub 51} alloy thin film as ferromagnetic (FM) layer. Several unusual features were observed in this system, viz.,: (i) the PEB was spontaneously established without any external magnetic field treatments, (ii) single-shifted loops were obtained rather than double-shifted ones, and (iii) the spontaneous PEB effect was accompanied by a reduction in perpendicular coercivity, H{sub c⊥} from 1024 to 632 Oe. The results of x–ray diffraction revealed the formation of IrMn (111) texture. Training effect studies indicate that the PEB effect is stable in this system with less than 5% variation in PEB value within 15 repetitive scans. Significant reduction in the PEB effect was found for the CoPt/IrMn films either grown or subjected to post-annealing under external magnetic field (H{sub ind}). The thickness dependence of PEB effect with respect to the FM and antiferromagnetic layers were also investigated and a largest PEB value of 533 Oe was obtained for the sample grown with 3-nm thick CoPt and 10-nm thick IrMn layers. The results of present study thus establish an opportunity to realize PEB effect in the absence of external field during fabrication.

36 MATERIALS SCIENCE↗

Plateau Phenomenon in Gradient Descent Training of RELU Networks: Explanation, Quantification, and Avoidance

The ability of neural networks to provide ‘best in class’ approximation across a wide range of applications is well-documented. Nevertheless, the powerful expressivity of neural networks comes to naught if one is unable to effectively train (choose) the parameters defining the network. In general, neural networks are trained by gradient descent type optimization methods,a stochastic variant thereof. In practice, such methods result in the loss function decreases rapidly at the beginning of training but then, after a relatively small number of steps, significantly slow down. The loss may even appear to stagnate over the period of a large number of epochs, only to then suddenly start to decrease fast again for no apparent reason. This so-called plateau phenomenon manifests itself in many learning tasks. The present work aims to identify and quantify the root causes of plateau phenomenon.analysis is carried out in the setting of univariate ReLU networks. No assumptions are made on the number of neurons relative to the number of training data, and our results hold for both the lazy and adaptive regimes. Here, the main findings are: plateaux correspond to periods during which activation patterns remain constant, where activation pattern refers to the number of data points that activate a given neuron; quantification of convergence of the gradient flow dynamics; and, characterization stationary points in terms solutions of local least squares regression lines over subsets of the training data. Based on these conclusions, we propose a new iterative training method, the Active Neuron Least Squares (ANLS), characterised by the explicit adjustment of the activation pattern at each step, which is designed to enable a quick exit from a plateau. Illustrative numerical examples are included throughout.

97 MATHEMATICS AND COMPUTING↗

High quality epitaxial thin films and exchange bias of antiferromagnetic Dirac semimetal FeSn

FeSn is a topological semimetal (TSM) and kagome antiferromagnet (AFM) composed of alternating Fe 3 Sn kagome planes and honeycomb Sn planes. This unique structure gives rise to exotic features in the band structures such as the coexistence of Dirac cones and flatbands near the Fermi level, fully spin-polarized 2D surface Dirac fermions, and the ability to open a large gap in the Dirac cone by reorienting the Néel vector. In this paper, we report the synthesis of high-quality epitaxial (0001) FeSn films by magnetron sputtering. Using FeSn/Py heterostructures, we show a large exchange bias effect that reaches an exchange field of 220 Oe at 5 K, providing unambiguous evidence of antiferromagnetism and strong interlayer exchange coupling in our films. Field cycling studies show steep initial training effects, highlighting the complex magnetic interactions and anisotropy. Importantly, our work provides a simple, alternative means to fabricate FeSn films and heterostructures, making it easier to explore the topological physics of AFM TSMs and develop FeSn-based spintronics.

36 MATERIALS SCIENCE↗