Search NASA⌕ Search

SEARCH · Search NASA

Results for “Trainability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

54 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Addressing the "Red-AI" trend of rising energy consumption by large-scale neural networks, this study investigates the measured energy consumption of training various fully connected neural network architectures. We introduce the BUTTER-E dataset, an augmentation to the BUTTER Empirical Deep Learning dataset, containing energy consumption and performance data from 41,129 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (trainable parameters), 8 "shapes", and 14 depths on both CPUs and GPUs using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use. Our analysis uncovers a surprising, hardware-mediated non-linear relationship between energy efficiency and network design, challenging the assumption that reducing the number of parameters or FLOPs is the best way to achieve greater energy efficiency. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy. Highlighting the need for cache-considerate algorithm development, we suggest a codesign approach to energy efficient network, algorithm, and hardware design. This work contributes to the fields of sustainable computing and Green AI, offering practical guidance for creating more energy-efficient neural networks and promoting sustainable AI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Enhancing Lattice Kinetic Schemes for Fluid Dynamics with Lattice-Equivariant Neural Networks

A new class of equivariant neural networks is presented, hereby dubbed lattice-equivariant neural networks (LENNs), designed to satisfy local symmetries of a lattice structure. The approach develops within a recently introduced framework aimed at learning neural network-based surrogate models’ lattice Boltzmann collision operators. Whenever neural networks are employed to model physical systems, respecting symmetries and equivariance properties has been shown to be key for accuracy, numerical stability, and performance. Here, hinging on ideas from group representation theory, trainable layers are defined whose algebraic structure is equivariant with respect to the symmetries of the lattice cell. In this work, the presented method naturally allows for efficient implementations, in terms of both memory usage and computational costs, supporting scalable training/testing for lattices in two spatial dimensions and higher (in which the size of symmetry group grows). The approach is validated and tested considering 2D and 3D flowing dynamics, both in laminar and turbulent regimes. It is compared with group-averaged-based symmetric networks and with plain, nonsymmetric, networks, showing how the presented approach unlocks the (a posteriori) accuracy and training stability of the former models and the train/inference speed of the latter networks. (LENNs are about one order of magnitude faster than group-averaged networks in 3D.) The work in this paper opens toward practical use of machine learning-augmented lattice Boltzmann CFD in real-world simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗
NASA NTRS1971

Trends in problem-solving research - Twelve recently described tasks.

Review of descriptions of the 12 problem-solving tasks developed since the last review (Ray, 1955) of this topic, indicating that the newer tasks are more sophisticated in design and provide for better experimental control than those used prior to 1953. Validity, reliability, sensitivity, trainability, problem structure, and problem difficulty are discussed as criteria for the selection of tasks to be used in studies of skilled problem-solving performance.

Coates, G. D.↗
NASA NTRS1973

Performance improvement in remote manipulation with time delay by means of a learning system.

A teleoperating system is presented that involves shared control between a human operator and a general-purpose computer-based learning machine. This setup features a trainable control network termed the autonomous control subsystem (ACS) which is able to observe the operator's control actions, learn the task at hand, and take appropriate control actions. A working ACS system is described that has been put in operation for the purpose of exploring the uses of a remote intelligence of this type. The expansion of the present system into a multifunctional learning machine capable of a greater degree of autonomy is also discussed.

Freedy, A.↗
NASA NTRS1977

Speech as a pilot input medium

The speech recognition system under development is a trainable pattern classifier based on a maximum-likelihood technique. An adjustable uncertainty threshold allows the rejection of borderline cases for which the probability of misclassification is high. The syntax of the command language spoken may be used as an aid to recognition, and the system adapts to changes in pronunciation if feedback from the user is available. Words must be separated by .25 second gaps. The system runs in real time on a mini-computer (PDP 11/10) and was tested on 120,000 speech samples from 10- and 100-word vocabularies. The results of these tests were 99.9% correct recognition for a vocabulary consisting of the ten digits, and 99.6% recognition for a 100-word vocabulary of flight commands, with a 5% rejection rate in each case. With no rejection, the recognition accuracies for the same vocabularies were 99.5% and 98.6% respectively.

Plummer, R. P.↗
NASA NTRS1991

An architectural approach to create self organizing control systems for practical autonomous robots

For practical industrial applications, the development of trainable robots is an important and immediate objective. Therefore, the developing of flexible intelligence directly applicable to training is emphasized. It is generally agreed upon by the AI community that the fusion of expert systems, neural networks, and conventionally programmed modules (e.g., a trajectory generator) is promising in the quest for autonomous robotic intelligence. Autonomous robot development is hindered by integration and architectural problems. Some obstacles towards the construction of more general robot control systems are as follows: (1) Growth problem; (2) Software generation; (3) Interaction with environment; (4) Reliability; and (5) Resource limitation. Neural networks can be successfully applied to some of these problems. However, current implementations of neural networks are hampered by the resource limitation problem and must be trained extensively to produce computationally accurate output. A generalization of conventional neural nets is proposed, and an architecture is offered in an attempt to address the above problems.

Greiner, Helen↗
NASA NTRS1991

Experiments on neural network architectures for fuzzy logic

The use of fuzzy logic to model and manage uncertainty in a rule-based system places high computational demands on an inference engine. In an earlier paper, the authors introduced a trainable neural network structure for fuzzy logic. These networks can learn and extrapolate complex relationships between possibility distributions for the antecedents and consequents in the rules. Here, the power of these networks is further explored. The insensitivity of the output to noisy input distributions (which are likely if the clauses are generated from real data) is demonstrated as well as the ability of the networks to internalize multiple conjunctive clause and disjunctive clause rules. Since different rules with the same variables can be encoded in a single network, this approach to fuzzy logic inference provides a natural mechanism for rule conflict resolution.

Keller, James M.↗
NASA NTRS1990

Distributed memory approaches for robotic neural controllers

The suitability is explored of two varieties of distributed memory neutral networks as trainable controllers for a simulated robotics task. The task requires that two cameras observe an arbitrary target point in space. Coordinates of the target on the camera image planes are passed to a neural controller which must learn to solve the inverse kinematics of a manipulator with one revolute and two prismatic joints. Two new network designs are evaluated. The first, radial basis sparse distributed memory (RBSDM), approximates functional mappings as sums of multivariate gaussians centered around previously learned patterns. The second network types involved variations of Adaptive Vector Quantizers or Self Organizing Maps. In these networks, random N dimensional points are given local connectivities. They are then exposed to training patterns and readjust their locations based on a nearest neighbor rule. Both approaches are tested based on their ability to interpolate manipulator joint coordinates for simulated arm movement while simultaneously performing stereo fusion of the camera data. Comparisons are made with classical k-nearest neighbor pattern recognition techniques.

Jorgensen, Charles C.↗
NASA NTRS1992

Analysis of fault-tolerant neurocontrol architectures

The fault-tolerance of analog parallel distributed implementations of a multivariable aircraft neurocontroller is analyzed by simulating weight and neuron failures in a simplified scheme of analog processing based on the functional architecture of the ETANN chip (Electrically Trainable Artificial Neural Network). The neural information processing is found to be only partially distributed throughout the set of weights of the neurocontroller synthesized with the backpropagation algorithm. Although the degree of distribution of the neural processing, and consequently the fault-tolerance of the neurocontroller, could be enhanced using Locally Distributed Weight and Neuron Approaches, a satisfactory level of fault-tolerance could only be obtained by retraining the degrated VLSI neurocontroller. The possibility of maintaining neurocontrol performance and stability in the presence of single weight of neuron failures was demonstrated through an automated retraining procedure of the neurocontroller based on a pre-programmed choice and sequence of the training parameters.

Troudet, T.↗
NASA NTRS1977

Automatic speech recognition research at NASA-Ames Research Center

A trainable acoustic pattern recognizer manufactured by Scope Electronics is presented. The voice command system VCS encodes speech by sampling 16 bandpass filters with center frequencies in the range from 200 to 5000 Hz. Variations in speaking rate are compensated for by a compression algorithm that subdivides each utterance into eight subintervals in such a way that the amount of spectral change within each subinterval is the same. The recorded filter values within each subinterval are then reduced to a 15-bit representation, giving a 120-bit encoding for each utterance. The VCS incorporates a simple recognition algorithm that utilizes five training samples of each word in a vocabulary of up to 24 words. The recognition rate of approximately 85 percent correct for untrained speakers and 94 percent correct for trained speakers was not considered adequate for flight systems use. Therefore, the built-in recognition algorithm was disabled, and the VCS was modified to transmit 120-bit encodings to an external computer for recognition.

Coler, Clayton R.↗
NASA NTRS1993

A pattern recognition system for locating small volvanoes in Magellan SAR images of Venus

The Magellan data set constitutes an example of the large volumes of data that today's instruments can collect, providing more detail of Venus than was previously available from Pioneer Venus, Venera 15/16, or ground-based radar observations put together. However, data analysis technology has not kept pace with data collection and storage technology. Due to the sheer size of the data, complete and comprehensive scientific analysis of such large volumes of image data is no longer feasible without the use of computational aids. Our progress towards developing a pattern recognition system for aiding in the detection and cataloging of small-scale natural features in large collections of images is reported. Combining classical image processing, machine learning, and a graphical user interface, the detection of the 'small-shield' volcanoes (less than 15km in diameter) that constitute the most abundant visible geologic feature in the more that 30,000 synthetic aperture radar (SAR) images of the surface of Venus are initially targeted. Our eventual goal is to provide a general, trainable tool for locating small-scale features where scientists specify what to look for simply by providing examples and attributes of interest to measure. This contrasts with the traditional approach of developing problem specific programs for detecting Specific patterns. The approach and initial results in the specific context of locating small volcanoes is reported. It is estimated, based on extrapolating from previous studies and knowledge of the underlying geologic processes, that there should be on the order of 10(exp 5) to 10(exp 6) of these volcanoes visible in the Magellan data. Identifying and studying these volcanoes is fundamental to a proper understanding of the geologic evolution of Venus. However, locating and parameterizing them in a manual manner is forbiddingly time-consuming. Hence, the development of techniques to partially automate this task were undertaken. The primary constraints for this particular problem are that the method must be reasonably robust and fast. Unlike most geological features, the small volcanoes of Venus can be ascribed to a basic process that produces features with a short list of readily defined characteristics differing significantly from other surface features on Venus. For pattern recognition purposes the relevant criteria include (1) a circular planimetric outline, (2) known diameter frequency distribution from preliminary studies, (3) a limited number of basic morphological shapes, and (4) the common occurrence of a single, circular summit pit at the center of the edifice.

Burl, M. C.↗
NASA NTRS1999

Gene Regulation Networks for Modeling Drosophila Development

This chapter will very briefly introduce and review some computational experiments in using trainable gene regulation network models to simulate and understand selected episodes in the development of the fruit fly, Drosophila Melanogaster.

gene regulation morphogenesis network model↗
NASA NTRS2009

Evolvable Neural Software System

The Evolvable Neural Software System (ENSS) is composed of sets of Neural Basis Functions (NBFs), which can be totally autonomously created and removed according to the changing needs and requirements of the software system. The resulting structure is both hierarchical and self-similar in that a given set of NBFs may have a ruler NBF, which in turn communicates with other sets of NBFs. These sets of NBFs may function as nodes to a ruler node, which are also NBF constructs. In this manner, the synthetic neural system can exhibit the complexity, three-dimensional connectivity, and adaptability of biological neural systems. An added advantage of ENSS over a natural neural system is its ability to modify its core genetic code in response to environmental changes as reflected in needs and requirements. The neural system is fully adaptive and evolvable and is trainable before release. It continues to rewire itself while on the job. The NBF is a unique, bilevel intelligence neural system composed of a higher-level heuristic neural system (HNS) and a lower-level, autonomic neural system (ANS). Taken together, the HNS and the ANS give each NBF the complete capabilities of a biological neural system to match sensory inputs to actions. Another feature of the NBF is the Evolvable Neural Interface (ENI), which links the HNS and ANS. The ENI solves the interface problem between these two systems by actively adapting and evolving from a primitive initial state (a Neural Thread) to a complicated, operational ENI and successfully adapting to a training sequence of sensory input. This simulates the adaptation of a biological neural system in a developmental phase. Within the greater multi-NBF and multi-node ENSS, self-similar ENI s provide the basis for inter-NBF and inter-node connectivity.

Curtis, Steven A.↗
NASA NTRSDate not supplied

Design and Analysis of Convolutional Neural Network for RF Signal Modulation Classification for In-Orbit Deployment

To effectively transmit data to and from satellites requires a complex and robust RF communication system. Commonly, several different types of signal modulations may be required to maximize satellite efficiency depending on a variety of unexpected channel impairments. We propose a neural network algorithm capable of learning these RF signal modulations using a supervised learning technique designed for low power, high-efficiency in-orbit deployment. The work presented demonstrates a convolutional neural network (CNN) capable of learning and recognizing a set of modulation schemes commonly used to transmit RF information. We are capable of recognizing the modulation scheme from the I and Q data channels directly, with no preprocessing or data conversion required other than breaking the incoming signal into a set of uniform normalized samples. We perform a network design and size analysis, showing that reasonably high accuracy can be obtained using networks with a relatively low number of trainable parameters. Given that a user of a system such as this may wish to receive a signal using a modulation scheme that the network has not previously learned, we demonstrate that transfer learning can learn new modulation schemes by retraining only the fully connected layers in the CNN. Thus, this type of network would excel in outer space deployment using high-efficiency transfer learning hardware. Modulation recognition can be performed through rapid feedforward computation, and the CNN training process is significantly simplified when learning new modulations is required.

CNN↗
NASA NTRS2023

Toward Physics-informed Neural Networks for 3D Multi-layer Cloud Mask Reconstruction

Three-dimensional (3D) cloud retrievals are critical for understanding their impact on climate and other applications such as aviation safety, weather prediction, and remote sensing. However, obtaining high-resolution and accurate vertical representation of clouds remains unsolved due to the limitations imposed by satellite instrumentation, viewing conditions, and the complexity of cloud dynamics. Cloud masks are essential for comprehending various cloud vertical properties, but deriving accurate 3D cloud masks from 2D satellite imagery data is a challenging task. To tackle these challenges, we introduce a physics-informed loss function for training deep learning models that can extend 2D cloud images into 3D cloud masks. The proposed loss, called CloudMask Loss, is composed of two domain knowledge-informed loss terms: one for evaluating cloud position and thickness, and the other for measuring the number of layers. By combining these loss terms, we improve the trainability of the deep learning models for more accurate and meaningful results. We apply the proposed loss function to different neural networks and demonstrate significant improvements in multi-layer cloud mask reconstruction. Utilizing the same neural network architecture, our proposed loss outperforms standard binary crossentropy loss in terms of multi-layer cloud classification accuracy, number of layers accuracy, and thickness mean absolute error (MAE). The proposed loss function can be readily integrated into various neural network architectures, resulting in substantial performance gains in 3D cloud mask generation.

multi-layer clouds↗
DOE OSTI2024

Exponential concentration in quantum kernel methods

Kernel methods in Quantum Machine Learning (QML) have recently gained significant attention as a potential candidate for achieving a quantum advantage in data analysis. Among other attractive properties, when training a kernel-based model one is guaranteed to find the optimal model’s parameters due to the convexity of the training landscape. However, this is based on the assumption that the quantum kernel can be efficiently obtained from quantum hardware. In this work we study the performance of quantum kernel models from the perspective of the resources needed to accurately estimate kernel values. We show that, under certain conditions, values of quantum kernels over different input data can be exponentially concentrated (in the number of qubits) towards some fixed value. Thus on training with a polynomial number of measurements, one ends up with a trivial model where the predictions on unseen inputs are independent of the input data. We identify four sources that can lead to concentration including expressivity of data embedding, global measurements, entanglement and noise. For each source, an associated concentration bound of quantum kernels is analytically derived. Lastly, we show that when dealing with classical data, training a parametrized data embedding with a kernel alignment method is also susceptible to exponential concentration. Our results are verified through numerical simulations for several QML tasks. Altogether, we provide guidelines indicating that certain features should be avoided to ensure the efficient evaluation of quantum kernels and so the performance of quantum kernel methods.

97 MATHEMATICS AND COMPUTING↗