SEARCH · Search NASA

Results for “hardware neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Exponential Escape Rate of Filamentary Incubation in Mott Spiking Neurons

Mott materials such as vanadium oxides, when subject to a strong applied voltage, present an inhomogeneous insulator-to-metal transition with formation of metallic filaments within the insulating bulk. This property is enabling the development of compact and power-efficient neuromorphic devices known as Mott neurons. However, the nature of the transition has not been fully understood yet, as it may be attributed to different effects, including Joule self-heating and hot-carrier injection. Moreover, the experimental determination of the threshold voltage needed to induce the transition has proven to be challenging, as the transition becomes increasingly unpredictable when the threshold is approached. The physical understanding of these issues would not only deepen our understanding of Mott insulators, but would also be an important step toward the realization of neuromorphic devices based on such materials. In this work we use numerical simulations based on the Mott resistor network model to study the nature of the filament incubation and formation process. Here, we show that both electronic and thermal effects, in the form of current density focusing and Joule self-heating, respectively, contribute to the filamentary incubation and growth. Remarkably, we find that the percolation of the metallic filaments near the threshold is intrinsically stochastic, qualitatively similar to the familiar Arrhenius activated behavior and to the stochastic firing of biological neurons. More precisely, we characterize the filament percolation as a Poisson point process, which has the same probability distribution as mathematical models of neuronal firing with an exponential escape rate. Finally, we support the numerical simulation results by performing experiments in VO 2 that are in agreement with the exponential escape rate behavior. Thus, we establish a functionality of Mott insulators that opens a path toward implementing neuromorphic hardware with quantum materials.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NASA NTRS1993

An application of artificial neural networks to experimental data approximation

As an initial step in the evaluation of networks, a feedforward architecture is trained to approximate experimental data by the backpropagation algorithm. Several drawbacks were detected and an alternative learning algorithm was then developed to partially address the drawbacks. This noniterative algorithm has a number of advantages over the backpropagation method and is easily implemented on existing hardware.

Meade, Andrew J., Jr.↗

DOE OSTI2020

Physics for neuromorphic computing

Neuromorphic computing takes inspiration from the brain to create energy-efficient hardware for information processing, capable of highly sophisticated tasks. Systems built with standard electronics achieve gains in speed and energy by mimicking the distributed topology of the brain. Scaling-up such systems and improving their energy usage, speed and performance by several orders of magnitude requires a revolution in hardware. Here, we discuss how including more physics in the algorithms and nanoscale materials used for data processing could have a major impact in the field of neuromorphic computing. We review striking results that leverage physics to enhance the computing capabilities of artificial neural networks, using resistive switching materials, photonics, spintronics and other technologies. We discuss the paths that could lead these approaches to maturity, towards low-power, miniaturized chips that could infer and learn in real time.

device physics↗

NASA NTRS2002

Implementation of Adaptive Digital Controllers on Programmable Logic Devices

Much has been made of the capabilities of FPGA's (Field Programmable Gate Arrays) in the hardware implementation of fast digital signal processing. Such capability also makes an FPGA a suitable platform for the digital implementation of closed loop controllers. Other researchers have implemented a variety of closed-loop digital controllers on FPGA's. Some of these controllers include the widely used proportional-integral-derivative (PID) controller, state space controllers, neural network and fuzzy logic based controllers. There are myriad advantages to utilizing an FPGA for discrete-time control functions which include the capability for reconfiguration when SRAM-based FPGA's are employed, fast parallel implementation of multiple control loops and implementations that can meet space level radiation tolerance requirements in a compact form-factor. Generally, a software implementation on a DSP (Digital Signal Processor) or microcontroller is used to implement digital controllers. At Marshall Space Flight Center, the Control Electronics Group has been studying adaptive discrete-time control of motor driven actuator systems using digital signal processor (DSP) devices. While small form factor, commercial DSP devices are now available with event capture, data conversion, pulse width modulated (PWM) outputs and communication peripherals, these devices are not currently available in designs and packages which meet space level radiation requirements. In general, very few DSP devices are produced that are designed to meet any level of radiation tolerance or hardness. The goal of this effort is to create a fully digital, flight ready controller design that utilizes an FPGA for implementation of signal conditioning for control feedback signals, generation of commands to the controlled system, and hardware insertion of adaptive control algorithm approaches. An alternative is required for compact implementation of such functionality to withstand the harsh environment encountered on spacecraft. Radiation tolerant FPGA's are a feasible option for reaching this goal.

Gwaltney, David A.↗

DOE OSTI2024

Investigating resource-efficient neutron/gamma classification ML models targeting eFPGAs

There has been considerable interest and resulting progress in implementing machine learning (ML) models in hardware over the last several years from the particle and nuclear physics communities. A big driver has been the release of the Python package, hls4ml, which has enabled porting models specified and trained using Python ML libraries to register transfer level (RTL) code. So far, the primary end targets have been commercial field-programmable gate arrays (FPGAs) or synthesized custom blocks on application specific integrated circuits (ASICs). However, recent developments in open-source embedded FPGA (eFPGA) frameworks now provide an alternate, more flexible pathway for implementing ML models in hardware. These customized eFPGA fabrics can be integrated as part of an overall chip design. In general, the decision between a fully custom, eFPGA, or commercial FPGA ML implementation will depend on the details of the end-use application. In this work, we explored the parameter space for eFPGA implementations of fully-connected neural network (fcNN) and boosted decision tree (BDT) models using the task of neutron/gamma classification with a specific focus on resource efficiency. We used data collected using an AmBe sealed source incident on Stilbene, which was optically coupled to an OnSemi J-series silicon photomultiplier (SiPM) to generate training and test data for this study. We investigated relevant input features and the effects of bit-resolution and sampling rate as well as trade-offs in hyperparameters for both ML architectures while tracking total resource usage. The performance metric used to track model performance was the calculated neutron efficiency at a gamma leakage of 10 -3 . The results of the study will be used to aid the specification of an eFPGA fabric, which will be integrated as part of a test chip.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

Implementing machine learning methods on QICK hardware for qubit readout & control

Quantum readout and control is a fundamental aspect of quantum computing that requires accurate measurement of qubit states. Errors emerge in all stages, from initialization to readout, and identifying errors in post-processing necessitates resource-intensive statistical analysis. In our work, we use a lightweight fully-connected neural network (NN) to classify states of a transmon system with no prior processing. Our NN accelerator yields higher fidelities (92%) than the classical matched filter method (84%). By exploiting the natural parallelism of NNs and their placement near the source of data on field-programmable gate arrays (FPGAs), we can achieve ultra-low latency on the Quantum Instrumentation Control Kit (QICK). Integrating machine learning methods on QICK opens several pathways for efficient real-time processing of quantum circuits.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Dilute Combustion Control Using Spiking Neural Networks

Dilute combustion with exhaust gas recirculation (EGR) in spark-ignition engines presents a cost-effective method for achieving higher levels of engine efficiency. At high levels of EGR, however, cycle-to-cycle variability (CCV) of the combustion process is exacerbated by sporadic occurrences of misfires and partial burns. Previous studies have shown that temporal deterministic patterns emerge at such conditions and certain combustion cycles have a significant influence over future events. Due to the complexity of the combustion process and the nature of CCV, harnessing all the deterministic information for control purposes has remained challenging even with physics based 0-D, 1-D, and high-fidelity computational fluid dynamics (CFD) models. In this study, we present a data-driven approach to optimize the combustion process by controlling CCV adjusting the cycle-to-cycle fuel injection quantity. Readily available data from in-cylinder pressure was used to train a spiking neural network (SNN) which learns the optimal way to manage fuel injection in order to reduce CCV while maintaining acceptable levels of fuel consumption. SNNs are particularly well suited for powertrain control applications due to their ability to be deployed on FPGA-based neuromorphic hardware which are small, inexpensive, and have a low power demand. The high-performance computing (HPC) resources of Oak Ridge National Laboratory were used to run an evolutionary-based training approach for choosing the best SNN configuration that minimizes the size of the network while achieving the desired goal. The neuromorphic hardware with the optimized SNN deployed was connected to the rapid prototyping engine control system for real-time control implementation and tested on a single cylinder version of a GM LNF 4-cylinder engine. The results show a significant reduction of CCV with a small percentage of additional fuel used to stabilize the charge.

33 ADVANCED PROPULSION SYSTEMS↗

NASA NTRS1989

Parallel asynchronous systems and image processing algorithms

A new hardware approach to implementation of image processing algorithms is described. The approach is based on silicon devices which would permit an independent analog processing channel to be dedicated to evey pixel. A laminar architecture consisting of a stack of planar arrays of the device would form a two-dimensional array processor with a 2-D array of inputs located directly behind a focal plane detector array. A 2-D image data stream would propagate in neuronlike asynchronous pulse coded form through the laminar processor. Such systems would integrate image acquisition and image processing. Acquisition and processing would be performed concurrently as in natural vision systems. The research is aimed at implementation of algorithms, such as the intensity dependent summation algorithm and pyramid processing structures, which are motivated by the operation of natural vision systems. Implementation of natural vision algorithms would benefit from the use of neuronlike information coding and the laminar, 2-D parallel, vision system type architecture. Besides providing a neural network framework for implementation of natural vision algorithms, a 2-D parallel approach could eliminate the serial bottleneck of conventional processing systems. Conversion to serial format would occur only after raw intensity data has been substantially processed. An interesting challenge arises from the fact that the mathematical formulation of natural vision algorithms does not specify the means of implementation, so that hardware implementation poses intriguing questions involving vision science.

Coon, D. D.↗

NASA NTRS2002

Implementation of Adaptive Digital Controllers on Programmable Logic Devices

Much has been made of the capabilities of Field Programmable Gate Arrays (FPGA's) in the hardware implementation of fast digital signal processing functions. Such capability also makes an FPGA a suitable platform for the digital implementation of closed loop controllers. Other researchers have implemented a variety of closed-loop digital controllers on FPGA's. Some of these controllers include the widely used Proportional-Integral-Derivative (PID) controller, state space controllers, neural network and fuzzy logic based controllers. There are myriad advantages to utilizing an FPGA for discrete-time control functions which include the capability for reconfiguration when SRAM- based FPGA's are employed, fast parallel implementation of multiple control loops and implementations that can meet space level radiation tolerance requirements in a compact form-factor. Generally, a software implementation on a Digital Signal Processor (DSP) device or microcontroller is used to implement digital controllers. At Marshall Space Flight Center, the Control Electronics Group has been studying adaptive discrete-time control of motor driven actuator systems using DSP devices. While small form factor, commercial DSP devices are now available with event capture, data conversion, Pulse Width Modulated (PWM) outputs and communication peripherals, these devices are not currently available in designs and packages which meet space level radiation requirements. In general, very few DSP devices are produced that are designed to meet any level of radiation tolerance or hardness. An alternative is required for compact implementation of such functionality to withstand the harsh environment encountered on spacemap. The goal of this effort is to create a fully digital, flight ready controller design that utilizes an FPGA for implementation of signal conditioning for control feedback signals, generation of commands to the controlled system, and hardware insertion of adaptive-control algorithm approaches. Radiation tolerant FPGA's are a feasible option for reaching this goal.

Gwaltney, David A.↗

DOE OSTI2025

Investigating parameter trainability in the SNAP-displacement protocol of a qudit system

Here, in this study, we explore the universality of Selective Number-dependent Arbitrary Phase (SNAP) and Displacement gates for quantum control in qudit-based systems. However, optimizing the parameters of these gates poses a challenging task. Our main focus is to investigate the sensitivity of training any of the SNAP parameters in the SNAP-Displacement protocol. We analyze conditions that could potentially lead to the Barren Plateau problem in a qudit system and draw comparisons with multi-qubit systems. The parameterized ansatz we consider consists of blocks, where each block is composed of hardware operations, namely SNAP and Displacement gates [Fösel et al 2020 Efficient cavity control with snap gates arXiv:2004.14256]. Applying Variational Quantum algorithm (VQA) with observable and gate cost functions, we utilize techniques similar to those in [McClean et al 2018 Barren plateaus in quantum neural network training landscapes Nat. Commun.9 1–6] and [Cerezo et al 2021 Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nat. Commun.12 1–12] along with the concept of t − design. Through this analysis, we make the following key observations: (a) The trainability of a SNAP-parameter does not exhibit a preference for any particular direction within our cost function landscape, (b) By leveraging the first and second moments properties of Haar measures, we establish new lemmas concerning the expectation of certain polynomial functions, and (c) utilizing these new lemmas, we identify a general condition that indicates an expected trainability advantage in a qudit system when compared to multi-qubit systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2025

Data Structure Alchemy

In an increasingly more data-driven world, the project set out to uncover the first principles of data-structure design, chart the immense design space they form, and build automation that can synthesize an optimal structure, or even a whole storage engine, for any given workload, hardware platform, and cost target. Data structures are at the center of every computational system and are directly responsible for its performance. Two core technical thrusts were defined: 1) Mapping design spaces for key data-centric abstractions (filters, hash functions, storage-engine layouts, neural-network topologies, blockchain protocols, image layouts, etc.). 2) Developing search & synthesis algorithms, initially analytical cost models, later neural-guided bi-level optimisers that navigate sextillions of candidate designs in seconds and materialise the best one as ready‐to-run code. This report distills the key insights, accomplishments, and impact.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

CFD-trained ANN Model for Approximating Near-occupant Condition in Real-time Simulations

The main drawback of Computational Fluid Dynamics (CFD) simulations has been the time and resource consuming nature which is not suitable for real-time applications. In this work, we first generated numerous CFD models of a given indoor space to obtain airspeed, temperature, and mean radiant temperature near an occupant as training data. Several artificial neural networks (ANN) models were trained using this CFD simulated data to approximate near real-time environmental conditions for a given occupant. This trained ANN model approach is a part of a real-time simulation of building operations using a combination of software and real hardware (HVAC equipment) approaches. The preliminary results suggest that the CFD- generated training data and the trained ANN model can accurately approximate such conditions in a real-time application, a method that has great potential in building simulation and building digital twin areas of research.

Zhang, Yun↗

DOE OSTI2022

Graph Neural Networks for Charged Particle Tracking on FPGAs

The determination of charged particle trajectories in collisions at the CERN Large Hadron Collider (LHC) is an important but challenging problem, especially in the high interaction density conditions expected during the future high-luminosity phase of the LHC (HL-LHC). Graph neural networks (GNNs) are a type of geometric deep learning algorithm that has successfully been applied to this task by embedding tracker data as a graph—nodes represent hits, while edges represent possible track segments—and classifying the edges as true or fake track segments. However, their study in hardware- or software-based trigger applications has been limited due to their large computational cost. In this paper, we introduce an automated translation workflow, integrated into a broader tool called hls4ml , for converting GNNs into firmware for field-programmable gate arrays (FPGAs). We use this translation tool to implement GNNs for charged particle tracking, trained using the TrackML challenge dataset, on FPGAs with designs targeting different graph sizes, task complexites, and latency/throughput requirements. This work could enable the inclusion of charged particle tracking GNNs at the trigger level for HL-LHC experiments.

Elabd, Abdelrahman↗

NASA NTRS1992

Learning a trajectory using adjoint functions and teacher forcing

A new methodology for faster supervised temporal learning in nonlinear neural networks is presented which builds upon the concept of adjoint operators to allow fast computation of the gradients of an error functional with respect to all parameters of the neural architecture, and exploits the concept of teacher forcing to incorporate information on the desired output into the activation dynamics. The importance of the initial or final time conditions for the adjoint equations is discussed. A new algorithm is presented in which the adjoint equations are solved simultaneously (i.e., forward in time) with the activation dynamics of the neural network. We also indicate how teacher forcing can be modulated in time as learning proceeds. The results obtained show that the learning time is reduced by one to two orders of magnitude with respect to previously published results, while trajectory tracking is significantly improved. The proposed methodology makes hardware implementation of temporal learning attractive for real-time applications.

Toomarian, Nikzad B.↗

DOE OSTI2020

Accelerating Radiation Computations for Dynamical Models With Targeted Machine Learning and Code Optimization

Abstract Atmospheric radiation is the main driver of weather and climate, yet due to a complicated absorption spectrum, the precise treatment of radiative transfer in numerical weather and climate models is computationally unfeasible. Radiation parameterizations need to maximize computational efficiency as well as accuracy, and for predicting the future climate many greenhouse gases need to be included. In this work, neural networks (NNs) were developed to replace the gas optics computations in a modern radiation scheme (RTE+RRTMGP) by using carefully constructed models and training data. The NNs, implemented in Fortran and utilizing BLAS for batched inference, are faster by a factor of 1–6, depending on the software and hardware platforms. We combined the accelerated gas optics with a refactored radiative transfer solver, resulting in clear‐sky longwave (shortwave) fluxes being 3.5 (1.8) faster to compute on an Intel platform. The accuracy, evaluated with benchmark line‐by‐line computations across a large range of atmospheric conditions, is very similar to the original scheme with errors in heating rates and top‐of‐atmosphere radiative forcings typically below 0.1 K day −1 and 0.5 W m −2 , respectively. These results show that targeted machine learning, code restructuring techniques, and the use of numerical libraries can yield material gains in efficiency while retaining accuracy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS1996

Spectral imaging applications: Remote sensing, environmental monitoring, medicine, military operations, factory automation and manufacturing

This paper reviews the activities at OKSI related to imaging spectroscopy presenting current and future applications of the technology. The authors discuss the development of several systems including hardware, signal processing, data classification algorithms and benchmarking techniques to determine algorithm performance. Signal processing for each application is tailored by incorporating the phenomenology appropriate to the process, into the algorithms. Pixel signatures are classified using techniques such as principal component analyses, generalized eigenvalue analysis and novel very fast neural network methods. The major hyperspectral imaging systems developed at OKSI include the Intelligent Missile Seeker (IMS) demonstration project for real-time target/decoy discrimination, and the Thermal InfraRed Imaging Spectrometer (TIRIS) for detection and tracking of toxic plumes and gases. In addition, systems for applications in medical photodiagnosis, manufacturing technology, and for crop monitoring are also under development.

National Defense↗

DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗

NASA NTRS2002

Evolutionary Design of a Robotic Material Defect Detection System

During the post-flight inspection of SSME engines, several inaccessible regions must be disassembled to inspect for defects such as cracks, scratches, gouges, etc. An improvement to the inspection process would be the design and development of very small robots capable of penetrating these inaccessible regions and detecting the defects. The goal of this research was to utilize an evolutionary design approach for the robotic detection of these types of defects. A simulation and visualization tool was developed prior to receiving the hardware as a development test bed. A small, commercial off-the-shelf (COTS) robot was selected from several candidates as the proof of concept robot. The basic approach to detect the defects was to utilize Cadmium Sulfide (CdS) sensors to detect changes in contrast of an illuminated surface. A neural network, optimally designed utilizing a genetic algorithm, was employed to detect the presence of the defects (cracks). By utilization of the COTS robot and US sensors, the research successfully demonstrated that an evolutionarily designed neural network can detect the presence of surface defects.

Ballard, Gary↗