Search NASA⌕ Search

SEARCH · Search NASA

Results for “Machine Learning Hardware”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Resource frugal optimizer for quantum machine learning

Quantum-enhanced data science, also known as quantum machine learning (QML), is of growing interest as an application of near-term quantum computers. Variational QML algorithms have the potential to solve practical problems on real hardware, particularly when involving quantum data. However, training these algorithms can be challenging and calls for tailored optimization procedures. Specifically, QML applications can require a large shot-count overhead due to the large datasets involved. In this work, we advocate for simultaneous random sampling over both the dataset as well as the measurement operators that define the loss function. We consider a highly general loss function that encompasses many QML applications, and we show how to construct an unbiased estimator of its gradient. This allows us to propose a shot-frugal gradient descent optimizer called Refoqus (REsource Frugal Optimizer for QUantum Stochastic gradient descent). Our numerics indicate that Refoqus can save several orders of magnitude in shot cost, even relative to optimizers that sample over measurement operators alone.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Real-time optimization of multi-cell industrial evaporative cooling towers using machine learning and particle swarm optimization

Existing electrical generating stations must operate with greater flexibility due to increasing renewable energy penetration on the electrical grid, and many coal-fired power stations have transitioned away from baseload operation to load-following operation to aid in grid stability. In cases where multiple independently controlled cooling tower cells are used in parallel for the cooling purposes of such stations, there is an opportunity to increase plant efficiency through data-driven optimization across their full load ranges. This work presents a novel application of real-time optimization using machine learning and particle swarm optimization on a multi-cell induced-draft cooling tower servicing a coal-fired power station under variable load. This is the first work to demonstrate simultaneous optimization of a multi-cell cooling tower, in addition to using machine learning for closed-loop control on a cooling tower. A novel control configuration is presented that ensures original control logic is not adversely affected and that the overall plant process is not disrupted using only existing hardware and operational data. To verify this methodology, the 12 independent cooling tower cells are simulated in parallel using historic operating data to demonstrate the effectiveness of real-time optimization compared to current practice. An artificial neural network is trained to predict overall cooling tower power consumption using only operational data and ambient conditions with an R2 value of greater than 0.96. The real-time optimization using particle swarm yields 6.7% annual energy usage savings compared to current practices, although the extent of the real-time savings varies greatly with both plant load and environmental conditions. This is particularly significant for a variable load situation because frequent ramping typically results in reduced overall efficiency. Furthermore, this proposed AI-based solution presents an opportunity to improve the overall heat rate of a load-following coal-fired power plant without the need to perform extensive first-principles modeling or add additional hardware to the cooling tower, resulting in more resources conserved and less overall emissions per unit of electricity generated.

42 ENGINEERING↗
DOE OSTI2016

FatMan vs. LittleBoy: Scaling Up Linear Algebraic Operations in Scale-Out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2× performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

FatMan vs. LittleBoy: Scaling up Linear Algebraic Operations in Scale-out Data Platforms

Linear algebraic operations such as matrix manipulations form the kernel of many machine learning and other crucial algorithms. Scaling up as well as scaling out such algorithms are highly desirable to enable efficient processing over millions of data points. To this end, we present a matrix manipulation approach to effectively scale-up each node in a scale-out data parallel platform such as Apache Spark. Specifically, we enable hardware acceleration for matrix multiplications in a distributed Spark setup without user intervention. Our approach supports both dense and sparse distributed matrices, and provides flexible control of acceleration by matrix density. We demonstrate the benefit of our approach for generalized matrix multiplication operations over large matrices with up to four billion elements. To connect the effectiveness of our approach with machine learning applications, we performed Gramian matrix computation via generalized matrix multiplications. Our experiments show that our approach achieves more than 2x performance speed-up, and up to 96.1% computation improvement, compared to a state of the art Spark MLlib for dense matrices.

Lim, Seung-Hwan↗
DOE OSTI2018

Fast inference of deep neural networks in FPGAs for particle physics

Here, recent results at the Large Hadron Collider (LHC) have pointed to enhanced physics capabilities through the improvement of the real-time event processing techniques. Machine learning methods are ubiquitous and have proven to be very powerful in LHC physics, and particle physics as a whole. However, exploration of the use of such techniques in low-latency, low-power FPGA (Field Programmable Gate Array) hardware has only just begun. FPGA-based trigger and data acquisition systems have extremely low, sub-microsecond latency requirements that are unique to particle physics. We present a case study for neural network inference in FPGAs focusing on a classifier for jet substructure which would enable, among many other physics scenarios, searches for new dark sector particles and novel measurements of the Higgs boson. While we focus on a specific example, the lessons are far-reaching. A companion compiler package for this work is developed based on High-Level Synthesis (HLS) called hls4ml to build machine learning models in FPGAs. The use of HLS increases accessibility across a broad user community and allows for a drastic decrease in firmware development time. We map out FPGA resource usage and latency versus neural network hyperparameters to identify the problems in particle physics that would benefit from performing neural network inference with FPGAs. For our example jet substructure model, we fit well within the available resources of modern FPGAs with a latency on the scale of 100 ns.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Fourier-based three-dimensional multistage transformer for aberration correction in multicellular specimens

High-resolution tissue imaging is often compromised by sample-induced optical aberrations that degrade resolution and contrast. Although wavefront sensor-based adaptive optics (AO) can measure these aberrations, such hardware solutions are typically complex, expensive to implement and slow when serially mapping spatially varying aberrations across large fields of view. Here we introduce AOViFT (adaptive optical vision Fourier transformer)—a machine learning-based aberration sensing framework built around a three-dimensional multistage vision transformer that operates on Fourier domain embeddings. AOViFT infers aberrations and restores diffraction-limited performance in puncta-labeled specimens with substantially reduced computational cost, training time and memory footprint compared to conventional architectures or real-space networks. We validated AOViFT on live gene-edited zebrafish embryos, demonstrating its ability to correct spatially varying aberrations using either a deformable mirror or postacquisition deconvolution. By eliminating the need for the guide star and wavefront sensing hardware and simplifying the experimental workflow, AOViFT lowers technical barriers for high-resolution volumetric microscopy across diverse biological samples.

Alshaabi, Thayer [Howard Hughes Medical Institute,↗
DOE OSTI2023

Comparison of Machine Learning Algorithms for Natural Gas Identification with Mixed Potential Electrochemical Sensor Arrays

Mixed-potential electrochemical sensor arrays consisting of indium tin oxide (ITO), La 0.87 Sr 0.13 CrO 3 , Au, and Pt electrodes can detect the leaks from natural gas infrastructure. Algorithms are needed to correctly identify natural gas sources from background natural and anthropogenic sources such as wetlands or agriculture. We report for the first time a comparison of several machine learning methods for mixture identification in the context of natural gas emissions monitoring by mixed potential sensor arrays. Random Forest, Artificial Neural Network, and Nearest Neighbor methods successfully classified air mixtures containing only CH 4 , two types of natural gas simulants, and CH 4 +NH 3 with >98% identification accuracy. The model complexity of these methods were optimized and the degree of robustness against overfitting was determined. Finally, these methods are benchmarked on both desktop PC and single-board computer hardware to simulate their application in a portable internet-of-things sensor package. The combined results show that the random forest method is the preferred method for mixture identification with its high accuracy (>98%), robustness against overfitting with increasing model complexity, and had less than 10 ms training time and less than 0.1 ms inference time on single-board computer hardware.

03 NATURAL GAS↗
DOE OSTI2020

miniGAN: a proxy application for generative adversarial networks

miniGAN is a python-based machine learning proxy application for generative adversarial networks, developed through the Exascale Computing Project's (ECP) ExaLearn project. It will be included in the main ECP proxy application and the machine learning proxy application suite. It is a proxy for ECP cosmological(CosmoFlow, ExaGAN) and wind energy(ExaWind) applications. miniGAN will be distributed to ECP hardware vendors as part of hardware codesign. miniGAN uses the Numpy/PyTorch/TensorFlow/Keras/Horovod frameworks and libraries. It also relies on the Kokkos and Kokkos-Kernels packages developed here at Sandia Labs. SAND2020-2038 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Ellis, John↗
DOE OSTI2019

Through the Windshield Driver Recognition

Biometric recognition of vehicle occupants in unconstrained environments is rife with a host of challenges. In particular, the complications arising from imaging through vehicle windshields provide a significant hurdle. Distance to target, glare, poor lighting, head pose of occupants, and speed of vehicle are some of the challenges. We explore the construction of a multi-unit computational camera system to mitigate these challenges in order to obtain accurate and consistent face recognition results. This paper documents the hardware components and software design of the computational imaging system. Also, we document the use of RCNN for face detection and GAN for machine learning-inspired High Dynamic Range Imaging, artifact removal, and image fusion.

Cornett, David↗
DOE OSTI2024

Intelligent Pixel Detectors: Towards a Radiation Hard ASIC with On-Chip Machine Learning in 28 nm CMOS

Detectors at future high energy colliders will face enormous technical challenges. Disentangling the unprecedented numbers of particles expected in each event will require highly granular silicon pixel detectors with billions of readout channels. With event rates as high as 40 MHz, these detectors will generate petabytes of data per second. To enable discovery within strict bandwidth and latency constraints, future trackers must be capable of fast, power efficient, and radiation hard data-reduction at the source. We are developing a radiation hard readout integrated circuit (ROIC) in 28nm CMOS with on-chip machine learning (ML) for future intelligent pixel detectors. We will show track parameter predictions using a neural network within a single layer of silicon and hardware tests on the first tape-outs produced with TSMC. Preliminary results indicate that reading out featurized clusters from particles above a modest momentum threshold could enable using pixel information at 40 MHz.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Towards Automatic and Agile AI/ML Accelerator Design with End-to-End Synthesis

Domain-specific designs offer greater energy efficiency and performance gain than general-purpose processors. For this reason, modern system-on-chips have a significant portion of their silicon area with custom accelerators. However, designing hardware by hand is laborious and time-consuming, given the large design space and the performance, power, and area constraints that are not realized in the software. Moreover, domain-specific algorithms (e.g., machine learning models) are evolving quickly, challenging the accelerator design further. To address these issues, this paper presents SODA Synthesizer, an automated open-source high-level ML framework to Verilog modular compiler targeting AI/ML Application-Specific Integrated Circuits (ASICs) accelerators. SODA tightly couples the Multi- Level Intermediate Representation (MLIR) compiler infrastructure [24] and open-source HLS approaches. Thus, SODA can support various ML frameworks and algorithms and can perform optimizations that combine specialized architecture templates and conventional HLS to generate the hardware modules. In addition, SODA’s closed-loop design space exploration (DSE) engine allows developers to perform end-to-end design space explorations on different metrics and technology nodes.

Zhang, Jeff↗
DOE OSTI2017

Scaling Deep Learning on GPU and Knights Landing clusters

The speed of deep neural networks training has become a big bottleneck of deep learning research and development. For example, training GoogleNet by ImageNet dataset on one Nvidia K20 GPU needs 21 days. To speed up the training process, the current deep learning systems heavily rely on the hardware accelerators. However, these accelerators have limited on-chip memory compared with CPUs. To handle large datasets, they need to fetch data from either CPU memory or remote processors. We use both self-hosted Intel Knights Landing (KNL) clusters and multi-GPU clusters as our target platforms. From an algorithm aspect, current distributed machine learning systems are mainly designed for cloud systems. These methods are asynchronous because of the slow network and high fault-tolerance requirement on cloud systems. We focus on Elastic Averaging SGD (EASGD) to design algorithms for HPC clusters. Original EASGD used round-robin method for communication and updating. The communication is ordered by the machine rank ID, which is inefficient on HPC clusters. First, we redesign four efficient algorithms for HPC systems to improve EASGD's poor scaling on clusters. Async EASGD, Async MEASGD, and Hogwild EASGD are faster \textcolor{black}{than} their existing counterparts (Async SGD, Async MSGD, and Hogwild SGD, resp.) in all the comparisons. Finally, we design Sync EASGD, which ties for the best performance among all the methods while being deterministic. In addition to the algorithmic improvements, we use some system-algorithm codesign techniques to scale up the algorithms. By reducing the percentage of communication from 87% to 14%, our Sync EASGD achieves 5.3x speedup over original EASGD on the same platform. We get 91.5% weak scaling efficiency on 4253 KNL cores, which is higher than the state-of-the-art implementation.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

High-Resolution Imaging of Unstained Polymer Materials

Electron microscopy has played an important role in polymer characterization. Traditionally, electron diffraction is used to study crystalline polymers while transmission electron microscopy is used to study microphase separation in stained block copolymers and other multiphase systems. We describe developments that eliminate the barrier between these two approaches - it is now possible to image polymer crystals with atomic resolution. The focus of this Review is on high-resolution imaging (30 Å and smaller) of unstained polymers. Recent advances in hardware allow for capturing numerous (as many as 105) low-dose images from an unperturbed specimen; beam damage is a significant barrier to high-resolution electron microscopy of polymers. Machine-learning-based software is then used to sort and average the images to retrieve pristine structural information from a collection of noisy images. Acknowledging the heterogeneity in polymer samples prior to averaging is essential. Molecular conformations in a wide range of amphiphilic block copolymers, polymerized ionic liquids, and conjugated polymers can be gleaned from two-dimensional projections (2D), three-dimensional (3D) tomograms, and four-dimensional (4D) scanning transmission electron microscopy (STEM) data sets where 2D diffraction patterns are taken as a function of position. Some methods such as phase contrast STEM have been used to image closely related materials such as metal-organic frameworks but not polymers. With improvements in hardware and software, such methods may soon be applied to polymers. Our goal is to provide a comprehensive understanding of the strategies toward the high-resolution imaging of radiation sensitive polymer materials at different length scales.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Tournament-Based Pretraining to Accelerate Federated Learning

Advances in hardware, proliferation of compute at the edge, and data creation at unprecedented scales have made federated learning (FL) necessary for the next leap forward in pervasive machine learning. For privacy and network reasons, large volumes of data remain stranded on endpoints located in geographically austere (or at least austere network-wise) locations. However, challenges exist to the effective use of these data. To solve the system and functional level challenges, we present an three novel variants of a serverless federated learning framework. We also present tournament-based pretraining, which we demonstrate significantly improves model performance in some experiments. Overall, these extensions to FL and our novel training method enable greater focus on science rather than ML development.

Baughman, Matt↗
DOE OSTI2018

Open Source Software for HPC

The computational power of HPC is beyond our comprehension when we hear that 5 quadrillion computations can happen in a matter of seconds, or that machine learning is changing the way everything works. But none of that happens in a vacuum, and the teams behind the scenes—the developers of the hardware, the operating systems, the data transfer protocols, and the applications themselves—are the unsung heroes of a world where faster is better and you'd better hope there's no bug in the software or the hardware to slow you down. HPC is most successful when all these aspects work together seamlessly. The stories that follow are a tribute to the hardworking teams behind the scenes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, heterogeneous quantum-probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability.

Mohseni, Masoud↗
DOE OSTI2023

SABATH: Surrogate AI Benchmarking Applications' Testing Harness

SABATH provides benchmarking infrastructure for evaluating scientific ML/AI models. It contains support for scientific machine learning surrogates from external repositories. The software dependences are explicitly exposed in the surrogate model definition, which allows the use of advanced optimization, communication, and hardware features.

Luszczek, Piotr [Univ. of Tennessee, Knoxville, TN↗