Search NASA⌕ Search

SEARCH · Search NASA

Results for “machine learning, HPC”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

The Livermore Brain: Massive Deep Learning Networks Enabled by High Performance Computing

The proliferation of inexpensive sensor technologies like the ubiquitous digital image sensors has resulted in the collection and sharing of vast amounts of unsorted and unexploited raw data. Companies and governments who are able to collect and make sense of large datasets to help them make better decisions more rapidly will have a competitive advantage in the information era. Machine Learning technologies play a critical role for automating the data understanding process; however, to be maximally effective, useful intermediate representations of the data are required. These representations or “features” are transformations of the raw data into a form where patterns are more easily recognized. Recent breakthroughs in Deep Learning have made it possible to learn these features from large amounts of labeled data. The focus of this project is to develop and extend Deep Learning algorithms for learning features from vast amounts of unlabeled data and to develop the HPC neural network training platform to support the training of massive network models. This LDRD project succeeded in developing new unsupervised feature learning algorithms for images and video and created a scalable neural network training toolkit for HPC. Additionally, this LDRD helped create the world’s largest freely-available image and video dataset supporting open multimedia research and used this dataset for training our deep neural networks. This research helped LLNL capture several work-for-others (WFO) projects, attract new talent, and establish collaborations with leading academic and commercial partners. Finally, this project demonstrated the successful training of the largest unsupervised image neural network using HPC resources and helped establish LLNL leadership at the intersection of Machine Learning and HPC research.

97 MATHEMATICS AND COMPUTING↗

Strategies for Integrating Deep Learning Surrogate Models with HPC Simulation Applications

The emerging trend of the convergence of high performance computing (HPC), machine learning/deep learning (ML/DL), and big data analytics presents a host of challenges for large-scale computing campaigns that seek best practices to interleave traditional scientific simulation-based workloads with ML/DL models. A portfolio of systematic approaches to incorporate deep learning into modeling and simulation serves a vital need when we support AI for science at a computing facility. In this paper, we evaluate several strategies for deploying deep learning surrogate models in a representative physics application on supercomputers at the Oak Ridge Leadership Computing Facility (OLCF). We discuss a set of recommended deployment architectures and implementation approaches. We analyze and evaluate these alternatives and show their performance and scalability up to 1000 GPUs on two mainstream platforms equipped with different deep learning hardware and software stacks.

Yin, Junqi↗

Characterizing Machine Learning I/O Workloads on Leadership Scale HPC Systems

High performance computing (HPC) is no longer solely limited to traditional workloads such as simulation and modeling. With the increase in the popularity of machine learning (ML) and deep learning (DL) technologies, we are observing that an increasing number of HPC users are incorporating ML methods into their workflow and scientific discovery processes, across a wide spectrum of science domains such as biology, earth science, and physics. This gives rise to a diverse set of I/O patterns than the traditional checkpoint/restart-based HPC I/O behavior. The details of the I/O characteristics of such ML I/O workloads have not been studied extensively for large-scale leadership HPC systems. This paper aims to fill that gap by providing an in-depth analysis to gain an understanding of the I/O behavior of ML I/O workloads using darshan - an I/O characterization tool designed for lightweight tracing and profiling. We study the darshan logs of more than 23, 000 HPC ML I/O jobs over a time period of one year running on Summit - the second-fastest supercomputer in the world. This paper provides a systematic I/O characterization of ML I/O jobs running on a leadership scale supercomputer to understand how the I/O behavior differs across science domains and the scale of workloads, and analyze the usage of parallel file system and burst buffer by ML I/O workloads.

Paul, Arnab↗

Sparse Deep Neural Network Inference using different Programming Models

Sparse deep neural networks have gained increasing attention recently in achieving speedups on inference with reduced memory footprints. However, the real-world applications are little shown with specialized optimizations, yet a wide variety of DNN tasks remain dense without exploiting the advantages of sparsity in networks. Recent work presented by MIT/IEEE/Amazon GraphChallenge has demonstrated significant speedups and various techniques. Still, we find that there is limited investigation of the impact of various Python and C\slash C++ based programming models to explore new opportunities in general cases. In this work, we provide performance evaluation through different programming models using CuPy, cuSPARSE, and OpenMP to discuss the advantages and disadvantages of our sparse implementations on single-GPU and multiple GPUs of NVIDIA DGX-A100 40GB/80GB platforms.

machine learning, HPC↗

MetaStrider: Architectures for Scalable Memory-centric Reduction of Sparse Data Streams

Reduction is an operation performed on the values of two or more key-value pairs that share the same key. Reduction of sparse data streams finds application in a wide variety of domains such as data and graph analytics, cybersecurity, machine learning, and HPC applications. However, these applications exhibit low locality of reference, rendering traditional architectures and data representations inefficient. This article presents MetaStrider, a significant algorithmic and architectural enhancement to the state-of-the-art, SuperStrider. Furthermore, these enhancements enable a variety of parallel, memory-centric architectures that we propose, resulting in demonstrated performance that scales near-linearly with available memory-level parallelism.

97 MATHEMATICS AND COMPUTING↗

Integrating Distributed-Memory Machine Learning into Large-Scale HPC Simulations

Lawrence Livermore National Laboratory (LLNL) is a national lab based in Livermore, California. It was established in 1952 to meet increasing national security needs. They specialize in earth and atmospheric science, bioscience and bioengineering, lasers and optical science and technology, advanced materials and manufacturing, high-energy-density science, nuclear, chemical, and isotopic science and technology, and HPC, simulation and data science ((cor, 2018)). LLNL is also home to a number of large Linux clusters and supercomputers which are used to support research in their areas of specialization (mac, 2018). One purpose of the LLNL supercomputers is to run highly detailed and accurate physics simulations. These simulations allow scientists to get data on experiments that would be too expensive, difficult, or dangerous to perform in reality. With the help of simulations scientists at LLNL are able to study processes in more depth than they would be able to otherwise.

97 MATHEMATICS AND COMPUTING↗

Outcomes of HPC User Support using a Science Gateway AI Assistant

High Performance Computing (HPC) is a vital resource for nuclear energy research, facilitating advanced simulations and complex modeling of the quantification and qualification of advanced reactor technology. However, a common gap in knowledge exists around utilizing HPC systems, particularly for nuclear energy researchers unfamiliar with specific HPC systems. A researcher may be well-versed in using one HPC system and understanding its associated processes. Yet, they might struggle when faced with a different HPC system and its unique processes. HPC support staff play a crucial role in addressing these challenges by providing educational resources and assisting users. However, they also face the challenge of maintaining these systems and ensuring they run efficiently for all users, a responsibility that can be challenging to scale effectively with the increasing demand and expansion of HPC systems. This paper addresses this knowledge gap with an artificial intelligence (AI) assistant that offers on-demand, site-specific HPC support for researchers. Idaho National Laboratory (INL) has deployed an AI assistant that is intended to supplement expert HPC support staff and assist nuclear energy researchers. This paper reports on a four-and-a-half-month study evaluating the integration of an AI assistant within a science gateway, with the goal of enhancing existing HPC support.

97 MATHEMATICS AND COMPUTING↗

Using HPC for AI/ML

A presentation providing information on why using HPC for AI/ML is valuable as well as how INL HPC can be used to perform AI/ML at scale.

97 MATHEMATICS AND COMPUTING↗