Search NASA⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 3396728

GLANCED-IO: Taming IO Optimization for Deep Learning at Scale

Hariharan Devarajan, Hari [Lawrence Livermore National Laboratory]·Nixon, William [University of Chicago]·Carns, Philip [Argonne National Laboratory]·Zheng, Huihuo [Argonne National Laboratory]·Madireddy, Sandeep [Argonne National Laboratory]·Foreman, Sam [Argonne National Laboratory]·Arcomano, Troy [Allen Institute for AI]·Ross, Robert [Argonne National Laboratory]·Gunawi, Haryadi [University of Chicago]·Sinurat, Ray
View original source ↗Read source document ↗Export metadata ↓

Abstract

The source did not provide an abstract. Follow the original record for more information.

Explore related subjects

Computer science

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Hariharan Devarajan, Hari [Lawrence Livermore National Laboratory], Nixon, William [University of Chicago], Carns, Philip [Argonne National Laboratory], Zheng, Huihuo [Argonne National Laboratory], Madireddy, Sandeep [Argonne National Laboratory], Foreman, Sam [Argonne National Laboratory], Arcomano, Troy [Allen Institute for AI], Ross, Robert [Argonne National Laboratory], Gunawi, Haryadi [University of Chicago], Sinurat, Ray. 2026-08-02. GLANCED-IO: Taming IO Optimization for Deep Learning at Scale. https://doi.org/10.2172/3396728

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2026

Expanding Access to Science Participation: A FAIR Framework for Petascale Data Visualization and Analytics

The massive data generated by scientists daily serve as both a major catalyst for new discoveries and innovations, as well as a significant roadblock that restricts access to the data. Here, our paper introduces a new approach to removing Big Data barriers and democratizing access to petascale data for the broader scientific community. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hiding the complexities of dealing with file systems or cloud services. We enable FAIR (Findable, Accessible, Interoperable, and Reusable) access to datasets such as NASA’s petascale climate datasets. Our paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction utilizes state-of-the art progressive compression algorithms and machine-learning insights to power scalable visualization dashboards for petascale data. The result provides users with the ability to identify extreme events or trends dynamically, expanding access to scientific data and further enabling discoveries. We validate our approach by improving the ability of climate scientists to visually explore their data via three fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution. These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Computer science↗