Search NASA⌕ Search

SEARCH · Search NASA

Results for “HPC visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The Exploitation of Data Reduction for Visualization

The disparity between the computational speed and storage bandwidth, as demonstrated in Figure 1, is a well known problem that grows with each successive generation. The visualization community is principally responding to this issue by using in situ to reduce which data must be written to storage. However, other communities are taking different, possibly complementary approaches. In particular, data compression is a common general approach to reduce storage demands. Data compression technologies are typically not designed with post processing in mind. The principal metrics measured are compression ratio, the improved bandwidth to storage, and the error introduced. It is assumed that data is inflated to its full size before any post processing can happen. Although when talking about bandwidth disparities, HPC’s dirty little secret is that no part of the memory nor interconnect hardware is increasing at the rate of computation. For example, the Summit supercomputer has a peak computation rate almost 10 times its predecessor, Titan, but only about 4 times the memory, less than twice the aggregate memory bandwidth, and almost no improvement in the interconnect bisection bandwidth. Naively inflating data for post processing does not help with limitations in the memory and interconnect systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

The primary challenge motivating this team’s work is the widening gap between the ability to compute information and to store it for subsequent analysis. This gap adversely impacts science code teams, who are able to perform analysis only on a small fraction of the data they compute, resulting in the very real likelihood of lost or missed science, when results are computed but not analyzed. Our approach is to perform as much analysis or visualization processing on data while it is still resident in memory, an approach that is known as in situ processing. The idea in situ processing was not new at the time of the start of this effort in 2014, but efforts in that space were largely ad hoc, and there was no concerted effort within the research community that aimed to foster production-quality software tools suitable for use by DOE science projects. In large, our objective was produce and enable use of production-quality in situ methods and infrastructure, at scale, on DOE HPC facilities, though we expected to have impact beyond DOE due to the widespread nature of the challenges, which affect virtually all large-scale computational science efforts. To achieve that objective, we assembled a unique team of researchers consisting of representatives from DOE national laboratories, academia, and industry, and engaged in software technology R&D, as well as engaged in close partnerships with DOE science code teams, to produce software technologies that were shown to run effectively at scale on DOE HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

SpotSDC: Revealing the Silent Data Corruption Propagation in High-Performance Computing Systems

We report the trend of rapid technology scaling is expected to make the hardware of high-performance computing (HPC) systems more susceptible to computational errors due to random bit flips. Some bit flips may cause a program to crash or have a minimal effect on the output, but others may lead to silent data corruption (SDC), i.e., undetected yet significant output errors. Classical fault injection analysis methods employ uniform sampling of random bit flips during program execution to derive a statistical resiliency profile. However, summarizing such fault injection result with sufficient detail is difficult, and understanding the behavior of the fault-corrupted program is still a challenge. In this article, we introduce SpotSDC, a visualization system to facilitate the analysis of a program's resilience to SDC. SpotSDC provides multiple perspectives at various levels of detail of the impact on the output relative to where in the source code the flipped bit occurs, which bit is flipped, and when during the execution it happens. SpotSDC also enables users to study the code protection and provide new insights to understand the behavior of a fault-injected program. Based on lessons learned, we demonstrate how what we found can improve the fault injection campaign method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

A Divergence Statistics Extension to VTK for Performance Analysis

This report follows the series of previous documents ([PT08, BPRT09b, PT09, BPT09, PT10, PB13], where we presented the parallel descriptive, correlative, multi-correlative, principal component analysis, contingency, k -means, order and auto-correlative statistics engines which we developed within the Visualization Tool Kit ( VTK ) as a scalable, parallel and versatile statistics package. We now report on a new engine which we developed for the calculation of divergence statistics, a concept which we hereafter explain and whose main goal is to quantify the discrepancy, in a stasticial manner akin to measuring a distance, between an observed empirical distribution and a theoretical, "ideal" one. The ease of use of the new diverence statistics engine is illustrated by the means of C++ code snippets. Although this new engine does not yet have a parallel implementation, it has already been applied to HPC performance analysis, of which we provide an example.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

HDF5 As a Vehicle for in Transit Data Movement

For in transit processing, one of the fundamental challenges is the efficient movement of data from producers to consumers. Exploiting the flexibility offered by the SENSEI generic in situ framework, we have developed a number of different in transit data transport mechanisms. In this work, we focus on the transport mechanism that leverages the HDF5 parallel I/O library, and investigate the performance characteristics of this transport mechanism. For in transit use cases at scale on HPC platforms, one might expect that an in transit data transport mechanism that uses faster layers of the storage hierarchy, such as DRAM memory, would always outperform a transport that uses slower layers of the storage hierarchy, such as an NVRAM-based persistent storage presented as a distributed file system. However, our test results show that the performance of the transport using NVRAM is competitive with the transport that uses socket-based data movement across varying levels of producer and consumer concurrency.}, booktitle = {Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Gu, Junmin↗
DOE OSTI2022

VisMetHack2022: Visualizing winds and surface variables from the ECMWF IFS 1-km nature run

This data collection was contributed to the Visualisation Hackathon 2022 (#VisMetHack2022), in conjunction with the Using ECMWF's Forecasts (UEF2022) workshop. The European Center for Medium-Range Weather Forecasts (ECMWF) and the Oak Ridge National Laboratory (ORNL) are pleased to announce access to the data collection from global 1-km nature run (NR) simulations using the Integrated Forecast System (IFS) with explicit convection. We invite you to join us in exploring this precursor to a digital twin of the earth! The NR simulations reveal unprecedented detail of the earth’s atmosphere, and the then outgoing Editor-in-Chief of AGU JAMES commended the project as one of “stunning ambitions,” enabled by computational capacity at scale. The project also won the 2020 HPCwire Readers Choice Award for Best Use of HPC in Physical Sciences. A set of two NR seasonal simulations have been completed, one corresponding to the northern hemispheric winter months (NDJF) and the other for the North Atlantic tropical cyclone season (ASO). The project used the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). The simulations were facilitated with an INCITE award from the US Department of Energy Office of Science. For the first seasonal run of four months (NDJF), the hydrostatic IFS model was initialized at 00Z on 1 November 2018. The NR for the TC season (AS) was initialized at 00Z on 1 August 2019. The NR simulations were constrained only by sea surface temperatures (SST) at the lower boundary. The IFS output was saved every 3 hours. After feedback and interest from the scientific community, the simulations were rerun for four specific extreme events, with output every 15 minutes. The special cases include a tropical cycle and three severe storm events over the continental USA.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Final Report Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

The primary challenge motivating this project is the widening gap between the ability to compute information and to store it for subsequent analysis. This gap adversely impacts science code teams, who can perform analysis only on a small fraction of the data they calculate, resulting in the substantial likelihood of lost or missed science, when results are computed but not analyzed. Our approach is to perform as much analysis or visualization processing on data while it is still resident in memory, which is known as in situ processing. The idea in situ processing was not new at the time of the start of this effort in 2014, but efforts in that space were largely ad hoc, and there was no concerted effort within the research community that aimed to foster production-quality software tools suitable for use by Department of Energy (DOE) science projects. Our objective was to produce and enable the use of production-quality in situ methods and infrastructure, at scale, on DOE high-performance computing (HPC) facilities, though we expected to have an impact beyond DOE due to the widespread nature of the challenges, which affect virtually all large-scale computational science efforts. To achieve this objective, we engaged in software technology research and development (R&D), in close partnerships with DOE science code teams, to produce software technologies that were shown to run efficiently at scale on DOE HPC platforms.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2019

Sensitivity and Uncertainty Workflow of Full System SIERRA Models Supporting High Consequence Applications

Credibility of end-to-end CompSim (Computational Simulation) models and their agile execution requires an expressive framework to describe, communicate and execute complex computational tool chains representing the model. All stakeholders from system engineering and customers through model developers and V&V partners need views and functionalities of the workflow representing the model in a manner that is natural to their discipline. In the milestone and in this report we define workflow as a network of computation simulation activities executed autonomously on a distributed set of computational platforms. The FY19 ASC L2 Milestone (6802) for the Integrated Workflow (IWF) project was designed to integrate and improve existing capabilities or develop new functionalities to provide a wide range of stakeholders a coherent and intuitive platform capable of defining and executing CompSim modeling from analysis workflow definition to complex ensemble calculations. The main goal of the milestone was to advance the integrated workflow capabilities to support the weapon system analysts with a production deployment in FY20. Ensemble calculations supporting program decisions include sensitivity analysis, optimization and uncertainty quantification. The goal of the L2 milestone aligned with the ultimate goal of the IWF project is to foster cultural and technical shift toward and integrated CompSim capability based on automated workflows. Specific deliverables were defined in five broad categories: 1) Infrastructure, including development of distributed-computing workflow capability, 2) integration of Dakota (Sandia's sensitivity, optimization and UQ engine) with SAW (Sandia Analysis Workbench), 3) ARG (Automatic Report Generator introspecting analysis artifacts and generating human-readable extensible and archivable reports), 4) Libraries and Repositories aiding capability reuse, and 5) Exemplars to support training, capturing best practices and stress testing of the platform. A set of exemplars was defined to represent typical weapon system qualification CompSim projects. Analyzing the required capabilities and using the findings to plan implementation of required capabilities ensured optimal allocation of development resources focused on production deployment after the L2 is completed. It was recognized early that the end-to-end modeling applications pose a considerable number of diverse risks, and a formal risk tracking process was implemented. The project leveraged products, capabilities and development tasks of IWF partners. SAW, Dakota, Cubit, Sierra, Slycat, and NGA (NexGen Analytics, a small business) contributed to the integrated platform developed during this milestone effort. New products delivered include: a) NGW (Next Generation Workflow) for robust workflow definition and execution, b) Dakota wizards, editor and results visualization, and c) the automatic report generator ARG. User engagement was initiated early in the development process eliciting concrete requirements and actionable feedback to assure that the integrated CompSim capability will have high user acceptance and impact. The current integrated capabilities have been demonstrated and are continually being tested by a set of exemplars ranging from training scenarios to computationally demanding uncertainty analyses. The integrated workflow platform has been deployed on both SRN (Sandia Restricted Network) and SCN (Sandia Classified Network). Computational platforms where the system has been demonstrated span from Windows (Creo the CAD platform chosen by Sandia) to Trinity HPC (Sierra and CTH solvers). Follow up work will focus on deployment at SNL and other sites in the nuclear enterprise (LLNL, KCNSC), training and consulting support to democratize the analysis agility, process health and knowledge management benefits the NGW platform provides.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

DISTRI: Distributed Multi-Facility HPC Simulator (DISTRI) v2.1

DISTRI is an advanced network simulator designed for multi-facility computational infrastructures with agentic behavior. It simulates HPC facilities where computational resources act as autonomous agents, making intelligent decisions about job scheduling, load balancing, and resource allocation. The simulator focuses on developing and testing decentralized algorithms that promote resilience and efficiency in multi-facility environments. Key Features: - Agentic Resource Behavior: Processors and DTNs act as autonomous agents with decision-making capabilities - Pheromone-Based Load Balancing: Decentralized load balancing inspired by ant colony optimization - Dual Topology Support: Mesh (normal operations) and Dumbell (network testing) topologies - Comprehensive TCP Simulation: Realistic TCP implementations with multiple congestion control algorithms - Failure Resilience Testing: Processor failure simulation with automatic job reassignment - Extensive Visualization: Detailed performance analysis and metrics collection - Research-Ready: Designed for algorithm development and benchmarking

Bez, Jean Luca [Lawrence Berkeley National Laborat↗
DOE OSTI2023

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

EVs@Scale Next-Gen Profiles - Fleet Utilization 2023

As U.S. fleet operators begin transitioning to electric vehicles (EVs), critical questions arise regarding how to manage this shift without disrupting fleet operations or placing undue stress on the electric grid. A major challenge for fleets is maintaining effective operational schedules while accommodating charging requirements, particularly with high-power charging (HPC) infrastructure, which presents grid stability concerns for utilities. Proposed solutions such as charging substations, megawatt charging systems (MCS), and smart charge management systems (SCMS) offer potential pathways forward, but their effectiveness depends on alignment with real-world fleet behavior and operational constraints. This report investigates the charging and utilization behavior of EV and EVSE fleets actively employing HPC technologies by conducting detailed case study analyses based on telematics data. A suite of predefined metrics—covering charging, routing, and other operational behaviors—is developed to evaluate the impact of fleet activities on grid infrastructure and identify opportunities for optimization. Results highlight variations in charging behavior across fleets, such as weekday versus weekend usage, diurnal charging trends, and the role of operational predictability in enabling SCMS effectiveness. While SCMS can help lower costs and improve energy efficiency for fleets with stable schedules, they may be insufficient for fleets with highly variable or long-haul operations, which may require more robust solutions like MCS. Visualization of aggregated hourly energy metrics reveals that while fleet behaviors are diverse, there are common temporal patterns that could inform infrastructure planning and energy management. These insights emphasize the need for fleet-specific charging strategies that minimize grid impact while supporting reliable fleet operations. Additionally, the report underscores the broader economic stakes of electrification, particularly in high-value markets such as freight, where misaligned transitions could stall EV adoption. By examining current EV and EVSE fleet deployments using predetermined standardized metrics, this study offers a foundation for developing technologies and operational frameworks that support scalable, grid-compatible electrification across a variety of fleet types while establishing a baseline understanding of operational behaviors. In doing so, we aim to ensure that future charging solutions reflect actual fleet needs and grid constraints—an essential step toward maintaining operational continuity and achieving a successful transition to electric fleet operations.

Charging↗
DOE OSTI2023

Data Center Facility Monitoring with Physics Aware Approach

U.S. Department of Energy's National Renewable Energy Laboratory (NREL) hosts one of the world's most energy-efficient HPC data centers; this system uses component-level warm-water liquid cooling to efficiently remove heat from the data center and capture it for reuse in the building or rejection to the atmosphere. Given the complexity of this system, building data-driven tools for holistically monitoring and operating the entire data center is a priority for ensuring maximal efficiency and resiliency. In this advanced smart facility, over one million metrics are recorded per minute using state-of-the-art streaming data architecture and software to capture and process the state of the system in real time. Here we detail two efforts to effectively analyze, visualize, and interpret this large volume streaming data. We have developed a novel, flexible system for identifying and visualizing individual metric anomalies and component performance across the data center through automatic metadata extraction and physically-motivated visualization for quick interpretation. Additionally, to directly connect system maintenance to data stream processing we explore a physics informed multi-metric drift and anomaly detection application to detect scale-build up in heat exchangers.

anomaly detection↗
DOE OSTI2025

HPC Campaign Management: Remote data access with user-defined error bound using ADIOS and ZFP

Remote access to large-scale scientific datasets, like those generated by combustion simulations or other high-performance computing (HPC) applications, presents a significant challenge. Downloading entire datasets is often impractical due to their size and the bandwidth limitations of typical networks. To address this challenge, we propose a novel approach that enables efficient remote access to large datasets distributed across multiple facilities. Our method enables technologies to download only the data values of a select variable, in a select region of interest, to a user-defined accuracy. For this purpose, we extended the ADIOS IO library to provide read functions with user-defined accuracy, a remote data server that understands multidimensional selections of specific variables, steps and accuracy from an ADIOS dataset, and which uses lossy compression on the remote site to reduce the data to be transferred back to the client. In addition, our extension of the ADIOS library collects metadata from multiple datasets in small files called Campaign Archives, which can be shared among project participants on any HPC, cloud or laptop, and which can easily facilitate the discovery of content and pointers to the data location as well as remote access to the data by local tools as if data was local. This feature called Campaign Management, enables a group of scientists to manage related datasets stored in multiple files, across multiple facilities as if it was in a single file/database. We demonstrate the effectiveness of our approach using a 1.5 TB dataset from the S3D combustion simulation on Frontier at the Oak Ridge Leadership Facility. Even a single variable from this dataset, at 64 GB, is too large to be processed on a standard laptop. We show two different reading patterns for 2D plots and 3D visualization, with careful settings that a scientist studying combustion data would do and show that running the same Python scripts on Frontier directly takes comparable time than running them on the local laptop with remote access to the data on Frontier.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗
DOE OSTI2023

High Performance Computing Facility Operational Assessment 2022: Oak Ridge Leadership Computing Facility

The Oak Ridge Leadership Computing Facility (OLCF) was established to accelerate scientific discovery by providing world-leading computational performance and advanced data infrastructure. As a US Department of Energy (DOE) Office of Science user facility, the OLCF has managed the successful deployment and operation of a succession of leadership-class resources dedicated to open science. In addition to these resources, the OLCF staff continually strive to develop innovative processes and technologies, improve security, and empower users through allocation management and comprehensive user support and training. These efforts support the advancement of science by the OLCF users and benefit high-performance computing (HPC) facilities around the world. In calendar year (CY) 2022, the OLCF supported 1,681 users and 570 projects and exceeded all targets for user satisfaction. The facility received an average satisfaction score of 4.6 out of 5 on the annual user survey, and 96% of respondents reported a high satisfaction rate with the OLCF overall. Of the 3,212 user tickets submitted in CY 2022, OLCF staff resolved 97% within 3 business days. The facility also introduced several new services for users this year, including weekly virtual office hours with subject matter experts from ORNL and vendor partners; new views in MyOLCF that allow users to analyze allocation and compute usage for a project; the ability to build and run containers on Summit; and improved data visualization support and training resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Improving Performance of M-to-N Processing and Data Redistribution in In Transit Analysis and Visualization

In an in transit setting, a parallel data producer, such as a numerical simulation, runs on one set of ranks M, while a data consumer, such as a parallel visualization application, runs on a different set of ranks N. One of the central challenges in this in transit setting is to determine the mapping of data from the set of M producer ranks to the set of N consumer ranks. This is a challenging problem for several reasons, such as the producer and consumer codes potentially having different scaling characteristics and different data models. The resulting mapping from M to N ranks can have a significant impact on aggregate application performance. In this work, we present an approach for performing this M-to-N mapping in a way that has broad applicability across a diversity of data producer and consumer applications. We evaluate its design and performance with a study that runs at high concurrency on a modern HPC platform. By leveraging design characteristics, which facilitate an “intelligent” mapping from M-to-N, we observe significant performance gains are possible in terms of several different metrics, including time-to-solution and amount of data moved.

Loring, Burlen↗
DOE OSTI2020

Improving Performance of M-to-N Processing and Data Redistribution in In Transit Analysis and Visualization

In an in transit setting, a parallel data producer, such as a numerical simulation, runs on one set of ranks M, while a data consumer, such as a parallel visualization application, runs on a different set of ranks N: One of the central challenges in this in transit setting is to determine the mapping of data from the set of M producer ranks to the set of N consumer ranks. This is a challenging problem for several reasons, such as the producer and consumer codes potentially having different scaling characteristics and different data models. The resulting mapping from M to N ranks can have a significant impact on aggregate application performance. In this work, we present an approach for performing this M-to-N mapping in a way that has broad applicability across a diversity of data producer and consumer applications. We evaluate its design and performance with a study that runs at high concurrency on a modern HPC platform. By leveraging design characteristics, which facilitate an ''intelligent'' mapping from M-to-N, we observe significant performance gains are possible in terms of several different metrics, including time-to-solution and amount of data moved.

Loring, Burlen↗
DOE OSTI2024

MemFriend: Understanding Memory Performance with Spatial-Temporal Affinity

In HPC applications, memory access behavior is one of the main factors affecting performance. Improving an application’s memory access behavior involves optimizing data layout and/or restructuring code, and requires studying spatial-temporal data locality. Existing data locality analyses focus on single-location metrics and are restricted to evaluating temporal locality. We introduce spatial-temporal affinity metrics that quantify temporal access proximity, forward access correlation, and nearby access correlation between pairs of memory locations. We describe methods for distinguishing between potential vs. realized affinity and for reasoning about affinity at multiple resolutions (3D, 2D, 1D). Finally, we construct spatial-temporal affinity signatures that classify memory behavior and that be used to reason about changes in software (data relayout, code refactoring) or hardware (caching, prefetching). We describe methods for signature visualization, interpretation, and quantitative comparison of signatures. We evaluate our methodology using applications with variants that contrast data structures, data layouts and algorithms. We show that spatial-temporal affinity analysis provides novel insights and enables predictive reasoning about application performance when contrasted with reuse distance analysis.

Suriyakumar, Yasodhadevi↗
DOE OSTI2017

Heat Transfer Efficiency in Various Prandtl Number Turbulent Flows using DNS approach

Convective heat transfer in turbulent flows utilized in many nuclear reactor designs with various coolants. While the traditional light water reactors (LWR) rely on water coolant, some advanced next-generation designs propose to use gas, molten salt, liquid metal coolants. Due to large presence of LWRs, most of the convective heat transfer correlations are focused on fluids with properties similar to water. However, some of the next-generation reactor designs cannot directly apply those correlations to advanced coolants, such as liquid metal, which has very low Prandtl number. Direct Numerical Simulation (DNS) is one of the promising approaches to study the details of fluid flow and heat transfer using first principles. The advances of high performance computing (HPC) in recent years made it possible to apply DNS to a wide variety of flow conditions. Kim and Moin studied heat transfer in fully developed turbulent channel flow using DNS. Kawamura et al. investigated the turbulent heat transfer in channel flow with low to medium-high Prandtl number fluid (ranging between 0.025 and 5.0). They take Reynolds number effects into consideration as well. However, most of these studies focus on medium and high Prandtl number. Low Prandtl number flow (like liquid metal) DNS studies are very limited. In the presented research the effect of low Prandtl number is studied in the turbulent channel flow. The second-order accurate spatial and time discretization is used for the flow solver in the presented work. The instantaneous velocity and temperature fields are visualized in the simulation. The effect of Prandtl number on the heat transfer is investigated by comparing different turbulent flow cases. In the same hydrodynamic conditions, the heat conduction enhances as the Prandtl number decreases in the low Prandtl number range. The behavior of the thermal boundary layer follows the empirical correlation. As the Prandtl number decreases, the conductive sublayer region becomes wider, where is out of the range of the empirical correlation. In the future work, we plan to estimate the Nusselt number as well to compare the total heat transfer rate with respect to different Prandtl number flow, especially the very low Prandtl number cases.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗