Search NASA⌕ Search

SEARCH · Search NASA

Results for “HPC visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Slycat™ User Manual

Slycat™ is a web-based system for performing data analysis and visualization of potentially large quantities of remote, high-dimensional data. Slycat™ specializes in working with ensemble data. An ensemble is a group of related data sets, which typically consists of a set of simulation runs exploring the same problem space. An ensemble can be thought of as a set of samples within a multi-variate domain, where each sample is a vector whose value defines a point in high-dimensional space. To understand and describe the underlying problem being modeled in the simulations, ensemble analysis looks for shared behaviors and common features across the group of runs. Additionally, ensemble analysis tries to quantify differences found in any members that deviate from the rest of the group. The Slycat™ system integrates data management, scalable analysis, and visualization. Results are viewed remotely on a user’s desktop via commodity web clients using a multi-tiered hierarchy of computation and data storage, as shown in Figure 1. Our goal is to operate on data as close to the source as possible, thereby reducing time and storage costs associated with data movement. Consequently, we are working to develop parallel analysis capabilities that operate on High Performance Computing (HPC) platforms, to explore approaches for reducing data size, and to implement strategies for staging computation across the Slycat™ hierarchy. Within Slycat™, data and visual analysis are organized around projects, which are shared by a project team. Project members are explicitly added, each with a designated set of permissions. Although users sign-in to access Slycat™, individual accounts are not maintained. Instead, authentication is used to determine project access. Within projects, Slycat™ models capture analysis results and enable data exploration through various visual representations. Although for scientists each simulation run is a model of real-world phenomena given certain conditions, we use the term model to refer to our modeling of the ensemble data, not the physics. Different model types often provide complementary perspectives on data features when analyzing the same data set. Each model visualizes data at several levels of abstraction, allowing the user to range from viewing the ensemble holistically to accessing numeric parameter values for a single run. Bookmarks provide a mechanism for sharing results, enabling interesting model states to be labeled and saved.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Adaptive Spatially Aware I/O for Multiresolution Particle Data Layouts

Large-scale simulations on nonuniform particle distributions that evolve over time are widely used in cosmology, molecular dynamics, and engineering. Such data are often saved in an unstructured format that neither preserves spatial locality nor provides metadata for accelerating spatial or attribute subset queries, leading to poor performance of visualization tasks. Furthermore, the parallel I/O strategy used typically writes a file per process or a single shared file, neither of which is portable or scalable across different HPC systems. We present a portable technique for scalable, spatially aware adaptive aggregation that preserves spatial locality in the output. We evaluate our approach on two supercomputers, Stampede2 and Summit, and demonstrate that it outperforms prior approaches at scale, achieving up to 2.5× faster writes and reads for nonuniform distributions. Furthermore, the layout written by our method is directly suitable for visual analytics, supporting low-latency reads and attribute-based filtering with little overhead.

Usher, Will↗
DOE OSTI2017

Contextual Compression of Large-Scale Wind Turbine Array Simulations

Data sizes are becoming a critical issue particularly for HPC applications. We have developed a user-driven lossy wavelet-based storage model to facilitate the analysis and visualization of large-scale wind turbine array simulations. The model stores data as heterogeneous blocks of wavelet coefficients, providing high-fidelity access to user-defined data regions believed the most salient, while providing lower-fidelity access to less salient regions on a block-by-block basis. In practice, by retaining the wavelet coefficients as a function of feature saliency, we have seen data reductions in excess of 94 percent, while retaining lossless information in the turbine-wake regions most critical to analysis and providing enough (low-fidelity) contextual information in the upper atmosphere to track incoming coherent turbulent structures. Our contextual wavelet compression approach has allowed us to deliver interative visual analysis while providing the user control over where data loss, and thus reduction in accuracy, in the analysis occurs. We argue this reduced but contextualized representation is a valid approach and encourages contextual data management.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Evaluating HPC Scheduling Strategies for Urgent Workloads

Scientific computing centers increasingly face workloads with diverse urgency requirements, driven by applications that demand rapid or even immediate execution. Appropriately configured scheduling policies can significantly improve both user satisfaction and overall cluster utilization. In this work, we present a systematic analysis of scheduler configurations under scenarios where a fraction of jobs have urgent computing needs. We evaluate multiple job scheduling simulators, develop a lightweight job-submission emulation framework, and create tools to analyze and visualize the resulting scheduling data. Our study identifies key trade-offs between responsiveness, fairness, and efficiency, and offers a set of practical scheduling configurations (particularly for Slurm) that can be tailored to HPC environments supporting mixed-urgency workloads.

Maheshwari, Ketan [ORNL] (ORCID:000000033800662X)↗
DOE OSTI2021

VERAIO Software Management Plan

VERAIO is a set of utility codes used to provide a common set of input and outputs to the Virtual Environment for Reactor Applications (VERA). VERA is a collection of several different computer codes that all have a common input and output. This prevents the need to manage input and output from each individual code, allowing for ease of use and reducing errors associated with code operability. The VERAIO utilities include VERAIn, VERAView, and VERARun. Each of these utilities is described below. VERAIn is an input processor that reads an ASCII input file generated by users, parses the file, performs some error checking, and writes an XML file to be read by other VERA codes. The main purpose of VERAIn is to provide a common input to all of the VERA codes so users only need to learn one input. VERAIn is written in Perl and uses YAML configuration files to provide flexibility. VERAView is a graphical user interface (GUI) that reads a VERA HDF output file and allows users to visualize results. VERAView is written in Python. VERARun is a script that drives the VERA execution in a high performance computing (HPC) environment. Work performed at the code level supports the Quality Assurance Program Plan (QAPP) (VERA-QA-001), and VERA Software Quality Assurance Plan (VERA-QA-002).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

VERAIO Software Management Plan

VERAIO is a set of utility codes used to provide a common set of inputs and outputs to the Virtual Environment for Reactor Applications (VERA). VERA is a collection of several different computer codes that all have a common input and output. This prevents the need to manage input and output from each individual code, allowing for ease of use and reducing errors associated with code operability. The VERAIO utilities include VERAIn, VERAView, and VERARun. Each of these utilities is described below. VERAIn is an input processor that reads an ASCII input file generated by users, parses the file, performs some error checking, and writes an XML file to be read by other VERA codes. The main purpose of VERAIn is to provide a common input to all of the VERA codes, so users only need to learn one input. VERAIn is written in Perl and uses YAML configuration files to provide flexibility. VERAView is a graphical user interface (GUI) that reads a VERA hierarchical data format (HDF) output file and allows users to visualize results. VERAView is written in Python. VERARun is a script that drives the VERA execution in a high performance computing (HPC) environment. Work performed at the code level supports the quality assurance program plan (QAPP) (VERA-QA-001) and the VERA Software Quality Assurance Plan (VERA-QA-002).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Next-Generation Analysis and Visualization Platform for Manufacturing and Engineering Workflows (Final Report)

Advanced manufacturing and engineering are fundamental to the economic strength and national security of the United States. With the introduction of advanced modeling and simulation practices in manufacturing and engineering over the last several decades, we are producing better products, more quickly, and at decreased cost. Through the adoption of high-performance computing (HPC) enabled workflows, in small- to medium-sized manufacturers and engineering firms, which employ fewer than 500 people, is renewing United States manufacturing and engineering. Despite successes, there are significant barriers to widespread adoption of these tools. Our Phase I SBIR project, Next-Generation Analysis and Visualization Platform for Manufacturing and Engineering Workflows, aims to address several of these barriers. Our project helped us evaluate the feasibility and benefits and mitigate risks of asynchronous processing for analysis and visualization workflows. We evaluated the impact of asynchronous, non-blocking communication can have on the user experience in a complex visualization application. Besides the impact on users, we wanted to understand the impacts on developers (development application programming interfaces (APIs))and generated applications (robustness and stability). Any asynchronous platform implementation is bound to rely on multiple threads which can make the programming model extraordinarily complex and prone to bugs. If not managed carefully, the complexity and resulting instability would be an unacceptable evolution for widely used production tools like ParaView and putting the success of the Phase II project at risk.

42 ENGINEERING↗
DOE OSTI2017

In Situ Visualization of Radiation Transport Geometry

The ultimate goal for radiation transport is to perform full-core reactor modelling and simulation. Advances in computational simulation bring this goal close to reality and the newest Monte Carlo transport codes have begun to shift to using accelerators that have become a stalwart in the supercomputing and HPC space. Within the reactor modelling and simulation community, Monte Carlo transport simulations are considered the gold standard for simulation. Through the use of "combinatorial geometry" (constructive solid geomtry), complex models can be used with fewer approximation compromises while at the same time scale to run on some of the largest supercomputers in the world.Unfortunately, the state-of-the-art for "combinatorial geometry" visualization is to decompose the geometry into a mesh. This approach could require a significant amount of memory which is antithetical to in situ visualization. To address this issue, we introduce a ray caster for visualizing combinatorial geometry in radiation transport code. By only using the accelerators for the radiation transport code and leaving the CPU cores idle, there is an opportunity to conduct on node in situ visualization with the idle CPU cores, something domain experts have up to this point been unable to do. By utilizing VTK-m, the visualization can be run on the CPU as this particular application demands, but also run on any architecture that is supported by VTK-m, enabling future re-use across different platforms.

Kim, Mark B.↗
DOE OSTI2016

Performance Modeling of In Situ Rendering

With the push to exascale, in situ visualization and analysis will continue to play an important role in high performance computing. Tightly coupling in situ visualization with simulations constrains resources for both, and these constraints force a complex balance of trade-offs. A performance model that provides an a priori answer for the cost of using an in situ approach for a given task would assist in managing the trade-offs between simulation and visualization resources. In this work, we present new statistical performance models, based on algorithmic complexity, that accurately predict the run-time cost of a set of representative rendering algorithms, an essential in situ visualization task. To train and validate the models, we conduct a performance study of an MPI+X rendering infrastructure used in situ with three HPC simulation applications. We then explore feasibility issues using the model for selected in situ rendering questions.

Larsen, Matthew↗
DOE OSTI2023

VTK-m: Visualization for the Exascale Era and Beyond

A recent trend in modern high-performance computing is the increasing use of hybrid architectures, where the vast majority of performance comes from accelerators. Modern accelerators are based on Graphics Processing Units (GPU) that contain many low power cores that in their aggregate provides an extremely high computation rate. Current and future CPU processors are requiring more explicit parallelism as each successive version of the hardware packs in more cores, and technologies like hyperthreading and vector operations require even more parallel processing to leverage each core’s full potential. As an example, the Frontier supercomputer installed at Oak Ridge National Laboratories recently hit a record breaking 1.1 exaflops1 on the LINPACK HPC benchmark [Shoemaker 2022]. The system contains 37632 AMD MI250x GPUs which requires more than half a billion threads to keep the system fully utilized [Khizeran 2022].VTK-m is a toolkit of scientific visualization algorithms for these emerging processor architectures. VTK-m supports the fine-grained concurrency for data analysis and visualization algorithms required to drive extreme scale computing by providing abstract models for data and execution that can be applied to a variety of algorithms across many different processor architectures.

Bolstad, Mark↗
DOE OSTI2015

Formulas for Fast Computation of Divergence Statistics Applied to Quantitative Performance Analysis

In an earlier work, we reported on the extension to the statistical analysis capability of the Visualization Tool Kit (VTK), which we developed for the calculation of divergence statistics, with the particular aim of providing quantitative means for High Performance Computing (HPC) performance analysis, of which we provided an example as well as user's manual. However, we did not provide the mathematical foundations for this work. In the current report, we fill this void with the complete derivation of the formulas which we used in the divergence statistics engine. This provides the foundations for future work which will aim at generalizing these formulas for more detailed HPC performance analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Towards Interactive, Reproducible Analytics at Scale on HPC Systems

The growth in scientific data volumes has resulted in a need to scale up processing and analysis pipelines using High Performance Computing (HPC) systems. These workflows need interactive, reproducible analytics at scale. The Jupyter platform provides core capabilities for interactivity but was not designed for HPC systems. In this paper, we outline our efforts that bring together core technologies based on the Jupyter Platform to create interactive, reproducible analytics at scale on HPC systems. Our work is grounded in a real world science use case-applying geophysical simulations and inversions for imaging the subsurface. Our core platform addresses three key areas of the scientific analysis workflow-reproducibility, scalability, and interactivity. We describe our implemention of a system, using Binder, Science Capsule, and Dask software. We demonstrate the use of this software to run our use case and interactively visualize real-Time streams of HDF5 data.

containers↗
DOE OSTI2018

Python-based in situ analysis and visualization

This work focuses on enabling the use of Python-based methods for the purpose of performing in situ analysis and visualization. This approach facilitates access to and use of a rapidly growing collection of Python-based, third-party libraries for analysis and visualization, as well as lowering the barrier to entry for user-written Python analysis codes. Beginning with a simulation code that is instrumented to use the SENSEI in situ interface, we present how to couple it with a Python-based data consumer, which may be run in situ, and in parallel at the same concurrency as the simulation. We present two examples that demonstrate the new capability. One is an analysis of the reaction rate in a proxy simulation of a chemical reaction on a 2D substrate, while the other is a coupling of an AMR simulation to Yt, a parallel visualization and analysis library written in Python. In the examples, both the simulation and Python in situ method run in parallel on a large-scale HPC platform.

Loring, Burlen↗
DOE OSTI2021

Catalyst Revised: Rethinking the ParaView in Situ Analysis and Visualization API

As in situ analysis goes mainstream, ease of development, deployment, and maintenance becomes essential, perhaps more so than raw capabilities. In this paper, we present the design and implementation of Catalyst, an API for in situ analysis using ParaView, which we refactored with these objectives in mind. Furthermore, our implementation combines design ideas from in situ frameworks and HPC tools like Ascent and MPICH.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

2016 CSSE L3 Milestone: Deliver In Situ to XTD End Users

This report summarizes the activities in FY16 toward satisfying the CSSE 2016 L3 milestone to deliver in situ to XTD end users of EAP codes. The Milestone was accomplished with ongoing work to ensure the capability is maintained and developed. Two XTD end users used the in situ capability in Rage. A production ParaView capability was created in the HPC and Desktop environment. Two new capabilities were added to ParaView in support of an EAP in situ workflow. We also worked with various support groups at the lab to deploy a production ParaView in the LANL environment for both desktop and HPC systems. . In addition, for this milestone, we moved two VTK based filters from research objects into the production ParaView code to support a variety of standard visualization pipelines for our EAP codes.

"In Situ" production↗
DOE OSTI2018

Performance-Portable Particle Advection with VTK-m

Particle advection is the fundamental kernel behind most vector field visualization methods. Yet, the efficient parallel computation of large amounts of particle traces remains challenging. This is exacerbated by the variety of hardware trends in today's HPC arena, including increasing core counts in classical CPUs, many-core designs such as the Intel Xeon Phi, and massively parallel GPUs. The dedicated optimization of a particle advection kernel for each individual target architecture is both time-consuming and error prone. In this paper, we propose a performance-portable algorithm for particle advection. Our algorithm is based on the recently introduced VTK-m system and chiefly relies on its device adapter abstraction. We demonstrate the general portability of our implementation across a wide variety of hardware. Finally, our evaluation shows that our hardware-agnostic algorithm has comparable performance to hardware-specific algorithms.

Pugmire, Dave↗
DOE OSTI2016

Big Data as a Service from an Urban Information System

Big Data has already proven itself as a valuable tool that lets geographers and urban researchers utilize large data re- sources to generate new insights. However, wider adoption of Big Data techniques in these areas is impeded by a number of difficulties in both knowledge discovery and data and sci- ence production. Typically users face such problems as dis- parate and scattered data, data management, spatial search- ing, insufficient computational capacity for data-driven anal- ysis and modelling, and the lack of tools to quickly visual- ize and summarize large data and analysis results. Here we propose an architecture for an Urban Information Sys- tem (UrbIS) that mitigates these problems by utilizing the Big Data as a Service (BDaaS) concept. With technolog- ical roots in High-performance Computing (HPC), BDaaS is based on the idea of outsourcing computations to dif- ferent computing paradigms, scalable to super-computers. UrbIS aims to incorporate federated metadata search, in- tegrated modeling and analysis, and geovisualization into a single seamless workflow. The system is under active devel- opment and is built around various emerging technologies that include hybrid and NoSQL databases, massively par- allel systems, GPGPU computing, and WebGL-based geo- graphic visualization. UrbIS is designed to facilitate using Big Data across multiple cities to better understand how ur- ban areas impact the environment and how climate change and other environmental change impact urban areas.

Sorokine, Alexandre↗