Search NASA⌕ Search

SEARCH · Search NASA

Results for “data visualizations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Parallel analysis tools and new visualization techniques for ultra-large climate data set

ParVis was a project funded under LAB 10-05: “Earth System Modeling: Advanced Scientific Visualization of Ultra-Large Climate Data Sets”. Argonne was the lead lab with partners at PNNL, SNL, NCAR and UC-Davis. This report covers progress from January 1st, 2013 through Dec 1st, 2014. Two previous reports covered the period from Summer, 2010, through September 2011 and October 2011 through December 2012, respectively. While the project was originally planned to end on April 30, 2013, personnel and priority changes allowed many of the institutions to continue work through FY14 using existing funds. A primary focus of ParVis was introducing parallelism to climate model analysis to greatly reduce the time-to-visualization for ultra-large climate data sets. Work in the first two years was conducted on two tracks with different time horizons: one track to provide immediate help to climate scientists already struggling to apply their analysis to existing large data sets and another focused on building a new data-parallel library and tool for climate analysis and visualization that will give the field a platform for performing analysis and visualization on ultra-large datasets for the foreseeable future. In the final 2 years of the project, we focused mostly on the new data-parallel library and associated tools for climate analysis and visualization.

54 ENVIRONMENTAL SCIENCES↗

Streaming Data in HPC Workflows Using ADIOS

The “IO Wall” problem, in which the gap between computation rate and data access rate grows continuously, poses significant problems to scientific workflows which have traditionally relied upon using the filesystem for intermediate storage between workflow stages. One way to avoid this problem in scientific workflows is to stream data directly from producers to consumers and avoiding storage entirely. However, the manner in which this is accomplished is key to both performance and usability. This paper presents the Sustainable Staging Transport, an approach which allows direct streaming between traditional file writers and readers with few application changes. SST is an ADIOS “engine”, accessible via standard ADIOS APIs, and because ADIOS allows engines to be chosen at run-time, many existing file-oriented ADIOS workflows can utilize SST for direct application-to-application communication without any source code changes. This paper describes the design of SST and presents performance results from various applications that use SST, for feeding model training with simulation data with substantially higher bandwidth than the theoretical limits of Frontier’s file system, for strong coupling of separately developed applications for multiphysics multiscale simulation, or for in situ analysis and visualization of data to complete all data processing shortly after the simulation finishes.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗

Open Source Platform for Live Processing of High Data Rate Electron Microscopy (SBIR Phase I Award Topic 15, Subtopic B, Final Scientific / Technical Report)

4D scanning transmission electron microscopy (STEM) is experiencing a revolution both in terms of data size, and data veracity, as are many other areas of experimental data acquisition. This presents enormous opportunities to make new scientific discoveries, but is also hamstrung by aging software that is no longer fit for purpose. Experimental data acquisition platforms must make the same investments in software infrastructure that the simulation community made decades ago to take advantage of high-performance computing (HPC) if they hope to realize the full potential of the latest advances made in detector technology. This requires rethinking the design and deployment of hardware, moving away from small, fixed commodity computing capabilities with closed interfaces to open, modular, scalable systems designed for parallelism and throughput. The technical approach will leverage best-in-class open source technologies, and develop new approaches strategically to build an open source platform for experimental data acquisition targeting high data rate electron microscopy. The project will develop a software platform that uses the power and performance of modern HPC resources to ingest the large volume of data produced by electron microscopes using direct electron detectors. A set of configurable pipelines will be used to enable the processing to be tailored to the problem being investigated with builtin visualization of data at interactive rates. This project will leverage investments made by the DOE and other agencies to provide a powerful software platform for the experimental data community working with most powerful electron microscopes, such as 4D STEM detectors. It will create a platform that serves as a reference for the community. Permissive open source licensing will make the platform available to all, with a simple deployment strategy offering the ability to keep data close to where it is generated. This advanced software platform will require customization for each deployment, creating opportunities for software services that are typically offered to extend open source software platforms.

97 MATHEMATICS AND COMPUTING↗

Community Resilience Indicator Analysis: Commonly Used Indicators from Peer-Reviewed Research (Updated for Research Published 2003-2021)

In 2017, FEMA’s National Integration Center (NIC) Technical Assistance (TA) Branch identified a need to establish a data-driven basis for prioritizing locations for TA investment and guiding local emergency management planning. To achieve this goal, FEMA tasked Argonne National Laboratory (Argonne) with identifying commonly used indicators of community resilience across the landscape of published peer-reviewed research. FEMA and Argonne completed the first Community Resilience Indicator Analysis (CRIA) in 2018 and repeated the process in 2022. The CRIA process begins with a literature review and cataloguing of published peer-reviewed assessment methodologies on social vulnerability and community resilience. The literature review findings are then filtered by inclusion criteria established by the CRIA research team to ensure the methodologies are: (1) Quantitative, (2) Data and methodology are publicly available, (3) Calculated at the county level or lower, (4) Examine generalized hazard risk (rather than a singular hazard), and (5) Focused on pre-disaster community conditions. After this, the research team identifies the commonly used indicators across these methodologies and selects the best data source for each indicator. Finally, the research team bins the data for visual display, conducts a correlation analysis and creates a composite index, the FEMA Community Resilience Index (FEMA CRI). In 2018, the CRIA identified eight resilience and vulnerability assessment methodologies and 20 commonly used indicators (indicators used in three or more of the eight methodologies). The FEMA CRI in 2018 was created from these 20 indicators and was produced for at the county level. The 2022 CRIA updated the literature review to expand the list of methodologies examined and followed the same process, resulting in an analysis of 14 methodologies published between 2003 and 2021 and 22 indicators identified as commonly used (indicators used in five or more of the 14 methodologies). In 2022, the research team produced the FEMA CRI at the county and the census tract levels. To make the CRIA data more accessible and more actionable, each individual indicator and the FEMA CRI is binned and included in FEMA’s Resilience Analysis and Planning Tool (RAPT). RAPT enables emergency managers and community partners to quickly visualize relative differences in potential resilience by county, tribe and census tract. By reviewing the data for each of these 22 indicators individually, emergency managers can gain insights for targeted outreach strategies, planning, mitigation investments and response and recovery operations. Communities, regional governments and others can use this data to better understand potential challenges to resilience. As the social science field of examining and validating indicators of resilience evolves, FEMA will update RAPT to provide emergency managers and community partners with additional data and tools to inform planning, mitigation, response and recovery. It is important to understand that the role of the emergency manager is not to change or to “improve” the data, but to plan appropriately for the community characteristics reflected in the data. These datasets are community characteristics that researchers have identified as important considerations for resilience. For example, people with disabilities may have greater challenges to be resilient to disasters. If a community has a high population of people with disabilities, the emergency manager(s) may need to create tailored preparedness outreach programs and strategies to ensure those residents have support if evacuation is necessary. Rather than label these indicators as an absolute measure of resilience, FEMA considers “potential challenges to resilience” a better frame to understand these indicators. Everyone is vulnerable to disasters. While scholars theorize that certain characteristics may make an individual or a household more socially vulnerable, the data does not reflect measures that individuals and/or communities have taken to address potential challenges, such as emergency management planning and outreach or household preparedness measures. To aid emergency managers in understanding how to use these indicators, calling them potential challenges to resilience supports a more positive and strategic application of the data in all phases of emergency management.

99 GENERAL AND MISCELLANEOUS↗

Solutions for Digital Video Transmission Technology Final Report CRADA No. TC02068.0

This Project aimed at development of software for seismic data processing based on the Geotool code developed by the American company Multimax., Inc. The Geotool code was written in early 90-es for the UNIX platform. Under Project# 2821, functions of the old Geotool code were transferred into a commercial version for the Microsoft XP and Vista platform with addition of new capabilities on visualization and data processing. The developed new version of the Geotool+ was implemented using the up-to-date tool Microsoft Visual Studio 2005 and uses capabilities of the .NET platform. C++ was selected as the main programming language for the Geotool+. The two-year Project was extended by six months and funding levels increased from 600,000 to $670,000. All tasks were successfully completed and all deliverables were met for the project even though both the industrial partner and LLNL principal investigator left the project before its final report.

58 GEOSCIENCES↗

Visibility Histograms and Visibility-Driven Transfer Functions

Direct volume rendering is an important tool for visualizing complex data sets. However, in the process of generating 2D images from 3D data, information is lost in the form of attenuation and occlusion. The lack of a feedback mechanism to quantify the loss of information in the rendering process makes the design of good transfer functions a difficult and time consuming task. In this paper, we present the general notion of visibility histograms, which are multidimensional graphical representations of the distribution of visibility in a volume-rendered image. In this paper, we explore the 1D and 2D transfer functions that result from intensity values and gradient magnitude. With the help of these histograms, users can manage a complex set of transfer function parameters that maximize the visibility of the intervals of interest and provide high quality images of volume data. We present a semiautomated method for generating transfer functions, which progressively explores the transfer function space toward the goal of maximizing visibility of important structures. Our methodology can be easily deployed in most visualization systems and can be used together with traditional 1D and 2D opacity transfer functions based on scalar values, as well as with other more sophisticated rendering algorithms.

42 ENGINEERING↗

Refining HPCToolkit for application performance analysis at exascale

As part of the US Department of Energy’s Exascale Computing Project (ECP), Rice University has been refining its HPCToolkit performance tools to better support measurement and analysis of applications executing on exascale supercomputers. To efficiently collect performance measurements of GPU-accelerated applications, HPCToolkit employs novel non-blocking data structures to communicate performance measurements between tool threads and application threads. To attribute performance information in detail to source lines, loop nests, and inlined call chains, HPCToolkit performs parallel analysis of large CPU and GPU binaries involved in the execution of an exascale application to rapidly recover mappings between machine instructions and source code. To analyze terabytes of performance measurements gathered during executions at exascale, HPCToolkit employs distributed-memory parallelism, multithreading, sparse data structures, and out-of-core streaming analysis algorithms. To support interactive exploration of profiles up to terabytes in size, HPCToolkit’s hpcviewer graphical user interface uses out-of-core methods to visualize performance data. The result of these efforts is that HPCToolkit now supports collection, analysis, and presentation of profiles and traces of GPU-accelerated applications at exascale. These improvements have enabled HPCToolkit to efficiently measure, analyze and explore terabytes of performance data for executions using as many as 64K MPI ranks and 64K GPU tiles on ORNL’s Frontier supercomputer. HPCToolkit’s support for measurement and analysis of GPU-accelerated applications has been employed to study a collection of open-science applications developed as part of ECP. This paper reports on these experiences, which provided insight into opportunities for tuning applications, strengths and weaknesses of HPCToolkit itself, as well as unexpected behaviors in executions at exascale.

Adhianto, Laksono↗

Ensemble Data Analysis ENvironment (EDEN)

The EDEN toolkit facilitates exploratory data analysis and visualization of global climate model simulation datasets. EDEN provides an interactive graphical user interface (GUI) that helps the user visually construct dynamic queries of the characteristically large climate datasets using temporal ranges, variable selections, and geographic areas of interest. EDEN reads the selected data into a multivariate visualization panel which features an extended implementation of parallel coordinates plots as well as interactive scatterplots. The user can query data in the visualization panel using mouse gestures to analyze different ranges of data. The visualization panel provides coordinated multiple views whereby selections made in one plot are propagated to the other plots.

Steed, ChadAllen↗

ALPINE Overview

Data-driven sampling enables probabilistic identification of interesting regions in the data automatically, prioritizing important regions. Applied in situ to Nyx, important halo regions are preserved.

97 MATHEMATICS AND COMPUTING↗

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

Preserving Command Line Workflow for a Package Management System using ASCII DAG Visualization

Package managers provide ease of access to applications by removing the time-consuming and sometimes completely prohibitive barrier of successfully building, installing, and maintaining the software for a system. A package dependency contains dependencies between all packages required to build and run the target software. Package management system developers, package maintainers, and users may consult the dependency graph when a simple listing is insufficient for their analyses. However, users working in a remote command line environment must disrupt their workflow to visualize dependency graphs in graphical programs, possibly needing to move files between devices or incur forwarding lag. Such is the case for users of Spack, an open source package management system originally developed to ease the complex builds required by supercomputing environments. To preserve the command line workflow of Spack, we develop an interactive ASCII visualization for its dependency graphs. Through interviews with Spack maintainers, we identify user goals and corresponding visual tasks for dependency graphs. We evaluate the use of our visualization through a command line-centered study, comparing it to the system's two existing approaches. We observe that despite the limitations of the ASCII representation, our visualization is preferred by participants when approached from a command line interface workflow.

97 MATHEMATICS AND COMPUTING↗

AKUNA – INTEGRATED TOOLSETS SUPPORTING ADVANCED SUBSURFACE FLOW AND TRANSPORT SIMULATIONS FOR ENVIRONMETAL MANAGEMENT

A next generation open source subsurface simulator and user environment for environmental management is being developed through a collaborative effort across Department of Energy National Laboratories. The flow and transport simulator, Amanzi, will be capable of modeling complex subsurface environments and processes using both unstructured and adaptive meshes at very fine spatial resolutions that require supercomputing-scale resources. The user environment, Akuna, provides users with a range of tools to manage environmental and simulator data sets, create models, manage and share simulation data, and visualize results. Underlying the user interface are core toolsets that provide algorithms for sensitivity analysis, parameter estimation, and uncertainty quantification. Akuna is open-source, cross platform software that is initially being demonstrated on the Hanford BC Cribs remediation site. In this paper, we describe the emerging capabilities of Akuna and illustrate how these are being applied to the BC Cribs site.

user interface, data management, uncertainty quant↗

Akuna - Integrated Toolsets Supporting Advanced Subsurface Flow and Transport Simulations for Environmental Management

A next generation open source subsurface simulator and user environment for environmental management is being developed through a collaborative effort across Department of Energy National Laboratories. The flow and transport simulator, Amanzi, will be capable of modeling complex subsurface environments and processes using both unstructured and adaptive meshes at very fine spatial resolutions that require supercomputing-scale resources. The user environment, Akuna, provides users with a range of tools to manage environmental and simulator data sets, create models, manage and share simulation data, and visualize results. Underlying the user interface are core toolsets that provide algorithms for sensitivity analysis, parameter estimation, and uncertainty quantification. Akuna is open-source, cross platform software that is initially being demonstrated on the Hanford BC Cribs remediation site. In this paper, we describe the emerging capabilities of Akuna and illustrate how these are being applied to the BC Cribs site.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Data co-processing for extreme scale analysis level II ASC milestone (4745).

Exascale supercomputing will embody many revolutionary changes in the hardware and software of high-performance computing. A particularly pressing issue is gaining insight into the science behind the exascale computations. Power and I/O speed con- straints will fundamentally change current visualization and analysis work ows. A traditional post-processing work ow involves storing simulation results to disk and later retrieving them for visualization and data analysis. However, at exascale, scien- tists and analysts will need a range of options for moving data to persistent storage, as the current o ine or post-processing pipelines will not be able to capture the data necessary for data analysis of these extreme scale simulations. This Milestone explores two alternate work ows, characterized as in situ and in transit, and compares them. We nd each to have its own merits and faults, and we provide information to help pick the best option for a particular use.

Rogers, David↗

Hurricane Harvey Flood

Online supplement to Attributable human-induced changes in the magnitude of flooding in the Houston, Texas region during Hurricane Harvey, by Michael Wehner and Christopher Sampson. Geospatial data to visualize the attributable flooding in the greater Houston during Hurricane Harvey as simulated by the Fathom hydraulical model in a geotiff file format is provided at https://portal.nersc.gov/cascade/Harvey/ . The public is encouraged to download this open access data and investigate their own neighborhoods of interest using free or commercial software. A detailed validation of the baseline simulated "flood that was" for every neighborhood is outside the scope of this article but individuals can compare the simulation to their own experiences. The public is then invited to compare the "flood that might have been" had there not been a human interference in the climate system to the simulated actual flood in these neighborhoods. The variety of scientific opinions as to the human effect on precipitation and their effect on the flood are presented as detailed in the table below. A pair of high risk scenarios of the effect of a future warmer climate and a hypothetical storm similar to Harvey is also provided.

54 ENVIRONMENTAL SCIENCES↗

Acoustic Emission-Based Methodology to Evaluate Delamination Crack Growth Under Quasi-static and Fatigue Loading Conditions

The aim of this study was to investigate the applicability of acoustic emission (AE) technique to evaluate delamination crack in glass/epoxy composite laminates under quasi-static and fatigue loading. To this aim, double cantilever beam specimens were subjected to mode I quasi-static and fatigue loading conditions and the generated AE signals were recorded during the tests. By analyzing the mechanical and AE results, an analytical correlation between the AE energy with the released strain energy and the crack growth was established. It was found that there is a 3rd degree polynomial correlation between the crack growth and the cumulative AE energy. Using this correlation the delamination crack growth was predicted under both the static and fatigue loading conditions. The predicted crack growth values was were in a good agreement with the visually recorded data during the tests. The results indicated that the proposed AE-based method has good applicability to evaluate the delamination crack growth under quasi-static and fatigue loading conditions, especially when the crack is embedded within the structure and could not be seen visually.

42 ENGINEERING↗