Results for “io optimizations” — Page 2

NASA NTRS2016

NASA WorldWind: Open Source Visualization Technology for Earth Observation

NASA WorldWind: Open Source Visualization Technology for Earth Observation WorldWind, open source virtual globe technology for Java, iOS, Android and Web, is provided by NASA and is architected as API-centric modular componentry. This enable it to be continually optimized and feature-enriched in ways that allow applications based on this SDK (Software Development Kit) to benefit Earth Observation, especially Open Science, with minimal or no adjustment for the decade ahead. The next-generation National Airspace System (NAS) aviation management system for the U.S. Federal Aviation Administration, FAA, uses WorldWind, as do applications currently being developed by the European Space Agency, along with several other US and European government agencies and industry partners. This presentation will demonstrate several NASA open source use cases for WorldWind technology that include advances being made to optimize access to NetCDF and HDF data via WebWorldWind.NASA WorldWind: Multidimensional Geospatial Web Platform The ability to see spatial data in its native context is essential for that data to be appreciated whether by the scientific community, policy and decision-makers or the general public. Recently, the accessibility of spatial data has dramatically improved. Without the need to install an application, spatial data can now be experienced via any web browser, mobile devices included. For developers, by simply updating the app on your server, the latest version of your application is now immediately available to your entire usercommunity. Unlike other virtual globes such as Google Earth, NASA World Wind offers something very special, full control to customize the interface with any features or functionalities you might need. You decide how the data is accessed and experienced. This allows you to provide maximum value of the information to your user community. The web version of NASA WorldWind (WebWorldWind) has made it possible for a whole new suite of applications for managing and sharing spatial data. Apps built with this web version are ideal for immediate social media type activity and also facilitate delivery of sophisticated data exchange scenarios such as weather and climate research, disaster response, personal navigation, and industrial-strength tracking for transportation, supply chain, aviation and satellites. WebWorldWind is an application component, not an app in itself. It is written in JavaScript and provides the real world geographic context for spatial data and information visualization, using a rich set of shapes and graphic primitives. WebWorldWind also provides platform independence, while accommodating any number of data types. Web WorldWind runs on any platform via a browser, i.e., Internet Explorer, Firefox, Chrome and Safari. Features include, 3D virtual globe, 2D map with multiple projection choices (Mercator, Polar, UPS, Equirectangular), imagery and elevation import, extensible, data retrieval (via REST, WMS, WCS, WFS, Bing, User-Defined), decluttering, measurement, accurate line-ofsight, subsurface visualization, and more.

Open Source Mapping↗

DOE OSTI2014

An Optimizing Compiler for Petascale I/O on Leadership-Class Architectures

In high-performance computing (HPC), parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our DOE project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final report summarizes the major achievements of the project and also points out promising future directions Two new sections in this report compared to the previous report are IOGenie and SSD/NVM-specific optimizations.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

EOSDIS Archive & Data Stewardship

NASA’s Earth Observing System Data and Information System’s (EOSDIS) was built to archive and distribute earth science data from flight and research programs. At the close of FY2020, the data collection had grown to over 42 petabytes distributed across the US. This presentation describes fundamentals associated with managing a free and open archive of this size for a worldwide, multi-discipline user community. The presentation is prepared for the Committee on Earth Observation Satellites (CEOS), which strives to enhance international coordination and data exchange and to optimize societal benefit. The Working Group on Information System and Services (WGISS) is a forum within CEOS for the collaboration with other international and domestic agencies io the development of Earth observation data archives, systems and services. This presentation reviews the construct of the EOSDIS archives, formats for long term archive, preservation of appropriate data and documents, and challenges facing the community.

earth science↗

DOE OSTI2021

I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis

Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.

Bez, Jean Luca↗

DOE OSTI2021

A surface acoustic wave hydrogen sensor with tin doped indium oxide layers for intermediate temperatures

We report high temperature surface acoustic wave (SAW) gas sensors with conducting sensing layers require tuning of the sheet conductivity for optimal response. Conducting metal oxides are attractive sensing materials for their tunable electronic properties and high thermal stability, amongst others. Here, we have investigated the application of indium oxide (IO) and indium tin oxide (ITO) films on langasite (LGS)-based SAW reflective delay line sensor devices for monitoring hydrogen at 350 °C. Specifically, we modeled the effect of the IO and ITO sensing layer thickness on the wave velocity, attenuation, and effective electromechanical coefficient. This was followed by an experimental demonstration of tuning of the ITO film sheet conductivity by controlling the dopant concentration and yielding an improvement in the sensor sensitivity. The current study provides a pathway towards the development of conductivity-based sensing layers for high temperature SAW gas sensors with improved sensitivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2015

An Optimizing Compiler for Petascale I/O on Leadership Class Architectures

In high-performance computing systems, parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final report summarizes the major achievements of the project and also points out promising future directions.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1987

Planetary atmospheres and aurorae

Observations of planetary atmospheres and auroras obtained by the IUE satellite observatory during the first 10 years of its operation are reviewed. Topics examined include the value of UV studies of atmospheric phenomena, the kinds of observations available prior to the launch of IUE in 1978, the composition and structure of the upper atmospheres below the homopause, the effects of the magnetosphere on the atmosphere above the homopause, excitation processes, and fundamental questions and scientific goals. Data on Jupiter, Saturn, Neptune and Uranus, and the Io plasma torus are presented in tables and graphs and briefly characterized. It is pointed out that the IUE has greatly advanced knowledge of the plantary atmospheres, despite the fact that its design was not optimized for planetary observations.

Moos, H. W.↗

DOE OSTI2024

exnehilo7/fielddata-lite-iOS-app

Field Expedition Routing and Navigation (FERN) is an IOS app funded by the Center for Bioenergy Innovation (CBI). It allows an end user to make trips to collect novel data points. It allows user to return to previously surveyed points to collect more data. It calculates the optimal route to use to sample points needed for return trips. A companion php/database application is required for full functionality. Links to this open source project are in the README. FERN can use the default GPS on an Iphone but it also optimized for Arrow Gold GPS. Other external GPS could be supported by modifying this code and adding the proper routines.

Hopp, Daniel↗

DOE OSTI2024

Performance Results on CPU/GPU Exascale Architectures for OMEGA: The Ocean Model for E3SM Global Applications

The US Department of Energy (DOE) conducts climate simulations on some of the world’s largest supercomputers. These exascale machines use heterogeneous architectures with both CPUs and GPUs, and scientific codes must adapt to make full use of this computing power. Los Alamos National Lab is developing Omega: The Ocean Model for E3SM Global Applications, which is specifically designed for modern exascale computers. It uses external libraries that have been optimized for a variety of architectures to run on different supercomputers. Omega is an unstructured-mesh ocean model based on TRiSK numerical methods. It will be the new ocean component of the DOE’s Energy Exascale Earth System Model (E3SM). The algorithms in Omega follow those of the current ocean component, MPAS-Ocean, but it will be written in C++ rather than Fortran to take advantage of the Kokkos performance portability library. Omega spatial operators are written as Kokkos kernels to run efficiently on both CPUs and GPUs. Work on Omega began in 2023 with a new C++ framework for unstructured mesh partitioning, halo exchanges, parallel IO, and Kokkos interfaces. The current version, Omega-0, is being developed to solve the shallow water equations and at present includes all of the tendency terms but not time stepping. Here we share the results of Omega-0 verification and performance testing. Verification includes unit tests implemented with CTest as well as convergence tests in Polaris, an in-house python package with a large suite of test problems. Performance tests compare simulations conducted on CPUs versus GPUs and across different architectures: tests are run on Frontier, which has AMD “Optimized 3rd Gen EPYC” CPUs and AMD MI250X GPUs, as well as Perlmutter, which is composed of AMD EPYC 7763 CPUs and NVIDIA A100 GPUs.

58 GEOSCIENCES↗

DOE OSTI2015

Photoelectron Spectroscopy and Theoretical Studies of Anion-pi Interactions: Binding Strength and Anion Specificity

Proposed in theory and confirmed to exist, anion–π interactions have been recognized as new and important non-covalent binding forces. Despite extensive theoretical studies, numerous crystal structural identifications, and a plethora of solution phase investigations, intrinsic anion–π interaction strengths that are free from complications of condensed phases’ environments, have not been directly measured in the gas phase. Herein we present a joint photoelectron spectroscopic and theoretical study on this subject, in which tetraoxacalix[2]arene[2]triazine 1, an electron-deficient and cavity self-tunable macrocyclic was used as a charge-neutral molecular host to probe its interactions with a series of anions with distinctly different shapes and charge states (spherical halides Cl⁻, Br⁻, I⁻, linear thiocyanate SCN⁻, trigonal planar nitrate NO₃⁻, pyramidic iodate IO₃⁻, and tetrahedral sulfate SO₄²⁻). The binding energies of the resultant gaseous 1:1 complexes (1•Cl⁻,1•Br⁻, 1•I⁻, 1•SCN⁻, 1•NO₃⁻, 1•IO₃⁻ and 1•SO₄²⁻) were directly measured experimentally, exhibiting substantial non-covalent interactions with pronounced anion specific effects. The binding strengths of Cl⁻, NO₃⁻, IO₃⁻ with 1 are found to be strongest among all singly charged anions, amounting to ca. 30 kcal/mol, but only about 40% of that between 1 and SO₄²⁻. Quantum chemical calculations reveal that all anions reside in the center of the cavity of 1 with anion–π binding motif in the complexes’ optimized structures, where 1 is seen to be able to self-regulate its cavity structure to accommodate anions of different geometries and three-dimensional shapes. Electron density surface and natural bond orbital charge distribution analysis further support anion–π binding formation. The calculated binding energies of the anions and 1 nicely reproduce the experimentally estimated electron binding energy increase. This work illustrates that size-selective photoelectron spectroscopy combined with theoretical calculations represent a powerful technique to probe intrinsic anion–π interactions and has potential to provide quantitative guest-host molecular binding strengths and unravel fundamental insights in specific anion recognitions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS1996

RANS-MP: A Portable Parallel Navier-Stokes Solver

RANS-MP, a new implementation of a single-grid Navier-Stokes solver using the diagonalized Beam-Warming approximate-factorization scheme, is presented. This first release of the completely rewritten solver employs the following optimizations: (1) Bi-directional multi-partition method for the ADI solver part; this improves granularity and load balance; (2) Improved cache usage through elimination of non-unit-stride array access (possible in part due to multi-partitioning); (3) Preprocessing of communicating boundary conditions to streamline logic during time stepping; (4) Truly parallel, high-performance I/O using the newly-developed MPI-IO library; (5) Elimination of large amounts of redundant operations through efficient use of workspace. Results of some realistic wing computations on the IBM SP2 computer will be presented. We will demonstrate that excellent absolute performance and scalability are obtained with RANS-MP, even for relatively small grid sizes. Besides high performance, an outstanding feature of RANS-MP is its true portability, due to the use of the portable message passing and I/O libraries MPI and MPI-IO.

VanderWijngaart, Rob F.↗

DOE OSTI2025

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1991

Advanced information processing system for advanced launch system: Hardware technology survey and projections

The major goals of this effort are as follows: (1) to examine technology insertion options to optimize Advanced Information Processing System (AIPS) performance in the Advanced Launch System (ALS) environment; (2) to examine the AIPS concepts to ensure that valuable new technologies are not excluded from the AIPS/ALS implementations; (3) to examine advanced microprocessors applicable to AIPS/ALS, (4) to examine radiation hardening technologies applicable to AIPS/ALS; (5) to reach conclusions on AIPS hardware building blocks implementation technologies; and (6) reach conclusions on appropriate architectural improvements. The hardware building blocks are the Fault-Tolerant Processor, the Input/Output Sequencers (IOS), and the Intercomputer Interface Sequencers (ICIS).

Cole, Richard↗

NASA NTRSDate not supplied

Innovative Features of NASA's Celestial Mapping System to Support Exploration in the Lunar South Pole

Introduction: NASA's Celestial Mapping System (CMS) is developed to address the need for 3D tools for planetary science investigations, mission planning, in-situ operations, in a 3D-first design constructed around a unified view of a planetary globe. At present CMS provides many critical functionalities that include 1) Equipment planning and optimized placement on Lunar surface 2) Line of sight (visibility ) analysis 3) Powerful measurement tools based on 3D terrain with realistic 3D models to represent rovers, astronauts and equipment 4) Visualization of de-rived mapping products (e.g. resource maps), and 5) Data engine for hosting new observations that are not available in other contemporary lunar data tools. CMS is built on the foundation of powerful NASA WorldWind globe engines. In near future, users will be able to simultaneously deploy CMS onto multiple hardware configurations and platforms such as Windows, Linux, iOS and Android. The users will also have the flexibility to update to the latest imagery and terrain datasets as they are being acquired (in real time) before and/or during the exploration mission. CMS is also capable of consumption and analysis of data from locally hosted and external sources. It supports Open Geospatial Consorti-um (OGC) data and file standards, with current integrations of datasets from the Astrogeology Science Center of USGS which include global and local data acquired from NASA (LRO, Clementine, Lunar Orbiter) and JAXA (SELENE/Kaguya) with the capability of integrating more datasets. With development experience in both the end-user application and planetary engine side, CMS is also able to adapt to newer Lunar cartography standards as they develop and become recognized by international geospatial panels. Overcoming Polar Distortions: 3D geospatial applications traditionally suffer from significant distortion of imagery at the poles due to following reasons – 1) distortions in the source imagery 2) Incompatible tessellation algorithm on the poles 3) map projections. In the lunar context, with the focus on the South pole, this is not acceptable. The CMS team is researching ways to address polar distortion of imagery with new tessellation algorithms and by reprojecting the data using projections that are more accurate in polar scenarios. Figure1 shows the potential error introduced by different tessellation methods, represented by the red and green circles for Shoemaker crater. There is ~2 Km difference in the placement of the crater. Line of Sight Analysis and Traverse Planning: We have developed a built-in line of sight analysis (LOS) tool in CMS that analyzes the terrain profile and obstructions and provides the visibility of a given terrain for a remote observer. Figure 2 shows the viewshed analysis on the PSR in Nobile region. The PSR was created with help of HORUS generated images. The yellow pin shows the observer location outside the PSR. The yellow area shows the visible part of PSR. The obstructed area with no visibility for the observer is shown in red. This analysis was ex-tended further to set different heights for various observers and then perform the viewshed analysis. Combining the different visibility profiles can help designing improved traverses within the crater.

Geospatial Mapping↗

NASA NTRSDate not supplied

Innovative Features of NASA's Celestial Mapping System to Support Exploration in the Lunar South Pole

Introduction: NASA's Celestial Mapping System (CMS) is developed to address the need for 3D tools for planetary science investigations, mission planning, in-situ operations, in a 3D-first design constructed around a unified view of a planetary globe. At present CMS provides many critical functionalities that include 1) Equipment planning and optimized placement on Lunar surface 2) Line of sight (visibility ) analysis 3) Powerful measurement tools based on 3D terrain with realistic 3D models to represent rovers, astronauts and equipment 4) Visualization of de-rived mapping products (e.g. resource maps), and 5) Data engine for hosting new observations that are not available in other contemporary lunar data tools. CMS is built on the foundation of powerful NASA WorldWind globe engines. In near future, users will be able to simultaneously deploy CMS onto multiple hardware configurations and platforms such as Windows, Linux, iOS and Android. The users will also have the flexibility to update to the latest imagery and terrain datasets as they are being acquired (in real time) before and/or during the exploration mission. CMS is also capable of consumption and analysis of data from locally hosted and external sources. It supports Open Geospatial Consorti-um (OGC) data and file standards, with current integrations of datasets from the Astrogeology Science Center of USGS which include global and local data acquired from NASA (LRO, Clementine, Lunar Orbiter) and JAXA (SELENE/Kaguya) with the capability of integrating more datasets. With development experience in both the end-user application and planetary engine side, CMS is also able to adapt to newer Lunar cartography standards as they develop and become recognized by international geospatial panels. Overcoming Polar Distortions: 3D geospatial applications traditionally suffer from significant distortion of imagery at the poles due to following reasons – 1) distortions in the source imagery 2) Incompatible tessellation algorithm on the poles 3) map projections. In the lunar context, with the focus on the South pole, this is not acceptable. The CMS team is researching ways to address polar distortion of imagery with new tessellation algorithms and by reprojecting the data using projections that are more accurate in polar scenarios. Figure1 shows the potential error introduced by different tessellation methods, represented by the red and green circles for Shoemaker crater. There is ~2 Km difference in the placement of the crater. Line of Sight Analysis and Traverse Planning: We have developed a built-in line of sight analysis (LOS) tool in CMS that analyzes the terrain profile and obstructions and provides the visibility of a given terrain for a remote observer. Figure 2 shows the viewshed analysis on the PSR in Nobile region. The PSR was created with help of HORUS generated images. The yellow pin shows the observer location outside the PSR. The yellow area shows the visible part of PSR. The obstructed area with no visibility for the observer is shown in red. This analysis was ex-tended further to set different heights for various observers and then perform the viewshed analysis. Combining the different visibility profiles can help designing improved traverses within the crater.

Geospatial Mapping↗

DOE OSTI2024

Optimal thermodynamic conditions to minimize kinetic by-products in aqueous materials synthesis

Abstract Phase diagrams offer substantial predictive power for materials synthesis by identifying the stability regions of target phases. However, thermodynamic phase diagrams do not offer explicit information regarding the kinetic competitiveness of undesired by-product phases. Here we propose a quantitative and computable thermodynamic metric to identify synthesis conditions under which the propensity to form kinetically competing by-products is minimized. We hypothesize that thermodynamic competition is minimized when the difference in free energy between a target phase and the minimal energy of all other competing phases is maximized. We validate this hypothesis for aqueous materials synthesis through two empirical approaches: first, by analysing 331 aqueous synthesis recipes text-mined from the literature; and second, by systematic experimental synthesis of LiIn(IO 3 ) 4 and LiFePO 4 across a wide range of aqueous electrochemical conditions. Our results show that even for synthesis conditions that are within the stability region of a thermodynamic Pourbaix diagram, phase-pure synthesis occurs only when thermodynamic competition with undesired phases is minimized.

Wang, Zheren (ORCID:0000000227429451)↗

NASA NTRS2004

A Preliminary ZEUS Lightning Location Error Analysis Using a Modified Retrieval Theory

The ZEUS long-range VLF arrival time difference lightning detection network now covers both Europe and Africa, and there are plans for further expansion into the western hemisphere. In order to fully optimize and assess ZEUS lightning location retrieval errors and to determine the best placement of future receivers expected to be added to the network, a software package is being developed jointly between the NASA Marshall Space Flight Center (MSFC) and the University of Nevada Las Vegas (UNLV). The software package, called the ZEUS Error Analysis for Lightning (ZEAL), will be used to obtain global scale lightning location retrieval error maps using both a Monte Carlo approach and chi-squared curvature matrix theory. At the core of ZEAL will be an implementation of an Iterative Oblate (IO) lightning location retrieval method recently developed at MSFC. The IO method will be appropriately modified to account for variable wave propagation speed, and the new retrieval results will be compared with the current ZEUS retrieval algorithm to assess potential improvements. In this preliminary ZEAL work effort, we defined 5000 source locations evenly distributed across the Earth. We then used the existing (as well as potential future ZEUS sites) to simulate arrival time data between source and ZEUS site. A total of 100 sources were considered at each of the 5000 locations, and timing errors were selected from a normal distribution having a mean of 0 seconds and a standard deviation of 20 microseconds. This simulated "noisy" dataset was analyzed using the IO algorithm to estimate source locations. The exact locations were compared with the retrieved locations, and the results are summarized via several color-coded "error maps."

Elander, Valjean↗

DOE OSTI2020

GPU Direct I/O with HDF5

Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.

Ravi, J↗