Search NASA⌕ Search

SEARCH · Search NASA

Results for “io optimizations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

SCORPIO: A scalable two-phase parallel I/O library with application to a large scale subsurface simulator

Inefficient parallel I/O is known to be a major bottleneck among scientific applications employed on supercomputers as the number of processor cores grows into the thousands. Our prior experience indicated that parallel I/O libraries such as HDF5 that rely on MPI-IO do not scale well beyond 10K processor cores, especially on parallel file systems (like Lustre) with single point of resource contention. Our previous optimization efforts for a massively parallel multi-phase and multi-component subsurface simulator (PFLOTRAN) led to a two-phase I/O approach at the application level where a set of designated processes participate in the I/O process by splitting the I/O operation into a communication phase and a disk I/O phase. The designated I/O processes are created by splitting the MPI global communicator into multiple sub-communicators. The root process in each sub-communicator is responsible for performing the I/O operations for the entire group and then distributing the data to rest of the group. This approach resulted in over 25X speedup in HDF I/O read performance and 3X speedup in write performance for PFLOTRAN at over 100K processor cores on the ORNL Jaguar supercomputer. This research describes the design and development of a general purpose parallel I/O library called Scorpio that incorporates our optimized two-phase I/O approach. The library provides a simplified higher level abstraction to the user, sitting atop existing parallel I/O libraries (such as HDF5) and implements optimized I/O access patterns that can scale on larger number of processors. Performance results with standard benchmark problems and PFLOTRAN indicate that our library is able to maintain the same speedups as before with the added flexibility of being applicable to a wider range of I/O intensive applications.

Sreepathi, Sarat↗
DOE OSTI2013

SCORPIO: A Scalable Two-Phase Parallel I/O Library With Application To A Large Scale Subsurface Simulator

Inefficient parallel I/O is known to be a major bottleneck among scientific applications employed on supercomputers as the number of processor cores grows into the thousands. Our prior experience indicated that parallel I/O libraries such as HDF5 that rely on MPI-IO do not scale well beyond 10K processor cores, especially on parallel file systems (like Lustre) with single point of resource contention. Our previous optimization efforts for a massively parallel multi-phase and multi-component subsurface simulator (PFLOTRAN) led to a two-phase I/O approach at the application level where a set of designated processes participate in the I/O process by splitting the I/O operation into a communication phase and a disk I/O phase. The designated I/O processes are created by splitting the MPI global communicator into multiple sub-communicators. The root process in each sub-communicator is responsible for performing the I/O operations for the entire group and then distributing the data to rest of the group. This approach resulted in over 25X speedup in HDF I/O read performance and 3X speedup in write performance for PFLOTRAN at over 100K processor cores on the ORNL Jaguar supercomputer. This research describes the design and development of a general purpose parallel I/O library, SCORPIO (SCalable block-ORiented Parallel I/O) that incorporates our optimized two-phase I/O approach. The library provides a simplified higher level abstraction to the user, sitting atop existing parallel I/O libraries (such as HDF5) and implements optimized I/O access patterns that can scale on larger number of processors. Performance results with standard benchmark problems and PFLOTRAN indicate that our library is able to maintain the same speedups as before with the added flexibility of being applicable to a wider range of I/O intensive applications.

High Performance Computing↗
DOE OSTI2014

An Optimizing Compiler for Petascale I/O on Leadership-Class Architectures

In high-performance computing (HPC), parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our DOE project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final report summarizes the major achievements of the project and also points out promising future directions Two new sections in this report compared to the previous report are IOGenie and SSD/NVM-specific optimizations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

I/O Bottleneck Detection and Tuning: Connecting the Dots using Interactive Log Analysis

Using parallel file systems efficiently is a tricky problem due to inter-dependencies among multiple layers of I/O software, including high-level I/O libraries (HDF5, netCDF, etc.), MPI-IO, POSIX, and file systems (GPFS, Lustre, etc.). Profiling tools such as Darshan collect traces to help understand the I/O performance behavior. However, there are significant gaps in analyzing the collected traces and then applying tuning options offered by various layers of I/O software. Seeking to connect the dots between I/O bottleneck detection and tuning, we propose DXT Explorer, an interactive log analysis tool. In this paper, we present a case study using our interactive log analysis tool to identify and apply various I/O optimizations. We report an evaluation of performance improvement achieved for four I/O kernels extracted from science applications.

Bez, Jean Luca↗
DOE OSTI2021

A surface acoustic wave hydrogen sensor with tin doped indium oxide layers for intermediate temperatures

We report high temperature surface acoustic wave (SAW) gas sensors with conducting sensing layers require tuning of the sheet conductivity for optimal response. Conducting metal oxides are attractive sensing materials for their tunable electronic properties and high thermal stability, amongst others. Here, we have investigated the application of indium oxide (IO) and indium tin oxide (ITO) films on langasite (LGS)-based SAW reflective delay line sensor devices for monitoring hydrogen at 350 °C. Specifically, we modeled the effect of the IO and ITO sensing layer thickness on the wave velocity, attenuation, and effective electromechanical coefficient. This was followed by an experimental demonstration of tuning of the ITO film sheet conductivity by controlling the dopant concentration and yielding an improvement in the sensor sensitivity. The current study provides a pathway towards the development of conductivity-based sensing layers for high temperature SAW gas sensors with improved sensitivity.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2015

An Optimizing Compiler for Petascale I/O on Leadership Class Architectures

In high-performance computing systems, parallel I/O architectures usually have very complex hierarchies with multiple layers that collectively constitute an I/O stack, including high-level I/O libraries such as PnetCDF and HDF5, I/O middleware such as MPI-IO, and parallel file systems such as PVFS and Lustre. Our project explored automated instrumentation and compiler support for I/O intensive applications. Our project made significant progress towards understanding the complex I/O hierarchies of high-performance storage systems (including storage caches, HDDs, and SSDs), and designing and implementing state-of-the-art compiler/runtime system technology that targets I/O intensive HPC applications that target leadership class machine. This final report summarizes the major achievements of the project and also points out promising future directions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2012

Efficient Databases for MPC Microdata (Final Report)

The purpose of this grant was to develop the theory and practice of high-performance databases for massive streamed datasets. Over the last three years, we have developed fast indexing technology, that is, technology for rapidly ingesting data and storing that data so that it can be efficiently queried and analyzed. During this project we developed the technology so that high-bandwidth data streams can be indexed and queried efficiently. Our technology has been proven to work data sets composed of tens of billions of rows when the data streams arrives at over 40,000 rows per second. We achieved these numbers even on a single disk driven by two cores. Our work comprised (1) new write-optimized data structures with better asymptotic complexity than traditional structures, (2) implementation, and (3) benchmarking. We furthermore developed a prototype of TokuFS, a middleware layer that can handle microdata I/O packaged up in an MPI-IO abstraction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

exnehilo7/fielddata-lite-iOS-app

Field Expedition Routing and Navigation (FERN) is an IOS app funded by the Center for Bioenergy Innovation (CBI). It allows an end user to make trips to collect novel data points. It allows user to return to previously surveyed points to collect more data. It calculates the optimal route to use to sample points needed for return trips. A companion php/database application is required for full functionality. Links to this open source project are in the README. FERN can use the default GPS on an Iphone but it also optimized for Arrow Gold GPS. Other external GPS could be supported by modifying this code and adding the proper routines.

Hopp, Daniel↗
DOE OSTI2024

Performance Results on CPU/GPU Exascale Architectures for OMEGA: The Ocean Model for E3SM Global Applications

The US Department of Energy (DOE) conducts climate simulations on some of the world’s largest supercomputers. These exascale machines use heterogeneous architectures with both CPUs and GPUs, and scientific codes must adapt to make full use of this computing power. Los Alamos National Lab is developing Omega: The Ocean Model for E3SM Global Applications, which is specifically designed for modern exascale computers. It uses external libraries that have been optimized for a variety of architectures to run on different supercomputers. Omega is an unstructured-mesh ocean model based on TRiSK numerical methods. It will be the new ocean component of the DOE’s Energy Exascale Earth System Model (E3SM). The algorithms in Omega follow those of the current ocean component, MPAS-Ocean, but it will be written in C++ rather than Fortran to take advantage of the Kokkos performance portability library. Omega spatial operators are written as Kokkos kernels to run efficiently on both CPUs and GPUs. Work on Omega began in 2023 with a new C++ framework for unstructured mesh partitioning, halo exchanges, parallel IO, and Kokkos interfaces. The current version, Omega-0, is being developed to solve the shallow water equations and at present includes all of the tendency terms but not time stepping. Here we share the results of Omega-0 verification and performance testing. Verification includes unit tests implemented with CTest as well as convergence tests in Polaris, an in-house python package with a large suite of test problems. Performance tests compare simulations conducted on CPUs versus GPUs and across different architectures: tests are run on Frontier, which has AMD “Optimized 3rd Gen EPYC” CPUs and AMD MI250X GPUs, as well as Perlmutter, which is composed of AMD EPYC 7763 CPUs and NVIDIA A100 GPUs.

58 GEOSCIENCES↗
DOE OSTI2015

Photoelectron Spectroscopy and Theoretical Studies of Anion-pi Interactions: Binding Strength and Anion Specificity

Proposed in theory and confirmed to exist, anion–π interactions have been recognized as new and important non-covalent binding forces. Despite extensive theoretical studies, numerous crystal structural identifications, and a plethora of solution phase investigations, intrinsic anion–π interaction strengths that are free from complications of condensed phases’ environments, have not been directly measured in the gas phase. Herein we present a joint photoelectron spectroscopic and theoretical study on this subject, in which tetraoxacalix[2]arene[2]triazine 1, an electron-deficient and cavity self-tunable macrocyclic was used as a charge-neutral molecular host to probe its interactions with a series of anions with distinctly different shapes and charge states (spherical halides Cl⁻, Br⁻, I⁻, linear thiocyanate SCN⁻, trigonal planar nitrate NO₃⁻, pyramidic iodate IO₃⁻, and tetrahedral sulfate SO₄²⁻). The binding energies of the resultant gaseous 1:1 complexes (1•Cl⁻,1•Br⁻, 1•I⁻, 1•SCN⁻, 1•NO₃⁻, 1•IO₃⁻ and 1•SO₄²⁻) were directly measured experimentally, exhibiting substantial non-covalent interactions with pronounced anion specific effects. The binding strengths of Cl⁻, NO₃⁻, IO₃⁻ with 1 are found to be strongest among all singly charged anions, amounting to ca. 30 kcal/mol, but only about 40% of that between 1 and SO₄²⁻. Quantum chemical calculations reveal that all anions reside in the center of the cavity of 1 with anion–π binding motif in the complexes’ optimized structures, where 1 is seen to be able to self-regulate its cavity structure to accommodate anions of different geometries and three-dimensional shapes. Electron density surface and natural bond orbital charge distribution analysis further support anion–π binding formation. The calculated binding energies of the anions and 1 nicely reproduce the experimentally estimated electron binding energy increase. This work illustrates that size-selective photoelectron spectroscopy combined with theoretical calculations represent a powerful technique to probe intrinsic anion–π interactions and has potential to provide quantitative guest-host molecular binding strengths and unravel fundamental insights in specific anion recognitions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimal thermodynamic conditions to minimize kinetic by-products in aqueous materials synthesis

Abstract Phase diagrams offer substantial predictive power for materials synthesis by identifying the stability regions of target phases. However, thermodynamic phase diagrams do not offer explicit information regarding the kinetic competitiveness of undesired by-product phases. Here we propose a quantitative and computable thermodynamic metric to identify synthesis conditions under which the propensity to form kinetically competing by-products is minimized. We hypothesize that thermodynamic competition is minimized when the difference in free energy between a target phase and the minimal energy of all other competing phases is maximized. We validate this hypothesis for aqueous materials synthesis through two empirical approaches: first, by analysing 331 aqueous synthesis recipes text-mined from the literature; and second, by systematic experimental synthesis of LiIn(IO 3 ) 4 and LiFePO 4 across a wide range of aqueous electrochemical conditions. Our results show that even for synthesis conditions that are within the stability region of a thermodynamic Pourbaix diagram, phase-pure synthesis occurs only when thermodynamic competition with undesired phases is minimized.

Wang, Zheren (ORCID:0000000227429451)↗
DOE OSTI2020

GPU Direct I/O with HDF5

Exascale HPC systems are being designed with accelerators, such as GPUs, to accelerate parts of applications. In machine learning workloads as well as large-scale simulations that use GPUs as accelerators, the CPU (or host) memory is currently used as a buffer for data transfers between GPU (or device) memory and the file system. If the CPU does not need to operate on the data, then this is sub-optimal because it wastes host memory by reserving space for duplicated data. Furthermore, this “bounce buffer” approach wastes CPU cycles spent on transferring data. A new technique, NVIDIA GPUDirect Storage (GDS), can eliminate the need to use the host memory as a bounce buffer. Thereby, it becomes possible to transfer data directly between the device memory and the file system. This direct data path shortens latency by omitting the extra copy and enables higher-bandwidth. To take full advantage of GDS in existing applications, it is necessary to provide support with existing I/O libraries, such as HDF5 and MPI-IO, which are heavily used in applications. In this paper, we describe our effort of integrating GDS with HDF5, the top I/O library at NERSC and at DOE leadership computing facilities. We design and implement this integration using a HDF5 Virtual File Driver (VFD). The GDS VFD provides a file system abstraction to the application that allows HDF5 applications to perform I/O without the need to move data between CPUs and GPUs explicitly. We compare performance of the HDF5 GDS VFD with explicit data movement approaches and demonstrate superior performance with the GDS method.

Ravi, J↗
DOE OSTI2019

Measurements and analytics of wide-area file transfers over dedicated connections

Distributed scientific and big-data computations are becoming increasingly dependent on access to remote files. Wide-area file transfers are supported by two basic schemes: (i) application-level tools, such as GridFTP, that provide transport services between file systems housed at geographically separated sites, and (ii) file systems mounted over wide-area networks, using mechanisms such as LNet routers that make them transparently available. In both cases, the file transfer performance critically depends on the configuration consisting of host, file, IO, and disk subsystems, which are complex by themselves, as well as on their complex compositions implemented using buffers and IO-network data transitions. We present extensive file transfer rate measurements collected over dedicated 10 Gbps connections with 0-366 ms round-trip times, using GridFTP and XDD file transfer tools, and Lustre file system extended over wide-area networks using LNet routers. Our test configurations are composed of: three types of host systems; XFS, Lustre, and ext3 file systems; and Ethernet and SONET wide-area connections. We present analytics based on the convexity-concavity of throughput profiles which provide insights into throughput and its superior or inferior trend compared to linear interpolations. We propose the utilization-concavity coefficient, a scalar metric that characterizes the overall performance of any file transfer method consisting of specific configuration and scheme. Our results enable performance optimizations by highlighting the significant roles of (i) buffer sizes and parallelism in GridFTP and XDD, and (ii) buffer utilization and credit mechanism in LNet routers.

Rao, Nageswara↗
DOE OSTI2021

Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression

Lossy compression techniques have demonstrated promising results in significantly reducing the scientific data size while guaranteeing the compression error bounds. However, one important yet often neglected side effect of lossy scientific data compression is its impact on the performance of parallel I/O. Our key observation is that the compressed data size is often highly skewed across processes in lossy scientific compression. To understand this behavior, we conduct extensive experiments where we apply three lossy compressors MGARD, ZFP, and SZ, which are specifically designed and optimized for scientific data, to three real-world scientific applications Gray-Scott simulation, WarpX, and XGC. Our analysis result demonstrates that the size of the compressed data is always skewed even if the original data is evenly decomposed among processes. Such skewness widely exists in different scientific applications using different compressors as long as the information density of the data varies across processes. We then systematically study how this side effect of lossy scientific data compression impacts the performance of parallel I/O. We observe that the skewness in the sizes of the compressed data often leads to I/O imbalance, which can significantly reduce the efficiency of I/O bandwidth utilization if not properly handled. In addition, writing data concurrently to a single shared file through MPI-IO library is more sensitive to the unbalanced I/O loads. Therefore, we believe our research community should pay more attention to the unbalanced parallel I/O caused by lossy scientific data compression.

Wang, Xinying↗
DOE OSTI2023

Spectroscopic features of dissolved iodine in pristine and gamma-irradiated nitric acid solutions

While iodine speciation is important for a wide range of nuclear activities, understanding the mechanisms of the transformations of iodine between chemical forms and the sensitivity of these transitions to solution conditions and exposure to radiation remains an active area of research. This work curates spectroscopic data from several experimental techniques and establishes their sensitivity and limitations in detecting changes in iodine speciation in both neutral and acidic regimes. The techniques include Raman spectroscopy, Fourier Transform Infrared (FTIR) spectroscopy, 127 I NMR spectroscopy, and ultraviolet-visible (UV-Vis) spectroscopy. Analysis of these data indicates that these commonly accessible spectroscopies often have dynamic ranges of measurable concentrations that do not always overlap between all techniques. The experimental techniques are disparately sensitive to iodide (I - ), molecular iodine (I 2 ), iodate (IO 3 - ), and periodate (IO 4 - ) species. Raman, FTIR, and NMR spectra were subsequentially analyzed using two-dimensional correlation analyses to generate high-resolution autocorrelation spectra. Here, the use of these spectroscopies is then extended to tracking acidification-induced and gamma irradiation-induced transformations of dissolved sodium iodate in deionized water and concentrated nitric acid. Both dissolution into nitric acid and irradiation with a gamma source are demonstrated to perturb the iodine speciation promoting their assembly into molecular iodine (I 2 ) and/or triiodide (I 3 - ). While I 2 and I 3 - species are undetectable with FTIR spectroscopy and 127 I NMR spectroscopy, the species can be detected with UV-Vis spectroscopy, and in some instances, I 3 - can be detected with Raman spectroscopy in the low wavenumber region. Ultimately, the results of this work provide a path to designing optimal combinations of techniques to detect forms of iodine across a wide range of concentrations and conditions.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2022

Throughput Analytics of Cloud Networks

A network of virtual machines at cloud server sites connected over virtual IO connections is a flexible, easily deployable, and cost-effective alternative to a physical network infrastructure with dedicated servers connected over leased fiber lines. We study the throughput performance of such a cloud network by collecting measurements over Google Cloud infrastructure spanning multiple continents. To study its ideal performance and impact of packet losses, we utilize its emulation using dedicated servers and connection hardware emulation devices. We compare the measurements over the cloud network to those over its emulation on a testbed. We examine the throughput profiles of both networks as a function of the round trip time and their utilization-concavity coefficients, estimated using measurements for common TCP versions. The throughput profile's concave-convex shape and its coefficient are critical indicators of the network performance, qualitatively and quantitatively, respectively. The results indicate their overall agreement between the production cloud network and its emulation using dedicated connections, and a near optimal throughput performance of the former except for a few under-performing connections. Also, the number of parallel flows is found to be a dominant factor in optimizing the throughput across various conditions and TCP versions.

Phanekham, Derek↗
DOE OSTI2015

Performance of an RPM based on Gd-lined plastic scintillator for neutron and gamma detection [ANIMMA--2015-IO-372]

A Radiation Portal Monitor (RPM) was developed by the Istituto Nazionale di Fisica Nucleare (INFN) and Ansaldo Nucleare (ANN) within the FP7 SCINTILLA European project. The system was designed to detect both gamma and neutron radiation with a single technology. It is conceived to monitor vehicle and cargo containers in transits across borders or ports, to find radioactive elements and to avoid illegal trafficking of strategic nuclear materials. The system is based on a {sup 3}He-free neutron detection technology using plastic scintillators coupled to Gadolinium to detect and discriminate gamma from neutron signals. During the 3 years of the SCINTILLA project the construction and test of the first two prototypes drove the definition of the final layout of a full RPM system consisting of two twin pillars as a portal for vehicle and cargo container scan. A custom System Control Software (SCS) manages the electronics of the RPM, the ancillary devices and the data analysis. The combination of the detector layout and of the software functionalities enables both to distinguish neutrons and gammas and to identify the energy range of a detected gamma source. The system was initially characterized via static tests with gamma and neutron sources in the INFN laboratory. These measurements were used to calibrate the detector, evaluate the response of the single pillars as well as of the full system, and optimize the RPM configuration and discrimination algorithm. During this phase, specific tests were performed to study the stability over time of the system, monitoring the measured the neutron and gamma count rates over periods of several weeks. The results allow us to demonstrate the reliability and robustness of the RPM. In a second time the RPM performance was studied via dynamic tests performed during the SCINTILLA test and benchmark campaigns. These measurements took place in the JRC ITRAP+10 facility at Ispra (Varese-Italy). The laboratory is equipped with an experimental set-up for dynamic tests of multiple systems according to international standards. The performed measurements utilized radioactive sources with activities selected according to ANSI and IEC standards to test the detector alarm performances in terms of gamma and neutron response, sensitivity to high gamma fields, sensitivity to moderated neutron sources as well as false alarm rates (FAR). In addition, the RPM was tested in challenging configurations exceeding the requirements set by international standards to determine the real limits of the system. The results obtained during these campaigns demonstrated that the system detection efficiency is not only compliant to international standards for its category, but often exceeds them, demonstrating the validity of the chosen technology and of the implemented layout. The positive performance also showed the effectiveness of the SCS and of its functionalities. To further demonstrate the system capabilities, a test in a real-life environment of the RPM is planned to happen in a near future by installing the detectors in a seaport. In this presentation I will give an overview of the RPM characteristics, of its performances as determined in the test campaign mentioned above and of future plans, to demonstrate how this technology can be an effective choice for the realization of {sup 3}He-free RPM detectors. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗