SEARCH · Search NASA

Results for “io optimizations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

45 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Measurements and analytics of wide-area file transfers over dedicated connections

Distributed scientific and big-data computations are becoming increasingly dependent on access to remote files. Wide-area file transfers are supported by two basic schemes: (i) application-level tools, such as GridFTP, that provide transport services between file systems housed at geographically separated sites, and (ii) file systems mounted over wide-area networks, using mechanisms such as LNet routers that make them transparently available. In both cases, the file transfer performance critically depends on the configuration consisting of host, file, IO, and disk subsystems, which are complex by themselves, as well as on their complex compositions implemented using buffers and IO-network data transitions. We present extensive file transfer rate measurements collected over dedicated 10 Gbps connections with 0-366 ms round-trip times, using GridFTP and XDD file transfer tools, and Lustre file system extended over wide-area networks using LNet routers. Our test configurations are composed of: three types of host systems; XFS, Lustre, and ext3 file systems; and Ethernet and SONET wide-area connections. We present analytics based on the convexity-concavity of throughput profiles which provide insights into throughput and its superior or inferior trend compared to linear interpolations. We propose the utilization-concavity coefficient, a scalar metric that characterizes the overall performance of any file transfer method consisting of specific configuration and scheme. Our results enable performance optimizations by highlighting the significant roles of (i) buffer sizes and parallelism in GridFTP and XDD, and (ii) buffer utilization and credit mechanism in LNet routers.

Rao, Nageswara↗

DOE OSTI2021

Unbalanced Parallel I/O: An Often-Neglected Side Effect of Lossy Scientific Data Compression

Lossy compression techniques have demonstrated promising results in significantly reducing the scientific data size while guaranteeing the compression error bounds. However, one important yet often neglected side effect of lossy scientific data compression is its impact on the performance of parallel I/O. Our key observation is that the compressed data size is often highly skewed across processes in lossy scientific compression. To understand this behavior, we conduct extensive experiments where we apply three lossy compressors MGARD, ZFP, and SZ, which are specifically designed and optimized for scientific data, to three real-world scientific applications Gray-Scott simulation, WarpX, and XGC. Our analysis result demonstrates that the size of the compressed data is always skewed even if the original data is evenly decomposed among processes. Such skewness widely exists in different scientific applications using different compressors as long as the information density of the data varies across processes. We then systematically study how this side effect of lossy scientific data compression impacts the performance of parallel I/O. We observe that the skewness in the sizes of the compressed data often leads to I/O imbalance, which can significantly reduce the efficiency of I/O bandwidth utilization if not properly handled. In addition, writing data concurrently to a single shared file through MPI-IO library is more sensitive to the unbalanced I/O loads. Therefore, we believe our research community should pay more attention to the unbalanced parallel I/O caused by lossy scientific data compression.

Wang, Xinying↗

DOE OSTI2023

Spectroscopic features of dissolved iodine in pristine and gamma-irradiated nitric acid solutions

While iodine speciation is important for a wide range of nuclear activities, understanding the mechanisms of the transformations of iodine between chemical forms and the sensitivity of these transitions to solution conditions and exposure to radiation remains an active area of research. This work curates spectroscopic data from several experimental techniques and establishes their sensitivity and limitations in detecting changes in iodine speciation in both neutral and acidic regimes. The techniques include Raman spectroscopy, Fourier Transform Infrared (FTIR) spectroscopy, 127 I NMR spectroscopy, and ultraviolet-visible (UV-Vis) spectroscopy. Analysis of these data indicates that these commonly accessible spectroscopies often have dynamic ranges of measurable concentrations that do not always overlap between all techniques. The experimental techniques are disparately sensitive to iodide (I - ), molecular iodine (I 2 ), iodate (IO 3 - ), and periodate (IO 4 - ) species. Raman, FTIR, and NMR spectra were subsequentially analyzed using two-dimensional correlation analyses to generate high-resolution autocorrelation spectra. Here, the use of these spectroscopies is then extended to tracking acidification-induced and gamma irradiation-induced transformations of dissolved sodium iodate in deionized water and concentrated nitric acid. Both dissolution into nitric acid and irradiation with a gamma source are demonstrated to perturb the iodine speciation promoting their assembly into molecular iodine (I 2 ) and/or triiodide (I 3 - ). While I 2 and I 3 - species are undetectable with FTIR spectroscopy and 127 I NMR spectroscopy, the species can be detected with UV-Vis spectroscopy, and in some instances, I 3 - can be detected with Raman spectroscopy in the low wavenumber region. Ultimately, the results of this work provide a path to designing optimal combinations of techniques to detect forms of iodine across a wide range of concentrations and conditions.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

DOE OSTI2022

Throughput Analytics of Cloud Networks

A network of virtual machines at cloud server sites connected over virtual IO connections is a flexible, easily deployable, and cost-effective alternative to a physical network infrastructure with dedicated servers connected over leased fiber lines. We study the throughput performance of such a cloud network by collecting measurements over Google Cloud infrastructure spanning multiple continents. To study its ideal performance and impact of packet losses, we utilize its emulation using dedicated servers and connection hardware emulation devices. We compare the measurements over the cloud network to those over its emulation on a testbed. We examine the throughput profiles of both networks as a function of the round trip time and their utilization-concavity coefficients, estimated using measurements for common TCP versions. The throughput profile's concave-convex shape and its coefficient are critical indicators of the network performance, qualitatively and quantitatively, respectively. The results indicate their overall agreement between the production cloud network and its emulation using dedicated connections, and a near optimal throughput performance of the former except for a few under-performing connections. Also, the number of parallel flows is found to be a dominant factor in optimizing the throughput across various conditions and TCP versions.

Phanekham, Derek↗

DOE OSTI2015

Performance of an RPM based on Gd-lined plastic scintillator for neutron and gamma detection [ANIMMA--2015-IO-372]

A Radiation Portal Monitor (RPM) was developed by the Istituto Nazionale di Fisica Nucleare (INFN) and Ansaldo Nucleare (ANN) within the FP7 SCINTILLA European project. The system was designed to detect both gamma and neutron radiation with a single technology. It is conceived to monitor vehicle and cargo containers in transits across borders or ports, to find radioactive elements and to avoid illegal trafficking of strategic nuclear materials. The system is based on a {sup 3}He-free neutron detection technology using plastic scintillators coupled to Gadolinium to detect and discriminate gamma from neutron signals. During the 3 years of the SCINTILLA project the construction and test of the first two prototypes drove the definition of the final layout of a full RPM system consisting of two twin pillars as a portal for vehicle and cargo container scan. A custom System Control Software (SCS) manages the electronics of the RPM, the ancillary devices and the data analysis. The combination of the detector layout and of the software functionalities enables both to distinguish neutrons and gammas and to identify the energy range of a detected gamma source. The system was initially characterized via static tests with gamma and neutron sources in the INFN laboratory. These measurements were used to calibrate the detector, evaluate the response of the single pillars as well as of the full system, and optimize the RPM configuration and discrimination algorithm. During this phase, specific tests were performed to study the stability over time of the system, monitoring the measured the neutron and gamma count rates over periods of several weeks. The results allow us to demonstrate the reliability and robustness of the RPM. In a second time the RPM performance was studied via dynamic tests performed during the SCINTILLA test and benchmark campaigns. These measurements took place in the JRC ITRAP+10 facility at Ispra (Varese-Italy). The laboratory is equipped with an experimental set-up for dynamic tests of multiple systems according to international standards. The performed measurements utilized radioactive sources with activities selected according to ANSI and IEC standards to test the detector alarm performances in terms of gamma and neutron response, sensitivity to high gamma fields, sensitivity to moderated neutron sources as well as false alarm rates (FAR). In addition, the RPM was tested in challenging configurations exceeding the requirements set by international standards to determine the real limits of the system. The results obtained during these campaigns demonstrated that the system detection efficiency is not only compliant to international standards for its category, but often exceeds them, demonstrating the validity of the chosen technology and of the implemented layout. The positive performance also showed the effectiveness of the SCS and of its functionalities. To further demonstrate the system capabilities, a test in a real-life environment of the RPM is planned to happen in a near future by installing the detectors in a seaport. In this presentation I will give an overview of the RPM characteristics, of its performances as determined in the test campaign mentioned above and of future plans, to demonstrate how this technology can be an effective choice for the realization of {sup 3}He-free RPM detectors. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2020

CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows

The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2003

Sleep, Circadian Rhythms, and Performance During Space Shuttle Missions

Sleep and circadian rhythms may be disturbed during spaceflight, and these disturbances can affect crewmembers' performance during waking hours. The mechanisms underlying sleep and circadian rhythm disturbances in space are not well understood, and effective countermeasures are not yet available. We investigated sleep, circadian rhythms, cognitive performance, and light-dark cycles in five astronauts prior to, during, and after the 16-day STS-90 mission and the IO-day STS-95 mission. The efficacy of low-dose, alternative-night, oral melatonin administration as a countermeasure for sleep disturbances was evaluated. During these missions, scheduled rest activity cycles were 20-35 minutes shorter than 24 hours. Light levels on the middeck and in the Spacelab were very low; whereas on the flight deck (which has several windows), they were highly variable. Circadian rhythm abnormalities were observed. During the second half of the missions, the rhythm of urinary cortisol appeared to be delayed relative to the sleep-wake schedule. Performance during wakefulness was impaired. Astronauts slept only about 6.5 hours per day, and subjective sleep quality was lower in space. No beneficial effects of melatonin (0.3 mg administered prior to sleep episodes on alternate nights) were observed. A surprising finding was a marked increase in rapid eye movement (REM) sleep upon return to Earth. We conclude that these Space Shuttle missions were associated with circadian rhythm disturbances, sleep loss, decrements in neurobehavioral performance, and alterations in REM sleep homeostasis. Shorter than 24-hour rest-activity schedules and exposure to light-dark cycles inadequate for optimal circadian synchronization may have contributed to these disturbances.

Neri, David F.↗

DOE OSTI2017

Challenges and Opportunities of User-Level File Systemsfor HPC

The performance gap between magnetic disks and data processing on HPC systems has become that huge that an efficient data processing can only be achieved by introducing non-volatile memory (NVRAM) as a new storage tier. Although the benefits of hierarchical storage have been adequately demonstrated to the point that the newest leadership class HPC systems will employ burst buffers, critical questions remain for supporting hierarchical storage systems, including: How should we present hierarchical storage systems to user applications, such that they are easy to use and that application code is portable across systems? How should we manage data movement through a storage hierarchy for best performance and resilience of data? How do the particular I/O use cases mandate the way we manage data? There have been many efforts to explore this space in the form of file systems, with increasingly more implemented at the user level. This is because it is relatively easy to swap in new, specialized user-level file systems for use by applications on a case-by-case basis, as opposed to the current mainstream approach of using general-purpose, system-level file systems which may not be optimized for HPC workloads and must be installed by administrators. In contrast, file systems at the user level can be tailored for specific HPC workloads for high performance and can be used by applications without administrator intervention. Many user-level file system developers have found themselves “having to reinvent the wheel” to implement various optimizations in their file systems. Thus, a main goal of this meeting was to bring together experts in I/O performance, file systems, and storage, and collectively explore the space of current and future problems and solutions for I/O on hierarchical storage systems in order to begin a community effort in enabling user-level file system support for HPC systems. We had a lively week of learning about each other’s approaches as well as unique I/O use cases that can influence the design of a community-driven file and storage system standards. The agenda for this meeting contained talks from participants on the following high level topics: HPC storage and I/O support today; what do HPC users need for I/O; existing userlevel file system efforts; object stores and other alternative storage systems; and components for building user-level file systems. The talks were short and intended to be conversation starters for more in-depth discussions with the whole group. The participants engaged in lengthy discussions on various questions that arose from the talks including: Are we ready to program to a memory hierarchy versus block devices? Are the needs of HPC users reflected in our existing file systems and storage systems? Should we drop or keep POSIX moving forward? What do we mean when we say “user-level file system”? Do we all mean the same thing? How should the IO 500 benchmark be defined so it is fair and useful? and How are stage-in and stage-out actually going to work? The report for this seminar contains a record of the talks from the participants as well as the resulting discussions. Our hope is that the effort initiated during this seminar will result in long-term collaborations that will benefit the HPC community as a whole.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Collaborative Project: Understanding Climate Model Biases in Tropical Atlantic and Their Impact on Simulations of Extreme Climate Events

Recent studies have revealed that among all the tropical oceans, the tropical Atlantic has experienced the most pronounced warming trend over the 20th century. Many extreme climate events affecting the U.S., such as hurricanes, severe precipitation and drought events, are influenced by conditions in the Gulf of Mexico and the Atlantic Ocean. It is therefore imperative to have accurate simulations of the climatic mean and variability in the Atlantic region to be able to make credible projections of future climate change affecting the U.S. and other countries adjoining the Atlantic Ocean. Unfortunately, almost all global climate models exhibit large biases in their simulations of tropical Atlantic climate. The atmospheric convection simulation errors in the Amazon region and the associated errors in the trade wind simulations are hypothesized to be a leading cause of the tropical Atlantic biases in climate models. As global climate models have resolutions that are too coarse to resolve some of the atmospheric and oceanic processes responsible for the model biases, we propose to use a high-resolution coupled regional climate model (CRCM) framework to address the tropical bias issue. We propose to combine the expertise in tropical coupled atmosphere-ocean modeling at Texas A&M University (TAMU) and the coupled land-atmosphere modeling expertise at Pacific Northwest National Laboratory (PNNL) to develop a comprehensive CRCM for the Atlantic sector within a general and flexible modeling framework. The atmospheric component of the CRCM will be the NCAR WRF model and the oceanic component will be the Rutgers/UCLA ROMS. For the land component, we will use CLM modified at PNNL to include more detailed representations of vegetation and soil hydrology processes. The combined TAMU-PNNL CRCM model will be used to simulate the Atlantic climate, and the associated land-atmosphere-ocean interactions at a horizontal resolution of 9 km or finer. A particular focus of the model development effort will be to optimize the performance of WRF and ROMS over several thousand of cores by focusing on both the parallel communication libraries and the I/O interfaces, in order to achieve the sustained throughput needed to perform simulations on such fine resolution grids. The CRCM model will be developed within the framework of the Coupler (CPL7) software that is part of the NCAR Community Earth System Model (CESM). Through efforts at PNNL and within the community, WRF and CLM have already been coupled via CPL7. Using the flux coupler approach for the whole CRCM model will allow us to flexibly couple WRF, ROMS, and CLM with each model running on its own grid at different resolutions. In addition, this framework will allow us to easily port parameterizations between CESM and the CRCM, and potentially allow partial coupling between the CESM and the CRCM. TAMU and PNNL will contribute cooperatively to this research endeavor. The TAMU team led by Chang and Saravanan has considerable experience in studying atmosphere-ocean interactions within tropical Atlantic sector and will focus on modeling issues that relate to coupling WRF and ROMS. The PNNL team led by Leung has extensive expertise in atmosphere-land interaction and will be responsible for improving the land surface parameterization. Both teams will jointly work on integrating WRF-ROMS and WRF-CLM to couple WRF, ROMS, and CLM through CPL7. Montuoro of the TAMU Supercomputing Center will be responsible for improving the MPI and Parallel IO interfaces of the CRCM. Both teams will contribute to the design and execution of the proposed numerical experiments and jointly perform analysis of the numerical experiments.

54 ENVIRONMENTAL SCIENCES↗