Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel input/output”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Futility

The Futility package contains the following: 1) Definition of the size of integers and real numbers; 2) A generic Unit test harness; 3) Definitions for some basic extensions to the Fortran language: arbitrary length strings, a parameter list construct, exception handlers, command line processor, timers; 4) Geometry definitions: point, line, plane, box, cylinder, polyhedron; 5) File wrapper functions: standard Fortran input/output files, Fortran binary files, HDF5 files; 6) Parallel wrapper functions: MPI, and Open MP abstraction layers, partitioning algorithms; 7) Math utilities: BLAS, Matrix and Vector definitions, Linear Solver methods and wrappers for other TPLs (PETSC, MKL, etc), preconditioner classes; 8) Misc: random number generator, water saturation properties, sorting algorithms.

Collins, Benjamin S.↗
DOE OSTI2022

Modeling pre-Exascale AMR Parallel I/O Workloads via Proxy Applications

The present work investigates the modeling of preexascale input/output (I/O) workloads of Adaptive Mesh Refinement (AMR) simulations through a simple proxy application. We collect data from the AMReX Castro framework running on the Summit supercomputer for a wide range of scales and mesh partitions for the hydrodynamic Sedov case as a baseline to provide sufficient coverage to the formulated proxy model. The non-linear analysis data production rates are quantified as a function of a set of input parameters such as output frequency, grid size, number of levels, and the Courant-Friedrichs-Lewy (CFL) condition number for each rank, mesh level and simulation time step. Linear regression is then applied to formulate a simple analytical model which allows to translate AMReX inputs into MACSio proxy I/O application parameters, resulting in a simple “kernel” approximation for data production at each time step. Results show that MACSio can simulate actual AMReX nonlinear “static” I/O workloads to a certain degree of confidence on the Summit supercomputer using the present methodology. The goal is to provide an initial level of understanding of AMR I/O workloads via lightweight proxy applications models to facilitate autotune data management strategies in anticipation of exascale systems.

Godoy, William↗
DOE OSTI2015

CS-Studio Scan System Parallelization

For several years, the Control System Studio (CS-Studio) Scan System has successfully automated the operation of beam lines at the Oak Ridge National Laboratory (ORNL) High Flux Isotope Reactor (HFIR) and Spallation Neutron Source (SNS). As it is applied to additional beam lines, we need to support simultaneous adjustments of temperatures or motor positions. While this can be implemented via virtual motors or similar logic inside the Experimental Physics and Industrial Control System (EPICS) Input/Output Controllers (IOCs), doing so requires a priori knowledge of experimenters requirements. By adding support for the parallel control of multiple process variables (PVs) to the Scan System, we can better support ad hoc automation of experiments that benefit from such simultaneous PV adjustments.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Julienne v1.0.0

Julienne is a compiler-portable unit-testing framework for Fortran software projects, including those that use the parallel/accelerator-programming features of Fortran 2023. Julienne achieves portability across compilers through minimalism and isolation. The minimal design ensures that Julienne uses only features supported by the majority of Fortran compilers. The isolation through zero dependencies ensures that no other projects block Julienne from building with a particular compiler. Julienne also contains with additional services that support its unit-testing code. These include functions for manipulating strings, command lines and input/output format strings; and a user-defined collective subroutine for verifying that all processes pass a test in parallel testing. Julienne's name derives from the term for vegetables sliced into thin strings: julienne vegetables. Julienne captures the authors' most frequently used thin slice of the Veggies and Sourcery software repositories while avoiding certain compiler limitations of the those two packages.

Rouson, Damian↗
DOE OSTI2020

High-performance computing in water resources hydrodynamics

In this work, we present a vision of future water resources hydrodynamics codes that can fully utilize the strengths of modern high-performance computing. The advances to computing power, formerly driven by the improvement of central processing unit processors, now focus on parallel computing and, in particular, the use of graphics processing units (GPUs). However, this shift to a parallel framework requires refactoring the code to make efficient use of the data as well as changing even the nature of the algorithm that solves the system of equations. These concepts along with other features such as the precision for the computations, dry regions management, and input/output data are analyzed in this paper. A 2D multi-GPU flood code applied to a large-scale test case is used to corroborate our statements and ascertain the new challenges for the next-generation parallel water resources codes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Kip, Version 1.0

This high-performance ray tracing library provides very fast rendering, compact code, type flexibility via C++ 'generic programming' techniques, and ease of use through an application programming interface (API) that functions independently of any GUI, on-screen display, or other enclosing application. Kip supports constructive solid geometry models based on any of several built-in shapes and operators and allows for new, user-defined shapes and operators. Additional features include basic texturing, input/output of models using a simple human-readable file format with full error checking and detailed diagnostics, and support for shared data parallelism. Kip is written in pure, ANSI Standard C++, is entirely platform independent, and is very easy to use, as it requires no build system, configuration or installation scripts, wizards, special preprocessing, makefiles, or external libraries.

Staley, Martin↗
DOE OSTI2016

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulk data movement framework for terabit networks, called LADS. LADS exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to benefit from hardware-level zero-copy, and operating system bypass capabilities when available. It can further improve data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and data transfer rates across the high speed networks. We also investigate the performance degradation problems of LADS due to I/O contention on the parallel file system (PFS), when multiple LADS tools share the PFS. We design and evaluate a meta-scheduler to coordinate multiple I/O streams while sharing the PFS, to minimize the I/O contention on the PFS. Finally, with our evaluations, we observe that LADS with meta-scheduling can further improve the performance by up to 14 percent relative to LADS without meta-scheduling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Parallel IO Libraries for Managing HEP Experimental Data

The computing and storage requirements of the energy and intensity frontiers will grow significantly during the Run 4 & 5 and the HL-LHC era. Similarly, in the intensity frontier, with larger trig ger readouts during supernovae explosions, the Deep Underground Neutrino Experiment (DUNE) will have unique computing challenges that could be addressed by the use of parallel and accelerated dataprocessing capabilities. Most of the requirements of the energy and intensity frontier experiments rely on increasing the role of high performance computing (HPC) in the HEP community. In this presentation, we will describe our ongoing efforts that are focused on using HPC resources for the next generation HEP experiments. The HEPCCE (High Energy Physics-Center for Computational Excellence) IOS (Input/Output and Storage) group has been developing approaches to map HEP data to the HDF5 , an IO library optimized for the HPC platforms to store the intermediate HEP data. The complex HEP data products are serialized using ROOT to allow for experiment independent general mapping approaches of the HEP data to the HDF5 format. The mapping approaches can be optimized for high performance parallel IO. Similarly, simpler data can be directly mapped into the HDF5, which can also be suitable for offloading into the GPUs directly. We will present our works on both complex and simple data model models.

Bashyal, Amit↗
DOE OSTI2021

Scalable FBP decomposition for cone-beam CT reconstruction

Filtered Back-Projection (FBP) is a fundamental compute intense algorithm used in tomographic image reconstruction. Cone-Beam Computed Tomography (CBCT) devices use a cone-shaped X-ray beam, in comparison to the parallel beam used in older CT generations. Distributed image reconstruction of cone-beam datasets typically relies on dividing batches of images into different nodes. This simple input decomposition, however, introduces limits on input/output sizes and scalability.We propose a novel decomposition scheme and reconstruction algorithm for distributed FPB. This scheme enables arbitrarily large input/output sizes, eliminates the redundancy arising in the end-to-end pipeline and improves the scalability by replacing two communication collectives with only one segmented reduction. Finally, we implement the proposed decomposition scheme in a framework that is useful for all current-generation CT devices (7th gen). In our experiments using up to 1024 GPUs, our framework can construct 40963 volumes, for real-world datasets, in under 16 seconds (including I/O).

Chen, Peng↗
DOE OSTI2024

DL_POLY Quantum 2.0: A modular general-purpose software for advanced path integral simulations

DL_POLY Quantum 2.0, a vastly expanded software based on DL_POLY Classic 1.10, is a highly parallelized computational suite written in FORTRAN77 with a modular structure for incorporating nuclear quantum effects into large-scale/long-time molecular dynamics simulations. This is achieved by presenting users with a wide selection of state-of-the-art dynamics methods that utilize the isomorphism between a classical ring polymer and Feynman’s path integral formalism of quantum mechanics. Here, the flexible and user-friendly input/output handling system allows the control of methodology, integration schemes, and thermostatting. DL_POLY Quantum is equipped with a module specifically assigned for calculating correlation functions and printing out the values for sought-after quantities, such as dipole moments and center-of-mass velocities, with packaged tools for calculating infrared absorption spectra and diffusion coefficients.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Pass-efficient methods for compression of high-dimensional turbulent flow data

The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of in situ compression algorithms to avoid overuse of memory. To ensure that post-processing operations are successful, this must be done in a way that a sufficiently accurate representation of the solution is stored. Moreover, in situations where the input/output system becomes a bottleneck in analysis, visualization, etc., or the execution of the PDE solver is expensive, the number of passes made over the data must be minimized. In the interest of addressing this problem, this work focuses on the utility of pass-efficient, parallelizable, low-rank, matrix decomposition methods in compressing high-dimensional simulation data from turbulent flows. Additionally, a particular emphasis is placed on using coarse representation of the data – compatible with the PDE discretization grid – to accelerate the construction of the low-rank factorization. This includes the presentation of a novel single-pass matrix decomposition algorithm for computing the so-called interpolative decomposition. The methods are described extensively and numerical experiments on two turbulent channel flow data are performed. In the first (unladen) channel flow case, compression factors exceeding 400 are achieved while maintaining accuracy with respect to first- and second-order flow statistics. In the particle-laden case, compression factors of 100 are achieved and the compressed data is used to recover particle velocities. These results show that these compression methods can enable efficient computation of various quantities of interest in both the carrier and disperse phases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Red Storm Case Study

Sandia and Cray Inc. co-developed Red Storm, a distributed memory, massively paralleled highperformance supercomputer modeled on ASCI Redl, to run computer codes used for conducting materials science simulations for national security. Supercomputers have some of the fastest highperformance systems available and are used primarily for scientific and engineering work requiring exceedingly high-speed computations. Unlike conventional computers, supercomputers have large storage capacity; more than one central processing unit to rapidly retrieve stored data and program instructions; and input/output capability.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Management and Storage of Scientific Data

Scientific discoveries rely heavily on efficient access, search, and management of massive data sets. Data management technologies have, for decades, provided foundational capabilities for scientific computing. Just as storage, input/output (I/O), and data management have been fundamental to simulation-based science for many years, so too are capable data-management technologies key to the success of today’s scientific workflows utilizing data intensive and machine learning (ML) techniques. The Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR) program has invested broadly in data-management research focused on high-performance computing (HPC) systems, from parallel file systems that store data to application software that makes these systems more productive. Still, advances in technology combined with growing diversity of supported science strongly motivate continued investment in this area. In January 2022, ASCR convened a workshop to identify priority research directions in the area of data management for high-performance and scientific computing. Attendees were challenged to identify promising approaches that would support the breadth of the DOE mission, including the explosion of artificial intelligence (AI) uses and the growing needs of experimental and observational science. Technological and science drivers were identified and considered as they relate to key aspects of data management such as interfaces, architectural design, and FAIR principles (Findable, Accessible, Interoperable, and Reusable). The thoughts of the workshop participants were distilled into a set of four priority research directions with the potential for high impact on DOE science. These research directions are summarized in the following pages.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2016

VERA and VERA-EDU 3.5 Release Notes

The Virtual Environment for Reactor Applications components included in this distribution include selected computational tools and supporting infrastructure that solve neutronics, thermal-hydraulics, fuel performance, and coupled neutronics-thermal hydraulics problems. The infrastructure components provide a simplified common user input capability and provide for the physics integration with data transfer and coupled-physics iterative solution algorithms. Neutronics analysis can be performed for 2D lattices, 2D core and 3D core problems for pressurized water reactor geometries that can be used to calculate criticality and fission rate distributions by pin for input fuel compositions. MPACT uses the Method of Characteristics transport approach for 2D problems. For 3D problems, MPACT uses the 2D/1D method which uses 2D MOC in a radial plane and diffusion or SPn in the axial direction. MPACT includes integrated cross section capabilities that provide problem-specific cross sections generated using the subgroup methodology. The code can be executed both 2D and 3D problems in parallel to reduce overall run time. A thermal-hydraulics capability is provided with CTF (an updated version of COBRA-TF) that allows thermal-hydraulics analyses for single and multiple assemblies using the simplified VERA common input. This distribution also includes coupled neutronics/thermal-hydraulics capabilities to allow calculations using MPACT coupled with CTF. The VERA fuel rod performance component BISON calculates, on a 2D or 3D basis, fuel rod temperature, fuel rod internal pressure, free gas volume, clad integrity and fuel rod waterside diameter. These capabilities allow simulation of power cycling, fuel conditioning and deconditioning, high burnup performance, power uprate scoping studies, and accident performance. Input/Output capabilities include the VERA Common Input (VERAIn) script which converts the ASCII common input file to the intermediate XML used to drive all of the physics codes in the VERA Core Simulator (VERA-CS). VERA component codes either input the VERA XML format directly, or provide a preprocessor which can convert the XML into native input. VERAView is an interactive graphical interface for the visualization and engineering analyses of output data from VERA. The python-based software is easy to install and intuitive to use, and provides instantaneous 2D and 3D images, 1D plots, and alpha-numeric data from VERA multi-physics simulations. Testing within CASL has focused primarily on Westinghouse four-loop reactor geometries and conditions with example problems included in the distribution.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Feedback lock-in: A versatile multi-terminal measurement system for electrical transport devices

Here, we present the design and implementation of a measurement system that enables parallel drive and detection of small currents and voltages at numerous electrical contacts to a multi-terminal electrical device. This system, which we term a feedback lock-in, combines digital control-loop feedback with software-defined lock-in measurements to dynamically source currents and measure small, pre-amplified potentials. The effective input impedance of each current/voltage probe can be set via software, permitting any given contact to behave as an open-circuit voltage lead or as a virtually grounded current source/sink. This enables programmatic switching of measurement configurations and permits measurement of currents at multiple drain contacts without the use of current preamplifiers. Our 32-channel implementation relies on commercially available digital input/output boards, home-built voltage preamplifiers, and custom open-source software. With our feedback lock-in, we demonstrate differential measurement sensitivity comparable to a widely used commercially available lock-in amplifier and perform efficient multi-terminal electrical transport measurements on twisted bilayer graphene and SrTiO 3 quantum point contacts. The feedback lock-in also enables a new style of measurement using multiple current probes, which we demonstrate on a ballistic graphene device.

47 OTHER INSTRUMENTATION↗
DOE OSTI2018

User-Oriented Modeling Tools for Advanced Hybrid and Climate-Appropriate Rooftop Air Conditioners

Hybrid unitary air conditioning systems offer a pathway to substantially reduce energy use and peak electrical demand for cooling, heating, and ventilation in commercial buildings. Hybrid air conditioners incorporate multiple subsystems that are carefully orchestrated to provide climate- and application-specific efficiency advantages. There are a multitude of hybrid system architectures, but common subsystems include: heat recovery ventilation, indirect evaporative cooling, desiccant dehumidification, variable speed fans, modulating dampers, and multi-stage or variable-speed vapor compression cooling. Categorically, hybrid systems can operate in numerous discrete modes. For example: indirect evaporative cooling may operate for periods when the subsystem provides adequate sensible cooling, then vapor compression cooling will be included when more cooling or dehumidification is necessary. Laboratory assessments, field studies, and simulations have demonstrated that hybrid unitary air conditioners could reduce energy use for cooling and ventilation by 30-90% depending on climate and application. Heretofore, it has been challenging - if not impossible - for practitioners to model hybrid air conditioners as part of building energy simulations; and the limitation has severely obstructed broader adoption of technologies in this class. In this project, we developed a new feature for EnergyPlus that enables modeling hybrid unitary air conditioning equipment for building energy simulations. This is a significant advancement for both theory and practice, and confers public benefit by enabling practitioners to evaluate this compelling efficiency technology as a part of building energy simulations. The feature is a black-box model that requires extensive performance data for each hybrid unitary product. In parallel, we also developed new features for the Technology Performance Exchange to enable manufacturers to submit performance data in a standard format that can be used with the hybrid unitary model in EnergyPlus. Additionally, through this project we expanded university educational resources, and university- manufacturing industry collaborations in the field of energy efficiency technology. Over two years, we involved 20 undergraduate students in ambitious research projects focused on modeling complex multi-mode mechanical systems, supported three mechanical engineering bachelor theses, established undergraduate apprenticeships with multiple industry partners, and involved those partners in the process of design, validation, and debugging for the new EnergyPlus feature. The EnergyPlus feature is described and discussed in an academic article, as well as in an engineering reference, and input/output reference documentation for EnergyPlus. The Technology Performance Exchange features are live and publicly accessible, our manufacturer partners are primed to submit initial product information and performance data to the exchange, and the EnergyPlus feature is scheduled for public release in Spring 2018 as a part of EnergyPlus v8.9.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2019

Parallel storage system with burst buffer appliance for storage of partitioned key-value store across a plurality of storage tiers

Parallel storage systems are provided with a burst buffer appliance for storage of a partitioned key-value store across a plurality of storage tiers. The burst buffer appliance is configured to store key-value data on first and second storage tiers comprising respective disjoint subsets of storage. One or more servers in the partitioned key-value store provide (i) an interface between one or more local key-value stores and one or more additional storage architectures, and/or (ii) a key value interface that provides access to at least one hardware key-value store. The local key-values are optionally sorted both locally by the local store and globally by the overall MDHIM framework. The local and global sorting can be leveraged to provide batch input/output (IO) operations that aggregate multiple requests into batch operations on one or more sorted ranges.

Bent, John M.↗
DOE OSTI2019

Accelerating DCA++ (Dynamical Cluster Approximation) Scientific Application on the Summit supercomputer

Optimizing scientific applications on today’s accelerator-based high performance computing systems can be challenging, especially when multiple GPUs and CPUs with heterogeneous memories and persistent non-volatile memories are present. An example is Summit, an accelerator-based system at the Oak Ridge Leadership Computing Facility (OLCF) that is rated as the world’s fastest supercomputer to-date. New strategies are thus needed to expose the parallelism in legacy applications, while being amenable to efficient mapping to the underlying architecture.In this paper we discuss our experiences and strategies to port a scientific application, DCA++, to Summit. DCA++ is a highperformance research application that solves quantum manybody problems with a cutting edge quantum cluster algorithm, the dynamical cluster approximation.Our strategies aim to synergize the strengths of the different programming models in the code. These include: (a) streamlining the interactions between the CPU threads and the GPUs, (b) implementing computing kernels on the GPUs and decreasing CPU-GPU memory transfers, (c) allowing asynchronous GPU communications, and (d) increasing compute intensity by combining linear algebraic operations.Full-scale production runs using all 4600 Summit nodes attained a peak performance of 73.5 PFLOPS with a mixed precision implementation.We observed a perfect strong and weak scaling for the quantum Monte Carlo solver in DCA++, while encountering about 2× input/output (I/O) and MPI communication overhead on the time-to-solution for the full machine run. Our hardware agnostic optimizations are designed to alleviate the communication and I/O challenges observed, while improving the compute intensity and obtaining optimal performance on a complex, hybrid architecture like Summit.

Balduzzi, Giovanni↗