Search NASA⌕ Search

SEARCH · Search NASA

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

Brim, Michael↗
DOE OSTI2023

5G Energy FRAME: The Design and Implementation of Data, Model, and Use Case (Year 2 Report)

This report summarizes the Year 2 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research (ASCR) Program. In this report, newest 5G equipment testing results are presented, along with two 5G-enabled AI/ML examples for grid applications; in addition, the work flow of grid edge, cloud, and High Performance Computing (HPC) platform is introduced, to support and interface the cross-domain simulation for power system transmission, distribution, and communication networks. Last but not least, the outlook for Year 3 work and the overarching impact of 5G Energy FRAME work to a multitude of stakeholders are provided. Additional 5G performance data now is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

ResStock: Annual Baseline Results with Component Loads

The ResStock Analysis Tool was developed by NREL with support from the U.S. Department of Energy to provide a new approach to large-scale residential analysis by combining large public and private data sources, statistical sampling, detailed sub hourly building simulations, and high-performance computing. This combination achieves unprecedented granularity and accuracy in modeling the diversity of the housing stock and the distributional impacts of building technologies in different communities. The annual baseline energy results from a national-scale ResStock run use typical meteorological year 3 (TMY3) files for energy simulations. Results include heating and cooling loads for individual components of each building. Component loads describe the heating/cooling load that can be attributed to specific elements of a home, such as heat transfer through walls or internal gains. Additionally, these results include the standard ResStock outputs for housing characteristics and numerous energy outputs by end-use and fuel. A snapshot of the ResStock version used to produce this data, including a configuration file for the run can be found using the Source Code resource link.

Array↗
DOE OSTI2024

Utilizing Distributed Heterogeneous Computing with PanDA in ATLAS

In recent years, advanced and complex analysis workflows have gained increasing importance in the ATLAS experiment at CERN, one of the large scientific experiments at LHC. Support for such workflows has allowed users to exploit remote computing resources and service providers distributed worldwide, overcoming limitations on local resources and services. The spectrum of computing options keeps increasing across the Worldwide LHC Computing Grid (WLCG), volunteer computing, high-performance computing, commercial clouds, and emerging service levels like Platform-as-a-Service (PaaS), Container-as-a-Service (CaaS) and Function-as-a-Service (FaaS), each one providing new advantages and constraints. Users can significantly benefit from these providers, but at the same time, it is cumbersome to deal with multiple providers, even in a single analysis workflow with fine-grained requirements coming from their applications’ nature and characteristics. In this paper, we will first highlight issues in geographically-distributed heterogeneous computing, such as the insulation of users from the complexities of dealing with remote providers, smart workload routing, complex resource provisioning, seamless execution of advanced workflows, workflow description, pseudointeractive analysis, and integration of PaaS, CaaS, and FaaS providers. We will also outline solutions developed in ATLAS with the Production and Distributed Analysis (PanDA) system and future challenges for LHC Run4.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Technical Characterization and Benefit Evaluation of 5G-Enabled Grid Data Transport and Applications

This report summarizes the Year 1 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research Program. 5G is a breakthrough technology that enables a fully mobile and connected society, and a 5G-enabled digital continuum will be one of the critical foundations for a clean energy economy and grid modernization. In collaboration with PNNL’s Advanced Wireless Communication team and Center for Advanced Technology Evaluation team, the project team has been evaluating the system performance of 5G testbeds in the PNNL 5G Innovation Studio, and has formulated a co-simulation test case of power system transmission, distribution, and communication (T&D&C) networks considering 5G technology and high penetration of distributed energy resources. The methodology developed in the 5G Energy FRAME project can be customized to fit different future grid scenarios to evaluate multiple (dynamic) configurations (computing, sensing, communication, environment) for different stakeholders. In summary, our main technical highlights in project Year 1 are as follows: 1) Technical characterization of 5G standalone architectures, 2) Formulation of co-simulation test case of T&D&C networks embedded with 5G, 3) Initial benefit evaluation of 5G communication platform for grid use cases, and 4) Additional extended discussions on edge computing, artificial intelligence and machine learning, and high-performance computing and cloud computing adoptions. In addition, a collection of system performance data is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

GridPACK: Library Architecture and Usability Assessment

The LLNL team has carried out extensive analysis of GridPACK code, including two major code reviews (each being a 2-day event) and a vendor emulation exercise, where the team integrated GridPACK library within a third-party application. The code analysis identified a number of areas for improvement of GridPACK code architecture, but it did not find fundamental flaws in the GridPACK design. The team found the GridPACK strategy of developing a software development kit (SDK) for power systems application development to be sound and well aligned with Office of Electricity long term objectives. The SDK aims to provide power systems software vendors an easy access to high performance computing (HPC) libraries.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Simulation of Reverse Osmosis Membrane Compaction Using Material Point Method

Reverse Osmosis (RO) is a promising technology to address the impending water-crisis in the upcoming decades. RO at high salinities and high pressures is challenging due to membrane compaction that changes its porosity and permeability. In this study, we present a simulation methodology for membrane structural mechanics to understand pore size distribution and permeability variations under high pressure. We use the material point method (MPM), that solves the solid mechanics equations in a Lagrangian framework. MPM provides many features that make it well-suited for simulating mesoporous membranes. The Lagrangian framework allows for large deformations, easy integration of constitutive models and direct import of complex geometries as particles. The spatial discretization in our MPM solver is achieved using linear or cubic-spline shape functions while the time integration is carried out using the explicit Euler method. A series of images containing detailed pore structures obtained from X-ray tomography is first converted to a collection of material points. Compressive loads are applied to the top layer of the membrane to simulate the application of pressure. The membrane deformation and pore size distribution before and after load application are reported and compared with the experimental measurements. The presentation discusses the numerical methods used, the performance of the solver on high-performance computing machines, and the results of membrane compact in detail.

compaction↗
DOE OSTI2024

Evaluating integration and performance of containerized climate applications on a Hewlett Packard Enterprise Cray system

Containers have taken over large swaths of cloud computing as the most convenient way of packaging and deploying applications. The features that containers offer for packaging and deploying applications translate to high performance computing (HPC) as well. At The National Oceanic and Atmospheric Administration, containers provide an easy way to build and distribute complex HPC applications, allowing faster collaboration, portability, and experiment computer environment reproducibility amongst the scientific community. The challenge arises when applications rely on message passing interface (MPI). This necessitates investigation into how to properly run these applications with their own unique requirements and produce performance on par with native runs. We investigate the MPI performance for benchmarks and containerized climate models for various containers covering selection of compiler and MPI library combinations from the Cray provided programming environments on the Cray XC supercomputer GAEA. Performance from the benchmarks and the climate models shows that for the most part containerized applications perform on par with the natively built applications when the system optimized Cray MPICH libraries are bound into the container, and the hybrid model containers have poor performance in comparison. We also describe several challenges and our solutions in running these containers, particularly challenges with heterogeneous jobs for the containerized model runs.

Abraham, Subil↗
DOE OSTI2016

Impact of Data Placement on Resilience in Large-Scale Object Storage Systems

Distributed object storage architectures have become the de facto standard for high-performance storage in big data, cloud, and HPC computing. Object storage deployments using commodity hardware to reduce costs often employ object replication as a method to achieve data resilience. Repairing object replicas after failure is a daunting task for systems with thousands of servers and billions of objects, however, and it is increasingly difficult to evaluate such scenarios at scale on realworld systems. Resilience and availability are both compromised if objects are not repaired in a timely manner. In this work we leverage a high-fidelity discrete-event simulation model to investigate replica reconstruction on large-scale object storage systems with thousands of servers, billions of objects, and petabytes of data. We evaluate the behavior of CRUSH, a well-known object placement algorithm, and identify configuration scenarios in which aggregate rebuild performance is constrained by object placement policies. After determining the root cause of this bottleneck, we then propose enhancements to CRUSH and the usage policies atop it to enable scalable replica reconstruction. We use these methods to demonstrate a simulated aggregate rebuild rate of 410 GiB/s (within 5% of projected ideal linear scaling) on a 1,024-node commodity storage system. We also uncover an unexpected phenomenon in rebuild performance based on the characteristics of the data stored on the system.

Carns, Philip↗
DOE OSTI2018

Fibre Channel Development Final Report CRADA No. TC-0340-92-C

The Fibre Channel (FC) Development project was designed to enhance U.S. industrial competitiveness by accelerating the standardization and subsequent industrial acceptance of the American National Standards Institute’s (ANSI) Fibre Channel Standard (now X3T1 1). It was also intended to give the principal U.S. participants a one-to two-year lead in developing heterogeneous high-speed computer networks, distributed computing through clustering, scientific visualization techniques, and high speed channels. The project was proposed to benefit U.S. industry as these high performance computing methods became generally available. In 1992-93, during the initial project phases, Fibre Channel (FC) was emerging as a new ANSI standard. It was envisioned as becoming the mechanism for data transfer between the next generation of computers, and between these computers and their peripherals. Using this channel standard, U.S. computer vendors were expecting to be able to provide high-speed interoperatillity between heterogeneous computers and their storage devices, with data transfer rates greater than a GBit/sec. Our project was composed of key principal players in the FC standardization effort from both U.S. industry and the Department of Energy’s@OEs) Lawrence Liverrnore National Laboratory (LLNL). The industrial participants were developing individual FC interfaces to operate with their platform products. LLNL’s.role was to be a neutral testing site and a facilitator for helping to resolve conflicting interpretations of the embryonic Fibre Channel specification. LLNL anticipated that conflicts in the interpretation of such a standard would arise and require neutral resolution and that such resolutions would best be achieved in cooperation with the ANSI FC Working Group. Project plans included transferring to U.S. industry any resilient FC testing capabilities at the project conclusion.

42 ENGINEERING↗
DOE OSTI2023

Simulation-based characterization of the variability of earthquake risk to buildings in the near-field

Recent advancements in high performance computing platforms and computational workflow for regional-scale simulations are enabling unprecedented modeling of fault-to-structure earthquake processes. Regional simulations resolving ground motions at frequencies relevant to engineered systems are becoming computationally viable and provide a new capability to improve understanding of the geographical distribution and intensity of risk to buildings and critical infrastructure. As computational capabilities advance, it is essential to move beyond illustrative single rupture realizations for scenario earthquake events towards the development of a full suite of rupture realizations that appropriately characterize the range of risk to building systems. The work described in this article investigates the application of a suite of fault rupture realizations with the objective of assessing near-fault, site-specific seismic demand variability for building structures. A representative high-performance regional-scale computational model is utilized to execute ground motion and building response simulations based on 18 kinematic rupture realizations of an M7 strike-slip scenario earthquake. The fault rupture models for the scenario earthquake are created by systematically perturbing the hypocenter location and stochastically generating rupture parameters (slip, rise time, rake angle) to represent a breadth of ground motion intensities resulting from the spatial and temporal variabilities of an earthquake rupture process. The resulting seismic demand variability for three-story (short period) and forty-story (long period) steel moment-resisting frame buildings is characterized in terms of the median and distribution of peak inter-story drift ratio for a range of near-fault sites. The full suite of 18 fault rupture realizations and approximately 280,000 nonlinear dynamic building simulations indicate that the three-story building undergoes higher median seismic demand and significantly greater variability of demand at a given site than the forty-story building, which has important implications for the level of certainty in predicting building performance during an earthquake. The simulations performed provide deeper insight into the relationship between fault rupture parameterization and building response, which is essential information for developing a representative suite of rupture realizations for specific earthquake scenarios.

58 GEOSCIENCES↗
DOE OSTI2018

A computational fluid dynamics simulation framework for ventricular catheter design optimization

OBJECTIVE Cerebrospinal fluid (CSF) shunts are the primary treatment for patients suffering from hydrocephalus. While proven effective in symptom relief, these shunt systems are plagued by high failure rates and often require repeated revision surgeries to replace malfunctioning components. One of the leading causes of CSF shunt failure is obstruction of the ventricular catheter by aggregations of cells, proteins, blood clots, or fronds of choroid plexus that occlude the catheter’s small inlet holes or even the full internal catheter lumen. Such obstructions can disrupt CSF diversion out of the ventricular system or impede it entirely. Previous studies have suggested that altering the catheter’s fluid dynamics may help to reduce the likelihood of complete ventricular catheter failure caused by obstruction. However, systematic correlation between a ventricular catheter’s design parameters and its performance, specifically its likelihood to become occluded, still remains unknown. Therefore, an automated, open-source computational fluid dynamics (CFD) simulation framework was developed for use in the medical community to determine optimized ventricular catheter designs and to rapidly explore parameter influence for a given flow objective. METHODS The computational framework was developed by coupling a 3D CFD solver and an iterative optimization algorithm and was implemented in a high-performance computing environment. The capabilities of the framework were demonstrated by computing an optimized ventricular catheter design that provides uniform flow rates through the catheter’s inlet holes, a common design objective in the literature. The baseline computational model was validated using 3D nuclear imaging to provide flow velocities at the inlet holes and through the catheter. RESULTS The optimized catheter design achieved through use of the automated simulation framework improved significantly on previous attempts to reach a uniform inlet flow rate distribution using the standard catheter hole configuration as a baseline. While the standard ventricular catheter design featuring uniform inlet hole diameters and hole spacing has a standard deviation of 14.27% for the inlet flow rates, the optimized design has a standard deviation of 0.30%. CONCLUSIONS This customizable framework, paired with high-performance computing, provides a rapid method of design testing to solve complex flow problems. While a relatively simplified ventricular catheter model was used to demonstrate the framework, the computational approach is applicable to any baseline catheter model, and it is easily adapted to optimize catheters for the unique needs of different patients as well as for other fluid-based medical devices.

Neurosciences & Neurology↗
DOE OSTI2020

Scalable Deep-Learning-Accelerated Topology Optimization for Additively Manufactured Materials

Topology optimization (TO) is a popular and powerful computational approach for designing novel structures, materials, and devices. Two computational challenges have limited the applicability of TO to a variety of industrial applications. First, a TO problem often involves a large number of design variables to guarantee sufficient expressive power. Second, many TO problems require a large number of expensive physical model simulations, and those simulations cannot be parallelized. To address these issues, we propose a general scalable deep-learning (DL) based TO framework, referred to as SDL-TO, which utilizes parallel schemes in high performance computing (HPC) to accelerate the TO process for designing additively manufactured (AM) materials. Unlike the existing studies of DL for TO, our framework accelerates TO by learning the iterative history data and simultaneously training on the mapping between the given design and its gradient. The surrogate gradient is learned by utilizing parallel computing on multiple CPUs incorporated with a distributed DL training on multiple GPUs. The learned TO gradient enables a fast online update scheme instead of an expensive update based on the physical simulator or solver. Using a local sampling strategy, we achieve to reduce the intrinsic high dimensionality of the design space and improve the training accuracy and the scalability of the SDL-TO framework. The method is demonstrated by benchmark examples and AM materials design for heat conduction. The proposed SDL-TO framework shows competitive performance compared to the baseline methods but significantly reduces the computational cost by a speed up of around 8.6x over the standard TO implementation.

Bi, Sirui↗
DOE OSTI2024

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

Standardizing power monitoring and control at exascale

Power API-the result of collaboration among national laboratories, universities, and major vendors-provides a range of standardized power management functions, from application-level control and measurement to facility-level accounting, including real-time and historical statistics gathering. Here, support is already available for Intel and AMD CPUs and standalone measurement devices.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Data Provenance Hybridization Supporting Extreme-Scale Scientific WorkflowApplications

As high performance computing (HPC) infrastructures continue to grow in capability and complexity, so do the applications that they serve. HPC and distributed-area computing (DAC) (e.g. grid and cloud) users are looking increasingly toward workflow solutions to orchestrate their complex application coupling, pre- and post-processing needs To gain insight and a more quantitative understanding of a workflow’s performance our method includes not only the capture of traditional provenance information, but also the capture and integration of system environment metrics helping to give context and explanation for a workflow’s execution. In this paper, we describe IPPD’s provenance management solution (ProvEn) and its hybrid data store combining both of these data provenance perspectives.

provenance↗