SEARCH · Search NASA

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Real‐time XFEL data analysis at SLAC and NERSC: A trial run of nascent exascale experimental data analysis

X‐ray scattering experiments using free electron lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID‐19 viral proteins). XFEL experiments are a challenge to computing in two ways: (i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; (ii) data‐collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min—sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS‐NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis within 10 min is a first for XFEL experiments and an important milestone if we are to keep up with data‐collection trends.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Accelerated Probabilistic Marching Cubes by Deep Learning for Time-Varying Scalar Ensembles

Visualizing the uncertainty of ensemble simulations is challenging due to the large size and multivariate and temporal features of en-semble data sets. One popular approach to studying the uncertainty of ensembles is analyzing the positional uncertainty of the level sets. Probabilistic marching cubes is a technique that performs Monte Carlo sampling of multivariate Gaussian noise distributions for positional uncertainty visualization of level sets. However, the technique suffers from high computational time, making interactive visualization and analysis impossible to achieve. This paper introduces a deep-learning-based approach to learning the level-set uncertainty for two-dimensional ensemble data with a multivariate Gaussian noise assumption. We train the model using the first few time steps from time-varying ensemble data in our workflow. We demonstrate that our trained model accurately infers uncertainty in level sets for new time steps and is up to 170X faster than that of the original probabilistic model with serial computation and 10X faster than that of the original parallel computation.

Han, Mengjiao↗

DOE OSTI2020

A fast particle-based approach for calibrating a 3-D model of the Antarctic ice sheet

We consider the scientifically challenging and policy-relevant task of understanding the past and projecting the future dynamics of the Antarctic ice sheet. The Antarctic ice sheet has shown a highly nonlinear threshold response to past climate forcings. Triggering such a threshold response through anthropogenic greenhouse gas emissions would drive drastic and potentially fast sea level rise with important implications for coastal flood risks. Previous studies have combined information from ice sheet models and observations to calibrate model parameters. These studies have broken important new ground but have either adopted simple ice sheet models or have limited the number of parameters to allow for the use of more complex models. These limitations are largely due to the computational challenges posed by calibration as models become more computationally intensive or when the number of parameters increases. Here, we propose a method to alleviate this problem: a fast sequential Monte Carlo method that takes advantage of the massive parallelization afforded by modern high-performance computing systems. We use simulated examples to demonstrate how our sample-based approach provides accurate approximations to the posterior distributions of the calibrated parameters. The drastic reduction in computational times enables us to provide new insights into important scientific questions, for example, the impact of Pliocene era data and prior parameter information on sea level projections. These studies would be computationally prohibitive with other computational approaches for calibration such as Markov chain Monte Carlo or emulation-based methods. We also find considerable differences in the distributions of sea level projections when we account for a larger number of uncertain parameters. For example, based on the same ice sheet model and data set, the 99th percentile of the Antarctic ice sheet contribution to sea level rise in 2300 increases from 6.5 m to 13.1 m when we increase the number of calibrated parameters from three to 11. With previous calibration methods, it would be challenging to go beyond five parameters. Here, this work provides an important next step toward improving the uncertainty quantification of complex, computationally intensive and decision-relevant models.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2019

Future SDN-HPON Control Plane Architecture and Protocol for On-Demand Terabit End-to-End Extreme-Scale Science Applications

The UC Davis team conducted architecture, algorithm and experimental studies for next generation ultra-high-bandwidth optical networks in support of extreme-scale science applications. In particular, over the course of this 3-year project, the UC Davis team achieved the following main accomplishments: Designed a network architecture for automated multi-domain service provisioning: Designed information exchange schemes between autonomous systems (domains) Designed application programming interfaces to orchestrate resource reservation in data centers, high performance computation facilities, and communication networks belonging to different autonomous systems Experimentally assessed the multi-domain network architecture in a distributed field trial set-up connecting premises in three continents Designed a broker-based orchestrated multi-domain optical networking framework supporting observe-analyze-act cycle based cognitive inter-domain service provisioning: Designed an inter-domain operation mechanism, e.g., topology abstraction, for the broker plane and domain managers to collaboratively accomplish inter-domain service provisioning while respecting the autonomy of each administrative domain Designed an alien wavelength monitoring scheme and an artificial neural network based optical signal-to-noise ratio estimator for quality-of-transmission aware inter-domain service provisioning Developed a deep neural network based multi-domain traffic predictor and a cognitive routing and spectrum assignment algorithm to realize knowledge-based autonomous traffic engineering in multi-domain networks Implemented an optical signal-to-noise ratio estimator and traffic predictor with Google’s Tensor flow and assessed their performance using experimental and synthetic data Demonstrated the benefits of the proposed cognitive schemes with experiments ona two-domain elastic optical network testbed and evaluations on a six-domain simulation framework Built a two-domain software-defined networking control plane testbed using the ONOS platform and demonstrated the workflow of broker and domain manager cooperation, including hierarchical performance monitoring, topology abstraction and cognitive resource allocation. Designed self-learning cognitive service provisioning schemes using advanced machine learning technologies: Designed a self-taught anomaly detection framework for optical networks, including an unsupervised data clustering module for pattern recognition, a supervised data regression and classification module for online anomaly detection, and the system operation principle Experimentally validated the effectiveness of the self-taught anomaly detection framework with various failure/anomaly scenarios. Designed a self-learning autonomic routing, modulation and spectrum assignment (RMSA) provisioning framework for elastic optical networks with deep reinforcement learning. Designed effective modeling and training mechanisms for the self-learning RMSA framework Conducted extensive numerical simulations with various traffic patterns and topologies to verify the advantage of the self-learning RMSA framework.

42 ENGINEERING↗

NASA NTRS1987

Systems simulations supporting NASA telerobotics

Two simulation and analysis environments have been developed to support telerobotics research at the Langley Research Center. One is a high-fidelity, nonreal-time, interactive model called ROBSIM, which combines user-generated models of workspace environment, robots, and loads into a working system and simulates the interaction among the system components. Models include user-specified actuator, sensor, and control parameters, as well as kinematic and dynamic characteristics. Kinematic, dynamic, and response analyses can be selected, with system configuration, task trajectories, and arm states displayed using computer graphics. The second environment is a real-time, manned Telerobotic Systems Simulation (TRSS) which uses the facilities of the Intelligent Systems Research Laboratory (ISRL). It utilizes a hierarchical structure of functionally distributed computers communicating over both parallel and high-speed serial data paths to enable studies of advanced telerobotic systems. Multiple processes perform motion planning, operator communications, forward and inverse kinematics, control/sensor fusion, and I/O processing while communicating via common memory. This paper describes both ROBSIM and TRSS, including their capability, status, and future plans. Also described is the architecture of ISRL and recent telerobotic system studies in ISRL.

Harrison, F. Wallace, Jr.↗

NASA NTRS1987

Systems simulations supporting NASA telerobotics

Two simulation and analysis environments have been developed to support telerobotics research at the Langley Research Center. One is a high-fidelity, nonreal-time, interactive model called ROBSIM, which combines user-generated models of workspace environment, robots, and loads into a working system and simulates the interaction among the system components. Models include user-specified actuator, sensor, and control parameters, as well as kinematic and dynamic characteristics. Kinematic, dynamic, and response analyses can be selected, with system configuration, task trajectories, and arm states displayed using computer graphics. The second environment is a real-time, manned Telerobotic Systems Simulation (TRSS) which uses the facilities of the Intelligent Systems Research Laboratory (ISRL). It utilizes a hierarchical structure of functionally distributed computers communicating over both parallel and high-speed serial data paths to enable studies of advanced telerobotic systems. Multiple processes perform motion planning, operator communications, forward and inverse kinematics, control/sensor fusion, and I/O processing while communicating via common memory. Both ROBSIM and TRSS, including their capability, status, and future plans are discussed. Also described is the architecture of ISRL and recent telerobotic system studies in ISRL.

Harrison, F. W., Jr.↗

DOE OSTI2023

$\mathrm{RADICAL}$-Pilot and $\mathrm{PMIx}$/$\mathrm{PRRTE}$: Executing Heterogeneous Workloads at Large Scale on Partitioned $\mathrm{HPC}$ Resources

Execution of heterogeneous workflows on high-performance computing (HPC) platforms present unprecedented resource management and execution coordination challenges for runtime systems. Task heterogeneity increases the complexity of resource and execution management, limiting the scalability and efficiency of workflow execution. Re-source partitioning and distribution of tasks execution over portioned re-sources promises to address those problems but we lack an experimental evaluation of its performance at scale. Here this paper provides a performance evaluation of the Process Management Interface for Exascale (PMIx) and its reference implementation PRRTE on the leadership-class HPC plat-form Summit, when integrated into a pilot-based runtime system called RADICAL-Pilot. We partition resources across multiple PRRTE Distributed Virtual Machine (DVM) environments, responsible for launching tasks via the PMIx interface. We experimentally measure the work-load execution performance in terms of task scheduling/launching rate and distribution of DVM task placement times, DVM startup and termination overheads on the Summit leadership-class HPC platform. Integrated solution with PMIx/PRRTE enables using an abstracted, standardized set of interfaces for orchestrating the launch process, dynamic process management and monitoring capabilities. It extends scaling capabilities allowing to overcome a limitation of other launching mechanisms (e.g., JSM/LSF). Explored different DVM setup configurations provide insights on DVM performance and a layout to leverage it. Our experimental results show that heterogeneous workload of 65,500 tasks on 2048 nodes, and partitioned across 32 DVMs, runs steady with resource utilization not lower than 52%. While having less concurrently executed tasks resource utilization is able to reach up to 85%, based on results of heterogeneous workload of 8200 tasks on 256 nodes and 2 DVMs.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Toward equitable environmental exposure modeling through convergence of data, open, and citizen sciences: an example of air pollution exposure modeling amidst increasing wildfire smoke

Exposure modeling is critical in environmental epidemiology and human health but may face challenges (e.g., skewed data, unequal error, context-insensitive validation, and computational demands). Modeling decisions reflect the intended use of the models and the values that modelers prioritize. We aimed to provide a conceptual framework and machine learning (ML) modeling protocols that address these issues. With 500m-gridded hourly PM 2.5 and O 3 levels in Illinois before, during, and after the 2023 Canadian wildfire season as a motivating example, we conducted modeling experiments to evaluate modeling methods, guided by three domains we propose based on theories of science: 1) Data Diversity, leveraging open and citizen science data to enhance inclusivity, parsimony, and representativeness; 2) Equitable Accuracy, ensuring fairly distributed uncertainties across subpopulations; and 3) Sustainable Modeling, balancing accuracy with reducing computational demands to promote accessibility for under-resourced researchers. Here, we found that ML with publicly available data can achieve high accuracy. Depending on methods, performance may vary substantially, even with identical input data. Large but skewed data may reduce performance. Misuse of cross-validation protocols can underestimate prediction error; although we observed R 2 s of ∼98 %, the modeled estimates varied significantly, indicating the need for careful model validation. By using new modeling protocols including representativeness-considered training and validation data and a new loss function, we achieved high agreement between estimates and ground-based measurements (e.g., R 2 = ∼90 % for PM 2.5 ; ∼80 % for O 3 ), equally distributed errors across sociodemographic strata and urban–rural divides, and reduction in computation time—from several weeks or months to a few days.

Exposure assessment↗

DOE OSTI2018

Comparison of Distributive Consensus Algorithms for AnomalyDetection on the Power-Grid

As an increasing number of devices in the power-grid rely on high-precision clocks to measure electrical properties and time stamp data, GPS receivers have become a vulnerability on the power-grid. GPS spoofing attacks, in which a fake signal is transmitted, can cause sensor measurement errors that can misinform protective equipment and grid operators for both the transmission and distribution grid. The technology being developed to secure precision clocks in the transmission grid are not appropriate for the distribution grid. Thus, there is a need to developed anomaly detection algorithms that are suitable for the hardware available on the distribution grid. Traditionally, a distributed detection framework is comprised of spatially distributed nodes which acquire observations about a quantity of interest and sends the data to the fusion center, where a global decision is made. Instead the distribution grid has low-power devices, such as smart inverters and microPMUs, that could be used to perform highly parallelized calculations. Having on node calculations is beneficial since it is more network and computation efficient, as the more calculations can be performed where the data is being captured instead of communicating across a slow network to a centralized fusion center. Instead, we could employ an alternate peer-to-peer type of information exchange on the distribution grid that can still reach a global decision. A decentralized approach to anomaly detection makes efficient use of the computational and network bandwidth that exists, and it also lacks the single point of failure of a centralized approach. Thus, a decentralized local information exchange and/or inference is preferred. Lastly, the unreliability of the distribution grid network connections and potential noise or error in the measurements must be considered. In order to produce an effective detection tool, a Byzantine-fault tolerant and asynchronous algorithm must be developed. Thus, the first phase of our project is to develop a distributed, decentralized, asynchronous, Byzantine-fault tolerant, collaborative autonomy-based anomaly detection algorithm to collectively analyze clusters of data for signs of a malicious attack.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

High Performance Geocomputation for Assessing Human Dynamics at Planet Scale

Settlements are key indicators of human presence on the landscape. Large scale mapping of human settlements and their morphology from very high-resolution satellite images is a critical step towards developing an interpretative understanding of population distribution and the sociocultural attributes of the built environment they live in. Convolutional neural network (CNN) based Deep Learning experiments indicates that such computations can be scaled to some of the largest high-performance computing (HPC) architectures. While early results are encouraging for developing settlement and corresponding population maps at unprecedented speed and spatial resolutions, characterizing human dynamics at planet scale with high temporal resolution will require the community to develop novel geocomputational infrastructures and ecosystems.

Bhaduri, Budhu↗

DOE OSTI2021

TAO Users Manual (Rev. 3.15)

The Toolkit for Advanced Optimization (TAO) focuses on the development of algorithms and software for the solution of large-scale optimization problems on high-performance architectures. Areas of interest include unconstrained and bound-constrained optimization, nonlinear least squares problems, optimization problems with partial differential equation constraints, and variational inequalities and complementarity constraints. The development of TAO was motivated by the scattered support for parallel computations and the lack of reuse of external toolkits in current optimization software. Our aim is to produce high-quality optimization software for computing environments ranging from workstations and laptops to massively parallel high-performance architectures. Our design decisions are strongly motivated by the challenges inherent in the use of large-scale distributed memory architectures and the reality of working with large, often poorly structured legacy codes for specific applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Toward A High-Performance Emulation Platform for Brain-Inspired Intelligent Systems Exploring Dataflow-Based Execution Model and Beyond

Brain-inspired computing is a novel computing technology based on neural morphological engineering, which draws lessons from methods of human brain information processing and storage. Combining with the high-performance computing (HPC) platform, they constitute the foundation of general artificial intelligence. However, current brain HPC platforms generally suffer from slow speed, poor scalability, and high energy consumption, which severely restrain its potential and circumscribe the development of general artificial intelligence. The dataflow model was first proposed in the 1970s, providing a novel idea for the development of HPC. In addition, the dataflow model shares similar information processing mechanisms with human's neural system, which makes dataflow models naturally suit the emulation of brain-inspired computing. Based on the contemporary progress of the dataflow model, the Codelet model was proposed. Through a fine-grained asynchronous program execution and resource allocation, the Codelet model successfully realized the distributed computing on the heterogeneous system, effectively improved the computing power and speed, and open up a new path to overcome the shortcomings of the existing high-performance computing technology. We propose a dataflow-based emulation platform, aiming at providing high-performance computing technology support for general brain-inspired intelligent system, as well as using characteristics of dataflow models to fully explore the potential of brain-inspired intelligence. As an example, we will select a convolutional neural network (LeNet5) that already has a spectacular user base to initially verify the superiority and feasibility of our proposal.

brain-inspired intelligent system↗

DOE OSTI2024

A performant energy-conserving particle reweighting method for Particle-in-Cell simulations

A new particle-based reweighting method is developed and demonstrated in the Aleph Particle-in-Cell with Direct Simulation Monte Carlo (PIC-DSMC) program. Novel splitting and merging algorithms ensure that modified particles maintain physically consistent positions and velocities. This method allows a single reweighting simulation to efficiently model plasma evolution over orders of magnitude variation in density, while accurately preserving energy distribution functions (EDFs). Demonstrations on electrostatic sheath and collisional rate dynamics show that reweighting simulations achieve accuracy comparable to fixed weight simulations with substantial computational time savings. This highly performant reweighting method is recommended for modeling plasma applications that require accurate resolution of EDFs or exhibit significant density variations in time or space.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Neglecting Model Parametric Uncertainty Can Drastically Underestimate Flood Risks

Abstract Floods drive dynamic and deeply uncertain risks for people and infrastructures. Uncertainty characterization is a crucial step in improving the predictive understanding of multi‐sector dynamics and the design of risk‐management strategies. Current approaches to estimate flood hazards often sample only a relatively small subset of the known unknowns, for example, the uncertainties surrounding the model parameters. This approach neglects the impacts of key uncertainties on hazards and system dynamics. Here we mainstream a recently developed method for Bayesian inference to calibrate a computationally expensive distributed hydrologic model. We compare three different calibration approaches: (a) stepwise line search, (b) precalibration or screening, and (c) the Fast Model Calibrations (FaMoS) approach. FaMoS deploys a particle‐based approach that takes advantage of the massive parallelization afforded by modern high‐performance computing systems. We quantify how neglecting parametric uncertainty and data discrepancy can drastically underestimate extreme flood events and risks. Precalibration improves prediction skill score over a stepwise line search. The Bayesian calibration improves the uncertainty characterization of model parameters and flood risk projections.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2019

LIVVkit 2.1: automated and extensible ice sheet model validation

A collection of scientific analyses, metrics, and visualizations for robustvalidation of ice sheet models is presented using the Land Ice Verificationand Validation toolkit (LIVVkit), version 2.1, and the LIVVkit Extensionsrepository (LEX), version 0.1. This software collection targets stand-aloneice sheet or coupled Earth system models, and handles datasets and analysesthat require high-performance computing and storage. LIVVkit aims to enableefficient and fully reproducible workflows for postprocessing, analysis, andvisualization of observational and model-derived datasets in a shareableformat, whereby all data, methodologies, and output are distributed to usersfor evaluation. Extending from the initial LIVVkit software framework, wedemonstrate Greenland ice sheet simulation validation metrics using thecoupled Community Earth System Model (CESM) as well as an idealizedstand-alone high-resolution Community Ice Sheet Model, version 2 (CISM2),coupled to the Albany/FELIX velocity solver (CISM-Albany or CISM-A). As oneexample of the capability, LIVVkit analyzes the degree to which modelscapture the surface mass balance (SMB) and identifies potential sources ofbias, using recently available in situ and remotely sensed data ascomparison. Related fields within atmosphere and land surface models, e.g.,surface temperature, radiation, and cloud cover, are also diagnosed. Appliedto the CESM1.0, LIVVkit identifies a positive SMB bias that is focusedlargely around Greenland's southwest region that is due to insufficientablation.

58 GEOSCIENCES↗

NASA NTRS2023

Performance, Inflow, and Tip Loss Characteristics of Rotors with Discontinuous Steps in Twist and Lift at the Blade-Tips

This paper examines the inflow and tip-loss characteristics of rotors incorporating indexed blade-tips that have discontinuous steps in twist or lift distribution. An extension of the classic two-dimensional Prandtl tip loss analysis is derived for indexed-tip rotors and proprotors. Additionally, parametric analysis of tip span and indexed-tip twist-steps was performed using computational fluid dynamics for hover and high-speed axial-flow conditions for proprotors. The intent of this work is to define a new class of active-rotor-twist-control designs for proprotors that avoids unnecessary aeroperformance compromises for rotor twist distributions for proprotors in all phases of flight.

rotors↗

DOE OSTI2019

Extreme Scale De Novo Metagenome Assembly

Metagenome assembly is the process of transforming a set of short, overlapping, and potentially erroneous DNA segments from environmental samples into the accurate representation of the underlying microbiomes's genomes. State-of-the-art tools require big shared memory machines and cannot handle contemporary metagenome datasets that exceed Terabytes in size. In this paper, we introduce the MetaHipMer pipeline, a high-quality and high-performance metagenome assembler that employs an iterative de Bruijn graph approach. MetaHipMer leverages a specialized scaffolding algorithm that produces long scaffolds and accommodates the idiosyncrasies of metagenomes. MetaHipMer is end-to-end parallelized using the Unified Parallel C language and therefore can run seamlessly on shared and distributed-memory systems. Experimental results show that MetaHipMer matches or outperforms the state-of-the-art tools in terms of accuracy. Ultimately, MetaHipMer scales efficiently to large concurrencies and is able to assemble previously intractable grand challenge metagenomes. We demonstrate the unprecedented capability of MetaHipMer by computing the first full assembly of the Twitchell Wetlands dataset, consisting of 7.5 billion reads - size 2.6 TBytes.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Experimental and computational analysis of bending fatigue failure in chopped carbon fiber chip reinforced composites

With a better balance among good mechanical performance, high freedom of design, and low material and manufacturing cost, chopped carbon fiber chip reinforced sheet molding compound (SMC) composites show great potential in different engineering applications. Here in this paper, bending fatigue behaviors of SMC composites considering the heterogeneous fiber orientation distributions have been thoroughly investigated utilizing both experimental and computational methods. First, four-point bending fatigue tests are performed with designed SMC composites, and the local modulus is adopted as a metric to represent the local fiber orientation of two opposing sides. Interestingly, SMC composites with and without large discrepancy in local modulus of opposing sides show different fatigue behaviors. Interrupted tests are conducted to explore the bending fatigue failure mechanism, and the damage processes of valid specimens are also closely examined. We find that the fatigue failure of SMC composites under four-point bending is governed by crack propagation instead of crack initiation. Because of this, the heterogeneous local fiber orientations of both sides of the specimen influence fatigue life. The microstructure of the lower side shows a direct influence while that of the upper side also exhibiting influence which becomes more prominent for high cycle fatigue cases. Furthermore, a hybrid micro–macro computational model is proposed to efficiently study the cyclic bending behavior of SMC composites. The region of interest is reconstructed with a modified random sequential absorption algorithm to conserve all the microstructural details including the heterogeneous fiber orientation, while the rest of the regions are modeled as homogenized macro-scale continua. Combined with a framework to capture the progressive fatigue damage under cyclic bending, the bending fatigue behaviors of SMC composites are accurately captured by the hybrid computational model comparing with our experimental analysis.

36 MATERIALS SCIENCE↗