Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Multi-Scale Thermo-Mechanical Modeling of Porous 3D Woven TPS Materials

This work summarizes the process to compute and analyze the thermal conductivity and mechanical properties of 3D woven TPS materials such as 3MDCP (3D Mid-Density Carbon-Phenolic). The PuMA software, developed at NASA Ames, was used to characterize different 3MDCP samples from their constituents' data, averaging their thermal conductivity and elastic properties in the three main directions to obtain their effective orthotropic thermo-mechanical properties. This was performed in multiple steps: firstly, TPS samples were digitally reconstructed using micro computed tomography (µCT) and their constituents were segmented; then, the porous matrix phase was analyzed at the micro-scale and these results were used, along with the fibers’ constituents information, to model the tows at the meso-scale; finally, results from the constituents were homogenized and used to model the thermo-mechanical behavior of the unit cell at the macro-scale. Additionally, to gain a better understanding of the tows’ morphology and distribution of the carbon-phenolic blended fibers, microscopy images of the tows’ cross-section were segmented using deep learning techniques and analyzed with PuMA. This workflow can be applied to any TPS material to computationally obtain its thermo-mechanical properties, enabling more informed TPS design and manufacturing choices.

Multi-Scale↗
DOE OSTI2019

BigPanDA: PanDA Workload Management System and its Applications beyond ATLAS

Modern experiments collect peta-scale volumes of data and utilize vast, geographically distributed computing infrastructure that serves thousands of scientists around the world. Requirements for rapid, near real-time data processing, fast analysis cycles and need to run massive detector simulations to support data analysis pose special premium on efficient use of available computational resources. A sophisticated Workload Management System (WMS) is needed to coordinate the distribution and processing of data and jobs in such environment. The ATLAS experiment at CERN uses PanDA (Production and Data Analysis) Workload Management System for managing the workflow for all data processing on over 150 data centers. While PanDAcurrently uses more than 250,000 cores with a peak performance of 0.3 petaFLOPS, it runs around 2 million jobs per day on hundreds of Grid sites and serving thousands of ATLAS users. In 2017 about 1.5 exabytes of data were processed with PanDA.In 2012 BigPanDA project project was started with aim to introduce new types of computing resources into ATLAS computing infrastructure, but also to offering PanDA features to different data-intensive applications for projects and experiments outside of ATLAS and High-Energy and Nuclear Physics. In this article we will present accomplishments and discuss possible directions for future work.

Svirin, Pavlo↗
DOE OSTI2024

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗
DOE OSTI2024

Correlative single-cell hard X-ray computed tomography and X-ray fluorescence imaging

Abstract X-ray computed tomography (XCT) and X-ray fluorescence (XRF) imaging are two non-invasive imaging techniques to study cellular structures and chemical element distributions, respectively. However, correlative X-ray computed tomography and fluorescence imaging for the same cell have yet to be routinely realized due to challenges in sample preparation and X-ray radiation damage. Here we report an integrated experimental and computational workflow for achieving correlative multi-modality X-ray imaging of a single cell. The method consists of the preparation of radiation-resistant single-cell samples using live-cell imaging-assisted chemical fixation and freeze-drying procedures, targeting and labeling cells for correlative XCT and XRF measurement, and computational reconstruction of the correlative and multi-modality images. With XCT, cellular structures including the overall structure and intracellular organelles are visualized, while XRF imaging reveals the distribution of multiple chemical elements within the same cell. Our correlative method demonstrates the feasibility and broad applicability of using X-rays to understand cellular structures and the roles of chemical elements and related proteins in signaling and other biological processes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Evolution of DUNE’s Production System

The DUNE experiment will start running in 2029 and record 30 PB/year of raw waveforms from Liquid Argon TPCs and photon detectors. The size of individual readouts can range from 100 MB to a typical 8 GB full readout of the detector, and even 100 TB for extended readouts from supernova candidates. These data then need to be cataloged, stored and distributed for processing worldwide. This massive amount of data and a heterogeneous computing environment necessitates a powerful and robust distributed computing infrastructure. In the process of building up that infrastructure, DUNE’s production system has recently undergone an overhaul, in which it has integrated 1) a new workflow management system (justIN) 2) a new data catalog (MetaCat) and 3) a state-of-the-art data management system (Rucio). Simulations of DUNE’s Far Detector and its prototypes ProtoDUNE Horizontal Drift (ProtoDUNE-HD) and ProtoDUNE Vertical Drift (ProtoDUNE-VD), as well as data from ProtoDUNE-HD serve as the first tests of this infrastructure.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Matilda v1.0: An R package for probabilistic climate projections using a reduced complexity climate model

A primary advantage to using reduced complexity climate models (RCMs) has been their ability to quickly conduct probabilistic climate projections, a key component of uncertainty quantification in many impact studies and multisector systems. Providing frameworks for such analyses has been a target of several RCMs used in studies of the future co-evolution of the human and Earth systems. In this paper, we present Matilda, an open-science R software package that facilitates probabilistic climate projection analysis, implemented here using the Hector simple climate model in a seamless and easily applied framework. The primary goal of Matilda is to provide the user with a turn-key method to build parameter sets from literature-based prior distributions, run Hector iteratively to produce perturbed parameter ensembles (PPEs), weight ensembles for realism against observed historical climate data, and compute probabilistic projections for different climate variables. This workflow gives the user the ability to explore viable parameter space and propagate uncertainty to model ensembles with just a few lines of code. The package provides significant freedom to select different scoring criteria and algorithms to weight ensemble members, as well as the flexibility to implement custom criteria. Additionally, the architecture of the package simplifies the process of building and analyzing PPEs without requiring significant programming expertise, to accommodate diverse use cases. We present a case study that provides illustrative results of a probabilistic analysis of mean global surface temperature as an example of the software application.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Optimizing the Energy Usage and Cognitive Value of Extreme-Scale Data-Analysis Approaches

This report serves as a record that we have successfully completed work on the proposal Optimizing the Energy Usage and Cognitive Value of Extreme Scale Analysis Approaches, or, more succinctly, ECX. This includes successful completion of all three phases of the project, findings as summarized below, strong publications across the fields represented in the proposal, and a set of artifacts (code, websites, and other documentation) available to the public. In total, the results of this project represent significant findings in each domain, and a toolkit that will allow the community to engage with this work, advancing it and contributing to it in the years ahead. They also form the basis for a successfully funded follow-on project.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A Parallel Graph Environment for Real-World Data Analytics Workflows

Economic competitiveness and national security depend increasingly on the insightful analysis of large data sets. The diversity of real-world data sources and analytic workflows impose challenging hardware and software requirements for parallel graph platforms. The irregular nature of graph methods is not supported well by the deep memory hierarchies of conventional distributed systems, requiring new processor and runtime system designs to tolerate memory and synchronization latencies. Moreover, the efficiency of relational table operations and matrix computations are not attainable when data is stored in common graph data structures. In this paper, we present HAGGLE, a high-performance, scalable data analytics platform. The platform’s hybrid data model supports a variety of distributed, thread-safe data structures, parallel programming constructs, and persistent and streaming data. An abstract runtime layer enables us to map the stack to conventional, distributed computer systems with accelerators. The runtime uses multithreading, active messages, and data aggregation to hide memory and synchronization latencies on large-scale systems.

Graph Analytics↗
DOE OSTI2020

Distributed Aerodynamic Control using Active Trailing-Edge Flaps for Large Wind Turbines

This work presents a numerical framework to investigate distributed aerodynamic control devices for application in large wind turbines. Tool capabilities were extended to facilitate multiple aerodynamic polar tables. The airfoil aerodynamics characteristics were automatically determined, and blade-pitch, generator-torque, and trailing-edge-flap controllers were tuned in-the-loop according to a specific blade design. This automated workflow allows analysis and optimization of trailing-edge flaps, enabling codesign studies. Results targeted reductions of root-flap-bending moment derivatives. The applied trailing-edge-flap control reduced the standard deviation of root-flap-bending moments by more than 6% and benefit related parameters, e.g., reduce blade-tip deflections, by up to 8%. Because of varying thrust distributions along the blade span, different flap designs have nonlinear characteristics in terms of the control objective and show best performance when located at the radial position with maximum thrust. In general, larger flaps provide a greater influence to reduce the target control objective.

17 WIND ENERGY↗
DOE OSTI2024

Physics-coupled data-driven design of high-temperature alloys

We present a materials design loop, which streamlines physics-coupled machine learning (ML) surrogate models to discover new alloy chemistries with improved properties. The efficacy is demonstrated by discovering a high-temperature alumina-forming austenitic (AFA) stainless steel with enhanced creep, followed by experimental validation. The ML models have been trained using a well-curated, highly consistent experimental dataset augmented with synthetic microstructural features from a computational thermodynamic approach. We have populated a large number of hypothetical AFA alloys to explore the high-dimensional composition space and have predicted their creep properties by providing the same synthetic input features obtained from the trained ML models. Uncertainties from the ML training were taken as thresholds for truncating predicted results to identify alloys with improved or deteriorated creep. Individual elemental compositions have been determined via probability density distribution analysis from the group of alloys at the top and bottom of the predicted creep values for further virtual and experimental validations. In conclusion, we anticipate that this workflow can be applied to screen desired conditions, such as chemistry and processing parameters, in high-dimensional space through physics-guided data analytics.

Alloy design↗
DOE OSTI2024

msdlive-cli-distro

MSD-LIVE, the MultiSector Dynamics – Living, Intuitive, Value-adding, Environment, is a flexible and scalable data and code management system combined with a distributed computational platform that will enable MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and multi-model workflows within a robust Community of Practice. MSD-LIVE will facilitate a new open, collaborative, resource-rich, technology-facilitated, community-driven way of doing MSD research.

Lansing, Carina↗
DOE OSTI2017

Singularity: Scientific containers for mobility of compute

Here we present Singularity, software developed to bring containers and reproducibility to scientific computing. Using Singularity containers, developers can work in reproducible environments of their choosing and design, and these complete environments can easily be copied and executed on other platforms. Singularity is an open source initiative that harnesses the expertise of system and software engineers and researchers alike, and integrates seamlessly into common workflows for both of these groups. As its primary use case, Singularity brings mobility of computing to both users and HPC centers, providing a secure means to capture and distribute software and compute environments. This ability to create and deploy reproducible environments across these centers, a previously unmet need, makes Singularity a game changing development for computational science.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Thermal Effects on Far-Field Distributed Acoustic Strain-Rate Sensors

Summary Fiber-optic cables cemented outside of the casing of an unconventional well measure crosswell strain changes during fracturing of neighboring wells with low-frequency distributed acoustic sensing (LF-DAS). As a hydraulic fracture intersects an observation well instrumented with fiber-optic cables, the fracture fluid injected at ambient temperatures can cool a section of the sensing fiber. Often, LF-DAS and distributed temperature sensing (DTS) cables are run in tandem, enabling the detection of such cooling events. The increasing use of LF-DAS for characterizing unconventional hydraulic fracture completions demands an investigation of the effects of temperature on the measured strain response by LF-DAS. Researchers have demonstrated that LF-DAS can be used to extract the temporal derivative of temperature for use as a differential-temperature-gradient sensor. However, differential-temperature-gradient sensing is predicated on the ability to filter strain components out of the optical signal. In this work, beginning with an equation for optical phase shift of LF-DAS signals, a model relating strain, temperature, and optical phase shift is explicitly developed. The formula provides insights into the relative strength of strain and temperature effects on the phase shift. The uncertainty in the strain-rate measurements due to thermal effects is estimated. The relationship can also be used to quantify uncertainties in differential-temperature-gradient sensors due to strain perturbations. Additionally, a workflow is presented to simulate the LF-DAS response accounting for both strain and temperature effects. Hydraulic fracture geometries are generated with a 3D fracture simulator for a multistage unconventional completion. The fracture width distributions are imported by a displacement discontinuity method (DDM) program to compute the strain rates along an observation well. An analytic model is used to approximate the temperature in the fracture. Using the derived formulae for optical phase shift, the model outputs are then used to compute the LF-DAS response at a fiber-optic cable, enabling the generation of waterfall plots including both strain and thermal effects. The model results suggest that before, during, and immediately following a fracture intersecting a well instrumented with fiber, the strain on the fiber drives the LF-DAS signal. However, at later times, as completion fluid cools the observation well, the temperature component of the LF-DAS signal can be equal to or exceed the strain component. The modeled results are compared to a published field case in an attempt to enhance the interpretation of LF-DAS waterfall plots. Finally, we propose a sensing configuration to identify the events when “wet fractures” (fractures with fluids) intersect the observation well.

Engineering↗
DOE OSTI2022

Assessing the impact of substrate-level enzyme regulations limiting ethanol titer in Clostridium thermocellum using a core kinetic model

Clostridium thermocellum is a promising candidate for consolidated bioprocessing because it can directly ferment cellulose to ethanol. Despite significant efforts, achieved yields and titers fall below industrially relevant targets. This implies that there still exist unknown enzymatic, regulatory, and/or possibly thermodynamic bottlenecks that can throttle back metabolic flow. By (i) elucidating internal metabolic fluxes in wild-type C. thermocellum grown on cellobiose via 13 C-metabolic flux analysis ( 13 C-MFA), (ii) parameterizing a core kinetic model, and (iii) subsequently deploying an ensemble-docking workflow for discovering substrate-level regulations, this paper aims to reveal some of these factors and expand our knowledgebase governing C. thermocellum metabolism. Generated 13 C labeling data were used with 13 C-MFA to generate a wild-type flux distribution for the metabolic network. Notably, flux elucidation through MFA alluded to serine generation via the mercaptopyruvate pathway. Using the elucidated flux distributions in conjunction with batch fermentation process yield data for various mutant strains, we constructed a kinetic model of C. thermocellum core metabolism (i.e. k-ctherm138). Subsequently, we used the parameterized kinetic model to explore the effect of removing substrate-level regulations on ethanol yield and titer. Upon exploring all possible simultaneous (up to four) regulation removals we identified combinations that lead to many-fold model predicted improvement in ethanol titer. In addition, by coupling a systematic method for identifying putative competitive inhibitory mechanisms using K-FIT kinetic parameterization with the ensemble-docking workflow, we flagged 67 putative substrate-level inhibition mechanisms across central carbon metabolism supported by both kinetic formalism and docking analysis.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

UQpy v4.1: Uncertainty quantification with Python

This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best practices were adopted. A new software development workflow significantly improved collaboration between team members, and continuous integration and automated testing ensured the robustness and reliability of software performance. Continuous deployment of UQpy allowed its automated packaging and distribution in system agnostic format via multiple channels, while a Docker image enables the use of the toolbox regardless of operating system limitations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Quantifying the Dynamics of Protein Self-Organization Using Deep Learning Analysis of Atomic Force Microscopy Data

The dynamics of protein self-assembly on the inorganic surface and the resultant geometric patterns are visualized using high-speed atomic force microscopy. The time dynamics of the classical macroscopic descriptors such as 2D fast Fourier transforms, correlation, and pair distribution functions are explored using the unsupervised linear unmixing, demonstrating the presence of static ordered and dynamic disordered phases and establishing their time dynamics. Here, the deep learning (DL)-based workflow is developed to analyze detailed particle dynamics and explore the evolution of local geometries. Finally, we use a combination of DL feature extraction and mixture modeling to define particle neighborhoods free of physics constraints, allowing for a separation of possible classes of particle behavior and identification of the associated transitions. Overall, this work establishes the workflow for the analysis of the self-organization processes in complex systems from observational data and provides insight into the fundamental mechanisms.

36 MATERIALS SCIENCE↗
DOE OSTI2020

GridOPTICS/GridPACK

GridPACK is a software framework consisting of a set of modules designed to simplify the development of programs that model the power grid and run on parallel, high performance computing platforms. It also contains several fully developed applications, including powerflow, dynamic simulation, state estimation, Kalman filter analysis (dynamic state estimation), contingency analysis and real time path rating. These applications can be used either standalone or as components in more complicated workflows that combine several different types of application together. The framework modules are available as a combination of libraries and software templates and consist of components for setting up and distributing power grid networks, support for modeling the behavior of individual buses and branches in the network, converting the network models to the corresponding algebraic equations, and parallel routines for manipulating and solving large algebraic systems. The framework also contains a module for distributing tasks evenly amongst computing resources, even if individual tasks vary widely in their execution times. Additional modules support input and output, basic statistical analysis of contingency based calculations, distributed data structures, as well as basic profiling and error management.

Palmer, Bruce↗