Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

A Semi-Empirical Framework for Interpreting Traveling Wave Ion Mobility Arrival Time Distributions

The inherent structural heterogeneity of biomolecules is an important biophysical property that is essential to their function, but is challenging to characterize experimentally. We present a workflow that rapidly and quantitatively assesses the conformational heterogeneity of peptides and proteins in the gas phase using traveling wave ion mobility (TWIM) arrival time distributions (ATDs). We have established a set of semi-empirical equations that model the TWIM ATD peak width and resolution across a wide range of wave amplitudes (V) and wave velocities (v). In addition, a conformational broadening parameter, δ, can be extracted from this analysis that reports on the contribution of conformational heterogeneity to the broadening of TWIM ATD peak width during ion mobility separation. We use this δ value to evaluate the conformational heterogeneity of a set of helical peptides, and our analysis correlates well with previous peak width observations reported for these ions. Furthermore, we use molecular dynamics simulations to independently investigate the general flexibility of these peptides in the gas phase, and generate similar trends found in experimental TWIM data. Finally, we extended our analysis to Avidin, a 64-kDa homotetramer, and quantify the structural heterogeneity of this intact complex using TWIM ATD data as a function of cross-linking. We observe an initial reduction in δ values as a function of cross-linker concentration, demonstrating the sensitivity of our δ value analysis to changes in flexibility of the assembly. .

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

ACDC (Automated Campbell Diagram Code) [SWR-26-042]

This application provides a web-based graphical user interface to generating Campbell Diagrams and visualizing mode shapes for OpenFAST turbine models. Determining the aeroelastic stability and dynamic characteristics of wind turbines is a critical step in turbine design and analysis. Historically, extracting natural frequencies and mode shapes from OpenFAST—the industry-standard whole-turbine simulation code—has been a fragmented and tedious process. It required manual model configuration, command-line linearization execution, and complex post-processing via proprietary scripts to handle rotating-frame dynamics. To address these workflow bottlenecks, we present the Automated Campbell Diagram Code (ACDC), an open-source graphical software tool developed by the National Laboratory of the Rockies (NLR) under the DOE-funded Distributed Wind Aeroelastic Modeling (dWAM) project. ACDC streamlines the end-to-end linearization and stability analysis workflow into a single, intuitive cross-platform application. The software guides users through OpenFAST model configuration, definition of operating points, and the automated execution of steady-state trim and linearization simulations. Under the hood, ACDC automates the complex mathematical post-processing steps required for rotating systems, including Multi-Blade Coordinate (MBC) transformations, eigenanalysis, and advanced modal tracking utilizing the Modal Assurance Criterion (MAC) and spectral clustering. Finally, ACDC processes these results to automatically generate Campbell diagrams and features a robust 3D visualization engine to animate full-system mode shapes. By eliminating the reliance on external post-processing environments and manual data manipulation, ACDC significantly accelerates dynamic analysis and lowers the barrier to entry for wind energy researchers and engineers.

Summerville, Brent [National Laboratory of the Roc↗
DOE OSTI2024

Advancements in Multiphysics Microdepletion Analysis of an eVinci TM -like Microreactor Leveraging OpenMC-CRAB Workflow

Nuclear microreactors (MRs) are a class of nuclear reactor technology, characterized by reduced dimensions, modular design, and reduced power output in contrast to conventional Light Water Reactors (LWRs). MRs are proposed for supplying electricity and eventual process heat to remote locations, such as military installations and disaster-affected areas. Current research work sponsored by the US Department of Energy Microreactor Program (MRP) is devoted to the development of novel modeling and simulation tools to better support MR vendors and regulatory bodies. Notably, the NRC is projected to utilize the CRAB multiphysics software driver for executing both design and beyond-design-basis accident analyses. Furthermore, the NRC has been utilizing the MELCOR code to calculate mechanistic source terms during accidents. Since MELCOR relies on isotopic inventory and reactor temperature/power profiles under accident conditions, which theoretically can be derived from CRAB, the goal is to establish a comprehensive CRAB-MELCOR computational framework. Past work was focused on testing and demonstrating CRAB's capability to generate results that can be used to inform mechanistic source term calculations in MELCOR. In particular, a computational workflow leveraging OpenMC-generated microscopic cross sections and CRAB was first applied to perform multiphysics microscopic depletion calculation followed by an accident scenario for a stylized microreactor problem. In fiscal year 2024, the research work has been focused on applying the OpenMC-CRAB workflow, which was first tested in fiscal year 2023, to a realistic 3D heat-pipe cooled MR problem representative of the eVinci TM design. The latter computational problem was developed with inputs from WEC to conserve selected neutronic and thermal characteristics of the eVinci TM design without releasing proprietary data. The results of this simulation, encompassing isotopic inventory, power density distribution, and kinetic parameters, will inform both MELCOR and the WEC-developed FATE code for mechanistic source terms calculations. The results from the two codes will then be compared for code verification purposes. This report contains the design characteristics of the realist heat pipe cooled microreactor developed as a use-case for the verification exercise, and the current results for the multiphysics microscopic depletion performed with the OpenMC-CRAB workflow. The results include eigenvalue as a function of time, power distribution at EOL, in addition to nuclides inventory's time evolution and spatial distribution. Finally, we report improvements to the workflow efficiency achieved through a collaboration with the NEAMS programs. Through this collaborative effort, we were able to strongly decrease the computational time for the multiphysics microdepletion calculation (i.e., from 17.4 hours to 5.7 hours on 280 processors) in addition to simplifying the interface to generate isotopics spatial distribution utilizable by FATE and MELCOR. Future work, including the improvement of the current microscopic cross-sections' library and the simulation of an accident scenario at EOL, is also discussed.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

A Conceptual Design for a Mobile Application to Support Infield Inventory Activities

This paper introduces an inventory assistant being developed at Oak Ridge National Laboratory (ORNL) that we believe will empower users to perform inventory activities at nuclear facilities more accurately, reliably, and quickly. Inventory activities at nuclear facilities are often conducted using pen and paper, which can be time-consuming, tedious, and susceptible to reading or transcription errors. The proposed inventory assistant would replace the paper-based process used by International Atomic Energy Agency (IAEA) inspectors, nuclear facility operators, or verification monitors to complete an inventory of nuclear and non-nuclear items. In general, the inventory assistant would ingest an inventory list, distribute assigned items from the inventory list to one or more mobile devices, enable inventory teams to record their observations in the field, and then enable an inventory lead to integrate and reconcile the observations to produce a final report. The assistant consists of two software components—one for the inventory teams to record observations in the field (In-Field Observations App [IFOA]) and one for the inventory lead to reconcile the inventory list with observations (Distribution, Integration, and Reconciliation Application [DIRA]). This paper introduces the overall workflow of the inventory assistant and describes the IFOA user experience in more detail. To demonstrate the concept, the authors present a use case of IAEA inspectors conducting item counting and tag checking activities of UF6 cylinders at a gas centrifuge enrichment plant with a large number of UF6 cylinders (e.g., thousands). These activities can currently require 30–40 person-days of inspection to complete. Based on experiences during an exercised performed at the IAEA by the ORNL team in 2016, we believe an inventory assistant could allow the IAEA to complete item counting and tag checking using the global identifier or the operator’s barcode in 8–10 person-days of inspection. We would expect other users (e.g., facility operators or verification monitors) to also benefit from significant time savings.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Conquering Data Chaos: Research Data Management with Kubernetes

Managing massive volumes of data and effectively making it accessible to researchers poses significant challenges and is a barrier to scientific discovery. In many cases, critical data is locked up in unwieldy file formats or one-off databases and is too large to effectively process on a single machine. This talk explores the role of Kubernetes, an open-source container orchestration platform, in addressing research data management challenges. I will discuss how we are using a set of publicly available open-source and home-grown tools in the National Renewable Energy Lab (NREL) Data, Analysis, and Visualization (DAV) group to help researchers overcome data-related bottlenecks. The talk will begin by providing an overview of the data challenges faced in research data management, including data storage, processing, and analysis. I will highlight Kubernetes' ability to handle large-scale data by leveraging containerization and distributed computing, including distributed storage. Kubernetes allows researchers to encapsulate data processing infrastructure and workflows into portable containers, enabling reproducibility and ease of deployment. Kubernetes can then schedule and manage the resource allocation of these containers to enable efficient utilization of limited computing resources, leading to more efficient data processing and analysis. I will discuss some limitations of traditional, siloed approaches to dealing with data and emphasize the need for solutions which foster collaboration. I will highlight how we are using Kubernetes at NREL to facilitate data sharing and cooperation among research teams. Kubernetes' flexible architecture enables the deployment of shared computing environments, such as Apache Superset, where researchers can seamlessly access and analyze shared datasets. Providing the ability to have one research team easily consume data generated by another, utilizing Kubernetes' as a central data platform, is one of the major wins we've encountered by adopting the platform. Finally, I will showcase real-world use cases from NREL where we have used Kubernetes to solve some persistent data challenges involving large volumes of sensor and monitoring data. I will discuss the challenges we encountered when creating our cluster and making it available as a production-ready resource. I will also discuss the specific suite of tools, including Postgres and Apache Druid for columnar and timeseries data, and Redpanda Kafka for streaming data we have deployed in our infrastructure, and the process that went into the selection of these tools.

collaborative environment↗
DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗
DOE OSTI2022

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗
DOE OSTI2023

Virtual Infrastructure Twins: Software Testing Platforms for Computing-Instrument Ecosystems

Science ecosystems are being built by federating computing systems and instruments located at geographically distributed sites over wide-area networks. These computing-instrument ecosystems are expected to support complex workflows that incorporate remote, automated AI-driven science experiments. Their realization, however, requires various designs to be explored and software components to be developed, in order to support the orchestration of distributed computations and experiments. It is often too expensive, infeasible, or disruptive for the entire ecosystem to be available during the typically long software development and testing periods. We propose a Virtual Infrastructure Twin (VIT) of the ecosystem that emulates its network and computing components, and incorporates its instrument software simulators. It provides a software environment nearly identical to the ecosystem to support early development and testing, and design space exploration. We present a brief overview of previous digital infrastructure twins that culminated in the VIT concept, including (i) the virtual science network environment for developing software-defined networking solutions, and (ii) the virtual federated science instrument environment for testing the federation software stack and remote instrument control software. We briefly describe VITs for Nion microscope steering and access to GPU systems.

Rao, Nageswara↗
DOE OSTI2026

Multipoint Aerostructural Optimization of Wind Turbine Rotors Using a Coupled Blade‐Resolved Aerostructural Solver

Physics‐based design optimization workflows thread the needle between computational cost limitations and simulation complexity, often compromising between modeling detail and the range of operating design conditions. Multipoint aerostructural optimization of wind turbine rotors has so far been confined to low‐fidelity analyses or to high‐fidelity studies with simplified structural models, leaving the most complex design trade‐offs unexplored. We close this gap by performing the first tightly coupled gradient‐based multipoint aerostructural rotor optimization using 3D aerodynamic and structural solvers with discrete coupled adjoints. The optimizer simultaneously varies blade planform, airfoil shapes, and structural thickness through more than 270 design variables, minimizing a weighted combination of rotor mass and power across multiple wind speeds. Applied to a modified DTU 10‐MW benchmark under conservative structural and aerodynamic constraints, our multipoint optimization reduces rotor mass by up to 36% and increases power by 12%–15% across the main operating conditions; biasing the objective toward power yields power gains up to 18% and a 17% mass reduction. For a nominal wind distribution, 3‐point rotor designs accounting for low RPM and high thrust conditions capture dominant trade‐offs and outperform single‐point designs. Adding two off‐design points changes individual‐condition power by less than 3% but leaves the weighted average within 0.5%, and the mass‐power bias has a stronger effect on the final design than the operating‐point weighting itself. Our framework extends naturally to richer load cases and site‐specific wind distributions, providing a basis for high‐fidelity multipoint design earlier in industrial workflows.

17 WIND ENERGY↗
DOE OSTI2021

Three practical workflow schedulers for easy maximum parallelism

Runtime scheduling and workflow systems are an increasingly popular algorithmic component in HPC because they allow full system utilization with relaxed synchronization requirements. There are so many special-purpose tools for task scheduling, one might wonder why more are needed. Use cases seen on the Summit supercomputer needed better integration with MPI and greater flexibility in job launch configurations. Preparation, execution, and analysis of computational chemistry simulations at the scale of tens of thousands of processors revealed three distinct workflow patterns. A separate job scheduler was implemented for each one using extremely simple and robust designs: file-based, task-list based, and bulk-synchronous. Comparing to existing methods shows unique benefits of this work, including simplicity of design, suitability for HPC centers, short startup time, and well-understood per-task overhead. All three new tools have been shown to scale to full utilization of Summit, and have been made publicly available with tests and documentation. This work presents a complete characterization of the minimum effective task granularity for efficient scheduler usage scenarios. Here, these schedulers have the same bottlenecks, and hence similar task granularities as those reported for existing tools following comparable paradigms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Novel Proppant Logging Technique for Infill Drilling of Unconventional Shale Wells

Summary During the development of an unconventional play, wells are drilled and completed in batches, and depending on the development plans, current and expected energy market trends, as well as other developmental considerations, new wells are drilled and hydraulically fractured later near existing producing laterals. This creates challenges in terms of optimizing resource recovery and reducing interwell communication. A novel approach is proposed that utilizes systematic composite sampling and analysis of drilling mud returns to look for and quantitatively identify sand particles. The workflow involves cleaning, drying, and segregation of samples into sizes of interest to us (size distribution of pumped proppant in offset parent wells). These samples are imaged at a very high resolution and analyzed for grains using characteristic optical imaging properties to classify proppant sand particles using computer vision algorithms. Further analysis, such as elemental compositional analysis, is used to validate the results from the imaging workflow. We present a case study from the Permian Basin, where a new child well was used as a test case to prove this technology at the Hydraulic Fracturing Test Site (HFTS-2) in Delaware Basin. We introduce new proppant parameters that help identify sustained proppant zones vs. localized propped fractures. We have used additional diagnostics and data collected at the test site to validate observations from the proppant log and have successfully interpreted significantly propped vs. unpropped zones. A key finding from this test has been the significant proppant transport distances observed away from parent wells. Observable proppant was found at a lateral distance of approximately 425 m for one set of parent wells and more than 915 m for another set of parent wells. While a major limitation of this technique is the sampling rate, given adequate sampling, the proposed technology represents a systematic and one-of-a-kind interpretation of spatial proppant distribution while drilling infill wells. It provides us with unique opportunities to better understand the current state of the reservoir being targeted, including zones that are likely highly drained relative to others, and how the planned hydraulic fracturing of child wells can be improved.

Energy & Fuels↗
NASA NTRS2021

PIXLISE-C: Exploring The Data Analysis Needs of NASA Scientists for Mineral Identification

NASA JPL scientists working on the micro x-ray fluorescence (microXRF) spectroscopy data collected from Mars surface perform data analysis to look for signs of past microbial life on Mars. Their data analysis workflow mainly involves identifying mineral com- pounds through the element abundance in spatially distributed data points. Working with the NASA JPL team, we identified pain points and needs to further develop their existing data visualization and analysis tool. Specifically, the team desired improvements for the process of creating and interpreting mineral composition groups. To address this problem, we developed an interactive tool that enables scientists to (1) cluster the data using either manual lasso-tool selection or through various machine learning clustering algorithms, and (2) compare the clusters and individual data points to make informed decisions about mineral compositions. Our preliminary tool supports a hybrid data analysis workflow where the user can manually refine the machine-generated clusters.

Davidoff, Scott↗
DOE OSTI2019

The Evolution of the Pegasus Workflow Management Software

Since 2001 the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. The paper concludes with selected lessons learned.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Comparative Analysis of Imaging and Measurements of Micrometer-Scale Fracture Aperture Fields Within a Heterogeneous Rock Using PET and X-ray CT

Knowledge of the spatial distribution of fracture apertures is essential for reliable characterization of flow and transport processes in fractured systems and for better understanding of physicochemical matrix–fracture interactions. Here, we propose and test two image-based methods, thereby extending the current experimental capabilities to characterize aperture size distribution in structurally heterogeneous geologic porous media noninvasively. The first approach utilizes an inversion method based on the dataset acquired from positron emission tomography (PET) and the second approach considers an extension of the classic missing attenuation technique that relies on clinical X-ray computed tomography (X-ray CT). Independent sets of imaging experiments are conducted on a fractured basalt core with heterogeneous matrix properties and aperture distributions to compare the two methodologies. A repeat of each experiment is conducted to verify the proposed workflows. The performance of these two imaging techniques is systematically evaluated through the analysis of signal-to-noise ratio, minimum fracture size detectability, and measurement errors. While both approaches provide a reliable estimation of fracture aperture distributions, PET yields a signal-to-noise ratio that is substantially higher than the corresponding X-ray CT measurements. Furthermore, uncertainties of the aperture values for PET are considerably lower ($\bar {\sigma}_{\text {d}} = 15\%$) compared to those obtained from X-ray CT ($\bar {\sigma}_{\text {d}} = 29\%$), allowing for the detection of minimum aperture sizes of 20 $\mu$m with 70% confidence level. Finally, these approaches provide key experimental tools for better understanding dynamic hydromechanical fracture properties in geologic systems.

58 GEOSCIENCES↗
DOE OSTI2024

Modeling Distributed Computing Infrastructures for HEP Applications

Predicting the performance of various infrastructure design options in complex federated infrastructures with computing sites distributed over a wide area network that support a plethora of users and workflows, such as the Worldwide LHC Computing Grid (WLCG), is not trivial. Due to the complexity and size of these infrastructures, it is not feasible to deploy experimental test-beds at large scales merely for the purpose of comparing and evaluating alternate designs. An alternative is to study the behaviours of these systems using simulation. This approach has been used successfully in the past to identify efficient and practical infrastructure designs for High Energy Physics (HEP). A prominent example is the Monarc simulation framework, which was used to study the initial structure of the WLCG. New simulation capabilities are needed to simulate large-scale heterogeneous computing systems with complex networks, data access and caching patterns. A modern tool to simulate HEP workloads that execute on distributed computing infrastructures based on the SimGrid and WRENCH simulation frameworks is outlined. Studies of its accuracy and scalability are presented using HEP as a case-study. Hypothetical adjustments to prevailing computing architectures in HEP are studied providing insights into the dynamics of a part of the WLCG and candidates for improvements.

Horzela, Maximilian↗
DOE OSTI2024

A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly-accurate analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapid generation of complex nanoscale atomic structures which enables fast, systematic sampling of realistic nanomaterial structures and can be used as a random structure generator for large, diverse synthetic datasets. Using Construction Zone, we develop an end-to-end machine learning workflow for training neural network models to analyze experimental atomic resolution HRTEM images on the task of nanoparticle image segmentation purely with simulated databases. Further, we study the data curation process to understand how various aspects of the curated simulated data—including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions—affect model performance across three benchmark experimental HRTEM image datasets. Using our workflow, we are able to achieve state-of-the-art segmentation performance on these experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data. Construction Zone and its documentation are available at https://github.com/lerandc/construction_zone.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Generation of random geological models using multi-randomization for machine learning

Generating high-fidelity geological models is essential for advancing machine learning (ML) methods in automated seismic interpretation. For instance, seismic images paired with corresponding fault labels are foundational for ML-based fault detection from seismic migration sections. While several open-access datasets of random geological models exist, open-source tools specifically designed to produce large volumes of such models for ML applications remain scarce. To address this gap, we present RGM (Random Geological Model), an open-source software package for efficiently generating 2D and 3D synthetic geological models tailored for ML workflows. RGM supports the creation of diverse model components, including medium property distributions (P-/S-wave velocities and density), seismic reflectivity images (i.e., synthetic migration sections), relative geological time, and discrete fault attributes such as probability, dip, strike, rake, and displacement. It also accommodates the creation of complex geological features such as salt bodies and unconformities. The model generation algorithm employs a multi-randomization strategy, yielding an effectively infinite-dimensional model space that encompasses a wide range of geological scenarios and associated seismic features. Furthermore, RGM incorporates a method to generate synthetic elastic migration images using analytical elastic reflection coefficients combined with frequency-dependent scaling. This functionality enables the creation of training datasets for ML models that leverage elastic seismic images. RGM is implemented in modern object-oriented Fortran, allowing users to flexibly control statistical parameters governing model variability. We demonstrate the capability, performance, and geological realism of the package through comprehensive 2D and 3D examples.

58 GEOSCIENCES↗