Search NASASearch

SEARCH · Search NASA

Results for “workflow management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Active learning path-dependent properties using a cloud-based materials acceleration platform

Solid state materials are central to many modern technologies in which a given material may be exposed to a variety of environments. The material properties often vary with the sequence of environments in an irreversible manner, resulting in a quintessential path-dependency in experimental observables. While sequential learning techniques have been effectively deployed for accelerating learning of state properties of materials, they often use a consistent environment path in all experiments. To elevate such techniques for making optimal decisions in experimental investigations of path-dependent properties, we introduce an iterated expected information gain acquisition function that optimizes over entire experimental trajectories. This approach is implemented within a cloud-based Materials Acceleration Platform architecture utilizing an event-driven stateful broker coupled with remote HELAO (Hierarchical Experimental Laboratory Automation and Orchestration) instances and an AI science manager. The platform's efficacy was demonstrated through a case study optimizing multi-step spectro-electrochemical experiments to identify optically stable potential windows in (Co–Ni–Sb)O z metal oxides. The system successfully integrated AI-driven experiment design, remote laboratory automation, and cloud-based data infrastructure, validating the platform's capability for managing complex, adaptive, path-dependent workflows in materials discovery.

Guevarra, Dan [California Institute of Technology

Software to Manage the Unmanageable

In 1995, NASA s Jet Propulsion Laboratory (JPL) contracted Redmond, Washington-based Lucidoc Corporation, to design a technology infrastructure to automate the intersection between policy management and operations management with advanced software that automates document workflow, document status, and uniformity of document layout. JPL had very specific parameters for the software. It expected to store and catalog over 8,000 technical and procedural documents integrated with hundreds of processes. The project ended in 2000, but NASA still uses the resulting highly secure document management system, and Lucidoc has managed to help other organizations, large and small, with integrating document flow and operations management to ensure a compliance-ready culture.

Source record

GDSA framework, a computational framework for complex modeling problems in radioactive waste management

This paper details a computational framework to produce automated, graphical workflows, and how this framework can be deployed to support complex modeling problems like those in nuclear engineering. Key benefits of the framework include: automating previously manual workflows; intuitive construction and communication of workflows through a graphical interface; and automated file transfer and handling for workflows deployed across heterogeneous computing resources. This paper demonstrates the framework's application to probabilistic post-closure performance assessment of systems for deep geologic disposal of nuclear waste. However, the framework is a general capability that can help users running a variety of computational studies.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING

Phase II Field Demonstration at Lansing Smith Generating Plant, Southport, Florida (Final Report)

The Final Technical Report: Field Test Design and Pressure Management Strategies for Phase II Field Demonstration of Optimal Pressure Control, Plume Management, and Produced Water Strategies presents the culmination of multi-year efforts under the U.S. Department of Energy’s Brine Extraction and Storage Test (BEST) program, led by EPRI in partnership with Gulf Power at the Plant Smith site near Panama City, Florida. The project was designed to evaluate and demonstrate the technical feasibility of managing subsurface pressures and fluid movement associated with large-scale CO₂ injection, using low-salinity water as a proxy fluid. Through a combination of field injection testing, reservoir modeling, and optimization studies, the research team developed and refined active and passive brine extraction strategies aimed at controlling injection-induced pressure buildup, mitigating risks of fault activation, and managing plume migration. The field demonstration incorporated a new injection well (TIW-2), a new monitoring/extraction well (TEMW-A), and an existing well (TIW-1) repurposed for passive pressure relief. Complementary geophysical monitoring was designed to track plume development and evaluate the effectiveness of pressure management strategies. The report details the integrated workflow encompassing wellfield development, regulatory permitting, model calibration, and survey design. It includes a comprehensive electromagnetic (EM) modeling and inversion study used to develop a cost-effective, time-lapse geophysical monitoring plan capable of imaging the evolving low-salinity plume within the highly saline Lower Tuscaloosa formation. Reservoir simulation results guided the design of the 17-month injection program and the timing of active extraction to maintain formation pressures below a hypothetical fault reactivation threshold. Supporting analyses evaluated potential injectivity risks related to clay fines migration and geochemical incompatibility, identifying practical mitigation measures such as salinity and pH control. The final design recommends a series of crosswell EM surveys—conducted before, during, and after injection—as the most effective and economical approach for plume imaging, supplemented by continuous downhole pressure and flow monitoring. Collectively, the study provides a field-ready framework for cost-effective pressure management and monitoring in support of future CO₂ storage projects.

01 COAL, LIGNITE, AND PEAT

Data Management in the Continuum: Cross-facility Object-based Data Transfers

Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC–Cloud and HPC–HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45 × over an optimized HDF5 usage and 15 × over the HDF5 with caching by using the new interface in PDC-XF.

Bez, Jean Luca

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)

Toward a Common Earth Data Publication Framework

Data publication is an essential activity for all data archives. Each of NASA's twelve Distributed Active Archive Centers (DAACs) have established publication workflows which account for the heterogeneous suite of missions, instruments, data providers, and datasets managed within the Earth Observation System Data and Information System (EOSDIS) program. Some aspects of data publication vary across DAACs: workflows range from manual to automatic, terms used to describe publication elements differ, and systems used to publish and manage data vary. Despite these differences, the DAAC data publication processes are generally the same: obtain the data and related information from data providers, describe the data with metadata and documentation, and release the data for access by the user community. In order to improve consistency and reduce the time required to publish data, we have developed a cross-DAAC initiative called the Common Earthdata Publication Framework (Earthdata Pub). Earthdata Pub seeks to: standardize communications and interactions with data providers; identify and standardize common workflows and steps in the data publication process; and design/implement a front-end system with features that include a common web interface, email & status tracking, and common application programming interfaces (APIs) to communicate with various DAAC-specific software components (services and applications) on the back-end. We will present the latest updates on this effort's progress and future plans.

data publication

Network performance analysis for HPC datacenters (net_perf) v1.0

The software has two main features: (1) identify data movement trends in HPC data centers that use network flow monitoring (2) analyze the performance of individual data flows under the existing data movement management strategy and identify performance bottlenecks that impede timely data availability for science workflows. Its main advantage is that it is tailored for HPC network traffic by considering HPC data movement management intricacies.

Giannakou, Anna

MADA: Multi-Agent Design Assistant

MADA (Multi-Agent Design Assistant) is a Large Language Model (LLM) powered multi-agent framework that coordinates specialized agents for complex design workflows. The system was designed for HPC workflows with the following agents in mind: 1) A Job Management Agent (JMA) launches and manages ensemble simulations on HPC systems, 2) a Geometry Agent (GA) generates meshes, and 3) an Inverse Design Agent (IDA) proposes new designs informed by simulation outcomes. Our framework reduces cumbersome manual workflow setup, and enables automated design exploration at scale. However, the software also enables users to rapidly create new multi-agent systems. Simply define new agents in a configuration file, giving each their own set of tools (via MCP), and then chat and prompt your new multi-agent system. Is

Gunnarson, BrianS [Lawrence Livermore National Lab

A decade of progress in understanding and managing legacy well integrity for geologic carbon storage

This study reviews a decade of research progress in legacy well integrity and risk management for geologic carbon storage (GCS) to commemorate the 20 th anniversary of the Intergovernmental Panel on Climate Change’s 2005 Special Report on Carbon Capture and Storage. In the past ten years, legacy well research has benefited from global efforts to constrain emissions from abandoned oil and gas wells, a continued focus on well materials performance in the presence of CO 2 -rich fluids, and practical experience gained through GCS implementation. Field measurements of abandoned well emissions show that leakage is not universal or catastrophic but forms a continuum of low-to-moderate fluxes that depend on isolation integrity and environmental attenuation. Materials research has constrained the conditions under which Portland cements exhibit self-sealing and non-sealing behaviors, and has identified the impact of geomechanical properties, non-uniform pathway apertures, multi-phase flow, and impurities in the CO 2 stream, on leakage pathways as important new areas for investigation. GCS projects at brownfield sites have inspired the creation of new workflows that integrate various tools and technologies to manage legacy well leakage risks. GCS implementation has also motivated a push towards scenario-based well modeling that directly informs permit applications. These advances inspire new research questions for the coming decade, particularly around the level of legacy well leakage risk that is environmentally acceptable and tolerable to stakeholders when sequestering millions of tonnes of CO 2 annually.

Carbon capture and storage

Interpretable Models for Workflow Differentiation in High-Performance Scientific Networks

Scientific workflows in high-performance networks spawn hundreds of interdependent flows that must be managed collectively—yet existing network classifiers treat each flow in isolation, leading to fragmented QoS decisions and missed interflow patterns. We present a novel traffic classification solution that operates at the workflow level, distinguishing entire filetransfer operations from streaming analytics by capturing how concurrent flows interact and burst together. We introduce a workflow identification window (WIW) that ingests raw packet headers from parallel flows into unified tensors, preserving the spatial-temporal patterns that differentiate scientific workflows. This approach achieves 98.7% accuracy using CNN, LSTM, and hybrid architectures, while maintaining 84% accuracy on production traffic collected a week later—demonstrating robustness to temporal drift. By integrating SHAP and GradCAM explainability, we reveal that early-packet timing patterns and cross-flow correlations drive classification decisions, providing operators with interpretable insights. Our system enables coherent workflow-level QoS enforcement and dynamic bandwidth allocation in scientific networks, eliminating manual per-flow configuration while maintaining classification latency at millisecond level.

Giannakou, Anna [LBL, Berkeley]

Workflow-Based Software Development Environment

The Software Developer's Assistant (SDA) helps software teams more efficiently and accurately conduct or execute software processes associated with NASA mission-critical software. SDA is a process enactment platform that guides software teams through project-specific standards, processes, and procedures. Software projects are decomposed into all of their required process steps or tasks, and each task is assigned to project personnel. SDA orchestrates the performance of work required to complete all process tasks in the correct sequence. The software then notifies team members when they may begin work on their assigned tasks and provides the tools, instructions, reference materials, and supportive artifacts that allow users to compliantly perform the work. A combination of technology components captures and enacts any software process use to support the software lifecycle. It creates an adaptive workflow environment that can be modified as needed. SDA achieves software process automation through a Business Process Management (BPM) approach to managing the software lifecycle for mission-critical projects. It contains five main parts: TieFlow (workflow engine), Business Rules (rules to alter process flow), Common Repository (storage for project artifacts, versions, history, schedules, etc.), SOA (interface to allow internal, GFE, or COTS tools integration), and the Web Portal Interface (collaborative web environment

Izygon, Michel E.

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM

ML-AMD/exa-pd

Exa-pd is a highly parallelizable workflow for constructing multi-element phase diagrams (PDs). It uses standard sampling techniques—molecular dynamics (MD) and Monte Carlo (MC)—as implemented in the LAMMPS package, to simultaneously sample multiple phases on a fine temperature–composition mesh for free-energy calculations. The workflow uses Parsl as a global controller to manage the MD/MC jobs to achieve massive parallelization with almost ideal scalability. The resulting free energies of both liquid and solid phases (including solid solutions) are then fed to CALPHAD modeling using the PYCALPHAD package for the construction of a multi-element PD.

Zhang, Feng [Ames Laboratory (AMES), Ames, IA (Uni

NASA Tech Briefs, November 2013

Topics include: Cryogenic Liquid Sample Acquisition System for Remote Space Applications; 5 Spatial Statistical Data Fusion (SSDF); GPS Estimates of Integrated Precipitable Water Aid Weather Forecasters; Integrating a Microwave Radiometer into Radar Hardware for Simultaneous Data Collection Between the Instruments; Rapid Detection of Herpes Viruses for Clinical Applications; High-Speed Data Recorder for Space, Geodesy, and Other High-Speed Recording Applications; Datacasting V3.0; An All-Solid-State, Room-Temperature, Heterodyne Receiver for Atmospheric Spectroscopy at 1.2 THz; Stacked Transformer for Driver Gain and Receive Signal Splitting; Wireless Integrated Microelectronic Vacuum Sensor System; Fabrication Method for LOBSTER-Eye Optics in <110> Silicon; Compact Focal Plane Assembly for Planetary Science; Fabrication Methods for Adaptive Deformable Mirrors; Visiting Vehicle Ground Trajectory Tool; Workflow-Based Software Development Environment; Mobile Thread Task Manager; A Kinematic Calibration Process for Flight Robotic Arms; Magnetostrictive Alternator; Bulk Metallic Glasses and Composites for Optical and Compliant Mechanisms; Detection of Only Viable Bacterial Spores Using a Live/Dead Indicator in Mixed Populations; and Intravenous Fluid Generation System.

Source record

Practical Battery Thermal Modeling Techniques

Lithium-ion batteries are thermo-electrochemical devices, whereby nearly every facet of their functionality and performance are thermally driven. As a result, it is important to have thermal modeling techniques that effectively capture the intricacies of both the electrochemical nature of the battery and also the complex thermal network that typically results from the design of the battery thermal management system. Here we present a thermal modeling workflow and a set of general assumptions for how to construct a thermal model of a Li-ion battery pack. We use a 14-cell bank of 18650-format Li-ion cells, loosely based on a proposed alternative battery design for Orion, as the example. Although the workflow is performed with Thermal Desktop and related utilities, the focus of this presentation is less about software specific techniques, but rather is focused on the assumptions and conditions that should be used in a model (regardless of the tool used to build the model). Example cases and results will be presented for charge, discharge, and thermal runaway.

lithium-ion battery

mzPeak: Designing a Scalable, Interoperable, and Future-Ready Mass Spectrometry Data Format

Advances in mass spectrometry (MS) instrumentation, such as higher resolution, faster scan speeds, and improved sensitivity, have significantly increased the volume and complexity of data. The growing adoption of imaging and ion mobility further amplifies these challenges across MS-based omics fields, including proteomics, metabolomics, and lipidomics. While these technologies unlock new possibilities, they also present significant challenges in data management, storage, and accessibility. Existing open formats, such as the XML-based community standards mzML and imzML, struggle to meet the demands of modern MS workflows due to their large file sizes, slow data access, and limited metadata support. Vendor-specific formats, while optimized for proprietary instruments, lack interoperability, comprehensive metadata support and long-term archival reliability. This white paper lays the groundwork for mzPeak, a next-generation community data format designed to address these challenges and support high-throughput, multi-dimensional MS workflows. By adopting a hybrid model that combines efficient binary storage for numerical data and both human and machine-readable metadata storage, mzPeak will reduce file sizes, accelerate data access, and offer a scalable, adaptable solution for evolving MS technologies. For researchers, mzPeak will enable enhanced interoperability across platforms, seamless support for complex workflows including ion mobility and MS imaging, and faster data access compared to existing community formats such as mzML. Its design will ensure data is managed in compliance with regulatory standards, essential for applications such as precision medicine and chemical safety, where long-term data integrity and accessibility are critical. For vendors, mzPeak provides a streamlined, open alternative to proprietary formats, reducing the burden of regulatory compliance while aligning with the industry's push for transparency and standardization. By offering a high-performance, interoperable solution, mzPeak positions vendors to meet customer demands for sustainable data management tools which will be able to handle emerging and future data types and workflows. mzPeak aspires to become the cornerstone of MS data management, empowering researchers, vendors, and developers to innovate and collaborate more effectively.

data formats