Search NASA⌕ Search

SEARCH · Search NASA

Results for “Workflow management systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Software and computing for Run 3 of the ATLAS experiment at the LHC

The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towards Run 4 is also provided.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRSDate not supplied

Practical Battery Thermal Modeling Techniques

Lithium-ion batteries are thermo-electrochemical devices, whereby nearly every facet of their functionality and performance are thermally driven. As a result, it is important to have thermal modeling techniques that effectively capture the intricacies of both the electrochemical nature of the battery and also the complex thermal network that typically results from the design of the battery thermal management system. Here we present a thermal modeling workflow and a set of general assumptions for how to construct a thermal model of a Li-ion battery pack. We use a 14-cell bank of 18650-format Li-ion cells, loosely based on a proposed alternative battery design for Orion, as the example. Although the workflow is performed with Thermal Desktop and related utilities, the focus of this presentation is less about software specific techniques, but rather is focused on the assumptions and conditions that should be used in a model (regardless of the tool used to build the model). Example cases and results will be presented for charge, discharge, and thermal runaway.

lithium-ion battery↗
DOE OSTI2024

Enabling Low-Overhead HT-HPC Workflows at Extreme Scale using GNU Parallel

GNU Parallel is a versatile and powerful tool for process parallelization widely used in scientific computing. This paper demonstrates its effective application in high-performance computing (HPC) environments, particularly focusing on its scalability and efficiency in executing large-scale high-throughput high-performance computing (HT-HPC) workflows. Through real-world examples, we highlight GNU Parallel’s performance across various HPC workloads, including GPU computing, container-based workloads, and node-local NVMe storage. Our results on two leading supercomputers, OLCF’s Frontier and NERSC’s Perlmutter, showcase GNU Parallel’s rapid process dispatching ability and its capacity to maintain low overhead even at extreme scales. We explore GNU Parallel’s application in massive parallel file transfers using a scheduled Data Transfer Node (DTN) cluster, emphasizing its broad utility in diverse scientific workflows. Beyond its direct application as a viable workflow manager, GNU Parallel can be employed in conjunction with other workflow systems as a "last-mile" parallelizing driver and as a quick prototyping tool to design and extract parallel profiles from application executions. We then argue that the potential for GNU Parallel to transform workflow management at extreme scales is substantial, paving the way for more efficient and effective scientific discoveries.

Maheshwari, Ketan↗
DOE OSTI2024

PIPES (Pipeline for Integrated Projects in Energy Systems) [SWR-24-89]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. https://github.com/nrel-pipes/pipes-api https://github.com/nrel-pipes/pipes-web https://github.com/nrel-pipes/nrel-pipes

Gu, Jianli↗
NASA NTRS1992

Consulting report on the NASA technology utilization network system

The purposes of this consulting effort are: (1) to evaluate the existing management and production procedures and workflow as they each relate to the successful development, utilization, and implementation of the NASA Technology Utilization Network System (TUNS) database; (2) to identify, as requested by the NASA Project Monitor, the strengths, weaknesses, areas of bottlenecking, and previously unaddressed problem areas affecting TUNS; (3) to recommend changes or modifications of existing procedures as necessary in order to effect corrections for the overall benefit of NASA TUNS database production, implementation, and utilization; and (4) to recommend the addition of alternative procedures, routines, and activities that will consolidate and facilitate the production, implementation, and utilization of the NASA TUNS database.

Hlava, Marjorie M. K.↗
DOE OSTI2026

iDDS: intelligent distributed dispatch and scheduling for workflow orchestration

The intelligent distributed dispatch and scheduling (iDDS) service is a versatile workflow orchestration system designed for large-scale, distributed scientific computing. iDDS extends traditional workload and data management by integrating data-aware execution, conditional logic, and programmable workflows, enabling automation of complex and dynamic processing pipelines. Originally developed for the ATLAS experiment at the large hadron collider, iDDS has evolved into an experiment-agnostic platform that supports both template-driven workflows and a Function-as-a-Task model for Python-based orchestration. This paper presents the architecture and core components of iDDS, highlighting its scalability, modular message-driven design, and integration with systems such as PanDA and Rucio. We demonstrate its versatility through real-world use cases: fine-grained tape resource optimization for ATLAS, orchestration of large Directed Acyclic Graph (DAG) workflows for the Rubin Observatory, distributed hyperparameter optimization for machine learning applications, active learning for physics analyses, and AI-assisted detector design at the electron–ion collider. By unifying workload scheduling, data movement, and adaptive decision-making, iDDS reduces operational overhead and enables reproducible, high-throughput workflows across heterogeneous infrastructures. We conclude with current challenges and future directions, including interactive, cloud-native, and serverless workflow support.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2019

EARTHDATA PUB: A Data Publication Workflow Solution for NASA’s EOSDIS

Each NASA Distributed Active Archive Center (DAAC) faces the challenge of dealing with an increasingly diverse number of publishable data products from diverse data producers. Data producers, on the other hand, may experience pain points when interacting with the EOSDIS for the first time or when publishing different data at different DAACs. As a result, there has been a growing need to develop a common software framework that serves as a common interface for data producers, rigorously defines the data publication procedure for DAAC staff, facilitates the management of various data publication processes, and tracks the progress of data publication. This software should also account for the different configurations at different DAACs. Currently, two primary data publication workflow and tracking tools exist in operation at the EOSDIS: Semi-Automated ingest System (SAuS) and Data Publication workflow Portal (DAPPeR). However, neither tool is cloud-ready. Automated data processing could be managed by Cumulus, an EOSDIS cloud-based data ingest, archive and management system. However, Cumulus does not support manual tasks or on-premise implementations. We propose to develop the Earthdata Publication Minimum Viable Product (Earthdata Pub MVP) -- a cloud-hosted solution that works with both cloud and on-premise systems and implements the communications and exchange requirements generated by the Earthdata Pub information architecture team.

Rice, Justin L.↗
DOE OSTI2025

A2SD: Accelerating Scientific Innovation Through Autonomous Discovery Systems

The 2025 Advancing Autonomous Scientific Discovery (A2SD) workshop convened researchers from academia, national laboratories, and industry to explore the transformative role of autonomy in scientific discovery. The workshop highlighted a convergence of artificial intelligence, robotics, and computational workflows into autonomous systems capable of accelerating the scientific process. Presentations and discussions spanned autonomous experimentation, intelligent workflow orchestration, digital twins, and agent-based systems for managing complex research ecosystems. Key challenges discussed included interoperability across heterogeneous infrastructures, near real-time data management under FAIR principles, reproducibility, and the integration of human oversight. The workshop also emphasized the need for modular software interfaces, federated learning models, and education initiatives to support a next-generation scientific workforce.

Taufer, Michela [University of Tennessee, Knoxvill↗
DOE OSTI2025

Data Management in the Continuum: Cross-facility Object-based Data Transfers

Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC–Cloud and HPC–HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45 × over an optimized HDF5 usage and 15 × over the HDF5 with caching by using the new interface in PDC-XF.

Bez, Jean Luca↗
DOE OSTI2026

kessel

Kessel is a tool to create and drive continuous integration (CI) and developer workflows through a unified interface across multiple code projects and environments. It serves as a driver and integration layer for build systems and package managers, providing a flexible library of reusable components to build and execute complex workflows consistently.

Berger, Richard [@lanl]↗
NASA NTRS2014

Exploration Technologies for Operations

Although the International Space Station (ISS) assembly has been completed, the Operations support teams continue to seek more efficient and effective ways to prepare for and conduct the ISS operations and future exploration missions beyond low earth orbit. This search for improvement has led to a significant collaboration between the NASA research and advanced software development community at NASA Ames Research Center and the Mission Operations community at NASA Johnson Space Center. Since 2001, NASA Ames Research Center has been developing and applying its advanced intelligent systems and human systems integration research to mission operations tools for several of the unmanned Mars missions operations. Since 2006, NASA Ames Research Center has also been developing and applying its advanced intelligent systems and human systems integration research to mission operations tools for manned operations support with the Mission Operations Directorate at NASA Johnson Space Center. This paper discusses the completion of the development and deployment of a variety of intelligent and human systems technologies adopted for manned mission operations. The technologies associated with the projects include advanced software systems for operations and human-centered computing. Human-centered computing looks to the processes and procedures that people do to perform any given job, then attempts to identify opportunities to improve these processes and procedures. In particular, for mission operations, improvements are quantified by specifically identifying how a tool can increase a persons efficiency, enhance a persons functional capability, andor improve the assurance of a persons decisions. The Ames development team has collaborated with the Mission Operations team to identify areas of efficiencies through technology infusion applications in support of the Plan, Train, and Fly activities of human-spaceflight mission operations. The specific applications discussed in this paper are in the areas of mission planning systems, mission operations design modeling and workflow automation, advanced systems monitoring, mission control technologies, search tools, training management tools, spacecraft solar array management, spacecraft power management, and spacecraft attitude planning. We discuss these specific projects between the Ames Research Center and the Johnson Space Centers Mission Operations Directorate, and how these technologies and projects are enhancing the mission operations support for the International Space Station. We also discuss the challenges, problems, and successes associated with long-distance and multi-year development projects between the research team at Ames and the Mission Operations customers at Johnson Space center. Finally, we discuss how these technology infusion applications and underlying technologies might be used in the future to support on-board operations of the crew and spacecraft systems as human exploration expands beyond low earth orbit to destinations in the solar system where communications delays will require more on-board autonomy and planning by the crew. Longer communications delays will require that the ground mission operations support will be primarily strategic in nature, while the tactical level of planning, systems monitoring and control, and failure analysisisolationrecovery will be the responsibility of both the spacecraft autonomous systems and the crew. Our expectation is that the technologies

mission operations↗
DOE OSTI2025

EnergyPlus-MCP: A model-context-protocol server for ai-driven building energy modeling

Traditional building energy modeling with the EnergyPlus building performance simulation engine requires domain expertise, programming skills, and intensive manual efforts limiting its effective adoption. This paper introduces EnergyPlus-MCP, the first open-source Model Context Protocol (MCP) server specifically designed for EnergyPlus simulation workflows, establishing a new foundational infrastructure for AI-driven building energy modeling. The MCP server implements a layered architecture with 35 specialized tools spanning model management, editing and analysis, HVAC and other systems configuration inspection, and simulation execution, enabling Large Language Models to interact with EnergyPlus through conversational interfaces. The server addresses critical workflow barriers by automating model validation, streamlining energy efficiency measures modification, and providing intelligent output management with interactive visualization. Through practical demonstrations using a multi-zone building retrofit analysis, we show how the EnergyPlus-MCP server significantly reduces manual efforts while maintaining full simulation rigor. By providing accessible natural language interfaces to sophisticated building energy analysis, this approach enables scalable deployment of simulation expertise across public and private organizations, educational institutions, and research teams, fundamentally transforming traditional building energy modeling practices.

AI↗
DOE OSTI2025

Osprey Framework v0.2.2

The Alpha Berkeley Framework is a software architecture for building agentic AI systems that coordinate multi-step workflows in scientific and industrial environments. It is based on a plan-first orchestration model, where natural language requests are translated into execution plans with explicit dependencies and optional human approval. The framework includes capability classification, which selects relevant tools on a per-task basis to keep orchestration efficient as the number of available tools grows. It incorporates task extraction methods that compress conversational context and integrate external resources such as databases, APIs, and knowledge bases into structured, machine-readable tasks. Execution is supported by modular services with checkpointing, artifact management, and error handling, allowing workflows to be paused, inspected, and resumed. The system is designed for deployment in production environments, supporting both local and containerized execution as well as integration with HPC clusters. Interfaces include command-line tools, browser-based workflows, and containerized services. The framework has been demonstrated in tutorial examples and deployed at the Advanced Light Source, where it coordinates accelerator control and analysis workflows.

Hellert, Thorsten [Lawrence Berkeley National Labo↗
DOE OSTI2025

Creating Apptainer Workflows with Docker-Compose-like Utilities

Creating Apptainer Workflows with Docker-Compose-like Utilities In this presentation, I will explore the utilization of a tool called process-compose, inspired by docker-compose, to create Apptainer-based services. This approach allows for easy deployment and management of fully containerized applications on High Performance Computing (HPC) systems without requiring elevated privileges. Benefits to the Ecosystem: By incorporating process-compose and Apptainer, I aim to address several key challenges in the HPC ecosystem: Simplified Workflow Management: Process-compose provides a user-friendly interface for defining and managing complex containerized application services, reducing the setup time and lowering the barrier to entry for new users. Enhanced Portability: Apptainer ensures that containerized applications can run consistently across different HPC environments, promoting greater portability and reducing compatibility issues. Process-compose is also a single binary that does not need to be installed by admin level users. Community Driven Solutions: This approach aligns with the goals of the High Performance Software Foundation (HPSF) to advance community-driven solutions. By sharing our experiences and insights, I hope to foster collaboration and innovation within the HPC community. Increased Productivity: The combination of process-compose and Apptainer streamlines the serve deployment process, allowing researchers and developers to focus more on their scientific work rather than the intricacies of system or service administration. Through this presentation, attendees will gain valuable insights into the practical implementation of containerized workflows on HPC systems, learn about the benefits of using process-compose and Apptainer, and understand how these tools can contribute to a more efficient HPC ecosystem.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Extreme-scale workflows: A perspective from the JLESC international community

The Joint Laboratory for Extreme-Scale Computing (JLESC) focuses on software challenges in high-performance computing systems to meet the needs of today’s science campaigns, which often require large resources, consist of multiple tasks, and generate vast amounts of data. In this context, extreme-scale workflows have been the key factor in enabling scientific discoveries by helping scientists automate the dependencies and data exchanges between workflow tasks, instead of managing those manually. Here, in this paper, we present representative extreme-scale workflows and feature workflow systems developed by JLESC participating institutions. We present lessons learned while developing these tools, alongside with the open challenges and future research directions in the field of extreme-scale workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Bridging paradigms: Designing for HPC-Quantum convergence

Here, this paper presents a comprehensive software stack architecture for integrating quantum computing (QC) capabilities with High-Performance Computing (HPC) environments. While quantum computers show promise as specialized accelerators for scientific computing, their effective integration with classical HPC systems presents significant technical challenges. We propose a hardware-agnostic software framework that supports both current noisy intermediate-scale quantum devices and future fault-tolerant quantum computers, while maintaining compatibility with existing HPC workflows. The architecture includes a quantum gateway interface, standardized APIs for resource management, and robust scheduling mechanisms to handle both simultaneous and interleaved quantum–classical workloads. Key innovations include: (1) a unified resource management system that efficiently coordinates quantum and classical resources, (2) a flexible quantum programming interface that abstracts hardware-specific details, (3) A Quantum Platform Manager API that simplifies the integration of various quantum hardware systems, and (4) a comprehensive tool chain for quantum circuit optimization and execution. We demonstrate our architecture through implementation of quantum–classical algorithms, including the variational quantum linear solver, showcasing the framework’s ability to handle complex hybrid workflows while maximizing resource utilization. This work provides a foundational blueprint for integrating QC capabilities into existing HPC infrastructures, addressing critical challenges in resource management, job scheduling, and efficient data movement between classical and quantum resources.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

TPSAS-NF1676L-33992-DND

The CERES Science Team integrates and fuses observations from 6 CERES instruments aboard the Terra, Aqua, S-NPP, and NOAA-20 missions with data from more than 20 other unique data sources. Following the November 2017 launch of CERES Flight Model 6 (FM6) onboard NOAA20, CERES has now amassed over 80 instrument-years of valuable Earth radiation budget data. The rapidly growing volume of CERES data coupled with the introduction of new data products alongside improvements to existing science algorithms fosters the requirement for faster, more flexible, and scalable data production and orchestration. New virtualized, cloud-centric compute hardware hosted by the NASA Langley Research Center’s (LaRC) Atmospheric Sciences Data Center (ASDC) provides an ideal environment for these ever-increasing data production demands for CERES. This poster discusses updates to the implementation of the CERES Data Management Team’s (DMT) CERES AuTomAted job Loading sYSTem (CATALYST), a custom data processing workflow engine for CERES, to use on-demand computing resources to perform automated CERES data production processing in a Linux-based container environment. Linux containers provide CERES the flexibility to build multiple production environments in containers tailored for specific workloads and allow effortless provisioning of resources based on the CERES Science Team’s data production requirements.

Thomas N. Hillyer↗
DOE OSTI2024

AI-assisted detector design for the EIC (AID(2)E)

Artificial Intelligence is poised to transform the design of complex, large-scale detectors like ePIC at the future Electron Ion Collider. Featuring a central detector with additional detecting systems in the far forward and far backward regions, the ePIC experiment incorporates numerous design parameters and objectives, including performance, physics reach, and cost, constrained by mechanical and geometric limits. This project aims to develop a scalable, distributed AI-assisted detector design for the EIC (AID(2)E), employing state-of-the-art multiobjective optimization to tackle complex designs. Supported by the ePIC software stack and using G EANT 4 simulations, our approach benefits from transparent parameterization and advanced AI features. The workflow leverages the PanDA and iDDS systems, used in major experiments such as ATLAS at CERN LHC, the Rubin Observatory, and sPHENIX at RHIC, to manage the compute intensive demands of ePIC detector simulations. Tailored enhancements to the PanDA system focus on usability, scalability, automation, and monitoring. Ultimately, this project aims to establish a robust design capability, apply a distributed AI-assisted workflow to the ePIC detector, and extend its applications to the design of the second detector (Detector-2) in the EIC, as well as to calibration and alignment tasks. Additionally, we are developing advanced data science tools to efficiently navigate the complex, multidimensional trade-offs identified through this optimization process.

97 MATHEMATICS AND COMPUTING↗