Search NASA⌕ Search

SEARCH · Search NASA

Results for “Modeling workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Scidac-Data: Enabling Data Driven Modeling of Exascale Computing

Here, the SciDAC-Data project is a DOE-funded initiative to analyze and exploit two decades of information and analytics that have been collected by the Fermilab data center on the organization, movement, and consumption of high energy physics (HEP) data. The project analyzes the analysis patterns and data organization that have been used by NOvA, MicroBooNE, MINERvA, CDF, D0, and other experiments to develop realistic models of HEP analysis workflows and data processing. The SciDAC-Data project aims to provide both realistic input vectors and corresponding output data that can be used to optimize and validate simulations of HEP analysis. These simulations are designed to address questions of data handling, cache optimization, and workflow structures that are the prerequisites for modern HEP analysis chains to be mapped and optimized to run on the next generation of leadership-class exascale computing facilities. We present the use of a subset of the SciDAC-Data distributions, acquired from analysis of approximately 71,000 HEP workflows run on the Fermilab data center and corresponding to over 9 million individual analysis jobs, as the input to detailed queuing simulations that model the expected data consumption and caching behaviors of the work running in high performance computing (HPC) and high throughput computing (HTC) environments. In particular we describe how the Sequential Access via Metadata (SAM) data-handling system in combination with the dCache/Enstore-based data archive facilities has been used to develop radically different models for analyzing the HEP data. We also show how the simulations may be used to assess the impact of design choices in archive facilities.

97 MATHEMATICS AND COMPUTING↗

ADEPT: A Pedagogical Framework for Integrating Agentic AI with Deterministic Scientific Workflows

The integration of Large Language Models (LLMs) into scientific research promises to accelerate discovery, yet a significant gap remains between the dynamic reasoning of Artificial Intelligence (AI) agents and the static, deterministic nature of canonical scientific workflows. This paper introduces ADEPT (Agentic Discovery and Exploration Platform for Tools), a reference architecture and pedagogical framework explicitly designed to bridge this gap. ADEPT's primary mission is to provide a transparent, "glass-box" environment where researchers and engineers can learn to effectively wrap established scientific software (e.g., BLAST, Nextflow pipelines) and compose it into reliable, agent-driven workflows. We describe its modular, multi-server architecture, which leverages the Model Context Protocol (MCP) for tool serving, LangGraph for robust agentic orchestration, and a secure nsjail-based sandbox for safe code execution. By prioritizing architectural clarity, safety, and modularity, ADEPT serves as an extensible blueprint for building trustworthy AI-augmented systems and fosters the collaborative development necessary to responsibly employ agentic AI for science. We provide practical examples of how to adapt and extend this framework, highlighting its utility in workforce development and AI-readiness capabilities across research and development projects.

97 MATHEMATICS AND COMPUTING↗

A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly-accurate analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapid generation of complex nanoscale atomic structures which enables fast, systematic sampling of realistic nanomaterial structures and can be used as a random structure generator for large, diverse synthetic datasets. Using Construction Zone, we develop an end-to-end machine learning workflow for training neural network models to analyze experimental atomic resolution HRTEM images on the task of nanoparticle image segmentation purely with simulated databases. Further, we study the data curation process to understand how various aspects of the curated simulated data—including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions—affect model performance across three benchmark experimental HRTEM image datasets. Using our workflow, we are able to achieve state-of-the-art segmentation performance on these experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data. Construction Zone and its documentation are available at https://github.com/lerandc/construction_zone.

36 MATERIALS SCIENCE↗

Data readiness pipeline patterns for scientific AI at scale: Insights from climate, fusion, life sciences, and materials

This article examines how data readiness for AI principles apply to large scientific datasets used to train foundation models. We analyze archetypal workflows across four representative domains—climate, nuclear fusion, life sciences, and materials—to identify common preprocessing patterns and domain‐specific constraints. We introduce a two‐dimensional readiness model that combines canonical preprocessing patterns with a five‐level operational readiness scale, both tailored to high‐performance computing (HPC) environments. This construct helps outline key challenges in transforming large‐scale scientific data into formats suitable for scalable AI training. Together, these dimensions form a conceptual maturity matrix that characterizes scientific data readiness and guides infrastructure development toward standardized, cross‐domain support for scalable and reproducible AI for science. Finally, we evaluate this maturity matrix in the context of case studies including ClimaX (climate), AFLOW (materials), OpenFold (proteomics), and DIII‐D fusion disruption‐prediction workflows, from which we distill lessons learned and provide recommendations to guide practitioners in developing robust AI‐readiness pipelines. Finally, we discuss remaining cross‐cutting challenges that persist across scientific domains.

97 MATHEMATICS AND COMPUTING↗

Panorama 360 (Final Report)

This is the final technical report for the DOE-funded Panorama 360 project. Panorama 360 provided a resource for the collection, analysis, and sharing of performance data about end-to-end scientific workflows executing on DOE facilities. The work focused on workflows that include experimental data generation at DOE facilities. The main activities of Panorama 360 include the development of: 1. A distributed repository that stores different types of workflow execution data (e.g., point and time series performance traces at fine- and coarse-grained levels); 2. A set of open-source data capture, curation, and publishing tools fully integrated with a state-of-the-art workflow management system that automates data ingestion to the repository and enables users to discover, query, and process data from the repository; 3. A set of analysis algorithms and machine learning based tools to perform analysis and characterization of the gathered data, which can be used to detect anomalous performance or system faults; and 4. Best practices and recommendations for workflow evaluation, analysis, execution, and architectures.

97 MATHEMATICS AND COMPUTING↗

CO2 Plume Imaging with Accelerated Deep Learning-based Data Assimilation Considering Multiple Realizations: Application to the Illinois Basin-Decatur Carbon Sequestration Project

We propose a fast and efficient deep learning workflow for near real-time data assimilation, forecasting and visualization of CO2 plume evolution in saline aquifer and demonstrate its application at a field site. Unlike the previous work, this study incorporates the impact of spatial heterogeneity using multiple realizations. In the proposed workflow, a neural network model utilizes available monitoring data such as downhole pressure measurements as input and predicts the propagating pressure ‘front’ using the diffusive time of flight (DTOF) map which is considered as representative reservoir image of the flow field. The DTOF is the arrival time of pressure front propagation, which can be computed by the Fast Marching Method rapidly without flow simulations. Reservoir model calibration can be implemented by selecting the training data samples that describe the predicted DTOF map based on observed data. The power and efficacy of our workflow is demonstrated by application to the Illinois Basin-Decatur Project.

CO2 plume imaging↗

A characterization of workflow management systems for extreme-scale applications

We present that the automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today’s computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. Finally, the paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems.

97 MATHEMATICS AND COMPUTING↗

Pipeline for Integrated Projects in Energy Systems (PIPES): A Tool for Integrated System Planning [Slides]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. This presentation introduces PIPES a multi-model tool for integrated system planning; it describes the underlying architecture, deep dives into common user workflows, and outlines the upcoming development roadmap beyond its current alpha state.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Rapid construction of metabolic models for a family of Cyanobacteria using a multiple source annotation workflow

Cyanobacteria are photoautotrophic prokaryotes that exhibit robust growth under diverse environmental conditions with minimal nutritional requirements. They can use solar energy to convert CO 2 and other reduced carbon sources into biofuels and chemical products. The genus Cyanothece includes unicellular nitrogen-fixing cyanobacteria that have been shown to offer high levels of hydrogen production and nitrogen fixation. The reconstruction of quality genome-scale metabolic models for organisms with limited annotation resources remains a challenging task. Here we reconstruct and subsequently analyze and compare the metabolism of five Cyanothece strains, namely Cyanothece sp. PCC 7424, 7425, 7822, 8801 and 8802, as the genome-scale metabolic reconstructions iCyc792, iCyn731, iCyj826, iCyp752, and iCyh755 respectively. We compare these phylogenetically related Cyanothece strains to assess their bio-production potential. A systematic workflow is introduced for integrating and prioritizing annotation information from the Universal Protein Resource (Uniprot), NCBI Protein Clusters, and the Rapid Annotations using Subsystems Technology (RAST) method. The genome-scale metabolic models include fully traced photosynthesis reactions and respiratory chains, as well as balanced reactions and GPR associations. Metabolic differences between the organisms are highlighted such as the non-fermentative pathway for alcohol production found in only Cyanothece 7424, 8801, and 8802. Our development workflow provides a path for constructing models using information from curated models of related organisms and reviewed gene annotations. This effort lays the foundation for the expedient construction of curated metabolic models for organisms that, while not being the target of comprehensive research, have a sequenced genome and are related to an organism with a curated metabolic model. Organism-specific models, such as the five presented in this paper, can be used to identify optimal genetic manipulations for targeted metabolite overproduction as well as to investigate the biology of diverse organisms.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluating FRI3D for Cost Savings in Fire Hazard Analysis at DOE Sites

A fire hazard analysis, required for many U.S. Department of Energy (DOE) facilities, is a complex, cumbersome, and costly process. Fire hazard analyses may be viewed as a checkbox, but ideally and in spirit with the DOE-STD-1066, the fire hazard analysis (FHA) should be a part of the workflow and used to help in modifications, maintenance, and improving operational safety. With current FHA development processes, it is both time and cost prohibitive for true integration. A tool called Fire Risk Investigation in 3D or FRI3D was developed under the DOE Light Water Reactor Sustainability program to simplify and automate many aspects of a fire probabilistic risk analysis for existing nuclear power plants. The FRI3D tool automates fire scenarios by combining approved fire simulation codes, U.S. Nuclear Regulatory Commission fire calculations methods, 3D modeling and visualization, and probabilistic risk analysis models into a single workflow supported with a user interface. FRI3D was initially designed for used in combination with a PRA, this case study, evaluated using FRI3D for a plant modification, determined the benefits that detailed fire modeling can have for U.S. Department of Energy facilities with or without a PRA model. It also looked at what tasks from DOE requirements could be reduced using the tool and what is needed to integrate fire hazard analysis into site workflow.

97 - MATHEMATICS AND COMPUTING↗

Machine learning enabling high-throughput and remote operations at large-scale user facilities

Imaging, scattering, and spectroscopy are fundamental in understanding and discovering new functional materials. Contemporary innovations in automation and experimental techniques have led to these measurements being performed much faster and with higher resolution, thus producing vast amounts of data for analysis. These innovations are particularly pronounced at user facilities and synchrotron light sources. Machine learning (ML) methods are regularly developed to process and interpret large datasets in real-time with measurements. However, there remain conceptual barriers to entry for the facility general user community, whom often lack expertise in ML, and technical barriers for deploying ML models. Herein, we demonstrate a variety of archetypal ML models for on-the-fly analysis at multiple beamlines at the National Synchrotron Light Source II (NSLS-II). We describe these examples instructively, with a focus on integrating the models into existing experimental workflows, such that the reader can easily include their own ML techniques into experiments at NSLS-II or facilities with a common infrastructure. The framework presented here shows how with little effort, diverse ML models operate in conjunction with feedback loops via integration into the existing Bluesky Suite for experimental orchestration and data management.

36 MATERIALS SCIENCE↗

3D reconstruction and neural rendering for adversarial machine learning

While evasion attacks on computer vision systems have been widely studied, creating attacks that remain effective under significant changes in viewpoint continues to be challenging. Traditional approaches often rely on affine transformations of images, but these approaches degrade at larger perspective shifts and often produce unrealistic or ineffective perturbations. Recent methods use differentiable renderers to improve viewpoint robustness, but they typically depend on manually constructed 3D models. We introduce a semi-automated pipeline that generates physically printable and perspective-invariant adversarial patches using only a small set of 2D images. Our method integrates 3D reconstruction, neural rendering, adversarial patch optimization, and an object detection victim model into a unified workflow. We use 2D Gaussian Splatting for high fidelity mesh reconstruction and FlexPara for surface parameterization that produces texture maps suitable for patch editing. Together, these components form a fully differentiable pipeline in PyTorch3D that links texture modification to model outputs, enabling efficient optimization of patches that remain effective across many viewpoints. The complete process, from image capture to patch printing and physical evaluation, can be completed within a few hours. We demonstrate the effectiveness of the resulting patches through attacks on the YOLOv8 object detection model and discuss remaining challenges and opportunities for improving robustness and scalability.

Singhvi, Vivaan [ORNL] (ORCID:0009000586288221)↗

The BayREN Integrated Commercial Retrofits (BRICR) Project: An Introduction and Preliminary Results

BayREN Integrated Commercial Retrofits (BRICR) is a DOE-funded project which aims to enhance the capacity of energy efficiency programs to recruit participants, develop retrofits, and measure outcomes in small and medium-sized commercial buildings, a sector notoriously hard to reach and expensive to serve, that accounts for 2/3 of US commercial floor space. To address these barriers, BRICR leverages existing incentives, financing, data, and open source software to facilitate two paths for comprehensive improvements: a deep energy retrofit, or serial upgrades integrated into capital improvement and maintenance cycles. BRICR is developing an integrated workflow for iterative energy modeling of commercial buildings for city energy program managers and auditors - starting with mass building-scale simulation based on public records and proceeding through audit, retrofit, and measurement and verification stages. BRICR builds on existing tools including LBNL’s CBES, NREL’s OpenStudio, PNNL’s Audit Template, and DOE’s BuildingSync and SEED PlatformTM. At each stage, BRICR uses available information to inform simulations (starting with public records but iteratively augmented with observations from energy program staff) to improve the quality of the models that inform decision making. This paper presents initial results from energy models of 1699 office and retail buildings in San Francisco. Building stock data from public records were translated to create a BuildingSync file for each building which was stored in SEED. Each BuildingSync file was then translated to multiple OpenStudio Workflow files for EnergyPlus simulation to estimate energy savings of energy conservation measures (ECM). Energy savings predictions for each ECM were written back to an updated BuildingSync file for each building and re-uploaded to SEED. The distribution of baseline energy simulations was calibrated against the publicly disclosed distribution of energy benchmarking data to increase confidence in results.

Piette, M↗

Hourly Electricity Demand Projections for Eight Combined Climate and Socioeconomic Scenarios

This dataset contains 40 years (1980-2019) of simulated historical hourly electricity demand (i.e., loads) and 80 years (2020-2099) of projected hourly loads for 54 Balancing Authorities (BAs) and 48 states plus the District of Columbia. Details about the scenarios and variables included in this dataset are in the readme.pdf file. The two primary models that created the dataset are a version of the Global Change Analysis Model with detailed sectoral resolution over the United States (GCAM-USA) and the Total ELectricity Loads (TELL) model. Links to the model source code and workflow for deriving the dataset are provided in an accompanying meta-repository: https://github.com/IMMM-SFA/burleyson-etal_2023_applied_energy. Projections are for four future climate scenarios that represent combinations of Representative Concentration Pathways (RCPs) 4.5 and 8.5 combined with two levels of climate model sensitivities: rcp45cooler, rcp45hotter, rcp85cooler, and rcp85hotter. The four climate scenarios are crossed with Shared Socioeconomic Pathways (SSPs) 3 and 5 to yield eight different future load projections: rcp45cooler_ssp3, rcp45cooler_ssp5, rcp45hotter_ssp3, rcp45hotter_ssp5, rcp85cooler_ssp3, rcp85cooler_ssp5, rcp85hotter_ssp3, and rcp85hotter_ssp5. The climate scenarios are from the IM3 Thermodynamic Global Warming (TGW) dataset which is linked below in the related metadata. The related metadata also contains links to a repository containing the raw GCAM-USA output files.

Climate Change↗

Hourly Electricity Demand Projections for Eight Combined Climate and Socioeconomic Scenarios

This dataset contains 40 years (1980-2019) of simulated historical hourly electricity demand (i.e., loads) and 80 years (2020-2099) of projected hourly loads for 54 Balancing Authorities (BAs) and 48 states plus the District of Columbia. Details about the scenarios and variables included in this dataset are in the readme.pdf file. The two primary models that created the dataset are a version of the Global Change Analysis Model with detailed sectoral resolution over the United States (GCAM-USA) and the Total ELectricity Loads (TELL) model. Links to the model source code and workflow for deriving the dataset are provided in an accompanying meta-repository: https://github.com/IMMM-SFA/burleyson-etal_2024_applied_energy. Projections are for four future climate scenarios that represent combinations of Representative Concentration Pathways (RCPs) 4.5 and 8.5 combined with two levels of climate model sensitivities: rcp45cooler, rcp45hotter, rcp85cooler, and rcp85hotter. The four climate scenarios are crossed with Shared Socioeconomic Pathways (SSPs) 3 and 5 to yield eight different future load projections: rcp45cooler_ssp3, rcp45cooler_ssp5, rcp45hotter_ssp3, rcp45hotter_ssp5, rcp85cooler_ssp3, rcp85cooler_ssp5, rcp85hotter_ssp3, and rcp85hotter_ssp5. The climate scenarios are from the IM3 Thermodynamic Global Warming (TGW) dataset which is linked below in the related metadata. The related metadata also contains links to a repository containing the raw GCAM-USA output files.

Climate Change↗

Comparison of Resource Platform Selection Approaches for Scientific Workflows

Cloud computing is increasingly considered as an additional computational resource platform for scientific workflows. The cloud offers opportunity to scale-out applications from desktops and local cluster resources. At the same time, it can eliminate the challenges of restricted software environments and queue delays in shared high performance computing environments. Choosing from these diverse resource platforms for a workflow execution poses a challenge for many scientists. Scientists are often faced with deciding resource platform selection trade-offs with limited information on the actual workflows. While many workflow planning methods have explored task scheduling onto different resources, these methods often require fine-scale characterization of the workflow that is onerous for a scientist. In this position paper, we describe our early exploratory work into using blackbox characteristics to do a cost-benefit analysis across of using cloud platforms. We use only very limited high-level information on the workflow length, width, and data sizes. The length and width are indicative of the workflow duration and parallelism. The data size characterizes the IO requirements. We compare the effectiveness of this approach to other resource selection models using two exemplar scientific workflows scheduled on desktops, local clusters, HPC centers, and clouds. Early results suggest that the blackbox model often makes the same resource selections as a more fine-grained whitebox model. We believe the simplicity of the blackbox model can help inform a scientist on the applicability of cloud computing resources even before porting an existing workflow.

97 MATHEMATICS AND COMPUTING↗

Beyond the Hype: Navigating the Promise and Pitfalls of Multi-Modal Models for Materials Science

Multi-modal models offer great potential for accelerating discovery in materials and chemical systems, but their adoption raises crucial questions: What materials science challenges are best addressed by multi-modal approaches? How do we weigh the benefits against the resource investment required for multi-modal data acquisition? And critically, how can we optimize experimental workflows to leverage these models effectively? In this presentation, I will delve into the development of multi-modal characterization and analytics, focusing on their application in the demanding fields of next-generation microelectronics and energy storage materials. I will share challenges encountered in designing these workflows, highlighting lessons learned and posing questions that remain unanswered.

AI↗

High-resolution hourly surrogate modeling framework for physics-based large-scale building stock modeling

Surrogate modeling can play a key role in reducing high computational burdens for large-scale physics-based modeling and uncertainty quantification. With the rapid development of large-scale building stock energy modeling, surrogate modeling has also begun to be widely applied in this field; however, most existing surrogate models lack hourly time resolution for regional-scale modeling, which is essential for understanding building demand profiles and grid impacts. Further, there is generally a lack of necessary data and feature engineering frameworks specific to building modeling for efficiently managing large datasets and complex computations. Furthermore, this paper proposes a modeling framework for large-scale (city-/region-scale), high-resolution, high-fidelity surrogate building stock energy models. Our developed framework consists of six modules: (1) building stock energy modeling (ComStock TM and ResStock TM ), (2) data engineering for large simulation data, (3) high performance computing workflow, (4) feature engineering, (5) machine learning model development, and (6) model performance evaluation. Two case studies apply the developed framework in both residential and commercial building stock analysis to demonstrate its computational efficiency and surrogate modeling accuracies. Results show that surrogate models, when efficiently trained using the HPC workflow module, reach a high level of modeling accuracy for two case studies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗