Search NASA⌕ Search

SEARCH · Search NASA

Results for “hybrid workflows”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Composable Programming of Hybrid Workflows for Quantum Simulation

We present a composable design scheme for the development of hybrid quantum/classical algorithms and workflows for applications of quantum simulation. Our object-oriented approach is based on constructing an expressive set of common data structures and methods that enable programming of a broad variety of complex hybrid quantum simulation applications. The abstract core of our scheme is distilled from the analysis of the current quantum simulation algorithms. Subsequently, it allows a synthesis of new hybrid algorithms and workflows via the extension, specialization, and dynamic customization of the abstract core classes defined by our design. We implement our design scheme using the hardware-agnostic programming language QCOR into the QuaSiMo library. To validate our implementation, we test and show its utility on commercial quantum processors from IBM, running some prototypical quantum simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

QuaSiMo: A composable library to program hybrid workflows for quantum simulation

Abstract A composable design scheme is presented for the development of hybrid quantum/classical algorithms and workflows for applications of quantum simulation. The proposed object‐oriented approach is based on constructing an expressive set of common data structures and methods that enables programming of a broad variety of complex hybrid quantum simulation applications. The abstract core of the scheme is distilled from the analysis of the current quantum simulation algorithms. Subsequently, it allows synthesis of new hybrid algorithms and workflows via the extension, specialisation, and dynamic customisation of the abstract core classes defined by the proposed design. The design scheme is implemented using the hardware‐agnostic programming language QCOR into the QuaSiMo library. To validate the implementation, the authors test and show its utility on commercial quantum processors from IBM and Rigetti, running some prototypical quantum simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Novel Proposals for FAIR, Automated, Recommendable, and Robust Workflows

Lightning talks of the Workflows in Support of Large-Scale Science (WORKS) workshop are a venue where the workflow community (researchers, developers, and users) can discuss work in progress, emerging technologies and frameworks, and training and education materials. This paper summarizes the WORKS 2022 lightning talks, which cover five broad topics: data integrity of scientific workflows; a machine learning-based recommendation system; a Python toolkit for running dynamic ensembles of simulations; a cross-platform, high-performance computing utility for processing shell commands; and a meta(data) framework for reproducing hybrid workflows.

Abhinit, Ishan↗
DOE OSTI2025

Bridging paradigms: Designing for HPC-Quantum convergence

Here, this paper presents a comprehensive software stack architecture for integrating quantum computing (QC) capabilities with High-Performance Computing (HPC) environments. While quantum computers show promise as specialized accelerators for scientific computing, their effective integration with classical HPC systems presents significant technical challenges. We propose a hardware-agnostic software framework that supports both current noisy intermediate-scale quantum devices and future fault-tolerant quantum computers, while maintaining compatibility with existing HPC workflows. The architecture includes a quantum gateway interface, standardized APIs for resource management, and robust scheduling mechanisms to handle both simultaneous and interleaved quantum–classical workloads. Key innovations include: (1) a unified resource management system that efficiently coordinates quantum and classical resources, (2) a flexible quantum programming interface that abstracts hardware-specific details, (3) A Quantum Platform Manager API that simplifies the integration of various quantum hardware systems, and (4) a comprehensive tool chain for quantum circuit optimization and execution. We demonstrate our architecture through implementation of quantum–classical algorithms, including the variational quantum linear solver, showcasing the framework’s ability to handle complex hybrid workflows while maximizing resource utilization. This work provides a foundational blueprint for integrating QC capabilities into existing HPC infrastructures, addressing critical challenges in resource management, job scheduling, and efficient data movement between classical and quantum resources.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Learning to Count Grave Sites for Cemetery Observation Models With Satellite Imagery

Understanding how people occupy open spaces is important for research in support of population modeling, policy, national security, emergency response, and sustainability. For the past decade, there has been an increase in research toward capturing and reporting population dynamics and patterns of life at the building level and in some open public spaces such as cemeteries and parks. This is done through observation models developed from local sociocultural information acquired at various spatiotemporal scales to inform night, day, and episodic population occupancy estimates (people/1000 sq ft). Sociocultural information for cemeteries and parks is scarcely available and often collected manually. The process is not only marred by inconsistencies but is laborious and time consuming. In this study, we leverage convolutional neural networks (CNNs) and satellite imagery to derive grave site counts as proxy variables to support scalable and accurate sociocultural data required in a population observation model. Through a hybrid workflow (weak localization plus regression model), we characterize a large scale automation process to counting of grave sites. We evaluate and demonstrate the efficacy of proposed workflow using out-of-data set large satellite imagery and establish its broader impact on cemetery observation models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Scalable Programming Workflows for Validation of Quantum Computers

Hybrid quantum-classical workflows have become standard methods for executing variational algorithms and other quantum simulation techniques, which are key applications for noisy intermediate scale quantum (NISQ) computers. Validating these simulations is an important task which helps gauge the progress of quantum computer development, and classical simulation can serve as a tool to this end. Both exact and more scalable approximate methods with quantifiable error bounds can be used in validation tasks where the applicable metrics include the distance from a calculable ground truth, the quality of an error model fit to data, etc. Here we present a library extension that includes methods for validation of quantum simulations based on scalable hybrid workflows executable on high performance computers. We provide examples that use approximate methods based on tensor networks and stabilizer simulators to bound the error of quantum simulations on NISQ hardware.

Nguyen, Thien↗
DOE OSTI2021

A Hybrid Energy System Workflow for Energy Portfolio Optimization

This manuscript develops a workflow, driven by data analytics algorithms, to support the optimization of the economic performance of an Integrated Energy System. The goal is to determine the optimum mix of capacities from a set of different energy producers (e.g., nuclear, gas, wind and solar). A stochastic-based optimizer is employed, based on Gaussian Process Modeling, which requires numerous samples for its training. Each sample represents a time series describing the demand, load, or other operational and economic profiles for various types of energy producers. These samples are synthetically generated using a reduced order modeling algorithm that reads a limited set of historical data, such as demand and load data from past years. Numerous data analysis methods are employed to construct the reduced order models, including, for example, the Auto Regressive Moving Average, Fourier series decomposition, and the peak detection algorithm. All these algorithms are designed to detrend the data and extract features that can be employed to generate synthetic time histories that preserve the statistical properties of the original limited historical data. The optimization cost function is based on an economic model that assesses the effective cost of energy based on two figures of merit: the specific cash flow stream for each energy producer and the total Net Present Value. An initial guess for the optimal capacities is obtained using the screening curve method. The results of the Gaussian Process model-based optimization are assessed using an exhaustive Monte Carlo search, with the results indicating reasonable optimization results. The workflow has been implemented inside the Idaho National Laboratory’s Risk Analysis and Virtual Environment (RAVEN) framework. The main contribution of this study addresses several challenges in the current optimization methods of the energy portfolios in IES: First, the feasibility of generating the synthetic time series of the periodic peak data; Second, the computational burden of the conventional stochastic optimization of the energy portfolio, associated with the need for repeated executions of system models; Third, the inadequacies of previous studies in terms of the comparisons of the impact of the economic parameters. The proposed workflow can provide a scientifically defendable strategy to support decision-making in the electricity market and to help energy distributors develop a better understanding of the performance of integrated energy systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Facilitating Staging-based Unstructured Mesh Processing to Support Hybrid In-Situ Workflows

In-situ and in-transit processing alleviate the gap between the computing and I/O capabilities by scheduling data analytics close to the data source. Hybrid in-situ processing splits data analytics into two stages: the data processing that runs in-situ aims to extract regions of interest, which are then transferred to staging services for further in-transit analytics. To facilitate this type of hybrid in-situ processing, the data staging service needs to support complex intermediate data representations generated/consumed by the in-situ tasks. Unstructured (or irregular) mesh is one such derived data representation that is typically used and bridges simulation data and analytics. However, how staging services efficiently support unstructured mesh transfer and processing remains to be explored. This paper investigates design options for transferring and processing unstructured mesh data using staging services. Using polygonal mesh data as an example, we show that hybrid in-situ workflows with staging-based unstructured mesh processing can effectively support hybrid in-situ workflows, and can significantly decrease data movement overheads.

data-driven↗
DOE OSTI2025

An open-source hybrid unstructured mesh - CAD fusion multiphysics analysis workflow in SALAMANDER

Plasma facing components in fusion devices will endure extreme neutron and heat fluxes. To facilitate their design using simulation tools, the open-source Fusion Module, Fusion ENergy Integrated multiphys-X (FENIX) framework is being developed to model these components with a high-fidelity multi-physics multi-dimensional approach. It can iteratively resolve couplings between all the physics at play, from neutron radiation, to thermomechanics, to near-wall plasma dynamics. This framework is based on the Multiphysics Object Oriented Simulation Environment (MOOSE), which is developed by a collaboration of US National Laboratories since 2008, for advanced nuclear, geomechanics simulations and other applications. FENIX couples numerous simulation tools, including OpenMC, the Tritium Migration Analysis Program v8, the NekRS CFD software, and most MOOSE modules. For the coupling of radiation transport and other physics, FENIX supports a hybrid workflow between Computer Assisted Design (CAD) and unstructured mesh geometries. The CAD can be generated from skinning the unstructured mesh, to enable a coarse geometry for efficient particle transport, but still resolving the local material compositions and temperature gradients. Neutron transport is performed using DAGMC on the CAD, and Cardinal, integrated in FENIX, maps tallied quantities, such as the heat deposition or the tritium generation rates, from a tally volumetric mesh to the other physics’ unstructured mesh. This coupling was exercised on a simplified tokamak geometry, coupling neutron transport with the heat conduction equation, and on a monoblock divertor problem, coupling additionally with tritium migration. Mesh convergence studies highlight the importance of the mapping conservativeness. Coupling with thermo-mechanics is further enabled by the generalization of the approach to moving meshes. The presentation will include these coupled analysis as well as an update on status of the FENIX framework.

70 - PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge.To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments.We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26—37x faster to capture and transmit provenance data; uses 5—7x less CPU; 2x less memory; transmits 2x less data; and consumes 2—2.5x less energy. ProvLight [1] and E2Clab [2] are available as open-source tools.

Rosendo, Daniel↗
DOE OSTI2022

Quantitative interpretation of time-lapse seismic data at Farnsworth field unit: Rock physics modeling, and calibration of simulated time-lapse velocity responses

Here, this study investigates the contribution of fluid saturation variation to the time-lapse velocity response by performing fluid substitution modeling. The methodology is exemplified by the time-lapse seismic monitoring of carbon dioxide at Farnsworth field unit (FWU). In order to evaluate the fluid distribution in a matured oil reservoir, the Southwest Regional Partnership (SWP) acquired multiple vertical seismic profile (VSP) surveys at different times during the CO 2 –water alternatinggas (WAG) injection period. In this work, we present a thorough methodology for computing the elastic response of the saturated rock for different fluid saturations using a site-specific petro-elastic model (PEM). The output from the PEM was combined with results from a fluid compositional model to compute the seismic velocities at times corresponding to each VSP survey. To produce a calibrated simulated response, the measured time-lapse seismic velocities were integrated into the numerical simulation model. The mismatches between the predicted and measured time-lapse velocities were minimized through an iterative calibration process using a trained artificial neural network proxy (ANN) coupled with a particle swarm optimizer (PSO). Our study indicates that the hybrid optimization workflow can effectively perform the history matching. With an accurate prediction of the hydrodynamic properties, the migration of CO 2 within the subsurface was modeled by predicting the spatial velocity distribution for a radius of 305 m around the injection well. The technology demonstrated and the expertise gained from this study can guide similar CO 2 -WAG projects.

58 GEOSCIENCES↗
DOE OSTI2022

MZA: A Data Conversion Tool to Facilitate Software Development and Artificial Intelligence Research in Multidimensional Mass Spectrometry

Modern mass spectrometry-based workflows employing hybrid instrumentation and orthogonal separations collect multidimensional data, potentially allowing deeper understanding in omics studies through adoption of artificial intelligence methods. However, the large volume of these rich data challenges existing data storage and access technologies, therefore precluding informatics advancements. Here we present MZA™ (pronounced m-za), the mass-to-charge (m/z) generic data storage and access tool designed to facilitate software development and artificial intelligence research in multidimensional mass spectrometry measurements. Composed by a data conversion tool and a simple file structure based on the HDF5 format, MZA provides easy, cross-platform and efficient programmatic access to raw MS-data, enabling fast development of new tools in data science programming languages such as Python and R. The software executable and example Python and R scripts are freely available at https://github.com/PNNL-m-q/mza.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Automated workflow for non-empirical Wannier-localized optimal tuning of range-separated hybrid functionals

Here, we introduce an automated workflow for generating non-empirical Wannier-localized optimally-tuned screened range-separated hybrid (WOT-SRSH) functionals. WOT-SRSH functionals have been shown to yield highly accurate fundamental band gaps, band structures, and optical spectra for bulk and 2D semiconductors and insulators. Our workflow automatically and efficiently determines the WOT-SRSH functional parameters for a given crystal structure and composition, approximately enforcing the correct screened long-range Coulomb interaction and an ionization potential ansatz. In contrast to previous manual tuning approaches, our tuning procedure relies on a new search algorithm that only requires a few hybrid functional calculations with minimal user input. We demonstrate our workflow on 23 previously studied semiconductors and insulators, reporting the same high level of accuracy. By automating the tuning process and improving its computational efficiency, the approach outlined here enables applications of the WOT-SRSH functional to compute spectroscopic and optoelectronic properties for a wide range of materials.

Gant, Stephen E. [University of California, Berkel↗
NASA NTRS2021

PIXLISE-C: Exploring The Data Analysis Needs of NASA Scientists for Mineral Identification

NASA JPL scientists working on the micro x-ray fluorescence (microXRF) spectroscopy data collected from Mars surface perform data analysis to look for signs of past microbial life on Mars. Their data analysis workflow mainly involves identifying mineral com- pounds through the element abundance in spatially distributed data points. Working with the NASA JPL team, we identified pain points and needs to further develop their existing data visualization and analysis tool. Specifically, the team desired improvements for the process of creating and interpreting mineral composition groups. To address this problem, we developed an interactive tool that enables scientists to (1) cluster the data using either manual lasso-tool selection or through various machine learning clustering algorithms, and (2) compare the clusters and individual data points to make informed decisions about mineral compositions. Our preliminary tool supports a hybrid data analysis workflow where the user can manually refine the machine-generated clusters.

Davidoff, Scott↗
DOE OSTI2021

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗
DOE OSTI2021

Practical CO2—WAG Field Operational Designs Using Hybrid Numerical-Machine-Learning Approaches

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Computational multiphysics modeling of radioactive aerosol deposition in diverse human respiratory tract geometries

The evaluation of aerosol exposure relies on generic mathematical models that assume uniform particle deposition profiles over the human respiratory tract and do not account for subject-specific characteristics. Here we introduce a hybrid-automated computational workflow that generates personalized particle deposition profiles in 3D reconstructed human airways from computed tomography scans using Computational Fluid and Particle Dynamics simulations. This is the first large-scale study to consider realistic airways variability, where 380 lower and 40 upper human respiratory tract 3D geometries are reconstructed and parameterized. The data is clustered into nine groups using random forest regression. Computational fluid and particle dynamics simulations are conducted on these representative geometries using a realistic heavy-breathing respiratory cycle and radioactive iodine-131 as a source term. Monte Carlo radiation transport simulations are performed to obtain detailed energy deposition maps. Our findings emphasize the importance of personalized studies, as minor respiratory tract variations notably influence deposition patterns rather than global parameters of the lower airways, observing more than 30% variance in the mass deposition fraction.

62 RADIOLOGY AND NUCLEAR MEDICINE↗