Search NASA⌕ Search

SEARCH · Search NASA

Results for “adaptive workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

In-memory integration of existing software components for parallel adaptive unstructured mesh workflows

Unstructured mesh methods, like finite elements and finite volumes, support the effective analysis of complex physical behaviors modeled by partial differential equations over general threedimensional domains. The most reliable and efficient methods apply adaptive procedures with a-posteriori error estimators that indicate where and how the mesh is to be modified. Although adaptive meshes can have two to three orders of magnitude fewer elements than a more uniform mesh for the same level of accuracy, there are many complex simulations where the meshes required are so large that they can only be solved on massively parallel systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

The Evolution of the Pegasus Workflow Management Software

Since 2001 the Pegasus Workflow Management System has evolved into a robust and scalable system that automates the execution of a number of complex applications running on a variety of heterogeneous, distributed high-throughput, and high-performance computing environments. Pegasus was built on the principle of separation between the workflow description and workflow execution, providing the ability to port and adapt the workflow based on the target execution environment. Through its user-driven research and development it has adapted to the needs of a number of scientific communities, utilizing and developing novel algorithms and software engineering solutions. This paper describes the evolution of Pegasus over time and provides motivations behind the design decisions. The paper concludes with selected lessons learned.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Quantum simulations of hadron dynamics in the Schwinger model using 112 qubits

Hadron wave packets are prepared and time evolved in the Schwinger model using 112 qubits of IBM’s 133-qubit Heron quantum computer ibm_torino. The initialization of the hadron wave packet is performed in two steps. First, the vacuum is prepared across the whole lattice using the recently developed SC-ADAPT-VQE algorithm and workflow. SC-ADAPT-VQE is then extended to the preparation of localized states, and used to establish a hadron wave packet on top of the vacuum. This is done by adaptively constructing low-depth circuits that maximize the overlap with an adiabatically prepared hadron wave packet. Due to the localized nature of the wavepacket, these circuits can be determined on a sequence of small lattices using classical computers, and then robustly scaled to prepare wave packets on large lattices for simulations using quantum computers. Time evolution is implemented with a second-order Trotterization. To reduce both the required qubit connectivity and circuit depth, an approximate quasilocal interaction is introduced. This approximation is made possible by the emergence of confinement at long distances, and converges exponentially with increasing distance of the interactions. Using multiple error-mitigation strategies, up to 14 Trotter steps of time evolution are performed, employing 13,858 two-qubit gates (with a CNOT depth of 370). The propagation of hadrons is clearly identified, with results that compare favorably with Matrix Product State simulations. Finally, prospects for a near-term quantum advantage in simulations of hadron scattering are discussed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

WorkflowHub: a registry for computational workflows

The rising popularity of computational workflows is driven by the need for repetitive and scalable data processing, sharing of processing know-how, and transparent methods. As both combined records of analysis and descriptions of processing steps, workflows should be reproducible, reusable, adaptable, and available. Workflow sharing presents opportunities to reduce unnecessary reinvention, promote reuse, increase access to best practice analyses for non-experts, and increase productivity. In reality, workflows are scattered and difficult to find, in part due to the diversity of available workflow engines and ecosystems, and because workflow sharing is not yet part of research practice. WorkflowHub provides a unified registry for all computational workflows that links to community repositories, and supports both the workflow lifecycle and making workflows findable, accessible, interoperable, and reusable (FAIR). By interoperating with diverse platforms, services, and external registries, WorkflowHub adds value by supporting workflow sharing, explicitly assigning credit, enhancing FAIRness, and promoting workflows as scholarly artefacts. The registry has a global reach, with hundreds of research organisations involved, and more than 800 workflows registered.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Employing artificial intelligence to steer exascale workflows with colmena

Computational workflows are a common class of application on supercomputers, yet the loosely coupled and heterogeneous nature of workflows often fails to take full advantage of their capabilities. We created Colmena to leverage the massive parallelism of a supercomputer by using Artificial Intelligence (AI) to learn from and adapt a workflow as it executes. Colmena allows scientists to define how their application should respond to events (e.g., task completion) as a series of cooperative agents. In this paper, we describe the design of Colmena, the challenges we overcame while deploying applications on exascale systems, and the science workflows we have enhanced through interweaving AI. The scaling challenges we discuss include developing steering strategies that maximize node utilization, introducing data fabrics that reduce communication overhead of data-intensive tasks, and implementing workflow tasks that cache costly operations between invocations. These innovations coupled with a variety of application patterns accessible through our agent-based steering model have enabled science advances in chemistry, biophysics, and materials science using different types of AI. In conclusion, our vision is that Colmena will spur creative solutions that harness AI across many domains of scientific computing.

Workflows↗
DOE OSTI2022

CSEM Fluid Monitoring Methodology Using Real Data Examples

Conference presentation at International Meeting for Applied Geoscience & Energy (IMAGE), Houston, Texas, August 28 – September 1, 2023. Using field data from hydrocarbon and CO 2 applications, we illustrate the importance of a workflow and adaption to the target on hand. Verifying the geophysical acquisition and processing steps with 3D modeling and checking them against a 3D anisotropic log-derived model maintains confidence in the workflow and minimizes the influence on the data. This allows us to predict data validity and to certify the data with respect to the borehole logs.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2023

Normality of I-V Measurements Using ML

There is an increased interest in instrument-computing ecosystems (ICEs) that support science workflows empowered by AI-automated experiments and computations in diverse areas. In particular, electrochemistry ICEs are promising for accelerating the design and discovery of electrochemical systems for energy storage and conversion, by automating significant parts of workflows that combine synthesis and characterization experiments with computations. They require the integration of flow controllers, solvent containers, pumps, fraction collectors, and potentiostats, all connected to an electrochemical cell, as illustrated in Fig. 1. These are specialized instruments with custom software that is not originally designed for network integration. We developed network and software solutions for electrochemical workflows that adapt system and instrument settings in real-time for multiple rounds of experiments. In particular, we developed Python wrappers for Application Programming Interfaces (APIs) of instrument commands and Pyro client-server modules that enable them to be executed from remote computers. The entire workflow is orchestrated by a Jupyter notebook running on a remote computer.

Al Najjar, Anees↗
DOE OSTI2021

Resin Assisted Capture Coupled with Isobaric Tandem Mass Tag Labeling for Multiplexed Quantification of Protein Thiol Oxidation

Reversible oxidative modifications on protein thiols have recently emerged as important mediators of cellular function. Herein we describe the detailed procedure of a quantitative redox proteomics method that utilizes resin-assisted capture (RAC) in combination with tandem mass tag (TMT) isobaric labeling and liquid chromatography-tandem mass spectrometry (LC-MS/MS) to allow multiplexed stochiometric quantification of oxidized protein thiols at the proteome level. The site-specific quantitative information on oxidized cysteine residues provides additional insight into the functional impacts of such modifications. The workflow is adaptable across many sample types including cultured cells (e.g. mammalian, prokaryotic) and whole tissues (e.g. heart, lung, muscle) which are initially lysed/homogenized with free thiols being alkylated to prevent artificial oxidation. The oxidized protein thiols are then reduced and captured by a thiol affinity resin which streamlines and simplifies the workflow steps by allowing the proceeding digestion, labeling, and washing procedures to be performed without additional transfer of proteins/peptides. Finally, labeled peptides are eluted and analyzed by LC-MS/MS to reveal comprehensive stoichiometric changes related to thiol oxidation across the entire proteome. This method greatly improves our understanding of the role of redox dependent regulation under physiological and pathophysiological states related to protein thiol oxidation.

RAC, TMT, thiol redox proteomics, cysteine, PTM st↗
DOE OSTI2024

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Image processing workflow yielding high contrast synchrotron nanoscale computed tomography data from Ni-YSZ electrodes

The operating lifetime of Ni-YSZ fuel electrodes used in solid oxide electrolysis cells and fuel cells (SOECs and SOFCs) is limited by Ni redistribution, one of the primary degradation mechanisms that must be overcome to extend the longevity and maximize the performance of SOECs and SOFCs. To achieve this, 3D microstructural data is needed to relate both initial performance and performance loss over time to microstructural properties and their evolution throughout operation under various conditions. However, 3D microstructure data remains relatively scarce within the literature due to multiple challenges in acquiring and analyzing such data reliably. This work presents a workflow for acquiring and processing synchrotron X-ray nanoscale computed tomography (nano-CT) data from Ni-YSZ electrodes. Parameters for each step in the nano-CT workflow are described up to the final result (a 3D reconstruction), with particular emphasis on image alignment using freely available software. Following the results of a parametric sweep of the image alignment step, high contrast, low signal-to-noise 3D nano-CT data is obtained with relatively short compute times. While the exact methods best suited to samples with different microstructural qualities, or similar Ni-YSZ nano-CT data obtained from other sources may deviate from the solution found herein, this work also generalizes the decision points and evaluation of each step to provide a starting point to adapt this workflow to other datasets.

08 HYDROGEN↗
DOE OSTI2026

Constructing a High‐Resolution Aftershock Catalog for the 2017 Mw 8.2 Tehuantepec Earthquake Sequence Using a Machine Learning–Based Workflow

The 8 September 2017 Mw 8.2 Tehuantepec earthquake was the largest instrumentally recorded normal‐faulting earthquake in Mexico. The mainshock occurred offshore within the Tehuantepec seismic gap, generating >30,000 aftershocks in the following year. We applied an open‐source, machine learning (ML)–assisted workflow to construct a high‐resolution aftershock catalog using data from temporary and permanent seismic networks in southern Mexico. The workflow integrates PhaseNet for phase detection; GaMMA for phase association; and VELEST, HypoInverse, and HypoDD for velocity modeling and relocation. We processed seven months of continuous waveform data from 29 broadband stations, including a temporary rapid‐response deployment that improved station coverage of the offshore rupture zone. To evaluate performance, we compared our results against analyst‐reviewed picks and event locations from the Servicio Sismológico Nacional catalog. The resulting catalog contains 11,374 relocated earthquakes and represents the most comprehensive published dataset for this sequence, incorporating the first full use of the temporary network. Relocated hypocenters show improved depth control and align well with the Slab2.0 subduction geometry, revealing clearer separation between offshore slab events and onshore crustal seismicity. This study demonstrates that combining ML‐based detection with established methods provides a scalable and reproducible approach for constructing high‐quality earthquake catalogs in tectonically complex environments and offers practical guidance for adapting similar workflows to other earthquake sequences.

Garcia, Marc [The University of Texas at El Paso, ↗
DOE OSTI2019

Learning-accelerated discovery of immune-tumour interactions

We present an integrated framework for enabling dynamic exploration of design spaces for cancer immunotherapies with detailed dynamical simulation models on high-performance computing resources. Our framework combines PhysiCell, an open source agent-based simulation platform for cancer and other multicellular systems, and EMEWS, an open source platform for extreme-scale model exploration. We build an agent-based model of immunosurveillance against heterogeneous tumours, which includes spatial dynamics of stochastic tumour–immune contact interactions. We implement active learning and genetic algorithms using high-performance computing workflows to adaptively sample the model parameter space and iteratively discover optimal cancer regression regions within biological and clinical constraints.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs

Error-bounded lossy compression is a critical technique for significantly reducing scientific data volumes. With ever-emerging heterogeneous high-performance computing (HPC) architecture, GPU-accelerated error-bounded compressors (such as CUSZ and cuZFP) have been developed. However, they suffer from either low performance or low compression ratios. To this end, we propose CUSZ+ to target both high compression ratios and throughputs. We identify that data sparsity and data smoothness are key factors for high compression throughputs. Our key contributions in this work are fourfold: (1) We propose an efficient compression workflow to adaptively perform run-length encoding and/or variable-length encoding. (2) We derive Lorenzo reconstruction in decompression as multidimensional partial-sum computation and propose a fine-grained Lorenzo reconstruction algorithm for GPU architectures. (3) We carefully optimize each of CUSZ kernels by leveraging state-of-the-art CUDA parallel primitives. (4) We evaluate CUSZ+ using seven real-world HPC application datasets on V100 and A100 GPUs. Experiments show CUSZ+ improves the compression throughputs and ratios by up to 18.4x and 5.3x, respectively, over CUSZ on the tested datasets.

Tian, Jiannan↗
DOE OSTI2025

Dial

A key step in almost all scientific endeavors is answering the question: Given this data I already collected, what new data do I expect will yield the most useful information toward my scientific objective? The area of (sequential) experimental design has long been investigating answers to this question, but in recent years techniques from the machine learning subfield of active learning are increasingly applied. Researchers need a simple software tool for active learning applied to experimental design that can easily integrate into their existing workflows. This computer code, Dial, provides a microservice in ORNL's INTERSECT ecosystem for active learning applied to experimental design. By being part of the INTERSECT ecosystem, Dial is simple to integrate into any INTERSECT-based workflow. Dial provides multiple backend options, where a backend is an implementation of a specific active learning method. Users can select the backend that performs best for their application. Developers can also add new backends as needed. At its core, Dial receives a set of pre-existing measurements and input parameter bounds and then recommends one or more new sets of parameters to measure. Dial also includes interfaces to other microservices in the INTERSECT ecosystem so that it can be incorporated into INTERSECT campaigns. Dial provides a simple, yet powerful interface to convert automated INTERSECT workflows into autonomous workflows that adapt based on the results that are obtained. A shared microservice for active learning prevents duplicated effort by each application team implementing its own adaptive design of experiments tool.

Drane, Lance [Oak Ridge National Laboratory (ORNL)↗
DOE OSTI2022

Adaptive elasticity policies for staging-based in situ visualization

In situ processing aims to alleviate the growing gap between computation and I/O capabilities by performing data processing close to the data source. In situ processing is widely used to process data generated by multiple data sources, including observation data from edge devices or scientific observational facilities and the simulation data generated by scientific computation on a high-performance computing (HPC) platform. For a scientific workflow that is run on an HPC platform and composed of a simulation program and an in situ data analytics or visualization (abbreviated as ana/vis) task, there is an implicit assumption that the computing resources assigned to the workflow keep static during the workflow execution. However, with the converging trend between the HPC and cloud computing platform, running the in situ ana/vis task in an elastic way is promising to decrease its overhead and improve its resource utilization rate. Resource elasticity represents the ability to change resource configurations such as the number of computing nodes/processes during workflow execution. An elastic job may dynamically adjust resource configurations; it may use a few resources at the beginning and more resources toward the end of the job when interesting data appear. However, it is hard to predict a priori how many computing nodes/processes need to be added/removed during the workflow execution to adapt to changing workflow needs. How to efficiently guide elasticity operations, such as growing or shrinking the number of processes used for in situ analysis during workflow execution, is an open-ended research question. In this article, we present adaptive elasticity policies that adopt workflow runtime information collected during workflow execution to predict how to trigger the addition/removal of processes in order to minimize in situ processing overhead. Taking in situ visualization tasks as an example, we integrate the presented elasticity policies into a staging-based elastic workflow and evaluate its efficiency in multiple elasticity scenarios. Compared with the situation without elasticity or with a static elasticity policy that uses a fixed number of processes for each rescaling operation, the adaptive elasticity policy can save overhead in finding a proper resource configuration and improve resource utilization efficiency. Furthermore, one experiment illustrates that the adaptive elasticity policy saves 41% of core-hours compared with the situation without the resource elasticity.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

Unstructured Grid Adaptation and Solver Technology for Turbulent Flows

Unstructured grid adaptation is a tool to control Computational Fluid Dynamics (CFD) discretization error. However, adaptive grid techniques have made limited impact on production analysis workflows where the control of discretization error is critical to obtaining reliable simulation results. Issues that prevent the use of adaptive grid methods are identified by applying unstructured grid adaptation methods to a series of benchmark cases. Once identified, these challenges to existing adaptive workflows can be addressed. Unstructured grid adaptation is evaluated for test cases described on the Turbulence Modeling Resource (TMR) web site, which documents uniform grid refinement of multiple schemes. The cases are turbulent flow over a Hemisphere Cylinder and an ONERA M6Wing. Adaptive grid force and moment trajectories are shown for three integrated grid adaptation processes with Mach interpolation control and output error based metrics. The integrated grid adaptation process with a finite element (FE) discretization produced results consistent with uniform grid refinement of fixed grids. The integrated grid adaptation processes with finite volume schemes were slower to converge to the reference solution than the FE method. Metric conformity is documented on grid/metric snapshots for five grid adaptation mechanics implementations. These tools produce anisotropic boundary conforming grids requested by the adaptation process.

Park, Michael A.↗
NASA NTRSDate not supplied

Kamodo’s Satellite Constellation Mission Planning Tool

Kamodo provides a functional model-agnostic interface to a growing collection of Heliophysics model outputs. The CCMC, in collaboration with the Geospace Dynamics Constellation Science Team, has recently developed Kamodo’s satellite constellation mission planning tool to perform reconstructions in any pair of dimensions, including time. The ‘reconstruction’ tool enables users to fly any 4-dimensional grid of satellites through a given model data set, reconstructing what the given constellation would observe during the mission. This capability facilitates determination of what satellite configuration is best for a given science question, even allowing comparison across multiple models. This tool, written in Python, is built upon Kamodo’s flythrough tool, which in turn depends on a growing network of model-specific interfaces. Since each model interface is designed with model-agnostic syntax, the flythrough tool and the satellite constellation mission planning tool also feature model-agnostic syntax. In this work, we will describe the basic analysis choices available in the tool and provide a variety of sample workflows. The tool is freely available at https://github.com/nasa/Kamodo for the public. We invite the community to use the reconstruction tool and adapt the provided workflows for their mission planning, and to contribute their own workflows to share with others.

software↗