Search NASA⌕ Search

SEARCH · Search NASA

Results for “adaptive workflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Experiences with user-centered design for the tigres workflow API

Scientific data volumes have been growing exponentially. This has resulted in the need for new tools that enable users to operate on and analyze data. Cyber infrastructure tools, including workflow tools, that have been developed in the last few years has often fallen short if user needs and suffered from lack of wider adoption. User-centered Design (UCD) process has been used as an effective approach to develop usable software with high adoption rates. However, UCD has largely been applied for user-interfaces and there has been limited work in applying UCD to application program interfaces and cyber infrastructure tools. We use an adapted version of UCD that we refer to as Scientist-Centered Design (SCD) to engage with users in the design and development of Tigres, a workflow application programming interface. Tigres provides a simple set of programming templates (e.g., sequence, parallel, split, merge) that can be can used to compose and execute computational and data transformation pipelines. In this paper, we describe Tigres and discuss our experiences with the use of UCD for the initial development of Tigres. Our experience-to-date is that the UCD process not only resulted in better requirements gathering but also heavily influenced the architecture design and implementation details. User engagement during the development of tools such as Tigres is critical to ensure usability and increase adoption.

Ramakrishnan, L↗

Predictive Modeling and Diagnostic Monitoring of Extreme Science Workflows (Final Report)

This proposal addresses a critical issue of performance prediction identified in the report from the ASCR \Computational Modeling of Big Networks (COMBINE)" workshop: "end-to-end performance is not predictable due to a variety of factors. Even when some performance forecasts or predictions can be made, they often cannot explain the reasons why some predictions fail." We will develop new analytical models to predict the end-to-end performance of scientific workflows on DOE computing infrastructures, and use simulations and experimentation to validate and refine these models, as well as to pinpoint the sources of model inaccuracy. We will also use these models to help diagnose application and infrastructure problems, and to adapt the system based on this diagnosis. This section provides background in the areas relevant to the proposed work. RPI’s specific tasks within the Panorama project are as follows: (1) Develop Aspen-Simulation interface for Workflow Model Driven Simulation. (2) Validate manual performance models of two target workflow scenarios with empirical measurement and simulation. (3) Extend ROSS-Aspen API to simulate workflow descriptions when required. (4) Validate Aspen performance models of two target workflow scenarios with automatic performance model empirical measurement and simulation. (5) Design and implement final system to automatically generate Aspen performance models from workflow descriptions (including methods to compensate for limitations of Aspen analytical models). (6) Validate improved Aspen performance models with target workflow on production infrastructure. To date, all the project milestones assigned to us where reached within the best of our abilities over the course of the project performance period. Below describes the key outcome from our collaborative research in a system named, Durango .

97 MATHEMATICS AND COMPUTING↗

Three-dimensional estimation of deciduous forest canopy structure and leaf area using multi-directional, leaf-on and leaf-off airborne lidar data

Airborne laser scanning (ALS) has been widely used to map gap probability and leaf area index (LAI) distribution at plot and landscape scales. As an indirect measurement, most ALS methods to estimate LAI combine waveform or point density information with supporting field measurements such as the leaf angle distribution, gap probability, or direct LAI measures. The development of a more independent estimation approach would facilitate more widespread use of existing ALS data to investigate patterns of forest structure and build realistic 3-D vegetation scenes to simulate remote sensing imagery and energy balance. Here, we develop a data processing workflow (named PVlad) using ALS point cloud apparent reflectance to estimate LAI and voxel-based leaf area density (LAD), aiming to reduce the need for associated field measurements such as the gap probability. The adaptation of the path volume (PV) concept derived from apparent reflectance integrates information from multi-directional ALS pulses, and quantifies the percentage exploration of each voxel for classification and occlusion correction, such that rigorous volumetric sampling approaches can be developed to derive LAI and LAD. The PVlad workflow was applied to discrete-return lidar data (Riegl VQ480i) acquired by NASA Goddard's LiDAR, Hyperspectral and Thermal Imager (G-LiHT) Airborne Imager during leaf-on (summer) and leaf-off (spring) conditions at the Smithsonian Environmental Research Center (SERC). The estimates of LAI and LAD captured structural differences between mature, logged, and intermediate-aged stands over eight deciduous forest plots. The derived LAI values were compared to field litter collection measurements, and the derived LAD vertical distribution was compared to the output of the VoxLAD model using terrestrial laser scan (TLS) field survey data. Using voxel sizes ranging from 0.5 m to 5 m, overall LAI estimation showed linear fitting coefficient bias and for 1 and 2 m voxel sizes, and vertical LAD distribution showed strong correlation with and for 0.5 and 1m voxel sizes. For every forest stand, upper-canopy LAD had a low variance for voxel sizes of ≤ . Application of PVlad to the G-LiHT and other similar ALS data archives enables the development of fine-resolution LAI map products, including voxelization of LAD for ecosystem science and radiative transfer simulations of remote sensing imagery or surface energy balance.

Tiangang Yin↗

Performance Characterization and Provenance of Distributed Task-based Workflows on HPC Platforms

Understanding performance and provenance of task-based workflows poses significant challenges, particularly in distributed configurations where resources are shared by multiple applications. Task-based workflow management systems further complicate performance predictability because of their dynamicity that subtly alters task execution order from run to run. In this paper we propose a layered characterization framework for performance and task provenance for Dask.distributed workflows running on high-performance computing (HPC) platforms. It collects data from jobs, the workflow management system, and the operating system to aid in understanding the performance of these workflows. Our approach encompasses three main contributions: first, an extension of Dask.distributed to capture high-fidelity task provenance using Mochi data services; second, the adaptation of the established HPC I/O characterization tool Darshan to gather high-fidelity I/O data, thereby enhancing the granularity of our analysis; and third, a framework to combine and process the collected data and provide helpful insights into performance characterization and reproducibility, alongside our lessons learned.

Dask↗

Accelerating Simulation for High-Fidelity PV Inverter System Reliability Assessment with High-Performance Computing

The overall cost of photovoltaic (PV) systems has shown a downward trend during the last decade; however, PV inverter failures account for the highest cost of operation and maintenance. To address this, reliability tools with powerful computation and better accuracy are required for the lifetime prediction and degradation evaluation of PV inverters. This paper proposes an event-driven parallel computing-based simulator. The proposed simulator applies high-performance computing techniques and other accessory optimization techniques-including cluster merging, adaptive model updates, and steady-state identification-to make reliability assessments for PV inverters under given input mission profiles and operating conditions with high efficiency and high fidelity. The main idea of the simulator and its workflow are introduced. Then, a demo PV inverter system simulator is implemented, and the speedup of the total simulations of the switching model reaches 123.03 times.

high-performance computing↗

Hybrid learning techniques for scientific data reduction with performance guarantees

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

Scientific Data Compression for Large Scale Computational Fluid Dynamics (CFD) Simulations

This Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and General Electric (GE) investigated methods for reducing the size of large computational fluid dynamics (CFD) simulation datasets using scientific data compression techniques. The work focused on adapting the MultiGrid Adaptive Reduction of Data (MGARD) compression framework and integrating it with high-performance I/O and visualization tools used in CFD workflows. MGARD uses hierarchical multilevel decomposition to enable error-controlled compression of floating-point scientific data while preserving quantities of interest. During the project, MGARD compression was integrated with the ADIOS I/O framework and visualization tools such as ParaView to enable efficient storage, transfer, and analysis of simulation data. The collaboration also explored approaches for improving compression performance for CFD data defined on unstructured meshes. Results demonstrate that scientific data compression can significantly reduce storage requirements and improve data management for large-scale CFD simulations.

97 MATHEMATICS AND COMPUTING↗

Myriad World Baseline: Global Geodemographic Estimates

The LandScan Myriad World Baseline (MWB) method produces global, residential (nighttime/home-location) gridded geodemographic estimates based on 5-year age/gender cohorts—at 30-arcsecond (≈1 km) resolution. MWB is designed to fill gaps where detailed, georeferenced survey data (e.g., Demographic and Health Surveys (DHS)) are missing or outdated, and to provide a baseline that can support human security analysis, including consequence assessment, “patterns of life” modeling, and scenario-based population futures. MWB’s workflow spatializes household-level age/gender characteristics from the GLOPOP-S dataset by conflating household and gridded expected relative wealth adapted from Global Gridded Relative Deprivation Index (GRDI), then adjusts them to a target year of interest. Age/gender estimates are then applied to harmonize lowest-administrative-level statistics with LandScan residential counts, yielding final geodemographic estimates. Two validation case studies are presented: Ghana (2021) and Tokyo/Kanagawa, Japan (2020), illustrating spatial variability in demographic cohorts and comparing MWB outputs to official gridded statistics. Results show close overall alignment relative to validation criteria including population pyramids and age-dependency ratios.

Tuccillo, Joe [ORNL] (ORCID:0000000259300943)↗

Loss Landscape Analysis for Reliable Quantized ML Models for Scientific Sensing

In this paper, we propose a method to perform empirical analysis of the loss landscape of machine learning (ML) models. The method is applied to two ML models for scientific sensing, which necessitates quantization to be deployed and are subject to noise and perturbations due to experimental conditions. Our method allows assessing the robustness of ML models to such effects as a function of quantization precision and under different regularization techniques -- two crucial concerns that remained underexplored so far. By investigating the interplay between performance, efficiency, and robustness by means of loss landscape analysis, we both established a strong correlation between gently-shaped landscapes and robustness to input and weight perturbations and observed other intriguing and non-obvious phenomena. Our method allows a systematic exploration of such trade-offs a priori, i.e., without training and testing multiple models, leading to more efficient development workflows. This work also highlights the importance of incorporating robustness into the Pareto optimization of ML models, enabling more reliable and adaptive scientific sensing systems.

Baldi, Tommaso [Pisa, Scuola Normale Superiore]↗

Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

The current focus in Autonomous Experimentation (AE) is on developing robust workflows to conduct the AE effectively. This entails the need for well-defined approaches to guide the AE process, including strategies for hyperparameter tuning and high-level human interventions within the workflow loop. This paper presents a comprehensive analysis of the influence of initial experimental conditions and in-loop interventions on the learning dynamics of Deep Kernel Learning (DKL) within the realm of AE in scanning probe microscopy. We explore the concept of the “seed effect,” where the initial experiment setup has a substantial impact on the subsequent learning trajectory. Additionally, we introduce an approach of the seed point interventions in AE allowing the operator to influence the exploration process. Using a dataset from Piezoresponse Force Microscopy on PbTiO 3 thin films, we illustrate the impact of the “seed effect” and in-loop seed interventions on the effectiveness of DKL in predicting material properties. The study highlights the importance of initial choices and adaptive interventions in optimizing learning rates and enhancing the efficiency of automated material characterization. This work offers valuable insights into designing more robust and effective AE workflows in microscopy with potential applications across various characterization techniques.

47 OTHER INSTRUMENTATION↗

Integrative Multi-PTM Proteomics Reveals Dynamic Global, Redox, Phosphorylation, and Acetylation Regulation in Cytokine-treated Pancreatic Beta Cells

Studying regulation of protein function at a systems level necessitates an understanding of the interplay among diverse post-translational modifications (PTMs). A variety of proteomics sample processing workflows are currently used to study specific PTMs but rarely characterize multiple types of PTMs from the same sample inputs. Method incompatibilities and laborious sample preparation steps complicate large-scale physiological investigations and can lead to variations in results. The single-pot, solid-phase-enhanced sample preparation (SP3) method for sample cleanup is compatible with different lysis buffers and amenable to automation, making it attractive for high-throughput multi-PTM profiling. Herein, we describe an integrative SP3 workflow for multiplexed quantification of protein abundance, cysteine thiol oxidation, phosphorylation, and acetylation. The broad applicability of this approach is demonstrated using cell and tissue samples, and its utility for studying interacting regulatory networks is highlighted in a time-course experiment of cytokine-treated ß-cells. We observed a swift response in global regulation of protein abundances consistent with rapid activation of JAK-STAT and NF-?B signaling pathways. Regulators of these pathways as well as proteins involved in their target processes displayed multi-PTM dynamics indicative of a complex cellular response stages: acute, adaptation, and chronic (prolonged stress). PARP14, a negative regulator of JAK-STAT, had multiple co-localized PTMs that may be involved in intraprotein regulatory crosstalk. Our workflow provides a high-throughput platform that can profile multi-PTMomes from the same sample set, which is valuable in unraveling the functional roles of PTMs and their co-regulation.

proteomics, PTM, automation, SP3, cysteine thiol o↗

Exploring electron-beam induced modifications of materials with machine-learning assisted high temporal resolution electron microscopy

Directed atomic fabrication using an aberration-corrected scanning transmission electron microscope (STEM) opens new pathways for atomic engineering of functional materials. In this approach, the electron beam is used to actively alter the atomic structure through electron beam induced irradiation processes. One of the impediments that has limited widespread use thus far has been the ability to understand the fundamental mechanisms of atomic transformation pathways at high spatiotemporal resolution. Here, we develop a workflow for obtaining and analyzing high-speed spiral scan STEM data, up to 100 fps, to track the atomic fabrication process during nanopore milling in monolayer MoS 2 . An automated feedback-controlled electron beam positioning system combined with deep convolution neural network (DCNN) was used to decipher fast but low signal-to-noise datasets and classify time-resolved atom positions and nature of their evolving atomic defect configurations. Through this automated decoding, the initial atomic disordering and reordering processes leading to nanopore formation was able to be studied across various timescales. Using these experimental workflows a greater degree of speed and information can be extracted from small datasets without compromising spatial resolution. This approach can be adapted to other 2D materials systems to gain further insights into the defect formation necessary to inform future automated fabrication techniques utilizing the STEM electron beam.

36 MATERIALS SCIENCE↗

Cross-Validation of Computational and Experimental Distributed Surface Pressures on the Space Launch System

This paper presents a new workflow for comparing experimental pressure-sensitive paint (PSP) data to computational fluid dynamic (CFD) simulations by way of mapping data from corresponding grids utilizing interpolation methods. In addition to generating quantitative and qualitative point-to-point comparisons between PSP and CFD data, this workflow extracts sectional loading data from both grids and generates lineload comparison charts for corresponding PSP and CFD runs. Experimental PSP data presented in this paper were taken from a 2016 NASA Ames Research Center Unitary Plan Wind Tunnel 11- by 11-Foot Transonic WindTunnel Facility test of the NASA Space Launch System. CFD simulation data for comparison purposes were generated using the FUN3D code. Overall, interpolation onto PSP grids versus CFD grids yields comparable surface pressure fields. However, lineload comparisons are easier to make on the CFD grid-mapped data due to the grid topology and the current capabilities of the lineload analysis tools at NASA Langley Research Center. This workflow is written using contemporary software (Python, Tecplot, PyTecplot), is compatible with existing tools at NASA Langley, and is developed to be adaptable depending on the situation.

SLS↗

DeepLynx Ecosystem 2025

Poor data integration and governance continue to plague complex engineering projects, resulting in missed cost, schedule, and performance targets. Departments operate in isolated systems with manual data exchange, creating fragmented information that compounds errors and leads to significant delays and cost overruns. The DeepLynx ecosystem addresses these challenges through an open-source, modular data management platform that transforms fragmented project data into an integrated digital thread. Built on a federated microservice architecture, the ecosystem comprises seven specialized tools centered around DeepLynx Nexus, a unified data catalog with hierarchical organization and graph-based navigation capabilities. The ecosystem includes: DeepLynx Stream for real-time timeseries data ingestion from industrial sources; DeepLynx Ingest for governed data uploads with formal review workflows; DeepLynx Lattice for ontology-based entity and relationship extraction; DeepLynx Run for workflow orchestration and secure AI/ML compute; DeepLynx Visualize for 3D digital twin visualization; and DeepLynx Insight for AI-assisted document analysis with traceable, grounded responses. Deployable in cloud, on-premise, or hybrid environments using containerized Docker applications and Helm charts, the DeepLynx ecosystem provides flexible infrastructure that adapts to organizational requirements. By consolidating project data into a unified data lake with role-based access controls and OAuth2 authentication, DeepLynx enables digital thread and digital twin capabilities that improve decision-making, reduce risk, and support complex engineering workflows throughout the project lifecycle.

42 - ENGINEERING↗

Quality assurance for online adapted treatment plans: Benchmarking and delivery monitoring simulation

Purpose: An important challenge facing online adaptive radiation therapy is the development of feasible and efficient quality assurance (QA). This project aimed to validate the deliverability of online adapted plans and develop a proof-of-concept online delivery monitoring system for online adaptive radiation therapy QA. Methods: The first part of this project benchmarked automatically online adapted prostate treatment plans using traditional portal dosimetry IMRT QA. The portal dosimetry QA results of online adapted plans were compared to original (unadapted) plans as well as randomly selected prostate IMRT plans from our clinic. In the second part, an online delivery monitoring system was designed and validated via a simulated treatment with intentional multileaf collimator (MLC) errors. This system was based on inputs from the dynamic machine information (DMI), which continuously reports actual MLC positions and machine monitor units (MUs) at intervals of 50 ms or less during delivery. Based on the DMI, the system performed two levels of monitoring/verification during the delivery: (1) dynamic monitoring of cumulative fluence errors resulting from leaf position deviations and visualization using fluence error maps (FEMs); and (2) verification of MLC positions against the treatment plan for potential errors in MLC motion and data transfer at each control point. Validation of the online delivery monitoring system was performed by introducing intentional systematic MLC errors (ranging from 0.5 to 2 mm) to the DMI files for both leaf banks. These DMI files were analyzed by the proposed system to evaluate the system’s performance in quantifying errors and revealing the source of errors, as well as to understand patterns in the FEMs. In addition, FEMs from 210 actual prostate IMRT beams were analyzed using the proposed system to further validate its ability to catch and identify errors, as well as establish error magnitude baselines for prostate IMRT delivery. Results: Online adapted plans were found to have similar delivery accuracy in comparison to clinical IMRT plans when validated with portal dosimetry IMRT QA. FEMs for the simulated deliveries with intentional MLC errors exhibited distinct patterns for different MLC error magnitudes and directions, indicating that the proposed delivery monitoring system is highly specific in detecting the source of errors. Implementing the proposed QA system for online adapted plans revealed excellent delivery accuracy: over 99% of leaf position differences were within 0.5 mm, and >99% of pixels in the FEMs had fluence errors within 0.5 MU. Patterns present in the FEMs and MLC control point analysis for actual patient cases agreed with the error pattern analysis results, further validating the system’s ability to reveal and differentiate MLC deviations. Calculation of the fluence map based on the DMI was performed within 2 ms after receiving each DMI input. Conclusions: The proposed online delivery monitoring system requires minimal additional resources and time commitment to the current clinical workflow while still maintaining high sensitivity to leaf position errors and specificity to error types. The presented online delivery monitoring system therefore represents a promising QA system candidate for online adaptive radiation therapy.

07 ISOTOPE AND RADIATION SOURCES↗

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows, In: 2016 Second International Workshop on Extreme Scale Programming Models and Middlewar (ESPM2)

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

Computer Science↗

Accelerating error correction in tomographic reconstruction

Abstract Spurred by recent advances in detector technology and X-ray optics, upgrades to scanning-probe-based tomographic imaging have led to an exponential growth in the amount and complexity of experimental data and have created a clear opportunity for tomographic imaging to approach single-atom sensitivity. The improved spatial resolution, however, is highly susceptible to systematic and random experimental errors, such as center of rotation drifts, which may lead to imaging artifacts and prevent reliable data extraction. Here, we present a model-based approach that simultaneously optimizes the reconstructed specimen and sinogram alignment as a single optimization problem for tomographic reconstruction with center of rotation error correction. Our algorithm utilizes an adaptive regularizer that is dynamically adjusted at each alternating iteration step. Furthermore, we describe its implementation in a software package targeting high-throughput workflows for execution on distributed-memory clusters. We demonstrate the performance of our solver on large-scale synthetic problems and show that it is robust to a wide range of noise and experimental drifts with near-ideal throughput.

Ali, Sajid (ORCID:0000000321864636)↗