Search NASA⌕ Search

SEARCH · Search NASA

Results for “provenance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Workflow Provenance in the Computing Continuum for Responsible, Trustworthy, and Energy-Efficient AI

As Artificial Intelligence (AI) becomes more pervasive in our society, it is crucial to develop, deploy, and assess Responsible and Trustworthy AI (RTAI) models, i.e., those that consider not only accuracy but also other aspects, such as explainability, fairness, and energy efficiency. Workflow provenance data have historically enabled critical capabilities towards RTAI. Provenance data derivation paths contribute to responsible workflows through transparency in tracking artifacts and resource consumption. Provenance data are well-known for their trustworthiness helping explainability, reproducibility, and accountability. However, there are complex challenges to achieve RTAI, which are further complicated by the heterogeneous infrastructure in the computing continuum (Edge-Cloud-HPC) used to develop and deploy models. As a result, a significant research and development gap remains between workflow provenance data management and RTAI. In this paper, we present a vision of the pivotal role of workflow provenance in supporting RTAI and discuss related challenges. We present a schematic view between RTAI and provenance, and highlight open research directions.

Santos Souza, Renan↗
DOE OSTI2023

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge.To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments.We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26—37x faster to capture and transmit provenance data; uses 5—7x less CPU; 2x less memory; transmits 2x less data; and consumes 2—2.5x less energy. ProvLight [1] and E2Clab [2] are available as open-source tools.

Rosendo, Daniel↗
DOE OSTI2023

ProvSec: Open Cybersecurity System Provenance Analysis Benchmark Dataset with Labels

System provenance forensic analysis has been studied by a large body of research work. This area needs fine granularity data such as system calls along with event fields to track the dependencies of events. While prior work on security datasets has been proposed, we found a useful dataset of realistic attacks and details that are needed for high-quality provenance tracking is lacking. We created a new dataset of eleven vulnerable cases for system forensic analysis. It includes the full details of system calls including syscall parameters. Realistic attack scenarios with real software vulnerabilities and exploits are used. For each case, we created two sets of benign and adversary scenarios which are manually labeled for supervised machine-learning analysis. In addition, we present an algorithm to improve the data quality in the system provenance forensic analysis. We demonstrate the details of the dataset events and dependency analysis of our dataset cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Performance Characterization and Provenance of Distributed Task-based Workflows on HPC Platforms

Understanding performance and provenance of task-based workflows poses significant challenges, particularly in distributed configurations where resources are shared by multiple applications. Task-based workflow management systems further complicate performance predictability because of their dynamicity that subtly alters task execution order from run to run. In this paper we propose a layered characterization framework for performance and task provenance for Dask.distributed workflows running on high-performance computing (HPC) platforms. It collects data from jobs, the workflow management system, and the operating system to aid in understanding the performance of these workflows. Our approach encompasses three main contributions: first, an extension of Dask.distributed to capture high-fidelity task provenance using Mochi data services; second, the adaptation of the established HPC I/O characterization tool Darshan to gather high-fidelity I/O data, thereby enhancing the granularity of our analysis; and third, a framework to combine and process the collected data and provide helpful insights into performance characterization and reproducibility, alongside our lessons learned.

Dask↗
DOE OSTI2023

Blockchain Empowered Provenance Framework for Sensor Identity Management and Data Flow Security in Fossil-based Power Plants

The overall goals of this project are divided into 3 important phases. First, a peer-to-peer SCADA network is to be established where identity profile of sensors from fossil-power plants will be stored in an immutable manner in order to verify data flow integrity in real time. Second, it is aimed to develop a Blockchain-based provenance platform to audit equipment operations along with data and process integrity violations. In this phase, various operational rules will be incorporated in the ledger on behalf of the operating sensor so its abnormal use can be easily detected. In addition, it is aimed to achieve provenance when the rate of transactions is exceedingly high, thus enabling to design a scalable provenance scheme that can work in real time when number of data collecting points or sensor units increase in fossil-based power plants. Finally, a working testbed will be developed to integrate the Blockchain platform with sensor-based fossil power plant comprising of Remote Terminal Units (RTU) and SCADA devices. Using multiple Raspberry-PIs mimicking the role of RTUs that are interfaced with multiple sensors devices will be deployed to serve as a critical component of fossil power plant and the performance metrics such as throughput, latency, bootstrapping time, etc., will be evaluated on the testbed.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

INL CRMO XRF Report 2025-03: Provenance Determinations for 854 Projectile Points from Idaho Falls District Bureau of Land Management Archaeological Collections

This report presents provenance determinations for 854 obsidian and fine-grained volcanic projectile points from Idaho Falls District Bureau of Land Management collections. Provenance determinations were made using energy dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

INL CRMO XRF Report 2025-04: Provenance Determinations for 116 Obsidian Artifacts from Upper Snake U.S. Bureau of Reclamation Archaeological Collections

This report presents provenance determinations for 116 obsidian artifacts from Upper Snake U.S. Bureau of Reclamation collections. Provenance determinations were made using energy dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

INL CRMO XRF Report 2025-05: Provenance Determinations for 23 Folsom and Midland Projectile Points from Eastern Idaho

This report presents provenance determinations for 23 obsidian and fine-grained volcanic Folsom and Midland projectile points from eastern Idaho. Provenance determinations were made using energy dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

58 GEOSCIENCES↗
DOE OSTI2025

INL CRMO XRF Report 2025-06: Provenance Determinations for 246 Obsidian and Fine-grained Volcanic Artifacts from Folsom-era Sites in Eastern Idaho

This report presents provenance determinations for 246 obsidian and fine-grained volcanic artifacts from Folsom-era sites in eastern Idaho. Provenance determinations were made using energy dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

58 GEOSCIENCES↗
DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗
DOE OSTI2025

yProv4ML: Effortless provenance tracking for machine learning systems

The rapid growth in interest in deep learning and foundation models (FMs) in particular, has attracted the attention of a diverse range of researchers thanks to their generalization ability. However, the advent of these techniques has also brought to light the lack of transparency and rigor in the way development is pursued. In particular, the inability to determine the number of epochs and other hyperparameters in advance presents challenges in identifying the best model. To address this challenge, machine learning frameworks such as MLFlow can automate the collection of this type of information. However, these tools capture data using proprietary formats and pose little attention to lineage. This paper proposes yProv4ML, a framework that captures provenance information generated during machine learning processes in PROV-JSON format, with minimal code modification.

Machine learning↗
DOE OSTI2024

Distributed Energy, Utility Scale: 30 Proven Strategies to Increase VPP Enrollment [Slides]

After decades of low or declining growth in electricity demand, the U.S. now faces a significant near-term need for new generation capacity and transmission and distribution infrastructure. Virtual Power Plants (VPPs) can meet a large portion of the gap between electricity supply and growing demand, but only if deployed at an increased scale. This study provides 30 proven strategies for scaling VPPs through increased enrollment based on in-depth interviews with utilities and VPP solutions providers that have achieved considerable scale or rapid growth in program deployment. The study includes specific actions for regulators, utilities, and VPP solutions providers to increase VPP enrollment and deliver important customer and utility benefits.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

INL CRMO XRF Report 2024-01: Provenance Determinations for 135 Haskett Projectile Points from Eastern Idaho

This report presents provenance determinations for 135 obsidian and fine-grained volcanic (FGV) Haskett projectile points from across eastern Idaho. These determinations were made using energy-dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

58 GEOSCIENCES↗
DOE OSTI2025

INL CRMO XRF Report 2025-02: Provenance Determinations for 1,124 Projectile Points from Lands and Collections Managed by the U.S. Department of Energy-Idaho Operations Office (DOE-ID)

This report presents provenance determinations for 1,124 obsidian and fine-grained volcanic (FGV) projectile points from lands and collections managed by the U.S. Department of Energy-Idaho Operations Office (DOE-ID). These determinations were made using energy-dispersive X-ray fluorescence (ED-XRF) spectrometry at the Idaho National Laboratory (INL) Cultural Resource Management Office (CRMO) under the direction of Dr. Kyle Freund.

58 GEOSCIENCES↗
DOE OSTI2025

Oxygen stable isotopes in the nuclear fuel cycle: Assessment of the potential for determining the fabrication and provenance history of anhydrous and hydrous uranium oxides

Determining the origin and history of interdicted nuclear materials is a central challenge in nuclear forensics. The oxygen stable isotope composition of uranium oxide compounds has emerged as a promising forensic signature, attracting increasing attention since the early 2000s. This review examines analytical techniques for measuring oxygen isotope compositions in uranium oxides and evaluates how the nuclear fuel production cycle introduces or modifies these isotopic signatures. The potential for forensic geolocation is explored through workflows that calibrate the relationship between environmental water oxygen isotopes and those found in uranium oxides. Key strengths and limitations of this approach are assessed, including gaps in knowledge related to isotope fractionation during specific stages of the fuel cycle, and processing facility water inputs. The importance of proper sample handling and storage under inert atmospheres, as well as a deeper understanding of both intra-sample oxygen isotope heterogeneity, and hydrous uranium oxide phase formation, is highlighted for improving the reliability of forensic interpretations. In conclusion, the development of uranium oxide standards with well-characterized δ 18 O values and international collaboration toward consensus on their use are identified as essential steps for advancing the field.

Attribution↗
DOE OSTI2024

Identifying Sample Provenance From SEM/EDS Automated Particle Analysis via Few-Shot Learning Coupled With Similarity Graph Clustering

Automated particle analysis (APA) provides a vast amount of compositional data via energy-dispersive X-ray spectroscopy along with size and shape data via scanning electron microscopy for individual particles in a sample. In many instances, APA data are leveraged to support identification of the source of a sample based on the detection of particles of a specific composition. Often, the particles that provide context make up a minuscule portion of the sample. Additionally, the interpretation of complex samples can be difficult due to the diversity of compositions both in the mixture and within a particle. In this work, we demonstrate a method to compute and cluster similarity graphs that describe inter-particle relationships within a sample using a multi-modal few-shot learning neural network. Here, as a proof-of-concept, we show that samples known to have been exposed to gunshot residue can be distinguished from samples occasionally mistaken for gunshot residue. Our workflow builds upon standard APA techniques and data processing methods to unveil additional information in a readily interpretable and quantitatively comparable format.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Managing Software Provenance to Enhance Reproducibility in Computational Research

Scientific processes rely on software as an important tool for data acquisition, analysis, and discovery. Over the years, sustainable software development practices have made progress in being considered as an integral component of research. However, management of computation-based scientific studies is often left to individual researchers who design their computational experiments based on personal preferences and the nature of the study. Here, we believe that the quality, efficiency, and reproducibility of computation-based scientific research can be improved by explicitly creating an execution environment that allows researchers to provide a clear record of traceability. This is particularly relevant to complex computational studies in high-performance computing (HPC) environments. In this article, we review the documentation required to maintain a comprehensive record of HPC computational experiments for reproducibility. We also provide an overview of tools and practices that we have developed to perform such studies around Flash-X, a multiphysics scientific software.

97 MATHEMATICS AND COMPUTING↗