Search NASA⌕ Search

SEARCH · Search NASA

Results for “provenance tracking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
DOE OSTI2026

Livewire: A Model Platform for Data Quality Assessment and AI Readiness Across DOE Missions

High-quality, well-governed data is essential for accelerating discovery and achieving operational excellence across DOE and national laboratory missions. The Livewire Data Platform is a DOE-supported platform that offers automated assessments of data quality, standardization, provenance, and Artificial Intelligence (AI) readiness. It allows researchers and data practitioners to systematically and easily evaluate datasets against established governance criteria and prepare them for advanced analytics. Livewire addresses critical challenges in DOE's data ecosystem with integrated capabilities for metadata validation, provenance tracking, and schema alignment. This platform's automated workflows assist users in identifying data quality gaps, enhancing interoperability between datasets collected from various stakeholders, and ensuring compliance with DOE data standards, all while reducing manual curation efforts. Additionally, we will discuss its AI readiness framework, which is being developed to prepare datasets for training models, developing advanced analytic tools, and machine learning applications. Using some of the more than one hundred tabular datasets on Livewire, processed with this open-source methodology, we will demonstrate how Livewire can serve as a model for scalable, standards-driven data management. This approach provides a pathway to leverage existing and future datasets within the DOE, boosting innovation and efficiency across national laboratories.

33 - ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Optimal control of the electron temperature profile in DIII-D using machine learning surrogate models

The viability of the tokamak as a potential fusion reactor depends on the ability to keep the plasma in a stable regime while achieving temperatures, densities, and confinement times that are as high as possible. Tokamak scenario development attempts to find plasma regimes that achieve all of these conditions and are accessible with a given set of hardware constraints. This requires the ability to control plasma properties such as the normalized beta, the internal inductance, safety factor, rotation, etc. One property that has received less attention than some of the others, but is no less critical to achieving high performance, is the electron temperature (T e ) profile. In this work, Linear Quadratic Integral (LQI) control is used to develop a controller for the electron temperature profile in DIII-D. The controller is based on a linearized model derived from the transport equation that describes the evolution of the electron temperature, and includes contributions from the neural network surrogate models NubeamNet and MMMnet. Furthermore, the controller is tested in simulation using COTSIM, and is proven capable of tracking a target T e profile.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

ECO: the Evidence and Conclusion Ontology, an update for 2022

The Evidence and Conclusion Ontology (ECO) is a community resource that provides an ontology of terms used to capture the type of evidence that supports biomedical annotations and assertions. Consistent capture of evidence information with ECO allows tracking of annotation provenance, establishment of quality control measures, and evidence-based data mining. ECO is in use by dozens of data repositories and resources with both specific and general areas of focus. ECO is continually being expanded and enhanced in response to user requests as well as our aim to adhere to community best-practices for ontology development. The ECO support team engages in multiple collaborations with other ontologies and annotating groups. Here we report on recent updates to the ECO ontology itself as well as associated resources that are available through this project. ECO project products are freely available for download from the project website (https://evidenceontology.org/) and GitHub (https://github.com/evidenceontology/evidenceontology). ECO is released into the public domain under a CC0 1.0 Universal license.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

ProvSec: Open Cybersecurity System Provenance Analysis Benchmark Dataset with Labels

System provenance forensic analysis has been studied by a large body of research work. This area needs fine granularity data such as system calls along with event fields to track the dependencies of events. While prior work on security datasets has been proposed, we found a useful dataset of realistic attacks and details that are needed for high-quality provenance tracking is lacking. We created a new dataset of eleven vulnerable cases for system forensic analysis. It includes the full details of system calls including syscall parameters. Realistic attack scenarios with real software vulnerabilities and exploits are used. For each case, we created two sets of benign and adversary scenarios which are manually labeled for supervised machine-learning analysis. In addition, we present an algorithm to improve the data quality in the system provenance forensic analysis. We demonstrate the details of the dataset events and dependency analysis of our dataset cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Workflow Provenance in the Computing Continuum for Responsible, Trustworthy, and Energy-Efficient AI

As Artificial Intelligence (AI) becomes more pervasive in our society, it is crucial to develop, deploy, and assess Responsible and Trustworthy AI (RTAI) models, i.e., those that consider not only accuracy but also other aspects, such as explainability, fairness, and energy efficiency. Workflow provenance data have historically enabled critical capabilities towards RTAI. Provenance data derivation paths contribute to responsible workflows through transparency in tracking artifacts and resource consumption. Provenance data are well-known for their trustworthiness helping explainability, reproducibility, and accountability. However, there are complex challenges to achieve RTAI, which are further complicated by the heterogeneous infrastructure in the computing continuum (Edge-Cloud-HPC) used to develop and deploy models. As a result, a significant research and development gap remains between workflow provenance data management and RTAI. In this paper, we present a vision of the pivotal role of workflow provenance in supporting RTAI and discuss related challenges. We present a schematic view between RTAI and provenance, and highlight open research directions.

Santos Souza, Renan↗
DOE OSTI2020

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

CORAL: A framework for rigorous self-validated data modeling and integrative, reproducible data analysis

Abstract Background Many organizations face challenges in managing and analyzing data, especially when relevant datasets arise from multiple sources and methods. Analyzing heterogeneous datasets and additional derived data requires rigorous tracking of their interrelationships and provenance. This task has long been a Grand Challenge of data science and has more recently been formalized in the FAIR principles: that all data objects be Findable, Accessible, Interoperable, and Reusable, both for machines and for people. Adherence to these principles is necessary for proper stewardship of information, for testing regulatory compliance, for measuring the efficiency of processes, and for facilitating reuse of data-analytical frameworks. Findings We present the Contextual Ontology-based Repository Analysis Library (CORAL), a platform that greatly facilitates adherence to all 4 of the FAIR principles, including the especially difficult challenge of making heterogeneous datasets Interoperable and Reusable across all parts of a large, long-lasting organization. To achieve this, CORAL's data model requires that data generators extensively document the context for all data, and our tools maintain that context throughout the entire analysis pipeline. CORAL also features a web interface for data generators to upload and explore data, as well as a Jupyter notebook interface for data analysts, both backed by a common API. Conclusions CORAL enables organizations to build FAIR data types on the fly as they are needed, avoiding the expense of bespoke data modeling. CORAL provides a uniquely powerful platform to enable integrative cross-dataset analyses, generating deeper insights than are possible using traditional analysis tools.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

CACTUS: A depleted monolithic active timing sensor using a CMOS radiation hard technology

The planned luminosity increase at the Large Hadron Collider in the coming years has triggered interest in the use of the particles' time of arrival as additional information in specialized detectors to mitigate the impact of pile-up. The required time resolution is of the order of tens of picoseconds, with a spatial granularity of the order of 1 mm. A time measurement at this precision level will also be of interest beyond the LHC and beyond high energy particle physics. We present in this paper the first developments towards a radiation hard Depleted Monolithic Active Pixel Sensor (DMAPS), with high-resolution time measurement capability. The technology chosen is a standard high voltage CMOS process, in conjunction with a high resistivity detector material, which has already proven to efficiently detect particles in tracking applications after several hundred of Mrad of irradiation.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Optimizing pixel tracklet searches for shorter lifetimes

Pixel tracklets, disappearing tracks reconstructed with only pixel hits, have proven to be a promising technique in LHC analyses to search for dark matter candidates at the LHC that belong to a nearly-degenerate electroweak multiplet. However, a Pseudo-Dirac electroweak doublet fermion, arguably the most interesting such possibility, has a shorter lifetime and therefore existing tracklet searches are less sensitive in this case. We assess the performance of a tracklet search optimized for shorter lifetimes by requiring only three pixel hits for the tracklet reconstruction, and by demanding an accompanying soft track for suppressing backgrounds. We estimate how far the sensitivity of existing searches can be extended into the region of parameter space with this optimized search.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Molten Salt Sampling Techniques and Analytical Approaches

Recent global interest in pyroprocessing and molten salt reactors has brought salt sampling methods and techniques back to the forefront of nuclear safeguards concerns. Issues with uranium supplies have also encouraged various countries to pursue advanced nuclear fuel cycles. Tracking nuclear material in molten salt has proven to be a challenge and updating molten salt sampling will greatly help in this endeavor. Molten salt is problematic to sample due to salt stratification, lack of homogeneity, solids, and difficulty with hot cell adaptations. Various salt sampling techniques have been used since before the 1960s including surface, spoon/spatula, and bar solidification. Since then, new types of sampling techniques have been developed to improve sampling results. These include rod/dip, pipet, suction, filtered sampling along with devices such as the Valve Core Sampler and the Multi-Level Sampler. These different approaches are being analyzed and improved upon along with developing requirements for an improved salt sampling device. Work continues to develop salt samplers that are more robust, easier to segment, collect at a specific depth, can work with filters, and can collect fines. Sampling parameters are also being narrowed in terms of stirring, settling time, filtration, depth, etc. In the future, we hope to address deficiencies for process control and nuclear material accountancy control by determining the best way to collect samples that minimizes contaminants and is representative. A compilation of salt sampling approaches, analyses techniques, and an evaluation of findings will be presented.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

Curifactory: A research experiment manager

Curifactory is a command line tool and framework for organizing Python experiment code, configuration parameters, and results. It is an opinionated and lightweight approach to workflow management infrastructure and is primarily intended to support researchers conducting experiments on one machine. This software was developed to support the reproducibility of results for several data science projects in the Nuclear Nonproliferation Division at Oak Ridge National Laboratory. Curifactory is intended to be a general framework and is not specific to machine learning or data science. It can aid in any field in which experiments are primarily computation-based studies and can be implemented in Python (e.g., high-energy physics, astronomy, computational chemistry). Here, the design emphasizes the automated caching of intermediate data analysis artifacts to speed up development involving computationally intensive tasks. It also allows for data provenance and experiment reproduction. Individual experiment runs are tracked through logs and their output reports, and entire copies of a run with all cached data and metadata can be exported for others to run using Curifactory on another machine. Curifactory experiments can either be integrated into a project from the beginning or can be written on top of an existing codebase without needing significant modification. A few important views of the Curifactory library can be seen in Figure 1.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Expanding Activity Allocation Models to Daily Activities: Tracking Simulated Agent Trips to Exercise Locations in Clarksville, TN

Exercise facilities have been proven to have numerous physical, mental, and psychological benefits, yet exercise facilities are still inaccessible to a large portion of the population. This study serves to explore the accessibility of fitness centres through geographical, demographic, and temporal lenses through an expansion of the UrbanPop framework that seeks to allocate simulated agents to fitness centres in the Clarksville metro to explore the effects of travel distances on different demographics throughout the week. Findings indicate that senior and retired demographics consistently travel longer distances to exercise in the larger Clarksville area, likely due to tendencies to live further from the center of the metropolitan area. Furthermore, all demographics tend to travel further distances to exercise on the weekends rather than the weekdays, indicating that travel distance can affect likelihood of agents to travel, especially on weekdays when many agents are in the workforce or participating in schooling.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2019

Using acoustic telemetry for high-resolution sablefish movement informing potential interactions with a tidal turbine

As the MHK industry moves forward to the commercial phase, knowledge of potential environmental effects need to be documented and established. At present, the potential for fish interactions and subsequent injury or mortality are a significant concern with instream turbines. Sensors, methodologies, and analyses need to be developed to detect collision or near-miss events. These data will reduce uncertainty of the likelihood of such events and provide information to regulators to make informed decisions on permitting and possible mitigation measures if needed. Acoustic telemetry is a primary method for studying fine-scale fish movements because it can remotely track fish in three dimensions with accuracy. The Juvenile Salmon Acoustic Telemetry System (JSATS) developed by Pacific Northwest National Laboratory has proven effective for fish passage estimates and fine-scale movement in river impoundments and near dams. Leveraging the success of previous JSATS applications and applying it to the marine environment, juvenile sablefish (Anoplopoma fimbria) were tagged and tracked using a fixed-position receiver array in a marine tidal channel for the first time. Applying this technology to future installations of MHK turbines will provide information on fish avoidance and evasion behavior as well as blade strike and potential injury and mortality.

Staines, Garrett J.↗
DOE OSTI2022

Data from: "Lab and Field Warming Similarly Advance Germination Date and Limit Germination Rate for High and Low Elevation Provenances of Two Widespread Subalpine Conifers"

This archive contains data used to draw conclusions in “Lab and Field Warming Similarly Advance Germination Date and Limit Germination Rate for High and Low Elevation Provenances of Two Widespread Subalpine Conifers”, by Kueppers et al. 2017. Cone collection and field experiments were conducted on Niwot Ridge, Colorado, with the latter completed within the Alpine Treeline Warming Experiment (ATWE). Laboratory germination experiments were conducted at the University of California Merced. Geospatial files are also provided for additional geographical context.There are four data file formats in this package: comma-separated values (.csv), Microsoft Excel (.xlsx), keyhole markup language (.kml), and ESRI shapefile (.shp). The latter two are geospatial file formats. .csv files can be opened using any simple text editor program such as TextEdit (Mac) and Notepad (Windows), and .xlsx files can be opened using Microsoft Excel or Google Sheets. The .kml file contains coordinate data, and is compatible with Google Earth and Google Maps. ESRI shapefiles (.shp) are compatible with any geospatial software able to read the file type, such as QGIS and ESRI’s ArcGIS Suite. This data user’s guide is available in as a .pdf, and can be opened using software such as Adobe Acrobat Reader.-----------------------------------------------------------------------------------------------------------------------------------Accurately predicting upslope shifts in subalpine tree ranges with warming requires understanding how future forest populations will be affected by climate change, as these are the seed sources for new tree line and alpine populations. Early life history stages are particularly sensitive to climate and are also influenced by genetic variation among populations. We tested the climate sensitivity of germination and initial development for two widely distributed subalpine conifers, using controlled-environment growth chambers with one temperature regime from subalpine forest in the Colorado Rocky Mountains and one 5 °C warmer, and two soil moisture levels. We tracked germination rate and timing, rate of seedling development, and seedling morphology for two seed provenances separated by ~300 m elevation. Warming advanced germination timing and initial seedling development by a total of ~2 weeks, advances comparable to mean differences between provenances. Advances were similar for both provenances and species; however, warming reduced the overall germination rate, as did low soil moisture, only for Picea engelmannii. A three-year field warming and watering experiment planted with the same species and provenances yielded responses qualitatively consistent with the lab trials. Together these experiments indicate that in a warmer, drier climate, P. engelmannii germination, and thus regeneration, could decline, which could lead to declining subalpine forest populations, while Pinus flexilis forest populations could remain robust as a seed source for upslope range shifts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Development of ML FPGA Filter for Particle Identification and Tracking in Real Time

Real-time data processing is a frontier field in experimental particle physics. Machine Learning methods are widely used and have proven to be very powerful in particle physics. The growing computational power of modern FPGA boards allows us to add more sophisticated algorithms for real time data processing. Many tasks could be solved using modern Machine Learning (ML) algorithms which are naturally suited for FPGA architectures. The FPGA-based machine learning algorithm provides an extremely low, sub-microsecond, latency decision and makes information-rich data sets for event selection. We report work has started to evaluate an FPGA based Machine Learning (ML) algorithm for a real-time particle identification and tracking with Transition Radiation Detector (TRD) and e/m calorimeter. The first target is the GlueX experiment, with a plan to build a TRD based on GEM technology. GlueX trigger latency is 3.3 μs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Simultaneous Two- and Three-Photon Deep Imaging of Autofluorescence in Bacterial Communities

The intrinsic fluorescence of bacterial samples has a proven potential for label-free bacterial characterization, monitoring bacterial metabolic functions, and as a mechanism for tracking the transport of relevant components through vesicles. The reduced scattering and axial confinement of the excitation offered by multiphoton imaging can be used to overcome some of the limitations of single-photon excitation (e.g., scattering and out-of-plane photobleaching) to the imaging of bacterial communities. In this work, we demonstrate in vivo multi-photon microscopy imaging of Streptomyces bacterial communities, based on the excitation of blue endogenous fluorophores, using an ultrafast Yb-fiber laser amplifier. Its parameters, such as the pulse energy, duration, wavelength, and repetition rate, enable in vivo multicolor imaging with a single source through the simultaneous two- and three-photon excitation of different fluorophores. Three-photon excitation at 1040 nm allows fluorophores with blue and green emission spectra to be addressed (and their corresponding ultraviolet and blue single-photon excitation wavelengths, respectively), and two-photon excitation at the same wavelength allows fluorophores with yellow, orange, or red emission spectra to be addressed (and their corresponding green, yellow, and orange single-photon excitation wavelengths). We demonstrate that three-photon excitation allows imaging over a depth range of more than 6 effective attenuation lengths to take place, corresponding to an 800 micrometer depth of imaging, in samples with a high density of fluorescent structures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Alaska Liquid Natural Gas Pipeline Front-End Engineering & Design (Final Technical Report)

The Alaska Gasline Development Corporation (AGDC) is Alaska’s natural gas infrastructure development corporation established in 2013. AGDC’s mission is to maximize the benefit of Alaska’s vast North Slope natural gas resources for Alaskans through the development of infrastructure necessary to move the gas into local and international markets. AGDC was identified for a Congressionally Directed Spending (CDS) project for funding in the Energy and Water Development and Related Agencies Appropriations Act, 2023 under the heading: “Congressionally Directed Energy Efficiency and Renewable Energy Projects.” The CDS included $\$$4,000,000 of direct funding, with required match funds, to move the project forward. Alaska’s North Slope holds America’s largest proven and conventional natural gas supply. The integrated Alaska LNG Project will deliver 3.5 billion cubic feet of natural gas per day from Alaska’s North Slope gas fields to Alaskans as well as to a marine terminal located at tidewater in Cook Inlet. Alaska LNG is an integrated gas infrastructure project with three major components: a gas treatment plant (GTP) located at Prudhoe Bay, an 807-mile (1,287 km) gas pipeline (Mainline Pipeline) to Southcentral Alaska with interconnections for in-state gas use, and a natural gas liquefaction facility (LNG Facility) in Nikiski, Alaska. The integrated Alaska LNG Project has several strategic advantages including proven gas resources, existing upstream infrastructure, an advantageous arctic climate for LNG production, proximity to LNG markets, a track record of reliability from a state that first began exporting LNG to Japan in 1969, and broad support from Alaskans. North Slope natural gas is a conventional resource and can be produced with minimal drilling at a fraction of the carbon dioxide emissions of shale gas from the Lower 48 states. Through the development of the Alaska LNG Project, Alaska can provide energy security to Alaskans and a stable source of LNG to the Asia-Pacific region for generations. The Alaska LNG Project has been progressed through Pre-Front-End Engineering Design (Pre-FEED) and has obtained all major federal and State of Alaska permits and authorizations to construct the project, including the Federal Energy Regulatory Commission (FERC) Order Granting Authorization Under Section 3 of the Natural Gas Act. On September 5, 2024, the U.S. Department of Energy (DOE), National Energy Technology Laboratory (NETL) awarded Project No. DE-FE0032307 to AGDC with the objective to progress the project to Front-End Engineering Design (FEED) entry for the Alaska LNG Project Phase 1 Pipeline. The award Start Date was made effective July 1, 2023, with a Period of Performance through June 30, 2025. On March 27, 2025, AGDC announced the execution of definitive commercial agreements with Glenfarne Alaska LNG, LLC, an affiliate of Glenfarne Group, LLC, (together as “Glenfarne”), to lead the development of the Alaska LNG Project and enter FEED for the Phase 1 Pipeline. Project activities are now funded and directed by this private sector partner who holds a 75% interest in 8 Star Alaska, LLC (8 Star). 8 Star holds the assets of the Alaska LNG Project. As planned, AGDC continues to hold 25% minority interest in 8 Star and will play a governance role moving forward with Alaska LNG. This definitive commercial agreement milestone led to the successful completion of AGDC’s Statement of Project Objectives (SOPO) for FEED entry and led to the completion of DOE Project No. DE-FE0032307. At conclusion of the SOPO, AGDC also reached the award’s maximum federal cost share of $\$$4,000,000. AGDC is, therefore, providing Final Technical Report to close out DOE Project No. DE-FE0032307.

02 PETROLEUM↗