Search NASA⌕ Search

SEARCH · Search NASA

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Closing the Gap between FAIR Data Repositories and Hierarchical Data Formats

Many in the scientific community, particularly in publicly funded research, are pushing to adhere to more accessible data standards to maximize the findability, accessibility, interoperability, and reusability (FAIR) of scientific data, especially with the growing prevalence of machine learning augmented research. Online FAIR data repositories, such as the Open Science Framework (OSF), help facilitate the adoption of these standards by providing frameworks for storage, access, search, APIs, and other features that create organized hubs of scientific data. However, the wider acceptance of such repositories is hindered by the lack of support of hierarchical data formats, such as Technical Data Management Streaming (TDMS) and Hierarchical Data Format 5 (HDF5), that many researchers rely on to organize their datasets. Various tools and strategies should be used to allow hierarchical data formats, FAIR data repositories, and scientific organizations to work more seamlessly together. A pilot project at Los Alamos National Laboratory (LANL) addresses the disconnect between them by integrating the OSF FAIR data repository with hierarchical data renderers, extending support for additional file types in their framework. The multifaceted interactive renderer displays a tree of metadata alongside a table and plot of the data channels in the file. This allows users to quickly and efficiently load large and complex data files directly in the OSF webapp. Users who are browsing files can quickly and intuitively see the files in the way they or their colleagues structured the hierarchical form and immediately grasp their contents. This solution helps bridge the gap between hierarchical data storage techniques and FAIR data repositories, making both of them more viable options for scientific institutions like LANL which have been put off by the lack of integration between them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Nondestructive Assay Technical Infrastructure Program Mission and Vision

The Nondestructive Assay Program (NDAP) mission and vision is achieved by identifying and accomplishing a set of programmatic goals that correspond with eight broad technical program elements. The NDAP was created as a result of Defense Nuclear Facility Safety Board Recommendation 2007-1, entitled “Safety-Related In Situ Nondestructive Assay of Radioactive Materials,” which emphasized the need to improve in situ measurements of radioactive material at US Department of Energy (DOE) defense nuclear facilities. An NDAP five-year plan will define tasks to accomplish specific goals identified in the NDAP mission and vision. This mission and vision is applicable for eight technical program elements: Hardware/Software Development, Algorithm Development and Nuclear Data, Uncertainty Quantification, Nuclear Materials, Staffing, Personnel, And Training, Data Management, Requirement and Standards, and Information Preservation and Dissemination. An additional program element is technical support, which provides daily execution management support for the NDAP, site scope, and deliverable tracking via site task managers, along with technical advisement and support from the technical support group. The NDAP mission and vision provides attributes with specific goals for each program element, and some goals benefit multiple program elements. The goals and attributes defined herein are implemented via an NDAP five-year execution plan that defines site work scope, budget, and deliverables, all of which are updated annually. The NDAP is designed to benefit nondestructive assay (NDA) needs to support DOE nuclear criticality safety programs, ensuring that NDA technology is sufficiently capable of guaranteeing the safety of those who handle, store, process, or transport fissionable materials in the complex. Especially important for the NDAP is to maximize capabilities to identify, characterize, and manage in situ fissile material deposits in process equipment to ensure nuclear criticality safety at processing facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

NDMAS Overview

Overview of the Nuclear Data Management and Analysis System detailed as: Overview of NDMAS System, NDMAS NQA-1 process, Data in NDMAS, and Future work and new additions.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

DMTN-107: Options for Alert Production in LSST Operations Year 1

This document reviews five options for alert production in LSST Operations Year 1 (LOY1), taking into account any implications on LSST formal requirements including up-scopes, down-scopes or explicit violations. The Data Management System Science Team's preferred option for maximizing LSST science is to generate template images from as much of the data from the commissioning and/or science verification phases as possible and use them to run Difference Image Analysis and alert production during LOY1. A proposal to increase the sky area covered during commissioning via a "filler" scheduler program is also presented. As a potential moderate up-scope, this study presents an option to build interim templates on a $\sim$monthly basis during LOY1, which could increase the accessible sky area by ~1000-2000 deg**2 per month, and should be reconsidered closer to the start of Operations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

An open-access simulated earthquake ground-motion database for an M7 Hayward Fault earthquake in the San Francisco Bay Region

Comprehensive understanding of earthquake ground motions, particularly in the near-fault region of large-magnitude events, is limited by gaps in strong-motion data. This challenge is prominent in areas with high seismic hazard but infrequent large earthquakes where data is sparse and difficult to interpret. These data limitations lead to uncertainties in the development of site-specific ground motions, which are crucial for engineering risk assessments. To address these challenges, physics-based regional-scale ground-motion simulations have been developed. With the emergence of exaflop-scale computing ecosystems, it is now possible to simulate regional earthquake processes at unprecedented fidelity and generate the large number of fault rupture realizations necessary to characterize both intra- and inter-event ground-motion variability. This article introduces a new database of simulated earthquake ground motions, created for applications in earthquake engineering, earthquake planning, and emergency response. The inaugural version of the database features simulated ground motions for a magnitude 7 Hayward Fault earthquake in the San Francisco Bay Region (SFBR), using the EarthQuake SIMulation (EQSIM) simulation framework and the Graves–Pitarka kinematic rupture model. The aim is to provide high-fidelity, spatially dense, three-component motions generated on the Department of Energy’s (DOE) newest generation of graphics processing unit (GPU)-accelerated supercomputers. These motions are being made openly available to the engineering, scientific, and disaster planning communities. In addition, this work develops protocols for the efficient dissemination of these large data sets and emphasizes community engagement to build confidence in their application. This article discusses the methodology behind the data, underlying software verification and validation, scalable data management, and a user interface for data access. The goal is to facilitate widespread use and elicit expert feedback to maximize the utility and exploitation of simulated motions. While the initial focus is on the San Francisco Region, simulations for additional regions will be added as the DOE program progresses.

Simulated ground-motion database↗
DOE OSTI2026

AI-Ready Control System for the Fermilab Accelerator Complex

Reliable, high-intensity operation of the Fermilab Accelerator Complex is critical to the success of the Long-Baseline Neutrino Facility and Deep Underground Neutrino Experiment. We describe the requirements and infrastructure necessary to support routine use of artificial intelligence and machine learning (AI/ML) in the accelerator control system. Three capabilities are identified: a machine learning operations (MLOps) framework standardizing the lifecycle of AI/ML automation from data management through deployment and monitoring; a data quality framework defining and enforcing standards required to build trustworthy AI/ML applications; and workflow integration with large language models to assist physicists, engineers, and operators with information retrieval, code development, and routine analysis. Use cases spanning beam diagnostics, beam control, and support system automation illustrate the technical requirements across the complex.

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

PoliMOR

PoliMOR is a scalable, automated, and customizable policy engine framework for multi-tiered parallel file systems. It is composed of single-purpose agents that handle tasks such as gathering file metadata, making policy decisions, and then executing actions based on those policies. These agents are designed to communicate using distributed message queues, allowing the number of individual agents to be scaled up as needed. PoliMOR automates the data management tasks by precluding the need for admin intervention. The agents in PoliMOR can be customized to integrate any utilities/tools that perform tasks like metadata scanning and data placement management.

Brumgard, Christopher↗
DOE OSTI2023

FAIR for AI: An interdisciplinary and international community building perspective

A foundational set of findable, accessible, interoperable, and reusable (FAIR) principles were proposed in 2016 as prerequisites for proper data management and stewardship, with the goal of enabling the reusability of scholarly data. The principles were also meant to apply to other digital assets, at a high level, and over time, the FAIR guiding principles have been re-interpreted or extended to include the software, tools, algorithms, and workflows that produce data. FAIR principles are now being adapted in the context of AI models and datasets. Here, we present the perspectives, vision, and experiences of researchers from different countries, disciplines, and backgrounds who are leading the definition and adoption of FAIR principles in their communities of practice, and discuss outcomes that may result from pursuing and incentivizing FAIR AI research. The material for this report builds on the FAIR for AI Workshop held at Argonne National Laboratory on June 7, 2022.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

AGR 5/6/7 Data Qualification Report for ATR Cycles 162B through 168A

This report provides the qualification status of experimental data for the Advanced Gas Reactor (AGR) 5/6/7 fuel irradiation. AGR-5/6/7 was conducted in the Advanced Test Reactor (ATR) at Idaho National Laboratory (INL) in support of development and qualification of tri-structural isotropic (TRISO) low-enriched fuel for use in high temperature gas-cooled reactors. The objectives of the AGR-5/6/7 experiments are to: (i) irradiate reference-design fuel particles to support fuel qualification, (ii) establish operating margins for the fuel beyond normal operating conditions, and (iii) provide irradiated-fuel performance data and irradiated-fuel samples for post-irradiation examination (PIE) and safety testing. The test train contains five separate capsules that were independently controlled and monitored. Each capsule contains multiple 12.51-mm-long compacts filled with low enriched uranium carbide/oxide (UCO) TRISO fuel particles. The primary objective of the AGR-5/6 test (Capsules 1, 2, 4, and 5) is to verify successful performance of the reference-design fuel under normal operating conditions. The AGR-7 test (Capsule 3) was designed to explore fuel performance at higher temperatures to demonstrate the capability of the fuel to withstand conditions beyond normal operating conditions in support of plant design and licensing. AGR 5/6/7 will also provide irradiated-fuel performance data on fission-gas release from failed particles during irradiation. The AGR-5/6/7 capsules were irradiated in the ATR northeast flux trap location. The experiment began on February 16, 2018 and ended on July 22, 2020, spanning nine ATR cycles over two and a half years. Thus, the AGR-5/6/7 fuel compacts were irradiated for a total of 360.9 effective full power days. The AGR 5/6/7 experiment was able to remain in the reactor core during all three Powered Axial Locator Mechanism (PALM) cycles (163A, 165A, and 167A) without overheating its fuel compacts. This report includes irradiation monitoring data from nine ATR Cycles: 162B, 163A, 164A, 164B, 165A, 166A, 166B, 167A, and 168A, as stored in the Nuclear Data Management and Analysis System (NDMAS). During irradiation, data records consisted of instantaneous measurements recorded every minute and provided by text files automatically every 2 hours. The AGR 5/6/7 data streams addressed in this report include thermocouple (TC) temperatures, sweep gas data (flow rates [capsule inlet, outlet, and downstream at detector], pressure, and moisture content), and Fission Product Monitoring System (FPMS) data (release rates and release to birth rate ratios [R/Bs]) for each of the five capsules. A total of 94,989,908 TC temperature and sweep gas data records were received and processed by NDMAS for AGR 5/6/7 irradiation. Of these records, 41,593,387 (or 43.7% of the total) met data collection and accuracy requirements and are labeled as Qualified. A total of 57,746,693 TC temperature readings were captured from 54 installed TCs. Among them, 10,034,676 TC temperature records (only 17.4%) were Qualified and 47,701,371 TC temperatures (or 82.6%) are Failed due to 48 TC failures (63.5%) and due to missing values (19.1%). To assess performance of the operational TCs, analysis of daily correlations between TCs found no evidence of virtual junction failure for any TCs. Analyses on control charts of TC temperature differences revealed trending in TC readings for TC2, 4, 5, and 13 in Capsule 3, but there is no conclusive indication of TC drift failure that caused those trends. Therefore, TC control charts are not used to disqualify TC data, but only for users’ consideration. For sweep gas flow rates, a total of 31,519,747 gas flow records (84.4%) are Qualified for use for AGR-5/6/7 experiment; 5,723,468 gas flow records (15.4%) are Failed due mostly to missing values; and 74,641 high sweep gas flow rates (0.2 %) are Trend. A large number of Failed missing TC temperature and gas flow values were caused by an error in the data output script that outputted a ‘NULL’ value when values were unchanged. This problem was fixed during the outage of Cycle 166B, which led to a substantially decreased number of missing values during the last three cycles. Nonetheless, a large amount of non-missing data remained because of the high data acquisition frequency (1-minute) and still provided sufficient data to effectively monitor the experiment as designed. For FPMS data, NDMAS received and processed fission product release and R/B data for nine ATR cycles, when ATR core reached full power during AGR 5/6/7 irradiation. These data consist of 110,388 release rate records and 110,388 R/B records for the twelve radionuclides (Kr 85m, Kr 87, Kr 88, Kr 89, Kr 90, Xe 131m, Xe 133, Xe 135, Xe 135m, Xe 137, Xe 138, and Xe 139) for each of the five capsules. Equivalent numbers of uncertainty records associated the release rates and R/B values were provided. To date, qualification status of the FPMS data stored in the NDMAS dat

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Pipeline for Integrated Projects in Energy Systems (PIPES): A Tool for Integrated System Planning [Slides]

The Pipeline for Integrated Projects in Energy Systems (PIPES) is a comprehensive project, data, and workflow management tool designed for integrated modeling teams. PIPES facilitates the management of data requirements, tasks, and progress tracking, serving as a higher-level integration layer that works across various data and modeling software. This tool integrates models, data, and tools to perform large-scale, integrated analysis work at scale. PIPES is designed to streamline integrated modeling projects, enhance collaboration, and ensure the quality and efficiency of data management and workflow processes. This presentation introduces PIPES a multi-model tool for integrated system planning; it describes the underlying architecture, deep dives into common user workflows, and outlines the upcoming development roadmap beyond its current alpha state.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Enabling Modular Autonomous Feedback‐Loops in Materials Science through Hierarchical Experimental Laboratory Automation and Orchestration

Abstract Materials acceleration platforms (MAPs) operate on the paradigm of integrating combinatorial synthesis, high‐throughput characterization, automatic analysis, and machine learning. Within a MAP, one or multiple autonomous feedback loops may aim to optimize materials for certain functional properties or to generate new insights. The scope of a given experiment campaign is defined by the range of experiment and analysis actions that are integrated into the experiment framework. Herein, the authors present a method for integrating many actions within a hierarchical experimental laboratory automation and orchestration (HELAO) framework. They demonstrate the capability of orchestrating distributed research instruments that can incorporate data from experiments, simulations, and databases. HELAO interfaces laboratory hardware and software distributed across several computers and operating systems for executing experiments, data analysis, provenance tracking, and autonomous planning. Parallelization is an effective approach for accelerating knowledge generation provided that multiple instruments can be effectively coordinated, which the authors demonstrate with parallel electrochemistry experiments orchestrated by HELAO. Efficient implementation of autonomous research strategies requires device sharing, asynchronous multithreading, and full integration of data management in experimental orchestration, which to the best of the authors’ knowledge, is demonstrated for the first time herein.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Cyber Infrastructure for the Smart Electric Grid

As electric power systems undergo a transformative upgrade with the integration of advanced technologies to enable the smarter electric grid, professionals who work in the area require a new understanding of the evolving complexity of the grid. Cyber Infrastructure for the Smart Electric Grid delivers a comprehensive overview of the fundamental principles of smart grid operation and control, smart grid technologies, including sensors, communication networks, computation, data management, and cyber security, and the interdependencies between the component technologies on which a smart grid's security depends. The book offers readers the opportunity to critically analyze the smart grid infrastructure needed to sense, communicate, compute, and control in a secure way.

communication networks↗
DOE OSTI2022

Echo: Data and Analysis Management [Slides]

Echo software capabilities are described. Echo provides a comprehensive suite of tools for data wrangling, management, and analysis. It is an object-oriented approach to data analysis designed for robust and scalable execution of complex analysis workflows written in MATLAB.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

An open source fast fluid dynamics model for data center thermal management

Although computational fluid dynamics (CFD) has been widely adopted to improve data center thermal management, the high computational demand limits its applications, such as multivariate optimal design and operation. Fast fluid dynamics (FFD), which has been applied for fast airflow simulation, shows great potential. However, few research applied FFD for optimal design and operation of data center thermal management. This research improves the FFD model for data centers and conducts a comprehensive evaluation and demonstration. First, the FFD model is improved by solving the advection and diffusion equations together using an upwind scheme instead of a semi-Lagrangian advection solver in the conventional FFD model. Second, new features for data centers are added, such as a pressure correction method to simulate plenum airflow and dynamic boundary conditions for IT racks. The new FFD model is first validated with two indoor environment cases and the results show that the new FFD model has slightly better overall prediction accuracy and faster speed compared to the conventional FFD model. It is also observed that both FFD models achieve acceptable accuracy, except for a few localized disparities with experimental data, which might be due to simplified handling of turbulence viscosity near the boundaries. Furthermore, validation with a real data center shows that the FFD model achieves a similar level of accuracy as CFD when compared to the experimental measurements with some level of uncertainties. It is then demonstrated for data center optimal design and operation, which saves 53.4–58.8% of annual energy while still meeting the thermal requirements. In conclusion, with a much faster speed and comparable accuracy compared to CFD, the FFD model parallelized on a graphics processing unit is promising for practical model-based data center early design and operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

The Ontologies Community of Practice: A CGIAR Initiative for Big Data in Agrifood Systems

Heterogeneous and multidisciplinary data generated by research on sustainable global agriculture and agrifood systems requires quality data labeling or annotation in order to be interoperable. As recommended by the FAIR principles, data, labels, and metadata must use controlled vocabularies and ontologies that are popular in the knowledge domain and commonly used by the community. Despite the existence of robust ontologies in the Life Sciences, there is currently no comprehensive full set of ontologies recommended for data annotation across agricultural research disciplines. In this paper, we discuss the added value of the Ontologies Community of Practice (CoP) of the CGIAR Platform for Big Data in Agriculture for harnessing relevant expertise in ontology development and identifying innovative solutions that support quality data annotation. The Ontologies CoP stimulates knowledge sharing among stakeholders, such as researchers, data managers, domain experts, experts in ontology design, and platform development teams.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Anomaly Detection, Localization and Classification using Drifting Synchrophasor Data Streams

With ongoing automation and digitization of the electric power system, several Phasor Measurement Units(PMUs) have been deployed for monitoring and control. PMU data can have multiple anomalies, and many of the researchers in the past have concentrated on training machine/deep learning algorithms offline for anomaly detection over PMU data (i.e., not in real time). These machine/deep learning algorithms, when trained offline on a sample rather than a population of the dataset, fail to consider the dynamic behavior of the power grid in real-time, resulting in low accuracy. Considering the dynamic behavior of the power grid (e.g., change in load, generation, distributed energy resources (DERs) switching, network, controls), the definition of data anomalies varies in time and requires online training. A fundamental challenge is to enable online (i.e., real-time) training of machine/deep learning algorithms for anomaly detection over streaming PMU data. While machine/deep learning is often desirable to manage data streams, training a deep learning algorithm over streaming PMU data is nontrivial due to changes in data statistics caused by dynamic streaming data. This paper proposes PMUNET: a novel device-level deep learning-based data-driven approach for anomaly detection, localization, and classification over streaming PMU data, using online learning and multivariate data-drift detection algorithm .Two variants of PMUNET, Dynamic data Change Driven Learning (DCDL) and Continuity Driven Learning (CDL), are proposed and compared. DCDL aims to train the deep learning algorithm whenever the definition of anomaly changes due to the power grid dynamics. On the other hand, CDL continuously trains the deep learning algorithm over the PMU data-stream. The experimental results verify that DCDL outperforms CDL and other efficient anomaly detection methods over multiple events such as faults and load/ generator/capacitor/DERs variations/switching for IEEE 14 and 39 Bus test system as well as real PMU industrial data. The result verifies that DCDL variant of PMUNET improves over existing approach with a gain of 2% - 10% in terms of accuracy, false-positive rate, and false-negative rate.

adversarial deep learning↗
DOE OSTI2022

LANL Contributions to pyiron: An integrated development environment for computational materials science

To support and accelerate the development of simulation protocols in atomistic modelling, we introduce an integrated development environment (IDE) for computational materials science called pyiron (http://pyiron.org). The pyiron IDE combines a web based source code editor, a job management system for build automation, and a hierarchical data management solution. The core components of the pyiron IDE are pyiron objects based on an abstract class, which links application structures such as atomistic structures, projects, jobs, simulation protocols and computing resources with persistent storage and an interactive user environment. The simulation protocols within the pyiron IDE are constructed using the Python programming language. To highlight key concepts of this tool as well as to demonstrate its ability to simplify the implementation and testing of simulation protocols we discuss two applications. In these examples we show how pyiron supports the whole life cycle of a typical simulation, seamlessly combines ab initio with empirical potential calculations, and how complex feedback loops can be implemented. While originally developed with focus on ab initio thermodynamics simulations, the concepts and implementation of pyiron are general thus allowing to employ it for a wide range of simulation topics.

Janssen, Jan↗