SEARCH · Search NASA

Results for “data management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Server rack for improved data center management

Methods and systems for data center management include collecting sensor data from one or more sensors in a rack; determining a location and identifying information for each asset in the rack using a set of asset tags associated with respective assets; communicating the sensor and asset location to a communication module; receiving an instruction from the communication module; and executing the received instruction to change a property of the rack.

Bermudez Rodriguez, Sergio A.↗

DOE OSTI2023

Server rack for improved data center management

Bermudez Rodriguez, Sergio↗

DOE OSTI2024

DOE Data Days 2023 Report

The DOE Data Days (D3) workshop brings together data managers, developers, researchers, and program managers across the Department of Energy (DOE) and national laboratories to highlight data management successes, identify potential synergies and common problems, and establish channels for collaboration across the DOE data management community. The fourth D3 workshop was held on October 24th to 26th, 2023 held entirely in-person at Lawrence Livermore National Laboratory (LLNL). The workshop was organized by a multi laboratory committee in an effort to bring data management practitioners at the DOE laboratories together to share their work and results, facilitating knowledge transfers and best practices across project teams. Tools and platforms to support data management and analysis are rapidly evolving and provide enormous opportunities. This report summarizes the important discussions and recommendations from the different working sessions and contains the agenda, submitted abstracts, presentations with links to recorded presentations, breakout session summaries, list of registered attendees, and lessons learned for future organizing committee. The report will be distributed to the DOE, each participating institution’s programmatic stakeholders, and attendees. The dedicated D3 website will host presentations, agenda, and report that is accessible by all labs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2024

An open-source data storage and visualization platform for collaborative qubit control

Developing collaborative research platforms for quantum bit control is crucial for driving innovation in the field, as they enable the exchange of ideas, data, and implementation to achieve more impactful outcomes. Furthermore, considering the high costs associated with quantum experimental setups, collaborative environments are vital for maximizing resource utilization efficiently. However, the lack of dedicated data management platforms presents a significant obstacle to progress, highlighting the necessity for essential assistive tools tailored for this purpose. Current qubit control systems are unable to handle complicated management of extensive calibration data and do not support effectively visualizing intricate quantum experiment outcomes. In this paper, we introduce Qubit Control Storage and Visualization ( QubiCSV ), a platform specifically designed to meet the demands of quantum computing research, focusing on the storage and analysis of calibration and characterization data in qubit control systems. As an open-source tool, QubiCSV facilitates efficient data management of quantum computing, providing data versioning capabilities for data storage and allowing researchers and programmers to interact with qubits in real time. The insightful visualization are developed to interpret complex quantum experiments and optimize qubit performance. QubiCSV not only streamlines the handling of qubit control system data but also improves the user experience with intuitive visualization features, making it a valuable asset for researchers in the quantum computing domain.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Integrating the PanDA Workload Management System with the Vera C. Rubin Observatory

The Vera C. Rubin Observatory will produce an unprecedented astronomical data set for studies of the deep and dynamic universe. Its Legacy Survey of Space and Time (LSST) will image the entire southern sky every three to four days and produce tens of petabytes of raw image data and associated calibration data over the course of the experiment’s run. More than 20 terabytes of data must be stored every night, and annual campaigns to reprocess the entire dataset since the beginning of the survey will be conducted over ten years. The Production and Distributed Analysis (PanDA) system was evaluated by the Rubin Observatory Data Management team and selected to serve the Observatory’s needs due to its demonstrated scalability and flexibility over the years, for its Directed Acyclic Graph (DAG) support, its support for multi-site processing, and its highly scalable complex workflows via the intelligent Data Delivery Service (iDDS). PanDA is also being evaluated for prompt processing where data must be processed within 60 seconds after image capture. This paper will briefly describe the Rubin Data Management system and its Data Facilities (DFs). Finally, it will describe in depth the work performed in order to integrate the PanDA system with the Rubin Observatory to be able to run the Rubin Science Pipelines using PanDA.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

Developing Sample Management Tools for the Irradiation Test Area (ITA)

The ITA Data Manager (IDM) has been ported from CERN’s Proton Irradiation Facility (IRRAD), where it was used previously as the IRRAD Data Manager (IDM), in order to meet the data management and sample tracking needs of the Irradiation Test Area (ITA), a newly developed facility at Fermilab (launching Fall 2020) operating under the Fermilab Test Beam Facility.

43 PARTICLE ACCELERATORS↗

DOE OSTI2024

MAPSTER: Automated Geospatial Data Sharing – Version 1.4.0

The US Department of Energy’s (DOE) Oak Ridge National Laboratory (ORNL) developed MAPSTER which is a geospatial data management tool that aggregates, organizes, and shares data from dispersed sources such as unmanned aerial systems (UAS). Built specifically for use in environments where communications may be limited, MAPSTER utilizes two key technologies to effectively manage data in the field and enable easy data sharing with authorized partners: Observer and Checkpoint. Observer is a lightweight software package on an edge device, such as a laptop, that automatically detects newly processed UAS data and sends to a central server called Checkpoint. Checkpoint is a centralized server at ORNL that receives and manages data from all Observer instances. Even in a very low bandwidth environment, Observer can still send information about the UAS data product almost instantly as it generates its own metadata package on the size of KB (kilobytes). MAPSTER is not only for UAS data but for any geospatial data collected at the austere edge and dispersed sources.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Location Identifiers, Metadata, and Map for Field Measurements at the East-Taylor Watershed Community Observatory, Colorado, USA (Version 3.3)

This dataset contains identifiers, metadata, and a map of the locations where field measurements have been conducted at the East-Taylor Watershed Community Observatory located in the Upper Colorado River Basin, United States. This is version 3.3 of the dataset and replaces the prior version 3.2 (see below for details on changes between the versions). Dataset description: The East River-Taylor Watershed is the primary field site of the Watershed Function Scientific Focus Area (WFSFA) and the Rocky Mountain Biological Laboratory. Researchers from several institutions generate highly diverse hydrological, biogeochemical, climate, vegetation, geological, remote sensing, and model data at the East-Taylor Watershed in collaboration with the WFSFA. Thus, the purpose of this dataset is to maintain an inventory of the field locations and instrumentation to provide information on the field activities in the East-Taylor Watershed and coordinate data collected across different locations, researchers, and institutions. The dataset contains (1) a README file with information on the various files, (2) three csv files describing the metadata collected for each surface point location, plot and region registered with the WFSFA, (3) csv files with metadata and contact information for each surface point location registered with the WFSFA, (4) a csv file with with metadata and contact information for plots, (5) a csv file with metadata for geographic regions and sub-regions within the watershed, (6) a compiled xlsx file with all the data and metadata which can be opened in Microsoft Excel, (7) a kml map of the locations plotted in the watershed which can be opened in Google Earth, (8) a jpg image of the kml map which can be viewed in any photo viewer, and (9) a zipped file with the registration templates used by the SFA team to collect location metadata. The zipped template file contains two csv files with the blank templates (point and plot), two csv files with instructions for filling out the location templates, and one compiled xlsx file with the instructions and blank templates together. Additionally, the templates in the xlsx include drop down validation for any controlled metadata fields. Persistent location identifiers (Location_ID) are determined by the WFSFA data management team and are used to track data and samples across locations. Dataset uses: This location metadata is used to update the Watershed SFA’s publicly accessible Field Information Portal (an interactive field sampling metadata exploration tool; https://wfsfa-data.lbl.gov/watershed/), the kml map file included in this dataset, and other data management tools internal to the Watershed SFA team. Version Information: The latest version of this dataset publication is version 3.3. This version contains 167 new point locations, 1 new plot, and 2 new geographic regions. Overall, there are a total of 1439 point locations, 75 plots, and 54 geographic regions. Additionally, the kml map of locations and image now includes two boundaries (Upper Ohio Creek (UO) and Carbon Creek (CA)) outside of the East River watershed (USGS HUC-10) and accompanying stream network that represents areas of focus. Refer to methods for further details on the version history. This dataset will be updated on a periodic basis with new measurement location information. Researchers interested in having their East-Taylor Watershed measurement locations added to this list should reach out to the WFSFA data management team at wfsfa-data@googlegroups.com. Acknowledgments: Please cite this dataset if using any of the location metadata in other publications or derived products. If using the location metadata for the 2018 NEON hyperspectral campaign, additionally cite Chadwick et al. (2020). doi:10.15485/1618130. This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231. Part of this work was performed at SLAC Accelerator Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-76SF00515.

2018 NEON and 2025 CHESS Campaigns↗

DOE OSTI2024

Operational experience and R&D results using the Google Cloud for High-Energy Physics in the ATLAS experiment

The ATLAS experiment at CERN relies on a Worldwide Distributed Computing Grid infrastructure to support its physics program at the Large Hadron Collider. ATLAS has integrated cloud computing resources to complement its Grid infrastructure and conducted an R&D program on Google Cloud Platform. These initiatives leverage key features of commercial cloud providers: lightweight configuration and operation, elasticity and availability of diverse infrastructures. Here this paper examines the seamless integration of cloud computing services as a conventional Grid site within the ATLAS workflow management and data management systems, while also offering new setups for interactive, parallel analysis. It underscores pivotal results that enhance the on-site computing model and outlines several R&D projects that have benefited from large-scale, elastic resource provisioning models. Furthermore, this study discusses the impact of cloud-enabled R&D projects in three domains: accelerators and AI/ML, ARM CPUs and columnar data analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Autonomous Tools for Attack Surface Reduction (Final Report)

The electric power grid is a complex critical infrastructure that forms the lifeline of modern society, and its secure and reliable operation is of paramount importance to national security and economic wellbeing. However, recent findings documented in authoritative sources indicate the threat of cyber-based attacks growing in numbers and sophistication. However, securing the grid against stealthy cyberattacks is a challenging task due to legacy nature of the infrastructure coupled with dynamic nature of threat landscape and ever-growing sophistication of the adversaries. Additionally, the grid’s attack surface continues to grow with the increased dependence on digital communications and control that now extends to each consumer through smart meters and distributed energy resources. Unfortunately, this expansive surface increases the grid’s vulnerability and further exposes critical control systems in both substations and control centers. To respond to this emerging need, we had successfully assembled an interdisciplinary team with academic- industry partnership to successfully conduct research, development, evaluation, demonstration, and commercialization of attack surface reduction tools, whose goal was to significantly reduce the cyber attack surface in the North American power grid. Our proposed project was a synergistic collaborative effort leveraging the synergistic expertise of the team members across power systems, cyber security and CPS security, testbeds, field deployments and demonstration, and successful commercialization. The following are the specific tasks that have been successfully completed two phases (2016-2020). Phase I: Task 1: Developed and implemented a robust Project Management and Data Management Plan, coupled with a well thought out Risk Mitigation Plan. Task 2.1: Developed a comprehensive framework that continually assesses and autonomously reduces the attack surface for the power grid control environment spanning across substations, control center and the SCADA network to significantly reduce the risks of cyber attacks. Task 2.2: Developed attack surface analysis techniques, metrics, and tools that assess the attack surface at multiple levels including the control center, substations, and the SCADA network. Task 2.3: Developed attack surface reduction techniques and tools that dynamically reduce attack surface and hence increase attacker’s cost without interfering in the critical functions of the system. Task 2.4: Prototyped, implemented, and quantitatively evaluated/validated the techniques and tools on a realistic industrial CPS security testbed environment by leveraging the unique resources of the team. Task 3: Developed Commercialization plan to transition the developed tools into power system industry stakeholders for a broader adoption by leveraging the expertise of our industrial members. Phase II: Task 4: Successfully completed field demonstration, verification, and evaluation of the effectiveness of the attack surface analysis and reduction techniques on a realistic utility testbed environment. This also involved the development of realistic scenarios, sound metrics, data sets, evaluation criteria, and documentation. Technology integration & Field demonstration: The project had significantly advanced the state-of-the-art research and practice in improving the cybersecurity of our nation’s power grid infrastructure against cyber threats. In particular, the proposed, designed, and deployed attack surface analysis and reduction algorithms and tools have contributed to significantly reducing the exposure and risk of the devices, substations, and the integrated SCADA/EMS/ DMS grid environment to cyber threat. Strong demonstration and evaluation techniques have verified the feasibility of the developed techniques on realistic cyber-physical testbeds and utility partner's real grid environment, and collaborative research and evaluation of attack surface reduction techniques (for wide-are monitoring and control) within a vendor (GE) EMS platform. The Attack Host Analyzer (AHA) tool that was developed through this project was made available through GitHub.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2020

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an open-source electronic hardware platform, and it consists of a microcontroller and two sensors. Finally, we read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Groundwater Monitoring Report, U.S. Department of Energy Y-12 National Security Complex, Oak Ridge, Tennessee

This report contains the groundwater and surface water monitoring data obtained during calendar year (CY) 2019 at the U.S. Department of Energy (DOE) Y-12 National Security Complex (Y-12) on the DOE Oak Ridge Reservation (ORR) in Oak Ridge, Tennessee. The monitoring data were obtained from wells, springs, and surface water sampling locations in three hydrogeologic regimes at Y-12. The Bear Creek Hydrogeologic Regime (Bear Creek Regime) encompasses a section of Bear Creek Valley (BCV) between the west end of Y-12 and the west end of the Bear Creek Watershed (directions are in reference to the Y-12 grid system, shown as Plant North. The Upper East Fork Poplar Creek Hydrogeologic Regime (East Fork Regime) encompasses the Y-12 industrial facilities and support structures in BCV. The Chestnut Ridge Hydrogeologic Regime (Chestnut Ridge Regime) encompasses a section of Chestnut Ridge directly south of Y-12. Background information in Section 2 of this report outlines the hydrogeologic framework for groundwater and surface water quality monitoring at Y-12 and includes an overview of the groundwater contamination in each hydrogeologic regime. Section 3 provides details regarding the groundwater and surface water sampling and analysis activities implemented under the Y-12 GWPP, including sampling locations and frequency, sample collection and handling, field measurements and laboratory analytes, quality assurance (QA)/quality control (QC) sampling, data management, and data quality assessment (DQA). However, the equivalent QA/QC or DQA information for the groundwater and surface water data associated with the monitoring programs implemented by UCOR are not included in this report and instead are deferred to referenced programmatic plans and reports issued by OREM and UCOR. Section 4 of this report presents a summary evaluation of the CY 2019 monitoring data with regard to the respective objectives of surveillance monitoring and exit pathway/perimeter monitoring. The evaluation is based primarily on the analytical results for the following principal groundwater contaminants at Y-12: nitrate, uranium, gross alpha activity, gross beta activity, and volatile organic compounds (VOCs). Section 5 summarizes the most significant findings with respect to the principal contaminants along with recommendations for any proposed changes to the ongoing groundwater and surface water quality monitoring performed under the Y-12 GWPP. Technical reports and plans cited in the narrative sections of the report are listed in Section 6. Narrative sections of this report reference several appendices. Figures (maps and diagrams) and data tables (excluding data summary tables incorporated in the narrative sections) are in Appendix A and Appendix B, respectively. Appendix C contains construction details for each well sampled during CY 2019 by either the Y-12 GWPP or UCOR, along with schematic diagrams for wells equipped with Westbay™ multiport sampling equipment or Barcad® pump systems. Appendix D supports the background summary discussion in Section 2 and provides more detailed information about the hydrogeologic framework for groundwater and surface water monitoring at Y-12, including the primary sources of groundwater contamination in each hydrogeologic regime. Results for all field measurements and laboratory analyses obtained by the Y-12 GWPP and UCOR are presented in Appendix E, which also includes the sample numbers for the QA/QC samples associated with groundwater and surface water monitoring performed by the Y-12 GWPP.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

ASME Code Qualification Plan for LPBF 316 SS

This report describes a plan to qualify laser powder bed fusion (LPBF) 316 stainless steel for use with the American Society of Mechanical Engineers (ASME) Boiler & Pressure Vessel Code Section III, Division 5 rules for metallic components in high temperature nuclear reactors. Accomplishing this goal would make the material and manufacturing process available to vendors for inclusion in the next generation of advanced, high temperature reactors. The general approach adopted here is to treat LPBF 316 as if it was a completely new material and to develop a plan to qualify the material according to the current ASME practices. One key goal of this work is to explore and develop accelerated qualification approaches that might reduce the time required to qualify new materials by reducing the need for long term testing. However, the qualification plan here does not employ any accelerated qualification approaches to provide a limiting, bounding description of the number, duration, and types of testing required to qualify LPBF 316 without such techniques and to describe a comprehensive dataset that could be used to explore and validate accelerated qualification approaches in the future. The report addresses the fundamental challenges to qualifying Advanced Manufacturing (AM) materials for high temperature applications and summarizes the ASME Section III qualification process as well as current efforts to qualify LBPF and DED 316 for low temperature applications. The report then discusses specific issues, both material and logistical, related to qualifying PBF 316 steel. The final chapters of the report describe a complete test plan designed to generate sufficient data to qualify the material as well as a data management plan for how to store and manage the data to eventually provide the test data packaged needed to qualify the material with ASME.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

Cosmological neutrino mass: a frequentist overview in light of DESI

Supplementary material to DESI publication "Cosmological neutrino mass: a frequentist overview in light of DESI" to comply with the data management plan. Data points as well as parabola parameters for each profile likelihood figure are provided.

Chebat, Domitille [CEA Paris-Saclay] (ORCID:000900↗

DOE OSTI2022

Exploring MDSplus data-acquisition software and custom devices

MDSplus is a software tool designed for data acquisition, storage, and analysis of complex scientific experiments. Over the years, MDSplus has primarily been used for data management for fusion experiments. This paper demonstrates that MDSplus can be used for a much wider variety of systems and experiments. We present a step-by-step tutorial describing how to create a simple experiment, manage the data, and analyze it using MDSplus and Python. To this end, a custom example device was developed to be used as the data source. This device was built on an opensource electronic hardware platform, and it consists of a microcontroller and two sensors. We read data from these sensors, store it in MDSplus, and use JupyterLab to visualize and process it. This project and code demo are available on the GitHub site at this URL: https://github.com/santorofer/MDSplusAndCustomeDevices

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

A case study on parallel HDF5 dataset concatenation for high energy physics data analysis

In High Energy Physics (HEP), experimentalists generate large volumes of data that, when analyzed, helps us better understand the fundamental particles and their interactions. This data is often captured in many files of small size, creating a data management challenge for scientists. In order to better facilitate data management, transfer, and analysis on large scale platforms, it is advantageous to aggregate data further into a smaller number of larger files. However, this translation process can consume significant time and resources, and if performed incorrectly the resulting aggregated files can be inefficient for highly parallel access during analysis on large scale platforms. In this paper, we present our case study on parallel I/O strategies and HDF5 features for reducing data aggregation time, making effective use of compression, and ensuring efficient access to the resulting data during analysis at scale. We focus on NOvA detector data in this case study, a large-scale HEP experiment generating many terabytes of data. Here, the lessons learned from our case study inform the handling of similar datasets, thus expanding community knowledge related to this common data management task.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Exploring Legacy Document Management Systems to Increase Access to Project Data

The current Electronic Data Management System (EDMS) makes it difficult to efficiently access information in critical time frames. This project explored ways to allow Environmental Support and Services staff the ability to efficiently access documents with necessary historic information while leveraging existing data classification methods and the capabilities of EDMS.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2020

NDMAS System and Process Description

The U. S. Department of Energy (DOE) has made a significant investment in research to develop the next generation of reactor technologies as well as to improve the performance and lengthen the life cycle of existing nuclear reactors. Data collected to demonstrate new concepts may also be used in the future to support licensing of these technologies. Provenance of these data must be preserved. The Nuclear Data Management and Analysis System (NDMAS) was established to manage and preserve data collected by fuels and materials research conducted by the high-temperature, gas-cooled reactor program. The scope of NDMAS is expanding to include other nuclear research programs that have the shared need to preserve the provenance of research data. Nuclear research funded by DOE is conducted by Idaho National Laboratory (INL), universities, other national laboratories, foreign research partners, and private companies. This research will generate a large amount of data from a variety of sources over a period of many years. Managing the data generated by the research and development projects presents a significant challenge for retaining data integrity and availability.

+↗