Search NASA⌕ Search

SEARCH · Search NASA

Results for “Metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,603 records · Page 89
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Evaluation of Data Catalog Software for Hanford Site Environmental Datasets

Environmental information and data underpin achievement of the U.S. Department of Energy (DOE) Office of Environmental Management (EM) mission at the Hanford Site. The Hanford Environmental Data Management (HEDM) Program is the DOE Richland Operations Office (RL) approach to develop and implement a formal program for managing environmental data and the associated records, materials, and systems at the Hanford Site. The current project, contract, organization, and contractor-specific efforts at managing environmental data sets are insufficient to provide orderly, long-term, site-wide access. A vital element to be created within the HEDM program plan is a catalog of data sources, called the Hanford Environmental Information and Data Index (HEIDI), that will enable long-term access and retrievability for the multiple independent sources of data that might otherwise be difficult to discover. This report compares leading open source and commercial data catalog platforms using criteria to assess the functionality needed to develop the HEIDI catalog of Hanford data sources that connects and exchanges data with established Hanford Local Area Network (HLAN) enterprise information technology systems. Proprietary platforms evaluated included ArcGIS Enterprise Sites, Junar, OpenDataSoft, and Socrata, and non-proprietary platforms included Energy Data eXchange (EDX), Comprehensive Knowledge Archive Network (CKAN), and DKAN (a Drupal-based open data portal based on CKAN). Capabilities supporting data discoverability, retrieval, and archival, as well as metadata standard requirements and integration into the HLAN were rated as either failing to meet requirements (F), meeting requirements (M), or exceeding requirements by delivering additional desired features (E). The lowest rating for any capability area was assigned as the overall rating for the platform. These findings enable DOE-RL and the contractors implementing the HEDM plan to focus on candidate tools likely to meet the requirements for implementing HEIDI. All of the platforms receiving an overall rating of ‘F’ were unable to be deployed on Hanford infrastructure or within dedicated cloud resources. A propriety software-as-a-service (SaaS) model of delivering a data catalog (e.g., found in software such as Junar and OpenDataSoft) favors consistency across customers at the expense of customization and configurable roles that are needed for Hanford work. Hosting data on a shared commercial platform places limits on dataset size (maximum of 240 Mb for OpenDataSoft), a significant limitation for HEIDI implementation. EDX, a government data catalog based on CKAN, received the ‘F’ rating due to an inability to incorporate authentication from HLAN into the system. Among platforms rated ‘M’ or ‘E’, only the Socrata platform had a SaaS delivery model. In contrast to other SaaS platforms, Socrata provided custom roles and gateways that allow local datasets to be incorporated into an online catalog. Socrata also complies with the Federal Risk and Authorization Management Program, a significant benefit for cloud-based management of Hanford data. The other platforms rated ‘M’ or ‘E’, ArcGIS Enterprise Sites, CKAN, and DKAN, provide fully self-hosted options, allowing for greater control and flexibility with the HEIDI catalog. These widely used tools have supportive communities of practice, extensive customization options, and demonstrated deployments that provide evidence that they can meet requirements, often deliver additional desired features, and work well with federal government systems. Completely customized alternatives built on a collection of applications were not evaluated because achieving similar performance to CKAN or DKAN requires substantial resources, especially in the absence of the active communities that have grown to support these tools. ArcGIS Enterprise Sites, Socrata, CKAN, and DKAN were evaluated as strong candidates for successful implementation with HEIDI.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

November 2021 Operational Highlights

Completed: 1. the initial tests of document workflows for Titan on the Red, which included standing up a software encryption capability to test data ingestion; 2. a software encryption capability in order to ingest the first data source by the end of the month; 3. a stand-alone computer for classified ontology data entry. Titan on the Red is an artificial intelligence/machine learning system to make digitizing, cataloging, and searching NSRC collections easier and more efficient. Created Online Vault backups, with one copy stored at LANL and one shipped to Lawrence Livermore National Laboratory. The Online Vault is a classified, searchable library of LANL’s nuclear weapons design and test history. Imported new Laboratory Directed Research and Development (LDRD) documents based on revised access categories. Bulk ingested nearly 10,000 documents via java-based PowerLoader. This application ingests metadata and content into the Online Vault to meet the requirements for the NSRC collections. Rehoused 300 linear feet of weapons physics documents in archival, acid-free storage folders and boxes.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

Deliverable D11 – Data Sharing, Storage, Security Protocols, and a Specification of a Potential Data Sharing Portal

Pacific Northwest National Laboratory (PNNL) and Technical University of Denmark (DTU) completed this deliverable as part of Work Package 2: Data Information Catalog for Distributed Wind Research for the International Energy Agency (IEA) Wind Technology Collaboration Programme Task 41: Enabling Wind to Contribute to a Distributed Energy Future. As part of the work plan, Deliverable D11 requires the development of data sharing, storage, and security protocols for metadata to be stored on the platform, if needed. The specification of a potential data sharing portal that expands on the catalog is also required.

17 WIND ENERGY↗
DOE OSTI2022

January 2022 Operational Highlights

The Indexing Lab is now complete and includes classified and unclassified computers, safes and work stations. Staff capture metadata and catalog materials, including classified reports, directors’ correspondence and Rocky Flats materials to ensure the holdings are discoverable and retrievable. Access protocols were updated for products related to the LANL Annual Assessment in the Online Vault (OLV) classified repository. The Annual Assessment is a yearly analysis of the weapons systems in the nation’s nuclear stockpile. The Standardization Quality Assessment (SQA) for the Online Vault (OLV) classified repository was completed and contained no findings. Beta testing, or customer validation, began for the metrics database that generates metrics reports and visualizations from NSRC customer projects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Position Papers for the ASCR Workshop on the Management and Storage of Scientific Data

The purpose of this workshop is to identify priority research directions in the area of data management for high-performance and scientific computing above and beyond HPC’s traditional "the parallel file system is the data-management system" model. Supporting the breadth of the DOE mission, including the explosion of AI uses and the growing needs of experimental and observational science, motivates revisiting our assumptions about data management. There are many facets of this topic to explore including: (1) Interfaces for accessing data that resides on traditional persistent storage as well as memory devices; (2) Storage-system architecture design that supports scientific workflows on varied hierarchical storage and networking devices; (3) Devising metadata management infrastructure to support FAIR principles (Findability, Accessibility, Interoperability, and Reusability); (4) Capturing provenance information about scientific data; (5) Utilizing AI to learn I/O patterns of emerging workloads for efficient data management; (6) Providing data management support for AI and complex workflows; and (7) Understanding the overlap between traditional storage systems and I/O (SSIO) efforts and data management. While the program committee has identified these topics as important areas for discussion, we welcome position papers from the community that propose additional topics of interest for discussion at the workshop. The workshop agenda will include breakout sessions for discussing these and selected topic areas to inform priority research directions for data management for high-performance and scientific computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Report for the ASCR Workshop on the Management and Storage of Scientific Data

The purpose of this workshop is to identify priority research directions in the area of data management for high-performance and scientific computing above and beyond HPC’s traditional "the parallel file system is the data-management system" model. Supporting the breadth of the DOE mission, including the explosion of AI uses and the growing needs of experimental and observational science, motivates revisiting our assumptions about data management. There are many facets of this topic to explore including: (1) Interfaces for accessing data that resides on traditional persistent storage as well as memory devices; (2) Storage-system architecture design that supports scientific workflows on varied hierarchical storage and networking devices; (3) Devising metadata management infrastructure to support FAIR principles (Findability, Accessibility, Interoperability, and Reusability); (4) Capturing provenance information about scientific data; (5) Utilizing AI to learn I/O patterns of emerging workloads for efficient data management; (6) Providing data management support for AI and complex workflows; and (7) Understanding the overlap between traditional storage systems and I/O (SSIO) efforts and data management. While the program committee has identified these topics as important areas for discussion, we welcome position papers from the community that propose additional topics of interest for discussion at the workshop. The workshop agenda will include breakout sessions for discussing these and selected topic areas to inform priority research directions for data management for high-performance and scientific computing.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A High-Throughput Computing Infrastructure to Generate Custom, Open Community Geothermal Datasets

The most significant challenge facing geothermal research, development, and deployment is a lack of comprehensive datasets describing the geological and economical properties of North America. Automated knowledge base construction, the process of designing algorithms to analyze text and images to programmatically build new datasets, is one possible solution to this problem. The xDD library of full-text scientific articles (https://xdd.wisc.edu) is one of the largest collections of open and controlled-access scientific documents available for knowledge base construction in the world, but it has been underutilized by experts in geothermal research. The xDD development team attributed the lack of engagement by software developers and geothermal researchers to two perceived shortcomings of the system. First, the workflow for obtaining data from xDD for local development and testing of data mining applications was unnecessarily abstruse and required significant manual intervention by xDD systems administrators. Second, although xDD already held articles from a broad cross-section of scientific literature with an emphasis on the geosciences, it did not have an explicit set of geothermal research documents that could serve as the nucleus of a geothermal data mining application. To address these issues, the Automated Data Extraction PlaTform (ADEPT) was proposed to extend the data distribution capabilities of the xDD system. The ADEPT extension added the following four key features to xDD: 1) integration of National Geothermal Data System (NGDS) documents into the xDD library to provide an explicitly geothermally-themed collection; 2) improved RESTful (i.e., https-protocol driven) web services for external partners to access xDD data for machine learning application development; 3) a web platform for end-users and xDD administrators to coordinate the development of data mining applications from the initial step of browsing available documents to the final stage of deploying a production-quality machine learning application on high-throughput computing infrastructure; and 4) the development of demonstration data mining applications to illustrate the new workflow to potential collaborators. A total of 21,674 geothermal documents from NGDS were fully ingested into the xDD library and the associated metadata is publicly available through the xDD web services; furthermore, the ADEPT web platform is now publicly accessible and fully live at https://xdd.wisc.edu/adept/.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Data Report: TurboWave I and II Data Release

The TurboWave I and II infrasound campaigns were conducted to examine short term variability in acoustic propagation at local and regional distances. The tests were conducted in nearly co-located regions at the Energetic Materials Research and Testing Center in Socorro, NM between 2019 and 2020 and recorded across a variety of acoustic microbarometer sensors. This report details the waveform data recorded from the experiment and coincides with data archival at the Incorporated Research Institutions for Seismology. The report includes a description of the experiment along with the types of data and instruments. The data release includes raw waveform data as well as metadata information.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Waveform Data Quality Assessment

Healthy seismoacoustic sensors and accurate metadata are critical to all the science objectives of both the Source Physics Experiment (SPE) Rock Valley Direct Comparison (RV/DC) and LowYield Physics Experiment-1 (LYNM PE-1). Both projects share similar seismoacoustic stations. We provide a plan on how to move forward with waveform data quality assessments (QA) to the working groups. The identification of stations with instrument response errors will be useful to researchers using amplitudes to study source effects and Earth attenuation. The data QA can also be useful for field technicians to help identify and correct problem sites.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Analysis of Conservation Voltage Reduction under Inverter-Based VAR-Support [Slides]

Conservation voltage reduction (CVR) is a common technique used by utilities to strategically reduce demand during peak periods. As penetration levels of distributed generation (DG) continue to rise and advanced inverter capabilities become more common, it is unclear how the effectiveness of CVR will be impacted and how CVR interacts with advanced inverter functions. In this work, we investigated the mutual impacts of CVR and DG from photovoltaic (PV) systems (with and without autonomous Volt-VAR enabled). The analysis was conducted on an actual utility dataset, including a feeder model, measurement data from smart meters and intelligent reclosers, and metadata for more than 30 CVR events triggered by the utility over the year. The installed capacity of the modeled PV systems represented 66% of peak load, but reached instantaneous penetrations reached up to 2.5x the load consumption over the year. While the objectives of CVR and autonomous Volt-VAR are opposed to one another, this study found that their interactions were mostly inconsequential since the CVR events occurred when total PV output was low.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Utility-Scale Solar, 2022 Edition: Empirical Trends in Deployment, Technology, Cost, Performance, PPA Pricing, and Value in the United States [Slides]

Berkeley Lab’s “Utility-Scale Solar, 2022 Edition” provides an overview of key trends in the U.S. market, with a focus on 2021. Highlights of this year’s update include: -A record of nearly 12.5 GWAC of new utility-scale PV capacity came online in 2021, bringing cumulative installed capacity to more than 51.3 GWAC across 44 states. -90% of all new utility-scale PV capacity added in 2021 uses single-axis tracking. -Median installed project costs declined to $\$1.35$/WAC (or $\$1.02$/WDC) in 2021. -Project-level capacity factors vary widely, from 9% to 35% (on an AC basis), with a sample median of 24%. The report explores drivers of this variation. -Utility-scale PV’s LCOE fell to $\$33$/MWh in 2021 ($\$27$/MWh if factoring in the federal investment tax credit, or ITC). -PPA prices have largely followed the decline in solar’s LCOE over time, but have recently stagnated and even moved slightly higher. Prices from a sample of recent contracts average around $\$20$/MWh (levelized) in the West and $\$30-40$/MWh elsewhere in the continental US. In 2021, solar’s average market value (defined in the report to include only energy and capacity value) rose by 55% to $\$47$/MWh and exceeded average wholesale prices in 13 of the 17 balancing authorities analyzed. -Adding battery storage is one way to increase the value of solar. Our public data file tracks metadata and PPA prices from 67 PV+battery hybrid projects that are already online or that have secured offtake arrangements. -At the end of 2021, there were at least 674 GW of utility-scale solar power capacity within the interconnection queues across the nation, 284 GW of which include batteries. For more information, and to explore related interactive data visualizations, go to utilityscalesolar.lbl.gov.

14 SOLAR ENERGY↗
DOE OSTI2021

Waste Compliance and Tracking System (WCATS) Version 3 Requirements Document

This document describes the end-user requirements for the Waste Compliance and Tracking System (WCATS) project in accordance with the WCATS Software Quality Management Plan, EPC-WMP-WCATSPLAN-001. The WCATS application shall support the generation, characterization, processing, and shipment of LANL radioactive, hazardous, and industrial waste. Regulatory drivers include RCRA hazardous waste, DOT shipping, NNSA nuclear material control and accountability, DOE nuclear safety, TSDF permit, and transuranic waste certification requirements. The system will utilize a task-based architecture that supports the spectrum of treatment, storage, disposal, administrative, and characterization based unit operations necessary to manage waste from cradle to grave. The application design shall readily accommodate new facilities, processes, workflow, signature requirements, and so forth, via end-user established metadata. WCATS will provide support for representing waste storage and disposal facilities, buildings, rooms, and grid layouts (x, y, z) to support waste and radioactive material inventory management. Nuclear material at risk (MAR), DOE hazard rating (e.g., Category II facility) compliance per DOE-STD-1027, and permit inventory requirements will be configurable for any storage or disposal facility, or waste operation, and the system will automatically evaluate and enforce those requirements. In addition, the application will support the characterization and management of the entire range of hazardous and radioactive wastes (TRU, MTRU, LLW, MLLW, hazardous waste, etc.) that might be colocated or processed at a permitted facility. Some capabilities not found in traditional systems include user-defined tank systems for liquid waste, user-defined work paths (i.e., sequence of operations), and an equipment subsystem for tracking the calibration, maintenance, and inspection of tools used to process waste, such as torque wrenches, scales, pH probes, etc. The application incorporates a desktop and mobile user interface as shown in Figure 1. The mobile interface supports field operations, such as waste item characterization, intra-facility transfers, internal and external audits, and shipment preparation and receipt.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗
DOE OSTI2023

An Approach to Data Management Planning for Protected Data Projects

This report describes what is required in a data management plan for data that needs to be protected in some fashion. Here we provide an overview of what a data manager should consider, including the data ingestion and various extract-transform-load processes, metadata considerations and documentation, to what might need to be accounted for in the event of data loss. In addition, this report includes two appendices: forms that, when filled out, make the user compliant with DOE data management requirements as well as additional requirements for handling protected data at ORNL.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2023

Data Archive and Portal (DAP) Platform for Solid Phase Processing Technologies

The scale and speed of data generated by modern scientific experiments have constantly challenged the research community to store, curate, manage and optimally use it to drive scientific discoveries. In this work, we have developed a data archive and portal (DAP) platform including analytics capabilities to collect, curate, and manage data and metadata stream for solid phase processing (SPP) techniques. We successfully hosted around ~347K files of data related to processing parameters, microscopic images, and spectroscopic data related to solid phase processing. The DAP platform for SPP will establish an enduring capability to support machine learning and grow collaboration at the intersection of materials science and data science.

36 MATERIALS SCIENCE↗
DOE OSTI2023

ASC Data Science Infrastructure Project June 2023 [Slides]

Data Science Infrastructure (DSI) Project offers a standardized but flexible way of storing data and associated metadata to support AI/ML and data intensive scientific workflows: Establish requirements via use case interviews; Develop DSI framework to support flexible, secure, and accessible data stores tailored to researcher needs; Augment existing institutional frameworks for code management, performance testing and file indexing via automated DSI interfaces; Deploy DSI framework to support key use cases in collaboration with domain scientists.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

DSI Python API Demo May 2023 [Slides]

Data Science Infrastructure (DSI) is developing searchable databases and workflows for simulation and experimental data derived from ASC clients. Short term goal: Make data easily accessible through metadata indexing and querying while respecting data permissions. Longer term goal: Use this data for data science activities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Utility-Scale Solar, 2023 Edition: Empirical Trends in Deployment, Technology, Cost, Performance, PPA Pricing, and Value in the United States [Slides]

Berkeley Lab’s “Utility-Scale Solar, 2023 Edition” presents analysis of empirical plant-level data from the U.S. fleet of ground-mounted photovoltaic (PV), PV+battery, and concentrating solar-thermal power (CSP) plants with capacities exceeding 5 MWAC (PV plants of 5 MWAC or less, including residential rooftop systems, are covered separately in Berkeley Lab’s companion annual report, Tracking the Sun). Highlights of this year’s update include: -10.4 GWAC of new utility-scale PV capacity came online in 2022, bringing cumulative installed capacity to more than 61.7 GWAC across 46 states. -94% of all new utility-scale PV capacity added in 2022 uses single-axis tracking. -Median installed project costs declined to $\$1.32$/WAC (or $\$1.07$/WDC) in 2022. -Plant-level capacity factors vary widely, from 9% to 35% (on an AC basis), with a sample median of 24%. The report explores drivers of this variation. -Utility-scale PV’s LCOE fell to $\$39$/MWh in 2022 ($\$29$/MWh if factoring in the federal investment tax credit, or ITC). -PPA prices have largely followed the decline in solar’s LCOE over time, but have recently stagnated and even moved slightly higher. Prices from a sample of recent contracts average around $\$20-30$/MWh (levelized) in the West and $\$30-40$/MWh elsewhere in the continental US. -In 2022, solar’s average market value (defined in the report to include only energy and capacity value) rose by 40% to $\$71$/MWh and exceeded average wholesale prices in 4 of the 7 ISOs/RTOs and 11 of 18 other balancing authorities analyzed. -Adding battery storage is one way to increase the value of solar. Our public data file tracks metadata and PPA prices from ~100 PV+battery hybrid projects that are already online or that have secured offtake arrangements. -the end of 2022, there were at least 947 GW of utility-scale solar power capacity within the interconnection queues across the nation, 456 GW of which include batteries. For more information, and to explore related interactive data visualizations, go to utilityscalesolar.lbl.gov.

14 SOLAR ENERGY↗