Search NASA⌕ Search

SEARCH · Search NASA

Results for “system metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Constraints on Future Analysis Metadata Systems in High Energy Physics

In high energy physics (HEP), analysis metadata comes in many forms—from theoretical cross-sections, to calibration corrections, to details about file processing. Correctly applying metadata is a crucial and often time-consuming step in an analysis, but designing analysis metadata systems has historically received little direct attention. Among other considerations, an ideal metadata tool should be easy to use by new analysers, should scale to large data volumes and diverse processing paradigms, and should enable future analysis reinterpretation. This document, which is the product of community discussions organised by the HEP Software Foundation, categorises types of metadata by scope and format and gives examples of current metadata solutions. Important design considerations for metadata systems, including sociological factors, analysis preservation efforts, and technical factors, are discussed. A list of best practices and technical requirements for future analysis metadata systems is presented. These best practices could guide the development of a future cross-experimental effort for analysis metadata tools.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Best Practices for Energy Management Information Systems Metadata Schemas

Fact sheet highlights best practices for metadata schemas and standard naming conventions, which improve the ability of an energy management information system (EMIS) to consistently analyze, visualize, and derive value from operational data. This best practice document is part of a series of fact sheets created to help accelerate the market adoption and use of EMIS in the federal sector. It provides an overview of best practices for using metadata in an EMIS stack, which can significantly reduce the initial time required to deploy an EMIS.

data tags↗
DOE OSTI2022

Data Catalog Project - A Browsable, Searchable, Metadata System

Modern experiments are typically conducted by large, extended, where researchers rely on other team members to produce much of the data they use. The experiments record very large numbers of measurements which can be difficult for users to find, access and understand. We are developing a system for users to annotate their data products with structured metadata, providing data consumers with a discoverable, browsable data index. Machine understandable metadata captures the underlying semantics of the recorded data, which can then be consumed by both programs, and interactively by users. Collaborators can use these metadata to select and understand recorded measurements.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2016

International Metadata Standards and Enterprise Data Quality Metadata Systems

Well-documented data quality is critical in situations where scientists and decision-makers need to combine multiple datasets from different disciplines and collection systems to address scientific questions or difficult decisions. Standardized data quality metadata could be very helpful in these situations. Many efforts at developing data quality standards falter because of the diversity of approaches to measuring and reporting data quality. The one size fits all paradigm does not generally work well in this situation. I will describe these and other capabilities of ISO 19157 with examples of how they are being used to describe data quality across the NASA EOS Enterprise and also compare these approaches with other standards.

data quality↗
DOE OSTI2022

Best Practices for Energy Management Information Systems Metadata Schemas

The Federal Energy Management Program (FEMP) promotes best practices for impactful utilization of Energy Management Information Systems (EMIS) at federal facilities. This best practice document is part of a series of fact sheets created to help accelerate the market adoption and use of EMIS in the federal sector. The use of standard naming conventions and a metadata schema, which may be referred to as 'data tags,' improves the ability of the EMIS to consistently analyze, visualize, and derive value from operational data. To help address these issues, this fact sheet highlights best practices for use of metadata in an EMIS stack, which can significantly reduce the initial time required to deploy an EMIS.

EMIS↗
NASA NTRS2007

Master Metadata Repository and Metadata-Management System

A master metadata repository (MMR) software system manages the storage and searching of metadata pertaining to data from national and international satellite sources of the Global Ocean Data Assimilation Experiment (GODAE) High Resolution Sea Surface Temperature Pilot Project [GHRSSTPP]. These sources produce a total of hundreds of data files daily, each file classified as one of more than ten data products representing global sea-surface temperatures. The MMR is a relational database wherein the metadata are divided into granulelevel records [denoted file records (FRs)] for individual satellite files and collection-level records [denoted data set descriptions (DSDs)] that describe metadata common to all the files from a specific data product. FRs and DSDs adhere to the NASA Directory Interchange Format (DIF). The FRs and DSDs are contained in separate subdatabases linked by a common field. The MMR is configured in MySQL database software with custom Practical Extraction and Reporting Language (PERL) programs to validate and ingest the metadata records. The database contents are converted into the Federal Geographic Data Committee (FGDC) standard format by use of the Extensible Markup Language (XML). A Web interface enables users to search for availability of data from all sources.

Armstrong, Edward↗
NASA NTRS2001

Syntactic and Semantic Validation without a Metadata Management System

The ability to maintain quality information is essential to securing the confidence in any system for which the information serves as a data source. NASA's Global Change Master Directory (GCMD), an online Earth science data locator, holds over 9000 data set descriptions and is in a constant state of flux as metadata are created and updated on a daily basis. In such a system, the importance of maintaining the consistency and integrity of these-metadata is crucial. The GCMD has developed a metadata management system utilizing XML, controlled vocabulary, and Java technologies to ensure the metadata not only adhere to valid syntax, but also exhibit proper semantics.

Pollack, Janine↗
DOE OSTI2021

Scalable augmented enumeration and metadata operations for large filesystems

Systems, apparatus, and methods are disclosed for performing scalable operations in a file system. Metadata entries in a namespace or directory tree are sharded across multiple file metadata servers. An augmented enumeration operation, such as listing a directory, is parallelized across the multiple file metadata servers, transparently to clients. Exemplary augmentation features can include filtering and sorting. Augmentation features can be executed concurrently with enumeration, prior to enumeration, after enumeration, or as a combination of these, and can utilize pre-built index structures or holding structures for intermediate results. Augmented enumeration operations can also include no-output operations such as changing file attributes or deleting a file, and cumulative operations such as counting total disk space usage. The parallelization is compatible with tree-level parallelization and storage-level parallelization. Disclosed technologies can be applied to other fields requiring scalable enumeration, such as database and network applications.

Grider, Gary A.↗
DOE OSTI2023

"PoliMOR: A Policy Engine \"Made-to-Order\" for Automated and Scalable Data Management in Lustre"

Modern supercomputing systems are increasingly reliant on hierarchical, multi-tiered file and storage system architectures due to cost-performance-capacity trade-offs. Within such multi-tiered systems, data management services are required to maintain healthy utilization, performance, and capacity levels. We present PoliMOR, a pragmatic and reliable policy-driven data management framework. PoliMOR is composed of modular, single-purpose agents that gather file system metadata and enforce policies on storage systems. PoliMOR facilitates automated and scalable data management with customizable agents tailored to HPC facility-specific storage systems and policies. Our evaluations demonstrate the scalability and performance of PoliMOR both by its individual agents and as a collective entity. We believe PoliMOR is widely applicable across HPC facilities with large-scale data management challenges and will garner interest from the HPC community, given its flexible and open-source nature.

George, Anjus↗
NASA NTRS1999

The Role of Metadata Standards in EOSDIS Search and Retrieval Applications

Metadata standards play a critical role in data search and retrieval systems. Metadata tie software to data so the data can be processed, stored, searched, retrieved and distributed. Without metadata these actions are not possible. The process of populating metadata to describe science data is an important service to the end user community so that a user who is unfamiliar with the data, can easily find and learn about a particular dataset before an order decision is made. Once a good set of standards are in place, the accuracy with which data search can be performed depends on the degree to which metadata standards are adhered during product definition. NASA's Earth Observing System Data and Information System (EOSDIS) provides examples of how metadata standards are used in data search and retrieval.

Pfister, Robin↗
DOE OSTI2021

Photovoltaic Data Acquisition (PVDAQ) Public Datasets

The NREL PVDAQ is a large-scale time-series database containing system metadata and performance data from a variety of experimental PV sites and commercial public PV sites. The datasets are used to perform on-going performance and degradation analysis. Some of the sets can exhibit common elements that effect PV performance (e.g. soiling). The dataset consists of a series of files devoted to each of the systems and an associated set of metadata information that explains details about the system hardware and the site geo-location. Some system datasets also include environmental sensors that cover irradiance, temperatures, wind speeds, and precipitation at the site.

Array↗
DOE OSTI2022

Automated Metadata Extraction: Challenges and Opportunities

Proper application of the FAIR data principles is what separates a vibrant data ecosystem, in which research data are frequently shared and reused, from a lifeless data graveyard. Automated metadata extraction systems have been proposed as a means of bolstering the findability, interoperability, and reusabil- ity of data repositories with little or no human intervention. These extraction systems mine metadata by crawling a repository and applying lightweight extractors that, for various types of file (e.g., image, CSV file), extract or synthesize relevant attributes. In practice, however, the automated creation of generally useful metadata is fraught with challenges. Data consumers may have different perspectives as to what metadata representations are useful, the standards for recording metadata tend to change over time, and the software model for processing updates can introduce unnecessary human and computational effort. Thus, generalizing extraction for a broad audience of data consumers is a difficult and relatively unsolved problem.In this work, we explore these challenges faced by extraction systems in the context of constructing our own extraction system for science data. We first define the metadata extraction problem and provide context to the issues faced in generalizing metadata. Additionally, we identify potential research directions to help alleviate many of these challenges for all automated extraction systems. Ultimately, this work represents a first step in designing ubiquitous metadata extraction systems that can maximize the value of research data while minimizing the human efforts required in doing so.

Skluzacek, Tyler↗
NASA NTRS2017

Metadata Evaluation and Improvement: Evolving Analysis and Reporting

ESIP Community members create and manage a large collection of environmental datasets that span multiple decades, the entire globe, and many parts of the solar system. Metadata are critical for discovering, accessing, using and understanding these data effectively and ESIP community members have successfully created large collections of metadata describing these data. As part of the White House Big Earth Data Initiative (BEDI), ESDIS has developed a suite of tools for evaluating these metadata in native dialects with respect to recommendations from many organizations. We will describe those tools and demonstrate evolving techniques for sharing results with data providers.

metadata recommendations↗
DOE OSTI2023

The fast camera (Fastcam) imaging diagnostic systems on the DIII-D tokamak

Two camera systems are installed on the DIII-D tokamak at the toroidal positions of 90° (90° system) and 225° (225° system), respectively. The cameras have two types of relay optics, namely, a coherent optical fiber bundle and a periscope system. The periscope system provides absolute intensity calibration stability while sacrificing resolution (10 lp/mm), while the fiber system provides high resolution (16 lp/mm) while sacrificing calibration stability. The periscope is available only for the 90° system. The optics of the 225° system were designed for view stability, repeatability, and easy maintenance. The cameras are located inside optimized neutron, x ray and magnetic shielding in order to reduce electronics damage, reboots, and magnetic and neutron interference, increasing the overall system reliability. An automated filter wheel, providing remote filter change, allows for remote wavelength selection. A software suite automates camera acquisition and data storage, allowing for remote operation and reduced operator involvement. System metadata is used to streamline the data analysis workflow, particularly for intensity calibration. Here, the spatial calibration uses multiple observable wall features, resulting in a reconstruction accuracy ≤2 cm.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

PVInsight (Final Technical Report)

Data generated from real-world photovoltaic (PV) systems represent significant opportunities for the industry–from digital operations and maintenance to real-time planning and forecasting. However, these data also come with substantial, unique challenges. A particular challenge is the analysis of unlabeled PV performance data, which we define as time-series measurements of real power production (or sometimes current or voltage) that do not have corresponding meteorological measurements (irradiance, temperature, etc.) or system configuration information (sometimes called system metadata). These challenges are amplified in the distributed rooftop sector, in which data quality, completeness, and metadata can be very poor.

14 SOLAR ENERGY↗
NASA NTRS2020

GGOS Bureau of Networks and Observations: Network Infrastructure and Related Activities

The GGOS Bureau of Networks and Observations works with the IAG Services (IVS, ILRS, IGS, IDS, IGFS, and PSMSL) to advocate for the expansion and upgrade of space geodesy networks for the maintenance and improvement of the reference frame and other applications, as well as for the integration with other techniques, including absolute gravity and sea level measurements from tide gauges. New sites are being established following the GGOS concept of “core” and co-location sites, and new technologies are being implemented to enhance performance in data yield as well as accuracy. The Bureau continues to meet with organizations to discuss possibilities, including partnerships, for new and expanded participation. The GGOS Network continues to grow as new stations join every year. The Bureau holds meetings frequently, providing the opportunity for representatives from the services to meet and share progress and plans, and to discuss issues of common interest. It also monitors the status and projects the evolution of the network based on information from the current and expected future participants. Of particular interest at the moment is the integration of gravity and tide gauge networks and the forthcoming establishment of the new absolute gravity reference frame. The IAG Committees and Joint Working Groups play an essential role in the Bureau activity. The Standing Committee on Performance Simulations and Architectural Trade-offs (PLATO) uses simulation and analysis techniques to project future network capability and to examine trade-off options. The Committee on Data and Information is working on a strategy for a GGOS metadata system for data products and a more comprehensive long-term plan for an all-inclusive system. The Committee on Satellite Missions is working to enhance communication with the space missions, to advocate for missions that support GGOS goals and to enhance ground systems support. The IERS Working Group on Site Survey and Co-location (also participating in the Bureau) is working to enhance standardization in procedures, outreach and to encourage new survey groups to participate and improve procedures to determine systems’ reference points, a crucial aid in the detection of technique-specific systematic errors. We will give a brief update on the status and projection of the network infrastructure of the next several years, and the progress and plans of the Committees/Working Group in their critical role in enhancing data product quality and accessibility to the users.

Carey Noll↗
DOE OSTI2022

Improved CdTe PLR Estimates: Self-Shading and Spectral Mismatch

The RdTools year-on-year method of estimating performance loss rate (PLR) employs a simple normalization to remove the confounding effect of irradiance and temperature variation. However, the normalization's assumption that PV production scales linearly with in-plane broadband irradiance is a worse approximation for CdTe and other technologies with larger spectral sensitivities than it is for the more common c-Si technology. Additionally, CdTe systems using single-axis trackers (-20% of installed US utility-scale capacity) are subject to self-shading in the morning and afternoon, introducing another nonlinearity between PV output and broadband irradiance. Ignoring these effects may subject the estimated PLR to increased uncertainty, and perhaps bias, depending on the character of their short- and long-term variability. In this work we show that including self-shading and spectral mismatch models in the normalization for tracking CdTe systems can result not only in tighter PLR confidence intervals but different median PLRs as well. The shading and spectral models are kept simple to maintain consistency with the RdTools ethos of not requiring detailed system metadata or unusual measurements.

CdTe↗
DOE OSTI2024

Quantifying Error in Photovoltaic Installation Metadata: Preprint

In this research, we quantify the level of metadata error for a fleet of 2860 photovoltaic (PV) systems, using metadata values provided by fleet owners. Using satellite imagery and time series analysis techniques available in open-source Python packages Panel-Segmentation and PVAnalytics, respectively, we evaluate the accuracy of PV system metadata such as location, azimuth, tilt, and mounting configuration (fixed tilt vs. tracking). We find that approximately 75% of provided latitude-longitude coordinates are within 190 meters of the actual solar installation. We were unable to link 7.8% of latitude-longitude coordinates to any solar installation via satellite imagery analysis. We evaluate the level of error in owner-provided mounting configuration (fixed tilt vs. single-axis tracking), finding only 8 systems with an incorrect mounting configuration. When evaluating azimuth and tilt parameters, we find that approximately 64% of the data is correct, with data for 860 systems (approximately 30%) not provided by system owners. To illustrate the importance of having correct solar metadata, we evaluate how incorrect metadata affects solar performance estimates by modeling system AC energy output at ground-truth vs. incorrect latitude-longitude coordinates, mounting configurations, and azimuth-tilt configurations. Energy output estimates can vary significantly if incorrect metadata parameters are used, with incorrect mounting configuration leading to the largest discrepancy with over 20% variation in expected energy output.

azimuth↗