SEARCH · Search NASA

Results for “Metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,351 records · Page 75

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Panel-Segmentation [SWR-21-18]

Panel-Segmentation contains the scripts for automated metadata extraction of solar PV installations, using satellite imagery coupled with computer vision techniques. In this package, the user can perform the following actions: *Automatically generate a satellite image using a set of lat-long coordinates, and a Google Maps API key. Users would need to set up a Google Cloud account and get a Maps Static API key. Please refer to Setting Up Google Maps Static API Key section for this process. *Perform image segmentation on the satellite image, to locate the solar array(s) in the image on a pixel-by-pixel basis, using an image segmentation model (panel_detection_model.pth). Get classification of the installation (rooftop, ground mounted fixed-tilt or tracking, carport, etc). *Perform azimuth estimation on each solar array cluster in the masked image. *Detect solar panels and get its latitude, longitude, and address within a geographic bounding box through the SOL-Searcher Pipeline. *Detect and calculate hurricane damage on solar installations given pre-hurricane and post-hurricane satellite imagery through the Hurricane Detection Pipeline. *Detect and calculate hail damage on solar installations given satellite imagery through the Hail Detection pipeline. *Convert NOAA MESH (Maximum Estimated Size of Hail) grib2 files into kml or geojson files. *Estimate tilt and azimuth of a solar array by processing USGS LiDAR data for the array’s location.

Edun, Ayobami↗

DOE OSTI2020

Laika BOSS (Binary Object Scanning System)

Laika BOSS (Binary Object Scanning System), often written as "Laikaboss", is an object scanner and intrusion detection system that strives to be scalable, flexible, and verbose. Its goal is to take arbitrary file input and extract all relevant subfiles and metadata for use in helping analysts determine the function and maliciousness of a file at a glance. This is an extension to the original code developed by Lockheed Martin.

Walkup, Gregory↗

DOE OSTI2018

AESD (Advanced Energy System Design) Records API [SWRs-17-50, 17-51, 17-52]

The Records API (application program interface) for Advanced Energy System Design (AESD) enables software that serves multidimensional record-oriented data to interoperate with software than uses such data. In the context of the Records API, multidimensional data records are simply tuples of real numbers, integers, and character strings, where each data value is tagged by a variable name, according to a pre-defined schema, and each record is assigned a unique integer identifier. Conceptually, these records are isomorphic to rows in a relational database, JSON objects, or key-value maps. Records servers might supply static datasets, sensor measurements that periodically update as new telemetry become available, or the results of simulations as the simulations generate new output. Records client software might display or analyze the data, but in the case of simulations the client request the creation of new ensembles for specified input parameters. It is also possible to chain records clients and servers together so that a client consuming data from a server might transform that data and serve it to additional clients. This minimalist API avoids imposing burdensome metadata, structural, or implementation requirements on developers by relying on open-source technologies that are readily available for common programming languages. In particular, the API has been designed to place the least possible burden on services that provide data. This document defines the message format for the Records API, a transport mechanism for communicating the data, and the semantics for interpreting it. The message format is specified as Google Protocol Buffers and the transport mechanism uses WebSockets. We discuss three major use cases for serving and consuming records data: (i) static data, (ii) dynamically augmented data, (iii) on-demand simulations, (iv) with filters, and (v) with bookmarks. Separate implementations of the API exist in C++, Haskell, JavaScript, Python, and R.

Bush, Brian↗

DOE OSTI2012

MyEMSL

MyEMSL provides the following features: user-assisted upload of data from scientific instruments to an archive, tracking data though the upload process, providing metadata for recovery of uploaded data, providing users with access to the data. These features are supplied by a mixture of open source software and internally developed software. MyESML also provides a notification system that allows the system to communicate with users and other parts of the MyEMSL system. This allows the system to support automated workflows and user interaction. Reprocessing of already ingested data is also supported.

Glass, Kevin↗

DOE OSTI2021

battery-data-toolkit

This software package contains utilities for ingesting battery cycling data from various equipment and converting it to a common file format including metadata. Further development of the package will include utilities for further processing the data for a variety of machine learning tasks.

PAULSON, NOAHHARRIS↗

DOE OSTI2021

Highly Extensible X-ray Diffraction Toolkit

This code has been previous released as open source under LLNL-CODE-529294. The HEXRD program comprises a library of X-ray diffraction analysis tools and a stand-alone GUI application. While there are many software packages that provide similar capabilities, HEXRD was designed to provide an extremely flexible and extensible basis for describing a broad array of X-ray diffraction instruments; this includes both mono- and poly-chromatic modalities as well as poly- and single-crystal samples. The high-level objects include Material: an abstraction for defining crystalline materials, including unit cell parameters and anisotropic elastic moduli Imageseries: an interface for series of diffraction images containing one or more frames, including provisions for metadata. Instrument: an abstraction of an X-ray diffraction instrument including support for the definition of multiple detector elements as well as powder, Laue, and mono-chromatic single/multi-grain diffraction analysis modalities (including attendant calibration routines). The hexrdgui package is a standalone PySide2/Qt5 GUI built on top of hexrd. The 0.8.1 release includes various bug fixes as well as some new functionality in the GUI, including: o Enhancements to the import tools for LLNL diffraction instruemtns (PXRDIP and TARDIS) o The ability to save 1-d LeBail fits of integrated diffraction spectra.

Barton, NathanR.↗

DOE OSTI2022

VizBrick

Brick (https://brickschema.org/) is a unified metadata schema to address the problem of building data standardization. Creating Brick models for building datasets means that the contents of the datasets are semantically described using the standard terms defined in the Brick ontology, and it will enable the benefits of data standardization, without having to recollect or reorganize the data. The challenge is that building brick models for building datasets leads to repeated manual trial and error processes, which can be time-consuming. VizBrick is a tool with a graphic/Web-based user interface that can assist users to create Brick models visually and interactively without having to understand the Resource Description Framework (RDF) syntax. VizBrick contains a web server that renders VizBrick web interface pages for browsers. The web server utilizes software components that (1) provide Brick ontology entity mapping to data column suggestions to users so that they can efficiently create their model; (2) provide keyword/Metadata-based search capability for easy find of relevant brick concepts and relations to their data columns

Lee, Sangkeun [Oak Ridge National Lab. (ORNL), Oak↗

DOE OSTI2016

Cyber-Physical Security Assessment (CyPSA) Toolset

CyPSA seeks to organize and gain insight into the diverse sets of data that a critical infrastructure provider must manage. Specifically CyPSA inventories, manages, and analyzes assets and relations among those assets. A variety of interfaces are provided. CyPSA inventories assets (both cyber and physical). This may include the cataloging of assets through a common interface. Data sources used to generate a catalogue of assets include PowerWorld, NPView, NMap Scans, and device configurations. Depending upon the role of the person using the tool the types of assets accessed as well as the data sources through which asset information is accessed may vary. CyPSA allows practitioners to catalogue relations among assets and these may either be manually or programmatically generated. For example, some common relations among assets include the following: Topological Network Data: Which devices and assets are connected and how? Data sources for this kind of information include NMap scans, NPView topologies (via Firewall rule analysis). Security Metrics Outputs: The output of various security metrics such as overall exposure. Configure Assets:CyPSA may eventually include the ability to configure assets including relays and switches. For example, a system administrator would be able to configure and alter the state of a relay via the CyPSA interface. Annotate Assets: CyPSA also allows practitioners to manually and programmatically annotate assets. Sources of information with which to annotate assets include provenance metadata regarding the data source from which the asset was loaded, vulnerability information from vulnerability databases, configuration information, and the output of an analysis in general.

Garcia, Luis↗

DOE OSTI2023

Data Science Infrastructure SOFTWARE

LANL science workflows generate complex data sets and ensembles of data requiring significant compute and storage resources. The Data Science Infrastructure (DSI) project focuses on data-driven approaches to make data more readily available to LANL projects. DSI workflows leverage metadata stored in data-agnostic databases, supported by an abstraction layer API to simplify searching and accessing data across simulation runs, experimental runs, filesystems and environments. The abstraction layer API allows the user to query a range of data types: raw output, processed data, configuration data, machine learning models, performance data, etc. In addition to the abstraction backend API, the DSI project is developing client-driven query APIs and UIs to support specific user workflows.

Turton, Terece↗

DOE OSTI2023

Earthdata Pub

Earthdata Pub is a toolset that provides forms and workflows so that a data producer can request archival of their NASA-funded data, NASA can determine which (if any) DAAC is appropriate to that data, and the selected DAAC can work with the data producer to get the data and metadata necessary for publication.

Broughton, Kimberly↗

DOE OSTI2024

MAPSTER

MAPSTER Observer is a light-weight software package that can be installed on field deployed computers which automatically manages data and creates metadata packages designed to organize large datasets, making them easy to retrieve and share.

Larson, MatthewD↗

DOE OSTI2024

Ingest

Ingest is an application that allows project owners to solicit data from users and require them to fill out metadata along with those uploads. It is an application developed in Elixir/Phoenix and is a web based platform.

Darrington, JohnW.↗

DOE OSTI2023

CEDS Differential Privacy (CEDSDP) v0.1

A Python package that provides differentially private queries optimized for energy systems' data. It may be used to publish queries such as clustering, averaging, metadata inference, etc. that are useful for a variety of grid-related analytics, including cyberattack detection.

Peisert, Sean↗

DOE OSTI2024

SysCaps (Language Interfaces for Simulation Surrogates of Complex Systems) [SWR-24-97]

You've found the official code repository for the paper "SysCaps: Language Interfaces for Simulation Surrogates of Complex Systems," presented at the Foundation Models for Science: Progress, Opportunities, and Challenges workshop at NeurIPS 2024. Our paper conjectures that interfaces (both text templates as well as conversational) makes interacting with simulation surrogate models for complex systems more intuitive and accessible for both non-experts and experts. "System captions", or SysCaps, are text-based descriptions of systems based on information contained in simulation metadata. Our paper's goal is to train multimodal regression models that take text inputs (SysCaps) and timeseries inputs (exogenous system conditions such as hourly weather) and regress timeseries simulation outputs (e.g. hourly building energy consumption). The experiments in our paper with building and wind farm simulators, which can be reproduced using this codebase, aim to help us understand whether a) accurate regression in this setting is possible and b) if so, how well can we do it. Paper: https://arxiv.org/abs/2405.19653

Emami, Patrick↗

DOE OSTI2024

pnnl/graph_merge

A novel method for pairwise graph comparison that includes both the graph structure and applicable metadata. The resulting visual component allows a greater and more interpretable understanding of the comparison than current methods

Dennis, Douglas [Pacific Northwest National Labora↗

DOE OSTI2025

OpenSAMPL

OpenSAMPL (Open Synchronization Analytics and Monitoring PLatform) is a Python framework for processing, loading, and managing clock probe time series data and metadata from different vendors.

Grant, Josh [Oak Ridge National Laboratory (ORNL),↗

DOE OSTI2025

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗

DOE OSTI2025

GenomeDepot v1.0

GenomeDepot is a web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of web-sites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, BLAST search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools.

Kazakov, Alexey [Lawrence Berkeley National Labora↗