Search NASA⌕ Search

SEARCH · Search NASA

Results for “datasets metadata”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Genomes OnLine Database (GOLD) v.8: overview and updates

The Genomes OnLine Database (GOLD) (https://gold.jgi.doe.gov/) is a manually curated, daily updated collection of genome projects and their metadata accumulated from around the world. The current version of the database includes over 1.17 million entries organized broadly into Studies (45 770), Organisms (387 382) or Biosamples (101 207), Sequencing Projects (355 364) and Analysis Projects (283 481). These four levels contain over 600 metadata fields, which includes 76 controlled vocabulary (CV) tables containing 3873 terms. GOLD provides an interactive web user interface for browsing and searching by a wide range of project and metadata fields. Users can enter details about their own projects in GOLD, which acts as a gatekeeper to ensure that metadata is accurately documented before submitting sequence information to the Integrated Microbial Genomes (IMG) system for analysis. In order to maintain a reference dataset for use by members of the scientific community, GOLD also imports projects from public repositories such as GenBank and SRA. Here, the current status of the database, along with recent updates and improvements are described in this manuscript.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

EAGLE-I Power Outage Data 2024

The provided EAGLE-I historic dataset includes power outage information at the county level for 2024 at 15-minute intervals collected by the EAGLE-I program at ORNL. The data has been collected from utility's public outage maps using an ETL process. The dataset details FIPS code, county name, state name, total number of customers without power, total customers per county, and a date/timestamp. For detailed metadata, refer to the linked metadata DOI.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Hyporheic zone, river, and groundwater metagenome resolved genomes and rpS3 genes in East River Watershed, Colorado USA Summer 2020, 2021

Here we present metagenome assembled genomes (MAGs) for the bacterial and archaeal communities from water filter collected across 8 locations along the East River Watershed, CO, and 1 nearby groundwater well. The purpose was to look for connectivity and similarities across the network and to see the impact of the groundwater. As a part of Lawrence Berkeley National Laboratory (LBNL) Watershed Science Focus Area (SFA), we assessed community composition and strain similarities between the sites and we also compared it to previous metagenomic studies within the watershed looking at floodplain (Matheus Carnevali et al. 2021) and hillslope (Lavy et al. 2019) microbiomes. Here we present metagenome assembled genomes (MAGs) for the bacterial and archaeal communities from filters across 8 locations during August 2020 and July 2021. This resulted in 32 samples. The groundwater sample was sequenced at UC Berkley's QB3. The other 31 samples were sequenced at University of Maryland. Metagenomes were assembled using four autobinners and the best bins were selected using dasTool. The genomes were dereplicated at 95% with dRep and the subset of winning genomes were manually curated based on visual inspection of taxonomic profile, GC content, coverage, and a set of 51 bacterial single copy genes (BSCG), and 38 archaeal signal copy genes (ASCG). The dataset includes a zip file of 311 genomes (HZ_River_SW_MAGS_Dereplicated_95.zip). The dataset additionally includes a zipped file of ribosomal protein small subunit 3 (rpS3) proteins from the hyporheic zone and river data (rpS3_Proteins_HZ_River.zip), a metadata file used to register associated samples with IGSNs (International Generic Sample Numbers) (samples.csv), a location metadata file (locations.csv). This work was supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

DNA↗
DOE OSTI2021

Supplemental Data for the Manuscript: Quantification of Manganese for ChemCam Mars and Laboratory Spectra Using a Multivariate Model

This dataset includes all of the data needed to validate and/or reproduce the manganese calibration model described in the manuscript. The reference database contains metadata for the new Mn-bearing standards, minerals, and mixtures that are > 2.9 wt.% MnO. In addition, the files include the MnO composition data for all standards used, non-normalized spectral data, mean peak area spectrum, results of outlier determination, RMSECV data, regression vectors, and Test Set predictions.

58 GEOSCIENCES↗
DOE OSTI2025

Legacy Survey of Space and Time Data Preview 1: CcdVisit searchable catalog

The Legacy Survey of Space and Time Data Preview 1 (DP1) is the first release of data from the NSF-DOE Vera C. Rubin Observatory. It consists of raw and calibrated single-epoch images, co-adds, difference images, detection catalogs, and other derived data products. DP1 is based on 1792 science-grade optical/near-infrared exposures acquired over 48 distinct nights by the Rubin Commissioning Camera, LSSTComCam, on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile during the first on-sky commissioning campaign in late 2024. DP1 covers a total of approximately 15 sq. deg. over seven roughly equally-sized non-contiguous fields, each independently observed in six broad photometric bands, ugrizy, spanning a range of stellar densities and latitudes and overlapping with external reference datasets. This dataset is a subset of the full data release consisting of a searchable catalog named CcdVisit. This catalog contains per-detector visit metadata. This catalog contains 16,071 rows with 50 columns.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Legacy Survey of Space and Time Data Preview 2: VisitDetector searchable catalog

We present Rubin Data Preview 2 (DP2), the second data preview from the NDF-DOE Vera C. Rubin Observatory. Data Preview 2 (DP2) comprises coadds, detection catalogs, and ancillary data products; and when fully released will also include single-epoch images and difference images. DP2 is derived from observations acquired by the LSST Science Camera (LSSTCam) on the Simonyi Survey Telescope at the Summit Facility on Cerro Pachón, Chile, primarily during the on-sky commissioning campaign between 2025-04-16 and 2025-09-21, supplemented by observations taken between 2025-10-25 and 2026-01-06 that overlap the commissioning footprint. The DP2 footprint comprises the Science Validation wide-area survey, five Deep Drilling Fields, and a number of targeted small-field regions, including Trifid and Lagoon, Prawn, M49, and New Horizons, all observed as part of the Rubin First Look campaign. Each field was imaged in up to six broad photometric bands, ugrizy, and coadded to produce deep imaging covering an estimated 3,000 deg2. The addition of single-visit-only areas expands the total DP2 footprint to an estimated 15,000 deg2, with coverage in at least one filter. The median per-visit PSF FWHM across the wide-area survey ranges from 1.17 arcsec in the z band to 1.26 arcsec in g and r bands. The deepest field, reaches estimated coadded 5σ depths of u=26 mag, g=26.8 mag, r=26.3 mag, i=26.1 mag, z=25.3 mag, y=23.9 mag. Based on a roughly five-month primary observing baseline and covering only part of the eventual LSST footprint, DP2's area, depth, and multiband coverage nonetheless support a broad range of early science investigations ahead of LSST Data Release This dataset is a subset of the full data release consisting of a searchable catalog named VisitDetector. This catalog contains per-detector visit metadata. This catalog contains 5,150,198 rows with 56 columns.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2008

Standardizing Interfaces for External Access to Data and Processing for the NASA Ozone Product Evaluation and Test Element (PEATE)

NASA's traditional science data processing systems have focused on specific missions, and providing data access, processing and services to the funded science teams of those specific missions. Recently NASA has been modifying this stance, changing the focus from Missions to Measurements. Where a specific Mission has a discrete beginning and end, the Measurement considers long term data continuity across multiple missions. Total Column Ozone, a critical measurement of atmospheric composition, has been monitored for'decades on a series of Total Ozone Mapping Spectrometer (TOMS) instruments. Some important European missions also monitor ozone, including the Global Ozone Monitoring Experiment (GOME) and SCIAMACHY. With the U.S.IEuropean cooperative launch of the Dutch Ozone Monitoring Instrument (OMI) on NASA Aura satellite, and the GOME-2 instrumental on MetOp, the ozone monitoring record has been further extended. In conjunction with the U.S. Department of Defense (DoD) and the National Oceanic and Atmospheric Administration (NOAA), NASA is now preparing to evaluate data and algorithms for the next generation Ozone Mapping and Profiler Suite (OMPS) which will launch on the National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory Project (NPP) in 2010. NASA is constructing the Science Data Segment (SDS) which is comprised of several elements to evaluate the various NPP data products and algorithms. The NPP SDS Ozone Product Evaluation and Test Element (PEATE) will build on the heritage of the TOMS and OM1 mission based processing systems. The overall measurement based system that will encompass these efforts is the Atmospheric Composition Processing System (ACPS). We have extended the system to include access to publically available data sets from other instruments where feasible, including non-NASA missions as appropriate. The heritage system was largely monolithic providing a very controlled processing flow from data.ingest of satellite data to the ultimate archive of specific operational data products. The ACPS allows more open access with standard protocols including HTTP, SOAPIXML, RSS and various REST incarnations. External entities can be granted access to various modules within the system, including an extended data archive, metadata searching, production planning and processing. Data access is provided with very fine grained access control. It is possible to easily designate certain datasets as being available to the public, or restricted to groups of researchers, or limited strictly to the originator. This can be used, for example, to release one's best validated data to the public, but restrict the "new version" of data processed with a new, unproven algorithm until it is ready. Similarly, the system can provide access to algorithms, both as modifiable source code (where possible) and fully integrated executable Algorithm Plugin Packages (APPs). This enables researchers to download publically released versions of the processing algorithms and easily reproduce the processing remotely, while interacting with the ACPS. The algorithms can be modified allowing better experimentation and rapid improvement. The modified algorithms can be easily integrated back into the production system for large scale bulk processing to evaluate improvements. The system includes complete provenance tracking of algorithms, data and the entire processing environment. The origin of any data or algorithms is recorded and the entire history of the processing chains are stored such that a researcher can understand the entire data flow. Provenance is captured in a form suitable for the system to guarantee scientific reproducability of any data product it distributes even in cases where the physical data products themselves have been deleted due to space constraints. We are currently working on Semantic Web ontologies for representing the various provenance information. A new web site focusing on consolidating informaon about the measurement, processing system, and data access has been established to encourage interaction with the overall scientific community. We will describe the system, its data processing capabilities, and the methods the community can use to interact with the standard interfaces of the system.

Tilmes, Curt A.↗
DOE OSTI2022

Leaf spectra, Feb2016-April2016, PA-SLZ, PA-PNM, PA-BCI: Panama

This data package contains leaf spectra data measured on a monthly basis from February to April, 2016. Measurements were taken at the Bosque Protector San Lorenzo (SLZ), Barro Colorado Island (BCI) and Parque Natural Metropolitano (PNM) NGEE Tropics sites in Panama. Data from the BCI site are only available for March, 2016. Within the attached zip file are PDF manuals for instruments used, a guide to data collection protocol, and metadata files, including a PDF containing metadata for the 2016 ENSO gas exchange campaign. Also included is an additional zip file "2016_ENSO_BNL_Leaf_Spectra_Archive.zip" with data in .csv format organized by site. This data was collected as part of the 2016 ENSO campaign. See related datasets (existing and future) for further sample details, leaf water potential data, LMA, and gas exchange and leaf chemistry data. VERSION 2 update. The identification of a species from the PNM site has been corrected as follows: the identification of the tree initially identified as Pseudosamanea guachapele (ALBIED) has been revised to Albizia adinocephala (ALBIAD). The updated data package includes revised data, metadata and protocol documents updated to reflect this change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A global ensemble of ocean wave climate projections from CMIP5-driven models

This dataset, produced through the Coordinated Ocean Wave Climate Project (COWCLIP) phase 2, represents the first coordinated multivariate ensemble of 21st Century global wind-wave climate projections available (henceforth COWCLIP2.0). COWCLIP2.0 comprises general and extreme statistics of significant wave height (H S ), mean wave period (T m ), and mean wave direction (θ m ) computed over time-slices 1979–2004 and 2081–2100, at different frequency resolutions (monthly, seasonally and annually). The full ensemble comprising 155 global wave climate simulations is obtained from ten CMIP5-based state-of-the-art wave climate studies and provides data derived from alternative wind-wave downscaling methods, and different climate-model forcing and future emissions scenarios. The data has been produced, and processed, under a specific framework for consistency and quality, and follows CMIP5 Data Reference Syntax, Directory structures, and Metadata requirements. Technical comparison of model skill against 26 years of global satellite measurements of significant wave height has been undertaken at global and regional scales. This new dataset provides support for future broad scale coastal hazard and vulnerability assessments and climate adaptation studies in many offshore and coastal engineering applications.

Science & Technology - Other Topics↗
DOE OSTI2024

Aerial imagery dataset of lost oil wells

Orphaned wells are wells for which the operator is unknown or insolvent. The location of hundreds of thousands of these wells remain unknown in the United States alone. Cost-effective techniques are essential to locate orphaned wells to address environmental problems. In this paper, we present a dataset consisting of 120,948 aerial images of recently documented orphan wells. Each of these 512 × 512 images is paired with segmentation masks that indicate the presence or absence of such well. These images, sourced from the National Agriculture Imagery Program, cover the continental United States with spatial resolutions ranging from 30 centimeters to 1 meter. Additionally, we included negative examples by selecting locations uniformly across the United States. Accompanying metadata includes the IDs and spatial resolution of the original images, which are available for free through the United States Geological Survey, and the pixel coordinates of documented orphaned wells identified in these images. This dataset is intended to support the development of deep-learning models that can help locating undocumented orphan wells from such imagery, thereby blunting the environmental damage they do.

Climate-change mitigation↗
NASA NTRS2020

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗
DOE OSTI2023

Global analysis of the yeast knockout phenome

Genome-wide phenotypic screens in the budding yeast Saccharomyces cerevisiae, enabled by its knockout collection, have produced the largest, richest, and most systematic phenotypic description of any organism. However, integrative analyses of this rich data source have been virtually impossible because of the lack of a central data repository and consistent metadata annotations. Here, we describe the aggregation, harmonization, and analysis of ~14,500 yeast knockout screens, which we call Yeast Phenome. Using this unique dataset, we characterized two unknown genes (YHR045W and YGL117W) and showed that tryptophan starvation is a by-product of many chemical treatments. Furthermore, we uncovered an exponential relationship between phenotypic similarity and intergenic distance, which suggests that gene positions in both yeast and human genomes are optimized for function.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Global normal spectral irradiance in Albuquerque: a one-year open dataset for PV research

This report describes the creation process and final content of a spectral irradiance dataset for Albuquerque NM. The spectral irradiance measurements were made using a dual-axis tracker; therefore, they represent global normal irradiance. The dataset combines spectroradiometer and weather measurements from a two-year period into a continuous calendar year. The data files are accompanied by extensive metadata as well as example calculations and graphs to demonstrate the potential uses of this database.

14 SOLAR ENERGY↗
DOE OSTI2025

Best Practices for Nuclear Experiment Data Preservation at Idaho National Laboratory: A Guide for Researchers and Reactor Operators

Preserving experimental data is essential for supporting advancements in nuclear science and ensuring the longevity of Idaho National Laboratory's contributions to reactor technology and safety. This report provides a comprehensive guide to best practices for experimental data management and preservation, focusing on standardized data formats, redundancy in storage, metadata documentation, and alignment with international standards. By following these recommendations, experimentalists and reactor operators can enhance the accessibility, reproducibility, and utility of critical datasets for regulatory review, validation computational methods, and future research.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Model Inputs, Outputs, and Scripts associated with: “Spatial microbial respiration variations in the hyporheic zones within the Columbia River Basin”

This data package is associated with the publication “Spatial microbial respiration variations in the hyporheic zones within the Columbia River Basin” published in the Journal of Geophysical Research: Biogeosciences (Son et al. 2022) available at doi: 10.1029/2021JG006654. This data package includes the key model inputs/outputs of the river corridor model for the Columbia River Basin (CRB) and the model source codes, which were used in the manuscript. The model is a carbon-nitrogen-coupled river corridor model (RCM), and the model is used to quantify hyporheic zone (HZ) aerobic and anaerobic respiration at the NHDPLUS stream reach scales. The RCM used in this study combines empirical substrate models derived from observations and three microbially driven reactions to compute respiration of the HZ for each National Hydrography Dataset (NHD) reach within the CRB. The reactions in HZs of each NHD reach include anaerobic respiration and two-step anaerobic respiration via denitrification. Our HZ respiration estimates are limited to the lotic (or flowing) stream/river systems, and do not account for the respiration process in water column. Note that the RCM only simulates the HZ’s contribution to the dissolved carbon dioxide (CO2) concentrations in the streams, and the CO2 emissions to the atmosphere are not modelled. The model computes at hourly timesteps because of the fast reaction rates. The key input data of the model are exchange flux, residence time, and stream solute (dissolved organic carbon (DOC), dissolved oxygen (DO), and nitrate concentrations). These inputs are constant over time and represent long-term averaged values.This modeling framework successfully quantified HZ respiration components over multiple scales. It revealed key mechanisms driving the spatial variation of HZ aerobic and anaerobic respiration in reaches with varying hydrologic and substrate conditions. Thus, this modeling study offers a testing hypothesis in different river system (e.g., climate and biomes) for the HZ respiration processes, and can be used as a sampling design tool for large-scale HZ experimental studies.This dataset contains five folders: (1) model_inputs, (2) model_outputs, (3) Rscripts, (4) figures, and (5) model_codes. It also contains a readme, file level metadata (FLMD), and data dictionary (dd). Please see the FLMD for a list of all the files contained in this data package and descriptions for each. The model_inputs folder contains the model inputs used to drive the model simulations. The model_outputs folder contains key model output files from the river corridor model. The Rscripts folder contains the Rscripts for pre- and post- processing model results. The figures folder contains the raw figures associated with the manuscript. The model_codes folder includes key model source codes/input files. All files are .jpg, .jpeg, .out, .e, .od, .dat, .sub, .F90, .0, .R, .sbx, .cpg, .sbn, .shx, .shp, .dbf, .prj, .tfw, .tif, .xml, .pdf, or .csv.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Model Inputs, Outputs, and Scripts associated with: “Combined effects of stream hydrology and land use on basin-scale hyporheic zone denitrification in the Columbia River Basin”

This data package is associated with the publication “Combined effects of stream hydrology and land use on basin‐scale hyporheic zone denitrification in the Columbia River Basin”, published in Water Resource Research (Son et al.2022) available at https://doi.org/10.1029/2021WR031131. This data package includes the key model inputs/outputs of the river corridor model for the Columbia River Basin (CRB) and the model source codes used in the manuscript. The model is a carbon-nitrogen-coupled river corridor model (RCM), and the model is used to quantify hyporheic zone (HZ) denitrification at the NHDPLUS stream reach scales. The RCM used in this study combines empirical substrate models derived from observations and three microbially driven reactions, including two-step denitrification and aerobic respiration, are considered within the HZ. The key input data of the model are exchange flux, residence time, and stream solute (dissolved organic carbon (DOC), dissolved oxygen (DO), and nitrate concentrations). These inputs are constant over time and represent long-term averaged values. This study uses the RCM to explore the spatial patterns of HZ denitrification across reaches with different sizes and land use in the CRB. Our main objective is to use the RCM as a virtual reality model, and the machine-learning models as surrogates that encapsulate the complexities of the physics-based model while identifying the importance of different variables that are not evident in the model conceptualization. We do not include a direct comparison of the modeled HZ denitrification and measurements; however, the RCM can capture the overall spatial patterns of the HZ denitrification because the model inputs and its reaction networks are based on well-established theory and a physical-based model. The combination of the model-based predictions and a machine-learning approach (e.g., random forest) is used to improve our understanding of what variables of the model are associated with spatial patterns of the modeled denitrification across reaches with different sizes and land uses, and to develop a proxy model using measurable variables to reproduce the simulated patterns.This dataset contains five folders: (1) model_inputs, (2) model_outputs, (3) Rscripts, (4) figures, and (5) model_codes. It also contains a readme, file level metadata (FLMD), and data dictionary (dd). Please see the FLMD for a list of all the files contained in this data package and descriptions for each. The model_inputs folder contains the model inputs used to drive the model simulations. The model_outputs folder contains key model output files from the river corridor model. The Rscripts folder contains the Rscripts for pre- and post- processing model results. The figures folder contains the raw figures associated with the manuscript. The model_codes folder includes key model source codes/input files. All files are .jpg, .jpeg, .out, .e, .od, .dat, .sub, .F90, .0, .R, .sbx, .cpg, .sbn, .shx, .shp, .dbf, .prj, .tfw, .tif, .xml, .pdf, or .csv.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

NASA GeneLab: Open Science for Life in Space

The NASA GeneLab project capitalizes on multi-omic technologies to maximize the return on spaceflight experiments. To do this, GeneLab maintains a publicly accessible database (GLDS) that houses spaceflight and spaceflight relevant multi-omics data and collaborates with NASA principal investigators and projects to generate additional omics data. GeneLab houses more than 350 transcriptomic, proteomic, metabolomic and epigenomic datasets from plant, animal and microbial experiments, with a growing number of these having been produced by the GeneLab Sequencing Lab. The GLDS contains rich metadata about each experiment and has integrated radiation dosimetry data from experiments flown on the Space Shuttle, International Space Station, and Free Flying spacecrafts. With the increasing amount and complexity of omics data being generated, GeneLab utilizes community-defined, common models for metadata and terminology so that omics data and results are discoverable and reliably reproducible. GeneLab uses the ISA-Tab specification and semantic model for organizing and representing omics metadata. In addition to metadata standards, data files must be open-source file or common exchange formats to ensure accessibility and usability by all users. To ease data ingestion and transfer, the web-based submission tool allows PIs a user-friendly user interface to curate, organize, and publish their space relevant omics data. In the more recent years, data curation and submission portal has incorporated the FAIR principles making data findable, accessible, interoperable, and reusable. To increase reusability of data, GeneLab has implemented an effort to present processed data in the GLDS in addition to the raw omics data. The processed data will enable interpretation of the data by a larger group of students, scientists and the general public. Standard pipelines for the transformation of raw data into visualizations were developed by four GeneLab Analysis Working Groups (animals, plants, microbes, multi-omics) comprised of over 200 scientists from NASA, industry, and academia. To explore the data, the GLDS provides users various tools for data analysis, collaborative workspace for file storage and sharing, and a visualization portal. The analysis platform built using the Galaxy toolshed provides access to a broad variety of users including those with limited bioinformatics experience and students to learn how to analyze spaceflight omics data. The visualization portal takes GeneLab one step closer to data democratization by removing all bioinformatics requisites to interpret transcriptomics data hosted in the repository. To train the next generation of scientists, NASA offers training programs such as GeneLab 4 High School (GL4HS) and GeneLab 4 Universities. NLM Curation at a Scale Workshop 2022 | NASA GeneLab (GL4U) to teach students bioinformatics and computational biology methods to analyze omics data. Discoveries made using GeneLab have begun and will continue to deepen our understanding of biology, advance the field of genomics, and help to discover cures for diseases, create better diagnostic tools, and ultimately allow astronauts to better withstand the rigors of long-duration spaceflight.

GeneLab↗
DOE OSTI2021

MOSIQS: Persistent Memory Object Storage With Metadata Indexing and Querying for Scientific Computing

Scientific applications often require high-bandwidth shared storage to perform joint simulations and collaborative data analytics. Shared memory pools provide a chance to satisfy such needs. Recently, a high-speed network such as Gen-Z utilizing persistent memory (PM) offers an opportunity to create a shared memory pool connected to compute nodes. However, there are several challenges to use scientific applications on the shared memory pool directly such as scalability, failure-atomicity, and lack of scientific metadata-based search and query. In this paper, we propose MOSIQS, a persistent memory object storage framework with metadata indexing and querying for scientific computing. We design MOSIQS based on the key idea that memory objects on PM pool can live beyond the application lifetime and can become the sharing currency for applications and scientists. MOSIQS provides an aggregate memory pool atop an array of persistent memory devices to store and access memory objects to accelerate scientific computing. MOSIQS uses a lightweight persistent memory key-value store to manage the metadata of memory objects, which enables memory object sharing. To facilitate metadata search and query over millions of memory objects resident on memory pool, we introduce Group Split and Merge (GSM), a novel persistent index data structure designed primarily for scientific datasets. GSM splits and merges dynamically to minimize the query search space and maintains low query processing time while overcoming the index storage overhead. MOSIQS is implemented on top of PMDK. We evaluate the proposed approach on many-core server with an array of real PM devices. Experimental results show that MOSIQS gains a 100% write performance improvement and executes multi-attribute queries efficiently with 2.7× less index storage overhead offering significant potential to speed up scientific computing applications.

97 MATHEMATICS AND COMPUTING↗