Search NASA⌕ Search

SEARCH · Search NASA

Results for “data lake”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Evaluate data lake design for the accelerator control system

Increasing precision in automation for modern particle accelerators not only creates a requirement to gather data from all devices but also demands scalable and high-performance data infrastructure with the capability of handling vast incoming device data. A well architected data lake is suitable for such a system which integrates real-time data acquisition, transient data caching, and long-term storage. This paper evaluates data lake architecture for an Accelerator Control System (ACS), focusing on two critical components of a data lake, data cache and long-term storage.

Jaikar, Amol [Fermilab]↗
DOE OSTI2025

Evaluation of Data Lake Design for the Accelerator Control System

In this modern world, the user expects faster processing and real-time response to operate accelerator control devices. The existing framework with its infrastructure does not have the ability to satisfy these future needs. Therefore, modernization is required to reach industry standards and develop a modular framework that can provide flexibility and dynamic scalability. The data lake architecture, comprising three layers, ingestion, processing, and data consumption, provides flexibility and scalability to meet current and future demands for the accelerator control system.

Jaikar, Amol [Fermilab]↗
DOE OSTI2025

Common Column Identification for Table Similarity Detection in Electrified Transportation Data Lakes

Electrified transportation often requires researchers and operators to interact with datasets from a wide range of sources and disciplines, such as transportation, power systems, public health, policies, and regulations. These datasets vary in quality and format, making it difficult to understand, preprocess, and identify key columns representing real-world entities or values for indexing and joining, which can negatively impact downstream analysis and operation. Existing solutions are limited, requiring extensive manual customization or data expertise to utilize. In this article, we propose a multi-layered approach to automatically identify key columns to expedite preprocessing and aid in analysis of electrified transportation data. Our method leverages a dynamic ontology to identify common fields and an information theory-based strategy for edge cases that are difficult to generalize. Evaluations on a number of datasets from data.gov and kaggle.com show improved performance of our methods over several baseline techniques, and our ablation analyses illustrate the efficacy of individual components of our method. Our case studies also demonstrate that our methods have the potential to improve analysis of electrified transportation data and aid in automatic integration of such datasets.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1975

The trophic classification of lakes using ERTS multispectral scanner data

Lake classification methods based on the use of ERTS data are described. Preliminary classification results obtained by multispectral and digital image processing techniques indicate satisfactory correlation between ERTS data and EPA-supplied water analysis. Techniques for determining lake trophic levels using ERTS data are examined, and data obtained for 20 lakes are discussed.

Blackwell, R. J.↗
DOE OSTI2025

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE's) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and helping users access data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data. This paper provides an update on recent improvements made to the GDR's data lakes and automated data pipelines, including: (1) streamlining the data lake intake process, (2) better educating users on the process and requirements through a new data lakes page, (3) adding data lake direct access links to GDR data lake submission pages, (4) implementing a DAS data pipeline to convert DAS data uploaded in SEG-Y format to a standardized hierarchical data format v5 (HDF5), (5) extending this pipeline to encompass data in the GDR data lake, (6) adding metadata requirements for geospatial data, (7) making user interface/user experience (UX) enhancements to the data pipelines' documentation pages, and (8) improving the GDR's data standards and pipelines pages to better guide users in ensuring that their data is standardized by the GDR's automated data pipelines. 2024 Geothermal Resources Council. All rights reserved.

accessibility↗
DOE OSTI2026

The Zooplankton International Geospatial dataset: A global repository of spatiotemporal freshwater zooplankton community composition data from lakes and reservoirs to support ecological research

Zooplankton transfer substantial energy in aquatic food webs and are used as indicators of environmental change. Syntheses of zooplankton community dynamics globally require datasets that span a wide range of environmental gradients; however, these datasets are limited due to methodological differences across programs, taxonomic inconsistencies, and a lack of standardized metadata. To reconcile these challenges, we created the Zooplankton International Geospatial (ZIG) dataset, which includes original zooplankton, water physical and chemical variables, and lake morphometric data from 311 inland lakes and reservoirs. ZIG includes waterbodies ranging in size from 0.005 to 82,100 km2 and spanning broad latitudinal (−47.26 to 64.90) and longitudinal ranges (−165.04 to 176.53). Temporal coverage for individual waterbodies ranges between 1 and 60 yr with sampling frequency ranging from annually to weekly. With its extensive coverage and content, we consider ZIG to be a cornerstone for future investigations of global scale lake biodiversity change.

Figary, Stephanie [Cornell University, Ithaca, NY]↗
DOE OSTI2026

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

accessibility↗
NASA NTRS2009

Monitoring Invasive Aquatic Vegetation in Lake Okeechobee, Florida, Using NDVI Derived from Modis Data

Lake Okeechobee, located in southern Florida, encompasses approximately 1,700 sq km and is a vital part of the Lake Okeechobee and Everglades ecosystem. Major cyanobacterial blooms have been documented in Lake Okeechobee since the 1970s and have continued to plague the ecosystem. Similarly, hydrilla, water hyacinth, and water lettuce have been documented in the lake and continue to threaten the ecosystem by their rapid growth. This study examines invasive aquatic vegetation occurrence through the use of the Normalized Difference Vegetation Index (NDVI) calculated on MOD09 surface reflectance imagery. Occurrence during 2008 was analyzed using the Time Series Product Tool (TSPT), a MATLAB-based program developed at John C. Stennis Space Center. This project tracked spatial and temporal variability of cyanobacterial blooms, and overgrowth of water lettuce, water hyacinth, and hydrilla. In addition, this study presents an application of Moderate Resolution Imaging Spectroradiometer (MODIS) data to assist in water quality management.

Woods, Kate↗
DOE OSTI2025

Open Energy Data Initiative (OEDI) FY22-24 (Final Technical Report)

Final technical report for the Open Energy Data Initiative (OEDI) project covering fiscal years FY22 through FY24. The DOE Open Energy Data Initiative (OEDI) is a partnership between the National Renewable Energy Laboratory (NREL), the U.S. Department of Energy (DOE), and major cloud providers including Amazon, Microsoft, and Google to provide universal access to big data in the cloud. At the heart of OEDI is a centralized repository of high-value energy research datasets aggregated from the U.S. Department of Energy's Program Offices, National Laboratories and other collaborators. It aggregates smaller, domain-specific repositories, allows direct data submissions, and includes support for big data through its energy data lakes. OEDI's data lakes make high-value data universally accessible and help researchers, collaborators and the general public overcome many of the obstacles to accessing and using big data.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
NASA NTRS2018

Lake Chad Total Surface Water Area as Derived from Land Surface Temperature and Radar Remote Sensing Data

Lake Chad, located in the middle of the African Sahel belt, underwent dramatic decreases in the 1970s and 1980s leaving less than ten percent of its 1960s surface water extent as open water. In this paper, we present an extended record (dry seasons 1988-2016) of the total surface water area of the lake (including both open water and flooded vegetation) derived using Land Surface Temperature (LST) data (dry seasons 2000-2016) from the NASA Terra MODIS sensor and EUMETSAT Meteosat-based LST measurements (dry seasons 1988-2001) from an earlier study. We also examine the total surface water area for Lake Chad using radar data (dry seasons 2015-2016) from the ESA Sentinel-1a mission. For the limited number of radar data sets available to us (18 data sets), we find on average a close match between the estimates from these data and the corresponding estimates from LST, though we find spatial differences in the estimates using the two types of data. We use these spatial differences to adjust the record (dry seasons 2000-2016) from MODIS LST. Then we use the adjusted record to remove the bias of the existing LST record (dry seasons 1988-2001) derived from Meteosat measurements and combine the two records. From this composite, extended record, we plot the total surface water area of the lake for the dry seasons of 1988-1989 through 2016-2017. We find for the dry seasons of 1988-1989 to 2016-2017 that the maximum total surface water area of the lake was approximately 16,800 sq. km (February and May, 2000), the minimum total surface water area of the lake was approximately 6400 sq. km (November, 1990), and the average was approximately 12,700 sq. km. Further, we find the total surface water area of the lake to be highly variable during this period, with an average rate of increase of approximately 143 sq. km per year.

Policelli, Frederick↗
DOE OSTI2024

2024 Annual Technology Baseline (ATB) Cost and Performance Data for Electricity Generation Technologies

These data provide the 2024 update of the Electricity Annual Technology Baseline (ATB). Starting in 2015 NREL has presented the ATB, consisting of detailed cost and performance data, both current and projected, for electricity generation and storage technologies. The ATB products now include data (Excel workbook, Tableau workbooks, and structured summary csv files), as well as documentation and user engagement via a website, presentation, and webinar. Starting in 2021, the data are cloud optimized and provided in the OEDI data lake. The data for 2015 - 2020 are can be found on the NREL Data Search Page. The website documentation can be found on the ATB Website.

Array↗
DOE OSTI2026

Soil and groundwater environmental sensor data, Wax Lake Delta, Louisiana, March 2023 - March 2024

This study evaluates how environmental parameters that integrate biogeochemical processes vary with water table fluctuations in the freshwater Wax Lake Delta (WLD) in Louisiana, U.S.A. This data package contains seven *.csv files and one Excel file that compiles all the data from the individual .csv files. This dataset reports high frequency (15-min) observations of water level, soil redox potential, specific conductance, and pH made for one year along elevation transects located on the older, proximal (OT) and younger, distal (YT) ends of a deltaic island. Water depth relative to the ground surface (cm; HOBO U20L-04; error ± 0.4 cm), water pH and temperature (HOBO MX2501), and specific conductance and temperature (HOBO U24-001) sensors were installed in March 2023. Water depth was corrected for barometric pressure recorded by a separate logger secured to a platform above the highest water level. Soil redox probes (SWAP ORP-40-4-B) were also installed in March 2023. Each probe had four Pt sensors (2 mm width) placed at 10 cm, 20 cm, 30 cm, and 40 cm below the ground surface. Redox data were referenced to an external Ag0/AgCl (3M KCl) reference probe placed in saturated ground and recorded on CR1000X dataloggers (Campbell Scientific) powered by solar panels. A second reference probe was positioned near the primary reference probe for backup and data correction. The tops of the soil redox probes and soil moisture probes were flush with the soil surface so that sensors are reported at their indicated depths below ground surface. Here, we report data collected between 15 March 2023 to 15 March 2024 for all sensors, with some differences due to exact dates of sensor placement or data gaps associated with sensor malfunction. For example, water depth at OT4 was not recorded between March to November 2023. Data flags indicate whether a value is valid (1) or was excluded from data analysis in the associated manuscript (-1).

EARTH SCIENCE > LAND SURFACE > SOILS↗
NASA NTRS1974

Analysis of Skylab MSS data, and calibration of thermal data over Lake Monroe, Indiana

The author has identified the following significant results. The SL/4-S192 data over northern Indiana looks good. The thermal band looks very encouraging. The preliminary analysis indicates so far that it will be difficult to distinguish some land use classes because of the lack of vegetation. Preliminary analysis of the thermal band in particular indicates that there appears to be a correlation of the hot spots in the data over the Ft. Wayne, Indiana area and its industrial centers.

Silva, L. F.↗
NASA NTRS1975

Utilization of satellite data for inventorying prairie ponds and lakes. LANDSAT-1 data were used to discriminate ponds and lakes for waterfowl management

The author has identified the following significant results. The mapping of open water as an indicator of waterfowl habitat quality was carried out by using two different recognition techniques, a single waveband thresholding approach and a multiple waveband approach termed proportion estimation. The single waveband technique has proven simple to implement. Its computer algorithm was rapid and accurately recognized prairie lakes and large ponds. The resultant products of this processing technique were thematic maps and statistical tabulations describing open surface water conditions. The maps served to portray visually the location and frequency of surface water bodies but usually necessitated additional interpretation.

Gilmer, D. S.↗
NASA NTRS1972

A study of Minnesota forests and lakes using data from earth resources technology satellites

This project is to foster and develop new applications of remote sensing under an interdisciplinary effort. Seven reports make up the specific projects presently being conducted throughout the State of Minnesota in cooperation with several agencies and municipalities. These are included under the general headings of: (1) applications of aerial photography and ERTS-1 data to agricultural, forest, and water resources management; (2) classification and dynamics of water and wetland resources of Minnesota; (3) studies of Lake Superior Bay; and (4) feasibility of detecting major air pollutants by earth-oriented satellite-borne sensors.

Source record↗
NASA NTRS1974

A study of Minnesota forests and lakes using data from Earth Resources Technology Satellites

Highlights of research and practical benefits are discussed for the following projects which utilized ERTS 1 data to provide municipal, state, federal, and industrial users with environmental resource information for the state of Minnesota: (1) forest disease detection and control; (2) evaluation of water quality by remote sensing techniques; (3) forest vegetation classification and management; (4) detection of saline soils in the Red River Valley; (5) snowmelt flood prediction; (6) remote sensing applications to hydrology; (7) Rice Creek watershed project; (8) water quality in Lake Superior and the Duluth Superior Harbor; and (9) determination of Lake Superior currents from turbidity patterns.

Source record↗