Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud-based”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Designing the User Experience for Earth Observation Data Services in the Cloud

NASA's Earth Observation (EO) inventory is projected to grow by an order of magnitude over the next 5-6 years. The current mode for working with EO data of downloading the data to a local machine (laptop, desktop or server) will be difficult to sustain for these upcoming volumes. Therefore, NASA is in the process of developing a capability to host large volume data in commercial cloud, with an eye toward encouraging data analysis in the cloud. However, in order for the user community to take advantage of this new mode of data interaction, the user experience must be redesigned. Cloud-hosted data brings new challenges, such as managing the costs of data egress and working with data in Web Object Storage instead of a Posix filesystem. However, it also brings new opportunities. Scaling data transformation processes may permit more synchronous data services with near-immediate response vs. cumbersome ordering systems with latencies of hours or days. Data co-location in the cloud can facilitate data integration and fusion. Highly scalable filesystems and databases in the cloud support data reorganization to facilitate analysis at scale. In the course of NASA's reimagining of the User Experience for EO data usage relies on end user input gathered through surveys, workshops and meetings (such as this). At the same time, we have embarked on a course of pedagogy and capacity building to help the user community evolve to cloud-based analysis.

Lynnes, Christopher↗
NASA NTRS2019

EARTHDATA PUB: A Data Publication Workflow Solution for NASA’s EOSDIS

Each NASA Distributed Active Archive Center (DAAC) faces the challenge of dealing with an increasingly diverse number of publishable data products from diverse data producers. Data producers, on the other hand, may experience pain points when interacting with the EOSDIS for the first time or when publishing different data at different DAACs. As a result, there has been a growing need to develop a common software framework that serves as a common interface for data producers, rigorously defines the data publication procedure for DAAC staff, facilitates the management of various data publication processes, and tracks the progress of data publication. This software should also account for the different configurations at different DAACs. Currently, two primary data publication workflow and tracking tools exist in operation at the EOSDIS: Semi-Automated ingest System (SAuS) and Data Publication workflow Portal (DAPPeR). However, neither tool is cloud-ready. Automated data processing could be managed by Cumulus, an EOSDIS cloud-based data ingest, archive and management system. However, Cumulus does not support manual tasks or on-premise implementations. We propose to develop the Earthdata Publication Minimum Viable Product (Earthdata Pub MVP) -- a cloud-hosted solution that works with both cloud and on-premise systems and implements the communications and exchange requirements generated by the Earthdata Pub information architecture team.

Rice, Justin L.↗
NASA NTRSDate not supplied

Standardizing Algorithm Documentation For Improved Scientific Data Understanding: The Algorithm Publication Tool Prototype

Algorithm Theoretical Basis Documents (ATBDs) are documents which accompany Earth observation data products generated from algorithms. While ATBDs are essential to scientific reproducibility, these key documents are not standardized and are often difficult to find. In this paper, we present the prototype Algorithm Publication Tool (APT), a cloud-based ATBD authoring and editing tool for NASA’s Earth science data systems. A standardized ATBD information model is also described as well as lessons learned from developing the prototype tool.

Kaylin Bugbee↗
NASA NTRSDate not supplied

ImageLabler: Labeling and Managing Image Data for Machine Learning in the Earth Sciences

While machine learning techniques for image classification have been around for a long time, storing and managing the vast number of images required as training data is still a problem for scientists. This is especially true for the field of Earth science, where only recently have experts begun using machine learning techniques for image-based phenomena classification. Image Labeler, a fast and scalable cloud-based tagging platform for Earth science images, seeks to improve upon existing methods of managing images and associated metadata, such as maintaining categorized folders of images on a local machine, a process that can be cumbersome and difficult to scale. The platform facilitates rapid development of image-based Earth science phenomena training datasets by allowing scientists to upload their existing imagery as well as extract new samples from open satellite imagery services made available through NASA’s Global Imagery Browse Service (GIBS). Image Labeler also supports GeoTIFF data, with capabilities such as displaying GeoTIFFs on an interactive map, drawing shapefiles over them, and tagging them with additional metadata. This allows scientists to perform spatiotemporal subsetting with geographic information and develop training data more quickly. Built using modern web technologies, Image Labeler includes additional capabilities such as team collaboration for large-scale image tagging projects. Users can download their data in a machine-learning-ready format, allowing scientists to spend time on experimentation rather than on the collection of training data. In this presentation, we demonstrate how Image Labeler seeks to become a one-stop image data management solution for machine learning applications in Earth science.

Ashish Acharya↗
NASA NTRS2019

An Integrated Data Analytics Platform

An Integrated Science Data Analytics Platform is an environment that enables the confluence of resources for scientific investigation. It harmonizes data, tools and computational resources which subsequently enable the research community to focus on the investigation rather than spending time on security, data preparation, management, etc. OceanWorks is a NASA technology integration project to establish a cloud-based Integrated Ocean Science Data Analytics Platform at NASA’s Physical Oceanography Distributed Active Archive Center (PO.DAAC) for big ocean science. It focuses on advancement and maturity by bringing together several NASA open-source, big data projects for parallel analytics, anomaly detection, in-situ to satellite data matchup, quality-screened data subsetting, search relevancy, and data discovery. Our communities are relying on data distributed through data centers such as the PO.DAAC, COAPS, NCAR, and many others to conduct their research. In typical investigations, scientists would engage in: search for data, evaluate the relevance of that data, download it, and then apply algorithms to identify trends. Such workflow cannot scale if the research involves a massive amount of data or multi-variate measurements. NASA’s Surface Water and Ocean Topography (SWOT) mission is expected to produce massive amount of observational data during its 3-year nominal mission. Collections like SWOT challenges all existing Earth Science data archival, distribution and analysis paradigms. In this paper, we will discuss how OceanWorks enhances the analysis of physical ocean data where the computation is done on an elastic cloud platform next to the archive to deliver fast, web-accessible services for working with oceanographic measurements.

Yang, Chaowei↗
NASA NTRS2021

Large-Scale High-Resolution Coastal Mangrove Forests Mapping Across West Africa With Machine Learning Ensemble and Satellite Big Data

Coastal mangrove forests provide important ecosystem goods and services, including carbon sequestration, biodiversity conservation, and hazard mitigation. However, they are being destroyed at an alarming rate by human activities. To characterize mangrove forest changes, evaluate their impacts, and support relevant protection and restoration decision making, accurate and up-to-date mangrove extent mapping at large spatial scales is essential. Available large-scale mangrove extent data products use a single machine learning method commonly with 30 m Landsat imagery, and significant inconsistencies remain among these data products. With huge amounts of satellite data involved and the heterogeneity of land surface characteristics across large geographic areas, finding the most suitable method for large-scale high-resolution mangrove mapping is a challenge. The objective of this study is to evaluate the performance of a machine learning ensemble for mangrove forest mapping at 20 m spatial resolution across West Africa using Sentinel-2 (optical) and Sentinel-1 (radar) imagery. The machine learning ensemble integrates three commonly used machine learning methods in land cover and land use mapping, including Random Forest (RF), Gradient Boosting Machine (GBM), and Neural Network (NN). The cloud-based big geospatial data processing platform Google Earth Engine (GEE) was used for pre-processing Sentinel-2 and Sentinel-1 data. Extensive validation has demonstrated that the machine learning ensemble can generate mangrove extent maps at high accuracies for all study regions in West Africa (92%–99% Producer’s Accuracy, 98%–100% User’s Accuracy, 95%–99% Overall Accuracy). This is the first-time that mangrove extent has been mapped at a 20 m spatial resolution across West Africa. The machine learning ensemble has the potential to be applied to other regions of the world and is therefore capable of producing high-resolution mangrove extent maps at global scales periodically.

coastal environment↗
NASA NTRS2020

Connecting Users and Applications with Po.daac Hosted GHRSST Data

The 80+ GHRSST public datasets represent a rich resource for sea surface temperature research and applications given their time series length, resolution, spatial coverage, varying measurement types and processing levels, and availability in the full spectrum of PO.DAAC tools and services ecosystem. The PO.DAAC has created a publicly accessible recipe suite for the user community to perform straightforward yet powerful computations on GHRSST data using python recipes, Jupyter notebooks, R, Matlab, and the NCO programming language. These recipes include numerical computations for regional and global SST trends, anomaly derivations, EOF analysis, climate signal reproduction, and ocean phenology. For example, one recipe reproduces a famous SST based warming figure from the Fourth National Climate Assessment (USA) while another focuses on quantifying the regional changes in ocean SST phenology. Most are python-based while some contain hybrid calls and leverage the NCO programming interface too. All are available on the PO.DAAC user forum (https://podaac.jpl.nasa.gov/forum/) and/or via the open source NASA GitHub repository (https://github.com/nasa/podaac_tools_and_services). Several are available in the Jupyter notebook framework including podaacypy (https://github.com/nasa/podaacpy), a recipe for GHRSST granule metadata discovery and application, and more recently a Jupyter notebook developed to support data analysis and visualization of a cloud-based Zarr formatted Level 4 MUR dataset in the AWS Open Data Registry. Throughout the summer of 2020, the PO.DAAC intends to add and migrate more of its numerical recipes to the Jupyter notebook framework and publish them on its open source GitHub repository.

Gentemann, Chelle↗
NASA NTRS2020

Model Checking as a Service: Towards Pragmatic Hidden Formal Methods

Executable models can be used to support all engineering activities in Model-Based Systems Engineering. Testing and simulation of such models can provide early feedback about design choices. How-ever, in today’s complex systems failures could arise due to subtle errors that are hard to find without checking all possible execution paths. Formal methods, and especially model checking can uncover such subtle errors, yet their usage in practice is limited due to the specialized expertise and high computing power required. There-fore we created an automated, cloud-based environment that can verify complex reachability properties on SysML State Machines using hidden model checkers. The approach and the prototype is illustrated using an example from the aerospace domain.

Karban, Robert↗
NASA NTRS2020

Names Don't Fly: Smart Filters for Profanity Detection and Classification in User-Generated Content

Generally, names associate with a person’s identity. But what if in the pretext of a legitimate name and given the opportunity, users of software provide names to online web forms that carry along offensive language, slurs, and other profanity that is then sent to Mars ? The answer is simple: they don’t fly. In this paper,we perform model explorations to detect and classify inappropriate content in the names submitted from people across the world to ‘Send Your Names to MARS’ public engagement campaign.We propose a novel pipeline approach, that can effectively overcome the issues of lack of negative samples, noisy labels by gathering expert knowledge over time with human(s) in the loop and data augmentation, and achieve high accuracy in classifying inappropriate names with very little or no context. We describe cloud-based infrastructure to deploy our application and run predictions on large-scale data through our pipeline and achieve significant speedup over offline processes, with enhanced reliability and security.

Soderstrom, Tomas↗
NASA NTRS2020

TopiQAL: Topic-aware Question Answering using Scalable Domain-specific Supercomputers

We all have questions. About today's temperature, scores of our favorite baseball team, the Universe, and about vaccine for COVID-19. Life, physical, and natural scientists have been trying to find answers to various topics using scientific methods and experiments, while computer scientists have built language models as a tiny step towards automatically answering all of these questions across domains given a little bit of context. In this paper, we propose an architecture using state-of-the-art Natural Language Processing language models namely Topic Models and Bidirectional Encoder Representations from Transformers (BERT) that can transparently and automatically retrieve articles of relevance to questions across domains, and fetch answers to topical questions related to COVID-19 current and historical medical research literature. We demonstrate the benefits of using domain-specific supercomputers like Tensor Processing Units (TPUs), residing on cloud-based infrastructure, using which we could achieve significant gains in training and inference times, also with very minimal cost.

Penberthy, Scott↗
NASA NTRSDate not supplied

Maximizing Earth Science Observations with Data Harmonization: Harmonized Landsat/Sentinel-2

In August 2021, NASA released science quality harmonized Landsat/Sentinel-2 products for both cloud-based access and direct download from the Land Processes Distributed Active Archive Center (LP DAAC). These 30-meter products, HLSS30 (Sentinel-2 component) and HLSL30 (Landsat component) are placed on the same grid and are generated and distributed fully in the cloud. Similarly, ESA is prototyping harmonized Landsat Sentinel-2 products at 10-meter resolution. The data production and science teams from NASA and ESA will present technical details on the data production system, data product status and availability, and the benefit of harmonizing instruments with similar sensing characteristics between the agencies.

Earth Science↗
NASA NTRS2022

Satellite-derived Cyanobacteria Frequency and Magnitude in Headwaters & Near-dam Reservoir Surface Waters of the Southern U.S.

Reservoirs are dominant features of the modern hydrologic landscape and provide vital services. However, the unique morphology of reservoirs can create suitable conditions for excessive algae growth and associated cyanobacteria blooms in shallow in-flow reservoir locations by providing warm water environments with relatively high nutrient inputs, deposition, and nutrient storage. Cyanobacteria harmful algal blooms (cyanoHAB) are costly water management issues and bloom recurrence is associated with economic costs and negative impacts to human, animal, and environmental health. As cyanoHAB occurrence varies substantially within different regions of a water body, understanding in-lake cyanoHAB spatial dynamics is essential to guide reservoir monitoring and mitigate potential public exposure to cyanotoxins. Cloud-based computational processing power and high temporal frequency of satellites enables advanced pixel-based spatial analysis of cyanoHAB frequency and quantitative assessment of reservoir headwater in-flows compared to near-dam surface waters of individual reservoirs. Additionally, extensive spatial coverage of satellite imagery allows for evaluation of spatial trends across many dozens of reservoir sites. Surface water cyanobacteria concentrations for sixty reservoirs in the southern U.S. were estimated using 300 m resolution European Space Agency (ESA) Ocean and Land Colour Instrument (OLCI) satellite sensor for a five year period (May 2016–April 2021). Of the reservoirs studied, spatial analysis of OLCI data revealed 98% had more frequent cyanoHAB occurrence above the concentration of >100,000 cells/mL in headwaters compared to near-dam surface waters (P < 0.001). Headwaters exhibited greater seasonal variability with more frequent and higher magnitude cyanoHABs occurring mid-summer to fall. Examination of reservoirs identified extremely high concentration cyanobacteria events (>1,000,000 cells/mL) occurring in 70% of headwater locations while only 30% of near-dam locations exceeded this threshold. Wilcoxon signed-rank tests of cyanoHAB magnitudes using paired-observations (dates with observations in both a reservoir's headwater and near-dam locations) confirmed significantly higher concentrations in headwater versus near-dam locations (p < 0.001).

Amber R Ignatius↗
NASA NTRS2022

Satellite-derived cyanobacteria frequency and magnitude in headwaters & near-dam reservoir surface waters of the Southern U.S.

Reservoirs are dominant features of the modern hydrologic landscape and provide vital services. However, the unique morphology of reservoirs can create suitable conditions for excessive algae growth and associated cyanobacteria blooms in shallow in-flow reservoir locations by providing warm water environments with relatively high nutrient inputs, deposition, and nutrient storage. Cyanobacteria harmful algal blooms (cyanoHAB) are costly water management issues and bloom recurrence is associated with economic costs and negative impacts to human, animal, and environmental health. As cyanoHAB occurrence varies substantially within different regions of a water body, understanding in-lake cyanoHAB spatial dynamics is essential to guide reservoir monitoring and mitigate potential public exposure to cyanotoxins. Cloud-based computational processing power and high temporal frequency of satellites enables advanced pixel-based spatial analysis of cyanoHAB frequency and quantitative assessment of reservoir headwater in-flows compared to near-dam surface waters of individual reservoirs. Additionally, extensive spatial coverage of satellite imagery allows for evaluation of spatial trends across many dozens of reservoir sites. Surface water cyanobacteria concentrations for sixty reservoirs in the southern U.S. were estimated using 300m resolution European Space Agency (ESA) Ocean and Land Colour Instrument (OLCI) satellite sensor for a five year period (May 2016–April 2021). Of the reservoirs studied, spatial analysis of OLCI data revealed 98% had more frequent cyanoHAB occurrence above the concentration of >100,000 cells/mL in headwaters compared to near-dam surface waters (P < 0.001). Headwaters exhibited greater seasonal variability with more frequent and higher magnitude cyanoHABs occurring mid-summer to fall. Examination of reservoirs identified extremely high concentration cyanobacteria events (>1,000,000 cells/mL) occurring in 70% of headwater locations while only 30% of near-dam locations exceeded this threshold. Wilcoxon signed-rank tests of cyanoHAB magnitudes using paired-observations (dates with observations in both a reservoir's headwater and near-dam locations) confirmed significantly higher concentrations in headwater versus near-dam locations (p < 0.001).

cyanobacteria↗
NASA NTRSDate not supplied

Joint ESA-NASA Multi-Mission Algorithm and Analysis Platform (MAAP)

The scientific community is faced with a need for greatly improved data sharing, analysis, visualization and advanced collaboration based firmly on open science principles. Recent and upcoming launches of new satellite missions with more complex and voluminous data, as well as the ever more urgent need to better understand the global carbon budget and related ecological processes, provided the immediate rational for the ESA-NASA Multi-mission Algorithm and Analysis Platform (MAAP). This highly collaborative joint project of ESA and NASA established a framework between ESA and NASA to share data, science algorithms and compute resources in order to foster and accelerate scientific research conducted by ESA and NASA EO data users. Presented to the public in October 2021, the current version of MAAP provides a common cloud-based platform with computing capabilities co-located with the data, a collaborative coding and analysis environment, and a set of interoperable tools and algorithms developed to support the estimation and visualization of global above-ground biomass. Data from the Global Ecosystem Dynamics Investigation (GEDI) mission on the International Space Station and the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) have been instrumental in the first products of MAAP including the first comprehensive map of Boreal above-ground Biomass and a current Global Biomass Harmonization Activity, but the platform is also being specifically designed to support the forthcoming ESA Biomass mission and incorporate data from the upcoming NASA-ISRO SAR (NISAR) mission. While these missions and the corresponding research which includes airborne, field, and calibration/validation data collection and analyses, provide a wealth of data and information relating to global biomass estimation, they also present data storing, processing and sharing challenges. The NISAR mission alone will produce about 80TB/day. These large data volumes present a challenge that would otherwise place accessibility limits on the scientific community and impact scientific progress. Other challenges being addressed by MAAP include: 1) Enabling researchers to easily discover, process, visualize and analyze large volumes of data from both agencies; 2) Providing a wide variety of data in the same coordinate reference frame to enable comparison, analysis, data evaluation, and data generation; 3) Providing a version-controlled science algorithm development environment that supports tools, co-located data and processing resources; and 4) Addressing intellectual property and sharing challenges related to collaborative algorithm development and sharing of data and algorithms. MAAP products can be explored on the MAAP Dashboard at https://earthdata.nasa.gov/maap-biomass or the joint platform entrance at scimaap.net. MAAP also can be accessed through individual NASA (https://maap-project.org) and ESA (https://esa-maap.org/) landing pages.

cloud computing↗
NASA NTRSDate not supplied

Envisioning the Future of International Earth Observations Collaboration: Area and Map Accuracy Estimation by Sampling

- Maps have errors, sometimes a lot of errors ⇒ obtaining information directly from map problematic; any use of the map will benefit from information about errors/uncertainty - IPCC: unbiasedness and uncertainty - By randomly drawing a sample from the study area, observing reference conditions at drawn locations, and constructing estimators – easily estimate areas and map accuracy ± uncertainty - Sampling design and analysis, relatively easy – collecting reference observations is not. Cloud-based solutions, e.g. AREA2, Collect Earth Online, facilitate the collection of reference observations - More research and guidance needed!

Pontus Olofsson↗
NASA NTRSDate not supplied

Imbalanced Multi-layer Cloud Classification with Advanced Baseline Imager (ABI) and CloudSat/CALIPSO Data

Clouds at different altitudes play different roles in Earth’s climate. Comprehensive understanding of overlapping clouds is important for climate and weather prediction. The East Pacific region is where El Ni˜no and La Ni˜na originate and where multi-layer clouds frequently occur. The overlap of clouds at different altitudes in this region increases the classification complexity for cloud-based climatological studies. Unlike prior work in cloud layer classification that assumes single layer or two-layer of clouds, in this work, we consider multi-layer cloud classification with 8 cloud-level classes (clear-sky, high, middle, low, high+middle, high+low, middle+low, high+middle+low). We develop and analyze machine learning models on features extracted from satellite images from the East Pacific regions collected by GOES Advanced Baseline Imager (ABI). These are used to classify CloudSat/CALIPSO observed multi-layer clouds. Due to the imbalanced nature of the data, we investigate the adoption of conventional resampling methods, as well as deep learning methods with data augmentation. In our experiments, we utilize the random forest classifier and Multilayer perceptron classifier with data augmentation methods to reduce the class imbalance during training. With these approaches, we achieve a classification accuracy of 83.6% without exploiting any ancillary information.

machine learning↗
DOE OSTI2025

Advanced Transmission Technologies – GETs and HPCs Session 1: ATT Foundations and Dynamic Line Ratings (DLRs)

The INL TADA GETs Cohort Session 1, held on November 4, 2025, convened experts to address the integration of advanced transmission technologies, including Grid-Enhancing Technologies (GETs) and High Performance Conductors (HPCs), with a focus on digital assurance challenges. The session highlighted the growing importance of cybersecurity, supply chain transparency, reliability, and business risk management in deploying GETs, especially Dynamic Line Ratings (DLRs). Participants examined how expanded attack surfaces, limited vendor pools, and new regulatory requirements—such as FERC Orders 881, 2023, and 1920—are influencing utilities and technology providers. The workshop underscored the need for cyber-informed engineering, secure-by-design principles, and practical risk management strategies, while fostering collaboration and knowledge sharing among industry peers. Technical discussions covered the evolution from static to dynamic line ratings, complexities of cloud-based architectures, and NERC CIP compliance challenges. The session concluded with a collaborative risk exercise and a preview of future workshops on advanced power flow control and transmission topology optimization, reinforcing the cohort’s commitment to advancing digital assurance in the energy sector.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Latency Analysis of the Nexus Digital Twin Framework

Real-time digital catalogs are increasingly relied upon to track metadata and connect disparate data sources for cloud-based data integration efforts. One such tool, Deeplynx Nexus is supporting real-time digital twin efforts through event-driven data integration and time-series queries. Nexus’s usefulness for these applications depends critically on how quickly individual records can be uploaded and downloaded, since delays directly affect the responsiveness of any system built on top of it. However, the actual latency a user should expect from Nexus has not been systematically measured before, particularly for the small, frequent transactions typical of live sensor feeds. Here we show that single-record round-trip latency is 61.1 ms on a local Nexus instance and 391.7 ms on the hosted production infrastructure, a roughly 6.4x difference driven primarily by fixed per-request overhead rather than data volume. This overhead dominates at small scale: comparing single-record and ten-record trials suggests approximately 56 ms of each single-record request is fixed connection and authentication cost rather than data-transfer time, meaning batching even a handful of records is substantially more efficient than transmitting them individually. At large batch sizes, this pattern reverses for uploads, which converge to near parity between local and hosted environments by 25,000-50,000 records, while download latency remains persistently 5.7-6.4x slower on hosted infrastructure even at scale. These results suggest that Nexus deployments intended for real-time digital twin applications should prioritize record batching over single-record transactions, and that download-path optimization on hosted infrastructure offers the largest remaining opportunity to reduce latency at scale. We anticipate these baseline measurements will serve as a reference point for future digital twin projects evaluating whether Nexus’s latency profile meets their real-time requirements, and as a benchmark for tracking the effect of future infrastructure or API changes.

99 - GENERAL AND MISCELLANEOUS↗