VEDA Visualization Exploration & Data Analysis
Explore the source record for details and available documents.
Data Science↗
SEARCH · Search NASA
Results for “cloud computing; data analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Study of sensor spectral responses and data processing algorithms and architectures for onboard feature identification
A computational model of the deterministic and stochastic processes involved in remote sensing is used to study spectral feature identification techniques for real-time onboard processing of data acquired with advanced earth-resources sensors. Preliminary results indicate that: Narrow spectral responses are advantageous; signal normalization improves mean-square distance (MSD) classification accuracy but tends to degrade maximum-likelihood (MLH) classification accuracy; and MSD classification of normalized signals performs better than the computationally more complex MLH classification when imaging conditions change appreciably from those conditions during which reference data were acquired. The results also indicate that autonomous categorization of TM signals into vegetation, bare land, water, snow and clouds can be accomplished with adequate reliability for many applications over a reasonably wide range of imaging conditions. However, further analysis is required to develop computationally efficient boundary approximation algorithms for such categorization.
Huck, F. O.↗
Using Big Data Technologies with Earth Science Data in HDF5: HDF5 Scalable Solutions
HDF5 (Hierarchical Data Format 5) is open-source, high-performance software that consists of an abstract data model, library, and fileformat used for storing and managing extremely large and/or complex data collections. NASA Earth Observing System (EOS) Data and Information Systems use HDF5 as an archival format to store remote sensing data from EOS satellites. HDF5 is also used to store other types of Geoscience and Strophysical data, e.g., seismic data and data from Low-Frequency Array (LOFAR) radio telescopes. Data stored in HDF5 has reached tens of petabytes and is growing at an accelerated rate.With the growing amout of HDF5 Earth Science data to analyze and process, scientists need to adopt big data technologies including new storage paradigms such as cloud and object storage. To run models and perform data analysis they also need to utilizied efficient and diverse ways to access data, from high-performance computing's (HPC) Message Passing Interface (MPI) I/O and deep memory hierarchies (DMH) to non-HPC frameworks such as Apache Hadoop, Spark, and Drill. The HDF Group continually works to enable usage of big data technologies in HDF software.
Knox, Larry↗
Hydrology Copilot: A Cloud-Native Ai System for Hydrological Data Analysis
The emergence of AI-driven Earth observation systems promises to broaden access to petabyte-scale geospatial data beyond domain specialists. However, translating this vision into operational scientific infrastructure requires addressing fundamental challenges in data virtualization, code transparency, and domain-specific reasoning. We present Hydrology Copilot, a cloud-native AI framework for natural-language-driven analysis of Earth observation data. To demonstrate operational capabilities at scale, we implement the system using NASA's North American Land Data Assimilation System version 3 (NLDAS-3), which provides surface meteorological forcing and land-surface model output across North and Central America at 1-km resolution, from which drought diagnostics are derived. The system integrates five core contributions: (1) scalable data virtualization using Kerchunk-based cloud optimized access, achieving a 1.5 to 4.6 times improvement in I/O latency across benchmark queries spanning regional single-day extractions (4.6 times speedup) to continental monthly aggregations (1.5 times speedup); (2) transparent code generation through Microsoft Azure AI Foundry agents that expose executable Python workflows for scientific verification; (3) persistent conversational memory enabling multi-turn analytical discourse across sessions; (4) intelligent query validation that enforces dataset boundaries and resolves ambiguous requests before execution; and (5) a multi-agent architecture coordinating query parsing, code generation, and visualization. We evaluate the system through drought-monitoring workflows, demonstrating reliable code generation, accurate results validated against reference computations and the operational U.S. Drought Monitor, and efficient operation across increasingly complex tasks. By bridging natural-language interfaces with rigorous hydrological analysis, Hydrology Copilot advances beyond proof-of-concept demonstrations to provide a deployable framework for operational Earth science applications.
Data virtualization↗
Airborne Cloud Computing Environment (ACCE)
Airborne Cloud Computing Environment (ACCE) is JPL's internal investment to improve the return on airborne missions. Improve development performance of the data system. Improve return on the captured science data. The investment is to develop a common science data system capability for airborne instruments that encompasses the end-to-end lifecycle covering planning, provisioning of data system capabilities, and support for scientific analysis in order to improve the quality, cost effectiveness, and capabilities to enable new scientific discovery and research in earth observation.
airborne science↗
Performance analysis of the Spaceborne Laser Ranging System
The performance of the Spaceborne Laser Ranging System in the determination of distances between ground-based targets is discussed. The system consists of an orbiting pulsed laser distance measurement system that sequentially measures the distance to retroreflector arrays on the ground. Intersite distances are then determined from an analysis of the vertical distance data and the spacecraft orbital dynamics. Results of computer simulations indicate that for a six-day observing period with 50 percent cloud cover, uncertainties of 1 to 3 cm are expected for intertarget separations of 50 to 1200 km, with vertical uncertainties from 1 to 12 cm. Furthermore, it is found that by redetermining relative target positions, precision can be increased to less than 0.3 cm to 1 cm in the baseline and to 0.4 to 4 cm in vertical distance. It is concluded that with these precisions, strain and strain rate can be determined for crustal dynamics and earthquake studies, and geodetic surveys can be performed.
Kahn, W. D.↗
Benchmark Comparison of Cloud Analytics Methods Applied to Earth Observations
Earth Observation data are a vital resource for studying long term changes, but the large data volumes can be challenging to analyze. Time series analysis in particular is hampered by the typical thin-time-slice file organization. We examine several potential solutions inspired in large part by the data-parallel methods that have arisen with cloud computing. These solutions include various combinations of data re-organization, spatial indexing, distributed storage and pre-computation that we term "Analytics Optimized Data Stores" (AODS). We find that even simple solutions (such as a data cube) produce more than an order of magnitude improvement; the best provide two to three orders of magnitude improvement. The most performant solutions have tradeoffs in terms of generality or storage footprint, but may nonetheless be useful components in data analytics frameworks where performance is critical.
parallel processing (computers)↗
Proposed Use of the NASA Ames Nebula Cloud Computing Platform for Numerical Weather Prediction and the Distribution of High Resolution Satellite Imagery
The development of the Nebula Cloud Computing Platform at NASA Ames Research Center provides an open-source solution for the deployment of scalable computing and storage capabilities relevant to the execution of real-time weather forecasts and the distribution of high resolution satellite data to the operational weather community. Two projects at Marshall Space Flight Center may benefit from use of the Nebula system. The NASA Short-term Prediction Research and Transition (SPoRT) Center facilitates the use of unique NASA satellite data and research capabilities in the operational weather community by providing datasets relevant to numerical weather prediction, and satellite data sets useful in weather analysis. SERVIR provides satellite data products for decision support, emphasizing environmental threats such as wildfires, floods, landslides, and other hazards, with interests in numerical weather prediction in support of disaster response. The Weather Research and Forecast (WRF) model Environmental Modeling System (WRF-EMS) has been configured for Nebula cloud computing use via the creation of a disk image and deployment of repeated instances. Given the available infrastructure within Nebula and the "infrastructure as a service" concept, the system appears well-suited for the rapid deployment of additional forecast models over different domains, in response to real-time research applications or disaster response. Future investigations into Nebula capabilities will focus on the development of a web mapping server and load balancing configuration to support the distribution of high resolution satellite data sets to users within the National Weather Service and international partners of SERVIR.
Limaye, Ashutosh S.↗
Remote Sensing of Aircraft Contrails Using a Field Portable Digital Array Scanned Interferometer
With a Digital Array Scanned Interferometer (DASI), we have obtained proof-of-concept observations with which we demonstrate DASI capabilities for the determination of contrail properties. These include the measurement of the cloud and soot microphysical parameters, as well, the abundances of specific pollutant species such as SO(sub x) or NO(sub x). From high quality hyperspectral data and using radiative transfer methods and atmospheric chemistry analysis in the data reduction and interpretation, powerful inferences concerning cloud formation, evolution and dissipation can be made. Under this sub-topic, we will integrate DASI with computer controlled scanning of the field-of-view to direct the sensor towards contrails and exhaust plumes for tracking the emitting vehicles. The optimum DASI wavelength sensitivity range for sensing contrails is 0.35 - 2.5 micron. DASI deploys on the ground or from aircraft to observe contrails in the vicinity. This enables rapid, accurate measurement of the temporal, spatial, and chemical evolution of contrails (or other plumes or exhaust sources) with a low cost, efficient sensor.
Smith, William Hayden↗
Validation Ice Crystal Icing Engine Test in the Propulsion Systems Laboratory at NASA Glenn Research Center
The Propulsion Systems Laboratory (PSL) is an existing altitude simulation jet engine test facility located at NASA Glenn Research Center in Clevleand, OH. It was modified in 2012 with the integration of an ice crystal cloud generation system. This paper documents the inaugural ice crystal cloud test in PSLthe first ever full scale, high altitude ice crystal cloud turbofan engine test to be conducted in a ground based facility. The test article was a Lycoming ALF502-R5 high bypass turbofan engine, serial number LF01. The objectives of the test were to validate the PSL ice crystal cloud calibration and engine testing methodologies by demonstrating the capability to calibrate and duplicate known flight test events that occurred on the same LF01 engine and to generate engine data to support fundamental and computational research to investigate and better understand the physics of ice crystal icing in a turbofan engine environment while duplicating known revenue service events and conducting test points while varying facility and engine parameters. During PSL calibration testing it was discovered than heated probes installed through tunnel sidewalls experienced ice buildup aft of their location due to ice crystals impinging upon them, melting and running back. Filtered city water was used in the cloud generation nozzle system to provide ice crystal nucleation sites. This resulted in mineralization forming on flow path hardware that led to a chronic degradation of performance during the month long test. Lacking internal flow path cameras, the response of thermocouples along the flow path was interpreted as ice building up. Using this interpretation, a strong correlation between total water content (TWC) and a weaker correlation between median volumetric diameter (MVD) of the ice crystal cloud and the rate of ice buildup along the instrumented flow path was identified. For this test article the engine anti-ice system was required to be turned on before ice crystal icing would occur. The ice crystal icing event, an uncommanded reduction in thrust, was able to be turned on and off by manipulating cloud TWC. A flight test point where no ice crystal icing event occurred was also duplicated in PSL. Physics based computational tools were successfully used to predict tunnel settings to induce ice buildup along the low pressure compression system flow path for several test points at incrementally lower altitudes, demonstrating that development of ice crystal icing scaling laws is potentially feasible. Analysis of PSL test data showed that uncommanded reduction in thrust occurs during ice crystal cloud on operation prior to fan speed reduction. This supports previous findings that the reduction of thrust for this test article is due to ice buildup leading to a restricted airflow from either physical or aerodynamic blockage in the engine core flow path.
Ice Crystal Icing↗
Validation Ice Crystal Icing Engine Test in the Propulsion Systems Laboratory at NASA Glenn Research Center
The Propulsion Systems Laboratory (PSL) is an existing altitude simulation jet engine test facility located at NASA Glenn Research Center in Cleveland, OH. It was modified in 2012 with the integration of an ice crystal cloud generation system. This paper documents the inaugural ice crystal cloud test in PSL--the first ever full scale, high altitude ice crystal cloud turbofan engine test to be conducted in a ground based facility. The test article was a Lycoming ALF502-R5 high bypass turbofan engine, serial number LF01. The objectives of the test were to validate the PSL ice crystal cloud calibration and engine testing methodologies by demonstrating the capability to calibrate and duplicate known flight test events that occurred on the same LF01 engine and to generate engine data to support fundamental and computational research to investigate and better understand the physics of ice crystal icing in a turbofan engine environment while duplicating known revenue service events and conducting test points while varying facility and engine parameters. During PSL calibration testing it was discovered than heated probes installed through tunnel sidewalls experienced ice buildup aft of their location due to ice crystals impinging upon them, melting and running back. Filtered city water was used in the cloud generation nozzle system to provide ice crystal nucleation sites. This resulted in mineralization forming on flow path hardware that led to a chronic degradation of performance during the month long test. Lacking internal flow path cameras, the response of thermocouples along the flow path was interpreted as ice building up. Using this interpretation, a strong correlation between total water content (TWC) and a weaker correlation between median volumetric diameter (MVD) of the ice crystal cloud and the rate of ice buildup along the instrumented flow path was identified. For this test article the engine anti-ice system was required to be turned on before ice crystal icing would occur. The ice crystal icing event, an uncommanded reduction in thrust, was able to be turned on and off by manipulating cloud TWC. A flight test point where no ice crystal icing event occurred was also duplicated in PSL. Physics based computational tools were successfully used to predict tunnel settings to induce ice buildup along the low pressure compression system flow path for several test points at incrementally lower altitudes, demonstrating that development of ice crystal icing scaling laws is potentially feasible. Analysis of PSL test data showed that uncommanded reduction in thrust occurs during ice crystal cloud on operation prior to fan speed reduction. This supports previous findings that the reduction of thrust for this test article is due to ice buildup leading to a restricted airflow from either physical or aerodynamic blockage in the engine core flow path.
Turbofan Engine↗
An eight-month climatology of marine stratocumulus cloud fraction, albedo, and integrated liquid water
As part of the FIRE/Extended Time Observations (ETO) program, extended time observations were made at San Nicolas Island (SNI) from March to October, 1987. Hourly averages of air temperature, relative humidity, wind speed and direction, solar irradiance, and downward longwave irradiance were recorded. The radiation sensors were standard Eppley pyranometers (shortwave) and pyrgeometers (longwave). The SNI data were processed in several ways to deduce properties of the stratocumulus covered marine boundary layer (MBL). For example, from the temperature and humidity the lifting condensation level, which is an estimate of the height of the cloud bottom, can be computed. A combination of longwave irradiance statistics can be used to estimate fractional cloud cover. An analysis technique used to estimate the integrated cloud liquid water content (W) and the cloud albedo from the measured solar irradiance is also described. In this approach, the cloud transmittance is computed by dividing the irradiance measured at some time by a clear sky value obtained at the same hour on a cloudless day. From the transmittance and the zenith angle, values of cloud albedo and W are computed using the radiative transfer parameterizations of Stephens (1978). These analysis algorithms were evaluated with 17 days of simultaneous and colocated mm-wave (20.6 and 31.65 GHz) radiometer measurements of W and lidar ceilometer measurements of cloud fraction and cloudbase height made during the FIRE IFO. The algorithms are then applied to the entire data set to produce a climatology of these cloud properties for the eight month period.
Fairall, C. W.↗
SEPAC data analysis in support of the environmental interaction program
The main objective is to conduct data analyses of SEPAC data and computer modeling to investigate spacecraft environment effects associated with injection of electron beam, plasma clouds, and neutral gas clouds from the Shuttle Orbiter. To understand the dependence of spacecraft charging potential on beam density and other plasma parameters, a two dimensional electrostatic particle code was used to simulate the injection of electron beams from an infinite conductor into a plasma. The ionization effects on spacecraft charging are examined by including interactions of electrons with neutral gases. A survey of the simulation results is presented and discussed.
Lin, C. S.↗
Spinoff 2015
Topics covered include: 3D Endoscope to Boost Safety, Cut Cost of Surgery; Audio App Brings a Better Night's Sleep Liquid Cooling Technology Increases Exercise Efficiency; Algae-Derived Dietary Ingredients Nourish Animals; Space Grant Research Launches Rehabilitation Chair; Vision Trainer Teaches Focusing Techniques at Home; Aircraft Geared Architecture Reduces Fuel Cost and Noise; Ubiquitous Supercritical Wing Design Cuts Billions in Fuel Costs; Flight Controller Software Protects Lightweight Flexible Aircraft; Cabin Pressure Monitors Notify Pilots to Save Lives; Ionospheric Mapping Software Ensures Accuracy of Pilots' GPS; Water Mapping Technology Rebuilds Lives in Arid Regions; Shock Absorbers Save Structures and Lives during Earthquakes; Software Facilitates Sharing of Water Quality Data Worldwide; Underwater Adhesives Retrofit Pipelines with Advanced Sensors; Laser Imaging Video Camera Sees through Fire, Fog, Smoke; 3D Lasers Increase Efficiency, Safety of Moving Machines; Air Revitalization System Enables Excursions to the Stratosphere; Magnetic Fluids Deliver Better Speaker Sound Quality; Bioreactor Yields Extracts for Skin Cream; Private Astronaut Training Prepares Commercial Crews of Tomorrow; Activity Monitors Help Users Get Optimum Sun Exposure; LEDs Illuminate Bulbs for Better Sleep, Wake Cycles; Charged Particles Kill Pathogens and Round Up Dust; Balance Devices Train Golfers for a Consistent Swing; Landsat Imagery Enables Global Studies of Surface Trends; Ruggedized Spectrometers Are Built for Tough Jobs; Gas Conversion Systems Reclaim Fuel for Industry; Remote Sensing Technologies Mitigate Drought; Satellite Data Inform Forecasts of Crop Growth; Probes Measure Gases for Environmental Research; Cloud Computing Technologies Facilitate Earth Research; Software Cuts Homebuilding Costs, Increases Energy Efficiency; Portable Planetariums Teach Science; Schedule Analysis Software Saves Time for Project Planners; Sound Modeling Simplifies Vehicle Noise Management; Custom 3D Printers Revolutionize Space Supply Chain; Improved Calibration Shows Images' True Colors; Micromachined Parts Advance Medicine, Astrophysics, and More; Metalworking Techniques Unlock a Unique Alloy; Low-Cost Sensors Deliver Nanometer-Accurate Measurements; Electrical Monitoring Devices Save on Time and Cost; Dry Lubricant Smooths the Way for Space Travel, Industry; and Compact Vapor Chamber Cools Critical Components.
Source record↗
RACORO Continental Boundary Layer Cloud Investigations: 1. Case Study Development and Ensemble Large-Scale Forcings
Observation-based modeling case studies of continental boundary layer clouds have been developed to study cloudy boundary layers, aerosol influences upon them, and their representation in cloud- and global-scale models. Three 60 h case study periods span the temporal evolution of cumulus, stratiform, and drizzling boundary layer cloud systems, representing mixed and transitional states rather than idealized or canonical cases. Based on in situ measurements from the Routine AAF (Atmospheric Radiation Measurement (ARM) Aerial Facility) CLOWD (Clouds with Low Optical Water Depth) Optical Radiative Observations (RACORO) field campaign and remote sensing observations, the cases are designed with a modular configuration to simplify use in large-eddy simulations (LES) and single-column models. Aircraft measurements of aerosol number size distribution are fit to lognormal functions for concise representation in models. Values of the aerosol hygroscopicity parameter, kappa, are derived from observations to be approximately 0.10, which are lower than the 0.3 typical over continents and suggestive of a large aerosol organic fraction. Ensemble large-scale forcing data sets are derived from the ARM variational analysis, European Centre for Medium-Range Weather Forecasts, and a multiscale data assimilation system. The forcings are assessed through comparison of measured bulk atmospheric and cloud properties to those computed in "trial" large-eddy simulations, where more efficient run times are enabled through modest reductions in grid resolution and domain size compared to the full-sized LES grid. Simulations capture many of the general features observed, but the state-of-the-art forcings were limited at representing details of cloud onset, and tight gradients and high-resolution transients of importance. Methods for improving the initial conditions and forcings are discussed. The cases developed are available to the general modeling community for studying continental boundary clouds.
Boundary layers↗
Global precipitation estimates from satellite - Using difference fields of outgoing long-wave radiation
Global precipitation estimates using satellite data are derived using difference fields of outgoing long-wave radiation (OLR). The difference fields consist of clear OLR minus cloudy OLR, which is a measure of long-wave cloud radiative forcing at the top of the earth-atmosphere system; and clear daytime OLR minus clear night-time OLR, which is a measure of the diurnal variation of surface heating. All geophysical parameters used to compute OLR are derived from an analysis of the HIRS2/MSU sounding data. The derived global precipitation estimates show good agreement with collocated raingage data over land.
Wu, Man-Li↗
Automatic analysis of stereoscopic satellite image pairs for determination of cloud-top height and structure
Results are presented on an automatic stereo analysis of cloud-top heights from nearly simultaneous satellite image pairs from the GOES and NOAA satellites, using a massively parallel processor computer. Comparisons of computer-derived height fields and manually analyzed fields show that the automatic analysis technique shows promise for performing routine stereo analysis in a real-time environment, providing a useful forecasting tool by augmenting observational data sets of severe thunderstorms and hurricanes. Simulations using synthetic stereo data show that it is possible to automatically resolve small-scale features such as 4000-m-diam clouds to about 1500 m in the vertical.
Hasler, A. F.↗
Hyperspectral Data Processing: an Opportunity for End-To-End Processing
The evolution and improvements in hyperspectral instrumentation are being matched by information technology improvements in science data processing and analysis. Research has improved techniques in both onboard and ground-based processing to support other high data volume instruments. Algorithms and hardware have evolved, permitting faster access to the observations. Cloud computing is taking the algorithms to the data. Technologies are being specifically designed to address high volume data sets and are an investment in the improvement of hyperspectral data processing.
science data processing↗