Search NASA⌕ Search

SEARCH · Search NASA

Results for “data retrieval”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Team 2: AIRS Only Retrieval

This slide presentation makes the case for the retrieval of data from the Atmospheric Infrared Sounder (AIRS) Advanced Microwave Sounding Unit (AMSU). AIRS only retrieval is not only a risk reduction for failure of AMSU, but also important because NWP centers are reluctant to assimilate AMSU twice.

cloudy regression↗

Analysis of CrIs/ATMS Using AIRS Version-7 Retrieval and QC Methodology

The objective of this research is to develop and implement an algorithm to analyze a long term data record of CrIS/ATMS observations so as to produce monthly mean gridded Level-3 products which are consistent with, and will serve as a seamless follow on to, those of AIRS Version-7. We feel the best way to achieve this result is to analyze CrIS/ATMS data using retrieval and Quality Control (QC) methodologies which are scientifically equivalent to those used in AIRS Version-7. We developed and implemented a single retrieval program that uses as input either AIRS/AMSU or CrIS/ATMS radiance observations, and has appropriate switches that take into account the spectral and radiometric differences between CrIS and AIRS. Our methodology is call CHART (Climate Heritage AIRS Retrieval Technique).

Susskind, Joel↗

Retrieval of biophysical parameters with AVIRIS and ISM: The Landes Forest, south west France

The first steps of an experiment for investigating the capability of airborne spectrometer data for retrieval of biophysical parameters of vegetation, especially water conditions are presented. Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) and ISM data were acquired in the frame of the 1991 NASA/JPL and CNES campaigns on the Landes, South west France, a large and flat forest area with mainly maritime pines. In-situ measurements were completed at that time; i.e. reflectance spectra, atmospheric profiles, sampling for further laboratory analyses of elements concentrations (lignin, water, cellulose, nitrogen,...). All information was integrated in an already existing data base (age, LAI, DBH, understory cover,...). A methodology was designed for (1) obtaining geometrically and atmospherically corrected reflectance data, (2) registering all available information, and (3) analyzing these multi-source informations. Our objective is to conduct comparative studies with simulation reflectance models, and to improve these models, especially in the MIR.

Zagolski, F.↗

Development of Integrated Programs for Aerospace-vehicle design (IPAD): Integrated information processing requirements

The engineering-specified requirements for integrated information processing by means of the Integrated Programs for Aerospace-Vehicle Design (IPAD) system are presented. A data model is described and is based on the design process of a typical aerospace vehicle. General data management requirements are specified for data storage, retrieval, generation, communication, and maintenance. Information management requirements are specified for a two-component data model. In the general portion, data sets are managed as entities, and in the specific portion, data elements and the relationships between elements are managed by the system, allowing user access to individual elements for the purpose of query. Computer program management requirements are specified for support of a computer program library, control of computer programs, and installation of computer programs into IPAD.

Southall, J. W.↗

Measurement of the Vertical Distribution of Aerosol by Globally Distributed MP Lidar Network Sites

The global distribution of aerosol has an important influence on climate through the scattering and absorption of shortwave radiation and through modification of cloud optical properties. Current satellite and other data already provide a great amount of information on aerosol distribution. However there are critical parameters that can only be obtained by active optical profiling. For aerosol, no passive technique can adequately resolve the height profile of aerosol. The aerosol height distribution is required for any model for aerosol transport and the height resolved radiative heating/cooling effect of aerosol. The Geoscience Laser Altimeter System (GLAS) is an orbital lidar to be launched by 2002. GLAS will provide global measurements of the height distribution of aerosol. The sampling will be limited by nadir only coverage. There is a need for local sites to address sampling, and accuracy factors. Full time measurements of the vertical distribution of aerosol are now being acquired at a number of globally distributed MP (micro pulse) lidar sites. The MP lidar systems provide profiling of all significant cloud and aerosol to the limit of signal attenuation from compact, eye safe instruments. There are currently six sites in operation and over a dozen planned. At all sites there are a complement of passive aerosol and radiation measurements supporting the lidar data. Four of the installations are at Atmospheric Radiation Measurement program sites. The aerosol measurements, retrievals and data products from the network sites will be discussed. The current and planned application of data to supplement satellite aerosol measurements is covered.

Spinhirne, James↗

Revamping Spacecraft Operational Intelligence

The EPOXI flight mission has been testing a new commercial system, Splunk, which employs data mining techniques to organize and present spacecraft telemetry data in a high-level manner. By abstracting away data-source specific details, Splunk unifies arbitrary data formats into one uniform system. This not only reduces the time and effort for retrieving relevant data, but it also increases operational visibility by allowing a spacecraft team to correlate data across many different sources. Splunk's scalable architecture coupled with its graphing modules also provide a solid toolset for generating data visualizations and building real-time applications such as browser-based telemetry displays.

Hwang, Victor↗

Millimeter-Wave Imaging Radiometer (MIR) Data Processing and Development of Water Vapor Retrieval Algorithms

This document describes the final report of the Millimeter-wave Imaging Radiometer (MIR) Data Processing and Development of Water Vapor Retrieval Algorithms. Volumes of radiometric data have been collected using airborne MIR measurements during a series of field experiments since May 1992. Calibrated brightness temperature data in MIR channels are now available for studies of various hydrological parameters of the atmosphere and Earth's surface. Water vapor retrieval algorithms using multichannel MIR data input are developed for the profiling of atmospheric humidity. The retrieval algorithms are also extended to do three-dimensional mapping of moisture field using continuous observation provided by airborne sensor MIR or spaceborne sensor SSM/T-2. Validation studies for water vapor retrieval are carried out through the intercomparison of collocated and concurrent measurements using different instruments including lidars and radiosondes. The developed MIR water vapor retrieval algorithm is capable of humidity profiling under meteorological conditions ranging from clear column to moderately cloudy sky. Simulative water vapor retrieval studies using extended microwave channels near 183 and 557 GHz strong absorption lines indicate feasibility of humidity profiling to layers in the upper troposphere and improve the overall vertical resolution through the atmosphere.

Chang, L. Aron↗

Effects of Tunable Data Compression on Geophysical Products Retrieved from Surface Radar Observations with Applications to Spaceborne Meteorological Radars

This paper presents results and analyses of applying an international space data compression standard to weather radar measurements that can easily span 8 orders of magnitude and typically require a large storage capacity as well as significant bandwidth for transmission. By varying the degree of the data compression, we analyzed the non-linear response of models that relate measured radar reflectivity and/or Doppler spectra to the moments and properties of the particle size distribution characterizing clouds and precipitation. Preliminary results for the meteorologically important phenomena of clouds and light rain indicate that for a 0.5 dB calibration uncertainty, typical for the ground-based pulsed-Doppler 94 GHz (or 3.2 mm, W-band) weather radar used as a proxy for spaceborne radar in this study, a lossless compression ratio of only 1.2 is achievable. However, further analyses of the non-linear response of various models of rainfall rate, liquid water content and median volume diameter show that a lossy data compression ratio exceeding 15 is realizable. The exploratory analyses presented are relevant to future satellite missions, where the transmission bandwidth is premium and storage requirements of vast volumes of data, potentially problematic.

cloud properties↗

Physical retrieval of precipitation water contents using multisensor microwave data and model constraints

A physical retrieval method is proposed for estimating precipitation water distributions and other geophysical parameters from the Special Sensor Microwave/Imager (SSM/I) data. In the proposed method, sensor antenna functions are explicitly included to accommodate varying channel resolution. An embedded one-dimensional cloud model is utilized to generate vertical distributions of precipitating and nonprecipitating water, and spectral solutions are sought for certain background parameters, such as humidity, which vary more slowly in the horizontal than the cloud and precipitation water contents. The method allows the incorporation of measurements from SSM/T and geostationary infrared measurements in addition to SSM/I data, as well as information from conventional sources (e.g., radiosondes) or numerical forecast model fields.

Olson, William S.↗

Compression through decomposition into browse and residual images

Economical archival and retrieval of image data is becoming increasingly important considering the unprecedented data volumes expected from the Earth Observing System (EOS) instruments. For cost effective browsing the image data (possibly from remote site), and retrieving the original image data from the data archive, we suggest an integrated image browse and data archive system employing incremental transmission. We produce our browse image data with the JPEG/DCT lossy compression approach. Image residual data is then obtained by taking the pixel by pixel differences between the original data and the browse image data. We then code the residual data with a form of variable length coding called diagonal coding. In our experiments, the JPEG/DCT is used at different quality factors (Q) to generate the browse and residual data. The algorithm has been tested on band 4 of two Thematic mapper (TM) data sets. The best overall compression ratios (of about 1.7) were obtained when a quality factor of Q=50 was used to produce browse data at a compression ratio of 10 to 11. At this quality factor the browse image data has virtually no visible distortions for the images tested.

Novik, Dmitry A.↗

Machine learning based noise reduction for satellite products: application to solar-induced fluorescence retrievals using simulated and real data

In the past two decades, global satellite measurements of terrestrial chlorophyll solar-induced fluorescence (SIF) have been used widely for a number of different applications related to physiology, phenology, and productivity of plants. However, SIF retrievals are inherently noisy due to the relatively small SIF spectral signature in comparison with observational noise. In this work, we examine how a spectral-based approach that employs principal component analysis along with a relatively shallow artificial neural network can be used to reduce noise and other artifacts in satellite level 2 (L2) products. We first apply the approach in a controlled environment in which radiance spectra are simulated with a full atmospheric and surface radiative transfer model for different scenarios including various SIF values that are known. Various levels of noise can be added to the simulated spectra. Resulting noisy and noise-reduced SIF retrievals are compared with the true values to assess performance. We then apply the noise reduction approach to real SIF derived from instruments flying on meteorological satellites. The results are evaluated by comparing SIF retrievals from different platforms with each other and with other independent data sets, showing enhanced capability to capture seasonal and interannual variability in SIF.

Chlorophyll fluorescence↗

EOS MLS Level 2 Data Processing Software Version 3

This software accepts the EOS MLS calibrated measurements of microwave radiances products and operational meteorological data, and produces a set of estimates of atmospheric temperature and composition. This version has been designed to be as flexible as possible. The software is controlled by a Level 2 Configuration File that controls all aspects of the software: defining the contents of state and measurement vectors, defining the configurations of the various forward models available, reading appropriate a priori spectroscopic and calibration data, performing retrievals, post-processing results, computing diagnostics, and outputting results in appropriate files. In production mode, the software operates in a parallel form, with one instance of the program acting as a master, coordinating the work of multiple slave instances on a cluster of computers, each computing the results for individual chunks of data. In addition, to do conventional retrieval calculations and producing geophysical products, the Level 2 Configuration File can instruct the software to produce files of simulated radiances based on a state vector formed from a set of geophysical product files taken as input. Combining both the retrieval and simulation tasks in a single piece of software makes it far easier to ensure that identical forward model algorithms and parameters are used in both tasks. This also dramatically reduces the complexity of the code maintenance effort.

Livesey, Nathaniel J.↗

Earth observing system. Output data products and input requirements, version 2.0. Volume 2: Analysis of IDS input requirements

On 18 Jan. 1991, NASA confirmed 29 Inter-Disciplinary Science (IDS) teams, each involving a group of investigators, to conduct interdisciplinary research using data products from Earth Observing System (EOS) instruments. These studies are multi-disciplinary and require output data products from multiple EOS instruments, including both FI and PI instruments. The purpose of this volume is to provide information on output products expected from IDS investigators, required input data, and retrieval algorithms. Also included in this volume is the revised analysis of the 'best' and 'alternative' match data products for IDS input requirements. The original analysis presented in the August 1991 release of the SPSO Report was revised to incorporate the restructuring of the EOS platform. As a result of the reduced EOS payload, some of EOS instruments were deselected and their data products would not be available for IDS research. Information on these data products is also presented.

Lu, Yun-Chi↗

MR-CDF: Managing multi-resolution scientific data

MR-CDF is a system for managing multi-resolution scientific data sets. It is an extension of the popular CDF (Common Data Format) system. MR-CDF provides a simple functional interface to client programs for storage and retrieval of data. Data is stored so that low resolution versions of the data can be provided quickly. Higher resolutions are also available, but not as quickly. By managing data with MR-CDF, an application can be relieved of the low-level details of data management, and can easily trade data resolution for improved access time.

Salem, Kenneth↗

Evaluation of cloud height, optical thickness, and phase retrievals from the CHROMA algorithm applied to Sentinel-3 OLCI data

We previously developed the Cloud Height Retrieval from O 2 Molecular Absorption (CHROMA) algorithm for the Ocean Color Instrument (OCI) on the new NASA Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission. Here, we apply CHROMA to observations from the Ocean Land Colour Instrument (OLCI) to guide expectations for PACE, as it will take some time to obtain large-scale validation data for OCI. We use cloud top height (CTH), phase, and (for liquid clouds) cloud optical thickness (COT) data from the ground-based Atmospheric Radiation Measurement (ARM) network to evaluate the OLCI retrievals. We found that OLCI and Moderate Resolution Imaging Spectroradiometer (MODIS) CTH compare similarly well to the ARM reference. OLCI has a tendency to underestimate CTH as CTH increases, and algorithm assumptions about cloud geometric thickness may contribute to this. ARM COT from multifilter shadowband radiometers (MFRSR) and Sun photometers are well-correlated with one another, albeit with a roughly 30 % offset on average; OLCI and MODIS COT agree more closely with the MFRSR data. OLCI retrieval uncertainty estimates show skill at telling low-uncertainty cases from high-uncertainty ones, although CTH uncertainties are underestimated. Additionally, we compare the OLCI data to satellite retrievals based on thermal infrared measurements from MODIS and Sea and Land Surface Temperature Radiometer (SLSTR) data. Differences are broadly consistent with physical expectations based on the A-band vs. thermal techniques, although one key challenge in such aggregated comparisons is different cloud masking sensitivities and algorithm failure rates meaning additional sampling differences are introduced. We conclude by discussing the transition to and possible enhancements for PACE OCI.

Sayer, Andrew M. [Univ. of Maryland Baltimore Coun↗

Climatology of Global Precipitation Measurement Mission Precipitation Regimes and Implications for Global Estimates of Vertical Winds

The Global Precipitation Measurement (GPM) mission Validation Network (VN) framework leverages over 118 ground-based polarimetric Doppler radars to validate a large subset of precipitation measurements and retrievals from the GPM Dual-frequency Precipitation Radar (DPR). Recently, GPM DPR reflectivity profiles within the VN have been classified according to their convective regime using unsupervised machine learning techniques. The archetypal regimes are stratiform, convective, mixed stratiform-convective (e.g., transition regions), and “other” (e.g., peripheral regions of light precipitation). Subcategories within these four primary regimes vary according to the characteristic depth of included reflectivity profiles, resulting in 12 main GPM DPR precipitation profile categories. Polarimetry of ground-based Doppler radars in the VN offers additional insights into the types of precipitation, while pairs of radars positioned near each other enable retrieval of vertical winds via dual-Doppler analysis. Geometrically matched to the DPR reflectivity profiles in the GPM VN, these ground-based data and retrievals contribute more detailed characterization of the distinct kinematic and microphysical structures associated with each of the 12 DPR precipitation regimes. DPR reflectivity profiles linked with wind in the VN are restricted to GPM overpasses of proximal radar pairs that allow dual-Doppler analysis. Although a limited subset of DPR profiles in the VN are matched with vertical motion, agreement between the reflectivity structures paired with wind data and those of the greater DPR dataset in the VN suggest that estimates of vertical motion may be inferred in regions without ground-based measurements. We present a climatology of the 12 convective regimes identified within the DPR VN dataset as well as early efforts to estimate the kinematic and microphysical structures of precipitation profiles within the greater GPM DPR dataset by applying machine learning techniques. Precipitation data paired with global estimates of vertical winds from these efforts offer early insight to and support upcoming missions to retrieve convective mass flux, including the Investigation of Convective Updrafts (INCUS) in the Tropics and the global Atmosphere Observing System (AOS).

Precipitation↗

GeneLab Analysis Working Group Kick-Off Meeting

Goals to achieve for GeneLab AWG - GL vision - Review of GeneLab AWG charter Timeline and milestones for 2018 Logistics - Monthly Meeting - Workshop - Internship - ASGSR Introduction of team leads and goals of each group Introduction of all members Q/A Three-tier Client Strategy to Democratize Data Physiological changes, pathway enrichment, differential expression, normalization, processing metadata, reproducibility, Data federation/integration with heterogeneous bioinformatics external databases The GLDS currently serves over 100 omics investigations to the biomedical community via open access. In order to expand the scope of metadata record searches via the GLDS, we designed a metadata warehouse that collects and updates metadata records from external systems housing similar data. To demonstrate the capabilities of federated search and retrieval of these data, we imported metadata records from three open-access data systems into the GLDS metadata warehouse: NCBI's Gene Expression Omnibus (GEO), EBI's PRoteomics IDEntifications (PRIDE) repository, and the Metagenomics Analysis server (MG-RAST). Each of these systems defines metadata for omics data sets differently. One solution to bridge such differences is to employ a common object model (COM) to which each systems' representation of metadata can be mapped. Warehoused metadata records are then transformed at ETL to this single, common representation. Queries generated via the GLDS are then executed against the warehouse, and matching records are shown in the COM representation (Fig. 1). While this approach is relatively straightforward to implement, the volume of the data in the omics domain presents challenges in dealing with latency and currency of records. Furthermore, the lack of a coordinated has been federated data search for and retrieval of these kinds of data across other open-access systems, so that users are able to conduct biological meta-investigations using data from a variety of sources. Such meta-investigations are key to corroborating findings from many kinds of assays and translating them into systems biology knowledge and, eventually, therapeutics.

GeneLab↗

Uncertainty in Soil Moisture Retrievals: an Ensemble Approach Using SMOS L-Band Microwave Data

The uncertainty of soil moisture (SM) retrievals from satellite brightness temperature (TB) observations depends primarily on the choice of radiative transfer model (RTM) parameters, prior SM information and TB inputs. This paper studies the sensitivity of several (quasi-)operational and experimental SM retrieval products from the Soil Moisture Ocean Salinity (SMOS) mission to these choices at 11 reference sites, located in 7 watersheds across the United States (US). Different literature-based RTM parameter sets cause large biases between retrievals. Whereas typical RTM parameter sets are calibrated for SM retrievals, it is shown that a parameter set carefully optimized for TB forward modeling can also be used for retrieving SM. It is also shown that the inclusion of dynamic prior SM estimates in a Bayesian retrieval scheme can strongly improve SM retrievals, regardless of the choice of RTM parameters, and that the use of multi-angular and multi-polarization TB does not necessarily lead to superior retrievals compared to retrievals based on TB data at a single incidence angle and polarization. The second part of this paper evaluates ensemble uncertainty metrics for SM retrievals obtained by propagating a wide range of RTM parameters through the RTM. As expected for bounded variables, the spread in the ensemble SM retrievals is smallest for wet and dry SM values and highest for intermediate SM values. After removal of the strong long-term mean bias associated with the RTM parameter values for individual ensemble members, the remaining anomaly ensemble SM spread of 0.037 cu m/cu m approximates the actual time series unbiased root-mean-square-difference of 0.042 cu m/cu m between ensemble mean retrievals and in situ reference data across the reference sites. However, the temporal variability in the anomaly ensemble spread reveals higher-order biases in the retrieval error, which should be accounted for when characterizing retrieval error.

Jan Quets↗