SEARCH · Search NASA

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

All sources NASA NTRS DOE OSTI

NASA NTRS1998

Data Mining and Optimization Tools for Developing Engine Parameters Tools

This project was awarded for understanding the problem and developing a plan for Data Mining tools for use in designing and implementing an Engine Condition Monitoring System. Tricia Erhardt and I studied the problem domain for developing an Engine Condition Monitoring system using the sparse and non-standardized datasets to be available through a consortium at NASA Lewis Research Center. We visited NASA three times to discuss additional issues related to dataset which was not made available to us. We discussed and developed a general framework of data mining and optimization tools to extract useful information from sparse and non-standard datasets. These discussions lead to the training of Tricia Erhardt to develop Genetic Algorithm based search programs which were written in C++ and used to demonstrate the capability of GA algorithm in searching an optimal solution in noisy, datasets. From the study and discussion with NASA LeRC personnel, we then prepared a proposal, which is being submitted to NASA for future work for the development of data mining algorithms for engine conditional monitoring. The proposed set of algorithm uses wavelet processing for creating multi-resolution pyramid of tile data for GA based multi-resolution optimal search.

Dhawan, Atam P.↗

NASA NTRS2002

Surface Deformation and Gravity Changes from Surface and Internal Loads

Air and space borne remote sensing have made it possible to monitor the mass and energy transport at various scales within the cryosphere-hydrosphere-atmosphere system. The recent surface mass balance (the rate of net gain of snow and ice at a geographic point) map for the Antarctic ice sheet is constructed by interpolating sparse in situ observations (about 1,800 points) with empirically calibrated satellite data of passive back emission of microwaves. The digital elevation model obtained from satellite radar altimetry is used to improve the delineation of the ice flow drainage basins. As important as these results are, the uncertainty remains up to about 2 mm/yr of eustatic sea level change with the net imbalance. In other words, we are still unable to determine even the sign of the contribution of the Antarctic ice sheet to contemporary sea level change. The problem is more likely with the discharge rather than accumulation.

Hager, Bradford H.↗

NASA NTRS2003

Terrain Portrayal for Head-Down Displays Flight Test

The Synthetic Vision Systems General Aviation (SVS-GA) element of NASA's Aviation Safety Program is developing technology to eliminate low visibility induced General Aviation (GA) accidents through the application of synthetic vision techniques. SVS displays present computer generated 3-dimensional imagery of the surrounding terrain to greatly enhance pilot's situation awareness (SA), reducing or eliminating Controlled Flight into Terrain (CFIT), as well as Low-Visibility Loss of Control (LVLOC) accidents. In addition to substantial safety benefits, SVS displays have many potential operational benefits that can lead to flight in instrument meteorological conditions (IMC) resembling those conducted in visual meteorological conditions (VMC). Potential benefits could include lower landing minimums, more approach options, reduced training time, etc. SVS conducted research will develop display concepts providing the pilot with an unobstructed view of the outside terrain, regardless of weather conditions and time of day. A critical component of SVS displays is the appropriate presentation of terrain to the pilot. The relationship between the realism of the terrain presentation and resulting enhancements of pilot SA and pilot performance has been largely undefined. Comprised of coordinated simulation and flight test efforts, the terrain portrayal for head-down displays (TP-HDD) test series examined the effects of two primary elements of terrain portrayal: variations of digital elevation model (DEM) resolution and terrain texturing. Variations in DEM resolution ranged from sparsely spaced (30 arc-sec/2,953ft) to very closely spaced data (1 arc-sec/98 ft). Variations in texture involved three primary methods: constant color, elevation-based generic, and photo-realistic, along with a secondary depth cue enhancer in the form of a fishnet grid overlay. The TP-HDD test series was designed to provide comprehensive data to enable design trades to optimize all SVS applications, as well as develop requirements and recommendations to facilitate the implementation and certification of SVS displays. The TP-HDD flight experiment utilized the NASA LaRC Cessna 206 Stationaire and evaluated eight terrain portrayal concepts in an effort to confirm and extend results from the previously conducted TP-HDD simulation experiment. A total of 15 evaluation pilots, of various qualifications, accumulated over 75 hours of dedicated research flight time at Newport News (PHF) and Roanoke (ROA), VA, airports from August through October, 2002. This report will present results from the portion of testing conducted at Roanoke, VA.

Hughes, Monica F.↗

NASA NTRS2016

Global Estimates of Fine Particulate Matter Using a Combined Geophysical-Statistical Method with Information from Satellites, Models, and Monitors

We estimated global fine particulate matter (PM(sub 2.5)) concentrations using information from satellite-, simulation- and monitor-based sources by applying a Geographically Weighted Regression (GWR) to global geophysically-based satellite-derived PM(sub 2.5) estimates. Aerosol optical depth from multiple satellite products (MISR, MODIS Dark Target, MODIS and SeaWiFS Deep Blue, and MODIS MAIAC) was combined with simulation (GEOS-Chem) based upon their relative uncertainties as determined using ground-based sun photometer (AERONET) observations for 1998−2014. The GWR predictors included simulated aerosol composition and land use information. The resultant PM(sub 2.5) estimates were highly consistent (R(sup 2) equals 0.81) with out-of-sample cross-validated PM(sub 2.5) concentrations from monitors. The global population-weighted annual average PM(sub 2.5) concentrations were 3-fold higher than the 10 micrograms per cubic meter WHO guideline, driven by exposures in Asian and African regions. Estimates in regions with high contributions from mineral dust were associated with higher uncertainty, resulting from both sparse ground-based monitoring, and challenging conditions for retrieval and simulation. This approach demonstrates that the addition of even sparse ground-based measurements to more globally continuous PM(sub 2.5) data sources can yield valuable improvements to PM(sub 2.5) characterization on a global scale.

aerosols↗

NASA NTRS1984

Tropospheric-stratospheric exchange, part 1.1A

Much of the observational evidence of large scale tropospheric-stratospheric exchange has been obtained by radiosonde and satellite radiane data. So far mesosphere-stratosphere-troposphere (MST) radars have made mininal contributions, in part due to their recent use as a meteorological tool, intermittent operation at some facilities and sparse geographic distribution. However, as more MST facilities come on-line in more locations, the good time and height resolution data throughtout the troposphere and much of the stratosphere obtainable by MST radars will enhance the detail of stratospheric and tropospheric circulations and interactions. On smaller scales MST radars have already been used to examine convective forcing from the troposphere into the stratosphere and subsequent launching of gravity waves (LARSEN et al., 1982). Observations of persistent turbulent layers in the stratosphere over Arecibo, attributable to inertial oscillations, appear to propagate away from a source region near the tropopause (SATO and WOODMAN, 1982). MST radars offer the availability of high resolution wind data in height and time needed to observe interactions between the troposphere and stratosphere. The lack of geographic coverage (e.g., equatorial regions) and insufficient data bases at many MST facilities presently inhibit studies of large-scale interactions. At present MST radars can be used to examine smaller scale interactions.

Cornish, C. R.↗

NASA NTRS2026

Incremental Learning for Passive Microwave Precipitation Retrievals using Advanced Technology Microwave Sounder

Spaceborne passive microwave (PMW) radiometry is central to global precipitation monitoring, yet retrieval uncertainties remain substantial, particularly for cross-track sounders whose variable footprints and channel configurations are optimized for atmospheric temperature and moisture profiling rather than precipitation. Consequently, existing operational products often exhibit angular-dependent biases, limited effective swath utilization, unrealistic rainfall probability distributions, and systematic misclassification of precipitation phase. These limitations are further compounded by the scarcity of globally accurate and representative precipitation observations, as training data from the Dual-frequency Precipitation Radar (DPR) and the Cloud Profiling Radar (CPR) are spatially sparse, lack uniform global coverage, and exhibit heterogeneous error characteristics across precipitation regimes. To address these challenges, this study presents a supervised retrieval algorithm that incrementally trains an ensemble of extreme gradient-boosted decision trees by augmenting base learners with pre-training on reanalysis data and post-training on coincident DPR and CPR observations matched with the Advanced Technology Microwave Sounder (ATMS). By transferring prior information from reanalysis to posterior constraints from radar observations and adopting a sequential detection–estimation strategy for precipitation phase and rate retrieval, the proposed approach yields retrievals across the full ATMS swath that are largely free from persistent deficiencies in current Global Precipitation Measurement (GPM) passive microwave operational products. In particular, the method resolves bimodal artifacts in rainfall retrievals and mitigates systematic high-latitude snowfall biases, including overestimation across the Arctic and underestimation across the Antarctic. Validation against independent Multi-Radar Multi-Sensor (MRMS) data over the Contiguous United States (CONUS) further demonstrates improved performance in precipitation phase detection and rate estimation relative to both reanalysis and current GPM PMW products.

Mahyar Garshasbi↗

NASA NTRS1995

Cloud Masking and Surface Temperature Distribution in the Polar Regions Using AVHRR and other Satellite Data

Surface temperature is one of the key variables associated with weather and climate. Accurate measurements of surface air temperatures are routinely made in meteorological stations around the world. Also, satellite data have been used to produce synoptic global temperature distributions. However, not much attention has been paid on temperature distributions in the polar regions. In the polar regions, the number of stations is very sparse. Because of adverse weather conditions and general inaccessibility, surface field measurements are also limited. Furthermore, accurate retrievals from satellite data in the region have been difficult to make because of persistent cloudiness and ambiguities in the discrimination of clouds from snow or ice. Surface temperature observations are required in the polar regions for air-sea-ice interaction studies, especially in the calculation of heat, salinity, and humidity fluxes. They are also useful in identifying areas of melt or meltponding within the sea ice pack and the ice sheets and in the calculation of emissivities of these surfaces. Moreover, the polar regions are unique in that they are the sites of temperature extremes, the location of which is difficult to identify without a global monitoring system. Furthermore, the regions may provide an early signal to a potential climate change because such signal is expected to be amplified in the region due to feedback effects. In cloud free areas, the thermal channels from infrared systems provide surface temperatures at relatively good accuracies. Previous capabilities include the use of the Temperature Humidity Infrared Radiometer (THIR) onboard the Nimbus-7 satellite which was launched in 1978. Current capabilities include the use of the Advance Very High Resolution Radiometer (AVHRR) aboard NOAA satellites. Together, these two systems cover a span of 16 years of thermal infrared data. Techniques for retrieving surface temperatures with these sensors in the polar regions have been developed. Errors have been estimated to range from 1K to 5K mainly due to cloud masking problems. With many additional channels available, it is expected that the EOS-Moderate Resolution Imaging Spectroradiometer (MODIS) will provide an improved characterization of clouds and a good discrimination of clouds from snow or ice surfaces.

Comiso, Joey C.↗

NASA NTRS1971

Interhemispheric comparison of atmospheric circulation features as evaluated from NIMBUS satellite data

Findings are presented for IRIS data from NIMBUS 3 in mapping the global ozone distribution. The seasonal and regional variations of ozone, especially in the Southern Hemisphere, reveal features that were not evident from the sparse ground-based ozone observation network in this hemisphere. A regression analysis was undertaken for temperature and height fields on radiance data. Spectrum analyses of upper wind data from the North American section and Australia were completed.

Reiter, E. R.↗

NASA NTRS1998

Data Mining and Optimization Tools for Developing Engine Parameters Tools

This project was awarded for understanding the problem and developing a plan for Data Mining tools for use in designing and implementing an Engine Condition Monitoring System. From the total budget of $5,000, Tricia and I studied the problem domain for developing ail Engine Condition Monitoring system using the sparse and non-standardized datasets to be available through a consortium at NASA Lewis Research Center. We visited NASA three times to discuss additional issues related to dataset which was not made available to us. We discussed and developed a general framework of data mining and optimization tools to extract useful information from sparse and non-standard datasets. These discussions lead to the training of Tricia Erhardt to develop Genetic Algorithm based search programs which were written in C++ and used to demonstrate the capability of GA algorithm in searching an optimal solution in noisy datasets. From the study and discussion with NASA LERC personnel, we then prepared a proposal, which is being submitted to NASA for future work for the development of data mining algorithms for engine conditional monitoring. The proposed set of algorithm uses wavelet processing for creating multi-resolution pyramid of the data for GA based multi-resolution optimal search. Wavelet processing is proposed to create a coarse resolution representation of data providing two advantages in GA based search: 1. We will have less data to begin with to make search sub-spaces. 2. It will have robustness against the noise because at every level of wavelet based decomposition, we will be decomposing the signal into low pass and high pass filters.

Dhawan, Atam P.↗

NASA NTRS1993

The growth of the UniTree mass storage system at the NASA Center for Computational Sciences

In October 1992, the NASA Center for Computational Sciences made its Convex-based UniTree system generally available to users. The ensuing months saw the growth of near-online data from nil to nearly three terabytes, a doubling of the number of CPU's on the facility's Cray YMP (the primary data source for UniTree), and the necessity for an aggressive regimen for repacking sparse tapes and hierarchical 'vaulting' of old files to freestanding tape. Connectivity was enhanced as well with the addition of UltraNet HiPPI. This paper describes the increasing demands placed on the storage system's performance and throughput that resulted from the significant augmentation of compute-server processor power and network speed.

Tarshish, Adina↗

NASA NTRS2012

Application of Satellite Gravimetry for Water Resource Vulnerability Assessment

The force of Earth's gravity field varies in proportion to the amount of mass near the surface. Spatial and temporal variations in the gravity field can be measured via their effects on the orbits of satellites. The Gravity Recovery and Climate Experiment (GRACE) is the first satellite mission dedicated to monitoring temporal variations in the gravity field. The monthly gravity anomaly maps that have been delivered by GRACE since 2002 are being used to infer changes in terrestrial water storage (the sum of groundwater, soil moisture, surface waters, and snow and ice), which are the primary source of gravity variability on monthly to decadal timescales after atmospheric and oceanic circulation effects have been removed. Other remote sensing techniques are unable to detect water below the first few centimeters of the land surface. Conventional ground based techniques can be used to monitor terrestrial water storage, but groundwater, soil moisture, and snow observation networks are sparse in most of the world, and the countries that do collect such data rarely are willing to share them. Thus GRACE is unique in its ability to provide global data on variations in the availability of fresh water, which is both vital to life on land and vulnerable to climate variability and mismanagement. This chapter describes the unique and challenging aspects of GRACE terrestrial water storage data, examples of how the data have been used for research and applications related to fresh water vulnerability and change, and prospects for continued contributions of satellite gravimetry to water resources science and policy.

Rodell, Matthew↗

NASA NTRS2020

Evaluating Drought Indices for Early Warning in East and Southern Africa

Sparsely populated regions of east and southern Africa often have little ground based data to monitor drought, crops, and water resources. The Regional Hydrologic Extremes Assessment System (RHEAS) is a NASA supported data assimilation framework that combines both a hydrologic and crop model, which provide another monitoring approach. RHEAS can also be run in forecast mode, providing outlook on drought and crop yield.

Miller, Sara↗

NASA NTRS1985

Assessment of Computer-based Geologic Mapping of Rock Units in the LANDSAT-4 Scene of Northern Death Valley, California

Geologists obtain low accuracy levels when maps derived from LANDSAT MSS data are compared with those made by conventional methods. Procedures developed for the IDIMS computer system and used to classify a subset of a TM image of the Death Valley, California - Nevada border are described. Despite the superior resolution, broader spectral coverage, and greater sensitivity inherent to the TM, the actual recorded measured accuracy was in the same narrow range (30 to 60%) recorded for MSS data from earlier LANDSATs. The supervised classification approach appears to be superior to the unsupervised approach when applied to vegetation-sparse surfaces composed of spectrally contrasting rock/soil units distributed in relatively flat to low relief terrain. As spatial resolution improves and optimal spectral bands for identifying rock materials are specified, use of classified multispectral remote sensing data from air and space when coupled with supporting field calibration and checks should become the dominant way in which geologic mapping is carried out in future decades.

Short, N. M.↗

NASA NTRS2000

Application of Statistical Methods of Rain Rate Estimation to Data From The TRMM Precipitation Radar

The TRMM Precipitation Radar is well suited to statistical methods in that the measurements over any given region are sparsely sampled in time. Moreover, the instantaneous rain rate estimates are often of limited accuracy at high rain rates because of attenuation effects and at light rain rates because of receiver sensitivity. For the estimation of the time-averaged rain characteristics over an area both errors are relevant. By enlarging the space-time region over which the data are collected, the sampling error can be reduced. However. the bias and distortion of the estimated rain distribution generally will remain if estimates at the high and low rain rates are not corrected. In this paper we use the TRMM PR data to investigate the behavior of 2 statistical methods the purpose of which is to estimate the rain rate over large space-time domains. Examination of large-scale rain characteristics provides a useful starting point. The high correlation between the mean and standard deviation of rain rate implies that the conditional distribution of this quantity can be approximated by a one-parameter distribution. This property is used to explore the behavior of the area-time-integral (ATI) methods where fractional area above a threshold is related to the mean rain rate. In the usual application of the ATI method a correlation is established between these quantities. However, if a particular form of the rain rate distribution is assumed and if the ratio of the mean to standard deviation is known, then not only the mean but the full distribution can be extracted from a measurement of fractional area above a threshold. The second method is an extension of this idea where the distribution is estimated from data over a range of rain rates chosen in an intermediate range where the effects of attenuation and poor sensitivity can be neglected. The advantage of estimating the distribution itself rather than the mean value is that it yields the fraction of rain contributed by the light and heavy rain rates. This is useful in estimating the fraction of rainfall contributed by the rain rates that go undetected by the radar. The results at high rain rates provide a cross-check on the usual attenuation correction methods that are applied at the highest resolution of the instrument.

Meneghini, R.↗

NASA NTRS2002

Application of Reconfigurable Computing Technology to Multi-KiloHertz Micro-Laser Altimeter (MMLA) Data Processing

The Multi-KiloHertz Micro-Laser Altimeter (MMLA) is an aircraft based instrument developed by NASA Goddard Space Flight Center with several potential spaceflight applications. This presentation describes how reconfigurable computing technology was employed to perform MMLA signal extraction in real-time under realistic operating constraints. The MMLA is a "single-photon-counting" airborne laser altimeter that is used to measure land surface features such as topography and vegetation canopy height. This instrument has to date flown a number of times aboard the NASA P3 aircraft acquiring data at a number of sites in the Mid-Atlantic region. This instrument pulses a relatively low-powered laser at a very high rate (10 kHz) and then measures the time-of-flight of discrete returns from the target surface. The instrument then bins these measurements into a two-dimensional array (vertical height vs. horizontal ground track) and selects the most likely signal path through the array. Return data that does not correspond to the selected signal path are classified as noise returns and are then discarded. The MMLA signal extraction algorithm is very compute intensive in that a score must be computed for every possible path through the two dimensional array in order to select the most likely signal path. Given a typical array size with 50 x 6, up to 33 arrays must be processed per second. And for each of these arrays, roughly 12,000 individual paths must be scored. Furthermore, the number of paths increases exponentially with the horizontal size of the array, and linearly with the vertical size. Yet, increasing the horizontal and vertical sizes of the array offer science advantages such as improved range, resolution, and noise rejection. Due to the volume of return data and the compute intensive signal extraction algorithm, the existing PC-based MMLA data system has been unable to perform signal extraction in real-time unless the array is limited in size to one column, This limits the ability of the MMLA to operate in environments with sparse signal returns and a high number of noise return. However, under an IR&D project, an FPGA-based, reconfigurable computing data system has been developed that has been demonstrated to perform real-time signal extraction under realistic operating constraints. This reconfigurable data system is based on the commercially available Firebird Board from Annapolis Microsystems. This PCI board consists of a Xilinx Virtex 2000E FPGA along with 36 MB of SRAM arranged in five separately addressable banks. This board is housed in a rackmount PC with dual 850MHz Pentium processors running the Windows 2000 operating system. This data system performs all signal extraction in hardware on the Firebird, but also runs the existing "software based" signal extraction in tandem for comparison purposes. Using a relatively small amount of the Virtex XCV2000E resources, the reconfigurable data system has demonstrated to improve performance improvement over the existing software based data system by an order of magnitude. Performance could be further improved by employing parallelism. Ground testing and a preliminary engineering test flight aboard the NASA P3 has been performed, during which the reconfigurable data system has been demonstrated to match the results of the existing data system.

Powell, Wesley↗

NASA NTRS2007

Using High Frequency Passive Microwave, A-train, and TRMM Data to Evaluate Hydrometer Structure in the NASA GEOS-5 Data Assimilation System

Validating water vapor and prognostic condensate in global models remains a challenging research task. Model parameterizations are still subject to a large number of tunable parameters; furthermore, accurate and representative in situ observations are very sparse, and satellite observations historically have significant quantitative uncertainties. Progress on improving cloud / hydrometeor fields in models stands to benefit greatly from the growing inventory ofA-Train data sets. ill the present study we are using a variety of complementary satellite retrievals of hydrometeors to examine condensate produced by the emerging NASA Modem Era Retrospective Analysis for Research and Applications, MERRA, and its associated atmospheric general circulation model GEOS5. Cloud and precipitation are generated by both grid-scale prognostic equations and by the Relaxed Arakawa-Schubert (RAS) diagnostic convective parameterization. The high frequency channels (89 to 183.3 GHz) from AMSU-B and MRS on NOAA polar orbiting satellites are being used to evaluate the climatology and variability of precipitating ice from tropical convective anvils. Vertical hydrometeor structure from the Tropical Rainfall Measuring Mission (TRMM) and CloudSat radars are used to develop statistics on vertical hydrometeor structure in order to better interpret the extensive high frequency passive microwave climatology. Cloud liquid and ice water path data retrieved from the Moderate Resolution Imaging Spectroradiometer, MODIS, are used to investigate relationships between upper level cloudiness and tropical deep convective anvils. Together these data are used to evaluate cloud / ice water path, gross aspects of vertical hydrometeor structure, and the relationship between cloud extent and surface precipitation that the MERRA reanalysis must capture.

Robertson, Franklin↗

NASA NTRS2016

Achieving Consistent Doppler Measurements from SDO/HMI Vector Field Inversions

NASA's Solar Dynamics Observatory is delivering vector magnetic field observations of the full solar disk with unprecedented temporal and spatial resolution; however, the satellite is in a highly inclined geosynchronous orbit. The relative spacecraft-Sun velocity varies by +/-3 kms−1 over a day, which introduces major orbital artifacts in the Helioseismic Magnetic Imager (HMI) data. We demonstrate that the orbital artifacts contaminate all spatial and temporal scales in the data. We describe a newly developed three-stage procedure for mitigating these artifacts in the Doppler data obtained from the Milne-Eddington inversions in the HMI pipeline. The procedure ultimately uses 32 velocity-dependent coefficients to adjust 10 million pixels-a remarkably sparse correction model given the complexity of the orbital artifacts. This procedure was applied to full-disk images of AR 11084 to produce consistent Dopplergrams. The data adjustments reduce the power in the orbital artifacts by 31 dB. Furthermore, we analyze in detail the corrected images and show that our procedure greatly improves the temporal and spectral properties of the data without adding any new artifacts. We conclude that this new procedure makes a dramatic improvement in the consistency of the HMI data and in its usefulness for precision scientific studies.

Schuck, Peter W.↗

NASA NTRS1991

Generation Of Surface Grids From Data Points

Computational procedure generates grids on complicated three-dimensional surfaces from sets of data points that lie on and specify those surfaces. Starting with grouping of possibly sparse surface points into lines and/or patches, procedure involves interpolation within and blending of lines and/or patches and possibly redistribution and reassembly of patches to obtain finished system of zonal patch grids that match at boundaries between them. Procedure semiautomated via computer program that performs all steps except selection of patches and interpolation points, left to discretion of user.

Luh, Raymond Ching-Chung↗