Search NASA⌕ Search

SEARCH · Search NASA

Results for “sparse data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Advances to a Global Agroclimatology Solar Insolation and Meteorological Parameter Data Base: Improved Solar Irradiance up to Hourly Temporal Resolution

A primary objective of NASA’s Prediction of Worldwide Energy Resource (POWER) project is to facilitate the use of NASA Earth Science data holdings within the energy, agricultural, and architectural industries. To this end daily averaged solar data from several NASA projects and metrological data from a NASA assimilation model have long been reformatted and via a user friendly web based data portal (https://power.larc.nasa.gov) at the native resolution of each data products. Potential users can access solar and metrological data in a column formatted DSSAT ASCII format by entering single site specific coordinates or from an area by entering the appropriate area coordinates. Upgrades to the POWER data portal have been implemented that result in a complete upgrade of the base solar insolation data products. From the years 1984 through 2000, a new version of the NASA/GEWEX Surface Radiation Budget (SRB) and Clouds is utilized that reduces the RMS relative to surface measurements. Additionally, Clouds and the Earth’s Radiant Energy System (CERES) SYNoptic 1x1 Degree (SYN1Deg) data products are utilized starting from January 1, 2001 through 3 months of real-time. The CERES Fast Longwave and SHortwave radiative Fluxes (FLASHFlux) is still used to provided daily data spanning from the end of SYN1Deg to within 7 days of real-time. Meteorological parameters now are taken from the NASA Modern Era Retrospective-analysis for Research and Applications (MERRA-2) data set which provides higher resolution data products (hourly and 0.5 x 625 degree) covering the entire globe. Besides updating the solar and meteorological data products, this new version features new data products such as photosynthetically active radiation (PAR), more cloud information, estimates of soil temperatures and improved options for long-term climatological data requests. More than 36+ years daily time slices are included in the combined solar and meteorological data sets. However, perhaps the most important innovation of the POWER GIS-enabled Web Services is the provision of hourly solar and meteorological data products beginning in Jan 1, 2001. The hourly values will enable more detailed modeling and crop analysis that incorporates the diurnal variability of these parameters. The new capability is made possible by utilizing both the CERES SYN1Deg and MERRA-2 data products that include parameters at these resolutions. An assessment of both the meteorological and the solar irradiance data are based upon comparisons with globally distributed surface observations. The meteorological observations from the National Center for Environmental Information’s “Integrated Surface Database” as well as the “Global Summary of the Day” (GSOD) data files. The estimates of solar insolation are compared to the Baseline Solar Radiation Network (BSRN) and other high quality surface measurement networks. Results from the uncertainty assessments demonstrates that the NASA’s meteorological and solar irradiance data can represent a viable alternative to surface observations, particularly in data sparse regions of the world.

solar irradiance↗
NASA NTRS2022

Trends in Europe Storm Surge Extremes Match the Rate of Sea-level Rise

Coastal communities across the world are already feeling the disastrous impacts of climate change through variations in extreme sea levels1. These variations reflect the combined effect of sea-level rise and changes in storm surge activity. Understanding the relative importance of these two factors in altering the likelihood of extreme events is crucial to the success of coastal adaptation measures. Existing analyses of tide gauge records agree that sea-level rise has been a considerable driver of trends in sea-level extremes since at least 1960. However, the contribution from changes in storminess remains unclear, owing to the difficulty of inferring this contribution from sparse data and the consequent inconclusive results that have accumulated in the literature. Here we analyse tide gauge observations using spatial Bayesian methods to show that, contrary to current thought, trends in surge extremes and sea-level rise both made comparable contributions to the overall change in extreme sea levels in Europe since 1960 . We determine that the trend pattern of surge extremes reflects the contributions from a dominant north–south dipole associated with internal climate variability and a single-sign positive pattern related to anthropogenic forcing. Our results demonstrate that both external and internal influences can considerably affect the likelihood of surge extremes over periods as long as 60 years, suggesting that the current coastal planning practice of assuming stationary surge extremes might be inadequate.

Francisco M Calafat↗
NASA NTRS2022

Remote sensing-based vegetation and soil moisture constraints reduce irrigation estimation uncertainty

Understanding the human water footprint and its impact on the hydrological cycle is essential to inform water management under climate change. Despite efforts in estimating irrigation water withdrawals in earth system models, uncertainties and discrepancies exist within and across modeling systems conditioned by model structure, irrigation parameterization, and the choice of input datasets. Achieving model reliability could be much more challenging for data-sparse regions, given limited access to ground truth for parameterization and validation. Here, we demonstrate the potential of utilizing remotely sensed vegetation and soil moisture observations in constraining irrigation estimation in the Noah-MP land surface model. Results indicate that the two constraints together can effectively reduce model sensitivity to the choice of irrigation parameterization by 7%–43%. It also improves the characterization of the spatial patterns of irrigation and its impact on evapotranspiration and surface soil moisture by correcting for vegetation conditions and irrigation timing. This study highlights the importance of utilizing remotely sensed soil moisture and vegetation measurements in detecting irrigation signals and correcting for vegetation growth. Integrating the two remote sensing datasets into the model provides an effective and less feature engineered approach to constraining the uncertainty of irrigation modeling. Such strategies can be potentially transferred to other modeling systems and applied to regions across the globe.

Wanshu Nie↗
NASA NTRSDate not supplied

Spaceborne Passive-Microwave Hail Detection: Global Climatologies, Validation, and Challenges Going Forward

In addition to the myriad threats that severe hailstorms pose to society, infrastructure and agriculture, severe hail is difficult to measure in situ, and surface-based hail reporting and detection methods are inconsistent and vulnerable to nonmeteorological biases. This motivates the use of spaceborne remote-sensing platforms to retrieve hail and construct climatologies in a globally uniform way. We have developed a hail detection algorithm that leverages the sensitivity of spaceborne passive-microwave radiometers to scattering by hail, particularly in the channels from 10 to 89 GHz. We use this retrieval to construct global passive-microwave climatologies of severe hail using the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Global Precipitation Measurement Mission (GPM) Microwave Imager, Advanced Microwave Scanning Radiometer for EOS (AMSR-E), and Advanced Microwave Scanning Radiometer 2 (AMSR2) sensors and are working to extend into the pre-TRMM era to the Special Sensor Microwave Imager/Sounder (SSMI/(S)) data. Using coincident Global Precipitation Measurement (GPM) Ku-band precipitation radar, we assessed this retrieval and several others in the literature for their effectiveness and regional variability. We find that this retrieval, which leverages a signature in the Minimum 19-GHz polarization corrected temperature (PCT) combined with the 37-GHz PCT depression normalized by tropopause height constrains the radar reflectivity most tightly, and gives the least appearance of regional biases compared to other passive-microwave approaches in the literature. Satellite platforms offer consistent observations, even in remote, data-sparse, and oceanic regions that ground-based networks exclude. There are, however, potential disconnects between the processes identified aloft by the satellite and the resultant weather at the ground, leading to uncertainties in the retrievals that may propagate into satellite-based climatologies, particularly in the Tropics, where there are abundant strong - but not necessarily hailing - storms that are strongly represented in the current satellite climatologies. We will discuss ongoing efforts to assess and mitigate the contributing factors to these uncertainties, chiefly among them the effects of non-uniform beam filling in the passive-microwave footprint, and the relationships between the size distributions of hailstones aloft and the dynamic processes and environments with which they interact throughout their trajectories.

Sarah D. Bang↗
NASA NTRSDate not supplied

Detecting Hail from Space: Algorithms, Climatologies, and Challenges Going Forward

In addition to the myriad threats that severe hailstorms pose to society, infrastructure and agriculture, severe hail is difficult to measure in situ, and surface-based hail reporting and detection methods are inconsistent and subject to geographical or societal biases. This motivates the use of spaceborne remote-sensing platforms to retrieve hail and construct climatologies in a globally uniform way. We have developed a hail detection algorithm that leverages the sensitivity of spaceborne passive-microwave radiometers to scattering by hail, particularly in the channels from 10 to 89 GHz. We use this retrieval to construct global climatologies of severe hail using several different spaceborne sensors: the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Global Precipitation Measurement Mission (GPM) Microwave Imager, Advanced Microwave Scanning Radiometer for EOS (AMSR-E), and Advanced Microwave Scanning Radiometer 2 (AMSR2) sensors and are working to extend into the late 1980’s using the Special Sensor Microwave Imager/Sounder (SSMI/(S)) data. Using coincident Global Precipitation Measurement (GPM) Ku-band precipitation radar, we assessed this retrieval and several others in the literature for their effectiveness and regional variability. We developed a passive-microwave algorithm that corresponds tightly to radar reflectivity and gives the least appearance of regional biases compared to other passive-microwave approaches in the literature. Satellite platforms offer consistent observations, even in remote, data-sparse, and oceanic regions that ground-based networks exclude. There are, however, potential disconnects between the processes identified aloft by the satellite and the resultant weather at the ground, leading to uncertainties in the retrievals that may propagate into satellite-based climatologies, particularly in the Tropics, where there are abundant strong - but not necessarily hailing - storms that are strongly represented in the current satellite climatologies. We will discuss ongoing efforts to assess and mitigate the contributing factors to these uncertainties, chiefly among them the effects of non-uniform beam filling in the passive-microwave footprint, and the relationships between the size distributions of hailstones aloft and the dynamic processes and environments with which they interact throughout their trajectories.

Sarah D Bang↗
NASA NTRSDate not supplied

Remote Sensing of Hail from Space: Retrievals, Climatologies, and Challenges Going Forward

In addition to the myriad threats that severe hailstorms pose to society, infrastructure and agriculture, severe hail is difficult to measure in situ, and surface-based hail reporting and detection methods are inconsistent and subject to geographical or societal biases. This motivates the use of spaceborne remote-sensing platforms to retrieve hail and construct climatologies in the most globally consistent way. Passive-microwave algorithms leverage the sensitivity of spaceborne passive-microwave radiometers to scattering by hail, particularly in the channels from 10 to 89 GHz. These retrievals are used to construct global climatologies of severe hail. The Bang and Cecil (2019) retrievals has been applied to several different spaceborne sensors: the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Global Precipitation Measurement Mission (GPM) Microwave Imager, Advanced Microwave Scanning Radiometer for EOS (AMSR-E), and Advanced Microwave Scanning Radiometer 2 (AMSR2) sensors and used to construct [near] global climatologies of severe hail. This retrieval, and others, have been tested using Global Precipitation Measurement (GPM) Ku-band precipitation radar, to assess their effectiveness and regional variability. A successful retrieval and climatology are those that correspond tightly to radar reflectivity and give minimal appearance of regional biases, especially with latitude. Satellite platforms offer consistent observations, even in remote, data-sparse, and oceanic regions that ground-based networks exclude. There are, however, potential disconnects between the processes identified aloft by the satellite and the resultant weather at the ground, leading to uncertainties in the retrievals that may propagate into satellite-based climatologies, particularly in the Tropics, where there are abundant strong - but not necessarily hailing - storms that are strongly represented in the current satellite climatologies. There are ongoing efforts to assess and mitigate the contributing factors to these uncertainties, chiefly among them the effects of non-uniform beam filling in the passive-microwave footprint, and the relationships between the size distributions of hailstones aloft and the dynamic processes and environments with which they interact throughout their trajectories.

Sarah D Bang↗
NASA NTRSDate not supplied

Spaceborne Remote Sensing of Hail: Retrievals, Climatologies, and Challenges Going Forward

In addition to the myriad threats that severe hailstorms pose to society, infrastructure and agriculture, severe hail is difficult to measure in situ, and surface-based hail reporting and detection methods are inconsistent and subject to geographical or societal biases. This motivates the use of spaceborne remote-sensing platforms to retrieve hail and construct climatologies in the most globally consistent way. Passive-microwave algorithms leverage the sensitivity of spaceborne passive-microwave radiometers to scattering by hail, particularly in the channels from 10 to 89 GHz. These retrievals are used to construct global climatologies of severe hail. The Bang and Cecil (2019) retrievals has been applied to several different spaceborne sensors: the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI), Global Precipitation Measurement Mission (GPM) Microwave Imager, Advanced Microwave Scanning Radiometer for EOS (AMSR-E), and Advanced Microwave Scanning Radiometer 2 (AMSR2) sensors and used to construct [near] global climatologies of severe hail. This retrieval, and others, have been tested using Global Precipitation Measurement (GPM) Ku-band precipitation radar, to assess their effectiveness and regional variability. A successful retrieval and climatology are those that correspond tightly to radar reflectivity and give minimal appearance of regional biases, especially with latitude. Satellite platforms offer consistent observations, even in remote, data-sparse, and oceanic regions that ground-based networks exclude. There are, however, potential disconnects between the processes identified aloft by the satellite and the resultant weather at the ground, leading to uncertainties in the retrievals that may propagate into satellite-based climatologies, particularly in the Tropics, where there are abundant strong - but not necessarily hailing - storms that are strongly represented in the current satellite climatologies. There are ongoing efforts to assess and mitigate the contributing factors to these uncertainties, chiefly among them the effects of non-uniform beam filling in the passive-microwave footprint, and the relationships between the size distributions of hailstones aloft and the dynamic processes and environments with which they interact throughout their trajectories.

Sarah D Bang↗
NASA NTRSDate not supplied

Applicability of Loads Estimation Techniques Using Sparse Acceleration Sensor Data to Spacecraft Structural Health Monitoring

The use of structural health monitoring systems on spacecraft structures can play a crucial role in ensuring the safety, reliability, and longevity of the structure by gathering and analyzing onboard sensor data. Of specific importance is monitoring for excessive loading at critical interfaces as any off-nominal structural excitations experienced by spacecraft structures can cause early unpredicted high structural life consumption or damage. The availability and cost of flight-certified sensors along with the size of spacecraft structures and allowable payload mass drives the need for a method to estimate loads using sparsely-located sensors. Numerous approaches such as physics-based, statistical learning, and physics-enhanced statistical learning algorithms have gained popularity among structural prognostics applications. However, developing noise-robust prediction models to assess loads and structural life predictions from a sparse multi-sensor data acquisition system can be a challenging task. This paper discusses the evaluation of physics-based versus machine-learning algorithms for predicting loads and structural life at mission critical locations on the spacecraft structure using a finite element loads analysis with the application of simulated noise and noise reduction techniques. To estimate the loads from accelerations, the physics-based algorithm leverages a loads transformation matrix from a Craig-Bampton reduced finite element model. A System Equivalent Reduction Expansion Process (SEREP) and a pseudo-inverse approach are considered to expand from the onboard sensor degrees of freedom to the Craig-Bampton model degrees of freedom. The machine learning algorithm provides a data driven solution/mapping of the sensor accelerations to the loads at the mission critical locations using a high dimensionality analysis. Although these strategies produce comparable loads prediction without noise, the limitations of these strategies with incorporating simulated noise and noise reduction techniques with low signal to noise ratio signals are evaluated. The study demonstrates the immense potential of statistical learning algorithms for sparse structural prognostic models and enhancing signal denoising techniques. These findings also highlight the need for noise-resilient prognostic models and low-noise data acquisition systems onboard spacecraft structures.

Spacecraft Structural Health Monitoring↗
NASA NTRS1984

An inerference assessment approach for a three-dimensional slotted tunnel with sparse wall pressure data

The various procedures referred to as wall interference assessment and correction procedures presume the existence of a surface distribution of data (usually static pressure) measured over a surface on or near the tunnel walls for each test point to be assessed. An alternative approach in which a reasonably sophisticated computer model of the test section flow would be fitted parametrically to a sparse set of measured data is presented. The measurements provides line distributions of static pressure near the center lines of the top, side and bottom walls. The development of a test section model incorporating explicit recognition of discrete slots of finite length with controlled flow reentry into the solid wall downstream portion of the tunnel is shown.

Kemp, W. B., Jr.↗
NASA NTRS2002

Southern Ocean Seasonal Net Production from Satellite, Atmosphere, and Ocean Data Sets

A new climatology of monthly air-sea O2 flux was developed using the net air-sea heat flux as a template for spatial and temporal interpolation of sparse hydrographic data. The climatology improves upon the previous climatology of Najjar and Keeling in the Southern Hemisphere, where the heat-based approach helps to overcome limitations due to sparse data coverage. The climatology is used to make comparisons with productivity derived from CZCS images. The climatology is also used in support of an investigation of the plausible impact of recent global warming an oceanic O2 inventories.

Keeling, Ralph F.↗
NASA NTRS2010

The Kepler DB, a Database Management System for Arrays, Sparse Arrays and Binary Data

The Kepler Science Operations Center stores pixel values on approximately six million pixels collected every 30-minutes, as well as data products that are generated as a result of running the Kepler science processing pipeline. The Kepler Database (Kepler DB) management system was created to act as the repository of this information. After one year of ight usage, Kepler DB is managing 3 TiB of data and is expected to grow to over 10 TiB over the course of the mission. Kepler DB is a non-relational, transactional database where data are represented as one dimensional arrays, sparse arrays or binary large objects. We will discuss Kepler DB's APIs, implementation, usage and deployment at the Kepler Science Operations Center.

McCauliff, Sean↗
NASA NTRS1993

Investigation of serendipitious WFC sources

The serendipitious WFC sources under investigation, i.e., those which just happened to lie in the field of view while another object was being studied, were disappointing. The integration times were chosen to suit the primary target not the serendipitous targets. UX UMa, CZ Ori, BI Ori, WX Cet and AR And were not detected. A long (approximately 17 ksec) pointed observation of UX Uma (PI Wood) has since been carried out (February 1993) and the data is expected shortly. The other observations were much more successful. V471 Tau was observed with the WFC for 6.5 hrs with the S1 filter and 1.1 hrs with the S2b filter. It was easily detected with a count rate of 0.03 cps in the S1 filter and 0.15 cps in the S2b filter. The oscillations were seen, even before the data was folded, as was expected from preliminary results from the survey (Barstow et al. 1992). The pointed observations provide a much better phase coverage of the oscillations than did the survey data where the coverage was sparse. These data will be presented in a paper with the PSPC data (PI's Robinson and Shipman) and the pulse profiles in the different wavelengths will be compared.

Robinson, Edward L.↗
NASA NTRS2015

A Formal Messaging Notation for Alaskan Aviation Data

Data exchange is an increasingly important aspect of the National Airspace System. While many data communication channels have become more capable of sending and receiving data at higher throughput rates, there is still a need to use communication channels efficiently with limited throughput. The limitation can be based on technological issues, financial considerations, or both. This paper provides a complete description of several important aviation weather data in Abstract Syntax Notation format. By doing so, data providers can take advantage of Abstract Syntax Notation's ability to encode data in a highly compressed format. When data such as pilot weather reports, surface weather observations, and various weather predictions are compressed in such a manner, it allows for the efficient use of throughput-limited communication channels. This paper provides details on the Abstract Syntax Notation One (ASN.1) implementation for Alaskan aviation data, and demonstrates its use on real-world aviation weather data samples as Alaska has sparse terrestrial data infrastructure and data are often sent via relatively costly satellite channels.

communications↗
NASA NTRS1984

Application of VISSR Atmospheric Sounder (VAS) data in weather analysis

A technique which analyzes irregularly spaced satellite data is described. An experiment with rawinsonde and VISSR Atmospheric Sounder (VAS) radiance measurements collected on March 6-7, 1982 is conducted to reveal the applicability of the technique. The rawinsonde data are analyzed on a 16 x 12 grid using the two pass analysis scheme of Barnes (1973). A scheme similar to the Barnes (1973) procedure is employed to produce gridded analysis of VAS data over a 200 x 15000 km region in central part of the U.S. The use of a correction pass on the initial gridded field is described; the technique is extremely effective on uniformly spaced observations. The incorporation of the limited fine mesh model to the scheme to analyze data in sparse and cloudy regions is examined. A comparison of rawinsonde data with VAS data is provided. The technique proves effective for studying cloudy and sparse areas with VAS data and produces a four-dimensional data set with significant mesoscale structure.

Jedlovec, G. J.↗
NASA NTRS1988

Data traffic reduction schemes for sparse Cholesky factorizations

Load distribution schemes are presented which minimize the total data traffic in the Cholesky factorization of dense and sparse, symmetric, positive definite matrices on multiprocessor systems with local and shared memory. The total data traffic in factoring an n x n sparse, symmetric, positive definite matrix representing an n-vertex regular 2-D grid graph using n (sup alpha), alpha is equal to or less than 1, processors are shown to be O(n(sup 1 + alpha/2)). It is O(n(sup 3/2)), when n (sup alpha), alpha is equal to or greater than 1, processors are used. Under the conditions of uniform load distribution, these results are shown to be asymptotically optimal. The schemes allow efficient use of up to O(n) processors before the total data traffic reaches the maximum value of O(n(sup 3/2)). The partitioning employed within the scheme, allows a better utilization of the data accessed from shared memory than those of previously published methods.

Naik, Vijay K.↗
DOE OSTI2025

Pulse: An Outlier Sensitive Downsampling Algorithm For Timeseries Data

Pulse is a downsampling algorithm for timeseries data. Frequently datasets become so large that visualization tools and web browsers cannot effectively render graphics due to memory constraints. Downsampling algorithms are commonly applied to minimize the quantity of data required to visualize important features or trends in the data, but some datasets are composed by distinct enough features and trends that most existing downsampling algorithms fail to preserve them. Pule was developed to downsample timeseries data for galvanostatic stack test data at the Idaho National Laboratory. These datasets were composed by approximately 4 million records, most of them being extremely uniform. However, during relatively brief time periods when the stack test changes state, for example when the test article is powered on, or a load is added, the data produce sparse asymptotes. No existing downsampling algorithm was capable of preserving the sparse asymptotes in electrolysis stack test data. Instead, we develop a downsampling algorithm that preserves important outliers in data, and otherwise aggressively downsamples uniform data. The algorithm has applications in other domains like seismology, in the measurement of earthquakes, or astronomy, in the measurement of quasars or transit photometry.

Woodruff, Nathan [Idaho National Laboratory (INL),↗