Results for “sparse data set” — Page 2

NASA NTRS2012

Microwave Satellite Data for Hydrologic Modeling in Ungauged Basins

An innovative flood-prediction framework is developed using Tropical Rainfall Measuring Mission precipitation forcing and a proxy for river discharge from the Advanced Microwave Scanning Radiometer for Earth Observing System (AMSR-E) onboard the National Aeronautics and Space Administration's Aqua satellite. The AMSR-E-detected water surface signal was correlated with in situ measurements of streamflow in the Okavango Basin in Southern Africa as indicated by a Pearson correlation coefficient of 0.90. A distributed hydrologic model, with structural data sets derived from remote-sensing data, was calibrated to yield simulations matching the flood frequencies from the AMSR-E-detected water surface signal. Model performance during a validation period yielded a Nash-Sutcliffe efficiency of 0.84. We concluded that remote-sensing data from microwave sensors could be used to supplement stream gauges in large sparsely gauged or ungauged basins to calibrate hydrologic models. Given the global availability of all required data sets, this approach can be potentially expanded to improve flood monitoring and prediction in sparsely gauged basins throughout the world

Distributed hydrologic modeling↗

NASA NTRS1984

An initial approach to the utilization of VAS satellite sounding data

The primary objective is the development of methods to utilize satellite soundings. To achieve that, a more precise understanding of the characteristics of satellite derived soundings is required. The initial approach in the utilization of satellite soundings is to find a way to combine rawinsondes and satellite data into a unified data set. If this can be done in a meaningful way, satellite data could be integrated into existing data and interpretation would be enhanced. In this attempt, both rawinsonde and satellite soundings will be decomposed by the use of the Fourier cosine series. Then, each harmonic will be compared to get a better understanding of the representativeness of the satellite sounding data. Harmonics taken from satellite and rawinsonde soundings will be combined to provide a unified data set that can be used both over data sparse ocean areas as well as over land where rawinsonde data are available.

Shin, Kyung-Sup↗

NASA NTRS1991

Effects of partitioning and scheduling sparse matrix factorization on communication and load balance

A block based, automatic partitioning and scheduling methodology is presented for sparse matrix factorization on distributed memory systems. Using experimental results, this technique is analyzed for communication and load imbalance overhead. To study the performance effects, these overheads were compared with those obtained from a straightforward 'wrap mapped' column assignment scheme. All experimental results were obtained using test sparse matrices from the Harwell-Boeing data set. The results show that there is a communication and load balance tradeoff. The block based method results in lower communication cost whereas the wrap mapped scheme gives better load balance.

Venugopal, Sesh↗

NASA NTRS2016

Soil Moisture Active Passive Mission L4_SM Data Product Assessment (Version 2 Validated Release)

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public Version 2 validated release scheduled for 29 April 2016. The assessment of the Version 2 L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to up-scaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the wide geographic range of the sparse network sites, and the global assessment of the assimilation diagnostics, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 2 validation and supports the validated release of the data. An analysis of the time average surface and root zone soil moisture shows that the global pattern of arid and humid regions are captured by the L4_SM estimates. Results from the core validation site comparisons indicate that "Version 2" of the L4_SM data product meets the self-imposed L4_SM accuracy requirement, which is formulated in terms of the ubRMSE: the RMSE (Root Mean Square Error) after removal of the long-term mean difference. The overall ubRMSE of the 3-hourly L4_SM surface soil moisture at the 9 km scale is 0.035 cubic meters per cubic meter requirement. The corresponding ubRMSE for L4_SM root zone soil moisture is 0.024 cubic meters per cubic meter requirement. Both of these metrics are comfortably below the 0.04 cubic meters per cubic meter requirement. The L4_SM estimates are an improvement over estimates from a model-only SMAP Nature Run version 4 (NRv4), which demonstrates the beneficial impact of the SMAP brightness temperature data. L4_SM surface soil moisture estimates are consistently more skillful than NRv4 estimates, although not by a statistically significant margin. The lack of statistical significance is not surprising given the limited data record available to date. Root zone soil moisture estimates from L4_SM and NRv4 have similar skill. Results from comparisons of the L4_SM product to in situ measurements from nearly 400 sparse network sites corroborate the core validation site results. The instantaneous soil moisture and soil temperature analysis increments are within a reasonable range and result in spatially smooth soil moisture analyses. The O-F residuals exhibit only small biases on the order of 1-3 degrees Kelvin between the (re-scaled) SMAP brightness temperature observations and the L4_SM model forecast, which indicates that the assimilation system is largely unbiased. The spatially averaged time series standard deviation of the O-F residuals is 5.9 degrees Kelvin, which reduces to 4.0 degrees Kelvin for the observation-minus-analysis (O-A) residuals, reflecting the impact of the SMAP observations on the L4_SM system. Averaged globally, the time series standard deviation of the normalized O-F residuals is close to unity, which would suggest that the magnitude of the modeled errors approximately reflects that of the actual errors. The assessment report also notes several limitations of the "Version 2" L4_SM data product and science algorithm calibration that will be addressed in future releases. Regionally, the time series standard deviation of the normalized O-F residuals deviates considerably from unity, which indicates that the L4_SM assimilation algorithm either over- or under-estimates the actual errors that are present in the system. Planned improvements include revised land model parameters, revised error parameters for the land model and the assimilated SMAP observations, and revised surface meteorological forcing data for the operational period and underlying climatological data. Moreover, a refined analysis of the impact of SMAP observations will be facilitated by the construction of additional variants of the model-only reference data. Nevertheless, the “Version 2” validated release of the L4_SM product is sufficiently mature and of adequate quality for distribution to and use by the larger science and application communities.

SMAP L4_SM↗

NASA NTRS1997

Applications of Data Assimilation to Analysis of the Ocean on Large Scales

It is commonplace to begin talks on this topic by noting that oceanographic data are too scarce and sparse to provide complete initial and boundary conditions for large-scale ocean models. Even considering the availability of remotely-sensed data such as radar altimetry from the TOPEX and ERS-1 satellites, a glance at a map of available subsurface data should convince most observers that this is still the case. Data are still too sparse for comprehensive treatment of interannual to interdecadal climate change through the use of models, since the new data sets have not been around for very long. In view of the dearth of data, we must note that the overall picture is changing rapidly. Recently, there have been a number of large scale ocean analysis and prediction efforts, some of which now run on an operational or at least quasi-operational basis, most notably the model based analyses of the tropical oceans. These programs are modeled on numerical weather prediction. Aside from the success of the global tide models, assimilation of data in the tropics, in support of prediction and analysis of seasonal to interannual climate change, is probably the area of large scale ocean modeling and data assimilation in which the most progress has been made. Climate change is a problem which is particularly suited to advanced data assimilation methods. Linear models are useful, and the linear theory can be exploited. For the most part, the data are sufficiently sparse that implementation of advanced methods is worthwhile. As an example of a large scale data assimilation experiment with a recent extensive data set, we present results of a tropical ocean experiment in which the Kalman filter was used to assimilate three years of altimetric data from Geosat into a coarsely resolved linearized long wave shallow water model. Since nonlinear processes dominate the local dynamic signal outside the tropics, subsurface dynamical quantities cannot be reliably inferred from surface height anomalies. Because of its potential for large scale synoptic coverage of the deep ocean, acoustic travel time data should be a natural complement to satellite altimetry. Satellite data give us vertical integrals associated with thermodynamic and dynamic processes.

Miller, Robert N.↗

NASA NTRS1974

Global detailed geoid computation and model analysis

Comparisons and analyses were carried out through the use of detailed gravimetric geoids which we have computed by combining models with a set of 26,000 1 deg x 1 deg mean free air gravity anomalies. The accuracy of the detailed gravimetric geoid computed using the most recent Goddard earth model (GEM-6) in conjunction with the set of 1 deg x 1 deg mean free air gravity anomalies is assessed at + or - 2 meters on the continents of North America, Europe, and Australia, 2 to 5 meters in the Northeast Pacific and North Atlantic areas, and 5 to 10 meters in other areas where surface gravity data are sparse. The R.M.S. differences between this detailed geoid and the detailed geoids computed using the other satellite gravity fields in conjuction with same set of surface data range from 3 to 7 meters.

Marsh, J. G.↗

NASA NTRS1984

Application of VISSR Atmospheric Sounder (VAS) data in weather analysis

A technique which analyzes irregularly spaced satellite data is described. An experiment with rawinsonde and VISSR Atmospheric Sounder (VAS) radiance measurements collected on March 6-7, 1982 is conducted to reveal the applicability of the technique. The rawinsonde data are analyzed on a 16 x 12 grid using the two pass analysis scheme of Barnes (1973). A scheme similar to the Barnes (1973) procedure is employed to produce gridded analysis of VAS data over a 200 x 15000 km region in central part of the U.S. The use of a correction pass on the initial gridded field is described; the technique is extremely effective on uniformly spaced observations. The incorporation of the limited fine mesh model to the scheme to analyze data in sparse and cloudy regions is examined. A comparison of rawinsonde data with VAS data is provided. The technique proves effective for studying cloudy and sparse areas with VAS data and produces a four-dimensional data set with significant mesoscale structure.

Jedlovec, G. J.↗

NASA NTRS2002

Southern Ocean Seasonal Net Production from Satellite, Atmosphere, and Ocean Data Sets

A new climatology of monthly air-sea O2 flux was developed using the net air-sea heat flux as a template for spatial and temporal interpolation of sparse hydrographic data. The climatology improves upon the previous climatology of Najjar and Keeling in the Southern Hemisphere, where the heat-based approach helps to overcome limitations due to sparse data coverage. The climatology is used to make comparisons with productivity derived from CZCS images. The climatology is also used in support of an investigation of the plausible impact of recent global warming an oceanic O2 inventories.

Keeling, Ralph F.↗

NASA NTRS1988

Integration of space and in situ observations to study global climate change

The use of model-based global data sets of atmospheric circulation for studying fundamental dynamical and physical processes is discussed, focusing on limitations of the available model-based data sets. Data from the Global Weather Experiment in 1979 were analyzed by two authorized level IIIb data centers in 1980 and in 1981. The analyses led to difference in data-sparse regions such as the tropics. Study areas which can be addressed by an internally-consistent long-term multivariate data set for the atmospheric circulation are considered, including mean climate, forcing for the ocean models, global hydrological cycle, atmospheric energetics, intraseasonal variability, land surface processes, and structure and variability of vertical velocity, divergence, and diabatic heating. It is concluded that the most comprehensive technique for integrating space and in situ observations to produce this type of data set would be a four-dimensional data assimilation system with a realistic physical model of the type employed in operational numerical weather prediction.

Bengtsson, L.↗

NASA NTRS1997

The Influence of Correlated Crustal Signals in Modelling the Main Geomagnetic Field

Algorithms used in geomagnetic main-field modelling have for the most part treated the noise in the field measurements as if it were white. A major component of the noise consists of the field due to magnetization in the crust and it has been realized for some time that such signals are highly correlated at satellite altitude. Hence approximation by white noise, while of undoubted utility, is of unknown validity. In this paper we study two plausible statistical models for the crustal magnetization, in which the magnetization is a realization of a stationary, isotropic, random process. At a typical satellite altitude the associated fields exhibit significant correlation over ranges as great as 15 deg. or more, which introduces off-diagonal elements into the covariance matrix, elements that have usually been neglected in modelling procedures. Dealing with a full covariance matrix for a large data set would present a formidable computational challenge, but fortunately most of the entries in the covariance matrix are so small that they can be replaced by zeros. The resultant matrix comprises only about 3 per cent non-zero entries and thus we can take advantage of efficient sparse matrix techniques to solve the numerical system. We construct several main-field models based on vertical-component data from a selected 5 deg. by 5 deg. data set derived from the Magsat mission. Models with and without off-diagonal terms are compared.

Rygaard-Hjalsted, C.↗

NASA NTRS1991

Generation Of Surface Grids From Data Points

Computational procedure generates grids on complicated three-dimensional surfaces from sets of data points that lie on and specify those surfaces. Starting with grouping of possibly sparse surface points into lines and/or patches, procedure involves interpolation within and blending of lines and/or patches and possibly redistribution and reassembly of patches to obtain finished system of zonal patch grids that match at boundaries between them. Procedure semiautomated via computer program that performs all steps except selection of patches and interpolation points, left to discretion of user.

Luh, Raymond Ching-Chung↗

NASA NTRS2022

MLtool: Universal Supervised Machine Learning Tool to Model Tabulated Data

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine learning↗

NASA NTRS2022

MLtool Python Code

Machine Learning (ML) is a subfield of Artificial Intelligence that gives computers the ability to learn from past data without being explicitly programmed. The predictive capabilities of ML models have already been used to facilitate several scientific breakthroughs. However, the practical application of ML is often limited due to the gaps in technical knowledge of its users. The common issue faced by many scientific researchers is the inability to choose the appropriate ML pipelines that are needed to treat real-world data, which is often sparse and noisy. To solve this problem, we have developed an automated Machine Learning tool (MLtool) that includes a set of ML algorithms and approaches to aid scientific researchers. The current version of MLtool is implemented as an object-oriented Python code that is easily extensible. It includes 44 different regression algorithms used to model data. MLtool helps users select the best model for their data, based on the scoring metrics used. Besides regression algorithms, MLtool also includes a suite of pre- and post-processing techniques such as missing value imputation, categorical variable encoding, input feature normalization, uncertainty quantification, exploratory data analysis (EDA), etc. MLtool was tested on several publicly available multi-dimensional data sets and was found capable of making accurate predictions.

Machine Learning↗

NASA NTRS1980

A hill-sliding strategy for initialization of Gaussian clusters in the multidimensional space

A hill sliding technique was devised to extract Gaussian clusters from the multivariate probability density estimate of sample data for the first step of iterative unsupervised classification. Each cluster was assumed to posses a unimodal normal distribution. A clustering function proposed distinguished elements of a cluster under formation from the rest in the feature space. Initial clusters were extracted one by one according to the hill sliding tactics. A dimensionless cluster compactness parameter was proposed as a universal measure of cluster goodness and used satisfactorily in test runs with LANDSAT multispectral scanner data. The normalized divergence, defined by the cluster divergence divided by the entropy of the entire sample data, was utilized as a general separability measure between clusters. An overall clustering objective function was set forth in terms of cluster covariance matrices, from which the cluster compactness measure could be deduced. Minimal improvement of initial data partitioning was evaluated by this objective function in eliminating scattered sparse data points. The hill sliding clustering technique developed herein has the potential applicability to decomposition any multivariate mixture distribution into a number of unimodal distributions when an appropriate distribution function to the data set is employed.

Park, J. K.↗

NASA NTRS2017

Machine Learning Approaches to Increasing Value of Spaceflight Omics Databases

The number of spaceflight bioscience mission opportunities is too small to allow all relevant biological and environmental parameters to be experimentally identified. Simulated spaceflight experiments in ground-based facilities (GBFs), such as clinostats, are each suitable only for particular investigations -- a rotating-wall vessel may be 'simulated microgravity' for cell differentiation (hours), but not DNA repair (seconds) -- and introduce confounding stimuli, such as motor vibration and fluid shear effects. This uncertainty over which biological mechanisms respond to a given form of simulated space radiation or gravity, as well as its side effects, limits our ability to baseline spaceflight data and validate mission science. Machine learning techniques autonomously identify relevant and interdependent factors in a data set given the set of desired metrics to be evaluated: to automatically identify related studies, compare data from related studies, or determine linkages between types of data in the same study. System-of-systems (SoS) machine learning models have the ability to deal with both sparse and heterogeneous data, such as that provided by the small and diverse number of space biosciences flight missions; however, they require appropriate user-defined metrics for any given data set. Although machine learning in bioinformatics is rapidly expanding, the need to combine spaceflight/GBF mission parameters with omics data is unique. This work characterizes the basic requirements for implementing the SoS approach through the System Map (SM) technique, a composite of a dynamic Bayesian network and Gaussian mixture model, in real-world repositories such as the GeneLab Data System and Life Sciences Data Archive. The three primary steps are metadata management for experimental description using open-source ontologies, defining similarity and consistency metrics, and generating testing and validation data sets. Such approaches to spaceflight and GBF omics data may soon enable unique insight into which measured phenomena correlate to biological mechanisms that are truly affected by spaceflight conditions; which are most likely to be confounded by other variables; and which are insufficiently characterized, significantly increasing existing and future science return from ISS and spaceflight missions.

Gentry, Diana↗

NASA NTRSDate not supplied

Computational Fluid Dynamics Simulations of the Transonic Dynamics Tunnel Airstream Oscillator System

This paper presents a Computational Fluid Dynamics (CFD) model of the flow in the NASA Langley Research Center Transonic Dynamics Tunnel (TDT) with the Airstream Oscillator System (AOS) in operation. The TDT is a continuous-flow, closed circuit wind tunnel with a 16- by 16-foot slotted test section with cropped corners. The tunnel was originally built as the 19-ft Pressure Tunnel in 1938, but it was converted to the current transonic tunnel in the 1950s, with capabilities to use either air or heavy gas (R-134a) as the test medium. The TDT was also fitted with an AOS which can be used to create a gust field in the test section. To date, no computational analyses of the TDT involving the AOS have been performed. In this study, experimental data acquired of an oscillating airstream in the tunnel will be used to calibrate the computational analyses. A significant motivation of this work is to attempt to compare computational gust velocities to those recorded in the TDT literature. In addition to a validation of this model with experimental data, it may also be possible to supplement the experimental data with computational data. The experimental data is rather sparse and at only a few Mach numbers. Computational results may be able to expand the AOS data set. Another motivation for this work is that the Integrated Adaptive Wing Technology Maturation (IAWTM) semi-span model of the high-aspect-ratio CRM with 10 trailing edge control surfaces is currently being fabricated and will be delivered to the NASA Langley Transonic Dynamics Tunnel (TDT) for testing in late 2020. Among tests to be conducted, gust load alleviation (GLA) will be demonstrated at transonic conditions using the AOS.

Computational Fluid Dynamics↗

NASA NTRS2015

Technical Report Series on Global Modeling and Data Assimilation: Soil Moisture Active Passive (SMAP) Project Assessment Report for the Beta-Release L4_SM Data Product - Volume 40

During the post-launch SMAP calibration and validation (Cal/Val) phase there are two objectives for each science data product team: 1) calibrate, verify, and improve the performance of the science algorithm, and 2) validate the accuracy of the science data product as specified in the science requirements and according to the Cal/Val schedule. This report provides an assessment of the SMAP Level 4 Surface and Root Zone Soil Moisture Passive (L4_SM) product specifically for the product's public beta release scheduled for 30 October 2015. The primary objective of the beta release is to allow users to familiarize themselves with the data product before the validated product becomes available. The beta release also allows users to conduct their own assessment of the data and to provide feedback to the L4_SM science data product team. The assessment of the L4_SM data product includes comparisons of SMAP L4_SM soil moisture estimates with in situ soil moisture observations from core validation sites and sparse networks. The assessment further includes a global evaluation of the internal diagnostics from the ensemble-based data assimilation system that is used to generate the L4_SM product. This evaluation focuses on the statistics of the observation-minus-forecast (O-F) residuals and the analysis increments. Together, the core validation site comparisons and the statistics of the assimilation diagnostics are considered primary validation methodologies for the L4_SM product. Comparisons against in situ measurements from regional-scale sparse networks are considered a secondary validation methodology because such in situ measurements are subject to upscaling errors from the point-scale to the grid cell scale of the data product. Based on the limited set of core validation sites, the assessment presented here meets the criteria established by the Committee on Earth Observing Satellites for Stage 1 validation and supports the beta release of the data. The validation against sparse network measurements and the evaluation of the assimilation diagnostics address Stage 2 validation criteria by expanding the assessment to regional and global scales.

ubRMSE↗

NASA NTRS1994

Kelvin waves in total column ozone

Tropical Kelvin waves have been observed previously in ozone mixing ratio data from the SBUV (Solar Backscatter Ultraviolet) and LIMS (Limb Infrared Monitor of the Stratosphere) instruments on board the Nimbus-7 satellite. The present study investigates Kelvin wave features in total column ozone, using version 6 data from the Total Ozone Mapping Spectrometer (TOMS) instrument (also on Nimbus-7). Results show eastward-propagating zonal waves 1-2 with periods approx. 5-15 days, amplitudes approx. 3-5 Dobson Units (1-2% of the time mean), and latitudinal symmetry typical of Kelvin waves. The analyses and a linear model in this study suggest that the primary source of the perturbations is slow Kelvin waves in the lower-to-middle stratosphere. Maximum Kelvin wave signatures occur in conjunction with westward lower-to-middle stratospheric equatorial zonal winds (a quasi-biennial oscillation (QBO) wind modulation effect). The significance of these results is that the TOMS data are shown to be useful for investigations with global coverage of a major component of tropical stratospheric dynamics, Kelvin waves. The TOMS data set with its excellent coverage and high quality should be useful in validating model studies in the relatively data sparse and dynamically difficult tropical region.

Ziemke, J. R.↗