Search NASA⌕ Search

SEARCH · Search NASA

Results for “data volume”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

An optimal GPS data processing technique

A formula is derived to optimally combine dual-frequency GPS (Global Positioning System) pseudorange and carrier phase data streams into a single equivalent data stream, reducing the data volume and computing time in the filtering process for parameter estimation by a factor of four. The resulting single data stream is that of carrier phase measurements with both data noise and bias uncertainty strictly defined. With this analytical formula the single stream of equivalent GPS measurements can be efficiently formed by simple numerical calculations without any degradation in data strength. The formulation for the optimally combined GPS data and their covariances are given in closed form. Carrier phase ambiguity resolution, when feasible, is improved due to the preservation of the full data strength with the optimal data combining process.

Wu, S. C.↗
NASA NTRSDate not supplied

Updates of MERRA-2 Data and Services at NASA GES DISC

Over40 years of NASA climate reanalysis datasets from the Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) are available at the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC). In addition to being used in traditional weather and climate research, MERRA-2 is also widely used in application studies of, e.g., wind and solar energy, air quality and health, food and drought, and heat waves. Two new MERRA-2 datasets were recently added at the GES DISC: (1) climate statistics derived fromMERRA-2 daily data to assist in the analysis of extreme temperature and precipitation events and of large-scale meteorological patterns from 1980 to the present and (2) gridded satellite and conventional observations processed in the MERRA-2 system, along with key statistics derived from the data assimilation, to help better understand how the quality of observations directly affect there analysis data. The GES DISC focuses its efforts on continually improving existing data services and to develop new data tools to satisfy various user communities. The newly added features include the following: Time series service: This is a new service for MERRA-2 data, which enables the easy and fast access of long-term hourly or daily time series at a location for popular parameters. The data is saved in a single le in Ascii format with a user- friendly structure. New analytic functions in the subsetter interface: Options for downloading daily minimum and maximum values have been added into the subsetter interface, in addition to the existing daily mean option, for all MERRA-2 and MERRA sub-daily products. Data format conversion to GeoTIFF has been implemented. New variables in Giovanni: Most monthly variables have been integrated into Giovanni, GESDISC’s online visualization and analysis tool. Due to the large data volume, hourly variables were selected based on user requests. More online information: New MERRA-2 documentation has been added: Data How-To, Data in Action, and FAQ. This presentation overviews two new MERRA-2 datasets and illustrates the new features of data services through a number of case studies. MERRA-2 data and services can be found at: https://disc.gsfc.gov/datasets?

Data management↗
NASA NTRS2022

Remote Sensing of Aerosol Water Fraction, Dry Size Distribution and Soluble Fraction Using Multi-Angle, Multi-Spectral Polarimetry

A framework to infer volume water fraction, soluble fraction and dry size distributions of fine mode aerosol from multi-angle, multi-spectral polarimetry retrievals of column-averaged ambient aerosol properties is presented. The method is applied to observations of the Research Scanning Polarimeter (RSP) obtained during two NASA aircraft campaigns, namely the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) and the Cloud, Aerosol, and Monsoon Processes-Philippines Experiment (CAMP2Ex). All aerosol retrievals are statistically evaluated using in situ data. Volume water fraction is inferred from the retrieved ambient real part of the refractive index, assuming a dry refractive index of 1.54 and by applying a volume mixing rule to obtain the effective ambient refractive index. The uncertainties in inferred volume water fraction resulting from this simplified model are discussed and estimated to be lower than 0.2 and decreasing with increasing volume water fraction. The daily mean retrieved volume water fractions correlate well with the in situ values with a mean absolute difference of 0.09. Polarimeter-retrieved ambient effective radius for daily data is shown to increase as a function of volume water fraction as expected. Furthermore, the effective variance of the size distributions also increases with increasing effective radius, which we show is consistent with an external mixture of soluble and insoluble aerosol. The relative variations of effective radius and variance over an observation period are then used to estimate the soluble fraction of the aerosol. Daily results of soluble fraction correlate well with in situ observed sulfate mass fraction with a correlation coefficient of 0.79. Subsequently, inferred water and soluble fractions are used to derive dry fine-mode size distributions from their ambient counterparts. While dry effective radii obtained in situ and from RSP show similar ranges, in situ values are generally substantially smaller during the ACTIVATE deployments, which may be due to biases in RSP retrievals or in the in situ observations, or both. Both RSP and in situ observations indicate the dominance of aerosol with low hygroscopicity during the ACTIVATE and CAMP2Ex campaigns. Furthermore, RSP indicates a high degree of external mixing of particles with low and high hygroscopicity. These retrievals of fine mode water volume fraction and soluble fraction may be used for the evaluation of water uptake in atmospheric models. Furthermore, the framework allows to estimate the variation in the concentration of fine-mode aerosol larger than a specific dry radius limit, which can be used as a proxy for the variation in cloud condensation nucleus concentrations. This framework may be applied to multi-angle, multi-spectral satellite data expected to be available in the near future.

Bastiaan Van Diedenhoven↗
NASA NTRS1975

Development and investigation of single-scan TV radiography for the acquisition of dynamic physiologic data

A light amplifier for large flat screen fluoroscopy was investigated which will decrease both its size and weight. The work on organ contouring was extended to yield volumes. This is a simple extension since the fluoroscopic image contains density (gray scale) information which can be translated as tissue thickness, integrated, yielding accurate volume data in an on-line situation. A number of devices were developed for analog image processing of video signals, operating on-line in real time, and with simple selection mechanisms. The results show that this approach is feasible and produces are improvement in image quality which should make diagnostic error significantly lower. These are all low cost devices, small and light in weight, thereby making them usable in a space environment, on the Ames centrifuge, and in a typical clinical situation.

Baily, N. A.↗
NASA NTRS1995

SeaWiFS technical report series. Volume 31: Stray light in the SeaWiFS radiometer

Some of the measurements from the Sea-viewing Wide Field-of-view Sensor (SeaWiFS) will not be useful as ocean measurements. For the ocean data set, there are procedures in place to mask the SeaWiFS measurements of clouds and ice. Land measurements will also be masked using a geographic technique based on each measurment's latitude and longitude. Each of these masks involves a source of light much brighter than the ocean. Because of stray light in the SeaWiFS radiometer, light from these bright sources can contaminate ocean measurements located a variable number of pixels away from a bright source. In this document, the sources of stray light in the sensor are examined, and a method is developed for masking measurements near bright targets for stray light effects. In addition, a procedure is proposed for reducing the effects of stray light in the flight data from SeaWiFS. This correction can also reduce the number of pixels masked for stray light. Without these corrections, local area scenes must be masked 10 pixels before and after bright targets in the along-scan direction. The addition of these corrections reduces the along-scan masks to four pixels before and after bright sources. In the along-track direction, the flight data are not corrected, and are masked two pixels before and after. Laboratory measurements have shown that stray light within the instrument changes in a direct ratio to the intensity of the bright source. The measurements have also shown that none of the bands show peculiarities in their stray light response. In other words, the instrument's response is uniform from band to band. The along-scan correction is based on each band's response to a 1 pixel wide bright sources. Since these results are based solely on preflight laboratory measurements, their successful implementation requires compliance with two additional criteria. First, since SeaWiFS has a large data volume, the correction and masking procedures must be such that they can be converted into computationally fast algorithms. Second, they must be shown to operate properly on flight data. The laboratory results, and the corrections and masking procedures that derive from them, should be considered as zeroeth order estimates of the effects that will be found on orbit.

Hooker, Stanford B.↗
NASA NTRS1978

Identification of wood energy resources in central Michigan

Existing biomass studies were compiled for determining their applicability in measuring forest biomass in an entirely new way. Over sixty tree-weight tables were prepared from existing tables or formulas. An estimate of forest biomass was made on a defined area by using Landsat Satellite data analysis, existing forest cover type maps and actual weighting of the entire biomass. Control plots were cruised for normal volume data and weight data, harvested and weighed to determine actual tonnage yields.

Hudson, W. D.↗
NASA NTRS1986

The Tasseled Cap de-mystified

The fundamental concepts on which the Tasseled Cap transformations of MSS and TM data are based - particularly the identification of inherent data structures - are explained and discussed. Emphasis on the structures present in data from any given sensor, which are themselves the expression of physical characteristics of scene classes, provides a number of advantages, including (a) reduction in data volume with minimal information loss; (b) spectral features which can be applied, without re-definition or adjustment, to any data set for a given sensor; (c) spectral features which can be directly associated with important physical parameters; and (d) easier integration of data from multiple sensors.

Crist, E. P.↗
NASA NTRS1989

Imaging spectrometry - Technology and applications

The development history and current status of NASA imaging-spectrometer (IS) technology are discussed in a review covering the period 1982-1988. Consideration is given to the Airborne IS first flown in 1982, the second-generation Airborne Visible and IR IS (AVIRIS), the High-Resolution IS being developed for the EOS polar platform, improved two-dimensional focal-plane arrays for the short-wave IR spectral region, and noncollinear acoustooptic tunable filters for use as spectral dispersing elements. Also examined are approaches to solving the data-processing problems posed by the high data volumes of state-of-the-art ISs (e.g., 160 MB per 600 x 600-pixel AVIRIS scene), including intelligent data editing, lossless and lossy data compression techniques, and direct extraction of scientifically meaningful geophysical and biophysical parameters.

Solomon, Jerry E.↗
NASA NTRS1993

An optimal GPS data processing technique for precise positioning

A mathematical formula to optimally combine dual-frequency GPS pseudorange and carrier phase (integrated Doppler) data streams into a single data stream is derived in closed form. The data combination reduces the data volume and computing time in the filtering process for parameter estimation by a factor of 4 while preserving the full data strength for precise positioning. The resulting single data stream is that of carrier phase measurements with both data noise and bias uncertainty strictly defined. With this mathematical formula the single stream of optimally combined GPS measurements can be efficiently formed by simple numerical calculations. Carrier phase ambiguity resolution, when feasible, is strengthened due to the preserved full data strength with the optimally combined data and the resulting longer wavelength for the ambiguity to be resolved.

Wu, Sien-Chong↗
NASA NTRS2005

A Testbed Demonstration of an Intelligent Archive in a Knowledge Building System

The last decade's influx of raw data and derived geophysical parameters from several Earth observing satellites to NASA data centers has created a data-rich environment for Earth science research and applications. While advances in hardware and information management have made it possible to archive petabytes of data and distribute terabytes of data daily to a broad community of users, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications in order to realize the full potential of these valuable datasets. In examining what is needed to enable this progress in the data provider environment that exists today and is expected to evolve in the next several years, we arrived at the concept of an Intelligent Archive in context of a Knowledge Building System (IA/KBS). Our prior work and associated papers investigated usage scenarios, required capabilities, system architecture, data volume issues, and supporting technologies. We identified six key capabilities of an IA/KBS: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting. Among these capabilities, large-scale data mining is perceived by many in the community to be an area of technical risk. One of the main reasons for this is that standard data mining research and algorithms operate on datasets that are several orders of magnitude smaller than the actual sizes of datasets maintained by realistic earth science data archives. Therefore, we defined a test-bed activity to implement a large-scale data mining algorithm in a pseudo-operational scale environment and to examine any issues involved. The application chosen for applying the data mining algorithm is wildfire prediction over the continental U.S. This paper reports a number of observations based on our experience with this test-bed. While proof-of-concept for data mining scalability and utility has been a major goal for the research reported here, it was not the only one. The other five capabilities of an WKBS named above have been considered as well, and an assessment of the implications of our experience for these other areas will also be presented. The lessons learned through the testbed effort and presented in this paper will benefit technologists, scientists, and system operators as they consider introducing IA/KBS capabilities into production systems.

Ramapriyan, Hampapuram↗
NASA NTRS2015

Managing Large Datasets for Atmospheric Research

Since the mid-1980s, airborne and ground measurements have been widely used to provide comprehensive characterization of atmospheric composition and processes. Field campaigns have generated a wealth of insitu data and have grown considerably over the years in terms of both the number of measured parameters and the data volume. This can largely be attributed to the rapid advances in instrument development and computing power. The users of field data may face a number of challenges spanning data access, understanding, and proper use in scientific analysis. This tutorial is designed to provide an introduction to using data sets, with a focus on airborne measurements, for atmospheric research. The first part of the tutorial provides an overview of airborne measurements and data discovery. This will be followed by a discussion on the understanding of airborne data files. An actual data file will be used to illustrate how data are reported, including the use of data flags to indicate missing data and limits of detection. Retrieving information from the file header will be discussed, which is essential to properly interpreting the data. Field measurements are typically reported as a function of sampling time, but different instruments often have different sampling intervals. To create a combined data set, the data merge process (interpolation of all data to a common time base) will be discussed in terms of the algorithm, data merge products available from airborne studies, and their application in research. Statistical treatment of missing data and data flagged for limit of detection will also be covered in this section. These basic data processing techniques are applicable to both airborne and ground-based observational data sets. Finally, the recently developed Toolsets for Airborne Data (TAD) will be introduced. TAD (tad.larc.nasa.gov) is an airborne data portal offering tools to create user defined merged data products with the capability to provide descriptive statistics and the option to treat measurement uncertainty.

Chen, Gao↗
NASA NTRS2019

Machine Learning-Based Atmospheric Phenomena Detection Platform

As the number of Earth pointing satellites has increased over the last several decades, the data volume retrieved from instruments onboard these satellites has also increased. It is expected that this trend will continue as more data intensive missions and small satellite constellations are launched. Currently, feature detection - namely atmospheric phenomena - in these datasets is performed manually and is thus not scalable with the growing data archives. Recent advancements in computational efficiency allow for the Earth science community to leverage machine learning to identify interesting atmospheric phenomena. Given the wide range of distinctive features in various atmospheric phenomena, a specialized machine learning model is required for accurate detection of these phenomena independently. The Phenomena Portal, developed at NASA IMPACT, is designed to provide visualization for the output from these machine learning models. In addition, detected events for each atmospheric phenomena are stored in a database that can be used to more easily use/subset larger spatiotemporal datasets. The user interface also incorporates additional features to enhance the user experience including spatiotemporal analysis, multiple base layer images, and a slider to filter events with lower probabilities of positive detection. Each detection supports user feedback on whether the detection is true or false that can then be stored and used to improve the machine learning model performance.

Gurung, Iksha↗
NASA NTRSDate not supplied

Quantifying Emergent Fluid Dynamics Using Reynolds-Interpolated Fluid Reduced-order Models

Fluid reduced-order models (ROMs) which capture the flow physics within the problem's physical domain are usually constrained in accuracy to only the parameter points, e.g. Reynolds and Mach numbers, at which reference data was provided. Interpolation-focused quantity-of-interest ROMs are often structured differently and fail to provide flow volume data with the same quality - if at all. In this paper, techniques which reside at the intersection of these two ROM schools - flow physics ROMs which can be interpolated within a parameter space of interest - are explored. Using a combination of existing and novel techniques, emergent physics are identified using a fluid ROM at parameter points which are not provided in the ROM's training data.

uncertainty quantification↗
NASA NTRSDate not supplied

Quantifying Emergent Fluid Dynamics Using Reynolds-Interpolated Fluid Reduced-order Models

Fluid reduced-order models (ROMs) which capture the flow physics within the problem's physical domain are usually constrained in accuracy to only the parameter points, e.g. Reynolds and Mach numbers, at which reference data was provided. Interpolation-focused quantity-of-interest ROMs are often structured differently and fail to provide flow volume data with the same quality - if at all. In this paper, techniques which reside at the intersection of these two ROM schools - flow physics ROMs which can be interpolated within a parameter space of interest - are explored. Using a combination of existing and novel techniques, emergent physics are identified using a fluid ROM at parameter points which are not provided in the ROM's training data.

uncertainty quantification↗
DOE OSTI2025

On-the-fly data set combinations with RNTuple

With the expected data volume increase for HL-LHC and the even more complex computing challenges set by future colliders, the need for efficient data storage and processing becomes more pressing. ROOT’s next-generation data format and I/O subsystem, RNTTuple, is designed to address these challenges. RNTTuple already demonstrates a clear improvement in storage and I/O efficiency, as well as overall stability and robustness with respect to its predecessor, TTTree. These improvements provide a solid baseline to introduce novel extensions to common high-energy and nuclear physics (HENP) workflows. Notably, many workflows could benefit from the ability to arbitrarily join and chain data set samples at runtime, which could reduce overall storage requirements and improve application runtime and ergonomics. In this paper, we present the RNTupleProcessor, which enables HENP data set combinations with RNTuple. We will discuss the main design considerations, present the interfaces to support data set combinations and show how they integrate in typical workflows.

de Geus, Florine Willemijn [CERN; Twente U., Ensch↗
NASA NTRS1981

NASA tracking and data acquisition in the 1990's - Support for low earth orbit missions

Requirements related to increases in data volume for missions projected for the 1990's could be met by increasing the number of satellites in the Tracking and Data Relay Satellite System (TDRSS) constellation or by providing a new tracking and data acquisition satellite system having greater capacity (gigabits), increased reliability, and more direct user-to-relay connectivity. The program to develop the heir to TDRSS for the 1990's, has been defined as Tracking and Data Acquisition System (TDAS). A description is presented of the system architectural considerations which have to be studied in order to develop a cost effective TDAS. Attention is given to basic TDAS design parameters, TDAS spacecraft architectures, system considerations, technology considerations, and TDAS constellation options involving 2, 3, and 4 satellites.

Schwartz, J. J.↗
NASA NTRS1992

Equivalent GPS measurements for efficient estimation process

Dual-frequency pseudorange and carrier phase data streams can be analytically combined into a single equivalent data stream, reducing the data volume and computing time in the filtering process for parameter estimation by a factor of 2 to 4. The resulting single data stream is that of carrier phase measurements with both data noise and bias uncertainty strictly defined. Based on these analytical formulas the equivalent GPS measurements can be formed by simple and efficient numerical calculations without any degradation in data strength. Formulation for the equivalent GPS measurements and their covariances are given in closed form; and a numerical simulation is performed to demonstrate the validity and effectiveness of the equivalent measurements.

Wu, S. C.↗
NASA NTRS2019

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗