Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗
NASA NTRS1998

Neural Network for Analysis of Hyperspectral Imagery

Each material has unique spectral distribution that can be viewed as its fingerprint. Advanced hyper and ultra spectral sensors will produce large amount of data. Therefore, advanced capabilities are required that can effectively deal with large data cubes, possibly in real time.

hyperspectral imagery neural network↗
NASA NTRS2015

Email-Based Informed Consent: Innovative Method for Reaching Large Numbers of Subjects for Data Mining Research

Since the 2010 NASA authorization to make the Life Sciences Data Archive (LSDA) and Lifetime Surveillance of Astronaut Health (LSAH) data archives more accessible by the research and operational communities, demand for data has greatly increased. Correspondingly, both the number and scope of requests have increased, from 142 requests fulfilled in 2011 to 224 in 2014, and with some datasets comprising up to 1 million data points. To meet the demand, the LSAH and LSDA Repositories project was launched, which allows active and retired astronauts to authorize full, partial, or no access to their data for research without individual, study-specific informed consent. A one-on-one personal informed consent briefing is required to fully communicate the implications of the several tiers of consent. Due to the need for personal contact to conduct Repositories consent meetings, the rate of consenting has not kept up with demand for individualized, possibly attributable data. As a result, other methods had to be implemented to allow the release of large datasets, such as release of only de-identified data. However the compilation of large, de-identified data sets places a significant resource burden on LSAH and LSDA and may result in diminished scientific usefulness of the dataset. As a result, LSAH and LSDA worked with the JSC Institutional Review Board Chair, Astronaut Office physicians, and NASA Office of General Counsel personnel to develop a "Remote Consenting" process for retrospective data mining studies. This is particularly useful since the majority of the astronaut cohort is retired from the agency and living outside the Houston area. Originally planned as a method to send informed consent briefing slides and consent forms only by mail, Remote Consenting has evolved into a means to accept crewmember decisions on individual studies via their method of choice: email or paper copy by mail. To date, 100 emails have been sent to request participation in eight HRP-funded studies. The development of the Remote Consent process, the laws allowing transmission of consent via electronic means, total metrics to date, and remaining challenges (e.g., response issues, use of International Partner data, biospecimens/genetic data) for the research use of LSAH/LSDA data will be described.

Lee, Lesley R.↗
NASA NTRS2008

Previously Unrecognized Large Lunar Impact Basins Revealed by Topographic Data

The discovery of a large population of apparently buried impact craters on Mars, revealed as Quasi- Circular Depressions (QCDs) in Mars Orbiting Laser Altimeter (MOLA) data [1,2,3] and as Circular Thin Areas (CTAs) [4] in crustal thickness model data [5] leads to the obvious question: are there unrecognized impact features on the Moon and other bodies in the solar system? Early analysis of Clementine topography revealed several large impact basins not previously known [6,7], so the answer certainly is "Yes." How large a population of previously undetected impact basins, their size frequency distribution, and how much these added craters and basins will change ideas about the early cratering history and Late Heavy Bombardment on the Moon remains to be determined. Lunar Orbiter Laser Altimeter (LOLA) data [8] will be able to address these issues. As a prelude, we searched the state-of-the-art global topographic grid for the Moon, the Unified Lunar Control Net (ULCN) [9] for evidence of large impact features not previously recognized by photogeologic mapping, as summarized by Wilhelms [lo].

Frey, Herbert V.↗
NASA NTRS2013

Data Compression Algorithm Architecture for Large Depth-of-Field Particle Image Velocimeters

A large depth-of-field particle image velocimeter (PIV) is designed to characterize dynamic dust environments on planetary surfaces. This instrument detects lofted dust particles, and senses the number of particles per unit volume, measuring their sizes, velocities (both speed and direction), and shape factors when the particles are large. To measure these particle characteristics in-flight, the instrument gathers two-dimensional image data at a high frame rate, typically >4,000 Hz, generating large amounts of data for every second of operation, approximately 6 GB/s. To characterize a planetary dust environment that is dynamic, the instrument would have to operate for at least several minutes during an observation period, easily producing more than a terabyte of data per observation. Given current technology, this amount of data would be very difficult to store onboard a spacecraft, and downlink to Earth. Since 2007, innovators have been developing an autonomous image analysis algorithm architecture for the PIV instrument to greatly reduce the amount of data that it has to store and downlink. The algorithm analyzes PIV images and automatically reduces the image information down to only the particle measurement data that is of interest, reducing the amount of data that is handled by more than 10(exp 3). The state of development for this innovation is now fairly mature, with a functional algorithm architecture, along with several key pieces of algorithm logic, that has been proven through field test data acquired with a proof-of-concept PIV instrument.

Bos, Brent↗
NASA NTRS1991

Volcanism on Venus as inferred from the morphometry of large shields

Morphometric data for the Venusian shields are compared with those for large volcanos on Mars and the Big Island shield in Hawaii. Reasonable assumptions and constraints on the chemical and physical properties of venusian magma, crust, and lithosphere are used with the morphometric data to estimate probable deep magma-source depths below the shields. Such data for selected large volcanos on Venus, Mars, and earth are presented. The flattened profiles of the Venusian shields are interpreted to be predominantly the result of the density contrast between lava and rocks between the source and surface, a relatively shallow depth to a primary magma source, low magma viscosity, high magma volumes and rates, and the inferred presence of flanking vents and lava tubes that carried lavas great distances from the summit. The measured shield heights are used in conjunction with a reasonable range of magma and rock densities to estimate the depths to deep magma sources, yielding an average of 25 km beneath the Venusian plains with a range depth of 3 to 40 km.

Schaber, G. G.↗
NASA NTRS1984

LANDSAT-4 Radiometric and Geometric Correction and Image Enhancement Results

Techniques were developed or improved to calibrate, enhance, and geometrically correct LANDSAT-4 satellite data. Statistical techniques to correct data radiometry were evaluated and were found to minimize striping and banding. Conventional techniques cause striping even with perfect calibration parameters. Intensity enhancement techniques were improved to display image data with large variation in intensity or brightness. Data were geometrically corrected to conform to a 1:100,000 map reference and image products produced with the map overlay. It is shown that these products can serve as accurate map products. A personal computer was experimentally used for digital image processing.

Bernstein, R.↗
NASA NTRS1987

Middle atmosphere density and models

The 80 to 130 km altitude region is our old ignorosphere, the region of the atmosphere that no one seems to be interested in and yet the critical region for shuttle entry and atmospheric braking. Comparison between the Air Force reference atmosphere and Shuttle Inertial Measurement Units data shows large fluctuations at high latitudes. New data sources are available now, such as the Arecibo and Millstone Hill ionospheric scatter radars. In the 20 to 80 km altitude range there is a reasonable quantity of data on the mean atmosphere, however, infomation on diurnal variability is needed. In the 20 to 80 km altitude range data is needed to identify systematic variations and models for the regions are preliminary. Unpredictable variations are observed: turbulence, storm effects, gravity waves.

Champion, K.↗
NASA NTRS1988

Modeling surface deformations at complex strike-slip plate boundaries

Two new versions of the physical base-traction model of Li and Rice (1987) have been developed to study the loading processes and surface deformation near plate boundaries with geometric complexities, including a shallow creeping fault segment and two subparallel faults. Predictions of surface velocity and surface slip rate for the region between the San Andreas fault and the Pacific coast at the latitude of the Parkfield-Cholame segment using model parameters based on geologic, geodetic, and seismic considerations were found to be in good agreement with contemporary geodetic field data (not used in constrainning model parameters). Also, predicted surface velocity for the Salton Sea-Coachella Valley area were fit to recently derived contemporary geodetic data and very large baseline interferometry data, althoug here the model parameters are not well constrained.

Li, Victor C.↗
NASA NTRS1996

WATCHMAN: A Data Warehouse Intelligent Cache Manager

Data warehouses store large volumes of data which are used frequently by decision support applications. Such applications involve complex queries. Query performance in such an environment is critical because decision support applications often require interactive query response time. Because data warehouses are updated infrequently, it becomes possible to improve query performance by caching sets retrieved by queries in addition to query execution plans. In this paper we report on the design of an intelligent cache manager for sets retrieved by queries called WATCHMAN, which is particularly well suited for data warehousing environment. Our cache manager employs two novel, complementary algorithms for cache replacement and for cache admission. WATCHMAN aims at minimizing query response time and its cache replacement policy swaps out entire retrieved sets of queries instead of individual pages. The cache replacement and admission algorithms make use of a profit metric, which considers for each retrieved set its average rate of reference, its size, and execution cost of the associated query. We report on a performance evaluation based on the TPC-D and Set Query benchmarks. These experiments show that WATCHMAN achieves a substantial performance improvement in a decision support environment when compared to a traditional LRU replacement algorithm.

Scheuermann, Peter↗
NASA NTRS2001

Short-Arc Analysis of Intersatellite Tracking Data in a Gravity Mapping Mission

A technique for the analysis of low-low intersatellite range-rate data in a gravity mapping mission is explored. The technique is based on standard tracking data analysis for orbit determination but uses a spherical coordinate representation of the 12 epoch state parameters describing the baseline between the two satellites. This representation of the state parameters is exploited to allow the intersatellite range-rate analysis to benefit from information provided by other tracking data types without large simultaneous multiple data type solutions. The technique appears especially valuable for estimating gravity from short arcs (e.g., less than 15 minutes) of data. Gravity recovery simulations which use short arcs are compared with those using arcs a day in length. For a high-inclination orbit, the short-arc analysis recovers low-order gravity coefficients remarkably well, although higher order terms, especially sectorial terms, are less accurate. Simulations suggest that either long or short arcs of GRACE data are likely to improve parts of the geopotential spectrum by orders of magnitude.

Rowlands, David D.↗
NASA NTRS2012

Graphics Processing Unit Assisted Thermographic Compositing

Objective Develop a software application utilizing high performance computing techniques, including general purpose graphics processing units (GPGPUs), for the analysis and visualization of large thermographic data sets. Over the past several years, an increasing effort among scientists and engineers to utilize graphics processing units (GPUs) in a more general purpose fashion is allowing for previously unobtainable levels of computation by individual workstations. As data sets grow, the methods to work them grow at an equal, and often greater, pace. Certain common computations can take advantage of the massively parallel and optimized hardware constructs of the GPU which yield significant increases in performance. These common computations have high degrees of data parallelism, that is, they are the same computation applied to a large set of data where the result does not depend on other data elements. Image processing is one area were GPUs are being used to greatly increase the performance of certain analysis and visualization techniques.

Ragasa, Scott↗
NASA NTRSDate not supplied

Supporting Global Air Quality Management Needs With A Flexible Data Fusion Tool for Estimation and Forecasting in Google Earth Engine

High spatial and temporal resolution air quality estimation and forecasting can be enhanced by combining global data sources, like chemical transport models and satellite remote sensing, with local information from regulatory and low-cost air quality monitors. Successful integration of data from these diverse sources is complicated by many factors, however, including differences in spatial and temporal resolution, data availability and latency issues, varying data quality, and large computational and data storage requirements. This presentation will provide an overview of a NASA-funded effort to develop the foundation for future operationalization of air quality forecasting for world-wide end-users and integration into their air quality management decision processes, which will be achieved in future phases of this multi-year project. We will summarize our progress in developing a data fusion system using the Google Earth Engine platform which can integrate model, satellite, and surface-level monitoring datasets to enhance estimation and forecasting of air-quality-relevant pollutants at sub-daily and sub-city scales. The tool is being developed in close cooperation with several city- and regional-level air quality managers in the USA and around the world. Our end-goal is to provide these air quality managers with the information they need to assess and anticipate the impacts of poor air quality, track changes in air quality due to ongoing mitigation efforts and land use changes, and identify ways to improve their air quality monitoring strategies. This presentation will focus on recent advances achieved through the project, including integration of multiple air quality datasets in a prototype data fusion system in Google Earth Engine, the quantification of uncertainties associated with our data fusion approach, and the development of user interfaces and visualization tools to convey air quality information in a way which best meets end-user needs.

Carl Malings↗
NASA NTRSDate not supplied

Supporting Global Air Quality Management Needs With A Flexible Data Fusion Tool for Estimation and Forecasting in Google Earth Engine

High spatial and temporal resolution air quality estimation and forecasting can be enhanced by combining global data sources, like chemical transport models and satellite remote sensing, with local information from regulatory and low-cost air quality monitors. Successful integration of data from these diverse sources is complicated by many factors, however, including differences in spatial and temporal resolution, data availability and latency issues, varying data quality, and large computational and data storage requirements. This presentation will provide an overview of a NASA-funded effort to develop the foundation for future operationalization of air quality forecasting for world-wide end-users and integration into their air quality management decision processes, which will be achieved in future phases of this multi-year project. We will summarize our progress in developing a data fusion system using the Google Earth Engine platform which can integrate model, satellite, and surface-level monitoring datasets to enhance estimation and forecasting of air-quality-relevant pollutants at sub-daily and sub-city scales. The tool is being developed in close cooperation with several city- and regional-level air quality managers in the USA and around the world. Our end-goal is to provide these air quality managers with the information they need to assess and anticipate the impacts of poor air quality, track changes in air quality due to ongoing mitigation efforts and land use changes, and identify ways to improve their air quality monitoring strategies. This presentation will focus on recent advances achieved through the project, including integration of multiple air quality datasets in a prototype data fusion system in Google Earth Engine, the quantification of uncertainties associated with our data fusion approach, and the development of user interfaces and visualization tools to convey air quality information in a way which best meets end-user needs.

Carl Malings↗
NASA NTRS2023

Deriving New Mixing Ratios for Venus Atmospheric Gases Using Data From the Pioneer Venus Large Probe Neutral Mass Spectrometer

We present the first published method to convert data obtained by the Pioneer Venus Large Probe Neutral Mass Spectrometer (LNMS) into units of mixing ratio (ppm) and volume percent (v%) against CO 2 and N 2 , the dominant Venus atmospheric gases, including conversion to density (kg m −3 ). These unit conversions are key to unlocking the untapped potential of the data, which represents a significant challenge given the scant calibration data in the literature. Herein, we show that our data treatments and conversions yield mixing ratios and volume percent values for H 2 O, N 2 , and SO 2 that are within error to those reported for the gas chromatograph (LGC) on the Pioneer Venus Large Probe (PVLP). For the noble gases, we developed strategies to correct for instrument biases by treating the data as a relative scale and using PVLP and Venera-based measurements as calibration points. Together, these methods, conversions, calibrations, and comparisons afford novel unit conversions for the LNMS data and yield unified measures for Venus’ atmosphere from the LNMS and LGC on the PVLP. - Conversion into mixing ratio (ppm), volume percent (v%), and density (kg m −3 ). - Mixing ratios are expressed against CO 2 and N 2 . - LNMS and LGC measurements on the PVLP are consistent.

Venus↗
NASA NTRS1991

Architecture of a large object-oriented database for remotely sensed data

Attention is given to the proposed Intelligent Information Fusion System (IIFS) within the framework of the Intelligent Data Management project at NASA-Goddard. IIFS is to use connectionist architectures to extract high-level attributes from incoming sensor images, and then send those characterizations and their associated ephemeris and ancillary image data to a large object-oriented database which will serve as the master catalog of sensor data. Important issues facing this project include the choice of rapid-access data structures (RADSs) for cataloging images by their high-level characterization, the implementation of efficient spatial data structures for cataloging images by their scene location, the automated population of such a database from a continuous stream of incoming ephemeris and ancillary data, and the translation and optimization of natural-language database queries so that RADSs are employed when appropriate.

Dorfman, Erik↗
NASA NTRS1978

Engineering data management: Experience and projections

Experiences in developing a large engineering data management system are described. Problems which were encountered are presented and projected to future systems. Business applications involving similar types of data bases are described. A data base management system architecture proposed by the business community is described and its applicability to engineering data management is discussed. It is concluded that the most difficult problems faced in engineering and business data management can best be solved by cooperative efforts.

Jefferson, D. K.↗