Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS1993

A practical large scale/high speed data distribution system using 8 mm libraries

Eight mm tape libraries are known primarily for their small size, large storage capacity, and low cost. However, many applications require an additional attribute which, heretofore, has been lacking -- high transfer rate. Transfer rate is particularly important in a large scale data distribution environment -- an environment in which 8 mm tape should play a very important role. Data distribution is a natural application for 8 mm for several reasons: most large laboratories have access to 8 mm tape drives, 8 mm tapes are upwardly compatible, 8 mm media are very inexpensive, 8 mm media are light weight (important for shipping purposes), and 8 mm media densely pack data (5 gigabytes now and 15 gigabytes on the horizon). If the transfer rate issue were resolved, 8 mm could offer a good solution to the data distribution problem. To that end Exabyte has analyzed four ways to increase its transfer rate: native drive transfer rate increases, data compression at the drive level, tape striping, and homogeneous drive utilization. Exabyte is actively pursuing native drive transfer rate increases and drive level data compression. However, for non-transmitted bulk data applications (which include data distribution) the other two methods (tape striping and homogeneous drive utilization) hold promise.

Howard, Kevin↗
NASA NTRS1977

Satellite measurements of ionic concentrations applied to low altitude incoherent scatter interpretations

The determination of ion and electron temperatures and electron density from incoherent scatter radar data has in the past involved assumptions concerning the ionospheric composition below 250 km. Using a very large data base of measurements of O(+), O2(+), NO(+), and N2(+) made by the Atmosphere Explorer C satellite, a model has been developed of the ionic concentration between 130 and 300 km as a function of solar zenith angle. As the effect of the new model is to increase the radar determinations of the temperature, this may explain previous discrepancies between Langmuir probe and incoherent scatter temperatures, where the comparisons have been made at these low altitudes

Torr, M. R.↗
NASA NTRS1993

Investigation of spectral analysis techniques for randomly sampled velocimetry data

It is well known that velocimetry (LV) generates individual realization velocity data that are randomly or unevenly sampled in time. Spectral analysis of such data to obtain the turbulence spectra, and hence turbulence scales information, requires special techniques. The 'slotting' technique of Mayo et al, also described by Roberts and Ajmani, and the 'Direct Transform' method of Gaster and Roberts are well known in the LV community. The slotting technique is faster than the direct transform method in computation. There are practical limitations, however, as to how a high frequency and accurate estimate can be made for a given mean sampling rate. These high frequency estimates are important in obtaining the microscale information of turbulence structure. It was found from previous studies that reliable spectral estimates can be made up to about the mean sampling frequency (mean data rate) or less. If the data were evenly samples, the frequency range would be half the sampling frequency (i.e. up to Nyquist frequency); otherwise, aliasing problem would occur. The mean data rate and the sample size (total number of points) basically limit the frequency range. Also, there are large variabilities or errors associated with the high frequency estimates from randomly sampled signals. Roberts and Ajmani proposed certain pre-filtering techniques to reduce these variabilities, but at the cost of low frequency estimates. The prefiltering acts as a high-pass filter. Further, Shapiro and Silverman showed theoretically that, for Poisson sampled signals, it is possible to obtain alias-free spectral estimates far beyond the mean sampling frequency. But the question is, how far? During his tenure under 1993 NASA-ASEE Summer Faculty Fellowship Program, the author investigated from his studies on the spectral analysis techniques for randomly sampled signals that the spectral estimates can be enhanced or improved up to about 4-5 times the mean sampling frequency by using a suitable prefiltering technique. But, this increased bandwidth comes at the cost of the lower frequency estimates. The studies further showed that large data sets of the order of 100,000 points, or more, high data rates, and Poisson sampling are very crucial for obtaining reliable spectral estimates from randomly sampled data, such as LV data. Some of the results of the current study are presented.

Sree, Dave↗
NASA NTRS1993

Visualization techniques to aid in the analysis of multi-spectral astrophysical data sets

The goal of this project was to support the scientific analysis of multi-spectral astrophysical data by means of scientific visualization. Scientific visualization offers its greatest value if it is not used as a method separate or alternative to other data analysis methods but rather in addition to these methods. Together with quantitative analysis of data, such as offered by statistical analysis, image or signal processing, visualization attempts to explore all information inherent in astrophysical data in the most effective way. Data visualization is one aspect of data analysis. Our taxonomy as developed in Section 2 includes identification and access to existing information, preprocessing and quantitative analysis of data, visual representation and the user interface as major components to the software environment of astrophysical data analysis. In pursuing our goal to provide methods and tools for scientific visualization of multi-spectral astrophysical data, we therefore looked at scientific data analysis as one whole process, adding visualization tools to an already existing environment and integrating the various components that define a scientific data analysis environment. As long as the software development process of each component is separate from all other components, users of data analysis software are constantly interrupted in their scientific work in order to convert from one data format to another, or to move from one storage medium to another, or to switch from one user interface to another. We also took an in-depth look at scientific visualization and its underlying concepts, current visualization systems, their contributions, and their shortcomings. The role of data visualization is to stimulate mental processes different from quantitative data analysis, such as the perception of spatial relationships or the discovery of patterns or anomalies while browsing through large data sets. Visualization often leads to an intuitive understanding of the meaning of data values and their relationships by sacrificing accuracy in interpreting the data values. In order to be accurate in the interpretation, data values need to be measured, computed on, and compared to theoretical or empirical models (quantitative analysis). If visualization software hampers quantitative analysis (which happens with some commercial visualization products), its use is greatly diminished for astrophysical data analysis. The software system STAR (Scientific Toolkit for Astrophysical Research) was developed as a prototype during the course of the project to better understand the pragmatic concerns raised in the project. STAR led to a better understanding on the importance of collaboration between astrophysicists and computer scientists.

Brugel, Edward W.↗
NASA NTRS1993

Visualization techniques to aid in the analysis of multispectral astrophysical data sets

The goal of this project was to support the scientific analysis of multi-spectral astrophysical data by means of scientific visualization. Scientific visualization offers its greatest value if it is not used as a method separate or alternative to other data analysis methods but rather in addition to these methods. Together with quantitative analysis of data, such as offered by statistical analysis, image or signal processing, visualization attempts to explore all information inherent in astrophysical data in the most effective way. Data visualization is one aspect of data analysis. Our taxonomy as developed in Section 2 includes identification and access to existing information, preprocessing and quantitative analysis of data, visual representation and the user interface as major components to the software environment of astrophysical data analysis. In pursuing our goal to provide methods and tools for scientific visualization of multi-spectral astrophysical data, we therefore looked at scientific data analysis as one whole process, adding visualization tools to an already existing environment and integrating the various components that define a scientific data analysis environment. As long as the software development process of each component is separate from all other components, users of data analysis software are constantly interrupted in their scientific work in order to convert from one data format to another, or to move from one storage medium to another, or to switch from one user interface to another. We also took an in-depth look at scientific visualization and its underlying concepts, current visualization systems, their contributions and their shortcomings. The role of data visualization is to stimulate mental processes different from quantitative data analysis, such as the perception of spatial relationships or the discovery of patterns or anomalies while browsing through large data sets. Visualization often leads to an intuitive understanding of the meaning of data values and their relationships by sacrificing accuracy in interpreting the data values. In order to be accurate in the interpretation, data values need to be measured, computed on, and compared to theoretical or empirical models (quantitative analysis). If visualization software hampers quantitative analysis (which happens with some commercial visualization products), its use is greatly diminished for astrophysical data analysis. The software system STAR (Scientific Toolkit for Astrophysical Research) was developed as a prototype during the course of the project to better understand the pragmatic concerns raised in the project. STAR led to a better understanding on the importance of collaboration between astrophysicists and computer scientists. Twenty-one examples of the use of visualization for astrophysical data are included with this report. Sixteen publications related to efforts performed during or initiated through work on this project are listed at the end of this report.

Brugel, E. W.↗
NASA NTRSDate not supplied

A Robust Schema for Storing and Managing Machine Learning Data and Models

- Machine Learning (ML) has enabled models that can improve efficiency and decrease computational cost - ML models are crucial in enabling Integrated Computational Materials Engineering (ICME) - Large data sets require robust means of storing ML data and models

Brandon L. Hearley↗
NASA NTRS1977

A laser technique for characterizing the geometry of plant canopies

The interception of solar power by the canopy is investigated as a function of solar zenith angle (time), component of the canopy, and depth into the canopy. The projected foliage area, cumulative leaf area, and view factors within the canopy are examined as a function of the same parameters. Two systems are proposed that are capable of describing the geometrical aspects of a vegetative canopy and of operation in an automatic mode. Either system would provide sufficient data to yield a numerical map of the foliage area in the canopy. Both systems would involve the collection of large data sets in a short time period using minimal manpower.

Vanderbilt, V. C.↗
NASA NTRS1993

The analytical representation of viscoelastic material properties using optimization techniques

This report presents a technique to model viscoelastic material properties with a function of the form of the Prony series. Generally, the method employed to determine the function constants requires assuming values for the exponential constants of the function and then resolving the remaining constants through linear least-squares techniques. The technique presented here allows all the constants to be analytically determined through optimization techniques. This technique is employed in a computer program named PRONY and makes use of commercially available optimization tool developed by VMA Engineering, Inc. The PRONY program was utilized to compare the technique against previously determined models for solid rocket motor TP-H1148 propellant and V747-75 Viton fluoroelastomer. In both cases, the optimization technique generated functions that modeled the test data with at least an order of magnitude better correlation. This technique has demonstrated the capability to use small or large data sets and to use data sets that have uniformly or nonuniformly spaced data pairs. The reduction of experimental data to accurate mathematical models is a vital part of most scientific and engineering research. This technique of regression through optimization can be applied to other mathematical models that are difficult to fit to experimental data through traditional regression techniques.

Hill, S. A.↗
NASA NTRS2004

Automatic Generation of Algorithms for the Statistical Analysis of Planetary Nebulae Images

Analyzing data sets collected in experiments or by observations is a Core scientific activity. Typically, experimentd and observational data are &aught with uncertainty, and the analysis is based on a statistical model of the conjectured underlying processes, The large data volumes collected by modern instruments make computer support indispensible for this. Consequently, scientists spend significant amounts of their time with the development and refinement of the data analysis programs. AutoBayes [GF+02, FS03] is a fully automatic synthesis system for generating statistical data analysis programs. Externally, it looks like a compiler: it takes an abstract problem specification and translates it into executable code. Its input is a concise description of a data analysis problem in the form of a statistical model as shown in Figure 1; its output is optimized and fully documented C/C++ code which can be linked dynamically into the Matlab and Octave environments. Internally, however, it is quite different: AutoBayes derives a customized algorithm implementing the given model using a schema-based process, and then further refines and optimizes the algorithm into code. A schema is a parameterized code template with associated semantic constraints which define and restrict the template s applicability. The schema parameters are instantiated in a problem-specific way during synthesis as AutoBayes checks the constraints against the original model or, recursively, against emerging sub-problems. AutoBayes schema library contains problem decomposition operators (which are justified by theorems in a formal logic in the domain of Bayesian networks) as well as machine learning algorithms (e.g., EM, k-Means) and nu- meric optimization methods (e.g., Nelder-Mead simplex, conjugate gradient). AutoBayes augments this schema-based approach by symbolic computation to derive closed-form solutions whenever possible. This is a major advantage over other statistical data analysis systems which use numerical approximations even in cases where closed-form solutions exist. AutoBayes is implemented in Prolog and comprises approximately 75.000 lines of code. In this paper, we take one typical scientific data analysis problem-analyzing planetary nebulae images taken by the Hubble Space Telescope-and show how AutoBayes can be used to automate the implementation of the necessary anal- ysis programs. We initially follow the analysis described by Knuth and Hajian [KHO2] and use AutoBayes to derive code for the published models. We show the details of the code derivation process, including the symbolic computations and automatic integration of library procedures, and compare the results of the automatically generated and manually implemented code. We then go beyond the original analysis and use AutoBayes to derive code for a simple image segmentation procedure based on a mixture model which can be used to automate a manual preproceesing step. Finally, we combine the original approach with the simple segmentation which yields a more detailed analysis. This also demonstrates that AutoBayes makes it easy to combine different aspects of data analysis.

Fischer, Bernd↗
NASA NTRS2014

A Web Architecture to Geographically Interrogate CHIRPS Rainfall and eMODIS NDVI for Land Use Change

Monitoring of rainfall and vegetation over the continent of Africa is important for assessing the status of crop health and agriculture, along with long‐term changes in land use change. These issues can be addressed through examination of long‐term precipitation (rainfall) data sets and remote sensing of land surface vegetation and land use types. Two products have been used previously to address these goals: the Climate Hazard Group Infrared Precipitation with Stations (CHIRPS) rainfall data, and multi‐day composites of Normalized Difference Vegetation Index (NDVI) from the USGS eMODIS product. Combined, these are very large data sets that require unique tools and architecture to facilitate a variety of data analysis methods or data exploration by the end user community. To address these needs, a web‐enabled system has been developed to allow end‐users to interrogate CHIRPS rainfall and eMODIS NDVI data over the continent of Africa. The architecture allows end‐users to use custom defined geometries, or the use of predefined political boundaries in their interrogation of the data. The massive amount of data interrogated by the system allows the end‐users with only a web browser to extract vital information in order to investigate land use change and its causes. The system can be used to generate daily, monthly and yearly averages over a geographical area and range of dates of interest to the user. It also provides analysis of trends in precipitation or vegetation change for times of interest. The data provided back to the end‐user is displayed in graphical form and can be exported for use in other, external tools. The development of this tool has significantly decreased the investment and requirements for end‐users to use these two important datasets, while also allowing the flexibility to the end‐user to limit the search to the area of interest.

Burks, Jason E.↗
NASA NTRS1995

Large Enhancements in the O/N2 Ratio in the Evening Sector of the Winter Hemisphere During Geomagnetic Storms

In this paper, we have looked for enhancements of the O/N2 ratio in data measured by the Dynamics Explorer 2 (DE 2) satellite in the middle latitudes of the winter hemisphere, based on a prediction that was made by the National Center for Atmospheric Research thermosphere/tonosphere general circulation model (NCAR-TIGCM) that such increases occur. The NCAR-TIGCM predicts that these enhancements should be seen throughout the low latitude region and in many middle latitude locations, but that the enhancements in O/N2 are particularly strong in the middle-latitude, evening-to-midnight sector of the winter hemisphere. When this prediction was used to look for these effects in DE 2 NACS (neutral atmosphere composition spectrometer) data, large enhancements in the O/N2 ratio (approx. 50 to 90%) were seen. These enhancements were observed during the main phase of a storm that occurred on November 24, 1982, and were seen in the same region of the winter hemisphere predicted by the NCAR-TIGCM. They are partially the result of the depletion of N2 and, as electron loss is dependent on dissociative recombination at F(sub 2) altitudes, they have implications for electron densities in this area. Parcel trajectories, which have been followed through the NCAR-TIGCM history file for this event, show that large O/N2 enhancements occur in this limited region in the winter hemisphere for two reasons. First, these parcels of air are decelerated by the antisunward edge of the ion convection pattern; individual parcels converge and subsidence occurs. Thus molecular-nitrogen-poor air is brought from higher to lower heights. Because neutral parcels that are found a little poleward of the equatorial edge of the eveningside convection pattern are swept inward toward the center of the auroral oval, the enhancements occur only in a very limited range of latitudes. Second, nitrogen-poor air is transported from regions close to the magnetic pole in the winter hemisphere. During geomagnetic storms, enhanced meridional winds are driven by the increased pressure-gradient force that is associated with intensified Joule heating in the auroral oval. These pressure-driven winds decrease rapidly on the dayside beyond the auroral oval where the parcels originate, limiting the region into which the parcels can be transported. Thus these two processes drive values of O/N2 in a limited region of the winter hemisphere, and reinforce only in the evening sector, causing large changes in this region.

Burns, A. G.↗
NASA NTRS2004

Making NASA Earth Observing System Satellite Data Accessible to the K-12 and Citizen Scientist Communities

The Atmospheric Sciences Data Center (ASDC) at NASA s Langley Research Center houses over 700 data sets related to Earth s radiation budget, clouds, aerosols and tropospheric chemistry. These data sets are produced to increase academic understanding of the natural and anthropogenic perturbations that influence global climate change. The Mentoring and inquirY using NASA Data on Atmospheric and earth science for Teachers and Amateurs (MY NASA DATA) project has been established to systematically support educational activities at all levels of formal and informal education by reducing these large data holdings to microsets that will be easily explored and understood by the K-12 and the amateur scientist communities

Moore, Susan W.↗
NASA NTRS2012

Preliminary Evaluation of MapReduce for High-Performance Climate Data Analysis

MapReduce is an approach to high-performance analytics that may be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. We are particularly interested in the potential of MapReduce to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we are prototyping a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. Our initial focus has been on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. Preliminary results suggest this approach can improve efficiencies within data intensive analytic workflows.

Duffy, Daniel Q.↗
NASA NTRS2012

QuakeSim 2.0

QuakeSim 2.0 improves understanding of earthquake processes by providing modeling tools and integrating model applications and various heterogeneous data sources within a Web services environment. QuakeSim is a multisource, synergistic, data-intensive environment for modeling the behavior of earthquake faults individually, and as part of complex interacting systems. Remotely sensed geodetic data products may be explored, compared with faults and landscape features, mined by pattern analysis applications, and integrated with models and pattern analysis applications in a rich Web-based and visualization environment. Integration of heterogeneous data products with pattern informatics tools enables efficient development of models. Federated database components and visualization tools allow rapid exploration of large datasets, while pattern informatics enables identification of subtle, but important, features in large data sets. QuakeSim is valuable for earthquake investigations and modeling in its current state, and also serves as a prototype and nucleus for broader systems under development. The framework provides access to physics-based simulation tools that model the earthquake cycle and related crustal deformation. Spaceborne GPS and Inter ferometric Synthetic Aperture (InSAR) data provide information on near-term crustal deformation, while paleoseismic geologic data provide longerterm information on earthquake fault processes. These data sources are integrated into QuakeSim's QuakeTables database system, and are accessible by users or various model applications. UAVSAR repeat pass interferometry data products are added to the QuakeTables database, and are available through a browseable map interface or Representational State Transfer (REST) interfaces. Model applications can retrieve data from Quake Tables, or from third-party GPS velocity data services; alternatively, users can manually input parameters into the models. Pattern analysis of GPS and seismicity data has proved useful for mid-term forecasting of earthquakes, and for detecting subtle changes in crustal deformation. The GPS time series analysis has also proved useful as a data-quality tool, enabling the discovery of station anomalies and data processing and distribution errors. Improved visualization tools enable more efficient data exploration and understanding. Tools provide flexibility to science users for exploring data in new ways through download links, but also facilitate standard, intuitive, and routine uses for science users and end users such as emergency responders.

Donnellan, Andrea↗
NASA NTRS2013

Hadoop for High-Performance Climate Analytics: Use Cases and Lessons Learned

Scientific data services are a critical aspect of the NASA Center for Climate Simulations mission (NCCS). Hadoop, via MapReduce, provides an approach to high-performance analytics that is proving to be useful to data intensive problems in climate research. It offers an analysis paradigm that uses clusters of computers and combines distributed storage of large data sets with parallel computation. The NCCS is particularly interested in the potential of Hadoop to speed up basic operations common to a wide range of analyses. In order to evaluate this potential, we prototyped a series of canonical MapReduce operations over a test suite of observational and climate simulation datasets. The initial focus was on averaging operations over arbitrary spatial and temporal extents within Modern Era Retrospective- Analysis for Research and Applications (MERRA) data. After preliminary results suggested that this approach improves efficiencies within data intensive analytic workflows, we invested in building a cyber infrastructure resource for developing a new generation of climate data analysis capabilities using Hadoop. This resource is focused on reducing the time spent in the preparation of reanalysis data used in data-model inter-comparison, a long sought goal of the climate community. This paper summarizes the related use cases and lessons learned.

analytics↗
NASA NTRS1993

Clementine: Anticipated scientific datasets from the Moon and Geographos

The Clementine spacecraft mission is designed to test the performance of new lightweight and low-power detectors developed at the Lawrence Livermore National Laboratory (LLNL) for the Strategic Defense Initiative Office (SDIO). A secondary objective of the mission is to acquire useful scientific data, principally of the Moon and the near-Earth asteroid Geographos. The spacecraft will be in an elliptical polar orbit about the Moon for about 2 months beginning in February of 1994 and it will fly by Geographos on August 31. Clementine will carry seven detectors each weighing less than about 1 kg: two Star Trackers wide-angle uv/vis wide-angle Short Wavelength IR (SWIR) Long-Wavelength IR (LWIR) and LIDAR (Laser Image Detection And Ranging) narrow-angle imaging and ranging. Additional presentations about the mission detectors and related science issues are in this volume. If fully successful Clementine will return about 3 million lunar images, a dataset with nearly as many bits of data (uncompressed) as the first cycle of Magellan and more than 5000 images of Geographos. The complete and efficient analysis of such large data sets requires systematic processing efforts. Described below are concepts for two such efforts for the Clementine mission: global multispectral imaging of the Moon and videos of the Geographos flyby. Other anticipated datasets for which systematic processing might be desirable include multispectral observations of Earth; LIDAR altimetry of the Moon with high-resolution imaging along each ground track; high-resolution LIDAR color along each lunar ground track which could be used to identify potential titanium-rich deposits at scales of a few meters; and thermal IR imaging along each lunar ground track (including nighttime observations near the poles).

Mcewen, A. S.↗
NASA NTRS2008

Parametric-Studies and Data-Plotting Modules for the SOAP

"Parametric Studies" and "Data Table Plot View" are the names of software modules in the Satellite Orbit Analysis Program (SOAP). Parametric Studies enables parameterization of as many as three satellite or ground-station attributes across a range of values and computes the average, minimum, and maximum of a specified metric, the revisit time, or 21 other functions at each point in the parameter space. This computation produces a one-, two-, or three-dimensional table of data representing statistical results across the parameter space. Inasmuch as the output of a parametric study in three dimensions can be a very large data set, visualization is a paramount means of discovering trends in the data (see figure). Data Table Plot View enables visualization of the data table created by Parametric Studies or by another data source: this module quickly generates a display of the data in the form of a rotatable three-dimensional-appearing plot, making it unnecessary to load the SOAP output data into a separate plotting program. The rotatable three-dimensionalappearing plot makes it easy to determine which points in the parameter space are most desirable. Both modules provide intuitive user interfaces for ease of use.

Source record↗