Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

A SURVEY OF METAL LINES AT HIGH REDSHIFT. II. SDSS ABSORPTION LINE STUDIES-O VI LINE DENSITY, SPACE DENSITY, AND GAS METALLICITY AT z{sub abs} {approx} 3.0

We have analyzed a large data set of O VI absorber candidates found in the spectra of 3702 Sloan Digital Sky Survey (SDSS) quasars, focusing on a subsample of 387 active galactic nuclei sight lines with an average S/N {>=}5.0, allowing for the detection of absorbers above a rest-frame equivalent width limit of W{sub r} {>=} 0.19 A for the O VI 1032 A component. Accounting for random interlopers mimicking an O VI doublet, we derive for the first time a secure lower limit for the redshift number density {Delta}N/{Delta}z for redshifts z{sub abs} {>=} 2.8. With extensive Monte Carlo simulations, we quantify the losses of absorbers due to blending with the ubiquitous Ly{alpha} forest lines and estimate the success rate of retrieving each individual candidate as a function of its redshift, the emission redshift of the quasar, the strength of the absorber, and the measured signal-to-noise ratio (S/N) of the spectrum by modeling typical Lyman forest spectra. These correction factors allow us to derive the 'incompleteness and S/N-corrected' redshift number densities of O VI absorbers: {Delta}N{sub O{sub VI,c}}/{Delta}z{sub c} (2.8 < z < 3.2) = 4.6 {+-} 0.3, {Delta}N{sub O{sub VI,c}}/{Delta}z{sub c} (3.2 < z < 3.6) = 6.7 {+-} 0.8, and {Delta}N{sub O{sub VI,c}}/{Delta}z{sub c} (3.6 < z < 4.0) = 8.4 {+-} 2.9. We can place a secure lower limit for the contribution of O VI to the closure mass density at the redshifts probed here: {Omega}{sub O{sub VI}}(2.8 < z < 3.2) {>=} 1.9 x 10{sup -8} h {sup -1}. We show that the strong lines we probe account for over 65% of the mass in the O VI absorbers; the weak absorbers, while dominant in line number density, do not contribute significantly to the mass density. Making a conservative assumption about the ionization fraction, O{sub VI}/O, and adopting the Anders and Grevesse solar abundance values, we derive the mean metallicity of the gas probed in our search: {zeta}(2.8 < z < 3.2) {>=} 3.6 x 10{sup -4} h, in good agreement with other studies. These results demonstrate that large spectroscopic data sets such as SDSS can play an important role in QSO absorption line studies, in spite of the relatively low resolution.

79 ASTRONOMY AND ASTROPHYSICS↗

Set processing in a network environment

A combination of a local network, a mass storage system, and an autonomous set processor serving as a data/storage management machine is described. Its characteristics include: content-accessible data bases usable from all connected devices; efficient storage/access of large data bases; simple and direct programming with data manipulation and storage management handled by the set processor; simple data base design and entry from source representation to set processor representation with no predefinition necessary; capability available for user sort/order specification; significant reduction in tape/disk pack storage and mounts; flexible environment that allows upgrading hardware/software configuration without causing major interruptions in service; minimal traffic on data communications network; and improved central memory usage on large processors.

Hardgrave, W. T.↗

Ripening of Rh Nanoparticle Catalysts in Reverse Water–Gas Shift via a Data-Driven Model Combining Physics, Theory, and Experiment

Degradation via sintering is an ongoing challenge that impedes the broad commercial success of supported metallic nanoparticle catalysts. To mitigate degradation via informed catalyst design and process operations, here we aim to disambiguate the underlying mechanisms of sintering by combining theory and experiment in a quantitative framework. While mechanistic sintering models exist, they only model a single sintering pathway, even though multiple sintering mechanisms can occur simultaneously or dominate at different stages of the process. Data-driven machine learning models have emerged as a means to represent complex processes through data regression. However, machine learning models have very large data needs and lack mechanistic insights due to their black-box encoding. To develop an interpretive model of catalyst degradation via sintering, we constructed a hybrid model combining mechanistic “physics-based” models and data-driven methods to obtain both reliable predictions and mechanistic insights regarding experimentally observed sintering phenomena. Focusing on nanoparticle sintering in the Rh–TiO 2 catalyst for the reverse water–gas shift (RWGS) reaction, the hybrid model couples a mechanistic term for Ostwald ripening with energy values calculated via density functional theory (DFT) with a parametric, data-driven discrepancy function term for unmodeled mechanisms. The hybrid model is trained using Bayesian inference with data collected from small-angle X-ray scattering (SAXS) in situ experiments wherein average nanoparticle diameter versus time was measured at three relevant operating temperatures. The calibrated hybrid model results show that an Ostwald ripening-only model parameterized with fixed DFT energies does not fully capture the time and temperature dependence of the SAXS-observed sintering kinetics, and that an additional functional contribution, or DFT energy calibration, is required to reconcile simulation and experiment. Analysis of the hybrid-model error confirms that the hybrid model outperforms both the purely mechanistic and purely data-driven alternatives in terms of expected predictive accuracy for time-evolving average particle sizes. Furthermore, the results support the hypothesis that the Ostwald ripening mechanism is less important for explaining the sintering phenomena as operating temperature increases under an assumed fixed DFT parameterization. This could be explained in one of two ways: either latent, unmodeled sintering mechanisms dominate at higher temperatures, or the DFT uncertainty increases with temperature. The proposed modeling approach directly links theory to experiments and simulations via a statistical hybrid modeling framework and can be extended to other catalytic systems to improve predictive models and mechanistic understanding.

Bayesian hybrid modeling↗

Principal component analysis of computed emission lines from protostellar jets

A very important issue concerning protostellar jets is the mechanism behind their formation. Obtaining information on the region at the base of a jet can shed light on the subject, and some years ago this was done through a search for a rotational signature in the jet line spectrum. The existence of such signatures, however, remains controversial. In order to contribute to the clarification of this issue, in this paper we show that principal component analysis (PCA) can potentially help to distinguish between rotation and precession effects in protostellar jet images. This method reduces the dimensions of the data, facilitating the efficient extraction of information from large data sets such as those arising from integral field spectroscopy. PCA transforms the system of correlated coordinates into a system of uncorrelated coordinates, the eigenvectors, ordered by principal components of decreasing variance. The projection of the data on these coordinates produces images called tomograms, while eigenvectors can be displayed as eigenspectra. The combined analysis of both can allow the identification of patterns correlated to a particular physical property that would otherwise remain hidden, and can help to separate the effects of physically uncorrelated phenomena in the data. These are, for example, rotation and precession in the kinematics of a stellar jet. In order to show the potential of PCA analysis, we apply it to synthetic spectro-imaging datacubes generated as an output of numerical simulations of protostellar jets. In this way we generate a benchmark with which a PCA diagnostics of real observations can be confronted. Using the computed emission line profiles for [O i]λ6300 and [S ii]λ6716, we recover and analyze the effects of rotation and precession in tomograms generated by PCA. We show that different combinations of the eigenvectors can be used to enhance and to identify the rotation features present in the data. Our results indicate that PCA can be useful for disentangling rotation from precession in jets with an inclination of the jet with respect to the plane of the sky as high as 45&deg;. We have been able to recover the initially imposed rotation jet profile for models at a moderate inclination angle (&straightphi;&NestedLessLess;15&deg;) and without precession.

79 ASTRONOMY AND ASTROPHYSICS↗

Nonpareil 3: Fast Estimation of Metagenomic Coverage and Sequence Diversity

Estimations of microbial community diversity based on metagenomic data sets are affected, often to an unknown degree, by biases derived from insufficient coverage and reference database-dependent estimations of diversity. For instance, the completeness of reference databases cannot be generally estimated since it depends on the extant diversity sampled to date, which, with the exception of a few habitats such as the human gut, remains severely undersampled. Further, estimation of the degree of coverage of a microbial community by a metagenomic data set is prohibitively time-consuming for large data sets, and coverage values may not be directly comparable between data sets obtained with different sequencing technologies. Here, we extend Nonpareil, a database-independent tool for the estimation of coverage in metagenomic data sets, to a high-performance computing implementation that scales up to hundreds of cores and includes, in addition, a k -mer-based estimation as sensitive as the original alignment-based version but about three hundred times as fast. Further, we propose a metric of sequence diversity ( N d ) derived directly from Nonpareil curves that correlates well with alpha diversity assessed by traditional metrics. We use this metric in different experiments demonstrating the correlation with the Shannon index estimated on 16S rRNA gene profiles and show that N d additionally reveals seasonal patterns in marine samples that are not captured by the Shannon index and more precise rankings of the magnitude of diversity of microbial communities in different habitats. Therefore, the new version of Nonpareil, called Nonpareil 3, advances the toolbox for metagenomic analyses of microbiomes.

59 BASIC BIOLOGICAL SCIENCES↗

Data Systems Dynamic Simulation - A total system for data system design assessments and trade studies

Data Systems Dynamic Simulation is a simulation system designed to reduce cost and time and increase the confidence and comprehensiveness of Data Systems Simulation. It is designed to simulate large data processing and communications systems from end-to-end or by subsystem. Those features relevant to system timing, control, sizing, personnel support activities, cost and external influences are modeled. Emphasis is placed on ease of use, comprehensive system performance measures, and extensive post simulation analysis capability. The system has been used to support trade studies of the NASA data system needs in the 1985 to 1990 time frame.

Hooper, J. W.↗

Computing the Lyapunov spectrum of a dynamical system from an observed time series

The paper examines the problem of accurately determining, from an observed time series, the Liapunov exponents for the dynamical system generating the data. It is shown that, even with very large data sets, it is clearly advantageous to utilize local neighborhood-to-neighborhood mappings with higher-order Taylor series rather than just local linear maps. This procedure is demonstrated on the Henon and Ikeda maps of the plane itself, the Lorenz system of three ordinary differential equations, and the Mackey-Glass delay differential equation.

Brown, Reggie↗

Statistical properties of the solar wind

We now have spacecraft observations spanning more than a complete 22-year solar cycle. PVO at Venus, IMP 8 at Earth, and Voyager 2 and Pioneers 10 and 11 in the outer heliosphere provide many years of data from locations scattered through the heliosphere. These large data sets enable us to look at the solar wind velocity, density, temperature and flow angles on a statistical basis. Comparison of results at the different locations will provide information on the evolution of the solar wind with radial distance and variation with latitude.

Richardson, J. D.↗

Automated Classification of Thermal Infrared Spectra Using Self-organizing Maps

Existing and planned space missions to a variety of planetary and satellite surfaces produce an ever increasing volume of spectral data. Understanding the scientific informational content in this large data volume is a daunting task. Fortunately various statistical approaches are available to assess such data sets. Here we discuss an automated classification scheme based on Kohonen Self-organizing maps (SOM) we have developed. The SUM process produces an output layer were spectra having similar properties lie in close proximity to each other. One major effort is partitioning this output layer into appropriate regions. This is prefonned by defining dosed regions based upon the strength of the boundaries between adjacent cells in the SOM output layer. We use the Davies-Bouldin index as a measure of the inter-class similarities and intra-class dissimilarities that determines the optimum partition of the output layer, and hence number of SOM clusters. This allows us to identify the natural number of clusters formed from the spectral data. Mineral spectral libraries prepared at Arizona State University (ASU) and John Hopkins University (JHU) are used to test and evaluate the classification scheme. We label the library sample spectra in a hierarchical scheme with class, subclass, and mineral group names. We use a portion of the spectra to train the SOM, i.e. produce the output layer, while the remaining spectra are used to test the SOM. The test spectra are presented to the SOM output layer and assigned membership to the appropriate cluster. We then evaluate these assignments to assess the scientific meaning and accuracy of the derived SOM classes as they relate to the labels. We demonstrate that unsupervised classification by SOMs can be a useful component in autonomous systems designed to identify mineral species from reflectance and emissivity spectra in the therrnal IR.

Roush, Ted L.↗

Efficient Kriging via Fast Matrix-Vector Products

Interpolating scattered data points is a problem of wide ranging interest. Ordinary kriging is an optimal scattered data estimator, widely used in geosciences and remote sensing. A generalized version of this technique, called cokriging, can be used for image fusion of remotely sensed data. However, it is computationally very expensive for large data sets. We demonstrate the time efficiency and accuracy of approximating ordinary kriging through the use of fast matrixvector products combined with iterative methods. We used methods based on the fast Multipole methods and nearest neighbor searching techniques for implementations of the fast matrix-vector products.

Memarsadeghi, Nargess↗

JPL Genesis and Rapid Intensification Processes (GRIP) Portal

Satellite observations can play a very important role in airborne field campaigns, since they provide a comprehensive description of the environment that is essential for the experiment design, flight planning, and post-experiment scientific data analysis. In the past, it has been difficult to fully utilize data from multiple NASA satellites due to the large data volume, the complexity of accessing NASA s data in near-real-time (NRT), as well as the lack of software tools to interact with multi-sensor information. The JPL GRIP Portal is a Web portal that serves a comprehensive set of NRT observation data sets from NASA and NOAA satellites describing the atmospheric and oceanic environments related to the genesis and intensification of the tropical storms in the North Atlantic Ocean. Together with the model forecast data from four major global atmospheric models, this portal provides a useful tool for the scientists and forecasters in planning and monitoring the NASA GRIP field campaign during the 2010 Atlantic Ocean hurricane season. This portal uses the Google Earth plug-in to visualize various types of data sets, such as 2D maps, wind vectors, streamlines, 3D data sets presented at series of vertical cross-sections or pointwise vertical profiles, and hurricane best tracks and forecast tracks. Additionally, it allows users to overlap multiple data sets, change the opacity of each image layer, generate animations on the fly with selected data sets, and compare the observation data with the model forecast using two independent calendars. The portal also provides the capability to identify the geographic location of any point of interest. In addition to supporting the airborne mission planning, the NRT data and portal will serve as a very rich source of information during the post-field campaign analysis stage of the airborne experiment. By including a diverse set of satellite observations and model forecasts, it provides a good spatial and temporal context for the high-resolution, but limited in space and time, airborne observations.

Knosp, Brian W.↗

Modeling and Simulation Tool to Improve Urban Search Planning (NA22 Final Report)

The venture project Tool to Improve Urban Search Planning (LL14-V-Urban Search Planning ToolPD3RS) supported two major types of effort: 1) development of a software prototype search planning tool, with the goal of transitioning the tool to operational users for evaluation and possible adoption; 2) enabling research to better understand the radiation characteristics of an urban environment, with implications for deployment of radiation detection systems in that environment. The first effort has resulted in prototype software, Optimization Planning Tool for Urban Search (OPTUS), that has been distributed to a limited set of potential users, including users within the NNSA emergency response community. The second effort supported various research topics including high fidelity radiation transport modeling in urban environments, measurement campaigns for model validation, development of vehicle-based high purity germanium (HPGe) detector systems and supporting data analysis, and computational capabilities for archiving and managing large data sets. While some of these related efforts have informed the models and software design of the OPTUS tool, in large part they pursued more basic research, some of which has evolved into separate DNN R&D projects. Project performers included Oak Ridge National Laboratory (ORNL), Lawrence Berkeley National Laboratory (LBNL) and the Remote Sensing Laboratory (RSL), in addition to Lawrence Livermore Laboratory (LLNL), the project lead. The project also supported university research, including several internships at the laboratories.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Parallel Graph Environment for Real-World Data Analytics Workflows

Economic competitiveness and national security depend increasingly on the insightful analysis of large data sets. The diversity of real-world data sources and analytic workflows impose challenging hardware and software requirements for parallel graph platforms. The irregular nature of graph methods is not supported well by the deep memory hierarchies of conventional distributed systems, requiring new processor and runtime system designs to tolerate memory and synchronization latencies. Moreover, the efficiency of relational table operations and matrix computations are not attainable when data is stored in common graph data structures. In this paper, we present HAGGLE, a high-performance, scalable data analytics platform. The platform’s hybrid data model supports a variety of distributed, thread-safe data structures, parallel programming constructs, and persistent and streaming data. An abstract runtime layer enables us to map the stack to conventional, distributed computer systems with accelerators. The runtime uses multithreading, active messages, and data aggregation to hide memory and synchronization latencies on large-scale systems.

Graph Analytics↗

Predicting Ecologically Important Vegetation Variables from Remotely Sensed Optical/Radar Data Using Neural Networks

A number of satellite sensor systems will collect large data sets of the Earth's surface during NASA's Earth Observing System (EOS) era. Efforts are being made to develop efficient algorithms that can incorporate a wide variety of spectral data and ancillary data in order to extract vegetation variables required for global and regional studies of ecosystem processes, biosphere-atmosphere interactions, and carbon dynamics. These variables are, for the most part, continuous (e.g. biomass, leaf area index, fraction of vegetation cover, vegetation height, vegetation age, spectral albedo, absorbed photosynthetic active radiation, photosynthetic efficiency, etc.) and estimates may be made using remotely sensed data (e.g. nadir and directional optical wavelengths, multifrequency radar backscatter) and any other readily available ancillary data (e.g., topography, sun angle, ground data, etc.). Using these types of data, neural networks can: 1) provide accurate initial models for extracting vegetation variables when an adequate amount of data is available; 2) provide a performance standard for evaluating existing physically-based models; 3) invert multivariate, physically based models; 4) in a variable selection process, identify those independent variables which best infer the vegetation variable(s) of interest; and 5) incorporate new data sources that would be difficult or impossible to use with conventional techniques. In addition, neural networks employ a more powerful and adaptive nonlinear equation form as compared to traditional linear, index transformations, and simple nonlinear analyses. These neural networks attributes are discussed in the context of the authors' investigations of extracting vegetation variables of ecological interest.

Kimes, Daniel S.↗

Methods and apparatus for extraction and tracking of objects from multi-dimensional sequence data

An object tracking technique is provided which, given: (i) a potentially large data set; (ii) a set of dimensions along which the data has been ordered; and (iii) a set of functions for measuring the similarity between data elements, a set of objects are produced. Each of these objects is defined by a list of data elements. Each of the data elements on this list contains the probability that the data element is part of the object. The method produces these lists via an adaptive, knowledge-based search function which directs the search for high-probability data elements. This serves to reduce the number of data element combinations evaluated while preserving the most flexibility in defining the associations of data elements which comprise an object.

Hill, Matthew L.↗

Methods and apparatus for extraction and tracking of objects from multi-dimensional sequence data

An object tracking technique is provided which, given: (i) a potentially large data set; (ii) a set of dimensions along which the data has been ordered; and (iii) a set of functions for measuring the similarity between data elements, a set of objects are produced. Each of these objects is defined by a list of data elements. Each of the data elements on this list contains the probability that the data element is part of the object. The method produces these lists via an adaptive, knowledge-based search function which directs the search for high-probability data elements. This serves to reduce the number of data element combinations evaluated while preserving the most flexibility in defining the associations of data elements which comprise an object.

Hill, Matthew L.↗

LANL High-Performance Data System (HPDS)

The Los Alamos High-Performance Data System (HPDS) is being developed to meet the very large data storage and data handling requirements of a high-performance computing environment. The HPDS will consist of fast, large-capacity storage devices that are directly connected to a high-speed network and managed by software distributed in workstations. The HPDS model, the HPDS implementation approach, and experiences with a prototype disk array storage system are presented.

Collins, M. William↗

The use of large-area spectral data in wheat yield estimation

Large-area relations between satellite spectral data and end-of-season crop yield were investigated. Green Index Number (GIN) values from Landsat MSS data of sample segments throughout the U.S. Great Plains winter wheat belt in 1978 were correlated to county USDA-SRS reported yields. A linear relation between GIN and yield appeared to exist up to GIN values of 40 or 50, covering cases of severe to moderate stress. In a test on 1978 Texas winter wheat at the county level, GIN values for sample segments in the counties were used in conjunction with an agronomic-meteorological yield model. The combined fit explained significantly more of the observed yield variation at the county level than the agromet model alone.

Barnett, T. L.↗