Search NASA⌕ Search

SEARCH · Search NASA

Results for “large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

Telescope Scientist on the Advanced X-ray Astrophysics Observatory

This period included many scientific observations made with the Chandra Observatory. The results, as is well known, are spectacular. Fortunately, the High Resolution Mirror Assembly (HRMA) performance continues to be essentially identical to that predicted from ground calibration data. The Telescope Scientist Team has improved the mirror model to provide a more accurate description to the Chandra observers and enable them to reduce the systematic errors and uncertainties in their data reduction. We also have made considerable progress in improving the scattering model. There also has been progress in the scientific program. At this time 58 distant clusters of galaxies have been observed. We are performing a systematic analysis of this rather large data set for the purpose of determining absolute distances utilizing the Sunyaev Zel'dovich effect. These observations also have been used to study the evolution of the cluster baryon mass function and the cosmological constraints which result from this evolution.

VanSpeybroeck, L.↗

A model for live mission data systems using the OAIS reference model

Space sciences are confronted with overwhelming volume of data. The data rates are increasing, the granularity of registered observations is continuously refining, and computer technology allows producing terabytes of images and catalogs. The inexpensive emerging storage technologies, combined with the availability of high-speed communications will offer the infrastructure for extremely large data repositories to be accessible on-line. Mission data will be quickly accessible almost immediately after it has been collected from space observations. On-line science will demand for new tools and technologies for data access, data analysis, and data discovery. These trends will enhance the archival operational concepts mainly related to the long-term information preservation, placing an equally important emphasis on rapid data production, and dissemination to consumers.

mission data systems srchive system OAIS data mana↗

Evaluating lossy data compression on climate simulation data within a large ensemble

High-resolution Earth system model simulations generate enormous data volumes, and retaining the data from these simulations often strains institutional storage resources. Further, these exceedingly large storage requirements negatively impact science objectives, for example, by forcing reductions in data output frequency, simulation length, or ensemble size. To lessen data volumes from the Community Earth System Model (CESM), we advocate the use of lossy data compression techniques. While lossy data compression does not exactly preserve the original data (as lossless compression does), lossy techniques have an advantage in terms of smaller storage requirements. To preserve the integrity of the scientific simulation data, the effects of lossy data compression on the original data should, at a minimum, not be statistically distinguishable from the natural variability of the climate system, and previous preliminary work with data from CESM has shown this goal to be attainable. However, to ultimately convince climate scientists that it is acceptable to use lossy data compression, we provide climate scientists with access to publicly available climate data that have undergone lossy data compression. In particular, we report on the results of a lossy data compression experiment with output from the CESM Large Ensemble (CESM-LE) Community Project, in which we challenge climate scientists to examine features of the data relevant to their interests, and attempt to identify which of the ensemble members have been compressed and reconstructed. We find that while detecting distinguishing features is certainly possible, the compression effects noticeable in these features are often unimportant or disappear in post-processing analyses. In addition, we perform several analyses that directly compare the original data to the reconstructed data to investigate the preservation, or lack thereof, of specific features critical to climate science. Overall, we conclude that applying lossy data compression to climate simulation data is both advantageous in terms of data reduction and generally acceptable in terms of effects on scientific results.

54 ENVIRONMENTAL SCIENCES↗

Real-time computer data system for the 40- by 80-foot wind tunnel facility at Ames Research Center

The background material and operational concepts of a computer-based system for an operating wind tunnel are described. An on-line real-time computer system was installed in a wind tunnel facility to gather static and dynamic data. The computer system monitored aerodynamic forces and moments of periodic and quasi-periodic functions, and displayed and plotted computed results in real time. The total system is comprised of several off-the-shelf, interconnected subsystems that are linked to a large data processing center. The system includes a central processor unit with 32,000 24-bit words of core memory, a number of standard peripherals, and several special processors; namely, a dynamic analysis subsystem, a 256-channel PCM-data subsystem and ground station, a 60-channel high-speed data acquisition subsystem, a communication link, and static force and pressure subsystems. The role of the test engineer as a vital link in the system is also described.

Cambra, J. M.↗

The Behavior of Filters and Smoothers for Strongly Nonlinear Dynamics

The Kalman filter is the optimal filter in the presence of known Gaussian error statistics and linear dynamics. Filter extension to nonlinear dynamics is non trivial in the sense of appropriately representing high order moments of the statistics. Monte Carlo, ensemble-based, methods have been advocated as the methodology for representing high order moments without any questionable closure assumptions (e.g., Miller 1994). Investigation along these lines has been conducted for highly idealized dynamics such as the strongly nonlinear Lorenz (1963) model as well as more realistic models of the oceans (Evensen and van Leeuwen 1996) and atmosphere (Houtekamer and Mitchell 1998). A few relevant issues in this context are related to the necessary number of ensemble members to properly represent the error statistics and, the necessary modifications in the usual filter equations to allow for correct update of the ensemble members (Burgers 1998). The ensemble technique has also been applied to the problem of smoothing for which similar questions apply. Ensemble smoother examples, however, seem to quite puzzling in that results of state estimate are worse than for their filter analogue (Evensen 1997). In this study, we use concepts in probability theory to revisit the ensemble methodology for filtering and smoothing in data assimilation. We use Lorenz (1963) model to test and compare the behavior of a variety implementations of ensemble filters. We also implement ensemble smoothers that are able to perform better than their filter counterparts. A discussion of feasibility of these techniques to large data assimilation problems will be given at the time of the conference.

Zhu, Yanqiu↗

Detecting And Characterizing Archetypes of Unintended Consequences in Engineered Systems

When designing engineered systems, the potential for unintended consequences of design policies or design decisions exists despite best intentions. Conditions that might cause the formation of unintended consequences are often known only in hindsight. However, since these conditions are associated with a single event, it is difficult to uncover the general patterns of conditions leading to unintended consequences. In this research, patterns of conditions associated with unintended consequences are learned from historical data and represented in the form of archetypes. While previous work using systems theoretic modeling has identified high-level archetypes, this work leverages a self-organizing map to learn archetypes of unintended consequences from human-tagged risk factors in a large data set of lessons learned from adverse events at NASA. The sixty-six identified archetypes contain patterns of conditions such as complexity and human-machine interaction associated with the formation of unintended consequences. To validate the archetypes, a sample of the archetypes is represented using system dynamics in order to illustrate that the identified archetypes are specialized versions of known high-level archetypes of unintended consequences. While the research is based upon a specific dataset, the archetypes apply to any engineered system and the pattern of leading indicators open a new path to manage unintended consequences and mitigate the magnitude of potentially adverse outcomes.

Hannah S Walsh↗

Mass Dependence of Galaxy–Halo Alignment in LOWZ and CMASS

We measure the galaxy-ellipticity (GI) correlations for the Sloan Digital Sky Survey Data Release 12 LOWZ and CMASS samples with the shape measurements from the DESI Legacy Imaging Surveys. We model the GI correlations in an N-body simulation with our recent accurate stellar–halo mass relation from the Photometric object Around Cosmic webs (PAC) method. The large data set and our accurate modeling turns out an accurate measurement of the alignment angle between central galaxies and their host halos. We find that the alignment of central elliptical galaxies with their host halos increases monotonically with galaxy stellar mass or host halo mass, which can be well described by a power law for the massive galaxies. We also find that central elliptical galaxies are more aligned with their host halos in LOWZ than in CMASS, which might indicate an evolution of galaxy–halo alignment, though future studies are needed to verify this is not induced by the sample selections. In contrast, central disk galaxies are aligned with their host halos about 10 times more weakly in the GI correlation. These results have important implications for intrinsic alignment (IA) correction in weak lensing studies, IA cosmology, and theory of massive galaxy formation.

79 ASTRONOMY AND ASTROPHYSICS↗

Implementation of a data management software system for SSME test history data

The implementation of a software system for managing Space Shuttle Main Engine (SSME) test/flight historical data is presented. The software system uses the database management system RIM7 for primary data storage and routine data management, but includes several FORTRAN programs, described here, which provide customized access to the RIM7 database. The consolidation, modification, and transfer of data from the database THIST, to the RIM7 database THISRM is discussed. The RIM7 utility modules for generating some standard reports from THISRM and performing some routine updating and maintenance are briefly described. The FORTRAN accessing programs described include programs for initial loading of large data sets into the database, capturing data from files for database inclusion, and producing specialized statistical reports which cannot be provided by the RIM7 report generator utility. An expert system tutorial, constructed using the expert system shell product INSIGHT2, is described. Finally, a potential expert system, which would analyze data in the database, is outlined. This system could use INSIGHT2 as well and would take advantage of RIM7's compatibility with the microcomputer database system RBase 5000.

Abernethy, Kenneth↗

Behavior of Filters and Smoothers for Strongly Nonlinear Dynamics

The Kalman filter is the optimal filter in the presence of known gaussian error statistics and linear dynamics. Filter extension to nonlinear dynamics is non trivial in the sense of appropriately representing high order moments of the statistics. Monte Carlo, ensemble-based, methods have been advocated as the methodology for representing high order moments without any questionable closure assumptions. Investigation along these lines has been conducted for highly idealized dynamics such as the strongly nonlinear Lorenz model as well as more realistic models of the means and atmosphere. A few relevant issues in this context are related to the necessary number of ensemble members to properly represent the error statistics and, the necessary modifications in the usual filter situations to allow for correct update of the ensemble members. The ensemble technique has also been applied to the problem of smoothing for which similar questions apply. Ensemble smoother examples, however, seem to be quite puzzling in that results state estimates are worse than for their filter analogue. In this study, we use concepts in probability theory to revisit the ensemble methodology for filtering and smoothing in data assimilation. We use the Lorenz model to test and compare the behavior of a variety of implementations of ensemble filters. We also implement ensemble smoothers that are able to perform better than their filter counterparts. A discussion of feasibility of these techniques to large data assimilation problems will be given at the time of the conference.

Zhu, Yanqui↗

Development of the Equatorial Infrared Catalogue from satellite data

More than 40,000 infrared measurements of stellar sources have been obtained since November, 1976 during the ongoing process of compiling an Equatorial Infrared Catalogue. Because of the problem of eliminating spurious sources, which has affected earlier space surveys, an extensive effort is being made to verify the sources by means of (1) repetitive observations by satellite sensors, (2) cross correlation with a large data base developed from ground-based and space surveys at other wavelengths, and (3) investigation of a significant subset of the sources with a ground-based infrared telescope. As sources are verified, they are transferred from a working list to a screened preliminary version of the catalogue. The catalogue comprises the only survey of a significant area of the sky that has been accomplished (or is presently planned) with positional accuracies of a few arc seconds at a wavelength of not less than 2 microns.

Heinsheimer, T. F.↗

Flexible language constructs for large parallel programs

The goal of the research described is to develop flexible language constructs for writing large data parallel numerical programs for distributed memory (MIMD) multiprocessors. Previously, several models have been developed to support synchronization and communication. Models for global synchronization include SIMD (Single Instruction Multiple Data), SPMD (Single Program Multiple Data), and sequential programs annotated with data distribution statements. The two primary models for communication include implicit communication based on shared memory and explicit communication based on messages. None of these models by themselves seem sufficient to permit the natural and efficient expression of the variety of algorithms that occur in large scientific computations. An overview of a new language that combines many of these programming models in a clean manner is given. This is done in a modular fashion such that different models can be combined to support large programs. Within a module, the selection of a model depends on the algorithm and its efficiency requirements. An overview of the language and discussion of some of the critical implementation details is given.

Rosing, Matthew↗

Assessment of CFD-based Response Surface Model for Ares I Supersonic Ascent Aerodynamics

The Ascent Force and Moment Aerodynamic (AFMA) Databases (DBs) for the Ares I Crew Launch Vehicle (CLV) were typically based on wind tunnel (WT) data, with increments provided by computational fluid dynamics (CFD) simulations for aspects of the vehicle that could not be tested in the WT tests. During the Design Analysis Cycle 3 analysis for the outer mold line (OML) geometry designated A106, a major tunnel mishap delayed the WT test for supersonic Mach numbers (M) greater than 1.6 in the Unitary Plan Wind Tunnel at NASA Langley Research Center, and the test delay pushed the final delivery of the A106 AFMA DB back by several months. The aero team developed an interim database based entirely on the already completed CFD simulations to mitigate the impact of the delay. This CFD-based database used a response surface methodology based on radial basis functions to predict the aerodynamic coefficients for M > 1.6 based on only the CFD data from both WT and flight Reynolds number conditions. The aero team used extensive knowledge of the previous AFMA DB for the A103 OML to guide the development of the CFD-based A106 AFMA DB. This report details the development of the CFD-based A106 Supersonic AFMA DB, constructs a prediction of the database uncertainty using data available at the time of development, and assesses the overall quality of the CFD-based DB both qualitatively and quantitatively. This assessment confirms that a reasonable aerodynamic database can be constructed for launch vehicles at supersonic conditions using only CFD data if sufficient knowledge of the physics and expected behavior is available. This report also demonstrates the applicability of non-parametric response surface modeling using radial basis functions for development of aerodynamic databases that exhibit both linear and non-linear behavior throughout a large data space.

Hanke, Jeremy L.↗

DEIMoS: An Open-Source Tool for Processing High-Dimensional Mass Spectrometry Data

We present DEIMoS: Data Extraction for Integrated Multidimensional Spectrometry, a Python application programming interface and command-line tool for high-dimensional mass spectrometry data analysis workflows, offering ease of development and access to efficient algorithmic implementations. Functionality includes feature detection, feature alignment, collision cross section calibration, isotope detection, and MS/MS spectral deconvolution, with the output comprising detected features aligned across study samples and characterized by mass, CCS, tandem mass spectra, and isotopic signature. Notably, DEIMoS operates on N-dimensional data, largely agnostic to acquisition instrumentation: algorithm implementations utilize all dimensions simultaneously to (i) offer greater separation between features, improving detection sensitivity, (ii) increase alignment/feature matching confidence among datasets, and (iii) mitigate convolution artifacts in tandem mass spectra. We demonstrate DEIMoS with LC-IMS-MS/MS data, demonstrating the advantages of a multidimensional approach in each data processing step.

Colby, Sean M↗

Super instruction architecture of petascale electronic structure software: the story

Theoretical methods in chemistry lead to algorithms for the computation of electronic energies and other properties of electronic wave functions that require large numbers of floating point operations and involve large data sets. Thus, computational chemists are very interested in using massively parallel computer systems and in particular the new petascale systems. In this paper we discuss a new programming paradigm that was developed at the Quantum Theory Project to construct electronic structure software that can scale to large numbers of cores of the order of 100,000 and beyond to solve problems in materials engineering relevant to the problems facing society today.

Chemistry↗

Data Preservation in High Energy Physics

Data from high-energy physics (HEP) experiments are collected with significant financial and human effort and are mostly unique. At the same time, HEP has no coherent strategy for data preservation and re-use. An inter-experimental Study Group on HEP data preservation and long-term analysis was convened at the end of 2008 and held two workshops, at DESY (January 2009) and SLAC (May 2009). This document is an intermediate report to the International Committee for Future Accelerators (ICFA) of the reflections of this Study Group. Large data sets accumulated during many years of detector operation at particle accelerators are the heritage of experimental HEP. These data sets offer unique opportunities for future scientific studies, sometimes long after the shut-down of the actual experiments: new theoretical input; new experimental results and analysis techniques; the quest for high-sensitivity combined analyses; the necessity of cross checks. In many cases, HEP data sets are unique; they cannot and most likely will not be superseded by data from newer generations of experiments. Once lost, or in an unusable state, HEP data samples cannot be reasonably recovered. The cost of conserving this heritage through a collaborative, target-oriented long-term data preservation program would be small, compared to the costs of past experimental projects or to the efforts to re-do experiments. However, this cost is not negligible, especially for collaborations close or past their end-date. The preservation of HEP data would provide today's collaborations with a secure way to complete their data analysis and enable them to seize new scientific opportunities in the coming years. The HEP community will benefit from preserved data samples through reanalysis, combination, education and outreach. Funding agencies would receive more scientific return, and a positive image, from their initial investment leading to the production and the first analysis of preserved data.

43 PARTICLE ACCELERATORS↗

Finite-time correlations boost large voltage angle fluctuations in electric power grids

Abstract Decarbonization in the energy sector has been accompanied by an increased penetration of new renewable energy sources in electric power systems. Such sources differ from traditional productions in that, first, they induce larger, undispatchable fluctuations in power generation and second, they lack inertia. Recent measurements have indeed reported long, non-Gaussian tails in the distribution of local voltage frequency data. Large frequency deviations may induce grid instabilities, leading in worst-case scenarios to cascading failures and large-scale blackouts. In this article, we investigate how correlated noise disturbances, characterized by the cumulants of their distribution, propagate through meshed, high-voltage power grids. For a single source of fluctuations, we show that long noise correlation times boost non-Gaussian voltage angle fluctuations so that they propagate similarly to Gaussian fluctuations over the entire network. However, they vanish faster, over short distances if the noise fluctuates rapidly. We furthermore demonstrate that a Berry–Esseen theorem leads to the vanishing of non-Gaussianities as the number of uncorrelated noise sources increases. Our predictions are corroborated by numerical simulations on realistic models of power grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗

AUTONOMOUS GAUSSIAN DECOMPOSITION

We present a new algorithm, named Autonomous Gaussian Decomposition (AGD), for automatically decomposing spectra into Gaussian components. AGD uses derivative spectroscopy and machine learning to provide optimized guesses for the number of Gaussian components in the data, and also their locations, widths, and amplitudes. We test AGD and find that it produces results comparable to human-derived solutions on 21 cm absorption spectra from the 21 cm SPectral line Observations of Neutral Gas with the EVLA (21-SPONGE) survey. We use AGD with Monte Carlo methods to derive the H i line completeness as a function of peak optical depth and velocity width for the 21-SPONGE data, and also show that the results of AGD are stable against varying observational noise intensity. The autonomy and computational efficiency of the method over traditional manual Gaussian fits allow for truly unbiased comparisons between observations and simulations, and for the ability to scale up and interpret the very large data volumes from the upcoming Square Kilometer Array and pathfinder telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗