SEARCH · Search NASA

Results for “massive data sets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Comparing Compressed and Full-Modeling analyses with FOLPS: implications for DESI 2024 and beyond

The Dark Energy Spectroscopic Instrument (DESI) will provide unprecedented information about the large-scale structure of our Universe. In this work, we study the robustness of the theoretical modelling of the power spectrum of F OLPS , a novel effective field theory-based package for evaluating the redshift space power spectrum in the presence of massive neutrinos. We perform this validation by fitting the AbacusSummit high-accuracy N -body simulations for Luminous Red Galaxies, Emission Line Galaxies and Quasar tracers, calibrated to describe DESI observations. We quantify the potential systematic error budget of F OLPS finding that the modelling errors are fully sub-dominant for the DESI statistical precision within the studied range of scales. Additionally, we study two complementary approaches to fit and analyse the power spectrum data, one based on direct Full-Modelling fits and the other on the ShapeFit compression variables, both resulting in very good agreement in precision and accuracy. In each of these approaches, we study a set of potential systematic errors induced by several assumptions, such as the choice of template cosmology, the effect of prior choice in the nuisance parameters of the model, or the range of scales used in the analysis. Furthermore, we show how opening up the parameter space beyond the vanilla ΛCDM model affects the DESI observables. These studies include the addition of massive neutrinos, spatial curvature, and dark energy equation of state. We also examine how relaxing the usual Cosmic Microwave Background and Big Bang Nucleosynthesis priors on the primordial spectral index and the baryonic matter abundance, respectively, impacts the inference on the rest of the parameters of interest. This paper pathways towards performing a robust and reliable analysis of the shape of the power spectrum of DESI galaxy and quasar clustering using F OLPS .

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2009

Satellite Contributions to Global Change Studies

By providing a global view with a level playing field (no region missed because of unfavorable surface conditions or political boundaries), satellites have made major contributions to improved monitoring and understanding of our constantly changing planet. The global view has allowed surprising realizations like the relative sparsity of lightning strikes over oceans and the large-scale undulations on the massive Antarctic ice sheet. It has allowed the tracking of all sorts of phenomena, including aerosols, both natural and anthropogenic, as they move with the atmospheric circulation and impact weather and human health. But probably nothing that the global view allows is more important in the long term than its provision. of unbiased data sets to address the issue of global change, considered by many to be among the most important issues facing humankind today. With satellites we can monitor atmospheric temperatures at all latitudes and longitudes, and obtain a global average that lessens the likelihood of becoming endlessly mired in the confusions brought about by the certainty of regional differences. With satellites we can monitor greenhouse gases such as CO2 not just above individual research stations but around the globe. With satellites we can monitor the polar sea ice covers, as we have done since the late 1970s, determining and quantifying the significant reduction in Arctic sea ice and the slight growth in Antarctic sea ice over that period, With satellites we can map the full extent and changes in the Antarctic stratospheric ozone depletions that were first identified from using a single ground station; and through satellite data we have witnessed from afar land surface changes brought about by humans both intentionally, as with wide-scale deforestation, and unintentionally, as with the decay of the Aral Sea. The satellite data are far from sufficient for all that we need in order to understand the global system and forecast its changes, as we also need sophisticated climate models, in situ process studies, and data sets that extend back well before the introduction of satellite technology. Nonetheless, the repetitive, global view provided by satellites is contributing in a major way to our improved recognition of how the Earth im changing, a recognition that is none too soon in view of the magnitude of the impacts that humans can now have.

Parkinson, Claire L.↗

NASA NTRS2010

Applied Meteorology Unit (AMU) Quarterly Report

By providing a global view with a level playing field (no region missed because of unfavorable surface conditions or political boundaries), satellites have made major contributions to improved monitoring and understanding of our constantly changing planet. The global view has allowed surprising realizations like the relative sparsity of lightning strikes over oceans and the large-scale undulations on the massive Antarctic ice sheet. It has allowed the tracking of all sorts of phenomena, including aerosols, both natural and anthropogenic,as they move with the atmospheric circulation and impact weather and human health. But probably nothing that the global view allows is more important in the long term than its provision of unbiased data sets to address the issue of global change, considered by many to be among the most important issues facing humankind today. With satellites we can monitor atmospheric temperatures at all latitudes and longitudes, and obtain a global average that lessens the likelihood of becoming endlessly mired in the confusions brought about by the certainty of regional differences. With satellites we can monitor greenhouse gases such as CO2 not just above individual research stations but around the globe. With satellites we can monitor the polar sea ice covers, as we have done since the late 1970s, determining and quantifying the significant reduction in Arctic sea ice and the slight growth in Antarctic sea ice over that period. With satellites we can map the full extent and changes in the Antarctic stratospheric ozone depletions that were first identified from a single ground station; and through satellite data we have witnessed from afar land surface changes brought about by humans both intentionally, as with wide-scale deforestation, and unintentionally, as with the decay of the Aral Sea. The satellite data are far from sufficient for all that we need in order to understand the global system and forecast its changes, as we also need sophisticated climate models, in situ process studies, and data sets that extend back well before the introduction of satellite technology. Nuomthc)cmm, the repetitive, global view provided by satellites is contributing in a major way to our improved recognition of how the Earth is changing, a recognition that is none too soon in view of the magnitude of the impacts that humans can now have.

Bauman, William↗

NASA NTRS1997

The Structure of the Broad-Line Region in Well-Studied AGNs

Large amounts of high quality UV and optical data have been obtained in massive multi-wavelength monitoring campaigns on a small number of active galactic nuclei, and these data are changing our understanding of the central engines in these sources in a fundamental way. Preliminary analyses have shown that more comprehensive approaches will be necessary to make full use of these data. We propose to undertake a complete set of photoionization equilibrium calculations with a state-of-the-art computer code in order to determine the radial structure of the broad-line region in a way that is consistent with the emission-line fluxes, profiles, and transfer functions.

Peterson, Bradley M.↗

NASA NTRS2005

NASA's Earth Observations of the Global Environment

A birds eye view of the Earth from afar and up close reveals the power and magnificence of the Earth and juxtaposes the simultaneous impacts and powerlessness of humankind. The NASA Electronic Theater presents Earth science observations and visualizations in an historical perspective. Fly in from outer space to Africa and Cape Town. See the latest spectacular images from NASA & NOAA remote sensing missions like Meteosat, TRMM, Landsat 7, and Terra, which will be visualized and explained in the context of global change. See visualizations of global data sets currently available from Earth orbiting satellites, including the Earth at night with its city lights, aerosols from biomass burning in the Middle East and Africa, and retreat of the glaciers on Mt. Kilimanjaro. See the dynamics of vegetation growth and decay over Africa over 17 years. New visualization tools allow us to roam & zoom through massive global mosaic images including Landsat and Terra tours of Africa and South America, showing land use and land cover change from Bolivian highlands. Spectacular new visualizations of the global atmosphere & oceans are shown. See massive dust storms sweeping across Africa and across the Atlantic to the Caribbean and Amazon basin. See ocean vortexes and currents that bring up the nutrients to feed tiny phytoplankton and draw the fish, pant whales and fisher- man. See how the ocean blooms in response to these currents and El Nino/La Nifia. We will illustrate these and other topics with a dynamic theater-style presentation, along with animations of satellite launch deployments and orbital mapping to highlight aspects of Earth observations from space.

King, Michael D.↗

NASA NTRS2013

Statistical Issues in Galaxy Cluster Cosmology

The number and growth of massive galaxy clusters are sensitive probes of cosmological structure formation. Surveys at various wavelengths can detect clusters to high redshift, but the fact that cluster mass is not directly observable complicates matters, requiring us to simultaneously constrain scaling relations of observable signals with mass. The problem can be cast as one of regression, in which the data set is truncated, the (cosmology-dependent) underlying population must be modeled, and strong, complex correlations between measurements often exist. Simulations of cosmological structure formation provide a robust prediction for the number of clusters in the Universe as a function of mass and redshift (the mass function), but they cannot reliably predict the observables used to detect clusters in sky surveys (e.g. X-ray luminosity). Consequently, observers must constrain observable-mass scaling relations using additional data, and use the scaling relation model in conjunction with the mass function to predict the number of clusters as a function of redshift and luminosity.

Galaxy↗

NASA NTRS1989

Image segmentation by iterative parallel region growing and splitting

The spatially constrained clustering (SCC) iterative parallel region-growing technique is applied to image analysis. The SCC algorithm is implemented on the massively parallel processor at NASA Goddard. Most previous region-growing approaches have the drawback that the segmentation produced depends on the order in which portions of the image are processed. The ideal solution to this problem (merging only the single most similar pair of spatially adjacent regions in the image in each iteration) becomes impractical except for very small images, even on a massively parallel computer. The SCC algorithm overcomes these problems by performing, in parallel, the best merge within each of a set of local, possibly overlapping, subimages. A region-splitting stage is also incorporated into the algorithm, but experiments show that region splitting generally does not improve segmentation results. The SCC algorithm has been tested on various imagery data, and test results for a Landsat TM image are summarized.

Tilton, James C.↗

NASA NTRS2011

AIRS Impact on Analysis and Forecast of an Extreme Rainfall Event (Indus River Valley 2010) with a Global Data Assimilation and Forecast System

A set of data assimilation and forecast experiments are performed with the NASA Global data assimilation and forecast system GEOS-5, to compare the impact of different approaches towards assimilation of Advanced Infrared Spectrometer (AIRS) data on the precipitation analysis and forecast skill. The event chosen is an extreme rainfall episode which occurred in late July 11 2010 in Pakistan, causing massive floods along the Indus River Valley. Results show that the assimilation of quality-controlled AIRS temperature retrievals obtained under partly cloudy conditions produce better precipitation analyses, and substantially better 7-day forecasts, than assimilation of clear-sky radiances. The improvement of precipitation forecast skill up to 7 day is very significant in the tropics, and is caused by an improved representation, attributed to cloudy retrieval assimilation, of two contributing mechanisms: the low-level moisture advection, and the concentration of moisture over the area in the days preceding the precipitation peak.

Reale, O.↗

NASA NTRS1991

Massive star evolution and SN 1987A

The evolution of massive stars through hydrogen and helium burning is addressed. A set of stellar evolutionary sequences for mass/solar mass of 15, 20, and 25, and metallicity of 0.002, 0.005, 0.007, 0.010, and 0.20 are presented; semiconvection is restricted to operating slower than the local thermal time scale. Using these sequences, simple models of the massive star content of the LMC are found to agree moderately well with the new observational data of Fitzpatrick and Garmany (1990). LMC supergiants were detected only in their postmain-sequence phases, so that 5-10 times more massive stars are there but not identified as such. It is argued that SN 1987A exhibits the normal evolution of a single star of about 20 solar mases having LMC abundances. Despite the variety of envelope behavior, the structure of the core at collapse is rather similar for the stars of a given mass. Variations due to different rates of mass loss are likely to be larger than those due to composition.

Arnett, David↗

DOE OSTI2024

Efficient distributed continual learning for steering experiments in real-time

Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fully available from the beginning), incremental training suffers from catastrophic forgetting (i.e., new patterns are reinforced at the expense of previously acquired knowledge). Training from scratch each time new training data becomes available would result in extremely long training times and massive data accumulation. Rehearsal-based continual learning has shown promise for addressing the catastrophic forgetting challenge, but research to date has not addressed performance and scalability. To fill this gap, we propose an approach based on a distributed rehearsal buffer that efficiently complements data-parallel training on multiple GPUs to achieve high accuracy, short runtime, and scalability. It leverages a set of buffers (local to each GPU) and uses several asynchronous techniques for updating these local buffers in an embarrassingly parallel fashion, all while handling the communication overheads necessary to augment input minibatches using unbiased, global sampling. We further propose a generalization of rehearsal buffers to support both classification and generative learning tasks, as well as more advanced rehearsal strategies (notably Dark Experience Replay, leveraging knowledge distillation). We illustrate this approach with a real-life HPC streaming application from the domain of ptychographic image reconstruction. Furthermore, we run extensive experiments on up to 128 GPUs of the ThetaGPU supercomputer to compare our approach with baselines representative of training-from-scratch (the upper bound in terms of accuracy) and incremental training (the lower bound). Results show that rehearsal-based continual learning achieves a top-5 validation accuracy close to the upper bound, while simultaneously exhibiting a runtime close to the lower bound.

Asynchronous data management↗

NASA NTRS2002

A Perspective of Our Planet's Atmosphere, Land, and Oceans: A View from Space

King, Michael D.↗

NASA NTRS1992

The X-ray spectrum of Abell 665

The X-ray spectrum of the Abell 665 cluster of galaxies has been determined by the Ginga satellite. The spectrum can be well fitted by an isothermal model with kT = 8.26(+0.95; -0.81) keV and metal abundance 0.49 +/- 0.16. Joint analysis with lower energy spectral data from the Einstein Observatory is used to constrain the temperature distribution in the cluster. For a particular form of the polytropic temperature model we find the best fit to be the isothermal case, and we set an upper limit on the polytropic index of 1.35 (99-percent confidence). The virial mass of the cluster has been determined from both the X-ray data and the galaxy velocity dispersion. The mass estimates from the optical and X-ray analyses are fully consistent with each other. Abell 665 is found to be 0.82(+0.41; -0.26) times as massive as Coma if the two clusters are assumed to have the same spatial distribution of virial mass.

Hughes, John P.↗

NASA NTRS1994

Design and implementation of a parallel unstructured Euler solver using software primitives

This paper is concerned with the implementation of a three-dimensional unstructured-grid Euler solver on massively parallel distributed-memory computer architectures. The goal is to minimize solution time by achieving high computational rates with a numerically efficient algorithm. An unstructured multigrid algorithm with an edge-based data structure has been adopted, and a number of optimizations have been devised and implemented to accelerate the parallel computational rates. The implementation is carried out by creating a set of software tools, which provide an interface between the parallelization issues and the sequential code, while providing a basis for future automatic run-time compilation support. Large practical unstructured grid problems are solved on the Intel iPSC/860 hypercube and Intel Touchstone Delta machine. The quantitative effects of the various optimizations are demonstrated, and we show that the combined effect of these optimizations leads to roughly a factor of 3 performance improvement. The overall solution efficiency is compared with that obtained on the Cray Y-MP vector supercomputer.

Das, R.↗

NASA NTRS2002

A Perspective of Our Planet's Atmosphere, Land, and Oceans: A View from Space

A birds eye view of the Earth from afar and up close reveals the power and magnificence of the Earth and juxtaposes the simultaneous impacts and powerlessness of humankind. The NASA Electronic Theater presents Earth science observations and visualizations in an historical perspective. Fly in from outer space to South America with its Andes Mountains and the glaciers of Patagonia, ending up close and personal in Buenos Aires. See the latest spectacular images from NASA & NOAA remote sensing missions like GOES, TRMM, Landsat 7, QuikScat, and Terra, which will be visualized and explained in the context of global change. See visualizations of global data sets currently available from Earth orbiting satellites, including the Earth at night with its city lights, aerosols from biomass burning in South America and Africa, and global cloud properties. See the dynamics of vegetation growth and decay over South America over 17 years, and its contrast to the North American and Africa continents. New visualization tools allow us to roam & zoom through massive global mosaic images from the Himalayas to the dynamics of the Pacific Ocean that affect the climate of South and North America. New visualization tools allow us to roam & zoom through massive global mosaic images including Landsat and Terra tours of South America and Africa showing land use and land cover change from Patagonia to the Amazon Basin, including the Andes Mountains, the Pantanal, and the Bolivian highlands. Landsat flyins to Rio Di Janeiro and Buenos Aires will be shows to emphasize the capabilities of new satellite technology to visualize our natural environment. Spectacular new visualizations of the global atmosphere & oceans are shown. See massive dust storms sweeping across Africa and across the Atlantic to the Caribbean and Amazon basin. See ocean vortexes and currents that bring up the nutrients to feed tiny phytoplankton and draw the fish, giant whales and fisherman. See how the ocean blooms in response to these currents and El Nino/La Nina climate changes. We will illustrate these and other topics with a dynamic theater-style presentation, along with animations of satellite launch deployments and orbital mapping to highlight aspects of Earth observations from space.

King, Michael D.↗

NASA NTRS1992

The design and implementation of a parallel unstructured Euler solver using software primitives

This paper is concerned with the implementation of a three-dimensional unstructured grid Euler-solver on massively parallel distributed-memory computer architectures. The goal is to minimize solution time by achieving high computational rates with a numerically efficient algorithm. An unstructured multigrid algorithm with an edge-based data structure has been adopted, and a number of optimizations have been devised and implemented in order to accelerate the parallel communication rates. The implementation is carried out by creating a set of software tools, which provide an interface between the parallelization issues and the sequential code, while providing a basis for future automatic run-time compilation support. Large practical unstructured grid problems are solved on the Intel iPSC/860 hypercube and Intel Touchstone Delta machine. The quantitative effect of the various optimizations are demonstrated, and we show that the combined effect of these optimizations leads to roughly a factor of three performance improvement. The overall solution efficiency is compared with that obtained on the CRAY-YMP vector supercomputer.

Das, R.↗

NASA NTRS1994

TOPEX/POSEIDON tides estimated using a global inverse model

Altimetric data from the TOPEX/POSEIDON mission will be used for studies of global ocean circulation and marine geophysics. However, it is first necessary to remove the ocean tides, which are aliased in the raw data. The tides are constrained by the two distinct types of information: the hydrodynamic equations which the tidal fields of elevations and velocities must satisfy, and direct observational data from tide gauges and satellite altimetry. Here we develop and apply a generalized inverse method, which allows us to combine rationally all of this information into global tidal fields best fitting both the data and the dynamics, in a least squares sense. The resulting inverse solution is a sum of the direct solution to the astronomically forced Laplace tidal equations and a linear combination of the representers for the data functionals. The representer functions (one for each datum) are determined by the dynamical equations, and by our prior estimates of the statistics or errors in these equations. Our major task is a direct numerical calculation of these representers. This task is computationally intensive, but well suited to massively parallel processing. By calculating the representers we reduce the full (infinite dimensional) problem to a relatively low-dimensional problem at the outset, allowing full control over the conditioning and hence the stability of the inverse solution. With the representers calculated we can easily update our model as additional TOPEX/POSEIDON data become available. As an initial illustration we invert harmonic constants from a set of 80 open-ocean tide gauges. We then present a practical scheme for direct inversion of TOPEX/POSEIDON crossover data. We apply this method to 38 cycles of geophysical data records (GDR) data, computing preliminary global estimates of the four principal tidal constituents, M(sub 2), S(sub 2), K(sub 1) and O(sub 1). The inverse solution yields tidal fields which are simultaneously smoother, and in better agreement with altimetric and ground truth data, than previously proposed tidal models. Relative to the 'default' tidal corrections provided with the TOPEX/POSEIDON GDR, the inverse solution reduces crossover difference variances significantly (approximately 20-30%), even though only a small number of free parameters (approximately equal to 1000) are actually fit to the crossover data.

Egbert, Gary D.↗

NASA NTRS1992

The design and implementation of a parallel unstructured Euler solver using software primitives

This paper is concerned with the implementation of a 3D unstructured-grid Euler-solver on massively parallel distributed-memory computer architectures. The goal is to minimize solution time by achieving high computational rates with a numerically efficient algorithm. An unstructured multigrid algorithm with an edge-based data-structure has been adopted, and a number of optimizations have been devised and implemented in order to accelerate the parallel computational rates. The implementation is carried out by creating a set of software tools, which ease the implementation of computational problems on parallel architecture machines by relieving the user of the low-level machine specific issues. The quantitative effect of the various optimizations are demonstrated, and we show that the combined effect of these optimizations leads to roughly a factor of three performance improvement. The overall solution efficiency is compared with that obtained on the CRAY-YMP vector supercomputer.

Das, R.↗

NASA NTRS1993

Data fusion with artificial neural networks (ANN) for classification of earth surface from microwave satellite measurements

A data fusion system with artificial neural networks (ANN) is used for fast and accurate classification of five earth surface conditions and surface changes, based on seven SSMI multichannel microwave satellite measurements. The measurements include brightness temperatures at 19, 22, 37, and 85 GHz at both H and V polarizations (only V at 22 GHz). The seven channel measurements are processed through a convolution computation such that all measurements are located at same grid. Five surface classes including non-scattering surface, precipitation over land, over ocean, snow, and desert are identified from ground-truth observations. The system processes sensory data in three consecutive phases: (1) pre-processing to extract feature vectors and enhance separability among detected classes; (2) preliminary classification of Earth surface patterns using two separate and parallely acting classifiers: back-propagation neural network and binary decision tree classifiers; and (3) data fusion of results from preliminary classifiers to obtain the optimal performance in overall classification. Both the binary decision tree classifier and the fusion processing centers are implemented by neural network architectures. The fusion system configuration is a hierarchical neural network architecture, in which each functional neural net will handle different processing phases in a pipelined fashion. There is a total of around 13,500 samples for this analysis, of which 4 percent are used as the training set and 96 percent as the testing set. After training, this classification system is able to bring up the detection accuracy to 94 percent compared with 88 percent for back-propagation artificial neural networks and 80 percent for binary decision tree classifiers. The neural network data fusion classification is currently under progress to be integrated in an image processing system at NOAA and to be implemented in a prototype of a massively parallel and dynamically reconfigurable Modular Neural Ring (MNR).

Lure, Y. M. Fleming↗