Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
NASA NTRS2019

Machine Learning for the Zwicky Transient Facility

The Zwicky Transient Facility is a large optical survey in multiple filters producing hundreds of thousands of transient alerts per night. We describe here various machine learning (ML) implementations and plans to make the maximal use of the large data set by taking advantage of the temporal nature of the data, and further combining it with other data sets. We start with the initial steps of separating bogus candidates from real ones, separating stars and galaxies, and go on to the classification of real objects into various classes. Besides the usual methods (e.g., based on features extracted from light curves) we also describe early plans for alternate methods including the use of domain adaptation, and deep learning. In a similar fashion we describe efforts to detect fast moving asteroids. We also describe the use of the Zooniverse platform for helping with classifications through the creation of training samples, and active learning. Finally we mention the synergistic aspects of ZTF and LSST from the ML perspective.

Ashish Mahabal↗
NASA NTRSDate not supplied

Nancy Grace Roman Space Telescope Observatory Implementation and Challenges

NASA’s Nancy Grace Roman Space Telescope (Roman) is a deep space infrared observatory with a Hubble-sized telescope and wide field of regard with a boresight view greater than 200 times that of the Hubble Wide Field Camera 3infrared field of view, that will conduct a suite of science surveys to characterize dark energy and expand the census of exoplanets in our galaxy. Roman will also demonstrate exoplanet coronagraphy with active wavefront control technology and provide general investigator opportunities for the science community. Roman is finishing the critical design phase and is planning for launch in 2026. It will operate at the second Sun-Earth-Moon Lagrange for a five-year primary mission life. The Observatory features a telescope with an existing, repurposed 2.4m primary mirror, a Wide Field Instrument with a near-infrared detector focal plane array and optical elements for imaging and spectroscopy, as well as a Coronagraph instrument technology demonstration for direct imaging and spectroscopy of exoplanets. The telescope and instruments are mounted to an Instrument Carrier for optical metering and isolation from Spacecraft-induced disturbances. The Spacecraft includes a Bus, Solar Array Sunshield, Deployable Aperture Cover, Outer Barrel Assembly, and Star Tracker/Inertial Reference Unit Bench. When fully integrated, Roman will be the largest Observatory assembled and tested at NASA’s Goddard Space Flight Center. The development of scientific satellites is challenging and pushes engineering boundaries to broaden scientific knowledge. The Roman mission implementation is a prime example and expected challenges have been amplified by the foundational decision to use existing, repurposed telescope components. Other unique aspects of the Roman mission, including its survey nature, large data volume, and Observatory packaging, create constrained design spaces that drive competing requirements across Observatory subsystems. Given these challenges, systems engineering has been a critical discipline in balancing implementation decisions and will continue to play a key role in the development of the Roman mission. This paper will discuss details of the Roman Observatory configuration, systems engineering challenges and the decision-making process used to mature the Roman Space Telescope from preliminary design to implementation.

Lisa M L Bartusek↗
NASA NTRS2022

Automatic variable selection in ecological niche modeling: A case study using Cassin’s Sparrow (Peucaea cassinii)

MERRA/Max provides a feature selection approach to dimensionality reduction that enables direct use of global climate model outputs in ecological niche modeling. The system accomplishes this reduction through a Monte Carlo optimization in which many independent MaxEnt runs, operating on a species occurrence file and a small set of randomly selected variables in a large collection of variables, converge on an estimate of the top contributing predictors in the larger collection. These top predictors can be viewed as potential candidates in the variable selection step of the ecological niche modeling process. MERRA/Max’s Monte Carlo algorithm operates on files stored in the underlying filesystem, making it scalable to large data sets. Its software components can run as parallel processes in a high-performance cloud computing environment to yield near real-time performance. In tests using Cassin’s Sparrow (Peucaea cassinii) as the target species, MERRA/Max selected a set of predictors from Worldclim’s Bioclim collection of 19 environmental variables that have been shown to be important determinants of the species’ bioclimatic niche. It also selected biologically and ecologically plausible predictors from a more diverse set of 86 environmental variables derived from NASA’s Modern-Era Retrospective Analysis for Research and Applications Version 2 (MERRA-2) reanalysis, an output product of the Goddard Earth Observing System Version 5 (GEOS-5) modeling system. We believe these results point to a technological approach that could expand the use global climate model outputs in ecological niche modeling, foster exploratory experimentation with otherwise difficult-to-use climate data sets, streamline the modeling process, and, eventually, enable automated bioclimatic modeling as a practical, readily accessible, low-cost, commercial cloud service.

John L. Schnase↗
NASA NTRSDate not supplied

Ocean and Earth System Modelling

Petascale supercomputing infrastructure + modelling and analysis capabilities + interdisciplinary upper-ocean expertise Multiscale ocean turbulence simulation Physical-biogeochemical interactions Analysis of large data sets from remote sensing and Earth system model ensembles

Ocean↗
NASA NTRSDate not supplied

Developing Fast and Accurate Radiative Transfer Models to Meet the Needs of Modern Satellite Remote Sensing Applications

Modern hyperspectral satellite remote sensors provide highly accurate measurements the Earth’s Top-of-Atmosphere (TOA) radiance, reflectance, or polarized spectra with hundreds to thousands of spectral channels and with millions of observations per day. The large data volume and high spectral dimensionality of the data pose challenges for retrieval algorithms. To process the satellite Level-1 data (e.g. calibrated TOA spectra) into Level-2 products (e.g. atmospheric and surface properties) using physical-based retrieval algorithms, accurate and fast Radiative Transfer Models (RTMs) are needed. RTMs are usually the limiting factor in determining the speed of a level-2 algorithm. For example, more than one million Line-by-Line (LBL) radiative transfer (RT) calculations are needed in order to properly capture the spectral contributions of important atmospheric molecules for an IR hyperspectral sensor with a spectral coverage from 3.5 m to 15 m or a solar hyperspectral sensor with spectral coverage from 0.25 m to 2.5 m. In this presentation, we will discuss advantages and disadvantages of different ways (e.g. correlated k and effective transmittance) to accelerate the speed of a fast RTM. We finally describe a Principal Component-based Radiative Transfer Model (PCRTM), which can calculate TOA radiance or reflectance spectra from 50 cm-1 to 40,000 cm-1 (200 m to 0.25 m). It has demonstrated very good accuracy relative to reference LBL RTMs and saves orders of magnitude in computational time. The PCRTM has been used in many satellite remote sensing applications. Examples include forward modeling in Level-2 and Level-3 retrieval algorithms, high fidelity satellite instrument simulators and instrument performance trade studies, spectral and radiometric accuracy characterizations of satellite Level-1 data, tools for inter-satellite calibrations, tools for satellite RTM lookup table generations, and tools for generating physically based training datasets for Artificial Intelligence (AI) algorithms.

climate data record↗
NASA NTRS1977

Orbital gamma-ray data and large-scale lunar problems

The orbital gamma-ray data yielded by the Apollo 15 and 16 missions have produced information about lunar differentiation. This information is also applicable to studying lunar petrogenesis using lunar samples. It is shown that the spatial variation in petrogenetic processes, as observed in the K/Th, Fe/Th, and Mg/Fe ratios, is of nearly the same magnitude as that of the lunar samples. The consistent relationship of both mare and nonmare basaltic surface chemical compositions, and the low surface elevation suggests a physical control on the movement of basaltic magma onto the lunar surface.

Hubbard, N. J.↗
NASA NTRS2002

Measurement of Unsteady Pressure Data on a Large HSCT Semispan Wing and Comparison with Analysis

Experimental data from wind-tunnel tests of the Rigid Semispan Model (RSM) performed at NASA Langley's Transonic Dynamics Tunnel (TDT) are presented. The primary focus of the paper is on data obtained from testing of the RSM on the Oscillating Turntable (OTT). The OTT is capable of oscillating models in pitch at various amplitudes and frequencies about mean angles of attack. Steady and unsteady pressure data obtained during testing of the RSM on the OTT is presented and compared to data obtained from previous tests of the RSM on a load balance and on a Pitch and Plunge Apparatus (PAPA). Testing of the RSM on the PAPA resulted in utter boundaries that were strongly dependent on angle of attack across the Mach number range. Pressure data from all three tests indicates the existence of vortical flows at moderate angles of attack. The correlation between the vortical flows and the unusual utter boundaries from the RSM/PAPA test is discussed. Comparisons of experimental data with analyses using the CFL3Dv6 computational fluid dynamics code are presented.

Scott, Robert C.↗
NASA NTRS2001

NASA's Information Power Grid: Large Scale Distributed Computing and Data Management

Large-scale science and engineering are done through the interaction of people, heterogeneous computing resources, information systems, and instruments, all of which are geographically and organizationally dispersed. The overall motivation for Grids is to facilitate the routine interactions of these resources in order to support large-scale science and engineering. Multi-disciplinary simulations provide a good example of a class of applications that are very likely to require aggregation of widely distributed computing, data, and intellectual resources. Such simulations - e.g. whole system aircraft simulation and whole system living cell simulation - require integrating applications and data that are developed by different teams of researchers frequently in different locations. The research team's are the only ones that have the expertise to maintain and improve the simulation code and/or the body of experimental data that drives the simulations. This results in an inherently distributed computing and data management environment.

Johnston, William E.↗
NASA NTRS1998

Method and system for data clustering for very large databases

Multi-dimensional data contained in very large databases is efficiently and accurately clustered to determine patterns therein and extract useful information from such patterns. Conventional computer processors may be used which have limited memory capacity and conventional operating speed, allowing massive data sets to be processed in a reasonable time and with reasonable computer resources. The clustering process is organized using a clustering feature tree structure wherein each clustering feature comprises the number of data points in the cluster, the linear sum of the data points in the cluster, and the square sum of the data points in the cluster. A dense region of data points is treated collectively as a single cluster, and points in sparsely occupied regions can be treated as outliers and removed from the clustering feature tree. The clustering can be carried out continuously with new data points being received and processed, and with the clustering feature tree being restructured as necessary to accommodate the information from the newly received data points.

Zhang, Tian↗
NASA NTRS2016

Development of the Model of Galactic Interstellar Emission for Standard Point-Source Analysis of Fermi Large Area Telescope Data

Most of the celestial gamma rays detected by the Large Area Telescope (LAT) on board the Fermi Gamma-ray Space Telescope originate from the interstellar medium when energetic cosmic rays interact with interstellar nucleons and photons. Conventional point-source and extended-source studies rely on the modeling of this diffuse emission for accurate characterization. Here, we describe the development of the Galactic Interstellar Emission Model (GIEM),which is the standard adopted by the LAT Collaboration and is publicly available. This model is based on a linear combination of maps for interstellar gas column density in Galactocentric annuli and for the inverse-Compton emission produced in the Galaxy. In the GIEM, we also include large-scale structures like Loop I and the Fermi bubbles. The measured gas emissivity spectra confirm that the cosmic-ray proton density decreases with Galactocentric distance beyond 5 kpc from the Galactic Center. The measurements also suggest a softening of the proton spectrum with Galactocentric distance. We observe that the Fermi bubbles have boundaries with a shape similar to a catenary at latitudes below 20deg and we observe an enhanced emission toward their base extending in the north and south Galactic directions and located within approximately 4deg of the Galactic Center.

gamma rays: diffuse background – gamma rays: gen↗
NASA NTRS2005

A Testbed Demonstration of an Intelligent Archive in a Knowledge Building System

The last decade's influx of raw data and derived geophysical parameters from several Earth observing satellites to NASA data centers has created a data-rich environment for Earth science research and applications. While advances in hardware and information management have made it possible to archive petabytes of data and distribute terabytes of data daily to a broad community of users, further progress is necessary in the transformation of data into information, and information into knowledge that can be used in particular applications in order to realize the full potential of these valuable datasets. In examining what is needed to enable this progress in the data provider environment that exists today and is expected to evolve in the next several years, we arrived at the concept of an Intelligent Archive in context of a Knowledge Building System (IA/KBS). Our prior work and associated papers investigated usage scenarios, required capabilities, system architecture, data volume issues, and supporting technologies. We identified six key capabilities of an IA/KBS: Virtual Product Generation, Significant Event Detection, Automated Data Quality Assessment, Large-Scale Data Mining, Dynamic Feedback Loop, and Data Discovery and Efficient Requesting. Among these capabilities, large-scale data mining is perceived by many in the community to be an area of technical risk. One of the main reasons for this is that standard data mining research and algorithms operate on datasets that are several orders of magnitude smaller than the actual sizes of datasets maintained by realistic earth science data archives. Therefore, we defined a test-bed activity to implement a large-scale data mining algorithm in a pseudo-operational scale environment and to examine any issues involved. The application chosen for applying the data mining algorithm is wildfire prediction over the continental U.S. This paper reports a number of observations based on our experience with this test-bed. While proof-of-concept for data mining scalability and utility has been a major goal for the research reported here, it was not the only one. The other five capabilities of an WKBS named above have been considered as well, and an assessment of the implications of our experience for these other areas will also be presented. The lessons learned through the testbed effort and presented in this paper will benefit technologists, scientists, and system operators as they consider introducing IA/KBS capabilities into production systems.

Ramapriyan, Hampapuram↗
NASA NTRS1991

Ozone climatology at Natal, Brazil, from in situ ozonesonde data

A large ozone-profile data set has been obtained through balloon ozonesonde soundings at Natal, Brazil, during 1978-1988. Maximum ozone concentrations occur during local spring (September-October), and minimum concentrations during late autumn (April-May); the seasonal variation is much larger in the troposphere than in the stratosphere. If there were no seasonal variation in the stratosphere, the seasonal variation observed in the troposphere alone would be sufficient to drive a total ozone column variation of about 5 percent. This is about half the size of the variation observed in the Natal Dobson spectrophotometer data.

Kirchhoff, V. W. J. H.↗
NASA NTRS1991

Three-dimensional displays for natural hazards analysis, using classified Landsat Thematic Mapper digital data and large-scale digital elevation models

Methods are described for using Landsat Thematic Mapper digital data and digital elevation models for the display of natural hazard sites in a mountainous region of northwestern Montana, USA. Hazard zones can be easily identified on the three-dimensional images. Proximity of facilities such as highways and building locations to hazard sites can also be easily displayed. A temporal sequence of Landsat TM (or similar) satellite data sets could also be used to display landscape changes associated with dynamic natural hazard processes.

Butler, David R.↗
NASA NTRS2012

Big Data Challenges for Large Radio Arrays

Future large radio astronomy arrays, particularly the Square Kilometre Array (SKA), will be able to generate data at rates far higher than can be analyzed or stored affordably with current practices. This is, by definition, a "big data" problem, and requires an end-to-end solution if future radio arrays are to reach their full scientific potential. Similar data processing, transport, storage, and management challenges face next-generation facilities in many other fields.

Combining↗
NASA NTRS1995

Fast Spatio-Temporal Data Mining from Large Geophysical Datasets

Use of the UCLA CONQUEST (CONtent-based Querying in Space and Time) is reviewed for performance of automatic cyclone extraction and detection of spatio-temporal blocking conditions on MPP. CONQUEST is a data analysis environment for knowledge and data mining to aid in high-resolution modeling of climate modeling.

knowledge discovery data mining climate modeling c↗