Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS1974

An analysis of pilot error-related aircraft accidents

A multidisciplinary team approach to pilot error-related U.S. air carrier jet aircraft accident investigation records successfully reclaimed hidden human error information not shown in statistical studies. New analytic techniques were developed and applied to the data to discover and identify multiple elements of commonality and shared characteristics within this group of accidents. Three techniques of analysis were used: Critical element analysis, which demonstrated the importance of a subjective qualitative approach to raw accident data and surfaced information heretofore unavailable. Cluster analysis, which was an exploratory research tool that will lead to increased understanding and improved organization of facts, the discovery of new meaning in large data sets, and the generation of explanatory hypotheses. Pattern recognition, by which accidents can be categorized by pattern conformity after critical element identification by cluster analysis.

Kowalsky, N. B.↗
NASA NTRS1979

Recombination of NO/+/ in the mid-latitude trough and the polar ionization hole

The dissociative recombination of NO(+) with electrons is further examined using a sample of Atmosphere Explorer data from the midlatitude ionization trough. The electron temperature for this data sample covered a range of 2500 K, thus allowing an accurate determination of the temperature dependence of the rate coefficient. The results confirm earlier determinations which were made from a large data base, and the rate coefficient alpha is found to be inversely proportional to the electron temperature. The possibility of a neglected process in the NO(+) chemistry is investigated which, if it were included, might result in a smaller temperature dependence of alpha. Such a source would require a direct dependence on electron temperature. A possible candidate is associative recombination of energetic N(D-2) with O. The results confirm recent measurements which find that this process cannot be significant. The NO(+) chemistry is a very sensitive indicator of chemical equilibrium conditions. By using measurements of the relevant parameters made in the polar ionization hole it is found that horizontal drifts do not appear to be significant in this region.

Torr, M. R.↗
NASA NTRS1987

Contextual classification on the massively parallel processor

Classifiers are often used to produce land cover maps from multispectral Earth observation imagery. Conventionally, these classifiers have been designed to exploit the spectral information contained in the imagery. Very few classifiers exploit the spatial information content of the imagery, and the few that do rarely exploit spatial information content in conjunction with spectral and/or temporal information. A contextual classifier that exploits spatial and spectral information in combination through a general statistical approach was studied. Early test results obtained from an implementation of the classifier on a VAX-11/780 minicomputer were encouraging, but they are of limited meaning because they were produced from small data sets. An implementation of the contextual classifier is presented on the Massively Parallel Processor (MPP) at Goddard that for the first time makes feasible the testing of the classifier on large data sets.

Tilton, James C.↗
NASA NTRS2002

Telescope Scientist on the Advanced X-Ray Astrophysics Observatory

This period included many scientific observations made with the Chandra Observatory. The results, as is well known, are spectacular. Fortunately, the HRMA performance continues to be essentially identical to that predicted from ground calibration data. The Telescope Scientist Team has improved the mirror model to provide a more accurate description to the Chandra observers and enable them to reduce the systematic errors and uncertainties in their data reduction. There also has been progress in the scientific program. At this time 47 distant clusters of galaxies have been observed. We are performing a systematic analysis of this rather large data set for the purpose of determining absolute distances utilizing the Sunyaev Zel'dovich effect.

VanSpeybroeck, L.↗
NASA NTRS2009

Data Mining at NASA: From Theory to Applications

This slide presentation demonstrates the data mining/machine learning capabilities of NASA Ames and Intelligent Data Understanding (IDU) group. This will encompass the work done recently in the group by various group members. The IDU group develops novel algorithms to detect, classify, and predict events in large data streams for scientific and engineering systems. This presentation for Knowledge Discovery and Data Mining 2009 is to demonstrate the data mining/machine learning capabilities of NASA Ames and IDU group. This will encompass the work done re cently in the group by various group members.

Srivastava, Ashok N.↗
NASA NTRS1985

Extended testing of a general contextual classifier using the massively parallel processor - Preliminary results and test plans

Earlier encouraging test results of a contextual classifier that combines spatial and spectral information employing a general statistical approach are expanded. The earlier results were of limited meaning because they were produced from small (50-by-50 pixel) data sets. An implementation of the contextual classifier on NASA Goddard's Massively Parallel Processor (MPP) is presented; for the first time the MPP makes feasible the testing of the classifier on large data sets (a 12-hour test on a VAX-11/780 minicomputer now takes 5 minutes on the MPP). The MPP is a Single-Instruction, Multiple Data Stream computer, consisting of 16,384 bit serial microprocessors connected in a 128-by-128 mesh array with each element having data transfer connections with its four nearest neighbors so that the MPP is capable of billions of operations per second. Preliminary results are given (with more expected for the conference) and plans are mentioned for extended testing of the contextual classifier on Thematic Mapper data sets.

Tilton, J. C.↗
NASA NTRS2019

Ask-The-Expert: Minimizing Human Review for Big Data Analytics Through Active Learning

In this CIF project, we worked toward semi-automating knowledge discovery from anomaly detection algorithms through the use of active learning. Active learning is an area of research within machine learning that uses an "expert in the loop" to learn from large data sets that have very few annotations or labels available, and where providing such labels is expensive. In our case, the task can be defined as the identification of safety events from flight operational data. Since traditional anomaly detection algorithms cannot differentiate between operationally relevant and irrelevant statistical anomalies, Subject Matter Experts (SMEs) have a lengthy and expensive burden of investigating every example identified by the detection algorithm, classifying and labeling them as relevant or irrelevant. Active learningidentifies the unlabeled example for which a label would most improve the classifier, asks the domain expert for a label, and repeats this process until there are no more resources (time, budget) available for labeling or a minimum required performance is reached. A positive label indicates an operationally significant safety event whereas a negative label indicates otherwise. Based on these few labels we propose to build an active learning system that utilizes the SME's time in the most effective manner by iteratively asking for labels for as few informative instances as possible. Our work was proposed to be a stepping stone toward implementation and deployment of the system with user interface to be pursued by the Aviation Operations and Safety Program (AOSP) given its interest in safety monitoring and discovery of safety incidents.

aviation safety↗
NASA NTRS2018

Examining Dense Data Usage near the Regions with Severe Storms in All-Sky Microwave Radiance Data Assimilation and Impacts on GEOS Hurricane Analyses

Many numerical weather prediction (NWP) centers assimilate radiances affected by clouds and precipitation from microwave sensors, with the expectation that these data can provide critical constraints on meteorological parameters in dynamically sensitive regions to make significant impacts on forecast accuracy for precipitation. The Global Modeling and Assimilation Office (GMAO) at NASA Goddard Space Flight Center assimilates all-sky microwave radiance data from various microwave sensors such as all-sky GPM Microwave Imager (GMI) radiance in the Goddard Earth Observing System (GEOS) atmospheric data assimilation system (ADAS), which includes the GEOS atmospheric model, the Gridpoint Statistical Interpolation (GSI) atmospheric analysis system, and the Goddard Aerosol Assimilation System (GAAS). So far, most of NWP centers apply same large data thinning distances, that are used in clear-sky radiance data to avoid correlated observation errors, to all-sky microwave radiance data. For example, NASA GMAO is applying 145 km thinning distances for most of satellite radiance data including microwave radiance data in which all-sky approach is implemented. Even with these coarse observation data usage in all-sky assimilation approach, noticeable positive impacts from all-sky microwave data on hurricane track forecasts were identified in GEOS-5 system. The motivation of this study is based on the dynamic thinning distance method developed in our all-sky framework to use of denser data in cloudy and precipitating regions due to relatively small spatial correlations of observation errors. To investigate the benefits of all-sky microwave radiance on hurricane forecasts, several hurricane cases selected between 2016-2017 are examined. The dynamic thinning distance method is utilized in our all-sky approach to understand the sources and mechanisms to explain the benefits of all-sky microwave radiance data from various microwave radiance sensors like Advanced Microwave Sounder Unit (AMSU-A), Microwave Humidity Sounder (MHS), and GMI on GEOS-5 analyses and forecasts of various hurricanes.

Kim, Min-Jeong↗
NASA NTRS2019

Accelerated Simulation of Air Pollution Using NVIDIA RAPIDS

Atmospheric chemistry models are a central tool to study and forecast the impact of air pollution on the environment, vegetation, and human health. However, the numerical simulation of chemical kinetics is computationally expensive due to the stiffness of the system of ordinary differential equations that describes atmospheric chemistry. Here we present an alternative approach to the computation of atmospheric chemistry based on machine learning. Our training data set is produced using the NASA Goddard Earth Observing System (GEOS) model with GEOS-Chem chemistry, run on the NASA Center for Climate Simulation (NCCS) Discover supercomputing cluster on 384 Intel Xeon Haswell cores. This model spends more than 50% of total run time on solving atmospheric chemistry. The data set contains as input features the air pollution concentrations before solving the differential equations, together with some key physical parameters such as temperature and sun intensity. As target variables we define the air pollution concentrations after solving the differential equations. Using Dask-cuDF and Dask-XGBoost on the NVIDIA RAPIDS platform on 8 Tesla V100 GPUs, we generate from this training set gradient boosted decision tree models that can reproduce the simulation of chemical kinetics. We do this on the NCCS Advanced Data Analytics Platform (ADAPT) science cloud environment. Our application takes full advantage of recent advances in Dask-XGBoost, such as multi-node and multi-GPU scaling for distributed training with large data sets. The increase in training data size enabled by this is critical to capture the full range of chemical environments encountered across the globe and all annual seasons.The boosted tree models offer good predictability and show many of the features of the full chemistry reference simulation. Further improvements can be achieved through mass balance considerations and by accounting for error correlations. We incorporate the boosted tree models into the GEOS reference model using XGBoost's C API. This enables a seamless integration of the GPU trained models into GEOS-Chem, which is written in Fortran and optimized for use in a massively parallel CPU environment. We show the benefits of this approach and discuss the potential speedup of this machine learning accelerated atmospheric chemistry model.

Keller, Christoph A.↗
NASA NTRS2012

The Cabauw Intercomparison Campaign for Nitrogen Dioxide Measuring Instruments (CINDI): Design, Execution, and Early Results

From June to July 2009 more than thirty different in-situ and remote sensing instruments from all over the world participated in the Cabauw Intercomparison campaign for Nitrogen Dioxide measuring Instruments (CINDI). The campaign took place at KNMI's Cabauw Experimental Site for Atmospheric Research (CESAR) in the Netherlands. Its main objectives were to determine the accuracy of state-ofthe- art ground-based measurement techniques for the detection of atmospheric nitrogen dioxide (both in-situ and remote sensing), and to investigate their usability in satellite data validation. The expected outcomes are recommendations regarding the operation and calibration of such instruments, retrieval settings, and observation strategies for the use in ground-based networks for air quality monitoring and satellite data validation. Twenty-four optical spectrometers participated in the campaign, of which twenty-one had the capability to scan different elevation angles consecutively, the so-called Multi-axis DOAS systems, thereby collecting vertical profile information, in particular for nitrogen dioxide and aerosol. Various in-situ samplers and lidar instruments simultaneously characterized the variability of atmospheric trace gases and the physical properties of aerosol particles. A large data set of continuous measurements of these atmospheric constituents has been collected under various meteorological conditions and air pollution levels. Together with the permanent measurement capability at the CESAR site characterizing the meteorological state of the atmosphere, the CINDI campaign provided a comprehensive observational data set of atmospheric constituents in a highly polluted region of the world during summertime. First detailed comparisons performed with the CINDI data show that slant column measurements of NO2, O4 and HCHO with MAX-DOAS agree within 5 to 15%, vertical profiles of NO2 derived from several independent instruments agree within 25% of one another, and MAX-DOAS aerosol optical thickness agrees within 20-30% with AERONET data. For the in-situ NO2 instrument using a molybdenum converter, a bias was found as large as 5 ppbv during day time, when compared to the other in-situ instruments using photolytic converters.

Atmospheric composition↗
NASA NTRS1989

Satellite/rocket ozone comparisons at Natal, Brazil

Comparisons are presented of satellite, rocket, and balloon ozone profiles near Natal, Brazil (5.9 deg S, 35.2 deg W). The low variability of stratospheric ozone at Natal during March and April of 1985 has allowed intercomparisons of reasonably large data sets, rather than a small number of paired satellite/in situ comparisons. There are sharp differences between the profile from the SBUV instrument on Nimbus 7 and the in situ measurements. These results support the conclusions of the NASA Ozone Trends Panel that there is an instrumental cause for the very large changes in upper stratospheric ozone seen by SBUV. Along with other comparisons, these results are being used in a reassessment of the SBUV instrument and its data reduction procedures. The agreement between the ozone profiles from the SAGE II instrument on the ERBS satellite and the rocket values is excellent over the full range of comparisons. Both SAGE II and ROCOZ-A must convert from altitude to pressure for intercomparisons with SME and with SBUV-type instruments. The conversion between pressure and altitude is as important as the ozone measurements, especially in the upper stratosphere where the scale height for ozone is approximately half that for pressure.

Barnes, Robert A.↗
NASA NTRS2004

GPS FOM Chimney Analysis using Generalized Extreme Value Distribution

Many a time an objective of a statistical analysis is to estimate a limit value like 3-sigma 95% confidence upper limit from a data sample. The generalized Extreme Value Distribution method can be profitably employed in many situations for such an estimate. . .. It is well known that according to the Central Limit theorem the mean value of a large data set is normally distributed irrespective of the distribution of the data from which the mean value is derived. In a somewhat similar fashion it is observed that many times the extreme value of a data set has a distribution that can be formulated with a Generalized Distribution. In space shuttle entry with 3-string GPS navigation the Figure Of Merit (FOM) value gives a measure of GPS navigated state accuracy. A GPS navigated state with FOM of 6 or higher is deemed unacceptable and is said to form a FOM 6 or higher chimney. A FOM chimney is a period of time during which the FOM value stays higher than 5. A longer period of FOM of value 6 or higher causes navigated state to accumulate more error for a lack of state update. For an acceptable landing it is imperative that the state error remains low and hence at low altitude during entry GPS data of FOM greater than 5 must not last more than 138 seconds. I To test the GPS performAnce many entry test cases were simulated at the Avionics Development Laboratory. Only high value FoM chimneys are consequential. The extreme value statistical technique is applied to analyze high value FOM chimneys. The Maximum likelihood method is used to determine parameters that characterize the GEV distribution, and then the limit value statistics are estimated.

Ott, Rick↗
NASA NTRS1991

Physical processes in comets

Post-Halley comets are known to be irregular objects with most nucleus activity very localized and with the dust coma capable of fragmentation and apparently being a source of gas. Older, 1-D strategies which assume steady isotropic outflow of material can give poor time-and-space averaged results, at best. With 2-D data, images through interference filters, it is hoped that: dust structures can be seen that give evidence of the proper geometry for data reduction; gradients along the axes of symmetry and evidence of fragmentation can be studied; and that evidence for gas abundance gradients associated with the dust can be found. High quality data from brighter comets can then be used to suggest improved data reduction procedures for fainter ones. To obtain such data, large image-quality interference filters were procured for use with a charge coupled device (CCD) camera at Lick Observatory, where the scale of the 1 m Nickel reflector is ideal for brighter comets. Whenever possible, data is taken simultaneously with other telescopes and equipment, especially spectroscopy at the Lick 3 m or infrared photometry at the Infrared Telescope Facility (IRTF) on Mauna Kea.

Newburn, Ray L., Jr.↗
NASA NTRS1985

Distribution of atmospheric aerosols and CO2 lidar backscatter simulation

Development of a Global Wind Measurement Satellite System (WINDSAT) (NOAA, 1981) requires a knowledge of the global characteristics of the free tropospheric and lower stratospheric aerosol. In particular, there is a need to document the behavior of the aerosol backscattering function, at CO2 laser wavelengths, beta sub CO2, as a function of space and time. There is, however, a relative lack of data for the free troposphere, particularly over the remoter regions of the globe, as compared with that for the boundary layer and the stratosphere. Moreover, because of variations in concentration that occur as a function of space and time, large data sets are required to obtain meaningful averages. A recent study by Kent et al. (1985) uses three distinct tropospheric aerosol data sets in order to obtain an improved global model of the general aerosol characteristics, including variation of beta sub CO2 with latitude, season, and altitude. The more important findings are summarized.

Kent, G. S.↗
NASA NTRS2005

University of Washington Participation in SAFARI-2000

This report presents a summary, of the participation in the 6-week field study in southern Africa. During the field study there were flown 119.02 research hours (31 research flights). In these flights the researchers obtained many unique data sets for evaluating the effects of biomass burning and other sources of particles and gases on the climate of southern Africa, and obtained simultaneous measurements with NASA ER-2 and Terra overflights. They also analyzed portions of the large data sets acquired in SAFARI-2000. They attended several SAFARI-2000 workshops, national and international conferences to present SAFARI-2000 results.

Hobbs, Peter V.↗
NASA NTRS2019

NASA Global Satellite and Model Data Products and Services for Tropical Cyclone Research

The lack of observations over vast tropical oceans is a major challenge for tropical cyclone research. Satellite observations and model reanalysis data play an important role in filling these- gaps. Established in the mid-1980's, the Goddard Earth Sciences Data and Information Services Center (GES DISC), as one of the 12 NASA data centers, archives and distributes data from several Earth science disciplines such as precipitation, atmospheric dynamics, atmospheric composition, hydrology, including well-known NASA satellite missions (e.g. TRMM, GPM) and model assimilation projects (MERRA-2). Acquiring datasets suitable for tropical cyclone research in a large data archive is a challenge for many, especially for those who are not familiar with satellite or model data. Over the years, the GES DISC has developed user-friendly data services. For example, Giovanni is an online visualization and analysis tool, allowing users to visualize and analyze over 2000 satellite- and model-based variables with a Web browser, without downloading data and software. In this chapter, we will describe data and services at the GES DISC with emphasis on tropical cyclone research. We will also present two case studies and discuss future plans.

Liu, Zhong↗
NASA NTRS1995

Interactive access and management for four-dimensional environmental data sets using McIDAS

This grant has fundamentally changed the way that meteorologists look at the output of their atmospheric models, through the development and wide distribution of the Vis5D system. The Vis5D system is also gaining acceptance among oceanographers and atmospheric chemists. Vis5D gives these scientists an interactive three-dimensional movie of their very large data sets that they can use to understand physical mechanisms and to trace problems to their sources. This grant has also helped to define the future direction of scientific visualization through the development of the VisAD system and its lattice data model. The VisAD system can be used to interactively steer and visualize scientific computations. A key element of this capability is the flexibility of the system's data model to adapt to a wide variety of scientific data, including the integration of several forms of scientific metadata.

Hibbard, William L.↗
NASA NTRS2002

Seasonal to Decadal-Scale Variability in Satellite Ocean Color and Sea Surface Temperature for the California Current System

Support for this project was used to develop satellite ocean color and temperature indices (SOCTI) for the California Current System (CCS) using the historic record of CZCS West Coast Time Series (WCTS), OCTS, WiFS and AVHRR SST. The ocean color satellite data have been evaluated in relation to CalCOFI data sets for chlorophyll (CZCS) and ocean spectral reflectance and chlorophyll OCTS and SeaWiFS. New algorithms for the three missions have been implemented based on in-water algorithm data sets, or in the case of CZCS, by comparing retrieved pigments with ship-based observations. New algorithms for absorption coefficients, diffuse attenuation coefficients and primary production have also been evaluated. Satellite retrievals are being evaluated based on our large data set of pigments and optics from CalCOFI.

Mitchell, B. Greg↗