Search NASA⌕ Search

SEARCH · Search NASA

Results for “Large data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

A Fast Implementation of the ISODATA Clustering Algorithm

Clustering is central to many image processing and remote sensing applications. ISODATA is one of the most popular and widely used clustering methods in geoscience applications, but it can run slowly, particularly with large data sets. We present a more efficient approach to ISODATA clustering, which achieves better running times by storing the points in a kd-tree and through a modification of the way in which the algorithm estimates the dispersion of each cluster. We also present an approximate version of the algorithm which allows the user to further improve the running time, at the expense of lower fidelity in computing the nearest cluster center to each point. We provide both theoretical and empirical justification that our modified approach produces clusterings that are very similar to those produced by the standard ISODATA approach. We also provide empirical studies on both synthetic data and remotely sensed Landsat and MODIS images that show that our approach has significantly lower running times.

Memarsadeghi, Nargess↗
NASA NTRS2008

James Webb Space Telescope - Applying Lessons Learned to I&T

The James Webb Space Telescope (JWST) is part of a new generation of spacecraft acquiring large data volumes from remote regions in space. To support a mission such as the JWST, it is imperative that lessons learned from the development of previous missions such as the Hubble Space Telescope and the Earth Observing System mission set be applied throughout the development and operational lifecycles. One example of a key lesson that should be applied is that core components, such as the command and telemetry system and the project database, should be developed early, used throughout development and testing, and evolved into the operational system. The purpose of applying lessons learned is to reap benefits in programmatic or technical parameters such as risk reduction, end product quality, cost efficiency, and schedule optimization. In the cited example, the early development and use of the operational command and telemetry system as well as the establishment of the intended operational database will allow these components to be used by the developers of various spacecraft components such that development, testing, and operations will all use the same core components. This will reduce risk through the elimination of transitions between development and operational components and improve end product quality by extending the verification of those components through continual use. This paper will discuss key lessons learned that have been or are being applied to the JWST Ground Segment integration and test program.

Johns, Alan↗
NASA NTRS2008

The Wide-Field Imaging Interferometry Testbed (WIIT): Recent Progress and Results

Continued research with the Wide-Field Imaging Interferometry Testbed (WIIT) has achieved several important milestones. We have moved WIIT into the Advanced Interferometry and Metrology (AIM) Laboratory at Goddard, and have characterized the testbed in this well-controlled environment. The system is now completely automated and we are in the process of acquiring large data sets for analysis. In this paper, we discuss these new developments and outline our future research directions. The WIIT testbed, combined with new data analysis techniques and algorithms, provides a demonstration of the technique of wide-field interferometric imaging, a powerful tool for future space-borne interferometers.

Rinehart, Stephen A.↗
NASA NTRS2007

A Fast Implementation of the Isodata Clustering Algorithm

Clustering is central to many image processing and remote sensing applications. ISODATA is one of the most popular and widely used clustering methods in geoscience applications, but it can run slowly, particularly with large data sets. We present a more efficient approach to IsoDATA clustering, which achieves better running times by storing the points in a kd-tree and through a modification of the way in which the algorithm estimates the dispersion of each cluster. We also present an approximate version of the algorithm which allows the user to further improve the running time, at the expense of lower fidelity in computing the nearest cluster center to each point. We provide both theoretical and empirical justification that our modified approach produces clusterings that are very similar to those produced by the standard ISODATA approach. We also provide empirical studies on both synthetic data and remotely sensed Landsat and MODIS images that show that our approach has significantly lower running times.

Memarsadeghi, Nargess↗
NASA NTRS2010

A Case Study: Using Delmia at Kennedy Space Center to Support NASA's Constellation Program

The presentation examines the use of Delmia (Digital Enterprise Lean Manufacturing Interactive Application) for digital simulation in NASA's Constellation Program. Topics include an overview of the Kennedy Space Center (KSC) Design Visualization Group tasks, NASA's Constellation Program, Ares 1 ground processing preliminary design review, and challenges and how Delmia is used at KSC, Challenges include dealing with large data sets, creating and maintaining KSC's infrastructure, gathering customer requirements and meeting objectives, creating life-like simulations, and providing quick turn-around on varied products,

Kickbusch, Tracey↗
NASA NTRS2011

Modeling Powered Aerodynamics for the Orion Launch Abort Vehicle Aerodynamic Database

Modeling the aerodynamics of the Orion Launch Abort Vehicle (LAV) has presented many technical challenges to the developers of the Orion aerodynamic database. During a launch abort event, the aerodynamic environment around the LAV is very complex as multiple solid rocket plumes interact with each other and the vehicle. It is further complicated by vehicle separation events such as between the LAV and the launch vehicle stack or between the launch abort tower and the crew module. The aerodynamic database for the LAV was developed mainly from wind tunnel tests involving powered jet simulations of the rocket exhaust plumes, supported by computational fluid dynamic simulations. However, limitations in both methods have made it difficult to properly capture the aerodynamics of the LAV in experimental and numerical simulations. These limitations have also influenced decisions regarding the modeling and structure of the aerodynamic database for the LAV and led to compromises and creative solutions. Two database modeling approaches are presented in this paper (incremental aerodynamics and total aerodynamics), with examples showing strengths and weaknesses of each approach. In addition, the unique problems presented to the database developers by the large data space required for modeling a launch abort event illustrate the complexities of working with multi-dimensional data.

Chan, David T.↗
NASA NTRS2012

Automated and Manual Rocket Crater Measurement Software

An update has been performed to software designed to do very rapid automated measurements of craters created in sandy substrates by rocket exhaust on liftoff. The previous software was optimized for pristine lab geometry and lighting conditions. This software has been enhanced to include a section for manual measurements of crater parameters; namely, crater depth, crater full width at half max, and estimated crater volume. The tools provide a very rapid method to measure these manual parameters to ease the burden of analyzing large data sets. This software allows for rapid quantization of the rocket crater parameters where automated methods may not work. The progress of spreadsheet data is continuously saved so that data is never lost, and data can be copied to clipboards and pasted to other software for analysis. The volume estimation of a crater is based on the central max depth axis line, and the polygonal shape of the crater is integrated around that axis.

Metzger, Philip↗
NASA NTRS2011

Provenance Challenges for Earth Science Dataset Publication

Modern science is increasingly dependent on computational analysis of very large data sets. Organizing, referencing, publishing those data has become a complex problem. Published research that depends on such data often fails to cite the data in sufficient detail to allow an independent scientist to reproduce the original experiments and analyses. This paper explores some of the challenges related to data identification, equivalence and reproducibility in the domain of data intensive scientific processing. It will use the example of Earth Science satellite data, but the challenges also apply to other domains.

Tilmes, Curt↗
NASA NTRS2012

Space Shuttle Orbiter Digital Outer Mold Line Scanning

The Space Shuttle Orbiters Discovery and Endeavor have been digitally scanned to produce post-flight configuration outer mold line surfaces. Very detailed scans of the windward side of these vehicles provide resolution of the detailed tile step and gap geometry, as well as the reinforced carbon carbon nose cap and leading edges. Lower resolution scans of the upper surface provide definition of the crew cabin windows, wing upper surfaces, payload bay doors, orbital maneuvering system pods and the vertical tail. The process for acquisition of these digital scans as well as post-processing of the very large data set will be described.

Campbell, Charles H.↗
NASA NTRS2013

Shape-induced Gravitational Sorting of Saharan Dust During Transatlantic Voyage: Evidence from CALIOP Lidar Depolarization Measurements

Motivated by the physical picture of shape-dependent air resistance and, consequently, shape-induced differential sedimentation of dust particles, we searched for and found evidence of dust particle asphericity affecting the evolution and distribution of dust-scattered light depolarization ratio (delta). Specifically, we examined a large data set of Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) observations of Saharan dust from June to August 2007. Observing along a typical transatlantic dust track, we find that (1) median delta is uniformly distributed between 2 and 5 km altitudes as the elevated dust leaves the west coast of Africa, thereby indicating uniformly random mixing of particle shapes with height; (2) vertical homogeneity of median delta breaks down during the westward transport: between 2 and 5 km delta increases with altitude and this increase becomes more pronounced with westward progress; (3) delta tends to increase at higher altitude (greater than 4 km) and decrease at lower altitude (less than 4 km) during the westward transport. All these features are captured qualitatively by a minimal model (two shapes only), suggesting that shape-induced differential settling and consequent sorting indeed contribute significantly to the observed temporal evolution and vertical stratification of dust properties. By implicating particle shape as a likely cause of gravitational sorting, these results will affect the estimates of radiative transfer through Saharan dust layers.

CALIPSO↗
NASA NTRS2015

Kinematic and Microphysical Control of Lightning Flash Rate over Northern Alabama

The Deep Convective Clouds and Chemistry (DC3) experiment seeks to examine the relationship between deep convection and the production of nitrogen oxides (NO (sub x)) via lightning (LNO (sub x)). A critical step in estimating LNO (sub x) production in a cloud-resolving model (CRM) without explicit lightning is to estimate the flash rate from available model parameters that are statistically and physically correlated. As such, the objective of this study is to develop, improve and evaluate lightning flash rate parameterizations in a variety of meteorological environments and storm types using radar and lightning mapping array (LMA) observations taken over Northern Alabama from 2005-2012, including during DC3. UAH's Advanced Radar for Meteorological and Operational Research (ARMOR) and the Weather Surveillance Radar - 1988 Doppler (WSR 88D) located at Hytop (KHTX) comprises the dual-Doppler and polarimetric radar network, which has been in operation since 2004. The northern Alabama LMA (NA LMA) in conjunction with Vaisala's National Lightning Detection Network (NLDN) allow for a detailed depiction of total lightning during this period. This study will integrate ARMOR-KHTX dual Doppler/polarimetric radar and NA LMA lightning observations from past and ongoing studies, including the more recent DC3 results, over northern Alabama to form a large data set of 15-20 case days and over 20 individual storms, including both ordinary multicell and supercell convection. Several flash rate parameterizations will be developed and tested, including those based on 1) graupel/small hail volume; 2) graupel/small hail mass, and 3) convective updraft volume. Sensitivity of the flash rate parameterizations to storm intensity, storm morphology and environmental conditions will be explored.

Carey, Lawrence D.↗
NASA NTRS2012

Obtaining Accurate Change Detection Results from High-Resolution Satellite Sensors

Multi-date acquisitions of high-resolution imaging satellites (e.g. GeoEye and WorldView), can display local changes of current economic interest. However, their large data volume precludes effective manual analysis, requiring image co-registration followed by image-to-image change detection, preferably with minimal analyst attention. We have recently developed an automatic change detection procedure that minimizes false-positives. The processing steps include: (a) Conversion of both the pre- and post- images to reflectance values (this step is of critical importance when different sensors are involved); reflectance values can be either top-of-atmosphere units or have full aerosol optical depth calibration applied using bi-directional reflectance knowledge. (b) Panchromatic band image-to-image co-registration, using an orthorectified base reference image (e.g. Digital Orthophoto Quadrangle) and a digital elevation model; this step can be improved if a stereo-pair of images have been acquired on one of the image dates. (c) Pan-sharpening of the multispectral data to assure recognition of change objects at the highest resolution. (d) Characterization of multispectral data in the post-image ( i.e. the background) using unsupervised cluster analysis. (e) Band ratio selection in the post-image to separate surface materials of interest from the background. (f) Preparing a pre-to-post change image. (g) Identifying locations where change has occurred involving materials of interest.

multispectral↗
NASA NTRS1973

A Modular Atmospheric Propagation Program (MAPP)

A computer model of the atmosphere has been developed that will solve a wide variety of lidar and atmospheric propagation problems. The lidar system calculations involve the prediction of path loss and lidar signal return for a variety of atmospheric conditions and lidar system configurations. The program contains a very large data base of absorption and attenuation coefficients for all of the naturally occurring gases and several gaseous pollutants such as S02, N02, NO, N20, 03, HCL, etc. The spectral range of this data base extends from the ultra-violet (1200 angstroms) through the infra-red. Several aerosol models are included in the program to account for the effects of this important scattering mechanism.

Wright, M. L.↗
NASA NTRS2017

Ask-the-expert: Active Learning Based Knowledge Discovery Using the Expert

Often the manual review of large data sets, either for purposes of labeling unlabeled instances or for classifying meaningful results from uninteresting (but statistically significant) ones is extremely resource intensive, especially in terms of subject matter expert (SME) time. Use of active learning has been shown to diminish this review time significantly. However, since active learning is an iterative process of learning a classifier based on a small number of SME-provided labels at each iteration, the lack of an enabling tool can hinder the process of adoption of these technologies in real-life, in spite of their labor-saving potential. In this demo we present ASK-the-Expert, an interactive tool that allows SMEs to review instances from a data set and provide labels within a single framework. ASK-the-Expert is powered by an active learning algorithm for training a classifier in the backend. We demonstrate this system in the context of an aviation safety application, but the tool can be adopted to work as a simple review and labeling tool as well, without the use of active learning.

software↗
NASA NTRS2017

Position Paper - pFLogger: The Parallel Fortran Logging framework for HPC Applications

In the context of high performance computing (HPC), software investments in support of text-based diagnostics, which monitor a running application, are typically limited compared to those for other types of IO. Examples of such diagnostics include reiteration of configuration parameters, progress indicators, simple metrics (e.g., mass conservation, convergence of solvers, etc.), and timers. To some degree, this difference in priority is justifiable as other forms of output are the primary products of a scientific model and, due to their large data volume, much more likely to be a significant performance concern. In contrast, text-based diagnostic content is generally not shared beyond the individual or group running an application and is most often used to troubleshoot when something goes wrong. We suggest that a more systematic approach enabled by a logging facility (or logger) similar to those routinely used by many communities would provide significant value to complex scientific applications. In the context of high-performance computing, an appropriate logger would provide specialized support for distributed and shared-memory parallelism and have low performance overhead. In this paper, we present our prototype implementation of pFlogger a parallel Fortran-based logging framework, and assess its suitability for use in a complex scientific application.

Fortran↗
NASA NTRS2017

POSITION PAPER - pFLogger: The Parallel Fortran Logging Framework for HPC Applications

In the context of high performance computing (HPC), software investments in support of text-based diagnostics, which monitor a running application, are typically limited compared to those for other types of IO. Examples of such diagnostics include reiteration of configuration parameters, progress indicators, simple metrics (e.g., mass conservation, convergence of solvers, etc.), and timers. To some degree, this difference in priority is justifiable as other forms of output are the primary products of a scientific model and, due to their large data volume, much more likely to be a significant performance concern. In contrast, text-based diagnostic content is generally not shared beyond the individual or group running an application and is most often used to troubleshoot when something goes wrong. We suggest that a more systematic approach enabled by a logging facility (or 'logger') similar to those routinely used by many communities would provide significant value to complex scientific applications. In the context of high-performance computing, an appropriate logger would provide specialized support for distributed and shared-memory parallelism and have low performance overhead. In this paper, we present our prototype implementation of pFlogger - a parallel Fortran-based logging framework, and assess its suitability for use in a complex scientific application.

Clune, Thomas L.↗
NASA NTRS2017

pFlogger: The Parallel Fortran Logging Utility

In the context of high performance computing (HPC), software investments in support of text-based diagnostics, which monitor a running application, are typically limited compared to those for other types of IO. Examples of such diagnostics include reiteration of configuration parameters, progress indicators, simple metrics (e.g., mass conservation, convergence of solvers, etc.), and timers. To some degree, this difference in priority is justifiable as other forms of output are the primary products of a scientific model and, due to their large data volume, much more likely to be a significant performance concern. In contrast, text-based diagnostic content is generally not shared beyond the individual or group running an application and is most often used to troubleshoot when something goes wrong. We suggest that a more systematic approach enabled by a logging facility (or 'logger)' similar to those routinely used by many communities would provide significant value to complex scientific applications. In the context of high-performance computing, an appropriate logger would provide specialized support for distributed and shared-memory parallelism and have low performance overhead. In this paper, we present our prototype implementation of pFlogger - a parallel Fortran-based logging framework, and assess its suitability for use in a complex scientific application.

Clune, Tom↗
NASA NTRS2017

Ask-the-Expert: Active Learning Based Knowledge Discovery Using the Expert

Often the manual review of large data sets, either for purposes of labeling unlabeled instances or for classifying meaningful results from uninteresting (but statistically significant) ones is extremely resource intensive, especially in terms of subject matter expert (SME) time. Use of active learning has been shown to diminish this review time significantly. However, since active learning is an iterative process of learning a classifier based on a small number of SME-provided labels at each iteration, the lack of an enabling tool can hinder the process of adoption of these technologies in real-life, in spite of their labor-saving potential. In this demo we present ASK-the-Expert, an interactive tool that allows SMEs to review instances from a data set and provide labels within a single framework. ASK-the-Expert is powered by an active learning algorithm for training a classifier in the back end. We demonstrate this system in the context of an aviation safety application, but the tool can be adopted to work as a simple review and labeling tool as well, without the use of active learning.

GUI↗