Search NASASearch

SEARCH · Search NASA

Results for “data cubes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Investigating Access Performance of Long Time Series with Restructured Big Model Data

Data sets generated by models are substantially increasing in volume, due to increases in spatial and temporal resolution, and the number of output variables. Many users wish to download subsetted data in preferred data formats and structures, as it is getting increasingly difficult to handle the original full-size data files. For example, application research users such as those involved with wind or solar energy, or extreme weather events are likely only interested in daily or hourly model data at a single point (or for a small area) for a long time period, and prefer to have the data downloaded in a single file. With native model file structures, such as hourly data from NASA Modern-Era Retrospective analysis for Research and Applications Version-2 (MERRA-2), it may take over 10 hours for the extraction of parameters-of-interest at a single point for 30 years. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is exploring methods to address this particular user need. One approach is to create value-added data by reconstructing the data files. Taking MERRA-2 data as an example, we have tested converting hourly data from one-day-per-file into different data cubes, such as one-month, or one-year. Performance is compared for reading local data files and accessing data through interoperable services, such as OPeNDAP. Results show that, compared to the original file structure, the new data cubes offer much better performance for accessing long time series. We have noticed that performance is associated with the cube size and structure, the compression method, and how the data are accessed. An optimized data cube structure will not only improve data access, but also may enable better online analysis services

reanalysis

Systems, computer-implemented methods, and tangible computer-readable storage media for wide-field interferometry

Disclosed herein are systems, computer-implemented methods, and tangible computer-readable storage media for wide field imaging interferometry. The method includes for each point in a two dimensional detector array over a field of view of an image: gathering a first interferogram from a first detector and a second interferogram from a second detector, modulating a path-length for a signal from an image associated with the first interferogram in the first detector, overlaying first data from the modulated first detector and second data from the second detector, and tracking the modulating at every point in a two dimensional detector array comprising the first detector and the second detector over a field of view for the image. The method then generates a wide-field data cube based on the overlaid first data and second data for each point. The method can generate an image from the wide-field data cube.

Lyon, Richard G.

PRSEUS Pressure Cube Test Data and Response

NASA s Environmentally Responsible Aviation (ERA) Program is examining the hybrid wing body (HWB) aircraft, among others, in an effort to increase the fuel efficiency of commercial aircraft. The HWB design combines features of a flying wing with features of conventional transport aircraft, and has the advantage of simultaneously increasing both fuel efficiency and payload. Recent years have seen an increased focus on the structural performance of the HWB. The key structural challenge of a HWB airframe is the ability to create a cost and weight efficient, non-circular, pressurized shell. Conventional round fuselage sections react cabin pressure by hoop tension. However, the structural configuration of the HWB subjects the majority of the structural panels to bi-axial, in-plane loads in addition to the internal cabin pressure, which requires more thorough examination and analysis than conventional transport aircraft components having traditional and less complex load paths. To address this issue, while keeping structural weights low, extensive use of advanced composite materials is made. This report presents the test data and preliminary conclusions for a pressurized cube test article that utilizes Boeing's Pultruded Rod Stitched Efficient Unitized Structure (PRSEUS), and which is part of the building block approach used for HWB development.

Lovejoy, Andrew E.

TPSAS-NF1676L-32235-DND

This presentation will provide a summary of the Open Data Cube initiative. In addition, a training manual will be provided to support Open Data Cube training for GEO and the Senegal government.

Brian Killough

TPSAS-NF1676L-32249-DND

This presentation will provide a summary of the Open Data Cube initiative. In addition, a training manual will be provided to support Open Data Cube training for the GEO Secretariat.

Brian Killough

TPSAS-NF1676L-32493-DND

The Committee on Earth Observation Satellites (CEOS) System Engineering Office (SEO) has supported the Open Data Cube (ODC) initiative to provide a data architecture solution that has value to its global users and increases the impact of EO satellite data. ODC is an open-source platform for processing satellite data. We have developed software products and tools around the core ODC that would help users perform machine learning on EO satellite data. The recent United Nations (UN) Sustainable Development Agenda provides a shared blueprint for peace and prosperity for people and for the planet, considering our current situation and helping to create a plan. The core of this agenda is a set of seventeen Sustainable Development Goals (SDGs), which represent an urgent call for action by all countries - both developed and developing - in a global partnership. The CEOS SEO team has recently developed and released a set of innovative Jupyter notebooks addressing UN SDGs 6.6.1 (spatial extents of water-related ecosystems), 11.3.1 (ratio of land consumption rate to population growth rate), and 15.3.1 (proportion of land that is degraded over total land area). These notebooks empower users by providing features that will assist with streamlining analysis ready data retrieval, processing, and visualization. We have recently incorporated several machine learning techniques in these notebooks. In this paper, we present the lessons learned from our experience on classifying land using supervised and unsupervised machine learning techniques using ODC framework for UN SDGs. We identify the current limitations of ODC to seamlessly support machine learning techniques. We propose features that would help machine learning, specifically within the ODC framework. We propose a thematic indexing/loading of data for both unsupervised learning as well as data annotation/labeling pipeline. Currently, ODC supports machine learning by separating data-management from the analysis process. It works as a mechanism to load cubes of data. ODC does not natively support features that are vital in machine learning such as validation splits, fair/balanced sampling, establishing load size constraints, etc. We believe that our proposed features will empower users by providing features that bring machine learning techniques closed to ODC. Enhancements to ODC to better accommodate machine learning techniques can assist in fulfilling UN SDGs such as 6.3.2, 6.4.2, 6.6.1, 11.3.1, 14.1.1, 15.1.1, 15.3.1, and 15.4.2.

Syed R Rizvi

An iterative method to deblend AGN-Host contributions for Integral Field spectroscopic observations

ABSTRACT We present a new iterative deblending method to separate the host galaxy (HG) and their Active Galactic Nuclei (AGNs) emission with the use of Integral Field spectroscopic (IFS) data. The method decomposes the resolved HG emission from the unresolved AGN emission by modelling the two-dimensional surface brightness (SB) profile of the point-spread function (PSF) and the two-dimensional SB HG continuum simultaneously per each monochromatic slide. Our method does not require any prior information about the observed SB profile or a detailed fitting of the PSF, making it ideal for the automatic analysis of large galaxy samples. In this work, we test the quality of our method, its advantages, and its disadvantages. We test our method by using a set of IFS mock data cubes to quantify the reliability of our deblending process and further compare our method with the qdblend3d analysis tool. Furthermore, we applied our method to three data cubes selected from the MaNGA survey according to the dominance of either its HG or its AGN. We show that our deblending method is capable of disengaging the bright, non-resolved AGN emission from the HG continuum and its narrow emission lines. However, the decoupling depends on how well the IFS spatially resolves the PSF, and on the relative flux intensity of the HG-AGN. Therefore, the method is ideal for disentangling the bright-flux contribution from AGN-dominated spectra.

Ibarra-Medel, H. (ORCID:0000000297906313)

The 3-Dimensional Inner and Outer Structure of Ejecta Around Eta Carinae as Detected by the STIS

The HST/STIS instrument was used successfully to perform a complete mapping of the Homunculus nebula at two wavelength ranges including H-alpha and H-beta with a spectral resolving power of about 5000 and a spatial resolution of 0.1". The individual spectra were merged to synthesize three-dimensional data cubes that contain a set of images of Eta Car with spatial resolution of 0.10 to 0.251, sliced at velocity increment of 10 -- 30 km/s. For the first time this unique method allows us to diagnose the origin of intrinsic narrow emission structure of the nebula with high spatial and velocity resolution. Our initial analysis revealed the inner emission structure appeared to trace an elongated bipolar shell (possibly other shells as well) with a scale size of an arcsecond (i.e., "little homunculus in the Homunculus"). Furthermore, the mapping data cube revealed that the "fan" or "paddle" -- often referred as the source of peculiar blue-shifted intrinsic emissions including the Strontium cloud -- is not the source of intrinsic emissions. The fan is not even a part of the equatorial disk, but is spatially separated from the peculiar emission structure. Indeed we suggest that the fan is a surface of the Northwest lobe, possibly revealed by a blowout of the equatorial disk. We will use a number of visualization techniques (tomographic animations and simple 3-D models) to show these structures. These new results have strong impact upon future numerical modelings of the Homunculus nebula and of understanding of the evolution of the ejecta powered by the central source(s).

Ishibashi, Kazunori

Between a Map and a Data Rod

A Digital Divide has long stood between how NASA and other satellite-derived data are typically archived (time-step arrays or maps) and how hydrology and other point-time series oriented communities prefer to access those data. In essence, the desired method of data access is orthogonal to the way the data are archived. Our approach to bridging the Divide is part of a larger NASA-supported data rods project to enhance access to and use of NASA and other data by the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS) and the larger hydrology community. Our main objective was to determine a way to reorganize data that is optimal for these communities. Two related objectives were to optimally reorganize data in a way that (1) is operational and fits in and leverages the existing Goddard Earth Sciences Data and Information Services Center (GES DISC) operational environment and (2) addresses the scaling up of data sets available as time series from those archived at the GES DISC to potentially include those from other Earth Observing System Data and Information System (EOSDIS) data archives. Through several prototype efforts and lessons learned, we arrived at a non-database solution that satisfied our objectivesconstraints. We describe, in this presentation, how we implemented the operational production of pre-generated data rods and, considering the tradeoffs between length of time series (or number of time steps), resources needed, and performance, how we implemented the operational production of on-the-fly (virtual) data rods. For the virtual data rods, we leveraged a number of existing resources, including the NASA Giovanni Cache and NetCDF Operators (NCO) and used data cubes processed in parallel. Our current benchmark performance for virtual generation of data rods is about a years worth of time series for hourly data (9,000 time steps) in 90 seconds. Our approach is a specific implementation of the general optimal strategy of reorganizing data to match the desired means of access. Results from our project have already significantly extended NASA data to the large and important hydrology user community that has been, heretofore, mostly unable to easily access and use NASA data.

machine learning

The Wide-Field Imaging Interferometry Testbed: Recent Results

We present recent results from the Wide-Field Imaging Interferometry Testbed (WIIT). The data acquired with the WIIT is "double Fourier" data, including both spatial and spectral information within each data cube. We have been working with this data, and starting to develop algorithms, implementations, and techniques for reducing this data. Such algorithms and tools are of great importance for a number of proposed future missions, including the Space Infrared Interferometric Telescope (SPIRIT), the Submillimeter Probe of the Evolution of Cosmic Structure (SPECS), and the Terrestrial Planet Finder Interferometer (TPF-I)/Darwin. Recent results are discussed and future study directions are described.

Rinehart, Stephen

Determining structure in molecular clouds

We descibe an automatic, objective routine for analyzing the clumpy structure in a spectral line position-position-velocity data cube. The algorithm works by first contouring the data at a multiple of the rms noise of the observations, then searches for peaks of emission which locate the clumps, and then follows them down to lower intensities. No a proiri clump profile is assumed. By creating simulated data, we test the performance of the algorithm and show that a contour map most accurately depicts internal structure at a contouring interval equal to twice the rms noise of the map. Blending of clump emission leads to small errors in mass and size determinations and in severe cases can result in a number of clumps being misidentified as a single unit, flattening the measured clump mass spectrum. The algorithm is applied to two real data sets as an example of its use. The Rosette molecular cloud is a 'typical' star-forming cloud, but in the Maddalena molecular cloud high-mass star formation is completely absent. Comparison of the two clump lists generated by the algorithm show that on a one-to-one basis the clumps in the star-forming cloud have higher peak temperatures, higher average densities, and are more gravitationally bound than in the non-star-forming cloud. Collective properties of the clumps, such as temperature-size-line-width-mass relations appear very similar, however. Contrary to the initial results reported in a previous paper (Williams & Blitz 1993), we find that the current, more thoroughly tes ted analysis finds no significant difference in the clump mass spectrum of the two clouds.

Williams, Jonathan P.

Interstellar and Ejecta Dust in the Cas A Supernova Remnant

The ejecta of the Cas A supernova remnant has a complex morphology, consisting of dense fast-moving line emitting knots and diffuse X-ray emitting regions that have encountered the reverse shock, as well as more slowly expanding, unshocked regions of the ejecta. Using the Spitzer 5-35 micron IRS data cube, and Herschel 70, 100, and 160 micron PACS data, we decompose the infrared emission from the remnant into distinct spectral components associated with the different regions of the ejecta. Such decomposition allows the association of different dust species with ejecta layers that underwent distinct nuclear burning histories, and determination of the dust heating mechanisms. Our decomposition identified three characteristic dust spectra. The first, most luminous one, exhibits strong emission features at approx. 9 and 21 micron, and a weaker 12 micron feature, and is closely associated with the ejecta knots that have strong [Ar II] 6.99 micron and [Ar III] 8.99 micron emission lines. The dust features can be reproduced by magnesium silicate grains with relatively low MgO-to-SiO2 ratios. A second, very different dust spectrum that has no indication of any silicate features, is best fit by Al2O3 dust and is found in association with ejecta having strong [Ne II] 12.8 micron and [Ne III] 15.6 micron emission lines. A third characteristic dust spectrum shows features that best matched by magnesium silicates with relatively high MgO-to-SiO2 ratio. This dust is primarily associated with the X-ray emitting shocked ejecta and the shocked interstellar/circumstellar material. All three spectral components include an additional featureless cold dust component of unknown composition. Colder dust of indeterminate composition is associated with [Si II] 34.8 micron emission from the interior of the SNR, where the reverse shock has not yet swept up and heated the ejecta. The dust mass giving rise to the warm dust component is about approx. 0.1solar M. However, most of the dust mass is associated with the unidentified cold dust component. Its mass could be anywhere between 0.1 and 1 solar M, and is primarily limited by the mass of refractory elements in the ejecta. Given the large uncertainty in the dust mass, the question of whether supernovae can produce enough dust to account for ISM dust masses in the local and high-z universe remains largely unresolved.

Interstellar

The evolution of the FIGARO data reduction system

The Figaro data reduction system originated at Caltech around 1983. It was based on concepts being developed in the U.K. by the Starlink organization, particularly the use of hierarchical self-defining data structures and the abstraction of most user-interaction into a set of 'parameter system' routines. Since 1984 it has continued to be developed at AAO, in collaboration with Starlink and Caltech. It was adopted as Starlink's main spectroscopic data reduction package, although it is by no means limited to spectra; it has operations for images and data cubes and even a few (very specialized) for four-dimensional data hypercubes. It continued to be used at Caltech and will be used at the Keck. It is also in use at a variety of other organizations around the world. Figaro was originally a system for VMS Vaxes. Recently it was ported (at Caltech) to run on SUN's, and work is underway at the University of New South Wales on a DecStation version. It is hoped to coordinate all this work into a unified release, but coordination of the development of a system by organizations covering three continents poses a number of interesting administrative problems. The hierarchical data structures used by Figaro allow it to handle a variety of types of data, and to add new items to data structures. Error and data quality information was added to the basic file format used, error information being particularly useful for infrared data. Cooperating sets of programs can add specific sub-structures to data files to carry information that they understand (polarimetry data containing multiple data arrays, for example), without this affecting the way other programs handle the files. Complex instrument-specific ancillary information can be added to data files written at a telescope and can be used by programs that understand the instrumental details in order to produce properly calibrated data files. Once this preliminary data processing was done the resulting files contain 'ordinary' spectra or images that can be processed by programs that are not instrument-specific. The structures holding the instrumental information can then be discarded from the files. Much effort has gone into trying to make it easy to write Figaro programs; data access subroutines are now available to handle access to all the conventional items found in Figaro files (main data arrays, error information, quality information etc), and programs that only need to access such items can be very simple indeed. A large number of Figaro users do indeed write their own Figaro applications using these routines. The fact that Figaro programs are written as callable subroutines getting information from the user through a small set of parameter routines means that they can be invoked in numerous ways; they are normally linked and run as individual programs (called by a small main routine that is generated automatically), but are also available linked to run under the ADAM data acquisition system and there is an interface that lets them be called as part of a user-written Fortran program. The long-term future of Figaro probably depends to a large extent on how successfully it manages the transition from being a VMS-only system to being a multi-platform system.

Shortridge, K.

Yohkoh/SXT x-ray synoptic maps of coronal brightness and temperature

The Yohkoh soft X-ray telescope (SXT) records on the order of 50 solar images per day in two different color filters. These provide material for the generation of synoptic maps, which compress the 3-dimensional data cube into two dimensions. We are creating synoptic maps from strips of data both at disk center and at different heights, including limb maps that are analogous to those produced by ground-based coronagraphs. The ratios of intensities in images taken in two filters provide estimates of the electron temperature in the range 1 - 3 x 10(exp 6) K. These are broad-band temperature maps; rather than maps created with discrete sampling as in the case of the coronal green and red lines. We discuss the properties of these maps and their application to the study of energy release in the corona.

Slater, G. L.

X-ray Mapping of Terrestrial and Extraterrestrial Materials Using the Electron Microprobe

Lunar samples returned from the Apollo program motivated development of the Bence-Albee algorithm for the rapid and accurate analysis of lunar materials, and established interlaboratory comparability through its common use. In the analysis of mineral and rock fragments it became necessary to combine micro- and macroscopic analysis by coupling electron-probe microanalysis (EPMA) with automated stage point counting. A coarse grid that included several thousand points was used, and initially wavelength-dispersive (WDS) and later energydispersive (EDS) data were acquired at discrete stage points using approx. 5 sec count times. A approx 50 micrometer beam diameter was used for WDS and up to 500 micrometer beam diameter for EDS analysis. Average analyses of discretely sampled phases were coupled with the point count data to calculate the bulk composition using matrix algebra. Use of a defocused beam resulted in a contribution from multiple phases to each analytical point, and the analytical data were deconvolved relative to end-member phase chemistry on the fly. Impressive agreement was obtained between WDS and EDS measurements as well as comparison with bulk chemistry obtained by other methods. In the 30 years since these methods were developed, significant improvements in EPMA automation and computer processing have taken place. Digital beam control allows routine collection of x-ray maps by EDS, and stage mapping for WDS is conducted continuously at slew speed and incrementally by sampling at discrete points. Digital pulse processing in EDS systems has significantly increased the throughput for EDS mapping, and the ongoing development of Si-drift detector systems promises mapping capabilities rivaling WDS systems. Spectrum imaging allows a data cube of EDS spectra to be acquired and sophisticated processing of the original data is possible using matrix algebra techniques. The study of lunar and meteoritic materials includes the need to conveniently: (1) Characterize the sample at microscopic and macroscopic scales with relatively high sensitivity, (2) Determine the modal abundance of minerals, and (3) Identify and relocate discrete features of interest in terms of size and chemistry. The coupled substitution of cations in minerals can result in significant variation in mineral chemistry, but at similar average Z, leading to poor backscattered-electron (BSE) contrast discrimination of mineralogy. It is necessary to discriminate phase chemistry at both the trace element level and the major element level. To date, the WDS of microprobe systems is preferred for mapping due to high throughput and the ability to obtain the necessary intensity to discriminate phases at both trace and major element concentrations. It is desirable to produce fully quantitative compositional maps of geological materials, which requires the acquisition of k-ratio maps that are background and dead-time corrected, and which have been corrected by phi(delta z> or an equivalent algorithm at each pixel. To date, turnkey systems do not allow the acquisition of k-ratio maps and the rigorous correction in this manner. X-ray maps of a chondrule from the Ourique meteorite, and a comb-layered xenolith from the San Francisco volcanic field, have been analyzed and processed to extract phase information. The Ourique meteorite presents a challenge due to relatively low BSE contrast, and has been studied using spectrum imaging. X-ray maps for Si, Mg, and FeK(alpha) were used to produce RGB images. The xenolith sample contains sector-zoned augite, olivine, plagioclase, and basaltic glass. X-ray maps were processed using Lispix and ImageJ software to produce mineral phase maps. The x-ray maps for Mg, Ca, and Ti were used with traceback to generate binary images that were converted to RGB images. These approaches are successful in discriminating phases, but it is desirable to achieve the methods that were used on lunar samples 30 years ago on current microprobe systems. Curnt research includes x-ray mapping analysis of the Dalgety Downs chondrite by micro x-ray fluorescence and spectrum imaging, in collaboration with Kenny Witherspoon of IXRF Systems and Dale Newbury of NIST.

Carpenter, P.

Optimal Reorganization of NASA Earth Science Data for Enhanced Accessibility and Usability for the Hydrology Community

A long-standing "Digital Divide" in data representation exists between the preferred way of data access by the hydrology community and the common way of data archival by earth science data centers. Typically, in hydrology, earth surface features are expressed as discrete spatial objects (e.g., watersheds), and time-varying data are contained in associated time series. Data in earth science archives, although stored as discrete values (of satellite swath pixels or geographical grids), represent continuous spatial fields, one file per time step. This Divide has been an obstacle, specifically, between the Consortium of Universities for the Advancement of Hydrologic Science, Inc. and NASA earth science data systems. In essence, the way data are archived is conceptually orthogonal to the desired method of access. Our recent work has shown an optimal method of bridging the Divide, by enabling operational access to long-time series (e.g., 36 years of hourly data) of selected NASA datasets. These time series, which we have termed "data rods," are pre-generated or generated on-the-fly. This optimal solution was arrived at after extensive investigations of various approaches, including one based on "data curtains." The on-the-fly generation of data rods uses "data cubes," NASA Giovanni, and parallel processing. The optimal reorganization of NASA earth science data has significantly enhanced the access to and use of the data for the hydrology user community.

data rods