Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

Theoretical Models of Astrochemical Processes

Interstellar chemistry provides a natural laboratory for studying exotic species and processes at densities, temperatures, and reaction rates. that are difficult or impractical to address in the laboratory. Thus, many chemical reactions considered too sloe by the standards of terrestrial chemistry, can be 'observed and modeled. Curious proposals concerning the nature and chemistry of complex interstellar organic molecules will be described. Catalytic reactions on "rain surfaces can, in principle, lead to a lame variety of species and this has motivated many laboratory and theoretical studies. Gas phase processes may also build lame species in molecular clouds. Future laboratory data and computational tools needed to construct accurate chemical models of various astronomical sources to be observed by Herschel and ALMA will be outlined.

Charnley, Steven↗
NASA NTRS2011

Superpixel-Augmented Endmember Detection for Hyperspectral Images

Superpixels are homogeneous image regions comprised of several contiguous pixels. They are produced by shattering the image into contiguous, homogeneous regions that each cover between 20 and 100 image pixels. The segmentation aims for a many-to-one mapping from superpixels to image features; each image feature could contain several superpixels, but each superpixel occupies no more than one image feature. This conservative segmentation is relatively easy to automate in a robust fashion. Superpixel processing is related to the more general idea of improving hyperspectral analysis through spatial constraints, which can recognize subtle features at or below the level of noise by exploiting the fact that their spectral signatures are found in neighboring pixels. Recent work has explored spatial constraints for endmember extraction, showing significant advantages over techniques that ignore pixels relative positions. Methods such as AMEE (automated morphological endmember extraction) express spatial influence using fixed isometric relationships a local square window or Euclidean distance in pixel coordinates. In other words, two pixels covariances are based on their spatial proximity, but are independent of their absolute location in the scene. These isometric spatial constraints are most appropriate when spectral variation is smooth and constant over the image. Superpixels are simple to implement, efficient to compute, and are empirically effective. They can be used as a preprocessing step with any desired endmember extraction technique. Superpixels also have a solid theoretical basis in the hyperspectral linear mixing model, making them a principled approach for improving endmember extraction. Unlike existing approaches, superpixels can accommodate non-isometric covariance between image pixels (characteristic of discrete image features separated by step discontinuities). These kinds of image features are common in natural scenes. Analysts can substitute superpixels for image pixels during endmember analysis that leverages the spatial contiguity of scene features to enhance subtle spectral features. Superpixels define populations of image pixels that are independent samples from each image feature, permitting robust estimation of spectral properties, and reducing measurement noise in proportion to the area of the superpixel. This permits improved endmember extraction, and enables automated search for novel and constituent minerals in very noisy, hyperspatial images. This innovation begins with a graph-based segmentation based on the work of Felzenszwalb et al., but then expands their approach to the hyperspectral image domain with a Euclidean distance metric. Then, the mean spectrum of each segment is computed, and the resulting data cloud is used as input into sequential maximum angle convex cone (SMACC) endmember extraction.

Thompson, David R.↗
NASA NTRS2012

Theoretical Models of Astrochemical Processes

Interstellar chemistry provides a natural laboratory for studying exotic species and processes at densities, temperatures, and reaction rates, that are difficult or impractical to address in the laboratory. Thus, many chemical reactions considered too slow by the standards of terrestrial chemistry, can be 'observed' and modeled. Various proposals concerning the nature and chemistry of complex interstellar organic molecules will be described. Catalytic reactions on grain surfaces can, in principle, lead to a large variety of species and this has motivated many laboratory and theoretical studies. Gas phase processes may also build large species in molecular clouds. Future laboratory data and computational tools needed to construct accurate chemical models of various astronomical sources to be observed by Herschel and ALMA will be outlined

Charnley, Steven B.↗
NASA NTRS2017

NASA's EOSDIS Cumulus: Ingesting, Archiving, Managing, and Distributing Earth Science Data from the Commercial Cloud

NASA's Earth Observing System Data and Information System (EOSDIS) has been working towards a vision of a cloud-based, highly-flexible, ingest, archive, management, and distribution system for its ever-growing and evolving data holdings. This system, Cumulus, is emerging from its prototyping stages and is poised to make a huge impact on how NASA manages and disseminates its Earth science data. This talk will outline the motivation for this work, present the achievements and hurdles of the past 18 months and will chart a course for the future expansion of the Cumulus expansion. We will explore on not just the technical, but also the socio-technical challenges that we face in evolving a system of this magnitude into the cloud and how we are rising to meet those challenges through open collaboration and intentional stakeholder engagement.

Earth Scienc↗
NASA NTRS2018

Accessing Data Stored in Amazon S3 Using the Hyrax OPeNDAP Server

For three years we have been investigating data storage and retrieval in the Amazon Cloud, striving to optimize both storage structures and (cache-enhanced retrieval processes. These optimizations are hidden from data users, who simply employ the Data Access Protocol (DAP), a popular, Web-based Application Programmer Interface (API) developed by OPeNDAP. We present our collected findings and their realization in the Hyrax data server. Specific techniques include caching data on spinning disk; accessing sharded data in place; optimizing the organizational structures for data within the flat key-value space of Simple Storage Service (S3); and employing an API extension that permits simultaneous operations on many datasets in a single request.

Data access↗
NASA NTRSDate not supplied

Pyroscopegridding: Geo-Leo Aerosol Data Fusion (an Open-Source Package)

The retrieval of aerosol optical depths (AODs) from sun-synchronous polar orbiting satellites, such as MODISs, VIIRSs, OMI, TROPOMI, etc., has been widely adopted as a method for obtaining information regarding particulate matter (PM) and related atmospheric processes. However, the advent of recently launched geostationary satellites, such as GOES-16/17/18, Himawari-8/9, and Meteosat Third Generation (MTG), has led to an increased temporal resolution of AOD observations (order of 10 minutes), resulting in typically one or more images per hour during daylight hours, compared to the once-per-day observations obtained from LEO satellites. By integrating these observations, the diurnal cycle of global AOD can be characterized at local, regional, and global scales. The scientific community is still examining the novel data from geostationary satellite observations and evaluating methods for effectively merging these observations with differing spatial and temporal resolutions. This presents a significant ""Big Data"" challenge, encompassing not only data storage, but also data discoverability, accessibility, and migration within cloud computing environments. This study presents our attempts at fusing Level 2 aerosol data from six satellites, three of which are geostationary (GOES-16/17 and Himawari-8) and three of which are polar orbiting (TERRA/MODIS, AQUA/MODIS, and SNPP-VIIRS), using the Dark Target aerosol retrieval algorithm. The ability to fuse remote sensing products on demand into desired temporal and spatial domains empowers researchers and practitioners to more efficiently work with satellite and sensor data. It is our hope that through making our open-source package and accompanying functionality available, the scientific community will have improved access to aerosol data processing resources.

Jennifer Wei↗
NASA NTRSDate not supplied

Efficient Access to MERRA-2 Reanalysis Data from the Cloud Archives

The Modern-Era Retrospective analysis for Research and Applications, Version (MERRA-2) dataset is now available in the AWS West Region-2 cloud. The free data downloading policy remains unchanged. The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) offers cloud services such as OPeNDP subsetting, and time series services. This presentation will showcase use cases that testing the procedures and find the most efficiency workflows to access and process the data for finding climate extremes. The use cases include a) accessing and processing data directly in the cloud S3 bucket without downloading it, and b) downloading subset data to process on the user’s local system.

data access, cloud computing, climate extremes↗
NASA NTRSDate not supplied

Surface Radiation Budget

The Surface Radiation Budget (SRB) data sets contain global 3-hourly, daily and monthly averages of surface longwave and shortwave radiative properties, cloud amount, and meteorological properties computed using models. The main input data for these models include cloud information, top-of-atmosphere radiances and profiles of atmospheric water vapor and temperature. Some of the input data include Earth Radiation Budget Energy (ERBE) top-of-atmosphere clear-sky albedo and International Satellite Cloud Climatology Project (ISCCP) radiances and cloud amount. SRB parameters derived for the renewable energy community are also available from the Surface meteorology and Solar Energy (SSE) data set. Other SRB data are available from Clouds and the Earth's Radiant Energy System (CERES) and Multi-angle Imaging SpectroRadiometer (MISR). [Mission Objectives] The objective of the SRB Project is to produce and archive a global data set of shortwave (SW) and longwave (LW) surface and top of the atmosphere parameters. The data generated in the SRB project may be used in conjunction with other data sets to facilitate the development of renewable energy resources and increase understanding of radiative properties within the meteorological community. [Temporal_Coverage: Start_Date=1983-07-01; Stop_Date=2005-06-30] [Spatial_Coverage: Southernmost_Latitude=-90; Northernmost_Latitude=90; Westernmost_Longitude=-180; Easternmost_Longitude=180].

Stackhouse, Paul W.↗
NASA NTRSDate not supplied

Open Source Application of Fusing Aerosol Products from GEO and LEO Satellites

Retrieving aerosol optical depths (AODs) from sun-synchronous polar orbiting (aka low earth orbit, LEO) satellites, such as MODISs, and VIIRSs, OMI, TROPOMI, etc, has become well-established as a tool for extracting information on particulate matter (PM) and related processes in the atmosphere. However, with recently launched geostationary satellites (GEO), such as GOES-16/17/18, and Himawari-8/9, and Meteosat Third Generation (MTG) they provide a much higher temporal resolution (order of 10 minutes), typically an image once or more per hour during daylight compared to LEO once per day. By combining these observations, we may be able to characterize the diurnal cycle of global AOD at the local, regional and global scale. While the science community is still exploring the new data from GEO observations, we have been thinking about how to properly combine/merge/fuse those data considering differences in their spatial and temporal resolutions. However, this poses a “Big Data” challenge. The big data challenge is not just about data storage, but also about data discoverability, and accessibility, and even more, about data migration/mirroring in the cloud-computing environment. This paper is merely showing some of the efforts and approaches we have attempted in fusing six satellites’ Level 2 aerosol data (three are from GEO (GOES-16/17 and Himawari-8), and the other three are from LEO (TERRA/MODIS, AQUA/MODIS, SNPP-VIIRS) from Dark Target (DT) aerosol retrieval algorithm. Having the on-demand capability of fusing remote sensing products onto the desired temporal and spatial domain enables researchers and application practitioners to better manipulate and work with satellite and sensor data. It is our hopeWe hope that by making such an open-source package, and the accompanying functionality, the scientific community will be granted easier access to aerosol data processing resources. The MEaSUREs Program (Making Earth System Data Records for Use in Research Environments) expands our understanding of the Earth's current system through atmospheric and surface measurements. In an effort to aid the scientific research component and improve open source methods, this project developed Python code for fusing six satellite Level 2 aerosol data (three are from geostationary satellites (GEO), and the other three are from low earth orbital satellites (LEO)) from Dark Target Aerosol Retrieval Algorithm.

Jennifer Wei↗
NASA NTRS1975

Man-Computer Interactive Data Access System (McIDAS). Continued development of McIDAS and operation in the GARP Atlantic tropical experiment

The complete output of the Synchronous Meteorological Satellite was recorded on one inch magnetic tape. A quality control subsystem tests cloud track vectors against four sets of criteria: (1) rejection if best match occurs on correlation boundary; (2) rejection if major correlation peak is not distinct and significantly greater than secondary peak; (3) rejection if correlation is not persistent; and (4) rejection if acceleration is too great. A cloud height program determines cloud optical thickness from visible data and computer infrared emissivity. From infrared data and temperature profile, cloud height is determined. A functional description and electronic schematics of equipment are given.

Suomi, V. E.↗
NASA NTRS2018

High Performance Space Computing with System-on-Chip Instrument Avionics for Space-based Next Generation Imaging Spectrometers (NGIS).

The emergent technology of system-on-chip (SoC) devices promises lighter, smaller, cheaper, and more capable and reliable space electronic systems that could help to unveil some of the most treasured secrets in our universe. This technology is an improvement over the technology that is currently used in space applications, which lags behind stateof-the-art commercial-off-the-shelf (COTS) equipment by several generations. SoC technology integrates all computational power required by next-generation space exploration science instruments onto a single chip. This presentation will describe a Xilinx Zynq-based data acquisition, cloud-screening and compression computing system that has been developed at the Jet Propulsion Laboratory (JPL) for JPL’s Next Generation Imaging Spectrometers (NGIS). The Xilinx Zynq-based Alpha Data hardware assembly fits into a 120mm by 190m by 40mm assembly and uses 9 watts at peak performance. The computing element is a Xilinx Zynq Z7045Q which includes a Kintex-7 FPGA (equivalent to 3 RAD Virtex5 FPGAs in terms of logic cell resources) and dual-core ARM Cortex-A9 Processors (equivalent to 10 RAD750 Power PCs in term of processing capability).

Smith, Adam↗
NASA NTRS2018

High Performance Space Computing with System-on-Chip Instrument Avionics for Space-based Next Generation Imaging Spectrometers (NGIS).

The emergent technology of system-on-chip (SoC) devices promises lighter, smaller, cheaper, and more capable and reliable space electronic systems that could help to unveil some of the most treasured secrets in our universe. This technology is an improvement over the technology that is currently used in space applications, which lags behind state-of-the-art commercial-off-the-shelf (COTS) equipment by several generations. SoC technology integrates all computational power required by next-generation space exploration science instruments onto a single chip. This presentation will describe a Xilinx Zynq-based data acquisition, cloud-screening and compression computing system that has been developed at the Jet Propulsion Laboratory (JPL) for JPL’s Next Generation Imaging Spectrometers (NGIS). The Xilinx Zynq-based Alpha Data hardware assembly fits into a 120mm by 190m by 40mm assembly and uses 9 watts at peak performance. The computing element is a Xilinx Zynq Z7045Q which includes a Kintex-7 FPGA (equivalent to 3 RAD Virtex5 FPGAs in terms of logic cell resources) and dual-core ARM Cortex-A9 Processors (equivalent to 10 RAD750 Power PCs in term of processing capability).

Dolinar, Sam↗
NASA NTRS2019

High performance space computing with system-on-chip instrument avionics for space-based Next Generation Imaging Spectrometers (NGIS)

The emergent technology of system-on-chip (SoC) devices promises lighter, smaller, cheaper, and more capable and reliable space electronic systems that could help to unveil some of the most treasured secrets in our universe. This technology is an improvement over the technology that is currently used in space applications, which lags behind state-of-the-art commercial-off-the-shelf (COTS) equipment by several generations. SoC technology integrates all computational power required by next-generation space exploration science instruments onto a single chip. This presentation will describe a Xilinx Zynq-based data acquisition, cloud-screening and compression computing system that has been developed at the Jet Propulsion Laboratory (JPL) for JPL’s Next Generation Imaging Spectrometers (NGIS). The Xilinx Zynq-based Alpha Data hardware assembly fits into a 120mm by 190m by 40mm assembly and uses 9 watts at peak performance. The computing element is a Xilinx Zynq Z7045Q which includes a Kintex-7 FPGA (equivalent to 3 RAD Virtex5 FPGAs in terms of logic cell resources) and dual-core ARM Cortex-A9 Processors (equivalent to 10 RAD750 Power PCs in term of processing capability).

Dolinar, Sam↗
NASA NTRS1990

ER-2 lidar measurements of stratocumulus cloud top structure on July 14, 1987

On July 14, 1987 NASA's ER-2 high altitude aircraft flew a mission to measure the structure of stratocumulus clouds off the coast of California. A flight pattern was executed so that the two-dimensional variability of the clouds could be detected. The technique of analysis of the lidar data to measure cloud tops follows. First each signal is searched for its maximum in return strength. This maximum is caused by scattering of the laser light off cloud particles or from the ocean surface. Next the variance of the signal return above the level of maximum backscatter is determined. Cloud top is assigned to a level (above the level of maximum backscatter) where the backscatter exceeds the average variance. This two-step process is necessary because the level of maximum backscatter does not correspond to the cloud top. Ocean surface returns are easily separated from cloud returns in this process, described in detail by Boers, Spinhirne, and Hart (1988). Analysis of the data so far has shown that there were very few breaks in the clouds. Furthermore the layer top was very flat with local oscillations not exceeding 30 m. Such small cloud top variations are still well within the range of detectability, because the precision of this technique of cloud top detection has previously been established to be 13 to 15 m. Data are presently being analyzed to compute cloud top distributions and fractional cloudiness. The aim of this research is to relate the fractional cloudiness to the mean thermodynamic structure of the boundary layer. Researchers plan to compute spectral scales of the cloud top variability in two dimensions to determine the orientation of the clouds with respect to the mean wind. Furthermore the lidar derived cloud top distribution will be used in the computation of the thermodynamic and radiation budget of the boundary layer.

Boers, Reinout↗
NASA NTRS2015

High Resolution Nature Runs and the Big Data Challenge

NASA's Global Modeling and Assimilation Office at Goddard Space Flight Center is undertaking a series of very computationally intensive Nature Runs and a downscaled reanalysis. The nature runs use the GEOS-5 as an Atmospheric General Circulation Model (AGCM) while the reanalysis uses the GEOS-5 in Data Assimilation mode. This paper will present computational challenges from three runs, two of which are AGCM and one is downscaled reanalysis using the full DAS. The nature runs will be completed at two surface grid resolutions, 7 and 3 kilometers and 72 vertical levels. The 7 km run spanned 2 years (2005-2006) and produced 4 PB of data while the 3 km run will span one year and generate 4 BP of data. The downscaled reanalysis (MERRA-II Modern-Era Reanalysis for Research and Applications) will cover 15 years and generate 1 PB of data. Our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS), a specialization of the concept of business process-as-a-service that is an evolving extension of IaaS, PaaS, and SaaS enabled by cloud computing. In this presentation, we will describe two projects that demonstrate this shift. MERRA Analytic Services (MERRA/AS) is an example of cloud-enabled CAaaS. MERRA/AS enables MapReduce analytics over MERRA reanalysis data collection by bringing together the high-performance computing, scalable data management, and a domain-specific climate data services API. NASA's High-Performance Science Cloud (HPSC) is an example of the type of compute-storage fabric required to support CAaaS. The HPSC comprises a high speed Infinib and network, high performance file systems and object storage, and a virtual system environments specific for data intensive, science applications. These technologies are providing a new tier in the data and analytic services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. In our experience, CAaaS lowers the barriers and risk to organizational change, fosters innovation and experimentation, and provides the agility required to meet our customers' increasing and changing needs

big data analysis↗
NASA NTRS2018

Hyperspectral Data Processing: an Opportunity for End-To-End Processing

The evolution and improvements in hyperspectral instrumentation are being matched by information technology improvements in science data processing and analysis. Research has improved techniques in both onboard and ground-based processing to support other high data volume instruments. Algorithms and hardware have evolved, permitting faster access to the observations. Cloud computing is taking the algorithms to the data. Technologies are being specifically designed to address high volume data sets and are an investment in the improvement of hyperspectral data processing.

science data processing↗