Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing; data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Detection of Ice Polar Stratospheric Clouds from Assimilation of Atmospheric Infrared Sounder Data

A novel technique is presented for detection of ice polar stratospheric clouds (PSCs) that form at extremely low temperatures in the lower polar stratosphere during winter. Temperature is a major factor in determining abundance of PSCs, which in turn provide surfaces for heterogeneous chemical reactions leading to ozone loss and radiative cooling. The technique infers the presence of ice PSCs using radiances from the Atmospheric Infrared Sounder (AIRS) in the Goddard Earth Observing System version 5 (GEOS-5) data assimilation system. Brightness temperatures are computed from short-term GEOS-5 forecasts for several hundred AIRS channels, using a radiation transfer module. The differences between collocated AIRS observations and these computed values are the observed-minus-forecast (O-F) residuals in the assimilation system. Because the radiation model assumes clear-sky conditions, we hypothesize that these O-F residuals contain quantitative information about PSCs. This is confirmed using sparse data from the Polar Ozone and Aerosol Measurement (POAM) III occultation instrument. The analysis focuses on 0-F residuals for the 6.79pm AIRS moisture channel. At coincident locations, when POAM III detects ice clouds, the AIRS O-F residuals for this channel are lower than -2K. When no ice PSCs are evident in POAM III data, the AIRS 0-F residuals are larger. Given this relationship, the high spatial density of AIRS data is used to construct maps of regions where 0-F residuals are lower than -2K, as a proxy for ice PSCs. The spatial scales and spatio-temporal variations of these PSCs in the Antarctic and Arctic are discussed on the basis of these maps.

Stajner, Ivanka↗

Modifying the Asynchronous Jacobi Method for Data Corruption Resilience

Moving scientific computation from high-performance computing (HPC) and cloud computing (CC) environments to devices on the edge, i.e., physically near instruments of interest, has received tremendous interest in recent years. Such edge computing environments can operate on data in situ, offering enticing benefits over data aggregation to HPC and CC facilities that include avoiding costs of transmission, increased data privacy, and real-time data analysis. Because of the inherent unreliability of edge computing environments, new fault-tolerant approaches must be developed before the benefits of edge computing can be realized. Motivated by algorithm-based fault tolerance, a variant of the asynchronous Jacobi (ASJ) method is developed that achieves resilience to data corruption by rejecting solution approximations from neighbor devices according to a bound derived from convergence theory. Numerical results on a two-dimensional Poisson problem show that the new rejection criterion, along with a novel approximation to the shortest path length on which the criterion depends, restores convergence for the ASJ variant in the presence of certain types data corruption. Numerical results are obtained for when the singular values in the analytic bound are approximated. Additional linear systems are also explored, one with a more dense sparsity pattern and one that includes advection. All results indicate that successful resilience to data corruption depends on whether the bound tightens fast enough to reject corrupted data before the iteration evolution deviates significantly from that predicted by the convergence theory defining the bound. This observation generalizes to future work on algorithm-based fault tolerance for other asynchronous algorithms, including upcoming approaches that leverage Krylov subspaces.

97 MATHEMATICS AND COMPUTING↗

Development and Validation of a Polar Cloud Algorithm for CERES

The objectives of this project, as described in the original proposal, were to develop an algorithm for diagnosing cloud properties over snow- and ice-covered surfaces, particularly at night, using satellite radiances from the Advanced Very High Resolution Radiometer (AVHRR) and High-resolution Infrared Radiation Sounder (HIRS) sensors. Products from this algorithm include a cloud mask and additional cloud properties such as cloud phase, amount, and height. The SIVIS software package, developed as a part of the CERES project, was originally the primary tool used to develop the algorithm, but as it is no longer supported we have had to pursue a new tool to enable the combination and analysis of collocated radiances from AVHRR and HIRS. This turned out to be a much larger endeavor than we expected, but we now have the data sets collocated (with many thanks to B. Baum for the fundamental code) and we have developed a nighttime cloud detection algorithm. Using this algorithm we have also computed realistic-looking cloud fractions from AVHRR brightness temperatures. A method to identify cloud phase has also been implemented. Atmospheric information from the TIROS Operational Vertical Sounder (TOVS) Polar Pathfinder Data Set, which includes temperature and moisture profiles as well as surface information, provides information required for determining cloud-top height.

Source record↗

Parameter Estimation in Atmospheric Data Sets

In this study the structure tensor technique is used to estimate dynamical parameters in atmospheric data sets. The structure tensor is a common tool for estimating motion in image sequences. This technique can be extended to estimate other dynamical parameters such as diffusion constants or exponential decay rates. A general mathematical framework was developed for the direct estimation of the physical parameters that govern the underlying processes from image sequences. This estimation technique can be adapted to the specific physical problem under investigation, so it can be used in a variety of applications in trace gas, aerosol, and cloud remote sensing. As a test scenario this technique will be applied to modeled dust data. In this case vertically integrated dust concentrations were used to derive wind information. Those results can be compared to the wind vector fields which served as input to the model. Based on this analysis, a method to compute atmospheric data parameter fields will be presented. .

Wenig, Mark↗

The Parallel System for Integrating Impact Models and Sectors (pSIMS)

We present a framework for massively parallel climate impact simulations: the parallel System for Integrating Impact Models and Sectors (pSIMS). This framework comprises a) tools for ingesting and converting large amounts of data to a versatile datatype based on a common geospatial grid; b) tools for translating this datatype into custom formats for site-based models; c) a scalable parallel framework for performing large ensemble simulations, using any one of a number of different impacts models, on clusters, supercomputers, distributed grids, or clouds; d) tools and data standards for reformatting outputs to common datatypes for analysis and visualization; and e) methodologies for aggregating these datatypes to arbitrary spatial scales such as administrative and environmental demarcations. By automating many time-consuming and error-prone aspects of large-scale climate impacts studies, pSIMS accelerates computational research, encourages model intercomparison, and enhances reproducibility of simulation results. We present the pSIMS design and use example assessments to demonstrate its multi-model, multi-scale, and multi-sector versatility.

crop modeling↗

Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows

Scientific workflows are used almost universally across science domains for solving complex and largescale computing and data analysis problems. The importance of workflows is highlighted by the fact that they have underpinned some of the most significant discoveries of the past decades. Many of these workflows have significant computational, storage, and communication demands, and thus must execute on a range of large-scale computer systems, from local clusters to public clouds and upcoming exascale HPC platforms. Managing these executions is often a significant undertaking, requiring a sophisticated and versatile software infrastructure. Historically, infrastructures for workflow execution consisted of complex, integrated systems, developed in-house by workflow practitioners with strong dependencies on a range of legacy technologies—even including sets of ad hoc scripts. Due to the increasing need to support workflows, dedicated workflow systems were developed to provide abstractions for creating, executing, and adapting workflows conveniently and efficiently while ensuring portability. While these efforts are all worthwhile individually, there are now hundreds of independent workflow systems. These workflow systems are created and used by thousands of researchers and developers, leading to a rapidly growing corpus of workflows research publications. The resulting workflow system technology landscape is fragmented, which may present significant barriers for future workflow users due to many seemingly comparable, yet usually mutually incompatible, systems that exist. In order to tackle some of the challenges described above, the DOE-funded ExaWorks and NSF-funded WorkflowsRI projects have organized in 2021 a series of events entitled the “Workflows Community Summit”. The third edition of the “Workflows Community Summit” explored workflows challenges and opportunities from the perspective of computing centers and facilities. This third summit builds on two prior summits (https://workflowsri.org/summits) that (i) established a high level vision for workflows research; and (ii) explored technical approaches for realizing that vision. The third summit brought together a small group of facilities representatives with the aim to understand how workflows are currently being used at each facility, how facilities would like to interact with workflow developers and users, how workflows fit with facility roadmaps, and what opportunities there are for tighter integration between facilities and workflows. This report documents and organizes the wealth of information provided by the participants before, during, and after the summit.

97 MATHEMATICS AND COMPUTING↗

Search for solar signals in GOES imagery and cloud climatology analysis

The digitization and analysis of 16mm GOES IR imagery to identify spatial and temporal variation in clouds is discussed, along with a cloud climatology analysis. Recently added to the Interactive Graphics Package was a Computer Image Analysis System. This system allows many new imaging techniques. One of the possibilities is the analysis of overall light intensity variations in six 10 degree square boxes across the intertropical convergence zone. This technique is discussed, as is a preliminary analysis of unnormalized data.

Markson, R. J.↗

The Matsu Wheel: A Cloud-Based Framework for Efficient Analysis and Reanalysis of Earth Satellite Imagery

Project Matsu is a collaboration between the Open Commons Consortium and NASA focused on developing open source technology for cloud-based processing of Earth satellite imagery with practical applications to aid in natural disaster detection and relief. Project Matsu has developed an open source cloud-based infrastructure to process, analyze, and reanalyze large collections of hyperspectral satellite image data using OpenStack, Hadoop, MapReduce and related technologies. We describe a framework for efficient analysis of large amounts of data called the Matsu "Wheel." The Matsu Wheel is currently used to process incoming hyperspectral satellite data produced daily by NASA's Earth Observing-1 (EO-1) satellite. The framework allows batches of analytics, scanning for new data, to be applied to data as it flows in. In the Matsu Wheel, the data only need to be accessed and preprocessed once, regardless of the number or types of analytics, which can easily be slotted into the existing framework. The Matsu Wheel system provides a significantly more efficient use of computational resources over alternative methods when the data are large, have high-volume throughput, may require heavy preprocessing, and are typically used for many types of analysis. We also describe our preliminary Wheel analytics, including an anomaly detector for rare spectral signatures or thermal anomalies in hyperspectral data and a land cover classifier that can be used for water and flood detection. Each of these analytics can generate visual reports accessible via the web for the public and interested decision makers. The result products of the analytics are also made accessible through an Open Geospatial Compliant (OGC)-compliant Web Map Service (WMS) for further distribution. The Matsu Wheel allows many shared data services to be performed together to efficiently use resources for processing hyperspectral satellite image data and other, e.g., large environmental datasets that may be analyzed for many purposes.

Correlations among the Optical Properties of Cirrus-Cloud Particles: Microphysical Interpretation

Cirrus measurements obtained with a ground-based polarization Raman lidar at 67.9 deg N in January 1997 reveal a strong positive correlation between the particle optical properties, specifically depolarization ratio delta(sub par) and extinction- to-backscatter (lidar) ratio S, for delta(sub par) less than approximately 40%, and an anti-correlation for delta(sub par) greater than approximately 40%. Over the length of the measurements the particle properties vary systematically. Initially, delta (sub par) approximately equals 60% and S approximately equals 10sr are observed. Then, with decreasing delta(sub par), S first increases to approximately 27sr (delta(sub par) approximately equals 40%) before decreasing to values around 10sr again (delta(sub par) approximately equals 20%). The analysis of lidar humidity and radiosonde temperature data shows that the measured optical properties stem from scattering by dry solid ice particles, while scattering by supercooled droplets, or by wetted or subliming ice particles can be excluded. For the microphysical interpretation of the lidar measurements, ray-tracing computations of particle scattering properties have been used. The comparison with the theoretical data suggests that the observed cirrus data can be interpreted in terms of size, shape, and, under the assumption that the lidar measurements of consecutive cloud segments can be mapped on the temporal development of a single cloud parcel moving along its trajectory, growth of the cirrus particles: Near the cloud top in the early stage of cirrus development, light scattering by nearly isometric particles that have the optical characteristics of hexagonal columns (short, column-like particles) is dominant. Over time the ice particles grow, and as the cloud base height extends to lower altitudes characterized by warmer temperatures they become morphologically diverse. For large S and depolarization values of approximately 40%, the scattering contributions of column- and plate-like particles are roughly the same. In the lower ranges of the cirrus clouds, light scattering is predominantly by plate-like ice particles. This interpretation assumes random orientation of the cirrus particles. Simulations with a simple model suggest, however, that the positive correlation between S and delta(sub par) which is observed for depolarization ratios less than 40% mainly at low cloud altitudes, can be alternatively explained by horizontal alignment of a fraction of the cirrus particle population.

Reichardt, J.↗

Land utilization and ecological aspects in the Sylhet-Mymensingh Haor Region of Bangladesh: An analysis of LANDSAT data

The use of remote sensing data from LANDSAT (ERTS) imageries in identifying, evaluating and mapping land use patterns of the Haor area in Bangladesh was investigated. Selected cloud free imageries of the area for the period 1972-75 were studied. Imageries in bands 4, 5 and 7 were mostly used. The method of analysis involved utilization of both human and computer services of information from ground, aerial photographs taken during this period and space imageries.

Chowdhury, M. I.↗

Low cost, microcomputer-based interactive analysis system for direct-reception and archived remote sensing data

A hardware/software system has been developed for use in meteorological field experiments and by developing countries which need low-cost remote sensing data analysis capabilities. The system can directly receive satellite transmissions of the AVHRR, TOVS, Argos, GOES, GMS, and Meteosat systems. Also, the system can process computer tapes from Landsat. Received data are ingested directly into the computer's disk storage, eliminating the need for a wide-band analog tape drive. Data may be interpreted in terms of cloud imagery, cloudtop temperature, sea surface temperature, vegetation index, and temperature/humidity profiles with overlays of geographic boundaries and latitude/longitude grids. The image processing, enhancement functions, and modeling capabilities of the system are discussed.

Vermillion, Charles H.↗

Results from Global Land-Surface Data Assimilation Methods

Realistic representation of the land surface is crucial in global climate modeling (GCM). Recently, the Mosaic land-surface Model (LSM) has been driven off-line using GEOS DAS (Goddard Earth Observing System Data Assimilation System) atmospheric forcing, forming the Off-line Land-surface Global Assimilation (OLGA) system. This system provides a computationally efficient test bed for land surface data assimilation. Here, we validate the OLGA simulation of surface processes and the assimilation of ISCCP surface temperatures. Another component of this study as the incorporation of the Physical-space Statistical Analysis System (PSAS) into OLGA, in order to assimilate surface temperature observations from the International Satellite Cloud Climatology Project (ISCCP). To counteract the subsequent forcing of the analyzed skin temperature back to the initial state following the analysis. incremental bias correction (IBC) was included in the assimilation. The IBC scheme effectively removed the time mean bias, but did not remove him in the mean diurnal cycle. Therefore, a diurnal him correction (DBC) scheme was developed, where the time-dependent bias was modeled with a sine wave parameterization. In addition, quality control of the ISCCP data and anisotropic temperature correction were implemented in PSAS. Preliminary results showed a substantial impact from the inclusion of PSAS and DBC that was visible in the surface meteorology fields and energy budget. Also, the monthly mean diurnal cycle from the experiment closely matched the diurnal cycle from the observations.

Radakovich, Jon D.↗

Watermarks in stream processing systems: semantics and comparative analysis of Apache Flink and Google cloud dataflow

Streaming data processing is an exercise in taming disorder: from oftentimes huge torrents of information, we hope to extract powerful and timely analyses. But when dealing with streaming data, the unbounded and temporally disordered nature of real-world streams introduces a critical challenge: how does one reason about the completeness of a stream that never ends? In this paper, we present a comprehensive definition and analysis of watermarks, a key tool for reasoning about temporal completeness in infinite streams.First, we describe what watermarks are and why they are important, highlighting how they address a suite of stream processing needs that are poorly served by eventually-consistent approaches:• Computing a single correct answer, as in notifications.• Reasoning about a lack of data, as in dip detection.• Performing non-incremental processing over temporal subsets of an infinite stream, as in statistical anomaly detection with cubic spline models.• Safely and punctually garbage collecting obsolete inputs and intermediate state.• Surfacing a reliable signal of overall pipeline health.Second, we describe, evaluate, and compare the semantically equivalent, but starkly different, watermark implementations in two modern stream processing engines: Apache Flink and Google Cloud Dataflow.

Akidau, Tyler↗

Building a Bilingual Google Earth Engine Dashboard to Increase Accessibility to Long-term Time Series Remote Sensing Data for Monitoring Saline System Changes in Chile’s Atacama Desert

Saline systems, consisting of salt flats, ponds, and marshes, provide vital water resources to wildlife and communities in northern Chile’s Atacama Desert, one of the driest regions in the world. Mining is extensive in the Atacama, which contains 30% of the world’s lithium reserves and is abundant in potassium and boron. The groundwater that feeds into salt marshes and ponds is extracted in large volumes for mining operations, limiting the availability of water for ecosystems. However, identifying long-term and large-scale environmental impacts from local lithium mining on the saline systems is limited by region inaccessibility and terrain variability. Open access satellite imagery and cloud computing technology has made studying Atacama saline systems feasible and allowed for collaboration across different agencies and countries. The NASA DEVELOP Program partnered with Chile’s la Universidad de La Serena and Servicio Nacional de Geología y Minería (SERNAGEOMIN) to create the Saline Analysis Tool (SalT) in Google Earth Engine (GEE). SalT is used to analyze the extent and distribution of remote saline systems in the Atacama from 1986 to the present day. The tool filters Landsat 5 Thematic Mapper (TM) and Landsat 8 Operational Land Imager (OLI) data from GEE’s data catalog and creates a single composite image per year for analysis. Additional output analyses include land cover classification, Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI) raster images that can be displayed on the map interface or exported. The tool can also generate time-lapse videos and charts displaying NDVI, NDWI, and land cover over time. A key feature of the tool is the use of a bilingual graphical user interface to make analysis accessible and customizable to different users’ needs—SalT provides options to select an analysis area, analysis time period, and outputs to display or export. The tool also incorporates new Earth observations as they are added to GEE’s catalog. The ability to easily visualize and analyze long-term remote sensing imagery will enable SERNAGEOMIN and la Universidad de la Serena to continually monitor changes in these saline systems and inform future land management policy.

NASA DEVELOP↗

Ames Contrail Simulation Model: Modeling Aviation Induced Contrails and the Computation of Contrail Radiative Forcing Using Air Traffic Data

NASA is developing traffic flow management strategies to reduce the impact of aviation on climate and improve efficiency of aircraft routes in the presence of limited airport and airspace capacity constraints, which are partly due to convective weather and natural calamities. Development of Strategic Traffic Flow Management (STFM) requires models of aircraft emissions and contrails, and models of their effect on climate. There is extensive literature on the modeling of contrails and the Radiative Forcing (RF) associated with contrails. This report captures the latest NASA developments in this research area and describes the integrated modeling, analysis, and software development to support STFM. Aircraft contrails are long, thin and often linear clouds triggered by aircraft engine exhausts in the high-altitude ice-saturated atmosphere. Contrails, similar to that of natural cirrus clouds, can impact global climate by reflecting shortwave radiation and trapping longwave radiation. Recent studies from the Intergovernmental Panel on Climate Change (IPCC) have shown that aircraft contrails are estimated to have greater impact on global warming than aircraft CO 2 emissions. The Ames Contrail Simulation Model (ACSM) presented in this report simulates the full life cycle of aircraft contrails, including their formation, dynamic evolution, and dissipation, and calculates the associated RF with actual meteorological and air traffic data. ACSM combines models from previous studies that focus on contrail formation and persistence based on the Schmidt-Appleman theoretical criteria, and it incorporates elements of cloud dynamics, microphysics, and climate modeling found in other surveyed contrail models, while also making adjustment for improved computational efficiency. In addition, ACSM is integrated with NASA's state-of-the-art flight simulation software for rapid assessment of aircraft contrail impacts. Applications include assessments of long-term global climate impact resulting from aviation-induced contrails and the design of optimal contrail-mitigation aircraft operation strategies.

aviation contrails↗

Computer simulation of the Pioneer 10 microwave occultation experiment

As Pioneer 10 approaches the planet Jupiter, its trajectory will carry it behind the planet, as seen from the earth. The microwave telemetry signal from the spacecraft will then pass through the Jovian atmosphere on its way to the earth, and analysis of the effects of the atmosphere on the signal should yield information on the atmospheric structure. To help define the capabilities of this experiment and to aid the analysis of the real data, we have simulated the occultation experiment on a computer by using model Jovian atmospheres and a predicted trajectory. The results indicate that the useful depth of penetration of the microwave signal will be governed by ammonia absorption rather than superrefractivity. The experiment is apparently far more sensitive to variations in cloud-top temperature than to variations in the hydrogen-helium ratio. Characteristic 'flashes' in the attenuation record of the signal may yield information on the location of the top of the Jovian convective zone.

Ungar, S. G.↗

Waterspout, Gust Fronts and Associated Cloud Systems

Nine waterspouts observed on five experimental days during the GATE period of observations are discussed. Primary data used are from 2 aircraft flying in different patterns, one above the other between 30 and 300 m. There is strong evidence associating whirl initiation with cumulus outflow. Computations prepared from estimates of convergence with the region suggest the possibility of vortex generation within 4 minutes. This analysis supports (1) the importance cumulus outflows may have in waterspout initiation and (2) the possibility that sea surface temperature gradients may be important in enabling waterspout development from modest size cumuli.

Simpson, J.↗

An Assessment of SeaWiFS and MODIS Ocean Coverage

Ocean coverages of SeaWiFS and MODIS were assessed for three seasons by considering monthly mean values of surface winds speeds and cloud cover. Mean and maximum coverages combined SeaWiFS and MODIS by considering combined coverages for ten-degree increments of the MODIS orbital mean anomaly. From this analysis the mean and maximum combined coverages for SeaWiFS and MODIS were determined for one and four-day periods for spring, summer, and winter seasons. Loss of coverage due to Sun glint and cloud cover were identified for both the individual and combined cases. Our analyses indicate that MODIS will enhance ocean coverage for all three seasons examined. ne combined SeaWiFS/MODIS show an increase of coverage of 42.2% to 48.7% over SeaWiFS alone for the three seasons studied; the increase in maximum one day coverage ranges from 47.5% to 52.0%. The increase in four-day coverage for the combined case ranged from 31.0% to 35.8% for mean coverage and 33.1 % to 39.2% for maximum coverage. We computed meridional distributions of coverages by binning the data into five-degree latitude bands. Our analysis shows a strong seasonal dependence of coverage. In general the meridional analysis indicates that increase in coverages for SeaWiFS/MODIS over SeaWiFS alone are greatest near the solar declination.

Woodward, Robert H.↗