SEARCH · Search NASA

Results for “cloud computing; data analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Evaluation of Historic Nuclear Cloud Measurement for Entrainment Parameter Determination

We evaluated the uncertainties associated with the entrainment parameters computed from nuclear cloud measurements taken during historic weapon testing campaigns. This evaluation found that variations in the way cloud heights and diameters were defined during the analysis of film records result in very large uncertainties in the entrainment parameters, severely limiting the utility of these data. For the measurements of atmospheric conditions at the time of the tests, the main sources of error are associated with the differences in the location and timing of the soundings.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

NASA NTRS2004

Analysis of TRMM Microphysical Measurements: Tropical Rainfall Measuring Mission (TRMM)

SPEC Incorporated participated in three of the four TRMM field campaigns (TEFLUN-A, TEFLUN-B and KWAJEX), installing and operating a cloud particle imager (CPI) and a high volume precipitation spectrometer (HVPS) on the SPEC Learjet in TEFLUN-A, the University of North Dakota Citation in TEFLUN-B and KWAJEX, and a CPI on the NASA DC-8 in KWAJEX. This report presents and discusses new software tools and algorithms that were developed to analyze microphysical data collected during these field campaigns, as well as scientific interpretations of the data themselves. Software algorithms were developed to improve the analysis of microphysical measurements collected by the TRMM aircraft during the field campaigns. Particular attention was paid to developing and/or improving algorithms used to compute particle size distributions and ice water content. Software was also developed in support of production of the TRMM Common Microphysical Product (CMP) data files. CMP data files for TEFLUN-A field campaign were produced and submitted to the DAAC. Typical microphysical properties of convective and stratiform regions from TEFLUN-A and KWAJEX clouds were produced. In general, it was found that in the upper cloud region near -20 to -25 C, stratiform clouds contain very high (greater than 1 per cubic centimeter) concentrations of small ice particles, which are suspected to be a residual from homogeneous freezing and sedimentation of small drops in a convective updraft. In the upper cloud region near -20 to -25 C, convective clouds contain aggregates, which are not found lower in the cloud. Stratiform clouds contain aggregates at all levels, with the majority in the lowest levels. Convective cloud regions contain much higher LWC and drop concentrations than stratiform regions at all levels, and higher LWC in the middle and upper regions. Stratiform clouds contain higher IWC than convective clouds only at the lowest level. Irregular shaped ice particles are found in very high concentrations throughout both convective and stratiform cloud regions. A striking difference in particle shape in cirrus formed in situ, cirrus formed from maritime anvils and cirrus formed from continental anvils. Over 50% of the mass of in situ cirrus ice particles is composed of bullet rosettes, while bullet rosettes are virtually non-existent in maritime and tropical anvils. Tropical anvils are composed of mostly singular, plates, capped columns, and blocky irregular shapes, while continental anvils have a much higher percentage of aggregates, some of which are chains of small spheroidal particles that appear to result from homogeneous freezing of drops. A correlation between high electric fields in continental anvils and the formation of aggregates is hypothesized.

Source record↗

NASA NTRS1986

Determination of cloud fields from analysis of HIRS2/MSU sounding data

IR and microwave remote sensing data collected with the HIRS2 and MSU sensors on the NOAA polar-orbiting satellites were evaluated for their effectiveness as bases for determining the cloud cover and cloud physical characteristics. Techniques employed to adjust for day-night alterations in the radiance fields are described, along with computational procedures applied to compare scene pixel values with reference values for clear skies. Sample results are provided for the mean cloud coverage detected over South America and Africa June 1979, with attention given to concurrent surface pressure and cloud top pressure values.

Susskind, J.↗

NASA NTRS1974

Altitude determination and descriptive analysis of clouds on ERTS-1 multispectral photography

A simple method to determine the approximate altitude of clouds is described, with the objective of refining their classification using only marginal data from the photographs. Results of the application of this method on photographs of the Goajira Peninsula, Paraguana Peninsula and the Central Coast of Venezuela are presented. Here, the altitudes computed are used to classify clouds and to identify the genus of others without typical form. Instability of air masses through clouds vertical development, and wind direction as well as other local climatic characteristics such as moisture content, loci of condensation, area, etc. are determined using repetitive coverage for the time interval of the photography. Applications for the regional and urban planning (including airport location and flights schedule) and natural resources evaluation are suggested.

Albrizzio, C.↗

NASA NTRS1996

Cloud Properties Derived from Surface-Based Near-Infrared Spectral Transmission

Surface based near-infrared cloud spectral transmission measurements from a recent precipitation/cloud physics field study are used to determine cloud physical properties and relate them to other remote sensing and in situ measurements. Asymptotic formulae provide an effective means of closely approximating the qualitative and quantitative behavior of transmission computed by more laborious detailed methods. Relationships derived from asymptotic formulae are applied to measured transmission spectra to test objectively the internal consistency of data sets acquired during the field program and they confirmed the quality of the measurements. These relationships appear to be very useful in themselves, not merely as a quality control measure, but also a potentially valuable remote-sensing technique in its own right. Additional benefits from this analysis have been the separation of condensed water (cloud) transmission and water vapor transmission and the development of a method to derive cloud liquid water content.

Pilewskie, Peter↗

NASA NTRS2014

Climate Analytics as a Service

Climate science is a big data domain that is experiencing unprecedented growth. In our efforts to address the big data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). CAaaS combines high-performance computing and data-proximal analytics with scalable data management, cloud computing virtualization, the notion of adaptive analytics, and a domain-harmonized API to improve the accessibility and usability of large collections of climate data. MERRA Analytic Services (MERRA/AS) provides an example of CAaaS. MERRA/AS enables MapReduce analytics over NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of key climate variables. The effectiveness of MERRA/AS has been demonstrated in several applications. In our experience, CAaaS is providing the agility required to meet our customers' increasing and changing data management and data analysis needs.

big data↗

NASA NTRS2016

GC31G-1182: Opennex, a Private-Public Partnership in Support of the National Climate Assessment

The NASA Earth Exchange (NEX) is a collaborative computing platform that has been developed with the objective of bringing scientists together with the software tools, massive global datasets, and supercomputing resources necessary to accelerate research in Earth systems science and global change. NEX is funded as an enabling tool for sustaining the national climate assessment. Over the past five years, researchers have used the NEX platform and produced a number of data sets highly relevant to the National Climate Assessment. These include high-resolution climate projections using different downscaling techniques and trends in historical climate from satellite data. To enable a broader community in exploiting the above datasets, the NEX team partnered with public cloud providers to create the OpenNEX platform. OpenNEX provides ready access to NEX data holdings on a number of public cloud platforms along with pertinent analysis tools and workflows in the form of Machine Images and Docker Containers, lectures and tutorials by experts. We will showcase some of the applications of OpenNEX data and tools by the community on Amazon Web Services, Google Cloud and the NEX Sandbox.

datasets↗

NASA NTRS1988

A modified ATI technique for nowcasting convective rain volumes over areas

This paper explores the applicability of the area-time-integral (ATI) technique for the estimation of the growth portion only of a convective storm (while the rain volume is computed using the entire life history of the event) and for nowcasting the total rain volume of a convective system at the stage of its maximum development. For these purposes, the ATIs were computed from the digital radar data (for 1981-1982) from the North Dakota Cloud Modification Project, using the maximum echo area (ATIA) no less than 25 dBz, the maximum reflectivity, and the maximum echo height as the end of the growth portion of the convective event. Linear regression analysis demonstrated that correlations between total rain volume or the maximum rain volume versus ATIA were the strongest. The uncertainties obtained were comparable to the uncertainties which typically occur in rain volume estimates obtained from radar data employing Z-R conversion followed by space and time integration. This demonstrates that the total rain volume of a storm can be nowcasted at its maximum stage of development.

Makarau, Amos↗

NASA NTRSDate not supplied

Cloud-Based Time Series Analysis of Extremes: Use Cases and Applications

"Extreme weather events, such as hurricanes, tornadoes, floods, droughts, heatwaves, and blizzards, can cause widespread damage, disrupting ecosystems, agricultural production, and economies. The frequency and intensity of these events have been increasing, likely due to climate change, raising concerns and the need for more accurate analysis and predictions. NASA's Goddard Earth Sciences Data and Information Services Center (GES DISC) has migrated its long-term historical datasets, including precipitation data from MERRA-2 reanalysis, GLDAS land data assimilation, and IMERG satellite observations, to the cloud. This cloud-stored data enables scientists and researchers to utilize cloud computing for advanced modeling and forecasting of extreme weather events, eliminating the need to download large datasets. In this presentation, we will provide an overview of the cloud-based data and services managed by GES DISC; demonstrate methods for accessing and analyzing time series data stored in the cloud; and compare results across various datasets to address critical questions related to extreme precipitation. We will present use cases including: 1. Determining the average total precipitation in California during January and February from 2000 to 2024, and identifying anomalous precipitation in 2021. 2. Calculating the 10, 20, 50, and 100-year return periods for maximum daily rainfall based on 25 years of historical precipitation data (2000-2024) for Maryland.

time series↗

NASA NTRS2019

Cloud Giovanni: Reining in Costs and Improving Performance with Analytical Data Stores Using Scalable Serverless Architecture

Giovanni is the Geospatial Interactive Online Visualization ANd aNalysis Infrastructure developed at NASA GES DISC which provides a simple and intuitive way to visualize, analyze, and access vast amounts of Earth science data. It receives large number of user requests each day for a variety of analysis and visualization services, which leads to the big data challenge of serving gradually increasing large data volumes with diverse statistical algorithms. We hereby propose a multi-dimensional accumulation method which provides fast and cost-efficient cloud analysis for diverse services including both area averaging and time averaging. This method involves the weighted volume integration over multiple variable dimensions (time and space), and is implemented in AWS using Athena providing serverless and highly scalable data analysis. Compared to the standard method, this approach dramatically reduced the computational time by order of magnitude with a minimal AWS cost incurred. For example, for a benchmark of 10-year area averaging over the 1x1 degree daily variable, the computational time was reduced from minutes to seconds, and the Athena cost is only $5 for 100,000 requests.

Zhang, Hailiang↗

NASA NTRS1989

Adaptation of the ISCCP cloud detection algorithm to combined AVHRR and SMMR arctic data

The International Satellite Cloud Climatology Project (ISCCP) cloud detection algorithm is applied to artic data, and modifications are suggested. Both Advanced Very High Resolution Radiometer (AVHRR) and Scanning Multichannel Microwave Radiometer (SMMR) data are examined. Synthetic AVHRR and SMMR data are also generated. Modifications suggested include the use of snow and ice data sets for the estimation of surface parameters, additional AVHRR channels, and surface class characteristic values when clear sky values cannot be obtained. Greatest improvement in computed cloud fraction is realized over snow and ice surfaces; over other surfaces all versions perform similarly. Since the use of SMMR for surface analysis increases the computational burden, its use may be justified only over snow and ice-covered regions.

Key, J.↗

NASA NTRS2014

MERRA Analytic Services: Meeting the Big Data Challenges of Climate Science Through Cloud-enabled Climate Analytics-as-a-service

Climate science is a Big Data domain that is experiencing unprecedented growth. In our efforts to address the Big Data challenges of climate science, we are moving toward a notion of Climate Analytics-as-a-Service (CAaaS). We focus on analytics, because it is the knowledge gained from our interactions with Big Data that ultimately produce societal benefits. We focus on CAaaS because we believe it provides a useful way of thinking about the problem: a specialization of the concept of business process-as-a-service, which is an evolving extension of IaaS, PaaS, and SaaS enabled by Cloud Computing. Within this framework, Cloud Computing plays an important role; however, we it see it as only one element in a constellation of capabilities that are essential to delivering climate analytics as a service. These elements are essential because in the aggregate they lead to generativity, a capacity for self-assembly that we feel is the key to solving many of the Big Data challenges in this domain. MERRA Analytic Services (MERRAAS) is an example of cloud-enabled CAaaS built on this principle. MERRAAS enables MapReduce analytics over NASAs Modern-Era Retrospective Analysis for Research and Applications (MERRA) data collection. The MERRA reanalysis integrates observational data with numerical models to produce a global temporally and spatially consistent synthesis of 26 key climate variables. It represents a type of data product that is of growing importance to scientists doing climate change research and a wide range of decision support applications. MERRAAS brings together the following generative elements in a full, end-to-end demonstration of CAaaS capabilities: (1) high-performance, data proximal analytics, (2) scalable data management, (3) software appliance virtualization, (4) adaptive analytics, and (5) a domain-harmonized API. The effectiveness of MERRAAS has been demonstrated in several applications. In our experience, Cloud Computing lowers the barriers and risk to organizational change, fosters innovation and experimentation, facilitates technology transfer, and provides the agility required to meet our customers' increasing and changing needs. Cloud Computing is providing a new tier in the data services stack that helps connect earthbound, enterprise-level data and computational resources to new customers and new mobility-driven applications and modes of work. For climate science, Cloud Computing's capacity to engage communities in the construction of new capabilies is perhaps the most important link between Cloud Computing and Big Data.

Data Analytics↗

DOE OSTI2025

IRIS-MEMFLOW: Data Flow-Enabled Portable Memory Orchestration in IRIS Runtime for Diverse Heterogeneity

Task-based programming models and execution paradigms provide a means to decompose a computation by expressing it as a graph in which each node represents a specific computation operating on memory objects and the edges define the dependencies in the execution flow. In this execution model, independent nodes in the graph can be executed concurrently in different computing devices, making it suitable for heterogeneous systems in which computing devices with different architectures coexist. However, careful memory orchestration across heterogeneous devices is needed because copies of the same memory object may reside in multiple devices during execution. Manually ensuring such an orchestration is quite challenging. Not only must an application developer guard against race conditions, but they must also optimize data movement between the host and devices because unnecessary data movement significantly impacts performance. To mitigate these challenges, we enhance the IRIS heterogeneous runtime and introduce IRIS-MEMFLOW–a data flow–enabled portable memory abstraction for seamlessly orchestrating memory in diverse heterogeneous computing environments. By using data-flow analysis, IRIS-MEMFLOW guards against race conditions while multiple heterogeneous devices access memory objects. IRIS-MEMFLOW also optimizes data movement between the host and devices without manual intervention. As a result, IRIS provides improved programming productivity, performance, and portability for multidevice heterogeneous executions in high-performance computing and cloud systems that run diverse architectures from different vendors. The efficacy of IRIS-MEMFLOW is evaluated through experiments that show its capability in terms of programming productivity, multidevice heterogeneity, portability, and low overhead versus the state of the art.

Monil, M. A. H. [ORNL] (ORCID:0000000334194037)↗

NASA NTRS1999

Sensitivity Analysis of Aerosol Retrieval Algorithms Using Two-Channel Satellite Radiance Data

We discuss the methodology of interpreting channel 1 and 2 AVHRR radiance data to retrieve tropospheric aerosol properties over the ocean and describe a detailed analysis of the sensitivity of monthly average retrievals to the assumed aerosol models. We use real AVHRR data and accurate numerical techniques for computing single and multiple scattering and spectral absorption of light in the vertically inhomogeneous atmosphere-ocean system. Our analysis shows that two-channel algorithms can provide significantly more accurate retrievals of the aerosol optical thickness than one-channel algorithms and that imperfect cloud screening is the largest source of errors in the retrieved optical thickness. Both underestimating and overestimating aerosol absorption as well as strong variability of the aerosol refractive index may lead to regional and/or seasonal biases in optical thickness retrievals. The Angstrom exponent appears to be the most invariant aerosol size characteristic and should be retrieved along with optical thickness as the second aerosol parameter.

Geogdzhayev, Igor V.↗

NASA NTRSDate not supplied

TPSAS-NF1676L-11082-DND

Analysis of one decade of radar-lidar and Geostationary Operational Environmental Satellite (GOES) observations at the Atmospheric Radiation Measurement (ARM) Program Southern Great Plains (SGP) site reveals that there is excellent agreement in the long-term mean cloud fractions (CF) derived from the surface and GOES data, and the CF is independent of temporal resolution and spatial scales for grid boxes of size 0.5o to 2.5Â°. When computed over a 0.5-hr (4-hr) period, cloud frequency of occurrence (FREQ) and amount when present (AWP) derived from the point surface data agree very well with the same quantities determined from GOES for a 0.5o (2.5o) region centered on the ARM SGP site. The values of FREQ (AWP) derived from the radar-lidar observations at a given altitude increase (decrease) as the averaging period increases from 5 min to 6 hours. Similarly, CF at a given altitude increases as the vertical resolution increases from 90 to 1000 m. The profiles of CF have distinct bimodal vertical distributions with a lower peak between 1 and 2 km and a higher one between 8 and 11 km. The 10-year mean total CF, 46.9%, varies seasonally from a summer minimum of 39.8% to a maximum of 54.6% during the winter. The annual mean CF is 1-2% less than that from previous studies, ~48-49%, because fewer clouds occurred during 2005 and 2006, especially during winter. The differences in single- and multi-layered CFs between this study and an earlier analysis can be explained by the different temporal resolutions used in the two studies where single-layered CFs decrease but multilayered CFs increase from a 5-min resolution to a 1-hr resolution. The vertical distribution of nighttime GOES high-cloud tops agrees well with surface observations, but during daytime, fewer high clouds are retrieved by the GOES analysis than seen from the surface observations. The frequencies of occurrence for both daytime and nighttime GOES low-cloud tops are significantly higher than surface observations, but the cloud fractions are in good agreement.

B Xi↗

NASA NTRSDate not supplied

Web-based Visualization and Analytics of Petascale Data: Equity as a Tide that Lifts All Boats

Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Data visualization↗

NASA NTRS1976

High resolution radiometric measurements of convective storms during the GATE experiment

Using passive microwave data from the NASA CV-990 aircraft and radar data collected during the Global Atmospheric Research Program Atlantic Tropical Experiment (GATE), an empirical model was developed relating brightness temperatures sensed at 19.35 GHz to surface rainfall rates. This model agreed well with theoretical computations of the relationship between microwave radiation and precipitation in the tropics. The GATE aircraft microwave data was then used to determine the detailed structure of convective systems. The high spatial resolution of the data permitted identification of individual cells which retained unique identities throughout their lifetimes in larger cloud masses and allowed analysis of the effects of cloud merger.

Fowler, G.↗

NASA NTRS1982

Satellite measurements of cloud reflectance and optical thickness

The cloud reflectance and the optical thickness have been computed with the aid of atmospheric models from the first eleven months (April 1970-February 1971) of backscattered ultraviolet (BUV) data over the Pacific Ocean. Both the cloud and the optical thickness are derived from the 380 nm channel by assuming that the entire IFOV (200 km) is filled by stratiform clouds. The analysis shows a large variability in the cloud reflectance in both the intertropic conversion zone (ITCZ) and the high latitudes. It is also found that for 90 percent of the time in the tropics, the clouds have an optical thickness of less than 10. An analysis of clouds with optical thickness between 10 and 20 shows that in the tropics the minimum frequency coincides with the dry zone at 2.5 deg S and the maximum frequency for clouds of optical thickness 10-20 is greater in summer than in winter and irrespective of the season; 50 percent of the time the clouds have an optical thickness less than 13.

Ahmad, Z.↗