Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33

Diversity in computing technologies and strategies for dynamic resource allocation

Here, High Energy Physics (HEP) is a very data intensive and trivially parallelizable science discipline. HEP is probing nature at increasingly finer details requiring ever increasing computational resources to process and analyze experimental data. In this paper, we discuss how HEP provisioned resources so far using Grid technologies, how HEP is starting to include new resource providers like commercial Clouds and HPC installations, and how HEP is transparently provisioning resources at these diverse providers.

97 MATHEMATICS AND COMPUTING↗

Final scientific report for DOE award title: Improving the Representation of Ice Sedimentation Rates in Global Climate Models

It is well known that cirrus clouds play a major role in regulating the earth’s climate, but the details of how this works are just beginning to be understood. This project targeted the main property of cirrus clouds that influence climate processes; the ice fall speed. That is, this project improves the representation of the mass-weighted ice particle fall velocity, V m , in climate models, used to predict future climate on global and regional scales. Prior to 2007, the dominant sizes of ice particles in cirrus clouds were poorly understood, making it virtually impossible to predict how cirrus clouds interact with sunlight and thermal radiation. Due to several studies investigating the performance of optical probes used to measure the ice particle size distribution (PSD), as well as the remote sensing results from our last ARM project, it is now well established that the anomalously high concentrations of small ice crystals often reported prior to 2007 were measurement artifacts. Advances in the design and data processing of optical probes have greatly reduced these ice artifacts that resulted from the shattering of ice particles on the probe tips and/or inlet tube, and PSD measurements from one of these improved probes (the 2-dimensional Stereo or 2D-S probe) are utilized in this project to parameterize V m for climate models. Our original plan in the proposal was to parameterize the ice PSD (in terms of temperature and ice water content) and ice particle mass and projected area (in terms of mass- and area-dimensional power laws or m-D/A-D expressions) since these are the microphysical properties that determine V m , and then proceed to calculate V m from these parameterized properties. But the 2D-S probe directly measures ice particle projected area and indirectly estimates ice particle mass for each size bin. It soon became apparent that the original plan would introduce more uncertainty in the V m calculations than simply using the 2D-S measurements to directly calculate V m . By calculating V m directly from the measured PSD, ice particle projected area and estimated mass, more accurate estimates of V m are obtained. These V m values were then parameterized for climate models by relating them to (1) sampling temperature and ice water content (IWC) and (2) the effective diameter (D e ) of the ice PSD. Parameterization (1) is appropriate for climate models having single-moment microphysical schemes whereas (2) is appropriate for double-moment microphysical schemes and yields more accurate V m estimates. These parameterizations were developed for tropical cirrus clouds, Arctic cirrus, mid-latitude synoptic cirrus and mid-latitude anvil cirrus clouds based on field campaigns in these regions. An important but unexpected result of this research was the discovery of microphysical evidence indicating the mechanisms by which ice crystals are produced in cirrus clouds. This evidence, derived from PSD measurements, indicates that homogeneous freezing ice nucleation dominates in mid-latitude synoptic cirrus clouds, whereas heterogeneous ice nucleation processes dominate in mid-latitude anvil cirrus. Based on these findings, D e was parameterized in terms of temperature (T) for conditions dominated by (1) homo- and (2) heterogeneous ice nucleation. From this, an experiment was designed for global climate models (GCMs). The net radiative forcing from cirrus clouds may be affected by the means ice is produced (homo- or heterogeneously), and this net forcing contributes to climate sensitivity (i.e. the change in mean global surface temperature resulting from a doubling of CO 2 ). The objective of this GCM experiment was to determine how a change in ice nucleation mode affects the predicted global radiation balance. In the first simulation (Run 1), the D e -T relationship for homogeneous nucleation is used at all latitudes, while in the second simulation (Run 2), the D e -T relationship for heterogeneous nucleation is used at all latitudes. For both runs, V m is calculated from D e . Two GCMs were used; the Community Atmosphere Model version 5 (CAM5) and a European GCM known as ECHAM5 (thanks to our European colleagues who collaborated with us). Similar results were obtained from both GCMs in the Northern Hemisphere mid-latitudes, with a net cooling of ~ 1.0 W m -2 due to heterogeneous nucleation, relative to Run 1. The mean global net cooling was 2.4 W m -2 for the ECHAM5 GCM while CAM5 produced a mean global net cooling of about 0.8 W m -2 . This dependence of the radiation balance on nucleation mode is substantial when one considers the direct radiative forcing from a CO 2 doubling is 4 W m -2 . The differences between GCMs in mean global net cooling estimates may demonstrate a need for improving the representation of cirrus clouds in GCMs, including the coupling between microphysical and radiative properties. Unfortunately, after completing this GCM experiment, we learned from the company that provided the 2D-S microphysical data that the data was corrupted due to a computer program coding problem. Therefore the microphysical data had to be reprocessed and reanalyzed, and the GCM experiments were redone under our current ASR project but using an improved experimental design.

54 ENVIRONMENTAL SCIENCES↗

Description of the Three-Dimensional Large-Scale Forcing Data from the 3D Constrained Variational Analysis (VARANAL3D)

This technical report introduces a Three-Dimensional Constrained Variational Analysis (3DCVA) (Tang and Zhang 2015) and its product of three-dimensional large-scale forcing data to drive single-column models (SCM), cloud-resolving models (CRM), and large-eddy simulation (LES) models, and to evaluate model results. The 3DCVA algorithm is an extension of the original 1D constrained variational analysis (1DCVA) (Zhang and Lin 1997, Zhang et al. 2001). The three-dimensional structure of the forcing data allows studies of spatial variation of the large-scale forcing fields and tests of physical parameterizations across scales. In the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) user facility, the 3D forcing data are assigned the datastream name varanal3d. In this technical report, 3DCVA will be used to refer to the algorithm, while VARANAL3D will be used to refer to the data product.

54 ENVIRONMENTAL SCIENCES↗

Development of Ensemble Neural Network Convection Parametrizations for Climate Models Using ARM Data

The novel neural network (NN) approach has been formulated and used for development of a NN ensemble stochastic convection parametrization for climate models. This fast parametrization is built based on data from Cloud Resolving Model (CRM) simulations initialized with and forced by TOGA-COARE data. The SAM (System for Atmospheric Modeling), developed by D. Randall, M. Khairoutdinov, and their collaborators, has been used for CRM simulations. The observational data are also used for validation of model simulations. The SAM-simulated data have been averaged and projected onto the GCM space of atmospheric states to implicitly define a stochastic convection parametrization. This parametrization is emulated using an ensemble of NNs. An ensemble of NNs with different NN parameters has been trained and tested. The inherent uncertainty of the stochastic convection parametrization derived in such a way is estimated. Due to these inherent uncertainties, NN ensemble is used to constitute a stochastic NN convection parametrization. The developed NN convection parametrization have been validated in a diagnostic CAM (CAM-NN) run vs. the control CAM run. Actually, CAM inputs have been used, at every time step of the control/original CAM integration, for parallel calculations of the NN convection parametrization (CAM-NN) to produce its outputs as a diagnostic byproduct. Total precipitation (P) and cloudiness (CLD) time series, diurnal cycles, and P and CLD distributions for the large Tropical Pacific Ocean for the parallel CAM-NN and CAM runs show similarity and consistency with the NCEP reanalysis. The P and CLD distributions for the tropical area for the parallel runs have been analyzed first for the TOGA-COARE boreal winter season (November 1992 through February 1993) and then for the winter seasons of the follow-up parallel decadal simulations. The obtained results are encouraging and practically meaningful. They show the validity of the NN approach. This constitutes an important practical conclusion of the study: the obtained results on NN ensembles as a stochastic physics parametrization show a realistic possibility of development of NN convection parametrization for climate (and NWP) models based on learning cloud physics from CRM/SAM simulated data.

97 MATHEMATICS AND COMPUTING↗

Scalable pCT Image Reconstruction Delivered as a Cloud Service

We describe a cloud-based medical image reconstruction service designed to meet a real-time and daily demand to reconstruct thousands of images from proton cancer treatment facilities worldwide. Rapid reconstruction of a three-dimensional Proton Computed Tomography (pCT) image can require the transfer of 100 GB of data and use of approximately 120 GPU-enabled compute nodes. The nature of proton therapy means that demand for such a service is sporadic and comes from potentially hundreds of clients worldwide. We thus explore the use of a commercial cloud as a scalable and cost-efficient platform for pCT reconstruction. To address the high performance requirements of this application we leverage Amazon Web Services' GPU-enabled cluster resources that are provisioned with high performance networks between nodes. To support episodic demand, we develop an on-demand multi-user provisioning service that can dynamically provision and resize clusters based on image reconstruction requirements, priorities, and wait times. We compare the performance of our pCT reconstruction service running on commercial cloud resources with that of the same application on dedicated local high performance computing resources. We show that we can achieve scalable and on-demand reconstruction of large scale pCT images for simultaneous multi-client requests, processing images in less than 10 minutes for less than $10 per image.

Cloud computing↗

Report for the ASCR Workshop on Visualization for Scientific Discovery, Decision-Making, and Communication

Visualization—the use of visual elements to explore data, form hypotheses, or convey conclusions—is an integral part of the scientific process. Starting from an initial exploration of new data to illustrating outcomes for the general public, visualization is one of the most intuitive and powerful modes of communication. With the explosion of new data sources and types, unprecedented volumes of data, and new technologies, such as virtual reality (VR) and artificial intelligence (AI), visualization has become increasingly essential but also ever more challenging. The Department of Energy’s (DOE) Office of Advanced Scientific Computing Research (ASCR) sponsored a Basic Research Needs workshop in January 2022 to understand the major opportunities and grand challenges in visualization tools and technologies for scientific computing as well as for DOE-relevant applications and goals in general. The workshop identified five priority research directions (PRDs) for visualization to support scientific discovery, decision making, and communication. The first three PRDs describe interconnected research themes addressing the need for new techniques to deal with complex data, uncertainty, and interpretability (PRD 1); the need for scalable and interoperable software stacks (PRD 2); and the challenges and opportunities inherent in new technologies, such as VR, cloud, or exascale computing (PRD 3). The remaining two PRDs describe foundational research themes that recognize the potential of visualizations to provide equitable access to information and to strengthen the scientific discourse (PRD 4); and the need to consider human factors when designing visualizations (PRD 5). Collectively, these PRDs form the pillars for a coherent, long-term research and development strategy in Visualization for Scientific Discovery, Decision-Making, and Communication in the context of the Office of Science’s mission scope.

97 MATHEMATICS AND COMPUTING↗

INTEGRATED WORKFLOW MANAGEMENT FOR PARTICLE ACCELERATOR SIMULATION

Supercomputing systems are used for a wide range of computationally demanding tasks in many fields of science and engineering. They play a key role in numerical simulation, in which mathematical models are computed in order to simulate the behavior of physical systems. Scientists and engineers that use supercomputers for numerical simulation often have their productivity limited by the need to manually organize and manage extremely large amounts of data that are often produced and consumed by the software programs run on these systems. Recognizing these limitations, Kitware Inc. (Clifton Park, NY) and SLAC National Accelerator Laboratory (Menlo Park, CA) are developing an advanced software platform that can reduce the cognitive overhead required by knowledge workers when using supercomputers for numerical simulation. Phase I of the project is complete and includes the development of new capabilities for organizing simulation project files, improvements to the user interface and overall usability, and deployment of a “middle tier” server to sit between user desktop machines and supercomputers to offload much of the data management workload. The project also developed prototype software for executing sequences of numerical simulations, and a prototype for migrating supercomputing software to cloud-based computing systems to provide a potential alternative to supercomputers with different logistical and price-to-performance tradeoffs.

Tourtellott, John↗

Closed Loop Geothermal Working Group: GeoCLUSTER App, Subsurface Simulation Results, and Publications

To better understand the heat production, electricity generation performance, and economic viability of closed-loop geothermal systems in hot-dry rock, the Closed-Loop Geothermal Working Group -- a consortium of several national labs and academic institutions has tabulated time-dependent numerical solutions and levelized cost results of two popular closed-loop heat exchanger designs (u-tube and co-axial). The heat exchanger designs were evaluated for two working fluids (water and supercritical CO2) while varying seven continuous independent parameters of interest (mass flow rate, vertical depth, horizontal extent, borehole diameter, formation gradient, formation conductivity, and injection temperature). The corresponding numerical solutions (approximately 1.2 million per heat exchanger design) are stored as multi-dimensional HDF5 datasets and can be queried at off-grid points using multi-dimensional linear interpolation. A Python script was developed to query this database and estimate time-dependent electricity generation using an organic Rankine cycle (for water) or direct turbine expansion cycle (for CO2) and perform a cost assessment. This document aims to give an overview of the HDF5 database file and highlights how to read, visualize, and query quantities of interest (e.g., levelized cost of electricity, levelized cost of heat) using the accompanying Python scripts. Details regarding the capital, operation, and maintenance and levelized cost calculation using the techno-economic analysis script are provided. This data submission will contain results from the Closed Loop Geothermal Working Group study that are within the public domain, including publications, simulation results, databases, and computer codes. GeoCLUSTER is a Python-based web application created using Dash, an open-source framework built on top of Flask that streamlines the building of data dashboards. GeoCLUSTER provides users with a collection of interactive methods for streamlining the exploration and visualization of an HDF5 dataset. The GeoCluster app and database are contained in the compressed file geocluster_vx.zip, where the "x" refers to the version number. For example, geocluster_v1.zip is Version 1 of the app. This zip file also contains installation instructions. **To use the GeoCLUSTER app in the cloud, click the link to "GeoCLUSTER on AWS" in the Resources section below. To use the GeoCLUSTER app locally, download the geocluster_vx.zip to your computer and uncompress this file. When uncompressed this file comprises two directories and the geocluster_installation.pdf file. The geo-data app contains the HDF5 database in condensed format, and the GeoCLUSTER directory contains the GeoCLUSTER app in the subdirectory dash_app, as app.py. The geocluster_installation.pdf file provides instructions on installing Python, the needed Python modules, and then executing the app.

15 GEOTHERMAL ENERGY↗

TWST Cloud Properties (COD, droplet effective radius, and thermodynamic phase). SAIL, AMF2, 2022-2023.

The dataset consists of calibrated zenith shortwave spectral radiances (440-1700 nm) and cloud optical depth (COD) retrievals approximately every second. Retrievals of cloud droplet effective radius and thermodynamic phase are currently done offline, but some of these retrievals are included in the dataset. Effective radius and phase are included for days with periods of sufficiently high COD. Effective radius and phase retrievals in the data set were computed at approximately 5-10 second intervals to reduce computation time. The retrievals can be performed at the full sample rate. The field of view is 0.5 degree.

54 ENVIRONMENTAL SCIENCES↗

SeaWiFS Technical Report Series. Volume 7: Cloud screening for polar orbiting visible and infrared (IR) satellite sensors

Methods for detecting and screening cloud contamination from satellite derived visible and infrared data are reviewed in this document. The methods are applicable to past, present, and future polar orbiting satellite radiometers. Such instruments include the Coastal Zone Color Scanner (CZCS), operational from 1978 through 1986; the Advanced Very High Resolution Radiometer (AVHRR); the Sea-viewing Wide Field-of-view Sensor (SeaWiFS), scheduled for launch in August 1993; and the Moderate Resolution Imaging Spectrometer (IMODIS). Constant threshold methods are the least demanding computationally, and often provide adequate results. An improvement to these methods are the least demanding computationally, and often provide adequate results. An improvement to these methods is to determine the thresholds dynamically by adjusting them according to the areal and temporal distributions of the surrounding pixels. Spatial coherence methods set thresholds based on the expected spatial variability of the data. Other statistically derived methods and various combinations of basic methods are also reviewed. The complexity of the methods is ultimately limited by the computing resources. Finally, some criteria for evaluating cloud screening methods are discussed.

Darzi, Michael↗

Recent advances in integrated hydrologic models: Integration of new domains

Over the past several decades, hydrologic models have advanced from independent models of the surface and subsurface to integrated models that can capture the terrestrial hydrologic cycle within one framework. In recent years, these coupled frameworks have seen the inclusion of biogeochemical processes, ecohydrology, sedimentation and erosion, cold region hydrology, anthropogenic activities, and atmospheric processes. This expansion is the result of increased computational, data, and modeling capabilities and capacities, as well as improved understanding of the processes that drive these integrated systems. Here, in this study, we review these recent advances to integrate new processes and systems into existing terrestrial hydrologic models and highlight the significant challenges and opportunities that remain. We identify that with so many models currently available and in development, selecting the most appropriate model is difficult, and we suggest a path for new or novice modelers to find the most appropriate code based on their needs. In addition, data required to parameterize and calibrate these models can often constrain their applicability and usefulness. However, advances in environmental sensors and measurement technology, in addition to data assimilation of non-traditional data (e.g. remote sensing, qualitative data) are providing new ways of addressing this issue. As we expand hydrologic models to integrate more processes and systems, our computational demands also increase. Recent and emerging advances in computational platforms, including cloud and quantum computing, in addition to the use of machine learning to capture some processes, will continue to support the use of increasingly larger and more complex, process-based models. Finally, we highlight that it is critical to develop state-of-the-science models that are accessible to all model users, not just those applied for research and development. We encourage continued development of diverse modeling platforms, considering the user needs, data availability, and computational resources.

54 ENVIRONMENTAL SCIENCES↗

Development of a Scalable Risk-informed Predictive Maintenance Cloud-based Strategy at Nuclear Power Plants

The fact that light-water reactor operation and maintenance costs are prohibitively expensive and contribute to the premature decommissioning of nuclear power plants is partly due to how the equipment is monitored. In recent years, cloud computing has emerged as a dominant technology, as its low cost, computing and storage adaptability, and ability to host applications across numerous virtual infrastructures potentially make it a cost-effective alternative to onsite storage and diagnostics. In this paper, a technological assessment is carried out on a provisional cloud deployment architecture for a nuclear power plant predictive monitoring system. This cloud-based monitoring system would enable maintenance and diagnostic analysts and other authorized plant users to remotely monitor equipment functionality, thus enabling early fault detection and effective predictive maintenance practices. To provide data processing and storage, sensor device networking, and database management, the Microsoft Azure cloud platform is utilized as part of the proposed cloud architecture; however, this analysis could be extended to other cloud computing service providers as well. The focus of this paper is on application of cloud resources for enabling predictive maintenance, identification of technological hurdles associated with moving to a cloud-computing-based architecture, and potential benefits from moving to a centralized cloud system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The special effort processing of FGGE data

The basic FGGE level IIb data set was enhanced. It focused on removing deficiencies in the objective methods of quality assurance, removing efficiencies in certain types of operationally produced satellite soundings, and removing deficiencies in certain types of operationally produced cloud tracked winds. The Special Effort was a joint NASA-NOAA-University of Wisconsin effort. The University of Wisconsin installed an interactive McIDAS capability on the Amdahl computer at the Goddard Laboratory of Atmospheric Sciences (GLAS) with one interactive video terminal at Goddard and the other at the World Weather Building. With this interactive capability a joint processing effort was undertaken to reprocess certain FGGE data sets. NOAA produced a specially edited data set for the special observing periods (SOPs) of FGGE. NASA produced an enhanced satellite sounding data set for the SOPs while the University of Wisconsin produced an enhanced cloud tracked wind set from the Japanese geostationary satellite images.

Source record↗

Stratus: An interactive steady state mixed layer model for personal computers

A steady-state, horizontally homogeneous, cloud-topped marine boundary layer model based primarily on the work of Lilly (1968) and Schubert et al., (1979) is presented. The conservative thermodynamic variables are equivalent potential temperature, theta(sub e), and total water mixing ratio, q + l. Some of the differences between this and Lilly's (1968) model are: radiation is allowed to penetrate into the boundary layer; cloud top values of longwave radiation, equivalent potential temperature, and water vapor mixing ratio are linear functions of height derived from climatological data at California coastal stations; and the closure assumption assumes a weighted average of Lilly's (1968) maximum and minimum entrainment theories. This model was programmed in FORTRAN and will run interactively on an IBM-compatible personal computer. The program allows the user to specify the geographical location, the wind speed, the sea-surface temperature, the large scale horizontal divergence, and the initial guess for cloud top height. Output includes the steady state values of cloud top and cloud base height, mixed layer equivalent potential temperature and total water mixing ratio, and the associated convective and radiative fluxes.

Guinn, Thomas A.↗

Two-stage atlas subset selection in multi-atlas based image segmentation

Purpose: Fast growing access to large databases and cloud stored data presents a unique opportunity for multi-atlas based image segmentation and also presents challenges in heterogeneous atlas quality and computation burden. This work aims to develop a novel two-stage method tailored to the special needs in the face of large atlas collection with varied quality, so that high-accuracy segmentation can be achieved with low computational cost. Methods: An atlas subset selection scheme is proposed to substitute a significant portion of the computationally expensive full-fledged registration in the conventional scheme with a low-cost alternative. More specifically, the authors introduce a two-stage atlas subset selection method. In the first stage, an augmented subset is obtained based on a low-cost registration configuration and a preliminary relevance metric; in the second stage, the subset is further narrowed down to a fusion set of desired size, based on full-fledged registration and a refined relevance metric. An inference model is developed to characterize the relationship between the preliminary and refined relevance metrics, and a proper augmented subset size is derived to ensure that the desired atlases survive the preliminary selection with high probability. Results: The performance of the proposed scheme has been assessed with cross validation based on two clinical datasets consisting of manually segmented prostate and brain magnetic resonance images, respectively. The proposed scheme demonstrates comparable end-to-end segmentation performance as the conventional single-stage selection method, but with significant computation reduction. Compared with the alternative computation reduction method, their scheme improves the mean and medium Dice similarity coefficient value from (0.74, 0.78) to (0.83, 0.85) and from (0.82, 0.84) to (0.95, 0.95) for prostate and corpus callosum segmentation, respectively, with statistical significance. Conclusions: The authors have developed a novel two-stage atlas subset selection scheme for multi-atlas based segmentation. It achieves good segmentation accuracy with significantly reduced computation cost, making it a suitable configuration in the presence of extensive heterogeneous atlases.

60 APPLIED LIFE SCIENCES↗

Calcium abundance variations in diffuse interstellar clouds

Observations are discussed which show that the Na I/H2 column-density ratio is relatively constant among different clouds where H2 is optically thin, but the Na I/Ca II column-density ratio varies. This implies that the gas-phase abundance of calcium is variable. It is proposed that the wide variation in the gas-phase calcium abundance can be explained by small variations in the calcium content of grains. The observed enhancement of the Ca II/Na I column-density ratio with cloud velocity is attributed to grain destruction in grain-grain collisions within the same cloud. The structure of shock waves propagating through the interstellar gas is computed, and the grain-destruction model is compared with observational data.

Jura, M.↗

Satellite observations of a monsoon depression

The exploration of a monsoon depression over Burma and the Bay of Bengal is discussed. Aircraft and satellite data were examined, with an emphasis on the Microwave Sounding Unit (MSU) aboard TIROS-N and the Scanning Multichannel Microwave Radiometer (SMMR) aboard Nimbus-7. The structure of the monsoon depression was found to be dominated by cumulus convection. The only systematic large scale behavior discerned was a propagation of the depression westward, and diurnal migration of contours of brightness temperature. These contours in the middle troposphere showed a gradient toward the north with the patterns migrating northward at night. From SMMR and dropwindsonde data, water vapor contents were found to be near 65 mm, increasing to more than 70 mm in the northeast Bay of Bengal. Cloud water contents reached about three mm. Rainfall rates exceeding 5.7 mm/h occurred over a small part of the storm area, while mean rainfall rates in areas of order 20,000 sq km reached approximately 0.5 mm/h. Measured MSU brightness temperatures were reconciled very well with dropwindsonde data and with airborne in situ observations of clouds (by photography) and hydrometeors (by radar). Diffuse scattering was determined to be important in computing brightness temperature.

Warner, C.↗