Virtualized Infiniband: Enabling HPC in the Cloud
Presentation describes virtualized infiniband and how it enables doing high performance computing level processing and file systems in a cloud environment. Generic benchmarks are included.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Presentation describes virtualized infiniband and how it enables doing high performance computing level processing and file systems in a cloud environment. Generic benchmarks are included.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 kilometers, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-kilometer scales are resolved in horizontal domains as large as 10,000 kilometers in two dimensions, and 1,000 x 1,000 square kilometers in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically-based parameterizations for climate models and numerical weather prediction models. A review of developments and applications of cloud models in the past, present and future will be presented in this talk. In particular, a new approach to using cloud-resolving models to study the interactions between clouds, precipitation and aerosols will be presented.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 km, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as l0,OOO km in two-dimensions, and 1,OOO x 1,OOO km2 in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically-based parameterizations for climate models and numerical weather prediction models. A review of developments and applications of cloud models in the past, present and future will be presented in this talk. In particular, a new approach to using cloud-resolving models to study the interactions between clouds, precipitation and aerosols will be presented.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 km, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as 10,000 km in two-dimensions, and 1,000 x 1,000 square kilometers in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically-based parameterizations for climate models and numerical weather prediction models. A review of developments and applications of cloud models in the past, present and future will be presented in this talk. In particular, a new approach to using cloud-resolving models to study the interactions between clouds, precipitation and aerosols will be presented.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 kilometers, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-kilometer scales are resolved in horizontal domains as large as 10,000 kilometers in two-dimensions, and 1,000 x 1,000 square kilometers in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. A review of developments, improvements and applications of cloud models (GCE and WRF) at Goddard will be presented in this talk. In particular, a new approach to using multi-scale modeling system to study the interactions between clouds, precipitation, aerosols and land will be presented.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 kilometers, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial-lkm scales are resolved in horizontal domains as large as 10,000 kilometers in two-dimensions, and 1,000 x 1,000 square kilometers in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. A review of developments, improvements and applications of cloud models (GCE and WRF) at Goddard wlll be is presented in this talk. In particular, a new approach to using multi-scale modeling system to study the interactions between clouds, precipitation, aerosols and land will be presented.
Numerical cloud models, which are based the non-hydrostatic equations of motion, have been extensively applied to cloud-scale and mesoscale processes during the past four decades. Because cloud-scale dynamics are treated explicitly, uncertainties stemming from convection that have to be parameterized in (hydrostatic) large-scale models are obviated, or at least mitigated, in cloud models. Global models will use the non-hydrostatic framework when their horizontal resolution becomes about 10 km, the theoretical limit for the hydrostatic approximation. This juncture will be reached one to two decades from now. Over the past generation, voluminous datasets on atmospheric convection have been accumulated from radar, instrumented aircraft, satellites, and rawinsonde measurements in field campaigns, enabling the detailed evaluation of models. Improved numerical methods have resulted in more accurate and efficient dynamical cores in models. Improvements have been made in the parameterizations of microphysical processes, radiation, boundary-layer effects, and turbulence; however, microphysical parameterizations remain a major source of uncertainty in all classes of atmospheric models. In recent years, exponentially increasing computer power has extended cloud-resolving-model integrations from hours to months, the number of computational grid points from less than a thousand to close to ten million. Three-dimensional models are now more prevalent. Much attention is devoted to precipitating cloud systems where the crucial 1-km scales are resolved in horizontal domains as large as 10,000 km in two-dimensions, and 1,000 x 1,000 sq km in three-dimensions. Cloud models now provide statistical information useful for developing more realistic physically based parameterizations for climate models and numerical weather prediction models. It is also expected that NWP and mesoscale model can be run in grid size similar to cloud resolving model through nesting technique. A review of developments, improvements and applications of cloud models (GCE and WRF) at Goddard will be presented in this talk. In particular, a new approach to using multi-scale modeling system to study the interactions between clouds, precipitation, aerosols and land will be presented.
The NASA CERES project has provided the climate quality observed TOA and computed surface fluxes to the scientific community. CERES instruments are onboard the Terra, Aqua, NPP, and NOAA-20 spacecraft. CERES uses MODIS and VIIRS imagers to retrieve cloud properties needed to convert CERES footprint radiance observations into fluxes using empirically derived angular directional models obtained during early CERES record. CERES utilizes geostationary sensors to infer the regional diurnal fluxes in between Terra and Aqua CERES observations. The imagers are also used to radiometrically scale the geostationary sensors (GEO) radiances to the imager calibration reference to ensure that the GEO derived cloud properties and broadband TOA fluxes are consistent in both space and time. Both the imager and GEO retrieved cloud properties are used to compute the surface fluxes. The Aqua-MODIS, SNPP-VIIRS, and NOAA-20 VIIRS afternoon imagers will also need to be radiometrically scaled to the same common calibration reference. Although all three imagers employ onboard solar diffusers, the calibration is not consistent over time due to instrument anomalies and ageing of the onboard calibrators. The CERES imager and GEO calibration group (IGCG) has been tasked to remove the imager channel calibration drifts for the next CERES reprocessing effort. The team will rely primarily on deep convective clouds, desert, and polar ice invariant targets to monitor the imager channel stability. The team uses ray-matched radiance pairs to radiometrically scale the SNPP and NOAA-20 VIIRS sensors to Aqua-MODIS. The radiometric scaling is further validated using geostationary imagers as transfer radiometers. The DSCOVR satellite was launched on February 25, 2015 and orbits around the L1 Lagrange point directly between the Earth and the sun. The EPIC instrument onboard DSCOVR employs a CCD array to image the Earth approximately every 2-hours. The EPIC sensor contains no onboard calibration systems. However, multiple inter-calibration studies have shown that the EPIC imager is very stable in time. This is likely due to the DSCOVR satellite being located about a 1.5M km from the Earth, where very little Earth reflected solar radiation degrades the optics. The excellent radiometric stability of EPIC allows the CERES IGCG to utilize the EPIC observations as a stable reference for monitoring the calibration stability of the three afternoon imagers, as well as to validate the radiometric scaling factors between them. Examples of the imager relative calibration using EPIC before and after radiometric scaling will be shown along with the results from the use of invariant targets to remove imager calibration drifts.
Visual observations of cloud cover are hindered at night due to inadequate illumination of the clouds. This usually leads to an underestimation of the average cloud cover at night, especially for the amounts of middle and high clouds, in climatologies on surface observations. The diurnal cycles of cloud amounts, if based on all the surface observations, are therefore in error, but they can be obtained more accurately if the nighttime observations are screened to select those made under sufficient moonlight. Ten years of nighttime weather observations from the northern hemisphere in December were classified according to the illuminance of moonlight or twilight on the cloud tops, and a threshold level of illuminance was determined, above which the clouds are apparently detected adequately. This threshold corresponds to light from a full moon at an elevation angle of 6 degrees or from a partial moon at higher elevation, or twilight from the sun less than 9 degrees below the horizon. It permits the use of about 38% of the observations made with the sun below the horizon. The computed diurnal cycles of total cloud cover are altered considerably when this moonlight criterion is imposed. Maximum cloud cover over much of the ocean is now found to be at night or in the morning, whereas computations obtained without benefit of the moonlight criterion, as in our published atlases, showed the time of maximum to be noon or early afternoon in many regions. Cloud cover is greater at night than during the day over the open oceans far from the continents, particularly in summer. However, near noon maxima are still evident in the coastal regions, so that the global annual average oceanic cloud cover is still slightly greater during the day than at night, by 0.3%. Over land, where daytime maxima are still obtained but with reduced amplitude, average cloud cover is 3.3% greater during the daytime. The diurnal cycles of total cloud cover we obtain are compared with those of ISCCP for a few regions; they are generally in better agreement if the moonlight criterion is imposed on the surface observations. Using the moonlight criterion, we have analyzed ten years (1982-1991) of surface weather observations over land and ocean, worldwide, for total cloud cover and for the frequency of occurrence of clear sky, fog and precipitation The global average cloud cover (average of day and night) is about 2% higher if we impose the moonlight criterion than if we use all observations. The difference is greater in winter than in summer, because of the fewer hours of darkness in the summer. The amplitude of the annual cycle of total cloud cover over the Arctic Ocean and at the South Pole is diminished by a few percent when the moonlight criterion is imposed. The average cloud cover for 1982-1991 is found to be 55% for northern hemisphere land, 53% for southern hemisphere land, 66% for northern hemisphere ocean, and 70% for southern hemisphere ocean, giving a global average of 64%. The global average for daytime is 64.6% for nighttime 63.3%.
Cloud optical properties, such as extinction-to-backscatter ratio and depolarization ratio, have a significant impact on the accuracy of cloud extinction retrievals from lidar systems because parameterizations of these variables are often used in non-ideal conditions to determine cloud type and optical depth. Statistics and trends of these optical parameters were analyzed for four years, 2003-2007, of Cloud Physics Lidar data during five projects of varying geographic locations. Extinction-to-backscatter ratio (at 532 nm) was derived by calculating the transmission loss through the cloud layer, while depolarization ratio was computed using the parallel and perpendicular polarized 1064 nm channels. The majority of the cloud layers yielded an S-ratio between 10 and 40 sr with the S-ratio frequency distribution centered at 25 sr for ice clouds, 21 sr for mixed phase clouds, and 11 sr for water clouds. On average for ice clouds, S ratio slightly decreased with decreasing temperature, while depolarization ratio increased significantly as temperatures decreased. Trends for water and mixed phase clouds were also observed. Ultimately, these observed trends in optical properties as a function of temperature and geographic location will improve current parameterizations of extinction-to-backscatter ratio, which consequently increases accuracy in cloud optical depth and radiative forcing estimates.
Ten years of nighttime weather observations from the Northern Hemisphere in December were classified according to the illuminance of moonlight or twilight on the cloud tops, and a threshold level of illuminance was determined, above which the clouds are apparently detected adequately. This threshold corresponds to light from a full moon at an elevation angle of 6 deg, light from a partial moon at higher elevation, or twilight from the sun less than 9 deg bvelow the horizon. It permits the use of about 38% of the observations made with the sun below the horizon. The computed diurnal cycles of total cloud cover are altered considerably when this moonlight criterion is imposed. Maximum cloud cover over much of the ocean is now found to be at night or in the morning, whereas computations obtained without benefit of the moonlight criterion, as in our published atlases, showed the time of maximum to be noon or early afternoon in many regions. The diurnal cycles of total cloud cover we obtain are compared with those of the International Satellite Cloud Climatology Project (ISCCP) for a few regions; they are generally in better agreement if the moonlight criterion is imposed on the surface observations. Using the moonlight criterion, we have analyzed 10 years (1982-91) of surface weather observations over land and ocean, worldwide, for total cloud cover and for the frequency of occurrence of clear sky, fog, and precipitation. The global average cloud cover (average of day and night) is about 2% higher if the moonlight criterion is imposed than if all observations are used. The difference is greater in winter than in summer, because of the fewer hours of darkness in summer. The amplitude of the annual cycle of total cloud cover over the Arctic Ocean and at the South Pole is diminished by a few percent when the moonlight criterion is imposed. The average cloud cover for 1982-91 is found to be 55% for Northern Hemisphere land, 53% for Southern Hemisphere land, 66% for Northern Hemisphere ocean, and 70% for Southern Hemisphere ocean, giving a global average of 64%. The global average for daytime is 64.6%; for nighttime 63.3%.
The development of a cloud model-radiative transfer model combination for computing average brightness temperature, T(B), is discussed. The cloud model and radiative transfer model used in this study are described. The relations between rain rate, cloud and rain water, cloud and precipitation ice, and upwelling radiance are investigated. The effects of the rain rate relations on T(B) under different climatological conditions are examined. The model-derived T(B) results are compared to the 92 and 183 GHz aircraft observations of Hakkarinen and Adler (1984, 1986) and the radar-estimated rain rate of Hakkarinen and Adler (1986); good correlation between the data is detected.
The interactions between horizontal ambient flow and divergent wind fields, such as those that obtain atop cumulonimbus complexes, were investigated (theoretically) kinematically. The following were observed from the results of the analyses. First, for a particular divergent field, the relative mass flux over the area of the nephsystem decreased as the strength of the horizontal flow increased. Secondly, while in some of the cases analyzed the interaction between the two flows only resulted in the fanning out of streamlines and a slight redistribution in the wind speed, in many cases backflows and a total reorganization of the wind field occurred. Backflows have a blocking effect on the horizontal flow. Some of the computed patterns were compared with upper level cloud vectors (from geostationary satellite photographs). The comparison indicated that the computed resultant wind field could be used to explain some features of such satellite-derived wind fields.
The effect of radiative heating and cooling by clouds on the available potential energy (APE) is theoretically discussed. It is shown that the cloud radiative contribution to the generation of APE is determined by the net cloud radiative heating and the efficiency factor, which is a function of the temperature distribution of the atmosphere. Results are presented for low and middle cloud effects for three atmospheric layers. Cloud radiative heating is found to be a single function of cloud optical thickness for all classes designed in terms of cloud top heights and optical thickness. Low clouds at low latitudes destroy APE an midclouds generate APE. A concept is developed to relate the cloud radiative heating to cloud heights and optical depths. Cloud-generated radiative heating is computed for January zonal mean conditions for low and midclouds. For both cases, the strongest influence is found in the low troposphere, with marked differences in signs and magnitudes. At extratropical latitudes, both cloud classes generate net radiative cooling. In the tropics, the effect of low cloud changes from net cooling to the net heating as the optical thickness increases, and midclouds cause net heating. A mechanism is described whereby this dependence produces a strong positive feedback effect on the development of SST anomalies in the tropical oceans.
Moist entropy is nearly conserved in adiabatic motion. It is redistributed rather than created by moist convection. Thus moist entropy and its equation, as a healthy direction, can be used to construct analytical and numerical models for the interaction between tropical convective clouds and large-scale circulations. Hence, an accurate equation of moist entropy is needed for the analysis and modeling of atmospheric convective clouds. On the basis of the consistency between the energy and the entropy equations, a complete equation of moist entropy is derived from the energy equation. The equation expresses explicitly the internal and external sources of moist entropy, including those in relation to the microphysics of clouds and precipitation. In addition, an accurate formula for the surface flux of moist entropy from the underlying surface into the air above is derived. Because moist entropy deals "easily" with the transition among three water phases, it will be used as a prognostic variable in the next generation of cloud-resolving models (e. g. a global cloud-resolving model) for low computational noise. Its equation that is derived in this paper is accurate and complete, providing a theoretical basis for using moist entropy as a prognostic variable in the long-term modeling of clouds and large-scale circulations.
An interactive man-computer system, termed WINDCO, which was developed to measure cloud motions from pictures obtained by the ATS-I and ATS-III satellites, is described. The system will measure motions to at least three knots at moderate cost in a real-time environment. Accuracy could be improved by a factor of 4 by incorporating high-resolution visible SMS data.
The estimate of surface irradiance on a global scale is possible through radiative transfer calculations using satellite-retrieved surface, cloud, and aerosol properties as input. Computed top-of-atmosphere (TOA) irradiances, however, do not necessarily agree with observation-based values, for example, from the Clouds and the Earth's Radiant Energy System (CERES). This paper presents amethod to determine surface irradiances using observational constraints of TOA irradiance from CERES. A Lagrange multiplier procedure is used to objectively adjust inputs based on their uncertainties such that the computed TOA irradiance is consistent with CERES-derived irradiance to within the uncertainty. These input adjustments are then used to determine surface irradiance adjustments. Observations by the Atmospheric Infrared Sounder (AIRS), Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO), CloudSat, andModerate Resolution Imaging Spectroradiometer (MODIS) that are a part of the NASA A-Train constellation provide the uncertainty estimates. A comparison with surface observations from a number of sites shows that the bias [root-mean-square (RMS) difference] between computed and observed monthlymean irradiances calculated with 10 years of data is 4.7 (13.3) W/sq m for downward shortwave and 22.5 (7.1) W/sq m for downward longwave irradiances over ocean and 21.7 (7.8) W m22 for downward shortwave and 21.0 (7.6) W/sq m for downward longwave irradiances over land. The bias andRMS error for the downward longwave and shortwave irradiances over ocean are decreased from those without constraint. Similarly, the bias and RMS error for downward longwave over land improves, although the constraint does not improve downward shortwave over land. This study demonstrates how synergetic use of multiple instruments (CERES,MODIS, CALIPSO, CloudSat, AIRS, and geostationary satellites) improves the accuracy of surface irradiance computations.
A program was developed for computing the scattering and absorption by arbitrarily oriented and randomly oriented prolate and oblate spheroids. This permits examination of the effect of particle shape for cases ranging from needles through spheres to platelets. Applications of this capability to aerosol and cloud problems are discussed. Initial results suggest that the effect of nonspherical particle shape on transfer of radiation through aerosol layers and cirrus clouds, as required for many climate studies, can be readily accounted for by defining an appropriate effective spherical particle radius.