Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Effect of dry large‐scale vertical motions on initial MJO convective onset

Abstract Anomalies of eastward propagating large‐scale vertical motion with ~30 day variability at Addu City, Maldives, move into the Indian Ocean from the west and are implicated in Madden‐Julian Oscillation (MJO) convective onset. Using ground‐based radar and large‐scale forcing data derived from a sounding array, typical profiles of environmental heating, moisture sink, vertical motion, moisture advection, and Eulerian moisture tendency are computed for periods prior to those during which deep convection is prevalent and those during which moderately deep cumulonimbi do not form into deep clouds. Convection with 3–7 km tops is ubiquitous but present in greater numbers when tropospheric moistening occurs below 600 hPa. Vertical eddy convergence of moisture in shallow to moderately deep clouds is likely responsible for moistening during a 3–7 day long transition period between suppressed and active MJO conditions, although moistening via evaporation of cloud condensate detrained into the environment of such clouds may also be important. Reduction in large‐scale subsidence, associated with a vertical velocity structure that travels with a dry eastward propagating zonal wavenumbers 1–1.5 structure in zonal wind, drives a steepening of the lapse rate below 700 hPa, which supports an increase in moderately deep moist convection. As the moderately deep cumulonimbi moisten the lower troposphere, more deep convection develops, which itself moistens the upper troposphere. Reduction in large‐scale subsidence associated with the eastward propagating feature reinforces the upper tropospheric moistening, helping to then rapidly make the environment conducive to formation of large stratiform precipitation regions, whose heating is critical for MJO maintenance.

Powell, Scott W.↗

Supercomputers, Clouds and Grids powered by BigPanDA for Brain studies

The PanDA WMS - Production and Distributed Analysis Workload Management System - has been developed and used by the ATLAS experiment at the LHC (Large Hadron Collider) for all data processing and analysis challenges. BigPanDA is an extension of the PanDA WMS to run ATLAS and non-ATLAS applications on Leadership Class Facilities and supercomputers, as well as traditional grid and cloud resources. The success of the BigPanDA project has drawn attention from other compute intensive sciences such as biology. In 2017, a pilot project was started between BigPanDA and the Blue Brain Project (BBP) of the Ecole Polytechnique Federal de Lausanne (EPFL) located in Lausanne, Switzerland. This proof of concept project is aimed at demonstrating the efficient application of the BigPanDA system to support the complex scientific workflow of the BBP, which relies on using a mix of desktop, cluster and supercomputers to reconstruct and simulate accurate models of brain tissue.

Computer Science↗

3.6 and 4.5 μm Spitzer phase curves of the highly irradiated hot jupiters WASP-19b and HAT-P-7b

We analyze full-orbit phase curve observations of the transiting hot Jupiters WASP-19b and HAT-P-7b at 3.6 and 4.5 μm, obtained using the Spitzer Space Telescope. For WASP-19b, we measure secondary eclipse depths of 0.485%±0.024% and 0.584%±0.029% at 3.6 and 4.5 μm, which are consistent with a single blackbody with effective temperature 2372 ± 60 K. The measured 3.6 and 4.5 μm secondary eclipse depths for HAT-P-7b are 0.156%±0.009% and 0.190%±0.006%, which are well described by a single blackbody with effective temperature 2667 ± 57 K. Comparing the phase curves to the predictions of one-dimensional and three-dimensional atmospheric models, we find that WASP-19b’s dayside emission is consistent with a model atmosphere with no dayside thermal inversion and moderately efficient day–night circulation. We also detect an eastward-shifted hotspot, which suggests the presence of a superrotating equatorial jet. In contrast, HAT-P-7b’s dayside emission suggests a dayside thermal inversion and relatively inefficient day–night circulation; no hotspot shift is detected. For both planets, these same models do not agree with the measured nightside emission. The discrepancies in the model-data comparisons for WASP-19b might be explained by high-altitude silicate clouds on the nightside and/or high atmospheric metallicity, while the very low 3.6 μm nightside planetary brightness for HAT-P-7b may be indicative of an enhanced global C/O ratio. We compute Bond albedos of 0.38 ± 0.06 and 0 (<0.08 at 1σ) for WASP-19b and HAT-P-7b, respectively. In the context of other planets with thermal phase curve measurements, we show that WASP-19b and HAT-P-7b fit the general trend of decreasing day–night heat recirculation with increasing irradiation.

79 ASTRONOMY AND ASTROPHYSICS↗

Leveraging the checkpoint-restart technique for optimizing CPU efficiency of ATLAS production applications on opportunistic platforms

Data processing applications of the ATLAS experiment, such as event simulation and reconstruction, spend considerable amount of time in the initialization phase. This phase includes loading a large number of shared libraries, reading detector geometry and condition data from external databases, building a transient representation of the detector geometry and initializing various algorithms and services. In some cases the initialization step can take as long as 10-15 minutes. Such slow initialization has a significant negative impact on overall CPU efficiency of the production job, especially when the job is executed on opportunistic, often short-lived, resources such as commercial clouds or volunteer computing. In order to improve this situation, we can take advantage of the fact that ATLAS runs large numbers of production jobs with similar configuration parameters (e.g. jobs within the same production task). This allows us to checkpoint one job at the end of its configuration step and then use the generated checkpoint image for rapid startup of thousands of production jobs. By applying this technique we can bring the initialization time of a job from tens of minutes down to just a few seconds. In addition to that we can leverage container technology for restarting checkpointed applications on the variety of computing platforms, in particular of platforms different from the one on which the checkpoint image was created. We will describe the mechanism of creating checkpoint images of Geant4 simulation jobs with AthenaMP (the multi-process version of the ATLAS data simulation, reconstruction and analysis framework Athena) and the usage of these images for running ATLAS Simulation production jobs on volunteer computing resources (ATLAS@Home) and on Supercomputers.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Fast All-Sky Radiation Model for Solar Applications (FARMS): A Brief Overview of Mechanisms, Performance, and Applications: Preprint

Solar radiation can be computed using radiative transfer models, such as the Rapid Radiation Transfer Model (RRTM) and its general circulation model applications, and used for various energy applications. Due to the complexity of computing radiation fields in aerosol and cloudy atmospheres, simulating solar radiation can be extremely time-consuming, but many approximations--e.g., the two-stream approach and the delta-M truncation scheme--can be utilized. To provide a new fast option for computing solar radiation, we developed the Fast All-sky Radiation Model for Solar applications (FARMS) by parameterizing the simulated diffuse horizontal irradiance and direct normal irradiance for cloudy conditions from the RRTM runs using a 16-stream discrete ordinates radiative transfer method. The solar irradiance at the surface was simulated by combining the cloud irradiance parameterizations with a fast clear-sky model, REST2. To understand the accuracy and efficiency of the newly developed fast model, we analyzed FARMS runs using cloud optical and microphysical properties retrieved using GOES data from 2009-2012. The global horizontal irradiance for cloudy conditions was simulated using FARMS and RRTM for global circulation modeling with a two-stream approximation and compared to measurements taken from the U.S. Department of Energy's Atmospheric Radiation Measurement Climate Research Facility Southern Great Plains site. Our results indicate that the accuracy of FARMS is comparable to or better than the two-stream approach; however, FARMS is approximately 400 times more efficient because it does not explicitly solve the radiative transfer equation for each individual cloud condition. Radiative transfer model runs are computationally expensive, but this model is promising for broad applications in solar resource assessment and forecasting. It is currently being used in the National Solar Radiation Database, which is publicly available from the National Renewable Energy Laboratory at http://nsrdb.nrel.gov.

14 SOLAR ENERGY↗

Enabling a systems biology knowledgebase with gaggle and firegoose

The overall goal of this project was to extend the existing Gaggle and Firegoose systems to develop an open-source technology that runs over the web and links desktop applications with many databases and software applications. This technology would enable researchers to incorporate workflows for data analysis that can be executed from this interface to other online applications. The four specific aims were to (1) provide one-click mapping of genes, proteins, and complexes across databases and species; (2) enable multiple simultaneous workflows; (3) expand sophisticated data analysis for online resources; and enhance open-source development of the Gaggle-Firegoose infrastructure. Gaggle is an open-source Java software system that integrates existing bioinformatics programs and data sources into a user-friendly, extensible environment to allow interactive exploration, visualization, and analysis of systems biology data. Firegoose is an extension to the Mozilla Firefox web browser that enables data transfer between websites and desktop tools including Gaggle. In the last phase of this funding period, we have made substantial progress on development and application of the Gaggle integration framework. We implemented the workspace to the Network Portal. Users can capture data from Firegoose and save them to the workspace. Users can create workflows to start multiple software components programmatically and pass data between them. Results of analysis can be saved to the cloud so that they can be easily restored on any machine. We also developed the Gaggle Chrome Goose, a plugin for the Google Chrome browser in tandem with an opencpu server in the Amazon EC2 cloud. This allows users to interactively perform data analysis on a single web page using the R packages deployed on the opencpu server. The cloud-based framework facilitates collaboration between researchers from multiple organizations. We have made a number of enhancements to the cmonkey2 application to enable and improve the integration within different environments, and we have created a new tools pipeline for generating EGRIN2 models in a largely automated way.

97 MATHEMATICS AND COMPUTING↗

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

15 GEOTHERMAL ENERGY↗

The Aerosol Model Benchmarking Repository: A toolkit for model intercomparison

The Aerosol Model Benchmarking Repository and Standards (AMBRS) project was initiated to provide tools and to establish community standards for benchmarking aerosol models. This report describes a set of open-source tools for building, running, and analyzing aerosol box model simulations in a standardized framework. The framework consists of three core components: AMBuilder, a CMake-based build system that compiles supported models consistently; AMBRS, a Python module that defines unified numerical experiments and executes them with aligned inputs; and PyParticle, an aerosol analysis package that standardizes output, computes diagnostics, and visualizes simulation results. Together, these tools enable reproducible intercomparison of aerosol schemes and support process-level evaluation of how model simplifications affect predictions of size distributions, cloud condensation nuclei activity, and other relevant properties relevant for the Earth-Energy system. Beyond its role in benchmarking, AMBRS provides a platform for studying aerosol processes across scales and can be used to generate training data for AI/ML applications in support of a broader hierarchical aerosol modeling strategy.

54 ENVIRONMENTAL SCIENCES↗

Machine learning approaches to streamline and enhance the analysis of multiscale imaging data for bioaerosol and soil particles

Bioaerosol and soil particles are ubiquitous in the environment. They are multicomponent and complex in nature displaying mixed inorganic and organic components. The way components are mixed in a bioaerosol sample is referred to as its mixing state. Soil particles are also a mixture of inorganic (mineral) and organic (soil organic matter) components. Bioaerosol particles contribute to a major fraction of coarse mode atmospheric particles, especially in the tropical areas, contributing up to 80 % of the particle mass concentration. The mixing state of particles is crucial to evaluate because it impacts several important environmental processes such as warm and cold cloud formation and radiation budget. Mixing states in aerosols are accompanied by chemical reactions across solid-liquid-gas interfaces. In this study, we utilized elemental compositions and microcopy images of thousands of atmospheric particles acquired by computer-controlled scanning electron microscope equipped with an energy-dispersive x-ray spectrometer to compute the mixing state of atmospheric particles. A 2D convolutional neural network (CNN), also known as convnet, was used to model the relationship between low resolution imaging data and higher resolution spectroscopy data, with the former as training input and the latter as target output. Two types of CNNs were implemented and tested; a basic CNN and an Inception-v3 network. For binary classification, the basic CNN achieved an accuracy of 84.29 % across all atom types, and the Inception-v3-like network achieved an accuracy 85.51 %. This study demonstrates the applicability of deep learning to handle large amounts of imaging/chemical spectroscopy data efficiently and evaluate particle mixing state from a range of environmental samples.

54 ENVIRONMENTAL SCIENCES↗

PlantSegNet: 3D point cloud instance segmentation of nearby plant organs with identical semantics

In this study, we introduce PlantSegNet, a novel neural network model for instance segmentation of nearby objects with similar geometric structures. Our work addresses the challenges of instance segmentation of plant point clouds, including the difficulty of annotating and labeling point clouds, the loss of local structural information in neural network components, and the generation of large numbers of incorrect small clusters due to poor choices of the loss function. One of the key contributions of our approach is a digital twin of sorghum, i.e., a procedural sorghum model, which was used to generate point clouds of sorghum fields. This allowed us to create a large-scale, annotated, synthetic dataset of sorghum plants that we used to train our PlantSegNet model. We demonstrated the effectiveness of our method in segmenting instances of sorghum leaves grown in outdoor field settings. To the best of our knowledge, this is the first study to address this specific instance segmentation problem for plants grown in such a setting. We compared our proposed method with other state-of-the-art methods for indoor settings, including SGPN and TreePartNet, on both synthetic and real data. Furthermore, our results show that PlantSegNet outperforms these methods regarding accuracy, robustness, and efficiency.

97 MATHEMATICS AND COMPUTING↗

Vehicle Lateral Offset Estimation Using Infrastructure Information for Reduced Compute Load

Accurate perception of the driving environment and a highly accurate position of the vehicle are paramount to safe Autonomous Vehicle (AV) operation. AVs gather data about the environment using various sensors. For a robust perception and localization system, incoming data from multiple sensors is usually fused together using advanced computational algorithms, which historically requires a high-compute load. To reduce AV compute load and its negative effects on vehicle energy efficiency, we propose a new infrastructure information source (IIS) to provide environmental data to the AV. The new energy–efficient IIS, chip–enabled raised pavement markers are mounted along road lane lines and are able to communicate a unique identifier and their global navigation satellite system position to the AV. This new IIS is incorporated into an energy efficient sensor fusion strategy that combines its information with that from traditional sensor. IIS reduce the need for camera imaging, image processing, and LIDAR use and point cloud processing. We show that IIS, when combined with traditional sensors, results in more accurate perception and localization outcomes and a reduced AV compute load.

Sharma, Sachin↗

ARMing the Edge: Demonstration of Edge Computing Field Campaign Report

Edge computing enables “next-to-instrument” control and intelligent data volume reduction and the potential for autonomous, adaptive measurement strategies such as for automated control of scan strategies for Doppler lidar (DL). Instruments with narrow bandwidth connections (e.g., ship and remote sites) can do scene determination and save phenomenon-appropriate data. For example, Doppler spectrum can be saved when clouds are detected by the instrument or automatic moment detection can take place in camera images and only preserve spectrum when non-monomodal spectra are detected. Automated control at the edge involves changing the sampling (temporal or scanning strategy) of an instrument to suit the phenomena both present and being studied (Jackson et al. 2020). Both data processing and instrument control introduces the possibility of a software-defined instrument.

54 ENVIRONMENTAL SCIENCES↗

Emulation to simulate low resolution atmospheric data

Climate simulations require significant compute power, they are complex and therefore it is time consuming to simulate them. We have developed an emulator to simulate unknown climate datasets. The emulator uses stochastic collocation and multi-dimensional in- terpolation to simulate the datasets. We have used the emulator to determine various physical quantities such as temperature, short and long wave cloud forcing, zonal winds etc. The emulation gives results which are very close to those obtained by simulations. The emulator was tested on 2 degree atmospheric datasets. The work evaluates the pros and cons of evaluating the mean first and inter- polating and vice versa. To determine the physical quantities, we have assumed them to be a function of time, longitude, latitude and a random parameter. We have looked at parameters that govern high stable clouds, low stable clouds, timescale for convection etc. The emulator is especially useful as it requires negligible compute times when compared to the simulation itself.

Rao, Vishwas Hebbur Venkata Subba↗

A Contrast Calibration Protocol for X-ray Speckle Visibility Spectroscopy

X-ray free electron lasers, with their ultrashort highly coherent pulses, opened up the opportunity of probing ultrafast nano- and atomic-scale dynamics in amorphous and disordered material systems via speckle visibility spectroscopy. However, the anticipated count rate in a typical experiment is usually low. Therefore, visibility needs to be extracted via photon statistics analysis, i.e., by estimating the probabilities of multiple photons per pixel events using pixelated detectors. Considering the realistic X-ray detector responses including charge cloud sharing between pixels, pixel readout noise, and gain non-uniformity, speckle visibility extraction relying on photon assignment algorithms are often computationally demanding and suffer from systematic errors. In this paper, we present a systematic study of the commonly-used algorithms by applying them to an experimental data set containing small-angle coherent scattering with visibility levels ranging from below 1% to ∼60%. We also propose a contrast calibration protocol and show that a computationally lightweight algorithm can be implemented for high-speed correlation evaluation.

47 OTHER INSTRUMENTATION↗

Optimizing cloud motion estimation on the edge with phase correlation and optical flow

Abstract. Phase correlation (PC) is a well-known method for estimating cloud motion vectors (CMVs) from infrared and visible spectrum images. Commonly, phase shift is computed in the small blocks of the images using the fast Fourier transform. In this study, we investigate the performance and the stability of the blockwise PC method by changing the block size, the frame interval, and combinations of red, green, and blue (RGB) channels from the total sky imager (TSI) at the United States Atmospheric Radiation Measurement user facility's Southern Great Plains site. We find that shorter frame intervals, followed by larger block sizes, are responsible for stable estimates of the CMV, as suggested by the higher autocorrelations. The choice of RGB channels has a limited effect on the quality of CMVs, and the red and the grayscale images are marginally more reliable than the other combinations during rapidly evolving low-level clouds. The stability of CMVs was tested at different image resolutions with an implementation of the optimized algorithm on the Sage cyberinfrastructure test bed. We find that doubling the frame rate outperforms quadrupling the image resolution in achieving CMV stability. The correlations of CMVs with the wind data are significant in the range of 0.38–0.59 with a 95 % confidence interval, despite the uncertainties and limitations of both datasets. A comparison of the PC method with constructed data and the optical flow method suggests that the post-processing of the vector field has a significant effect on the quality of the CMV. The raindrop-contaminated images can be identified by the rotation of the TSI mirror in the motion field. The results of this study are critical to optimizing algorithms for edge-computing sensor systems.

54 ENVIRONMENTAL SCIENCES↗

Under canopy light detection and ranging–based autonomous navigation

This study describes a light detection and ranging (LiDAR)–based autonomous navigation system for an ultralightweight ground robot in agricultural fields. The system is designed for reliable navigation under cluttered canopies using only a 2D Hokuyo UTM–30LX LiDAR sensor as the single source for perception. Its purpose is to ensure that the robot can navigate through rows of crops without damaging the plants in narrow row–based and high–leaf–cover semistructured crop plantations, such as corn ( Zea mays ) and sorghum ( Sorghum bicolor ). The key contribution of our work is a LiDAR–based navigation algorithm capable of rejecting outlying measurements in the point cloud due to plants in adjacent rows, low–hanging leaf cover or weeds. The algorithm addresses this challenge using a set of heuristics that are designed to filter out outlying measurements in a computationally efficient manner, and linear least squares are applied to estimate within–row distance using the filtered data. Moreover, a crucial step is the estimate validation, which is achieved through a heuristic that grades and validates the fitted row–lines based on current and previous information. The proposed LiDAR–based perception subsystem has been extensively tested in production/breeding corn and sorghum fields. In such variety of highly cluttered real field environments, the robot logged more than 6 km of autonomous run in straight rows. These results demonstrate highly promising advances to LiDAR–based navigation in realistic field environments for small under–canopy robots.

42 ENGINEERING↗

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository

The Department of Energy's (DOE's) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and helping users access data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data. This paper provides an update on recent improvements made to the GDR's data lakes and automated data pipelines, including: (1) streamlining the data lake intake process, (2) better educating users on the process and requirements through a new data lakes page, (3) adding data lake direct access links to GDR data lake submission pages, (4) implementing a DAS data pipeline to convert DAS data uploaded in SEG-Y format to a standardized hierarchical data format v5 (HDF5), (5) extending this pipeline to encompass data in the GDR data lake, (6) adding metadata requirements for geospatial data, (7) making user interface/user experience (UX) enhancements to the data pipelines' documentation pages, and (8) improving the GDR's data standards and pipelines pages to better guide users in ensuring that their data is standardized by the GDR's automated data pipelines. 2024 Geothermal Resources Council. All rights reserved.

accessibility↗

Impact of Multiple Scattering on Longwave Radiative Transfer Involving Clouds

Abstract General circulation models (GCMs) are extensively used to estimate the influence of clouds on the global energy budget and other aspects of climate. Because radiative transfer computations involved in GCMs are costly, it is typical to consider only absorption but not scattering by clouds in longwave (LW) spectral bands. In this study, the flux and heating rate biases due to neglecting the scattering of LW radiation by clouds are quantified by using advanced cloud optical property models, and satellite data from Cloud‐Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO), CloudSat, Clouds and the Earth's Radiant Energy System (CERES), and Moderate Resolution Imaging Spectrometer (MODIS) merged products (CCCM). From the products, information about the atmosphere and clouds (microphysical and buck optical properties, and top and base heights) is used to simulate fluxes and heating rates. One‐year global simulations for 2010 show that the LW scattering decreases top‐of‐atmosphere (TOA) upward flux and increases surface downward flux by 2.6 and 1.2 W/m 2 , respectively, or approximately 10% and 5% of the TOA and surface LW cloud radiative effect, respectively. Regional TOA upward flux biases are as much as 5% of global averaged outgoing longwave radiation (OLR). LW scattering causes approximately 0.018 K/d cooling at the tropopause and about 0.028 K/d heating at the surface. Furthermore, over 40% of the total OLR bias for ice clouds is observed in 350–500 cm −1 . Overall, the radiative effects associated with neglecting LW scattering are comparable to the counterpart due to doubling atmospheric CO 2 under clear‐sky conditions.

54 ENVIRONMENTAL SCIENCES↗