Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Improving the Accuracy of the National Solar Radiation Database (1998-2016): Preprint

The National Solar Radiation Database (NSRDB) provides gridded satellite-based data sets for the United States and other parts of North and South America using cloud properties retrieved from the Geostationary Operational Environmental (GOES) series of satellites. These cloud properties in combination with ancillary atmospheric parameters are used in the Physical Solar Model (PSM) to compute solar radiation. This paper seeks to 1) describe the changes made in the PSM for the recent update to the National Solar Radiation Database (NSRDB 1998-2016) and 2) examine the performance of this update by comparing it to high-quality ground-based measurements. We summarize the changes to the input data for the radiative transfer models as well as updates for the processing algorithms. We also analyze the performance and accuracy of the NSRDB across the United States. Additional details of the data set and model are described in (1), (2), (3), and (4).

14 SOLAR ENERGY↗

Developing and testing capabilities for simulating cases with heterogeneous land/water surfaces in a novel atmospheric large eddy simulation code

Large eddy simulations (LES) are the primary computational tool used to simulate high Reynolds number three-dimensional turbulent flows. In the context of earth system sciences, particularly atmospheric science, LES are uniquely able to resolve the scales of atmospheric motion that are key for building process-level understanding of boundary layer turbulence, atmosphere-surface interaction, clouds, and cloud-aerosol-chemistry interaction, and are a core limited-area modeling capability. Increasing demands are being placed on LES code bases as growing high performance computing resources allow LES to address a wider range of scientific problems. In addition, LES are emerging as a source of high-quality machine learning training data. These demands necessitate an agile and extensible code base that allows the model to quickly adapt to emergent needs. However, LES have largely relied on legacy Fortran code bases that lack flexibility. A new, Python-based LES capability called Predicting INteractions of Aerosol and Clouds in Large Eddy Simulation (PINACLES) has been developed as part of the Department of Energy’s Earth System Model Development (ESMD) program area’s Enabling Aerosol-cloud interactions at Global convection-permitting scalES (EAGLES) project. PINACLES was developed from the ground up with a philosophy of maximizing scientific throughput, by attempting to optimize for both model throughput and software extensibility. The initial development of PINACLES delivered a state-of-the-art idealized LES capability solving the non-hydrostatic anelastic equations of motion with doubly periodic boundary conditions and idealized homogenous surface boundary conditions. Here we provide a final report on the outcomes of a fiscal year 2021 Seed Laboratory Directed Research Project that extended PINACLES in two key ways. First, PINACLES was coupled to a state-of-the-art land surface model enabling it to simulate spatially inhomogeneous land-atmosphere interactions that are known to control key atmospheric processes. Second, the dynamical core of PINACLES was modified to permit non-periodic boundary conditions. This model enhancement enables simulation of realistic cases with boundary conditions prescribed from atmospheric reanalysis and enables nested simulations conducted on a hierarchy of computational domains with increasing resolution. Together, these extensions to PINACLES make it a formidable modeling capability and expand its potential application to diverse components of DOE’s atmospheric science portfolio.

42 ENGINEERING↗

Retrieval of Areal-averaged Spectral Surface Albedo from Transmission Data Alone: Computationally Simple and Fast Approach

We introduce and evaluate a simple retrieval of areal-averaged surface albedo using ground-based measurements of atmospheric transmission alone at five wavelengths (415, 500, 615, 673 and 870nm), under fully overcast conditions. Our retrieval is based on a one-line semi-analytical equation and widely accepted assumptions regarding the weak spectral dependence of cloud optical properties, such as cloud optical depth and asymmetry parameter, in the visible and near-infrared spectral range. To illustrate the performance of our retrieval, we use as input measurements of spectral atmospheric transmission from Multi-Filter Rotating Shadowband Radiometer (MFRSR). These MFRSR data are collected at two well-established continental sites in the United States supported by the U.S. Department of Energy’s (DOE’s) Atmospheric Radiation Measurement (ARM) Program and National Oceanic and Atmospheric Administration (NOAA). The areal-averaged albedos obtained from the MFRSR are compared with collocated and coincident Moderate Resolution Imaging Spectroradiometer (MODIS) white-sky albedo. In particular, these comparisons are made at four MFRSR wavelengths (500, 615, 673 and 870nm) and for four seasons (winter, spring, summer and fall) at the ARM site using multi-year (2008-2013) MFRSR and MODIS data. Good agreement, on average, for these wavelengths results in small values (≤0.01) of the corresponding root mean square errors (RMSEs) for these two sites. The obtained RMSEs are comparable with those obtained previously for the shortwave albedos (MODIS-derived versus tower-measured) for these sites during growing seasons. We also demonstrate good agreement between tower-based daily-averaged surface albedos measured for “nearby” overcast and non-overcast days. Thus, our retrieval originally developed for overcast conditions likely can be extended for non-overcast days by interpolating between overcast retrievals.

atmospheric transmission↗

Optical Particle Measurements during EPCAPE Field Campaign Report

This campaign requested the deployment of the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) User Facility optical particle counter (OPC) at the first ARM Mobile Facility (AMF1) located at the Scripps Pier in La Jolla, California during the Eastern Pacific Cloud Aerosol Precipitation Experiment (EPCAPE). The addition of the OPC was requested for two reasons. (1) Close the gap between the scanning mobility particle sizer (SMPS) and aerodynamic particle sizer (APS) size distribution from the Aerosol Observing System (AOS) measurements. (2) Principal investigator Petters has been working with Tracking Aerosol Convection Interaction Experiment (TRACER) data to compute particle fluxes from Doppler lidar (Petters et al. 2024). Briefly, backscatter flux is obtained using the eddy covariance technique using the Doppler vertical velocity and attenuated backscatter. Building upon prior studies, we were able to relate backscatter to particle number concentration by calibrating the lidar retrievals against optical particle counter-measured ground-based aerosol size distribution and radiosonde-interpolated relative humidity at lidar sample height. Performing similar analysis was of interest to EPCAPE to better understand the emissions and vertical transport of large particles into the overlying stratus clouds. However, as stated above, this analysis requires an optical size distribution that covers the 0.3-30-μm-diameter size range. The OPC was deployed between 2023-04-14 and 2024-02-14. The deployment, data quality analysis, and data archiving was handled by the DOE ARM instrument mentor team without additional involvement by the principal investigator. Data quality was marked as “routine” for the majority of the campaign.

54 ENVIRONMENTAL SCIENCES↗

Estimation of the cloud transmittance from radiometric measurements at the ground level

The extinction of solar radiation due to the clouds is more significant than due to any other atmospheric constituent, but it is always difficult to be modeled because of the random distribution of clouds on the sky. Moreover, the transmittance of a layer of clouds is in a very complex relation with their type and depth. A method for estimating cloud transmittance was proposed in Paulescu et al. (Energ. Convers. Manage, 75 690–697, 2014). The approach is based on the hypothesis that the structure of the cloud covering the sun at a time moment does not change significantly in a short time interval (several minutes). Thus, the cloud transmittance can be calculated as the estimated coefficient of a simple linear regression for the computed versus measured solar irradiance in a time interval Δt. The aim of this paper is to optimize the length of the time interval Δt. Radiometric data measured on the Solar Platform of the West University of Timisoara during 2010 at a frequency of 1/15 seconds are used in this study.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Unprecedented cloud resolution in a GPU-enabled full-physics atmospheric climate simulation on OLCF’s summit supercomputer

Clouds represent a key uncertainty in future climate projection. While explicit cloud resolution remains beyond our computational grasp for global climate, we can incorporate important cloud effects through a computational middle ground called the Multi-scale Modeling Framework (MMF), also known as Super Parameterization. This algorithmic approach embeds high-resolution Cloud Resolving Models (CRMs) to represent moist convective processes within each grid column in a Global Climate Model (GCM). The MMF code requires no parallel data transfers and provides a self-contained target for acceleration. This study investigates the performance of the Energy Exascale Earth System Model-MMF (E3SM-MMF) code on the OLCF Summit supercomputer at an unprecedented scale of simulation. Hundreds of kernels in the roughly 10K lines of code in the E3SM-MMF CRM were ported to GPUs with OpenACC directives. A high-resolution benchmark using 4600 nodes on Summit demonstrates the computational capability of the GPU-enabled E3SM-MMF code in a full physics climate simulation.

58 GEOSCIENCES↗

GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond

Recent progress in Artificial Intelligence (AI) techniques, the large-scale availability of high-quality data, as well as advances in both hardware and software to efficiently process these data, are transforming a range of fields from computer vision and natural language processing to autonomous driving and healthcare. For example, the availability of high-resolution geographic data and high-performance computing techniques together with deep learning fuel progress in fast and accurate object detection. Recent examples of GeoAI work include the detections of terrain features and densely-distributed building footprints, information extraction from scanned historical maps, semantic classification (e.g. LiDAR point clouds), novel methods for spatial interpolation, and advances in traffic forecasting. Similarly, machine learning and natural language processing are facilitating the extraction of geographic information from unstructured (textual) data, such as news articles and Wikipedia as well as the matching of natural features in multiple gazetteers.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Micropulse Lidar Cloud Mask (MPLCMASK) Value-Added Product for the Fast-Switching Polarized Micropulse Lidar Technical Report

Lidar backscattered signal is a useful tool for identifying vertical cloud structure in the atmosphere in optically thin clouds. Cloud boundaries derived from lidar signals are a necessary input for popular ARM data products, such as the Active Remote Sensing of Clouds (ARSCL) product. An operational cloud boundary algorithm (Wang and Sassen 2001) has been implemented for use with the ARM Micropulse Lidar (MPL) systems. In addition to retrieving cloud boundaries above 500 m, the value-added product (VAP) named Micropulse Lidar Cloud Mask (MPLCMASK) applies lidar-specific corrections (i.e., range-square, background, deadtime, and overlap) as described in Campbell et al. (2002) to the measured backscattered lidar. Depolarization ratio is computed using the methodology developed by Flynn et al. (2007) for polarization-capable MPL systems. The cloud boundaries output from MPLCMASK will be the primary lidar cloud mask for input to the ARSCL product and will be applied to all MPL systems, including historical data sets.

54 ENVIRONMENTAL SCIENCES↗

Spectral binning for energy production calculations and multijunction solar cell design

Currently, most solar cells are designed for and evaluated under standard spectra intended to represent typical spectral conditions. However, no single spectrum can capture the spectral variability needed for annual energy production (AEP) calculations, and this shortcoming becomes more significant for series-connected multijunction cells as the number of junctions increases. For this reason, AEP calculations are often performed on very detailed yearlong sets of data, but these pose 2 inherent challenges: (1) These data sets comprise thousands of data points, which appear as a scattered cloud of data when plotted against typical parameters and are hence cumbersome to classify and compare, and (2) large sets of spectra bring with them a corresponding increase in computation or measurement time. Here, we show how a large spectral set can be reduced to just a few 'proxy' spectra, which still retain the spectral variability information needed for AEP design and evaluation. The basic 'spectral binning' methods should be extensible to a variety of multijunction device architectures. In this study, as a demonstration, the AEP of a 4-junction device is computed for both a full set of spectra and a reduced proxy set, and the results show excellent agreement for as few as 3 proxy spectra. This enables much faster (and thereby more detailed) calculations and indoor measurements and provides a manageable way to parameterize a spectral set, essentially creating a 'spectral fingerprint,' which should facilitate the understanding and comparison of different sites.

14 SOLAR ENERGY↗

Machine Learning Toolkit for Extreme Scale

Support Vector Machines (SVM) is a popular machine learning technique, which has been applied to a wide range of domains such as science, finance, and social networks for supervised learning. MaTEx undertakes the challenge of designing a scalable parallel SVM training algorithm for large scale systems, which includes commodity multi-core machines, tightly connected supercomputers and cloud computing systems. Several techniques are proposed for improved speed and memory space usage including adaptive and aggressive elimination of samples for faster convergence , and sparse format representation of data samples. Several heuristics for earliest possible to lazy elimination of non-contributing samples are considered in MaTEx. In many cases, where an early sample elimination might result in a false positive, low overhead mechanisms for reconstruction of key data structures are proposed. The proposed algorithm and heuristics are implemented and evaluated on various publicly available datasets

Source record↗

Adaptive Computing (AC) [SWR-24-106]

The Adaptive Computing (AC) software stack supports goal-based computing, for which a simulation workload is created on the fly adapting to the results of calculations. Application-specific code defines an objective, which may be to solve an optimization problem or to train a surrogate model with minimal uncertainty. Then, the AC driver decides where in the design parameter space to run simulations to best achieve that objective. This process is iterative and online; as new data is returned from simulations, the AC driver chooses new simulations to run. The AC driver can strategically run simulations on distributed hardware resources (including high performance computing machines, cloud resources, and edge devices) to maximize throughput and obey resource constraints.

Griffin, Kevin [National Renewable Energy Laborato↗

Adaptive Computing (AC) (Open Source) [SWR-24-106]

The Adaptive Computing (AC) software stack supports goal-based computing, for which a simulation workload is created on the fly, adapting to the results of calculations. Application-specific code defines an objective, which may be to solve an optimization problem or to train a surrogate model with minimal uncertainty. Then, the AC driver decides where in the design parameter space to run simulations to best achieve that objective. This process is iterative and online; as new data is returned from simulations, the AC driver chooses new simulations to run. The AC driver can strategically run simulations on distributed hardware resources (including high performance computing machines, cloud resources, and edge devices) to maximize throughput and obey resource constraints.

Griffin, Kevin [National Laboratory of the Rockies↗

Machine Learning Toolkit for Extreme Scale

Support Vector Machines (SVM) is a popular machine learning technique, which has been applied to a wide range of domains such as science, finance, and social networks for supervised learning. MaTEx undertakes the challenge of designing a scalable parallel SVM training algorithm for large scale systems, which includes commodity multi-core machines, tightly connected supercomputers and cloud computing systems. Several techniques are proposed for improved speed and memory space usage including adaptive and aggressive elimination of samples for faster convergence , and sparse format representation of data samples. Several heuristics for earliest possible to lazy elimination of non-contributing samples are considered in MaTEx. In many cases, where an early sample elimination might result in a false positive, low overhead mechanisms for reconstruction of key data structures are proposed. The proposed algorithm and heuristics are implemented and evaluated on various publicly available datasets

Source record↗

A Method for Projecting Cloud Shadows Onto a Central Receiver Field to Predict Receiver Damage

This work demonstrates methods of mapping high-spatial-resolution direct normal irradiance (DNI) data from satellites, Total Sky Imagers (TSIs), and analogous data sources onto a heliostat field for characterizing the spatial and temporal variation of the incident flux on a central receiver tower during cloud transient events. The mapping methods are incorporated into an optical software module that interfaces with CoPylot–SolarPILOT’s python API– to provide computationally efficient optical simulation of the heliostat field and the solar power tower. Eventually, this optical model will be incorporated into optimization models whereby a plant operator can understand the effects of cloud transient events on overall power production and receiver lifetime due to creep-fatigue damage and therefore make better informed decisions about receiver shutdown events. By more accurately modelling the effects of cloud events on receiver flux maps, this work may determine the magnitude and frequency of thermal cycling on receiver tubes and panels using actual or realistic cloud shapes instead of averaged DNI values–which may undercount the total cycle number. This work may also prevent unnecessary plant shutdowns due to overly precautionary control strategies and characterize the relative impact of various cloud types on receiver life. We plan to eventually integrate this methodology into the System Advisor Model (SAM) to improve performance model accuracy during periods of cloudiness. In this paper, we demonstrate generating DNI maps and mapping them to a solar field in CoPylot using 10 m resolution data from publicly available Sentinel-2 satellite data over the Crescent Dunes plant.

Mullin, Matthew↗

Laser scanner data processing and 3D modeling using a free and open source software

The laser scanning is a technology that allows in a short time to run the relief geometric objects with a high level of detail and completeness, based on the signal emitted by the laser and the corresponding return signal. When the incident laser radiation hits the object to detect, then the radiation is reflected. The purpose is to build a three-dimensional digital model that allows to reconstruct the reality of the object and to conduct studies regarding the design, restoration and/or conservation. When the laser scanner is equipped with a digital camera, the result of the measurement process is a set of points in XYZ coordinates showing a high density and accuracy with radiometric and RGB tones. In this case, the set of measured points is called “point cloud” and allows the reconstruction of the Digital Surface Model. Even the post-processing is usually performed by closed source software, which is characterized by Copyright restricting the free use, free and open source software can increase the performance by far. Indeed, this latter can be freely used providing the possibility to display and even custom the source code. The experience started at the Faculty of Engineering in Catania is aimed at finding a valuable free and open source tool, MeshLab (Italian Software for data processing), to be compared with a reference closed source software for data processing, i.e. RapidForm. In this work, we compare the results obtained with MeshLab and Rapidform through the planning of the survey and the acquisition of the point cloud of a morphologically complex statue.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

mzapy : An Open-Source Python Library Enabling Efficient Extraction and Processing of Ion Mobility Spectrometry-Mass Spectrometry Data in the MZA File Format

We have recently reported MZA, a new and simple mass spectrometry data structure based on the broadly supported HDF5 format and created to facilitate software development. While this format is inherently supportive of application development, the availability of a core library with standard mass spectrometry utilities greatly facilitates fast software development. Here, we present a Python library, mzapy, for efficient extraction and processing of mass spectrometry data in the MZA format. In addition to raw data extraction, mzapy contains supporting utilities enabling tasks including calibration, signal processing, peak finding, and generating plots. Being implemented in pure Python with minimal and largely standardized dependencies makes mzapy uniquely suited to application development in the multi-omics domain. The free and open source mzapy is built with extensibility in mind, and future development will support cloud computing and artificial intelligence/machine learning applications. The software source code is freely available at https://github.com/PNNL-m-q/mzapy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Portable Acceleration of CMS Computing Workflows with Coprocessors as a Service

Computing demands for large scientific experiments, such as the CMS experiment at the CERN LHC, will increase dramatically in the next decades. To complement the future performance increases of software running on central processing units (CPUs), explorations of coprocessor usage in data processing hold great potential and interest. Coprocessors are a class of computer processors that supplement CPUs, often improving the execution of certain functions due to architectural design choices. We explore the approach of Services for Optimized Network Inference on Coprocessors (SONIC) and study the deployment of this as-a-service approach in large-scale data processing. In the studies, we take a data processing workflow of the CMS experiment and run the main workflow on CPUs, while offloading several machine learning (ML) inference tasks onto either remote or local coprocessors, specifically graphics processing units (GPUs). With experiments performed at Google Cloud, the Purdue Tier-2 computing center, and combinations of the two, we demonstrate the acceleration of these ML algorithms individually on coprocessors and the corresponding throughput improvement for the entire workflow. This approach can be easily generalized to different types of coprocessors and deployed on local CPUs without decreasing the throughput performance. We emphasize that the SONIC approach enables high coprocessor usage and enables the portability to run workflows on different types of coprocessors.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Introduction: Neuromorphic Materials

The explosive growth in data collection and the need to process it efficiently, as well as the desire to automate increasingly complex tasks in transportation, medical care, manufacturing, security and many other fields have motivated a growing interest in neuromorphic computing. Unlike the binary, transistorbased ON/OFF logic gates and separate logic and memory functionalities employed in digital computing, neuromorphic computing is inspired by animal brains that use interconnected synapses and neurons to perform processing, storage and transmission of information at the same location, while only consuming ~20 W or less of power. Motivated by the brain’s efficiency, adaptability, self-learning and resiliency qualities, neuromorphic computing can be broadly defined as an approach to processing and storing information using hardware and algorithms inspired by models of biological neural systems. Present research in neuromorphic computing encompasses approaches that vary significantly in their degree of neuro-inspiration, from systems that only incorporate features such as asynchronous, event-driven operation or use crossbar arrays of non-volatile memory (NVM) elements to accelerate deep neural networks (DNNs), to designs that embrace the extreme parallelism, sparsity, reconfigurability, adaptability, complexity and stochasticity observed in nervous systems. The term ‘neuromorphic’ computing is often credited to Carver Mead, who in the 1980s investigated Si-based analog electronics to replicate functions of the animal retina. Earlier important advances in this field include the work of Frank Rosenblatt, who proposed the concept of the perceptron, Bernard Widrow, who used this concept to build one of the first analog neural networks, the Adaline and many other researchers (see ref. 6 for an historical perspective on neuromorphic computing). With the recent increase in the use of artificial intelligence and large language models, and rising concerns over the associated energy costs, interest in neuromorphic hardware has expanded rapidly. According to some estimates, driven largely by the drastic growth in the training use of artificial intelligence (AI) models using the current computing architectures, the energy cost of computing is projected to reach the energy supply worldwide by 2045. Furthermore, while this is not a realistic outcome, it means that, if more efficient computing technologies are not developed -- soon -- the world will soon become one where demand for energy and market constraints limit the continued increase of societal access to AI and cloud services from data centers. Data centers used for training and use of these models consume hundreds of terawatt hours of electricity, already past 4% of the US electricity demand.

Circuits↗