Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2013

Cloud Properties and Radiative Heating Rates for TWP

A cloud properties and radiative heating rates dataset is presented where cloud properties retrieved using lidar and radar observations are input into a radiative transfer model to compute radiative fluxes and heating rates at three ARM sites located in the Tropical Western Pacific (TWP) region. The cloud properties retrieval is a conditional retrieval that applies various retrieval techniques depending on the available data, that is if lidar, radar or both instruments detect cloud. This Combined Remote Sensor Retrieval Algorithm (CombRet) produces vertical profiles of liquid or ice water content (LWC or IWC), droplet effective radius (re), ice crystal generalized effective size (Dge), cloud phase, and cloud boundaries. The algorithm was compared with 3 other independent algorithms to help estimate the uncertainty in the cloud properties, fluxes, and heating rates (Comstock et al. 2013). The dataset is provided at 2 min temporal and 90 m vertical resolution. The current dataset is applied to time periods when the MMCR (Millimeter Cloud Radar) version of the ARSCL (Active Remotely-Sensed Cloud Locations) Value Added Product (VAP) is available. The MERGESONDE VAP is utilized where temperature and humidity profiles are required. Future additions to this dataset will utilize the new KAZR instrument and its associated VAPs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Cloud forensics and incident response platform

A system, method, and device for cloud forensics and incident response is provided. In an embodiment, a computer-implemented method for performing cloud forensics and incident response includes intercepting, by a cloud incident response module (CIRM), communication between a virtual machine (VM) and a hypervisor. The method also includes extracting, by the CIRM, data from the communication between the VM and the hypervisor according to a forensic policy. Intercepting and extracting the data are transparent to the VM and to the hypervisor. Intercepting and extracting the data are independent of the VM and the hypervisor.

Urias, Vincent↗
DOE OSTI2023

60 Microlensing Events from the Three Years of Zwicky Transient Facility Phase One

Abstract Microlensing events have historically been discovered throughout the Galactic bulge and plane by surveys designed solely for that purpose. We conduct the first multiyear search for microlensing events on the Zwicky Transient Facility (ZTF), an all-sky optical synoptic survey that observes the entire visible northern sky every few nights. We discover 60 high-quality microlensing events in the 3 yr of ZTF-I using the bulk lightcurves in the ZTF Public Data Release 5.19 of our events are found outside of the Galactic plane (∣ b ∣ ≥ 10°), nearly doubling the number of previously discovered events in the stellar halo from surveys pointed toward the Magellanic Clouds and the Andromeda galaxy. We also record 1558 ongoing candidate events as potential microlensing that can continue to be observed by ZTF-II for identification. The scalable and computationally efficient methods developed in this work can be applied to future synoptic surveys, such as the Vera C. Rubin Observatory’s Legacy Survey of Space and Time and the Nancy Grace Roman Space Telescope, as they attempt to find microlensing events in even larger and deeper data sets.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Concentric Spherical GNN for 3D Representation Learning

Learning 3D representations that generalize well to arbitrarily oriented inputs is a challenge of practical importance in applications varying from computer vision to physics and chemistry. We propose a novel multi-resolution convolutional architecture for learning over concentric spherical feature maps, of which the single sphere representation is a special case. Our hierarchical architecture is based on alternatively learning to incorporate both intra-sphere and inter-sphere information. We show the applicability of our method for two different types of 3D inputs, mesh objects, which can be regularly sampled, and point clouds, which are irregularly distributed. We also propose an efficient mapping of point clouds to concentric spherical images, thereby bridging spherical convolutions on grids with general point clouds. We demonstrate the effectiveness of our approach in improving state-of-the-art performance on 3D classification tasks with rotated data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Deliver Signal Phase and Timing (SPAT) for Energy Optimization of Vehicle Cohort Via Cloud-Computing and LTE Communications

Predictive Signal Phase and Timing (SPAT) message set is one fundamental building block for vehicle-to-infrastructure (V2I) applications such as Eco-Approach and Departure (EAD) at traffic signal controlled urban intersections. Among the two complementary communication methods namely short-range sidelink (PC5) and long-range cellular radio link (Uu), this paper documents the work with long-range link: the complete data chain includes connecting to the traffic signals via existing backhaul communication network, collecting the raw signal phase state data, predicting the signal state changes and delivering the SPAT data via a geofenced service to requests over HTTP protocols. An Application Programming Interface (API) library is developed to support various cellular data transmission reduction and latency improvement techniques. An emulation-based algorithm is applied to predict the traffic signal state changes to provide adequate prediction horizon (e.g., at minimum 2 minutes) for the cohort energy optimization. In fact, the same connectivity and SPAT delivery methodology has been applied to traffic signalized intersections nationwide in the United States upon public agency approvals for access to their firewalled traffic control network and signal control systems or directly to individual controllers. This methodology proves its effectiveness and potential for rapid growth of such SPAT deliveries at mass production scale without needing infrastructure hardware retrofit or excessive communication means. To support the energy optimization of light and heavy-duty vehicle cohorts of mixed automation and propulsion systems (EV, ICE and hybrid), the connection and SPAT deliveries at two sites were completed, including public roads in Washtenaw County, Michigan and closed track test sites at American Center for Mobility (ACM) in Ypsilanti, Michigan. However, only closed test track results at ACM will be presented in this paper. A neuroevolution based optimizer is developed and implemented to control the speed of a vehicle cohort with different propulsion systems and automation levels. Closed track tests showed significant energy savings of the cohort operation.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Performance Profile of Transformer Fine-Tuning in Multi-GPU Cloud Environments

The study presented here focuses on performance characteristics and trade-offs associated with running machine-learning tasks in multi-GPU environments on both on-site cloud computing resources and commercial cloud services (Azure). Specifically, this study examines these tradeoffs by examining the performance of training and fine-tuning of transformer-based deep-learning (DL) networks on clinical notes and data, a task of critical importance in the medical domain. To this end, we perform DL-related experiments on the widely deployed NVIDIA V100 GPUs and on the newer A100 GPUs connected via NVLink or PCIe. This study analyzes the execution time of major operations to train DL models and investigate popular options to optimize each of them. We examine and present the findings on the impacts that various operations (e.g. data loading into GPUs, training, fine-tuning), optimizations, and system configurations (single vs. multi-GPU, NVLink vs. PCIe) have on the overall training performance.

Begoli, Edmon↗
DOE OSTI2022

Adaptive stabilization of quantum circuits executed on unstable devices

Conventional computers have evolved to device components that demonstrate failure rates of 10 −17 or less, while current quantum computing devices typically exhibit error rates of 10 −2 or greater. This raises concerns about the reliability and reproducibility of the results obtained from quantum computers. The problem is highlighted by experimental observation that today’s NISQ devices are inherently unstable. Remote quantum cloud servers typically do not provide users with an ability to calibrate the device themselves. Using inaccurate characterization data for error mitigation can have devastating impact on reproducibility. In this study, we investigate if one can infer the critical channel parameters dynamically from the noisy binary output of the executed quantum circuit and use it to improve program stability. An open question however is how well does this methodology scale. We discuss the efficacy and efficiency of our adaptive algorithm using canonical quantum circuits such as the uniform superposition circuit. Our metric of performance is the Hellinger distance between the post-stabilization observations and the reference (ideal) distribution.

Dasgupta, Samudra↗
DOE OSTI2020

Progress on the National Solar Radiation Data Base (NSRDB): A new DNI computation

This study introduces a new technique to compute direct normal irradiance (DNI) for improving the National Solar Radiation Data Base (NSRDB). A finite-surface integration algorithm is developed to compute solar radiation in differential solid angles and efficiently infer its contribution to a surface perpendicular to the solar direction. A lookup table of cloud bi-directional transmittance distribution function (BTDF) is developed by use of the discrete ordinates radiative transfer (DISORT) model for possible solar and observing directions and various cloud optical and microphysical properties. In each solar incident direction, DNI is given by the cloud BTDFs from approximately 200 differential solid angles. The simulated DNI is calibrated and evaluated using surface observations at the National Renewable Energy Laboratory's (NREL's) Solar Energy Research Laboratory (SRRL) and the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) facility.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Robust implementation of generative modeling with parametrized quantum circuits

Although the performance of hybrid quantum-classical algorithms is highly dependent on the selection of the classical optimizer and the circuit ansätze, a robust and thorough assessment on-hardware of such features has been missing to date. From the optimizer perspective, the primary challenge lies in the solver’s stochastic nature, and their significant variance over the random initialization. Therefore, a robust comparison requires one to perform several training curves for each solver before one can reach conclusions about their typical performance. Since each of the training curves requires the execution of thousands of quantum circuits in the quantum computer, such a robust study remained a steep challenge for most hybrid platforms available today. In this work, we leverage on Rigetti’s Quantum Cloud Services (QCS™) to overcome this implementation barrier, and we study the on-hardware performance of the data-driven quantum circuit learning (DDQCL) for three different state-of-the-art classical solvers, and on two-different circuit ansätze associated to different entangling connectivity graphs for the same task. Additionally, we assess the gains in performance from varying circuit depths. To evaluate the typical performance associated with each of these settings in this benchmark study, we use at least five independent runs of DDQCL towards the generation of quantum generative models capable of capturing the patterns of the canonical Bars and Stripes dataset. In this experimental benchmarking, the gradient-free optimization algorithms show an outstanding performance compared to the gradient-based solver. In particular, one of them had better performance when handling the unavoidable noisy objective function to be minimized under experimental conditions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Emulator of PR‐DNS: Accelerating Dynamical Fields With Neural Operators in Particle‐Resolved Direct Numerical Simulation

Abstract Particle‐resolved direct numerical simulations (PR‐DNS) play an increasing role in investigating aerosol‐cloud‐turbulence interactions at the most fundamental level of processes. However, the high computational cost associated with high resolution simulations poses considerable challenges for large domain or long duration simulation using PR‐DNS. To address these issues, here we present an emulator of the complex physics‐based PR‐DNS developed by use of the data‐driven Fourier Neural Operator (FNO) method. The effectiveness of the method is showcased by presenting turbulence and temperature fields in a two‐dimensional space. The results demonstrate high accuracy at various resolutions and the emulator is two orders of magnitude cheaper in terms of computational demand compared to the physics‐based PR‐DNS model. Furthermore, the FNO emulator exhibits strong generalization capabilities for different initial conditions and ultra‐high‐resolution without the need to retrain models. These findings highlight the potential of the FNO method as a promising tool to simulate complex fluid dynamics problems with high accuracy, computational efficiency, and generalization capabilities, enhancing our understanding of the aerosol‐cloud‐precipitation system.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

ornladios/ADIOS2

ADIOS 2: The Adaptable Input Output (I/O) System version 2 is an open-source framework that addresses scientific data management challenges, e.g. scalable parallel I/O, as we approach the exascale era in high-performance computing (HPC). ADIOS 2 bindings are available in C++, C, Fortran, Python and can be used on supercomputers, personal computers, and cloud systems running on Linux, macOS and Windows. ADIOS 2 has out-of-the-box support for MPI and serial environments.

ECP↗
DOE OSTI2025

New Particle Formation and Growth in the Houston Atmosphere During TRACER (Final Report)

From 2020-2025, researchers from UC Irvine, UC Riverside, and Colorado State University collaborated on a Department of Energy-funded project to understand how airborne particles form and grow in urban atmospheres, conducting an intensive field campaign in Houston, Texas during summer 2022. Using advanced instruments to measure gas-phase chemicals, particle composition, and a specialized chamber to study particle growth, the team discovered that sulfur-containing compounds from industrial and power plant emissions are the dominant driver of new particle formation in Houston, with particles typically forming locally in the city and growing as air moves away in the urban plume. The research revealed an important methodological insight: measurements from fixed ground stations can be misleading when interpreting how particles actually evolve as air masses move, which has significant implications for how scientists worldwide interpret atmospheric observations. These findings improve understanding of urban air quality and help reduce uncertainties in climate models, since these particles play critical roles in cloud formation and Earth's radiation balance, while also providing detailed information about ultrafine particle composition relevant to public health. The project trained three doctoral students, developed enhanced computer models for urban particle formation, and made all data publicly available through the DOE Atmospheric Radiation Measurement data archive for use by the broader scientific community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

Challenges and opportunities in understanding microbial communities with metagenome assembly (accompanied by IPython Notebook tutorial)

Metagenomic investigations hold great promise for informing the genetics, physiology, and ecology of environmental microorganisms. Current challenges for metagenomic analysis are related to our ability to connect the dots between sequencing reads, their population of origin, and their encoding functions. Assembly-based methods reduce dataset size by extending overlapping reads into larger contiguous sequences (contigs), providing contextual information for genetic sequences that does not rely on existing references. These methods, however, tend to be computationally intensive and are again challenged by sequencing errors as well as by genomic repeats. While numerous tools have been developed based on these methodological concepts, they present confounding choices and training requirements to metagenomic investigators. To help with accessibility to assembly tools, this review also includes an IPython Notebook metagenomic assembly tutorial. This tutorial has instructions for execution any operating system using Amazon Elastic Cloud Compute and guides users through downloading, assembly, and mapping reads to contigs of a mock microbiome metagenome. Despite its challenges, metagenomic analysis has already revealed novel insights into many environments on Earth. As software, training, and data continue to emerge, metagenomic data access and its discoveries will to grow.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n -Moment Bulk Microphysics Scheme

The small-scale microphysical processes governing the formation of precipitation particles cannot be resolved explicitly by cloud resolving and climate models. Instead, they are represented by microphysics schemes that are based on a combination of theoretical knowledge, statistical assumptions, and fitting to data (“tuning”). Historically, tuning was done in an ad hoc fashion, leading to parameter choices that are not explainable or repeatable. Recent work has treated it as an inverse problem that can be solved by Bayesian inference. The posterior distribution of the parameters given the data—the solution of Bayesian inference—is found through computationally expensive sampling methods, which require over $\mathcal{O}$(10 5 ) evaluations of the forward model; this is prohibitive for many models. We present a proof of concept of Bayesian learning applied to a new bulk microphysics scheme named “Cloudy,” using the recently developed Calibrate-Emulate-Sample (CES) algorithm. Cloudy models collision-coalescence and collisional breakup of cloud droplets with an adjustable number of prognostic moments and with easily modifiable assumptions for the cloud droplet mass distribution and the collision kernel. The CES algorithm uses machine learning tools to accelerate Bayesian inference by reducing the number of forward evaluations needed to $\mathcal{O}$(10 2 ). It also exhibits a smoothing effect when forward evaluations are polluted by noise. In a suite of perfect-model experiments, we show that CES enables computationally efficient Bayesian inference of parameters in Cloudy from noisy observations of moments of the droplet mass distribution. In an additional imperfect-model experiment, a collision kernel parameter is successfully learned from output generated by a Lagrangian particle-based microphysics model.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Sensing Electrical Networks Securely & Economically (SENSE)

The growing adoption of distributed energy resources (DERs) like battery energy storage systems and roof top solar/PV and the rapid penetration of electric vehicles (EVs), the electric grid is undergoing a major transformation with elevated stress on legacy grid assets. Despite a lot of expenditure to address these challenges, both in dollars and manpower, utilities have not been able to receive the value that was promised. The gains have been most visible at the transmission and substation level, especially where the main objective was improving operational and economic efficiency for the utility. Improving visibility and control at a few select points enhances the existing and established paradigm of centralized command and control. With changing load patterns, load types and the overall transition to an “active grid”, the centralized control and coordination paradigm gets challenged. To address the challenges, a new architecture and mechanism is needed, one that supports decentralized control and decision making, extracting value streams at the grid edge, particularly as the changes are fueled by transitions occurring in the distribution system. To address this, a communications and data processing platform, “GAMMA” was developed and demonstrated through the project. At the heart of the platform, are distributed, intelligent edge nodes with sensing and compute capabilities, that can record and analyze information locally. They are embedded in sensors and actuators specific to different distribution system applications. Phase 1 of the project focused on developing novel sensor technology that can be used for monitoring utility pole top distribution transformers. The sensors were designed with the objective of being low-cost, communicating with the GAMMA cloud using novel “delay-tolerant” networking using Bluetooth and a secure mobile application. They were non-intrusive in nature so that they can be installed quickly in the field, resulting in overall low cost of deployment and operations. Following the successful completion of Phase 1, the team manufactured 100 units for a field demonstration in Phase 2. The field demonstration was carried out on two real feeder systems with the local utility partner. In total, 100 sensors were installed and operated over a period of 6 months in the state of Georgia. The platform is operational end to end, with the cloud infrastructure deployed on a distributed, serverless environment that can serve multiple data streams, an analytics engine and a portal to securely view the data from multiple assets. The data collected through the GAMMA Mobile Phone app showcased the viability of the novel delay tolerant networking architecture, and the data processing algorithms developed through the course of the project, were successful in extracting important information about the overall network, improving the utility’s visibility and situational awareness in the distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

The E3SM Diagnostics Package (E3SM Diags v2.7): a Python-based diagnostics package for Earth system model evaluation

Abstract. The E3SM Diagnostics Package (E3SM Diags) is a modern, Python-based Earth system model (ESM) evaluation tool (with Python module name e3sm_diags), developed to support the Department of Energy (DOE) Energy Exascale Earth System Model (E3SM). E3SM Diags provides a wide suite of tools for evaluating native E3SM output, as well as ESM data on regular latitude–longitude grids, including output from Coupled Model Intercomparison Project (CMIP) class models. E3SM Diags is modeled after the National Center for Atmospheric Research (NCAR) Atmosphere Model Working Group (AMWG, 2022) diagnostics package. In its version 1 release, E3SM Diags included a set of core essential diagnostics to evaluate the mean physical climate from model simulations. As of version 2.7, more process-oriented and phenomenon-based evaluation diagnostics have been implemented, such as analysis of the quasi-biennial oscillation (QBO), the El Niño–Southern Oscillation (ENSO), streamflow, the diurnal cycle of precipitation, tropical cyclones, ozone and aerosol properties. An in situ dataset from DOE's Atmospheric Radiation Measurement (ARM) program has been integrated into the package for evaluating the representation of simulated cloud and precipitation processes. This tool is designed with enough flexibility to allow for the addition of new observational datasets and new diagnostic algorithms. Additional features include customizable figures; streamlined installation, configuration and execution; and multiprocessing for fast computation. The package uses an up-to-date observational data repository maintained by its developers, where recent datasets are added to the repository as they become available. Finally, several applications for the E3SM Diags module were introduced to fit a diverse set of use cases from the scientific community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

Supplement to LA-UR-17-21218: Application to SSVD

We apply the formalism derived in LA-UR-17-21218 to the prescription for an RMI-based self-similar velocity distribution (SSVD) derived by Ham- merberg et al.. We compute analytically the true [mt(t)] and inferred [mi(t)] ejecta mass arriving at the piezoelectric sensor for several shots de- scribed in the literature and compare the results to the data. We nd that while the \RMI + SSVD" prescription gives rise to decent estimates for the nal accumulated mass at the pin, the time-dependent accumulation rises too sharply and linearly to agree with data. We also compute the time-dependent pressure and voltage at the sensor, and compare the latter to data. The pres- sure does not rise smoothly from zero, instead exhibiting a strong surge as the leading edge of the ejecta cloud arrives, which produces an initial sharp spike in the voltage trace, which is not observed. These inconsistencies result from a discontinuity in the prescribed self-similar velocity distribution at maximum relative velocity.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Landscape-scale Characterization of Arctic Tundra Vegetation Composition, Structure, and Function with a Multi-sensor Unoccupied Aerial System: Supporting Data.

Airborne remote sensing data collected using the Brookhaven National Laboratory's (BNL) heavy-lift unoccupied aerial system (UAS) octocopter platform, the Osprey, operated by the Terrestrial Ecosystem Science and Technology (TEST) group (https://www.bnl.gov/testgroup). This package includes data from three flights flown over the NGEE-Arctic Council, Kougarok and Teller sites in July, 2018. The Osprey is a multi-sensor UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) and thermal infrared (TIR) surface 'skin' temperature imagery, as well as surface reflectance at 1 nm intervals in the visible to near-infrared spectral range 350 - 1000 nm measured at regular intervals along each flight path. Derived image products include ortho-mosaiced RGB and TIR images, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, digital terrain model (DTM), and a canopy height model (CHM). In addition, a VNIR surface reflectance file is provided for the trigger locations collected during each flight campaign. Ancillary aircraft data, flight mission parameters, and general flight conditions provided by the onboard flight and data collection computers are also included. Unprocessed and processed data products are included in this package (processing levels 0-3). Data and metadata are provided as text (*.txt, *.json, *.kml, *hdr, *.enp), tabular (*.dat, *.csv, *.waypoint, ENVI format (no extension)), point cloud (*.laz) and image (*.jpg, *.tif, *png) formats. This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy?s Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗