Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Fostering Geothermal Machine Learning Success: Elevating Big Data Accessibility and Automated Data Standardization in the Geothermal Data Repository: Preprint

The Department of Energy's (DOE) Geothermal Data Repository (GDR) has implemented improvements to both its data lakes and its data standards and automated data pipelines. The GDR data lakes have reduced storage and compute-related barriers to using large geothermal datasets, enabling these large datasets to be accessed by anyone with a modern computer and internet access. More recently, the GDR has been working to further reduce barriers through streamlining the data intake process, educating users on the process and requirements, and aiding users in accessing data from the data lakes. These improvements have augmented the quantity of datasets the GDR is able to accept into its data lakes and have enabled users who are new to cloud tools to access these datasets more easily, overall increasing the accessibility of big geothermal data for use in machine learning and other projects. In addition, the GDR now has built-in data standards and pipelines for drilling data, geospatial data, and distributed acoustic sensing (DAS) data. These standardization efforts aim to enhance the real-world applicability of geothermal machine learning outcomes by improving the quality of training data. Specifically, through standardizing high-value datasets, the GDR is reducing project-specific data curation requirements, thus allowing more time for actual research. By automating this process, the burden of standardization is lifted from the user, ultimately increasing the availability of standardized data.

accessibility↗
DOE OSTI2013

THE BOLOCAM GALACTIC PLANE SURVEY. VIII. A MID-INFRARED KINEMATIC DISTANCE DISCRIMINATION METHOD

We present a new distance estimation method for dust-continuum-identified molecular cloud clumps. Recent (sub-)millimeter Galactic plane surveys have cataloged tens of thousands of these objects, plausible precursors to stellar clusters, but detailed study of their physical properties requires robust distance determinations. We derive Bayesian distance probability density functions (DPDFs) for 770 objects from the Bolocam Galactic Plane Survey in the Galactic longitude range 7. Degree-Sign 5 {<=} l {<=} 65 Degree-Sign . The DPDF formalism is based on kinematic distances, and uses any number of external data sets to place prior distance probabilities to resolve the kinematic distance ambiguity (KDA) for objects in the inner Galaxy. We present here priors related to the mid-infrared absorption of dust in dense molecular regions and the distribution of molecular gas in the Galactic disk. By assuming a numerical model of Galactic mid-infrared emission and simple radiative transfer, we match the morphology of (sub-)millimeter thermal dust emission with mid-infrared absorption to compute a prior DPDF for distance discrimination. Selecting objects first from (sub-)millimeter source catalogs avoids a bias towards the darkest infrared dark clouds (IRDCs) and extends the range of heliocentric distance probed by mid-infrared extinction and includes lower-contrast sources. We derive well-constrained KDA resolutions for 618 molecular cloud clumps, with approximately 15% placed at or beyond the tangent distance. Objects with mid-infrared contrast sufficient to be cataloged as IRDCs are generally placed at the near kinematic distance. Distance comparisons with Galactic Ring Survey KDA resolutions yield a 92% agreement. A face-on view of the Milky Way using resolved distances reveals sections of the Sagittarius and Scutum-Centaurus Arms. This KDA-resolution method for large catalogs of sources through the combination of (sub-)millimeter and mid-infrared observations of molecular cloud clumps is generally applicable to other dust-continuum Galactic plane surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Large Eddy Simulations of Turbulent and Buoyant Flows in Urban and Complex Terrain Areas Using the Aeolus Model

Fast and accurate predictions of the flow and transport of materials in urban and complex terrain areas are challenging because of the heterogeneity of buildings and land features of different shapes and sizes connected by canyons and channels, which results in complex patterns of turbulence that can enhance material concentrations in certain regions. To address this challenge, we have developed an efficient three-dimensional computational fluid dynamics (CFD) code called Aeolus that is based on first principles for predicting transport and dispersion of materials in complex terrain and urban areas. The model can be run in a very efficient Reynolds average Navier–Stokes (RANS) mode or a detailed large eddy simulation (LES) mode. The RANS version of Aeolus was previously validated against field data for tracer gas and radiological dispersal releases. As a part of this work, we have validated the Aeolus model in LES mode against two different sets of data: (1) turbulence quantities measured in complex terrain at Askervein Hill; and (2) wind and tracer data from the Joint Urban 2003 field campaign for urban topography. As a third set-up, we have applied Aeolus to simulate cloud rise dynamics for buoyant plumes from high-temperature explosions. For all three cases, Aeolus LES predictions compare well to observations and other models. These results indicate that Aeolus LES can be used to accurately simulate turbulent flow and transport for a wide range of applications and scales.

42 ENGINEERING↗
DOE OSTI2012

Comparison of Simulated and Observed Continental Tropical Anvil Clouds and Their Radiative Heating Profiles

Vertically pointing millimeter-wavelength radar observations of anvil clouds extending from mesoscale convective systems (MCSs) that pass over an Atmospheric Radiation Measurement Program (ARM) field site in Niamey, Niger, are compared to anvil structures generated by the Weather Research and Forecasting (WRF) mesoscale model using six different microphysical schemes. The radar data provide the statistical distribution of the radar reflectivity values as a function of height and anvil thickness. These statistics are compared to the statistics of the modeled anvil cloud reflectivity at all altitudes. Requiring the model to be statistically accurate at all altitudes is a stringent test of the model performance. The typical vertical profile of radiative heating in the anvil clouds is computed from the radar observations. Variability of anvil structures from the different microphysical schemes provides an estimate of the inherent uncertainty in anvil radiative heating profiles. All schemes underestimate the optical thickness of thin anvils and cirrus, resulting in a bias of excessive net anvil heating in all of the simulations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

A Reference Model for Virtual Machine Launching Overhead

Cloud bursting is one of the key research topics in the cloud computing communities. A well designed cloud bursting module enables private clouds to automatically launch virtual machines (VMs) to public clouds when more resources are needed. One of the main challenges in developing a cloud bursting module is to decide when and where to launch a VM so that all resources are most effectively and efficiently utilized and the system performance is optimized. However, based on system operational data obtained from FermiCloud, a private cloud developed by the Fermi National Accelerator Laboratory for scientific workflows, the VM launching overhead is not a constant. It varies with physical resource utilization, such as CPU and I/O device utilizations, at the time when a VM is launched. Hence, to make judicious decisions as to when and where a VM should be launched, a VM launching overhead reference model is needed. In this paper, we first develop a VM launching overhead reference model based on operational data we have obtained on FermiCloud. Second, we apply the developed reference model on FermiCloud and compare calculated VM launching overhead values based on the model with measured overhead values on FermiCloud. Our empirical results on FermiCloud indicate that the developed reference model is accurate. We believe, with the guidance of the developed reference model, efficient resource allocation algorithms can be developed for cloud bursting process to minimize the operational cost and resource waste.

Wu, Hao↗
DOE OSTI2014

Cloud-Based Model Calibration Using OpenStudio: Preprint

OpenStudio is a free, open source Software Development Kit (SDK) and application suite for performing building energy modeling and analysis. The OpenStudio Parametric Analysis Tool has been extended to allow cloud-based simulation of multiple OpenStudio models parametrically related to a baseline model. This paper describes the new cloud-based simulation functionality and presents a model cali-bration case study. Calibration is initiated by entering actual monthly utility bill data into the baseline model. Multiple parameters are then varied over multiple iterations to reduce the difference between actual energy consumption and model simulation results, as calculated and visualized by billing period and by fuel type. Simulations are per-formed in parallel using the Amazon Elastic Cloud service. This paper highlights model parameterizations (measures) used for calibration, but the same multi-nodal computing architecture is available for other purposes, for example, recommending combinations of retrofit energy saving measures using the calibrated model as the new baseline.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2012

Companies Reaching for the Clouds for Computing Power

By now, we’ve likely all at least heard of cloud computing, and to some extent may grasp what it’s all about. But after delving into a recent article in The New York Times, I came to realize just how big of a deal it is--much bigger than my own limited experience with it had allowed me to see. Cloud computing is the use of hardware or software computing resources that are delivered as a service over a network, typically via the web. The gist of it is, almost anything you can imagine doing with your computer system doesn’t have to physically exist on your system or in your office in order to be accessible to you. You can entrust remote services with your data, software, and computation. It’s easier, and also much less expensive.

Madison, Alison L.↗
DOE OSTI2015

Collaborative Project: High-resolution Global Modeling of the Effects of Subgrid-Scale Clouds and Turbulence on Precipitating Cloud Systems

We proposed to implement, test, and evaluate recently developed turbulence parameterizations, using a wide variety of methods and modeling frameworks together with observations including ARM data. We have successfully tested three different turbulence parameterizations in versions of the Community Atmosphere Model: CLUBB, SHOC, and IPHOC. All three produce significant improvements in the simulated climate. CLUBB will be used in CAM6, and also in ACME. SHOC is being tested in the NCEP forecast model. In addition, we have achieved a better understanding of the strengths and limitations of the PDF-based parameterizations of turbulence and convection.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Physics-Informed Machine Learning from Observations for Clouds, Convection, and Precipitation Parameterizations and Analysis

Fusing learning from Earth observations from space and from the ground (e.g., ARM) with newly developed interpretable and generalizable physics-informed model structures to improve the parameterization of clouds and convection and advance the simulation, understanding and analysis of hydrological extreme events. This covers the prescribed foci of (2) predictive modeling using a hierarchy of models and (3) insight gleaned from complex data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Development of a Multi-Robot System for Autonomous Inspection of Nuclear Waste Tank Pits

This paper introduces the overall design plan, development timeline, and preliminary progress of the Autonomous Pit Exploration System project. This project aims to develop an advanced multi-robot system for the efficient inspection of nuclear waste-storage tank pits. The project is structured into three phases: Phase 1 involves data collection and interface definition in collaboration with Hanford Site experts and university partners, focusing on tank riser geometry and hardware solutions. Phase 2 includes the selection of sensors and robot components, detailed mechanical design, and prototyping. Phase 3 integrates all components into a cohesive system managed by a master control package which also incorporates digital twin and surrogate models, and culminates in comprehensive testing and validation at a simulated tank pit at the Idaho National Laboratory. Additionally, the system’s communication design ensures coordinated operation through shared data, power, and control signals. For transportation and deployment, an electric vehicle (EV) is chosen to support the system for a full 10 h shift with better regulatory compliance for field deployment. A telescopic arm design is selected for its simple configuration and superior reach capability and controllability. Preliminary testing utilizes an educational robot to demonstrate the feasibility of splitting computational tasks between edge and cloud computers. Successful simultaneous localization and mapping (SLAM) tasks validate our distributed computing approach. More design considerations are also discussed, including radiation hardness assurance, SLAM performance, software transferability, and digital twinning strategies.

Nuclear waste management↗
DOE OSTI2025

A new method for diagnosing effective radiative forcing from aerosol–cloud interactions in climate models

Aerosol–cloud interactions (ACIs) are a leading source of uncertainty in estimates of the historical effective radiative forcing (ERF). One reason for this uncertainty is the difficulty in estimating the ERF from aerosol–cloud interactions (ERFaci) in climate models, which typically requires multiple calls to the radiation code. Most commonly used methods also cannot disentangle the contributions from different processes to ERFaci. Here, we develop a new, computationally efficient method for estimating the shortwave (SW) ERFaci from liquid clouds using histograms of monthly averaged cloud fraction partitioned by cloud droplet effective radius (r e ) and liquid water path (LWP). Multiplying the histograms with SW cloud radiative kernels gives the total SW ERFaci from liquid clouds, which can be decomposed into contributions from the Twomey effect, LWP adjustments, and cloud fraction (CF) adjustments. We test the method with data from five CMIP6-era models, using the Moderate Resolution Imaging Spectroradiometer (MODIS) satellite instrument simulator to generate the histograms. Our method gives similar total SW ERFaci estimates to other established methods in regions of prevalent liquid cloud and indicates that the Twomey effect, LWP adjustments, and CF adjustments have contributed -0.34 ± 0.23, -0.22 ± 0.13, and -0.09 ± 0.11 W m -2 , respectively, to the effective radiative forcing of the climate since 1850 in the ensemble mean (95 % confidence). These results demonstrate that widespread adoption of a MODIS r e –LWP joint histogram diagnostic would allow the SW ERFaci and its components to be quickly and accurately diagnosed from climate model outputs, a crucial step for reducing uncertainty in the historical ERF.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Collisional excitation of C2H− by H2: New interaction potential and scattering calculations

Interstellar anions play an important role in astrochemistry as being tracers of the physical and chemical conditions in cold molecular clouds and circumstellar gas. The local thermodynamic equilibrium is generally not fulfilled in media where anions are detected and radiative and collisional data are required to model the observed lines. The C2H− anion has not yet been detected in the interstellar medium; however, collisional data could be used for non-LTE models that would help in identifying the most intense lines. For this purpose, we have computed the first 4D potential energy surface (PES) of the C2H−–H2 complex using an explicitly correlated coupled-cluster approach. The PES is characterized by a single deep minimum with a well-depth of 924.96 cm−1. From this interaction potential, we derived excitation cross sections and rate coefficients of C2H− induced by collisions with para- and ortho-H2. The results obtained for collisions with para-H2 are compared to previous calculations performed using a 2D-PES obtained from an average over H2 rotations.

Chemistry↗
DOE OSTI2018

Feature Detection From LiDAR Point Cloud and Aerial Imagery Using Distance Dependent Chinese Restaurant Process [Slides]

It is possible to integrate data captured by UAS from multiple data modalities spatially. We can detect features by combining information across images and LiDAR point clouds. Performance issues exist with the current algorithms in place & could be improved. We hope to test it for at least 500 iterations. Post processing of data takes enormous efforts in case of data generated by proprietary software tools…We intend to develop a better framework for seamless integration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Structure Perception in 3D Point Clouds

Understanding human perception is critical to the design of effective visualizations. The relative benefits of using 2D versus 3D techniques for data visualization is a complex decision space, with varying levels of uncertainty and disagreement in both the literature and in practice. This study aims to add easily reproducible, empirical evidence on the role of depth cues in perceiving structures or patterns in 3D point clouds. We describe a method to synthesize a 3D point cloud that contains a 3D structure, where 2D projections of the data strongly resemble a Gaussian distribution. We performed a within-subjects structure identification study with 128 participants that compared scatterplot matrices (canonical 2D projections) and 3D scatterplots under three types of motion: rotation, xy-translation, and z-translation. We found that users could consistently identify three separate hidden structures under rotation, while those structures remained hidden in the scatterplot matrices and under translation. This work contributes a set of 3D point clouds that provide definitive examples of 3D patterns perceptible in 3D scatterplots under rotation but imperceptible in 2D scatterplots.

data analysis↗
DOE OSTI2021

Structure Perception in 3D Point Clouds: Preprint

Understanding human perception is critical to the design of ef- fective visualizations. The relative benefits of using 2D versus 3D techniques for data visualization is a complex decision space, with varying levels of uncertainty and disagreement in both the liter- ature and in practice. This study aims to add easily reproducible, empirical evidence on the role of depth cues in perceiving structures or patterns in 3D point clouds. We describe a method to synthesize a 3D point cloud that contains a 3D structure, where 2D projec- tions of the data strongly resemble a Gaussian distribution. We performed a within-subjects structure identification study with 128 participants that compared scatterplot matrices (canonical 2D projections) and 3D scatterplots under three types of motion: rota- tion, xy-translation, and z-translation. We found that users could consistently identify three separate hidden structures under ro- tation, while those structures remained hidden in the scatterplot matrices and under translation. This work contributes a set of 3D point clouds that provide definitive examples of 3D patterns per- ceptible in 3D scatterplots under rotation but imperceptible in 2D scatterplots.

data analysis↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
DOE OSTI2021

Coding the Computing Continuum: Fluid Function Execution in Heterogeneous Computing Environments

Advances in network technologies have greatly decreased barriers to accessing physically distributed computers. This newfound accessibility coincides with increasing hardware specialization, creating exciting new opportunities to dispatch workloads to the best resource for a specific purpose, rather than those that are closest or most easily accessible. We present Delta, a service designed to intelligently schedule function-based workloads across a distributed set of heterogeneous computing resources. Delta implements an extensible architecture in which different predictors and scheduling algorithms can be integrated to provide dynamically evolving estimates of function execution times on different resources-estimates that can be used to determine the most appropriate location for execution. We describe predictors for function runtime, data transfer time, and cold-start resource provisioning and configuration delay; dynamic learning methods that update predictor models over time; and scheduling strategies that take into account both function and endpoint information. We show that these methods can halve workload makespan when compared with a strategy that selects the fastest resource, and decrease makespan by a factor of five when compared to a round robin strategy, when deployed on a heterogeneous testbed with resources ranging from a Raspberry Pi to a GPU node in an academic cloud.

Computing continuum↗