Search NASA⌕ Search

SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Transforming Energy Through Computational Excellence: NREL's Computational Science Center

Computational methods underpin advancing the science and engineering of energy efficiency, sustainable transportation, renewable power technologies, and developing a knowledge base to optimize energy systems. NREL's Computational Science Center (CSC) proudly focuses on providing the service of computing, advancing the science of computing, and enabling NREL's clean energy mission.

applied mathematics↗

Cumulus: NASA's Cloud Based Distributed Active Archive Center Prototype

NASAs Earth Science Data System (ESDS) Program serves as a central cog in order to facilitate the implementation of NASA's Earth Science strategic plan. Since 1994, the ESDS Program has committed to the full and open sharing of Earth science data obtained from NASA instruments to all users. One of the key responsibilities of the ESDS Program is to continuously evolve the entire data and information system to maximize returns on the collected NASA data. An independent review was conducted in 2015 to holistically review the EOSDIS in order to identify gaps. The review recommendations were to investigate two areas: one, whether commercial cloud providers offer potential for storage, processing, and operational efficiencies, and two, the potential development of new data access and analysis paradigms. In response, ESDS has initiated several prototypes investigating the advantages and risks of leveraging cloud computing. This paper describes one such prototyping activity named Cumulus. Cumulus is being designed and developed as a "native" cloud-based data ingest, archive and management system that can be used for all future NASA Earth science data streams. Cumulus will foster design of new analysisvisualization tools that can leverage collocated data from all of the distributed DAACs as well as elastic cloud computing resources.

Earth Science Informatics↗

WE-B-BRD-01: Innovation in Radiation Therapy Planning II: Cloud Computing in RT

As defined by the National Institute of Standards and Technology, cloud computing is “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.” Despite the omnipresent role of computers in radiotherapy, cloud computing has yet to achieve widespread adoption in clinical or research applications, though the transition to such “on-demand” access is underway. As this transition proceeds, new opportunities for aggregate studies and efficient use of computational resources are set against new challenges in patient privacy protection, data integrity, and management of clinical informatics systems. In this Session, current and future applications of cloud computing and distributed computational resources will be discussed in the context of medical imaging, radiotherapy research, and clinical radiation oncology applications. Learning Objectives: Understand basic concepts of cloud computing. Understand how cloud computing could be used for medical imaging applications. Understand how cloud computing could be employed for radiotherapy research.4. Understand how clinical radiotherapy software applications would function in the cloud.

60 APPLIED LIFE SCIENCES↗

Smart connected worker edge platform for smart manufacturing: Part 1—Architecture and platform design

Abstract The challenge of sustainably producing goods and services for healthy living on a healthy planet requires simultaneous consideration of economic, societal, and environmental dimensions in manufacturing. Enabling technology for data driven manufacturing paradigms like Smart Manufacturing (a.k.a. Industry 4.0) serve as the technological backbone from which sustainable approaches to manufacturing can be implemented. Unfortunately, these technologies are typically associated with broader and deeper factory automation that is often too expensive and complex for the small and medium sized manufacturers (SMMs) that comprise the majority of manufacturing business in the USA and for whom their most valuable asset are the people whose jobs automation while replace. This paper describes an edge intelligent platform to integrate internet‐of‐things technologies with computing hardware, software, computational workflows for machine learning, and data ingestion, enabling SMMs to transition into smart manufacturing paradigms by leveraging the intelligence of their people. The platform leverages consumer grade electronics and sensors (affordable and portable), customized software with open source software packages (accessible), and existing communication network infrastructures (scalable). The software systems are implemented via Kubernetes orchestration of Docker containerization to ensure scalability and programmability. The platform is adaptive via computational workflow engines that produce information from data by processing with low‐cost edge computing devices while efficiently accessing resources of cloud servers as needed. The proposed edge platform connects workers to technological resources that provide computational intelligence (i.e., silicon‐based sensing and computation for data collection and contextualization) to enable decision making at the edge of advanced manufacturing.

Kim, Yoon G.↗

An Assessment of New Satellite Data Products for the Development of a Long-Term Global Solar Resource at 10-100 km

A project representing an effort to reprocess the NASA based solar resource data sets is reviewed. The effort represented a collaboration between NASA, NOAA, NREL and the SUNY-Albany and aimed to deliver a 10 km resolution, 3-hourly data set spanning from 1983 through near-present. Part of the project was to transition project capability to NREL for annual processing to extend data set. Due to delays in the key input project called ISCCP, we evaluate only Beta versions of this data set and also introduce the potential use of another NASA Langley based cloud data set for the CERES project. The CERES project uses these cloud properties to compute global top-of-atmosphere and surface fluxes at the 1x1 degree resolution. Here, we also briefly discuss these data sets in potential usage for solar resource benchmarking.

Stackhouse, Paul W., Jr.↗

HPC and Cloud Convergence Beyond Technical Boundaries: Strategies for Economic Sustainability, Standardization, and Data Accessibility

At the IEEE/ACM International Conference for High-Performance Computing, Networking, Storage, and Analysis (SC23), held in Denver, experts discussed the convergence of high-performance computing and cloud computing. Experts explored how this integration could address current scientific computing limitations, enhance computational capabilities, and foster global collaboration while focusing on economic, security, technical, and community challenges and opportunities.

97 MATHEMATICS AND COMPUTING↗

New Developments for Physically-based Falling Snow Retrievals over Land in Preparation for GPM

The NASA Global Precipitation Measurement mission (GPM) concept centers on deploying a Core spacecraft carrying a dual-frequency precipitation radar and a microwave radiometric imager with channels from 10 to 183 GHz to serve as a precipitation physics observatory and a calibration reference to unify a constellation of dedicated and operational passive microwave sensors. Because of the extended orbit of the Core (plus or minus 65 deg) and the enhanced dual frequency radar and high frequency radiometer, GPM will be able to sense falling snow precipitation and light rain over land. Accordingly, GPM has partnered with the Canadian CloudSat/CALIPSO Validation Project (C3VP) to obtain observations to provide one of several important ground-based validation data sets around which the falling snow models and retrieval algorithms can be further developed and tested. In this work we compare and correlate the long time series (Nov.'06 - March '07) measurements of precipitation rate from parsivels to the passive (89, 150, 183 plus or minus 1, plus or minus 3, plus or minus 7 GHz) observations of NOAA's AMSU-B radiometer. We separate the comparisons into categories of no precipitation, liquid rain and falling snow precipitation. We found that there are similar TBs (especially at 89 and 150 GHz) for cases with falling snow and for non-precipitating cases. The comparisons indicate that surface emissivity contributions to the satellite observed TB over land can add uncertainty in detecting and estimating falling snow. The newest results show that by computing brightness temperatures based on CARE radiosonde data and a rough estimate of surface emissivity show that the cloud ice scattering signal in the AMSU-B data is detected. That is the differences in computed TB and AMSU-B TB for precipitating and non-precipitating cases are unique such that the precipitating and non-precipitating cases can be identified. These results require that the radiosonde releases are within an hour of the AMSU-B data. Forest fraction, snow cover, and measured emissivities were combined to calculate the surface emissivities.

Jackson, Gail S.↗

Bounding Methods for Heterogeneous Lidar-derived Navigational Geofences

Safe Unmanned Aerial Vehicle (UAV) operations near the ground require navigation methods that avoid fixed obstacles such as buildings, power lines and trees. Aerial lidar surveys of ground structures are available with the precision and accuracy to geolocate obstacles, but the high volume of raw survey data can exceed the compute power of onboard processors and the rendering ability of ground-based flight planning maps. Representing ground structures with bounding polyhedra instead of point clouds greatly reduces the data size and can enable effective obstacle avoidance, as long as the bounding geometry envelopes the structures with high spatial fidelity. This report describes in detail four methods to compute bounding geometries of ground obstacles from lidar point clouds. The four methods are: 1) 2.5D Maximum Elevation Box, 2) 2.5D Ground Map Extrusion, 3) 3D Bounding Cylinder, and 4) 3D Bounding Box. The methods are applied to five point cloud datasets from lidar surveys of UAV flight research sites in Georgia and Virginia with an average point spacing that ranges from 0.1m to 0.6m. The methods are assessed using survey areas with geometrically heterogeneous ground structures: buildings, vegetation, power lines, and sub-meter structures such as road signs and guy wires. The 2.5D Maximum Elevation Box method is useful for simple structures. The 2.5D Ground Map Extrusion method efficiently encloses vegetation, but requires handdrawn ground footprints. The 3D Bounding Cylinder method excels at enclosing linear structures such as power lines and fences. The 3D Bounding Box method excels at enclosing planar structures such as buildings. The methods are compared on the basis of data compression and boundary fidelity on selected areas. The 2.5D methods yield the highest data compression but the polyhedra produced by them enclose significant amounts of empty space. Boundary fidelity is superior for the 3D methods, though this fidelity comes at the cost of a roughly thirtyfold lower data compression ratio than the 2.5D Maximum Elevation Box method. A mix of these output geometries is proposed for autonomous UAV navigation with limited on-board computing. Both the accuracy and spatial detail of emerging satellite-based survey technology lower than that of aerial lidar scanning survey technology. Sub-meter structures and thin linear structures are not reliably mapped at present by satellite-based surveys.

Moore, Andrew J.↗

Web-based Visualization and Analytics of Petascale Data: Equity as a Tide that Lifts All Boats

Scientists generate petabytes of data daily to help uncover environmental trends or behaviors that are hard to predict. For example, understanding climate simulations based on the long-term average of temperature, precipitation, and other environmental variables is essential to predicting and establishing root causes of future undesirable scenarios and assessing possible mitigation strategies. While supercomputer centers provide a powerful infrastructure for generating petabytes of simulation output, accessing and analyzing these datasets interactively remains challenging on multiple fronts. This paper presents an approach to managing, visualizing, and analyzing petabytes of data within a browser on equipment ranging from the top NASA supercomputer to commodity hardware like a laptop. Our novel data fabric abstraction layer allows user-friendly querying of scientific information while hid-ing the complexities of dealing with file systems or cloud services.We also optimize network utilization while streaming from petas-cale repositories through state-of-the-art progressive compression algorithms. Based on this abstraction, we provide customizable dashboards that can be accessed from any device with any inter-net connection, enabling interactive visual analysis of vast amounts of data to a wide range of users - from top scientists with access to leadership-class computing environments to undergraduate students of disadvantaged backgrounds from minority-serving institutions. We focus on NASA’s use of petascale climate datasets as an example of particular societal impact and, therefore, a case where achieving equity in science participation is critical. We validate our approach by improving the ability of climate scientists to visually explore their data via two fully interactive dashboards. We further validate our approach by deploying the dashboards and simplified training materials in the classroom at a minority-serving institution.These dashboards, released in simplified form to the general public, contribute significantly to a broader push to democratize the access and use of climate data.

Data visualization↗

Data screening and preprocessing for Landsat MSS data

Two computer algorithms are presented. The first, called SCREEN, is used to automatically identify pixels representing clouds, cloud shadows, snow, water, or anomalous signals in Landsat-2 data. The second, called XSTAR, compensates Landsat-2 data for the effects of atmospheric haze, without requiring ground measurements or ground references. The presentation of these algorithms includes their theoretical background, algebraic details, and performance characteristics. Verification of the algorithms has for the present been limited to Landsat agricultural data. Plans for further development of the XSTAR technique are also presented.

Lambeck, P. F.↗

Multi-Angle Implementation of Atmospheric Correction for MODIS (MAIAC). Part 3: Atmospheric Correction

This paper describes the atmospheric correction (AC) component of the Multi-Angle Implementation of Atmospheric Correction algorithm (MAIAC) which introduces a new way to compute parameters of the Ross-Thick Li-Sparse (RTLS) Bi-directional reflectance distribution function (BRDF), spectral surface albedo and bidirectional reflectance factors (BRF) from satellite measurements obtained by the Moderate Resolution Imaging Spectroradiometer (MODIS). MAIAC uses a time series and spatial analysis for cloud detection, aerosol retrievals and atmospheric correction. It implements a moving window of up to 16 days of MODIS data gridded to 1 km resolution in a selected projection. The RTLS parameters are computed directly by fitting the cloud-free MODIS top of atmosphere (TOA) reflectance data stored in the processing queue. The RTLS retrieval is applied when the land surface is stable or changes slowly. In case of rapid or large magnitude change (as for instance caused by disturbance), MAIAC follows the MODIS operational BRDF/albedo algorithm and uses a scaling approach where the BRDF shape is assumed stable but its magnitude is adjusted based on the latest single measurement. To assess the stability of the surface, MAIAC features a change detection algorithm which analyzes relative change of reflectance in the Red and NIR bands during the accumulation period. To adjust for the reflectance variability with the sun-observer geometry and allow comparison among different days (view geometries), the BRFs are normalized to the fixed view geometry using the RTLS model. An empirical analysis of MODIS data suggests that the RTLS inversion remains robust when the relative change of geometry-normalized reflectance stays below 15%. This first of two papers introduces the algorithm, a second, companion paper illustrates its potential by analyzing MODIS data over a tropical rainforest and assessing errors and uncertainties of MAIAC compared to conventional MODIS products.

Lyapustin, A.↗

A Technique Using Principal Component Analysis to Compare Seasonal Cycles of Earth Radiation from CERES and Model Computations

A method for quantitatively comparing the seasonal cycles of two global data sets is presented. The seasonal cycles of absorbed solar radiation (ASR) and outgoing longwave radiation (OLR) have been computed from an eight-year data set from the Clouds and Earth's Radiant Energy System (CERES) scanning radiometers and from a model data set produced by the NASA Goddard Space Flight Center Global Modeling and Assimilation Office. To compare the seasonal cycles from these two data sets, principal component (PC) analysis is used, where the PCs express the time variations and the corresponding empirical orthogonal functions (EOFs) describe the geographic variations. Ocean has a long thermal response time compared to land, so land and ocean are separated for the analysis. The root-mean square values for the seasonal cycles of ASR and OLR are extremely close for the two data sets. The first three PCs are quite close, showing that the time responses and magnitudes over the globe are very similar. The agreement between the two sets of PCs is quantified by computing the matrix of inner products of the two sets. For ASR over land, the first PCs of CERES and the model agree to better than 99.9%. The EOF maps are similar for most of the globe, but differ in a few places, and the agreement of the EOF maps is likewise quantified. Maps of differences between the annual cycles show regions of agreement and disagreement.

Smith, G. Louis↗

Earth Science Data Fusion with Event Building Approach

Objectives of the NASA Information And Data System (NAIADS) project are to develop a prototype of a conceptually new middleware framework to modernize and significantly improve efficiency of the Earth Science data fusion, big data processing and analytics. The key components of the NAIADS include: Service Oriented Architecture (SOA) multi-lingual framework, multi-sensor coincident data Predictor, fast into-memory data Staging, multi-sensor data-Event Builder, complete data-Event streaming (a work flow with minimized IO), on-line data processing control and analytics services. The NAIADS project is leveraging CLARA framework, developed in Jefferson Lab, and integrated with the ZeroMQ messaging library. The science services are prototyped and incorporated into the system. Merging the SCIAMACHY Level-1 observations and MODIS/Terra Level-2 (Clouds and Aerosols) data products, and ECMWF re- analysis will be used for NAIADS demonstration and performance tests in compute Cloud and Cluster environments.

Lukashin, C.↗

Cloud Governance at Scale

Operating and maintaining a large multi-tenant ecosystem in the cloud requires scalable solutions to unique technical and process challenges. The Cloud Computing model grants significant permissions to development teams that traditionally were reserved for Data-Center Administrators and Supply-Chain Managers. Earthdata Cloud has worked to re-cast traditional data-center management into a sensible cloud-first model. This talk discusses some of our challenges, solutions, and way ahead.

financial controls↗

Advanced Computing Support at NASA Advanced Supercomputing (NAS) Division

NAS (NASA Advanced Supercomputing) Division: Advanced Computing: High-End Computing Capability (HECC) Project; Quantum Artificial Intelligence Lab (QuAIL); NASA Earth Exchange (NEX); Data Analytics Infrastructure for NASA; Cloud Services for Science & Engineering Projects.

Advanced Computing↗

Effectiveness of Deep Learning Trained on SynthCity Data for Urban Point-Cloud Classification

3D object recognition is one of the most popular areas of study in computer vision. Many of the more recent algorithms focus on indoor point clouds, classifying 3D geometric objects, and segmenting outdoor 3D scenes. One of the challenges of the classification pipeline is finding adequate and accurate training data. Hence, this article seeks to evaluate the accuracy of a synthetically generated data set called SynthCity, tested on two mobile laser-scan data sets. Varying levels of noise were applied to the training data to reflect varying levels of noise in different scanners. The chosen deep-learning algorithm was Kernel Point Convolution, a convolutional neural network that uses kernel points in Euclidean space for convolution weights.

Geology↗

Technical Characterization and Benefit Evaluation of 5G-Enabled Grid Data Transport and Applications

This report summarizes the Year 1 work of Pacific Northwest National Laboratory’s (PNNL’s) 5G Fabricated Resource and Asset Management Encompassment for energy infrastructure (Energy FRAME) project funded by the Department of Energy Office of Science’s Advanced Scientific Computing Research Program. 5G is a breakthrough technology that enables a fully mobile and connected society, and a 5G-enabled digital continuum will be one of the critical foundations for a clean energy economy and grid modernization. In collaboration with PNNL’s Advanced Wireless Communication team and Center for Advanced Technology Evaluation team, the project team has been evaluating the system performance of 5G testbeds in the PNNL 5G Innovation Studio, and has formulated a co-simulation test case of power system transmission, distribution, and communication (T&D&C) networks considering 5G technology and high penetration of distributed energy resources. The methodology developed in the 5G Energy FRAME project can be customized to fit different future grid scenarios to evaluate multiple (dynamic) configurations (computing, sensing, communication, environment) for different stakeholders. In summary, our main technical highlights in project Year 1 are as follows: 1) Technical characterization of 5G standalone architectures, 2) Formulation of co-simulation test case of T&D&C networks embedded with 5G, 3) Initial benefit evaluation of 5G communication platform for grid use cases, and 4) Additional extended discussions on edge computing, artificial intelligence and machine learning, and high-performance computing and cloud computing adoptions. In addition, a collection of system performance data is shared through the publicly available weblink, https://www.pnnl.gov/projects/5g-energy-frame/publications

24 POWER TRANSMISSION AND DISTRIBUTION↗

Aerosol and cloud microphysics covariability in the northeast Pacific boundary layer estimated with ship-based and satellite remote sensing observations: NE Pacific Aerosol-Cloud Interactions

Ship measurements collected over the northeast Pacific along transects between the port of Los Angeles (33.7°N, 118.2°W) and Honolulu (21.3°N, 157.8°W) during May to August 2013 were utilized to investigate the covariability between marine low cloud microphysical and aerosol properties. Ship-based retrievals of cloud optical depth (τ) from a Sun photometer and liquid water path (LWP) from a microwave radiometer were combined to derive cloud droplet number concentration Nd and compute a cloud-aerosol interaction (ACI) metric defined as ACICCN = ∂ ln(Nd)/∂ ln(CCN), with CCN denoting the cloud condensation nuclei concentration measured at 0.4% (CCN0.4) and 0.3% (CCN0.3) supersaturation. Analysis of CCN0.4, accumulation mode aerosol concentration (Na), and extinction coefficient (σext) indicates that Na and σext can be used as CCN0.4 proxies for estimating ACI. ACICCN derived from 10 min averaged Nd and CCN0.4 and CCN0.3, and CCN0.4 regressions using Na and σext, produce high ACICCN: near 1.0, that is, a fractional change in aerosols is associated with an equivalent fractional change in Nd. ACICCN computed in deep boundary layers was small (ACICCN = 0.60), indicating that surface aerosol measurements inadequately represent the aerosol variability below clouds. Satellite cloud retrievals from MODerate-resolution Imaging Spectroradiometer and GOES-15 data were compared against ship-based retrievals and further analyzed to compute a satellite-based ACICCN. Satellite data correlated well with their ship-based counterparts with linear correlation coefficients equal to or greater than 0.78. Combined satellite Nd and ship-based CCN0.4 and Na yielded a maximum ACICCN = 0.88–0.92, a value slightly less than the ship-based ACICCN, but still consistent with aircraft-based studies in the eastern Pacific.

Painemal, David↗