SEARCH · Search NASA

Results for “cloud computing, data cloud”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Report on the Second ARM Mobile Facility (AMF2) Roll, Pitch, and Heave (RPH) Stabilization Platform: Design and Evaluation

One of the primary objectives of the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Climate Research Facility’s second Mobile Facility (AMF2) is to obtain reliable measurements of solar, surface, and atmospheric radiation, as well as cloud and atmospheric properties, from ocean-going vessels. To ensure that these climatic measurements are representative and accurate, many AMF2 instrument systems are designed to collect data in a zenith orientation. A pillar of the AMF2 strategy in this effort is the use of a stable platform. The purpose of the platform is to 1) mitigate vessel motion for instruments that require a truly vertical orientation and keep them pointed in the zenith direction, and 2) allow for accurate positioning for viewing or shading of the sensors from direct sunlight. Numerous ARM instruments fall into these categories, but perhaps the most important are the vertically pointing cloud radars, for which vertical motions are a critical parameter. During the design and construction phase of AMF2, an inexpensive stable platform was purchased to perform the stabilization tasks for some of these instruments. The first table compensated for roll, pitch, and yaw (RPY) and was reported upon in a previous technical report (Kafle and Coulter, 2012). Subsequently, a second table was purchased specifically for operation with the Marine W-band cloud radar (MWACR). Computer programs originally developed for RPY were modified to communicate with the new platform controller and with an inertial measurements platform that measures true ship motion components (roll, pitch, yaw, surge, sway, and heave). This platform could not be tested dynamically for RPY because of time constraints requiring its deployment aboard the container ship Horizon Spirit in September 2013. Hence the initial motion tests were conducted on the initial cruise. Subsequent cruises provided additional test results. The platform, as tested, meets all the design and performance criteria established for its use. This is a report of the results of those efforts and the critical points in moving forward

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

A semi-Lagrangian method for detecting and tracking deep convective clouds in geostationary satellite observations

Automated methods for the detection and tracking of deep convective clouds in geostationary satellite imagery have a vital role in both the forecasting of severe storms and research into their behaviour. Studying the interactions and feedbacks between multiple deep convective clouds (DCC), however, poses a challenge for existing algorithms due to the necessary compromise between false detection and missed detection errors. We utilise an optical flow method to determine the motion of deep convective clouds in GOES-16 ABI imagery in order to construct a semi-Lagrangian framework for the motion of the cloud field, independently of the detection and tracking of cloud objects. The semi-Lagrangian framework allows severe storms to be simultaneously detected and tracked in both spatial and temporal dimensions. For the purpose of this framework we have developed a novel Lagrangian convolution method and a number of novel implementations of morphological image operations that account for the motion of observed objects. These novel methods allow the accurate extension of computer vision techniques to the temporal domain for moving objects such as DCCs. By combining this framework with existing methods for detecting DCCs (including detection of growing cores through cloud top cooling and detection of anvil clouds using brightness temperature), we show that the novel framework enables reductions in errors due to both false and missed detections compared to any of the individual methods, reducing the need to compromise when compared with existing frameworks. The novel framework enables the continuous tracking of anvil clouds associated with detected deep convection after convective activity has stopped, enabling the study of the entire life cycle of DCCs and their associated anvils. Furthermore, we expect this framework to be applicable to a wide range of cases including the detection and tracking of low-level clouds and other atmospheric phenomena. In addition, this framework may be used to combine observations from multiple sources, including satellite observations, weather radar and reanalysis model data.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2019

GMLS-Nets: A Framework for Learning from Unstructured Data

Data fields sampled on irregularly spaced points arise in many applications in the sciences and engineering. For regular grids, Convolutional Neural Networks (CNNs) have been successfully used to gaining benefits from weight sharing and invariances. We generalize CNNs by introducing methods for data on unstructured point clouds based on Generalized Moving Least Squares (GMLS). GMLS is a nonparametric technique for estimating linear bounded functionals from scattered data, and has recently been used in the literature for solving partial differential equations. By parameterizing the GMLS estimator, we obtain learning methods for operators with unstructured stencils. In GMLS-Nets the necessary calculations are local, readily parallelizable, and the estimator is supported by a rigorous approximation theory. We show how the framework may be used for unstructured physical data sets to perform functional regression to identify associated differential operators and to regress quantities of interest. The results suggest the architectures to be an attractive foundation for data-driven model development in scientific machine learning applications.

42 ENGINEERING↗

DOE OSTI2012

Heterogeneous ice nucleation and water uptake by field-collected atmospheric particles below 273 K

Atmospheric ice formation induced by particles with complex chemical and physical properties through heterogeneous nucleation is not well understood. Heterogeneous ice nucleation and water uptake by ambient particles collected from urban environments in Los Angeles and Mexico City are presented. Using a vapour controlled cooling system equipped with an optical microscopy, the range of onset conditions for ice nucleation and water uptake by the collected particles was determined as a function of temperature (200{273 K) and relative humidity with respect to ice (RHice) up to water saturation. Three distinctly different types of authentic atmospheric particles were investigated including soot particles associated with organics/inorganics, inorganic particles of marine origin coated with organic material, and Pb/Zn containing inorganic particles apportioned to anthropogenic emissions relevant to waste incineration. Single particle characterization was provided by micro-spectroscopic analyses using computer controlled scanning electron microscopy with energy dispersive analysis of X-rays (CCSEM/EDX) and scanning transmission X-ray microscopy with near edge X-ray absorption ne structure spectroscopy (STXM/NEXAFS). Above 230 K, signicant differences in water uptake and immersion freezing effciencies of the different particle types were observed. Below 230 K, the particles exhibited high deposition ice nucleation effciencies and formed ice at RHice values well below homogeneous ice nucleation limits. The data show that the chemical composition of these eld{collected particles plays an important role in determining water uptake and immersion freezing. Heterogeneous ice nucleation rate coeffcients, cumulative ice nuclei (IN) spectrum, and IN activated fraction for deposition ice nucleation are derived. The presented ice nucleation data demonstrate that anthropogenic and marine particles comprising of various chemical and physical properties exhibit distinctly different ice nucleation effciencies and can serve as effcient IN at atmospheric conditions typical for cirrus and mixed phase clouds. This indicates a potential link between human activities and cloud formation, and thus climate.

Environmental Molecular Sciences Laboratory↗

DOE OSTI2021

MemXCT: Design, Optimization, Scaling, and Reproducibility of X-Ray Tomography Imaging

Here, this work extends our previous research entitled "MemXCT: Memory-centric X-ray CT Reconstruction with Massive Parallelization" that was originally published at SC19 conference (Hidayetoglu et al., 2019) with reproducibility of the computational imaging performance. X-ray computed tomography (XCT) is regularly used at synchrotron light sources to study the internal morphology of materials at high resolution. However, experimental constraints, such as radiation sensitivity, can result in noisy or undersampled measurements. Further, depending on the resolution, sample size and data acquisition rates, the resulting noisy dataset can be in the order of terabytes. Advanced iterative reconstruction techniques can produce high-quality images from noisy measurements, but their computational requirements have made their use an exception rather than the rule. We propose a novel memory-centric approach that avoids redundant computations at the expense of additional memory complexity. We develop a memory-centric iterative reconstruction system, MemXCT, that uses an optimized SpMV implementation with two-level pseudo-Hilbert ordering and multi-stage input buffering. We evaluate MemXCT on various supercomputer architectures involving KNL and GPU. MemXCT can reconstruct a large (11Kx11K) mouse brain tomogram in 10 seconds using 4096 KNL nodes (256K cores). The results presented in our original article at the SC19 were based on large-scale supercomputing resources. The MemXCT application was selected for the Student Cluster Competition (SCC) Reproducibility Challenge and evaluated on a variety of cloud computing resources by universities around the world in the SC20 conference. We summarize the results of the top-ranked SCC Reproducibility Challenge teams and identify the most pertinent measures for ensuring the reproducibility of our experiments in this article.

47 OTHER INSTRUMENTATION↗

NASA NTRS1998

The 1985 Biomass Burning Season in South America: Satellite Remote Sensing of Fires, Smoke, and Regional Radiative Energy Budgets

Using satellite imagery, more than five million square kilometers of the forest and cerrado regions over South America are extensively studied to monitor fires and smoke during the 1985 biomass burning season. The results are characterized for four major ecosystems, namely: (1) tropical rain forest, (2) tropical broadleaf seasonal, (3) savannah/grass and seasonal woods (SGW), and (4) mild/warm/hot grass/shrub (MGS). The spatial and temporal distribution of fires are examined from two different methods using the multispectral Advanced Very High Resolution Radiometer Local Area Coverage data. Using collocated measurements from the instantaneous scanner Earth Radiation Budget Experiment data, the direct regional radiative forcing of biomass burning aerosols is computed. The results show that more than 70% of the fires occur in the MGS and SGW ecosystems due to agricultural practices. The smoke generated from biomass burning has negative instantaneous net radiative forcing values for all four major ecosystems within South America. The smoke found directly over the fires has mean net radiative forcing values ranging from -25.6 to -33.9 W m(exp -2). These results confirm that the regional net radiative impact of biomass burning is one of cooling. The spectral and broadband properties for clear-sky and smoke regions are also presented that could be used as input and/or validation for other studies attempting to model the impact of aerosols on the earth-atmosphere system. These results have important applications for future instruments from the Earth Observing System (EOS) program. Specifically, the combination of the Visible Infrared Scanner and Clouds and the Earth's Radiant Energy System (CERES) instruments from the Tropical Rainfall Measuring Mission and the combination of Moderate Resolution Imaging Spectrometer and CERES instruments from the EOS morning crossing mission could provide reliable estimates of the direct radiative forcing of aerosols on a global scale, thereby reducing the uncertainties in current global aerosol radiative forcing values.

Christopher, Sundar A.↗

NASA NTRS2005

Lunar and Planetary Science XXXVI, Part 18

Topics discussed include: PoDS: A Powder Delivery System for Mars In-Situ Organic, Mineralogic and Isotopic Analysis Instruments Planetary Differentiation of Accreting Planetesimals with 26Al and 60Fe as the Heat Sources Ground-based Observation of Lunar Surface by Lunar VIS/NIR Spectral Imager Mt. Oikeyama Structure: First Impact Structure in Japan? Central Mounds in Martian Impact Craters: Assessment as Possible Perennial Permafrost Mounds (Pingos) A Further Analysis of Potential Photosynthetic Life on Mars New Insight into Valleys-Ocean Boundary on Mars Using 128 Pixels per Degree MOLA Data: Implication for Martian Ocean and Global Climate Change; Recursive Topography Based Surface Age Computations for Mars: New Insight into Surficial Processes That Influenced Craters Distribution as a Step Toward the Formal Proof of Martian Ocean Recession, Timing and Probability; Laser-induced Breakdown Spectroscopy: A New Method for Stand-Off Quantitative Analysis of Samples on Mars; Milk Spring Channels Provide Further Evidence of Oceanic, >1.7-km-Deep Late Devonian Alamo Crater, Southern Nevada; Exploration of Martian Polar Residual Caps from HEND/ODYSSEY Data; Outflow Channels Influencing Martian Climate: Global Circulation Model Simulations with Emplaced Water; Presence of Nonmethane Hydrocarbons on Pluto; Difference in Degree of Space Weathering on the Newborn Asteroid Karin; Circular Collapsed Features Related to the Chaotic Terrain Formation on Mars; A Search for Live (sup 244)Pu in Deep-Sea Sediments: Preliminary Results of Method Development; Some Peculiarities of Quartz, Biotite and Garnet Transformation in Conditions of Step-like Shock Compression of Crystal Slate; Error Analysis of Remotely-Acquired Mossbauer Spectra; Cloud Activity on Titan During the Cassini Mission; Solar Radiation Pressure and Transient Flows on Asteroid Surfaces; Landing Site Characteristics for Europa 1: Topography; and The Crop Circles of Europa.

Source record↗

DOE OSTI2019

GMLS-Nets: Scientific Machine Learning Methods for Unstructured Data

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2014

Range-Specific High-Resolution Mesoscale Model Setup: Data Assimilation

Mesoscale weather conditions can have an adverse effect on space launch, landing, and ground processing at the Eastern Range (ER) in Florida and Wallops Flight Facility (WFF) in Virginia. During summer, land-sea interactions across Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS) lead to sea breeze front formation, which can spawn deep convection that can hinder operations and endanger personnel and resources. Many other weak locally driven low-level boundaries and their interactions with the sea breeze front and each other can also initiate deep convection in the KSC/CCAFS area. Some of these other boundaries include the Indian River breeze front, Banana River breeze front, outflows from previous convection, horizontal convective rolls, convergence lines from other inland bodies of water such as Lake Okeechobee, the trailing convergence line from convergence of sea breeze fronts due to the shape of Cape Canaveral, frictional convergence lines from the islands in the Bahamas, convergence lines from soil moisture differences, convergence lines from cloud shading, and others. All these subtle weak boundary interactions often make forecasting of operationally important weather very difficult at KSC/CCAFS during the convective season (May-Oct). These convective processes often build quickly, last a short time (60 minutes or less), and occur over small distances, all of which also poses a significant challenge to the local forecasters who are responsible for issuing weather advisories, watches, and warnings. Surface winds during the transition seasons of spring and fall pose the most difficulties for the forecasters at WFF. They also encounter problems forecasting convective activity and temperature during those seasons. Therefore, accurate mesoscale model forecasts are needed to aid in their decision making. Both the ER and WFF would benefit greatly from high-resolution mesoscale model output to better forecast a variety of unique weather phenomena. Global and national scale models cannot properly resolve important local-scale weather features at each location due to their horizontal resolutions being much too coarse. Therefore, a properly tuned model at a high resolution is needed to provide improved capability. This task is a multi-year effort in which the Applied Meteorology Unit (AMU) will tune the Weather Research and Forecasting (WRF) model individually for each range. The goal of the first year, the results of which are in this report, was to tune the WRF model based on the best model resolution and run time while using reasonable computing capabilities. To accomplish this, the ER and WFF supported the tasking of the AMU to perform a number of sensitivity tests in order to determine the best model configuration for operational use at each of the ranges to best predict winds, precipitation, and temperature (Watson 2013). This task is a continuation of that work and will provide a recommended local data assimilation (DA) and numerical forecast model design optimized for the ER and WFF to support space launch activities. The model will be optimized for local weather challenges at both ranges.

Global and national scale models↗

NASA NTRS2008

FPGA Coprocessor for Accelerated Classification of Images

An effort related to that described in the preceding article focuses on developing a spaceborne processing platform for fast and accurate onboard classification of image data, a critical part of modern satellite image processing. The approach again has been to exploit the versatility of recently developed hybrid Virtex-4FX field-programmable gate array (FPGA) to run diverse science applications on embedded processors while taking advantage of the reconfigurable hardware resources of the FPGAs. In this case, the FPGA serves as a coprocessor that implements legacy C-language support-vector-machine (SVM) image-classification algorithms to detect and identify natural phenomena such as flooding, volcanic eruptions, and sea-ice break-up. The FPGA provides hardware acceleration for increased onboard processing capability than previously demonstrated in software. The original C-language program demonstrated on an imaging instrument aboard the Earth Observing-1 (EO-1) satellite implements a linear-kernel SVM algorithm for classifying parts of the images as snow, water, ice, land, or cloud or unclassified. Current onboard processors, such as on EO-1, have limited computing power, extremely limited active storage capability and are no longer considered state-of-the-art. Using commercially available software that translates C-language programs into hardware description language (HDL) files, the legacy C-language program, and two newly formulated programs for a more capable expanded-linear-kernel and a more accurate polynomial-kernel SVM algorithm, have been implemented in the Virtex-4FX FPGA. In tests, the FPGA implementations have exhibited significant speedups over conventional software implementations running on general-purpose hardware.

Pingree, Paula J.↗

NASA NTRS2008

Radiative Susceptibility of Cloudy Atmospheres to Droplet Number Perturbations: 2. Global analysis from MODIS

Global distributions of albedo susceptibility for areas covered by liquid clouds are presented for 4 months in 2005. The susceptibility estimates are based on expanded definitions presented in a companion paper and include relative cloud droplet number concentration (CDNC) changes, perturbations in cloud droplet asymmetry parameter and single-scattering albedo, atmospheric/surface effects, and incorporation of the full solar spectrum. The cloud properties (optical thickness and effective radius) used as input in the susceptibility calculations come from MODIS Terra and Aqua Collection 5 gridded data. Geographical distributions of susceptibility corresponding to absolute ( absolute cloud susceptibility ) and relative ( relative cloud susceptibility ) CDNC changes are markedly different indicating that the detailed nature of the cloud microphysical perturbation is important for determining the radiative forcing associated with the first indirect aerosol effect. However, both types of susceptibility exhibit common characteristics such as significant reductions when perturbations in single-scattering properties are omitted, significant increases when atmospheric absorption and surface albedo effects are ignored, and the tendency to decrease with latitude, to be higher over ocean than over land, and to be statistically similar between the morning and afternoon MODIS overpasses. The satellite-based susceptibility analysis helps elucidate the role of present-day cloud and land surface properties in indirect aerosol forcing responses. Our realistic yet moderate CDNC perturbations yield forcings on the order of 1-2 W/sq m for cloud optical property distributions and land surface spectral albedos observed by MODIS. Since susceptibilities can potentially be computed from model fields, these results have practical application in assessing the reasonableness of model-generated estimates of the aerosol indirect radiative forcing.

Oreopoulos, Lazaros↗

NASA NTRS2013

A Supplementary Clear-Sky Snow and Ice Recognition Technique for CERES Level 2 Products

Identification of clear-sky snow and ice is an important step in the production of cryosphere radiation budget products, which are used in the derivation of long-term data series for climate research. In this paper, a new method of clear-sky snow/ice identification for Moderate Resolution Imaging Spectroradiometer (MODIS) is presented. The algorithm's goal is to enhance the identification of snow and ice within the Clouds and the Earth's Radiant Energy System (CERES) data after application of the standard CERES scene identification scheme. The input of the algorithm uses spectral radiances from five MODIS bands and surface skin temperature available in the CERES Single Scanner Footprint (SSF) product. The algorithm produces a cryosphere rating from an aggregated test: a higher rating corresponds to a more certain identification of the clear-sky snow/ice-covered scene. Empirical analysis of regions of interest representing distinctive targets such as snow, ice, ice and water clouds, open waters, and snow-free land selected from a number of MODIS images shows that the cryosphere rating of snow/ice targets falls into 95% confidence intervals lying above the same confidence intervals of all other targets. This enables recognition of clear-sky cryosphere by using a single threshold applied to the rating, which makes this technique different from traditional branching techniques based on multiple thresholds. Limited tests show that the established threshold clearly separates the cryosphere rating values computed for the cryosphere from those computed for noncryosphere scenes, whereas individual tests applied consequently cannot reliably identify the cryosphere for complex scenes.

Radkevich, Alexander↗

DOE OSTI2019

GMLS-Nets: A Framework for Learning from Unstructured Data

Data fields sampled on irregularly spaced points arise in many applications in the sciences and engineering. For regular grids, Convolutional Neural Networks (CNNs) have been successfully used to gaining benefits from weight sharing and invariances. We generalize CNNs by introducing methods for data on unstructured point clouds based on Generalized Moving Least Squares (GMLS). GMLS is a non-parametric technique for estimating linear bounded functionals from scattered data, and has recently been used in the literature for solving partial differential equations. By parameterizing the GMLS estimator, we obtain learning methods for operators with unstructured stencils. In GMLS-Nets the necessary calculations are local, readily parallelizable, and the estimator is supported by a rigorous approximation theory. We show how the framework may be used for unstructured physical data sets to perform functional regression to identify associated differential operators and to regress quantities of interest. The results suggest the architectures to be an attractive foundation for data-driven model development in scientific machine learning applications

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Validation of Photovoltaic Modeling Tool Solargraf Against Measured Data

Solargraf is a cloud-based 3D design tool by Enphase Energy that allows users to design solar and storage systems with a variety of elements. Through a Cooperative Research and Development Agreement (CRADA), Enphase Energy collaborated with the National Renewable Energy Laboratory (NREL) to validate Solargraf's 3D design simulation against measured PV system performance. This study follows the same methodology of similar validation studies completed at NREL. The predicted performance results from simulations in both Solargraf and NREL's System Advisor Model (SAM) tool were compared with measured data to evaluate performance predictions.

14 SOLAR ENERGY↗

DOE OSTI2021

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗

DOE OSTI2023

Machine Learning for Advanced Building Construction

High-efficiency retrofits can play a key role in reducing carbon emissions associated with buildings if processes can be scaled-up to reduce cost, time, and disruption. Here we demonstrate an artificial intelligence/computer vision (AI/CV)-enabled framework for converting exterior build scans and dimensional data directly into manufacturing and installation specifications for overclad panels. In our workflow point clouds associated with LiDAR-scanned buildings are segmented into a facade feature space, vectorized features are extracted using an iterative random-sampling consensus algorithm, and from this representation an optimal panel design plan satisfying manufacturing constraints is generated. This system and the corresponding construction process is demonstrated on a test facade structure constructed at the National Renewable Energy Laboratory (NREL). We also include a brief summary of a techno-economic study designed to estimate the potential energy and cost impact of this new system.

build scans↗

DOE OSTI2014

A statistical analysis of seeds and other high-contrast exoplanet surveys: massive planets or low-mass brown dwarfs?

We conduct a statistical analysis of a combined sample of direct imaging data, totalling nearly 250 stars. The stars cover a wide range of ages and spectral types, and include five detections (κ And b, two ∼60 M {sub J} brown dwarf companions in the Pleiades, PZ Tel B, and CD–35 2722B). For some analyses we add a currently unpublished set of SEEDS observations, including the detections GJ 504b and GJ 758B. We conduct a uniform, Bayesian analysis of all stellar ages using both membership in a kinematic moving group and activity/rotation age indicators. We then present a new statistical method for computing the likelihood of a substellar distribution function. By performing most of the integrals analytically, we achieve an enormous speedup over brute-force Monte Carlo. We use this method to place upper limits on the maximum semimajor axis of the distribution function derived from radial-velocity planets, finding model-dependent values of ∼30-100 AU. Finally, we model the entire substellar sample, from massive brown dwarfs to a theoretically motivated cutoff at ∼5 M {sub J}, with a single power-law distribution. We find that p(M, a)∝M {sup –0.65} {sup ±} {sup 0.60} a {sup –0.85} {sup ±} {sup 0.39} (1σ errors) provides an adequate fit to our data, with 1.0%-3.1% (68% confidence) of stars hosting 5-70 M {sub J} companions between 10 and 100 AU. This suggests that many of the directly imaged exoplanets known, including most (if not all) of the low-mass companions in our sample, formed by fragmentation in a cloud or disk, and represent the low-mass tail of the brown dwarfs.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Semantic Property Graph for Scalable Knowledge Graph Analytics

Graphs are a natural and fundamental representation to describe entities, relationships, activities, and evolution of complex systems. Many domains such as communication, citation, procurement, biology, social media, and transportation can be modeled as a set of entities and their relationships. Resource Description Framework (RDF) and Labeled Property Graph (LPG) are two of the most used data models to encode information in a graph. Both models are similar in terms of using basic graph elements such as nodes and edges but differ in terms of the modeling approach, expressibility, serialization, and target applications. RDF is a flexible data exchange model for expressing information about entities but it tends to a have high memory footprint and inefficient storage, which does not make it a natural choice to perform scalable graph analytics. In contrast, LPG has gained traction as a reliable model to perform scalable graph analytic tasks such as sub-graph matching, network alignment, and real-time knowledge graph query. It provides efficient storage, fast traversal, and flexibility to model various real-world domains. At the same time, the LPG lacks the support of a formal knowledge representation such as an ontology to provide automated knowledge inference. We propose Semantic Property Graph (SPG) as a logical projection of reified RDF into the LPG model. SPG continues to use RDF ontology to define the type hierarchy of the projected graph and validate it against a given ontology. We present a framework to convert reified RDF graphs into SPG using two different computing environments. We also present cloud-based graph migration capabilities using Amazon Web Services.

Purohit, Sumit↗