Vakhnin, A. | Search NASA

DOE OSTI2015

Component based dataflow processing framework

In this paper we present SOA based CLAs12 event Reconstruction and Analyses (CLARA) framework used to develop Earth Science multi-sensor data fusion, processing, and analytics applications (NAIADS: NASA JLAB collaboration). CLARA design focus is on two main traits: a) real-time data stream processing, and b) service oriented architecture (SOA) in a flow based programming (FBP) paradigm. Data driven and data centric architecture of CLARA presents an environment for developing agile, elastic, multilingual data processing applications. The CLARA framework presents solutions, capable of processing large volumes of data interactively and substantially faster than batch systems.

Gyurjyan, V.↗

NASA NTRS2015

Earth Science Data Fusion with Event Building Approach

Objectives of the NASA Information And Data System (NAIADS) project are to develop a prototype of a conceptually new middleware framework to modernize and significantly improve efficiency of the Earth Science data fusion, big data processing and analytics. The key components of the NAIADS include: Service Oriented Architecture (SOA) multi-lingual framework, multi-sensor coincident data Predictor, fast into-memory data Staging, multi-sensor data-Event Builder, complete data-Event streaming (a work flow with minimized IO), on-line data processing control and analytics services. The NAIADS project is leveraging CLARA framework, developed in Jefferson Lab, and integrated with the ZeroMQ messaging library. The science services are prototyped and incorporated into the system. Merging the SCIAMACHY Level-1 observations and MODIS/Terra Level-2 (Clouds and Aerosols) data products, and ECMWF re- analysis will be used for NAIADS demonstration and performance tests in compute Cloud and Cluster environments.

Lukashin, C.↗