Data Management Strategies for Scientific Applications in Cloud Environments
Explore the source record for details and available documents.
Engineering topics
Publications and source records attributed to Plale, Beth.
Explore the source record for details and available documents.
Provenance of scientific data is a key piece of the metadata record for the data's ongoing discovery and reuse. Provenance collection systems capture provenance on the fly, however, the protocol between application and provenance tool may not be reliable. Consequently, the provenance record can be partial, partitioned, and simply inaccurate. We use a workflow emulator that models faults to construct a large 10GB database of provenance that we know is noisy (that is, has errors). We discuss the process of generating the provenance database, and show early results on the kinds of provenance analysis enabled by the large provenance.
The Open Provenance Model is a model of provenance that is designed to meet the following requirements: (1) To allow provenance information to be ex- changed between systems, by means of a compatibility layer based on a shared provenance model. (2) To allow developers to build and share tools that operate on such a provenance model. (3) To deFIne provenance in a precise, technology- agnostic manner. (4) To support a digital representation of provenance for any “thing, whether produced by computer systems or not. (5) To allow multiple levels of description to coexist. (6) To deFIne a core set of rules that identify the valid inferences that can be made on provenance representation. This docu- ment contains the speciFIcation of the Open Provenance Model (v1.1) resulting from a commChallenge.
Workflows have been used to model repeatable tasks or operations in manufacturing, business process, and software. In recent years, workflows are increasingly used for orchestration of science discovery tasks that use distributed resources and web services environments through resource models such as grid and cloud computing. Workflows have disparate re uirements and constraints that affects how they might be managed in distributed environments. In this paper, we present a multi-dimensional classification model illustrated by workflow examples obtained through a survey of scientists from different domains including bioinformatics and biomedical, weather and ocean modeling, astronomy detailing their data and computational requirements. The survey results and classification model contribute to the high level understandingof scientific workflows.