Publication

Tracing and using data lineage for pipeline processing in Astro-WISE

Mwebaze, J., Boxhoorn, D. & Valentijn, E. A., Jan-2013, In : Experimental Astronomy. 35, 1-2, p. 131-155 25 p.

Research output: Contribution to journalArticleAcademicpeer-review

Most workflow systems that support data provenance primarily focus on tracing lineage of data. Data provenance by data lineage provides the derivation history of data including information about services and input data that contributed to the creation of a data product. We show that tracing lineage by means of full backward chaining not only enables users to share, discover and reuse the data, but also supports scientific data processing through storage, retrieval and (re)processing of digitized scientific data. In this paper, we present Astro-WISE, a distributed system for processing, analyzing and disseminating wide field imaging astronomical data. We show how Astro-WISE traces lineage of data and how it facilitates data processing, retrieval, storage and archiving. Particularly we show how it solves issues related to the changing data items typical for the scientific environment, such as physical changes in calibrations, our insight in these changes and improved methods for deriving results.

Original languageEnglish
Pages (from-to)131-155
Number of pages25
JournalExperimental Astronomy
Volume35
Issue number1-2
Publication statusPublished - Jan-2013

    Keywords

  • Data lineage, Provenance, Pipeline data processing, Astro-WISE, PROVENANCE

Download statistics

No data available

ID: 5784552