Extreme data lineage in ad-hoc astronomical data processing
PhD ceremony: Mr. J. Mwebaze, 16.15 uur, Academiegebouw, Broerstraat 5, Groningen
Dissertation: Extreme data lineage in ad-hoc astronomical data processing
Promotor(s): prof. E.A. Valentijn
Faculty: Mathematics and Natural Sciences
This research addresses the problems encountered in tracing and using lineage of data, pixels and code with ad hoc astronomical data processing. We describe a data model for ad hoc-based data processing and provide methods for tracing, querying and viewing its lineage. We show that data, pixel and code lineage can be used to facilitate efficient processing through incremental re-computation, as well as semi-automatic modification of processing steps, and sub-image processing.
We describe an implementation of these techniques in an all-in-one system, Astro-WISE, that allows a scientist to archive raw data, calibrate data, perform post-calibration scientific analysis and archive all results in one environment. Astro-WISE combines the use of data, pixel and code lineage to support multiple surveys with a novel way to process wide-field astronomical data in a distributed environment.
We present an example to demonstrate Astro-WISE's efficiency in searching for rare astronomical objects: we discover several of the most distant quasars in the Universe.
Last modified: | 13 March 2020 01.01 a.m. |
More news
-
24 May 2024
Lustrum 410 in pictures
Lustrum 410 in pictures: We are already over half way through our lustrum! Enjoy a few photos of the lustrum so far.
-
22 May 2024
Raffaella Carloni and Jonas Göbel winners Ben Feringa Impact Award 2024
Raffaella Carloni and Jonas Göbel received the Ben Feringa Impact Awards 2024 for the categories of researchers and students on Tuesday 21 May.
-
22 May 2024
2.9 million for sustainable management salt marshes Wadden Sea
Prof. Chris Smit (GELIFES) and Prof. Kai Jensen (University of Hamburg) will receive an NWA grant of 2.9 million euros for research that will contribute to sustainable managemnt of the salt marshes of the Wadden Sea.