Extreme data lineage in ad-hoc astronomical data processing
PhD ceremony: Mr. J. Mwebaze, 16.15 uur, Academiegebouw, Broerstraat 5, Groningen
Dissertation: Extreme data lineage in ad-hoc astronomical data processing
Promotor(s): prof. E.A. Valentijn
Faculty: Mathematics and Natural Sciences
This research addresses the problems encountered in tracing and using lineage of data, pixels and code with ad hoc astronomical data processing. We describe a data model for ad hoc-based data processing and provide methods for tracing, querying and viewing its lineage. We show that data, pixel and code lineage can be used to facilitate efficient processing through incremental re-computation, as well as semi-automatic modification of processing steps, and sub-image processing.
We describe an implementation of these techniques in an all-in-one system, Astro-WISE, that allows a scientist to archive raw data, calibrate data, perform post-calibration scientific analysis and archive all results in one environment. Astro-WISE combines the use of data, pixel and code lineage to support multiple surveys with a novel way to process wide-field astronomical data in a distributed environment.
We present an example to demonstrate Astro-WISE's efficiency in searching for rare astronomical objects: we discover several of the most distant quasars in the Universe.
Last modified: | 13 March 2020 01.01 a.m. |
More news
-
17 June 2025
ERC Advanced grants for three UG researchers
The European Research Council has awarded ERC Advanced grants to Prof. Inga Kamp, Prof. Wouter Roos and Prof. Syuzanna Harutyunyan.
-
13 June 2025
Team 'Lord of the Roads' second at RDW Self Driving Challenge
The University of the North team 'Lord of the Roads', in which students from educational institutions Noorderpoort, Hanzehogeschool and the University of Groningen collaborated, came second in the RDW Self Driving Challenge (SDC). The team competed...
-
12 June 2025
Those most affected by modern agriculture
Farmers only grow a limited number of crops these days, which has significant consequences for the animals that live there. Raymond Klaassen researches what adjustments farmers could make to improve the conditions for the species most affected by...