Publication

Data Validation Beyond Big Data

Valentijn, E. A., 1-Jun-2018, VST in the Era of the Large Sky Surveys: Proceedings of the conference held 5-8 June, 2018 in Naples, Italy. p. 17

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademic

From KiDs to Euclid OU-Ext to Euclid data validation. For the OmegaCAM@VST datahandling we have build and operated the distributed information system Astro-WISE. Astro-WISE was successfully used for the processing of KiDS data and particularly its built in extreme data-lineage facilitated the quality control and re-processing of the data with improved calibrations and improved code. Many of the aspects of the Astro-WISE approach will be applied in the data centric information system being build for the data processing for the Euclid satellite. However, the large amounts of data from Euclid in combination with the required much higher accuracies and danger of plural hidden systematics and biases forces to anticipate a new era beyond the Big data hype: data validation. In popular terms discriminating facts and fakes. I will discuss some new steps towards advanced data validation, such as build in dynamical reference systems in the OU-Ext approach, the validation of and by machine learning, and applying extreme data lineage to trace the roots and dependencies of data products.
Original languageEnglish
Title of host publicationVST in the Era of the Large Sky Surveys
Subtitle of host publicationProceedings of the conference held 5-8 June, 2018 in Naples, Italy
Pages17
Publication statusPublished - 1-Jun-2018
EventVST in the era of the large sky surveys - INAF - Osservatorio Astronomico di Capodimonte, Naples, Italy
Duration: 5-Jun-20188-Jun-2018

Conference

ConferenceVST in the era of the large sky surveys
CountryItaly
CityNaples
Period05/06/201808/06/2018

Event

VST in the era of the large sky surveys

05/06/201808/06/2018

Naples, Italy

Event: Conference

Download statistics

No data available

ID: 62946972