Publication

PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions

van der Sluis, I., Kloppenburg, L. & Redeker, G., 11-Dec-2016, Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. p. 131-139 9 p.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

This paper presents a tool to investigate the design of multimodal instructions (MIs), i.e., instructions that contain both text and pictures. The benefit of including pictures in information presentation has been established, but the characteristics of those pictures and of their textual counterparts and the
relation(s) between them have not been researched in a systematic manner. We present the PAT Workbench, a tool to store, annotate and retrieve MIs based on a validated coding scheme with currently 42 categories that describe instructions in terms of textual features, pictorial elements, and relations between text and pictures. We describe how the PAT Workbench facilitates collaborative annotation and inter-annotator agreement calculation. Future work on the tool includes expanding its functionality and usability by (i) making the MI annotation scheme dynamic for adding relevant features based on empirical evaluations of the MIs, (ii) implementing algorithms for automatic tagging of MI features, and
(iii) implementing automatic MI evaluation algorithms based on results obtained via e.g. crowdsourced assessments of MIs.
Original languageEnglish
Title of host publicationProceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016
Pages131-139
Number of pages9
Publication statusPublished - 11-Dec-2016
EventLT for DH: Language Technology Resources and Tools for Digital Humanities - Osaka, Osaka, Japan
Duration: 11-Dec-201611-Dec-2016
https://www.clarin-d.net/en/current-issues/lt4dh

Workshop

WorkshopLT for DH: Language Technology Resources and Tools for Digital Humanities
Abbreviated titleLT4DH
CountryJapan
CityOsaka
Period11/12/201611/12/2016
Internet address

Event

LT for DH: Language Technology Resources and Tools for Digital Humanities

11/12/201611/12/2016

Osaka, Japan

Event: Workshop

    Keywords

  • annotation multimodal evaluation instructions

Download statistics

No data available

ID: 65103324