Publication

PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions

van der Sluis, I., Kloppenburg, L. & Redeker, G., 11-Dec-2016, Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. p. 131-139 9 p.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

APA

van der Sluis, I., Kloppenburg, L., & Redeker, G. (2016). PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions. In Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016 (pp. 131-139)

Author

van der Sluis, Ielka ; Kloppenburg, Lennart ; Redeker, Gisela. / PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions. Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. 2016. pp. 131-139

Harvard

van der Sluis, I, Kloppenburg, L & Redeker, G 2016, PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions. in Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. pp. 131-139, LT for DH: Language Technology Resources and Tools for Digital Humanities, Osaka, Japan, 11/12/2016.

Standard

PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions. / van der Sluis, Ielka; Kloppenburg, Lennart; Redeker, Gisela.

Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. 2016. p. 131-139.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Vancouver

van der Sluis I, Kloppenburg L, Redeker G. PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions. In Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016. 2016. p. 131-139


BibTeX

@inproceedings{257b77890f8743c3815c1036ba1444c5,
title = "PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions",
abstract = "This paper presents a tool to investigate the design of multimodal instructions (MIs), i.e., instructions that contain both text and pictures. The benefit of including pictures in information presentation has been established, but the characteristics of those pictures and of their textual counterparts and therelation(s) between them have not been researched in a systematic manner. We present the PAT Workbench, a tool to store, annotate and retrieve MIs based on a validated coding scheme with currently 42 categories that describe instructions in terms of textual features, pictorial elements, and relations between text and pictures. We describe how the PAT Workbench facilitates collaborative annotation and inter-annotator agreement calculation. Future work on the tool includes expanding its functionality and usability by (i) making the MI annotation scheme dynamic for adding relevant features based on empirical evaluations of the MIs, (ii) implementing algorithms for automatic tagging of MI features, and(iii) implementing automatic MI evaluation algorithms based on results obtained via e.g. crowdsourced assessments of MIs.",
keywords = "annotation multimodal evaluation instructions",
author = "{van der Sluis}, Ielka and Lennart Kloppenburg and Gisela Redeker",
note = "In E. Hinrichs, M. Hinrichs, T. Trippel (eds.), Proceedings of COLING 2016, Twenty-sixth International Conference on Computational Linguistics: Proceedings of the LT for DH workshop: Language Technology Resources and Tools for Digital Humanities",
year = "2016",
month = "12",
day = "11",
language = "English",
isbn = "978-4-87974-708-2",
pages = "131--139",
booktitle = "Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016",

}

RIS

TY - GEN

T1 - PAT Workbench: Annotation and evaluation of text and pictures in multimodal instructions

AU - van der Sluis, Ielka

AU - Kloppenburg, Lennart

AU - Redeker, Gisela

N1 - In E. Hinrichs, M. Hinrichs, T. Trippel (eds.), Proceedings of COLING 2016, Twenty-sixth International Conference on Computational Linguistics: Proceedings of the LT for DH workshop: Language Technology Resources and Tools for Digital Humanities

PY - 2016/12/11

Y1 - 2016/12/11

N2 - This paper presents a tool to investigate the design of multimodal instructions (MIs), i.e., instructions that contain both text and pictures. The benefit of including pictures in information presentation has been established, but the characteristics of those pictures and of their textual counterparts and therelation(s) between them have not been researched in a systematic manner. We present the PAT Workbench, a tool to store, annotate and retrieve MIs based on a validated coding scheme with currently 42 categories that describe instructions in terms of textual features, pictorial elements, and relations between text and pictures. We describe how the PAT Workbench facilitates collaborative annotation and inter-annotator agreement calculation. Future work on the tool includes expanding its functionality and usability by (i) making the MI annotation scheme dynamic for adding relevant features based on empirical evaluations of the MIs, (ii) implementing algorithms for automatic tagging of MI features, and(iii) implementing automatic MI evaluation algorithms based on results obtained via e.g. crowdsourced assessments of MIs.

AB - This paper presents a tool to investigate the design of multimodal instructions (MIs), i.e., instructions that contain both text and pictures. The benefit of including pictures in information presentation has been established, but the characteristics of those pictures and of their textual counterparts and therelation(s) between them have not been researched in a systematic manner. We present the PAT Workbench, a tool to store, annotate and retrieve MIs based on a validated coding scheme with currently 42 categories that describe instructions in terms of textual features, pictorial elements, and relations between text and pictures. We describe how the PAT Workbench facilitates collaborative annotation and inter-annotator agreement calculation. Future work on the tool includes expanding its functionality and usability by (i) making the MI annotation scheme dynamic for adding relevant features based on empirical evaluations of the MIs, (ii) implementing algorithms for automatic tagging of MI features, and(iii) implementing automatic MI evaluation algorithms based on results obtained via e.g. crowdsourced assessments of MIs.

KW - annotation multimodal evaluation instructions

M3 - Conference contribution

SN - 978-4-87974-708-2

SP - 131

EP - 139

BT - Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH) at COLING2016

ER -

ID: 65103324