Generating Multimodal References

van der Sluis, I. & Krahmer, E., 2007, In : Discourse Processes. 44, 3, p. 145-174

Research output: Contribution to journalArticleAcademicpeer-review

Copy link to clipboard


This paper presents a new computational model for the generation of multimodal referring expressions, based on observations in human communication. The algorithm is an extension of the graph-based algorithm proposed by Krahmer et al. (2003) and makes use of a so-called Flashlight Model for pointing. The Flashlight Model accounts for various types of pointing gestures of different precisions. Based on a notion of effort the algorithm produces referring
expressions combining language and pointing gestures. The algorithm is evaluated using two production experiments, with which spontaneous data is gathered on controlled input. The output of the algorithm coincides to a large extent with the utterances of the participants. However, an important difference is that the participants tend to produce overspecified referring expressions
while the algorithm generates minimal ones. We briefly discuss ways to generate overspecified multimodal references.
Original languageEnglish
Pages (from-to)145-174
JournalDiscourse Processes
Issue number3
Publication statusPublished - 2007

Download statistics

No data available

ID: 13576379