Evaluating Algorithms for the Generation of Referring Expressions: Going beyond Toy Domains.

van der Sluis, I., Gatt, A. & van Deemter, K., 2007, Proceedings of the RANLP-2007. Borovets, Bulgaria

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Copy link to clipboard


We describe a corpus-based evaluation method-
ology, applied to a number of classic algorithms
in the generation of referring expressions. Fol-
lowing up on earlier work involving very simple
domains, this paper deals with the issues asso-
ciated with domains that contain ‘real-life’ ob-
jects of some complexity. Results indicate that
state of the art algorithms perform very differ-
ently when applied to a complex domain. More-
over, if a version of the Incremental Algorithm is
used then it becomes of huge importance to se-
lect a good preference order, because some pref-
erence orders are prone to generating very un-
natural output. Finding good preference orders,
however, can be difficult, as we show. These re-
sults should contribute to a growing debate on
the evaluation of nlg systems, arguing in favour
of carefully constructed balanced and semanti-
cally transparent corpora.
Original languageEnglish
Title of host publicationProceedings of the RANLP-2007
Place of PublicationBorovets, Bulgaria
Publication statusPublished - 2007

Download statistics

No data available

ID: 13577819