TY - JOUR

T1 - Multimodal character viewpoint in quoted dialogue sequences

AU - Stec, Kashmiri

AU - Huiskes, Mike

AU - Wieling, Martijn

AU - Redeker, Gisela

PY - 2017/5/5

Y1 - 2017/5/5

N2 - We investigate the multimodal production of character viewpoint in spoken American English narratives by performing complementary qualitative and quantitative analyses of two quoted dialogues, focusing on the storyteller’s use of character viewpoint gestures, character intonation, character facial expression, spatial orientation and gaze. A micro-analysis revealed that the extent of multimodal articulation depends on (i) the quoted speaker, with different multimodal articulatory patterns found for quotes by the speaker’s past self vs. a third-person character, and (ii) the position of the quoted utterance within the quoted dialogue, with mid-dialogue utterances garnering less co-articulation than initial or final utterances within the quoted dialogue. We further investigated these observations using a quantitative approach, which was based on generalized additive modeling (GAM). The GAM analysis revealed different multimodal patterns for each quoted character, as indicated by the number of co-produced multimodal articulators. These patterns were found to hold regardless of the quote’s position within the narrative. We discuss these findings with respect to previous work on multimodal quotation.

AB - We investigate the multimodal production of character viewpoint in spoken American English narratives by performing complementary qualitative and quantitative analyses of two quoted dialogues, focusing on the storyteller’s use of character viewpoint gestures, character intonation, character facial expression, spatial orientation and gaze. A micro-analysis revealed that the extent of multimodal articulation depends on (i) the quoted speaker, with different multimodal articulatory patterns found for quotes by the speaker’s past self vs. a third-person character, and (ii) the position of the quoted utterance within the quoted dialogue, with mid-dialogue utterances garnering less co-articulation than initial or final utterances within the quoted dialogue. We further investigated these observations using a quantitative approach, which was based on generalized additive modeling (GAM). The GAM analysis revealed different multimodal patterns for each quoted character, as indicated by the number of co-produced multimodal articulators. These patterns were found to hold regardless of the quote’s position within the narrative. We discuss these findings with respect to previous work on multimodal quotation.

KW - multimodality

KW - co-speech gesture

KW - viewpoint

KW - direct speech

KW - quotation

KW - CO-SPEECH GESTURE

KW - SIGN

KW - HAND

U2 - 10.5334/gjgl.255

DO - 10.5334/gjgl.255

M3 - Article

VL - 2

JO - Glossa: a journal of general linguistics

JF - Glossa: a journal of general linguistics

SN - 2397-1835

IS - 1

M1 - 39

ER -