Audiovisual processing in aphasic and non-brain-damaged listeners. The whole is more than the sum of its parts

PhD ceremony: Ms. D.A. Hessler, 16.15 uur, Aula Academiegebouw, Broerstraat 5, Groningen

Dissertation: Audiovisual processing in aphasic and non-brain-damaged listeners. The whole is more than the sum of its parts

Promotor(s): prof. Y.R.M. Bastiaanse

Faculty: Arts

Speech perception is a task that is commonly fulfilled without much effort. Only when processing is disturbed, e.g. after a brain damage, we notice its complexity. Dörte Hessler investigated this phenomenon. Not only auditory but also audiovisual processing of speech sounds is addressed.

The research firstly showed that individuals with aphasia (a language disorder resulting from brain damage) have more difficulties in recognizing small differences between speech sounds than larger. Speech sounds can differ in the manner of articulation, the place of articulation and the fact whether the vocal cords vibrate. Speech sounds differing in all three ways were easier to recognize than those differing in only one way. The most difficult distinction was found for sounds that differed only with regard to the vibration of the vocal cords (e.g. the difference between p and b). Measurements of brain reactions of listeners without a language disorder complemented these findings: the brain waves showed a larger reaction when the difference between sounds was small. This is probably due to the additional attention that is necessary to process the small differences.

The research project furthermore showed that visual support (speechreading), which has a positive influence on perception, is not limited to clearly visible features of sounds, such as the place of articulation, but also on the manner of articulation and the vocal cord vibrations. Even individuals without brain-damage show an effect of speechreading: their reaction times decrease when they have to choose a target sound. Additionally, also their brain reactions were influenced: auditory and audiovisual input lead to clearly distinctive reaction patterns. Processing was easier when the sound was presented audiovisually.