Prosody and gesture in multilingual speakers

Funded by the VW foundation (Febraury 2014 - August 2015)

Project leaders: Prof. Dr. Ulrike Gut, WWU and Dr. Mandana Seyfeddinipur, School of Oriental and African Studies, London

Project members: Julius Hassemer, Dr. Sophie Salffner, Nicole Stein

The project explores the alignment of gestures and prosodic events in multilingual speakers' spontaneous speech. The prosodic systems of the languages of the world have been shown to vary widely (e.g. Hirst & di Cristo 1998, van der Hulst 1999, Gussenhoven 2004, Jun 2005): languages use the phonetic parameters pitch, duration and loudness in very different ways and have distinctly diverging means of marking prosodically communicative events such as focus, contrast and shared knowledge. Recent research has shown that this prosodic diversity leads to fascinating contact phenomena in multilingual speakers. For instance, speakers of a tone language when speaking an intonation language combine tone and pitch accents in an entirely novel way (see Lim 2009 for Chinese Singapore speakers of English and Gut 2005 for Hausa, Igbo and Yoruba speakers of English).

Speaking is inextricably accompanied by gestures, which in turn appear to co-occur with prosodic events. There is evidence that in English and Dutch, languages that have word stress, pointing and beat gestures tend to be aligned with the stressed syllable (Levelt et al. 1985, DeRuiter 2000, McClave 1998, Loehr 2007, 2012, Krahmer & Swerts 2007, Rochet Capellan et al. 2008). Which prosodic unit or event gestures are aligned with in languages without stress remains an open question. Equally unexplored is the question of how multilingual speakers of languages with and without stress will anchor their gestures in the speech signal.

The aim of this pilot project is to provide first answers to these questions and to test methodologies of exploring them at a larger scale. It focusses on the prosodic marking and gestural alignment of contrastive focus on the one hand and the prosodic alignment of beat gestures on the other. Speakers of two language pairs - Iwaidja and English and Ikaan and English - will be investigated. Each of these language pairs comprises two languages with typologically distant prosodic systems of which one is well-documented (English) and the other not. The pilot project will thus be able to (i) provide a first description of the prosodic marking of focus in Iwaidja and Ikaan, (ii) provide first insights into the question of which prosodic events beat gestures are aligned with in languages without stress, (iii) describe the result of prosodic contact in speakers of two typologically distinct languages and (iv) provide first findings on the gestural contact in multilingual speakers of English.