Gestural input during bookreading: A value of a pragmatically specific situation?

Transcription

1 Gestural input during bookreading: A value of a pragmatically specific situation? Katharina J. Rohlfing, Angela Grimminger & Kerstin Nachtigäller Bielefeld University, CITEC Workshop Multimodality and Language Learning

2 Outline Introduction Why is the book situation special? A study by Rowe & Pan (2004) Study 1: Longitudinal study Study 2: Comparison of pragmatically different situations Results

3 Introduction

4 Maternal pointing in language learning frequently accompanies object labeling (Murphy, 1978) reinforces the message conveyed in speech (Iverson et al., 1999; Rowe, 2000) helps to establish periods of joint attention (Iverson et al., 1994) to 14 month old children mediates children s vocabulary skills at 54 months (Rowe & Goldin-Meadow, 2009)

5 Semantic coherence and language learning gesture paves the way for language (Iverson & Goldin-Meadow, 2005) speech-gesture combination as a precursor to two-word utterances (Butcher & Goldin-Meadow, 2000; McEachern & Haynes, 2004) gestures in input remain constant (Özçalişkan & Goldin-Meadow, 2005)

6 Situations in language learning different situations trigger different input (Yont et al., 2003; Flynn & Masur, 2007) early books foster the development of,reading routines (Kümmerling-Meibauer & Meibauer, 2005; 2011)

7 Gestures in book reading situations: questions does pointing frequency differ by context (bookreading versus toy play)? what is the relationship between maternal pointing in book reading situations and child subsequent vocabulary?

8 Gesturing in bookreading Rowe & Pan, 2004

9 Gestures in book reading situations participants: 68 low-income mothers and 2 year old children Rowe & Pan, 2004 mother and child were videotaped interacting at home, beginning with a book, then moving on to toys for a total of 10 minutes

10 Gestures in book reading situations Rowe & Pan, 2004 communicative intent: directing attention, discussing joint focus, negotiating immediate activity (Ninio et al., 1994) contribution of gesture to speech: reinforces, disambiguates, adds information lexical items in gesture and speech conversational function: eliciting vs. non eliciting

11 Gestures in book reading situations: results (1) mothers spent 18 % of time in bookreading Rowe & Pan, 2004 the majority of maternal pointing (54 %) gestures occurred during bookreading mothers produced 4.6 points / minute during bookreading vs. 0.7 during toy play

12 Gestures in book reading situations: results (2) mothers discussed joint focus significantly more in bookreading Rowe & Pan, 2004 in bookreading, the gestures were mostly used to reinforce the verbal message children were more likely to respond with a label to mother s point in bookreading maternal points-per-minute during bookreading was positively correlated with children s receptive vocabulary a year later

13 Gestures in book reading situations: conclusions Rowe & Pan, 2004 although dyads may spend relatively brief periods of time interacting around a book, this form of interaction appears to be one that is particularly facilitative of child vocabulary development maternal pointing during bookreading reinforces the verbal message and is often used to discuss a joint focus of attention maternal pointing during bookreading is positively associated with children s subsequent vocabulary comprehension children were more likely to provide a correct verbal label than is the case in response to maternal points during toy play

14 Our research questions Rowe & Pan (2004) found a correlation between maternal pointing behavior (during book situation) to their 24 month-olds and children s receptive vocabulary skills at 36 months of age What kind of gestural behavior is elicited in book reading situations? Which children s age is especially sensitive for maternal gestural behavior? Does the book content influence maternal gestural behavior? What is the value of the book reading situation?

15 1. Longitudinal study

16 Participants conditions Nouns Relations 4 (2 f and 2 male) 7 (4 f and 3 male) age M SD M SD 1. visit 11;11 0;11 11;23 0;11 2. visit 14;17 0;11 14;16 0;13 3. visit 18;11 0;12 17;26 0;13

17 Relations Nouns Book conditions

18 Procedure

19 Dependent measures gestural and verbal behavior semantic coherence of gestural behavior (reinforcing or supplementing language) no other higher-level coding ELFRA

20 What kind of gestural behavior is elicited in book reading situations?

21 Gestures: Types deictic iconic / symbolic conventionalized manipulative (Rohlfing, 2011) beats

22 Semantic coherence co-expressivity of gesture and speech: semantic power (Marcos et al., 2003) different types of gestures: different meaning pointing gestures (Iverson et al., 1999): reinforcing supplementing

23 Hypotheses very specific gestural behavior occurs during the book reading situation (ritualization of book reading strategies) gestural forms facilitate the reference resolution

24 Maternal gestures during bookreading

25 Taxonomy communication iconic gestures different forms of pointing saccades (Rohlfing, 2011) one stroke manipulative gestures action

26 Saccades: Taxonomy real to book object s relation object s features grouping

27 Saccades seem to be a means in interaction with early-verbal children (Rohlfing, 2011) suggest a correspondence of entities object to object object to picture triggered mostly in the relation condition

28 Which children s age is especially sensitive for maternal gestural behavior?

29 Hypotheses maternal gestural behavior at the age of 14 months correlates with children s vocabulary skills (Rowe et al., 2008) reinforcing gestural behavior (in the sense of gestural motherese) correlates with children s vocabulary skills maternal overall gestural behavior will remain constant (Özçalişkan & Goldin- Meadow, 2005)

30 Results maternal gestural behavior showed a weak relationship to children s gestural and verbal behavior in the investigated situation maternal overall gestures at the age of 11 marginally positively correlated with child s verbal behavior rs(11) = 0.57, p < 0.06 maternal reinforcing pointing at the age of 14 marginally positively correlated with child s overall gestural behavior rs(11) = 0.54, p < 0.09 maternal reinforcing pointing at the age of 18 marginally positively correlated with child s overall gestural behavior rs(11) = 0.58, p < 0.06

31 Results: at 14 months of age maternal gestural behavior in general pointing behavior reinforcing pointing behavior positively correlated with productive and receptive ELFRA 14

32 Results: at 14 months of age (long term) pointing behavior of the mother rs(9) = , p < 0.08 rs(9) = 0.74*, p < 0.05 productive ELFRA 18 the number of maternal words within deictic gestures at 14 months of age, there seems to be a relationship between maternal gestural behavior and children s later verbal (but not gestural) skills even though all ELFRAs correlated with each other, we found no longitudinal effects for the advantage of maternal gestural behavior

33 Results: at 14 months of age children s deictic gestures with speech rs = 0.63* rs = 0.48 mothers deictic gestures with speech rs = 0.7 * rs = 0.86* ELFRA 18 ELFRA % of the variance

34 Longitudinal results: deictic gesture overall Deictic Gestures 25 ** 20 mean number Mother Child children's age

35 Longitudinal results 100 Proportions of deictic gestures" with speech 80 mean proportions Mother Child children's age

36 Longitudinal results for the dyad, especially the use of deictic gestures increases with the age from 11 to 18 months: F(2,9) = 11.32, p < 0.01, η 2 = 0.7 for the children, proportion of gestures with language is increasing with age, especially at 14 months: F(2,9) = 11.32, p < 0.01, η 2 = months seems to be an age, at which children s gestural behavior with language receives a boost

37 Longitudinal results mothers deictic gestures with speech ELFRA18 more finetuning to language more words m o r e g e s t u r e s

38 Does the book content influence maternal gestural behavior?

39 Hypotheses some types of maternal gestural behavior will be influenced by the content of the book since discourse is influenced by it (Nachtigäller & Rohlfing, 2011) more gestural behavior will be triggered in relation book, since in this condition, there are more objects to point at different types of saccades will be elicited in different book contents

40 Results: Different book contents (1) 25! Mothers' deictic gestures! number of deictic gestures! 20! 15! 10! 5! Nouns Relations 0! 11! 14! 18! Age in months!

41 Results: Different book contents (2) 25! Children's deictic gestures! number of deictic gestures! 20! 15! 10! 5! Nouns Relations 0! 11! 14! 18! Age in months!

42 Results: Different book contents no statistical difference for the gestural behavior of the mother nor the child with respect to the book conditions in contrast to the hypothesis, relation condition does not trigger more gestures

43 Results: Saccades in different book contents saccades (real book): significantly more in the noun condition saccades (group; relations): significantly more in the relation condition

44 Results: Discussion book content does not influence gestural behavior of the dyad some types of pointing saccades can be triggered by the book content noun condition seems to elicit very concrete / framed behavior (Kümmerling- Meibauer & Meibauer, 2005; 2011)

45 2. Cross-sectional comparison What is the value of the book reading situation?

46 Assumptions effect of nonverbal gestural behavior on language skills method: nonverbal behavior in relation to verbal behavior (not to time as in Rowe & Pan, 2004) pointing within speech words within pointing

47 Participants and conditions conditions Book Free play time about 5 minutes about 25 minutes gender 8 female and 9 male age M = 14;21 SD = 0;12 ELFRA 24 from 10 children

48 Focus on the relevant pragmatics in both conditions only gestures within joint book reading were considered we excluded all pragmatic impairments mother was talking about something else (e.g. drinking) child s attention was distracted

49 Results 2 (child and mother) x 2 (book and free play) ANOVA for pointing behavior mothers gesture more than children (8.52 versus 2.25) no main effect of situation (5.35 for book versus 5.41 for free play) 2 x 2 ANOVA for words in deictic gestures mothers being more verbal (136.8 words versus 3.1 words in children), especially in free play more words in free play (103.98) than in book situation (35.88)

50 Predictors of ELFRA 24 Book Free play deictic gestures β = 0.4, p = 0.24 β = 0.4, p = 0.24 pointing β = 0.51, p = 0.13 β = 0.4, p = 0.25 reinforcing pointing pointing with Speech words within pointing β = 0.46, p = 0.18 β = 0.37, p = 0.3 β = 0.51, p = 0.13 β = 0.38, p = 0.28 β = 0.62, p = 0.05 β = 0.35, p = 0.3

51 Results: predictors of the ELFRA 24 the number of maternal words within deictic gestures rs(10) = 0.69* ns productive ELFRA 24 productive ELFRA 18 verbal or nonverbal behavior of overall maternal pointing in

52 Results: relations to developing languge skills maternal reinforcing pointing in book rs(13) = 0.63* rs(13) = 0.45 receptive ELFRA 14 productive ELFRA 14 maternal supplementing pointing in book rs(13) = 0.73** rs(13) = productive ELFRA 18 receptive ELFRA 18 gestural motherese: reinforcing to younger, supplementing to older children

53 Results the only predictor of later language skills is: Mothers words in pointing gestures in book reading situations no other predictors either from child s behavior or from free play the content of the words (reinforcing or supplementing) seems to be related to language skills as well

54 Discussion book situation is particularly facilitative of child language development, because: it is recurrent, thus it provides routines for communicative behavior (Kümmerling-Meibauer & Meibauer, 2005) it transports cultural forms of action organization (Dausendschön-Gay, 2003) it clearly frames reference intentions

55 Conclusions and implications pointing in book reading: facilitates reference resolution by very specific pointing patterns (saccades) pointing is tangible in book reading 14 months seems to be an age, at which gestural behavior of the mother receives a boost and children are especially sensitive to it gestural behavior of the mothers with speech at the age of 14 months can be especially important / helpful for further language learning book content does not change the gestural behavior of the mother it does not matter what is in the book, just read it! mothers words provided within her pointing during book reading (but not free play) predict children s later language skills book reading seems to be especially facilitative of child s language learning

56 Thank you for your attention! This work was supported by the Dilthey Fellowship (Research Initiative Focus on Humanities) of the VolkswagenStiftung Many thanks to our Research Assistants: Benjamin Koch Sina Rauchut Inka Wallstabe