. Niparko, J. K. (2006). Speech Recognition at 1-Year Follow-Up in the Childhood

Transcription

1 Psychology 230: Research Methods Lab A Katie Berg, Brandon Geary, Gina Scharenbroch, Haley Schmidt, & Elizabeth Stevens Introduction: Overview: A training program, under the lead of Professor Jeremy Loebach, was previously developed to help new adult cochlear implant users learn to hear. Users have taken part in a long term training paradigm that has had them identify speech sounds (phonemes, words, sentences), speakers, and environmental sounds. Stimuli for the cochlear implant users have been obtained from multiple recorded speaker sources. The primary issue is that not all speakers are equally intelligible, and if performance is to be judged on how well cochlear implant users can understand a speaker, speaker intelligibility is critical. Our project aims to explore speaker intelligibility. Additionally we desire to quantify how intelligible our speakers are, and how that contributes to the training data that is being gathered. To do that, we need to test talker intelligibility and come up with a metric by which to judge them. Categories and Analysis: 1) Cochlear Implants: The category of Cochlear Implants was chosen in order to cover the basics of how implants work. An understanding of cochlear implants is essential to come to an understanding of the devices that our participants interact with. The articles concerning cochlear implants cover the basics of how implants work, information about deaf culture, the effectiveness of implants, how effectiveness is measured, and the current training options for implant users. Sources here also provide information about cochlear implant simulators and possible methods for testing participants on various speech phrases (normal, unpredictable, anomalous). The sources provide an essential backdrop to our study. Eisenberg, L. S., Johnson, K. C., Martinez, A. S., Cokely, C. G., Tobey, E. A., Quittner, A. L.,... Niparko, J. K. (2006). Speech Recognition at 1-Year Follow-Up in the Childhood Development after Cochlear Implantation Study: Methods and Preliminary Findings. Audiology & Neurotology, 11(4), doi: This study explores comparisons of development between of 3-5 year old children with and without cochlear implants (CI). They use a hierarchical approach for evaluating speech recognition to best assess CI and normal hearing children. To test the language abilities of each group, recordings were played in a sound field and the children identified the words spoken. This is relevant to our study because it shows how cochlear implants have been tested in the past and compared to normal hearing abilities.

2 Gale, E. (2011). Exploring perspectives on cochlear implants and language acquisition within the deaf community. Journal of Deaf Studies and Deaf Education, 16(1), doi: This article provides a broad overview about how cochlear implants were first adapted as a mode of communication for deaf individuals and the reactions that followed this novel invention. Cochlear implants sparked heated debate within the deaf community almost instantly after they were first employed in the 1980s. This article is relevant to our research project because it explains many of the insecurities deaf individuals have about the device itself and the mode of communication that is still used today. Loebach, J. L., Bent, T., & Pisoni, D. B. (2008). Multiple routes to the perceptual learning of speech. Acoustical Society of America, 124(1), doi: / This source uses normal hearing individuals to explore how training influenced linguistic and indexical tasks when adapting to cochlear implant simulations. They found that training on an indexical task requiring high levels of controlled attention is equally as beneficial as a linguistic task. These findings are interesting, but more importantly, this article explains the cochlear implant simulators we will be using and utilized similar recordings (including anomalous and meaningful phrases). Loebach, J. L., & Pisoni, D. B. (2008). Perceptual learning of spectrally degraded speech and environmental sounds Acoustical Society of America, 123(2), doi: / This study explored training for cochlear implant users. It tested how well normal hearing people could learn words, sentences, or environmental sounds when played in a cochlear implant simulator. After each group learned their sounds, all participants were tested on general speech and environmental sounds to see which condition generalized best to each of the two types of sounds. They found that speech didn't generalize to environmental sounds, but environmental sounds generalized to speech. This information is very useful moving forward because it will contribute to future training programs for new implant users. It is relevant to our project because it is Professor Loebach's previous work. We will focus on speech sounds rather than environmental, but it is still important to keep this finding in mind. Nourski, K. V., Etler, C. P., Brugge, J. F., Oya, H., Kawasaki, H., Reale, R. A.,... Howard, M. A. (2013). Direct Recordings from the Auditory Cortex in a Cochlear

3 Implant User. Jaro-Journal of the Association for Research in Otolaryngology, 14(3), doi: DOI /s This article provides a broad overview of how the cochlear implant stimulates the auditory nerve of those with severe-to-profound hearing loss to provide them with a mode of communication. Understanding how the human auditory cortex responds to a cochlear implant is key to understanding the strengths and weaknesses of these devices. This article is relevant to our research project because we will be looking at the different variables required for clear speech recognition in those with cochlear implants and understanding the basics of how the device functions creates a strong foundation for this research. Seung, H., Holmes, A., & Colburn, M. (2005). Twin Language Development: A Case Study of a Twin with a Cochlear Implant and a Twin with Typical Hearing. The Volta Review, 105(2), This article examines the language development of a pair of twins, one of whom received a cochlear implant at 20 months. The study showed that the hearing-loss twin experienced a delay in expressive vocabulary performance at 41 months. This delay, however, can be accounted for when the length of implant use is considered. Overall, this article is important to understanding speaker intelligibility because it provides some background information about cochlear implants and their effectiveness. Tyler, R. S., Witt, S. A., Dunn, C. C., & Wang, W. (2010). Initial development of a spatially separated speech-in-noise and localization training program. Journal of the American Academy of Audiology, 21(6), doi: This study explores a possible training program for cochlear implant (CI) usersspecifically to hear speech-in-noise. Their pilot data shows that their training program is effective for hearing in noise and localization. Overall this study shows that though there is no large-scale training program for new CI users, there is some effort to develop one and to help users hear better. This is important to our study because our data will ultimately help develop a training program for CI users. 2) Speaker Intelligibility: Speaker Intelligibility was chosen in order to operationalize what is implied when using the term intelligibility in regards to speech. Ultimately, speaker intelligibility is studied to understand the relation of comprehensibility for our participants in the midst of distracting noises or the influences which impact the clarity and perception of speech.

4 Our sources cover the various characteristics which distinguish different speakers from one another and how they are perceived by cochlear implant users in real-life situations. Barker, Jon, & Cooke, Martin. (2007). Modelling speaker intelligibility in noise. Speech Communication, 49(5), doi: This study compared listeners performance on a multi-speaker speech-in-noise task. Listeners were presented a range of signal-to-noise ratios in speechshaped noise and were asked to identify three keywords in a simple six-word sentences. A glimpsing model, which combined a simulation of energetic masking with speaker-dependent statistical models, was able to predict not only the intelligibility of individual speakers to a remarkable degree. This is relevant to our study because we will be asking listeners to identify keywords using a range of signal-to-noise ratios. Bradlow, Ann R., Torretta, Gina M., & Pisoni, David B. (1996). Intelligibility of normal speech I: Global and fine-rained acoustic-phonetic talker characteristics. Speech Communication, 20(3-4), doi: Using 200 college student listeners, "global" talker characteristics (gender, fundamental frequency (F0), and speaking rate) were investigated. Also, several fine-grained acoustic-phonetic talker-characteristics were analyzed as correlates of overall intelligibility. Results suggest that a substantial portion of variability in normal speech intelligibility is traceable to specific acoustic-phonetic characteristics of the talker. This study is relevant to our research because it is important to know what characteristics make a good talker versus a bad talker. Eisenberg, Leon, Berlin, Charles I., Dill, Anne, & Frank, Sheldon. (1968). Class and race effects on the intelligibility of monosyllables. Child Development, 39(4), doi: This study looked at differences in intelligibility scores between 160 Negro and white schoolchildren of low and middle socioeconomic class families. The most intelligible speech was generated by the educated speakers, regardless of the race of the speaker or the listeners. Negro children showed generally poorer listening scores than whites, and Negro speakers generated slightly poorer intelligibility

5 scores than whites. Great variability within speakers overcame the effects of race and class. This is important to our study because it is important to take race and ethnicity into account because it might have an effect on whether the listener thinks a speaker is intelligible or not. Fu, Qian-Jie, Chinchilla, Sherol, & Galvin, JohnJ. (2004). The Role of Spectral and Temporal Cues in Voice Gender Discrimination by Normal-Hearing Listeners and Cochlear Implant Users. Journal of the Association for Research in Otolaryngology, 5(3), doi: /s This study investigated the relative importance of temporal and spectral cues in voice gender discrimination and vowel recognition by normal-hearing subjects listening to an acoustic simulation of cochlear implant speech processing and by cochlear implant users. The performance of cochlear implant listeners was similar to that of normal-hearing subjects listening to comparable speech processing (4 8 spectral channels). The results suggest that both spectral and temporal cues contribute to voice gender discrimination and that temporal cues are especially important for cochlear implant users to identify the voice gender when there is reduced spectral resolution. This is important to our study because we are developing a training program for cochlear implantees. Their unique listener characteristics, like reliance on temporal cues, is important to note. Laures, Jacqueline S., & Weismer, Gary. (1999). The effects of a flattened fundamental frequency on intelligibility at the sentence level. Journal of Speech, Language, and Hearing Research, 42(5), The effect of a flattened fundamental frequency (FFF) contour on sentence intelligibility was the focus of this study. Speech intelligibility was assessed using 2 measures: one involving word transcription and the other interval scaling. Results showed that both measures were significantly lower when the FFF contour was flattened, as compared with naturally varying contours. Further studies are needed to look at possible causes for these results. Fundamental frequency is relevant to our study because it is one of the characteristics we will look at when deciding which speakers are most intelligible. Monsen, Randall B. (1978). Toward measuring how well hearing-impaired children speak. Journal of Speech & Hearing Research, 21(2), Intelligibility scores for a group of 37 hearing-impaired and 2 normally hearing adolescents were compared with 9 acoustically measured speech variables (e.g.,

6 measurements of consonant production, vowel production, and prosody). Two variables alone accounted for about 70% of the variance in the intelligibility scores. To validate these correlations, intelligibility scores from another 30 listeners were predicted and then compared with another group of normal listeners. This study is important to decide what variables are effective in measuring speech intelligibility. This is relevant to our study because this study identified 9 variables that we can use to measure speech intelligibility. Pennington, Lindsay, & Miller, Nick. (2007). Influence of listening conditions and listener characteristics on intelligibility of dysarthric speech. Clinical Linguistics & Phonetics, 21(5), doi: This study compared intelligibility scores of dysarthric speech achieved under a standard listening condition with those obtained in non-standard conditions and investigated the effect of listener age, gender and familiarity with speaker accent on intelligibility scores. No differences were observed in the mean intelligibility scores across standard and non-standard listening conditions. No main effects of listener age, gender or familiarity of accent were observed. These results suggest that when conducting intelligibility research listening conditions should be standardized and that adult listeners of either gender and any age may be used to test intelligibility outcomes providing hearing acuity is taken into consideration. This is relevant to our study because it is important to consider gender and age differences when deciding what makes an intelligible speaker. 3) Talker Characteristics: There are many factors influencing the perception of speech. One major component of speech perception by those who have cochlear implants is focused largely on the characteristics of the speaker they are listening to. These talker characteristics may include lexical, phonetic, and contextual characteristics of individual and interactive speech. The articles below all correspond with this category because they focus on more specific talker characteristics that fit under those broader categories of speech in general. These characteristics include gender, rate/rhythm of speech, age, dialect, vowel pronunciation discrepancies, and tone of speech. We chose talker characteristics because the previously mentioned topics are very vital to our research project due to the fact that we are examining which specific talker characteristics lead to speaker intelligibility and thus, clear speech perception. Bradlow, A. R., Torretta, G. M., & Pisoni, D. B. (1996). Intelligibility of normal speech I: Global and fine-grained acoustic-phonetic talker characteristics. Speech Communication, 20(3 4), doi:

7 This article looks at the various talker characteristics that delve into speaker intelligibility. Characteristics specifically included in this article include gender, F0, and speaker rate. The article also looks at a component of talker vowel spaces in their patterns of speech and how that correlates with overall speaker intelligibility. This article is useful for our research project because it explains valuable information about how to improve speech synthesis and recognition strategies for special populations, which in our case includes those of the deaf community with cochlear implants. Chang, Y.-p., & Qian-Jie, F. (2006). Effects of Talker Variability on Vowel Recognition in Cochlear Implants. Journal of Speech, Language, and Hearing Research, 49(6), This article examines the effects of different talker characteristic variability on the vowel recognition of those with cochlear implants and those without. It explains how differences in speech perception are significantly influenced by individual talker characteristics. This article is relevant to our research project because it focuses on many of the different talker characteristics we will be testing and the effects they have on the speech reception of those with cochlear implants. Clopper, C. G., & Pisoni, D. B. (2006). The Nationwide Speech Project: A new corpus of American English dialects. Speech Communication, 48(6), doi: This article examines the speech characteristics of both male and female talkers who also represent the primary regional varieties of American English. The recordings of speech that were gathered are used for research on the perception of dialect variation. This article is relevant to our research project because it explains another talker characteristic that influences speech reception. Dilley, L. C., Wieland, E. A., Gamache, J. L., McAuley, J. D., & Redford, M. A. (2013). Age-Related Changes to Spectral Voice Characteristics Affect Judgments of Prosodic, Segmental, and Talker Attributes for Child and Adult Speech. Journal of Speech Language and Hearing Research, 56(1), doi: Doi / (2012/ ) This article looks at how age-related talker characteristics influence the overall perception of speech. In particular, it focuses on investigating whether the

8 perceived age differences of the speakers were associated with differences in judgments of prosodic, segmental, and talker attributes. This article is relevant to our research project because it focuses deeply on a specific talker characteristic, age, and this could lead to a better understanding of how each of these different characteristics affect speech perception in those with cochlear implants. Reel, L. A., & Hicks, C. B. (2012). Selective Auditory Attention in Adults: Effects of Rhythmic Structure of the Competing Language. Journal of Speech Language and Hearing Research, 55(1), doi: Doi / (2011/ ) This article looks at two specific areas of talker characteristics - the differences between the characteristics in talkers of different mixed-gender pairings, and the rhythmic structure of the language itself of the competing speech. This particular article is very relevant to our research project because will also be testing these two types of talker characteristics in our methods of experimentation. 4) Code Switching: Style-switching or Code-switching refers to switching between different styles of speech or variations in language. This aspect of our study was chosen as a category particularly because speaker intelligibility is not limited to one single language, dialect, or environment. In order to encompass language variations found in varying countries, languages, or environments it is critical to examine code-switching. All of the articles used refer to the differences in intelligibility between clear speech (which would be more formal or scripted speech; for example an audio recording) and conversational speech (which would be more informal or a typical conversation in a crowded environment). Gagné, J., Masterson, V., Munhall, K. G., & Bilida, N. (1994). Across talker variability in auditory, visual, and audiovisual speech intelligibility for conversational and clear speech. Journal of the Academy of Rehabilitative Audiology, 27, Retrieved from This study compared the intelligibility of speakers using clear speech and conversational speech. Clear speech effects were also studied. Overall, clear speech tokens were more intelligible than conversational speech tokens, but the results indicated that clear speech effects could not be produced consistently.

9 Gagné, J., Rochette, A., & Charest, M. (2002). Auditory, visual and audiovisual clear speech. Speech Communication, 37 (3-4), doi: This study also compared the intelligibility of vowels in conversational and clear speech. The results of the study indicated that clear speech and conversational speech are distinctly different, and clear speech tends to improve speaker intelligibility. Krause, J. C., & Braida, L. D. (2002). Investigating alternative forms of clear speech: The effects of speaking rate and speaking mode on intelligibility. Journal of the Acoustical Society of America, 112(5), Retrieved from This study examined the effects of speaking rate and speaking mode on listener intelligibility. Intelligibility in clear and conversational speech was compared. The article has been requested through ILLiad, but has not been received yet.