Computational Psycholinguistics. Lecture 9: Models of Spoken-Word Recognition

Transcription

1 Computational Psycholinguistics Lecture 9: Models of Spoken-Word Recognition Andrea Weber Computerlinguistik Universität des Saarlandes Overview lectures This week: Short summary of last week Models of word recognition... switching lecturers... The role of morphology in word recognition Andrea Weber Computational Psycholinguistics 2 1

2 Stages of lexical processing In sum... Initial contact Lexical selection Word recognition Lexical access and integration Andrea Weber Computational Psycholinguistics 3 Competition in lexical processing In general: the more word candidates match the incoming speech signal, the more competition there is and the slower recognition proceeds. Furthermore, we not only find parallel activation of candidates that match in onset (beaker-beetle), but also in rhyme (beaker-speaker) Importantly, word candidates can get activated by any part of the speech input, not just the onset (embedding) Andrea Weber Computational Psycholinguistics 4 2

3 Competition (more realistic) /s/ /st/ /stei/ /steip/ sad psychology staple snake stay stupid... tape staple tiger stay stupid... apron tape staple stay apron tape staple Andrea Weber Computational Psycholinguistics 5 Influencing factors... Factors influencing lexical processing: Word length Uniqueness point Word frequency Neighborhood size Phonological priming Semantic priming Concreteness Andrea Weber Computational Psycholinguistics 6 3

4 Fixations over time, Tanenhaus et al. (1996) Andrea Weber Computational Psycholinguistics 7 Validate the paradigm It has been shown that fixations are influenced by properties of the language system (this would not be the case if the results just reflect participants using strategies, by-passing the normal speech comprehension system) For instance, effects of lexical frequency were replicated (Dahan, Magnuson, & Tanenhaus, 2001) We know about frequency effects from other paradigms (high frequent words are faster recognized than low frequent words) Using eye tracking, it was shown that high frequent competitors are fixated more often and earlier than low frequent competitors Also, the time course and probabilities of eye movements closely correspond to response probabilities derived from TRACE simulations (Allopenna, Magnuson, & Tanenhaus, 1998) Andrea Weber Computational Psycholinguistics 8 4

5 Lexikal frequency, Dahan et al. (2001) Click on the bench competitor bell (low frequ.) target bench (low frequ.) competitor bed (high frequ.) Andrea Weber Computational Psycholinguistics 9 TRACE predictions, Allopenna et al. (1998) Andrea Weber Computational Psycholinguistics 10 5

6 Linking hypothesis The close match between predicted and observed fixation patterns allowed the following linking hypothesis (link between lexical activation and eye movements): The activation of the name of a picture determines the probability that a subject will shift attention to that picture and thus make a saccadic eye movement to fixate it. Andrea Weber Computational Psycholinguistics 11 More results on lexical access Besides the studies that were necessary to establish eye tracking as a valid paradigm for word recognition, many other studies found interesting results; among others Competition not only for onset overlap, but also for rhyme overlap (e.g., beaker-speaker) Subcategorical influences (e.g., voice onset time) Use of morphosyntactic information to restrict competition (e.g., gender information in preceding article: le canon-la cassette) Influence of semantic categories (e.g., lock-key) Adjectival modifiers and referential noun phrases (e.g, thouch the plain red square) Influence of preceding verb information (e.g., noch nie flog eine Fliege so hoch-fliese) Influence of prosody (e.g., contrastive focus) Activation of two lexica during non-native word recognition Andrea Weber Computational Psycholinguistics 12 6

7 Referential NPs, Tanenhaus et al. (1996) Four blocks were shown within one display; the blocks differed in three dimensions (marking, color, shape) The instructions referred to a block using a noun phrase with adjectives The display determined which word disambiguated the target block with respect to the other alternatives From a syntactic perspective, adjectives modify the head noun in a NP. Thus the interpretation of an adjective might not occur until the head noun However, in the interpretation of the utterance not the head noun itself is modified, but rather the entity in the discourse model to which the noun refers Thus, when the visual context circumscribes the domain of the referent, interpretation of an adjective could take place as soon as it is encountered. Andrea Weber Computational Psycholinguistics 13 Referential NPs, Tanenhaus et al. (1996) Touch the plain red square early disambiguation mid disambiguation late disambiguation Andrea Weber Computational Psycholinguistics 14 7

8 Referential NPs, Tanenhaus et al. (1996) Launch times for first saccades to target object were faster in the early disambiguation condition than either the mid or late condition. However, no difference was found between mid and late condition * early mid late The point of disambiguation was determined by the instructions in conjunction with the display influenced when eye movements occurred; moreover eye movements to the target object began shortly after the disambiguating word. Andrea Weber Computational Psycholinguistics 15 Prosody and reference resolution (Eberhard, Spivey- Knowlton, Sedivy, & Tanenhaus, 1995) (a) Touch the large red square. When does the utterance become unambiguous? Two objects are large, thus on the lexical level disambiguation takes places at red. In (a), anticipatory eye movements to target during red. When large is emphasized with a high pitch accent, we know that the accent is interpreted as contrastive (the large and the small square). In (b), anticipatory eye movements to target already during LARGE. This is seen as evidence, that prosodic information is used immediately to resolve reference ambiguity. (b) Touch the LARGE red square. Andrea Weber Computational Psycholinguistics 16 8

9 Summary What is lexical processing The means by which words are being recognized What are the stages of lexical processing Initial contact, lexical selection, word recognition, lexical access and integration What influences the process of spoken-word recognition Length, uniqueness point, frequency, neighborhood, phonological and semantic priming, concreteness How eye tracking came into play validating the paradigm by replicating competition and frequency effects, establishing a strong linking hypothesis Some things we learned about lexical access since 1995 Rhyme competition, subcategorical influences, morphosyntactic constraints, semantic field, verb information, prosody,. Andrea Weber Computational Psycholinguistics 17 Models of word recognition Search models (e.g., Forster, 1976) - serial comparison Perceptual input has no direct access to lexical entries Instead, a complete perceptual representation of the stimulus is constructed Forster s model is for both reading and listening Direct Access models - parallel comparison Logogen Model (Morton, 1969), not domain specific Cohort Model (Marlsen-Wilson and Welsh, 1978), spoken words TRACE (McClelland and Elman, 1986), spoken words Shortlist/Merge (Norris, 1994; Norris, McQueen, & Cutler, 2000), spoken words NAM (Neighborhood activation model; Luce, 1998) FLMP (Fuzzy logical model of perception; Massaro, 1998) Andrea Weber Computational Psycholinguistics 18 9

10 Lexical Search Theory (Forster, 1976) A complete perceptual representation of the perceived stimulus is constructed. The perceptual representation is then compared with the representation of what words look like or sound like in access files There are three access files: Orthographic (reading) Phonological (listening) Syntactic/semantic (language production) The access files are organized in a series of bins, and only one bin needs to be searched to find a match Comparisons of the representation of the perceived stimulus are one by one against representations in the bins. Serial position within a bin affects the speed with which it is accessed (i.e., ordered by lexical frequency) Meaning of words is not stored within the access files, rather representations in the access files have a pointer to meaning information in a master file in semantic memory Andrea Weber Computational Psycholinguistics 19 Sketch of lexical search model Orthographic access file Phonological access file Syntactic/semantic access file Bins arranged in order of decreasing frequency house /haus/ HOUSE house castle Master file Cross referencing Andrea Weber Computational Psycholinguistics 20 10

11 Lexical search model In short, lexical access in the search model corresponds to going to the right access file and comparing stimulus with access code Frequency effects can be explained by the ranking of the bins But for example, repetition priming is more difficult to explain. Suggestion: Not lexical access, but memory trace which facilitates decision. Also we know by now, that activation of word candidates begin before a complete word has been presented Andrea Weber Computational Psycholinguistics 21 Direct Access models - The Logogen model In Morton s Logogen model (1969), perceptual input feeds into feature counters called logogens Each word is represented by a logogen (logos=word; genus=birth, origin, class) If the perceptual input contains a feature of a particular word, then the feature count of its logogen increases. Features can be visual or auditory (vertical stem in a letter or plosive air release in stop consonants) Each logogen has a threshold at which it fires - when it fires, the word is said to have been identified and its meaning becomes available Andrea Weber Computational Psycholinguistics 22 11

12 Frequency effect in Logogen cat [kæt] cot [kot] High frequency words have a lower threshold for firing - they are fired easier/ faster Andrea Weber Computational Psycholinguistics 23 Spreading activation in semantic network cradle baby bed hospital nurse animal dentist doctor bird mammal canary fever rain heat delirium sun ostrich green grass yellow Andrea Weber Computational Psycholinguistics 24 12

13 Sketch of Logogen model Cognitive system Auditory stimulus Auditory analysis Auditory attributes Logogen system Visual stimulus Visual analysis Visual attributes When a logogen fires there are two outputs: One to the response buffer for speech One to the cognitive system for comprehension Response buffer Andrea Weber Computational Psycholinguistics 25 Logogen Model Frequency effects: explained as increased experience of a word resulting in a higher resting activation for high frequency relative to low frequency words. (this is also how repetition priming works) Semantic priming: Because of the bi-directional flow of information between the cognitive system and logogen system, activation from one logogen spreads (indirectly) to those for related words. Because activated logogens do not return to their resting level immediately, the primed target will require less perceptual input to be activated to its individual threshold, and hence less time than an unrelated target Neighbourhood effects: original model proposed before such findings were discovered, therefore not developed to account for this finding Andrea Weber Computational Psycholinguistics 26 13

14 The Cohort Model (continuous activation model) First version appeared in 1978 by Marslen-Wilson and Welsh Cohort: A group of words that are in a common candidate set during lexical selection Continuous mapping of acoustic-phonetic information to the lexicon Example rabbit : Cohort 1: all words with [r] Cohort 2: all words with [rae] Cohort 3: all words with [raeb] Onset of next word can be anticipated (uniqueness point) The model has still a strong emphasis on word onsets 3 levels of word recognition Contact with acoustic-phonetic representation of speech input (activation of cohorts based on feature overlap) Selection process (sensitive to syntactic/ semantic constraints) Integration to discourse Andrea Weber Computational Psycholinguistics 27 The Cohort model The emphasis on word onset is potentially problematic (how to recover from early errors in perception) Many words do not have a uniqueness point, but how to recognize then the onset of a word in a speech stream stay cool /steik/ the /k/ could still be part of the first word and until we hear the next sound /u/, we don t know that /k/ must be a word onset (example see last lecture) Andrea Weber Computational Psycholinguistics 28 14

15 TRACE (interactive continuous model) TRACE is fully computationally implemented connectionist model Connectionist systems rely on parallel processing of sub-systems, using statistical properties instead of logical rules to transform information (they are assumed to reflect functional properties of the brain) First version appeared in 1986 McClelland & Elman 3 levels of processing elements (nodes) with resting activation values Features (e.g., [+voice], [+nasal]) Phonemes (e.g., /d/, /g/) Lexical entries (words) Activation spreads from features to phonemes to words Bi-directional connections between nodes within each level and at adjacent levels (top-down influence!!!) Inhibition and facilitation between nodes of a level Andrea Weber Computational Psycholinguistics 29 Sketch of TRACE words /arm/ /gelt/ /elf/ /glut/ FEEDBACK phonemes /g/ /a/ /e/ /l/ /f/ /t/ /i/ spreading activation inhibition features [+velar] [+plosive] [+nasal] speech input Geld Andrea Weber Computational Psycholinguistics 30 15

16 TRACE model Frequency effects are explained by frequently occurring units being weighted higher Phonological priming is explained by weights being temporarily changed before returning to normal Semantic priming is explained by words activating related words Lexical processing can constrain phonetic processing in TRACE (feedback) This makes it, however, hard to identify nature of mispronunciations. Although a mispronounced word activates the wrong set of phonemes, TRACE will be unable to to tell at lexical level which phoneme was mispronounced because top-down feedback will correct the errorful information at the phoneme level TRACE predicts that lexical involvement in phonetic categorization should build up over time (as a word becomes more activated, it will send stronger feedback to phoneme nodes); however, empirical evidence shows that the bias to label ambiguous sounds so as to form a word decreases with time Andrea Weber Computational Psycholinguistics 31 Shortlist (autonomous continuous model) Similar to TRACE but no influences from higher levels on lower levels!! Shortlist is fully computationally implemented connectionist model Two distinct stages Bottom-up information determines set of candidate words for shortlist (candidates can span different proportions of input) Only short-listed candidates compete with each other Activation only spreads from phoneme level to word level (no top-down influence!!!) Bi-directional connections between nodes within a level but not at adjacent levels (no top-down influence!!!) Inhibition and facilitation between nodes of a level Andrea Weber Computational Psycholinguistics 32 16