Avoidance The Problem of Avoidance The fact that animals can learn to avoid aversive stimuli presented a serious challenge to theorists in the 1940s and 50s. There is no ongoing shock prior to the response, and no shock immediately after. Thus there appeared to be no immediate consequence of the behavior that could serve to reinforce it. Yet avoidance behavior was acquired and maintained. This problem led to a series of attempts to account theoretically for avoidance behavior, each involving a new type of avoidance conditioning procedure. The Shuttlebox A shuttlebox is a test chamber divided into two compartments by a low hurdle or by a doorway (pictured). The grid floor of the shuttlebox can be electrified to deliver the aversive stimulus (shock). It is also equipped with a light or speaker for delivering a warning stimulus. 1
Two-Way Shuttlebox Avoidance: The Procedure In two-way avoidance, the subject (typically a rat) is placed in one compartment of a shuttlebox. After a short while, a tone sounds and continues for 20 seconds, at which time the shocker is turned on. If the rat shuttles to the other compartment during the shock, this turns off the shock and tone. If the rat shuttles before the shock, the tone ends immediately and no shock is delivered. After a short while, the tone comes on again and the rat must shuttle back to the first compartment to terminate the tone or the tone and shock. Two-Way Shuttlebox Avoidance: Performance Initially the rat waits until the shock begins, then runs, sooner or later ending up in the other compartment and ending the shock. After more experience, we find the rat orienting toward the hurdle or doorway during the tone, then running quickly to the other compartment when the shock commences. After even more experience, the rat jumps the gun and runs before the shock begins. Soon the rat is efficiently running to the opposite compartment each time the tone comes on, thus avoiding the shock on most occasions. Accounting For Shuttlebox Avoidance Mowrer s two-factor theory attempts to explain shuttlebox avoidance by appear to two factors: Classical conditioning: During the early phases, each tone ends in shock. This classically conditions fear to the tone. Instrumental conditioning: Now that the tone elicits strong fear, any response that immediately terminates the tone will be negatively reinforced (escape). Thus, the rats are not shuttling during the tone in order to avoid the shock; they are shuttling in order to terminate the (now aversive) tone. 2
Sidman Avoidance This procedure, developed by Murray Sidman in the early 1950s, investigated avoidance in an operant chamber equipped with a lever: In the absence of lever-pressing, a ½ second shock was scheduled every 5 seconds (the S-S or shock-shock interval. After a lever-press, shock was delivered at the end of a 20- second R-S or response-shock interval, if no further responses occurred, after which the S-S interval was reinstated. Each lever-press during the R-S interval reset the R-S timer to the beginning, postponing the shock. In this procedure, there is no explicit CS as there is in two -way shuttlebox avoidance. Performance on Sidman Avoidance Schedules Subjects (typically rats) acquire lever-press avoidance responding on this schedule and tend to respond rather steadily at a rate of about one leverpress every second or so. A few times during each session, the rat may fail to respond during the R-S interval and then receives a shock. Often several minutes elapse after the start of the session during which even a well-trained animal fails to respond on the lever. This phenomenon has been called warm-up. Sidman s Explanation for Sidman Avoidance Performance Sidman proposed an explanation the may be called differential punishment of other behavior. According to this explanation, Responding on the lever is never followed closely by shock because of the R-S interval. Any other ongoing behavior occasionally will be paired with shock delivery, tending to suppress those behaviors. Eventually the only behavior not suppressed is lever-pressing; as a result it occurs frequently. 3
Anger s CATS Theory of Avoidance Douglas Anger proposed an alternative to Sidman s differential punishment theory: Time-correlated internally -generated stimuli mark the passage of time during the R-S interval. Those occurring just before shock delivery (near the end of the R-S interval) become aversive through classical conditioning. Anger called these conditioned aversive temporal stimuli or CATS. A lever-press resets the interval and thus replaces highly aversive end-interval stimuli with less aversive stimuli marking the beginning of the interval. Lever-presses are thus negatively reinforced by escape from CATS. This is essentially Mowrer stwo-factor theory applied to Sidman avoidance. Sidman s Evidence Against CATS Sidman created a variant of his original avoidance schedule. In this variant, a tone sounded during the last few seconds of the R-S interval. The rats now waited for the tone to sound. They then casually approached the lever, pressing it at the last moment before the shock would have been delivered. This explicit CS seemed to act more as a discriminative stimulus for responding than as an elicitor of fear. Although the rats could have kept it from appearing, they did not. Herrnstein and Hineline s (1966) Avoidance Schedule Herrnstein and Hineline (1966) devised an avoidance schedule that eliminated internal, timecorrelated stimuli that might function as CATS: Two variable-time (VT) shock programs were arranged to run simultaneously. In the absence of lever-pressing, a VT 2-minute schedule determined shock delivery. Each response transferred control over shock delivery to the second, VT 4-minute schedule. Each shock-delivery transferred control back to the VT 2- minute schedule. By responding at least once after each shock, the rats could spend most of the session on the VT 2-minute schedule, thus avoiding about half of the shocks that they would have received if they had never responded. 4
Explaining Performance on the Herrnstein & Hineline Schedule Most rats acquired the avoidance response, although only after long training. Note that if the VT 2-minute schedule were about to schedule a shock, a response at that moment would be followed immediately by shock. All that the subjects could learn on this schedule is that shocks tend to occur less often in the presence of responding than in the absence of responding. Herrnstein and Hineline therefore proposed that what reinforces avoidance responding on this schedule is shock-frequency reduction. This explanation refers to the long-term average effect of responding (a molar explanation) rather than to the immediate effect of responding (a molecular explanation). Seligman s Cognitive Theory of Avoidance Martin Seligman proposed that, when responding on traditional avoidance schedules, subjects learn expectancies: They learn to expect that a shock will be delivered if they do not respond. They learn to expect that a shock will not be delivered if they do respond. Subjects prefer not to receive a shock; so they respond. Essentially, this theory states that the reinforcer for avoidance responding is the avoidance of the shock. Avoidance Responding: The Problem of Extinction Early theorists attempted to extinguish avoidance responding in the shuttlebox by disconnecting the shocker. However, subjects continued to respond, hour after hour. Extinction eventually occurred, but only after a very long time. This led some theorists to propose that there must be something special about avoidance responding that made it especially resistant to extinction. 5
The Proper Extinction of Avoidance Is disconnecting the shocker the proper extinction procedure for avoidance responding? No. In this procedure, responses continue to be followed by the absence of shock, just as before. So long as the subject responds reliably, there are no opportunities to learn that shocks no longer occur in the absence of responding. The proper method of extinction would be to deliver the shocks whether or not the subject responded. With this procedure, extinction is quite rapid. The Cognitive Explanation The slow extinction of avoidance responding following disconnection of the shocker can be understood readily in terms of Seligman s cognitive theory of avoidance: Responding continues to be followed by shock absence, confirming the expectation learned during training. Failing to respond no longer leads to shock, but because the subject almost always respond, there are few opportunities to experience this change and revise the expectation. Because subjects continue to expect shock if they do not respond, they continue to respond. Differences in Ease of Acquisition Bob Bolles noticed that some forms of avoidance behavior seemed easier for subjects to acquire than others: One-way shuttlebox avoidance: Rapidly acquired Two-way shuttlebox avoidance: Many trials required Lever-press avoidance: Many, many trials, if ever; very difficult to get subjects responding Bolles developed an explanation for these differences that appealed to SSDRs. 6
SSDRs Organisms have evolved innate behavior patterns to deal with various threats. Bolles termed these behaviors SSDRs, or species-specific defense reactions. In general, SSDRs include: Fighting (self-defense when attacked) Fleeing (running, preferable to a place of safety) Freezing (immobility, when the threat is still relatively distant or uncertain) Bolles Explanation for Differences in Acquisition Rate One-Way Avoidance Subject has only to move from a place of danger to a place of safety. This is compatible with the subject s innate reaction (fleeing). Two-Way Avoidance Subject has to move from a place of danger to one only immediately less dangerous. Neither compatible nor incompatible with fleeing. Lever-Press Avoidance No place to run or hide, nothing to attack, so subject tends to freeze. Generally incompatible with lever-pressing (although subjects sometimes attack the lever). Is Lever-Press Avoidance Really Harder to Learn? My own (limited) experience with lever-press avoidance (in this case, Sidman avoidance) does not confirm Bolles assertion that this type of avoidance is difficult for rats to acquire. In Sidman avoidance, the frequent shocks programmed under the S-S interval tend to break up freezing, increasing the chances that the subject will depress the lever in the course of its activities. In other situations involving relatively infrequent shocks (averaging one per two minutes), rats almost always acquired the required lever-pressing behavior within a session or two. Freezing tended to go away after subjects had some experience in the situation. 7