Schedules of Reinforcement

Transcription

1 Schedules of Reinforcement Types of Schedule Schedule Performance Analysis What Is a Schedule of Reinforcement? A schedule of reinforcement arranges a contingency (relationship) between an operant and the delivery of a reinforcer. Continuous reinforcement (CRF) Every response is reinforced. Partial or intermittent schedule Not every response is reinforced. Schedules may also specify relationships between reinforcement contingencies and discriminative stimuli. Schedules of Reinforcement as Feedback Functions In a simple control system: a perceptual input is compared to an internallyspecified reference state for that perception. The difference between them creates an error signal, which drives an output. The output affects the environment so as to push the perceptual input toward its reference value (negative feedback), thus closing the loop. A schedule of reinforcement specifies how the output will affect the perceptual input; it thus constitutes an environmental feedback function. 1

2 Control-System Diagram Showing Feedback Function reference Input function input error comparator Output funct Output (key-pecks) disturbances Feedback (grain accesses) Feedback function Reinforcement schedule, e.g. FR-5 Basic Intermittent Schedules Schedules may be based on number of responses, passage of time, or both. Ratio Schedules Reinforcement after a specified number of responses have been completed. Interval Schedules Reinforcement follows first response after the passage of a specified amount of time. Differential Reinforcement Schedules Reinforcement based on the spacing of responses in time or on response omission. Fixed Ratio Schedules Reinforcement after a fixed number of responses has occurred. Symbolized FR-x, where x is the fixed ratio requirement. Example: FR-5: Five responses per reinforcement. Delivery of reinforcement resets the response counter back to zero. 2

3 Responding on Fixed Ratio Schedules Break-and-run pattern: High rate sustained during ratio completion Pause follows delivery of reinforcer ( postreinforcement pause ) Pause length increases with ratio size. Too high a ratio leads to ratio strain. FR-200 FR-120 Analysis of Performance on Fixed-Ratio Schedules In ratio schedules, the rate of reinforcement depends directly on the rate of responding. Because reinforcement is more effective at short than at long delays, high response rates are more strongly reinforced than low ones. This positive feedback loop drives rate during the ratio run to the maximum. After reinforcement, alternate sources of potential reinforcement for various activities compete. Returning to the lever is only weakly encouraged owing to the delay between return to the lever and reinforcer delivery that is imposed by the ratio requirement. This explains the post-reinforcement pause, and why the pause is longer for higher ratios. Variable Ratio Schedules Reinforcement after completion of a variable number of responses; schedule specified by the average number of responses per reinforcement. Symbolized VR-x, where x is the average ratio requirement. Example: VR-20: An average of 20 responses is required before the reinforcer will be delivered, but actual ratio varies unpredictably after each reinforcement. 3

4 Performance on Variable Ratio Schedules Variable ratio schedules maintain a relatively high, steady rate of responding. Little or no evidence of post-reinforcement pausing Too high a ratio produces ratio strain. VR-173 Analysis of Performance on Variable-Ratio Schedules On ratio schedules, rate of reinforcement depends directly on rate of responding. This positive feedback loop drives responding upward toward the maximum. Because VR schedules occasionally present very short ratios, return to the lever is sometimes almost immediately reinforced, allowing this behavior to compete effectively against other behaviors. This explains the lack of post-reinforcement pause. Lack of pauses also rules out fatigue explanation for post-reinforcement pauses in FR schedules. Fixed Interval Schedules Reinforcement follows the first response to occur after a specified fixed interval is over. Delivery of reinforcer resets the interval timer to zero. Symbolized FI x, where x is the interval size in seconds or minutes. Examples: FI 30-s, FI 1-m 4

5 Performance on Fixed Interval Schedules Low or zero rate begins to increase perhaps half-way into the interval, accelerates to the end. This pattern is called the fixed interval scallop because of its scalloped or fluted appearance. FI 4-m Analysis of Performance on Fixed Interval Schedules It is evident that rats can time the interval; therefore there must be internally generated stimuli that correlate with different times since reinforcement. These may function as discriminative stimuli; those occurring near the end of the interval are more closely associated with reinforcement, therefore generate higher rates of responding. Skinner demonstrated that an external clock stimulus would produce an inverted scallop if the clock were run backwards, supporting this analysis. Stable performance may represent a tradeoff between responding too soon (wasted effort) and responding too late (reduced rate of reinforcement). Variable Interval Schedules Reinforcer delivered immediately following the first response to occur after the current interval is over. Interval size varies unpredictably after each reinforcer delivery; schedule specified by the average interval size. Symbolized as VI x, where x is the average interval length in seconds or minutes Examples: VI 20-s, VI 3-m 5

6 Performance on Variable Interval Schedules VI schedules tend to sustain a relatively moderate but steady rate of responding. As the average interval size increases, response rate decreases. Sustain lower rates than VR schedules that produce the same rate of reinforcement VI 3-m Analysis of Performance on Variable-Interval Schedules On variable-interval schedules, the probability that a response will be immediately followed by reinforcer delivery is roughly constant. Because the currently timed interval is sometimes very short, returning to the lever is occasionally followed almost immediately by reinforcer delivery, encouraging immediate return to responding (no pauses or scallops). Reinforcement rate is nearly independent of response rate, above some minimum. As the only way to know whether a reinforcer has been set up by the schedule is to respond, this encourages a moderate, steady rate of responding. Differential Reinforcement Schedules Differential reinforcement of high rates (DRH) A response is reinforced if it occurs within x seconds of the previous response Differential reinforcement of low rates (DRL) A response is reinforced if it occurs no sooner than x seconds after the previous response Differential reinforcement of other behavior (DRO) A reinforceris delivered after x seconds, but only if no response has occurred. Also called an omission schedule. 6

7 Analysis of Differential Reinforcement Performance Inter-response times (IRTs ) vary from response to response. Only those IRTs meeting the schedule requirement are followed immediately by reinforcement. These become more probable, other IRTs relatively less probable, and performance changes in the direction required by the schedule: Short IRTs for DRH, long IRTs for DRL, infinite IRTs for DRO. Applications The simple schedules just described were not intended to model real-world contingencies, but rather, to explore how schedule properties affect the patterning of behavior. Nevertheless, we can draw some parallels between these laboratory schedules and reinforcement contingencies found outside the laboratory. Some examples follow. Piece-work Some jobs pay so much per item produced. The pay rate depends directly on the rate of production of pieces; thus this is a ratio schedule. Where each item requires several steps to complete, breaks will tend to occur after an item is completed rather than in the middle of assembly a post-reinforcement pause. 7

8 Gambling The slot machine is an excellent example. Each response (put money in slot, pull lever) brings you closer to a pay -off. The faster you play, the sooner you win. How many responses you will have to make before a pay -off varies unpredictably after each win. It s a variable-ratio schedule! And what do we know about VR schedules? They generate a high, steady rate of play. Just what the house wants! Waiting for Company to Arrive Your guests are expected at 8 pm. As the time approaches, you glance out the front window to see if anyone has pulled into your driveway. The closer to 8 pm it gets, the more often you glance. If they are punctual, the first glance after 8 pm will find them in the drive. It s (approximately) a fixed-interval schedule, complete with FI scallop. Calling the Plumber One of the water pipes at your house has sprung a leak, and you are desperate to get your plumber out to fix it. He doesn t have an answering machine, so you call, call, call, call, call. Finally, he answers the phone. It s not how often you called that counted, but whether enough time had passed for him to be back from wherever he went. But you had no idea how long that would be. It s a variable-interval schedule, and it generated a moderate, steady rate of calling. 8