SIMULATED SLOT-MACHINE PLAY WITH CONCURRENT VARIABLE RATIO AND RAIVDOM RATIO SCHEDULES OF REINFORCEMENT1

Transcription

1 Psychological Reports, 1980, Psychological Reports 1980 SIMULATED SLOT-MACHINE PLAY WITH CONCURRENT VARIABLE RATIO AND RAIVDOM RATIO SCHEDULES OF REINFORCEMENT1 RUSSELL T. HURLBURT, TERRY J. KNAPP, AND STEVEN H. KNOWLES University of Nevada, LJ Vegas Summary.-Variable and random ratio schedules of reinforcement are conceptually contrasted and experimentally compared in a computer simulated "slot machine" task. 20 subjects did not behave differentially in terms of either choice of game or employment of strategy. The variable ratio (VR) schedule of reinforcement, where reinforcement is contingent upon a varying number of responses, has been singled out as the most conspicuous and powerful element in the environmental circumstances which generate and maintain gambling behavior. Skinner, for example, has said on numerous occasions that the VR schedule lies at the heart of all gambling devices (Skinner, 1969, p. 19). Following his lead, secondary accounts of behavior analysis universally offer the slot machine as a prototypical example of a VR schedule of reinforcement (Ferster & Perrott, 1968; Whaley & Malott, 1971), and similarly students in introductory psychology courses are usually offered the slot-machine'exernplar as a way of understanding and remembering how a VR schedule is defined and what effects on behavior it produces (Mc- Connell, 1977). This paper will show that slot machines do not pay off on the same kind of VR schedule as is most frequently programmed in laboratory experiments, but pay off rather on a random ratio (RR) schedule, a special case of VR schedules. This theoretical distinction between VR and RR schedules is not an original contribution of this paper (Catania & Reynolds, 1968); nonetheless, given the widespread misunderstandings, an amplification of the difference is in order. The paper then goes beyond the theoretical distinction to face the empirical issue, which may be stated: Does a human distinguish between a VR and an RR schedule in a typical gambling game? In a ratio schedule, reinforcement is withheld for a set number of responses and then made available following the next response. If the sequence of numbered responses is anything other than an unending series of the same number, then a VR schedule is defined. Now there are, of course, an infinite number of possible sequences, even if the mean value of the elements of the sequence is 'A lengthier version of this paper was presented at the 4th National Conference on Gambling, Reno, Nevada, December, Request reprints from R. T. Hurlburt or T. J. Knapp, Department of Psychology, Univrrsicy of Nevada, Las Vegas, 4505 Maryland Parkway, Las Vegas, NV

2 636 R. T. HURLBURT, ET AL. specified. Two ways of generating sequences of numbers are of most concern to us here. The first, usually called by operant conditioners "variable ratio," even though the term is an overgeneralization, is here exemplified by a mean ratio specified to be 3.5. The numbers, 1, 2, 3, 4, 5, and 6 are to be arranged in variable order to form the VR sequence. If done repeatedly and randomly, this will result in an indefinitely long sequence of digits which may serve as the run lengths on a VR schedule whose mean run length is 3.5. One-sixth of all runs will be of length 1, another sixth of length 2, another sixth of length 3, etc. The second method of generating the sequence, precisely called "random ratio" by all users, is exemplified as follows: use a roulette-type wheel with seven sections, 5 of which are red, 2 black. Count the number of spins necessary to come up black; use this number of spins as the first element in the sequence. Then begin spinning the wheel again, counting the number of spins until black appears. This number becomes the second element in the sequence. This procedure can also be continued to generate an indefinitely long sequence of numbers which may serve as run lengths on the special kind of VR schedule called an RR schedule. Here again, the sequence will contain elements whose mean is 3.5, but the elements themselves will range from 1 to an indefinitely large number, that is, at some point it will take a very large number of spins for the roulette wheel to stop at black. It is clear that both a VR and an RR schedule can be specified by a sequence of run lengths. However, the distribution of lengths is much different in the two cases. To see one important difference between these distributions, consider two gaming contraptions on either of which one may bet a penny. The first is programmed by the kind of mechanical devices employed in operant psychology laboratories to generate VR schedules and the second programmed by the interworkings of a traditional slot machine (Nolan, 1970), char IS, an RR schedule. One essential difference between these two devices concerns the probability of payoff on the n"' response, given the number of betting responses (o) since the last payoff (reinforcement). The traditional slot machine (RR) has a constant probability of payoff for any given pull of the lever (trial); this is not true for the VR schedule. In the VR case, the probability of payoff increases with each successive pull since the last payoff. This distinction has practical importance in the sense that a VR slot machine becomes a game of skill, viz., the player may estimate where he is in a run, if a variable number of coins is allowed to be played at each pull. The difference between a variable ratio schedule and a random ratio schedule has been noted by operant psychologists (Catania, 1968), and both schedules extensively though independently examined for their effects on the behavior of lower organisms (Thompson & Grabowski, 1972). However, one finds little consideration of this issue in the animal literature, and none at all

3 SCHEDULES OF REINFORCEMENT 637 in the experimental literature concerning human performance on a VR or RR or concurrent VR-RR schedule (Matthews, Shimoff, Catania, & Sagvolden, 1977). The present study undertakes to provide preliminary data on whether humans distinguish VR and RR schedules. METHOD Twenty volunteers were solicited from an undergraduate course in psychology. Each subject received extra credit in the course and was allowed to keep whatever winnings accrued during the experiment. An Online teletype computer terminal provided instructions to the subjects, access to the VR and RR schedules, and stored a permanent record of each subject's data. The instructions indicated that two pairs of "slot machines" would be available for play. During each pair, two slot-machine-type games would be available for play, and the subject could choose to play either game at any time, alternating between games whenever desired. The two games were identical except that one paid on a VR schedule, the other on an RR schedule (subjects had been told nothing about sghedules). Prior to each pair, subjects were given a fixed number of experience or training trials on each game and then supplied with 200 "quarters" with which to play either or both of the games available. A minimum number of pulls was required. Following the playing of both pairs, the subjects collected their earnings at the rate of 1 cent for each dollar they had remaining or a total of about a dollar for each subject. At any time during his participation, the subject could request the computer to provide a display of his current stake. This display was a drawing of a paper cup with the filled depth of quarters approximately indicated. Thus, the subject had access only to approximately, not exactly, how well he was doing at any given time. This was done to simulate as closely as possible the cues the typical slot machine player receives regarding winnings. Furthermore, the printed record of the subject's responses disappeared from view, prohibiting the subject from explicitly reviewing the sequence of wins and losses. On each play of each game, the subject could choose to play from one to five quarters. Payoff was strictly proportional to number of coins played. Each pair provided a VR and an RR game which were matched with respect to expected value. One pair, called here the "6" pair, used 6 pulls as the upper limit on the VR game, while the other, called the "20 pair, used 20 pulls as the upper limit on the VR game. All subjects served in all conditions and were randomly assigned and appropriately counterbalanced across comparisons of the "6" pair and the "20" pair of VR-RR schedules.

4 R. T. HURLBURT, ET AL. RESULTS The data for the present analysis concern the subjects' percent of choice of the VR or RR schedule as a function of the frequency of payoff during training trials, and as a function of the pair of games played. Hence, individual subject's data were scored for percent of trials in which they chose VR over RR, the data summed across individuals, and the difference between schedules and betwee9 frequency of payoff during training trials statistically assessed. Subjects did not choose at a statistically significant level to play either the VR or the RR "slot machine" more often in either pair. More subjects did choose (x2 = 9.53, df = 1, fl <.Ol) to play the "slot machine" that paid off most frequently during the training trials. Each individual subject's data were subjected to his own chi-square analyses to check whether that subject employed a consistent strategy of varying the number of coins bet as a function of the number of trials since the last payoff. Each of the 20 subjects played four games, resulting in 80 chi-squared analyses. With the level of significance set at P <.05, 28 of these analyses indicated that some consistent strategy was employed'in varying the sequence of number of coins played. Of these 28, 22 sequences were such as would maximize payoff if done' while playing the VR "slot machine" but which would have no effect on the RR schedule. Interestingly, however, these subjects' strategies were about evenly divided between the VR and RR "slot machines," indicating a lack of discrimination between the two schedules on the part of the subjects. DISCUSSION With computer simulated "slot machines" subjects did not behave differentially, in terms of either choice of game or employment of a strategy, whether they were playing on a VR or RR schedule of reinforcement. This was so at both a low and high schedule value. Frequency of payoff during training trials, however, did clearly influence the subjects' choice of which "slot machine" to play. The game which paid off most often during the training for each pair was subsequently played more frequently. These data leave ambiguous the behavioral significance of the theoretical distinction between VR and RR schedules of reinforcement. The apparent inability of human subjects to respond differentially to concurrent schedules has been noted by earlier researchers, and recent evidence (Matthews, Shimoff, Catania, & Sagvolden, 1977) suggests that the manner in which the subjects are introduced to the schedules may be critical. Shaping is apparently more likely than verbal instructions to lead to differential responding. However, it seems at this point likely that subjects can learn to discriminate between the VR and RR schedules if the situation is properly constructed. The fact that 22 of 28 strategies are such that they would maximize payoff on VR may be taken as evidence that subjects are making some fine discriminations

5 SCHEDULES OF REINFORCEMENT 639 in this experiment but had not yet performed the target discrimination between schedules. That is, if subjects can acquire a particular strategy, it is likely that they eventually could differentiate between two games, in one of which the strategy was successful while unsuccessful in the ocher. REFERENCES CATANIA, A. C. (Ed.) Contemporary research in opwant behavior. Scott, Foresman, Glenview, IL: CATANIA, A. C., & REYNOLDS, G. S. A quantitative analysis of the responding maintained by interval schedules of reinforcement. ]ournal of the Experimental Analysis of Behavior, 1968, 11, FERSTBR, C. B., & PERROTT, M. C. Behavior principles. New York: New Century, MATTHEWS, B. A., SHIMOFF, E., CATANIA, A. C., & SAGVOLDEN, T. Uninstructed human responding: sensitivity to ratio and interval contingencies. Iotrrnal of the Experimental Analysis of Behavior, 1977, 27, MCCONNELL, J. V. Understanding human behavior. Rinehart & Winston, (2nd ed.) New York: Holt, NOLAN, W. I. The facts on slots. Las Vegas: Gambler's Book Club, SKINNER, B. F. Contingencies of reinforcement: a theoretical analysis. New York: Appleton-Century-Crofts, THOMPSON, T., & GRABOWSKI, J. G. Reinforcement schedules arzd multlopwant arulysts. New York: Appleton-Century-Crofts, WHALEY, D. L., & ~ O T T R., W. Elementa~y principles of behavior. Appleton-Century-Crofts, New York: Accepted August 14, 1980.