EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION ON BEHAVIOR MAINTAINED UNDER TEMPORALLY DEFINED SCHEDULES OF DELAYED SIGNALED REINFORCEMENT

The Psychological Record, 2010, 60, 115 122 EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION ON BEHAVIOR MAINTAINED UNDER TEMPORALLY DEFINED SCHEDULES OF DELAYED SIGNALED REINFORCEMENT Marco A. Pulido and Guillermo Martínez Laboratorio de Condicionamiento Operante Universidad Intercontinental, México The present study assessed the effects of systematically separating the cue from the response in temporally defined schedules of delayed signaled reinforcement. Identical schedules were used to study the effects of the independent variable on response acquisition and response maintenance. In the first experiment, 8 groups of 3 naïve rats were exposed to 1 of 8 temporally defined schedules that differed in both the duration of a response opportunity and responsesignal temporal separation. In the second experiment, 3 rats were exposed to the previously described schedules using a within-subjects design. Results in both experiments showed response rate as a decreasing function of responsesignal temporal separation. The findings could be the result of the combination of delaying conditioned reinforcement and blocking the response selected for reinforcement. Key words: Signal-response temporal separation, temporally defined schedules of delayed, signaled reinforcement, response acquisition, response maintenance, rats Chained schedules of reinforcement typically have been used to study conditioned reinforcement, based on the premise that responding in the early links of the chain is maintained by the reinforcing properties of the stimulus that accompanies the next link of the schedule. One way to validate this premise is to compare response rates in the initial links of equivalent chained and tandem schedules. If responding in the early links of chained schedules is maintained by the conditioned reinforcement properties of the signal, then response rates should be considerably lower in the tandem schedule than in the chained schedule. In agreement with a conditioned reinforcement account of chained schedules, Gollub s (1977) review of experiments that compare tandem and chained schedules suggests that response rates in the initial link of two-component, fixed-interval (FI) chained schedules would The authors would like to thank the Facultad de Psicología and the Instituto de Posgrado e Investigación of the Universidad Intercontinental for their support in the preparation and conduction of this study. The authors would also like to thank Dr. Ben Williams for his comments on a previous version of this article. Correspondence concerning this article should be addressed to Marco A. Pulido, Av. Universidad 1330, A, 1102. Colonia del Carmen Coyoacán, C.P. 04100, México, DF. E-mail: mpulido@ uic.edu.mx

102 PULIDO AND MARTÍNEZ be higher than those observed in an equivalent tandem schedule. Gollub s conclusions, however, are not in agreement with the data produced by several studies. Malagodi, De Weese, and Johnston (1973) found that response rates in the initial link of two-component, FI tandem and chained schedules were homogeneously low. In another study, Wallace, Osborne, and Fantino (1982) found that response rates in the initial link of two-component tandem schedules were higher than those produced by equivalent chained schedules. Kelleher and Fry (1962) found a similar result using three-component, FI tandem and chained schedules. Response rates in the initial links of threeand five-component, fixed-ratio (FR) chained schedules were also lower than those produced by equivalent tandem schedules (Jwaideh, 1973). In summary, research developed to assess a conditioned reinforcer account of stimulus function in chained schedules of reinforcement, by means of comparing response rates produced by both chained and tandem schedules, has failed to produce conclusive evidence. Leaving the question of why conditioned reinforcement predictions of responding in tandem and chained schedules have not produced the expected results unanswered (and it is beyond the aim of the present study to answer this question), some scientists have developed other procedures that may help them assess stimulus properties in chained reinforcement schedules. It has also led other scientists to reconsider the pertinence of the concept of conditioned reinforcement in behavior analysis (see, for instance, Fantino & Romanowich, 2007; Squires, 1972). Royalty, Williams, and Fantino (1987) reasoned that variables that modulate primary reinforcer value should similarly change conditioned reinforcement value; thus, if delayed food sustains lower response rates than immediate reinforcement (Sizemore & Lattal, 1977; Skinner, 1938; Williams, 1976), delayed stimulus change in chained schedules should accordingly sustain lower response rates than immediate stimulus change. In order to assess this possibility, Royalty et al. exposed pigeons to three-component, variable interval (VI) chained schedules where schedule transition could occur immediately (e.g., VI 33 s, VI 33 s, VI 33 s) or after a 3-s delay (e.g., VI 30 s FT 3 s, VI 33 s, VI 33 s). The scientists assessed the effect of delaying component transition in the initial, middle, and final links of the chain. In agreement with their hypothesis, results showed that response rates were lower when stimulus change was delayed. This effect was consistent in both the initial and middle links of the chain for all subjects; when signal transition was delayed in the terminal link of the schedule, 4 out of 6 subjects produced lower response rates under delayed signaled transitions. The temporal separation between an operant response and a stimulus change in chained schedules was used by Royalty et al. (1987) to determine if a cue presented within the framework of a chained schedule acquired conditioned reinforcement properties. (See, however, Staddon & Cerutti, 2003, for an alternative interpretation of signal effects in chained schedules.) However, response-cue separation in chained reinforcement schedules is an experimental manipulation that appears in a number of studies with different theoretical interests. The following account presents published studies within different theoretical frameworks but with a common independent variable, responsesignal temporal separation in chained schedules. Their presentation is relevant because they permit the reader to follow the experimental manipulation across different experimental procedures and assess the different results and interpretations associated with the independent variable.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 103 Tombaugh and Tombaugh (1971) exposed naïve rats to a fixed-ratio (FR) 1, fixed-time (FT) 7.5-s chained schedule and varied the placement of a 1.5-s visual cue across the delay interval. Their results showed that response latency was high when no cue was present and low with a continuous signal; intermediate and very similar results were found when the signals were located at the beginning or end of the delay interval. After response acquisition was accomplished, Tombaugh and Tombaugh exposed the experimental subjects to an extinction condition. During extinction, response latencies in both the continuous and late signal conditions were considerably shorter than those obtained under the no-cue and early-cue conditions. The scientists interpreted their results in Pavlovian terms, suggesting that temporal contiguity between the conditioned stimulus and the unconditioned stimulus is an important variable for both respondent and operant behavior. Lieberman, Davidson, and Thomas (1985) exposed naïve pigeons to a random-interval (RI) 20-s, FT 6-s chained schedule where a 1-s change in illumination followed the RI component immediately or 3 s after the emission of the response that initiated the FT. Reinforcement was delivered at the end of the schedule if the peck that initiated the FT component was delivered at the side of the response key selected by the experimenter as correct. Results showed that correct choices for both the immediate and delayed signal conditions were very similar. The scientists interpreted their finding as evidence that delayed signals did not reduce drive in pigeons. Williams, Preston, and De Kervor (1990) conducted a series of experiments designed to assess blocking of the response-reinforcer association in delayed reinforcement schedules. In a first experiment, they exposed previously shaped pigeons to a four-component multiple schedule where the basic schedule consisted of a tandem VI 60 s, FT 8 s and signal placement during the FT was systematically varied. In one component, no signal was presented during the FT; in another component, a 6-s signal was presented during the first part of the FT; and in still another component, a 6-s signal was presented during the last part of the FT. Finally, in an immediate reinforcer condition, the FT was substituted for an 8-s FI (where the last response in the FI produced reinforcement). Results showed that response rates were higher in the immediate reinforcer condition, followed by response rates produced by the condition in which the signal was presented at the beginning of the FT. The lowest response rates were produced by the unsignaled and the late signal conditions, respectively. In a second experiment, the same multiple schedule with four components was again presented to previously shaped pigeons; however, signal duration was shorter (2 s) and the early signal condition was replaced by a procedure in which a cue was presented in the middle of the FT (from 3 to 5 s). Results showed that response rates were high in the immediate reinforcer condition, intermediate in the unsignaled condition, and similarly low in the two signaled conditions. In a third experiment, the scientists exposed experimentally naïve rats to an FR 1, FT 30-s reinforcement schedule. Rats were divided into two groups. In the first group no signal was present during the schedule; in the second group a light was present during the last 5s of the FT. Results showed that subjects exposed to the second condition required more sessions to reach the different response acquisition criteria used in the study. The scientists concluded that their studies suggest that a signal immediately preceding reinforcement

104 PULIDO AND MARTÍNEZ delivery blocks the response-reinforcer association (Williams et al., 1990, p. 395). Williams (1999) exposed groups of experimentally naïve rats to a FR 1, FT 30-s tandem schedule. In one condition a 5-s signal was presented immediately following the response; in another condition a responseproduced signal was presented in the last 5 s of the delay interval; in a third group no signal was presented during the delay interval. Results showed that response rates were high in the immediate signal condition, intermediate in the unsignaled condition, and low in the late signal condition. Williams suggested that response acquisition in the late signal condition could be attributed to the blocking of the response-reinforcer association. Taken together, the studies designed to evaluate the effects of responsesignal temporal separation in reinforcement schedules are difficult to assess. The Royalty et al. (1987) study shows that separation of the response from the signal has clear detrimental effects on response rate maintenance; however, the Tombaugh and Tombaugh (1971) study suggests that a late signal may enhance resistance to extinction, and the Lieberman et al. (1985) study shows that delayed cue presentation may not affect acquisition of a discrimination response. The Williams et al. (1990) study appears to make a clear case in replicating the Royalty et al. findings; however, the effects of response-signal temporal separation differ (especially when comparing Experiments 1 and 2, where the first experiment suggests that the independent variable may have a graded effect and the second suggests that it may produce a steep decline in response rate), and it is impossible to determine why they differ because experimental procedures vary considerably across the three experiments. Lastly, the Williams (1999) study appears to confirm the findings produced in the third experiment conducted by Williams et al.; however, different dependent variables were used in each study and the experiments assess only a limited amount of parametric variations in response-signal temporal separation (only early and late cue presentation). In view of the contradictory findings produced by the literature so far, and the relative difficulty in comparing studies that differ considerably in procedure, the purpose of the present study was to assess the effects of response-signal temporal separation using two different dependent variables (response acquisition and maintenance). To facilitate the comparison of the results produced by each dependent variable, identical schedule contingencies were used in each case. The effects of delayed reinforcement have been studied using a variety of different experimental procedures; however, in most cases an increase in delay-of-reinforcement duration produces a concomitant increase in programmed interreinforcer interval (Lattal, 1987). As a number of studies have found that change in reinforcement frequency modules response rate, reinforcement rate cannot be ignored as an extraneous variable in delay-of-reinforcement studies. One procedure that has been successfully used to control programmed reinforcement rate when varying delayof-reinforcement duration was designed by Weil (1984). This scientist exposed pigeons to temporally defined schedules (Schoenfeld & Cole, 1972) that consisted of a repetitive time cycle of fixed duration (T) that was divided into two components (t d and t ). The first response during t d produced reinforcement at the end of the cycle; responses during t had no programmed consequences. Weil reasoned that by locating t d at the beginning of the cycle and systematically changing its duration within the

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 105 cycle, he could vary delay-of-reinforcement duration without concomitantly changing programmed interreinforcer interval. Despite initially discouraging results (failure to produce a delay gradient), Weil s procedure has been used to produce a delay gradient using response acquisition (Bruner, Pulido, & Escobar, 1999, 2000) and response maintenance as a dependent variable (Pulido & López, 2006). Additionally, the well-documented finding that signaled delays produce higher response rates than unsignaled delays (Richards, 1981; Richards & Hittesdorf, 1978) has been replicated using Weil s procedure (Pulido, Rubí, & Backer, 2009). Thus, a second purpose of the present study was to explore the effects of delayed conditioned reinforcement under a procedure that allows the independent manipulation of programmed reinforcement rate and response-reinforcer temporal separation. Experiment 1 A chained schedule of reinforcement differs from a temporally defined schedule of signaled, delayed reinforcement in a number of ways. First, component transition is response contingent in the chained schedule but response independent in a temporally defined schedule. Second, a temporally defined schedule is a repetitive time cycle of fixed duration and thus a limited hold contingency is periodically in effect; no such contingency is in effect in a chained schedule. A third difference between these schedules has to do with the response identified for reinforcement. This response occurs in the first link of the schedule in a temporally defined program and in the last link in a chained schedule. These differences make the procedure appealing for research for several reasons. First, if the response targeted for reinforcement is only responsible for cue presentation and has no effect on component transition, then change in response frequency may be attributed to the signal alone. (In contrast, change in response frequency in a chained schedule can be equally attributed to component transition and change in the exteroceptive stimulus.) Additionally, a limited hold contingency guarantees that the interreinforcer interval remains constant regardless of change in delay duration. Lastly, the fact that no response is required in the last link of the schedule to produce reinforcement allows the procedure to be classified as a variable delay-of-reinforcement procedure (Lattal, 1987). In summary, the procedure used in the present study may allow a straightforward assessment of the effects of signal and reinforcement delay in reinforcement schedules. Thus the purpose of the first experiment was to try to replicate the Williams et al. (1990; Experiment 3) and Williams (1999) findings using a temporally defined schedule of signaled, delayed reinforcement. The ultimate goal of the study was to produce data that may help determine the most typical effect of delayed signal presentation in delayed reinforcement schedules. Method Subjects. Twenty-four naïve, male Wistar Lewis rats were used as subjects. All subjects were approximately 5 months old at the beginning of the study. Each subject s weight was registered on 5 consecutive days under free-feeding conditions to determine ad libitum body weight; food was then restricted until all subjects reached 80% of their free-feeding weight. Subjects were kept at their prescribed body weights throughout

106 PULIDO AND MARTÍNEZ the experiment by supplementary feeding following each experimental session. Subjects were kept in the laboratory vivarium under constant temperature conditions and a 12-hr light-dark cycle (lights on at 7:00 a.m.). All experimental subjects were kept in individual cages with free access to water. Apparatus. Sessions were conducted in a custom-built rodent operantconditioning chamber made of transparent Plexiglas. The space in which the subjects were studied measured 18.5 cm in height by 23.5 cm in length by 23.5 cm in depth. A stainless steel lever made of a 3-cm bar topped by a metal disk 2 cm in diameter was placed on the front wall of the chamber. The lever was placed 5.5 cm above the floor and 11 cm apart from each wall. The lever required a force of at least 24 g for depression. A depression of the lever produced an audible click and was counted as a response. A metal plate 5 cm in diameter and located 2 cm below and to the right of the lever was used as a pellet receptacle. A BRS-LVE, PDH-020 pellet dispenser delivered four.25- mg pellets in each emission. Pellets were produced by remolding pulverized Purina Nutri Cubes. Two 1.1-W, 28-Vdc pilot lights with a translucent glass cover were used to illuminate the experimental chamber. One light was located inside the box, 7 cm above the food receptacle. The second light was placed outside the chamber, pasted directly on the center of its Plexiglas ceiling. A Sonalert that delivered an 87.62-dB auditory signal was attached to the external front wall of the experimental chamber, 5 cm to the left of the lever. The conditioning chamber was housed inside a larger, soundattenuating wooden box equipped with a ventilating fan. Experimental events were programmed and recorded using an IBM-compatible 386 microcomputer equipped with an industrial automation card (Advantech PC-Labcard 725) coupled to a relay rack. Procedure. During the first session, with the lever absent from the chamber, each rat was exposed to a magazine training procedure. Magazine training consisted of 30 consecutive response-independent food deliveries using a FT 30-s schedule. All experimental subjects consumed the food in the tray after just one exposure to the schedule. On the second session (and on 30 additional consecutive sessions) the lever was inside the experimental chamber and groups of 3 subjects each were exposed to one of the eight different reinforcement schedules. Schedules consisted of a 32-s temporally defined schedule of reinforcement (Schoenfeld & Cole, 1972). The schedule consisted of a repetitive time cycle of fixed duration (T). The first response emitted during the cycle produced reinforcement at the end of T. In all experimental groups, schedules consisted of two different components that alternated within the reinforcement cycle (t d and t ). A response emitted during t d produced reinforcement at the end of the cycle; responses during t were recorded but had no programmed consequences. The experiment can be conceptualized as a between-groups factorial design with two factors: (a) t d duration (4 s or 8 s) and (b) signal conditions (unsignaled, immediate signal, briefly delayed signal, and delayed signal). The t d duration was varied in order to produce two delay-of-reinforcement values. Under t d = 4 s conditions, the first response emitted during the first 4 s of the T cycle produced reinforcement after 32 s had elapsed. In this arrangement nominal delay lasts at least 28 s; however, as this procedure allows responding to occur throughout t d and t, nominal delay will not forcefully equal obtained delay (and thus the arrangement may be characterized as a variable delay-

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 107 of-reinforcement procedure [Lattal, 1987; Schoenfeld, Cole, Lang, & Mankoff, 1973]). In a similar fashion, when t d = 8 s, the first response emitted during the first 8 s of the reinforcement cycle produced food at the end of T. In this later case, nominal delay lasts at least 24 s; however, as was previously discussed, obtained delay can be considerably shorter. Signal presentation during the experiment was arranged along the following lines. In the unsignaled condition, no programmed exteroceptive stimulus was presented during the reinforcement cycle. In the immediatesignal condition a change of illumination (the pilot light located inside the box went out and the light located on the top of the box was activated) and the activation of the Sonalert were produced by the first response emitted during t d. Both visual and audible cues were turned off at the end of t d. As the signals occurred until the first response during t d was emitted, signal duration in this particular condition could be considerably shorter than t d value. In the briefly delayed signal condition, the previously described cues were similarly produced by the first response emitted during t d but were presented to the subject at the end of t d. In this particular experimental condition, maximum response-signal temporal separation could not exceed t d duration but could be considerably shorter depending on the precise moment a response was emitted during t d. In the t d = 4 s condition, cue presentation lasted 4 s; in the t d = 8 s condition, cue presentation lasted 8 s. In the delayed signal condition, the first response during t d produced the previously described visual and audible cues; however, tone and illumination change did not occur until the last few seconds of the reinforcement cycle. In t d = 4 s condition, the signal was presented exactly at the onset of the 28th second and remained present until the end of the cycle; in the t d = 8 s condition, the signal appeared at the onset of the 24th second and remained present until the end of the cycle. Figure 1 shows a schematic representation of the experimental procedure. t d (sec) 4 s 8 s Unsignaled t d Immediate Signal Briefly Delayed Signal t Signal Delayed Signal T = 32 s T = 32 s Figure 1. Schematic representation of the experimental procedure. S R

108 PULIDO AND MARTÍNEZ Sessions were conducted 6 days per week at approximately the same time each day. Each session lasted 1 hr or the time necessary to obtain 30 reinforcers, whichever occurred first. Unsignaled Immediate Signal Briefly Delayed Signal Delayed Signal Consecutive Sessions t d = 4 s Consecutive Sessions t d = 8 s Figure 2. Response rate per minute for each subject on each experimental session for all experimental conditions.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 109 Results Most of the graphic and inferential analyses presented in this article were designed and conducted using blocks of five sessions. This has been a classic way of analyzing data in delay-of-reinforcement literature because it both allows data variability to appear and allows the inferential tests enough degrees of freedom to operate and identify significant differences between groups (see, e.g., Chung, 1965; Hearst, 1962; Weil, 1984; and Williams, 1976). The data in this study will be analyzed in congruence with the procedures used in these classic studies (and for the same reasons). Figure 2 shows mean response rate per minute as a function of exposure to the different schedules, for each session and for each subject. The graphs located on the left side of the figure correspond to those conditions where t d = 4 s; the graphs located on the right side of the figure correspond to those conditions where t d = 8 s. The unsignaled conditions are located at the top of the figure, followed respectively by the immediate signal condition, the briefly delayed signal condition, and the delayed signal condition. In general, response rates produced under the short t d value were lower than those produced when t d = 8 s. Also, the number of nonresponding animals was higher with the short t d value than with the long t d value (8 vs. 4 subjects). In both t d durations response rates produced under unsignaled delay conditions were comparatively low. The effects of response-signal temporal separation appear to vary according to t d duration. Under the short t d value, response rates decreased in an orderly manner as the temporal separation between the response and the signal increased. In contrast, response rates produced under the long t d value were relatively low in both immediate and delayed signal conditions and comparatively higher in the briefly delayed signal condition. To assess the effects of the independent variables on the acquisition of free-operant responding, a two-factor analysis of variance was conducted. The analysis assessed the effects of t d duration and response-signal separation on the response rates of the last five sessions. The results showed that the main effect of t d duration reached statistical significance, F(1, 119) = 9.035, p =.003. The main effect of response-signal temporal separation also reached statistical significance, F(3, 112) = 11.16, p =.000, as did the interaction between the independent variables, F(3, 112) = 8.408, p =.000. To further assess the effects of response-signal separation on response rate, a linear regression analysis was carried out using response-signal temporal separation as an independent variable and response rate of the last five experimental sessions as a dependent variable. Separate regression analyses were carried out for both t d conditions; Table 1 shows the results for both analyses. Table 1 Linear Regression Between Response-Signal Temporal Separation and Response Rate Experimental Condition R 2 F Coef. t p t d = 4 s.267 16.98.157 4.12.000 t d = 8 s.003 0.11.023 0.33.741 Table 1 shows that the slope of the regression equation for the short t d value is statistically different from zero; this regression equation has negative slope,

110 PULIDO AND MARTÍNEZ suggesting that separating the signal from the response decreases response rate. In contrast, the slope of the regression equation for the long t d value is not statistically different from zero. Figure 3 shows local response-rate-per-minute distribution in 8-s bins of the interreinforcer interval. The graphs located on the right side of the figure correspond to the 8-s t d duration condition; the graphs located on the left side of the figure correspond to the shorter t d duration. The different response-signal temporal relations are presented from top to bottom. Unsignaled Immediate Signal Briefly Delayed Signal Delayed Signal 8-s Subintervals of T t d = 4 s 8-s Subintervals of T t d = 8 s Figure 3. Mean response rate per minute in consecutive 8-s bins of the reinforcement cycle for all subjects and experimental conditions in the last five experimental sessions.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 111 Under the short t d value, response patterns for all conditions appear relatively flat, and response rate descended in an orderly manner as the response-signal interval increased. In immediate signal conditions, a slight increase in response rate appears during the third bin for Subjects F1 and F3. The briefly delayed signal produces an important decrease in response rate in the second bin for Subject G9 and a more conspicuous decline for Subject G7. Under delayed signal conditions, response rates produced by Subject G11 are relatively high during the first two bins and comparably lower in the last two bins. Under the long t d value, response rates per minute are relatively higher in the briefly delayed signal condition and lower in both the immediate and delayed signal condition (with the exception of G4, whose response rates are the second highest in the long t d condition). Under the immediate signal condition, response rates decline slightly during the second bin for Subjects E7 and E8. Under briefly delayed signal conditions, signal presentation during the second bin sharply increases response rate for Subject G1; in contrast, signal presentation during the second bin has a slightly inhibiting effect on response rate for Subject G2 and no noticeable effect for Subject G3. Discussion In general, with the exception of the results produced by the subjects exposed to the long t d duration and immediate signal, the data produced by the present experiment align with those produced by Williams, Preston, and De Kervor (1990) and Williams (1999). Under the short t d duration, response rates were an orderly and decreasing function of lengthening response-signal temporal separation; under the long t d duration condition, subject performance in the immediate signal condition remains somewhat of a mystery, but the fact that response rates in the briefly delayed signal condition were generally higher than those produced under the delayed signal condition is congruent with the aforementioned studies. Several different explanations may be provided to explain the anomalous findings produced under the long t d value and the immediate signal. First, response acquisition studies do not guarantee that the experimental subjects will come into contact with prescribed reinforcement contingencies (especially if response rates are very low, as is the case in the immediate signal condition). Another possibility has to do with the experimental procedure used in the present study, which allows signal duration to be considerably shorter than t d duration in immediate signal conditions. Thus it is possible that obtained signals in immediate signal conditions under the long t d value were particularly small, and several studies have suggested that signal duration modifies response rate in chained reinforcement schedules (Pulido, Rubí, & Backer, 2009; Schaal & Branch, 1988, 1990). This last hypothesis cannot be further explored under the present circumstances as no provision was made to measure obtained signal duration under immediate signal conditions. Another possibility suggests that the effects of response-signal separation on operant behavior depend on interreinforcer interval duration. Several studies have suggested that signal effects on operant behavior are strongly determined by interreinforcer interval (Schaal, Odum, & Shahan, 2000; Schaal, Schuh, & Branch, 1992). In both the present study and the Royalty et al. (1987) experiment, relatively

112 PULIDO AND MARTÍNEZ long interreinforcer intervals were used (32 s and 99 s, respectively). In contrast, interreinforcer intervals in both the Tombaugh and Tombaugh (1971) study and the Lieberman et al. (1985) experiment were comparatively brief (7.5 s and 6 s, respectively). Thus the possibility that response-signal separation effects on operant behavior depend on the specific temporal parameters of the interreinforcer interval cannot be excluded. Apparently, under a short interreinforcer interval, response-signal temporal separation maintains an inverse function with response rate. However, under richer reinforcer contingencies it may have no effects (as in the Lieberman et al. study) or response-enhancing effects (as in the Tombaugh & Tombaugh study). The fact that mean reinforcement rate per minute under the long t d value is considerably higher than that obtained under the short t d value (.322 >.144; t = 2.09(16), p =.004) is in general agreement with the argument that signal effects may be controlled by interreinforcer interval. Future studies may help clarify this issue. Experiment 2 To date, the experiments conducted in order to assess the effects of response-signal temporal separation in delay-of-reinforcement studies have used different procedures and dependent variables. The purpose of the present study was to assess the effects of this independent variable using a single experimental procedure across different dependent variables. The first experiment presented in this study assessed the effects of responsesignal temporal separation on response acquisition of free-operant responding. As has been mentioned before (Pulido, Sosa, & Valadez, 2006), response acquisition is a problematic dependent variable because schedule contingencies are difficult to assess when the subject fails to respond (or responds at very low rates). In the first experiment presented in this study, at least 12 subjects responded at very low rates (did not reach three responses per minute in any experimental session) and thus their contact with schedule contingencies cannot be taken for granted. The purpose of the second experiment was to guarantee that all subjects would be exposed to the different schedule contingencies by replicating the first experiment but using a within-subjects experimental design and response maintenance as a dependent variable. Method Subjects. Three naïve, male Wistar Lewis rats were used as subjects. The subjects were littermates and were 90 days old at the beginning of the study. Each subject s weight was registered on 5 consecutive days under free-feeding conditions to determine ad-libitum body weight; food was then restricted until the subjects reached 80% of their free-feeding weight. Subjects were kept at their prescribed body weights throughout the experiment by means of supplementary feeding following each experimental session. Subjects were kept in the laboratory vivarium under constant temperature conditions and a 12-hour light dark cycle (lights on at 7:00 a.m.). All experimental subjects were kept in individual cages with free access to water. Apparatus. The same apparatus and equipment used in the first experiment were employed to register and program reinforcement contingencies in the second experiment.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 113 Procedure. During the first session, with the lever absent from the chamber, each rat was exposed to a magazine training procedure. Magazine training consisted of 30 consecutive response-independent food deliveries using a FT 30-s schedule. All experimental subjects consumed the food on the tray after just one exposure to the schedule. During the second session (and during 30 additional consecutive sessions), the lever was placed inside the experimental chamber and the 3 animals were exposed to a 32-s temporally defined schedule of reinforcement. The schedule consisted of a repetitive time cycle of fixed duration (T). The first response emitted during t d produced reinforcement at the end of the cycle. This first program was used to develop consistent responding at the beginning of the experiment and was later used as a baseline condition throughout the study. After 30 sessions of exposure to this schedule, subjects were exposed to all experimental conditions described in Experiment 1 in different order, with the exception of the unsignaled delay condition. The unsignaled delay condition was not assessed because the relatively short lifespan of rodents made it difficult to guarantee that they would finish the experiment. Table 2 shows the particular sequence of experimental conditions that each subject received. Table 2 Sequence of Experimental Conditions Subject I1 I2 I3 t d Signal t d Signal t d Signal 1 8 8-16 4 28-32 8 25-32 2 8 0-8 4 0-4 8 8-16 3 4 4-8 8 25-32 8 0-8 4 4 28-32 8 8-16 4 0-4 5 8 25-32 4 4-8 4 28-32 6 4 0-4 8 0-8 4 4-8 Each experimental condition was preceded and followed by the previously detailed baseline condition. Both experimental and baseline conditions were in effect for at least 15 sessions; once the 15th session was reached, response rates were studied daily in order to determine if the stability criterion had been met. Performance was considered stable if response rates in five consecutive sessions did not differ by more than 20% of their common mean. Sessions were conducted 6 days per week at approximately the same time each day. Each session lasted 1 hr or the time necessary to obtain 30 reinforcers, whichever occurred first. Results Figure 4 shows in the y axis response rate per minute for the 3 experimental subjects on the last 5 days of each experimental condition.

114 PULIDO AND MARTÍNEZ The x axis shows the three different experimental conditions separated by the baseline conditions. The graph located in the top part of the figure shows the data produced under the t d = 4 s condition; the graph located in the bottom part of the figure shows the data produced under the t d = 8 s condition. t d = 4 s BL Immediate BL Brief BL Delayed t d = 8 s BL Immediate BL Brief BL Delayed Figure 4. Response rate per minute for the 3 subjects in the last 5 days for all experimental conditions. Experimental conditions are separated by their corresponding baselines. Under t d = 4 s conditions, response rates reached their higher level in the immediate signal condition for two of the subjects (I1 and I3); response rates for Subject I2 remained comparatively low. Under briefly delayed signal conditions, response rates for Subjects I1 and I3 diminished considerably; however, response rates for Subject I2 increased notably. Response rates in the delayed signal condition were the lowest obtained for the 3 subjects under t d = 4 s conditions. Response rates for Subject I2 decreased dramatically, whereas response rates for Subjects I1 and I3 showed only a modest decrease in response rate. Under t d = 8 s conditions, response rates reached their higher level in the immediate signal phase for all 3 subjects. Under the briefly signaled delay, response rates decreased significantly for Subject I2 and more modestly for Subjects I1 and I3. Under delayed signal conditions, response rates for Subject I2 once again decreased dramatically; however, response rates under delayed signal conditions appear to be very similar to those produced under briefly delayed signal conditions for Subjects I1 and I3.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 115 To further assess the effects of the independent variables on response maintenance, a two-factor analysis of variance was conducted. The analysis assessed the effects of t d duration and response-signal temporal separation on response rates produced during the last five sessions. Results showed that the main effect of t d duration did not reach statistical significance, F(1, 89) =.145, p =.704. The main effects of response-signal temporal separation reached statistical significance, F(1, 84) = 13.441, p =.000. No interaction effects were found, F(1, 84) = 000, p = 1.0. To further assess the effects of response-signal separation on response rate, a linear regression analysis was carried out using responsesignal temporal separation as an independent variable and response rate of the last five experimental sessions as a dependent variable. Separate regression analyses were carried out for both t d conditions; Table 3 shows the results for both analyses. Table 3 Linear Regression Between Response-Signal Temporal Separation and Response Rate Experimental Condition R 2 F Coef. t p t d = 4 s.589 22.8.101 4.77.000 t d = 8 s.451 35.34.087 5.94.000 Table 3 shows that the slope of the regression equations (for both t d = 4 s and t d = 8 s) are statistically different from zero; both regression equations have negative slopes, suggesting that separating the signal from the response decreases response rate. A comparison of the regression coefficients from both equations suggests changes in response-signal temporal separation have more robust effects under the t d = 4 s condition than under the longer t d value. Figure 5 shows local response rate distribution in 8-s bins of the interreinforcer interval. The graphs located at the bottom of the figure correspond to the 8-s t d duration condition; the graphs located at the top of the figure correspond to the shorter t d duration. The three columns in the figure correspond to different response-signal temporal relations. Figure 5 suggests that responding during the interreinforcer interval was more or less homogeneous for all subjects and experimental conditions. Response patterns appear particularly flat under t d = 4 s conditions. Under the immediate signal condition and the short t d duration, response rates appear to be an inverse function of cycle duration at least for Subjects I1 and I3. A similar pattern can be observed for the 3 subjects under the immediate signal condition and the long t d duration. Under the briefly delayed signal condition and the long t d duration, response rates were high at the beginning of the reinforcement cycle and relatively lower during the rest of the cycle. In the delayed signal condition, for both t d values, response rates for all subjects slightly decreased at the end of the reinforcement cycle (probably suggesting that the signal had acquired discriminative properties that control feeder-visiting behavior).

116 PULIDO AND MARTÍNEZ t d = 4 s t d = 8 s 8-s subintervals of T 8-s subintervals of T 8-s subintervals of T Immediate Signal Briefly Delayed Signal Delayed Signal Figure 5. Mean response rate per minute in consecutive 8-s bins of the reinforcement cycle for all subjects and experimental conditions in the last five experimental sessions. Discussion The results produced by this study suggest that response rate is an inverse function of response-signal temporal separation. This finding is in general agreement with the data produced by Royalty et al. (1987), Williams et al. (1990), and Williams (1999) and in disagreement with the results produced by Tombaugh and Tombaugh (1971) and Lieberman et al. (1985). Response rates were generally high under immediate signal conditions and gradually declined as the response-signal interval increased. This finding was general across different t d values (and thus across different programmed delays). Taken together, the present study, the Royalty et al. experiment, and the Williams experiments make a strong case for a conditioned reinforcement account of stimulus function in chained reinforcement schedules. Unfortunately, the present data provide no information that may help explain the reasons for the negative findings produced in the Tombaugh and Tombaugh and Lieberman et al. studies. Pulido and López (2006) have produced data that suggest that delay-of-reinforcement effects on operant behavior sustained by temporally defined schedules may develop slowly and only after stable performance has been achieved. In both the Tombaugh and Tombaugh study and the Lieberman et al. study, subjects were exposed to the experimental contingencies without previous shaping and no attempt was made by the scientists to determine if stable performance had occurred. Thus it is possible that, as in Experiment 1 of this study, the animals in both the Tombaugh and Tombaugh study and the Lieberman et al. experiment failed to adequately probe the reinforcement contingencies programmed by the scientists.

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 117 General Discussion In general, both the first and second experiments suggest that the operation of separating the response selected for reinforcement from visual and auditory stimuli correlated with reinforcement has inhibiting effects on operant behavior. This finding replicates results produced by Royalty et al. (1987) using three-component chained schedules. The results also replicate the findings produced by Williams et al. (1990) using different experimental procedures and Williams (1999), using a two-component chained schedule, as well as results produced using four-component chained schedules recently published by Bejarano and Hackenberg (2007). The present findings also agree with data produced by Lieving, Reilly, and Lattal (2006) using an observing response. These scientists exposed experienced pigeons to multiple schedules where a food-correlated luminous stimulus could be produced by treadle pressing and hopper access was produced by key pecking. In general, Lieving et al. found that treadle pressing rates were a decreasing function of increasing response-signal temporal separation. The results of the present study also extend previous findings by suggesting that the effects of response-signal temporal separation are general across two different dependent variables (response acquisition and steady state behavior). Some scientists have suggested that the concept of conditioned reinforcement adds little to our understanding of behavior maintained by chained schedules (Staddon, 1983). The present study (together with the Royalty et al., 1987, Leiving et al., 2009, & Williams, 1982, 1999, experiments) suggests that conditioned reinforcement may have an important role in explaining behavior produced by chained schedules. A review of the research produced so far also suggests how little we know about this phenomenon. The present study, as well as those preceding it, suggests that interreinforcer interval could have an important role in modulating signal effects when the visual or auditory cues are separated from the response selected for reinforcement. The systematic assessment of the effects of this variable on chained reinforcement schedules is still a pending assignment for behavior scientists. This article has sustained the premise that the inverse relationship between response-signal temporal separation and response rate found in chained reinforcement schedules may only be interpreted in terms of a conditioned reinforcement delay gradient. However, Williams (1982, 1999, 2001) has suggested that signal-placement effects during delay interval may be understood in different ways. Specifically, Williams has suggested that a signal occurring in the last portion of a delay interval will block the response-reinforcer association and thus inhibit response maintenance and acquisition. The basic argument behind this premise is that, as the signal immediately precedes reinforcement, it becomes a better predictor of this event and thus prevents the remote response from becoming associated with its consequence. The problem with Williams s argument is that we can find no way of empirically distinguishing between response inhibition produced by response-signal delay duration and a blocking phenomenon. It is possible that both delay and blocking contribute to the overall effects. Perhaps a future study may develop a strategy that can help assess the exact contribution of each variable.

118 PULIDO AND MARTÍNEZ In any case, the results produced by the present study suggest that the effects of separating a signal from a response have considerable generality across experimental procedures. The manipulation decreases response rate even when reinforcement delivery occurs more than a minute away from the signal (as in the Royalty et al., 1987, study) or much less than 30 s away from the signal (as in the present study). The finding is also general across different procedures designed to promote component transition (the same finding is produced by a contingent and by a noncontingent component transition). In a similar vein, the same finding is produced by a procedure constrained by a limited hold contingency (as in the present study) and without this constraint (as in the Royalty et al. study). The finding also appears to be general across different animal species, as essentially the same findings were produced by pigeons (in the Royalty et al. study) and rats (in both the Williams, 1999, and the present study). The results from the present study are also interesting as few experiments have been conducted so far that allow the comparison of the effects of the same independent variables on different dependent variables. The results produced in the present study are in general agreement with those produced by Pulido, López, and Lanzagorta (2005) and Pulido and López (2006) that show that both steady state performance and response acquisition lead to similar conclusions about the effects of signaled and unsignaled delay of reinforcement on operant behavior. However, results produced in the immediate signal condition of the long t d value in the first experiment suggest that insufficient exposure to reinforcement contingencies in responseacquisition studies may produce anomalous data. The aforementioned results also suggest that 30 experimental sessions may not be sufficient to produce stable behavior in some conditions and that prolonged experimental exposure to reinforcement schedules may not compensate for low or nonexistent operant behavior levels in experimental subjects. Local response rates in both the first and second experiment suggest that subjects respond in a more or less homogeneous way throughout the reinforcement cycle. Local response rates also suggest that signal effects on behavior may be highly varied (especially in the response-acquisition study), producing notable response-enhancing effects in Subject G5 and responseinhibiting effects in Subject G9. Much of this variability disappears in the second study where a stringent stability criterion was in effect and the only constant effect is a gradual decline in response rate as reinforcement delivery approaches. No direct attempt to measure obtained delays was made in the present study; however, local response rates suggest that responding was more or less constant throughout the cycle and thus obtained delays must have been considerably shorter than nominal delays. A basic assumption in the Tombaugh and Tombaugh (1971) study was that a late signal (a signal occurring at the end of the reinforcement cycle) should enhance responding in a chained schedule because the temporal separation between the signal and food delivery should enhance the Pavlovian association between the signal and the unconditioned stimulus. The data produced in this study suggests that a late signal could acquire discriminative properties that increase the probability of food receptacle visits (as suggested by the decline in response rates in the fourth bin of the subjects in the second experiment). In opposition to the Tombaugh and Tombaugh hypothesis, no evidence was found of an overall increase in response rate in the late

EFFECTS OF RESPONSE-SIGNAL TEMPORAL SEPARATION 119 signal condition. It may be argued that Tombaugh and Tombaugh used a different interreinforcer interval than the one used in this study; however, research has shown that long interreinforcer intervals enhance Pavlovian conditioning (Prokasy & Whaley, 1963; Salafia, Mis, Terry, Bartosiak, & Daston, 1973), and the aforementioned scientists used an extremely short interval (7.5 s). These facts suggest that other variables should be explored in order to account for the anomalous Tombaugh and Tombaugh data. One possibility has to do with uncontrolled exteroceptive stimuli during the offset of the delay interval. Tombaugh and Tombaugh retracted the lever from the chamber once the response identified for reinforcement had been emitted (and thus all of their experimental procedures had immediate exteroceptive feedback). Future studies may assess the effects of this extraneous variable on delayed signal reinforcement procedures. In conclusion, the results from the present study (as well as other studies presented in this article) suggest that separating the response from the signal in delayed-reinforcement procedures decreases response rate. This effect suggests exteroceptive stimuli presented during delay interval may function as conditioned reinforcers, blocking stimuli, or both. However, the general issue regarding stimulus functions in chained schedules is far from solved. As was mentioned before, a number of experiments involving the experimental manipulation assessed in this study have failed to replicate the finding, and the culprit variables have yet to be both identified and assessed. Furthermore, the literature regarding tandem and chained schedule comparisons has produced results that are basically incongruent with a conditioned reinforcement account of signal functions in delayed-reinforcement procedures. This last brief outline suggests that signal functions in chained reinforcement schedules still require a major theoretical reformulation that may provide a more comprehensive account of their effects. In a similar vein, this area of research still awaits a vast parametric analysis that may allow scientists to predict signal effects according to schedule parameters. In any case, after more than 40 years since Kelleher and Fry (1962) published their paper on chained reinforcement schedules, much research and theoretical development regarding signal functions in chained reinforcement schedules is still pending. References BEJARANO, R., & HACKENBERG, T. D. (2007). IRT-stimulus contingencies in chained schedules: Implications for the concept of conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 88, 215 227. BRUNER, C., PULIDO, M. A., & ESCOBAR, R. (1999). Response acquisition and maintenance with a temporally defined schedule of delayed reinforcement. Revista Mexicana de Análisis de la Conducta, 25, 379 391. BRUNER, C., PULIDO, M. A., & ESCOBAR, R. (2000). La adquisición del palanqueo con programas temporales de reforzamiento demorado. Revista Mexicana de Análisis de la Conducta, 26, 91 103. CHUNG, S.-H. (1965). Effects of delayed reinforcement in a concurrent situation, Journal of the Experimental Analysis of Behavior, 8, 439 444. FANTINO, E., & ROMANOWICH, P. (2007). The effect of conditioned reinforcement rate on choice. A review. Journal of the Experimental Analysis of Behavior, 87, 409 421.

120 PULIDO AND MARTÍNEZ GOLLUB, L. R. (1977). Conditioned reinforcement: Schedule effects. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 288 312). Englewood Cliffs, NJ: Prentice-Hall. HEARST, E. (1962). Delayed alternation in the pigeon. Journal of the Experimental Analysis of Behavior, 5, 225 228. JWAIDEH, A. R. (1973). Responding under chained and tandem fixed-ratio schedules. Journal of the Experimental Analysis of Behavior, 19, 259 267. KELLEHER, R. T., & FRY, W. T. (1962). Stimulus functions in chained fixedinterval schedules. Journal of the Experimental Analysis of Behavior, 5, 167 173. LATTAL, K. A. (1987). The effect of delay and intervening events on reinforcement value. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior (Vol. 5). Mahwah, NJ: Lawrence Erlbaum Associates. LIEBERMAN, D. A., DAVIDSON, F. H., & THOMAS, G. V. (1985). Marking in pigeons: The role of memory in delayed reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 11, 611 624. LIEVING, G. A., REILLY, M. P., & LATTAL, K. A. (2006). Disruption of responding maintained by conditioned reinforcement: Alterations in response-conditioned-reinforcer relations. Journal of the Experimental Analysis of Behavior, 86, 197 209. MALAGODI, E. F., DE WEESE, J., & JOHNSTON, J. M. (1973). Second order schedules: A comparison of chained, brief stimulus, and tandem procedures. Journal of the Experimental Analysis of Behavior, 20, 447 460. PROKASY, W. F., & WHALEY, F. L. (1963). Intertrial interval range shift in classical eyelid conditioning. Psychological Reports, 12, 55 58. PULIDO, M. A., & LÓPEZ, L. (2006). Delay of reinforcement effects under temporally defined schedules of reinforcement. Revista Mexicana de Análisis de la Conducta, 32, 39 53. PULIDO, M. A., LÓPEZ, L., & LANZAGORTA, N. (2005). Effects of contingent and non-contingent signals during delay interval on response acquisition by rats. Revista Mexicana de Análisis de la Conducta, 31, 227 244. PULIDO, M. A., RUBÍ, M., & BACKER, C. (2009). Effects of signal duration on response acquisition with delayed reinforcement under temporally defined schedules of reinforcement. Revista Mexicana de Análisis de la Conducta, 34, 147-164.. PULIDO, M. A., SOSA, R., & VALADEZ, L. (2006). Adquisición de la operante libre bajo condiciones de reforzamiento demorado: Una revisión. Acta Comportamentalia, 14, 5 21. RICHARDS, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35, 145 152. RICHARDS, R. W., & HITTESDORF, W. M. (1978). Inhibitory stimulus control under conditions of signaled and unsignaled delay of reinforcement. The Psychological Record, 28, 615 625. ROYALTY, P., WILLIAMS, B. A., & FANTINO, E. (1987). Effects of delayed conditioned reinforcement in chain schedules. Journal of the Experimental Analysis of Behavior, 47, 41 56.