Impulsivity in students with serious emotional disturbance: the interactive effects of reinforcer rate, delay, and quality

We conducted two studies extending basic matching research on self-control and impulsivity to the investigation of choices of students diagnosed as seriously emotionally disturbed. In Study 1 we examined the interaction between unequal rates of reinforcement and equal versus unequal delays to reinforcer access on performance of concurrently available sets of math problems. The results of a reversal design showed that when delays to reinforcer access were the same for both response alternatives, the time allocated to each was approximately proportional to obtained reinforcement. When the delays to reinforcer access differed between the response alternatives, there was a bias toward the response alternative and schedule with the lower delays, suggesting impulsivity (i.e., immediate reinforcer access overrode the effects of rate of reinforcement). In Study 2 we examined the interactive effects of reinforcer rate, quality, and delay. Conditions involving delayed access to the high-quality reinforcers on the rich schedule (with immediate access to low-quality reinforcers earned on the lean schedule) were alternated with immediate access to low-quality reinforcers on the rich schedule (with delayed access to high-quality reinforcers on the lean schedule) using a reversal design. With 1 student, reinforcer quality overrode the effects of both reinforcer rate and delay to reinforcer access. The other student tended to respond exclusively to the alternative associated with immediate access to reinforcers. The studies demonstrate a methodology based on matching theory for determining influential dimensions of reinforcers governing individuals' choices.     

Resistance to Extinction in the Steady State and in Transition.

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behavioral contrast and behavioral momentum.

In Exp 1, 5 pigeons were trained to peck a key on multiple schedules of food reinforcement. The reinforcer rate was constant in 1 component and varied between conditions in the alternated component. In the constant component, steady-state response rate and its resistance to both prefeeding and extinction were inversely related to the reinforcer rate in the alternated component. Thus, resistance to both prefeeding and to extinction, like response rate, exhibits behavioral contrast. In Exp 2, a time-out period between schedule components eliminated contrast effects on steady-state response rate but not on resistance to extinction. The resistance-to-change results contradict expectations derived from current quantitative accounts of steady-state operant behavior and suggest instead that resistance to change depends on the contingency between component stimuli and reinforcers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus.

In Exp I, 16 New Zealand white rabbits were trained to perform an instrumental head-raising response for sucrose reward. A jaw-movement CR was established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a VI 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a VR 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Exp II (6 Ss), a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in Exp I, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of nonreinforcement of the CS. In Exp III (16 Ss), an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. (53 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The delay-reduction hypothesis: Extension to three-alternative choice.

Choice was assessed for 2 and 3 VI schedules arranged in the terminal links of concurrent-chains schedules (Exps I–IV) and simple concurrent schedules (Exp IV). Ss were 14 male White King pigeons. When the 3rd VI schedule provided a relatively high rate of reinforcement, its addition resulted in a divergence of the distribution of responses between the 2 other schedules (Exps I and II). When the 3rd VI schedule provided a sufficiently low rate of reinforcement, its addition resulted in a convergence of the distribution of responses between the other schedules. Both sets of results are consistent with the delay-reduction hypothesis but not the constant-ratio rule. In Exp IV, the 3rd VI schedule was selected so that the delay-reduction hypothesis required no change in the distribution of responses between the other 2 schedules. This experiment also assessed the effects of adding a 3rd VI schedule in a simple concurrent schedule. In both cases the distribution of responses between the other 2 schedules was not affected systematically. Results support an extension of the delay-reduction hypothesis to 3-alternative choice. (35 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Primacy of interresponse-time reinforcement in accounting for rate differences under variable-ratio and variable-interval schedules.

Pigeons' keypeck rates under VR schedules are typically higher than under VI schedules when between-schedule reinforcement rates are equated. Exp I, with 4 male White Carneaux pigeons, reproduced this between-schedule rate difference in a multiple VR-VI schedule. However, when the short interresponse times (IRTs) typically reinforced under VRs were required for VI reinforcement, between-component rate differences diminished. Exp II (4 Ss) replicated Exp I except that long IRTs reinforced under VI schedules were required for VR reinforcement. This manipulation eliminated between-schedule rate differences. In Exp III (4 Ss), VR/VI and VI/VR tandems had the same correlation between response and reinforcement rates. In a simulation, C. P. Shimp's (1969) IRT response rule was used as an algorithm to mimic between-schedule rate difference. Findings show that IRT reinforcement (a molecular factor), rather than the feedback function between response rate and reinforcement rate (a molar factor), accounts for the higher response rates under VR schedules. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pacemaker rate in the behavioral theory of timing.

The assumption by the behavioral theory of timing that pacemaker rate is proportional to reinforcer density was tested in 2 experiments involving 5 homing pigeons. Ss discriminated between the 1st and 2nd halves of a 50-sec trial. Responses on a left key were reinforced at variable intervals for the 1st 25 sec of the trial, and right-key responses were reinforced at variable intervals during the 2nd 25 sec. In Exp 1, overall reinforcement rate was varied by manipulating the intertrial interval (ITI) duration. Pacemaker rate, estimated by fitting predictions to psychometric functions, was an inverse function of ITI. In Exp 2, reinforcer duration was manipulated as a means of altering reinforcer density. Pacemaker rate was found to be directly related to reinforcer duration. Results support the assumption that pacemaker rate is determined by reinforcer density. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Response–reinforcer associations after caudate-putamen lesions in the rat: Spatial discrimination and overshadowing–potentiation effects in instrumental learning.

Exp I, with 12 male hooded rats, demonstrated that Ss with caudate-putamen lesions exhibited an impairment in the acquisition and reversal of a spatial maze task when compared with unoperated control Ss. Exp II, with 24 Ss, investigated leverpress responding supported by a VI schedule in 3 groups of Ss: a group with caudate-putamen lesions, a group with lesions of the posterior cortex, and an unoperated control group. The presentation of a 0.5-sec, response-contingent light correlated with reinforcement generated an elevated response rate in the 2 operated groups but tended to suppress responding in the control group, perhaps by overshadowing the response–reinforcer relation. Only the group with cortical lesions maintained the elevated rate when the light was uncorrelated with food delivery. Exp III confirmed for these same Ss that caudate-putamen lesions produced a spatial learning deficit. No deficit was observed in the posterior cortex group. It is suggested that caudate-putamen lesions disrupt the mechanism underlying the response–reinforcer association on which spatial maze learning and free operant responding in part depend. (48 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus". Correction to Lovibond.

Reports an error in the article, "Facilitation of Instrumental Behavior by a Pavlovian Appetitive Conditioned Stimulus" by Peter F. Lovibond (Journal of Experimental Psychology: Animal Behavior Processes, 1983, Jul, Vol. 9, No. 3, pp. 225-247)." Part of the second sentence on page 227 was omitted, and the correction is presented here. (The following abstract of this article originally appeared in record 1984-08705-001.) In Exp I, 16 New Zealand white rabbits were trained to perform an instrumental head-raising response for sucrose reward. A jaw-movement CR was established to a 2-sec CS by pairing it with sucrose; a control stimulus was unpaired with sucrose. Instrumental responding maintained by a VI 40-sec schedule was enhanced during 10-sec presentations of the paired, but not the unpaired, CS. Responding on a VR 15 schedule was unaffected except on trials on which the pre-CS baseline response rate was low; in such cases the paired CS caused a long-lasting acceleration of responding. Noncontingent presentation of the sucrose reinforcer itself briefly suppressed responding but had no long-term effect. In Exp II (6 Ss), a CS that had been conditioned at a 10-sec duration produced the same pattern of effects as in Exp I, indicating that facilitation resulted from CS presentation rather than from the frustrative effects of nonreinforcement of the CS. In Exp III (16 Ss), an inhibitory CS blocked facilitation by the excitatory CS but did not itself affect instrumental responding. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cocaine seeking by rats is a goal-directed action.

In 2 experiments rats were trained to self-administer intravenous cocaine on chained schedules using different responses in the initial (drug-seeking) and terminal (drug-taking) links. In both between- (Exp 1) and within-subject designs (Exp 2), the drug-taking response was then either extinguished or reinforced in the absence of the opportunity to perform the seeking response. In a subsequent extinction test with the seeking manipulanda alone, the rate of drug seeking was reduced after the prior extinction of the associated taking response. An additional group trained with a sucrose reinforcer showed a comparable devaluation effect. These findings demonstrate that seeking responses for cocaine and food rewards are mediated by a representation of the contingency between seeking response and the opportunity to take the reward. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stimulus–response overshadowing: Effects of signaled reward on instrumental responding as measured by response rate and resistance to change.

If every food reinforcer delivered on a VI schedule is preceded by a brief flash of light, rats leverpress for food more slowly than if light flashes are given independent of the food and their responding. This retardation in rate has been attributed to the overshadowing of the response–food association by the light–food association. Such an explanation assumes that response rate is positively correlated to the response association strength. Exps I and II investigated this account by using prefeeding and extinction procedures to test the strength of responding in 44 Long-Evans rats that had received food-correlated or random-light signals. Acquisition response rate was lower in the signaled condition (consistent with previous studies), but responding was more resistant to both extinction and satiation, regardless of whether the light cues were presented during the extinction/prefeeding phase. Exp III, with 27 Ss, revealed no difference between the signaled and random conditions either in terms of acquisition response rates or resistance to satiation when a VR schedule of reinforcement was used. Results are inconsistent with an overshadowing account of the acquisition rate difference. Instead, the signal seems to enhance the rat's sensitivity to the contingencies present on VI schedules. (44 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How pigeons estimate rates of prey encounter.

Pigeons were trained on operant schedules simulating successive encounters with prey items. When items were encountered on VI schedules, birds were more likely to accept a poor item (long delay to food) the longer they had just searched, as if they were averaging prey density over a short memory window (Exp 1). Responding as if the immediate future would be like the immediate past was reversed when a short search predicted a long search next time (Exp 2). Experience with different degrees of environmental predictability appeared to change the length of the memory window (Exp 3). The results may reflect linear waiting (J. J. Higa et al, 1991), but they differ in some respects. The findings have implications for possible mechanisms of adjusting behavior to current reinforcement conditions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relative persistence of behavior: A fundamental measure of relative reinforcing effects.

Relative persistence of behavior is measured in terms of rates of behavior at a higher schedule value divided by rates of behavior at a lower schedule value and then multiplied by 100 to express the result as a percentage. Relative persistence of behavior is a direct function of reinforcer size over a broad range of values. In 1 study, food was the reinforcer; however, in most studies of persistence, drug reinforcers were used, and the drugs were taken orally or intravenously. A variety of reinforcing drugs were studied: barbiturates, opioids, psychomotor stimulants, a dissociative anesthetic, and alcohol. Both ratio and interval schedules were used to study relative persistence of behavior in human participants, male and female rhesus monkeys (Macaca mulatta), and rats. Thus, studies of persistence are based on results of an extensive set of conditions. The results support the proposition that relative persistence of behavior is a fundamental measure of relative reinforcing effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of reinforcement on infants' performance in an auditory detection task.

Localization responses to octave-band noises with center frequencies at 400 and 4,000 Hz were obtained from 12-mo-old infants, first without reinforcement and with a 5-sec response interval and then with reinforcement and an unlimited response interval. The percentage of correct responses was substantially greater in the reinforced than in the nonreinforced condition. In Exp II, 12-mo-old Ss were tested in nonreinforced and reinforced sessions, as in Exp I, except that both sessions incorporated the 5-sec response interval. Again, performance was superior in the reinforced session. It is suggested that auditory detection techniques that omit reinforcement may be yielding attentional thresholds rather than thresholds of audibility. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delayed partial reinforcement as an analogue of partial reinforcement.

Combined 2 levels of sucrose (8 and 64%) with 2 delay conditions (12-sec delay and immediate reinforcement) and 2 reinforcement schedules (100 and 50%). Ss were 64 naive female albino sprague-dawley rats given magazine and barpress training and extinction. Acquisition results showed that the highest level of performance was produced by high incentive level, immediate reinforcement, and 100% schedule of reinforcement. In extinction the same relationships were obtained, except that 50% rather than 100% schedules produced superior performance. It is concluded that under conditions where the number of reinforcements were equated, greater persistence of nonreinforced responding was produced by contrasting immediate reinforcement with nonreinforcement as compared to contrasting delayed reinforcement with nonreinforcement. (french summary) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choice behavior in transition: Development of preference with ratio and interval schedules.

In Exp 1, the choice responses of 8 pigeons were observed during 50 periods of transition. Each condition began with equal probabilities of reinforcement on 2 response keys and switched to unequal probabilities. With the ratio of the 2 probabilities held constant, preference for the higher probability developed more rapidly when the 2 probabilities were high than when they were low. In Exp 2, each condition began with 2 equal variable interval (VI) schedules, but later 1 key delivered 60%, 75%, or 90% of the reinforcers. The rate of approach to asymptotic performance was roughly the same with all 3 reinforcement percentages. These and previous results pose difficulties for some well-known models of acquisition, but the results are well described by a simple model that states that the strength of each response is independently increased by reinforcement and decreased by nonreinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Instrumental responding remains sensitive to reinforcer devaluation after extensive training.

Four experiments, with 140 male rats, investigated performance of an instrumental response following postconditioning devaluation of the reinforcer. Exps I and II, which examined whether extended training would make the instrumental response insensitive to such devaluation, found substantial decreases in the performance of both moderately and extensively trained instrumental responses when their reinforcers were paired with an intraperitoneal injection of LiCl (.5 mg/kg). Both experiments revealed that there was some residual performance of a response whose reinforcer had been devalued. Exp III showed residual responding to be attributable neither to unconditioned levels of responding nor to the inadequacy of the devaluation procedure, while Exp IV revealed the amount of residual responding to be the same whether or not the reinforcer was earned by another response during aversion training. Results suggest that a portion of instrumental responding depends on response–reinforcer learning even after extensive training. However, with both moderate and extensive training, some portion of responding is independent of the current value of the reinforcer. (47 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fixed-time schedules attenuate extinction-induced phenomena in the treatment of severe aberrant behavior

We compared the effects of extinction (EXT) and fixed-time (FT) schedules as treatment for severe problem behavior displayed by 3 individuals with developmental disabilities. First, functional analyses identified the reinforcers maintaining aberrant behavior for all 3 individuals. Next, EXT and FT schedules were compared using a multielement design. During EXT, the reinforcer maintaining problem behavior was withheld. During FT, the reinforcers were presented response independently at preset intervals. Results showed that FT schedules were generally more effective than EXT schedules in reducing aberrant behavior. FT schedules may be used in situations when extinction-induced phenomena are problematic.     

Effects of d-amphetamine on responding of squirrel monkeys maintained under second-order schedules of food presentation, electric shock presentation or stimulus-shock termination

The effects of d-amphetamine (0.01-5.6 mg/kg i.m.) were studied on lever pressing of squirrel monkeys maintained under various second-order schedules by a visual stimulus (S) that, with separate monkeys, was occasionally paired with the presentation of either food, electric shock or with the termination of a stimulus in the presence of which shocks occurred. Under one condition, the first response after 5 min produced a 3-sec stimulus change and the fourth stimulus change was followed immediately by food delivery, electric shock presentation or by the termination of a stimulus in the presence of which shocks occurred [fixed-ratio (FR); fixed-interval (FI) [FR 4 (FI 5-min:S)]. The effects of d-amphetamine were also studied under the food- and shock-presentation schedules when food or shock occurred only once, at the end of each session, after completion of 53n 3-min fixed-intervals all of which ended with a brief stimulus change [FR 10 (FI 3-min : S)]. Under a third condition, each thirtieth response produced the 3-sec brief stimulus (FR 30 : S) and the first FR 30 completed after 5 min elapsed produced the stimulus followed by food or, with separate monkeys, electric shock [FI 5-min (FR 30:S)]. Low to intermediate doses of d-amphetamine (0.03-0.3 mg/kg) generally increased and higher doses (0.56-5.6 mg/kg) decreased responding under all conditions. The effects of d-amphetamine on responding maintained by brief stimuli under different types of second-order schedules are generally similar, regardless of the type of reinforcing event or particular second-order schedule.     

Assessment of variables affecting choice and application to classroom interventions.

Reports the results of a brief, computer-based assessment of reinforcer dimensions affecting choice and the application of information yielded by the assessment in informing interventions for classroom disruptions of 2 students (aged 10 & 11 yrs) with attention deficit hyperactivity disorder (ADHD). The assessment involved successive presentations of 2 concurrent sets of math problem options, each of which was associated with competing reinforcer or response dimensions (reinforcer rate, quality, delay, and response effort) in a counterbalanced fashion. The results showed that choice (time allocated problems within each set) was differentially affected by the reinforcer or response dimensions. The authors then investigated the effectiveness of interventions informed by the assessment in reducing the students' classroom disruptions. Following baseline, levels of the reinforcer dimension shown by the assessment to be most influential were alternated in the context of a reversal design. For 1 student, the authors evaluated the effects of immediate versus delayed access to reinforcers contingent on not exceeding a specified number of daily disruptions. For the other student, they evaluated the effects of high quality versus low quality reinforcers on the rate of disruptions. (PsycINFO Database Record (c) 2011 APA, all rights reserved)