Responding under basic schedules of reinforcement in the crow.

Conditioned 5 crows to key-peck for food reinforcement using standard operant-conditioning apparatus and procedures. Responding under fixed-ratio, variable-interval, variable-ratio, and fixed-interval schedules of reinforcement was successfully maintained for each S with substantial schedule requirements. Terminal patterns of response under each of the schedules were similar to those displayed by other species, with the exception that crows paused longer after reinforcement, and responded more slowly under fixed-interval schedules, than pigeons usually do. Functional relationships between measures of performance and schedule requirements were quite consistent with reports on other species. (22 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Timing in Choice Experiments.

In Experiment 1, pigeons chose between variable- and fixed-interval schedules. The timer for 1 schedule was reset by a reinforcement on that schedule or on either schedule. In both cases, the pigeons timed reinforcement on each schedule from trial onset. The data further suggest that their behavior reflects 2 independent processes: 1 deciding when a response should be emitted and responsible for the timing of the overall activity, and the other determining what this response should be and responsible for the allocation of behavior between the 2 response keys. Results from Experiment 2, which studied choice between 2 fixed-interval schedules, support those 2 conclusions. These results have implications for the study of operant choice in general. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Acquisition of preference in concurrent chains: Comparing linear-operator and memory-representational models.

Two experiments were conducted to test predictions of 2 models for acquisition of preference in concurrent chains. Pigeons were trained with pairs of fixed-interval and variable-interval terminal-link schedules in a successive-reversal design. Results showed that acquisition rate was faster when fixed-interval schedules preceded and when variable-interval schedules followed the reversal and was approximately constant when the same pair of schedules was used. These results were predicted by both linear-operator and memory-representational models. However, only the linear-operator model predicted effects of schedule duration, Overall, these results demonstrated that a simple linear-operator model can account for the major features of preference acquisition in concurrent chains and suggest that choice in transition and steady state may provide convergent validation of a single delay-discounting function. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Enhanced response acceleration or suppression produced by response-independent food presentations in rats with septal lesions

In two separate experiments, the response rates of rats with septal lesions were compared with those of control rats when response-independent food was presented while subjects were responding on a differential-reinforcement-of-low-rate schedule of reinforcement (Experiment 1) or according to a conjunctive differential-reinforcement-of-high-rate, fixed-ratio schedule (Experiment 2). Free-food deliveries resulted in acceleration of responding of subjects working on the low-rate schedule and in suppression of responding of those animals working on the high-rate schedule. Both of these effects were localized to a brief time period immediately following the free-food deliveries, and baseline rates of responding were not altered. The acceleration and suppression of responding that occurred on the respective schedules were greatly enhanced in rats with septal lesions. This finding supports the suggestion that these animals are hyperreactive to the reinforcing and/or eliciting properties of discrete stimulus events.     

Relaxation measured by EMG as a function of vibrotactile stimulation

Key pecking of two pigeons was maintained under a multiple schedule of food presentation. In the presence of one keylight stimulus responding produced food according to a fixed-interval 5-min schedule. Additionally, during this component, each 50th response produced electric shock. When a different keylight stimulus was present, key pecking resulted in food delivery under a variable-interval 3-min schedule. Responding was suppressed by shock presentation (punishment) but was still positively accelerated throughout each fixed-interval cycle; steady response rates occurred during the alternate component when only the variable-interval schedule was in effect. Overall rates of punished responding were largely unchanged with d-amphetamine (0.1-3.0 mg/kg); unpunished responding was generally either increased slightly or was decreased. Pentobarbital and chlordiazepoxide (1.0-17.0 mg/kg) administered alone increased both punished and unpunished responding at most doses. Combinations of d-amphetamine with either pentobarbital or chlordiazepoxide produced increases in punished responding that exceeded those obtained with either of these drugs alone. The combined effects of d-amphetamine and either pentobarbital or chlordiazepoxide on unpunished responding depended on the individual dose combinations. Combinations of d-amphetamine with pentobarbital or chlordiazepoxide produced effects on both punished and unpunished responding that differed substantially from those obtained when any of these drugs were administered separately.     

Predictable and unpredictable rewards produce similar changes in dopamine tone.

Unpredicted rewards trigger more vigorous phasic responses in midbrain dopamine (DA) neurons than predicted rewards. However, recent evidence suggests that reward predictability may fail to influence DA signaling over longer scales: In rats passively receiving rewarding electrical brain stimulation, the concentration of DA in dialysate obtained from nucleus accumbens probes was similar regardless of whether reward onset was predictable (G. Hernandez et al., 2006). The present experiment followed up on these findings by requiring the rats to work for the rewarding stimulation, thus confirming whether they indeed learned the timing and predictability of reward delivery. Performance under fixed-interval and variable-interval schedules was compared, and DA levels in the nucleus accumbens were measured by means of in vivo microdialysis. The observed patterns of operant responding indicate that the rats working under the fixed-interval schedule learned to predict the time of reward availability, whereas the rats working under the variable-interval schedule did not. Nonetheless, indistinguishable changes in DA concentration were observed in the 2 groups. Thus, reward predictability had no discernable effect on a measure believed to track the slower components of DA signaling. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Apprentissage vicariant dans une situation de conditionnement operant libre.

Investigated the quantitative and qualitative effects of fixed-ratio (FR) and fixed-interval (FI) reinforcement schedules on a free operant behavior acquired in a vicarious learning situation. 84 medical students underwent the direct or vicarious conditioning session. Complete vicarious acquisition was obtained with the FR schedule. With the FI schedule, response rates were lower than those recorded in the nonvicarious situation, but the temporal distribution of responses was inadequate and the behavioral pause was too brief following presentation of reinforcing stimuli. Observation of the entire conditioning session including the extinction phase did not lead to a more rapid extinction. It is suggested that the processes involved in the learning of the FR-controlled operant activity might be less complex and more immediately available to Ss, while in the FI condition the low perceptual saliency of relevant temporal factors might hinder the vicarious acquisition of the operant behavior. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.     

Food-deprivation level alters the effects of morphine on pigeons' key pecking

Four pigeons pecked response keys under a multiple fixed-ratio 30 fixed-interval 5-min schedule of food presentation. Components alternated separated by 15-s timeouts; each was presented six times. Pigeons were maintained at 70%, 85%, and greater than 90% of their free-feeding weights across experimental conditions. When response rates were stable, the effects of morphine (0.56 to 10.0 mg/kg) and saline were investigated. Morphine reduced response rates in a dose-dependent manner under the fixed-ratio schedule and at high doses under the fixed-interval schedule. In some cases, low doses of morphine increased rates under the fixed-interval schedule. When pigeons were less food deprived, reductions in pecking rates occurred at lower doses under both schedules for 3 of 4 birds compared to when they were more food deprived. When pigeons were more food deprived, low doses of morphine increased rates of pecking in the initial portions of fixed intervals by a greater magnitude. Thus, food-deprivation levels altered both the rate-decreasing and rate-increasing effects of morphine. These effects may share a common mechanism with increased locomotor activity produced by drugs and with increased drug self-administration under conditions of more severe food deprivation.     

Schedule-induced kinesic and taxic behavioral stereotypy in the pigeon.

Two experiments explored the functional character of 2 schedule-induced interim behaviors (pacing and retreat) and 1 terminal behavior (keypecking) that developed on fixed-time (FT) schedules of food delivery with a keylight that increased in brightness throughout the interstimulus interval. In Experiment 1, this ramp procedure was compared with a less discriminable ramp, and FT and Random Time (RT) procedures without a signal. Decreased discriminability expanded keypecking in the trial. The FT schedule eliminated only keypecking and the RT procedure eliminated keypecking and retreat while pacing remained. In Experiment 2, predictive and unpredictive ramps were added to the RT procedure. The data suggest that schedule-induced stereotypy can be divided into kinesic stereotypy (pacing), which arises from repeated reinforcer presentations, and taxic stereotypy, which is tied to an increase (keypecking) or decrease (retreat) in the momentary probability of reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interresponse-time reinforcement and behavior under aperiodic reinforcement schedules: A case study using computer modeling.

Computer modeling was used to investigate the extent to which response rates under variable-interval and variable-ratio schedules are compatible with a simple process of interresponse time (IRT) reinforcement, as argued by Peele, Casey, and Silberberg (1984). Their computer model was duplicated, as well as its principal result of a large response rate difference between the interval and ratio schedules. After their model was run under a variety of interval and ratio schedules, it was found that the response rates produced did not exhibit patterns of sensitivity to schedule parameter variation found experimentally. Furthermore, the model predicted a large response rate difference between a variable-ratio and a "linear feedback" variable-interval schedule, contrary to the results of McDowell and Wixted (1986). We concluded that simple IRT reinforcement was probably not adequate as an explanation of schedule effects under aperiodic interval and ratio schedules, although a modification of the Peele et al. model incorporating behaviors that were not measured operants could exhibit schedule sensitivity. This suggested that realistic molecular models of schedule phenomena must involve more than simple IRT reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Molecular mechanisms resulting in pathogenic anti-mouse erythrocyte antibodies in New Zealand black mice

A procedure was developed with pigeons to extend the experimental analysis of punished behavior and the effects of anxiolytic drugs. Under this procedure the completion of a fixed-ratio requirement on a changeover key switched between two variable-interval schedules of reinforcement that were programmed on a second response key. Under one schedule, correlated with a green keylight, key pecks produced only food; under the second schedule, correlated with a red keylight, key pecks produced both food and electric shock. Pigeons were switched into the component with shock if they did not enter that component within 5 min. Parameter values of the variable-interval schedules were manipulated systematically and the effects of two clinically active anxiolytic drugs, buspirone and chlordiazepoxide, were examined. Responding was suppressed during the component with shock (punishment) and, under non-drug conditions, pigeons infrequently switched into the punishment component; changeover responses occurred rapidly when switched into the punishment component. Both buspirone (0.1-3.0 mg/kg) and chlordiazepoxide (3.0-30 mg/kg) increased punished responding at doses that had little effect on unpunished responding; d-amphetamine (0.3-5.6 mg/kg), which was studied only under one parameter of the variable-interval schedule, produced greater decreases in rates of punished responding than in unpunished responding.(ABSTRACT TRUNCATED AT 250 WORDS)     

Two- versus three-alternative concurrent-chain schedules: A test of three models.

Two experiments with pigeons used concurrent-chain procedures with variable-interval schedules as initial links and different delays to food as terminal links. Two schedules were present in all sessions, but a 3rd schedule was alternately present and absent in successive sessions. When the 3rd schedule delivered food with no terminal-link delay, the presence of this schedule led to an increase in preference for the schedule with the shorter terminal link of the 2 unchanged schedules. When the terminal-link delay for the 3rd schedule was 30 s, the presence of this schedule led to a decrease in preference for the schedule with the shorter terminal link of the 2 unchanged schedules. These results are inconsistent with the predictions of R. Grace's (1994) contextual-choice model, but they are consistent with 2 other mathematical models- delay-reduction theory and the hyperbolic value-added model. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using visual reinforcement to establish stimulus control of responding of Siamese fighting fish (Betta splendens).

Stimulus control of ring swimming was studied with male Siamese fighting fish (Betta splendens) using 2-component multiple schedules in which the components were correlated with the presence or absence of air bubbles in the water. In Experiment 1, either response-independent mirror presentations or extinction was juxtaposed with immediate response-dependent mirror presentations. Rates of ring swimming generally were higher with immediate reinforcement than with either response-independent mirror presentations or extinction. In Experiment 2, different durations of response-dependent mirror presentations were juxtaposed. Generally, higher rates of ring swimming occurred with 15-s than with 0-, 1-, or 3-s durations. Results demonstrate that stimulus control of responding can be established with these fish under several conditions of differential reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

System-specific differences in behavior regulation: Overrunning and underdrinking in molar nondepriving schedules.

In two experiments we tested the molar regulation prediction that animals adjust schedule performance to reduce deviations from baseline response totals. Both experiments constrained the baseline drink-burst length under molar nondepriving schedules but allowed rats to continue running without drinking. In Experiment 1, rats were required to run in order to drink. In Experiment 2, water was delivered independently of running by fixed-time (FT) schedules. Under the run-to-drink contingency, rats exceeded their baseline amounts of running (overrunning) but failed to maintain their baseline water intake (underdrinking). The total amount of running that did not lead to drinking approximated a baseline running. Under the FT schedules, rats again underdrank, but total running approximated baseline. These results do not support previous studies that have shown molar equlibrium effects under nondepriving reciprocal schedules. We conclude that (a) contingent running may not substitute for independent running; (b) intermittent access to water reduces the total instigation for drinking; (c) molar regulation differs under reciprocal and nonreciprocal schedules; and (d) more dynamic, system-specific regulatory models need to be developed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

d-Amphetamine, operant history, and variable-interval performance

The effects of d-amphetamine on the bar-pressing of rats maintained under a variable-interval schedule of water reinforcement were examined as a function of the operant history of the subjects. One group of rats initially received 51 sessions of exposure to a fixed-ratio 20 schedule, while a second group received equivalent exposure to an interresponse-time-greater-than-12-sec schedule. Mean group response rate when stable was over ten times as high under the fixed-ratio schedule as under the interresponse-time-greater-than-12-sec schedule. Response rates of the two groups largely converged across 47 sessions of exposure to a variable-interval 60-second schedule, at which time response rates for both groups appeared stable. Acute administration of d-amphetamine sulfate similary affected mean response rates of both groups: A 0.25 mg/kg dose did not obviously affect rate, while doses of 0.5, 1.0, and 2.0 mg/kg produced dose-dependent rate decreases. These results indicate that the efficacy of operant history as a determinant of drug effects may be limited to circumstances where current contingencies do not exercise powerful and direct control over behavior.     

Superordinate categorization via learned stimulus equivalence: Quantity of reinforcement, hedonic value, and the nature of the mediator.

Three experiments examined superordinate categorization via stimulus equivalence training in pigeons. Experiment 1 established superordinate categories by association with a common number of food pellet reinforcers, plus it established generalization to novel photographic stimuli. Experiment 2 documented generalization of choice responding from stimuli signaling different numbers of food pellets to stimuli signaling different delays to food reinforcement. Experiment 3 indicated that different numbers of food pellets did not substitute as discriminative stimuli for the photographic stimuli with which the food pellets had been paired. The collective results suggest that the effective mediator of superordinate categories that are established via learned stimulus equivalence is not likely to be an accurate representation of the reinforcer, neither is it likely to be a distinctive response that is made to the discriminative stimulus. Motivational or emotional mediation is a more likely account. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of electric-shock delivery on schedule-induced water intake: delay of shock, shock intensity, and body-weight loss

In each of four experiments, schedule-induced water intake in the rat was studied under fixed-time 40-sec food delivery. Experiments I and II studied the temporal relationship between response-independent electric-shock delivery and licking. Shock was delivered under a variable-time 60-sec schedule. A lick-dependent delay was imposed so that licking and shock delivery were systematically separated in time by a minimum of 1 to 15 sec. Over a wide range of shock intensities the data failed to reveal a consistent delay-of-shock effect. Similar shock intensities led to similar reduction of water intake at each delay of shock interval. Experiments III and IV studied the effects of body-weight loss on water intake during independent shock delivery. In Experiment III, shock was delivered under variable-time 60-sec with a minimum separation between shock and licking of 5 sec. In Experiment IV, shock was delivered under variable-time 180-sec. The minimum separation between shock and licking was 10 sec. In each study, the resistance of water intake to suppression by shock delivery increased as the degree of body-weight loss increased. Schedule-induced water intake was affected more by shock when the animal was maintained at 90% of free-feeding weight than at 70%.     

Learning the temporal dynamics of behavior.

This study presents a dynamic model of how animals learn to regulate their behavior under time-based reinforcement schedules. The model assumes a serial activation of behavioral states during the interreinforcement interval, an associative process linking the states with the operant response, and a rule mapping the activation of the states and their associative strength onto response rate or probability. The model fits data sets from fixed-interval schedules, the peak procedure, mixed fixed-interval schedules, and the bisection of temporal intervals. The major difficulties of the model came from experiments that suggest that under some conditions animals may time 2 intervals independently and simultaneously. (PsycINFO Database Record (c) 2011 APA, all rights reserved)