From contrast to reinforcement: Role of response contingency in anticipatory contrast.

Intake of a 0.15% saccharin solution is suppressed if access to the saccharin is followed by access to 32% sucrose in brief daily pairings. The present series of four experiments was concerned with factors that lead to this anticipatory contrast effect (suppressed saccharin intake) rather than a reinforcement effect. In Experiment 1, anticipatory contrast was obtained with an autoshaping procedure (no lick requirement on the initial tube), and degree of contrast did not vary as a function of intersolution interval in the range of 0–25 s. Experiments 2 and 3 showed that requirements of 10, 100, 200, or 400 licks on the first tube available led to a reinforcement effect in latency, but a requirement of 0 licks (autoshaping procedure) led to a contrast effect in licks and latency. In Experiment 4, a group with a 200-contingent-lick requirement showed a reinforcement effect in latency, but a group yoked to this contingent group showed a contrast effect in both latency and licks. Overall, the results suggest that anticipatory contrast occurs under conditions of a "relaxed" instrumental contingency. The data are discussed in terms of control of behavior by stimulus–stimulus, response–stimulus, and stimulus–response associations, and the results are related to behavioral contrast, to flavor–outcome associations, and to "misbehavior" produced by Pavlovian-instrumental interactions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Performance of goldfish trained on multiple schedules of response-independent reinforcement.

In 3 experiments, 2 with appetitively elicited target striking and a 3rd with aversively elicited shuttling, 42 goldfish were trained on variable time schedules of response-independent reinforcement with or without a correlation between conditioned stimulus color and probability of reinforcement. Unlike pigeons in keypecking situations, Ss responded strongly even on uncorrelated schedules. In the 2nd and 3rd experiments—in which density of reinforcement and contingency were unconfounded—the level of response to a color was determined only by probability of reinforcement, independently of stimulus-reinforcer contingency. Except on the assumption that both measured responses are products entirely of adventitious response-reinforcer control, which seems particularly unlikely in the case of shuttling, the results cast doubt on the generality of the contingency interpretation of classical conditioning. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Duration of signals for intertrial reinforcement and nonreinforcement in random control procedures.

In the random control procedure, responding to a conditioned stimulus (target CS) is prevented when the probability of unsignaled, unconditioned stimuli (UCS) in the intertrial interval (ITI) is equal to the probability of the UCS in the presence of the target CS. Three experiments used an autoshaping procedure with White Carneaux pigeons to examine the effects of the temporal duration of signals for the ITI UCS (cover CSs) and for concomitant periods of nonreinforcement. In Experiment 1, a short duration cover, but not a long duration cover, resulted in responding to the target CS. In Experiment 2, an explicit CS– cue during periods of nonreinforcement did not affect target acquisition. In Experiment 3, a long CS–, but not a short cover CS, was a sufficient condition for the acquisition of responding to the target CS. These results imply that the acquisition of responding to a target CS requires a discriminable period of nonreinforcement that is long relative to the target CS duration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Parametric manipulation of interresponse-time contingency independent of reinforcement rate.

The pecking of 4 White Carneaux pigeons was reinforced under a modified interval-percentile procedure that allowed independent manipulation of overall reinforcement (RFM) rate and the degree to which RFM depended on interresponse-time (IRT) duration. Results show that increasing the contingency, as measured by the phi coefficient, between RFM and long IRTs while controlling the overall rate of RFM systematically increased the frequency of those IRTs and decreased response rate under both of the RFM rates studied. Increasing RFM rate also generally increased response rate, particularly under weaker IRT contingencies. Random-interval schedules with comparable RFM rates generated response rates and IRT distributions similar to those obtained with moderate-to-high IRT RFM contingencies. It is suggested that IRT RFM contingencies inherent in random-interval and constant-probability VI schedules exercise substantial control over responding independent of overall RFM rate effects. The IRT RFM contingencies inherent in these schedules may actually mask the effects of overall RFM rate; thus differences in response rate as a function of RFM rate when IRT RFM is eliminated may be underestimated. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of contingency violations on the extinction of a conditioned fear inhibitor and a conditioned fear excitor.

In 4 experiments, 192 male Holtzman and Sprague-Dawley rats were used in a conditioned-suppression paradigm to assess the effects of contingency variations on responding to a conditioned stimulus (CS) inhibitor (CS–) and a conditioned stimulus excitor (CS+). In Exp I, various unconditioned stimulus/stimuli (UCS) frequencies were equated across the presence and absence of a CS– in the context of either background cues (continuous-trial procedure) or an explicit neutral event (discrete-trial procedure). With both procedures, a CS-alone treatment enhanced inhibition, whereas treatments involving 50 or 100% reinforcement for the CS– eliminated inhibition without conditioning excitation to that CS. The latter outcome also occurred in Exp II, with discrete-trial training equating considerably reduced UCS frequencies for the presence and absence of the CS–. In further evidence that inhibition was eliminated without conditioning excitation to the CS–, Exp III showed that a novel CS did not acquire excitation when 25, 50, or 100% reinforcement was equated across the presence and absence of that CS in the context of a discrete-trial event. Using the procedures of Exp I, Exp IV showed that a CS+ was extinguished by a CS-alone treatment but was substantially maintained by treatments involving 50 or 100% uncorrelated reinforcement. These effects for a CS+ and a CS– implicate CS–UCS contiguity, rather than contingency, as the factor determining the extinction of a CS. (33 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

More rapid associative change with retraining than with initial training.

Retraining was compared with initial acquisition in 4 magazine approach experiments with rats and 1 autoshaping experiment with pigeons. The levels of performance were matched prior to reinforcement and nonreinforcement for stimuli with a history of both training and extinction and stimuli with only a minimal history of training. Under these conditions, a previously extinguished stimulus was more vulnerable both to the incremental effects of reinforcement and to the decremental effects of nonreinforcement compared with a stimulus that had only been minimally trained. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay versus probability of reinforcement in discrimination learning.

29 male Carneaux pigeons' responses to compound stimuli were measured as a function of changes in reinforcement delay and probability. A 2-key autoshaping procedure was used in which 2-compound stimuli were successively presented. One compound stimulus was simultaneously correlated with immediate reinforcement and a low reinforcement probability, whereas the other compound was simultaneously correlated with various durations of delayed reinforcement and a high reinforcement probability. Results indicate that reinforcement delay and probability had an interactive effect in discrimination learning. Specifically, the control exerted by a high-probability compound decreased as the reinforcement delay correlated with it increased. Within a critical range of delay values, high- and low-probability compounds exerted equal degrees of control; for even longer delays, the low-probability compound exerted greater control. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of the rates of associative change during acquisition and extinction.

Five experiments used a compound test procedure to compare the rate parameters for the associative changes resulting from reinforcement and nonreinforcement. Experiments I and 4, using a magazineapproach procedure in rats, found initial acquisition to proceed more rapidly but to generalize less broadly than extinction. Experiments 2 and 5 repeated these observations in an autoshaping preparation with pigeons. Experiment 3 found no evidence for differential disruption of acquisition and extinction in testing. These results were obtained in a test procedure that compares responding with stimulus compounds in order to remove the differences in overall performance, which have complicated inferences about associative changes in earlier experiments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Simultaneous temporal processing.

Seven studies assessed the ability of 36 male albino Norway rats to process temporal information from 2 internal clocks simultaneously and independently. In the 1st 6 experiments, a light stimulus signaled an overall interval (OI) between the beginning of a trial and the availability of food reinforcement; a sound stimulus was used to signal shorter intervals that divided the OIs into equal segments. When there was a fixed temporal relation between the final segment signal and the availability of reinforcement, there was a double-scallop pattern of responding throughout the segmented OI; the function relating response rate (FRRR) to time during the segment intervals was similar to the FRRR to time in unsegmented OIs; a change in response rate occurred at the time that a normally presented segment signal was omitted. Results show that Ss timed the OI and the segment intervals simultaneously and independently without interference. In Exp VII, a light stimulus was used on some trials and a sound stimulus was used on others to signal a discrete-trial 50-sec peak procedure. When these 2 signals were presented in compound, there was a leftward shift of the response function that suggested that Ss timed both signals simultaneously. (81 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behavioral contrast and behavioral momentum.

In Exp 1, 5 pigeons were trained to peck a key on multiple schedules of food reinforcement. The reinforcer rate was constant in 1 component and varied between conditions in the alternated component. In the constant component, steady-state response rate and its resistance to both prefeeding and extinction were inversely related to the reinforcer rate in the alternated component. Thus, resistance to both prefeeding and to extinction, like response rate, exhibits behavioral contrast. In Exp 2, a time-out period between schedule components eliminated contrast effects on steady-state response rate but not on resistance to extinction. The resistance-to-change results contradict expectations derived from current quantitative accounts of steady-state operant behavior and suggest instead that resistance to change depends on the contingency between component stimuli and reinforcers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Response rate and sensitivity to the molar feedback function relating response and reinforcement rate on VI+ schedules of reinforcement.

In 5 experiments, the author examined rats' sensitivity to the molar feedback function relating response rate to reinforcement rate on schedules of reinforcement. These studies demonstrated that, at lower rates of responding, rats' performance on variable ratio (VR), variable interval (VI), and variable interval with linear feedback loop (VI+) schedules was determined largely by reinforcement of interresponse times; response rates were faster on VR than on both VI and VI+ schedules. In contrast, when procedures were adopted to maintain high rates of response, rats showed sensitivity to the molar characteristics of the schedules; they responded as fast on a VI+ schedule as on a VR schedule and faster on both of these schedules than on a yoked VI schedule. When the variance of response rate was manipulated, this factor was noted as an important element in determining sensitivity to the molar characteristics of the schedule. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Substitutability in time allocation.

Describes a method of measuring substitutability, using concurrent schedules of reinforcement. It is assumed that an animal allocates its time to maximize utility (according to a certain function) within the constraints imposed by the choice situation (the limited time available and the schedules of reinforcement). This assumption implies that when it chooses among various rates and amounts of a single reinforcer, the animal allocates time proportionally to the value of the alternatives (matching). But when the animal chooses among various rates and amounts of different reinforcers, its time allocation deviates from matching. This deviation provides a measure of the substitutability of the commodities. (53 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Partial reinforcement reduces the associative change produced by nonreinforcement.

Four autoshaping experiments with pigeons investigated the associative decrement produced by nonreinforcement of stimuli with a history of partial or continuous reinforcement. In each experiment, one keylight was reinforced on a 25% schedule and one on a 100% schedule. Then nonreinforced presentations of each stimulus were accompanied by different diffuse stimuli. In each experiment, the diffuse stimulus nonreinforced in the presence of the 100% excitor developed inhibition more rapidly than the diffuse stimulus nonreinforced in the presence of the 25% excitor. This inhibition was measured by transfer to another excitor reinforced on 100% (Experiment 1) or 25% (Experiment 2) schedule. The same difference was observed when the 25% excitor underwent a period of 100% reinforcement (Experiment 3) or was associatively stronger than the 100% excitor (Experiment 4). These results suggest that partial reinforcement acts in part to reduce the subsequent effectiveness of a nonreinforcement in producing associative change. This may contribute to the partial reinforcement extinction effect. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pedunculopontine tegmental nucleus lesions impair stimulus–reward learning in autoshaping and conditioned reinforcement paradigms.

The role of the pedunculopontine tegmental nucleus (PPTg) in stimulus–reward learning was assessed by testing the effects of PPTg lesions on performance in visual autoshaping and conditioned reinforcement (CRf) paradigms. Rats with PPTg lesions were unable to learn an association between a conditioned stimulus (CS) and a primary reward in either paradigm. In the autoshaping experiment, PPTg-lesioned rats approached the CS+ and CS– with equal frequency, and the latencies to respond to the two stimuli did not differ. PPTg lesions also disrupted discriminated approaches to an appetitive CS in the CRf paradigm and completely abolished the acquisition of responding with CRf. These data are discussed in the context of possible cognitive function of the PPTg, particularly in terms of lesion-induced disruptions of attentional processes that are mediated by the thalamus. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Hippocampal function is required for feedback control of an internal clock's criterion.

A discrete-trial peak-interval procedure was used to evaluate the effects of hippocampal damage on the control of an internal clock's criterion. Rats first received either lesions of the fimbria-fornix or sham operations. Following surgery rats were trained on a 20-s peak-interval procedure and later were transferred to a 10-s peak-interval procedure. Rats with sham operations were maximally responsive about the time that reinforcement was sometimes made available (10 or 20 s) and showed an oscillation of successive peak-time values similar to biological feedback control systems. In contrast, rats with fimbria-fornix lesions were maximally responsive at a time about 20% earlier than the time that reinforcement was made available (8 or 16 s) and showed no control of successive peak-time values. Taken together, these results demonstrate that a fimbria-fornix lesion reduces the remembered time of reinforcement stored in reference memory, interferes with the internal control of temporal criteria stored in working memory, and has no effect on the animal's sensitivity to stimulus duration or the acquisition of a new temporal criterion. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Degraded contingency revisited: Posttraining extinction of a cover stimulus attenuates a target cue's behavioral control.

In a Pavlovian conditioning situation, unsignaled outcome presentations interspersed among cue-outcome pairings attenuate conditioned responding to the cue (i.e., the degraded contingency effect). However, if a nontarget cue signals these added outcomes, responding to the target cue is partially restored (i.e., the cover stimulus effect). In 2 conditioned suppression experiments using rats, the effect of posttraining extinction of the cover stimulus was examined. Experiment 1 found that this treatment yielded reduced responding to the target cue. Experiment 2 replicated this finding, while demonstrating that this basic effect was not due to acquired equivalence between the target cue and the cover stimulus. These results are consistent with the extended comparator hypothesis interpretation of the degraded contingency and cover stimulus effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Strain differences in avoidance conditioning as a function of the classical CS-US contingency.

Examined rates of shuttle box avoidance responding in 3 strains of rats as a function of classical and instrumental contingencies in 2 experiments. Ss were a total of 126 female albino Fischer, Lewis, and Long-Evans rats. In Exp I, during classical conditioned-stimulus-unconditioned-stimulus (CS-UCS) pairings in the absence of an avoidance contingency, there were large differences between the 3 strains in rates of anticipatory responding to the CS. The same pattern of differences was observed in Exp II when the avoidance contingency was added. None of the instrumental contingencies of CS termination, UCS termination, or the avoidance contingency differentially affected the strains. Classically elicited anticipatory responses and their compatibility with the required avoidance response were viewed as central factors in both the acquisition and maintenance of skeletal avoidance responses. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of intertrial food presentations on anticipatory goal-tracking in the rat

Four experiments examined the sensitivity of anticipatory goal-tracking in the rat to stimulus-food contingency. Contingency was manipulated by varying the probability of food delivery in the absence of a food-tray-light or clicker conditional stimulus (CS), while holding constant the probability of food coincident with the CS. CS control of anticipatory food tray investigation was examined after a period of context extinction in all experiments. Acquisition of stimulus control was undermined by the scheduling of intertrial food deliveries (Experiment 1). The rate of intertrial food deliveries influenced subsequent acquisition of CS control when all intertrial food deliveries were omitted (Experiment 2). When intertrial food deliveries were added to the training regimen subsequent to acquisition of CS control, that control was impaired (Experiments 3 and 4).     

Hippocampal lesions and associated learning in the pigeon.

The performance of pigeons with hippocampal lesions was compared with that of unoperated and neostriatal-lesioned control Ss in 3 experiments. In Experiment 1, hippocampal-lesioned birds were retarded in the acquisition and the maintenance levels of autoshaped responding. However, the deficit was attenuated following the addition of a response contingency to the autoshaping schedule. In Experiment 2, the hippocampal-lesioned birds showed impaired performance on a differential reinforcement of low rates of responding schedule. From the high levels of responding in Experiment 2, underresponding was observed in hippocampal-lesioned birds relative to control Ss on return to the autoshaping schedule in Experiment 3. Results are interpreted in terms of impaired classical conditioning in hippocampal-lesioned birds. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Response suppression on DRL by rats with septal damage.

The effectiveness of the DRL contingency in suppressing response rates of septal and normal male Long-Evans hooded rats was investigated by using a multiple-DRL-yoked-VI schedule. The yoking procedure equated the interreinforcement times on the 2 schedules. Each schedule was in effect for half of each session, and the change in schedule was signaled by the presence or absence of a cue light. Schedule order and DRL delay requirement were varied. For both normal and septal rats, the response rates were higher in the VI than in the DRL component; this demonstrated that the responding of septals as well as normals was suppressed by the differential reinforcement of a particular class of IRTs. A sharp difference in the level of responding occurred at the point of transition from one component of the multiple schedule to the other, thus providing evidence of a discrimination between the 2 schedules for both normals and septals. It is concluded that the responding of septals is suppressed by the DRL contingency and not controlled solely by the density and distribution of reinforcement. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)