Influence of interresponse time reinforcement on signalled-reward effect.

Examined the conditions under which presenting a signal for reinforcement decreases or increases the rate of leverpressing in rats. Response rate on a variable-interval (VI) schedule of reinforcement was decreased by a brief signal for reinforcement. In Experiment 1, requiring 1 short IRT to occur on completion of the VI requirement attenuated this response decrement. Requiring three responses to be emitted within a minimum amount of time at the end of the VI resulted in the reinforcement signal's enhancement of response rates. Experiment 2 replicated this increase in responding and showed that the reinforcement signal attenuated response rates on a simple VI schedule that yielded the same overall rate of reinforcement. In Experiment 3, the reinforcement signal enhanced response rates when 3 responses were required at the end of a VI schedule, but the signal attenuated response rates when the 3 responses could occur at any time in relation to the VI. These results suggest that the pattern of responding emitted immediately prior to reinforcement is a critical factor in determining the effect of reinforcement signal on response rate. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using visual reinforcement to establish stimulus control of responding of Siamese fighting fish (Betta splendens).

Stimulus control of ring swimming was studied with male Siamese fighting fish (Betta splendens) using 2-component multiple schedules in which the components were correlated with the presence or absence of air bubbles in the water. In Experiment 1, either response-independent mirror presentations or extinction was juxtaposed with immediate response-dependent mirror presentations. Rates of ring swimming generally were higher with immediate reinforcement than with either response-independent mirror presentations or extinction. In Experiment 2, different durations of response-dependent mirror presentations were juxtaposed. Generally, higher rates of ring swimming occurred with 15-s than with 0-, 1-, or 3-s durations. Results demonstrate that stimulus control of responding can be established with these fish under several conditions of differential reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Partial reinforcement reduces the associative change produced by nonreinforcement.

Four autoshaping experiments with pigeons investigated the associative decrement produced by nonreinforcement of stimuli with a history of partial or continuous reinforcement. In each experiment, one keylight was reinforced on a 25% schedule and one on a 100% schedule. Then nonreinforced presentations of each stimulus were accompanied by different diffuse stimuli. In each experiment, the diffuse stimulus nonreinforced in the presence of the 100% excitor developed inhibition more rapidly than the diffuse stimulus nonreinforced in the presence of the 25% excitor. This inhibition was measured by transfer to another excitor reinforced on 100% (Experiment 1) or 25% (Experiment 2) schedule. The same difference was observed when the 25% excitor underwent a period of 100% reinforcement (Experiment 3) or was associatively stronger than the 100% excitor (Experiment 4). These results suggest that partial reinforcement acts in part to reduce the subsequent effectiveness of a nonreinforcement in producing associative change. This may contribute to the partial reinforcement extinction effect. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Free-operant performance on variable interval schedules with a linear feedback loop: No evidence for molar sensitivities in rats.

Four experiments examined rats' sensitivity to molar and molecular factors on instrumental schedules of reinforcement. rats were exposed to a variable interval schedule with a positive feedback loop (VI+), such that faster responding led to a shorter interreinforcement interval. In Experiments 1 and 2, rats responded faster on a variable response (VR) schedule than on either a VI schedule matched for reinforcement rate or a VI + schedule matched for the feedback function. In Experiment 3, rats responded no differently on a VI schedule than they did on a VI+ schedule with equated rates of reinforcement. In Experiment 4, rats responded faster on a VI+ schedule with an interresponse time requirement yoked to that experienced on a VR schedule, than on a VI+ schedule with the same feedback function as the VR schedule. Taken together these results suggest that rats are more sensitive to the molecular than the molar properties of the schedules. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Limbic lesions and the energizing, aversive, and inhibitory effects of non-reward in rats.

Tested 30 male albino Wistar rats with bilateral lesions in the amygdala, septum, hippocampus, stria terminalis, and fornix on a multiple reinforcement schedule in which barpressing in one component was associated with VI reinforcement (S+) and the other with extinction (S–). Responses on a 2nd lever turned off S– for 5-sec periods during the extinction component. All groups, with the exception of Ss with amygdaloid lesions exhibited behavioral contrast. Ss with hippocampal or fornical lesions showed greater resistance to extinction. Response rates on the lever that turned off S– were higher after stria terminalis and septal lesions, whereas lower rates were obtained from Ss with amygdaloid lesions. It is concluded that amygdaloid lesions attenuate the energizing and aversive effects of nonreward, septal and stria terminalis lesions increase the aversive effects, and hippocampal and fornical lesions interfere with the inhibitory effects of nonreward. (French summary) (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Schedules of reinforcement as determinants of human causality judgments and response rates.

Experiments examined the effect of relationships between a response and an outcome on human judgments of causal effectiveness. In Experiment 1, the time between outcomes obtained on a variable ratio (VR) schedule became the intervals for a yoked variable interval (VI) schedule. Response rates were higher on the VR than on the VI schedule. In Experiment 2, the number of responses required per outcome on a VR schedule were matched to that on a master VI 20-s schedule. Both ratings of causal effectiveness and response rates were higher in the VR schedule. In Experiment 3, tandem VI fixed-ratio (FR) schedules produced higher rates and judgments than equivalent conjunctive VI FR schedule. In Experiment 4, a VI schedule with a reinforcement requirement for a short interresponse time (IRT) produced higher rates and judgments than a simple VI schedule. These results corroborate the view that schedules are a determinant of both response rates and causal judgments. Few current theories of causal judgment predict this pattern of results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Variable ratio control of the spitting response in the archer fish (Toxotes jaculator).

Archer fish (Toxotes jaculator) hunt for prey by spitting water at airborne insects. In groups a fish engaged in spitting may lose its prey to a competitor and thereby experience nonreinforcement. To account for the maintenance of spitting in a competitive context, we propose that this species-specific response functions as an operant controlled by a socially induced variable ratio (VR) schedule of reinforcement. To determine whether VRs can control spitting, we trained 3 fish independently in a conditioning apparatus to spit at a food dispenser. After continuous reinforcement the fish were placed on extinction and then reconditioned. Then, they were exposed to increasing VR values, extinction, and reconditioning. All 3 fish showed evidence of conditioning, extinction, schedule control, and a partial reinforcement extinction effect. Operant control of spitting in isolated fish suggests that socially induced VRs can maintain spitting in groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The observing-response procedure: A novel method to study drug-associated conditioned reinforcement.

In this experiment, the observing-response procedure was adapted for use with drug self-administration. Rats' responding for oral ethanol was sometimes reinforced on a random-ratio schedule, whereas at other times it had no effect (i.e., extinction). Behavior producing stimuli associated with the otherwise unsignaled random-ratio and extinction periods (i.e., observing behavior) was acquired and maintained. In a vehicle control condition, both self-administration and observing behavior decreased, but observing decreased less rapidly proportionally to baseline than vehicle consumption. Thus, conditioned reinforcers may have persistent effects that are relatively independent of the current status of the primary reinforcer. The procedure allows long-term study of drug-associated conditioned reinforcement and provides independent indexes of the conditioned reinforcing and discriminative stimulus effects of drug stimuli. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Enhanced cue function of olfactory stimulation in mice with septal lesions.

Trained a total of 121 male hybrid normal mice and mice with septal lesions on a go-no-go discrimination task (multiple fixed interval of 15 sec, 15-sec extinction schedule) in 4 experiments. When the discrimination stimulus (the one which indicated to the S whether responding at the end of the interval would be reinforced) was a pellet of food (delivered at the start of the interval) or odor of food or non-nutritive substance (present throughout the 15-sec interval), acquisition of septum-damaged mice was enhanced. These lesions did not, however, alter performance when the discrimination was cued by a buzzer or flashing light. Results suggest that septal lesions produce an increased reactivity to olfactory stimuli and to stimuli associated with the delivery of food reinforcement. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Transfer of value from S+ to S- in a simultaneous discrimination.

Value transfer theory has been proposed by L. von Fersen et al (see record 1991-29523-001) to account for transitive inference effects, in which following training on 4 simultaneous discriminations (A?+?B–, B?+?C–, C?+?D–, D?+?E–) pigeons showed a preference for B over D. According to this theory, some of the value of reinforcement acquired by each stimulus always associated with reinforcement (S+) transfers to the stimulus associated with nonreinforcement (S–). In the transitive inference experiment, C (associated with both reward and nonreward) could transfer less value to D than A (associated only with reward) could transfer to B. Support for value transfer theory was demonstrated in 2 experiments, involving a total of 20 pigeons, in which an S– presented in the context of an S+ was preferred over an S– presented in the context of a stimulus to which responses were sometimes reinforced (S±). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Responses eliminated by noncontingent or negatively contingent reinforcement recover in extinction

Autoshaped key pecking in pigeons was eliminated by presenting reinforcers only during non-CS periods (negatively contingent reinforcement) or in both non-CS and CS periods (noncontingent reinforcement). In either case, when all reinforcers were subsequently removed (simple extinction), responding recovered strongly (Experiment 1). Recovery in extinction occurred only if the CS was in a conditioned state when non-CS reinforcers were introduced (Experiment 2). Recovery from noncontingent reinforcement was virtually complete, since total responding in extinction after response elimination was not less than in control groups extinguished without an intervening response-elimination phase (Experiment 3). Recovery also occurred for nonautoshapable, instrumentally reinforced key pecking (Experiment 4). The hypothesis that recovery is due to reinstatement of the non-CS stimulus conditions of acquisition (absence of food) was not supported (Experiments 5 and 6). Other accounts of recovery are considered.     

Discrimination learning during the first year: Stimulus and positional cues.

In four studies, 3-, 6-, and 9-month-old human infants were tested in a discrimination learning task in which visual fixation to a particular stimulus or lateral position was reinforced with an auditory stimulus. In Experiment 1, all age groups exhibited acquisition, extinction, and reinstatement of fixation to the reinforced target or position. Experiment 2 revealed that 3-month-olds retained the positional discrimination but not the stimulus discrimination after a 5-min delay between acquisition and extinction; older infants retained both types of discriminations. In Experiments 3 and 4 we investigated a possible developmental shift in the dominance of positional versus stimulus cues by training infants on displays in which stimulus and position were confounded and then by dissociating the cues on test trials. Results from both experiments indicated positional cue dominance for young infants and stimulus cue dominance for older infants. The findings are discussed in terms of differences in the attentional demands elicited by proprioceptive versus exteroceptive cues. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An experimental analysis of steady-state response rate components on variable ratio and variable interval schedules of reinforcement.

Three experiments explored a novel approach to analyzing the different components of response rate that are produced by exposure to free-operant schedules of reinforcement. It has been suggested that overall response rate comprises a tendency to initiate responding, and to continue to respond once the bout is initiated. Previous post hoc analyses of interresponse times (IRT) data have suggested several features of these different aspects of responding that the current experimental procedure broadly confirmed. Increasing the size of a variable interval (VI) schedule decreases the number of “burst-initiation” responses, but has less effect on responding once the burst has been initiated (Experiment 1); that the major difference between a variable ratio (VR) schedule and a VI schedule, is not in the number of “burst-initiation” responses, but in the number of “within-burst” responses, with shorter interresponse times that are emitted, with greater numbers of such “within-burst” responses being emitted on a VR schedule (Experiments 2 and 3). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Resistance to Extinction in the Steady State and in Transition.

Three experiments with pigeons explored the constancy of reinforcer omission during extinction conjectured by rate estimation theory. Experiment 1 arranged 3-component multiple variable-interval (VI) schedules with a mixture of food and extinction trials within each session. Reinforcers omitted to an extinction criterion increased with food-trial reinforcer rate. Experiment 2 arranged 3-component multiple VI schedules where components differed in rate or number of reinforcers. Resistance to extinction depended on the training reinforcer rate but not on the number of reinforcers omitted. Experiment 3 replicated the partial-reinforcement extinction effect within subjects in a discrete-trial procedure and found that more reinforcers were omitted in continuous- than in partial-reinforcement trials. A model of extinction based on behavioral momentum theory accounted for all the data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of acquisition training schedule on extinction and reinstatement of cocaine self-administration in male rats.

Partial reinforcement is known to increase resistance to extinction (Rn) relative to training with continuous reinforcement. This phenomenon, referred to as the partial reinforcement extinction effect, is one of the most robust in learning and conditioning studies. Experiment 1 investigated manipulations known to affect the partial reinforcement extinction effect and determined their possible relevance for drug use patterns. Male rats received intravenous cocaine self-administration training under partial reinforcement (FR-10) training or continuous reinforcement (FR-1) conditions with either a low (0.25 mg/kg infusion) or a high cocaine dose (1.00 mg/kg infusion). Animals were placed on an extinction (recurrent nonreward) schedule for 10 days (1-hr sessions) prior to being tested for cue-induced reinstatement (single 2-hr session). Experiment 2 involved acquisition of cocaine self-administration under FR-1 conditions of short training (15 days) or extended training (30 days) with a low dose (0.25 mg/kg infusion) or a medium dose (0.50 mg/kg infusion) of cocaine reward prior to extinction or reinstatement. Experiment 1 showed that rats trained with FR-10-high dose outcomes exhibited greater Rn than the remaining groups. Additionally, FR-10-high dose and FR-10-low dose rats were more likely to return to active drug seeking during the reinstatement test. In Experiment 2, rats trained under FR-1-medium dose conditions were more persistent during extinction following short acquisition training than comparable rats experiencing extended acquisition training. The reinstatement test was conducted following extinction, in which it was observed that overtraining under FR-1-medium dose reward schedules resulted in a decrease in the tendency to return to active drug seeking. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Importance of trials versus accumulating time across trials in partially reinforced appetitive conditioning.

Four experiments with rats examined partial reinforcement in appetitive conditioning. In Experiment 1, adding nonreinforced trials to a continuous reinforcement schedule slowed acquisition, whereas deleting reinforcers did not. Trial massing suppressed performance and learning. In Experiment 2, conditioning with a short conditioned stimulus (CS) was rapid, and partial reinforcement with a short CS was as effective as continuous reinforcement with equal accumulated time in the CS. In Experiment 3, conditioning was nevertheless influenced by the probability of reinforcement. In Experiments 3 and 4, conditioning was especially disrupted when nonreinforced trials preceded reinforced trials closely in time. The results underscore the importance of temporal variables in conditioning but are more consistent with trial-based accounts than time-accumulation accounts of conditioning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Skinner's early research: From reflexology to operant conditioning.

The facts of Skinner's research in the 1930s on the acquisition of operant behavior are combined with his own later comments. Skinner discovered that a single reinforcement is enough for conditioning of an arbitrary response. The combination of successive extinction curves after single reinforcements within 1 session led to the 1st schedule of intermittent reinforcement. Operant conditioning also could be arranged to generate new forms of behavior by shaping by successive approximation. Skinner was first influenced by the then dominant terminology of reflexology, but he soon rejected this stimulus–response tradition by demonstrating that eliciting stimuli play no role in operant conditioning. Theoretical implications of Skinner's early research are compared and contrasted with other theories of conditioning at the time. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential outcomes enhance accuracy of delayed matching to sample but not resistance to change.

Three experiments assessed the relation between the differential outcomes effect and resistance to change of delayed matching-to-sample performance. Pigeons produced delayed matching-to-sample trials by responding on variable interval schedules in two components of a multiple schedule. In the same-outcome component, the probability of reinforcement was the same for both samples (.9 in Experiments 1 and 2, .5 in Experiment 3); in the different-outcomes component, the probability of reinforcement was .9 for one sample and .1 for the other. In all three experiments, the forgetting functions in the different-outcomes component were higher and shallower than in the same-outcomes component. When total reinforcement was greater in the same-outcomes component (Experiments 1 and 2), resistance to disruption by prefeeding, intercomponent food, extinction, or flashing lights typically was greater in that component. In Experiment 3, when total reinforcement was equated, resistance to disruption was similar across components. Thus, the level and slope of forgetting functions depended on differential reinforcement correlated with the samples, but the resistance to change of forgetting functions depended on total reinforcement in a component. Both aspects of the results can be explained by a model of delayed matching to sample performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Extinction and reinforcement of alternative behavior.

Examined the course of extinction of 1 behavior as a function of the presence and subsequent absence of reinforcement for alternative behavior. 4 experiments were conducted, using a total of 102 male hooded rats in 3 experiments and 30 adult White Carneaux pigeons in the 4th. Major findings are: (a) Topography and reinforcement schedule for alternative behavior had little differential effect. (b) High-frequency reinforcement of alternative behavior increased suppressive effects of extinction, and low-frequency reinforcement did not. (c) Reinforced alternative behavior maintained for a relatively long period decreased subsequent recovery of the original response programed for extinction, resulting in a substantial saving in total number of extinction responses. (d) Temporary reinforcement of alternative behavior produced more permanent suppression in the context of simple extinction than in the context of SD (change in stimulus) periods in discrimination learning. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The delay-reduction hypothesis: Extension to three-alternative choice.

Choice was assessed for 2 and 3 VI schedules arranged in the terminal links of concurrent-chains schedules (Exps I–IV) and simple concurrent schedules (Exp IV). Ss were 14 male White King pigeons. When the 3rd VI schedule provided a relatively high rate of reinforcement, its addition resulted in a divergence of the distribution of responses between the 2 other schedules (Exps I and II). When the 3rd VI schedule provided a sufficiently low rate of reinforcement, its addition resulted in a convergence of the distribution of responses between the other schedules. Both sets of results are consistent with the delay-reduction hypothesis but not the constant-ratio rule. In Exp IV, the 3rd VI schedule was selected so that the delay-reduction hypothesis required no change in the distribution of responses between the other 2 schedules. This experiment also assessed the effects of adding a 3rd VI schedule in a simple concurrent schedule. In both cases the distribution of responses between the other 2 schedules was not affected systematically. Results support an extension of the delay-reduction hypothesis to 3-alternative choice. (35 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)