Timing and the control of variation.

Two rat experiments shed light on how variation in behavior is regulated. Experiment 1 used the peak procedure. On most trials, the 1st bar press more than 40 s after signal onset ended the signal and produced food. Other trials lasted much longer and ended without food. On those trials, the variability of bar-press duration increased greatly after the 1st response more than 40 sec after signal onset. In Experiment 2, which asked whether the increase was due to the omission of expected reward or the decrease in reward expectation, reward expectation had a strong effect on response duration, whereas omission of expected reward had little effect. In both experiments, response rate and response duration changed independently, suggesting that they reflect different parts of the underlying mechanism. In Experiment 1, response durations implied that timing of the signal was more accurate than the rate-vs-time function might suggest. Experiment 2 suggested that lowering reward expectation increases variation in response form. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Importance of trials versus accumulating time across trials in partially reinforced appetitive conditioning.

Four experiments with rats examined partial reinforcement in appetitive conditioning. In Experiment 1, adding nonreinforced trials to a continuous reinforcement schedule slowed acquisition, whereas deleting reinforcers did not. Trial massing suppressed performance and learning. In Experiment 2, conditioning with a short conditioned stimulus (CS) was rapid, and partial reinforcement with a short CS was as effective as continuous reinforcement with equal accumulated time in the CS. In Experiment 3, conditioning was nevertheless influenced by the probability of reinforcement. In Experiments 3 and 4, conditioning was especially disrupted when nonreinforced trials preceded reinforced trials closely in time. The results underscore the importance of temporal variables in conditioning but are more consistent with trial-based accounts than time-accumulation accounts of conditioning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Summation of symbols by pigeons (Columba livia): The importance of number and mass of reward items.

In each of 3 experiments, different sets of 4 pigeons (Columba livia) were trained to discriminate between 2 visual symbols that covered wells containing food items that varied in number, mass, or both. In Experiment 1, the symbols were associated with 0, 1, 3, 5, 7, or 9 pieces of grain reward. The pigeons learned to choose the symbol corresponding to the larger reward, and on summation tests, they chose the pair of symbols that summed to the larger total reward. When number of food pellets was varied but mass of reward was held constant in Experiment 2, preference for the larger number symbols failed to appear. When number was held constant and mass was varied in Experiment 3, the pigeons showed a clear preference for the larger mass symbols on single-symbol and summation tests. These findings show that pigeons summate the value of symbols and are more likely to represent symbols by mass of food reward than by number of food items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of chlordiazepoxide, buspirone and cocaine on behavior suppressed by timeout presentation

Male Wistar rats were exposed to a two-component multiple schedule: a random-interval 30 s schedule of pellet presentation and a conjoint random-interval 30 s schedule of pellet presentation, random-interval 2 s schedule of timeout 10 s presentation. Once responding had stabilized subjects were injected intraperitoneally with vehicle, chlordiazepoxide (1-30 mg/kg), buspirone (0.1-4.2 mg/kg) or cocaine (1-30 mg/kg), 15 min before the start of the experimental session. Before drug administration, punished response rates were less than 30% of unpunished response rates for four of the six subjects, and 60% and 75% for the other two. Low doses of chlordiazepoxide (1 and 3 mg/kg) increased punished responding (range 25-300%), and slightly increased unpunished response rates (by 25% in all but one subject, whose rates increased by 75%). The higher doses of chlordiazepoxide (10-30 mg/kg) dose-dependently decreased response rates in both components. The lower doses of buspirone (0.1 and 0.3 mg/kg) either did not affect, or decreased response rates in both components of the schedule; the higher doses produced dose-dependent decreases. Low doses of cocaine (1, 3 and 5.6 mg/kg) did not affect response rates in either component of the multiple schedule, whereas higher doses produced a dose-dependent decrease in response rates, except for one subject whose punished response rates increased substantially. The behavioral effects of chlordiazepoxide and buspirone observed in the present experiment were similar to those observed in experiments in which response rates were suppressed by shock presentation.     

Changes in medial prefrontal cortical dopamine levels associated with response-contingent food reward: an electrochemical study in rat

Voltammetry was used to monitor in rats changes in medial prefrontal cortex (PFC) dopamine (DA) levels associated with response-contingent presentation of a condensed milk reward. During two initial training sessions, minor DA signal fluctuations were seen when animals consumed a standard 30 sec (0.2 ml) meal earned on a continuous reinforcement schedule. There was no evidence of experience-dependent changes in these fluctuations. Under delayed reinforcement conditions, lever-presses were followed by DA signal increases that were time-locked to the delay duration, and these were followed by signal decreases when animals eventually received the reward. Such decreases became more pronounced when the standard rate of milk delivery was tripled, but were attenuated when milk delivery was reduced to half the usual rate. Withholding earned milk resulted in signal increases. In contrast, DA signal increases were observed during milk consumption when the standard meal duration was unexpectedly shortened to 15 sec or lengthened to 60 or 90 sec. Orderly changes in DA signal were also observed under partial reinforcement conditions. Unreinforced responses were associated with DA signal decreases, whereas transient increases were seen during the 30 sec meal that followed reinforced responses. These findings indicate that response-contingent reward presentation elicits synchronous changes in PFC DA transmission. They also suggest that the DA input to PFC is activated when rewards are presented under conditions that deviate from those that the animals had come to expect, particularly so when the temporal structure of learned associations is altered.     

The effects of acquisition training schedule on extinction and reinstatement of cocaine self-administration in male rats.

Partial reinforcement is known to increase resistance to extinction (Rn) relative to training with continuous reinforcement. This phenomenon, referred to as the partial reinforcement extinction effect, is one of the most robust in learning and conditioning studies. Experiment 1 investigated manipulations known to affect the partial reinforcement extinction effect and determined their possible relevance for drug use patterns. Male rats received intravenous cocaine self-administration training under partial reinforcement (FR-10) training or continuous reinforcement (FR-1) conditions with either a low (0.25 mg/kg infusion) or a high cocaine dose (1.00 mg/kg infusion). Animals were placed on an extinction (recurrent nonreward) schedule for 10 days (1-hr sessions) prior to being tested for cue-induced reinstatement (single 2-hr session). Experiment 2 involved acquisition of cocaine self-administration under FR-1 conditions of short training (15 days) or extended training (30 days) with a low dose (0.25 mg/kg infusion) or a medium dose (0.50 mg/kg infusion) of cocaine reward prior to extinction or reinstatement. Experiment 1 showed that rats trained with FR-10-high dose outcomes exhibited greater Rn than the remaining groups. Additionally, FR-10-high dose and FR-10-low dose rats were more likely to return to active drug seeking during the reinstatement test. In Experiment 2, rats trained under FR-1-medium dose conditions were more persistent during extinction following short acquisition training than comparable rats experiencing extended acquisition training. The reinstatement test was conducted following extinction, in which it was observed that overtraining under FR-1-medium dose reward schedules resulted in a decrease in the tendency to return to active drug seeking. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A random-walk model of accuracy and reaction time applied to three experiments on pigeon visual discrimination.

A random-walk model of visual discrimination is described and applied to reaction time (RT) distributions from three discrete-trial experiments with pigeons. Experiment 1 was a two-choice hue discrimination task with multiple hues. Choice percentages changed with hue discriminability; RTs were shortest for the least and most discriminable stimuli. Experiments 2 and 3 used go/no-go hue discriminations. Blocks of sessions differed in reward probability associated with a variable red stimulus in Experiment 2 and with a constant green stimulus in Experiment 3. Changes in hue had a large effect on response percentage and a small effect on RT; changes in reward shifted RT distributions on the time axis. The “random-walk, pigeon” model applied to these data is closely related to Ratcliff's diffusion model (Ratcliff, 1978; Ratcliff & Rouder, 1998). Simulations showed that stimulus discriminability affected the speed with which evidence accumulated toward a response threshold, in line with comparable effects in human subjects. Reward probability affected bias, modeled as the amount of evidence needed to reach one threshold rather than the other. The effects of reward probability are novel, and their isolation from stimulus effects within the decision process can guide development of a broader model of discrimination. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Degree of representation of the matching concept in pigeons (Columba livia).

In Experiment 1, 12 pigeons (Columba livia) were trained on a simultaneous matching-to-sample task with 2 stimuli and then tested with 2 novel stimuli. Half of the birds were trained with a fixed ratio schedule requirement of 1 (FR1) or 20 (FR20) pecks on the sample stimulus. None of the birds showed any evidence of concept-mediated transfer. In Experiment 2, 12 pigeons were trained with 3 stimuli and then tested with the same novel stimuli used in Experiment 1. Half of the birds in each group were trained with either an FR1 or FR20 requirement on the sample stimulus. Two of the FR20 birds showed high levels of transfer to the novel stimuli similar to that of monkeys in a previous study. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Duration of signals for intertrial reinforcement and nonreinforcement in random control procedures.

In the random control procedure, responding to a conditioned stimulus (target CS) is prevented when the probability of unsignaled, unconditioned stimuli (UCS) in the intertrial interval (ITI) is equal to the probability of the UCS in the presence of the target CS. Three experiments used an autoshaping procedure with White Carneaux pigeons to examine the effects of the temporal duration of signals for the ITI UCS (cover CSs) and for concomitant periods of nonreinforcement. In Experiment 1, a short duration cover, but not a long duration cover, resulted in responding to the target CS. In Experiment 2, an explicit CS– cue during periods of nonreinforcement did not affect target acquisition. In Experiment 3, a long CS–, but not a short cover CS, was a sufficient condition for the acquisition of responding to the target CS. These results imply that the acquisition of responding to a target CS requires a discriminable period of nonreinforcement that is long relative to the target CS duration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of context on responding during a compound stimulus.

In 4 experiments, the authors used rats to examine the strength of responding during a clicker-tone compound in the presence of a light, after the auditory stimuli had individually been paired with food in the presence of the same light. Experiment 1 demonstrated a higher rate of responding during the compound when the duration of the light was short rather than long. In Experiments 2, 3, and 4, the long duration light was used as a signal for food in a conditional discrimination involving the tone and the clicker. Responding on test trials with the clicker-tone compound during the light was enhanced by this treatment and resulted in a level of performance that was no different from that observed when the duration of the light was short. The results are more compatible with a configural than an elemental theory of associative learning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Progressive ratio performance in rats with gustatory thalamus lesions.

To determine whether damage of the gustatory thalamus (the parvicellular region of the ventroposteromedial nucleus) disrupts the perceived value of reinforcing stimuli, the performance of rats with bilateral, electrophysiologically guided, electrolytic lesions of the area was examined in Experiment 1 with a progressive-ratio schedule of reinforcement. Lesioned rats showed normal concentration-dependent changes in break point (an index of the amount of effort a subject will expend to obtain reinforcement) and in consummatory responding (licking) for the sucrose reward. In the reward comparison procedure of Experiment 2, however, the same lesioned rats failed to show morphine-induced suppression of alanine consumption. The results provide no support for the view that the thalamic taste area is involved in the perception of the absolute reinforcing value of gustatory stimuli. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of pramipexole on impulsive choice in male wistar rats.

Clinical reports, primarily with Parkinson's disease patients, note an association between the prescribed use of pramipexole (and other direct-acting dopamine agonist medications) and impulse control disorders, particularly pathological gambling. Two experiments examined the effects of acute pramipexole on rats' impulsive choices where impulsivity was defined as selecting a smaller–sooner over a larger–later food reward. In Experiment 1, pramipexole (0.1 to 0.3 mg/kg) significantly increased impulsive choices in a condition in which few impulsive choices were made during a stable baseline. In a control condition, in which impulsive choices predominated during baseline, pramipexole did not significantly change the same rats' choices. Experiment 2 explored a wider range of doses (0.01 to 0.3 mg/kg) using a choice procedure in which delays to the larger–later reinforcer delivery increased across trial blocks within each session. At the doses used in Experiment 1, pramipexole shifted choice toward indifference regardless of the operative delay. At lower doses of pramipexole (0.01 and 0.03 mg/kg), a trend toward more impulsive choice was observed at the 0.03 mg/kg dose. The difference in outcomes across experiments may be due to the more complex discriminations required in Experiment 2, that is, multiple discriminations between changing delays within each session. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Schedule-induced kinesic and taxic behavioral stereotypy in the pigeon.

Two experiments explored the functional character of 2 schedule-induced interim behaviors (pacing and retreat) and 1 terminal behavior (keypecking) that developed on fixed-time (FT) schedules of food delivery with a keylight that increased in brightness throughout the interstimulus interval. In Experiment 1, this ramp procedure was compared with a less discriminable ramp, and FT and Random Time (RT) procedures without a signal. Decreased discriminability expanded keypecking in the trial. The FT schedule eliminated only keypecking and the RT procedure eliminated keypecking and retreat while pacing remained. In Experiment 2, predictive and unpredictive ramps were added to the RT procedure. The data suggest that schedule-induced stereotypy can be divided into kinesic stereotypy (pacing), which arises from repeated reinforcer presentations, and taxic stereotypy, which is tied to an increase (keypecking) or decrease (retreat) in the momentary probability of reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex differences in the relationship between activity and weight loss in the rat.

Access to a running wheel combined with restricted feeding produced body weight loss at an equivalent rate in male and female litter-mate rats (Experiment 1). Thus, despite weighing less and running more, females were not more vulnerable to this procedure. When factors influencing weight loss were varied, no sex difference was found in adaptation to a new feeding schedule or in the effect of single versus group housing (Experiment 2). The apparent critical difference was that body weight loss increased running in males but not in females (Experiment 3). In all rats, rapid recovery of body weight occurred when food access was no longer restricted (Experiment 1), suggesting that "activity-based anorexia" is a misnomer for weight loss by rats in a running wheel. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of interresponse time reinforcement on signalled-reward effect.

Examined the conditions under which presenting a signal for reinforcement decreases or increases the rate of leverpressing in rats. Response rate on a variable-interval (VI) schedule of reinforcement was decreased by a brief signal for reinforcement. In Experiment 1, requiring 1 short IRT to occur on completion of the VI requirement attenuated this response decrement. Requiring three responses to be emitted within a minimum amount of time at the end of the VI resulted in the reinforcement signal's enhancement of response rates. Experiment 2 replicated this increase in responding and showed that the reinforcement signal attenuated response rates on a simple VI schedule that yielded the same overall rate of reinforcement. In Experiment 3, the reinforcement signal enhanced response rates when 3 responses were required at the end of a VI schedule, but the signal attenuated response rates when the 3 responses could occur at any time in relation to the VI. These results suggest that the pattern of responding emitted immediately prior to reinforcement is a critical factor in determining the effect of reinforcement signal on response rate. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fixed-ratio schedules increase generalized self-control: Preference for large rewards despite high effort or punishment.

Three experiments investigated the effects of FR reinforcement on generalized self-control involving high effort and punishment. In Experiment 1, rats received food in a runway for the completion of each round trip (continuous-reinforcement group) or every fifth round trip (FR group). Control rats received food at the same temporal intervals as these groups but without any instrumental requirement. When all rats were next given a series of choices between a large food reward requiring high lever force versus a small reward requiring low lever force, the FR rats showed the greatest self-control. In Experiments 2 and 3, rats were rewarded on a continuous or FR schedule followed by choice between a large food reward accompanied by intermittent shock vs a small or absent food reward without shock. The FR rats again showed the greatest self-control. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pigeons (Columba livia) associate time intervals with symbols in a touch screen task: Evidence for ordinality but not summation.

The present experiments examined whether pigeons can sum symbols that are associated with various temporal consequences in a touch screen apparatus. Pigeons were trained to discriminate between two visual symbols that were associated with 0, 1, 2, 3, 4, or 5 s either of delay to 4 s of hopper access (delay group) or duration of hopper access (reward group). In Experiment 1, the pigeons in both groups learned to select the symbol associated with the more favorable outcome, and they successfully transferred this discrimination to novel symbol pairs. However, when tested with 2 pairs of symbols associated with different summed durations, they responded on the basis of a simple response rule rather than the sum of the symbol pair. In Experiment 2, the reward group was presented with four symbols at once and was allowed to successively choose one symbol at a time. All pigeons chose the symbols in order from largest to smallest. This indicates that pigeons formed an ordered representation of symbols associated with different time intervals, even though they did not sum the symbols. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choice in the time-left procedure and in concurrent chains with a time-left terminal link

In two experiments, rats chose between a standard fixed-duration food-associated stimulus and a stimulus whose duration was the time remaining to reinforcement in an elapsing comparison interval. In Experiment 1, 4 rats responded in a time-left procedure wherein a single initial-link variable-interval schedule set up two potential terminal links simultaneously. As time elapsed in the initial-link schedule, the choice was between a standard fixed-interval 30-s terminal link and a time-left terminal link whose programmed interval requirement equaled 90 s minus the elapsed time in the initial link. Rats generally responded more on the lever with the shortest programmed terminal-link duration, but the temporal parameters of the procedure were found to vary with response distributions. Contrary to previous reports, therefore, time-left data were well predicted by choice models that make no assumptions about animal timing. In Experiment 2, 8 rats responded on a concurrent-chains schedule with independent variable-interval initial links and a time-left terminal link in one of the choice schedules. On the time-left lever, the programmed terminal-link delay equaled 90 s minus the elapsed time in the time-left initial link. On the standard lever, terminal-link responses were reinforced according to a variable-interval schedule whose average value varied over four conditions. Relative time-left initial-link responses increased in the elapsing time-left initial-link schedule as the time-left terminal link became shorter relative to the standard terminal link. Scalar expectancy theory failed to predict the resultant data, but a modified version of the delay-reduction model made good predictions. An analysis of the elaboration of scalar expectancy theory for variable delays demonstrated that the model is poorly formulated for arithmetically distributed delays.     

Attenuation and enhancement of instrumental responding by signals for reinforcement on a variable interval schedule.

In three experiments we used pigeons responding on a variable interval schedule to examine the effects of signaling reinforcement on instrumental performance. Using a four-component multiple schedule, we found in Experiment 1 that when reinforcement was delayed by 0.5 s for two of the components, response rates were attenuated by signaling reinforcement relative to performance in the unsignaled delay condition. A delay of 3 s in the remaining two components resulted in an enhancement of responding when the delay was signaled. An enhancement of responding occurred when we used a scheduled 0.5-s delay when the procedure ensured that responding was prevented during the delay interval by extinguishing the keylight (Experiment 2) or by resetting the delay timer when responses were emitted during the delay (Experiment 3). These results are discussed with reference to the theories of signaling reinforcement during instrumental responding. (PsycINFO Database Record (c) 2010 APA, all rights reserved)