Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell.

The authors investigated the contribution of the nucleus accumbens (NAc) core and shell to effort-based decision making using a discounting procedure. Selection of 1 lever delivered a smaller, 2-pellet reward immediately, whereas the other lever delivered a 4-pellet reward after a fixed ratio of presses (2, 5, 10, or 20) that increased over 4 blocks of 10 discrete choice trials. Subsequent testing employed an equivalent delays procedure, whereby the relative delay to reward delivery after selection of either option was equalized. In well-trained rats, inactivation of the core, but not the shell, via infusion of GABA A/B agonists muscimol/baclofen reduced preference for the high-effort option under standard conditions and also when rats were tested using an equivalent delays procedure. However, inactivation of the core did not alter preference for 4-pellet versus 2-pellet rewards when the relative costs of each option were the same (1 press). Thus, the NAc core, but not the shell, appears to be part of a neural circuit that biases choice toward larger rewards associated with a greater effort cost. Furthermore, the contributions by the NAc core to this form of decision making can be dissociated from its role in delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Discriminative cues indicating reward magnitude continue to determine reaction time of rats following lesions of the nucleus accumbens

The role of the nucleus accumbens in incentive motivation is accepted but poorly understood. In this study, we examined in the rat one aspect of motivated behaviour which might be mediated by the nucleus accumbens, namely the translation of a motivational signal (the expected value of a reward) into motor output (responding for the reward). Rats were trained in a reaction time task in which on each trial they received one, two or three pellets. The number of pellets for each trial was randomly determined in advance and signalled to the rats by cue lights. Rats responded with faster reaction times as the size of the expected reward increased. Following ibotenic acid lesions of the nucleus accumbens, there was no difference in the pattern or the speed of reaction times. Although lesions of the nucleus accumbens did not disconnect the motivational system from the motor system, it is possible that the nucleus accumbens is involved in the learning of the incentive salience of external stimuli. Therefore, after postoperative testing the cue contingencies were reversed. Initially, the cues continued to be interpreted according to their prior significance, but eventually both the lesioned rats and the control group acquired the new relationship and did so in equivalent times. We conclude that the nucleus accumbens is not involved in the acquisition or expression of the processes whereby the expectation of rewards of different value is translated into a motor initiation signal.     

Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats.

The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) are important neural regions in responding adaptively to changes in the incentive value of reward. Recent evidence suggests these structures may be differentially engaged in effort and cue-guided choice behavior. In 2 T-maze experiments, we examined the effects of bilateral lesions of either BLA or OFC on (1) effortful choices in which rats could climb a barrier for a high reward or select a low reward with no effort and (2) effortful choices when a visual cue signaled changes in reward magnitude. In both experiments, BLA rats displayed transient work aversion, choosing the effortless low reward option. OFC rats were work averse only in the no cue conditions, displaying a pattern of attenuated recovery from the cue conditions signaling reward unavailability in the effortful arm. Control measures rule out an inability to discriminate the cue in either lesion group. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Predictable and unpredictable rewards produce similar changes in dopamine tone.

Unpredicted rewards trigger more vigorous phasic responses in midbrain dopamine (DA) neurons than predicted rewards. However, recent evidence suggests that reward predictability may fail to influence DA signaling over longer scales: In rats passively receiving rewarding electrical brain stimulation, the concentration of DA in dialysate obtained from nucleus accumbens probes was similar regardless of whether reward onset was predictable (G. Hernandez et al., 2006). The present experiment followed up on these findings by requiring the rats to work for the rewarding stimulation, thus confirming whether they indeed learned the timing and predictability of reward delivery. Performance under fixed-interval and variable-interval schedules was compared, and DA levels in the nucleus accumbens were measured by means of in vivo microdialysis. The observed patterns of operant responding indicate that the rats working under the fixed-interval schedule learned to predict the time of reward availability, whereas the rats working under the variable-interval schedule did not. Nonetheless, indistinguishable changes in DA concentration were observed in the 2 groups. Thus, reward predictability had no discernable effect on a measure believed to track the slower components of DA signaling. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of varied preshift reward magnitude on successive negative contrast effects in rats.

A total of 110 male Sprague-Dawley albino rats, distributed across 3 experiments, received simple instrumental conditioning trials in a straight runway. In each experiment the conditions of reward prior to a shift to small reward were varied between groups. Collectively, results indicate that the extent of the negative contrast effect depends upon the difference between pre- and postshift incentive levels and that Ss exposed to varied reward magnitude training average the incentive values of these rewards. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal Discounting When the Choice Is Between Two Delayed Rewards.

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?

What roles do mesolimbic and neostriatal dopamine systems play in reward? Do they mediate the hedonic impact of rewarding stimuli? Do they mediate hedonic reward learning and associative prediction? Our review of the literature, together with results of a new study of residual reward capacity after dopamine depletion, indicates the answer to both questions is 'no'. Rather, dopamine systems may mediate the incentive salience of rewards, modulating their motivational value in a manner separable from hedonia and reward learning. In a study of the consequences of dopamine loss, rats were depleted of dopamine in the nucleus accumbens and neostriatum by up to 99% using 6-hydroxydopamine. In a series of experiments, we applied the 'taste reactivity' measure of affective reactions (gapes, etc.) to assess the capacity of dopamine-depleted rats for: 1) normal affect (hedonic and aversive reactions), 2) modulation of hedonic affect by associative learning (taste aversion conditioning), and 3) hedonic enhancement of affect by non-dopaminergic pharmacological manipulation of palatability (benzodiazepine administration). We found normal hedonic reaction patterns to sucrose vs. quinine, normal learning of new hedonic stimulus values (a change in palatability based on predictive relations), and normal pharmacological hedonic enhancement of palatability. We discuss these results in the context of hypotheses and data concerning the role of dopamine in reward. We review neurochemical, electrophysiological, and other behavioral evidence. We conclude that dopamine systems are not needed either to mediate the hedonic pleasure of reinforcers or to mediate predictive associations involved in hedonic reward learning. We conclude instead that dopamine may be more important to incentive salience attributions to the neural representations of reward-related stimuli. Incentive salience, we suggest, is a distinct component of motivation and reward. In other words, dopamine systems are necessary for 'wanting' incentives, but not for 'liking' them or for learning new 'likes' and 'dislikes'.     

Temporal horizons in decision making.

The perception of duration is crucial when we make choices between immediate and delayed rewards. Immediate rewards are valued more than the same rewards if they are delayed. Preferences for earlier rewards become even stronger when the reward can be received within a subjectively proximate time frame. A “rational” decision to wait for a delayed reward may be overruled by an “impulsive” choice when the option for the closer reward falls within this proximate time range. Based on findings on circadian and circannual physiological rhythms, we suggest that there are 2 time units that are both biologically and culturally determined and have an impact on human experience and behavior: the day and the year. We highlight results of a neuroimaging study showing that rewards with delays up to 1 year are discounted differently than reward delays longer than 1 year. This duration-dependent discounting is associated with specific brain activation in the striatum. We present various conceptualizations of subjective time incorporated in parametric models of intertemporal decisions that may lead to a better understanding of human choice behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cocaine exposure causes long-term increases in impulsive choice.

In this study, the authors examined the long-term effects of prior exposure to cocaine on a delay-discounting task commonly used to measure impulsive choice. Male Long-Evans rats received daily intraperitoneal injections of 30 mg/kg cocaine HCl or saline for 14 days. Following 3 weeks of withdrawal, rats began training. On each trial, rats were given a choice between 2 levers. A press on 1 lever resulted in immediate delivery of a single 45-mg food pellet, and a press on the other resulted in delivery of 4 pellets after a delay period. Impulsive choice was defined as preference for the small immediate over the large delayed reward. Three months after treatment, cocaine-exposed rats displayed increased impulsive choice behavior. They also showed less anticipatory responding (entries into the food trough) during the delays prior to reward delivery, indicating that the enhanced impulsive choice in these rats may be related to deficits in bridging the delay between response and reward. These data demonstrate that cocaine exposure can cause enduring increases in impulsive choice behavior, consistent with observations in human subjects with drug addictions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Dopamine, effort, and decision making: Theoretical comment on Bardgett et al. (2009).

Forebrain dopamine (DA) systems are thought to be a critical component of the brain circuitry regulating behavioral activation, work output during instrumental behavior, and effort-related decision making. Tasks that offer animals choices between alternatives that require different degrees of effort can be used to assess effort-related choice behavior. Rats treated with DA antagonists, or with accumbens DA depletions, tend to show reduced selection of instrumental behaviors with high response requirements, and instead they choose to engage in food-seeking behaviors that involve less effort. The accompanying article by Bardgett et al. (see record 2009-04037-002) describes a novel effort-discounting task that involves the modification of a previously developed T-maze choice procedure (Salamone et al., 1994). Each arm of the maze contained different magnitudes of food reinforcement, and in order to obtain the higher magnitude reward, the rats had to climb a barrier in that arm of the maze. With training, rats were able to climb successively higher barriers to obtain the larger amount of food, and the choice between the high barrier arm and the no-barrier arm with the smaller reward served as a template for assessing the effects of dopaminergic drugs. D1 and D2 family antagonists, as well as the DA releasing agent amphetamine, were able to produce a bidirectional modulation of choice behavior, while drugs that act on D3 receptors were ineffective. These studies illustrate features of the neurochemical regulation of effort-related decision making, and may have implications for the understanding of both natural and pathological features of motivation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Muscarinic receptor antagonism of the nucleus accumbens core causes avoidance to flavor and spatial cues.

Pharmacological blockade of muscarinic receptors in the nucleus accumbens reduces food intake and instrumental behaviors that are reinforced by food delivery. Nucleus accumbens muscarinic antagonism may specifically suppress the hedonic or reinforcing effects of food, thus blocking its capacity to direct behavior. Alternatively, muscarinic receptor blockade may cause a negative hedonic state that interferes with appetitive learning and food intake. In these experiments, rats received infusions of scopolamine methyl bromide (10 μg/0.5 μl) into the nucleus accumbens core, following exposure to a novel flavor of liquid diet (Experiment 1) or prior to being placed into a place preference apparatus (Experiment 2). In both experiments, nucleus accumbens muscarinic receptor antagonism caused subsequent avoidance of the paired cue (flavor or spatial location). This effect was specific to cholinergic manipulation; no conditioned taste avoidance was observed after pairing the novel flavor with nucleus accumbens core antagonism of N-methyl-D-aspartate, dopamine D?, or opioid receptors (Experiment 3). These experiments confirm previous reports of a critical role for striatal acetylcholine in modulating goal-directed behaviors, but suggest caution when interpreting behavioral effects of pharmacological manipulation of striatal acetylcholine. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Nucleus accumbens opioid, GABAergic, and dopaminergic modulation of palatable food motivation: Contrasting effects revealed by a progressive ratio study in the rat.

The current studies were designed to evaluate whether incentive motivation for palatable food is altered after manipulations of opioid, GABAergic, and dopaminergic transmission within the nucleus accumbens. A progressive ratio schedule was used to measure lever-pressing for sugar pellets after microinfusion of drugs into the nucleus accumbens in non-food-deprived rats. The mu opioid agonist D-Ala2, NMe-Phe4, Glyol5-enkephalin and the indirect dopamine agonist amphetamine induced a marked increase in break point and correct lever-presses; the GABAA agonist muscimol did not affect break point or lever-presses. The data suggest that opioid, dopaminergic, and GABAergic systems within the accumbens differentially modulate food-seeking behavior through mechanisms related to hedonic evaluation of food, incentive salience, and control of motor feeding circuits, respectively. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of antidepressant drugs on dopamine D1 and D2 receptor expression and dopamine release in the nucleus accumbens of the rat

This study examined the effect of repeated treatment with the antidepressant drugs, fluoxetine, desipramine and tranylcypromine, on dopamine receptor expression (mRNA and binding site density) in sub-regions of the nucleus accumbens and striatum of the rat. The effect of these treatments on extracellular levels of dopamine in the nucleus accumbens was also measured. Experiments using in situ hybridisation showed that the antidepressants caused a region-specific increase in D2 mRNA, this effect being most prominent in the nucleus accumbens shell. In contrast, none of the treatments increased D1 mRNA in any of the regions examined. Measurement of D2-like binding by receptor autoradiography, using the ligand [3H]YM-09151-2, revealed that both fluoxetine and desipramine increased D2-like binding in the nucleus accumbens shell; fluoxetine had a similar effect in the nucleus accumbens core. Tranylcypromine, however, had no effect on D2-like binding in the nucleus accumbens but decreased binding in the striatum. In micro-dialysis experiments, our data showed that levels of extracellular dopamine in the nucleus accumbens were not altered in rats treated with either fluoxetine or desipramine, but increased by tranylcypromine. From our findings, we propose that the antidepressant drugs tested enhance dopamine function in the nucleus accumbens through either increased expression of post-synaptic D2 receptors (fluoxetine and desipramine) or increased dopamine release (tranylcypromine).     

Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively.

For rats that bar pressed for intracranial electrical stimulation in a 2-lever matching paradigm with concurrent variable interval schedules of reward, the authors found that the time allocation ratio is based on a multiplicative combination of the ratio of subjective reward magnitudes and the ratio of the rates of reward. Multiplicative combining was observed in a range covering approximately 2 orders of magnitude in the ratio of the rates of reward (from about 1:10 to 10:1) and an order of magnitude change in the size of rewards. After determining the relation between the pulse frequency of stimulation and subjective reward magnitude, the authors were able to predict from knowledge of the subjective magnitudes of the rewards and the obtained relative rates of reward the subject's time allocation ratio over a range in which it varied by more than 3 orders of magnitude. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Functional neuroimaging of intertemporal choice models: A review.

People often forsake a larger reward later for a smaller reward sooner. The process of devaluing the larger, later prize is called temporal discounting or delay discounting, which lies at the core of intertemporal choice. Here, we describe the methodology and findings of research on the mechanisms of intertemporal choice, with a focus on those that utilize functional MRI (fMRI). We consider the neural bases for the most common economic models of intertemporal choice and examine whether these models require neural processes that are common or distinct across types of decision making. Considered as a whole, current research points to potentially distinct contributions from brain systems associated with valuation and with prospective thought, which may be reflected in separable foci in posterior cingulate cortex. Based on open questions in the field, we suggest two core goals for future research: identifying aspects of valuation that are unique to intertemporal choice and evaluating direct or indirect interactions between delay and prize magnitude. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential involvement of the basolateral amygdala, orbitofrontal cortex, and nucleus accumbens core in the acquisition and use of reward expectancies.

In this study, the authors tested the hypothesis that the basolateral amygdala (BLA), orbitofrontal cortex (OFC), nucleus accumbens core (NA-core), and the extended hippocampus mediate different aspects of the development-maintenance of unique reward expectancies produced by the differential outcomes procedure (DOP). Rats were trained with either DOP or a nondifferential outcomes procedure (NOP) on a simple discrimination task. Fornix lesions did not affect either version of the task, demonstrating that the extended hippocampal system has no role in stimulus-outcome (S-O) associations. In contrast, in the DOP condition, BLA lesions impaired performance throughout training, OFC lesions impaired choice accuracy only in the later maintenance phase, and NA-core lesions resulted in enhanced learning. These results suggest that BLA and OFC are important for establishment (BLA) and behavioral maintenance (OFC) of S-O associations, whereas the NA-core is not needed and can in fact impede using multiple S-O associations. No impairments were observed in the NOP condition, demonstrating that these structures are not critical to stimulus-response learning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed.

A recently developed mathematical model (DMOD) also predicts that Ss prefer the unpredictable reward situation under conditions that substantially decrease aversiveness of unpredictable nonreward (Daly & Daly, 1982). Because a high proportion of reinforced trials (lenient schedule) and alcohol decrease aversive conditioning, these variables were tested with rats in 5 E-maze experiments. A choice to 1 side of the maze resulted in a stimulus uncorrelated with reward outcome (unpredictable). A choice to the other side resulted in stimuli correlated with reward and nonreward (predictable). Stimuli were not visible until after the choice was made. A lenient reinforcement schedule resulted in preference for the unpredictable reward situation if rewards were not delayed. Alcohol resulted in preference for the unpredictable reward situation if a medium 5-pellet reward was given. A lenient reinforcement schedule combined with an alcohol injection resulted in faster acquisition of the preference for the unpredictable reward situation than did a lenient schedule combined with a saline control injection. These results pose a major challenge to most theories, yet were predicted by DMOD. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Repeated cocaine augments excitatory amino acid transmission in the nucleus accumbens only in rats having developed behavioral sensitization

Rats were pretreated with daily cocaine or saline injections for 1 week. The rats treated with daily cocaine were separated into two groups: a sensitized group of animals demonstrating > 20% increase in motor activity on the last injection compared with the first injection of daily cocaine, and a nonsensitized group showing < 20% elevation. At 2-3 weeks after the last daily injection, four experiments were performed to assess changes in excitatory amino acid (EAA) transmission in the nucleus accumbens produced by repeated cocaine administration. (1) Rats were challenged with a microinjection of AMPA into the shell or core of the nucleus accumbens. The sensitized rats demonstrated greater motor activity than did the saline-pretreated or nonsensitized animals after AMPA injection into either subnucleus. (2) It was shown that the behavioral distinction between sensitized, nonsensitized, and control rats in behavioral responsiveness to AMPA was not mediated by differences in AMPA-induced dopamine release. (3) The extracellular content of glutamate was measured after a cocaine challenge given at 21 d of withdrawal. Cocaine elevated the levels of glutamate in the core of sensitized rats, but not of nonsensitized or control rats. (4) Microinjection of the non-NMDA antagonist 6-cyano-7-nitroquinoxaline-2,3-dione into the core abolished the augmented motor response to a cocaine challenge in sensitized rats, but was without effect on cocaine-induced motor activity in nonsensitized animals. These results indicate that repeated cocaine administration increases EAA transmission in the nucleus accumbens only in rats that develop behavioral sensitization to cocaine.     

Fixed-ratio schedules increase generalized self-control: Preference for large rewards despite high effort or punishment.

Three experiments investigated the effects of FR reinforcement on generalized self-control involving high effort and punishment. In Experiment 1, rats received food in a runway for the completion of each round trip (continuous-reinforcement group) or every fifth round trip (FR group). Control rats received food at the same temporal intervals as these groups but without any instrumental requirement. When all rats were next given a series of choices between a large food reward requiring high lever force versus a small reward requiring low lever force, the FR rats showed the greatest self-control. In Experiments 2 and 3, rats were rewarded on a continuous or FR schedule followed by choice between a large food reward accompanied by intermittent shock vs a small or absent food reward without shock. The FR rats again showed the greatest self-control. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Age-related changes in the capacity, rate, and modulation of dopamine uptake within the striatum and nucleus accumbens of Fischer 344 rats: an in vivo electrochemical study

Age-related changes in the capacity, rate, and modulation of dopamine (DA) uptake within the striatum and the nucleus accumbens core of Fischer 344 rats were investigated using in vivo electrochemical recordings coupled with local drug application techniques. Equimolar amounts of DA were pressure ejected into the striatum and the nucleus accumbens of 6-, 12-, 18-, and 24-month old rats. The DA ejections produced larger DA signal amplitudes in the older rats, suggesting age-related differences in the capacity to clear extracellular DA. Within the striatum, the capacity and rate of DA uptake were reduced by 50% in the aged groups (18 and 24 months) compared with the younger rats (6 and 12 months). In the nucleus accumbens, significant reductions in DA uptake capacity and rate were observed in the 24-month group. In both brain regions and in all age groups studied, the rate of DA uptake was found to be concentration-dependent until a maximal rate was reached. The maximum rate of DA transport was significantly reduced in both the striatum and the nucleus accumbens of aged rats (18 and 24 months versus 6 and 12 months). The ability of nomifensine, an inhibitor of the DA transporter, to modulate DA signal amplitudes in the striatum and the nucleus accumbens was also decreased with age (24 months versus 6 months). Taken together, these findings demonstrate substantial age-related deficits in DA uptake processes within the striatum and the nucleus accumbens, consistent with the hypothesis that DA uptake may be slowed in aged animals to compensate for reductions in DA release.