Self-administered cocaine causes long-lasting increases in impulsive choice in a delay discounting task.

Cocaine use is associated with high levels of impulsive choice (preference for immediate over delayed rewards), but it is not clear whether cocaine use causes elevated impulsive choice, or whether elevated impulsive choice is solely a predisposing factor for cocaine use. This study examined the effects of prior cocaine self-administration on rats performing a delay discounting task commonly used to measure impulsive choice. Male Long-Evans rats were implanted with intravenous catheters, and following recovery, were trained to self-administer 30 mg/kg/day cocaine HCl (approx. 0.5 mg/kg/infusion) for 14 consecutive days (a control group received yoked intravenous saline infusions). Following three weeks of withdrawal, all rats were food-restricted and began training on the delay discounting task in standard operant chambers. On each trial, rats were given a choice between two levers. A press on one lever delivered a small food reward immediately, and a press on the other delivered a large food reward after a variable delay period. Rats that self-administered cocaine displayed greater impulsive choice (enhanced preference for the small immediate over the large delayed reward, as reflected by shorter indifference points) compared to controls, but were no different from controls on a “probabilistic discounting” task in which they chose between small certain and large uncertain rewards. These data suggest that self-administered cocaine can cause lasting elevations in impulsive choice, and that the high levels of impulsive choice observed in human cocaine users may be due in part to long-term effects of cocaine on brain function. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal Discounting When the Choice Is Between Two Delayed Rewards.

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cocaine exposure causes long-term increases in impulsive choice.

In this study, the authors examined the long-term effects of prior exposure to cocaine on a delay-discounting task commonly used to measure impulsive choice. Male Long-Evans rats received daily intraperitoneal injections of 30 mg/kg cocaine HCl or saline for 14 days. Following 3 weeks of withdrawal, rats began training. On each trial, rats were given a choice between 2 levers. A press on 1 lever resulted in immediate delivery of a single 45-mg food pellet, and a press on the other resulted in delivery of 4 pellets after a delay period. Impulsive choice was defined as preference for the small immediate over the large delayed reward. Three months after treatment, cocaine-exposed rats displayed increased impulsive choice behavior. They also showed less anticipatory responding (entries into the food trough) during the delays prior to reward delivery, indicating that the enhanced impulsive choice in these rats may be related to deficits in bridging the delay between response and reward. These data demonstrate that cocaine exposure can cause enduring increases in impulsive choice behavior, consistent with observations in human subjects with drug addictions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Now or later? Striatum and insula activation to immediate versus delayed rewards.

Neuroimaging studies on delay discounting tasks that use reward delays ranging from minutes to days have implicated the insula and striatum in the processing of intertemporal decisions. This study aimed at assessing whether these brain regions would also be involved in decision-making when subjects have to wait through the delays within the range of seconds. Using functional MRI in 13 healthy volunteers, we repeatedly presented monetary options with delays that differed within the range of multiple seconds. Using a region-of-interest approach, we found significant activation in the bilateral anterior insula and striatum when subjects chose either the immediate (smaller) or delayed (larger) option. In particular, insular activation was observed after the response and the delay, when the outcome of the immediate or the delayed choice was shown. Significantly greater activation was observed in the ventroanterior striatum while subjects chose the immediate, as opposed to the delayed, options and also after receiving the outcome of waiting through the longer delay option. The evidence presented here indicates that both the ventral striatum and the insula are involved in the processing of choosing delay options as well as the consequences of choices with delays in the seconds' range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward discounting as a measure of impulsive behavior in a psychiatric outpatient population.

Impulsivity has been operationalized as a choice of an immediate smaller reward over a larger delayed or uncertain reward. This study examined a procedure that measures reward preference under these contingencies in psychiatric outpatients considered either at a high or low risk for engaging in impulsive behavior depending on their psychiatric diagnoses. The participants' rates of delay and uncertainty reward discounting were compared with their performances on a behavioral inhibition task and responses on a self-report personality impulsivity measure. The high-risk participants discounted delayed rewards more sharply and scored higher on the self-report impulsivity measure relative to the low-risk participants. Delay and uncertainty discounting were modestly correlated, but no other relationships were found between the other measures. Results from this study indicate that delay-discounting tasks may be sensitive to at least one form of impulsive behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Exposure to rewards and the delay of gratification.

Gave 111 children aged 6, 7, 8, and 9 yrs old a choice between an immediate reward and a larger, delayed reward. Half of the Ss in each age group saw the rewards before choosing, while the other half did not. Exposure to the rewards led to more choices of delayed reward for 7 yr olds. The 6 yr olds predominantly chose immediate rewards regardless of exposure, while the 8 and 9 yr olds predominantly chose the delayed reward regardless of exposure. 2 explanations are offered for the results for the 7 yr olds. In one, the results are attributed to the effect of exposure on value and/or expectancy of the delayed reward. In the other, it is suggested that exposure orients attention to the value, rather than the immediacy, of the alternatives. In this view, delay of gratification in young children reflects a form of decentration. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD.

To investigate the preference for unpredictable rewards predicted by the present author and J. T. Daly's (see record 1983-20275-001) modification, known as the DMOD model, of R. A. Rescorla and A. R. Wagner's (1972) previous model of reinforcement, the present author conducted 5 E-maze experiments with 144 male Holtzman rats. In Exps I–V, Ss were given a choice between receiving reward and nonreward in a situation in which stimuli were correlated with reward outcome (predictable situation) vs a situation in which the stimuli were uncorrelated with reward outcome (unpredictable situation). Preference for the unpredictable situation occurred under the following conditions: small (1 37-mg pellet) immediate rewards, small delayed (15-sec) rewards when the cues correlated with reward outcome were absent during the delay interval, large (15 pellets) immediate rewards when a difficult discrimination was required, and when the stimulus predicting nonreward was present at the choice point. Preference for the predictable situation was strongest if reinforcement was delayed and large or if the stimulus predicting reward was present at the choice point. A weaker preference for the predictable situation occurred if reinforcement was immediate, large, and required a simple discrimination or if reinforcement was large and delayed and the cues that correlated with reward outcome were absent during the delay interval. Findings support the predictions of the DMOD model of appetitive learning. (33 ref) (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Pathological gamblers, with and without substance abuse disorders, discount delayed rewards at high rates.

Pathological gambling is classified as a disorder of impulse control, yet little research has evaluated behavioral indices of impulsivity in gamblers. The rates at which rewards delayed in time are subjectively devalued may be a behavioral marker of impulsivity. This study evaluated delay discounting in 60 pathological gamblers and 26 control participants. Gamblers were divided into those with (n?=?21) and without (n?=?39) substance use disorders. A hypothetical $1,000 reward was delayed at intervals ranging from 6 hr to 25 years, and immediate rewards varied from $1 to $999. Pathological gamblers discounted delayed rewards at higher rates than control participants, and gamblers with substance use disorders discounted delayed rewards at higher rates than non-substance-abusing gamblers. These data provide further evidence that rapid discounting of delayed rewards may be a feature central to impulse control and addictive disorders, including pathological gambling. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell.

The authors investigated the contribution of the nucleus accumbens (NAc) core and shell to effort-based decision making using a discounting procedure. Selection of 1 lever delivered a smaller, 2-pellet reward immediately, whereas the other lever delivered a 4-pellet reward after a fixed ratio of presses (2, 5, 10, or 20) that increased over 4 blocks of 10 discrete choice trials. Subsequent testing employed an equivalent delays procedure, whereby the relative delay to reward delivery after selection of either option was equalized. In well-trained rats, inactivation of the core, but not the shell, via infusion of GABA A/B agonists muscimol/baclofen reduced preference for the high-effort option under standard conditions and also when rats were tested using an equivalent delays procedure. However, inactivation of the core did not alter preference for 4-pellet versus 2-pellet rewards when the relative costs of each option were the same (1 press). Thus, the NAc core, but not the shell, appears to be part of a neural circuit that biases choice toward larger rewards associated with a greater effort cost. Furthermore, the contributions by the NAc core to this form of decision making can be dissociated from its role in delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay discounting of real and hypothetical rewards.

The degree to which real and hypothetical rewards were discounted across delays ranging from 6 hr to 1 year was explored in a within-subjects design. An adjusting-amounts procedure was used to estimate the subjective value of real and hypothetical rewards at each delay. A hyperbolic discounting function provided a significantly better fit to individual participants' preferences than did an exponential function. No significant effect of reward type on degree of hyperbolic discounting or area under the discounting curves was detected. These findings offer some support for the validity of using hypothetical rewards to estimate discounting rates in substance-abusing and other populations, but caution is suggested because this support is gleaned from a failure to detect an effect of reward type. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of reward expectation on behavior-related neuronal activity in primate striatum

Rewards constitute important goals for voluntary behavior. This study aimed to investigate how expected rewards influence behavior-related neuronal activity in the anterior striatum. In a delayed go-nogo task, monkeys executed or withheld a reaching movement and obtained liquid or sound as reinforcement. An initial instruction picture indicated the behavioral reaction to be performed and the reinforcer to be obtained after a subsequent trigger stimulus. Movements varied according to the reinforcers predicted by the instructions, suggesting that animals differentially expected the two outcomes. About 250 of nearly 1,500 neurons in anterior parts of caudate nucleus, putamen, and ventral striatum showed typical task-related activations that reflected the expectation of instructions and trigger, and the preparation, initiation, and execution of behavioral reactions. Strikingly, most task-related activations occurred only when liquid reward was delivered at trial end, rather than the reinforcing sound. Activations close to the time of reward showed similar preferences for liquid reward over the reinforcing sound, suggesting a relationship to the expectation or detection of the motivational outcome of the trial rather than to a "correct" or "end-of-trial" signal. By contrast, relatively few activations in the present task occurred irrespective of the type of reinforcement. In conclusion, many of the behavior-related neurons investigated in the anterior striatum were influenced by an upcoming primary liquid reward and did not appear to code behavioral acts in a motivationally neutral manner. Rather, these neurons incorporated information about the expected outcome into their behavior-related activity. The activations influenced by reward several seconds before its occurrence may constitute a neuronal basis for the retrograde effects of rewards on behavioral reactions.     

Interactions between the prefrontal cortex and amygdala during delay discounting and reversal.

Interactions between the prefrontal cortex and amygdala are thought to be critical for reward anticipation. Alterations in reward anticipation that lead to an inability to wait for rewards or a diminished capacity to change behavior when doing so would be optimal are often termed impulsivity and compulsivity, respectively. Distinct regions of the prefrontal cortex may support decreased impulsivity through self-control and decreased compulsivity through flexibility. However, both self-control and flexibility appear to involve the amygdala. Using a delay discounting paradigm, the current investigation found that inactivation and disconnection of the medial prefrontal cortex and basolateral amygdala led rats to become more impulsive by affecting preference for smaller immediate over larger delayed rewards. Conversely, inactivation and disconnection of the orbitofrontal cortex and amygdala led rats to become more compulsive as demonstrated by an inability to flexibly reverse stimulus–reward relationships in an odor reversal task. The current findings support a double dissociation between orbitofrontal cortex–amygdala interactions for odor reversal and medial prefrontal cortex–amygdala interactions for delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay Discounting of Potentially Real and Hypothetical Rewards: II. Between- and Within-Subject Comparisons.

Prior studies comparing discounting of delayed hypothetical or potentially real rewards have reported no differences, but they used within-subjects designs. This raises the possibility that participants remembered their choices in one condition and repeated them in the other. In Experiment 1, between-subjects comparisons were made with an adjusting-amount procedure. No significant effect of reward type on delay discounting was detected. Experiment 2 increased the proportion of real rewards and made between- and within-subject comparisons. These comparisons also failed to reveal a significant effect of reward type. Although these findings are consistent with prior findings, caution is urged because choices involving hypothetical rewards have yet to be compared with choices involving real rewards (i.e., the consequences of every choice are obtained) in an experiment using forced-choice trials and steady-state methodology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward prediction in primate basal ganglia and frontal cortex

Reward information is processed in a limited number of brain structures, including fronto-basal ganglia systems. Dopamine neurons respond phasically to primary rewards and reward-predicting stimuli depending on reward unpredictability but without discriminating between rewards. These responses reflect 'errors' in the prediction of rewards in correspondence to learning theories and thus may constitute teaching signals for appetitive learning. Neurons in the striatum (caudate, putamen, ventral striatum) code reward predictions in a different manner. They are activated during several seconds when animals expect predicted rewards. During learning, these activations occur initially in rewarded and unrewarded trials and become subsequently restricted to rewarded trials. This occurs in parallel with the adaptation of reward expectations by the animals, as inferred from their behavioral reactions. Neurons in orbitofrontal cortex respond differentially to stimuli predicting different liquid rewards, without coding spatial or visual features. Thus, different structures process reward information processed in different ways. Whereas dopamine neurons emit a reward teaching signal without indicating the specific reward, striatal neurons adapt expectation activity to new reward situations, and orbitofrontal neurons process the specific nature of rewards. These reward signals need to cooperate in order for reward information to be used for learning and maintaining approach behavior.     

Scalar expectancy theory and choice between delayed rewards.

In this article scalar expectancy theory is applied to variable and fixed delays to reward. It is assumed that all delays are represented in memory with scalar variance and that subjects choose between alternatives by sampling from the memory distributions associated with each and choosing the smaller delay. This simple scheme is shown to entail four common findings in the choice literature: (a) approximate matching of choice ratios to reward ratios (the matching law) when both alternatives are associated with variable delays scheduled with constant probability; (b) undermatching, in which choice is closer to indifference than matching, when both alternatives are variable but scheduled with uniform distributions; (c) overmatching, in which choice is more extreme than matching, when both alternatives are fixed delays; and (d) preference for variable delays scheduled with a constant probability over fixed delays. Overmatching and Weber's law are illustrated in experiments using the time-left procedure (Gibbon & Church, 1981). The preference for variable delay is demonstrated in this procedure, followed by study of a unique variable schedule of delays for which the theoretical account predicts, and the data confirm, the elimination of the preference for variability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choosing problem-solving rewards and Halloween prizes: Delay of gratification and preference for symbolic reward as a function of development, motivation, and personal investment.

Conducted 3 experiments to generate and cross-validate a developmental account of reward preferences in the lifelike condition in which available rewards differ on both the immediate-delayed and material-symbolic dimensions. 429 children within a broad range of developmental levels (CA 2?–20 yrs) selected from arrays in which immediate-material, immediate-symbolic, delayed-material, and delayed-symbolic items were available. In each experiment more mature Ss delayed gratification to maximize reward magnitude and chose rewards symbolic of success more often than did less mature Ss. Analyses indicated that the 2 reward dimensions were both highly salient in Ss' thinking and that developmental effects on the 2 dimensions could be neatly integrated within a 3-stage developmental scale. Developmental effects on this scale and on the separate dimensions persisted across changes in the particular reward items available and changes in location from school testing room to more naturalistic Halloween settings. Findings indicate that preference for symbolic rewards tends to be more pronounced among the mentally retarded than the nonretarded but cast doubt upon previous interpretations of that group difference. Finally, children's Halloween prize choices suggested that preference for symbolic reward is a function of one's developmental level, the achievement being rewarded, and the extent of one's personal investment in that achievement. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Imagining the future: Degraded representations of future rewards and events in schizophrenia.

Over the course of life, most people work toward temporally distant rewards such as university degrees or work-related promotions. In contrast, many people with schizophrenia show deficits in behavior oriented toward long-term rewards, although they function adequately when rewards are more immediately present. Moreover, when asked about possible future events, individuals with schizophrenia show foreshortened future time perspectives relative to healthy individuals. Here, we take the view that these deficits are related and can be explained by cognitive deficits. We compared the performance of participants with schizophrenia (n = 39) and healthy participants (n = 25) on tasks measuring reward discounting and future event representations. Consistent with previous research, we found that relative to healthy participants, those with schizophrenia discounted the value of future rewards more steeply. Furthermore, when asked about future events, their responses were biased toward events in the near future, relative to healthy participants' responses. Although discounting and future representations were unrelated in healthy participants, we found significant correlations across the tasks among participants with schizophrenia, as well as correlations with cognitive variables and symptoms. Further analysis showed that statistically controlling working memory eliminated group differences in task performance. Together these results suggest that the motivational deficits characteristic of schizophrenia relate to cognitive deficits affecting the ability to represent and/or evaluate distant outcomes, a finding with important implications for promoting recovery from schizophrenia. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Discounting of delayed rewards in opioid-dependent outpatients: Exponential or hyperbolic discounting functions?

Recent theories of substance abuse have used value discounting of delayed rewards to partly explain the decision to take drugs. Normative-economic theory holds that an exponential function describes the effects of delay on discounting, whereas the matching law posits a hyperbolic discounting function. The ability of these functions to describe 18 human heroin-dependent individuals' monetary- and heroin-reward delay-discounting functions was assessed. In the 1st condition, participants chose between immediate and delayed hypothetical monetary rewards. Delayed rewards were $1,000, and the immediate reward amount was adjusted until choices reflected indifference. In the 2nd condition, participants chose between immediate and delayed heroin (the delayed amount was that which each participant reported he or she could purchase with $1,000). The hyperbolic function produced significantly higher R2 values and significantly lower sums of squared error values. Consistent with previous findings, delayed heroin rewards were discounted at a significantly higher rate than were delayed monetary rewards. (PsycINFO Database Record (c) 2010 APA, all rights reserved)