Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes.

Previous research has shown that the value of large future rewards is discounted less steeply than is the value of small future rewards. These experiments extended this line of research to probabilistic rewards. Two experiments replicated the standard findings for delayed rewards but demonstrated that amount has an opposite effect on the discounting of probabilistic rewards. That is, large probabilistic amounts were discounted at the same or higher rates than small amounts. Although amount had opposite effects on the discounting of delayed and probabilistic rewards, nevertheless, the same form of mathematical function accurately described discounting of both types of reward. The findings suggest that fundamentally similar, but not identical, processes are involved in decision making regarding delayed and probabilistic rewards. The implications of these findings for impulsivity and self-control are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A multivariate assessment of individual differences in sensation seeking and impulsivity as predictors of amphetamine self-administration and prefrontal dopamine function in rats.

Drug abuse vulnerability has been linked to sensation seeking (behaviors likely to produce rewards) and impulsivity (behaviors occurring without foresight). Since previous preclinical work has been limited primarily to using single tasks as predictor variables, the present study determined if measuring multiple tasks of sensation seeking and impulsivity would be useful in predicting amphetamine self-administration in rats. Multiple tasks were also used as predictor variables of dopamine transporter function in the medial prefrontal and orbitofrontal cortexes, as these neural systems have been implicated in sensation seeking and impulsivity. Rats were tested on six behavioral tasks as predictor variables to evaluate sensation seeking (locomotor activity, novelty place preference, and sucrose reinforcement on a progressive ratio schedule) and impulsivity (delay discounting, cued go/no-go, and passive avoidance), followed by d-amphetamine self-administration (0.0056–0.1 mg/kg infusion) and kinetic analysis of dopamine transporter function as outcome variables. The combination of these predictor variables into a multivariate approach failed to yield any clear relationship among predictor and outcome measures. Using multivariate approaches to understand the relation between individual predictor and outcome variables in preclinical models may be hindered by alterations in behavior due to training and thus, the relation between various individual differences in behavior and drug self-administration may be better assessed using a univariate approach in which a only a single task is used as the predictor variable. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Effects of methylphenidate on discounting of delayed rewards in attention deficit/hyperactivity disorder.

Impulsivity is a central component of attention deficit/hyperactivity disorder (ADHD). Delay discounting, or a preference for smaller, immediate rewards over larger, delayed rewards, is considered an important aspect of impulsivity, and delay-related impulsivity has been emphasized in etiological models of ADHD. In this study, we examined whether stimulant medication, an effective treatment for ADHD, reduced discounting of delayed experiential and hypothetical rewards among 49 children (ages 9–12 years) with ADHD. After a practice day, participants completed a 3-day double-blind placebo-controlled acute medication assessment. Active doses were long-acting methylphenidate (Concerta), with the nearest equivalents of 0.3 and 0.6 mg/kg TID immediate-release methylphenidate. On each testing day, participants completed experiential (real-world money in real time) and hypothetical discounting tasks. Relative to placebo, methylphenidate reduced discounting of delayed experiential rewards but not hypothetical rewards. Broadly consistent with etiological models that emphasize delay-related impulsivity among children with ADHD, these findings provide initial evidence that stimulant medication reduces delay discounting among those with the disorder. The results also draw attention to task parameters that may influence the sensitivity of various delay discounting measures to medication effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pathological gamblers, with and without substance abuse disorders, discount delayed rewards at high rates.

Pathological gambling is classified as a disorder of impulse control, yet little research has evaluated behavioral indices of impulsivity in gamblers. The rates at which rewards delayed in time are subjectively devalued may be a behavioral marker of impulsivity. This study evaluated delay discounting in 60 pathological gamblers and 26 control participants. Gamblers were divided into those with (n?=?21) and without (n?=?39) substance use disorders. A hypothetical $1,000 reward was delayed at intervals ranging from 6 hr to 25 years, and immediate rewards varied from $1 to $999. Pathological gamblers discounted delayed rewards at higher rates than control participants, and gamblers with substance use disorders discounted delayed rewards at higher rates than non-substance-abusing gamblers. These data provide further evidence that rapid discounting of delayed rewards may be a feature central to impulse control and addictive disorders, including pathological gambling. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward discounting as a measure of impulsive behavior in a psychiatric outpatient population.

Impulsivity has been operationalized as a choice of an immediate smaller reward over a larger delayed or uncertain reward. This study examined a procedure that measures reward preference under these contingencies in psychiatric outpatients considered either at a high or low risk for engaging in impulsive behavior depending on their psychiatric diagnoses. The participants' rates of delay and uncertainty reward discounting were compared with their performances on a behavioral inhibition task and responses on a self-report personality impulsivity measure. The high-risk participants discounted delayed rewards more sharply and scored higher on the self-report impulsivity measure relative to the low-risk participants. Delay and uncertainty discounting were modestly correlated, but no other relationships were found between the other measures. Results from this study indicate that delay-discounting tasks may be sensitive to at least one form of impulsive behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interaction of the amygdala with the frontal lobe in reward memory

Five cynomolgus monkeys (Macaca fascicularis) were assessed for their ability to associate visual stimuli with food reward. They learned a series of new two-choice visual discriminations between coloured patterns displayed on a touch-sensitive monitor screen; the feedback for correct choice was delivery of food. Normal learning in this task is known to be dependent on the amygdala. The monkeys received brain lesions which were designed to disconnect the amygdala from interaction with other brain structures thought to be involved in this memory task. All the monkeys received an amygdalectomy in one hemisphere and lesions in the other hemisphere of some of the projection targets of the amygdala, namely the ventral striatum, the mediodorsal thalamus and the ventromedial prefrontal cortex. The rate of learning new problems was assessed before and after each operation. Disconnection of the amygdala from the ventral striatum was without effect on learning rate. An earlier study had shown that disconnection of the amygdala from either the mediodorsal thalamus or the ventromedial prefrontal cortex produced only a mild impairment, significantly less severe than that produced by bilateral lesions of any of these three structures. The present results show, however, that disconnection of the amygdala from both the mediodorsal thalamus and the ventromedial prefrontal cortex in the same animal, by crossed unilateral lesions of the amygdala in one hemisphere and of both the mediodorsal thalamus and the ventromedial prefrontal cortex in the other hemisphere, produces an impairment as severe as that which follows bilateral lesions of any of these three structures.(ABSTRACT TRUNCATED AT 250 WORDS)     

Delay discounting in current and former marijuana-dependent individuals.

Studies have found that a variety of drug-dependent groups discount delayed rewards more than matched controls. This study compared delay discounting for a hypothetical $1,000 reward among dependent marijuana users, former dependent marijuana users, and matched controls. Discounting of marijuana was also assessed in the currently marijuana-dependent group. No significant differences in discounting were detected among the groups; however, currently dependent users showed a trend to discount money more than the other 2 groups. Within the dependent marijuana group, marijuana was discounted more than money, and discounting for money and marijuana was significantly and positively correlated. Regression analyses indicated that delay discounting was more closely associated with tobacco use than marijuana use. A variety of questionnaires were also administered, including impulsivity questionnaires. Dependent marijuana users scored as significantly more impulsive on the Impulsiveness subscale of the Eysenck Impulsiveness–Venturesomeness–Empathy questionnaire than controls. However, the 3 groups did not significantly differ on several other personality questionnaires, including the Barratt Impulsivity Scale—11. The Stanford Time Perception Inventory Present–Fatalistic subscale was positively correlated with money and marijuana discounting, indicating that a greater sense of powerlessness over the future is related to greater delay discounting. Results suggest that current marijuana dependence may be associated with a trend toward increased delay discounting, but this effect size appears to be smaller for marijuana than for previously examined drugs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal discounting in choice between delayed rewards: The role of age and income.

This study examined the effects of age and income on temporal discounting (i.e., the decrease in the subjective value of a reward as the delay to its receipt increases). The value of delayed hypothetical monetary rewards was discounted at similar rates by adults of different ages but similar income levels, but at different rates by adults of similar age but different income levels. Specifically, lower income older adults showed a greater degree of temporal discounting than did either upper income older adults or upper income younger adults, but there were no age differences in discounting between the upper income groups. Comparison of these findings with those of a previous study (Green, Fry, & Myerson, 1994) suggests that impulsivity in decision making declines rapidly in young adulthood, reaching stable levels in the 30s. Further, age and income appear to interact in determining the impulsivity of decision making by adults. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Disinhibitory psychopathology and delay discounting in alcohol dependence: Personality and cognitive correlates.

Increased discounting of delayed rewards may reflect a decision bias that contributes to excessive use of alcohol and more generally, to an impulsive, disinhibitory predisposition that is characterized by a preference for immediate over long-term rewards. The current study examined the association between delay discounting of rewards and the covariation among several types of disinhibitory problems that are often comorbid with alcohol dependence (AD). Lifetime problems with alcohol, marijuana, other drugs, childhood conduct disorder, and adult antisocial behavior were assessed in a sample of 426 young adults, 257 of whom had a lifetime diagnosis of AD. Higher delay discounting rates were associated with the covariation among all domains of disinhibitory problems and were not uniquely associated with any one domain. Higher delay discounting rates also were associated with lower intelligence, lower working memory capacity, and higher trait impulsivity. The results suggest that increased delay discounting of rewards may reflect aspects of a general vulnerability to externalizing, disinhibitory disorders. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell.

The authors investigated the contribution of the nucleus accumbens (NAc) core and shell to effort-based decision making using a discounting procedure. Selection of 1 lever delivered a smaller, 2-pellet reward immediately, whereas the other lever delivered a 4-pellet reward after a fixed ratio of presses (2, 5, 10, or 20) that increased over 4 blocks of 10 discrete choice trials. Subsequent testing employed an equivalent delays procedure, whereby the relative delay to reward delivery after selection of either option was equalized. In well-trained rats, inactivation of the core, but not the shell, via infusion of GABA A/B agonists muscimol/baclofen reduced preference for the high-effort option under standard conditions and also when rats were tested using an equivalent delays procedure. However, inactivation of the core did not alter preference for 4-pellet versus 2-pellet rewards when the relative costs of each option were the same (1 press). Thus, the NAc core, but not the shell, appears to be part of a neural circuit that biases choice toward larger rewards associated with a greater effort cost. Furthermore, the contributions by the NAc core to this form of decision making can be dissociated from its role in delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward prediction in primate basal ganglia and frontal cortex

Reward information is processed in a limited number of brain structures, including fronto-basal ganglia systems. Dopamine neurons respond phasically to primary rewards and reward-predicting stimuli depending on reward unpredictability but without discriminating between rewards. These responses reflect 'errors' in the prediction of rewards in correspondence to learning theories and thus may constitute teaching signals for appetitive learning. Neurons in the striatum (caudate, putamen, ventral striatum) code reward predictions in a different manner. They are activated during several seconds when animals expect predicted rewards. During learning, these activations occur initially in rewarded and unrewarded trials and become subsequently restricted to rewarded trials. This occurs in parallel with the adaptation of reward expectations by the animals, as inferred from their behavioral reactions. Neurons in orbitofrontal cortex respond differentially to stimuli predicting different liquid rewards, without coding spatial or visual features. Thus, different structures process reward information processed in different ways. Whereas dopamine neurons emit a reward teaching signal without indicating the specific reward, striatal neurons adapt expectation activity to new reward situations, and orbitofrontal neurons process the specific nature of rewards. These reward signals need to cooperate in order for reward information to be used for learning and maintaining approach behavior.     

Delay discounting of real and hypothetical rewards.

The degree to which real and hypothetical rewards were discounted across delays ranging from 6 hr to 1 year was explored in a within-subjects design. An adjusting-amounts procedure was used to estimate the subjective value of real and hypothetical rewards at each delay. A hyperbolic discounting function provided a significantly better fit to individual participants' preferences than did an exponential function. No significant effect of reward type on degree of hyperbolic discounting or area under the discounting curves was detected. These findings offer some support for the validity of using hypothetical rewards to estimate discounting rates in substance-abusing and other populations, but caution is suggested because this support is gleaned from a failure to detect an effect of reward type. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Positive mood effects on delay discounting.

Delay discounting is the process by which the value of an expected reward decreases as the delay to obtaining that reward increases. Individuals with higher discounting rates tend to prefer smaller immediate rewards over larger delayed rewards. Previous research has indicated that personality can influence an individual's discounting rates, with higher levels of Extraversion predicting a preference for immediate gratification. The current study examined how this relationship would be influenced by situational mood inductions. While main effects were observed for both Extraversion and cognitive ability in the prediction of discounting rates, a significant interaction was also observed between Extraversion and positive affect. Extraverted individuals were more likely to prefer an immediate reward when first put in a positive mood. Extraverts thus appear particularly sensitive to impulsive, incentive-reward-driven behavior by temperament and by situational factors heightening positive affect. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls.

Fifty-six heroin addicts and 60 age-matched controls were offered choices between monetary rewards ($11–$80) available immediately and larger rewards ($25–$85) available after delays ranging from 1 week to 6 months. Participants had a 1-in-6 chance of winning a reward that they chose on one randomly selected trial. Delay-discounting rates were estimated from the pattern of participants' choices. The discounting model of impulsiveness (G. Ainslie, 1975) implies that delay-discounting rates are positively correlated with impulsiveness. On average, heroin addicts' discount rates were twice those of controls (p?=?.004), and discount rates were positively correlated with impulsivity as measured by self-report questionnaires (p?     

Neural basis of individual differences in impulsivity: Contributions of corticolimbic circuits for behavioral arousal and control.

The objective of the current study was to analyze the neural correlates of behavioral arousal and inhibitory control as they relate to individual differences in impulsivity via well-established functional MRI amygdala reactivity and prefrontal inhibitory control paradigms in healthy adult subjects. Impulsivity correlated positively with activity of the bilateral ventral amygdala, parahippocampal gyrus, dorsal anterior cingulate gyrus (BA 32), and bilateral caudate. Conversely, impulsivity correlated negatively with activity of the dorsal amygdala and ventral prefrontal cortex (BA 47). Together, these findings suggest that dispositional impulsivity is influenced by the functional interplay of corticolimbic behavioral arousal and control circuits. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats.

The orbitofrontal cortex (OFC) and basolateral nucleus of the amygdala (BLA) are important neural regions in responding adaptively to changes in the incentive value of reward. Recent evidence suggests these structures may be differentially engaged in effort and cue-guided choice behavior. In 2 T-maze experiments, we examined the effects of bilateral lesions of either BLA or OFC on (1) effortful choices in which rats could climb a barrier for a high reward or select a low reward with no effort and (2) effortful choices when a visual cue signaled changes in reward magnitude. In both experiments, BLA rats displayed transient work aversion, choosing the effortless low reward option. OFC rats were work averse only in the no cue conditions, displaying a pattern of attenuated recovery from the cue conditions signaling reward unavailability in the effortful arm. Control measures rule out an inability to discriminate the cue in either lesion group. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Delay Discounting of Potentially Real and Hypothetical Rewards: II. Between- and Within-Subject Comparisons.

Prior studies comparing discounting of delayed hypothetical or potentially real rewards have reported no differences, but they used within-subjects designs. This raises the possibility that participants remembered their choices in one condition and repeated them in the other. In Experiment 1, between-subjects comparisons were made with an adjusting-amount procedure. No significant effect of reward type on delay discounting was detected. Experiment 2 increased the proportion of real rewards and made between- and within-subject comparisons. These comparisons also failed to reveal a significant effect of reward type. Although these findings are consistent with prior findings, caution is urged because choices involving hypothetical rewards have yet to be compared with choices involving real rewards (i.e., the consequences of every choice are obtained) in an experiment using forced-choice trials and steady-state methodology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay of gratification in psychopathic and nonpsychopathic offenders.

Delay of gratification is a prototypical measure of self-control that merits systematic investigation in psychopaths. White male prisoners were provided with repeated opportunities to select an immediate response with uncertain reward or a delayed response with a higher rate of reward under 1 of 3 incentive conditions. Psychopaths' performance depended on their level of trait anxiety and incentive condition: Whereas low-anxious psychopaths were relatively unwilling to delay when omission of expected rewards also incurred monetary punishments, they displayed relative superior performance when the task involved rewards only. Findings complement those for passive avoidance learning in psychopaths and suggest that inhibitory self-control in low-anxious psychopaths is somewhat impaired under conditions involving a combination of monetary rewards and punishments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward expectancy in primate prefrontal neurons

The prefrontal cortex is important in the organization of goal-directed behaviour. When animals are trained to work for a particular goal or reward, reward 'expectancy' is processed by prefrontal neurons. Recent studies of the prefrontal cortex have concentrated on the role of working memory in the control of behaviour. In spatial delayed-response tasks, neurons in the prefrontal cortex show activity changes during the delay period between presentation of the cue and the reward, with some of the neurons being spatially specific (that is, responses vary with the cue position). Here I report that the delay activity in prefrontal neurons is dependent also on the particular reward received for the behavioural response, and to the way the reward is given. It seems that the prefrontal cortex may monitor the outcome of goal-directed behaviour.     

Functional disconnection of a prefrontal cortical–dorsal striatal system disrupts choice reaction time performance: Implications for attentional function.

This series of experiments investigated the role of a prefrontal cortical–dorsal striatal circuit in attention, using a continuous performance task of sustained and spatially divided visual attention. A unilateral excitotoxic lesion of the medial prefrontal cortex and a contralateral lesion of the medial caudate-putamen were used to "disconnect" the circuit. Control groups of rats with unilateral lesions of either structure were tested in the same task. Behavioral controls included testing the effects of the disconnection lesion on Pavlovian discriminated approach behavior. The disconnection lesion produced a significant reduction in the accuracy of performance in the attentional task but did not impair Pavlovian approach behavior or affect locomotor or motivational variables, providing evidence for the involvement of this medial prefrontal corticostriatal system in aspects of visual attentional function. (PsycINFO Database Record (c) 2010 APA, all rights reserved)