An argument against dual valuation system competition: Cognitive capacities supporting future orientation mediate rather than compete with visceral motivations.

The dynamic inconsistency of preference is well documented in behavioral research, but its basis remains controversial. In this article, the authors summarize recent functional MRI (fMRI) work in the domain of intertemporal choice, specifically considering evidence bearing on the hypothesis that delay discounting in humans is determined by competition between an evolutionarily older system that discounts precipitously with delay (System 1) and a newer system that exhibits very little discounting (System 2). The authors argue that neuroimaging evidence does not support the hypothesized separate and competing value systems. While it is clear that the sophisticated cognitive capacities that lead to greater valuation of larger later alternatives (e.g., selective attention and self-signaling) depend critically on neocortical structures, these capacities affect intertemporal choice through mediation of (rather than competition with) older cortical and subcortical structures central to reward and motivation. Taken together, neuroimaging evidence supports the alternative hypothesis that intertemporal choice is guided by a single valuation system. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal Discounting When the Choice Is Between Two Delayed Rewards.

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell.

The authors investigated the contribution of the nucleus accumbens (NAc) core and shell to effort-based decision making using a discounting procedure. Selection of 1 lever delivered a smaller, 2-pellet reward immediately, whereas the other lever delivered a 4-pellet reward after a fixed ratio of presses (2, 5, 10, or 20) that increased over 4 blocks of 10 discrete choice trials. Subsequent testing employed an equivalent delays procedure, whereby the relative delay to reward delivery after selection of either option was equalized. In well-trained rats, inactivation of the core, but not the shell, via infusion of GABA A/B agonists muscimol/baclofen reduced preference for the high-effort option under standard conditions and also when rats were tested using an equivalent delays procedure. However, inactivation of the core did not alter preference for 4-pellet versus 2-pellet rewards when the relative costs of each option were the same (1 press). Thus, the NAc core, but not the shell, appears to be part of a neural circuit that biases choice toward larger rewards associated with a greater effort cost. Furthermore, the contributions by the NAc core to this form of decision making can be dissociated from its role in delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Now or later? Striatum and insula activation to immediate versus delayed rewards.

Neuroimaging studies on delay discounting tasks that use reward delays ranging from minutes to days have implicated the insula and striatum in the processing of intertemporal decisions. This study aimed at assessing whether these brain regions would also be involved in decision-making when subjects have to wait through the delays within the range of seconds. Using functional MRI in 13 healthy volunteers, we repeatedly presented monetary options with delays that differed within the range of multiple seconds. Using a region-of-interest approach, we found significant activation in the bilateral anterior insula and striatum when subjects chose either the immediate (smaller) or delayed (larger) option. In particular, insular activation was observed after the response and the delay, when the outcome of the immediate or the delayed choice was shown. Significantly greater activation was observed in the ventroanterior striatum while subjects chose the immediate, as opposed to the delayed, options and also after receiving the outcome of waiting through the longer delay option. The evidence presented here indicates that both the ventral striatum and the insula are involved in the processing of choosing delay options as well as the consequences of choices with delays in the seconds' range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A role for dorsal and ventral hippocampus in inter-temporal choice cost-benefit decision making.

Previous studies suggest a preferential role for dorsal hippocampus (dHPC) in spatial memory tasks, whereas ventral hippocampus (vHPC) has been implicated in aspects of fear and/or anxiety. In this study, we tested the hypothesis that vHPC may be a critical subregion for performance on a delay-based, cost-benefit decision making task. Rats chose between the two goal arms of a T maze, one containing an immediately available small reward, the other containing a larger reward that was only accessible after a delay. dHPC, vHPC, and complete hippocampal (cHPC) lesions all reduced choice of the delayed high reward (HR) in favor of the immediately available low reward (LR). The deficits were not due to a complete inability to remember which reward size was associated with which arm of the maze. When an equivalent 10-s delay was introduced in both goal arms, all rats chose the HR arm on nearly all trials. The deficit was, however, reinstated when the inequality was reintroduced. Our results suggest an important role for both dHPC and vHPC in the extended neural circuitry that underlies intertemporal choice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward discounting as a measure of impulsive behavior in a psychiatric outpatient population.

Impulsivity has been operationalized as a choice of an immediate smaller reward over a larger delayed or uncertain reward. This study examined a procedure that measures reward preference under these contingencies in psychiatric outpatients considered either at a high or low risk for engaging in impulsive behavior depending on their psychiatric diagnoses. The participants' rates of delay and uncertainty reward discounting were compared with their performances on a behavioral inhibition task and responses on a self-report personality impulsivity measure. The high-risk participants discounted delayed rewards more sharply and scored higher on the self-report impulsivity measure relative to the low-risk participants. Delay and uncertainty discounting were modestly correlated, but no other relationships were found between the other measures. Results from this study indicate that delay-discounting tasks may be sensitive to at least one form of impulsive behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal horizons in decision making.

The perception of duration is crucial when we make choices between immediate and delayed rewards. Immediate rewards are valued more than the same rewards if they are delayed. Preferences for earlier rewards become even stronger when the reward can be received within a subjectively proximate time frame. A “rational” decision to wait for a delayed reward may be overruled by an “impulsive” choice when the option for the closer reward falls within this proximate time range. Based on findings on circadian and circannual physiological rhythms, we suggest that there are 2 time units that are both biologically and culturally determined and have an impact on human experience and behavior: the day and the year. We highlight results of a neuroimaging study showing that rewards with delays up to 1 year are discounted differently than reward delays longer than 1 year. This duration-dependent discounting is associated with specific brain activation in the striatum. We present various conceptualizations of subjective time incorporated in parametric models of intertemporal decisions that may lead to a better understanding of human choice behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Self-administered cocaine causes long-lasting increases in impulsive choice in a delay discounting task.

Cocaine use is associated with high levels of impulsive choice (preference for immediate over delayed rewards), but it is not clear whether cocaine use causes elevated impulsive choice, or whether elevated impulsive choice is solely a predisposing factor for cocaine use. This study examined the effects of prior cocaine self-administration on rats performing a delay discounting task commonly used to measure impulsive choice. Male Long-Evans rats were implanted with intravenous catheters, and following recovery, were trained to self-administer 30 mg/kg/day cocaine HCl (approx. 0.5 mg/kg/infusion) for 14 consecutive days (a control group received yoked intravenous saline infusions). Following three weeks of withdrawal, all rats were food-restricted and began training on the delay discounting task in standard operant chambers. On each trial, rats were given a choice between two levers. A press on one lever delivered a small food reward immediately, and a press on the other delivered a large food reward after a variable delay period. Rats that self-administered cocaine displayed greater impulsive choice (enhanced preference for the small immediate over the large delayed reward, as reflected by shorter indifference points) compared to controls, but were no different from controls on a “probabilistic discounting” task in which they chose between small certain and large uncertain rewards. These data suggest that self-administered cocaine can cause lasting elevations in impulsive choice, and that the high levels of impulsive choice observed in human cocaine users may be due in part to long-term effects of cocaine on brain function. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Positive mood effects on delay discounting.

Delay discounting is the process by which the value of an expected reward decreases as the delay to obtaining that reward increases. Individuals with higher discounting rates tend to prefer smaller immediate rewards over larger delayed rewards. Previous research has indicated that personality can influence an individual's discounting rates, with higher levels of Extraversion predicting a preference for immediate gratification. The current study examined how this relationship would be influenced by situational mood inductions. While main effects were observed for both Extraversion and cognitive ability in the prediction of discounting rates, a significant interaction was also observed between Extraversion and positive affect. Extraverted individuals were more likely to prefer an immediate reward when first put in a positive mood. Extraverts thus appear particularly sensitive to impulsive, incentive-reward-driven behavior by temperament and by situational factors heightening positive affect. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Discounting of hypothetical and potentially real outcomes in nicotine-dependent and nondependent samples.

Discounting is a behavioral phenomenon in which the value of an outcome diminishes as a function of its increased delay or decreased probability and is related to substance abuse research because of its theoretical ties with behavioral models of impulsive choice. Research to date suggests that hypothetical outcomes used in discounting research yield data that are indistinguishable from those using potentially real outcomes. However, the extant literature focuses primarily on delay discounting in non-drug-using humans and has not examined whether hypothetical outcomes yield disproportionate numbers of nonsystematic response patterns. In two experiments, we compared hypothetical and potentially real monetary outcomes in delay and probability discounting tasks in terms of rates of discounting and the frequency of nonsystematic response patterns. In Experiment 1, 61 adults reported no smoking, binge drinking, or illicit drug use in the past year. Experiment 2 included a community sample of nicotine-dependent adults (N = 36). In both experiments, discounting for hypothetical and potentially real outcomes yielded similar data, replicating and extending a growing literature pointing to the empirical equivalence of these outcomes. These findings are relevant to research on discounting that is frequently used in the study of substance use and other impulse-control behaviors. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Sex hormonal modulation of hyperbolic discount factor in men.

Hormones in the brain have many types of visceral influences on social behavior and economic decision-making. Although hormone-associated visceral effects attract attention in neuroeconomics, little is known regarding the relationship between sex hormones and hyperbolic discounting. Furthermore, although the “discount factor” (a parameter indicating patience in intertemporal choice) is economically relevant and has neural correlates, most neuroeconomic studies on intertemporal choice have been focusing on neuromodulation of logged discount rates, which sometimes results in difficulty in interpretation. This study was aimed to examine linear correlations between chronic sex hormone levels and hyperbolic discount factors of gains and losses in healthy male students. Participants’ salivary sex hormone (testosterone) levels were also assessed. We observed a positive linear relationship between testosterone levels and hyperbolic discount factor of gains; while no linear relationship between testosterone levels and discount factor of losses was observed. The results indicate that (i) chronic elevation in testosterone may be associated with patience (indicated by a discount factor) in intertemporal choice on gains, (ii) testosterone is unrelated to discounting of delayed losses. Implications for sex hormone-related visceral effects on problematic decision-making (e.g. addiction) and possible neuroendocrinological mechanism (e.g. conversion of testosterone into female hormones in the brain) are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interactions between the prefrontal cortex and amygdala during delay discounting and reversal.

Interactions between the prefrontal cortex and amygdala are thought to be critical for reward anticipation. Alterations in reward anticipation that lead to an inability to wait for rewards or a diminished capacity to change behavior when doing so would be optimal are often termed impulsivity and compulsivity, respectively. Distinct regions of the prefrontal cortex may support decreased impulsivity through self-control and decreased compulsivity through flexibility. However, both self-control and flexibility appear to involve the amygdala. Using a delay discounting paradigm, the current investigation found that inactivation and disconnection of the medial prefrontal cortex and basolateral amygdala led rats to become more impulsive by affecting preference for smaller immediate over larger delayed rewards. Conversely, inactivation and disconnection of the orbitofrontal cortex and amygdala led rats to become more compulsive as demonstrated by an inability to flexibly reverse stimulus–reward relationships in an odor reversal task. The current findings support a double dissociation between orbitofrontal cortex–amygdala interactions for odor reversal and medial prefrontal cortex–amygdala interactions for delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Alcohol demand, delayed reward discounting, and craving in relation to drinking and alcohol use disorders.

A behavioral economic approach to alcohol use disorders (AUDs) emphasizes both individual and environmental determinants of alcohol use. The current study examined individual differences in alcohol demand (i.e., motivation for alcohol under escalating conditions of price) and delayed reward discounting (i.e., preference for immediate small rewards compared to delayed larger rewards) in 61 heavy drinkers (62% with an AUD). In addition, based on theoretical accounts that emphasize the role of craving in reward valuation and preferences for immediate rewards, craving for alcohol was also examined in relation to these behavioral economic variables and the alcohol-related variables. Intensity of alcohol demand and delayed reward discounting were significantly associated with AUD symptoms, but not with quantitative measures of alcohol use, and were also moderately correlated with each other. Likewise, craving was significantly associated with AUD symptoms, but not with alcohol use, and was also significantly correlated with both intensity of demand and delayed reward discounting. These findings further emphasize the relevance of behavioral economic indices of motivation to AUDs and the potential importance of craving for alcohol in this relationship. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay Discounting of Potentially Real and Hypothetical Rewards: II. Between- and Within-Subject Comparisons.

Prior studies comparing discounting of delayed hypothetical or potentially real rewards have reported no differences, but they used within-subjects designs. This raises the possibility that participants remembered their choices in one condition and repeated them in the other. In Experiment 1, between-subjects comparisons were made with an adjusting-amount procedure. No significant effect of reward type on delay discounting was detected. Experiment 2 increased the proportion of real rewards and made between- and within-subject comparisons. These comparisons also failed to reveal a significant effect of reward type. Although these findings are consistent with prior findings, caution is urged because choices involving hypothetical rewards have yet to be compared with choices involving real rewards (i.e., the consequences of every choice are obtained) in an experiment using forced-choice trials and steady-state methodology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Intertemporal choice: Neuronal and psychological determinants of economic decisions.

This introductory article provides a short overview of empirical and theoretical articles presented in the special issue on psychological and neural models of intertemporal decision making, which is divided into 2 parts. The first part consists of contributions presenting different models of intertemporal choice. These contributions provide an overview of current conceptualizations; that is, providing several psychological and neural frameworks, investigating how memory processes are related to the anticipation of time, and showing how the perception of time underlies our decisions about the future. A final article deals with factors that influence choices on environmental policies where the consequences of decisions are delayed by decades or more. The second upcoming part is concerned with functional neuroimaging of intertemporal decision making. Two reviews of studies in neuroimaging and 2 empirical articles examine the questions of which brain regions (and associated functions) are involved in deciding on options with different temporal consequences. We hope this volume will be conducive in developing a better understanding of intertemporal decision making as part of complex sets of neural processes as well as psychological factors that include cognitive reasoning, emotional states, and the interconnected perception of time. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of methylphenidate on discounting of delayed rewards in attention deficit/hyperactivity disorder.

Impulsivity is a central component of attention deficit/hyperactivity disorder (ADHD). Delay discounting, or a preference for smaller, immediate rewards over larger, delayed rewards, is considered an important aspect of impulsivity, and delay-related impulsivity has been emphasized in etiological models of ADHD. In this study, we examined whether stimulant medication, an effective treatment for ADHD, reduced discounting of delayed experiential and hypothetical rewards among 49 children (ages 9–12 years) with ADHD. After a practice day, participants completed a 3-day double-blind placebo-controlled acute medication assessment. Active doses were long-acting methylphenidate (Concerta), with the nearest equivalents of 0.3 and 0.6 mg/kg TID immediate-release methylphenidate. On each testing day, participants completed experiential (real-world money in real time) and hypothetical discounting tasks. Relative to placebo, methylphenidate reduced discounting of delayed experiential rewards but not hypothetical rewards. Broadly consistent with etiological models that emphasize delay-related impulsivity among children with ADHD, these findings provide initial evidence that stimulant medication reduces delay discounting among those with the disorder. The results also draw attention to task parameters that may influence the sensitivity of various delay discounting measures to medication effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay discounting of real and hypothetical rewards.

The degree to which real and hypothetical rewards were discounted across delays ranging from 6 hr to 1 year was explored in a within-subjects design. An adjusting-amounts procedure was used to estimate the subjective value of real and hypothetical rewards at each delay. A hyperbolic discounting function provided a significantly better fit to individual participants' preferences than did an exponential function. No significant effect of reward type on degree of hyperbolic discounting or area under the discounting curves was detected. These findings offer some support for the validity of using hypothetical rewards to estimate discounting rates in substance-abusing and other populations, but caution is suggested because this support is gleaned from a failure to detect an effect of reward type. (PsycINFO Database Record (c) 2010 APA, all rights reserved)