Distributed versus exclusive preference in discrete-trial choice.

Two experiments on discrete-trial choice examined the conditions under which pigeons would exhibit exclusive preference for the better of two alternatives as opposed to distributed preference (making some choices for each alternative). In Experiment 1, pigeons chose between red and green response keys that delivered food after delays of different durations, and in Experiment 2 they chose between red and green keys that delivered food with different probabilities. Some conditions of Experiment 1 had fixed delays to food and other conditions had variable delays. In both experiments, exclusive or nearly exclusive preference for the better alternative was found in some conditions, but distributed preference was found in other conditions, especially in Experiment 2 when key location varied randomly over trials. The results were used to evaluate several different theories about discrete-trial choice. The results suggest that exclusive preference for one alternative is a frequent outcome in discrete-trial choice. When distributed preference does occur, it is not the result of inherent tendencies to sample alternatives or to match response percentages to the values of the alternatives. Rather, distributed preference may occur when two factors (such as reinforcer delay and position bias) compete for the control of choice, or when the consequences for the two alternatives are similar and difficult to discriminate. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differential effects on effort discounting induced by inactivations of the nucleus accumbens core or shell.

The authors investigated the contribution of the nucleus accumbens (NAc) core and shell to effort-based decision making using a discounting procedure. Selection of 1 lever delivered a smaller, 2-pellet reward immediately, whereas the other lever delivered a 4-pellet reward after a fixed ratio of presses (2, 5, 10, or 20) that increased over 4 blocks of 10 discrete choice trials. Subsequent testing employed an equivalent delays procedure, whereby the relative delay to reward delivery after selection of either option was equalized. In well-trained rats, inactivation of the core, but not the shell, via infusion of GABA A/B agonists muscimol/baclofen reduced preference for the high-effort option under standard conditions and also when rats were tested using an equivalent delays procedure. However, inactivation of the core did not alter preference for 4-pellet versus 2-pellet rewards when the relative costs of each option were the same (1 press). Thus, the NAc core, but not the shell, appears to be part of a neural circuit that biases choice toward larger rewards associated with a greater effort cost. Furthermore, the contributions by the NAc core to this form of decision making can be dissociated from its role in delay discounting. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rapid acquisition of choice and timing in pigeons.

Pigeons were trained on a concurrent-chains procedure in which the initial link associated with the shorter terminal-link delay to food changed unpredictably across sessions. In the minimal-variation condition, delays were always 10 s and 20 s, whereas in the maximal-variation condition delays were generated pseudorandomly for each session. On some terminal links, food was withheld to obtain measures of temporal control. Measures of choice (log initial-link response ratios) and timing (start and stop times on no-food trials) showed temporal control and stabilized within the 1st half of each session. In the maximal-variation condition, choice was a nonlinear function of the log delay ratio, consistent with a categorical discrimination but contrary to models based on the matching law. Residuals from separate regressions of log response and log start and stop time ratios on log delay ratios were positively correlated. Overall, results support cognitive models that assume that initial-link choice is based on an all-or-none decision process, and that choice and timing are mediated by a common representation of delay. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pigeons prefer conditional stimuli over their absence: A comment on Roberts et al. (2009).

Recently, Roberts et al. (2009) have suggested that pigeons performing delayed matching-to-sample appear unwilling to request to see the sample again (or even for the first time) prior to choice, even if that choice would result in an increase in matching accuracy. In each of their four experiments, however, presentation (Experiments 3 & 4) or representation of the sample (Experiments 1 & 2) resulted in an added delay to reinforcement. Thus, the pigeons had to choose between an immediate reinforcer on about 50% of the trials and a delayed reinforcer on a significantly higher percentage of the trials. In the present research, when we equated the two alternatives for delay to reinforcement, we found that pigeons generally showed a significant preference for trials with a relevant sample over trials with an irrelevant sample. When the contingencies were reversed, most of the pigeons reversed their preference. Although these results do not present evidence for metacognition, they do show that pigeons are sensitive to the potential for a higher probability of reinforcement when delay to reinforcement is controlled. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Tests of an equivalence rule for fixed and variable reinforcer delays.

An adjusting procedure was used to measure choices between fixed and variable delays before reinforcement in 3 White Carneaux and 1 racing pigeon. The schedule of delays that followed a peck at the red key (the standard key) was constant throughout a condition, but it varied across conditions. A peck at the green key (the adjusting key) led to a delay whose duration was systematically increased or decreased depending on a S's choices. The purpose of these adjustments was to estimate an indifference point—a delay that equated a S's preferences for the 2 alternatives. Results were used to test a simple equation that predicts indifference points between fixed and variable delays. The results from 11 conditions were used to estimate the 2 parameters of the equation, and the equation was then used to predict, with what was considered reasonable success, the results from 10 other conditions. Results suggest that when Ss register their preferences with a single, brief response, choices between fixed and variable delays are predictable and consistent with a unidimensional scale of value. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Preference for delayed reward as a function of age, intelligence, and length of delay interval.

"Using a simple choice measure of preference for delayed reward on elementary school children… ranging in age from 5 to 12, and under five different lengths of delay interval, the following hypotheses were tested and confirmed: (a) preference for delayed reward is positively related to age; (b) positively to intelligence; (c) and negatively to length of the delay interval… . An additional finding was that subjects preferring the immediate reward tend to have more variable future time perspectives and that length of time perspective is slightly related to IQ, but not to age." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Human risky choice: Delay sensitivity depends on reinforcer type.

The present study was designed to help bridge the methodological gap between human and nonhuman animal research in delay-based risky choice. In Part 1, 4 adult human subjects made repeated choices between variable-time and fixed-time schedules of 30-s video clips. Both alternatives had equal mean delays of 15 s, 30 s, or 60 s. Three of 4 subjects strongly preferred the variable-delay alternative across all conditions. In Part 2, these 3 subjects were then provided pairwise choices between 2 variable-time schedules with different delay distributions. Subjects generally preferred the variable-delay distributions with a higher probability of short-reinforcer delays, consistent with accounts based on nonlinear discounting of delayed reinforcement. There was only weak correspondence between experimental results and verbal reports. The overall pattern of results is inconsistent with prior risky choice research with human subjects but is consistent with prior results with nonhuman subjects, suggesting that procedural differences may be a critical factor determining risk-sensitivity across species. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Temporal Discounting When the Choice Is Between Two Delayed Rewards.

The present experiments extend the temporal discounting paradigm from choice between an immediate and a delayed reward to choice between 2 delayed rewards: a smaller amount of money available sooner and a larger amount available later. Across different amounts and delays, the data were consistently well described by a hyperbola-like discounting function, and the degree of discounting decreased systematically as the delay to the sooner reward increased. Three theoretical models (the elimination-by-aspects, present-value comparison, and common-aspect attenuation hypotheses) were evaluated. The best account of the data was provided by the common-aspect attenuation hypothesis, according to which the common aspect of the choice alternatives (i.e., the time until the sooner reward is available) receives less weight in the decision-making process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choice between fixed and variable delays to reinforcement in the adjusting-delay procedure and concurrent chains.

A generalization of the contextual choice model ([CCM]; R. C. Grace, see record 1994-28397-001) is presented that describes preference between fixed and variable schedules in concurrent chains and the adjusting-delay procedure. In Exp 1A and 1B, fixed-delay indifference values for pigeons for a series of variable-delay schedules were obtained, using the adjusting-delay procedure, to test a Weber's law invariance and the effectiveness of several models. These data were then used to generate, in Exp 2, pairs of fixed and variable schedules that should have been equally preferred in concurrent chains. As predicted, deviation from indifference was small. These results indicate that (a) the determiners of value in the 2 procedures are the same, (b) the delay of reinforcement gradient is best represented as a power function with a delay threshold, and (c) a single model (CCM) can accurately describe data in the 3 most popular operant choice paradigms. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cocaine exposure causes long-term increases in impulsive choice.

In this study, the authors examined the long-term effects of prior exposure to cocaine on a delay-discounting task commonly used to measure impulsive choice. Male Long-Evans rats received daily intraperitoneal injections of 30 mg/kg cocaine HCl or saline for 14 days. Following 3 weeks of withdrawal, rats began training. On each trial, rats were given a choice between 2 levers. A press on 1 lever resulted in immediate delivery of a single 45-mg food pellet, and a press on the other resulted in delivery of 4 pellets after a delay period. Impulsive choice was defined as preference for the small immediate over the large delayed reward. Three months after treatment, cocaine-exposed rats displayed increased impulsive choice behavior. They also showed less anticipatory responding (entries into the food trough) during the delays prior to reward delivery, indicating that the enhanced impulsive choice in these rats may be related to deficits in bridging the delay between response and reward. These data demonstrate that cocaine exposure can cause enduring increases in impulsive choice behavior, consistent with observations in human subjects with drug addictions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Application of a cognitive-distance model to learning in a simulated travel task.

A cognitive-distance model for choice, obtained by specializing a general class of models for categorization, was tested in a situation simulating the task of controlling speed of a vehicle in tasks defined by different relations between speed and probability of delay. Subjects exhibited significant learning whenever delay schedules permitted greater-than-chance performance, but on the average they did not approach optimal performance in the sense of choosing speeds so as to maximize distance attained in allowed time. Evidence was obtained that subjects encoded information about probabilities of delay and distributions of distance attained at different speeds quite accurately in memory and that suboptimal performance was due primarily to imperfect discrimination among representations of choice alternatives on a cognitive scale of expected distance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influences of delay and rate of reinforcement on discrete-trial choice.

An adjusting procedure was used to measure 4 White Carneaux pigeons' preferences among alternatives that differed in the duration of a delay before reinforcement and of an intertrial interval (ITI) after reinforcement. In most conditions, a peck at a red key led to a fixed delay, followed by reinforcement, a fixed ITI, and then the beginning of the next trial. A peck at a green key led to an adjustable delay, reinforcement, and then the next trial began without an ITI. The purpose of the adjusting delay was to estimate an indifference point, or a delay that made an S approximately indifferent between the 2 alternatives. As the ITI for the red key increased from 0 to 60 sec, the green-key delay at the indifference point increased systematically, but only slightly. The fact that there was some increase showed that Ss' choices were controlled by more than simply the delay to the next reinforcer. An analysis that ignored reinforcement rate but that considered the delays between a choice response and the reinforcers on subsequent trials was able to account for most of the obtained increases in green-key delays. It is concluded that in this type of discrete-trial situation, rate of reinforcement exerts little or no control over choice behavior. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD.

To investigate the preference for unpredictable rewards predicted by the present author and J. T. Daly's (see record 1983-20275-001) modification, known as the DMOD model, of R. A. Rescorla and A. R. Wagner's (1972) previous model of reinforcement, the present author conducted 5 E-maze experiments with 144 male Holtzman rats. In Exps I–V, Ss were given a choice between receiving reward and nonreward in a situation in which stimuli were correlated with reward outcome (predictable situation) vs a situation in which the stimuli were uncorrelated with reward outcome (unpredictable situation). Preference for the unpredictable situation occurred under the following conditions: small (1 37-mg pellet) immediate rewards, small delayed (15-sec) rewards when the cues correlated with reward outcome were absent during the delay interval, large (15 pellets) immediate rewards when a difficult discrimination was required, and when the stimulus predicting nonreward was present at the choice point. Preference for the predictable situation was strongest if reinforcement was delayed and large or if the stimulus predicting reward was present at the choice point. A weaker preference for the predictable situation occurred if reinforcement was immediate, large, and required a simple discrimination or if reinforcement was large and delayed and the cues that correlated with reward outcome were absent during the delay interval. Findings support the predictions of the DMOD model of appetitive learning. (33 ref) (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Choice between fixed and variable delays with different reward amounts.

Choice between a variable and a fixed food source was studied in pigeons. The variable source yielded different durations of access to food (changed across experimental conditions) uniquely associated with equiprobable delays of either 20 sec or 60 sec, timed from the choice point. A comparison, elapsing alternative (time-left procedure [J. Gibbon and R. M. Church; see PA, Vol 67:2812]), was associated with fixed access to feeder after a 60-sec delay, timed from the beginning of the trial. Preference was best characterized as reflecting an average of the 2 local rates of reinforcement in the variable alternative (the average of the ratio of each standard amount and its associated delay). The averaging rule characterized initial choice link data well, but terminal link responding showed that Ss also remembered the 2 delay–amount combinations separately. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Delay of punishment and choice behavior in the rat.

Trained 4 groups of female Sprague-Dawley albino rats (N = 24) to press each of 2 bars for reward delayed 120 sec. A different set of distinctive cues prevailed between bar press and food depending on which bar was pressed. For experimental Ss shocks were then introduced following each response to the preferred bar. The response-shock interval was 1.5, 12, or 96 sec. Experimental Ss' preference shifted from the shock bar; preference remained the same for no-shock controls. The rate of changes was a negative function of delay of shock, but final percentage of responses to the nonshock bar was the same for all delays. Results support the hypothesis that delay of punishment as such has no effect on choice at asymptote. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fixed and variable ratios and delays: Further tests of an equivalence rule.

A discrete-trial procedure was used to measure 4 White Carneaux pigeons' choices between FR and VR schedules and between fixed and variable delays before reinforcement. A peck at a green key produced a reinforcement schedule that was constant within a condition but varied across conditions. A peck at a red key produced a ratio schedule (or, in other conditions, a simple delay) whose size was increased or decreased many times a session, depending on the S's previous choices. The purpose of these adjustments was to estimate an indifference point—a ratio size (or delay duration) at which the S chose each key about equally often. The results were used to test the present author's (see record 1985-19333-001) equivalence rule for choices between fixed and variable schedules. This rule predicted the major trends in the obtained indifference points from both ratio and delay conditions, but better predictions were generated with a more complex equation that included parameters reflecting the Ss' sensitivities to delay of reinforcement and to events of different probabilities. It is concluded that a successful equivalence rule must include parameters that can be adjusted to describe the effects of delay and probability in a given experimental setting. Once these parameters are estimated, however, choices involving both fixed and variable delays and FRs and VRs can be accurately predicted with the same equation. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An argument against dual valuation system competition: Cognitive capacities supporting future orientation mediate rather than compete with visceral motivations.

The dynamic inconsistency of preference is well documented in behavioral research, but its basis remains controversial. In this article, the authors summarize recent functional MRI (fMRI) work in the domain of intertemporal choice, specifically considering evidence bearing on the hypothesis that delay discounting in humans is determined by competition between an evolutionarily older system that discounts precipitously with delay (System 1) and a newer system that exhibits very little discounting (System 2). The authors argue that neuroimaging evidence does not support the hypothesized separate and competing value systems. While it is clear that the sophisticated cognitive capacities that lead to greater valuation of larger later alternatives (e.g., selective attention and self-signaling) depend critically on neocortical structures, these capacities affect intertemporal choice through mediation of (rather than competition with) older cortical and subcortical structures central to reward and motivation. Taken together, neuroimaging evidence supports the alternative hypothesis that intertemporal choice is guided by a single valuation system. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Now or later? Striatum and insula activation to immediate versus delayed rewards.

Neuroimaging studies on delay discounting tasks that use reward delays ranging from minutes to days have implicated the insula and striatum in the processing of intertemporal decisions. This study aimed at assessing whether these brain regions would also be involved in decision-making when subjects have to wait through the delays within the range of seconds. Using functional MRI in 13 healthy volunteers, we repeatedly presented monetary options with delays that differed within the range of multiple seconds. Using a region-of-interest approach, we found significant activation in the bilateral anterior insula and striatum when subjects chose either the immediate (smaller) or delayed (larger) option. In particular, insular activation was observed after the response and the delay, when the outcome of the immediate or the delayed choice was shown. Significantly greater activation was observed in the ventroanterior striatum while subjects chose the immediate, as opposed to the delayed, options and also after receiving the outcome of waiting through the longer delay option. The evidence presented here indicates that both the ventral striatum and the insula are involved in the processing of choosing delay options as well as the consequences of choices with delays in the seconds' range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)