Observing response acquisition: Preference for unpredictable appetitive rewards obtained under conditions predicted by DMOD.

To investigate the preference for unpredictable rewards predicted by the present author and J. T. Daly's (see record 1983-20275-001) modification, known as the DMOD model, of R. A. Rescorla and A. R. Wagner's (1972) previous model of reinforcement, the present author conducted 5 E-maze experiments with 144 male Holtzman rats. In Exps I–V, Ss were given a choice between receiving reward and nonreward in a situation in which stimuli were correlated with reward outcome (predictable situation) vs a situation in which the stimuli were uncorrelated with reward outcome (unpredictable situation). Preference for the unpredictable situation occurred under the following conditions: small (1 37-mg pellet) immediate rewards, small delayed (15-sec) rewards when the cues correlated with reward outcome were absent during the delay interval, large (15 pellets) immediate rewards when a difficult discrimination was required, and when the stimulus predicting nonreward was present at the choice point. Preference for the predictable situation was strongest if reinforcement was delayed and large or if the stimulus predicting reward was present at the choice point. A weaker preference for the predictable situation occurred if reinforcement was immediate, large, and required a simple discrimination or if reinforcement was large and delayed and the cues that correlated with reward outcome were absent during the delay interval. Findings support the predictions of the DMOD model of appetitive learning. (33 ref) (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Scalar expectancy theory and choice between delayed rewards.

In this article scalar expectancy theory is applied to variable and fixed delays to reward. It is assumed that all delays are represented in memory with scalar variance and that subjects choose between alternatives by sampling from the memory distributions associated with each and choosing the smaller delay. This simple scheme is shown to entail four common findings in the choice literature: (a) approximate matching of choice ratios to reward ratios (the matching law) when both alternatives are associated with variable delays scheduled with constant probability; (b) undermatching, in which choice is closer to indifference than matching, when both alternatives are variable but scheduled with uniform distributions; (c) overmatching, in which choice is more extreme than matching, when both alternatives are fixed delays; and (d) preference for variable delays scheduled with a constant probability over fixed delays. Overmatching and Weber's law are illustrated in experiments using the time-left procedure (Gibbon & Church, 1981). The preference for variable delay is demonstrated in this procedure, followed by study of a unique variable schedule of delays for which the theoretical account predicts, and the data confirm, the elimination of the preference for variability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Ingestion of environmentally contaminated Lake Ontario salmon by laboratory rats increases avoidance of unpredictable aversive nonreward and mild electric shock.

To determine what behavioral changes are caused by consumption of Lake Ontario salmon, a 30% diet of Lake Ontario or control Pacific Ocean salmon was fed to rats for 20 days. In Experiments 1 and 2 (preference-for-predictability E-maze test), rats fed Lake Ontario salmon developed a preference for predictable food rewards more quickly than did the control rats. In Experiments 3 (passive avoidance) and 4 (conditioned suppression), rats fed Lake Ontario salmon suppressed responding to food far more after the introduction of mild electric shocks than did control rats. All results supported the hypothesis that ingestion of Lake Ontario salmon, contaminated with polychlorinated biphenyls, mercury, lead, etc., increases the reactivity of rats to aversive events. The results were successfully simulated by DMOD, a mathematical model of learning, using the assumption that rats fed Lake Ontario salmon find unpredictable nonreward and mild shock more aversive. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fixed-ratio schedules increase generalized self-control: Preference for large rewards despite high effort or punishment.

Three experiments investigated the effects of FR reinforcement on generalized self-control involving high effort and punishment. In Experiment 1, rats received food in a runway for the completion of each round trip (continuous-reinforcement group) or every fifth round trip (FR group). Control rats received food at the same temporal intervals as these groups but without any instrumental requirement. When all rats were next given a series of choices between a large food reward requiring high lever force versus a small reward requiring low lever force, the FR rats showed the greatest self-control. In Experiments 2 and 3, rats were rewarded on a continuous or FR schedule followed by choice between a large food reward accompanied by intermittent shock vs a small or absent food reward without shock. The FR rats again showed the greatest self-control. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Noradrenaline and partial reinforcement in rats.

Lesions of the dorsal noradrenergic bundle produced by 6-hydroxydopamine, which depleted telencephalic noradrenaline by over 95%, were found to produce resistance to extinction in male albino Wistar rats trained on a continuously reinforced schedule for food reward but not to alter the extinction rate after partial reinforcement training. These results confirm the dorsal bundle extinction effect reported previously (S. T. Mason and S. D. Iversen, 1977) but suggest that it is confined to continuously reinforced situations. The pattern of results is consistent with a role for the dorsal bundle in attentional processes but appears to contradict the predictions required if the dorsal bundle were to have a role in frustrative nonreward. (45 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Transfer of value from S+ to S- in a simultaneous discrimination.

Value transfer theory has been proposed by L. von Fersen et al (see record 1991-29523-001) to account for transitive inference effects, in which following training on 4 simultaneous discriminations (A?+?B–, B?+?C–, C?+?D–, D?+?E–) pigeons showed a preference for B over D. According to this theory, some of the value of reinforcement acquired by each stimulus always associated with reinforcement (S+) transfers to the stimulus associated with nonreinforcement (S–). In the transitive inference experiment, C (associated with both reward and nonreward) could transfer less value to D than A (associated only with reward) could transfer to B. Support for value transfer theory was demonstrated in 2 experiments, involving a total of 20 pigeons, in which an S– presented in the context of an S+ was preferred over an S– presented in the context of a stimulus to which responses were sometimes reinforced (S±). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of acquisition training schedule on extinction and reinstatement of cocaine self-administration in male rats.

Partial reinforcement is known to increase resistance to extinction (Rn) relative to training with continuous reinforcement. This phenomenon, referred to as the partial reinforcement extinction effect, is one of the most robust in learning and conditioning studies. Experiment 1 investigated manipulations known to affect the partial reinforcement extinction effect and determined their possible relevance for drug use patterns. Male rats received intravenous cocaine self-administration training under partial reinforcement (FR-10) training or continuous reinforcement (FR-1) conditions with either a low (0.25 mg/kg infusion) or a high cocaine dose (1.00 mg/kg infusion). Animals were placed on an extinction (recurrent nonreward) schedule for 10 days (1-hr sessions) prior to being tested for cue-induced reinstatement (single 2-hr session). Experiment 2 involved acquisition of cocaine self-administration under FR-1 conditions of short training (15 days) or extended training (30 days) with a low dose (0.25 mg/kg infusion) or a medium dose (0.50 mg/kg infusion) of cocaine reward prior to extinction or reinstatement. Experiment 1 showed that rats trained with FR-10-high dose outcomes exhibited greater Rn than the remaining groups. Additionally, FR-10-high dose and FR-10-low dose rats were more likely to return to active drug seeking during the reinstatement test. In Experiment 2, rats trained under FR-1-medium dose conditions were more persistent during extinction following short acquisition training than comparable rats experiencing extended acquisition training. The reinstatement test was conducted following extinction, in which it was observed that overtraining under FR-1-medium dose reward schedules resulted in a decrease in the tendency to return to active drug seeking. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward magnitude and partial reinforcement effects in a single runway.

The partial reinforcement acquisition effect (PRAE) in running speeds and the frustration effect (activity following nonreward compared with reward) were measured simultaneously in an alley whose goal-box floor was a stabilimeter. Experimental groups of 9 male Charles River albino rats each received 50 or 100% reinforcement combined factorially with 3 magnitudes of reward (1, 3, or 9 pellets). A control group of 18 Ss was never rewarded. The size of the PRAE was a direct function of reward magnitude, and crossing of 50 and 100% curves was found for all alley segments, including the goal segment. The frustration effect (FE) was present by the 2nd day of training for the 3- and 9-pellet groups, and the size of the FE was directly related to reward magnitude. The present study is unique in that (a) the findings were free from the effects of reward contrast, (b) behavior antecedent to the goal indicated that incentive was effectively manipulated, and (c) an unrewarded control group was used. (31 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effort as a determinant of choice in rats.

Studied the effect of response effort on rate of responding and choice behavior in a 2-bar situation in 2 experiments with 17 male Sprague-Dawley rats. Exp I used a limited test session and tested at a single FR ratio. Exp II employed a free behavior situation, in which total daily water intake was obtained by barpressing. Ratio requirements beginning with continuous reinforcement and followed by sessions of FR 5, 10, 20, 40, 80, 120, and 160 were tested. Relations between response rate and preference behavior were schedule dependent. On low-ratio schedules the highest response rates and preference were for the light bar. As the ratios were increased, however, response rates became higher and preference was switched to the heavy bar. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Dopamine receptor blockade and extinction differentially affect behavioral variability.

Dopamine (DA) neurons respond to unexpected food delivery and are inhibited during the omission of expected reward. DA receptor blockade mimics some, but not all, aspects of nonreward (extinction) conditions. It was therefore of interest to ask whether DA receptor blockade produces extinction-like increases in behavioral variability in addition to its well-known operant response-suppressing effects. In the current experiment, rats were trained drug-free on an operant task in which they pressed on a keyboard. Two of the keys led to food on a continuous reinforcement schedule. Both response rates and behavioral variability were measured. Test day administration of D? and D? antagonists SCH23390 and raclopride, like extinction, suppressed responding but, unlike extinction, did not lead to an increase in variability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Elimination of vicarious reinforcement effects within an implicit reward situation.

The presence of cognitive evaluations of reward conditions in vicarious reward situations set in typical classrooms has been suggested in theoretical literature. The present study investigated this suggestion by examining the effect of free- versus no-talk conditions between target and peer Ss under vicarious reward conditions. Ss were 18 boys and 14 girls from a typical 4th-grade classroom. Data collected on a task involving reproduction of the letters of the alphabet showed that vicarious reinforcement effects occurred when verbal communication of reward enjoyment to nonreward Ss was possible but not when such communication was restricted. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Changes in self-stimulation preference as a function of incentive of alternative rewards.

Tested 24 male Wistar rats on a 23-hr food and water deprivation schedule for preference between lever pressing for electrical stimulation of the lateral hypothalamus and liquid rewards, which varied in taste and caloric content. Preference for self-stimulation diminished as the alternative liquid became more palatable: with a saccharine-glucose solution as the alternative, Ss showed an equal preference for hypothalamic stimulation and the liquid reward. In longer 6-hr tests, Ss deprived of water for 24 and 48 hr. initially showed a preference for self-stimulation over water but eventually satisfied their water deficit by pressing for water. Neglect of physiological needs during self-stimulation appears to be more a function of the alternatives available and the length of the test session than intrinsic properties of the brain stimulation. (French summary) (15 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The kinetics of choice: An operant systems analysis.

The behaving organism and its environment constitute interacting elements of a feedback system. Starting with a kinetic model of the effects of reinforcement on preference and incorporating feedback functions describing the effects of responding on reinforcement rate, a systems analysis of concurrent reinforcement schedule situations was developed. Correct predictions were generated of both the form of the acquisition of preference and of asymptotic preference levels for choice between probabilistic reinforcement schedules (i.e., VI and VR). A kinetic model was then formulated at the level of switching behavior. All predictions at the molar (preference) level followed also from this molecular analysis. In addition, the switching model correctly predicted a constant sum of transition probabilities (local switching rates) and a decrease in overall switching rate as both preference strength and the difference between reinforcement rates increased. The present systems analysis is seen as being unique among recent theoretical efforts concerned with schedules of reinforcement in its ability to describe molecular as well as molar properties of both transition-state and steady-state behavior. (41 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Does intrinsic motivation theory explain the adverse effects of rewards on immediate task performance?

In an effort to answer the question posed in the title, we assessed the effects of rewards on the immediate task performance of preschool children in two studies. Both studies had within-subjects, repeated measures designs, and both yielded highly consistent results showing a detrimental effect of reward on the Peabody Picture Vocabulary Test and on the Goodenough-Harris Draw-a-Man test. Performance decrements were confined to sessions in which subjects were rewarded; when rewarded subjects were shifted to nonreward, their performance improved dramatically. Although these studies were not concerned with the effects of reward on intrinsic motivation, the findings appear to present theoretical difficulties for current cognitive-motivational explanations of the adverse effects of material rewards on immediate task performance. An alternative viewpoint that material rewards can produce a temporary regression in psychological functioning is suggested. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Some effects of "suspicious" versus 'trusting" training schedules.

The structure of child training was translated into the paradigm of learning in a human engineering situation. The parent is conceived as having a role for the trainee (the child) to learn and employs a schedule of reinforcement. The "suspicious" training schedule involved the trainer always assuming an incorrect response has been given, and uses punitive action (electric shock); "trusting" schedule assumes a correct response, reward being a green light to proceed. Ss were college students; the task involved deciding which of 2 statements reflected greater psychopathology. S had the choice of revealing his selection or not. The hypothesis was that the S exposed to the "suspicious" routine would be more inclined to bring his response to the attention of the evaluater to correct the impression of being wrong, and to more readily learn to adopt trainers' frame of reference. In general, the results support the hypotheses. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of the nucleus accumbens core in impulsive choice, timing, and reward processing.

The present series of experiments aimed to pinpoint the source of nucleus accumbens core (AcbC) effects on delay discounting. Rats were trained with an impulsive choice procedure between an adjusting smaller sooner reward and a fixed larger later reward. The AcbC-lesioned rats produced appropriate choice behavior when the reward magnitude was equal. An increase in reward magnitude resulted in a failure to increase preference for the larger later reward in the AcbC-lesioned rats, whereas a decrease in the larger later reward duration resulted in normal alterations in choice behavior in AcbC-lesioned rats. Subsequent experiments with a peak timing (Experiments 2 and 3) and a behavioral contrast (Experiment 4) indicated that the AcbC-lesioned rats suffered from decreased incentive motivation during changes in reward magnitude (Experiments 2 and 4) and when expected rewards were omitted (Experiments 2 and 3), but displayed intact anticipatory timing of reward delays (Experiments 2 and 3). The results indicate that the nucleus accumbens core is critical for determining the incentive value of rewards, but does not participate in the timing of reward delays. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The signal-generated partial reinforcement extinction effect.

The five experiments reported here distinguish between memory of the reinforcement event just experienced on a preceding trial, called event-generated memory, and a reinforcement event memory activated by a signal associated with the memory, called signal-generated memory. In each experiment, a group of rats received discrimination training in Phase 1 to establish some stimulus as a signal for nonreward (N) and then consistent reinforcement (CRF) in Phase 2, in which the signal for N was presented on selected rewarded (R) trials. I hypothesized that this treatment would activate a memory of nonreward, S{N} and that this signal-generated S{N} would become a signal for reward much like event-generated S{N} does during conventional partial reinforcement (PRF) training. Rats that received this training had substantially higher resistance to extinction than did CRF and nonsignaled control groups, an effect named the signal-generated partial reinforcement extinction effect (PREE). The effect was shown to be due to presenting a signal associated with N in the CRF context and not to intrinsic properties of the signal or other characteristics of the training experience; the effect was also shown to be as large as the conventional PREE. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cocaine exposure causes long-term increases in impulsive choice.

In this study, the authors examined the long-term effects of prior exposure to cocaine on a delay-discounting task commonly used to measure impulsive choice. Male Long-Evans rats received daily intraperitoneal injections of 30 mg/kg cocaine HCl or saline for 14 days. Following 3 weeks of withdrawal, rats began training. On each trial, rats were given a choice between 2 levers. A press on 1 lever resulted in immediate delivery of a single 45-mg food pellet, and a press on the other resulted in delivery of 4 pellets after a delay period. Impulsive choice was defined as preference for the small immediate over the large delayed reward. Three months after treatment, cocaine-exposed rats displayed increased impulsive choice behavior. They also showed less anticipatory responding (entries into the food trough) during the delays prior to reward delivery, indicating that the enhanced impulsive choice in these rats may be related to deficits in bridging the delay between response and reward. These data demonstrate that cocaine exposure can cause enduring increases in impulsive choice behavior, consistent with observations in human subjects with drug addictions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Limbic lesions and the energizing, aversive, and inhibitory effects of non-reward in rats.

Tested 30 male albino Wistar rats with bilateral lesions in the amygdala, septum, hippocampus, stria terminalis, and fornix on a multiple reinforcement schedule in which barpressing in one component was associated with VI reinforcement (S+) and the other with extinction (S–). Responses on a 2nd lever turned off S– for 5-sec periods during the extinction component. All groups, with the exception of Ss with amygdaloid lesions exhibited behavioral contrast. Ss with hippocampal or fornical lesions showed greater resistance to extinction. Response rates on the lever that turned off S– were higher after stria terminalis and septal lesions, whereas lower rates were obtained from Ss with amygdaloid lesions. It is concluded that amygdaloid lesions attenuate the energizing and aversive effects of nonreward, septal and stria terminalis lesions increase the aversive effects, and hippocampal and fornical lesions interfere with the inhibitory effects of nonreward. (French summary) (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reward, distraction, and the overjustification effect.

A number of experiments have recently demonstrated that extrinsic constraints and rewards can produce lower levels of intrinsic interest in subsequent free-choice situations. This effect has been considered to be the result of a shift in the self-perceived locus of motivation from intrinsic to extrinsic but has also been explained as resulting from the distracting qualities of reward procedures. The latter hypothesis implies that reward and nonreward distractors will produce decreases in intrinsic interest and that these decreases will dissipate over multiple-trial procedures as a result of adaptation. On the other hand, the attribution explanation predicts that rewards or other extrinsic constraints will produce decreases in interest that are stable or strengthened over time. The present experiment, using 132 male and female undergraduates, involved manipulation of 3 levels of the reward/distraction variable (reward, nonreward/distraction, and a nonreward/no-distraction control) crossed with 3 levels of initial trial participation (10, 25, or 50 trials). The results indicate that rewards produced a constant decrease in interest over trials, consistent with the attribution explanation. While there was some evidence for a temporary disruption in intrinsic interest due to nonreward distraction, no support was obtained for a distraction interpretation of the effects of rewards on free-choice behavior. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)