Influences of delay and rate of reinforcement on discrete-trial choice.

An adjusting procedure was used to measure 4 White Carneaux pigeons' preferences among alternatives that differed in the duration of a delay before reinforcement and of an intertrial interval (ITI) after reinforcement. In most conditions, a peck at a red key led to a fixed delay, followed by reinforcement, a fixed ITI, and then the beginning of the next trial. A peck at a green key led to an adjustable delay, reinforcement, and then the next trial began without an ITI. The purpose of the adjusting delay was to estimate an indifference point, or a delay that made an S approximately indifferent between the 2 alternatives. As the ITI for the red key increased from 0 to 60 sec, the green-key delay at the indifference point increased systematically, but only slightly. The fact that there was some increase showed that Ss' choices were controlled by more than simply the delay to the next reinforcer. An analysis that ignored reinforcement rate but that considered the delays between a choice response and the reinforcers on subsequent trials was able to account for most of the obtained increases in green-key delays. It is concluded that in this type of discrete-trial situation, rate of reinforcement exerts little or no control over choice behavior. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Molar function depends on molecular structure of behavior.

Pigeons chose between green side keys, then waited a shorter or longer time before pecking a center key, and finally chose between red side keys. Two successive left choices (to green and then red) with a shorter wait intervening, or 2 successive right choices with a longer wait intervening, were intermittently reinforced with food. The 2 required waiting times and the relative frequency of reinforcement for the shorter reinforced pattern were varied. Molar preference, defined in terms of coherent responses that satisfied the molecular reinforcement contingency, conformed to the highly adaptive matching level, but molar preference, defined in terms of incoherent responses, did not. The molar matching result therefore generalizes to responses with complex molecular structures provided that analyses distinguish between coherent and incoherent responses. The results are compatible with the idea that awareness can facilitate adaptation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Pigeons' spatial memory: IV. Effects of intertrial interval manipulations on delayed matching of key location.

In Exp I, the effect of intertrial interval (ITI) duration on 3 Silver King pigeons' delayed matching of key location was investigated. Trials began with a ready stimulus (brief operation of the grain feeder); then one randomly chosen key from a 3?×?3 matrix was lit briefly as the sample. After a delay (retention interval) of 1, 4 or 8 sec, the sample key was lit again, along with another key. A peck at the key that had served as the sample (correct comparison) produced food reinforcement, whereas an incorrect peck led directly to the ITI. The ITI was 2.5 or 25 sec. Matching accuracy was lower at the shorter ITI and was linearly related to the log of the ratio of the ITI to the delay interval. In Exp II, noncontingent food reinforcement presented during the 25-sec ITI lowered matching accuracy of 5 Ss. In Exp III, reinforcement was given for pecking a key (correct, incorrect, or irrelevant) presented during the ITI. Reinforcement for pecking any key had a facilitative effect on matching accuracy. Results are discussed in relation to models of spatial memory, the apparent parallels between processes in delayed matching and classical conditioning, and the notion that expectancy of reinforcement during the matching trial facilitates accurate choice of the correct key. (French abstract) (37 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Discrimination of temporal relations by pigeons.

In 4 experiments, pigeons were tested on a duration comparison task involving the successive presentation of two visual stimuli that varied in duration from trial to trial. Following presentation of the durations, 2 choice keys were lit, and reinforcement for choices was based on the temporal relation between durations of the pair. In Exp 1, the range of durations was varied over conditions. Responding changed as an orderly function of the ratio of the 2 durations. There was a decrease in discrimination accuracy as average duration increased over conditions. In Exp 2, Ss were transferred from a task involving spatially differentiated choices to one involving hue-differentiated choices. Performance was similar to that of Exp 1. In Exp 3, Ss were exposed to a single duration range that included many durations from the 4 ranges of Exp 1. Discrimination accuracy was comparable on 3 problem duration categories within this range and declined slightly for problems in the 4th and longest category. Manipulation of absolute reinforcement rate in Exp 4 resulted in no change in discrimination accuracy, suggesting that the decline in accuracy over conditions of Exp 1 could not be attributed to decreases in reinforcement rate that accompanied lengthier durations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fixed and variable ratios and delays: Further tests of an equivalence rule.

A discrete-trial procedure was used to measure 4 White Carneaux pigeons' choices between FR and VR schedules and between fixed and variable delays before reinforcement. A peck at a green key produced a reinforcement schedule that was constant within a condition but varied across conditions. A peck at a red key produced a ratio schedule (or, in other conditions, a simple delay) whose size was increased or decreased many times a session, depending on the S's previous choices. The purpose of these adjustments was to estimate an indifference point—a ratio size (or delay duration) at which the S chose each key about equally often. The results were used to test the present author's (see record 1985-19333-001) equivalence rule for choices between fixed and variable schedules. This rule predicted the major trends in the obtained indifference points from both ratio and delay conditions, but better predictions were generated with a more complex equation that included parameters reflecting the Ss' sensitivities to delay of reinforcement and to events of different probabilities. It is concluded that a successful equivalence rule must include parameters that can be adjusted to describe the effects of delay and probability in a given experimental setting. Once these parameters are estimated, however, choices involving both fixed and variable delays and FRs and VRs can be accurately predicted with the same equation. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Long-term retention of a complex operant in pigeons.

Examined the effects of a 60-day retention interval on sequence performance when White Carneaux pigeons were required to peck each of 2 keys 4 times in any order for reinforcement. In Exp I, with 20 Ss, it was shown that if the retention interval contained no interpolated experimental experience, it had no effect on sequence performance. If Ss pecked a key for food on a VI schedule during the retention interval, sequence disruption occurred. In Exp II, with 10 Ss, it was found that variations in the location and color of a key pecked during VI had no effect on disruption. In Exp III, with 20 Ss, it was found that disruption did not occur when Ss were simply placed in the experimental chamber during the retention interval, or given response-independent food, or given VI reinforcement for hopping on a foot treadle. In Exp IV, with 15 Ss, disruption was found even when interpolated keypeck training demanded a temporal pattern different from what had occurred on the sequence task. In Exp V, with 5 Ss, it was demonstrated that disruption could be prevented if Ss were exposed to an alternation of the sequence and VI procedures during initial acquisition. (40 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Tests of an equivalence rule for fixed and variable reinforcer delays.

An adjusting procedure was used to measure choices between fixed and variable delays before reinforcement in 3 White Carneaux and 1 racing pigeon. The schedule of delays that followed a peck at the red key (the standard key) was constant throughout a condition, but it varied across conditions. A peck at the green key (the adjusting key) led to a delay whose duration was systematically increased or decreased depending on a S's choices. The purpose of these adjustments was to estimate an indifference point—a delay that equated a S's preferences for the 2 alternatives. Results were used to test a simple equation that predicts indifference points between fixed and variable delays. The results from 11 conditions were used to estimate the 2 parameters of the equation, and the equation was then used to predict, with what was considered reasonable success, the results from 10 other conditions. Results suggest that when Ss register their preferences with a single, brief response, choices between fixed and variable delays are predictable and consistent with a unidimensional scale of value. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Same-different texture discrimination and concept learning by pigeons.

Investigated the acquisition and transfer of a "same-different" conditional discrimination using multidimensional visual texture stimuli in pigeons. Using a choice task, 4 pigeons were reinforced for discriminating different displays, created from aggregated differences in element color or shape, from uniform displays, in which all elements were identical. Discrimination of these 2 display types was readily acquired by the pigeons when they were required to locate and peck the contrasting target region of the different displays. The pigeons showed high levels of discrimination transfer to novel texture stimuli both during acquisition and in 2 subsequent transfer tests. The results suggest that pigeons may be able to learn a generalized same-different concept when promoted by the use of large numbers of multielement stimuli during training. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Training with controlled reinforcer density: Implications for models of timing.

Pigeons discriminated between the 1st and 2nd halves of a trial. Trial duration was varied both within and between sessions in a 2-alternative free-operant psychophysical choice procedure. Two keys were illuminated either both red or both green at the beginning of a trial. Half the trials were long (red keys), and half were short (green keys). Session duration was kept constant across conditions. Left-key responding was reinforced only during the 1st half of a trial, and right-key responding was reinforced only in the 2nd half of a trial. Contrary to predictions of the behavioral theory of timing (P. R. Killeen and J. G. Fetterman, see record 1988-28629-001), estimates of pacemaker period increased with increases in the trial duration despite constancy of reinforcement rate. Weber fractions were relatively constant across timed intervals, consistent with Weber's law. The present experiment is discussed with regard to implications for current theories of timing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Simon Effect in Pigeons.

Pigeons pecked left versus right keys contingent upon the color presented at 1 of those locations. Spatial-response latencies were shorter when the color appeared at the same location as the required response than at the opposite location. This Simon effect occurred when the stimulus on the alternative key was constant, varied from trial to trial, or changed when the color cue appeared and when the reinforcement probability for correct responses was the same on corresponding as on noncorresponding trials. Humans performing the same task by touching the keys also showed the Simon effect. These findings demonstrate that for pigeons, too, a relevant symbolic cue activates a spatial code that produces faster responses at the location corresponding with the activated code. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The dynamics of conditioning and extinction.

Pigeons responded to intermittently reinforced classical conditioning trials with erratic bouts of responding to the conditioned stimulus. Responding depended on whether the prior trial contained a peck, food, or both. A linear persistence–learning model moved pigeons into and out of a response state, and a Weibull distribution for number of within-trial responses governed in-state pecking. Variations of trial and intertrial durations caused correlated changes in rate and probability of responding and in model parameters. A novel prediction—in the protracted absence of food, response rates can plateau above zero—was validated. The model predicted smooth acquisition functions when instantiated with the probability of food but a more accurate jagged learning curve when instantiated with trial-to-trial records of reinforcement. The Skinnerian parameter was dominant only when food could be accelerated or delayed by pecking. These experiments provide a framework for trial-by-trial accounts of conditioning and extinction that increases the information available from the data, permitting such accounts to comment more definitively on complex contemporary models of momentum and conditioning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Task dependence of slowing after pyramidal lesions in monkeys.

Studied the effect of lesions in a medullary pyramid in 2 experiments with 9 vervet monkeys (Cercopithecus aethiops). (a) Ss were trained to press a key with a rear-projected circle, presented together with different numbers of keys with ellipses. Two to eight choices were presented in random sequence. Discrimination was measured with a titration schedule, and short choice reaction times were reinforced selectively. Choice reaction time and the function relating it to the number of choices were unaffected by lesions reducing the area of a transverse section of the pyramid up to 93%. (b) A repetitive response controlled by an FR schedule was performed simultaneously with a holding response with the other hand, and the 2 hands changed function after each food pellet. Lesions reducing the pyramid only 25% increased interresponse times of the phasic response with the contralateral hand. Exp I and previous studies are interpreted to indicate that slowing after pyramidal lesions is dependent on behavioral context. Exp II showed that a simple repetitive movement is affected when the rate is high. The static reponse was more clearly affected that the phasic response, which indicates a role for the pyramidal tract in posture. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Memory strategies in pigeons' performance of a radial-arm-maze analog task.

Pigeons were presented with an analog of a radial-arm-maze task involving 5 response keys. In Experiment 1, pigeons were exposed to either a linear (L) or a two-dimensional matrix (M) array of the 5 keys, and the keys were either each of a distinctive hue (H) or were all white (W). Acquisition was facilitated both by M and by H. In Experiment 2, increasing the number of pecks that constituted a choice from 5 to 20 for half of the LW birds had little effect on performance. In Experiment 3, a delay was interpolated after a number of choices that varied across trials. All groups except the LW 5-peck group showed little disruption in performance with increasing delay (up to a maximum of 1 hr). Inverted-U-shaped point-of-delay interpolation functions were found for each group. The data suggest that pigeons retrospectively coded choices already made when the delay occurred early in the trial and prospectively coded choices yet to be made when the delay occurred late in the trial. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Establishment and generalization of non-compliant responding in children.

Evaluated the performance of 3 nursery-school children in a 2-option keypressing task. An instruction was given in each trial to respond on 1 of the keys, as designated by color or form cues. Several reinforcement schedules for compliance were presented. High rates of noncompliance were produced by contingencies favoring this choice. Noncompliant responding showed persistent generalization to new color and form cues and in the presence of a new E, even though contingencies did not favor generalization. Noncompliance also persisted for many sessions when contingencies were revised to favor compliance. Results have implications for the study of generalized imitation, generalized compliance, and the problem of noncompliance in problem children. (French summary) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mechanisms underlying the effects of unsignaled delayed reinforcement on key pecking of pigeons under variable-interval schedules

Three experiments were conducted to test an interpretation of the response-rate-reducing effects of unsignaled nonresetting delays to reinforcement in pigeons. According to this interpretation, rates of key pecking decrease under these conditions because key pecks alternate with hopper-observing behavior. In Experiment 1, 4 pigeons pecked a food key that raised the hopper provided that pecks on a different variable-interval-schedule key met the requirements of a variable-interval 60-s schedule. The stimuli associated with the availability of the hopper (i.e., houselight and keylight off, food key illuminated, feedback following food-key pecks) were gradually removed across phases while the dependent relation between hopper availability and variable-interval-schedule key pecks was maintained. Rates of pecking the variable-interval-schedule key decreased to low levels and rates of food-key pecks increased when variable-interval-schedule key pecks did not produce hopper-correlated stimuli. In Experiment 2, pigeons initially pecked a single key under a variable-interval 60-s schedule. Then the dependent relation between hopper presentation and key pecks was eliminated by arranging a variable-time 60-s schedule. When rates of pecking had decreased to low levels, conditions were changed so that pecks during the final 5 s of each interval changed the keylight color from green to amber. When pecking produced these hopper-correlated stimuli, pecking occurred at high rates, despite the absence of a peck-food dependency. When peck-produced changes in keylight color were uncorrelated with food, rates of pecking fell to low levels. In Experiment 3, details (obtained delays, interresponse-time distributions, eating times) of the transition from high to low response rates produced by the introduction of a 3-s unsignaled delay were tracked from session to session in 3 pigeons that had been initially trained to peck under a conventional variable-interval 60-s schedule. Decreases in response rates soon after the transition to delayed reinforcement were accompanied by decreases in eating times and alterations in interresponse-time distributions. As response rates decreased and became stable, eating times increased and their variability decreased. These findings support an interpretation of the effects of delayed reinforcement that emphasizes the importance of hopper-observing behavior.     

Local Proactive Interference in Delayed Matching to Sample: The Role of Reinforcement.

Discriminability in delayed matching to sample was lower when the samples on consecutive trials differed compared with when samples on consecutive trials were the same. This local proactive interference occurred when correct choices on the previous trial were reinforced but not when correct choices on the previous trial were not reinforced. When the choice on the previous trial was incorrect, discriminability was higher on different consecutive trials than on same trials. These effects were amplified by varying the ratio of reinforcers for correct choices, as predicted by a model that attributes local proactive interference to an interaction between control by the sample on the current trial and the influence of reinforcers for correct choices on previous trials. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Superordinate category formation in pigeons: Association with a common delay or probability of food reinforcement makes perceptually dissimilar stimuli functionally equivalent.

Training associated pairs of perceptually dissimilar stimulus classes with a common delay or probability of food reinforcement in pigeons. Then, different choice responses were trained to 1 component class in each pair. In a choice test, the untrained class in each pair occasioned the same response as did the choice-trained class. In a 3rd experiment, 2 classes had reinforcement delays of 1 s and 15 s, respectively, and 2 other classes had reinforcement probabilities of 0.1 and 0.9. Then, 1 choice response was reinforced to a class previously associated with a better condition of reinforcement (e.g., 1-s delay or 1.0 probability), and a different response was reinforced to a class previously associated with a worse condition of reinforcement (0.1 probability or 0-s delay). Testing with all classes suggested that categorization was based on the relative reinforcement or hedonic value and not on the parametric details of reinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Choice behavior in transition: Development of preference with ratio and interval schedules.

In Exp 1, the choice responses of 8 pigeons were observed during 50 periods of transition. Each condition began with equal probabilities of reinforcement on 2 response keys and switched to unequal probabilities. With the ratio of the 2 probabilities held constant, preference for the higher probability developed more rapidly when the 2 probabilities were high than when they were low. In Exp 2, each condition began with 2 equal variable interval (VI) schedules, but later 1 key delivered 60%, 75%, or 90% of the reinforcers. The rate of approach to asymptotic performance was roughly the same with all 3 reinforcement percentages. These and previous results pose difficulties for some well-known models of acquisition, but the results are well described by a simple model that states that the strength of each response is independently increased by reinforcement and decreased by nonreinforcement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

New Method for Prediction of Loads in Steel Reinforced Soil Walls

The paper describes a new working stress design methodology introduced by the writers for geosynthetic reinforced soil walls (K-Stiffness Method) that is now extended to steel reinforced soil walls. A large database of full-scale steel reinforced soil walls (a total of 20 fully instrumented wall sections) was used to develop the new design methodology. The effects of global wall stiffness, soil strength, reinforcement layer spacing, and wall height were investigated. Results of simple statistical analyses using the ratio of measured to predicted peak reinforcement loads (i.e., method bias) demonstrate the improved prediction accuracy. The AASHTO Simplified Method results in an average method bias of 1.1 with a coefficient of variation (COV) of 45%, whereas the proposed K-Stiffness Method results in an average bias of 0.95 and a COV of 32%. Soil strength was found to have limited influence on reinforcement loads for steel reinforced soil walls, especially for high shear strength soils, while global wall stiffness and wall height had a major influence on reinforcement loads.     

A new consequence of Simpson's paradox: Stable cooperation in one-shot prisoner's dilemma from populations of individualistic learners.

Theories of choice in economics typically assume that interacting agents act individualistically and maximize their own utility. Specifically, game theory proposes that rational players should defect in one-shot prisoners' dilemmas (PD). Defection also appears to be the inevitable outcome for agents who learn by reinforcement of past choices, because whatever the other player does, defection leads to greater reinforcement on each trial. In a computer simulation and 4 experiments, the authors show that, apparently paradoxically, when players' choices are correlated by an exogenous factor (here, the cooperativeness of the specific PD chosen), people obtain greater average reinforcement for cooperating, which can sustain cooperation. This effect arises from a well-known statistical paradox, Simpson's paradox. The authors speculate that this effect may be relevant to aspects of real-world human cooperative behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)