Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods.

Since its establishment, psychology has struggled to find valid methods for studying thoughts and subjective experiences. Thirty years ago, Ericsson and Simon (1980) proposed that participants can give concurrent verbal expression to their thoughts (think aloud) while completing tasks without changing objectively measurable performance (accuracy). In contrast, directed requests for concurrent verbal reports, such as explanations or directions to describe particular kinds of information, were predicted to change thought processes as a consequence of the need to generate this information, thus altering performance. By comparing performance of concurrent verbal reporting conditions with their matching silent control condition, Ericsson and Simon found several studies demonstrating that directed verbalization was associated with changes in performance. In contrast, the lack of effects of thinking aloud was merely suggested by a handful of experimental studies. In this article, Ericsson and Simon's model is tested by a meta-analysis of 94 studies comparing performance while giving concurrent verbalizations to a matching condition without verbalization. Findings based on nearly 3,500 participants show that the “think-aloud” effect size is indistinguishable from zero (r = –.03) and that this procedure remains nonreactive even after statistically controlling additional factors such as task type (primarily visual or nonvisual). In contrast, procedures that entail describing or explaining thoughts and actions are significantly reactive, leading to higher performance than silent control conditions. All verbal reporting procedures tend to increase times to complete tasks. These results suggest that think-aloud should be distinguished from other methods in future studies. Theoretical and practical implications are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A quasi-qualitative investigation of strategies used in qualitative categorization.

This study investigates how analysts approach the task of initially categorizing qualitative data, what analysis strategies increase or decrease the testimonial validity of the categories, whether the data should be presented as a whole or in meaningful units, the effects of analysts' familiarity with the material, and the cognitive strategies associated with testimonial validity judgments. 30 upper level undergraduates (aged 19–42 yrs) analyzed essay data for "themes" while engaged in a think-aloud task. The essay writers then ranked the sets of categories in terms of overall quality and rated them on dimensions relevant to testimonial validity while thinking aloud. Think-aloud protocols for the best and worst approaches are examined. Implications for the use of qualitative research methods are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The suppression of exciting thoughts.

We examined how the suppression of an exciting thought influences sympathetic arousal as indexed by skin conductance level (SCL). Subjects were asked to think aloud as they followed instructions to think about or not to think about various topics. Experiment 1 showed that trying not to think about sex, like thinking about sex, elevates SCL in comparison to thinking about or not thinking about less exciting topics (e.g., dancing). Experiment 2 revealed that the suppression of the thought of sex yielded SCL elevation whether or not subjects believed their think-aloud reports would be private or public, and it also revealed that the effect dissipated over the course of a few minutes. Experiment 3 found such dissipation again but showed that subsequent intrusions of the suppressed exciting thought are associated with further elevations in SCL over 30 min. Because such an association was not found when subjects were trying to think about the exciting thought, it was suggested that the suppression of exciting thoughts might be involved in the production of chronic emotional responses such as phobias and obsessive preoccupations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Strategy use on standardized reading comprehension tests.

Strategy use and its impact on standardized reading test performance were investigated. High school students were randomly assigned to 2 groups, standardized test and main idea, with separate control and think-aloud conditions. In the standardized think-aloud group, students thought aloud while taking a portion of a reading test, consisting of passages accompanied by several questions each. In the main idea conditions, students read the same passages with the lower level questions removed and answered a single multiple-choice question about the main idea of each passage, having been told that the task was not a test. Both groups used strategies, and students in the standardized test condition made significantly greater use of strategies than students in the main idea condition. Significant between-group differences were found in use of rereading. In comparisons between the think-aloud and control conditions, thinking aloud was found to have a significant detrimental effect on students' ability to identify passage main ideas. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding unfamiliar words in young, young-old, and old-old adults: Inferential processing and the abstraction-deficit hypothesis.

Previous research suggests that older adults derive interpretations of unfamiliar words that are less precise than those of young adults (D. McGinnis & E. M. Zelinski, 2000). Thirty-one adults aged 18-37, 27 aged 65-74, and 28 aged 75-87 read passages containing unfamiliar words (1 per passage) and were asked to think aloud during reading. After reading each passage, participants selected meaning-relevant cues and rated the quality of 4 definition options. Compared with the 2 younger groups, the oldest group rated thematic and irrelevant definitions significantly higher, and their think-aloud protocols included more generalized inferences. Results pertaining to cue selection were not significant. Taken together, these results suggest that age differences in meaning derivation may be related to inferential processing that is overgeneralized, providing support for the abstraction-deficit hypothesis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reflections on introspection.

Reviews theoretical positions on introspection and its methods and the validity of the observations gained. Whether introspection is a method at all, whether one can be trained to perform introspection, and what the results of the process are, are discussed. Introspection as data and introspection as a method are distinguished; "data" refers to the statements or reports of introspection which can be used and measured as can any behavior. Introspection as method, however, involves "data" accessible only by observation. Several classifications of introspective methods are proposed (self-observation, self-reports, and thinking aloud) and related to historical definitions of psychology (e.g., W. Wundt, E. B. Titchener, and J. B. Watson). It is concluded that arguments about introspection and its methods are merely a question of what one is interested in; introspection is a technique which yields information that cannot be obtained in another way and arguments over its validity or definition are theoretically unhelpful. (21 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The cyclops and the twelve-eyed toad: William James and the unity-disunity problem in psychology.

One of the dominating features of the psychology and philosophy of William James is a deep concern with the monism–pluralism issue. Because of James's fascination with monistic and pluralistic perspectives and the related problem of unity and disunity, he would have been keenly interested in current concerns about the diversity of psychology. His advice regarding diversity might be especially informative because it would be based on a lifetime of thought devoted to the theoretical and practical problems attending monistic and pluralistic orientations. This article briefly examines James's pluralistic vision of the world especially as expressed in his mature philosophical treatises. Attention is then directed to the advice James might have offered to those who seek disciplinary unity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Windows on comprehension: Reading comprehension processes as revealed by two think-aloud procedures.

To address methodological questions regarding use of the think-aloud (TA) procedure and theoretical questions regarding the roles of prior knowledge and strategy use in reading comprehension, 24 college students each read 3 passages in 3 different presentation modes (marked TA, unmarked TA, and control) and answered essay comprehension questions. There was no effect of presentation mode on essay scores. TA comments were coded into 4 categories, 2 of which were significantly correlated with comprehension scores for marked but not unmarked passages. The authors conclude that the marked procedure elicited more veridical protocols. A second coding and analysis of the marked protocols showed that students who scored high on the comprehension test were more likely to have made many TA comments reflecting a "knowledge-transforming" approach to the text. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Following the Wrong Footsteps: Fixation Effects of Pictorial Examples in a Design Problem-Solving Task.

Two experiments examined possible negative transfer in nonexperts from the use of pictorial examples in a laboratory design problem-solving situation. In Experiment 1, 89 participants were instructed to "think aloud" and were assigned to 1 of 3 conditions: (a) control (standard instructions), (b) fixation (inclusion of a problematic example, describing its problematic elements), or (c) defixation (inclusion of a problematic example, with instructions to avoid using problematic elements). Negative transfer due to examples was measured both quantitatively and qualitatively through verbal protocols. Verbal protocols (N = 176) were analyzed for participants' reasons for reference to the examples. In Experiment 2, fixation to examples was evaluated in nonverbalizing participants (N = 60). Results of both experiments suggest that (a) although participants consulted the problem instructions, they tended to follow the examples even when they included inappropriate elements and (b) the fixation effects can be diminished with the use of defixating instructions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

We talk, therefore we think? A cultural analysis of the effect of talking on thinking.

The Western assumption that talking is connected to thinking is not shared in the East. The research examines how the actual psychology of individuals reflects these different cultural assumptions. In Study 1, Asian Americans and European Americans thought aloud while solving reasoning problems. Talking impaired Asian Americans' performance but not that of European Americans. Study 2 showed that participants' beliefs about talking and thinking are correlated with how talking affects performance, and suggested that cultural difference in modes of thinking can explain the difference in the effect of talking. Study 3 showed that talking impaired Asian Americans' performance because they tend to use internal speech less than European Americans. Results illuminate the importance of cultural understanding of psychology for a multicultural society. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The effects of 17 alpha-methyltestosterone, methandrostenolone, and nandrolone decanoate on the rat estrous cycle

In making clinical judgements, it is argued that midwives use 'shortcuts' or heuristics based on estimated probabilities to simplify the decision-making task. Midwives (n = 30) were given simulated patient assessment situations of high and low complexity and were required to think aloud. Analysis of verbal protocols showed that subjective probability judgements (heuristics) were used more frequently in the high than low complexity case and predominated in the last quarter of the assessment period for the high complexity case. 'Representativeness' was identified more frequently in the high than in the low case, but was the dominant heuristic in both. Reports completed after each simulation suggest that heuristics based on memory for particular conditions affect decisions. It is concluded that midwives use heuristics, derived mainly from their clinical experiences, in an attempt to save cognitive effort and to facilitate reasonably accurate decisions in the decision-making process.     

Review of A history of modern experimental psychology: From James and Wundt to cognitive science.

Reviews the book, A history of modern experimental psychology: From James and Wundt to cognitive science by George Mandler (see record 2007-05052-000). George Mandler, a longtime researcher in the area of memory and cognition, has gathered together his notes and selected bits from previous publications to assemble a new book cast as a brief history of the emergence of cognitive psychology. Mandler draws us to the positive impact Behaviourism had on the development of Cognitive Psychology. Mandler's book stands as an outline of the past, not a history. Its value rests with the perspective that comes from someone who has been thinking, researching and writing about topics central to Cognitive Psychology for over 40 years. He has been a witness to change, someone who has even participated in them, so his insights are valuable and directive. I would have enjoyed Mandler's book to a greater extent if, rather than chronologically reporting events, he had attempted to provide a gestalt of the emergence of cognitive psychology, one that would have located the articulate in the inarticulate of research practise and concept development in societies caught in the rift of redefinition. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Witmer, James, and Münsterberg.

Briefly comments on the L. T. Benjamin article (see record 1996-03525-010) regarding the contributions of Lightner Witmer to the field of applied psychology. The present author notes an incident documented in the correspondence between Hugo Münsterberg and William James as having occurred when the APA was to have its meeting at Harvard University in the early part of the 20th century. Witmer's publishing of critical comments about James' work are said to have so incensed Münsterberg, the Philosophy/Psychology Department Chair at Harvard, that he declared that the meetings would not be held at Harvard unless Witmer was expelled from APA. James intervened, and the meetings were held. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Overcoming inefficient reading skills.

Although experienced readers vary considerably in reading skill, skill measures often are uncorrelated with literal comprehension. According to the Compensatory-Encoding Model of reading, less automated reading skills and a small verbal working-memory capacity can be surmounted by slowing reading rate, pausing, looking back, and by other means. This important prediction is largely untested. In the present study, 76 readers were assessed on their levels of verbal efficiency. They were also recorded thinking aloud while reading text. Protocols were analyzed for evidence of compensation deployment. Analyses revealed that those with less automated reading skills deployed them more often. As expected, verbal efficiency was uncorrelated with literal comprehension but verbal working-memory capacity was positively correlated with inferential comprehension. Educational implications are derived. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Narrative comprehension in Alzheimer's disease: Assessing inferences and memory operations with a think-aloud procedure.

Objective: To further understanding of text comprehension abilities in Alzheimer's disease (AD), a think-aloud protocol was used to examine the role of inferencing and the memory operations used to produce inferences. Method: Twenty participants with AD and 20 cognitively healthy older adults (OA) read narratives, pausing to talk aloud after each sentence. A verbal protocol analysis developed by Trabasso and Magliano (1996a) was used to code participants' utterances into inferential and noninferential clause types; inferential statements were then coded to identify the memory operation used in their generation. Results: Compared with OA controls, the AD participants showed poorer story comprehension, d = 2.0, produced fewer inferences, d = .67, and were less skilled at providing explanations of story events, d = 1.27, and in using prior text information to explain outcomes, d = .90. The AD group also appeared to rely more on the activation of world knowledge, d = .58, which contributed to less effective inferences and produced more incoherent noninferential statements, d = 1.05. Poorer text comprehension for the AD group was associated with poorer verbal memory abilities, r's > .55, and poorer use of prior text events when producing explanatory inferences, r = .42. Conclusions: The findings suggest that the memory difficulties of the AD group appear to be an important cognitive factor interfering with their ability to integrate story events through the use of inferences and to create a global coherence to support text comprehension. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Criticality is perilous.

This essay reviews four longer arguments regarding the status of critical thinking on psychology. The commentary understands criticality as somehow sitting at the edge of theory itself. Given Psychology's perpetual identity crisis, the question of its edges and thus where criticality should nibble is brought to bear on the basic premises of the paper. Two of the papers are addressed in terms of the implied metapsychology they would presuppose and where that might break down (Teo's understanding of trauma). Two essays are addressed in terms of being a form of reading: one founded in adumbrations of desire and the other in a faith in science. The authors are asked if their work might not be informed by recent trends philosophy of science and through particular psychoanalytic lenses. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Strategic processing during comprehension.

This study addresses 3 questions: How flexible are readers when reading strategically? How is strategic processing affected by properties of the text? and Do some strategies lead to better text retention than others? Participants read short narratives and thought aloud with an instruction to either explain, predict, associate, or understand. The think-aloud protocols were used to predict sentence reading times for other participants who read silently with the same strategies. The results indicated that readers are capable of strategically controlling the inferences that they generate. However, strategic control comes at some cost in that it limits the resources devoted to other inferences. Furthermore, strategic processing is heavily constrained by a text. Text-based explanations occurred when there was an identifiable causal antecedent in the prior text. Knowledge-based inferences occurred when there were no antecedents and when new characters and objects were introduced. These effects occurred across reading strategies. Reading to explain led to better memory, but only when reading silently. (PsycINFO Database Record (c) 2010 APA, all rights reserved)