Receiver operating characteristics (ROCs) in recognition memory: A review.

Receiver operating characteristic (ROC) analysis is being used increasingly to examine the memory processes underlying recognition memory. The authors discuss the methodological issues involved in conducting and analyzing ROC results, describe the various models that have been developed to account for these results, review the behavioral empirical literature, and assess the models in light of those results. The empirical literature includes studies of item recognition, relational recognition (e.g., source and associative tests), as well as exclusion and remember-know tasks. Nine empirical regularities are described, and a number of unresolved empirical issues are identified. The results indicate that several common classes of recognition models, such as pure threshold and pure signal detection models, are inadequate to account for recognition memory, whereas several hybrid models that incorporate a signal detection-based process and a threshold recollection or attention process are in better agreement with the results. The results indicate that there are at least 2 functionally distinct component/processes underlying recognition memory. In addition, the ROC results have various implications for how recognition memory performance should be measured. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Regularities of source recognition: ROC analysis.

Source memory has become the focus of a growing number of investigations in a variety of fields. An appropriate model for source memory is, therefore, of increasing importance. A simple 2-dimensional signal-detection model of source recognition is presented. The receiver operating characteristics (ROCs) obtained from 3 experiments are then used to test the model. The data demonstrate 3 regularities: convex ROCs, z-ROCs with linear slopes of 1.00, and slightly concave z-ROCs. Two of these regularities support the model. The 3rd requires a revision of the model. This revised model is fitted to the data. The implications of these regularities for other theories are also discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Separating mnemonic process from participant and item effects in the assessment of ROC asymmetries.

One of the most influential findings in the study of recognition memory is that receiver operating characteristic (ROC) curves are asymmetric about the negative diagonal. This result has led to the rejection of the equal-variance signal detection model of recognition memory and has provided motivation for more complex models, such as the unequal-variance signal detection and dual-process models. Here, the authors test the possibility that previous demonstrations of ROC asymmetry do not reflect mnemonic process but rather reflect distortions due to averaging data over items. Application of a hierarchical unequal-variance signal detection model reveals that asymmetries are in fact a real phenomenon and do not reflect distortions from averaging data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Dual-Process Models of Associative Recognition in Young and Older Adults: Evidence From Receiver Operating Characteristics.

In 3 experiments, young and older adults studied lists of unrelated word pairs and were given confidence-rated item and associative recognition tests. Several different models of recognition were fit to the confidence-rating data using techniques described by S. Macho (2002, 2004). Concordant with previous findings, item recognition data were best fit by an unequal-variance signal detection theory model for both young and older adults. For both age groups, associative recognition performance was best explained by models incorporating both recollection and familiarity components. Examination of parameter estimates supported the conclusion that recollection is reduced in old age, but inferences about age differences in familiarity were highly model dependent. Implications for dual-process models of memory in old age are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Investigating strength and frequency effects in recognition memory using type-2 signal detection theory.

Criterion- versus distribution-shift accounts of frequency and strength effects in recognition memory were investigated with Type-2 signal detection receiver operating characteristic (ROC) analysis, which provides a measure of metacognitive monitoring. Experiment 1 demonstrated a frequency-based mirror effect, with a higher hit rate and lower false alarm rate, for low frequency words compared with high frequency words. In Experiment 2, the authors manipulated item strength with repetition, which showed an increased hit rate but no effect on the false alarm rate. Whereas Type-1 indices were ambiguous as to whether these effects were based on a criterion- or distribution-shift model, the two models predict opposite effects on Type-2 distractor monitoring under some assumptions. Hence, Type-2 ROC analysis discriminated between potential models of recognition that could not be discriminated using Type-1 indices alone. In Experiment 3, the authors manipulated Type-1 response bias by varying the number of old versus new response categories to confirm the assumptions made in Experiments 1 and 2. The authors conclude that Type-2 analyses are a useful tool for investigating recognition memory when used in conjunction with more traditional Type-1 analyses. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Six Regularities of Source Recognition.

In recent work, researchers have shown that source-recognition memory can be incorporated in an extended signal detection model that covers both it and item-recognition memory (A. Hilford, M. Glanzer, K. Kim, & L. T. DeCarlo, 2002). In 5 experiments, using learning variables that have an established effect on item recognition, the authors tested further implications of that extended model. The results establish 6 source-recognition regularities that support the model. An important regularity in that set is the linkage of source and item recognition: Any learning variable that increases item recognition increases source recognition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Word frequency and receiver operating characteristic curves in recognition memory: Evidence for a dual-process interpretation.

Dual-process models of the word-frequency mirror effect posit that low-frequency words are recollected more often than high-frequency words, producing the hit rate differences in the word-frequency effect, whereas high-frequency words are more familiar, producing the false-alarm-rate differences. In this pair of experiments, the authors demonstrate that the analysis of receiver operating characteristic (ROC) curves provides critical information in support of this interpretation. Specifically, when participants were required to discriminate between studied nouns and their plurality reversed complements, the ROC curve was accurately described by a threshold model that is consistent with recollection-based recognition. Further, the plurality discrimination ROC curves showed characteristics consistent with the interpretation that participants recollected low-frequency items more than high-frequency items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Context effects on remembering and knowing: The expectancy heuristic.

Three experiments are reported examining the effect of context on remember-know judgments. In Experiments 1 and 2, medium-frequency words were intermixed with high-frequency or low-frequency words at study or at test, respectively. Remember responses were greater for medium-frequency targets when they were studied or tested among high-frequency, as compared with low-frequency, words. The authors proposed a decision-based mechanism called "the expectancy heuristic" to explain why remember responses were more likely when items were studied or tested in the context of words that were relatively less distinct. According to the expectancy heuristic, when items on a recognition test exceed an expected level of memorability they will be given a remember judgment but when they do not, but are still more familiar than new words, they will be given a know judgment. Experiment 3, which varied expectancies about the strength of tested targets, demonstrated the use of the expectancy heuristic, indicating that it operates by selectively influencing the remember criterion rather than by influencing recollection of studied items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Aging selectively impairs recollection in recognition memory for pictures: Evidence from modeling and receiver operating characteristic curves.

Young and older adults were tested on recognition memory for pictures. The Yonelinas high threshold (YHT) model, a formal implementation of 2-process theory, fit the response distribution data of both young and older adults significantly better than a normal unequal variance signal-detection model. Consistent with this finding, nonlinear z-transformed receiver operating characteristic curves were obtained for both groups. Estimates of recollection from the YHT model were significantly higher for young than for older adults. This deficit was not a consequence of a general decline in memory; older adults showed comparable overall accuracy and in fact a nonsignificant increase in their familiarity scores. Implications of these results for theories of recognition memory and the mnemonic deficit associated with aging are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the role of individual items in recognition memory and metacognition: Challenges for signal detection theory.

The authors tested the role of individual items in recognition memory using a forced-choice paradigm with face stimuli. They constructed distractor stimuli using morphing procedures that were similar to two parent faces and then compared a studied morph against an unstudied morph that was similar to two studied parents. The similarity of the parent faces was carefully balanced so that the choosing rates for the studied and unstudied morphs were approximately equal. Despite being equally likely to choose the studied and the unstudied morph, participants were systematically more confident when choosing the studied morph. This result is incompatible with Gaussian signal detection theory, even with unequal variances for targets and distractors. The authors propose an extension of an extant sampling model, SimSample, which provides a qualitative and quantitative account of the confidence and recognition dissociation. The results suggest that observers make contact with individual items when making recognition judgments with faces and that the structure of the sampling and decision process naturally leads to this dissociation of confidence and recognition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Signal detection theory with finite mixture distributions: Theoretical developments with applications to recognition memory.

An extension of signal detection theory (SDT) that incorporates mixtures of the underlying distributions is presented. The mixtures can be motivated by the idea that a presentation of a signal shifts the location of an underlying distribution only if the observer is attending to the signal; otherwise, the distribution is not shifted or is only partially shifted. Thus, trials with a signal presentation consist of a mixture of 2 (or more) latent classes of trials. Mixture SDT provides a general theoretical framework that offers a new perspective on a number of findings. For example, mixture SDT offers an alternative to the unequal variance signal detection model; it can also account for nonlinear normal receiver operating characteristic curves, as found in recent research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Signal Detection Theory Analyses of Semantic Priming in Word Recognition.

Word recognition, semantic priming, and cognitive impenetrability research have used signal detection theory (SDT) measures to separate perceptual and postperceptual processes. In the D. Norris (1986) checking model and model simulation (D. Norris, 1995), priming alters only postperceptual word decision criteria: Stimulus-related priming reduces uncertainty, increasing sensitivity; stimulus-unrelated priming increases false alarms more than hits, reducing sensitivity. This work is cited as strong evidence that criterion changes can alter perceptual sensitivity and that SDT is inappropriate for investigating complex cognitive processes. The authors' current SDT ideal observer analysis of the model demonstrates that related priming does not directly alter sensitivity and that unrelated priming increases only false-alarm rate, reducing sensitivity. This analysis provides new perspectives on SDT concepts of complex decision processing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A dynamic stimulus-driven model of signal detection.

Signal detection theory forms the core of many current models of cognition, including memory, choice, and categorization. However, the classic signal detection model presumes the a priori existence of fixed stimulus representations—usually Gaussian distributions—even when the observer has no experience with the task. Furthermore, the classic signal detection model requires the observer to place a response criterion along the axis of stimulus strength, and without theoretical elaboration, this criterion is fixed and independent of the observer's experience. We present a dynamic, adaptive model that addresses these 2 long-standing issues. Our model describes how the stimulus representation can develop from a rough subjective prior and thereby explains changes in signal detection performance over time. The model structure also provides a basis for the signal detection decision that does not require the placement of a criterion along the axis of stimulus strength. We present simulations of the model to examine its behavior and several experiments that provide data to test the model. We also fit the model to recognition memory data and discuss the role that feedback plays in establishing stimulus representations. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Variations in recollection: The effects of complexity on source recognition.

Whether recollection is a threshold or signal detection process is highly controversial, and the controversy has centered in part on the shape of receiver operating characteristics (ROCs) and z-transformed ROCs (zROCs). U-shaped zROCs observed in tests thought to rely heavily on recollection, such as source memory tests, have provided evidence in favor of the threshold assumption, but zROCs are not always as U-shaped as threshold theory predicts. Source zROCs have been shown to become more linear when the contribution of familiarity to source discriminations is increased, and this may account for the existing results. However, another way in which source zROCs may become more linear is if the recollection threshold begins to break down and recollection becomes more graded and Gaussian. We tested the “graded recollection” account in the current study. We found that increasing stimulus complexity (i.e., changing from single words to sentences) or increasing source complexity (i.e., changing the sources from audio to videos of speakers) resulted in flatter source zROCs. In addition, conditions expected to reduce recollection (i.e., divided attention and amnesia) had comparable effects on source memory in simple and complex conditions, suggesting that differences between simple and complex conditions were due to differences in the nature of recollection, rather than differences in the utility of familiarity. The results suggest that under conditions of high complexity, recollection can appear more graded, and it can produce curved ROCs. The results have implications for measurement models and for current theories of recognition memory. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

On the equivalence of two recognition measures of short-term memory.

2 tests of short-term, recognition memory were compared. An extension of signal-detection theory was used to generate prediction from 1 test to the results of the other. These predictions were investigated for an individual S's memory of about 100 nonsense syllables. The errors in predictions were about what might be expected assuming binomial variability in the 2 measures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing a neurocomputational model of recollection, familiarity, and source recognition.

The authors assess whether the complementary learning systems model of the medial temporal lobes (Norman & O'Reilly, 2003) is able to account for source recognition receiver operating characteristics (ROCs). The model assumes that recognition reflects the contribution of a hippocampally mediated recollection process and a cortically mediated familiarity process. The hippocampal process is found to produce threshold output functions that lead to U-shaped zROCs, whereas the cortical process produces Gaussian signal detection functions and linear zROCs. The model is consistent with several dual process theories of recognition and is capable of producing the types of zROCs observed in studies of item and source recognition. In addition, the model makes the novel prediction that as the level of feature similarity across items increases, the ability of the hippocampus to encode distinct representations for each stimulus will diminish, and the threshold nature of recollection will break down, leading source zROCs to become more linear. The authors conducted 3 new behavioral source experiments that confirmed the model's prediction. The results demonstrate that the model provides a viable account of item and source recognition performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Examining classification criteria: A comparison of three cut score methods.

This study compared 3 different methods of creating cut scores for a screening instrument, T scores, receiver operating characteristic curve (ROC) analysis, and the Rasch rating scale method (RSM), for use with the Behavioral and Emotional Screening System (BESS) Teacher Rating Scale for Children and Adolescents (Kamphaus & Reynolds, 2007). Using the BESS norm data set, we compared the methods across 7 classification indices. Additional information about accuracy was used with a subset of children who had been given a prior diagnosis for selected disorders. The results showed that the methods were generally in concordance, with similarities identified across methods. RSM and ROC analysis methods performed similarly, with both methods identifying the same optimal cut-point. The method based on T scores appeared to be more conservative, identifying a lower cut score as optimal. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Remember-Know: A Matter of Confidence.

This article critically examines the view that the signal detection theory (SDT) interpretation of the remember-know (RK) paradigm has been ruled out by the evidence. The author evaluates 5 empirical arguments against a database of 72 studies reporting RK data under 400 different conditions. These arguments concern (a) the functional independence of remember and know rates, (b) the invariance of estimates of sensitivity, (c) the relationship between remember rates and overall hit and false alarm rates, (d) the relationship between RK responses and confidence judgments, and (e) dissociations between remember and overall hit rates. Each of these arguments is shown to be flawed, and despite being open to refutation, the SDT interpretation is consistent with existing data from both the RK and remember-know-guess paradigms and offers a basis for further theoretical development. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Item Recognition Memory and the Receiver Operating Characteristic.

Four experiments were conducted to investigate the effects of study time, study repetition, semantic and orthographic similarity, and category length on item recognition memory receiver operating characteristics (ROCs). Analyses of ROC shape rejected A. P. Yonelinas's (1994) dual-process model. The normal unequal variance signal-detection model provided a better account of the data, except for a small but consistent excess of high-confidence errors. It was found that z-transformed ROC slope was increased by similarity, category length, and study item repetition, rejecting R. Ratcliff, G. McKoon, and M. Tindall's (1994) "constancy-of-slopes" generalization for these variables, but slope was relatively unaffected by massed study time. (PsycINFO Database Record (c) 2010 APA, all rights reserved)