Color‐difference formula performance for several datasets of small color differences based on visual uncertainty

Visual uncertainty, while reported, is not used routinely when evaluating color‐difference formula performance in comparison with visual data; rather, data are analyzed assuming no uncertainty; that is, repeating the experiment would result in the identical average results. Previously, Shen and Berns developed three methods to determine whether a color‐difference formula was well‐fitting, under‐fitting, or over‐fitting visual data when visual uncertainty was considered, the method dependent on how the uncertainty was reported and the colorimetric sampling of the color‐difference stimuli. The “nonellipsoid standard error method” was used in the current analyses. Three datasets were evaluated: BFD‐P, Leeds, and Witt. For the BFD‐P data, incorporating visual uncertainty led to the same performance results as the average results, that CIEDE2000 was an improvement over CIE94, which was an improvement over CIELAB. For the Witt data, incorporating visual uncertainty led to the same performance results as the average results, that CIEDE2000 and CIE94 had equivalent performance, both an improvement over CIELAB. However, both formulas under‐fitted the visual results; thus, neither formula was optimal. For the Leeds dataset, the visual uncertainty analysis did not support the improvement of CIEDE2000 over CIE94 that occurred when evaluating the average results. Both formulas well fit the visual data. These analyses also provided insight into the tradeoffs between the number of color‐difference pairs and the number of observations when fitting a local contour of equal perceived color difference: In particular, increasing the number of observations was more important than increasing the number of color‐difference pairs. Finally, average standard error could be used to approximate visual uncertainty defined using STRESS. © 2010 Wiley Periodicals, Inc. Col Res Appl, 2011