Evaluation of the consistency of wine quality assessments from expert wine tasters

Background and Aims: The aim of this study was to assess the ability of experienced wine tasters to consistently assign quality scores to both red and white wines. Methods and Results: Wine quality scores were collected over a 15‐year period from 571 experienced wine tasters. Consistency was measured by correlating the scores given to duplicate presentations of wines, calculating the pooled variation in repeat scores and assessing their ability to allocate duplicate presentations of the same wine to the same quality category. Although the majority of tasters showed statistically significant consistency, their individual abilities varied considerably and, in general, their ability to consistently score one wine type was a poor predictor of their consistency in scoring the other. Tasters were better able to allocate duplicate presentations of red wines to the same category than white wines, and red wine consistency was improved by combining the scores of three assessors as is done in the Australian wine show system. Conclusions: The ability of experienced wine tasters to consistently rate wines for overall quality varied greatly between individuals, but was generally better for red wines than for whites. Consistency was improved by combining the scores from a small team of tasters. Significance of the Study: The study demonstrates the need to conduct replicate tastings when assessing wines for quality as adequate taster repeatability cannot be guaranteed. Furthermore, using the combined score of a small team of tasters generally results in more consistent quality assessments.