Forecast aggregation via recalibration |
| |
Authors: | Brandon M. Turner Mark Steyvers Edgar C. Merkle David V. Budescu Thomas S. Wallsten |
| |
Affiliation: | 1. Stanford University, 450 Serra Mall, Stanford, CA, 94305, USA 2. University of California, Irvine, CA, 92697, USA 3. University of Missouri, 319 Jesse Hall, Columbia, MO, 65211, USA 4. Fordham University, 441 East Fordham Road, Bronx, NY, 10458, USA 5. University of Maryland, 8082 Baltimore Avenue, College Park, MD, 20740, USA
|
| |
Abstract: | It is known that the average of many forecasts about a future event tends to outperform the individual assessments. With the goal of further improving forecast performance, this paper develops and compares a number of models for calibrating and aggregating forecasts that exploit the well-known fact that individuals exhibit systematic biases during judgment and elicitation. All of the models recalibrate judgments or mean judgments via a two-parameter calibration function, and differ in terms of whether (1) the calibration function is applied before or after the averaging, (2) averaging is done in probability or log-odds space, and (3) individual differences are captured via hierarchical modeling. Of the non-hierarchical models, the one that first recalibrates the individual judgments and then averages them in log-odds is the best relative to simple averaging, with 26.7 % improvement in Brier score and better performance on 86 % of the individual problems. The hierarchical version of this model does slightly better in terms of mean Brier score (28.2 %) and slightly worse in terms of individual problems (85 %). |
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|