Scoring procedures for multiple criteria decision aiding with robust and stochastic ordinal regression

We propose several scoring procedures for transforming the results of robustness analysis to a univocal recommendation. We use a preference model in form of an additive value function, and assume the Decision Maker (DM) to provide pairwise comparisons of reference alternatives. We adapt single- and multi-stage ranking methods to select the best alternative or construct a complete ranking by exploiting four types of outcomes: (1) necessary preference relation, (2) pairwise outranking indices, (3) extreme ranks, and (4) rank acceptability indices. In each case, a choice or ranking recommendation is obtained without singling out a specific value function. We compare the proposed scoring procedures in terms of their ability to suggest the same recommendation as the one obtained with the Decision Maker׳s assumed “true” value function. To quantify the results of an extensive simulation study, we use the following comparative measures (including some newly proposed ones): (i) hit ratio, (ii) normalized hit ratio, (iii) Kendall׳s τ, (iv) rank difference measure, and (v) rank agreement measure. Their analysis indicates that to identify the best “true” alternative, we should refer to the acceptability indices for the top rank(s), whereas to reproduce the complete “true” ranking it is most beneficial to focus on the expected ranks that alternatives may attain or on the balance between how much each alternative outranks and is outranked by all other alternatives.