Speaker
Description
The contribution aims at discussing some preliminary results on the evaluation of prediction performance for the class of mixture models with uncertainty (Piccolo and Simone, 2019). The ultimate goal of the analysis is the evaluation of the extent by which the uncertainty specification constitutes an added value for prediction of ordinal scores. A small simulation study is presented to assess prediction performance of competing models under miss-specification. The Ranked Probability Score is chosen as scoring rule since it is the most suited to deal with ordinal data, without the assignment of numerical scores to category. Finally, a variable selection procedure based on prediction performance can be outlined on a case study for the prediction of subjective probability to survive. Comparisons with cumulative link models are illustrated for the sake of completeness. Preliminary findings discussed in Simone and Piccolo (2022) indicate that uncertainty modelling improves prediction performance substantially. Hence, it is important to assess the information quality of the baseline preference model (the Binomial, for instance). To this aim, we introduce a new utility measure for preference models when contaminated with alternative uncertainty specifications in the sense proper to the framework of Information Quality. As a result, the mixing weight of the chosen feeling component within the mixture can be explicitly interpreted in terms of model predictive ability.
Keywords: CUB models; Predictability; Ranked Probability Score; Ordinal Data
References:
D. Piccolo, R. Simone (2019). The class of CUB models: statistical foundations, inferential issues and empirical evidence. STATISTICAL METHOD AND APPLICATIONS, Volume 28, pages 389-435.
R. Simone and D. Piccolo (2022). On the predictability of a class of ordinal data models. In A. Balzanella, M. Bini, C. Cavicchia, and R. Verde, editors, Book of short papers SIS 2022, 51st Scientific Meeting of the Italian Statistical Society, pages 1053–1058. Pearson.
