Performance Evaluation: Subjectivity

Identifier:  imarina:6300887

Handle: https://hdl.handle.net/20.500.11797/imarina6300887

Authors:  Perez-Laborda, Alejandro

Abstract:
© 2020, Springer Nature B.V. The number of situations that require individual judgments and evaluations, and that may be object of different sources of conscious and unconscious biases is endless. This paper proposes a practical score aggregation procedure that attempts to reduce and mitigate the influence of bias in subjective judgments. The argument is based on the idea that bias is associated with deviations from the panel mean and/or deviations from the judges’ grading style. Consequently, the procedure is not specific to a particular type of bias, but rather addresses general forms of bias. We also discuss a set of desirable properties. The proposed score aggregation procedure is then applied to a unique data set from the 2000 Summer Olympic Games diving competition.