顺序会歪曲评估吗?：从大型钢琴比赛的观察数据与现场实验得到的佐证

This study investigates why “serial order effects”—systematic distortions in judgment caused by the presentation order of candidates—arise in sequential expert evaluations, and whether such effects can be mitigated. Focusing on Japan’s largest piano competition, we combine long-term observational data from 2004–2022 with data from a cluster-randomized controlled trial (RCT) conducted in 2023. The result drawn from the observational data reveals a robust serial order effect: performers who appear earlier in the sequence consistently receive lower scores. This disadvantage is particularly pronounced in competitions with fewer participants and with highly skilled performers. These patterns are difficult to reconcile with fatigue-based explanations and instead suggest a calibration mechanism, whereby judges—motivated to maintain internal consistency—avoid extreme evaluations in the early stages when the overall distribution of performance quality is still uncertain, thereby suppressing initial scores. We then run a RCT to evaluate an information treatment that explicitly informs judges about the existence and magnitude of serial order effects using historical data. The results show no strong evidence that the intervention significantly mitigates order effects overall. However, we find suggestive evidence that the serial order effect is partially mitigated for highly skilled performers. The results suggest the need for institutional or procedural solutions to address order effects in sequential evaluation settings.