Did you watch the Winter Olympics this year? With some sports, it’s easy to determine the winner: the first cross-country skier to cross the finish line, the fastest bobsledding team down the track, or the hockey team that scores the most goals. But in other Olympic sports, like figure skating, it can be difficult to measure who wins because competitors are scored more subjectively.
If you’ve ever watched a skating competition and thought an athlete received a score that was either too high or too low—you’re not alone. That’s subjectivity at work. And in recent years, skating has come under fire for not doing more to standardize its scoring methods. For instance, in the past, the skating community has allowed judges to score athletes from their own countries, which can create a conflict of interest and lead to bias.
Just like an ABO Oral Exam, eliminating bias in your judges/examiners is critical. Bias can have a significant impact on the results. According to some scholars, studies featuring data from recent international skating competitions has revealed evidence of measurable bias. By contrast, the ABO Oral Exam is structured to prevent and eliminate bias. In addition to working with a team of 500 excellent Examiners, we have some additional safeguards built into the program: for example, we don’t allow Examiners to evaluate candidates they know or have trained; we set standardized criteria for what constitutes passing and failing and review it in our post-exam panel grading sessions; and we use six different Examiners to evaluate a single candidate so that a score is never entirely dependent on an interaction with one ABO Examiner.
Some experts believe that if figure skating adjusted for judges' severity in real time, it would improve the fairness of the competitions. We’re a step ahead in the testing community! Maybe they should employ a psychometrician on the Olympic planning committee to keep an eye on the scoring practices…