By Sarah Schnabel, PhD, Director of Assessment and Psychometrics
Knowledge-based (summative) Quarterly Questions are designed to assess everyday clinical knowledge essential to the practice of ophthalmology. While our item writers are encouraged to develop questions that challenge diplomates to think, the ultimate criterion for whether an item makes it onto an assessment is whether it is factual and clinically relevant. Our first priority in test construction is good content representativeness.
In the case of Quarterly Questions, there is no “ideal” difficulty level. That said, I interpret the average score of approximately 85% to mean that the test is not too easy (thus, there is some room for learning), but that the pass rate (over 99% in 2019) means that the passing standard correctly reflects what we know to be true: the vast majority of ABO diplomates are keeping up.
The knowledge-based Quarterly Questions are a mix of items you’ll perceive as harder and easier depending upon your knowledge base, level of specialization, and other factors. Although a group of subject-matter experts have determined that each question is likely clinically relevant to most diplomates, it isn’t possible to construct an assessment that perfectly aligns with each individual’s practice. When you perceive an item as harder or answer an item incorrectly, reflect on whether learning more about that topic would be beneficial to your practice. Over time, perhaps you’ll find fewer and fewer items that seem “hard!”
The Quarterly Questions platform allows you to provide us with feedback on the questions. There is a comment field available for each question and we encourage you to let us know if you learned something, if the question was relevant, or if there was some way we could improve upon the knowledge we are testing in that question. Your feedback helps us learn more about what you find relevant to your practice.