Ask a Test Scientist: Can Test Feedback Help Physicians Learn to Self-Assess More Accurately?

Our ability to accurately self-assess is based on many factors, among them, our likelihood to employ cognitive biases. The most significant of those biases is overconfidence. And while having tremendous confidence in our abilities as professionals is crucial, an imbalance between confidence and competence can skew the way we view our own performance.

The Dunning-Kruger Effect

You may have heard of the Dunning-Kruger effect, which helps to explain therelationship between confidence and competence. In their original 1999 study[1], Justin Kruger and David Dunning asked individuals to take a test and then estimate their own performance. High scorers were fairly accurate when asked how they performed. Those on the lower performing end, however, could not rate their performance accurately and often perceived that they performed better than they actually did.

What might this mean to practicing physicians? While the Dunning-Kruger study did not look at overconfidence specific to medicine, similar studies that followed have shown that physicians and medical students experience similar struggles in evaluating their own level of medical knowledge[2,3]. Sometimes, we just don’t know what we don’t know.

But physicians can improve their accuracy in making cognitive judgments with exposure to meaningful feedback, like the detailed feedback that participants in the American Board of Ophthalmology’s Quarterly Questions program receive.

The Value of Feedback

According to research, detailed test feedback that explains the rationale for the correct answer results in greater retention of information than just “right/wrong” feedback[4].

But the benefit of feedback doesn’t end there. A study conducted in 2008[5] had undergraduate students take a multiple-choice test and estimate their confidence after each response. This process might sound familiar to you if you’ve tried the Quarterly Questions program, which also asks for your confidence level after answering each question. These researchers found that when feedback was provided on each question, the examinees were more accurate in their confidence ratings on a subsequent test.

In other words, estimating your confidence with a question you’ve answered and then receiving feedback (a rationale for the correct answer) has two benefits: 1) the feedback you receive increases the likelihood that you’ll know the answer to that question in the future, and 2) some preliminary research has shown that the act of simply assessing your confidence level may lead to more accurate self-assessment judgments when assessed on that material again in the future.


The takeaway here is that assessments that provide feedback are not only valuable in terms of learning information you didn’t already know—they may help you get better at accurately assessing yourself. No one can know everything or be right all of the time, but using feedback and reflecting on common cognitive errors you may not even realize you’re making can go a long way in helping you to increase your knowledge andself-awareness—and provide the very best care for your patients.

1. Kruger J, Dunning D. Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77(6):1121-1134. doi:

2. Berner ES, EdD, Graber ML, MD. Overconfidence as a cause of diagnostic error in medicine. American Journal of Medicine, The. 2008;121(5):S2-S23.

3. Borracci RA, Arribalzaga EB. The incidence of overconfidence and underconfidence effects in medical student examinations. Journal of Surgical Education. 2018;75(5):1223-1229. doi:

4. Wojcikowski K, Kirk L. Immediate detailed feedback to test-enhanced learning: An effective online educational tool. Med Teach. 2013;35(11):915-919.

5. Butler AC, Karpicke JD, Roediger HL. Correcting a metacognitive error: Feedback increases retention of low-confidence correct responses. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2008;34(4):918-928.

#askapsychometrician #psychometrics #selfassessment #feedback

© 2020 American Board of Ophthalmology