#examdevelopment

Dec 17, 2019
2 min

Ask a Test Scientist: Why Did My Quarterly Questions Score Change?

Last month, you may have received an email from the ABO indicating that your score on the 2019 Quarterly Questions assessment changed. You might be wondering why and how that happened. Each year, the ABO reviews the performance of all questions in the Quarterly Questions activity during a process known as Key Validation. If a question is deemed “flawed” during this review, individuals are given credit for the item regardless of what answer he or she entered. What is a flawed

Nov 5, 2019
5 min

Thank You, Fall 2019 Examiners!

Examiners for the American Board of Ophthalmology are diplomates selected for their embodiment of the Board's commitment to excellence and high-quality ophthalmic care. ABO Examiners lead the oral examination process; many also share their expertise in the development of examination materials. The success of the Oral Examination process in providing a fair and comprehensive assessment of each candidate for Board Certification depends on each Examiner's knowledge, skill, chara

Aug 29, 2019
3 min

Ask a Test Scientist: Why Do We Need Two Kinds of Quarterly Questions?

On July 1, the American Board of Ophthalmology released the article-based questions for the 2019 Quarterly Questions program. If you’re participating in the program, you know that during Quarter 3, you chose 5 journal articles to read at your own pace and then answered a total of 10 questions about them. You probably noticed that these questions were different than the knowledge-based questions released in Quarters 1 and 2 of each year. Today I hope to answer: why do we have

Jul 30, 2019
3 min

Subject Matter Experts Meet in Minnesota to Assemble Future Board Exams

From July 25-26 in Minneapolis, MN, the American Board of Ophthalmology conducted its annual exam development meeting to support the creation of hundreds of high-quality test items for the Written Qualifying Examination, Oral Examination, and Quarterly Questions assessment program. Over the past several months, the subject matter experts listed by committee below have volunteered their time and clinical insights to writing, evaluating, and editing multiple-choice and oral exa

Jul 2, 2019
2 min

Ask a Test Scientist: Why can't CME replace the assessment component of MOC?

The Quarterly Questions program, which consists of 40 knowledge-based and 10 article-based questions annually, is the activity through which diplomates obtain credit for Maintenance of Certification (MOC) Part III: Assessment of Knowledge, Judgment, and Skills. A popular question I have been asked is: why can’t participation in CME courses replace the need for an assessment? When it comes to answering these questions, I always start by explaining the purpose of the assessment

Apr 23, 2019
1 min

ABO Psychometrician Publishes Article in CLEAR Exam Review

In the Spring 2019 edition of the CLEAR Exam Review, American Board of Ophthalmology Psychometrician Sarah D. Schnabel, PhD, discusses why assessment in medical continuing certification must be three things: relevant, dynamic, and frequent. In the article, Dr. Schnabel makes the case for ongoing summative assessment in board certification -- the same idea behind the ABO's Quarterly Questions program framework: "When a certifying board has reason to believe that certificate ho

Apr 3, 2019
12 min

Recognizing ABO Volunteers During National Volunteer Month

Volunteers make the work of the American Board of Ophthalmology possible. The extraordinary contributions of nearly 600 examiners and examination development volunteers enable the ABO to deliver high-quality assessments and professional development activities to thousands of ophthalmologists each year. Volunteers for the American Board of Ophthalmology are diplomates selected to examine at the oral examination, develop items for written and oral examinations, and serve as spe

Apr 3, 2019
3 min

Ask a Test Scientist: What's the lifecycle of a Quarterly Question?

To assemble a single year of the Quarterly Questions assessment program, it takes at least 530 test questions—enough to cover every subspecialty in ophthalmology from cataract to uveitis—along with additional questions based on more than 40 articles. So where does all of this content come from? And how does it make its way on to your screen as part of the assessment? This post explains the process of writing a Quarterly Questions item and how that moves through the process of

Feb 5, 2019
2 min

Ask a Test Scientist: Who Writes ABO Test Questions?

Diplomates often wonder “who writes the questions we see on tests?” Board-certified ophthalmologists volunteer annually to develop questions (items) for the ABO examinations. The process usually begins in February, when the volunteer item-writers receive training on developing good items. Psychometric research has found that the best way to capture someone’s knowledge in a multiple-choice question is to keep the item focused on a single topic, ask a straightforward question,

Dec 5, 2018
5 min

11 Ways To Get Involved With the ABO

The impetus for board certification in American medicine did not come from the federal government or even public watchdogs. It arose more than a century ago from practicing physicians like you who wanted to address concerns about the qualifications and training of individuals claiming to be specialists in ophthalmology. In the early 1900s, leaders of our profession came together and established a framework for board certification in a medical specialty that would go on to set

Nov 7, 2018
8 min

2018 American Board of Ophthalmology Volunteer Honor Roll

New Examiners Beardsley, Robert M. - Eugene, OR Birkholz, Emily S. - Mankato, MN Birnbach, Charles D. - Lake Forest Park, WA Bishop, Frank M. - Chula Vista, CA Carolan, James A. - Novato, CA Cavuoto, Kara M. - Miami, FL Chan, Clement K. - Palm Springs, CA Giangiacomo, Annette L. - Decatur, GA Goldstein, Jeffrey B. - Asheville, NC Hegleh, Joseph A. - Sarasota, FL Kim, Brian Y. - Williston, VT Larson, Scott A. - Iowa City, IA Lin, Amy - Salt Lake City, UT Mohadjer, Yasaman - Gu

Aug 4, 2018
2 min

Thank You, 2018 Exam Development Volunteers

The American Board of Ophthalmology thanks the following groups of diplomate-volunteers who contributed their knowledge and ophthalmic expertise to the 2018 Exam Development Meeting in Philadelphia from August 2-4. Cataract Geoffrey Emerick, M.D. Robert Fish, M.D. Gregory Katz, M.D. Yousuf Khalifa, M.D. Amy Lin, M.D. Leslie Olsakovsky, M.D. Bhavna Sheth, M.D. Sonal Tuli, M.D. Brian Welcome, M.D. Comprehensive Jane Bailey, M.D. Sean Breit, M.D. John Campagna, M.D. Francesca Gi

Aug 1, 2018
1 min

Observer Calls Oral Examination a "Model of Professionalism"

Ophthalmology Times recently published a guest editorial by James Lifton, MBA, a public member of the American Board of Medical Specialties (ABMS) Board of Trustees who observed the American Board of Ophthalmology's June Oral Examination. In the editorial, which summarized his experience at the examination and his perceptions of the process, Lifton wrote that the Oral Examination, "offers a model of professionalism to ophthalmologists, most just beginning their careers...[p]h

Jul 1, 2018
2 min

The Science of Testing: How Key Validation Ensures Fairness on ABO Examinations

Key validation is a process whereby examination items (questions) are reviewed to ensure they are fair. After the administration of an examination, an item analysis is performed by a psychometrician. This analysis provides information about how many examinees answered each item correctly, which options (A, B, C, or D) were selected most often, and which items positively discriminated higher from lower performers. This analysis doesn’t in and of itself identify an unfair or “f

Jun 1, 2018
2 min

Two Facts About ABO Standards That May (Pleasantly) Surprise You

In early May, 18 board-certified ophthalmologists met with American Board of Ophthalmology Psychometrician Sarah D. Schnabel, Ph.D., to set standards for the Written Qualifying Examination (WQE) for initial certification. Here are the two most important things they wanted you to know about ABO standards: 1) It’s always possible for all examinees to pass an ABO examination. Though it’s not likely that every examinee will pass on his or her first try, it’s important to understa

Apr 30, 2018
8 min

Thank You to All Our Volunteers

National Volunteer Month is a perfect time to spotlight the hard work of board-certified ophthalmologists around the country who advance excellence in ophthalmology by giving generously of their time and expertise. Each year, hundreds of busy clinicians write and review questions and content for ABO examinations and Quarterly Questions, set rigorous standards for achievement in ophthalmology, and enable the successful administration of oral examinations. Here are the members

Apr 29, 2018
1 min

The Science of Testing: Why AIG is No Substitute for SMEs

Artificial intelligence (AI) is one of the hottest topics in technology today. (Just ask Siri or Alexa.) But what if we could ask these virtual assistants to order our groceries, tell us the weather, and write examination content? It sounds a bit farfetched, but Automatic Item Generation (AIG), a major innovation in the science of test development, could change the way assessment organizations like certification boards develop multiple-choice questions. Multiple-choice questi

Mar 26, 2018
2 min

The Science of Testing: Similarities and DIFs

Do men and women answer test questions about ophthalmology differently? What about younger versus older ophthalmologists? In general, that’s unlikely. But there may be instances where two subgroups of equally qualified ophthalmologists approach the same test question and arrive at different answers. Test developers use what’s called “DIF” or Differential Item Functioning, to examine item performance variability between examinees of comparable proficiency in different demograp

Jan 17, 2018
1 min

The Science of Testing: Why Do We Write Four Options for Multiple-Choice Questions?

The American Board of Ophthalmology writes multiple-choice examination questions (items) with four options: A, B, C, and D. One of those options is correct (the key). Why do we write four options? This is a hotly debated topic among psychometricians. Some researchers prefer three options, some think four is best, and some advocate for as many options as are reasonable in the context of the question. The research on this topic has focused on whether distractors (incorrect opti

Dec 11, 2017
2 min

The Science of Testing: How Does the ABO Create Reliable and Valid Examinations?

Note from the CEO: The ABO written and oral examinations are created through the dedicated collaboration of hundreds of volunteers along with ABO Directors and the Board’s office staff. A consistent comment from first-time volunteers is, “I had no idea how difficult it is to write good questions and how much work goes into this process.” To allow diplomates and candidates a glimpse “behind the curtain,” the ABO’s psychometrician, Sarah Schnabel, will contribute regular postin