One of the main responsibilities of the American Board of Ophthalmology’s psychometric and examination development team is determining what questions will appear its assessments. Every few years, the ABO undergoes a process to evaluate the content specifications for each examination. Content specifications (also called the content outline or test blueprint) indicate what percentage of the examination is allocated to each subject area. These specifications may also dictate what percentage of the questions on the exam are recall questions versus interpretation questions, the percentage of diagnosis versus management questions, and more. The ABO follows the guidelines in the Standards for Educational and Psychological Testing as closely as possible when developing and revising content specifications.
To begin a content specification review, a group of volunteer subject matter experts convene to evaluate the test purpose statement. It is important to begin with a clear understanding of what the test is designed to measure. Then, a separate group of subject matter experts defines subject areas (domains) for the assessment (such as lens and cataract; cornea, external disease and anterior segment; glaucoma, etc.) Both of these groups are carefully selected so that there is representation by individuals from the major practice areas (e.g., oculoplastics, neuro-ophthalmology, retina, etc.), practice settings (academic and private practice), time since initial certification, and history of volunteering with the ABO. You can see recent lists of subject matter expert volunteers involved in this process here. This step is meant to ensure that the input into the test content is based upon a wide variety of perspectives.
Next, this list of domains is circulated to all diplomates in the form of a survey. This is often referred to as a practice analysis survey. You may recall completing a practice analysis survey in late 2016 or early 2017 for the Written Qualifying Examination and Maintenance of Certification exams. The survey asks board-certified ophthalmologists how important each domain (or sub-domain) is for safe and effective practice for a board-certified ophthalmologist. These results are compiled and analyzed, which results in a recommended weighting for each of the domain areas (e.g., lens and cataract: 12%; cornea, external disease and anterior segment: 13%; glaucoma: 13%, etc.) The survey results also provide guidance on the recommended weighting of the number of items that focus on presentation, diagnostic testing, and management/outcomes.
Finally, these weights are approved by the ABO Board of Directors and posted on the ABO website. The ABO examination development team assembles test forms to meet these specifications for each and every test administration to ensure the appropriate representation of content on all versions of the exam. Click here to view all the content outlines currently in use by the ABO.