The Quarterly Questions program, which consists of 40 knowledge-based and 10 article-based questions annually, is the activity through which diplomates obtain credit for Maintenance of Certification (MOC) Part III: Assessment of Knowledge, Judgment, and Skills. A popular question I have been asked is: why can’t participation in CME courses replace the need for an assessment?
When it comes to answering these questions, I always start by explaining the purpose of the assessment, Quarterly Questions, which is primarily a summative assessment. A summative assessment is often called an assessment of learning. It is used to determine whether (or the extent to which) an individual has learned or mastered some knowledge or skill. In contrast, a formative assessment is used to provide feedback to the examinee to encourage ongoing learning (AERA, APA, & NCME, 2014) and is often called assessment for learning. In MOC, the ABO is committed to both types of assessment: summative, to ensure the level of medical knowledge has met the standards for board certification, and formative, to assist the diplomate in identifying knowledge gaps.
Educational psychology research has found that in terms of reliably assessing a broad knowledge domain, multiple-choice questions (MCQs) are one of the most successful tools (some would argue they are the single best tool). That said, if we can validate tools that are just as, or even more reliable, valid, and feasible than MCQs, we would consider using them in the ABO MOC program. Unfortunately, participation in CME activities, although likely very educational for most physicians, do not provide the ABO nor the public with any assurance that doctors who have attended have learned, assimilated, and applied the information presented. While it is likely that participants enrich their knowledge and understanding through CME, and we recognize such efforts as part of MOC Part II: Lifelong Learning and Self-Assessment, the only way we can verify such is through summative assessment. If we could prove that the knowledge base and outcomes of doctors were higher because they attended certain meetings or CME courses, then the ABO would consider those activities as alternative options. However, as you can imagine, building that validity argument is challenging, and requires summative assessment for its development.
The ABO is committed to providing a summative assessment that is relevant to the diplomate, statistically reliable, and a valid measure of ophthalmic knowledge. As new assessment tools that meet these criteria become available, the menu of options for MOC activities will grow.
Reference
American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME]. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.