When your organization seeks ACCME accreditation, they provide an array of criteria to meet in order to have your offering accredited. In its publication “The ACCME Accreditation Requirements (Updated November 2017)” they present 35 such criteria. According to this publication, found here in this document.
CME providers who seek full accreditation must meet criteria 1-13. Additional criteria are met to receive what the ACCME refers to as “Accreditation with Commendation.”
What we’re going to look at briefly in this article are some of the criteria that address evaluating the impact of your educational intervention on the healthcare providers who take the course, and on their organizations and communities.
Traditionally, the standard for evaluating learning was to administer a posttest following the treatment and measure knowledge recall, analysis, and application of the intended learning outcomes. Sometimes this is paired with a pretest. The ACCME still allows for this method, but it is by no means the only acceptable method to evaluate learners and receive accreditation.
The criteria we’ll examine briefly are numbers 11, 12, 36. 37, and 38. Methods used to measure these vary and are applicable across criteria.
Briefly, Criteria 11 and 12 are required for full accreditation. Looking at the effect of your intervention, Criterion 11 requires that you analyze changes in learner competence and performance and in patient outcomes, while Criterion 12 requires you to gather data or information on how well your CME mission has been met.
Criteria 36 and 37 appear similar to 11 and 12, but in addition to analyzing change, these criteria for Accreditation with Commendation ask you to demonstrate improvement in learner performance and healthcare quality, respectively. And Criterion 38 looks beyond the immediate target of the intervention and to its broader impact on patients and their communities.
That said, what are acceptable ways for evaluation to be accomplished? The ACCME and American Board of Internal Medicine offer suggestions for evaluation mechanisms for CME and Maintenance of Certification (MOC) interventions, with an appropriate evaluation method, participation threshold, and feedback method for each mechanism. This can be found in online here in this document.
Key to each of the non-test oriented methods is meaningful participation by the learner. This helps “keep everyone honest,” as it were.
This document emphasizes that while this is a suggested list, it is not an exhaustive list. Your CME producers and designers may come up with other creative means of evaluating how well learning outcomes were achieved and your CME mission met.
So the bottom line is, not all ACCME accredited learner evaluations are multiple choice pretests and posttests. ACCME has a wealth of information including publications available online at www.acme.org that can help you understand what your evaluation options are. I would suggest starting with the two publications mentioned in this article.