ABMS Member Boards The American Boards of: • • • • • • • • • • • •
Allergy and Immunology Anesthesiology Colon & Rectal Surgery Dermatology Emergency Medicine Family Medicine Internal Medicine Medical Genetics Neurological Surgery Nuclear Medicine Obstetrics & Gynecology Ophthalmology
• • • • • • • • • • • •
Orthopaedic Surgery Otolaryngology Pathology Pediatrics Physical Medicine and Rehabilitation Plastic Surgery Preventive Medicine Psychiatry & Neurology Radiology Surgery Thoracic Surgery Urology
1
ABMS as a Self-regulatory Organization » ABMS is largest self-regulatory group of physicians in the United States » ~ 750,000 practicing physicians are certified by one (or more) of the 24 ABMS Member Boards » ~ 65% have time-limited certificates (93% projected by 2020) » ~ 315,000 participating in MOC (increasing by ~30-50 K/ year)
2
History of MOC
1917
1969
1970
1972
1973
Boards adopted Family Medicine principles of issues first timeSpecialty Boardlimited certificates recertification movement begins COCERT formed to develop guidelines for recertification process
First recertification policies established by Family Medicine and Internal Medicine
1982
1998
1999
Many boards begin administering recertification exams
MOC concept introduced; Task Force on Competence created
2000
2006
Boards commit to MOC; adopt four MOC components
Six general competencies established with ACGME
2009
Common MOC Standards adopted, implemented
All Boards receive approval for their MOC programs
What is Maintenance of Certification? (ABMS MOC®) A lifelong learning process designed to document that physician specialists, certified by one of the Member Boards of ABMS, maintain the necessary competencies to provide quality patient care.
4
ABMS Maintenance of Certification Four Parts I. Professional standing (licensure) * II. Lifelong learning and self-assessment* III. Cognitive expertise (examination) IV. Practice performance assessment* * Parts
I, II and IV modified by MOC Standards adopted March 2009
5
ABMS Maintenance of Certification ABMS/ACGME Competencies 2
1 Practice-based Learning & Improvement
4 Medical Knowledge
3 Patient Care and Procedural Skills
5 Interpersonal & Communication Skills
Systems-based Practice
6 Professionalism
6
Recent Events/Trends Related to Simulation » Conceptual Development of “MOC-CME” » Simulation and MOC Workshop » Re-constitution of ABMS Committee on Research and Evaluation Procedures (COREP) • Simulation Working Group
7
MOC CME: Simulation-based Methods » Simulation approaches consistent with evidence-base underlying effective CME • • • • • •
Often integrates needs and outcomes assessment Interactive vs. passive Live vs. print-based Multi-media vs. single media Multi-format vs. single format Episodic/continual vs. single administration
» Multiple competencies 8
Possible Themes » Simulation could be useful in MOC, if: • Skills associated with performance could be identified, • Defensible standards could be established
» Controversy regarding the value of investment in team training » Procedural training / assessment: • Quantifiable measures important: ? time, checklist sequence, or simulator-related complication • Assessing operative performance: - Technical skills and decision-making - Cognitive, cognitive/motor, & motor elements - High fidelity not absolutely essential
Themes from Educational Session » Key features of simulation-based medical education (SBME): • Mastery learning • Deliberate practice
» Research linking simulation-based education to real patient outcomes
Possible Themes » Multiple examples of effective simulation: • Clinical ultrasonography skills in critical care - Mixed correlation between cognitive and psychomotor testing
• ClinSim for Part II of ABFM - Form of cognitive simulation using Bayesian model
• Simulation in immersive practice environments - Important features: deliberate practice, patient safety protected, assesses integration of knowledge and skills, authentic environment and context (minimized ‘gaming’)
Possible Themes » Multiple examples of effective simulation: • ACOG Simulation Consortium - Simulation facilitates development of consistent standards and allows focus on core skills relevant to different procedures
• ABA/ASA work on rare and infrequent events - Network of simulation centers - Events linked to closed patient claims
Themes from Educational Session » Simulation in High-stakes Assessment (MOC Part III) • Addresses important skills (assessment drives learning) • Efficient and able to be standardized
» Summative assessment increases psychometric demands • Reliability, validity, research agenda • Scoring approach critical - Varies across level of learner
• Content important
» Challenges: • Cost, logistics, setting standards
Simulation in MOC: Pros and Cons » Pros • Authenticity • Patient safety • Fairness re: consistent portrayal/scoring • Includes competencies not currently addressed • Initial data showing improved outcomes
» Cons • • • •
Cost Resources Expertise Credibility to physician examinees • Assessment anxiety
ABMS Discussion: MOC and Part IV » Complex relationship between Parts II and IV • Simulation as Part II could model Part IV assessment • Simulation-based learning could serve as the educational intervention in Part IV activity • Measure à simulation-based intervention à re-measure
» Focus of simulation • • • • •
Team training Low frequency / high impact New procedures Practice-based learning (Communication)
Potential Advantages of Simulation in MOC » Consistency in administration & scoring • Ensuring ‘competence’ at entry • Demonstrating technical proficiency - Patient safety implications
» Adaptability to practice » Flexibility in scheduling » Ensures coverage of important content » Enables immediate and specific feedback
16
Potential Disadvantages of Simulation in MOC » Cost » Simulation Centers • Geographic distribution • Consistent approaches to simulation and assessment • Variable accreditation standards across specialties
» Transferability to practice (validity)
17
MOC Part IV » Question of relevance for Part IV – ‘practice assessment’ » Not appropriate for ‘everyday’ practice,… » Allows assessment of conditions/presentations that are infrequent but important • Anesthesiology – Malignant hyperthermia • Internal Medicine – Anaphylaxis • Cross specialty – Disaster medicine; bioterrorism
18
Target for Assessment
Observable Practice Potential Practice Professional Field Melnick, Med Ed 2002
19
MOC Part IV » Possible criteria for Part IV simulation activity 1. Infrequent / important content identified via practice profile 2. Implemented using performance improvement framework 1. Measure / intervention (learning or improvement) / Re-measure
» Not used alone - complementary to other Part IV performance improvement activities
20
Simulation in MOC: Conclusions » Potential place relevant to all 4 parts » Value: • Flexibility for Parts l, ll, and IV • Standardization and authenticity for Part lll • Part IV – potential to addresses important, but not easily measured, content
» Requirements: • Evidence to support the value in MOC - Transferability to practice performance - Leads to improvement in competence/performance 21
What: A 6 hour simulation program focused on personal improvement, deliberate practice of less-used skills, and expert feedback. How: Provision of MOC "credit" for educational efforts in the network of simulation centers across the country, the Simulation Academy of SAEM Why: Engaging and meaningful educational opportunities for life-long learners An obligation to use the best tools available for ongoing maintenance of certification. Based on 2006 data, there were 58 simulation centers managed by EM programs nationally
Satisfaction of ABMS MOC parts II and IV
Training and Evaluation Environment
Content
Supportive
Relevant
"Safe" environment
Concrete
Evidence based
Best practices
Learning Contract
Free of bias
Model of Practice
ABEM KSAs
Learning Conversation
Honest
Procedural competencies
Clinical Guidelines
Rigorous
Timely
Simulation-based Maintenance of Certification for Emergency Medicine
Every Three years
MOC option
CME option
Variety of topics
Catalog of courses available on website
MOL option
Covers "core" content
New techniques
Variety of locations
Output measures
Variety of dates
Pre-arrival
Arrival
On completion
Pre-test
CME credit
Reading
Case-based e-sims
Satisfaction survey Highly interactive event 4-6 hours
Certificate of completion Verification letter
Post-test Baseline Knowledge Data Medical decision collection making
Registration Complete
Pre-Post Measures
Data collection
Decay analysis Learning Outcomes
Data collection
Initial Offerings in Simulation-based Maintenance of Certification for Emergency Medicine
Diplomates would have an opportunity to select from an initial panel of four courses. Any of the selections would fulfill the MOC requirement. Selectivity allows the adult learner: autonomy and self-determination to refine and tailor to individual life experiences and knowledge content relevant to specific goals self-relevancy practical skills and deliberate practice
Emergency Trauma Care Management
The Emergent Airway
Critical Resource Management, the Crashing Patient
Incorporating Emergency Ultrasonography into Clinical Care
Simulation for Maintenance of Certification Planning Guide
What are we looking for from our certification? What do we get? What do we give to our profession? When the American Board of Medical Specialties set out the guidelines for Maintenance of Certification (MOC), that board described a need to maintain the care of patients at the highest quality. That is what we all seek. Over time we have read reams of written materials, taken a score of exams, maintained our licensure, and done a good job taking care of our patients. New devices, new techniques, and new technologies have come along which have made our work safer and more complex. We have maintained.
Clinical Simulation for Patient Safety and Performance Improvement
Here is an opportunity to learn by doing, to advance our skills at our own pace, and to refresh our thinking. We can learn together through simulations that engage and challenge, guided by colleagues who are knowledgeable and supportive. Clinical simulation has come a long way since the CPR dummy. These devices keep pace with the most current technologies used in our clinical work. It is time to get something substantial back from our MOC process. The experience will be challenging, and effortful, no doubt, but the rewards will be relevant and substantial.
Establishing a US-wide MOC network will take time and substantial effort to coordinate. With any process of this magnitude, careful planning is key, followed by proof-of-concept testing in the field. Initial steps include: vetting of curricula site visits at prospective centers creation of data-sharing agreements establishing a shared IS infrastructure Already in progress, several potential initial sites are being evaluated, using a rubric adapted from several simulation accreditation bodies. At this time, Northwestern University has been designated the coordinating center for this effort. This is, in part, due to the proximity of SAEM headquarters, and through the leadership of that center’s director in the field of MOC using simulation.
2
The goal is to have approximately six centers up and running a series of shared CME-MOC courses within the first year. In future steps, approximately thirty centers will be identified to serve the over 25,000 diplomates in Emergency Medicine. Each center will commit to providing an outstanding learning experience to ABEM’s diplomates, to share data on course and learner performance, and to contribute to future course offerings. Each center will provide, at minimum one SMOCEM course per month.
3
What are we looking for from our certification? What do we get? What do we give to our profession? When the American Board of Medical Specialties set out the guidelines for Maintenance of Certification (MOC), that board described a need to maintain the care of patients at the highest quality. That is what we all seek. Over time we have read reams of written materials, taken a score of exams, maintained our licensure, and done a good job taking care of our patients. New devices, new techniques, and new technologies have come along which have made our work safer and more complex. We have maintained.
Clinical Simulation for Patient Safety and Performance Improvement
Here is an opportunity to learn by doing, to advance our skills at our own pace, and to refresh our thinking. We can learn together through simulations that engage and challenge, guided by colleagues who are knowledgeable and supportive. Clinical simulation has come a long way since the CPR dummy. These devices keep pace with the most current technologies used in our clinical work. It is time to get something substantial back from our MOC process. The experience will be challenging, and effortful, no doubt, but the rewards will be relevant and substantial.
Establishing a US-wide MOC network will take time and substantial effort to coordinate. With any process of this magnitude, careful planning is key, followed by proof-of-concept testing in the field. Initial steps include: vetting of curricula site visits at prospective centers creation of data-sharing agreements establishing a shared IS infrastructure Already in progress, several potential initial sites are being evaluated, using a rubric adapted from several simulation accreditation bodies. At this time, Northwestern University has been designated the coordinating center for this effort. This is, in part, due to the proximity of SAEM headquarters, and through the leadership of that center’s director in the field of MOC using simulation.
2
The goal is to have approximately six centers up and running a series of shared CME-MOC courses within the first year. In future steps, approximately thirty centers will be identified to serve the over 25,000 diplomates in Emergency Medicine. Each center will commit to providing an outstanding learning experience to ABEM’s diplomates, to share data on course and learner performance, and to contribute to future course offerings. Each center will provide, at minimum one SMOCEM course per month.
3
What are these ‘initial courses’ all about? In general, these courses are intensely hands-on and practical. There is a roughly 50:50 ratio of new content and didactic material and hands-on and deliberate practice. They use best-practices in the field of simulation for clinical education to ensure a learning experience of the highest quality. Learners will find that instructors are experienced simulation practitioners who bring generous respect for professional learners’ life experiences. The best simulation experiences are bolstered by adult learning theory, recognizing that when an adult learner has control over the nature, timing, and direction of the learning process, the entire experience is facilitated.
What will centers need to have to participate? ^ŝŵƵůĂƟŽŶĐĂĚĞŵLJĞŶƚĞƌ ǀĂůƵĂƟŽŶ A vetting process will identify features of your simulation program which may be important to future Simulation-based Maintenance of Certification in Emergency Medicine activities. SMOCEM activities are designed to: - Provide a relevant and meaningful experience to the ABEM Diplomate - Accomplish the MOC Part IV requirements set by ABEM. Centers which will engage in SMOCEM activities will demonstrate: - A history of high quality educational offerings. - Faculty resources necessary to provide quality education - Procedures for faculty training and evaluation. - An infrastructure appropriate for regular and frequent SMOCEM activities including: o equipment, o space allocation, o assessment infrastructure o technology support o administrative and financial support o SMOCEM course director responsible for event quality - Policies and procedures that assure a confidential and secure environment for participants. - A faculty which includes current ABEM diplomates who participate in the educational offerings
ĚƵĐĂƟŽŶĂůKīĞƌŝŶŐƐ Selected centers and programs will have current curricular offerings, including examples of curricula and objectives for:
Who is paying for all of this? A common question. It is a question that requires a deeper analysis of the current practices in continuing professional education, and how these efforts have been supported in the past. One might ask first: Is the current process working? What do professionals get from their current continuous learning options? Who is paying for this now? Are these sources getting anything of value in return? If a practicing physician takes a SMOCEM course, then they come away with a high-value, hands-on, practical, and relevant experience that is germane to their work environment. These courses generate CME credits, and will be paid for by the same mechanisms that physicians currently use to obtain CME. This time, the courses will be more personal, and will enhance the practice of emergency medicine through new essential skills and refreshment of skills last used in residency.
4
1. Post Graduate Education training (CME or Practicing Physicians 2. Interprofessional Education (teams of physicians and nurses, etc) 3. Graduate Medical Education (GME) training. Selected centers and programs will have a curriculum and scenario development process: a. Needs assessments b. Development of Objectives c. Selection of Methods d. Creation of Materials e. Delivery f. Assessment of Learner Performance g. Assessment of Materials h. Measures of overall effectiveness
/ŶƐƚƌƵĐƚŽƌƐĂŶĚWĞƌƐŽŶŶĞů Selected centers and programs will have a process which ensures instructor quality. The Simulation Academy will offer faculty development programs for those interested in becoming facilitators.
5
What are these ‘initial courses’ all about? In general, these courses are intensely hands-on and practical. There is a roughly 50:50 ratio of new content and didactic material and hands-on and deliberate practice. They use best-practices in the field of simulation for clinical education to ensure a learning experience of the highest quality. Learners will find that instructors are experienced simulation practitioners who bring generous respect for professional learners’ life experiences. The best simulation experiences are bolstered by adult learning theory, recognizing that when an adult learner has control over the nature, timing, and direction of the learning process, the entire experience is facilitated.
What will centers need to have to participate? ^ŝŵƵůĂƟŽŶĐĂĚĞŵLJĞŶƚĞƌ ǀĂůƵĂƟŽŶ A vetting process will identify features of your simulation program which may be important to future Simulation-based Maintenance of Certification in Emergency Medicine activities. SMOCEM activities are designed to: - Provide a relevant and meaningful experience to the ABEM Diplomate - Accomplish the MOC Part IV requirements set by ABEM. Centers which will engage in SMOCEM activities will demonstrate: - A history of high quality educational offerings. - Faculty resources necessary to provide quality education - Procedures for faculty training and evaluation. - An infrastructure appropriate for regular and frequent SMOCEM activities including: o equipment, o space allocation, o assessment infrastructure o technology support o administrative and financial support o SMOCEM course director responsible for event quality - Policies and procedures that assure a confidential and secure environment for participants. - A faculty which includes current ABEM diplomates who participate in the educational offerings
ĚƵĐĂƟŽŶĂůKīĞƌŝŶŐƐ Selected centers and programs will have current curricular offerings, including examples of curricula and objectives for:
Who is paying for all of this? A common question. It is a question that requires a deeper analysis of the current practices in continuing professional education, and how these efforts have been supported in the past. One might ask first: Is the current process working? What do professionals get from their current continuous learning options? Who is paying for this now? Are these sources getting anything of value in return? If a practicing physician takes a SMOCEM course, then they come away with a high-value, hands-on, practical, and relevant experience that is germane to their work environment. These courses generate CME credits, and will be paid for by the same mechanisms that physicians currently use to obtain CME. This time, the courses will be more personal, and will enhance the practice of emergency medicine through new essential skills and refreshment of skills last used in residency.
4
1. Post Graduate Education training (CME or Practicing Physicians 2. Interprofessional Education (teams of physicians and nurses, etc) 3. Graduate Medical Education (GME) training. Selected centers and programs will have a curriculum and scenario development process: a. Needs assessments b. Development of Objectives c. Selection of Methods d. Creation of Materials e. Delivery f. Assessment of Learner Performance g. Assessment of Materials h. Measures of overall effectiveness
/ŶƐƚƌƵĐƚŽƌƐĂŶĚWĞƌƐŽŶŶĞů Selected centers and programs will have a process which ensures instructor quality. The Simulation Academy will offer faculty development programs for those interested in becoming facilitators.
5
Selected centers and programs will have robust support from (non-instructor) personnel and a support infrastructure, including administrative and IT functions, and the security measures in place to assure participant confidentiality.
What are the ‘initial courses’ ? Advances in Emergency Management of Critically Ill patients (adult and peds—non-trauma) Advances in Trauma care
^ƉĂĐĞĂŶĚ ƋƵŝƉŵĞŶƚ Our goal is to initially enlist centers which can handle 20-30 learners, and has the appropriate educational, storage and support facilities to make events comfortable for learners Certainly, participating centers will have available a sufficient number and type of relevant simulation technologies (manikins, virtual reality, computer-based simulation, etc.) to deploy the curriculum. These centers will require access to a number of other clinical devices including;; Ultrasound equipment Fluid infusers, ventilators Airway trainers and adjuncts Pediatric and neonatal resuscitation equipment Adult and pediatric trauma equipment AV and related software systems will be needed to facilitate debriefing and assessment as well as information technology (IT) used to schedule and organize courses and curriculum.
KƚŚĞƌŬĞLJŝƐƐƵĞƐŝŶĐůƵĚĞ͗
ƵƐƚŽŵĞƌͲƌĞůĂƚĞĚƐĞƌǀŝĐĞ͕ ĂŶŽŶŐŽŝŶŐ ǀĂůƵĂƟŽŶŽĨWƌŽŐƌĂŵ īĞĐƟǀĞŶĞƐƐ͕ĂŶĚ Ă,ŝƐƚŽƌLJŽĨŽůůĂďŽƌĂƟŽŶ
6
Advances in Emergency Medicine Procedures Advances in Ultrasound Advances in Airway Management
[not all centers will offer all courses] Events are based in adult learning theory and use “real- time peer review and reflection, arguably the most effective approach to adult learning” The goal “is to refresh and revitalize skills and to reveal new techniques and protocols” Events will provide practicing emergency physicians with: - opportunities to receive supportive feedback on simulation- based performance, based on objective standards;; - opportunities to practice with newer techniques or devices sufficient to demonstrate competence;; - a personalized review of the skills required to maintain competence;; and - tools to continue practice at or near their home environment. 7
Selected centers and programs will have robust support from (non-instructor) personnel and a support infrastructure, including administrative and IT functions, and the security measures in place to assure participant confidentiality.
What are the ‘initial courses’ ? Advances in Emergency Management of Critically Ill patients (adult and peds—non-trauma) Advances in Trauma care
^ƉĂĐĞĂŶĚ ƋƵŝƉŵĞŶƚ Our goal is to initially enlist centers which can handle 20-30 learners, and has the appropriate educational, storage and support facilities to make events comfortable for learners Certainly, participating centers will have available a sufficient number and type of relevant simulation technologies (manikins, virtual reality, computer-based simulation, etc.) to deploy the curriculum. These centers will require access to a number of other clinical devices including;; Ultrasound equipment Fluid infusers, ventilators Airway trainers and adjuncts Pediatric and neonatal resuscitation equipment Adult and pediatric trauma equipment AV and related software systems will be needed to facilitate debriefing and assessment as well as information technology (IT) used to schedule and organize courses and curriculum.
KƚŚĞƌŬĞLJŝƐƐƵĞƐŝŶĐůƵĚĞ͗
ƵƐƚŽŵĞƌͲƌĞůĂƚĞĚƐĞƌǀŝĐĞ͕ ĂŶŽŶŐŽŝŶŐ ǀĂůƵĂƟŽŶŽĨWƌŽŐƌĂŵ īĞĐƟǀĞŶĞƐƐ͕ĂŶĚ Ă,ŝƐƚŽƌLJŽĨŽůůĂďŽƌĂƟŽŶ
6
Advances in Emergency Medicine Procedures Advances in Ultrasound Advances in Airway Management
[not all centers will offer all courses] Events are based in adult learning theory and use “real- time peer review and reflection, arguably the most effective approach to adult learning” The goal “is to refresh and revitalize skills and to reveal new techniques and protocols” Events will provide practicing emergency physicians with: - opportunities to receive supportive feedback on simulation- based performance, based on objective standards;; - opportunities to practice with newer techniques or devices sufficient to demonstrate competence;; - a personalized review of the skills required to maintain competence;; and - tools to continue practice at or near their home environment. 7
Simulation for Maintenance of Certification Course Offerings
What are the ‘initial courses’ ? Advances in Emergency Management of Critically Ill patients (adult and peds—non-trauma) Advances in Trauma care Advances in Emergency Medicine Procedures Advances in Ultrasound Advances in Airway Management
[not all centers will offer all courses] Events are based in adult learning theory and use “real- time peer review and reflection, arguably the most effective approach to adult learning” The goal “is to refresh and revitalize skills and to reveal new techniques and protocols” Events will provide practicing emergency physicians with: - opportunities to receive supportive feedback on simulationbased performance, based on objective standards; - opportunities to practice with newer techniques or devices sufficient to demonstrate competence; - a personalized review of the skills required to maintain competence; and - tools to continue practice at or near their home environment. 2
Advances in Emergency Management of Critically Ill patients (adult and peds—non-trauma)
Course Format Instructor to Learner Ratio: 5:1 2.5 hours Didactic Content High fidelity simulation Course Objectives At course completion, the learner will have sucessfully managed simulations of lifethreatening illness presented in adult and pediatric cases.
Day 1: AM Didactics Learning Scenarios Day1: PM
Day 2: AM Practice and Self Assessment Optional Remedial Self Assessment
Learning Scenarios Self Assessment
The learner will also demonstrate the ability to perform relevant procedures, diagnostic skills, and will manage a team of caregivers in simulation scenarios.
Techniques and Technologies Pre-course Preparation delivered electronically via Learning Management System Lecture format for updates and to refine knowledge in Plenary sessions High fidelity simulation scenarios in group ratios of 1:5 Practical skills training in CVC placement and estimation of central venous pressure via task trainers and ultrasound Self-assessment scenarios with high fidelity simulators and task trainers in ratios of 1:3 Post-course survey delivered electronically via Learning Management System Course discounts provided for those interested in participating in educational research
3
Didactic Topics and Simulation Scenarios Emergency Management of the Critically Ill Pediatric Patient Neonatal and Pediatric Medical Resuscitation Sepsis Diabetic Ketoacidosis Accidental Ingestions Cardiomyopathy Procedures: Rapid Sequence Intubation Interosseus Lines Emergency Management of the Critically Ill Adult Patient Adult Resuscitation Sepsis Toxidromes Hypertensive Crisis and Neurologic Emergencies Procedures: Transvenous Pacemaker Placement Assessment of Central Venous Pressure by Bedside Ultrasound Central Line Placement via Ultrasound Guidance General Topics Team Management and Coordinated Communication
4
Process map [Network Operations]: MOC in Emergency Medicine Simulation Network Coordinating Center Content Blueprint
Professional Learner
Learning Center
Facilitator
Determines Capacity
Provides Availability
Internet Course selections and Registration
Prepares Center Course Registration Create Facilitator Tools Attends Event
M M
Prepares Facilitator
Receives Training
Learning Event
Facilitates/ Provides Instruction
Content Feedback Learner Feedback
Provides Feedback
Pooled facilitator feedback
Provides Content Feedback
Provides Learner Feedback Collates/Analyzes Feedback
Learner Feedback Center feedback
Generates CME Certificate Compensates Center and Faculty
Facilitator feedback
CME-MOC Certificate Facility Fee
Faculty Fee Vozenilek 12-20-10
