Laura Hempstead, DO, FACOFP AODME April 22/2015
Disclosures, Goals and Objectives Nothing to disclose Identify strategies to overcome initial barriers to starting a CCC Discuss ways to use a core competency of the block daily feedback evaluation Utilize alarm bells to identify underperforming residents early and intervene Develop a global behaviorally based evaluation linking milestones to the evaluation.
Clinical Competency Committee Nursing and Ancillary Personnel Evaluations Procedural Evaluations OSCE Mock Orals ITE End-of- Rotation Evaluations Sim Lab Clinical Competency Committee Self Evaluations Case Logs Unsolicited Comments Student Evaluations Clinic Workplace Evaluations Peer Evaluations Assessment of Milestones Patient/ Family Evaluations
Who should be members of CCC Program director/associate PD Core faculty from IM, OB, family medicine center, other key rotations Residency coordinator Clinic manager/pcmh manager Behaviorist and/or social worker Invite advisors when their residents are being discussed Input from chief residents and RICs crucial
The milestones are overwhelming, how do we get started? Faculty education on content of milestones Resident education on content of milestones Quick-reference sheets provided for all faculty and residents Alarm bell reference sheets provided for all faculty and residents
More on Milestones: Individualizing them to your program Set aside a full half day (you may need 2) Tape the individual milestones to the walls/place on tables around a large room Provide 2-3 colors of sticky notes to each meeting attendee (include all interested residents/faculty and support staff) Each attendee writes their suggestions for behaviors on one color, suggestions for curriculum improvement in another color, fellowship suggestions in a 3 rd color.
Milestone Template
CCC Agenda Template Week 1 review multirater evaluations on the 7 residents being evaluated for the block. Week 2 review core competency of the block daily feedback evaluations on the 7 residents being evaluated for the block Week 3 OB resident in charge (RIC) and faculty present at beginning of meeting - strengths and areas for improvement of each OB team member Week 4 IM RIC and faculty present strengths and areas for improvement of each IM team member Every week/additional meetings if needed milestone evaluations for the 7 residents of the block, discuss residents with IEPs not fully addressed during weekly meetings. Monitor success of evaluation tools. Implement new evaluation processes and tools if indicated.
Evaluations used at UMKC Family Medicine Residency Program General rotation Evaluation behaviorally based, used for all rotations (Faculty to resident, peer to peer) Multi-rater evaluation (nursing, non-physician staff) Alarm Bells evaluation (on the fly, not mandatory) Daily outpatient feedback evaluation RIC evaluation (Faculty to RIC, Resident to RIC) Quarterly self-evaluation Portfolio (Test scores, longitudinal projects etc.)
Core Competency of the Block Evaluation Medical Knowledge
Alarm Bells resource
Resident Quarterly Self-assessment Free Text document Addresses between 1 and 3 of the milestone behaviors for each core competency Most of these behaviors are not readily observable by faculty Completed after 1 st and 3 rd quarters of each academic year.
Resident Quarterly Self-assessment Medical Knowledge: MK-1 Describe your plan of study to improve medical knowledge: Address how you use the In- Service exam to guide areas for further reading and study. Include comments about your daily study strategy. Patient Care : PC3 Describe how you utilize PCMH principles in your patient care and your goals for improvement. Examples: motivational interviewing, pre-visit planning, setting goals with patients, shared decision making, using community resources etc. Interpersonal skills and Communication: C-1, C-2, C-3, P-4 Provide an example of how you manage stress. Comment on your ability to remain calm in stressful situations. Comment on your stress management goals. Describe how you give non-judgmental feedback to patients, staff and peers. Lists goals for improvement.
Resident Quarterly Self-assessment Professionalism: P-2, P-4 Describe how you maintain work-life balance and utilize wellness principles in your own life. Describe how you exhibit professionalism including a comment on your timeliness, accountability and sense of responsibility. List specific goals you for improvement. Practice Based Learning and Improvement: PBLI-2, P-2, P-5, C-4 Describe your plans for future electives and how these electives will benefit your future practice. Describe the apps and resources you use to assist with patient care. Describe how you use the library resources. System Based Practice and Improvement: SBP-2, PBLI-1 Describe how you report medical errors. Discuss how you participate in M&M to learn how to improve our system. List system improvement areas you are working on. Discuss how you use protocols and evidence-based guidelines to improve patient safety and transitions of care. Outline your plans to improve in this area. Describe your QI project/research Project (PGY1 and 2 describe topics of interest and how you are acquiring the tools necessary to complete a research project.)
Global Rotation Evaluations Behaviorally based Questions limited to 12 Requires faculty training - Has the option to rate each area evaluated as appropriate for PGY level (globally) or evaluate specific milestones Doesn t link to every milestone. (These are addressed using Core Competency of the block evaluations and resident self-evaluations.)
Overall Rotation Evaluation expectations Level 0 Alarm Bell Behavior. Resident needs continuous supervision, probation or improvement plan. Some behaviors could trigger immediate suspension. (Comments required) Level 1 Level of medical student/new intern in first 6 months of training. If upper level resident is performing at this level, recommend improvement plan in comments. Level 2 Progressing PGY1 or PGY2 level Level 3 Progressing PGY3 level. Level 4 Resident has achieved target milestone for independent practice Level 5 - Highly accomplished physician, mentor and leader (Comments required) N/A If you did not observe this behavior during your rotation.
Rapport: Establishes an ongoing relationship with patients that inspires trust and confidence, elicits cultural factors that impact patient health. (PC-2, PC-4, SPB-4, C-1, P-3) 0-Unable to provide clear verbal explanations to patients or health care team, unable to work effectively with patient and family members. Complains and projects negative attitudes with patients and families. Poor interpersonal skills promote lack of confidence. 1-Communicates clearly with patients at appropriate literacy level and treats patients respectfully. Recognizes cultural impact on health. 2-Creates and sustains nurturing relationships with patients and families, uses minimal jargon and considers patient s literacy level. Provides consistent patient education and considers cultural factors. 3- Creates collaborative relationships with patients which foster patient self-management and trust. Incorporates patient s beliefs and cultural practices into health care goal setting with patients. 4-Creates nurturing relationships with difficult and complex patients, and those with cultural barriers to care. Partners well with complex patients to meet their health care needs. 5-Is a leader in cultural proficiency. Develops organizational policies and educational resources to support patient self-management skills and health care goal setting.
Global Rotation Evaluations Behaviorally based Questions limited to 12 Requires faculty training - Has the option to rate each area evaluated as appropriate for PGY level (globally) or evaluate specific milestones Doesn t link to every milestone. (These are addressed using Core Competency of the block evaluations and resident self-evaluations.)
Faculty response to new evaluations Old habits die hard! Alarm bells utilized more initially, faculty are reverting to face-to-face conversation to express concerns Core competency of the block adopted by several faculty not used regularly by others Global rotation has received positive feedback. Shorter than our prior evaluations. There was some initial confusion. Comments section utilized more formatively after training.
References http://acgme.org/acgmeweb/portals/0/pdfs/milestones/f amilymedicinemilestones.pdf Swing SR. The ACGME outcome project: Retrospective and prospective. Med Teach. 2007;29:648 654 http://journals.lww.com/academicmedicine/fulltext/2015/ 04000/Competencies,_Milestones,_and_Entrustable.1.aspx
Questions/Discussion