Analysis of a Clinical Evaluation Tool Teresa Connolly, PhD, RN, CNRN Brenda Owen, MSN, CNM, RN Glenda Robertson, MA, RN Joan Ward, MS, RN, CNE

Transcription

1 HEADLINE Analysis of a Clinical Evaluation Tool Teresa Connolly, PhD, RN, CNRN Brenda Owen, MSN, CNM, RN Glenda Robertson, MA, RN Joan Ward, MS, RN, CNE

2 Session Disclosure 1. Approval Statement: This continuing education activity was approved by the Western Multi-State Division, an accredited approver by the American Nurses Credentialing Center s Commission on Accreditation. Arizona, Colorado, Idaho, and Utah Nurses Associations are members of the Western Multi-State Division. Approval # Criteria for Successful Completion: To receive contact hours, participants must check-in to the session using the barcode scanner, attend the entire session and then complete both the session evaluation and full conference evaluation by July 31, Conflicts of Interest: This educational activity does not include any content that relates to the products and/or services of a commercial interest that would create a conflict of interest. 4. Commercial Support: There is no commercial support being received for this session.

3 Learning Objectives Explain how a QSEN based, clinical evaluation tool aims to measure nursing students clinical performance in the areas of essential skills, professional behaviors, and communication with diverse patients, populations, and healthcare teams. Describe essential elements to include for a clinical evaluation tool based on our pilot results Incorporate key findings when developing or revising your own clinical evaluation tool

4 Key Findings

5 Goals for Redesigning our Tool Create leveling of evaluation throughout all undergraduate clinical courses Ease of use and clarity of clinical evaluation Tie our clinical learning experiences together: hospital clinical site and simulation evaluation PURPOSE OF NEW CLINICAL EVALUATION TOOL

6 Reference Guide One document that has all the elements that students could be evaluated on throughout their clinical courses Speaks to the diversity of the courses, clinical sites, and scholars not focused on just inhospital care

7 METHODS

8 Methods Used QSEN As Guiding Principles Patient-Centered Care Teamwork and Collaboration Quality Improvement Evidence-Based Practice Safety Informatics

9 Methods Reviewed Current Undergraduate Clinical Evaluation Tools Components Sub-Categorized Into QSEN Themes & Elements Developed

10 Methods Domain (QSEN) Patient Centered Care Theme Cultural Diversity and Inclusiveness Element Explain the role a patient s culture plays in their healthcare

11 Methods Domain (QSEN) Patient Centered Care Leveling Bloom s Taxonomy Theme Cultural Diversity and Inclusiveness Beginning Understanding Element Explain the role a patient s culture plays in their healthcare Intermediate Applying Advanced Evaluating

12 Likert Scale Patient Centered Care* Recognize the patient or designee as the source of control and full partner in providing compassionate and coordinated care based on respect for patient s preferences, values, and needs. Cultural diversity and inclusiveness Element Element Element 0 Never 1 Rarely 2 Sometimes 3 Often 4 Always No Opportunity

13 Pass/Fail Determination The current tool was used for the formal determination of clinical pass/fail decisions Pilot-tool Failure- any theme receiving a 0 at the end of the course All other scores meant either beginning to develop competency or had developed proficiency

14 Pilot Population All courses with a clinical component (two groups each) o Med-Surg I o Pediatrics o Obstetrics o Mental Health o Complex Care o Community Health o Immersion (only 1 group) Experienced clinical scholars were used for the pilot o At least 1 year with the CON and past experience with multiple rotations at that agency Sections were included to evaluate experiences in the Clinical agencies and Simulation/lab settings

15 RESULTS

16 Results Response to the surveys Respondents Percentage Total Educators (Clinical instructors, course faculty, SIM/Lab) 64% N=20 Learners 47% N=75

17 Results Do both the current tool and Pilot tool produce the same results? YES! 100% agreement on pass/fail decisions One clinical failure during the pilot period and it was identified on both tools.

18 Results Which measurement scale did you prefer to be evaluated on? Respondents Percentage Dichotomous Likert Educators (Clinical instructors, course faculty, SIM/Lab) 20% 60% Learners 22% 78%

19 Results Did raters believe the Pilot tool gave them an accurate rating of the students performance compared to the current tool Yes! No significant difference in raters evaluation of the tool as an assessment of student performance Pilot tool was slightly less effective for communicating strengths and weaknesses Comment boxes and mid-term/final evaluation sections were purposely not included on the Pilot tool due to redundancy o Themes potentially too abstract o Not part of the natural language used by preceptors

20 Results Did the Pilot tool contain feedback about student/my performance? Respondents Percentage Disagree Neutral Agree Educators (Clinical instructors, course faculty, SIM/Lab) 25% 0% 65% Learners 9% 14% 77%

21 Results Did the current tool contain feedback about student/my performance? Respondents Educators (Clinical instructors, course faculty, SIM/Lab) Percentage Disagree Neutral Agree 15% 10% 60% Learners 0% 6% 94%

22 Results Were any of the themes inappropriate or were any themes missing? No Instructors felt themes were appropriate Some specific items were irrelevant (i.e. setting bed height for outpatient clinical setting) Major item people wanted more content was on patient and family communication

23 Results Which method do you prefer to evaluate professionalism in the clinical setting? Respondents Educators (Clinical instructors, course faculty, SIM/Lab) Percentage Graded Contract Grade and contract 5% 25% 55% Learners 6% 59% 34%

24 QUALITATIVE DATA

25 Qualitative Data Was the tool valid based on agreement with preceptors qualitative feedback? Not necessarily Preceptors comments were coded into the themes used on the Pilot tool Some themes were never discussed as either a strength or weakness Unable to place some comments into the themes-- for example: The student has a bright smile. We noted some negative relationships--the instructor stated it was a strength on the Likert scale (3-4) and documented it as a weakness in the comments

26 Qualitative Data Did the tool tie the Simulation/lab and agency based clinical experiences together NO Pilot tool is a method of individual evaluation and simulation/lab clinical settings requires evaluation as a group.

27 Discussion

28 Discussion Despite reliability issues there are positive signs for validity of the QSEN-based themes The Clinical Evaluation Tool (CET) has the advantage of the Likert scale

29 Discussion What we kept from the Pilot tool o QSEN Leveling Likert o Professionalism contract and rated theme

30 Discussion Specific changes made o Combined/eliminated redundant themes Decreased the number of elements to choose from Used more of the natural instructor language as anchors

31 Pilot Tool Patient Centered Care* Recognize the patient or designee as the source of control and full partner in providing compassionate and coordinated care based on respect for patient s preferences, values, and needs. Communication with the Patient Recognizes the boundaries of therapeutic relationships Describes patient findings accurately, completely, and concisely to the instructor/preceptor Explains the elements of competent, confident and professional bedside manner to the instructor/preceptor Asks patient for permission to provide nursing care Utilizes a wide range of effective communication strategies and interpersonal skills to appropriately establish, maintain, re-establish and terminate the nurse-client relationship Utilizes communication, motivational interviewing (verbal and nonverbal) that is patient/population specific: Open-ended, Affirming (empathetic), Reflective (reflect content, emotion, intention), Summarizes message Responds to patient s questions or comments appropriately and in a timely manner Maintains competent, confident and professional bedside manner Demonstrates professional communication Assess own level of communication skill during encounters with patients, families, and/or population Adapts communication style (verbal and nonverbal) appropriately to the situation, patient, family, or population 0 Never 1 Rarely 2 Sometimes 3 Often 4 Always No Opportunity

32 Clinical Evaluation Tool Patient Centered Care* Recognize the patient or designee as the source of control and full partner in providing compassionate and coordinated care based on respect for patient s preferences, values, and needs. 0 Never MIDTERM 1 Rarely 2 Sometimes 3 Often 4 Always No Opportunity 0 Never 1 Rarely FINAL 2 Sometimes 3 Often 4 Always No Opportunity Communication with the Patient and Family Recognizes the boundaries of therapeutic relationships Utilizes therapeutic communication strategies to maintain a professional bedside manner Adapts therapeutic communication style as appropriate to the situation

33 Patient Centered Care* Recognize the patient or designee as the source of control and full partner in providing compassionate and coordinated care based on respect for patient s preferences, values, and needs. 0 Never MIDTERM 1 Rarely 2 Sometimes 3 Often 4 Always No Opportunity 0 Never 1 Rarely FINAL 2 Sometimes 3 Often 4 Always No Opportunity Cultural Diversity and Inclusiveness MS-1 OB PRAC BEGINNING Recognizes the influence personally held beliefs and/or experiences have on clinical performance INTERMEDIATE Provides care that incorporates an awareness of human diversity, including cultural, ethnic, and social factors. ADVANCED Integrates patient/ community values, preferences, and expressed needs throughout the care giving process

34 Discussion Other considerations: Quality improvement- not obtainable in our undergraduate clinical experience o Undergraduate vs. graduate levels o Applied to an RN to BSN program- in this program QI could be part of the evaluation as many are working in agencies where they can implement a QI project o Incorporation of minimum required elements and adding stretch goals at the next highest level

35 Additional Plans An orientation to the tool and curriculum for all clinical scholars is being developed Plan implementation of the new tool in Fall 2017 Conversion to on-line form in Spring 2018

36 Key findings & Questions?

37 Session Evaluation To evaluate this session, please do the following: Go to 2017necintherockies.sched.com/mobile (the online schedule) on your mobile device Click on the session you attended Press Feedback Survey Complete survey for the session *Remember to enter your unique identifier located on the back of your badge when completing survey.

38 HEADLINE Thank You!