Evaluating Teaching Effectiveness of a Patient Safety Communication Technique S U S A N A. R E E V E S, E D D, R N D A R T M O U T H - H I T C H C O C K M E D I C A L C E N T E R C O L B Y - S A W Y E R C O L L E G E T H E J A M E S M. J E F F O R D S C E N T E R F O R P O L I C Y S T U D I E S A T T H E U N I V E R S I T Y O F V E R M O N T
A Story of Communication
A Study of Teaching Effectiveness
4. Incorporation into Practice Factors that enable or prohibit knowledge, skills and attitudes transfer into practice 1. Practice Environment Safety Regulatory Mandates Ethical Imperatives Organizational Culture 3. Evaluation of Knowledge Attitudes Skills (Competency Evaluation) 2. Academic Environment Content (What?, When?, Pedagogy) Attributes of Environment Attributes of Teacher The Practice/Education Innovation Cycle
Research Purpose To explore the new graduates experience with the use of I-SBAR-R technique in practice in order to evaluate the effectiveness of the learning experience they had in their undergraduate program
Research Questions What was the experience of new nurses when using the I- SBAR-R technique in their practice? Would additional teaching, practice or information, or the use of an alternative pedagogy during their undergraduate nursing education been more useful to them to learn the skill? Are the nurses continuing to use the technique in practice?
Methods Research sites Small, liberal arts college with baccalaureate nursing program Rurally situated Academic Medical Center
Methods (cont.) Research participants
Methods (cont.) Data Collection Document Review Interviews
Research Design Realist Evaluation WHAT is it about this intervention that works, for WHOM, in what CIRCUMSTANCES, in what RESPECTS and WHY -Pawson, Greenhalgh, Harvey & Walsh, 2005, p. 21
Realist Evaluation Employs the use of a middle range theory a hypothesis about how an intervention or a program will work Followed by an examination of context, mechanisms and outcome
Middle Range Theory: Pre-professional nursing students who are didactically taught the knowledge, attitude and skill of I-SBAR-R, followed by reinforcement in the clinical education setting, will incorporate the skill in their new graduate nursing practice. Table 1: The Teaching Intervention Context Mechanism(s) Outcome Senior Student Nurses taught rationale and skill of I-SBAR-R in senior-level baccalaureate course Senior Student Nurses required to use I-SBAR-R in actual clinical communication interaction with nurses during shift report and during a patient status change communication with an MD Didactic lecture imparts knowledge Articles re: Medical Error Case Study Practicing a new skill in a real situation builds confidence and competence in skill Needed to accomplish this skill for final evaluative review Students engaged Practiced a scenario with peer in class All senior students completed the skill, with debrief by faculty.
Findings
Context #1 Surgical Unit Mechanism -Nursing Residency Program (orientation training) stressed the use of I-SBAR-R and had practice modules for simulation -The preceptor (an experience nurse responsible for on-unit clinical orientation of a new nurse) also had knowledge of the technique and used in practice thus providing reinforcement through mentoring -Preponderance of nursing staff on the floor graduated within 0-5 years and had patient safety communication techniques taught in school and used them routinely in practice. (Older nurses did not.) -Creates an organizing framework and helps with sense-making in chaotic environment -Physicians pressure nurses to get to the point. A strong sense of being too rushed to communicate Outcome Participant routinely uses I-SBAR-R technique in practice
Context #4 Medical Unit Mechanism No organized orientation or use of simulation for I-SBAR-R Preceptor did not use I-SBAR-R to conduct hand-over communication. Recommended that participant find what works for you method for hand-over communication Written report forms structured in SBAR language Physicians are too rushed to listen to structured communication, regardless of whether or not the patient is known to them Participant uses a modified form of I-SBAR-R that she tailored for use in her specific practice area Outcome Doesn t tend to use the B (Background) as it takes too much time
Context #5 Medical Unit Mechanism Significant I-SBAR-R focus during orientation using simulation Preceptor uses I-SBAR-R and diligent in supporting participant to acquire the comfort in using the skill routinely Needing to learn to tailor I-SBAR-R to conform to physician variation in what elements are desired in a verbal exchange about patients, especially with on-service and off-service physicians Takes time to build confidence in verbal communication of patient information, whether in hand-over or reporting a change in condition (weeding out what is important and what is extraneous information) Outcome Using I-SBAR-R in practice, but learning to tailor according to physician preference
Context #6 Medical Unit Mechanism No formal orientation/simulation Multiple preceptors all with individual methods of structuring communication for hand-over and communicating change in patient status to physicians/others Pressure by physicians to get to the point Knowledge that physicians are stressed and busy Reminds self that a primary role she plays is patient advocacy Modified I-SBAR-R in use Outcome Typically drops the B for background
Context #8 Critical Care Unit Mechanism Focus in orientation on I-SBAR-R, including simulated experience Preceptor use of I-SBAR-R in practice with strong encouragement for participant to adopt practice Utilization of skill by majority of staff nurses within the work unit Physician expectation to receive information in I-SBAR-R format Outcome Routinely uses I-SBAR-R in practice
Context #2&3 Deliberate Dialogue
Significant Variation in Use of Technique Orientation + Preceptor + Increased Number of Recently Graduated Nurses = Uptake Absence of any one of the above saw decrease in use
The dropping of the B to save time.
Physician Interactions Rushed, pressured, dismissive, uncomprehending The doctors are always rushed they want to know why you called, what you want, and they want it in a hurry. Doctors just don t give you the time to say all you have to say. At least in my experience the physician has never given me the chance to say the full SBAR. As soon as you give the situation, the physician is already telling you what they want you to do.
Physician Interactions Need to tailor communication to individual preferences The value of patient advocacy was related to a nurse s persistence in communication with physicians I know I will never get into trouble if I am advocating for a patient. It is part of being a patient advocate getting exactly the response I think I need.
The Issue of Clinician Confidence It is an awkward place to be calling the doctors when maybe you weren t right, or what you thought was important wasn t. I don t know what the right thing to do is yet in a situation, so the recommendation thing is hard.
The use of I-SBAR-R to ground one s self Sometimes, you just feel like, Oh My God! Do I feel safe? That little bit of structure to keep me organized when I feel like I m in the weeds helps. It is a tool and I rely on the tool to keep me sane.
Teaching Effectiveness Curriculum Timing Innovation in Pedagogy for I-SBAR-R Training Need for More Interprofessional Communication Training Didactic Clinical (shift reports, change in condition, rounds) PHONE TRAINING
Realist evaluations do not produce generalizable results because the conclusions are highly contextual. Accordingly, a realist evaluation may lead, at best, to tentative recommendations. Pawson, et al. (2005)
Recommendations for Teaching Earlier Exposure to Content Consider Combination of I-SBAR-R introduction with SOAP Documentation Full Shift Reports During Clinical (8/12 hour) RESILIENCY LOW CONFIDENCE TRAINING MORE INTERPROFESSIONAL TRAINING
Recommendations for Practice Knowledge of How Different Contexts Effect Uptake Concerted Efforts to Educate ALL Practicing Clinicians Challenge the Necessity for Customized Communication Orientation is Enculturation Time
Summary Insights into the evaluation of a teaching intervention were highly valuable Benefit of Realist Evaluation (the 50% phenomena) Accreditation and regulatory pressures continue and challenge to squeeze new elements into already jammed curricula in schools of nursing and medicine are significant
There are few things more compelling in health care today than the need to improve patient safety As preprofessional educators, we are stewards of the health care system s future. As such, the responsibility is ours to make sure we are doing whatever we can to make the system better, more responsive and safer.
Acknowledgements The James M. Jeffords Center for Policy Studies at the University of Vermont Colby-Sawyer College Dartmouth-Hitchcock Medical Center Former Students/Participants