2/15/2017. Continuous Quality Improvement as a Strategy to Improve NCLEX Scores. In Objectives

Transcription

1 2 Objectives Continuous Quality Improvement as a Strategy to Improve NCLEX Scores Cheryl L Mee MSN, MBA, RN Susan Sportsman PHD, RN, ANEF, FAAN Analyze aggregate results of faculty made tests, standardized examinations, and past NCLEX RN aggregate scores to identify weaknesses in curriculum and individual student performance. Use data to revise curriculum to improve identified weaknesses. Develop student remediation tactics that foster positive outcomes. Implement teaching learning actions that prepare students to successfully pass NCLEX exam. 3 First time, USA Education NCLEX Pass Rate, Percent Passing Category Total RN Diploma BSN ADN PN In ,236 novice nurses graduated from a prelicensure RN program but were unable to practice after taking the NCLEX for the first time! 1863 PNs did not pass on the first attempt 1

2 5 6 A path to performance improvement Act Plan Act Plan Act Plan Study Do Study Do Act Plan Study Do W Edwards Deming Aguayo, Rafael (1990), Dr. Deming, The American Who Taught the Japanese About Quality, Simon & Schuster, p. 76 Study Do 7 8 Methods to Predict Students who will be successful Grades in Previous Academic Programs Admission Scores Student Desire Essays Past Experiences Recommendations College Success Predictors Traditional Assessment Methods High School GPA GPA in other college courses Placement Tests Critical Thinking Tests-Writing Tests 2

3 9 10 Non-Traditional Students Delayed enrollments Attends part time Works full time (35 hrs or more /week) Financially independent (financial aid) Has dependents other than a spouse Single parent GED Just admit the smart students! It is easier to predict who will pass than it is to predict those that fail Non-Traditional Students are on a continuum An anonymous University President Are you using Admission Assessment Testing? Testing for success: A logistic regression analysis to determine which pre-admission exam best predicts success in an associate degree in nursing program Analysis - 5 years first to compare different admission assessment exams Determine which of three preadmission examinations best predicted success in an associate degree in nursing program. (NLN, HESI, ATI) HESI Admission Assessment examination scores can explain success in the nursing program better than the other tests included in this study. Manieri et al. (2015). Teaching and Learning in Nursing. 10,

4 13 14 The Problem with Admission Criteria There is no one criteria or tests score that will guarantee choosing the successful student. Success in various disciplines vary depending upon the competencies needed in the discipline. Concern about personal biases At Risk Students Clarifying the at risk profile, using admission criteria (A2 and other measures) Identifying students who meet the at risk criteria Identifying support strategies for at risk throughout the curriculum that go beyond the support given to all students Determine a process for identifying students who demonstrate at risk behaviors throughout the semester Are we singling out students for failure? NCLEX-Test Plan: Integrated Processes Nursing Process Caring Communication and Documentation Teaching/Learning Cultural/Spiritual Client Needs Categories and Distribution of Content unchanged Safe and Effective Care: - Management of Care - Safety and Infection Control 16-22% 8-14% 17-23% 9-15% 17-23% 9-15% Health Promotion and Maintenance 6-12% 6-12% 6-12% Psychosocial Integrity 6-12% 6-12% 6-12% Physiological Integrity - Basic Care and Comfort - Pharmacological and Parenteral Therapies - Reduction of Risk Potential - Physiological Adaptation 6-12% 13-19% 10-16% 11-17% 6-12% 12-18% 9-15% 11-17% 6-12% 12-18% 9-15% 11-17% 16 4

5 Should we teach to the test? 17 Steps to Success 18 Curriculum Data Analysis Policies Remediation Product Integration Student Preparation Considerations What research is available that demonstrates that your exams are reliable and valid? Are exams questions piloted to ensure that questions are psychometrically sound? Are the exams base on the NCLEX blueprint? How often is exam content updates? Qualifications of the item writers? Use of data? Security? Data Analysis Focus Cohort Data one point in time (cohort specific variables can affect data) Trending Cohort Data identify ongoing patterns 5

6 Are you reviewing and using data after each standardized exam? 21 Summary of Cohort Data Look for normal distribution patterns or bell curve Sample data student data Student name 6

7 25 26 Does the data make sense? Review cohort score in categories and number of items Does the mean cohort score make sense? Does the distribution of individual scores make sense? Does the individual student score correlate with students performance? Item detail Institution score vs. National score Are you trending exam data across time? 7

8 29 30 Trending vs Individual Exam Scores Trends and item details. Potential consistent curriculum weaknesses Curriculum gaps Curriculum strengths Exam Data -- CQI Who is reviewing the cohort data? Who is responsible for trending data and identifying concerns in cohorts? What happens to identified cohort concerns/weaknesses? Are all faculty involved and responsible for data analysis? Is exam data correlating with other assessment data? Is exam data trended to determine ongoing concerns? Are data analysis activities documented? Are identified interventions documented and follow up data analysis should reflect improvement 8

9 Remediation Develop student remediation tactics that foster positive outcomes. Integrative Review, multiple databases, articles The process of identifying the need to take action to remedy a situation that, if left unresolved, will result in unfavorable outcomes, whereas implementing intervention strategies will successfully address the situation. Culleiton, 2009 Remediation on the Rise! % of schools in a national sample required remediation for nursing students when assessment benchmarks were not achieved (Nibert, Young, & Britt, 2003) 35 Remediation frequently based on test performance Test Performance can be affected by non academic and personal factors affect test performance Test performance % of schools surveyed required nursing students to remediate after the initial failure to meet a benchmark and of these 37% required proof of remediation completion (Barton, Willson, Langford, & Schreiner, 2014). Remediation Test Anxiety Self Efficacy 9

10 38 Individualized Remediation Individual or group remediation? Individual remediation impacts performance Standardized exams help identify deficiencies Multiple strategies impacted NCLEX success Student contracts increased ownership and elevates the importance of remediation 39 Cohort Remediation Remediation for All? Targets the group s weaknesses Target test strategy and test anxiety Credited course Built into curriculum 10

11 41 Who needs remediation and when? Should remediation be for only high risk students? How do you define the high-risk student? Course scores and scores on standardized exams Groups participating in enhanced remediation had significantly higher NCLEX scores. Build value of remediation early Consider program long attention to remediation Remediation is a form of relearning and relearning not only enhances retention, it facilitates reacquisition of forgotten information. (Rawson and Dunlosky, 2013) Measuring remediation? Remediation strategies vary Quantifying the amount of remediation? Faculty expert opinion Time Measurable activities Retesting post remediation Barton, et al. (2014). Administrative Issues Journal, 4(2), HESI Validity 10 11

12 45 46 Remediation Policy Research Type of remediation interventions varied and progression and remediation policies were effective. No consistency in type of remediation is likely an indication that the progression and remediation policy motivated the students to study. (Morrison et al. 2002) Mandatory remediation HESI exit scores were significantly higher in programs that required remediation after failing to achieve benchmark. (Barton, Willson, Langford, and Schreiner, 2014) Instilling the Value of Remediation New Grads at work given standardized exams, remediation and retest. Students reported that in nursing program the test and remediation was not explained to them. They were more receptive when understood potential Curriculum Revision Act Plan Act Plan Study Do Study Do Act Plan Study Do 12

13 / Curriculum Validity 49 What Data is Available? NCLEX Test Blue Print (Detailed, Educator) 50 How can you tell? Review NCLEX Test Plan Review faculty made tests (test blue print) Review aggregate results of standardized tests against benchmark Represents practice environment Meets professional standards Client Need Categories Safe and Effective Care Management of Care Safety and Infection Control Health Maintenance and Promotion Psychological Integrity Physiological Integrity Basic Care and Comfort Pharmacology and Parenteral Therapy Risk Reduction Physiological Adaptation Integrated Processes Nursing Process Caring Communication and Documentation Teaching and Learning Culture and Spiritualty (new) What Data is Available? NCLEX Test Design Activities Related Content Standardized test data (Specialty, Exit) Results from faculty made tests Teacher-made and standardized exams comparison Teacher made Most common Less expensive More control of content Match chosen teaching materials Standardized Tests Nationally normed Diverse expert item writers Emphasis on higher order thinking Multilogical thinking Peer reviewed and edited Decrease legal risk Mapped to national or nursing organization content outline 13

14 53 54 Research on teacher-made exams Not Linked to NCLEX Test plan Not Peer Review No analysis of distractor discrimination 1 No assessment of cultural bias 1 Frequent low cognitive level 2 Violates good item writing protocol 2 Items too easy 3 Distractors often not chosen 1,3 Stimulates student anxiety 4 1. Killingsworth (2013); 2. Tarrant (2006); 3. Clifton & Schriner (2010); 4. Bailey (2012) Improving Faculty-made tests Map curriculum to NCLEX-test plan Tag Test questions to: NCLEX-Category Cognitive level Peer Review tests, using testing protocol Cognitive level Analyze test results Compare student results in faculty made exams to scores in standardized tests 55 Test Data Analysis: CQI Process 1. Analyze aggregate HESI data to identify weaknesses, particularly in NCLEX categories. 2. Compare poor performance on NCLEX Categories in HESI with low aggregate scores in test items tagged in identified NCLEX category. 3. Is there congruence between the two? If so, evaluate: Objectives Teaching-Learning activities Serendipity How might this CQI Process be integrated into the Program Evaluation? (Table Discussion) 14

15 58 Definition of facilitator Facilitated Discussion: Using the Group Process to Make Change A guide or discussion leader for the group Facilitation provides leadership without taking the reigns Get others to assume responsibility and take the lead Should the facilitator have a dog in the fight? Process Facilitator: How Methods and Procedures How relationships are maintained Tools being used The roles or norms set Content Leadership : What The subjects for discussion The Task The problem being solved The decision being made Groups The agenda Items Dynamics The goals Climates 15

16 61 Differences between Facilitators and Facilitative Leaders Characteristics Facilitator Facilitative leader Group Membership Third Party Group Leader Involvement in Substantive Issues Use of expertise Decision-making power No Substantially neutral Process Expert May have general content knowledge Deeply Involved in issue Process and content expert Yes Facilitation in Difficult Situations Getting your own house in order The Environment is disrupted Has the group made every effort to be collegial and supportive? Stop wishing the difficult folks would stop being difficult. You may be able to control the environment to reduce disruptions, but you can not control other people s behaviors. Participant interrupts frequently: Allowing the disrupter to manipulate the group in order to keep the peace rewards the troublemaker for objectionable behavior. Action: Redirect the speaker: Ok Mary, let s hear from Jane now. 16

17 65 66 Rude or Attacking Participant verbally attacks facilitator or another member of the committee Action: Maintain selfcontrol. Identify inappropriate behavior: Let s all take a deep breathe, We want to remain civil in our discussion. Mary, Let s discuss this concern at another time. The Chronic Devil s Advocate Assign unlikely people to discuss both sides assign the devil s advocate to the opposite side Acknowledge the perspective, then ask if there are other reactions, What solution would you have to resolve this issue? The Broken Record Different Philosophies or Beliefs Person won t give up on an idea. Acknowledge the importance of this to him. State clearly that others do not support this idea Ask for intent, versus positions Clarify needs and beliefs-understanding drivers helps move to win-win Identify points of agreement, then determine how differences will be resolved and who gets to decide 17

18 69 70 What if there is no agreement? Not all decisions are majority vote. Be honest about who makes the final decision, before the work begins Set time limits, in some cases Teaching-Learning Process to integrate NCLEX Categories NCLEX Test Plan 71 How can activity statements or content examples help us plan teaching-learning activities? Management of Care: 72 Class Simulation/Lab Clinical Activity Statement Maintain client confidentiality and privacy Related Content Participate in cost effective care* 18

19 73 74 Safety and Infection Control Health Promotion and Maintenance Activity Statement Use ergonomic principles when providing care (e.g., assistive devices, proper lifting) Assess peer s actions which moving, listing patients Content Reporting of Incident. Event, Irregular, Occurrence, Variance Identify need/situation where reporting of incident/ event/ irregular occurrence/variance is appropriate Activity Statement Provide information for prevention and treatment of high risk health behaviors (e.g., smoking cessation, safe sexual practices, drug education) Content Identify risk factors for disease/illness (e.g., age, gender, ethnicity, lifestyle) CLASS: Use of NCLEX Test Plan Simulation Lab Specific Objectives based on NCLEX Test Plan congruent with class objectives Mapping for each syllabus Learning activities in class Mapping of Test Items Test Items written at the application level or higher Analysis of faculty-made and standardized test Remediation of individuals Remediation of Class Tracking and Revision of Course 19

20 77 78 Question: In your typical Clinical experience--- Which categories do you typically emphasize? Which categories are not likely to be emphasized? Do you correlate assignments related to weaknesses in testing? Intentional Framework for Planning Clinical Experiences Most clinical experiences (and pre- and post-clinical preparation) should feature learning objectives related to: 1. Patient- Centered Care Nursing Process Physiological (Basic Care/ Comfort; Pharmacological/Parenteral Therapies; Reduction of Risk) Communication 2. Learning Objectives related to: Caring Health Promotion & Maintenance Psychological Integrity Management of Care Physiological Adaptation Safety/Infection Control Cultural & Spiritual 3. Post-Clinical Conference and/or Debriefing Clinical Objectives : Nursing Process and Caring is integrated throughout 1. Implement an appropriate focused assessment, including VS. (Physiological Integrity: Reduction of Risk) 2. Implement, evaluate & document morning care required for selected patients (Physiological Integrity: Basic Care/Comfort, Communication) 3. Educate client about medications (Physiological Integrity: Pharm, Teaching/Learning 4. Assess skin and implement measures to maintain skin integrity (Physiological Integrity: Basic Care/Comfort) 5. Provide nonpharmacological comfort measures and evaluate patient response. (Physiological Integrity: Basic Care/Comfort) 6. Use therapeutic communication techniques to provide client support (Psychological Integrity; Communication) Other Clinical Objectives Management of Care Utilize information resources to enhance the care provided to a patient with a Fx hip (e.g., evidenced- based research, information technology, policies and procedures) Health Promotion and Maintenance Determine potential area of health risk. Assess client s readiness to learn, learning preferences and barriers to learning (Also risk reduction, teaching learning) Physiological Integrity: Physiological Adaptation Relate the physiological effects of a fx hip to the best practices of care. Safety /Infection Control Scavenger Hunt: Strategies to insure safety 20

21 82 References Group Activity: Choose a Clinical objective and plan a clinical experience Barton, et al. (2014). Administrative Issues Journal, 4(2), Mee, C. L., & Hallenbeck, V.J. (2015). Selecting Standardized Tests in Nursing Education. Journal of Professional Nursing. 31(6), Mee, C.L. and Schreiner, B. (2016). Remediation in Nursing Education Today: Review of the Literature and Considerations for Future Research. Journal of Nursing Regulation. 7(1), Morrison, S., Free, K., & Newman, M. (2002). Do progression and remediation policies improve NCLEX-RN pass rates? Nurse Educator, 27(2), Culleiton, A. L. (2009). Remediation: A closer look in an educational context. Teaching and Learning in Nursing, 4 (1),