IUE School of Nursing and Health Sciences, Campus assessment and evaluation report summary 2015-2016 Systematic assessment and evaluation is an integral part of the School of Nursing and Health Sciences. To that end, the School has an undergraduate curriculum committee and an undergraduate evaluation committee and has delegated the assessment/program evaluation functions for the graduate program to the graduate program committee. Masters of Science in Nursing (MSN) Program I. Review Mission and Vision The IU East School of Nursing Mission and Vision was reviewed and approved by Graduate Program faculty. II. III. IV. Review Policies/MSN handbook APG policies were reviewed, revised, and approved. MSN handbook was reviewed, revised and approved for 2016. The graduate council voted to change the format of the MSN handbook, student policies are now located within the student handbook. Clinical policies chapter/section was also included for the FNP program. Course mapping The cumbersome nature of the course map template in addition to adjunct faculty contributions led the graduate faculty to reconsider the utility of the course map template and discussion led to development of a more user friendly course mapping process. The graduate council voted to include the course map grid and add the evaluation mapping tool to each syllabus. This new process then allows for automatic review/revision of each course when the syllabus is reviewed/revised each time the course is taught. New course syllabus template takes effect summer 2016. Curriculum revision Graduate faculty approved addition of the 10 th MSN Program Learning Outcome: 10. Engage in life-long learning activities that contribute to professional development as well as to the advancement of nursing. V. MSN program assessment and evaluation: Based on feedback from the ACEN MSN-FNP accreditation visit faculty are reviewing and revising the systematic evaluation plan for fall 2016. The 2013 ACEN standards and the 10 MSN program learning outcomes have been added, faculty will continue to refine and work on the benchmarks and measurements for each standard. It is the hope of the graduate council that work/refinement will be complete by December, 2016. A. 2015/2016 MSN goals met: a. ACEN Change report visit: Successful ACEN accreditation visit, spring 2016 for FNP program- Met b. Implement HESI exams for pathophysiology, pharmacology and physical assessment for FNP program assessment plan- Met (analysis is underway).
c. Implement Typhon data collection service for student immunization, CPR, and clinical experience tracking- Met, all FNP students from the two cohorts have successfully obtained and are utilizing Typhon for patient case documentation. d. Continue regional and IUPUI campus collaborations, especially for the FNP program- Ongoing, currently IUSB provides a regional FNP director for IU East as well as IU Kokomo. We have continued to collaborate with the ADM and EDUC core courses, NURS Y510 and NURS Y520 with IU Kokomo. e. Continue to refine the portfolio process and integrate into Typhon tracking software- Due to interrater reliability issues, the Graduate Council to suspend the MSN portfolio process at this time. f. Continue to recruit and retain faculty for the FNP and administration concentrations- Amanda Carmack was hired full time and teaches in the ADM concentration, Cindy Evans was hired full time and teaches the three P courses (Pathophysiology, Pharmacology and Physical Assessment) and FNP clinical, we are currently finishing interviews for the full time FNP position. B. Graduation and Retention Five students graduated spring of 2016 in the administration concentration. Currently there are 38 students in the MSN program: 24 FNP; 8 Education; and, 6 Administration with cohorts graduating in fall 2016, spring 2017, spring 2018 and fall 2018. Three additional administration students have been admitted to begin fall of 2016 (graduate fall 2018). One education student withdrew prior to classes beginning spring 2016. One education student scheduled to graduate fall 2016 has decreased course load due to illness, scheduled to graduate fall 2017. C. FNP admits second cohort. FNP program admitted a second cohort spring of 2016. First director Amber Essman, MSN, CNP, FNP-BC, CNE resigned summer of 2015. Dr. Mario Ortiz from IUSB became the regional director for FNP programs for IU East and IU Kokomo, Dr. Ortiz resigned and Caitlyn Vlaeminck has taken over as regional FNP director. Jennifer Bow, was hired part time to serve as clinical site coordinator, this position has proven to be very valuable regarding placement of FNP students for their clinical experiences. D. ACEN Accreditation ACEN one-day focused visit for the addition of the FNP program was held in March, 2016. Visit was successful and ACEN has recommended that the IU East MSN program current accreditation be extended to the FNP program as well. The ACEN board of directors will vote in July of 2016. The next ACEN visit for the MSN program will be fall of 2018. E. Exit surveys Administration exit surveys. Five MSN Administration students graduated, only one has responded to the Skyfactor exit survey to date. Additional request to participate have been issued. F. Employer surveys Skyfactor participation was low, employer surveys will be sent out again.
G. Alumni surveys Skyfactor alumni survey response rate was 90! Results include: Alumni parameter topic Results/Benchmark Response rate 90 Pursue education since graduating 36 enrolled in doctoral programs MSN Essentials- I: Background for practice from 11 parameters met benchmark of >50 science and humanties. Two parameters were below 50 Need to evaluate Biopychosocial and Genetics. Q051. Importance of Skill/Ability and Knowledge - How important is this skill/ability and this knowledge to a successful nursing career? Integrate scientific findings from the following in order to support continual improvement of nursing care across diverse settings: Biopsychosocial fields Key Text N (N) (1) Not at all 0 (0) (2) 0 (0) (3) 0 (0) (4) Moderately 5 (50) (5) 1 (10) (6) 2 (20) (7) Extremely 2 (20) Resp = 90.9 N = 10 Mean = 5.10 Std Dev = 1.22 Q052. Importance of Skill/Ability and Knowledge - How important is this skill/ability and this knowledge to a successful nursing career? Integrate scientific findings from the following in order to support continual improvement of nursing care across diverse settings: Genetics Key Text N (N) (1) Not at all 0 (0) (2) 1 (10) (3) 1 (10) (4) Moderately 3 (30) (5) 2 (20) (6) 2 (20) (7) Extremely 1 (10) Resp = 90.9 N = 10 Mean = 4.60 Std Dev = 1.43 MSN Essentials- II: Organizational and systems leadership MSN Essentials- III: Quality improvement and safety MSN Essentials- IV: Translating and integrating scholarship into practice MSN Essentials- V: Informatics and healthcare technologies MSN Essentials- VI: Health Policy and Advocacy MSN Essentials- VII: Interprofessional Collaboration for Improving Patient and Population Health Outcomes MSN Essentials- VIII: Clinical prevention and population health for improving health MSN Essentials- IX: Master s level nursing practice 12 parameters met benchmark of >50, ranging from 70-90.9. 10 parameters met benchmark of > 50, ranging from 66.7-90. 8 parameters met benchmark of >50, ranging from 70-90. 8 parameters met benchmark of > 50, ranging from 70-100. 4 parameters met benchmark of > 50, ranging from 80-88.9. 6 parameters met benchmark of >50, ranging from 70-100. 7 parameters met benchmark of >50, ranging from 70-100. 8 parameters met benchmark of >50, ranging from 60-100.
ACEN Standard- III: Students 10/15 parameters met benchmark of >50, ranging from 55.6-81.8. 5 parameters below 50 To be analyzed, evaluated and action plan developed by graduate council. Q028. Nursing Program Promoted Career - To what extent did the MSN program: Increase your earning potential Key Text N (N) (1) Not at all 0 (0) (2) 1 (9.1) (3) 1 (9.1) (4) Moderately 3 (27.3) (5) 1 (9.1) (6) 2 (18.2) (7) Extremely 3 (27.3) Resp = 100.0 N = 11 Mean = 5.00 Std Dev = 1.65 Q033. Co-Curricular Activities - To what extent have the following school activities during your master's level education contributed to your success as a nurse: Internship Key Text N (N) (1) Not at all 1 (12.5) (2) 1 (12.5) (3) 1 (12.5) (4) Moderately 1 (12.5) (5) 1 (12.5) (6) 1 (12.5) (7) Extremely 2 (25) Resp = 72.7 N = 8 Mean = 4.38 Std Dev = 2.12 Q036. Co-Curricular Activities - To what extent have the following school activities during your master's level education contributed to your success as a nurse: Interaction with alumni Key Text N (N) (1) Not at all 1 (10) (2) 1 (10) (3) 1 (10) (4) Moderately 3 (30) (5) 0 (0) (6) 2 (20) (7) Extremely 2 (20) Resp = 90.9 N = 10 Mean = 4.40 Std Dev = 1.96 Q037. Co-Curricular Activities - To what extent have the following school activities during your master's level education contributed to your success as a nurse: Student organization participation Key Text N (N) (1) Not at all 1 (12.5) (2) 1 (12.5) (3) 1 (12.5) (4) Moderately 3 (37.5) (5) 0 (0) (6) 1 (12.5) (7) Extremely 1 (12.5) Resp = 72.7 N = 8 Mean = 3.88 Std Dev = 1.83 Q038. Co-Curricular Activities - To what extent have the following school activities during your master's level education contributed to your success as a nurse: Participation in volunteer activities Key Text N (N) (1) Not at all 0 (0) (2) 0 (0) (3) 0 (0) (4) Moderately 6 (66.7) (5) 0 (0) (6) 3 (33.3) (7) Extremely 0 (0) Resp = 81.8 N = 9 Mean = 4.67 Std Dev = 0.94 ACEN Standard VI: Outcomes 2 parameters met benchmark of >50, ranging from 54.5-81.8. H. Scholarly Project Evaluations April 2016- Five, R590 Scholarly projects were successfully completed. Results: Each student ranked their mentor and the scholarly project with agree and/or strongly agree for meeting the MSN program learning outcomes. Four of the five students filled out the course/mentor evaluation. I. Certified Nurse Educator Results: Two additional graduates have taken the CNE and passed successfully, 100 pass rate since program inception.
Graduate council adopted NLN statistics for benchmarking for Nursing Education majors taking the CNE self-assessment: VI. Graduate faculty development: On May 10, 2016 the school of nursing and health sciences brought in International Expert, Dr. Tim Bristol (NurseTim) for graduate teaching development. Six full time and two adjunct graduate fauclty members attended. Topics included: 1) Overview of effective learning in graduate education. 2) Overview of the future of graduate nursing education. 3) Higher order thinking and lesson plans. 4) QSEN for patient care and QSEN for student care. 5) Active learning on the graduate level. 6) Personal, peer and professional assessment. Evaluations were positive and ranked the day high. VII. 2016/2017 Goals: 1) Continue regional and IUPUI campus collaborations, especially for the FNP program. 2) Continue to recruit and retain faculty for the FNP and administration concentrations. 3) Perform needs assessment for a Clinical Nurse Leader MSN concentration and begin approval process. 4) Continue to recruit for administration and education concentrations. Bachelor of Science in Nursing (BSN) Program I. Mission and Vision The School of Nursing and Health Sciences updated and approved the mission and vision statements. II. Review if Polices and Handbooks Policies continued to be renewed on a rotating basis by undergraduate committees appropriate to the policy (students, curriculum, testing, admission, progression, graduation, and evaluation). The RN to BSN Handbook was updated and is available online for RN to BSN students via the RN to BSN information site. The BSN Handbook is in the process of being revised and updated and should be available to students by Fall 2016. III. Course Mapping Faculty under the direction of the undergraduate curriculum committee completed the development of course outcomes for all didactic and clinical courses; mapping assessment to outcomes and to the BSN Essentials of Baccalaureate Education. RN to BSN mobility courses are in the process of being mapped with a fall 2016 completion date. IV. Curriculum Revision
The faculty developed and approved a revised BSN curriculum. This curriculum is a concept-based curriculum as opposed to a content-based curriculum. The curriculum was approved by IU East Faculty Senate and the Indiana State Board of Nursing. ACEN will be notified of this change as part of the ACEN self-study for reaccreditation. Implementation is scheduled for Fall 2017. The current BSN curriculum will be phased out with the last class graduating spring 2019. V. Assessment and Evaluation 2015-2016 goals Continue to investigate factors related to poor NCLEX-RN performance. Implement and evaluate policy changes. Goal met The faculty continued to review overall data, including ATI predictor scores, ATI proctored exams scores, course grades, and remediation efforts. Faculty chose to add ATI Pulse to the cadre of student data available. ATI Pulse provides NCLEX-RN pass rate probabilities at the end of each semester to assist us in identifying at-risk students earlier. ExamSoft testing software was utilized to better mirror NCLEX-RN testing as well as to promote mapping of questions to the NCLEX-RN test plan. The testing policy was revised to integrate the use of ExamSoft. Beginning fall 2015, all out of sequence student completed a success plan with the Associate Dean of Undergraduate Nursing programs to better prepare them for re-entry into nursing coursework and clinical experiences. Use of the ATI capstone course for NCLEX-RN preparation was piloted in the spring of 2016. Its use seemed to have little impact on student NCLEX-RN performance and its use will be re-evaluated in the fall of 2016 when all students have completed NCLEX-RN testing. Active learning strategies continued to be used in the classroom. Review clinical experiences for medical-surgical nursing clinical, revise as necessary. Goal met Faculty developed clinical course outcomes for all BSN clinical and lab courses during the 2015-2016 academic year. Each course has moved to using simulation in addition to actual clinical experiences. Clinical content was realigned, especially in the NURS S470 course where fewer critical care experiences were used and there was an increase in more geriatric and rehabilitation experiences. Each med-surg clinical has also implemented 2-3 clinical simulations per course. Gain approval for the implementation of a Concept-Based Curriculum for the BSN program. Goal Met. A concept-based curriculum was developed and approved by the undergraduate nursing faculty. The curriculum was approved by the IU East faculty senate and the Indiana State Board of Nursing. It will be presented as part of the ACEN self-study for ACEN notification. Six additional faculty attended conferences on teaching in a concept-based curriculum during the 2015-2016 year. Target implementation is fall 2017. Implement revised evaluation plan. Goal Met The School of Nursing the revised and approved the BSN evaluation plan May 2015. We worked with SkyFactor to collect data via an alumni survey and an employer survey in addition to the student exit survey. The alumni survey results were discussed with faculty. There were insufficient numbers of employers identified from the alumni surveys to aggregate data on the employer survey.
Assessment Data Faculty reviewed the NCLEX-RN scores and ATI Comprehensive predictor scores for NCLEX-RN success for the BSN class graduating May 2015 and ATI Pulse and Comprehensive Predictor scores for the BSN class of May 2016. NCLEX-RN pass rate for the 2015 reporting year was 84.3, up 10 from the previous year. The ATI testing and remediation plan was reviewed and enhanced use. Each course in the program is now required to integrate the use of ATI proctored content exams. The NURS S485 course requires the use of multiple ATI practice exams to better prepare students for the NCLEX- RN. A Hurst NCLEX-RN review was held in January of 2015, May 2015, and January 2016 for senior students. This provided an opportunity to begin NCLEX-RN preparation early. Seniors in the BSN program took the ATI predictor exam in spring 2016. 26 of 68 students scored less than an 85 probability of passing NCLEX-RN. The Associate Dean of Undergraduate Programs developed individual success plans for these students. After one month, students retested on a parallel version of the ATI predictor. All but 8 students hit the 85 probability benchmark on the second attempt. Students who did not hit the benchmark had revised success plans developed for them. After another month, these students tested with the ATI 2016 comprehensive predictor. All but two hit the 85 benchmark. These students were given a remediation plan prior to a June graduation to enhance NCLEX-RN success. Faculty reviewed EBI Benchmarking exit results for those graduating May 2015 and trended in comparison to previous graduates. Skyfactor (formerly EBI) is a national company whose focus is to complete exit, alumni, and employer assessment surveys and benchmark results nationally and against chosen peer institutions. Faculty reviewed the survey results in regularly scheduled faculty meetings. The score of 5 is set as a benchmark indicating a good level. There are 14 factors evaluated. Exit Survey Factor Longitudinal Data for 2011-2015 1. Satisfaction: Quality of Nursing Instruction N Std Dev 2015 59 4.87 0.00 1.16 2014 69 4.27 0.60 1.13 2013 5.11-0.24 2012 5.11-0.24 2011 5.31-0.44
Factor 2. Satisfaction: Work and Class Size Factor 3. Satisfaction: Course Lecture and Interaction Factor 4. Satisfaction: Facilities and Administration N Std Dev 2015 59 5.08 0.00 1.09 2014 68 4.45 0.63 1.24 2013 5.49-0.41 2012 5.36-0.28 2011 5.33-0.25 N Std Dev 2015 59 5.56 0.00 1.02 2014 69 5.02 0.54 1.15 2013 5.78-0.22 2012 5.90-0.34 2011 5.92-0.36 N Std Dev 2015 59 4.44 0.00 1.20 2014 68 4.31 0.13 1.31 2013 5.02-0.58 2012 5.09-0.65 2011 5.14-0.70 Factor 5. Satisfaction: Classmates Factor 6. Satisfaction: Advisor N Std Dev 2015 59 5.20 0.00 1.21 2014 68 4.81 0.39 1.32 2013 5.32-0.12 2012 5.40-0.20 2011 5.42-0.22 N Std Dev 2015 59 5.64 0.00 1.41 2014 68 5.12 0.52 1.53 Factor 7. Learning: Professional Values
N Std Dev 2015 57 5.88 0.00 0.97 2014 68 5.65 0.23 1.16 2013 6.02-0.14 2012 5.87 0.01 2011 5.71 0.17 Factor 8. Learning: Core Competencies Factor 9. Learning: Technical Skills Factor 10. Learning: Core Knowledge N Std Dev 2015 58 5.47 0.00 1.11 2014 68 5.44 0.03 1.06 2013 5.82-0.35 2012 5.82-0.35 2011 5.74-0.27 N Std Dev 2015 56 5.45 0.00 1.19 2014 68 5.59-0.14 1.22 2013 5.83-0.38 2012 5.71-0.26 2011 5.63-0.18 N Std Dev 2015 58 5.45 0.00 1.00 2014 68 5.35 0.10 1.11 2013 5.80-0.35 2012 5.66-0.21 2011 5.56-0.11 Factor 11. Learning: Role Development Factor 12. Overall Satisfaction N Std Dev 2015 58 5.00 0.00 1.06 2014 68 4.88 0.12 1.15 2013 5.34-0.34 2012 5.30-0.30 2011 5.18-0.18
N Std Dev 2015 57 4.87 0.00 1.64 2014 68 3.85 1.02 1.79 2013 5.61-0.74 2012 5.11-0.24 2011 5.50-0.63 Factor 13. Overall Learning N Std Dev 2015 57 5.77 0.00 1.13 2014 68 5.44 0.33 1.14 2013 6.14-0.37 Factor 14. Overall Program Effectiveness N Std Dev 2015 57 5.37 0.00 1.20 2014 68 4.74 0.63 1.22 2013 5.86-0.49 2012 5.08 0.29 2011 5.23 0.14
Recommendations for the IUE Nursing Program as established by EBI. Regression Variables Impact on Overall Program Effectiveness Performance of Factors Factor R 2 ΔR 2 Value Description Mean Description Top Priority Factor 10. Learning: Enhanced Professional Skills 0.624 0.624 6.53 Extreme Impact 5.25 Good Monitor Factor 1. Satisfaction: Nursing Program Promoted Successful Career 0.000 0.000 0.00 No Impact 5.06 Good Factor 2. Satisfaction: School Activities Contributed to Success 0.000 0.000 0.00 No Impact 3.24 Poor Factor 4. Learning: Enhanced Patient Relationships and Care 0.000 0.000 0.00 No Impact 5.17 Good Factor 6. Learning: Enhanced Problem Solving 0.000 0.000 0.00 No Impact 5.14 Good Factor 8. Learning: Enhanced Community Health Care 0.000 0.000 0.00 No Impact 5.15 Good Factor 12. Learning: Enhanced Management Skills 0.000 0.000 0.00 No Impact 5.05 Good Notes The dependent variable in this regression is the factor Overall Program Effectiveness. The factors above are categorized by predictor status and then sorted by Impact Value. NOTE: EBI uses a significance value of p <.05 to calculate predictors. Based on the number of responses and/or the behavior of those responses, the significance value has been increased to p <.09. The Top Priority quadrant reflects those factors that are lower performing but have significant impact on Overall Program Effectiveness. The Maintain or Improve quadrant reflects factors that are high performing and have significant impact on Overall Program Effectiveness. The Maintain quadrant reflects factors that are high performing but have little if any impact on Overall Program Effectiveness. The Monitor quadrant reflects factors that are lower performing but have little if any impact on Overall Program Effectiveness. Impact on Overall Program Effectiveness: Impact is a scaled value of the variance from a range of 0 (no impact on Overall Program Effectiveness) to 7 (an extreme impact on Overall Program Effectiveness). Impact Description: Extreme: Impact > 6; High: Impact > 5; Moderate: Impact > 4; Slight: Impact > 3; Negligible: Impact > 2; No Impact: Impact = 0 Performance Description: Superior: Mean >= 6.7; Excellent: Mean >= 5.62; Good: Mean >= 4.54; Fair: Mean >= 3.46; Poor: Mean >= 2.38; Very Poor: Mean >= 1.3; Extremely Poor: Mean < 1.3
EBI results were discussed in the Undergraduate Evaluation committee and with all nursing faculty at a nursing meeting. The Undergraduate Evaluation committee made the following recommendations to the faculty based on the data. Faculty discussion summary is highlighted under each recommendation a. Streamlining the use of ATI Discussed how the use of ATI could be better integrated throughout the program with use of non-proctored exams, ATI pulse which identifies students at risk earlier, and how to better prepare students for the comprehensive predictor. b. Consider the use of paid tutors for all levels of students. Use of paid tutors may be budget prohibitive. Consider student volunteers to assist. Look at the potential of assigning faculty to tutoring responsibilities. c. Consider the addition and use of online/hybrid courses This is not possible as courses in this program were not approved for online or hybrid offerings. d. Continue to work with career placement and planning Planning and placement works with senior faculty to do resume and interviewing education. Continue with health facility job fairs. e. Explain the significance of non-nursing courses Discussed the need to have students understand about the need for a broad based education representative of a college educated individual. Work with pre-nursing advisors to begin this discussion early. f. Enhance wound care management skills This is pulling out just one skill area based on this assessment. Look into opportunities to provide students with more clinical experience associated with wound care. g. Consider having a NCLEX-RN review class We already have a HURST review twice during the final semester, once in January and once in May. Rather than do our own, if we believe HURST is not sufficient then look at other review courses. h. Testing outside of normal class time Testing is part of the educational experience and should not be in addition to class time. i. Eliminating technology/fees for student to streamline costs University fees cannot be eliminated. We should look toward ways to decrease other costs such as the resources and textbooks required. j. Move NURS S485 back to Monday so can facilitate capstone and management clinical. Not sure this recommendation is based on data. This will probably not occur. k. Address administration and faculty professionalism Dean discussed actions of incivility and unprofessionalism that will no longer be tolerated. Discussed how to address professionalism and civility in the classroom with students as well as how to role model it.
Exit Survey Factor Analysis 1-15: A score of 75 is deemed good, a score of 71-74 indicates needs work, and less than 70 indicates there are significant issues. Factor 1. Satisfaction: Nursing Program Promoted Successful Career Factor 2. Satisfaction: School Activities Contributed to Success Factor 3. Learning: Importance of Patient Relationships and Care Factor 4. Learning: Enhanced Patient Relationships and Care
Factor 5. Learning: Importance of Problem Solving Factor 6. Learning: Enhanced Problem Solving Factor 7. Learning: Importance of Community Health Care Factor 8. Learning: Enhanced Community Health Care
Factor 9. Learning: Importance of Professional Skills Factor 10. Learning: Enhanced Professional Skills Factor 11. Learning: Importance of Management Skills Factor 12. Learning: Enhanced Management Skills
Factor 13. Overall Satisfaction Factor 14. Overall Learning Faculty reviewed these results in May 2016 and will continue to brainstorm on ways to engage students in school activities as well as how to work on problem solving and enhancing management skills. The overall satisfaction score was concerning and faculty will continue to strategize on how this can be impacted. Goals for 2016-2017. 1. Continue investigating factors that contribute to NCLEX-RN performance. 2. Add faculty member tutoring for sophomore level students to enhance foundational knowledge acquisition. 3. Move forward with course development and approvals for the new BSN curriculum. 4. Model professionalism and civility and set the expectation for all members of the School of Nursing and Health Sciences. 5. Prepare for ACEN re-accreditation site visit, including completion of self-study.