Program Assessment Plan AAS in Paramedicine and Paramedic (EMT P) Certificate Part I Assessment Summary 2005 2006 A. Program Mission Statement Mission It is the mission of the Emergency Medical Services (EMS) Department to facilitate student learning and meet the needs of the EMS community by providing a relevant and current curriculum that is based on sound educational principles. The EMS Department is committed to using appropriate instructional strategies, making effective use of instructional resources and continuously assessing student academic achievement for the purpose of ongoing improvement. Purpose The Paramedic Certificate Program provides educational and clinical opportunities that foster sensitivity, compassion, and a high level of integrity and responsibility for the available student population. Development of communication skills; personal safety, safety of the crew, patient and bystanders; patient assessment; patient care; lifting and moving patients; transporting patients; transfer of care; patient advocacy and the attainment of professional standards form the foundation for the practice of an EMT. Graduates of this program will enter the workforce with entry-level skills that can be applied in a variety of healthcare settings. Continual assessment of student learning for the purpose of improvement has a high priority in this program. B. Intended Learning Outcomes for Students in Paramedic Certificate Program 1. Students will be able to orally discuss two patient scenarios. 2. Students will be able to demonstrate patient assessment/management of the trauma patient. 3. Students will be able to demonstrate cardiac arrest management. 4. Students will be able to demonstrate appropriate pediatric care for the seriously ill patient. 5. Students will be able to demonstrate adult airway management. 6. Students will be able to demonstrate the skills involved with intravenous therapy. 7. Students will be able to demonstrate random basic skills verification. C. Benchmarks The intended outcomes will be assessed by the final practical examination (evaluation). Outside practitioners directly evaluate students per attached final examination evaluation rubric. The final practical evaluation occurs when the students completes all paramedic classes EMS 200 classes. This a capstone experience. The desired outcome is that 100% of the students successfully pass the paramedic final practical examination with a 70% or better. 1
D. Assessment Results 1. Historical Context Currently, the EMS faculty regularly discusses student progress in the lab portions of EMS 200 classes. As the class continues through the semester, faculty assess how students are performing, review how skills are being taught and discuss ways that changes can be made in order for students to achieve a high level of competency for the particular skills being taught. (There is periodic skill testing during the semester.) 2. Current Year Data Results 2005-2006: 27 were the total number of students who took the Paramedic final practical examination. What the chart below indicates is that out of 27 students taking the practical 27 students passed all testing stations. National Registry Practical Exam Pass Rate 30 # of Students Passing 20 10 0 Oral Scenarios Patient Assess/Mng Trauma Dynamic/Static Cardiac Arrest Pediatric intraosseous infusion/intubation Stations Adult Airway Management IV/Medications Random Skill Verification What the chart above does not show is how many students needed to retest a station to pass on a second attempt. Naturally, those numbers would be good to identify. Currently, this information is not available and in future assessments we will collect that information. 3. Analysis: After reviewing the final practical testing data as it is currently seen there was a 100% pass rate on the practical examination. We know that some students needed to retest a station to pass. At this time, we are unable to retrieve this information from the National Registry offices where the data is collected. Again, we will attempt to get that information in the future. The faculty feels that the outcomes were what was expected and are pleased with the results. The actual paramedic final practical consists of 11 testing stations. For the sake of creating a more usable, easily read document we did combined appropriate stations. These stations evaluate the following: 1. Students will be able to orally discuss patient assessment/management of the medical patient and patient assessment/management of the trauma patient. 2. Students will be able to demonstrate on a manikin or live simulate patient, patient assessment/management of the trauma patient. 3. Students will be able to demonstrate cardiac arrest management in a dynamic setting (the ekg rhythm strip chances as the patient is treated) as well as identifying static(still paper rhythm strips) ekg strips. 4. Students will be able to demonstrate appropriate care and skills of the pediatric patient and demonstrating the skills of intraosseous infusion and pediatric intubation. 2
5. Students will be able to demonstrate adult airway management by properly performing adult intubation and the use of the adult combi-tube. 6. Students will be able to demonstrate the skills involved with intravenous therapy by correctly starting an IV and pushing an IV bolus of medication. 7. Students will be able to demonstrate a basic skill by randomly picking one of the following skills: - bleeding control and shock management. - immobilization the seated spinal injured patient. - immobilization the supine spinal injured patient. E. Use of Results: Get feedback to lab instructors and lecturers to review strengths and weaknesses of the program. Adjust lecture and lab classes to improve outcomes of both the practical and National Registry examinations. Tabulate results (missed skills and pass/fail of all candidates). Tabulate numbers of students that passed the entire test on the first go, on the second go (this information will be examined in future assessments). Part II Assessment Plan for 2006-2007 A. Intended Outcomes 1. Students will be able to orally discuss two patient scenarios. 2. Students will be able to demonstrate patient assessment/management of the trauma patient. 3. Students will be able to demonstrate cardiac arrest management. 4. Students will be able to demonstrate appropriate pediatric care for the seriously ill patient. 5. Students will be able to demonstrate adult airway management. 6. Students will be able to demonstrate the skills involved with intravenous therapy. 7. Students will be able to demonstrate random basic skills verification. The above will be assessed by a practical examination and written examination. The National Registry Written Examination is taken after the student completes the class and successfully passes the National Registry Paramedic Final Practical Examination. In addition to the above measures we will also send out surveys to both the graduate students (Paramedic Graduate Survey) and to their employers (Paramedic Graduate Employer Survey). B. Identify Assessment Procedures/Methods The above intended outcomes will be assessed by both the National Registry Paramedic Final Practical Examination and the National Registry Paramedic written examination. For the practical examination evaluators from outside the program will directly evaluate students on a set of prescribed criteria for each station.(final examination evaluation rubric). The final practical evaluation occurs when students complete all the paramedic classes. This a capstone experience. 3
National Registry Paramedic Practical Examination Assess the students by using the National Registry Practical Examination using the specific evaluation tools for each of the intended outcomes. The examination sheets have multiple components plus a list of critical criteria that the student must meet. All Paramedic Programs in the State of Colorado use the same practical testing sheets developed by the National Registry of EMTs and Paramedics. This program will: Set up evaluation stations. Outside practitioners are hired and used to directly evaluate students per the final examination evaluation rubric. Hire patients and assistants. Get final practical paperwork to evaluation staff Tabulate results (missed skills and pass/fail of all candidates). National Registry Paramedic Written Examination As of January 2007 al Paramedic class completers will take the National Registry Paramedic Written Examination online. The testing criteria will be the same as the paper/pencil examination. The student must pass each section of the examination with a 70% or higher. The expectation is that all students will pass the test on the first attempt. Paramedic Graduate Survey (see attachment) The graduate survey gives us an opportunity to get feedback from the students about six months after program completion (particularly in cognitive domain). Student feedback is quite important for the continued success of the Paramedic Program. The survey is in the form 22 questions which the first 17 require an answer of one of the following: Strongly agree Agree Neutral Disagree Strongly disagree The last few questions are long answer that asks specific questions regarding the Program particularly how the students needs were met and suggestions for improvement. Paramedic Graduate Employer Survey The employer survey gives us an opportunity to get feedback from the students employers regarding their performance in field about six months after program completion (particularly in cognitive domain). Employer feedback is critical for the continued success of the Paramedic Program. The survey is in the form 21 questions which the first 17 require an answer of one of the following: Strongly agree Agree Neutral 4
Disagree Strongly disagree The last few questions are long answer that asks specific questions regarding the Program particularly how the employers see how the students perform in the field after completing the program and suggestions for improvement. C. Benchmarks The intended outcomes will be assessed by the National Registry Paramedic Final Practical Examination, the scores from the National Registry Paramedic Written Examination, the Paramedic Graduate Survey and the Paramedic Graduate Employer Survey. For the practical, outside practitioners directly evaluate students using the final examination evaluation rubric. The final practical evaluation occurs when the students complete all paramedic classes (EMS 200 series). This a capstone experience. The desired outcome is that 100% of the students successfully pass the final practical examination and 100% pass the National Registry Examination. The National Registry Written Examination is taken after the student completes the class and successfully passes the National Registry Paramedic Final Practical Examination. The desired outcome for this examination is that 100% of the students will successfully pass the written test on the first attempt with a score of 70% or better. (The National Registry scores are only available to the EMS Department if students give their written permission to the National Registry to allow their scores to be released.) For the written, the students will take an on-line written examination with the expected outcome that all students (100%) will pass all sections of the examination with a 70% or higher. For both the surveys we have an expectation that both the students and employers will be stratified with both the education received and the performance of the paramedic. The results are tabulated using the Likert scale of 1-5 with three and higher being the desired score of 100% of both students and employers. The validity and reliability of the survey has been established by the Committee on Accreditation of Emergency Medical Services Professions (CoAEMSP). 5