Chapman University Annual Learning Outcomes Assessment Report

Chapman University Annual Learning Outcomes Assessment Report Degree Program Information 2017-2018 Academic Degree Program: Doctor of Pharmacy (PharmD) Brief Description of the Program: The Pharm D program at CUSP is an accelerated 3-year-round program where student pharmacists are required to complete six (T1 T6) 15-week didactic trimesters followed by six 6-week Advanced Pharmacy Practice Experiences (APPE) [ 4 required + 2 electives] at various pharmacy practice settings during T7 and T8 prior to graduation. Degree: Doctor of Pharmacy Number of students currently enrolled: Class of 2018 (graduated 5/18/18; N = 76) + Class of 2019 (N =78) + Class of 2020 (N = 90) = Total of 244 Contact Person Name: Siu Fun Wong, PharmD Title: Associate Dean of Assessment and Scholarship & Professor of Oncology Email Address: sfwong@chapman.edu Section 1 - Learning Outcome To assure compliance with the ACPE (Accreditation Council of Pharmacy Education) standards and guidelines, the CUSP program is designed as an assessment-driven program. The CUSP program assessment plan is comprehensive in its design and purposefully measures the key components of student learning toward the CUSP Student Learning Outcomes consisting of 5 domains that are designed to graduate practice-ready, innovative, and compassionate pharmacists. We hope to enable our student pharmacist graduates to enter the workforce and provide contemporary direct patient care within their scope of practice as an independent practitioner and as part of a healthcare team. These SLOs are mapped to the Chapman University ILOs, 2013 AACP (American Association of Colleges of Pharmacy) CAPE Outcomes, the North American Pharmacist Licensure Examination (NAPLEX) Competency Statements, and the ACPE Educational Outcomes. Out of the 5 domains, it is anticipated that Domain 5 will primarily be achieved during the experiential education curriculum in T7 and T8. Domain 2, a prime focus of the PharmD program, is completely designed to align with the Pharmacists Patient Care Process (PPCP) which is developed by the Joint Commission of Pharmacy Practitioners (www.jcpp.net) as the profession s systematic approach for pharmacist provision of care, regardless of the type of service or the pharmacy practice setting. 1

CUSP Student Learning Outcomes (Approved August 2014) Domain 1 Establish Personal and Professional Development 1.1 Maintain healthy lifestyles to promote long term health and wellbeing (Promoter) 1.2 Identify, create, implement, reflect and modify plans for personal and professional development for the purpose of individual growth (Self-awareness) 1.3 Demonstrate professional behaviors and values consistently and abide by the profession's code of ethics (Professionalism) 1.4 Demonstrate appropriate ethical and legal choices in the treatment of patients and their families in relation to the healthcare system, adhering to institutional and professional standards and regulations (Professional) 1.5 Demonstrate the ability to listen, communicate, empathize, educate and interact with any individual across the spectrum of age, gender, social and cultural background, educational level and health care circumstance without one's personal bias or preference. (Communicator) 1.6 Taking personal responsibility to advocate through contributions of time, medical expertise and other sources of support for practices and policies that improve the health of individual patients and populations (Advocate) 1.7 Demonstrate an understanding of the manner in which people of diverse cultures and belief systems perceive and respond to various health conditions and illnesses, and appropriately address gender and cultural biases in health care delivery. (Includer) 1.8 Develop visions, missions, and goals to empower team members by actively listening, gather input or feedback, and foster collaboration (Leader) 1.9 Develop new ideas and approaches to improve quality or overcome barriers to advance the profession or practice (Innovator) Domain 2 Deliver Patient-Centered Care 2.1 Gather, interpret, and evaluate patient information 2.1.1 Conduct patient and/or caregiver interview to gather appropriate information 2.1.2 Review and screen patient medical records 2.1.3 Select and/or use instruments and techniques to assess patients 2.1.4 Identify and interpret laboratory and diagnostic findings 2.1.5 Identify signs, symptoms and etiology of medical conditions 2.1.6 Identify and evaluate risk factors relevant to a patient's medical conditions 2

2.1.7 Determine need for referral 2.2 Identify and evaluate patient's current pharmacologic and non-pharmacologic therapy (Drugs, Dietary Supplement, Lifestyle, Self-care, and Complementary and Alternative medicine) 2.2.1 Review current therapy for appropriate use of current therapy (e.g. indication, therapeutic duplication or omission, usage errors) 2.2.2 Using fundamental knowledge listed below to perform pharmacologic assessments of the patient's therapy: 2.2.2.1 Anatomy and physiology 2.2.2.2 Pathology/Pathophysiology 2.2.2.3 Microbiology 2.2.2.4 Immunology 2.2.2.5 Biochemistry/Biotechnology 2.2.2.6 Molecular biology/genetics 2.2.2.7 Medicinal Chemistry 2.2.2.8 Pharmacology 2.2.2.9 Toxicology 2.2.2.10 Pharmaceutics/Biopharmaceutics 2.2.2.11 Pharmacokinetics/Clinical Pharmacokinetics 2.2.2.12 Pharmacogenomics/Genetics 2.2.2.13 Pharmacognosy and CAMs 2.3 Develop and Initiate Pharmacologic and Non-Pharmacologic Therapy (Drugs, Dietary Supplement, Lifestyle, Self-care, and Complementary and alternative medicine) 2.3.1 Design and implement a comprehensive patient-specific treatment plan using evidence-based medicine 2.3.2 Ensure patient-specific cultural, economic and quality of life factors while balancing individual, healthcare system and societal needs 2.3.3 Demonstrate safe and accurate preparation, compounding, dispensing and administration of patient's therapy 2.3.4 Prepare appropriate written, oral, and other communications to health care team, patients/caregivers, and organizations 2.3.5 Create written records of clinical encounters using standard terminology and formats 3

2.4 Manage Patient Therapy Domain 3 2.4.1 Plan and perform ongoing patient monitoring and evaluations to reach therapeutic goals and outcomes 2.4.2 Recognize therapeutic failure, patient tolerability & safety issues, nonadherence & misuse 2.4.3 Provide therapeutic alternative or modifications, if appropriate 2.4.4 Maintain appropriate written, oral, and other communications to health care team, patients/caregivers, and organizations 2.4.5 Maintain written records of clinical encounters using standard terminology and formats Deliver Population-Based Health 3.1 Recommend universal screening measures for common diseases 3.2 Provide health care information to promote health or prevent progression of medical conditions 3.3 Recognize risk factors for substance abuse and diversion, and provide resources to address and manage 3.4 Assess and manage emergency preparedness and travel medicine 3.5 Evaluate and apply health-related information, treatment guidelines, literature and resources 3.6 Select systems and technology for safe and efficient distribution of medications 3.7 Demonstrate safe and accurate techniques in the preparation, compounding, dispensing, administration and safe handling of medications, including biohazardous products 3.8 Plan, implement, and conduct health education strategies, interventions and program Domain 4 Engage in Inter-professional Education and Practice 4.1 Communicate effectively with patients, families, healthcare team members, and other colleagues 4.2 Function as an active and engaging member of an inter-professional health care team 4.3 Determine referral to other healthcare providers to optimize patient care Domain 5 Deliver Advanced Biomedical Pharmaceutical Research and Pharmacy Practice The student pharmacists will gain advanced knowledge and skills in a selected health sciencesrelated discipline using structured educational pathways provided through CUSP curriculum. e.g. Research (Basic Science, clinical, translational, or comparative effectiveness) 4

Specialty Pharmacy Practice (e.g. cardiology, pediatrics, emergency medicine) Pharmacy Administration (e.g. Health Policy Management, Medication Use System Management) Drug Development (Focus: governmental & industry) Section 2 - Process: The courses and their assessments in the CUSP curriculum are guided by the 2013 AACP CAPE Outcomes for PharmD degree training and ACPE Accreditation Standards 2016 and designed to be comprehensive, continuous, and cumulative. The emphasis of the pedagogy intends to promote independent and active learners. To assure the delivery of the CUSP SLO through our curriculum, all the course objectives developed by the course coordinator are included in each of the CUSP course syllabi from T1 T6. Each of the course objectives is mapped to the SLOs and ACPE 2016 Appendix 1 which provides the database for the CUSP course to SLO curriculum map (Appendix A) and to ACPE Appendix 1 curriculum map (Appendix B). Each year, faculty are provided opportunities to update the curriculum map based on the feedback they received from the faculty and course evaluations, assessment data, and professional updates. Lastly, content mapping (Primary criteria: Course #, Subject, Appendix 1; Secondary criteria: Contact hours, Intended Cognitive levels, Lesson Delivery and Assessment Methodology) serve as the micro-level mapping to the actual delivery of the curriculum content (Appendix C). CUSP uses all these mapping mechanisms as a guide for curriculum development in building the advancement of cognitive levels in the areas of foundation knowledge, skills, learning methodology (e.g., progression of active learning model) and competency assessments (e.g. percentage of higher cognitive level examination questions in P1 to P2, and complexity of clinical skills) of the student pharmacists as they progress in the curriculum to monitor the depth and breadth, and the readiness to deliver the curriculum to reach its goals. The knowledge-based assessments of the PharmD program is predominately conducted using Integrated Exam (IE) where a 2.5-3 hour knowledge-based examination is conducted every 3 weeks during each of the 6 trimesters, constituting 5 IEs per trimester. These examinations include items from approx. 80-90% of all didactic courses in the curriculum consisting of 3 items per contact hour (averaging 130 items per exam) and presented to each student in a random order. To provide faculty with personalized curriculum development feedback, the Office of Assessment and Scholarship (OAS), in collaboration with the faculty, created a detailed category coding system through ExamSoft (our electronic examination platform) to code each of the integrated examination items in multiple categories, which include CUSP SLO, Cognitive Level, Course Number, Disciplines (ACPE Appendix 1), Faculty Author, Question Type, Subject, Drug Classification, Population by Age, and Special Population. Faculty use these category domains to code each of their integrated examination items in Examsoft. This coding system allows us to analyze student performance across all these categories (cross-sectional and longitudinal) as well as determine where specific content is delivered and assessed in the curriculum. Performance (Skill)-based assessments of professional competencies are assessed through various measures including, but not limited to, the Observational Structured Clinical Examinations (OSCEs) administered at the end of every trimester from T1-T6, and various other performance-based in-class assessments and experiential education activities/assessments (eg, 5

Pharmacist Care Laboratory Series, direct observation of immunization administration, direct observation of blood pressure measurement, simulation activities, and monitoring and tracking of experiential education and other professional activities). OSCE is an acronym for Objective Structured Clinical Examination, an assessment method that is based on objective testing and direct observation of student pharmacist performance during planned clinical encounters (also called interactions or test stations). The OSCE assessments at CUSP are also designed with the same approach. A 2-station OSCE is conducted at the end of each of the 6 trimesters. Each OSCE assesses for specific skills and attitudes according to the CUSP OSCE Blueprint that is developed to align with the curriculum topics and cognitive level. OSCEs are graded as PASS or NO PASS based on the analytical checklist and global assessment rubrics. For grading of the analytical checklist, the sum of each observed item will serve as the score for the analytical checklist. A minimal performance level (MPL) is established for each OSCE case using the standard setting process conducted by CUSP faculty. The total possible score for the global assessment remains at a total of 20 from T1 thru T6. A student s analytical and global score may be normalized if an analysis of variance (ANOVA) shows a significant difference among the grader s average scores. The normalization process adjusts scores so that the average number of points lost by students in each grader s group is the same. A student with a score that is 1.5 of the standard deviation may not be eligible for normalization. A passing score is determined according to the Trimester-based weighted scores of the analytical checklist and the global assessment using the equation: Passing Score = [(MPL x % MPL of analytical applied) x %weight of analytical] + [(Total possible global score x % global applied) x % weight of global] The assessment tools are then mapped to the skill-based CUSP SLO. In the last 2 years, AACP has created an initiative generating a list of Entrustable Professional Activities (EPA) to set standards for graduating student pharmacist in becoming practice ready upon graduation and beyond. CUSP has incorporated EPAs as another assessment indicator to align the EPA goals with the skill-based courses such as the 1-year long Health Care Delivery series during Year 2 and the experiential education (IPPEs and APPEs) of our curriculum with the intent to evaluate student pharmacists cross-sectionally and longitudinally. Should the EPAs developed by AACP become widely accepted by other Schools of Pharmacy, CUSP will have data to compare our students assessment of their entrustability in these professional activities to students of other pharmacy schools, as well as being able to compare our faculty s assessment of our students entrustability to faculty s assessment of their students at other schools of pharmacy. Another elements of skill-based assessments are through the Advanced Experiential Education Experiences (APPE) curriculum which begins in the third professional year in Trimesters 7 and 8 following completion of all didactic curricular requirements. The APPE rotations are designed to allow student pharmacists to engage in advanced learning activities, enable direct patient care experiences, facilitate interprofessional interactions and practice, and advance professional development, confidence, and growing independence under the guidance of experiential educators. The objectives and criteria of APPE rotations are mapped to CUSP SLOs, EPA, and the activities in the Standards 2016 Appendix 2. The APPE curriculum comprises of six 6-unit consecutive rotations, which includes a minimum of 40 hours per week for six weeks equating to 240 hours per rotation, and 1440 total contact hours. Student pharmacists are assessed on these rotations through a formative evaluation at the midpoint (end of Week 3) and summative final evaluation (end of Week 6) for each rotation. The evaluation for each rotation assesses student performance and demonstration of CUSP SLOs based upon the level of supervision (degree of entrustability) required. Each experiential educator uses the levels of supervision designated with the following scoring system to rate each SLO associated with the rotation: 4 = General Direction, 3 6

= Intermittent Supervision Required, 2 = Reactive Supervision Required, 1 = Direct Supervision Required, 0 = Observer Only, and NA = Not Applicable. Rotation grades are reported on a 4.0 scale and assigned according to CUSP s standard GPA based grading scale. To pass each APPE rotation, the student pharmacist must meet the designated number of mandatory contact hours as confirmed by the experiential educator for each rotation as well as complete the Student Evaluation of Experiential Site and Educator document prior to the end of the 6 weeks. Student pharmacists must earn scores of 70% (equivalent to GPA of 2.0) or greater per rotation in order to pass. For consistency, all cognitive level mapping and examination item coding in CUSP use the same 4 levels which are defined as below. The cognitive levels are designed to monitor for the development of critical thinking skills and practice independency of the learners. 1F = Foundational - Knowledge is recalled and can be re-communicated in new ways. Relationships between facts or concepts are realized to demonstrate comprehension. 2I = Intermediate - Data can be used from various sources to solve problems. Concepts can be applied to new contexts. Information and concepts can be explored, compared, organized. 3A = Advanced - Judgment and analysis using sound evidence/criteria is regularly used. Problem solving is performed regularly using critical thinking skills such as analysis, evaluation, inductive and deductive reasoning. 4M = Mastery - Ability to develop independent ideas and to generate the most optimal solution to problems while considering various viewpoints. Integration of prior learning is used regularly As the CUSP program matures, additional assessments are being implemented. These include: correlative studies of IEs to standardized exams, longitudinal data reporting, and comparative analyses between cohorts to monitor for program progress and quality sustainability. Many of these reports are shared with faculty and student pharmacists through group presentations and individually. The data and reports are being shared in Section III below. To validate the proper utilization of the integrated exam to support for curriculum or assessment decision making, CUSP consistently conduct data analyses to achieve this goal. Part of the data were presented in last year s report with the addition of Pharmacy Curriculum Outcomes Assessment (PCOA) correlative analyses presented below in this report. With the execution of over 60 integrated exams consisting of 8,131 items cumulatively, we have achieved an KR-20 of over 80% in each of our integrated exams, and average biserial of over 0.2 in each of the integrated exams. Student Performances in integrated exams are tracked individually as well as in cohort manner. OAS at CUSP conducts correlation studies to determine the ability of the integrated exam outcome predictability of student overall performances and success in standardized examination (e.g. PCOA). 7