INSTRUCTIONAL DESIGN AND ASSESSMENT Design of a Pharmaceutical Care Laboratory: A Survey of Practitioners

Transcription

1 INSTRUCTIONAL DESIGN AND ASSESSMENT Design of a Pharmaceutical Care Laboratory: A Survey of Practitioners Rasma S. Chereson, PhD, Rhonda Bilger, BS, Shannon Mohr, BS, and Cindy Wuller, MS St. Louis College of Pharmacy Submitted January 12, 2004; accepted July 10, 2004; published January 21, Objectives. To obtain practitioners opinions on the skills necessary for a pharmaceutical care practice and their assessment of our students competencies in relevant activities. The goal was to use these data to design a pharmaceutical care laboratory course that would prepare students for current as well as future practice of pharmacy. Methods. Two hundred ninety-one questionnaires were sent to preceptors involved in the experiential program at the St. Louis College of Pharmacy. They were asked to indicate on a 5-point Likert scale their opinion of the relative importance of various skills and abilities in a pharmaceutical care practice and to assess our students competence in various areas. Results. According to practitioners, the most important skills in a successful pharmaceutical care practice are patient counseling, profile review for drug-related problems, interpretation and verification of prescriptions, drug information skills, and communication with health care professionals. Documentation of interventions, use of home diagnostic devices, and physical assessment were identified as areas in which students competencies needed to be addressed. Conclusions. The results of this survey were used in the development of the Pharmaceutical Care Laboratory course to help ensure its relevance to the contemporary as well as future practice of pharmacy. The findings of this study indicate that while the focus of the laboratory should be patient care activities, we should continue to integrate these with traditional elements of pharmacy practice. Keywords: pharmaceutical care, laboratory, curriculum INTRODUCTION With the advent of pharmaceutical care and the doctor of pharmacy as the first-professional degree program, curricula at schools of pharmacy across the country have been changed to reflect these developments in pharmacy practice. As part of these changes, upper-level dispensing laboratories have been modified, physically as well as instructionally, to focus more on patient care skills than on technical skills. These new laboratories are called pharmaceutical care laboratories, skills laboratories, integrated practice skills laboratories, and pharmacy practice skills laboratories. They vary in length from 1-semester to 6-semester sequences. The latter are often integrated to include pharmaceutics, pharmacology, medicinal chemistry, and therapeutics. Laboratory instructors have creatively modified their laboratories to better prepare students for pharmacy practice that reflects and incorporates the concepts of pharmaceutical care Many have also assessed the success of these new laboratories in helping the students develop the knowledge and skills needed to provide patient care Corresponding Author: Rasma S. Chereson, PhD. Address: St. Louis College of Pharmacy, 4588 Parkview Place, St. Louis, MO Tel: Fax: rchereson@stlcop.edu. 19 A new Pharmaceutical Care Laboratory course was developed as part of the curricular revision for the doctor of pharmacy degree program at the St. Louis College of Pharmacy. This is a 1-semester course taught in the fifth year (third-professional year) of the curriculum. The goals of this course are to help prepare graduates who are competent and confident in recognizing and solving drug-related problems, and who approach practice with an attitude of personal responsibility for providing effective and ethical patient care. The laboratory accomplishes these goals by providing opportunities for the students to apply knowledge they have gained in their didactic courses in a pharmacy setting to achieve positive patient outcomes, and to practice and apply the concepts of pharmaceutical care. The students are given the opportunity to practice and refine a wide variety of professional skills and behaviors. These include critical thinking and problem-solving skills, drug information skills, prescription-processing skills, and medication administration and physical assessment skills. Not only is the course new, the physical facility is new as well. The St. Louis College of Pharmacy underwent a total renovation, including construction of a Pharmaceutical Care Laboratory containing 18 work stations, each with its own computer, a compounding area, a sterile product

2 Table 1. Profile of Respondents, N=162 Characteristic Respondents, n (%) Practice Setting Independent 38 (23.5) Chain 69 (42.6) Hospital 36 (22.2) Degree BS 144 (88.9) PharmD 16 (9.9) Year Degree Obtained (10.5) (30.9) (29.0) (27.2) room, a conference room for group discussions and presentation, 2 patient counseling rooms, and a physical assessment room. Goals of the Study Our current curriculum, which includes the new Pharmaceutical Care Laboratory course, evolved through the collaborative efforts of faculty members from all disciplines. Our pharmacy practice faculty members were actively involved in this process and had considerable input in reviewing the course. We also obtained ideas for activities and outcomes for the new laboratory from colleagues at other schools of pharmacy, the literature, the CAPE Educational Outcomes and the anticipated direction of the profession. However, in this process, there was a significant lack of practitioner input. In planning the Pharmaceutical Care Laboratory course, we wanted to ensure that the content and laboratory activities reflected and helped prepare our students for the future of pharmacy practice, but also remained relevant to contemporary pharmacy practice. Our graduating students had to have the skills to be effective in the profession, and it was our responsibility to help prepare them to use their knowledge and skills in a professional setting. To achieve this, we felt obtaining practitioner input regarding the practice opportunities that should be included in the laboratory was essential, ie, we wanted to know which skills and abilities they felt were essential for a successful pharmaceutical care practice. Practitioners had the real world experience to help us design a laboratory that would enable students to become effective practitioners when they graduated. One approach we took to accomplish this was to involve practitioners in our Pharmacy Practice Laboratory as instructors. The practitioners insight and experience made the instruction more relevant for the students. Another approach we used to obtain practitioner 20 input was to have a series of meetings of a focus group consisting of practitioners from various areas of pharmacy practice. This was useful, but still offered the view of a limited number of pharmacists. To obtain input from a broader base of practitioners, we conducted a survey of all hospital and retail preceptors who participated in our experiential program. They were asked to give us their opinion of the importance of various skills and abilities in a pharmaceutical care practice. The results of this study aided us in the design and development of relevant laboratory activities. A second goal of this investigation was to determine our current students abilities to perform various skills in practice. The preceptors were asked to assess our students competencies in various skills based on their observations during their externship rotations. This assessment of our students strengths and weaknesses enabled us to determine the areas we should focus on in the development of the laboratory. METHODS Questionnaires were sent to all retail and hospital preceptors (291) who participated in the experiential program at the St Louis College of Pharmacy. The preceptors were asked to indicate, using a 5-point Likert scale (1= unimportant to 5 = very important) their opinion of the importance of various skills and abilities in a pharmaceutical care practice. They were also asked to assess the level of competency of students they had recently precepted (in the previous 5 years) in the same skills and abilities, also using a 5-point Likert scale (1 = low competency to 5 = high competency). One hundred sixty-two surveys were returned for a response rate of 56%. The data were analyzed using the SPSS (SPSS Inc, Chicago, Ill). Means were obtained for each question for each of 3 practice settings (chain, independent, and hospital), as well as overall means. Correlation coefficients were determined between the perceived importance of each skill and the ability of our students to perform that skill. A high negative correlation coefficient would indicate a possible area of concern: an important skill in which our students had low competence, or, an unimportant skill in which our students were highly competent. The former would indicate not enough emphasis being placed on that area, and the latter might indicate too much time being spent on a skill unnecessarily. RESULTS The demographic profile of the survey respondents is shown in Table 1. Chain pharmacists represented the largest group of respondents (n = 69, 42.6%).

3 Table 2. Practitioners' Perception of the Importance of Various Skills/Abilities in a Pharmaceutical Care Practice Skill/Ability The results are summarized in Table 2. In general, pharmacists felt that the majority of skills/abilities listed in the questionnaire were important (mean > 3) in a pharmaceutical care practice. Only 3 skills had an overall mean < 3: parenteral product preparation (the mean of 2.7 was due to retail pharmacists feeling it was unimportant), use and administration of immunizations (mean = 2.8), and extemporaneous compounding (mean = 2.9). The 10 skills/abilities deemed most important in a pharmaceutical care practice were patient counseling, interpretation and verification of prescriptions, profile review for drug-related problems, communication with other health care professionals, prescription processing (enter prescription into computer and prepare for dispensing), drug information skills, monitoring of drug therapy, nonprescription recommendations, and counseling and drug administration techniques. Interestingly, the majority of these are patient-centered, pharmaceutical care skills, indicating that practitioners are in agreement with academicians regarding the knowledge and skills needed for the future. Patient counseling was viewed as the most 21 Mean (SD) Chain Independent Hospital Overall 1. Patient counseling 4.82 (0.42) 4.89 (0.32) 4.59 (0.78) 4.78 (0.53) 2. Profile review for drug related problems (interactions, duplications, 4.64 (0.64) 4.63 (0.69) 4.65 (0.65) 4.65 (0.64) contraindications, etc.) 3. Documentation of interventions 4.20 (0.85) 4.32 (0.78) 4.15 (0.89) 4.22 (0.83) 4. Monitoring of drug therapy (for compliance, adverse reactions, 4.20 (0.88) 4.35 (0.63) 4.59 (0.56) 4.37 (0.75) therapeutic outcomes) 5. Communication with physicians and other health care professionals 4.40 (0.75) 4.65 (0.48) 4.68 (0.47) 4.56 (0.62) 6. Determination of appropriateness of medication 3.91 (1.07) 4.08 (0.72) 4.50 (0.62) 4.16 (0.94) 7. Knowing how to use a patient chart 3.42 (1.28) 3.43 (1.19) 4.56 (0.70) 3.78 (1.21) 8. Drug information skills 4.37 (0.88) 4.42 (0.69) 4.45 (0.62) 4.43 (0.76) 9. OTC recommendation and counseling 4.68 (0.70) 4.68 (0.53) 3.47 (0.96) 4.36 (0.91) 10. Drug administration techniques (eg, inhalation devices) 4.40 (0.76) 4.57 (0.50) 3.68 (1.07) 4.31 (0.83) 11. Patient interview 3.96 (1.01) 4.22 (0.89) 3.82 (1.04) 4.04 (0.98) 12. Use of home diagnostic devices 3.85 (0.90) 4.03 (0.73) 3.15 (0.99) 3.71 (0.92) 13. Physical patient assessment 3.06 (1.05) 3.28 (0.85) 3.24 (1.02) 3.17 (1.02) 14. Laboratory value interpretation 2.97 (1.03) 3.27 (1.10) 4.47 (0.61) 3.51 (1.13) 15. Interpret prescription and verify for completeness and accuracy 4.91 (0.33) 4.84 (0.37) 4.32 (0.84) 4.74 (0.56) 16. Enter prescription into computer and onto patient profile 4.80 (0.50) 4.59 (0.72) 3.91 (1.00) 4.49 (0.83) 17. Prepare prescription for dispensing 4.75 (0.66) 4.61 (0.73) 3.76 (1.07) 4.45 (0.92) 18. Extemporaneous compounding 2.74 (0.95) 3.68 (0.91) 2.47 (0.90) 2.87 (1.04) 19. Parenteral product preparation 2.21 (1.05) 2.49 (1.22) 3.59 (1.10) 2.74 (1.23) 20. Disease state management 3.88 (0.89) 4.16 (0.73) 4.36 (0.74) 4.10 (0.84) 21. Selection of source of drug supply 3.08 (1.13) 3.25 (1.11) 2.65 (0.92) 3.05 (1.08) 22. Use and administration of immunizations 2.55 (1.17) 2.95 (0.94) 2.94 (1.07) 2.83 (1.10) 23. Selection of drug/dosage form 3.72 (0.99) 4.11 (0.94) 3.94 (1.01) 3.91 (0.98) *Scale: 1 = unimportant to 5 = very important important skill (mean = 4.8). All respondents rated the importance of skills in patient counseling at 3 or higher, and 79.6% rated these skills at 5 (very important). This was followed closely by prescription interpretation and verification (mean = 4.74). As expected, there were differences of opinion on the importance of various skills depending on the pharmacists practice settings. For example, hospital pharmacists felt that knowing how to use a patient chart was very important (mean = 4.6), but, not surprisingly, chain and independent pharmacists did not (mean response was 3.4 for both groups ). Similarly, hospital pharmacists also felt laboratory value interpretation was important (mean = 4.5), but chain pharmacists did not (mean = 3.0). Figure 1 shows additional skills/abilities for which there were significant differences in perceived importance due to practice setting. The greatest agreement between practice settings on the importance of various skills occurred for drug information, profile review, patient counseling, documentation of interventions, physical assessment, and communication with physicians.

4 Table 3. Practitioners'/Preceptors' Assessment of Students' Competency in Various Skills/Abilities Mean (SD) Skill/Ability Chain Independent Hospital Overall 1. Patient Counseling 3.82 (0.96) 3.94 (0.91) 3.41 (0.73) 3.76 (0.94) 2. Profile review for drug related problems (interactions, duplications, 3.77 (0.80) 3.84 (0.77) 3.45 (0.83) 3.68 (0.84) contraindications, etc.) 3. Documentation of interventions 3.07 (0.73) 3.29 (1.00) 2.78 (0.93) 3.07 (0.88) 4. Monitoring of drug therapy (for compliance, adverse reactions, 3.35 (0.83) 3.57 (0.82) 3.17 (0.75) 3.36 (0.84) therapeutic outcomes) 5. Communication with physicians and other health care professionals 3.68 (0.91) 3.79 (0.96) 3.22 (0.94) 3.57 (0.98) 6. Determination of appropriateness of medication 3.50 (0.80) 3.93 (1.01) 3.53 (0.80) 3.61 (0.89) 7. Knowing how to use a patient chart 3.58 (1.03) 3.73 (0.88) 3.28 (1.08) 3.43 (1.07) 8. Drug information skills 3.87 (0.82) 4.33 (0.71) 3.74 (0.86) 3.91 (0.85) 9. OTC recommendation and counseling 3.35 (1.19) 3.61 (1.00) 3.16 (0.83) 3.38 (1.08) 10. Drug administration techniques (e.g. inhalation devices) 3.64 (0.90) 3.76 (0.94) 3.22 (0.85) 3.53 (0.95) 11. Patient interview 3.45 (1.03) 3.59 (0.95) 3.21 (0.71) 3.42 (0.94) 12. Use of home diagnostic devices 2.85 (1.04) 3.13 (1.06) 2.55 (1.04) 2.86 (1.05) 13. Physical patient assessment 2.83 (0.79) 3.05 (0.74) 2.57 (0.76) 2.81 (0.79) 14. Laboratory value interpretation 3.21 (0.86) 3.53 (1.22) 3.16 (1.05) 3.20 (1.04) Prescription Processing: 15. Interpret prescription and verify for completeness and accuracy 4.17 (1.04) 4.21 (0.78) 3.97 (0.85) 4.08 (0.94) 16. Enter prescription into computer and onto patient profile 4.09 (1.11) 4.21 (0.78) 3.52 (0.98) 3.95 (1.01) 17. Prepare prescription for dispensing 4.27 (1.05) 4.33 (0.60) 3.90 (0.71) 4.18 (0.87) Extemporaneous Compounding: 18. Perform appropriate calculations 3.96 (0.82) 4.13 (0.68) 3.93 (0.92) 3.99 (0.83) 19. Accurately weigh and measure ingredients 4.19 (0.80) 4.47 (0.68) 4.04 (0.93) 4.21 (0.81) 20. Use appropriate procedure to compound the product 3.96 (0.99) 4.23 (0.73) 3.88 (0.93) 4.02 (0.90) Parenteral Product Preparation: 21. Perform appropriate calculations 3.45 (0.93) 4.22 (0.83) 3.90 (0.87) 3.81 (0.96) 22. Use correct aseptic technique 3.55 (0.82) 4.13 (0.83) 3.29 (1.24) 3.44 (1.14) *Scale: 1 = low competency to 5 = high competency Figure 1. Practice setting differences in importance of various skills/abilities in a pharmaceutical care practice. Skills and abilities not listed on the survey but which many pharmacists indicated were important included dealing with third parties (primarily insurance companies), personnel relationships, and having knowledge of herbal medications. The preceptors assessment of our students competency is summarized in Table 3. The preceptors felt that 22 our students performance of the majority of the skills evaluated in the survey instrument was more than satisfactory (mean > 3.0). The mean was less than 3.0 for only 2 of the skills listed: use of home diagnostic devices (mean = 2.9) and physical patient assessment (mean = 2.8). Other areas of possible concern indicated by the results included documentation of interventions (mean = 3.1) and laboratory value interpretation (mean = 3.2). Our students were assessed as being the most competent in prescription processing skills, including prescription interpretation and verification (mean = 4.1), and in extemporaneous compounding skills (mean = 4.1). There were distinct differences in how pharmacists in different practice settings assessed our students abilities. Figure 2 indicates these differences for several of the skills. In general, hospital pharmacists rated our students competency lower than retail pharmacists; independent pharmacists, on the other hand, consistently rated our students competency higher than both chain and hospital pharmacists.

5 Figure 2. Practice setting differences in assessment of students' competence in various skills/abilities. DISCUSSION Not surprisingly, there was considerable difference of opinion among practitioners regarding the importance of various skills and abilities in a pharmaceutical care practice depending on their practice setting. This was especially true of the more site-specific skills, such as laboratory value interpretation, using a patient chart, and parenteral product preparation. This presents a dilemma in designing the laboratory activities, since our goal is to prepare a generalist practitioner. But there was also considerable agreement on certain skills and abilities, regardless of the practice setting. All pharmacists were in agreement that patient counseling, profile review for detecting and resolving drug-related problems, communication with health care professionals, drug information skills, documentation of interventions, and monitoring of drug therapy were important. The practitioners also agreed that skills and abilities associated with a more traditional pharmacy practice were very important, eg, prescription processing received an overall mean in importance of 4.7. With the diversity of focus of various practice settings and our goal of educating a generalist practitioner, identification of ability outcomes for the laboratory, and laboratory activities, was a complex issue. Students will be exposed to most of the practice skills to varying degrees: those for which there was not general agreement as to importance may only be touched upon, whereas those that all pharmacists felt were important will receive greater emphasis. When the practitioners assessed our students competencies in the various skills, independent pharmacists consistently rated them higher than did chain and hospital pharmacists. Perhaps this may be explained by the fact that independent pharmacists may have more time to devote to the students, enhancing their learning and improving their competency in the various skills. This may also be a self-selecting phenomenon: students who have requested independent pharmacies for their 23 community rotations may be students who are more interested in patient care skills and more motivated to perform these skills. Hospital pharmacists generally rated our students competencies lower than did independent and chain pharmacists. This may be attributed to the fact that most of our students work in community pharmacies and have little or no hospital experience. Our goal in the overall competency assessment was to identify areas in which our students need improvement so that we may address these in the laboratory. These areas included documentation of interventions, use of home diagnostic devices and physical assessment. Limitations of the study were that this was a regional survey, only 16 of the 162 respondents had a PharmD degree, and that it was a subjective analysis by participants. CONCLUSIONS The goal of the Pharmaceutical Care Laboratory course is to help prepare students to effectively practice in a setting where pharmaceutical care is the focus. It is designed to provide the students with opportunities to develop skills and habits to solve patient care problems. It is also intended to help instill an attitude of personal responsibility for providing patient care. The emphasis is on skills and abilities that are relevant and important to the future, as well as contemporary practice of pharmacy. Thus, practitioner input was important in the development of the laboratory. The data we obtained were used along with extensive input from other professional sources and our own view of the future of pharmacy practice to design the laboratory. Our goal was to integrate the traditional elements of the profession with the skills needed for the future. This study identified which skills and abilities practitioners perceive as important in a pharmaceutical care practice. In addition, feedback from preceptors regarding our students abilities to perform the various skills included on the survey instrument helped us determine the level of emphasis that should be placed on these skills in the laboratory. Increased focus on those areas that were perceived as important, but in which our students abilities were not strong, will better prepare our students to provide patient care in a professional, competent manner. The challenge for us was incorporating the wide variety of activities into a 1-semester laboratory that would give students opportunities to practice those skills deemed important. The results of this survey underscored the importance of not forsaking the traditional aspects of pharmacy practice while we planned a laboratory course that would help prepare our students for the future. While educators have the responsibility to anticipate and lead change in pharmacy practice, and not simply meet current demands, we must also

6 be cognizant of our responsibility to help prepare students to assume their professional role today. With these considerations in mind, we made some significant changes in our current Pharmacy Practice Laboratory. While we will continue many of our ongoing activities, we will also introduce new and/or expanded activities. Specifically, students will assess patients, devise a patient-specific plan, and develop monitoring parameters for assessing a patient s progress, and revise plans as needed to ensure positive patient outcomes. Students will thus provide all aspects of pharmaceutical care including processing medication orders; compounding specialized dosage forms; preparing sterile products; assessing, educating, and monitoring patients; and providing information to other health care professionals. With these changes, the Pharmaceutical Care Laboratory course will provide our students the opportunity to develop and practice lifelong skills that will help prepare them for current as well as future pharmacy practice. ACKNOWLEDGEMENTS The authors would like to acknowledge the generous assistance of our colleagues, Dr. Peter Hurd and Dr. Timothy McPherson, in the data analysis and preparation of this manuscript. REFERENCES 1. Wellman GS, Larson R. Using web-based prescription simulations as an active learning tool in an integrated practice skills laboratory. Am J Pharm Educ. 2002;66: Buring SM, Brown B, Doherty M, et al. Designing a pharmacy practice skills laboratory to synchronize skill development with didactic learning in cardiovascular therapeutics. Am J Pharm Educ. 2002;66:91S. 3. Cobb HH, Francisco G, Johnson T, et al. Preparing students to preserve the profession of pharmacy. Am J Pharm Educ. 2000;64 (Winter Supplement):128S. 4. Pugh CB, Garvin CL, Peters PH. Second year pharmacy skills laboratory: opportunity for innovation. Am J Pharm Educ. 1999;63 (Winter Supplement):99S. 5. Peterson CD. Concept pharmacy: NSDU s vision for the future. Am J Pharm Educ. 2000;64:131S. 6. Newton GD, Tracy TS, Popovich NG. Development and implementation of an integrating pharmacy practice laboratory. Am J Pharm Educ. 1990;54: Ginsburg DB, Atkins AD. Use of videotaped counseling to improve pharmacy students communication skills: descriptive report [abstract]. ASHP Midyear Clinical Meeting;1989;24: Fuhrman LC, Brady TF, Buff WE, et al. Enhancing student learning by bringing pharmacy practitioners into the classroom. Am J Pharm Educ. 1998;62:131S. 9. Van Riper GC, Billow JA. Laboratory exercises in pharmacy practice. J Pharm Teach. 1993;4: Brown TA. Renovation of a dispensing laboratory into a multifunctional teaching area. Am J Pharm Educ. 1999;63:87S. 11. Deloatch KH, Brock TP, Pittman AW. Design and renovation of a pharmaceutical care teaching space. Am J Pharm Educ. 1998;62 (Winter Supplement):129S Adamcik B, Airmet D. Multi-cohort learning; teaching pharmacy students about compliance, counseling, and mentoring. Am J Pharm Educ. 1998;62: Deloatch K, Brock T. UNC pharmaceutical care laboratory: where pharmacy education and practice meet. Carolina J Pharm. 1996;76: Sorense TD. Use of contract learning in a pharmaceutical care practice laboratory. Am J Pharm Educ. 1999;63:74S. 15. Remmel RP, Zimmerman CL. Design of a new PharmD curriculum at the University of Minnesota. Am J Pharm Educ. 1994;58:118S. 16. Marquess JG, Wickman JM, Pirkle J. Incorporating a pharmacy based immunization training program into a pharmacy practice laboratory course. Am J Pharm Educ. 2000;64:95S. 17. Deninger M, Catney C. Using the WebCT student presentation tool for students collaborative projects in a pharmacy practice laboratory course. Am J Pharm Educ. 2000;64:117S. 18. Marquess JG, Wickman JM, Marshall LL. Design, development and implementation of a pharmaceutical care based pharmacy practice laboratory focusing on disease state management. Am J Pharm Educ. 1998;62:97S. 19. Triplett JW,Oh T, Aceves-Blumenthal C. Modular pharmacy practice laboratory course integrating role-playing scenarios with community and hospital practice. Am J Pharm Educ. 1992;56: Shrewsbury RP. Hybrid CD-Internet delivery system for pharmaceutical care laboratory instruction. Am J Pharm Educ. 2000;64:106S. 21. Brock MS, Deloatch KH. Pharmaceutical care in education: helping pharmacists find their feathers. Pharm Times. 1996;62:1HPT-3HPT. 22. Austin Z, Tabak D. Design of a new professional practice laboratory using standardized patients. Am J Pharm Educ. 1998;62: From barks to bytes: the evolution of dispensing. The Wisconsin Pharmacy Graduate. 1998;Spring: Chambers SL, Schmittgen J, Allan CR. Evaluation of peer teaching in a pharmaceutical care laboratory. Am J Pharm Educ. 2000;64: Cobb HH, Thomas PC, Schramm LC, et al. Assessing student perception of large- and small-group instruction. Am J Pharm Educ. 2001;65 (Winter Supplement):104S. 26. Fant WK, Wall A, Brown B, et al. Professional and general abilities assessment in a pharmacy practice skills laboratory sequence. Am J Pharm Educ. 2001;65:113S. 27. Cobb H, Schramm L, Thomas P, et al. Assessing a pharmacy skills laboratory in a new Doctor of Pharmacy semester curriculum. Am J Pharm Educ. 1999;63:75S. 28. Whelan AM, Mansour S, Farmer P. Outcomes-based integrated hybrid PBL curriculum. Am J Pharm Educ. 2002;66: O Sullivan TA, Narducci W. Measuring practice skills: experience with a problem-based final examination format for a dispensing laboratory course. Am J Pharm Educ. 1999;63:77S. 30. Schramm LC, Thomas PC, Cobb HH, et al. Survey of community externship preceptors on student preparation and performance. Am J Pharm Educ. 1998;62:115S. 31. Schramm L, Thomas P, Cobb H, et al. Survey of hospital externship preceptors on student preparation and performance. Am J Pharm Educ. 1999;63:74S.