Instructional Design and Assessment

Transcription

1 Instructional Design and Assessment Evaluation of a Pharmacy Continuing Education Program: Long-Term Learning Outcomes and Changes in Practice Behaviors Nancy F. Fjortoft, PhD a ; Amy H. Schwartz, PharmD b a Midwestern University Chicago College of Pharmacy b Creighton University School of Pharmacy and Health Professions Objectives. To assess the long-term outcomes from a 3-month, curriculum-based pharmacy continuing education (CPE) program on lipid management and hypertension services, particularly changes in practice behaviors. Methods. An interactive lipid management and hypertension services pharmacy continuing education program focusing on the development of cognitive and psychomotor skills was developed and offered. A pre- and post-program survey was developed to assess participant knowledge related to 12 cognitive and psychomotor concepts derived from program objectives. Participants were asked about their practice settings and percentage of time engaged in practice-related activities, and whether the program influenced subsequent practice behaviors, specifically the provision of clinical services. Results. Forty-six of 49 participants completed the preprogram survey with 65% and 37% postprogram response rates at 6 and 12 months, respectively. Significant improvements in participant knowledge base and skills were observed between pre- and both post-survey administrations. No change in the percentage of time spent providing clinical services was observed at 6 months or at 12 months. Conclusions. The findings suggest that pharmacists can realize long-term gains in cognitive/psychomotor skills from curriculum-based interactive CPE. However, the benefits derived did not significantly affect practice behaviors, including the provision of clinical services. Keywords: continuing pharmacy education, practice behavior, learning outcomes, lipid management, hypertension INTRODUCTION Continuing pharmacy education (CPE) is well accepted in the profession of pharmacy. Fifty-one boards of pharmacy in the United States require CPE as a prerequisite to relicensure. 1 The American Council on Pharmaceutical Education (ACPE) has endorsed CPE, and in conjunction with representatives from the profession, has developed an accreditation process for approving providers and CPE certificate programs. Adherence with the ACPE standards Corresponding Author: Nancy F. Fjortoft, PhD. Mailing Address: Midwestern University Chicago College of Pharmacy, st Street, Downers Grove, IL Tel: Fax: nfjort@midwestern.edu. is required to maintain provider status. 2 While CPE appears to be thriving and well regulated, an essential question remains unanswered: does participation in CPE enable the individual pharmacist to change and enhance his or her practice? A curriculum-based CPE program was developed and assessed in an attempt to address this question. The objective of this paper is to describe the long-term cognitive and psychomotor gains of participants in a 3-month, curriculum-based CPE program. In addition, changes in pharmacists practice behavior are also examined. In an attempt to answer the question Is continuing professional education effective in positively changing practice behavior? Nona et al 3 identified 142 studies that evaluated outcomes associated with continuing professional education, 12 of which pertained to pharmacy. The remainder, in descending order based on number of studies, involved medicine, nursing, allied health, and dentistry. The predominate assessment tools were ques- 1

2 tionnaires, examinations, and audits. Continuing professional education in the health sciences overall was effective in changing participants attitudes, knowledge, and performance and in improving patient outcomes. 3 However, none of the studies identified had evaluated changes in pharmacists attitudes or examined the effect of CPE on patient outcomes. Five of the studies described improved pharmacists knowledge. Seven studies examined changes in pharmacists behavior. Of those, 4 noted positive changes in behavior, 2 reported mixed results, and 1 reported no change in behavior. Whether these changes were long term or short term is not known. A later review article by Davis, 4 identified over 100 studies that examined the effectiveness of educational interventions with continuing medical education (CME), in changing physicians knowledge, skills, and behavior, and patient outcomes. Interventions included non-interactive printed materials, formal live courses and seminars, and audits and outreach visits. More than two thirds of the studies found that the educational intervention resulted in a positive change in physician behavior and almost half produced improved patient health care outcomes. Additionally, the delivery method used for the educational program had a profound impact on whether it positively affected physicians knowledge, skills, behavior, and patient outcomes. Programs that used traditional didactic sessions (ie, passive learning) did not enhance physician performance or patient outcomes. In an updated review of published CME interventions by the same authors, their conclusion remained the same: interactive CME sessions were more likely to change performance and patient outcomes than traditional didactic sessions. 5 The same group of investigators recently authored a review in The Cochrane Collaboration, the objective of which was to assess the effectiveness of professional educational meetings (all disciplines) on practice and health care outcomes. 6 Similar to what was observed for CME, the findings suggest interactive workshops are more likely than traditional didactic sessions to result in positive changes in professional practice. Two recent studies assessed the impact of CPE certificate programs on pharmacist knowledge, skills, and behavior. Certificate programs are curriculumbased CPE courses that require participants to demonstrate learned competencies based on stated objectives. These programs are typically longer in duration ( 30 contact hours); and involve a mixture of selfstudy, live seminars, and experiential training, as well as formative and summative assessments. 7-9 Patterson 10 evaluated a year-long Pharmaceutical Care Certificate Program (PCCP) that consisted of live, interactive workshops and distance-learning components and found significant changes in documentation of clinical interventions and monitoring of patient outcomes between PCCP group and a control group; however, the sample size was very modest. Barner and Bennett 11 described the findings from a pre- and post-test analysis of their 9-month PCCP, which used home study and monthly live interactive workshops. The pharmacists participating in this study felt more confident in their ability to provide pharmaceutical care at the end of the program. Significant changes in the identification and resolution of drugrelated problems and patient monitoring were observed. Post-test analyses did not identify any significant differences in the frequency of performing these functions. In contrast to the reviews presented earlier regarding CME, CPE programs, based on a limited number of studies, have demonstrated limited changes in pharmacist practice behaviors. The difference may be because different personalities are attracted to medicine as opposed to pharmacy, or because of differences in practice opportunities. Also specific details regarding the content and delivery of CPE programs were not provided, making direct comparisons difficult. Further analysis is required to determine why differences in the impact of CME and CPE on physicians and pharmacists exist. Therefore, the objective of this study was to assesses the long-term cognitive, psychomotor, and practice behavior changes of participants in a curriculum-based CPE program. METHODS Course Description and Population Demographics We developed a 3-month long, curriculum-based CPE program entitled Lipid Management and Hypertension Services. Although similar in structure to a certificate program, the decision was made administratively not to promote it as such because several requirements for categorization were not met. The program was supported through registration fees and an unrestricted educational grant from Pfizer, Inc. The program utilized a combination of self-study and 3 live, interactive workshops. The self-study materials were print-based and factual in nature. Upon completion, each participant received 20 contact hours of CPE. Program goals were to provide pharmacists with the knowledge and skills necessary to develop and implement all aspects of a clinical service targeted for patients 2

3 with lipid disorders and/or hypertension. The first workshop reviewed the pathophysiology and pharmacotherapeutic considerations associated with lipid disorders (dyslipidemias) and hypertension. Faculty reviewed self-study materials and the self-assessment and facilitated case discussions. Participants were taught simple physical assessment techniques such as blood pressure measurement. The second workshop provided a systematic approach to the development of a clinical service, covering such issues as policies and procedures, regulatory requirements, patient documentation systems, etc. Experiential training included demonstrations regarding the procurement, use, and maintenance of a Cholestech LDX machine (Cholestech Corporation, Hayward, CA). The third and final workshop outlined the business aspects of clinical services, reviewed the development of a business plan, plans for the physical layout of the facility, and marketing and reimbursement strategies. Recruitment began in October 2000, at which time announcements were sent to college alumni and all licensed pharmacists in the state of Illinois. Selfstudy materials were mailed to registrants in early December of the same year. The live workshops were conducted monthly from January 2001 through March 2001 on the campus of a suburban university. Fortynine participants enrolled in the course. The enrollment was limited to 50 to maximize small group interaction and skill development. Survey Instrument Development and Data Collection A simple 2-page, pretest survey instrument was developed from program objectives (Appendix 1). The first 12 questions on the survey instrument used a 5-point adjectival scale with discrete responses to assess participants perceptions of their knowledge of the program objectives at the beginning of the program. Questions addressed both cognitive and psychomotor skills. The survey instrument gathered participant demographics including practice-related issues such as practice setting and percentage of time spent engaged in particular professional activities (prescription processing, clinical practice, management, education, and research). The pretest was administered to participants during the first workshop in January Survey instruments were coded for follow-up data collection and analysis. The posttest was mailed to participants 6 months after program completion (October 2001) along with a cover letter, self-addressed, stamped envelope, and $1.00 as a small incentive (Appendix 2). The posttest differed slightly from the pretest in that participants were asked if they were currently providing clinical services, and if so whether the CPE program provided assistance. Participants who responded no to this question were asked to explain why. A list of barriers was provided to facilitate responses. A write-in option was also available. A second post-test was mailed to participants 12 months after course completion (March 2002), along with a cover letter and self-addressed, stamped envelope, without incentive. The 12-month survey did not assess barriers. Statistical Analyses Data analysis was performed using SPSS, version 11.5 for Windows. 12 The Wilcoxon Signed Ranks Test was used to determine differences between pretest and posttest scores (6- and 12-month assessments) for the 12 program objectives (assessing cognitive and psychomotor skills). Student t-tests were used to identify differences between the percentages of time engaged in practice-related activities. Lastly, descriptive statistics were used to categorize sample demographics and changes in practice behaviors. RESULTS Forty-six participants completed the pretest survey, resulting in a response rate of 94%. Three participants chose not to participate in the study for unknown reasons. The response rates for the 6-and 12-month surveys were 32 (65%) and 18 (37%), respectively. Participant demographics are listed in Table 1. A little over half of the participants were female and had a bachelor of science (BS) degree in pharmacy. Approximately one third worked in either a hospital or retail setting. The mean time since initial licensure was 14.5 years. The medians of participants self-assessment scores of their knowledge and skills for the 12 program objectives are listed in Table 2. Improvements were noted for all of the objectives at 6-months and sustained through the 12-month assessment. The items with decreasing scores at 12 months were those objectives dealing primarily with operational and administrative responsibilities. Results from Wilcoxon Signed Ranks analyses are presented in Table 3. Significant improvements in 10 of the 12 program objectives were noted at 6 months. Physical assessment skills and pharmacological and nonpharmacologic treatment options for hypertension were the only objectives for which significant changes were not observed. Significant improvement in 9 of the 12 program objectives was noted at 12 months. 3

4 Table 1. Demographics of Participants Demographic n (%) Gender Male 21 (46) Female 25 (54) Degree BS 26 (56.5) PharmD 20 (43.5) Practice Setting Hospital 16 (35) Independent retail 5 (11) Corporate chain retail 11 (24) Consultant 1 (2) Other* 13 (28) Years Licensed 14.5 (11.95) *Other: pharmaceutical industry, long-term care, and ambulatory care Mean + standard deviation Table 2. Participants Median Self-assessment Scores* for Achievement of the Learning Objectives Objective Prior to CPE (n=49) 6 months after CPE (n=25) 12 months after CPE (n=18) Pharmacologic and nonpharmacologic treatment options for hyperlipidemia Developing treatment plans for patients with hyperlipidemia Physical assessment skills (ie, blood pressure measurement) Pharmacologic and nonpharmacologic treatment options for hypertension Developing treatment plans for patients with hypertension Developing policies and procedures congruent with regulatory requirements Operation and maintenance of the Cholestech LDX Equipment, supplies, and documentation requirements Physical layout and workflow requirements Reimbursement terminology and strategies Role and development of a business plan Marketing terminology and strategies *Scale: 1 = poor to 5 = excellent. CPE = continuing pharmacy education 4

5 Table 3. Differences in Self-Assessment Scores for Program Objectives After CPE 6 Months After CPE 12 Months After CPE Objective Z * P Z * P Pharmacologic and nonpharmacologic treatment options for hyperlipidemia Developing treatment plans for patients with hyperlipidemia Physical assessment skills (ie, blood pressure measurement) Pharmacologic and nonpharmacologic treatment options for hypertension Developing treatment plans for patients with hypertension Developing policies and procedures congruent with regulatory requirements Operation and maintenance of the Cholestech LDX Equipment, supplies and documentation requirements Physical layout and workflow requirements Reimbursement terminology and strategies Role and development of a business plan Marketing terminology and strategies * Wilcoxon Signed Ranks Test Alpha < 0.05 Findings regarding the percentage of time spent in practice-related activities are reported in Table 4. No significant differences between preanalysis and postanalysis were observed for any of the practicerelated activities. Comparisons of pre-test and post-test data regarding the provision of clinical services are presented in Table 5. Fifteen of the 46 original participants (33%) provided some type of clinical service at the beginning of the program. At 6 months this number decreased to 7 of 32 respondents (22%). At 12 months, 6 of 18 participants were providing some type of clinical service (33%). Of the 32 respondents to the 6-month survey, 13 cited barriers to the provision of clinical services (Table 6). Twelve additional respondents chose not to answer this question. The top 3 barriers were lack of financial support, time, and structural constraints. DISCUSSION The findings from this study support the conclusion that pharmacists are seeing long-term cognitive gains from curriculum-based interactive CPE. However, cognitive gains are not translating into positive changes in practice behavior. Study participants did not report changes in the percentage of time spent on prescription processing, clinical service, management, education, or research at either the 6-month follow-up or the 12-month follow-up. We expected to see an increase in the percentage of time spent in clinical services and a decrease in the percentage of time spent processing prescriptions. Participants did not report an increase in the number of clinical services provided. Rather, a decrease was seen. The studies presented earlier by Barner and Bennett 11 and Patterson 10 also noted that improvements in knowledge and skills do not necessarily translate into improvements in practice behavior. One of the limitations noted for both of these studies was the time between program completion and follow-up assessment, ie, a 3-month period. The authors suggested that a longer duration might enhance implementation rates; however, the present findings do not support this. What is contributing to the difference observed between the effectiveness of continuing education in medicine and other health disciplines and CPE? Why are other healthcare providers, particularly physicians, 5

6 better able to translate knowledge and skills into improved practice? As previously mentioned, personality types may drive discipline selection; however, it is unlikely that this factor alone influences learning Table 4. Mean Percentage of Time Spent Engaged in Practice-Related Activities Activity Prior to CPE* %(SD) 6 Months After CPE %(SD) 12 Months After CPE %(SD) Prescription processing (33.57) (35.34) (37.05) Clinical practice (31.27) ( (35.46) Management (17.95) (30.61) (25.59) Education 8.50 (9.85) 8.50 (6.42) (9.35) Research 2.29 (5.40) 3.14 (3.63) 1.17 (2.92) *CPE= continuing pharmacy education SD = standard deviation Table 5. Number and Percentage of Participants Providing Clinical Services Services Prior to CPE (%) 6 Months After CPE (%) 12 Months After CPE (%) Lipids 0 (0) 2 (7) 1 (5.5) Hypertension 3 (7) 1 (3) 1 (5.5) Both 2 (5) 3 (10) 0 (0) Neither 28 (65) 24 (77) 12 (67) Other * 10 (23) 1 (3) 4 (22) * Other: hematology, oncology, anticoagulation, and diabetes Table 6. Reported Barriers to Providing Clinical Services (N=24) Barrier n* Lack of financial support 3 Lack of time 3 Structural problems 2 Conflict of interest 1 Work as consultant 1 Left pharmacy 1 Temporarily out of work 1 No response 12 *n = number of respondents reporting styles. 13 A more plausible explanation for the difference may be related to the actual and perceived barriers experienced by pharmacists. Pharmacists may encounter 6 types of barriers to practice environmental, communication, educational, financial, legal, and attitudinal, each of which has internal and external influences. In most instances more than one type of barrier is 14, 15 present. 6

7 Overcoming barriers is challenging as they are not typically discrete and may exist on many levels. Lack of financial support, time, and managerial responsibilities were the most frequently reported barriers identified by program participants. Participants did not specifically identify attitudinal, communication, and educational barriers as contributing to the provision of clinical services. A recent report by Amsler et al 16 suggests that personal attitudinal barriers may be less prevalent than previously reported. The results from this study and similar evaluations suggest CPE alone provides insufficient guidance to overcome barriers. It is unclear why such a large number of participants chose not to provide information regarding barriers to implementing practice changes. One possible reason may be the relatively small sample size. Participants may have been concerned that they were identifiable. Additional reasons include limited time to complete the survey, or it was not possible to provide a simple explanation for the barriers to implementation that they have faced. If interactive, curriculum-based continuing education programs are only part of the solution to improving or influencing changes in clinical practice, what other measures need to be undertaken? In their thoughtful series of articles entitled Transitions in Pharmacy Practice: Part 1-5, Holland and Nimmo 13, propose a model for change that extends beyond the provision of educational programming. These authors propose that, regardless of the nature of a proposed change in practice, there are 3 sets of conditions that must be simultaneously satisfied before the change is likely to be implemented. If any of the conditions is not met or is not met simultaneously with the others, it is predicted that the process of achieving change is likely to falter. 20 The 3 conditions are: the availability of or access to appropriate learning resources; motivational strategies; and an appropriate/acceptable practice environment. Individual, administrative, and institutional buy in or support is necessary for success. Is there a way then to enhance the CPE process to influence practitioner performance and health care (patient) outcomes? Both Nona 3 and Davis 4 recommended that needs assessments be performed prior to program development. Scott et al 21 provided findings from a statewide assessment to determine future CPE offerings. Interestingly, the findings by these authors support the model proposed by Holland and Nimmo, 17 who recommended that program development be based on the skills, professional socialization, and judgment of the participants. Gender, age, practice locale, and position or specialty influence an individual s choices regarding CPE. The delivery method used in a CPE course will depend on the subject matter and the audience. Interactivity and interest in the subject matter influence retention of learned materials. Discussions about identifying and overcoming barriers are essential to facilitating post hoc implementation of the concepts learned. Formative and summative evaluations should be incorporated into the CPE courses to ascertain changes in practice behaviors and health care outcomes. Study Limitations This study has limitations. The time between course completion and posttest administration may not have been long enough for participants to build the infrastructure required to provide clinical services. In addition, the sample size was small, thus limiting the validity of any generalizations made. The decrease in response rate between pretest, 6-month posttest, and 12-month posttest was also a study limitation. The respondent pharmacists self-reported perceptions of knowledge, skills, and practice behavior. Directly observing and testing the pharmacists may have resulted in more accurate data, but was not possible for this study. In addition, the self-study material mailed to participants prior to the first data collection may have masked the true baseline knowledge of the participants. The analysis examining the percentage of time that the participants spent engaged in practice-related activities did not take into consideration vacancies in staffing, promotions, or career changes that may have occurred between survey distributions. In addition, the 5 categories of practice-related activities may have been too broad. Participants may indeed be spending the same or more time on prescription processing, but they may be performing those activities with a higher level of patient care skills. Lastly the 12-month survey did not address the issue of barriers. Both of these omissions may account for some of the instability or regression towards the mean observed. CONCLUSIONS The findings presented suggest that pharmacists can realize long-term cognitive/psychomotor benefits from curriculum-based interactive CPE. However, the benefits derived did not positively affect practice-related behaviors, including the provision of clinical services. The results of this and earlier studies suggest that educationally sound CPE is not enough to facilitate changes in clinical practice. 7

8 The ultimate goal of CPE is to improve clinical practice and patient outcomes. In order to move towards these goals, CPE providers may consider assessing the educational needs, background, and interests of potential participants in order to determine the most appropriate educational strategies. The teaching methods selected should be directed towards development of the desired knowledge and skills of participants. Reinforcement through professional practice and additional educational support may be required to sustain learning. Individual motivation and administrative and institutional support are the other essential components for success. Discussions regarding implementation barriers are essential, especially for practice-based programs. The development of long-term relationships between peers, ie, mentoring or professional support systems, may assist CPE participants with overcoming barriers, as well as facilitate change and enhance professional socialization. Finally, a continued effort towards long-term assessment of CPE programs is necessary to monitor progress and determine future quality improvement measures. 5. Davis D, O Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education. JAMA. 1999;282: Cochrane Collaboration. Continuing education meetings and workshops: effects on professional practice and health care outcomes. J Cont Educ Health Prof. 2001;21: Suveges LG, Blank JW. Development of a conceptual model for certificate programs in pharmacy. Am J. Pharm. Educ. 1992;56: Fjortoft NF, Engle J P, Henkel N, Joseph J. How effective are certificate courses in continuing pharmacy education. US Pharmacist. 1997;9: Smith JL. Planning and implementation of certificateprograms in pharmacy: a review of six programs in the literature. Am J Pharm Educ. 2001;65: Patterson BD. Distance education in a rural state: Assessing change in pharmacy practice as a result of a pharmaceutical care certificate program. Am J Pharm Educ. 1999;63: Barner JC, Bennett RW. Pharmaceutical care certificate program: assessment of pharmacists implementation into practice. J Am Pharm Assoc. 1999;39: SPSS for Windows, Release 11.5, SPSS, Inc. Chicago IL (2002). ACKNOWLEDGEMENTS We would like to thank Kathleen Chavanau, PharmD, Melissa Hogan, PharmD, Kathleen Snella, PharmD and David Zgarrick, PhD, for their assistance in the development and implementation of the CPE program. The pharmacy continuing education program described in this paper and the evaluation study was funded in part through an unrestricted educational grant from Pfizer, Inc. 13. Nimmo CM, Holland RW. Transitions in pharmacy practice, part 4: Can a leopard change its spots? Am J Health-Syst Pharm. 1999;56: American College of Clinical Pharmacy Clinical Practice Affairs Committee. Clinical pharmacy practice in noninstitutional settings. Pharmacother. 1992;12: Carter BL, Helling DK. Ambulatory care pharmacy services: has the agenda changed? Ann Pharmacother. 2000;34: Amsler MR, Murray MD, Tierney WM, Brewer N, Harris LE, Marrero DG, Weinberger M. Pharmaceutical care in chain pharmacies: beliefs and attitudes of pharmacists and patients. J Am Pharm Assoc. 2001;41: REFERENCES 17. Holland RW, Nimmo CM. Transitions, part 1: beyond pharmaceutical care. Am J Health-Syst Pharm. 1999;56: Survey of Pharmacy Law, NABP, Park Ridge, IL Nimmo CM, Holland RW. Transitions in pharmacy practice, part 2: who does what and why? Am J Health-Syst Pharm. 1999;56: American Council on Pharmaceutical Education. Continuing Education Manual: Continuing Pharmaceutical Education Provider Approval Program. 1 st ed. Chicago, Ill: American Council on Pharmaceutical Education; June Available at: Accessed on: March 1, Nona DA, Kenny R, Johnson DK. The effectiveness of continuing education as reflected in the literature of the health professions. Am J Pharm Educ. 1988:52: Davis D. Does CME Work? An analysis of the effect of educational activities on physician performance or health care outcomes. Intl J Psych Med. 1998;28: Holland RW, Nimmo CM. Transitions in pharmacy practice, part 3: effecting change the three-ring circus. Am J Health- Syst Pharm. 1999;56: Nimmo CM, Holland RW. Transitions in pharmacy practice, part 5: walking the tightrope of change. Am J Health-Syst Pharm. 2000;57: Scott VG, Amonkar MM, Madhavan SS. Pharmacists preferences for continuing education and certificate programs. Ann Pharmacother. 2001;35:

9 Appendix 1. Preprogram survey Lipid Management and Hypertension Services Pre-Program Survey Please rate your current (pre-course) knowledge of the following items on a scale from 1 to 5 with 1 = poor, 2 = fair, 3 = average, 4 = above average, and 5 =excellent: 1. Pharmacologic and nonpharmacologic treatment options for hyperlipidemia 2. Developing treatment plans for patients with hyperlipidemia 3. Physical assessment skills (i.e. blood pressure measurement) 4. Pharmacologic and nonpharmacologic treatment options hypertension 5. Developing treatment plans for patients with hypertension 6. Ability to develop policies and procedures for a clinical service that are in compliance with regulatory requirements 7. Operation and maintenance of the Cholestech LDX 8. Equipment, supplies and documentation requirements for patient care activities 9. Physical layout and workflow requirements 10. Reimbursement terminology and strategies 11. Development and role of a business plan 12. Marketing terminology and strategies 13. Which of the following clinical services are you currently providing? A. Hyperlipidemia B. Hypertension C. Both (hyperlipidemia and hypertension) D. Neither (you are not currently providing clinical services for either condition) 14. Are you providing any other types of clinical services? If so please describe: 15. Why did you decide to participate in this continuing education course? A. To learn how to develop a hyperlipidemia service B. To learn how to develop a hypertension service C. To learn how to develop a hyperlipidemia and hypertension service D. For information and continuing education credit E. Other (please describe): Please tell us a little bit about yourself 16. My gender is: Male Female 9

10 17. I have been a licensed pharmacist for years (fill in the blank with number of years). 18. My current primary practice setting can best be described as: A. Hospital B. Independent retail C. Corporate chain retail D. Other 19. I have the following degree(s): BS in Pharmacy Doctor of Pharmacy 20. My current job title is (fill in the blank). 21. For your current primary position, fill in the percentage of time per week on average that you spend in each of the five broad pharmacy practice functions: A. Prescription processing (%) B. Clinical activities (%) C. Management activities (%) D. Education activities (%) E. Research activities (%) Your responses will be analyzed in the aggregate and confidentiality will be strictly enforced. You will receive an identical survey in about 6 months time. We appreciate your response! Thank You! 10

11 Appendix 2. Postprogram Survey. The following questions were added to the pre-program survey to make the post-program survey. Lipid Management and Hypertension Services Post-Program Survey Cont. If you are currently providing clinical services, how did the Lipid Management and Hypertension Services program assist you in your practice? A. Gave me the skills needed to develop a new service B. Assisted me in the modification of existing services C. I have not yet made any changes to my practice Please provide a brief description of your circumstances: If your answer to question 20 was C, can you please tell us why? A. Lack of time B. Lack of interest C. Do not have the financial support D. Structural (i.e. logistics) E. Other (please describe): 11