Evaluation of a Clinical Medical Librarian Program

Evaluation of a Clinical Medical Librarian Program at the Yale Medical Library BY BETTE GREENBERG, Head Librarian, Reference SARA BATTISON, Senior Reference Librarian MADELEINE KOLISCH, Reference Librarian MARTHA LEREDU, Reference Librarian* Yale Medical Library Yale University New Haven, Connecticut ABSTRACT A clinical medical librarian (CML) program in a large university-based teaching hospital setting is viewed by the majority of clinicians as education-oriented, with slightly fewer clinicians viewing it as patient care oriented. The CML service has been utilized for research purposes only when it is clear that case-relevant information can evolve into research intended to have "clinical" impact. This study reports the results of a questionnaire circulated among clinicians receiving CML support by a large medical school library. Results indicate that (1) the CML effected a change in information-seeking behavior by the clinicians-they obtained information that would not have been available to them if the CML had not been present in patientmanagement conferences; (2) relevancy of information provided by the CML was judged by the clinicians to be very high; (3) the accuracy of the CML's search, coupled with the rapidity of delivery, was found to be highly satisfactory; (4) acceptance of the CML within the patient care setting was acknowledged by the majority of clinicians, who contact the CML in-hospital and overwhelmingly prefer to do so; and (5) there was no statistically significant variation in the manner in which different medical specialties use the services of a CML. These findings justify implementation of a clinical medical librarian program, on a modified basis, as an additional service to already existing reference services offered by a large medical school library. IN 1974 a detailed analysis of the first 2,500 MEDLINE search requests processed by the reference staff of the Yale Medical Library (YML) indicated that only 15% of MEDLINE searches were for patient care purposes while 85% of MEDLINE searches were for research or educational purposes. Based on the results of that study [1], it was determined that clinicians use the services of the library less than researchers, and *Presently employed as Medical Librarian, Veteran's Administration Hospital, Washington, D.C. thought was given to creating a new service that would meet the library needs of clinical users. A clinical medical librarian (CML) program, seen as the best extension service the library could provide in an effort to meet clinical information needs, was discussed and implemented in the spring of 1975. The objectives of the YML/ CML Program were: (1) To rapidly provide relevant documents from the literature to clinicians in order to enhance the quality of patient care, and to provide clinicians with a readily available educational resource for patient care purposes; (2) To influence the clinician's informationseeking behavior for case-relevant information; (3) To establish the medical librarian's role as a member of the patient care team. The unknown factors influencing this service were: (1) Could the library assimilate this service into its daily reference activities without the addition of new staff? (2) Would the program be successful in a modified version with the CML participating in-hospital only a few hours a week? (3) Could the reference librarian adapt to providing information within a nontraditional setting? (4) Would the program be accepted by the clinicians? Previous reports of CML programs have been given by Lamb [2, 3], Algermissen [4], Roach and Addington [5], Colaianni [6], Staudt, Halbrook, and Brodman [7], and Schnall and Wilson [8]. Although these programs were adapted to fit the needs of a particular institutional setting (with, for example, CMLs operating in a full-time capacity or on a modified basis), the results of these reports indicate a similarity of views. In order to have a successful program, the CML must be able to identify and accurately search the literature. The 319

GREENBERG AND OTHERS presence of a CML in a clinical setting enables the CML to obtain a more comprehensive view of clinical information needs and thus enhances the accuracy of the literature provided. Any assessment of relevance must approximate that of the clinicians. Delivery of information must be provided rapidly, preferably within the same day or, at the very least, within twenty-four hours. The information must be documented to anticipate future users' needs and in order to avoid duplication. DESCRIPrION OF THE YML/CML PROGRAM The four professional reference librarians were assigned as CMLs to the departments of pediatrics, psychiatry, internal medicine, and three surgical subspecialties (neurosurgery, orthopedic surgery, and urology) at the Yale-New Haven Hospital. The hospital does not have central library facilities to serve the clinical needs of its staff, but most departments have small collections of library materials. t more than three hours per week of attendance at patient-management conferences was allowed, with not more than four hours per week of in-library literature searching permitted. One part-time assistant, the only staff added for the purposes of the YML/CML Program, was employed for photocopying, pulling material from the shelves, and delivering the information to each of the four departments. Total staff-hours allotted for the CML Program were estimated at forty-three hours per week, or 1.2 FTE (.7 FTE professional plus.5 FTE nonprofessional). The average number of information requests processed per department per month for an eighteen-month period was 18.3. MEDLINE searches have been the primary method of processing information requests, with manual literature searches processed only 5% of the time. On the average, two to three articles have been photocopied for each literature search. Delivery of photocopied articles has been made usually within the same day. Files of photocopied articles and searches have been maintained by each CML. A departmental newsletter has been distributed regularly, announcing the searches that have been requested, and, in some cases, a list of newly received books. This newsletter has been very well received, disseminating information and generating requests for copies of key articles already on file. EVALUATION OF THE YML/CML PROGRAM Evaluation of the YML/CML Program was needed in order to justify the existence of this ex- 320 tension service, as well as to establish whether the program's original objectives had been met. Because the population of potential users per department was relatively large, it was decided that a controlled sample would be the best objective measure. An analysis of the CML service would be more accurate if clinicians surveyed had had an adequate trial; those clinicians who had not used the services of a CML, or who had only used these services occasionally, would probably decline to participate in an in-depth evaluation. Therefore, a preliminary questionnaire was designed in order: (1) to request the clinician to participate in a further, in-depth evaluation; (2) to determine the reason why some clinicians had not used the service; (3) to determine, if the service had been used, how many times it had been used; and (4) to advertise the service to clinicians who might be unaware of the YML/CML Program. The clinicians who agreed to participate in an in-depth evaluation became the controlled sample. Fifty-three percent of the clinicians who received the preliminary questionnaire indicated that they were willing to participate in a further evaluation. DESIGN OF THE FINAL QUESTIONNAIRE Considerable thought and time were devoted to designing the final questionnaire. We were most interested in developing a measure that would determine whether or not the original objectives had been met and that would indicate if these objectives had changed during the course of the program. In addition, we were interested in any qualitative or quantitative differences apparent in the way different medical specialties used the CML service. A questionnaire was designed to obtain a census of opinion as well as to analyze relationships between particular variables. "Closed" questions were thought to be more objective measures, lending themselves to more-accurate interpretation than "open-ended" questions, which would solicit a variety of responses difficult to analyze. The respondents were encouraged to explain any negative answers. Efforts were made to eliminate any bias, library jargon was deliberately avoided, and questions were kept as short and direct as possible. The need for accuracy, honesty, and confidentiality of response was stressed. DATA COLLECTION AND ANALYSIS The final questionnaire was sent to the ninetyeight clinicians who had indicated that they would be willing to participate in a further evaluation. (See Table 1.)

EVALUATION OF A CLINICAL MEDICAL LIBRARIAN PROGRAM TABLE 1 NUMBER OF RESPONDENTS PER DEPARTMENT AND PERCENTAGE OF RESPONSE RATE Department.of.of question- question- % of naire naire response recipients respondents Pediatrics 21 15 75 Psychiatry 23 23 100 Medicine 14 14 100 Surgery 40 21 53 Total 98 73 74 Questionnaires were numbered so that followup could be accomplished if necessary. The overall response rate from departments was 74%. The combined results from all four departments are listed in the Appendix. RESULTS OF SURVEY First Objective The most important aspect of the YML/CML Program, from the library's point of view, has been to help meet the clinician's need for information to enhance patient care. The results, however, indicate that the program is viewed by the majority of clinicians (overall mean* = 2.85) as education-oriented with only a slightly smaller amount (overall mean = 2.73) indicating that the service is viewed as supporting patient care. (See Table 2.) In answer to question II, 6, "Has the information provided by the CML directly altered or affected patient care?", respondents indicated that they regard CML support as moderate (overall mean = 2.64). In addition, the need for information for research purposes was seen as a byproduct of the CML Program (overall mean = 2.47). The relevancy of the information provided by the CML was evaluated by questions II, 4,9: That is, "Have the articles chosen for you by the CML satisfied your information needs?" and "Did the CML accurately pinpoint the subject in the literature?" These factors were considered important in determining whether or not the CML Program had been a success or a failure. The clinicians responded that the information provided was exceptionally relevant (overall mean 3.45). This = result, coupled with the response to the accuracy *Throughout this paper mean figures refer to a scale of 1-4. Department TABLE 2 UTILIZATION OF CML. INFORMATION Patient care Utilization Education Research Pediatrics 2.87* 2.73 2.40 Psychiatry 2.74 2.61 2.30 Medicine 2.86 3.29 2.57 Surgery 2.71 2.90 2.62 of the CML in searching the literature (overall mean = 3.48), indicates a high degree of satisfaction with the service. (See Table 3.) The evaluation also revealed an unanticipated educational benefit of the CML service. The response to question II, 2, "Has the CML service substantially improved your awareness of new developments or research in your field?", was significant (overall mean = 2.74) because this was not originally an objective of the CML Program. (See Table 4.) The personalized delivery of information was measured by questions II, 7, 8: "Has the CML service saved you time?" and TABLE 3 RELEVANCY OF INFORMATION To what extent To what extent have the articles did the CML chosen for you accurately Department by the CML pinpoint the satisfied your subject in the information needs? literature? Pediatrics 3.53 3.53 Psychiatry 3.39 3.52 Medicine 3.57 3.21 Surgery 3.57 3.57 TABLE 4 VALUE OF QUESTION-SPECIFIC INFORMATION IN RELATION TO HEIGHTENED AWARENESS OF DEVELOPMENTS IN CLINICIANS' SPECIALTY Has the CML service substantially improved your awareness of new developments or research in your field? Pediatrics 2.60 Psychiatry 2.96 Medicine 2.64 Surgery 2.67 321

GREENBERG AND OTHERS TABLE 5 DELIVERY OF INFORMATION To what extent has the CML saved you time? To what extent has the information delivery time been satisfactory? Pediatrics 3.80 3.93 Psychiatry 3.70 3.87 Medicine 3.57 3.79 Surgery 3.90 3.90 "Has the information delivery time been satisfactory?" The replies indicate that the YML/CML Program has more than achieved this aspect of the primary objective. Overall means to both these questions were 3.75 and 3.88, respectively. (See Table 5.) Second Objective One important ramification of the YML/CML Program has been the change in the clinician's information-seeking behavior as a result of the CML being present in patient-management conferences. In answer to question II, 1, "Has your pattern of seeking information changed since the CML joined your unit?", the responses were very positive (overall mean = 3.04). (See Table 6.) When asked by question II, 5, "Would you have found the information elsewhere if not provided by the CML?", the respondents indicated that there was some information the clinicians would not have obtained if the CML had not been present (overall mean = 2.41). The sharing of information is common among colleagues. Therefore, question, II, 3 "Have you shared with other members of your department information provided by the CML?", received a very positive response (overall mean = 3.29). This 322 TABLE 6 CHANGES IN INFORMATION-SEEKING BEHAVIOR Department To what extent Towhato extent would you have hasyour pattern of seeingorman information changed? found the information elsewhere if not provided by the CML? Pediatrics 2.93 2.33 Psychiatry 3.09 2.39 Medicine 2.71 2.57 Surgery 3.29 2.38 TABLE 7 INFORMATION EXCHANGE AMONG CLINICIANS, INDICATING RELIANCE ON THE LITERATURE To what extent have you shared with other members of your department information provided you by the CML? Pediatrics 3.20 Psychiatry 3.30 Medicine 3.36 Surgery 3.29 TABLE 8 PRIMARY SOURCES OF CLINICAL INFORMATION Department Col- Litera- CML Other leagues ture Pediatrics 60% 40% 0% 0% Psychiatry 39% 52% 9% 0% Medicine 29% 64% 7% 0% Surgery 24% 62% 14% 0% demonstrates that department members other than the original requester also benefited from a particular request for clinical information. (See Table 7.) This response is related to question II, 11, "What is your primary source of clinical information?" The results of this question indicate that the literature serves as the primary source of information for 47% of our respondents. (See Appendix, II, 11.) We feel that, whether the information is supplied by a colleague or by a CML, the information is more highly valued if substantiated by reference to the literature. (See Table 8.) Third Objective To what degree is the CML seen as a member of the patient care team and how well does the CML interact with the team members? With the exception of the issue of "acceptance," little has been written on the behavioral aspects of the CML role. As Lamb [2] has mentioned, winning the acceptance of the members of a unit can be the first obstacle, and possibly the most important one, to overcome. How well does the clinician think the CML has been integrated into a unit? It has been recognized by Kaluzny that innovation within the health care system is basically dependent upon a two-phase process: The first phase involves the decision by someone in authority to implement an innovation within the organization. The second phase involves acceptance or rejection

EVALUATION OF A CLINICAL MEDICAL LIBRARIAN PROGRAM TABLE 9 How WELL Do You THINK THE CML HAS BEEN INTEGRATED INTO YOUR UNIT?. responding %of total response tatall 0 0 Slightly 7 10 Mostly 36 49 Fully 30 41 Total 73 of the innovation by those within the organization who are affected [9]. The response to question III, 8, "How well do you think the CML has been integrated into your unit?", indicates that clinicians feel the CML has been mostly integrated (overall mean = 3.32). (See Table 9.) One of the most crucial factors to consider in justifying a CML service, especially one on a modified basis, is whether clinicians view the presence of a CML in-hospital as necessary. In answer to question III, 2, "How would you prefer to contact the CML?", approximately 52% preferred contact within the hospital, as opposed to any other routes of communication. This reinforces the presence of the CML in-hospital and corroborates the findings of Schnall and Wilson [8], who found that when the information need was viewed as "clinical," the presence of a CML was required in-hospital. (See Table 10.) The fact that 30% of the respondents preferred contact by telephone is indicative of actual interaction with clinicians during periods when a CML was not present in patient-management conferences. The value of the CML in-hospital is not, as yet, entirely correctly perceived by TABLE 10 PREFERENCE OF IN-LIBRARY VERSUS IN-HOSPITAL CONTACT WITH THE CML (How WOULD YOU PREFER TO CONTACT THE CML?). responding %of response In person at conferences 38 52 In person in the library 10 14 By phone 22 30 Through a third person 0 0 Other (no preference) 3 4 Total 73 clinicians. The value of awareness of the full case presentation cannot be disregarded. Seldom can a clinical problem be successfully negotiated by phone without extensive questioning by.the CML. A patient-management problem referred directly to a CML differs from other in-library inquiries by its very nature and content. We received an ambiguous response to question III, 3, "As a rule, when you request information from the CML have you done previous searching?" Overall, 58% responded that they do their own searching before asking the CML. This may mean that they consult colleagues, search personal or departmental reprint files or directly search the literature. However, although the CML is not always turned to first, an overwhelming 92% of the respondents replied "" when asked by question III, 4, "Do you go to the CML as a last resort?" (See Table 11.) One of the most critical indications of acceptance was measured by questions III, 5-7, "Do you feel the CML has adequate subject knowledge to function effectively?", "Have you sensed any communication barriers in dealing with the CML?", and "Does the presence of the CML in the clinical conference restrict or in any way alter normal proceedings?" On an interpersonal level, these issues of communication were thought to be significant variables. Ninety-five percent replied "" to the issue of "communication barriers." According to 96% of the total sample, the CML does not restrict or alter normal proceedings in the clinical conference. (See Table 12.) In fact, respondents commented that the patient-management conferences were enhanced by the presence of a CML. TABLE 1 I CLINICIANS' DEMAND FOR CML INFORMATION As a rule, when you request information Do yug from the to the Department CML, have last resort? you done previous searching? Pediatrics 47% 53% 7% 93% Psychiatry 43% 57% 9% 91% Medicine 79% 21% 21% 79% Surgery 71% 29% 0% 100% 323

TABLE 12 CML's ABILITY TO INTERACT INTERPERSONALLY AND INTELLECTUALLY IN HOSPITAL Question Have you sensed any communication barriers in dealing with the CML? 5% 95% Do you feel the CML has adequate subject knowledge to function effectively? 97% 3% Does the presence of the CML in the clinical conference restrict or in any way alter normal proceedings? 4% 96% GREENBERG AND OTHERS CONCLUS ION From the results of this evaluation, it appears that a clinical medical librarian program can be implemented on a modified basis as an additional service to already existing reference services. The CML Program at the Yale Medical Library, serving departments at Yale-New Haven Hospital, is viewed by the majority of clinicians as education-oriented, with slightly fewer clinicians responding that it has been patient-care oriented. However, the difference between the distribution of responses indicating educational purposes (overall mean = 2.85) and those indicating patient care purposes (overall mean = 2.75) is statistically insignificant. In a university-based teaching hospital, distinctions between patient care and education are not rigid, and material for education is seen as having an impact on patient care and vice versa. These results support our conclusion that the primary goal of enhancing patient care has been achieved by the YML/CML Program. The relevancy of information provided by the CMLs was judged by the clinicians to be very high, and there was a corresponding acknowledgment that there had been a change in, the clinicians' patterns of seeking information, as a result of the CML service. The issues of "acceptance" and how well the CML has been integrated into the units at Yale- New Haven Hospital are critical to the success and future of the service provided. We have found that the measure of acceptance of a CML program by the clinicians must be preceded by a decision by someone in authority to implement such an innovative service. Acceptance or rejection by those who actually utilize the service then follows. This evaluation of a CML service attests to the high level of utilization of a CML in a patient care 324 setting. The benefits of the service to the clinicians are multidimensional. The time-saving factor, one of the original objectives of the service, has been demonstrated by the fact that the clinicians' time has been less taxed as a result of obtaining information via the CML. There is also a hidden monetary savings because the physicians' time is spent more effectively when it is not necessary for them to obtain this information themselves. A review of the results revealed no statistically significant variation in the manner in which different medical specialties use the services of a CML. t all clinicians within the hospital community can be served, because of staff limitations, but to offer such a program, even on a limited basis, is better than not to offer it at all. The program helps to close the gap that exists between a research library and its users who have the need for support of clinical information. In addition, the YML/CML Program has had a beneficial effect on staff development; each clinical medical librarian has acquired further subject knowledge within her own speciality. It has enhanced the medical librarian's ability to communicate with users within the library. A factor which cannot be overemphasized is that the librarian's skills are being fully utilized. The clinical community of users has been made aware of the resources and services of the library. This reinforces the changing image of librarianship and the growing appreciation of the role of the librarian within a patient care setting. REFERENCES 1. GREENBERG, B.; BREEDLOVE, B.; and BERGER, W. MEDLINE demand profiles: an analysis of requests for clinical and research information. Bull. Med. Libr. Assoc. 65: 22-30, Jan. 1977. 2. LAMB, G. An innovative medical school: an innovative approach to health science information. Paper presented June 11-15, 1972, at the 71st Annual Meeting of the Medical Library Association, San Diego-Coronado, Calif. 3. LAMB, G. Bridging the information gap. Hosp. Libr. 1: 2-4, v. 15, 1976. 4. ALGERMISSEN, V. Biomedical librarians in a patient care setting at the University of Missouri-Kansas City School of Medicine. Bull. Med. Libr. Assoc. 62: 354-358, Oct. 1974. 5. ROACH, A. A., and ADDINGTON, W. W. The effects of an information specialist on patient care and medical education. J. Med. Educ. 50: 176-180, Feb. 1975. 6. COLAIANNI, L. A. Clinical medical librarians in a private teaching-hospital setting. Bull. Med. Libr. Assoc. 63: 410-411, Oct. 1975. 7. STAUDT, C.; HALBROOK, B.; and BRODMAN, E. A Bull. Med. Libr. Assoc. 66(3) July 1978

EVALUATION OF A CLINICAL MEDICAL LIBRARIAN PROGRAM clinical librarians' program: an attempt at evaluation. Bull. Med. Libr. Assoc. 64: 236-238, Apr. 1976. 8. SCHNALL, J. G., and WILSON, J. W. Evaluation of a clinical medical librarianship program at a university health sciences library. Bull. Med. Libr. Assoc. 64: 278-283, July 1976. 9. KALUZNY, A. D. Innovation in health services: theoretical framework and review of research. Health Serv. Res. 9: 101-120, Summer 1974. Received February 13, 1978; accepted March 31, 1978. II. APPENDIX CLINICAL MEDICAL LIBRARIAN (CML) SERVICE Analysis ofquestionnaire I. OBJECTIVES The Clinical Medical Librarian (CML) Service has as its goal to make information for patient care more immediately and readily available to the clinician. 1. How often have you used the information supplied by the CML directly for patient care? (1) Never (2) Occasionally (3) Usually (4) Always N* = 0(0%) N = 27 (37%) N = 34(47%) N = 12 (16%) Overall meant = 2.73 2. How often have you used the CML-supplied information for educational purposes? (1) Never (2) Occasionally (3) Usually (4) Always N = 0(0%) N = 28 (38%) N = 28 (38%) N = 17 (23%) Overall mean = 2.85 3. How often has the information provided by the CML led you to further research? (I) Never (2) Occasionally (3) Usually (4) Always N = 2(3%) N = 37(51%) N = 32 (44%) N = 2(3%) Overall mean = 2.47 4. If you have answered "Never" to any of the above, please explain: 2(3%) not currently in research. QUALITY AND EFFECTIVENESS 1. Has your pattern of seeking information changed since the CML joined your unit? N= 1(1%) N = 20(27%) N= 27(37%) N = 25(34%) 2. Has the CML service substantially improved your awareness of new developments or research in your field? N = 5(7%) N = 23(32%) N = 31(42%) N = 14(19%) Overall mean = 2.74 3. Have you shared with other nembers of your department information provided by the CML? N= 1(1%) N= 11(15%) N= 27(37%) N = 34(47%) Overall mean = 3.29 4. Have the articles chosen for you by the CML satisfied your information needs? N = 0(0%) N = 1(1%) N = 34(47%) N = 37(51%) Overall mean = 3.45 5. Would you have found the information elsewhere if not provided by the CML? N = 5(7%) N = 39(53%) N = 23(32%) N = 6(8%) Overall mean = 2.41 6. Has the information provided by the CML directly altered or affected patient care? N = 5(7%) N = 26(36%) N % 32(43%) N = 10(14%) Overall mean = 2.64 *N indicates the number of responses. tmean figures refer to a scale of 1-4. 325

GREENBERG AND OTHERS III. APPENDIX (Continued) 7. Has the CML service saved you time? N = 0(0%) N = 2(3%) N = 4(19%) N = 57(78%) Overall mean = 3.75 8. Has the information delivery time been satisfactory? N = 0(0%) N = 0(0%) N = 9(12%) N = 64(88%) Overall mean = 3.88 9. Did the CML accurately pinpoint the subject in the literature? N = 0(0%) N = 4(5%) N = 30(41%) N = 39(53%) Overall mean = 3.48 10. Will you turn to the CML for information in the future? N = 0(0%) N = I (1%) N = 19(26%) N = 53(73%) Overall mean = 3.71 11. What is your primary source of clinical information? (1) Colleagues (2) Theliterature (3) TheCML (4) Other N = 29 (40%) N = 34 (47 %) N = 8(11%) N = 2(3%) INTERACTION AND COMMUNICATION 1. How do you contact the CML? (Check as many as are appropriate) (1) In person at conferences (2) In person in the library (3) By phone (4) Through a third person N = 58 (79%) N = 37 (51 %) N = 43 (59%) N = 7 (10%) 2. How would you prefer to contact the CML? (Select only one) (1) In person at conferences (2) In person in the library N = 38 (52%) N = 10(14%) (3) By phone (4) Through a third person (5) Other N = 22 (30%) N = 0(0%) N = 3(4%) 3. As a rule, when you request information from the CML, have you done previous searching? N = 42(58%) N = 31(42%) 4. Do you go to the CML as a last resort? N = 6(8%) N = 67 (92%) 5. Do you feel the CML has adequate subject knowledge to function effectively? N = 71(97%) N = 2(3%) 6. Have you sensed any communication barriers in dealing with the CML? N = 4(5%) N = 69 (95%) 7. Does the presence of the CML in the clinical conference restrict or in any way alter normal proceedings? N = 3(4%) N = 70 (96%) 8. How well do you think the CML has been integrated into your unit? (1) t at all (2) Slightly (3) Mostly (4) Fully N = 0(0%) N=7(10%) N=36(49%) N=30(41%) Overall mean = 3.32 326