BY CONTINUOUS COMPUTER MONITORING

A new method of evaluating a family planning program using a computerized approach is described. The precoded record form devised for this purpose is discussed and some early experiences presented. A STATEWIDE FAMILY PLANNING PROGRAM: EVALUATION BY CONTINUOUS COMPUTER MONITORING Gary Richard Snyder, M.D.; Charles J. Bures; J. King B.E. Seegar, Jr., M.D.; and John L. Pitts, M.D., F.A.P.H.A. Introduction SINCE 1965, the local health departments in Maryland have offered family planning services in all of their 23 counties and Baltimore City. Exclusive of Baltimore City, the program has grown from serving 3,065 patients in fiscal year 1965 to serving 6,445 patients in fiscal year 1968. Many problems arose when we attempted to evaluate this program. After careful analysis, it was felt that most of these problems came from utilizing the "classic parameters" of study: that is, new, carry-over, and revisit patients; number of patients given oral contraceptives; number of intrauterine devices inserted, and so on. We felt that a much truer picture would be obtained if, in addition to the above, the number of patients actively participating in the program could be defined and determined at any time. Thus, the number and the rate of change of the number of patients actively participating in the program would serve as a gross index of (1) how active a given program was in recruiting new patients, (2) how well the service was continued by the patients, (3) how effective was the program's follow-up effort. Thus, the need was shown for a new system of obtaining this information that would be simple, labor saving, accurate, and which would make the data available quickly. Methods Accordingly, the medical and statistical record forms then in use were analyzed.* Items of information that others'-4 felt important were studied. We then decided to discard the previously used medical and statistical records and design a new record form.* The new form is precoded and meets the need for medical and statistical information. Only items that were of operational importance, and produced facts on which the program administrators could make informed and appropriate decisions about the program, were included. The new form has two parts, for initial visit and for revisit records. Both are precoded. The initial-visit record consists of three sheets of paper. Because it was felt necessary to follow an individual patient *These forms are available upon request from the author. AUGUST. 1969 1317

continuously, the initial record is prenumbered to give each patient her own number. On the front page is the patient's appropriate identifying legend along with the social, demographic, and medical data necessary for compilation of family planning statistics. The second sheet is a carbon copy of the first. On the back of the second sheet is an outline for the medical history. Sheet 3 has an outline for a complete gynecological physical examination and appropriate laboratory reports. On the back of this is a place to summarize Papanicolaou smears and to begin progress notes. An additional page for supplementary notes is available. The original or top page of this record is filled in, separated, and sent to the central office for processing; the carbon copy (sheet 2) is left at the clinic. The other sheet is also retained by the clinic as the beginning of the patient's medical record. After processing, the computer completes on the revisit form all the information obtained on the initial record. These revisit records are then sent weekly to the originating clinic, where they are filed with the patient's record. On the back of the revisit record are precoded reasons why a patient might terminate family planning services. When the patient returns to the clinic, the information on the form is verified. If at this later date the information stated is still correct, nothing need be done. If, however, any item of information is not correct or has changed, the appropiate response is circled. The record also separates, with one copy being retained by the clinic and the other forwarded to the central office. The system then repeats itself. In addition, at the bottom of the revisit record the computer records in English, for the four most recent visits, any change of contraceptive method, change in the type of IUD used, any complication or treatment, and a record of delinquency. On both the records, an area outlined in heavy lines is completed by the physician and the rest of the record by an interviewer. After consultation with Dr. J. Howard Beard, health officer of Anne Arundel County, we decided to pretest this system in that county, which has clinics in 13 locations dispensing family planning assistance and advice at weekly to monthly intervals, and serves over 1,200 patients annually. Its indigent population in need of family planning is estimated to be 5,360 women. The staff was oriented to the new computerized records, and operation began as of April 1, 1968. The pretest, lasting three months, was successful and pointed up only minor changes that needed to be made in the records and the system. By July 1, 1968, the nursing staff of each county in Maryland had been oriented, and operation began in every county but one as of that date. Definitions To be able to determine the number of patients actively participating in the program, the following new definitions were devised: Active Patient A patient is assumed to be actively participating in the program from the date she is seen in the clinic until the date of her next appointment to see a physician at the clinic. Delinquent Patient A patient becomes delinquent if she has not presented herself as of the date of her next appointment. A patient using oral contraceptives or cycled to receive an IUD is given 30 days of delinquency. A patient using any other method is given 90 days. Inactive Patient After a patient has completed her delinquency period and still has not returned to the clinic, she is classified as inactive for another 90 days. Terminated Patient After the 90-day period of inactivity has heen completed, the patient is classified as terminated. A patient may be terminated at any time by the clinic. New Patient This denotes that an individual is given 1318 VOL. 59, NO. 8. A.J.P.H.

EVALUATING FAMILY PLANNING PROGRAMS family planning service for the first time in the current fiscal year and did not receive family planning services in the preceding fiscal year. Carry-Over Patient This means that an individual was given family planning services in the preceding fiscal year and has returned for the first time in the current fiscal year. Revisit Patient The term designates an individual being given family planning services in the current fiscal year who has already received services in the current fiscal year. Computer System The computer program is designed to maintain a continuously updated master file for each patient. Upon receipt in the central office, the records are checked clerically for completeness. Weekly, all the records received within the preceding week are punched onto IBM cards and verified. Every item contained on the initial record is punched. On the revisit record, only the items in which a change has been noted are punched. The data are then checked for completeness and validity in a card to tape edit. About 2.5 per cent of the cards punched and verified are either incomplete or invalid. These errors are corrected and reentered into the system immediately. The data then undergo a two-part updating procedure. Again, invalid information is detected. So far this has amounted to 0.1 per cent of the cards entered. These errors are corrected and reentered in the next weekly updating. At this point, an updated master file on tape for each atient has been created. The appropriate report tapes are generated from the updated master file, viz., new revisit records to be printed, and monthly, quarterly, and yearly reports. The entire cost of developing and operating this system through September 1, 1968, was approximately $8,000, plus about 70 per cent of the time of an Epidemic Intelligence Service Officer. A more detailed breakdown will be found in Table 1. Output At present, each clinic is presented with a monthly report consisting of the following: 1. An alphabetical roster listing all patients on each clinic's rolls. Here will appear the patient's name, family planning number, address, telephone number, date of her next appointment, and a status code conforming to the definitions above. 2. An alphabetical listing of all delinquent patients. The identifying information above is also contained here. 3. An alphabetical listing of all inactive and terminated patients, including the same identifying information. 4. A summary table showing, for the current month and for the year to date, the number of new patients, carry-over patients, revisit patients, and how many patient visits this represents. 5. A summary table showing how many patients are active, delinquent, inactive, and terminated as of the date of the report. 6. A return rate showing the percentage of patients that were delinquent, inactive, or terminated and became active again. 7. For patients who return, an average interval of return will be calculated. In the near future, the monthly report will also contain the following: (a) An alphabetical listing of patients sched- Table 1-Estimated cost of Family Planning Project: totals Machine time through August, 1968 $1,659.00 Analyst effort through September, 1968 (360 hours @ $8 per hour) 2,880.00 Programming effort through August 31, 1968 (actual $ paid) 1,737.50 Keypunch-Keyverify through August, 1968 (@ $5 per hour) 500.00 Forms 1,108.60 Postage-Approximately 50.00 EIS Officer TOTAL $7,935.10 AUGUST. 1969 1319

uled to return to the clinic for a medical examination in the current month. (b) An alphabetical listing of patients on oral contraceptives due to return to the clinic for a supply visit. An individual patient will appear on this list every 90 days. (c) An alphabetical listing of patients who have not had a Papanicolaou smear in the preceding year. A quarterly report is distributed containing detailed cross-tabulations of the activity measures, viz., active, delinquent, inactive, and terminated-by age, race, religion, education, marital status, economic basis for service, number of times pregnant, number of living children, number of live births under 5 pounds 8 ounces, number of fetal deaths, interval since last pregnancy terminated, how many more children does the patient want, main source of referral, time taken to get to clinic, and method of contraception. Complication and treatment are cross-tabulated by method. Cytology, reason for missing previous appointment, patient's main reason for visit, whether the patient used the method satisfactorily, and nonmedical reason for method change or discontinuance are tabulated. Soon this report will also contain a tabulation of reasons for termination, and a modified multiple decrement life table calculated for all methods of contraception. Results The most gratifying results have been in the testimony of the personnel in the field using this system. The amount of time required to fill in an individual record for a new patient has been cut from 15 minutes with the old system to about 7 minutes with this new system. Another benefit has been the promptness with which the statistical information is now available. Because the system has been in operation in Anne Arundel County for only six months, and the rest of the counties for three months, no relationship between a patient's active participation in the program and the various parameters obtained has been ascertained. We anticipate that characteristics of those patients that tend to drop out of the program and those that tend to continue will be defined. All of the tabulations should be more meaningful as woman-years of experience accrue. Summary Because of the importance of the Family Planning Program in the clinics conducted by the Health Department in Maryland, and the many administrative problems that arose in attempts to evaluate this program, a new method of evaluation was devised in 1967. The greatest need was to develop a means of evaluation which would identify individual women served by the program as active, delinquent, inactive, or terminated patients. Medical and statistical records previously used were discarded and a new record form was designed. This precoded form meets the need for computerized statistical information as well as medical information. Precoding permits much more rapid recording of statistical and medical information by the personnel utilizing the form. The information on each patient is continuously updated, so that current information about the program is available. It is felt that this system is a valuable tool in program evaluation, a way to make more efficient use of personnel time, and a method to augment follow-up of patients utilizing family planning services. REFERENCES 1. Chandrasekaran, C., and Freymann, M. "Evaluating Community Family Planning Programs." In: Public Health and Population Change. Edited by Sheps, M. C., and Ridley, J. C. University of Pittsburgh Press, 1965. 2. Kantner, John F., and Stephan, Frederick F. Evaluation of Program Objectives in Family Planning. World Population Conference, 1 320 VOL. 59, NO. 8. A.J.P.H.

EVALUATING FAMILY PLANNING PROGRAMS Vol. II, Fertility, Family Planning, Mor- A Guide for State and Local Agencies. tality. United Nations, 1965, p. 278. Newv York: American Public Health Asso- 3. Mack, Newell B. Needed: Standardized ciation, 1968. Data for Action Programs. Studies in Family Planning. The Population Council, ACKNOWLEDGMENT - The assistance of the No 12, 1963. Division of Nursing of the Maryland State 4. Wishik, Samuel. "Evaluation of Family Department of Health in this endeavor is Planning Programs." In: Family Planning: deeply appreciated. Dr. Snyder is an Epidemic Intelligence Service Officer in Family Planning Evaluation, Epidemiology Program, National Communicable Disease Center, Health Services and Mental Health Administration, PHS, Department of Health, Education, and Welfare, Atlanta, Ga. He is assigned to Johns Hopkins School of Hygiene and Public Health, Department of Population and Family Health (615 North Wolfe Street), Baltimore, Md. 21205. Mr. Bures is Systems Analyst, Division of Data Processing, Dr. Seegar is Chief, Maternity and Family Planning Section, Division of Maternal and Child Health, and Dr. Pitts is Chief, Division of Maternal and Child Health in the Maryland State Department of Health. This paper was presented before a Joint Session of the Health Officers, Maternal and Child Health, and Public Health Nursing Sections of the American Public Health Association at the Ninety-Sixth Annual Meeting in Detroit, Mich., November 13, 1968. Attention Social Workers in Public Health Join your colleagues and become a member of the Conference of Social Workers in APHA. In these days, when social concerns in health programs are increasingly demanding the attention of persons in the health field, we need to build our membership in order to have an important voice in the Association's present or proposed programs. The Conference has been organized to: * Provide a structure which will permit social workers who are members of APHA to get together to present and discuss social issues, trends and problems in, and related to health and medical care, and to take any action it may desire. * Provide a structure which represents social work in public health to which officers, staffs, sections, or committees of the APHA may relate. * Provide Social Work leadership in relating and promoting social work thinking and participation in APHA. How to Join If you are a member of APHA send $2 annual dues payable to C.SW-APHA to Catherine M. Casey, Children's Bureau, SRS, 26 Federal Plaza, New York, N. Y. 10007 If you are not a member of APHA we urge you to join today, and then join the Conference of Social Workers in APHA. AUGUST, 1969 1321

This article has been cited by: 1. D L Kruegel. 1973. Numerator analysis of fertility and family planning in Maryland. American Journal of Public Health 63:6, 509-516. [Citation] [PDF] [PDF Plus]