OFFICE OF THE AUDITOR

OFFICE OF THE AUDITOR DEPARTMENT OF ENVIRONMENTAL HEALTH PUBLIC HEALTH INSPECTION DIVISION FOOD SAFETY PROGRAM PERFORMANCE AUDIT SEPTEMBER 2008 Dennis J. Gallagher Auditor

Dennis J. Gallagher City and County of Denver 201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000, FAX 720-913-5247 www.denvergov.org/auditor September 18, 2008 Nancy Severson, Manager Department of Environmental Health City and County of Denver Dear Ms. Severson: Attached is the Auditor s Office Audit Services Division s performance audit report of the Public Health Inspection Division s Food Safety Program. The audit period was for January 1, 2007 through December 31, 2007. The purpose of the audit was to evaluate whether the Division inspected all licensed facilities in accordance with the Division s policies and procedures, performed inspections efficiently and effectively, enforced retail food safety regulations adequately, and to assess whether internal controls in place were adequate. The audit identified several areas where efficiencies could be improved to help the Division complete required inspection activities in a timely manner. For example, audit work identified manual practices that could be eliminated if the Division pursued technological inspection tools currently available. Technological tools could also help offset staffing levels that are lower than benchmarking jurisdictions, allowing Division inspectors to dedicate more time to food facilities. We commend the Department of Environmental Health and the Public Health Inspection Division for taking proactive steps to protect the citizens, workers and visitors to the City and County of Denver by enrolling in the Draft Voluntary Retail Food Regulatory Program Standards, a voluntary set of high benchmarking standards developed by the US Food and Drug Administration. If you have any questions, please call Kip Memmott, Director of Audit Services, at 720-913-5029. Sincerely, DJG/mm Dennis J. Gallagher Auditor To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation. 1

Nancy Severson September 18, 2008 Page Two cc: Honorable John Hickenlooper, Mayor Honorable Members of City Council Members of Audit Committee Ms. Kelly Brough, Chief of Staff Mr. Claude Pumilia, Chief Financial Officer Mr. Chris Henderson, Chief Operating Officer Mr. David Fine, City Attorney Ms. Lauri Dannemiller, City Council Executive Staff Director Ms. Beth Machann, Controller 2

TABLE OF CONTENTS Transmittal Letter 1 Table of Contents 3 Auditor s Report 4 Executive Summary 6 Background, Scope, Objective, and Methodology 7 Findings and Recommendations 11 Exhibit A Department of Environmental Health Response 22 3

Dennis J. Gallagher Auditor City and County of Denver 201 West Colfax Ave., Dept. 705 Denver, Colorado 80202 720-913-5000, FAX 720-913-5247 www.denvergov.org/auditor AUDITOR S REPORT We have completed an audit of the Public Health Inspection Division s Food Safety Program. The purpose of the audit was to evaluate whether the Division inspected all licensed facilities in accordance with the Division s policies and procedures, performed inspections efficiently and effectively, enforced retail food safety regulations adequately, and to assess whether internal controls in place were adequate. This audit was included in the Auditor s Office Audit Services Division s Annual Audit Plan and is authorized pursuant to the City and County of Denver Charter, Article V, Part 2, Section 1, General Powers and Duties of Auditor. We conducted our audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain reasonable assurance about internal controls and compliance with applicable regulations. An audit includes examining, on a test basis, evidence supporting the accuracy and thoroughness of the food facility inspection processes of the Public Health Inspection Division. The audit identified several areas where efficiencies could be improved to help the Division complete required inspection activities in a timely manner. For example, audit work identified manual practices that could be eliminated if the Division pursued technological inspection tools currently available. Audit work included an evaluation of two Division practices where no exceptions were noted. First, audit testing showed that the Division s use of civil penalties as an enforcement mechanism was consistent and that civil penalties were assessed according to the Division s Civil Penalty Assessment Guideline. Second, audit testing indicated that complaints of foodborne illness were consistently investigated and that the Division consistently ensured that required corrective action was taken. We commend the Department of Environmental Health and the Public Health Inspection Division for taking proactive steps to protect the citizens, workers and visitors to the City and County of Denver by enrolling in the Food and Drug Administration s Draft Voluntary National Retail Food Regulatory Program Standards, a voluntary set of high benchmarking standards developed by the US Food and Drug Administration. We extend our appreciation to the personnel who assisted and cooperated with us during the audit. Audit Services Division Kip R. Memmott, CGAP, CICA Director of Audit Services To promote open, accountable, efficient and effective government by performing impartial reviews and other audit services that provide objective and useful information to improve decision making by management and the people. We will monitor and report on recommendations and progress towards their implementation. 4

September 18, 2008 Page Two Staff: Dick Wibbens, CPA, Audit Manager Nancy Howe, Audit Supervisor Anita Thompson, CICA, Lead Auditor Jane Harlow, CICA, Senior Auditor Mary Mutchler, CICA, Staff Auditor 5

PUBLIC HEALTH INSPECTION DIVISION FOOD SAFETY PROGRAM EXECUTIVE SUMMARY FOR THE PERIOD JANUARY 1, 2007 THROUGH DECEMBER 31, 2007 This summary highlights the finding of our performance audit of the Public Health Inspection (PHI) Division s Food Safety Program which is more fully described in the Findings and Recommendations section beginning on page 11. The Department of Environmental Health s response to our findings is included in Exhibit A on page 22 of this report. 1. Opportunities Exist to Enhance Food Safety Inspection Operations For 2007, audit work determined that routine inspections were not consistently completed in accordance with each facility s risk level category assignment and were not always performed on a timely basis. Additionally, newly licensed food facilities were not always entered into the PHI database resulting in no inspections for these establishments for a period of time. Several factors hinder the Division s efforts in these areas. For example, many of the Division s current processes are paper-based and manual, increasing the Division s resource needs and the risk of mistakes and gaps. Further, audit research indicated that some Divisional policies could be unnecessarily limiting the time inspectors have available to complete routine inspections. Additionally, it appears that the Food Safety Program s staffing levels are below FDA recommendations and below levels reported by benchmarking jurisdictions. Timely inspections help decrease the risk that the public may not be adequately protected from foodborne illnesses. As a result of audit work, we make recommendations to provide more tools and flexibility to help the Public Health Inspection Division enhance operations and better meet its operational responsibilities. 6

PUBLIC HEALTH INSPECTION DIVISION FOOD SAFETY PROGRAM BACKGROUND, SCOPE, OBJECTIVE AND METHODOLOGY FOR THE PERIOD JANUARY 1, 2007 THROUGH DECEMBER 31, 2007 Background The ultimate goal of all retail food regulatory programs is to reduce or eliminate the occurrence of illnesses and deaths from food produced at the retail level. According to the United States Food and Drug Administration s (FDA) 2005 Food Code, An estimated 76 million illnesses, 325,000 hospitalizations, and 5,000 deaths are attributable to foodborne illness in the United States each year. For some consumers, foodborne illness results only in mild, temporary discomfort or lost time from work or other daily activity. For others.foodborne illness may have serious or long-term consequences, and most seriously, may be life threatening. 1 Denver s Public Health Inspection Division (PHI or the Division), under the direction of the Department of Environmental Health, is responsible for inspecting food establishments in Denver. The Division also inspects other regulated facilities such as swimming pools, child care facilities, emergency medical vehicles, and body art businesses. The purpose of this audit was to review the Division s Food Safety Program. The mission of the Public Health Inspection Division s Food Safety Program is to reduce the incidence of foodborne disease in Denver s food establishments. PHI enforces compliance with regulations to minimize the risk of communicable and infectious diseases by providing full (also known as routine) inspections, re-inspections, investigations of general and foodborne illness complaints, technical assistance, education and enforcement actions in regulated facilities. While performing inspections, inspectors look for critical and non-critical violations. Critical violations directly relate to foodborne illness, and include violations such as improper temperature controls or bare-hand contact with ready-to-eat foods. Non-critical violations are not as serious but if left uncorrected, can become a serious problem. Examples of non-critical violations include improperly labeled food or inadequate ventilation. If a critical violation is cited during an inspection which cannot be corrected while the inspector is onsite, the inspector must return to the facility to ensure compliance with food safety regulations. When re-inspections do not result in compliance, PHI also has several enforcement mechanisms to help ensure compliance with food safety regulations under the authority of Chapter 23 of the Denver Revised Municipal Code (DRMC). Prior to taking an actual enforcement action, PHI can request an Administrative Conference, which is a voluntary meeting to establish an understanding between the operator, inspector, supervisor, and/or director on the necessary actions needed to bring the facility into compliance. The following are specific enforcement actions the Division can execute for non-compliant establishments. Order Inspectors can order facilities to dispose of food that has unapproved ingredients or cross contamination or to dispose of unapproved chemicals. Inspectors can also order the operator to clean the facility or repair equipment. 1 U.S. Food and Drug Administration 2005 Food Code. Available at http://www.cfsan.fda.gov/~dms/fc05-int.html 7

BACKGROUND, SCOPE, OBJECTIVE, AND METHODOLOGY Summons The Division can summon owners to court for failure to comply with official orders. Civil Penalties The Division is authorized to levy civil penalties up to $2,000 based on the history of non-compliance by the facility. Closure The Division can issue an order to cease and desist. Grounds for closure due to imminent health risk may include, but are not limited to, no hot water, sewage problems, no utilities, fire, pest infestation, contaminated food, foodborne illness outbreak, extreme un-cleanliness, or a combination of the above. Revocation or Suspension The Manager of the Department of Environmental Health can recommend that the Department of Excise and Licenses revoke or suspend a business license for continually non-compliant establishments jeopardizing the public s health and safety. While the Division does not individually track each type of enforcement, auditors were able to verify all of the above enforcements were used to ensure compliance by using direct observation as well as a review of food facility files. PHI s Participation in the FDA s Draft Voluntary National Food Standards Program The FDA administers a voluntary initiative called the Draft Voluntary National Retail Food Regulatory Program Standards. 2 The Program Standards provide a framework for continuous improvement to help ensure the safety and security of the food supply at the retail level. Under this program, the FDA details criteria for jurisdictions to achieve best practices in food safety. The criteria specified are not minimum criteria, but rather represent a high benchmark to which a regulatory retail food program can aspire. Jurisdictions conduct a self-assessment of their program and subsequently obtain a FDA verification audit to confirm the accuracy of the selfreported status of standards. The Program Standards address the following nine program areas: 1. Regulatory Foundation 2. Trained Regulatory Staff 3. Inspection Program Based on HACCP Principles 4. Uniform Inspection Program 5. Foodborne Illness and Food Defense Preparedness and Response 6. Compliance and Enforcement 7. Industry and Community Relations 8. Program Support and Resources 9. Program Assessment 2 FDA National Retail Food Regulatory Program. Available at http://www.cfsan.fda.gov. 8

BACKGROUND, SCOPE, OBJECTIVE, AND METHODOLOGY As of November, 2007, the FDA website indicated that the Division had self-reported completion of Foodborne Illness and Food Defense Preparedness and Response (Standard Five). Additionally, FDA personnel have verified Denver s completion of Compliance and Enforcement (Standard Six) and Industry and Community Relations (Standard Seven). 3 Denver compares favorably with the national completion averages for this program. According to the FDA website, other jurisdictions self-reported completing an average of 1.3 standards and jurisdictions with FDA-verified compliance averaged.5 standards (i.e., many jurisdictions have not even completed the implementation of one standard). Of the 184 enrolled jurisdictions, only 12 have verifiably complied with more of these standards than the City and County of Denver. Audit Scope The audit examined the Division s Food Safety Program to ensure: that all licensed food establishments were inspected in accordance with the requirements outlined in the Policies and Procedures of the Public Health Inspection Division, inspections were performed efficiently and effectively, and that retail food safety regulations were adequately enforced. The audit period was January 1, 2007 through December 31, 2007. Audit Objective The objectives of this audit were: To determine if all licensed food facilities were consistently recorded on PHI s Master Establishment List; To determine if licensed food facilities were inspected according to their risk level; To determine if re-inspections were completed as ordered, and to evaluate their effectiveness; To evaluate the enforcement process including the process for investigating reports of foodborne illness; and To compare the Division s practices to comparable and best practice jurisdictions. Audit Methodology The steps taken to meet the objectives included, but were not limited to: Interviewing personnel at various City agencies to understand procedures; Shadowing staff on inspections of establishments to understand the inspection process; Running and reviewing reports of all establishments licensed in Denver; 3 In December 2007, the Division also completed self-reporting for Standard Three. However, as of July 2008 this was not reflected on the FDA website. 9

BACKGROUND, SCOPE, OBJECTIVE, AND METHODOLOGY Comparing PHI s Master Establishment List to the list of licensed facilities maintained by the Department of Excise and Licenses to ensure all licensed establishments are brought forward to PHI s inspection list; Testing to ensure establishments were inspected and re-inspected according to Division policy. Specifically, auditors tested a sample of 71 (2.2%) of 3,282 food establishments 4 to determine whether they were inspected the minimum number of times per year as required by their risk classification and within the required timeframe. The sample was chosen utilizing the Division s list of establishments and selecting every 45 th establishment. Auditors also tested a randomly chosen sample of 30 establishments to determine the timeliness of re-inspections and to ensure critical violations were corrected; Reviewing PHI s foodborne illness investigation practices; and Comparing the Division s program and practices to local and best practice jurisdictions. 5 4 The facilities tested did not include facilities classified as carts, ice cream vendors, mobile units, and facilities with a temporary license. 5 The five benchmarking jurisdictions were Boulder County CO, Tri-County CO (Adams, Arapahoe and Douglas Counties), Maricopa County AZ, Fairfax County VA, and the City of Plano TX. Boulder County and Tri-County were chosen because they are in Colorado, Maricopa County was chosen because it has met three FDA verified best practices standards, and Fairfax County and the City of Plano were chosen because each has met five FDA verified best practices standards (the highest for all local jurisdictions). 10

PUBLIC HEALTH INSPECTION DIVISION FOOD SAFETY PROGRAM FINDINGS AND RECOMMENDATIONS FOR THE PERIOD JANUARY 1, 2007 THROUGH DECEMBER 31, 2007 1. Opportunities Exist to Enhance Food Safety Inspection Operations Opportunities exist to strengthen the Environmental Health Department s Food Safety Program. For 2007, audit work determined that routine inspections were not consistently completed in accordance with each facility s risk level category assignment and were not always performed on a timely basis. Additionally, newly licensed food facilities were not always entered into the PHI database. As discussed beginning on page 13 of this report, several factors hinder the Department from addressing these issues. If inspections are not completed consistently and in a timely manner, the overall result is an increased risk that the public may not be adequately protected from foodborne illnesses. The following discusses specific issues identified by audit work as well as our recommendations for correcting such issues. Inspection Requirements Not Consistently Met Audit testing determined that in 2007 the minimum number of routine inspections was not consistently completed according to each facility s risk level and was not always performed on a timely basis. According to PHI policy, each food facility is assigned a risk level category of high, medium, or low at the initial inspection upon a menu review. The assigned risk level is based on factors directly contributing to foodborne illness found during epidemiological 6 investigations. Facilities categorized as high risk must have a routine inspection at least once every four months, medium risk at least once every six months, and low risk at least once every 12 months. 7 Audit work included testing 71 establishments to determine whether they were inspected the minimum number of times required by policy. Using the assigned risk levels for these 71 establishments, auditors determined that they required a total of 131 full inspections in 2007. Testing determined that 25 (19%) of the required 131 inspections were not completed for 22 (31%) of the 71 establishments. This corresponded to Division reports indicating that 19% of inspections were not completed for all establishments regulated by the Division in 2007. Figure 1 below illustrates the results of audit testing. It shows the percentage of inspections not completed and the percentage of establishments that did not get all required inspections, divided into high, medium and low risk categories. As the figure illustrates, the largest percentages are associated with establishments in the high risk category. Division personnel stated that they try to ensure all establishments are inspected at least once a year. Additionally, they include reinspections when evaluating inspection completion numbers, and that when re-inspections are included, all facilities are at least visited the minimum number of times per year according to their assigned risk category. 6 Epidemiology is the scientific and medical study of the causes and transmission of disease within a population or the origin and development characteristics of a specific disease. 7 Public Health Inspection Division Policies and Procedures Manual, Section 3.1.3 and 3.1.4 11

FINDINGS AND RECOMMENDATIONS This is supported by testing, where auditors found all low-risk establishments in the sample were inspected at least once. While this practice ensures a minimum amount of attention for every establishment, we believe it does not constitute compliance with a policy that requires a minimum number of routine inspections. Additionally, when the Division cannot complete all inspections required by policy, the high risk establishments, which require more full inspections, are more likely to be affected. Figure 1: Percent of Inspections and Establishments Not Completed (By Risk Level) Establishments Inspections 63.6% 32% 28.3% 0% 16.7% High Medium 31% 19.1% 0% Average Low 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% In addition, further testing on the sample of 71 establishments showed inspections were not consistently completed in a timely manner. The PHI Policies and Procedures Manual requires that inspections shall be performed within two weeks of the due date of inspection. 8 However, audit testing determined that in 2007, 66% of the high risk, 67% of the medium risk, and 22% of the low risk establishments were not completed within two weeks of the inspection due date. Newly Licensed Food Facilities Not Consistently Entered into the PHI Database In addition to issues identified with inspection frequency and timeliness, audit testing determined that the PHI database did not capture all licensed retail food facilities within the City and County. Specifically, in the cases where inspections were required, 86 (2.6%) of the 3,312 licensed retail food facilities were not in the PHI database. 9 While this represents a small percentage of establishments, the fact that they were not in the database means that they were not inspected thereby increasing the risk of a foodborne illness. 8 Public Health Inspection Division Policies and Procedures, Section 3.1.4 9 There are legitimate reasons why a business may receive a license but not require an inspection. For example, a facility may be inspected by the United States Department of Agriculture (USDA), a facility may only sell prepackaged, non-potentially hazardous food or drinks, or a facility may receive a temporary license for a non-profit charitable event which does not require inspection. Taken from DRMC Chapter 23, Section 1-103. 12

FINDINGS AND RECOMMENDATIONS Several Factors Hinder the Department from Meeting Inspection Requirements As discussed below several factors impede PHI s ability to complete the inspections and reinspections required by policy, to complete them within the timeframe specified by Division policies, and to ensure all food establishments are on their listing of facilities subject to inspection. The Division s Current Inspection Process is a Manual, Paper-Based System The Division relies on a manual-based system and has not utilized some technological tools that are available to increase the efficiency of the inspection process. Audit work identified several process inefficiencies associated with the manual system used by the Division: o Inspectors spend a large amount of time recording inspections and reinspections As part of audit work, auditors accompanied inspectors during full inspections and re-inspections. During these shifts, auditors observed that inspections typically took between one and one-half hours to three hours to complete depending on the complexity of the facility. Inspectors spent approximately, one-third to one-half of this time completing the inspection form upon completion of the inspection. One PHI inspector noted that re-inspections should take approximately 15 minutes. However, one of the re-inspections observed by auditors lasted 60 minutes, with 30 minutes spent completing the inspection form. The inspection forms are not completed during inspections because they need to be filled out legibly and have to stay clean, otherwise, the scanner will not be able to read them. Also, inspectors enter thorough comments and notes regarding each inspection to aid with re-inspections or enforcement actions. Auditors also accompanied inspectors from Jefferson County, who look for the same violations as the Division s inspectors. During these shifts, inspectors completed full inspections in less than one hour and re-inspections in less than 10 minutes. These inspectors used handheld devices to record inspection results 10 and reported that the devices saved them a lot of time during inspections. Using a paper-based system results in a disproportionate reliance on each inspector s individual skills such as their memory, organizational and note-taking skills as opposed to entering the inspection notes directly into the handheld device as the inspection takes place. When auditors accompanied PHI inspectors, the inspectors either used a note pad or relied on memory to track the violations noted 10 Auditors accompanied inspectors from Jefferson County to learn more about what improvements could be expected from a handheld inspection device. These handheld devices were small, user-friendly and contained all necessary information for inspections. For example, if a re-inspection was needed, the device would prompt the inspector to go on the date specified at the original inspection. During the re-inspection, the device listed all of the previously cited critical violations and notes so the inspector could easily check them off if the facility had corrected the problem. An additional attribute of these devices is that if an inspector is unavailable, another inspector can easily fill in without having to obtain specific handwritten notes from the unavailable inspector. 13

FINDINGS AND RECOMMENDATIONS during the inspection. At the end of the inspection, each violation and notation has to be transcribed onto the inspection form. The forms were filled out at the conclusion of the inspections due to their cumbersome nature and the necessity of keeping the forms legible and clean. In addition, when re-inspections were necessary, each PHI inspector had to track which violations needed to be resolved and when they should return to the facility to perform the re-inspection due to the process timeliness limitations discussed below. Audit testing indicated that four (4.6%) of 87 critical violations were not corrected by the end of 2007. However, on three of the four uncorrected violations, the form included a note that all violations were corrected, but only three of the ten individual violations were not marked as corrected on the form. In this situation the inspector may not have specifically looked for the other violations or may have observed that the violations were corrected but did not accurately record this information. The Division s reliance on individual inspectors notes presents an opportunity for more violations to fall through the cracks. o Entering inspection data into the database is time-consuming and costly The process of manually tracking all time and inspection results is briefly outlined in Figure 2. Figure 2: Inspection Documentation Process As illustrated in the figure, PHI personnel spend two to four weeks processing the inspection forms. This includes completing, reviewing and scanning the inspection form, verifying the scanned data, and filing the form. 14

FINDINGS AND RECOMMENDATIONS Although the database contains each violation cited and tracks when reinspections should occur, the reports produced using the data are often obsolete by the time they are available. More specifically, a re-inspection typically occurs within two weeks of the original inspection. Therefore, if a violation or reinspection appears in the database three or four weeks after the violation was originally cited, then the re-inspection should have already occurred by the time the report is produced. This delay hinders the Division s efficiency, reduces their ability to utilize the information in the database, and limits the effectiveness of reports which could be used to assist PHI in carrying out its responsibilities. PHI s Methodology for Setting a Facility s Risk Level Lacks Flexibility The Division s current policy for setting a facility s risk level is based solely on the level of food handling. This policy reduces PHI s flexibility to safely adjust the number of inspections required in times of need such as during busy seasons, occurrences of large community events or to address limited resource issues. Some other jurisdictions have implemented options for establishing risk and have adjusted their inspection frequencies accordingly. For example, Fairfax County, VA will allow a high risk establishment to be inspected only twice per year if the prior year s history includes less than five critical violations. 11 Three of the other benchmark cities also use risk analyses beyond the level of food handling to determine if all establishments will be inspected more than twice per year. 12 The Division could utilize additional levels of risk to increase or reduce the number of inspections an establishment needs. The US Food and Drug Administration s (FDA) Food Code requires that regulatory agencies inspect a food establishment at least once every six months. The State of Colorado has adopted an inspection frequency requirement for retail food establishments that is consistent with the FDA s Food Code. Federal and state regulations serve as minimum requirements, so the City can adopt a more stringent inspection frequency requirement. However, because the City s inspection frequency requirements exceed the FDA s, we believe the Division could safely reduce the number of inspections it requires at certain times as a way to add flexibility to its inspection requirements. Adjusting inspection frequency based on an additional risk evaluation could provide establishments with incentive to adopt practices to reduce their inspection frequency. Conversely, more frequent inspections could serve as a type of penalty for establishments with numerous or uncorrected critical violations as well as provide a way to monitor such establishments. PHI s Methodology for Setting Inspection Due Dates Contributes to Missed Inspections Under the Division s current policy, an inspection due date is calculated based on the completion date of the previous inspection. For example, because PHI s policy requires that high risk establishments be inspected at least once every four months, when an inspection of a high risk establishment is completed, the next due date is four 11 Fairfax County has achieved FDA Standard 3 which addresses risk assignments and inspection frequency. 12 Maricopa County, AZ, Tri-County, CO, and the City of Plano, TX. 15

FINDINGS AND RECOMMENDATIONS months from the date of the completed inspection. When routine inspections are not timely, this scheduling methodology causes subsequent due dates to get pushed back, often into the next year. This can result in establishments getting fewer inspections each year than is required under PHI policy. If this situation happens often enough, or if the inspection is performed late enough, then a high risk establishment could be consistently under-inspected. For example, audit work identified several high risk establishments that were consistently inspected only twice per year as opposed to the three inspections (i.e., one every four months) required. Table 1 shows how untimely inspections combined with the Division s scheduling methodology can influence the number of times per year a facility is inspected. Table 1: Inspection Delay Can Reduce Inspection Completion Facility 1 st Inspection Delayed 2 nd Inspection Delayed 3 rd Inspection Due Date 1 32 days 13 days Pushed into the next year 2 63 days 53 days Pushed into the next year 3 96 days 60 days Pushed into the next year Total Time Added 45 days 116 days 156 days Audit testing demonstrated that of the 14 high risk establishments that were not inspected three times in 2007, nine could be attributed to the fact that the third inspection came due in 2008 as a result of delays in the first and second inspections. In addition to creating shortfalls, the fluctuating due date may also create an unnecessary increase in inspection frequency. For example, one high risk establishment was repeatedly inspected earlier than the target date resulting in a fourth inspection being completed during the year. PHI s Re-inspection Policy Requires Significant Resources PHI s policies and procedures require that inspectors follow up on all critical violations. According to PHI records, there were 5,672 routine inspections and 2,540 re-inspections completed in 2007. Testing showed that for high risk establishments, which typically need more reinspections, a re-inspection was needed for 57% of all establishments with critical violations noted. Of these, violations at 40% of the establishments were corrected by the first re-inspection. However, additional re-inspections (third or fourth visits to the facility) were needed for 17% of the establishments. This is a significant resource commitment that takes inspectors time and takes from the time they have available to complete all their responsibilities, including conducting routine full inspections. Benchmarking work indicated that Fairfax County sends letters to establishments with critical violations. They conduct follow-up inspections if needed, but reported that they rarely have repeat offenders. Additionally, the State of Colorado allows inspectors to document violations on a form called the Critical Item Violation Correction Sheet. A responsible party at the establishment must complete and return the form, certifying that the violations were corrected. This type of flexible re-inspection approach can free 16

FINDINGS AND RECOMMENDATIONS inspector time by reducing the need to physically return to the establishment, while attempting to hold the establishment responsible for correcting critical violations deemed not to be an imminent hazard to public health. Lack of Coordinated Efforts Between PHI and the Department of Excise and Licenses Contributes to Missed Inspections In addition to manual processes, audit work found that limited communication and coordination between the City s Department of Excise and Licenses and PHI contributed to food facilities not being entered into the PHI database. Consequently, these establishments are not placed on inspectors inspection lists and therefore, unless detected later, may never be inspected. The licensing process established by the Department of Excise and Licenses requires that food facilities be inspected before a license is issued. Involving PHI in the licensing process is a control for the City to ensure that all food facilities are inspected at the time of licensure. After the initial inspection for licensure, PHI manually delivers this inspection paperwork to administrative personnel, who enter the data into the PHI database to ensure routine inspections are done in the future. However, the manual nature of the Division s process limits its effectiveness. This is supported by audit testing which identified 86 (2.6%) of 3,312 licensed food facilities that were not entered into the PHI database. This resulted in these facilities going without inspection for an average of 2.5 years. Reconciling the data between PHI and Excise and Licenses is a manual, time-consuming process and audit work determined that periodic reconciling was not completed by PHI personnel to ensure that all licensed establishments were included in the PHI database. A contributing factor that adds to the difficulty of performing reconciliations is that the two Departments use different identifiers for food establishments. For example, Excise and Licenses tracks facilities based on their business license (or file) number while PHI creates a unique establishment ID for their facilities. However, staff at the City s Technology Services Department created a program for audit staff that queried the Excise and Licenses database and provided a list of newly licensed facilities that can be used to confirm whether these new facilities were captured in PHI s database. PHI could reduce the amount of time needed to perform reconciliations by utilizing this program. PHI s Staffing Level is Lower Than Other Jurisdictions and FDA Recommendations The Division s staffing level appears to be contributing to the difficulty in completing the required inspections. Audit analysis of PHI s staffing and work load measures indicate that even when fully staffed, the Division s staffing level is lower than the jurisdictions surveyed and below ratios recommended by the FDA s Draft Voluntary National Retail Food Regulatory Program Standards. More specifically, the Division s ratio of facilities per inspector is higher than all the benchmark jurisdictions surveyed. Table 2 shows PHI s staffing ratio compared to these jurisdictions. 17

FINDINGS AND RECOMMENDATIONS Table 2: Comparison of Establishments per Inspector Ratio Jurisdiction Number of Establishments Number of Inspectors (FTE s) Ratio of Establishments per Inspector Tri-County, CO 4,172 23.5 178:1 Maricopa County, AZ 18,150 87.5 207:1 City of Plano, TX 1,450 6.5 223:1 Fairfax County, VA 3,200 14 229:1 Boulder County, CO 1,420 4.5 316:1 Average 5,678 27.2 231:1 Denver, CO (PHI) 3,861 10.5 368:1 As the table shows, the average ratio for the benchmark jurisdictions is almost 231 facilities per inspector. At 368 facilities per inspector, PHI s ratio is 60% higher than the average. PHI would need to have 16.7 inspectors to reach the average level. Further, four of the five jurisdictions reported that they had completed 100% of the required inspections for 2007. The only jurisdiction reporting that they were unable to complete all inspections was Boulder County, the jurisdiction with the next highest ratio, who reported that 40% of their required inspections in 2007 were not completed. During the audit period, the Division s staffing capacity was 10.5 full-time equivalent inspectors. These inspectors were required to complete an estimated 7,028 routine or full inspections, not including temporary licensed facility inspections. This corresponds to approximately 669 full inspections for each inspector. According to the FDA s best practice criteria, one full-time equivalent inspector should be devoted to 280-320 inspections each per year. 13 This means that PHI s food inspectors are responsible for more than double the number of inspections than recommended by the FDA. It should be noted that the FDA s Program Standards represent a high benchmark as only 7 of 184 jurisdictions that have completed the FDA s self-assessment have reported meeting this standard, and none have been verified by the FDA. The FDA further recommends that each establishment be allocated eight to ten hours of attention per year for activities such as inspections, re-inspections, risk assessment reviews, and onsite training demonstrations. 14 Each PHI inspector has approximately 1,456 hours available 15 for fieldwork each year. With 1,456 hours available, each inspector can devote eight hours per year to only 182 establishments. Because each PHI inspector is responsible for about 368 establishments, they currently have time to devote only four hours to each establishment per year. 16 13 FDA National Retail Food Regulatory Program Standard 8. Available at http://www.cfsan.fda.gov. 14 FDA National Retail Food Regulatory Program Standard 8. Available at http://www.cfsan.fda.gov. 15 The Division expects each inspector to spend 70% of their time in the field. This allows for sick, vacation and holiday leave as well as administrative tasks and mandatory training. 52 weeks per year x 40 hours per week x 70% = 1,456 hours available. 16 1,456 available field hours divided by 368 establishments equals 3.96 hours per establishment. 18

FINDINGS AND RECOMMENDATIONS Audit work found that the number of retail food facilities in Denver increased about 10% between 2005 and 2007. 17 As Denver has grown, there has also been an increase in large venues and temporary events. During this time the number of food safety inspectors employed by PHI has remained constant. Additionally, the City s practice of routinely seeking vacancy savings through staff openings results in fewer than 10.5 inspectors at various times. Identified Weaknesses Result in Inefficiency and Increased Public Health Risks Beyond the value of the physical inspection of food facilities, the presence of inspectors also provides an opportunity for them to communicate with, establish relationships with, and educate owners and operators of food establishments. PHI s inability to conduct all required inspections not only causes some food facilities going without detailed, regular full inspections, but can also result in an environment where inspectors face significant time pressures to perform inspections, possibly resulting in increased errors, especially when a large number of re-inspections are needed. In these instances inspectors have less time to ensure their routine and re-inspections are complete and thorough. Although no evidence was found during audit work, the manual processes discussed previously including the need to transcribe notes, combined with the limited time to conduct a large number of inspections increases the risk of inspectors limiting their documentation due to the time needed to record violations and the necessity of conducting an onsite re-inspection. If inspections are not completed and violations are not corrected, the overall result is the increased risk that the public may not be adequately protected from foodborne illness. Technological Tools May Help Increase Efficiency - The Division s current efforts to obtain additional technological tools for inspections should result in several efficiencies. 18 Specifically, less time should be needed to perform full inspections and re-inspections, allowing inspectors to complete more of them, spend more time at the facilities, and provide more educational assistance to operators. Several of the benchmark entities already employ such technology. For example, three of the five 19 benchmarking jurisdictions reported using either laptop computers or handheld/palm devices to record inspections and the other two jurisdictions will be using electronics before the end of 2008. Personnel from the Jefferson County Department of Health and Environment stated that having inspection data go straight into the database (without having to scan the inspection reports) has saved them approximately 3-5 minutes in administrative time per inspection. With over 7000 inspections in 2007, replacing this piece of the inspection process could potentially save the Division about 350-580 labor hours per year. The use of technology may also provide some relief to supervisors and management at PHI, who often perform inspections to help ease the inspectors workload. In addition, it should reduce the amount of paper that must be routed for approval, scanning and filing, and the time it takes to get inspection data into the database. When the data is entered into the database more quickly, 17 According to PHI, there were 4250 facilities in 2005, 4550 facilities in 2006, and 4677 facilities in 2007. 18 In 2008 the PHI Division requested almost $29,000 in Seed Capital funds from the City for handheld computer devices and printers for field inspectors. In the application, PHI stated that they would realize about.75 FTE savings equal to $28,355 in annual salary. Additionally, the accompanying reduction in labor needs would allow them to devote.75 FTE to duties other than filing, copying, faxing and scanning. 19 Fairfax VA, Plano TX, and Maricopa County AZ are currently using electronic devices. 19

FINDINGS AND RECOMMENDATIONS Division staff will be able to use it when conducting re-inspections, eliminating the need to rely on their own hand-written notes. Recommendations To provide more tools and flexibility to help the Public Health Inspection Division strengthen operations, meet its operational responsibilities, and enhance its ability to protect the public, we recommend that the Division: 1. Continue to pursue budget funding to obtain more technological resources such as handheld electronic inspection recording devices to help inspectors complete all the full inspections and re-inspections required by policy. 2. Develop another level of risk assessment based on the performance of each food establishment as a way to add flexibility to the number of inspections required and encourage establishments to adopt stringent food safety practices. We recommend that this risk assignment be in addition to the original high, medium or low risk level currently assigned to each establishment. 3. Adjust the inspection scheduling methodology set forth in the Division s policies for inspections that are not completed when due. When inspections are delayed, the next full inspection should remain on or close to the establishment s originally scheduled timeframe. To keep a reasonable amount of time between inspections, the Division should consider requiring a minimum amount of time, such as 30 days, between inspections. Additionally, when determining the number of inspections completed, management should include only full inspections to obtain an accurate and consistent measure of performance. 4. Consider developing additional options for following up on critical violations that could reduce the need for physical re-inspections. 5. Work with the City s Excise and Licenses and Technology Services Departments to develop a consistent method of identifying new establishments in order to ensure all newly licensed facilities are captured on the PHI inspection list. The Division should regularly reconcile information on new facilities with information in the PHI database. In addition, PHI may want to consider using a more comprehensive identifier, such as the facility s license number, to more easily complete reconciliations. 6. Consider pursuing additional inspector FTEs to help complete all required duties. However, in light of the cost of implementing computerized inspection tools combined with the Division s limited budget, we believe the Division should first pursue the recommendations above. The Division could then evaluate their ability to complete all required inspections and re-inspections, and if necessary, pursue funding for additional staff based on a staff needs analysis. 20

EXHIBIT A DEPARTMENT OF ENVIRONMENTAL HEALTH RESPONSE PHI FOOD SAFETY 21

EXHIBIT A DEPARTMENT OF ENVIRONMENTAL HEALTH RESPONSE PHI - FOOD SAFETY 22