IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-35230) ON A LEARNING AND INNOVATION CREDIT

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No:ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-35230) Human Development Sector Unit South Caucasus Country Department Europe and Central Asia Region ON A LEARNING AND INNOVATION CREDIT IN THE AMOUNT OF SDR 4.0 MILLION (US$ 5.0 MILLION EQUIVALENT) TO THE REPUBLIC OF AZERBAIJAN FOR A HEALTH REFORM PROJECT MARCH 30,

2 CURRENCY EQUIVALENTS ( Exchange Rate Effective 03/22/2007 ) Currency Unit = New Azeri Manat New Azeri Manat 1.00 = US$ 1, US$ 1.00 = New Azeri Manat Fiscal Year January 1 - December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy MED Ministry of Economic Development CDH Central District Hospital MIS Management Information System CEM Country Economic Memorandum MOF Ministry of Finance CIS Commonwealth of Independent States MOH Ministry of Health CPS Country Partnership Strategy MTEF Medium Term Expenditure Framework DP District Polyclinic MTR Mid Term Review ECA Europe and Central Asia ORS Oral Rehydration Salt FAP Feldsher Ambulatory Point PAD Project Appraisal Document GDP Gross Domestic Product PCU Project Coordination Unit GOA Government of Azerbaijan PIU Project Implementation Unit HPPU Health Policy & Planning Unit RDF Revolving Drug Fund HR LIL Health Reform Learning and State Program on Poverty Reduction and SPPRED Innovation Loan Sustainable Development HSRP Health Sector Reform Project SUB Village Site Hospitals IBTA-II Second Institutional Building Technical Assistance Project SVA Village Doctor Ambulatories IDA International Development Association UNICEF United Nations Chidren's Fund IMC International Medical Corps USAID United States Agency for International Development WHO World Health Organization Vice President: Shigeo Katsu Country Director: D-M Dowsett-Coirolo Sector Manager: Armin H. Fidler Project Team Leader: Enis Baris 2

3 Azerbaijan Health Reform LIL CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners...34 Annex 1. Project Costs and Financing...35 Annex 2. Outputs by Component...36 Annex 3. Economic and Financial Analysis (including assumptions in the analysis)...39 Annex 4. Bank Lending and Implementation Support/Supervision Processes...40 Annex 5. Beneficiary Survey Results (if any)...42 Annex 6. Stakeholder Workshop Report and Results (if any)...43 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR...45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders...56 Annex 9. List of Supporting Documents...58 Annex 10. Additional Annexes Other Outcomes and Impacts List of Individuals Interviewed

4 A. Basic Information Country: Azerbaijan Project Name: Health Reform LIL Project ID: P L/C/TF Number(s): IDA ICR Date: 03/27/2007 ICR Type: Core ICR Lending Instrument: LIL Borrower: Original Total Commitment: Environmental Category: C Implementing Agencies: Ministry of Health Cofinanciers and Other External Partners: B. Key Dates GOVERNMENT OF AZERBAIJAN XDR 4.0M Disbursed Amount: XDR 3.5M Process Date Process Original Date Revised / Actual Date(s) Concept Review: 10/25/2000 Effectiveness: 10/12/ /12/2001 Appraisal: 04/17/2001 Restructuring(s): Approval: 06/12/2001 Mid-term Review: 09/15/2003 Closing: 12/31/ /30/2006 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Satisfactory Moderate Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately Satisfactory Quality of Supervision: Satisfactory Implementing Agency/Agencies: Moderately Satisfactory Overall Bank Performance: Satisfactory Overall Borrower Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project No at any time (Yes/No): Quality at Entry (QEA): None Rating i

5 Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes Satisfactory Quality of Supervision (QSA): None D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration Compulsory health finance 7 7 Health Theme Code (Primary/Secondary) Health system performance Primary Primary Injuries and non-communicable diseases Secondary Secondary Other communicable diseases Secondary Secondary Rural services and infrastructure Secondary Secondary E. Bank Staff Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn Country Director: D-M Dowsett-Coirolo Judy M. O'Connor Sector Manager: Armin H. Fidler Armin H. Fidler Project Team Leader: Enis Baris Michael Mills ICR Team Leader: ICR Primary Author: Enis Baris Panagiota Panopoulou F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project's overall development objective was to implement alternative approaches to strengthen and reform district primary health care services. The project also had two component specific outcomes: (i) increasing the knowledge and capacity among MOH officials to design and implement appropriate primary health care reforms, and (ii) providing improved primary health care services, and increasing utilization of primary health care services in facilities in targeted districts. Key indicators The HR LIL had as key indicators the following: ii

6 Development by the government of a mid-term strategy for health reform, with World Bank assistance; Development and use by the district health authorities of district-specific annual work plans; Development by the government of a policy paper on improving health services and access for the poor; Improvements in access, quality, and utilization of primary health care services in the targeted districts; Increased knowledge among the staff in the MOH and the targeted districts about strategies for strengthening and reforming PHC services. In addition, the HR LIL had indicators related to component outcomes. The indicators of increased knowledge and capacity in the MOH were: Regular meetings of MOH and inter-ministerial colleagues to discuss PHC reforms; Development of a model and plan for PHC reforms for the country; MOH officials trained to become familiar with health financing options and issues; Adoption of a national essential drug policy and formulary for use at facilities supported by the MOH; Hardware for management information system installed and key MOH and targeted district staff trained to use it. The indicators of strengthened primary health care services in targeted districts were: An increase of 40 percent in the number of patients seen at reformed PHC facilities; An increase of 20 percent in the proportion on infants in the population that receive immunization (DPT3) on time; An increase of 30 percent in the proportion of pregnant women in the population who have at least six prenatal visits; An increase of 50 percent in the proportion of adult patients seen in the reformed PHC facilities for whom a blood pressure is recorded in the patients' medical records; A decrease of 50 percent in the proportion of patients seen in the reformed PHC facilities who receive antibiotics by means of injection; A decrease of 25 percent in the per capita number of hospital and polyclinic beds; An increase of 20 percent in patient satisfaction with access and quality of care provided in the reformed PHC facilities. Revised Project Development Objectives (as approved by original approving authority) The project development objectives and key indicators did not change during the project life. (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Development by the government of a mid-term strategy for health reform, with World Bank assistance. iii

7 Value quantitative or Qualitative) No mid-term strategy for health reform. An explicit health reform strategy issued by the government. A concept paper on 'Health Care Reform in the Republic of Azerbaijan' issued by the MOH. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 2 : Development and use by the district health authorities of district-specific annual work plans (rationalization plans). Value quantitative or Qualitative) No rationalization plans. Plans have been completed and fully implemented in project districts. Plans have been completed, but only partially implemented in project districts. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 3 : Development by the government of a policy paper on improving health services and access for the poor. Value quantitative or Qualitative) A policy paper on improving health No policy paper on services and improving health services access for the poor and access for the poor. issued by the government. The SPPRED is developed by the government with emphasis on health sector issues. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 4 : Improvements in access, quality, and utilization of primary health care services in targeted districts. Value quantitative or Qualitative) Low level of access, quality and utilization of PHC services in targeted districts. Improved access and quality and increased utilization of PHC services in targeted districts. Improved access and quality and increased utilization of PHC services in targeted districts as demonstrated through the baseline and evaluation surveys. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % See Intermediate Outcome Indicators achievement) Indicator 5 : Increased knowledge among the staff in the MOH and the targeted districts about iv

8 Value quantitative or Qualitative) strategies for strengthening and reforming PHC services. Limited knowledge and understanding of staff in the MOH and targeted districts. Increased knowledge among the staff in the MOH and targeted districts about strategies for strengthening and reforming PHC services. MOH staff participate in national and international training events. 3,000 district staff are trained locally in 23 different subjects. Post training tests show increased knowledge of district staff. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Value (quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Development of a model and plan for PHC reforms for the country. No model and plan for PHC reforms for the country. A model and plan for PHC reforms issued by the government. Actual Value Achieved at Completion or Target Years No model and plan for PHC reforms issued by the government. But preparation and approval of a health sector reform project including a component on strengthening service delivery. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 2 : Regular meetings of MOH and inter-ministerial colleagues to discuss PHC reforms. Value (quantitative or Qualitative) No regular meetings of MOH and interministerial colleagues. Regular meetings of MOH and interministerial colleagues. No regular meetings of MOH and interministerial colleagues PHC reforms. But great v

9 involvement of MOF, MED and Pension Fund in the preparation of the second health project. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 3 : MOH officials trained to become familiar with health financing options and issues. Value (quantitative or Qualitative) MOH officials have no training on health care financing options and issues. MOH officials receive training on health care financing options and issues. MOH officials receive training at local and international events. Results of health care financing studies and Health Sector Note presented to MOH officials. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 4 : Adoption of a national essential drug policy and formulary for use at facilities supported by MOH. Value (quantitative or Qualitative) Development of No essential drug list, no essential drug list, national drug formulary national drug and no standard treatment formulary and protocols. standard treatment protocols. Essential drug list, national drug formulary and standard treatment protocols developed and adopted. Date achieved 10/12/ /30/ /30/2006 Comments (incl. % achievement) Indicator 5 : Hardware for management information system installed and key MOH and targeted district staff trained to use it. Value (quantitative or Qualitative) Insufficient hardware for management information system and district staff not trained. Hardware for management information system installed and key MOH and targeted district staff trained to use it. Date achieved 10/12/ /30/2005 vi

10 Comments (incl. % achievement) Indicator 6 : Value (quantitative or Qualitative) This indicator cannot be assessed. The MOH cancelled this sub-component financed the necessary equipment on its own. An increase of 40% in the number of patients seen at reformed PHC facilities. Average no. of patients per doctor Average no. of patients per month. FAP: per doctor per month. 40% increase. 141 (7.6%). SVA FAP: 131. SVA (29.6%). SUB: SUB: 158. DP: (10.5%). DP: 533 (6.6%). Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 7 : An increase of 20% in the proportion of infants in the population that receive immunization (DPT3). Value (quantitative or Qualitative) Polio: 89. DPT3: 87. Measles: 87. BCG: 85. Hepatitis B: % increase of baseline value. Polio: 17.8%. DPT3: 17.4%. Measles: 17.4%. BCG: 17%. Hepatitis B: 17.4%. Polio: 96 (7.9%). DPT3: 96 (10.3%). Measles: 94 (8.0%). BCG: 95 (11.8%). Hepatitis B: 89 (2.3%). Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 8 : An increase of 30% in the proportion of pregnant women in the population who have at least six prenatal visits. Value (quantitative or Qualitative) 30% increase of baseline value. Percentage of women Percentage of with no prenatal visits: women with no 41%. No. of average prenatal visits: prenatal visits per pregant 12.3%. No. of woman: 2.4. average prenatal visits: Percentage of women with no prenatal visits: 15% (63% decrease). No. of average prenatal visits per pregant woman: 3.9 (62% increase). Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 9 : An increase of 50% in the proportion of adult patients seen in the reformed PHC facilities for whom a blood pressure is recorded in the patients' medical records. Value (quantitative or Qualitative) No recording of blood 50% increase of pressure in rural facilities. baseline value. Recording of blood pressure in district polyclinics, but no recording in rural vii

11 facilities. Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 10 : A decrease of 50% in the proportion of patients seen in the reformed PHC facilities who receive antibiotics by means of injection. Value (quantitative or Qualitative) Injection of prescribed antibiotics (% to oral+injection). SVA: 64%. SUB: 30%. DP: 10%. Total: 34.7%. 50% decrease of baseline value. SVA: 32%. SUB: 15%. DP: 5%. Total: 17.4%. Injection of prescribed antibiotics (% to oral+injection). SVA: 55% (14% decrease). SUB: 15% (50% decrease). DP: 10% (no change). Total: 26.7% (23.1% decrease). Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 11 : A decrease of 25% in the per capita number of hospital and polyclinic beds. Value (quantitative or Qualitative) No. of hospitals beds per 1,000 population in 25% decrease. intervention districts: 8.4. No. of hospitals beds per 1,000 population in intervention districts: 8.4. No change. Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) Indicator 12 : An increase of 20% in patient satisfaction with access and quality of care provided in the reformed PHC facilities. Value (quantitative or Qualitative) 20% increase of Patient satisfaction in baseline value: rural PHC facilities: 21%. 4.2%. Patient satisfaction in rural PHC facilities: 71% (144% increase). Date achieved 12/31/ /30/ /31/2004 Comments (incl. % achievement) viii

12 G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 06/28/2001 Satisfactory Satisfactory /13/2001 Satisfactory Satisfactory /15/2002 Satisfactory Unsatisfactory /08/2002 Satisfactory Satisfactory /06/2002 Satisfactory Satisfactory /16/2003 Satisfactory Satisfactory /20/2003 Satisfactory Satisfactory /22/2004 Satisfactory Satisfactory /20/2004 Satisfactory Satisfactory /04/2005 Satisfactory Satisfactory /27/2005 Satisfactory Satisfactory /29/2006 Satisfactory Satisfactory 4.97 H. Restructuring (if any) Not Applicable I. Disbursement Profile ix

13 1. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative) 1.1 Context at Appraisal (brief summary of country macroeconomic and structural/sector background, rationale for Bank assistance) Country macroeconomic background During the early 1990s, Azerbaijan experienced a sharp output decline; however, starting 1997, there was a resumption of growth mostly attributed to investment activity in oil and oil related sectors. Although the overall picture in the late 1990s was one of a macroeconomic success story in a transition country, a deeper analysis revealed that economic progress was unequal and that major social problems persisted. Following a tight fiscal policy since early 1995, public expenditure declined by about one third in revenues in relation to GDP as compared to Fiscal adjustment hit harder public investment and the non-wage component of current expenditures in social sectors. At the same time, in the second half of the 1990s, there was a substantial concern that poverty might not have decreased, even though, at that time, there was no data to provide a definitive judgment. Nevertheless, it was clear that inequality in the distribution of expenditure had increased and that strengthening and reform of social services were urgently needed. Structural/sector background Following independence in 1991, Azerbaijan's health status indicators worsened considerably. Even during the Soviet period, health outcomes in Azerbaijan were worse than those in Western European countries. However, starting in mid-1980s the health status of the population began to stagnate as it did for all former Soviet republics, and by the early 1990s, it deteriorated sharply. Despite this, since 1995, many national health status statistics (i.e., infant mortality, maternal mortality, life expectancy, etc.) reported improvements and by 2000 health outcomes had reached their pre-independence levels. Contrary to official statistics, the evidence from social assessments and household surveys in the early 2000s suggested that much of the apparent improvement was because of under-reporting, as an increasing number of households did not use public facilities from which official statistics were generated. Moreover, there were substantial regional variations in reported health status and worse health outcomes in rural areas as compared to urban ones. The collapse of public finances after independence resulted in a drastic decrease of public resources for the health sector. In real terms, health expenditure in Azerbaijan dropped from US$148 per person in 1991 to US$18 in Public expenditure on health amounted to US$4.41 per capita in 1997, representing no more than 25 percent of per capita total expenditure on health. The reduced budget resulted in drug shortages, dilapidated buildings, obsolete equipment, lack of food and heating in health facilities, low wage levels, and late payment of salaries. The insufficient wage levels and lack of internal controls led to an increase in informal payments. Lack of public financing was coupled with inefficiencies in resource use. The health sector was characterized by overprovision of hospital beds and medical staff. Hospitals were financed on the basis of health personnel and number of beds (historical budgets) and thus had no incentives to reduce either of the two. The average length of stay was long (17.5 days in 2000) while the average bed occupancy rate was less than 30 percent. There was high reliance on specialist doctors at the primary care level. Family medicine did not exist, nor did an effective referral system. The competence level of medical staff was low and the majority of treatment 5

14 protocols were outdated. Lack of public resources and inefficiencies in resource use led to reduction in access to and quality of health services. There were various reasons for this: the high informal cost of care, the lack of essential drugs and equipment, and the poor quality of facilities. Consequently, half of the population reported not seeking care from the formal health system even when acutely ill. Results from household surveys indicated that in mid-1990s private expenditure amounted to four times the level of public expenditure on health. To address the serious problems facing the health sector, in 1998 the government established a Health Reform Commission to develop a reform strategy and in 1999 a document entitled 'General Concept of Health Care Reorientation' was produced. The strategy envisioned, among other things, setting primary care as a priority, reforming the sanitary and epidemiological services, introducing principles of insurance, reforming the pharmaceutical sector, privatizing public facilities, introducing fee-for-service, creating a system of licensing, certification, and accreditation, and reforming the health information system. The government's strategy appeared to be focusing on cost recovery for medical services, albeit with fee exemptions for selected social groups. The Ministry of Health (MOH) also developed draft documents on mandatory health insurance, and in 1999 a Law on Health Insurance was adopted by the Parliament. Nonetheless, at the time, there were serious doubts by the Bank team about the fiscal and institutional capacity to introduce a mandatory health payroll tax given the already heavy tax burden on enterprises and the financial position of the public pension system. Country Assistance Strategy The objective of the FY00-02 Country Assistance Strategy (CAS) was "to persuade and work with authorities and civil society to help create the appropriate institutional and policy framework which would steer the country to the path of good governance and equitable development." The CAS set three program targets: (i) to support radical reform of public sector institutions, (ii) to strengthen the regulatory and business environment for private sector development, and (iii) to invest in social development including poverty alleviation measures. In the health sector, the CAS had the objective of addressing the inefficient health financing system and over-reliance on specialist facilities and physicians. Rationale for Bank Assistance The Health Reform Project was a Learning and Innovation Credit, henceforth HR LIL. The HR LIL was the first Word Bank health operation in Azerbaijan and it aimed at addressing two of the three program targets of the CAS: contributing to poverty alleviation and supporting public sector reform. The focus of the project was on strengthening the MOH to design and carry out primary health care (PHC) reforms through: (i) building up institutional capacity and training officials working on health sector reform policies, and (ii) enhancing critical PHC services and assisting the most vulnerable in five targeted districts. The project built on the ongoing, at the time, and largely successful pilot program of UNICEF in Guba. The project represented an initial effort on part of the government in the process of health reform, and was an important instrument in measuring the impact of the strategies adopted, ascertaining whether and why these strategies were successful or not, refining the direction of reform efforts, and building support and momentum for the expansion of the health reform program. The project was also an opportunity for the Bank to work with the Government of Azerbaijan (GOA) for the first time and hence to be able to explore ways of providing support to a new client both at national and district level, and to ensure complementarity between the policy dialogue on health reform and the Poverty Reduction Strategy agenda. 6

15 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) Project Development Objective The project's overall development objective was to implement alternative approaches to strengthen and reform district primary health care services. The project also had two componentspecific outcomes: (i) to increase the knowledge and capacity among MOH officials to design and implement appropriate primary health care reforms, and (ii) to provide improved primary health care services, and increase utilization of primary health care services in facilities in targeted districts. Key indicators The HR LIL had as key indicators the following: Development by the government of a mid-term strategy for health reform, with World Bank assistance; Development and use by the district health authorities of district-specific annual work plans; Development by the government of a policy paper on improving health services and access for the poor; Improvements in access, quality, and utilization of primary health care services in the targeted districts; Increased knowledge among the staff in the MOH and the targeted districts about strategies for strengthening and reforming PHC services. In addition, the HR LIL had indicators related to component outputs/outcomes. The indicators of increased knowledge and capacity in the MOH were: Regular meetings of MOH and inter-ministerial colleagues to discuss PHC reforms; Development of a model and plan for PHC reforms for the country; MOH officials trained to become familiar with health financing options and issues; Adoption of a national essential drug policy and formulary for use at facilities supported by the MOH; Hardware for management information system installed and key MOH and targeted district staff trained to use it. The indicators of strengthened primary health care services in targeted districts were: An increase of 40 percent in the number of patients seen at reformed PHC facilities; An increase of 20 percent in the proportion on infants in the population that receive immunization (DPT3) on time; An increase of 30 percent in the proportion of pregnant women in the population who have at least six prenatal visits; An increase of 50 percent in the proportion of adult patients seen in the reformed PHC facilities for whom a blood pressure is recorded in the patients' medical records; A decrease of 50 percent in the proportion of patients seen in the reformed PHC facilities who receive antibiotics by means of injection; A decrease of 25 percent in the per capita number of hospital and polyclinic beds; 7

16 An increase of 20 percent in patient satisfaction with access and quality of care provided in the reformed PHC facilities. 1.3 Revised PDO and Key Indicators (as approved by original approving authority), and reasons/justification The project development objectives and key indicators did not change during the project life. 1.4 Main Beneficiaries, original and revised (briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) The largest component of the project (Component 2. District Level Primary Health Care Reform) focused on improving the health, nutrition and population outcomes in targeted districts in Azerbaijan by improving access to and quality of health care services. The project aimed at establishing and equipping up to 16 newly developed outpatient clinics (each clinic serving approximately 8,000-10,000 population) in each of the five districts. The clinics addressed the leading causes of morbidity and mortality in the country by providing PHC services, including: maternity care; preventive and curative care of newborn, children, and infants; reproductive health; prevention and management of STIs/HIV/AIDS; prevention and curative care for leading infectious diseases e.g., TB, malaria, diarrhea, and acute respiratory diseases; prevention and management of cardiovascular diseases; and, prevention of iodine deficiency. The project aimed at improving the health status of poor populations by emphasizing provision of services outside the capital city and focusing on districts that are primarily rural and/or geographically isolated (i.e., Nakhchivan). The districts were also selected on the basis of qualitative and quantitative indicators that included proxy indicators for poverty. The project also improved quality of health care services and management of the health care system by providing approximately 1,300 hours of training to district health authorities on management and clinical topics. Furthermore, all physicians in the participating districts underwent approximately 1,200 of clinical in-service training on PHC topics and leading causes of mortality and morbidity in the country. Finally, the project financed a variety of national and international training events and observational tours to other transitional countries for government officials. 1.5 Original Components (as approved) The HR LIL project had three components: Component 1. Capacity Building for Health Policy Reform (estimated total cost US$0.87 million). The primary objectives of this component were to: complement and support the districtlevel component (Component 2), and to explore ways to stimulate a national dialogue on policy changes necessary for health reform in Azerbaijan; generate support in the government for the reform process itself; introduce a common understanding of terms and objectives; perform critical assessments related to health financing reform; and strengthen the capacity in the MOH to plan, implement, and evaluate health reforms. Sub-component 1.1. Analysis and Planning Development (estimated total cost US$0.13 million). This sub-component aimed at supporting the development of appropriate models for PHC, and envisaged a variety of national and international training events and observational 8

17 tours for government officials to other transition countries where more efficient and cost effective health care delivery models had been implemented. Sub-component 1.2. Health Financing Reform (estimated total cost US$0.34 million). This subcomponent was designed to provide training, survey work, and technical assistance support in three main areas: financial planning of the health system (including analysis of user fees and equity, estimation of the content and cost of a basic health care package, and analysis of health care financing options); the feasibility and possible development of health insurance; and analysis of private health spending. It also included staff training. Sub-component 1.3. Pharmaceutical Policy Development (estimated total cost US$0.09 million). This sub-component aimed at supporting the development and implementation of policy reforms in the form of appropriate treatment protocols, essential drug policy, national drug formulary, promotion of rational drug use, and decentralized decision-making. This sub-component also aimed at supporting limited pharmaceutical sector studies, focusing on lessons learned from the evaluation of the UNICEF-supported pilots of revolving drug funds. Sub-component 1.4. Management Information System Development (estimated total cost US$0.31 million). This sub-component aimed at supporting the strengthening of the health information system to assist in the management of the reformed health services, as well as to enhance monitoring, evaluation, supervision and staff development. Component 2: District Level Primary Health Care Reform (estimated total cost US$3.63 million). This component was designed to support actual implementation and extension of district-level PHC reforms that have been previously piloted by UNICEF. This component would support reforms in targeted districts to enhance and implement the UNICEF-supported program already piloted elsewhere in the country; rationalize PHC services; improve the quality of and access to PHC services; and strengthen the management and clinical capabilities of district health personnel. This component envisaged providing material support and staff training to five districts (Xacmaz, Samkir, Salyan, Goycay, and Sarur) with five other districts (Gusar, Gazakh, Sabirabad, Kurdamir and Babek) acting as "control" districts. In these control districts, data would be collected on key project indicators. The implementation of the component was to be done in five stages: 1. Rationalization of health care services was expected to be undertaken, with substantial community involvement, to plan the PHC reform for each district, reduce the number of hospital beds, and rationalize medical staff. 2. The implementation of the PHC models was to be supported by field monitors and clinical facilitators. They were expected to provide on-site technical assistance and evaluation to doctors and nurses in the primary care clinics, as well as facilitate and monitor the uptake of new diagnostic and treatment methods and guidelines. Community involvement, through the creation of district and community steering committees, was to ensure that community needs were met. 3. Civil works were to be carried out to refurbish approximately 16 primary health care facilities (including a Central District PHC Center) in each district to improve access to PHC services. This aimed at making the health services more attractive and conducive to better quality care. A small amount of money was to be provided to each district to make basic repairs in the health centers in accordance with the rationalization plan. The provision of good quality primary health care services also required the availability of basic medical equipment, supplies, and cold chain equipment to support primary care. Each participating district was to be provided with an initial quantity of essential drugs 9

18 valued at about $0.50 per capita to establish a Revolving Drug Fund (RDF). 4. Training was envisaged to strengthen the management capabilities of district health personnel to implement specific reforms, i.e., use and maintenance of RDFs; rational drug use; use and application of health services utilization information; use of computers; public health management; and financial management. 5. Clinical in-service training was planned using the 16 existing and modified UNICEF training modules, as well as about four new modules in multiple clinical subject areas. The Bank credit was expected to finance the entire cost of the district component, with the exception of civil works where there would be a cost-sharing with the government. It was envisaged that the implementation of Component 2 would be contracted to UNICEF because of its successful ongoing experience with the implementation of a similar program elsewhere in the country. The two objectives of this arrangement were: (i) for UNICEF to have major responsibility (with the chief doctors, other health staff and the communities) for the success of investments in the five pilot districts; and (ii) for the MOH staff to learn more about the design and implementation of PHC reform programs and thus increase the government's capacity for later extending such reforms to other districts in the country. Component 3: Project Coordination and Evaluation (estimated total cost US$1.0 million). This component supported the operation of a Project Coordination Unit (PCU), staffed by project management and technical staff, and a strong monitoring and evaluation program. The evaluation of the project was the responsibility of the MOH, assisted by international technical assistance. To support Component 2, this sub-component was to finance a series of household surveys and observational studies to get baseline and end-of project data in both the intervention districts and the control districts in order to assess the outcomes and the impact of the project. 1.6 Revised Components Component 1. There was no substantial restructuring of this component during project implementation. However, the implementation of sub-component 1.4. Management Information System (MIS) Development was significantly delayed due to a disagreement between the Bank and the counterpart in the MOH, the Department of Information and Statistics Administration, over specifications of hardware and software to be procured under the project. In addition, an agreement was reached for the sake of economies of scale and consistency that all procurement of hardware and software for the MOH would be carried out under the HR LIL and all technical assistance under another World Bank-financed project, the Second Institutional Building Technical Assistance project (IBTA-II). While the bidding for the supply and installation of the MIS was completed in the Summer 2004, the MOH canceled the bid arguing that technical specifications had to be revised. The Bank agreed with the MOH and a new bidding process was initiated. Nevertheless, eventually, the MOH decided to cancel the procurement of MIS and use its own resources to procure the necessary hardware to upgrade the system and carry out training. At the request of the MOH, savings from the MIS sub-component were used to purchase goods (i.e., ambulances). Component 2. There was no substantial restructuring of this component during project implementation. Component 2 included setting up a scheme of RDFs in pilot districts based on the high drug cost and the scarcity of some essential drugs in the market at project preparation. However, during project implementation, the market conditions changed leading to wider availability of drugs and reduction in prices. Following a July 2003 UNICEF study that showed that prices of project delivered essential drugs were nearly identical to those of their equivalents in the market and that there was wider drug availability in the market, the MOH, in agreement 10

19 with the Bank, decided to discontinue the implementation of the RDFs and to allow for the free distribution of all drugs thus far procured. Other savings from procurement of essential drugs were used to purchase basic furniture for refurbished PHC facilities in pilot districts. 6.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) Extensions of project closing date. The original closing date of the HR LIL was December 31, The closing date was extended three times and the project closed on September 30, The first extension, from January 1, 2004 to September 30, 2005, was due to delays in project implementation (see Section 7.2) as well as due to the fact that the original project implementation period of 3.5 years was too short, particularly for a first Bank-supported health project in Azerbaijan. The government requested the extension of the project's closing date in order to complete: (i) studies on health financing and pharmaceuticals; (ii) the full implementation of the MIS; and (iii) the civil works and their assessment under Component 2. The extension would also allow more time between the baseline survey (carried out in November 2002) and the evaluation as well as use of savings under the UNICEF contract for additional training activities. The second extension from October 1, 2005 to December 31, 2005 was granted to keep the allocation for Category 3(a) for consultant services open until December 31, 2005 in order to use the remaining credit funds to finance the following consultant services: (i) a final audit for the HR LIL; and (ii) an architect for hospital design and an environmental assessment for the followup operation (Health Sector Reform Project, HSRP) which was under preparation. The third extension covered the period from January 1, 2006 to September 30, In December 2005, the Bank reviewed the status of outstanding payments under the project and estimated that once the final replenishment was done, there were still likely to remain roughly US$930,000, partly due to the SDR/US$ exchange rate fluctuations. The Bank advised the government of this situation and the MOH requested another extension until September 30, 2006 to use these remaining credit funds to purchase some necessary equipment for the MOH and to finance some critical activities as part of the preparation of the HSRP. The activities financed during the third extension period included: (i) the project implementation/preparation team which managed the completion of the HR LIL and assisted in the preparation of the HSRP; (ii) purchase of necessary office equipment and furniture for the PCU; (iii) purchase of IT and other office equipment for 40 MOH staff; (iv) two architects (an international and a local one) for the assessment of construction sites for the preparation of the HSRP; (v) an auditing firm to carry out the project audit; and (vi) goods and equipment to address emergency needs with the Avian Influenza outbreak in the country in the Winter of Change of PCU In the Fall of 2005, there were important changes in the MOH affecting both the leadership of the ministry and the PCU of the HR LIL. In September 2005, following a ministerial decision, the PCU for the HR LIL was dissolved and the Minister announced that regular ministerial staff would oversee the implementation of the HR LIL until its, then, closing date (December 31, 2005). However, in November 2005, there was a change of ministers in the MOH and the new Minister requested a third extension of the closing date (from January 1, 2006 to September 30, 2006) and the continuation of the preparation of the HSRP. A new project team was hired soon after to manage the completion of the HR LIL and to assist in the preparation of the HSRP. 11

20 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) Quality at entry is rated satisfactory by this ICR. The project was based on sound background analysis. Although the project preparation period seemed to be relatively short (approximately 7 months from the Concept Review to Board approval), the involvement of the World Bank in the health sector in Azerbaijan dated back to Between 1992 and 2000, the Bank was engaged in policy dialogue with the government on the health sector and carried out a systematic situation analysis well documented in various Bank studies (CAS, CEM, poverty assessments, as well as a concept paper on health in 1998). This process enabled the Bank to have a good understanding of the main sector's issues and the necessary reform actions. However, the difficult political climate, serious concerns regarding transparency and accountability, and the low institutional capacity in the sector did not allow for the preparation of a project earlier than The project expanded on a previously successful health pilot program and took into account the lessons learned from that operation. The focus of Component 2 of the HR LIL was to enhance critical PHC services and assist the most vulnerable groups in five selected districts, building on the ongoing pilot program already implemented by UNICEF. UNICEF established a pilot program in 1996 as part of an attempt to reform the financing and delivery of PHC to ensure access to effective, efficient and equitable services. By the time of project preparation, the UNICEF program had recently been evaluated by an external team and the evaluation concluded that there was enough evidence of success to justify further development. The lessons highlighted in the UNICEF evaluation were incorporated in the design of the HR LIL and the Bank team, taking into account the high risk nature of the operation, opted for an arrangement under which UNICEF would be contracted by the government to implement Component 2 apart from civil works, while the PCU would administer the implementation of Component 1 and the civil works under Component 2. The rationale for expanding on the UNICEF pilot drew on its largely successful outcomes, while the rationale for selecting UNICEF as a co-implementer was based on the previous collaboration between UNICEF and the MOH on district health care reform. Furthermore, UNICEF staff was instrumental in implementing the pilot program and had an already established relation with the various actors in the health sector. UNICEF's participation was also expected to provide an additional level of control and accountability, and contribute to closer donor collaboration. The project adopted a participatory approach consistent with lessons learned from other Bank operations. Based on lessons derived from Bank operations in other countries, participatory approaches enhance project ownership, and contribute to better project implementation. Implementation of the district component was expected to involve considerable participation from beneficiaries and community groups. In addition, the district administration and the health care providers were expected to contribute greatly in the preparation and implementation of the rationalization plans. These actors were brought together for the first time in order to discuss and propose solutions to the challenges faced by the health sector in the selected districts. The PDO was well defined and focused on an outcome for which the project could reasonably be held accountable for. The project's objective was to be achieved by actions 12

21 taken at the central level, through increased knowledge and capacity of the MOH officials, but also at the local (district) level, through the development of rationalization plans, refurbishment of facilities, provision of equipment and pharmaceuticals, and training. Given the limited amount of resources made available by the credit, the project initial duration period (3.5 years), and the limited institutional capacity, the project was rightly designed to support the implementation of a relatively strong package of primary care reforms in only five districts. The lessons drawn from the implementation of reforms at the five selected districts was expected to enhance the ability of the MOH to carry out similar reforms in the rest of the country. The importance placed by the project on the primary health sector in Azerbaijan was in line with health sector developments in other CIS countries where there was a clear shift from secondary and tertiary level health services towards a family medicine model. The limited number of project components reflected the clear and realistic project objectives, while at the same time it did not compromise expectations regarding what the project could achieve. The first component targeted the central level, while the second component the district level, albeit close interaction between MOH officials and district authorities was a key factor for the successful implementation of both components. Although the project did not cover areas such as health care financing, it financed a major study on this subject, as well as other studies on pharmaceuticals, a needs assessment and rationalization plans for the pilot districts. These studies were later complemented by a full-scale Health Sector Review (2005) and served as the analytical basis for the preparation of the second World Bank-financed health operation in Azerbaijan. A different project design might have contributed to a more successful implementation of the project. Although the implementation of Component 2 was successfully completed by UNICEF, the fact that an important component was executed by an external agency limited the strengthening of the in-house capacity at the MOH and resulted in a lower level of project ownership. In addition, the project envisaged that funds for the refurbishment of facilities would not be available for use until rationalization plans were completed by the districts and approved by the MOH. However, there was no legal covenant attached to this condition. As a result, rationalization plans were prepared but they were not fully implemented, thus leading to no reduction on number of hospital beds or health personnel. 2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) The implementation of Component 2 was considerably delayed and it started only in the Summer of Soon after project approval, the MOH and UNICEF signed an Agreement for the implementation of Component 2. However, it took approximately a year for the implementation to start as there was a disagreement between the Ministry of Finance (MOF) and the Ministry of Health (on one side) and the World Bank and UNICEF (on the other side) on the procedure of drawing down credit funds. The Ministry of Health wanted credit funds committed under the UNICEF contract to be deposited in the account of the MOH at the International Bank of Azerbaijan. This was against standard disbursements procedures of the Bank and in disagreement with UNICEF. UNICEF agreed in accepting an annual blanket withdrawal application based on the work program of the year, instead of the whole amount of the contract as stipulated in the contract signed by the MOH. This disagreement resulted in major delays in implementing the contract with UNICEF and the project as a whole, and project implementation was rated unsatisfactory in March Following negotiations between the Bank and the Government, the latter agreed to proceed with the initial withdrawal application of funds in late 13