Uzbekistan: Woman and Child Health Development Project

Transcription

1 Validation Report Reference Number: PVR-331 Project Number: Loan Number: 2090 September 2014 Uzbekistan: Woman and Child Health Development Project Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank JPIB joint project implementation bureau M&E monitoring and evaluation MDG Millennium Development Goal MOH PCR PHC SVP TA WCH Ministry of Health project completion report primary health care selski vrachebny punkt (rural medical center) technical assistance woman and child health NOTE In this report, $ refers to US dollars. Key Words asian development bank, independent evaluation department, infant mortality rate, maternal mortality ratio, millennium development goals, primary health care, project completion report, project validation report, uzbekistan, woman and child health development project, world bank The guidelines formally adopted by the Independent Evaluation Department (IED) on avoiding conflict of interest in its independent evaluations were observed in the preparation of this report. To the knowledge of IED management, there were no conflicts of interest of the persons preparing, reviewing, or approving this report. In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, IED does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT BASIC DATA Project Number: PCR Circulation Date: Nov 2012 Loan Number: 2090 PCR Validation Date: Sep 2014 Project Name: Woman and Child Health Development Project Country: Uzbekistan Approved ($ million) Actual ($ million) Sector: Health and Social Total Project Costs: Protection ADB Financing: ($ million) ADF: 0.00 Loan: Borrower: OCR: Beneficiaries: Others: Cofinancier: Total Cofinancing: Approval Date: 23 Sep 2004 Effectiveness Date: 31 Jan May 2005 Signing Date: 2 Nov 2004 Closing Date: 30 Jun May 2012 Project Officers: K. Saleh R. Mamatkulov Y. Shiroishi Y. Shiroishi N. Talipova Location: ADB headquarters ADB headquarters ADB headquarters Uzbekistan Resident Mission Uzbekistan Resident Mission From Oct 2004 Mar 2005 Jul 2005 Apr 2008 Jul 2008 To Feb 2005 Jun 2005 Mar 2008 Jun 2008 May 2012 Validator: K. Hardjanti, Consultant Peer Reviewer: H. Son, Principal Evaluation Specialist, IED1 Quality Reviewer: E. Gozali, Principal Evaluation Specialist, IED1 Director: W. Kolkma, Director, IED1 ADB = Asian Development Bank; ADF = Asian Development Fund; IED1 = Independent Evaluation Department, Division 1; OCR = ordinary capital resources; PCR = project completion report. A. Rationale I. PROJECT DESCRIPTION 1. Prior to independence from the Soviet Union in 1990, the indicators for woman and child health (WCH) in Uzbekistan were better than average in the Central Asian republics. However, these indicators have stagnated or declined since then, as follows: (i) maternal mortality ratio, estimated at 34 per 100,000 live births in 2001, stalled since the mid-1990s with wide geographic disparities; (ii) infant mortality rate at 62 deaths per 1,000 live births in 2002 not much changed compared with the early 1990s, and almost twice higher in rural areas than in urban areas; and (iii) HIV/AIDS registered cases almost doubled in the past 2 years, and tuberculosis incidence also doubled over the past decade (prior to project approval). If this trend continued, Uzbekistan would not attain the health-related Millennium Development Goals (MDGs). 2. As regards WCH, Uzbekistan had a good health care system but it lacked in quality. Of about 95% of women who received antenatal care, only 39% received it within the first trimester of pregnancy. Ninety eight percent of women delivered their babies in health facilities guided by a skilled health attendant. However, pregnancy or delivery complications were the primary cause of maternal mortality, which suggested poor prenatal care despite high attendance and inadequately managed high-risk cases.

4 2 3. WCH services were fragmented and limited in the selski vrachebny punkt ([SVP] or Russian for rural medical center). Not much rationalization has been made in primary health care (PHC) beyond the first level of outpatient care. The SVPs were the first clinics that provided PHC services. For women, although pre- and postnatal care and counseling were provided at SVPs and policlinics, obstetric care was given in hospitals. Delivery hospitals offered services for normal deliveries, but care for complicated deliveries and high-risk neonates was limited, and exposure to safe motherhood and baby-friendly hospital initiatives was minimal. The referral hospitals that accepted delivery complications were also weak. WCH care rationalization and a service integration plan were crucial within the health system framework. 4. Therefore, the Government of Uzbekistan in 2002 adopted a series of long-term state programs for WCH. To support these programs, particularly addressing gaps in WCH services, the government requested the assistance of the Asian Development Bank (ADB). ADB and the World Bank agreed with the government to develop complementary projects, with the World Bank supporting PHC and health financing, and ADB supporting WCH. 1 This broader approach aimed to help disseminate information to the public and provide better and earlier access to WCH services at the PHC level so that hospital services could be optimized. B. Expected Impact 5. The project 2 was to improve the health of women and children in Uzbekistan. Its direct impact to WCH was to reduce unnecessary use of hospitals and provide better information to consumers. C. Objectives or Expected Outcome 6. The project s broad objectives or expected outcomes were to (i) support health sector reforms, and (ii) increase the efficiency of the health care delivery system for WCH. D. Outputs 7. The project had four components: (i) strengthening WCH services; (ii) supporting finance, information, and management; (iii) building a blood safety program; and (iv) improving project management. 8. The expected outputs of these components were as follows: (i) Component 1 build PHC and a first referral network for WCH with improved services and capacity of medical and paramedical personnel to offer international standards of health care; (ii) Component 2 increase the efficiency of the health system by improving payment mechanisms, management, and access to timely information; (iii) Component 3 reduce the spread of blood-borne infectious diseases, including HIV/AIDS, by building a nationwide blood safety program; and (iv) Component 4 strengthen the capacity of the Ministry of Health (MOH) in project management, monitoring, and evaluation. 1 The project was prepared in parallel with the World Bank s Health-II project. These two projects have complementary components. 2 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan and Technical Assistance Grant to Uzbekistan for the Woman and Child Health Development Project. Manila.

5 3 E. Provision of Inputs 9. At project appraisal, the total project cost was estimated at $70.0 million $40.0 million from ADB s ordinary capital resources and $30.0 million from the government. The final actual project costs increased substantially because of the expansion in scope to $188.9 million mostly from the government. ADB financed $38.1 million and the government financed $150.8 million. Unused loan funds of $1.9 million were canceled at project closing. 10. The project focused on only 6 of the total 13 provinces at appraisal. During the midterm review, the government requested ADB to expand project coverage and equip district and provincial WCH referral facilities in the seven remaining provinces. 3 In January 2008, ADB approved a reallocation of loan proceeds to expand project activities. 11. Subsequently, the government requested another reallocation of the loan proceeds in October This request was to equip additional maternity facilities, and finance laboratory and storage equipment for blood centers. ADB approved the request in November 2010 and extended the loan closing date until 31 December 2011, enabling the completion of equipment procurement and installation. By reallocating loan proceeds and extending the loan closing date, the government was able to maximize the achievement of project objectives and utilize loan funds. F. Implementation Arrangements 12. The Ministry of Health (MOH) was designated as the executing agency of the project, with the deputy minister for WCH in the MOH as the project coordinator. An interagency project steering committee provided overall project guidance. A joint project implementation bureau (JPIB) was set up to support the implementation of both the ADB- and the World Bank-financed health-i and health-ii projects in Uzbekistan. The collaboration and coordination of development partners health reform activities through the JPIB was a salient feature in project implementation. 13. This collaborative implementation arrangement enabled complementary and timely support for health system financing and policy reforms, which were mainly supported by the World Bank health-i project and were crucial for the success of the ADB project. In addition, it integrated WCH in PHC and made it possible for the WCH and blood safety components to cover the entire country. 14. The technical assistance (TA) attached to the loan focused on the Capacity Building for Woman and Child Health Development (a grant of $300,000) to (i) strengthen the MOH s capacity to deal with health issues by improving its understanding of provider and consumer behaviors, and the causes of morbidity and mortality; (ii) help the government optimize and rationalize WCH care services, including human resource planning; and (iii) increase the effectiveness and delivery of services through quality monitoring and improvements. The TA was completed in December 2007, and was rated satisfactory. 4 With ADB s approval, the MOH and the JPIB used TA inputs effectively. The TA consultants (i) highlighted the need to focus on neonatal care; (ii) identified the needs of health professionals; (iii) recommended the 3 During appraisal, $18.0 million was earmarked for equipping the selski vrachebny punkt (rural medical center) in the project area. However, the government decided to equip the SVPs with the World Bank s assistance through its health projects. 4 ADB Technical Assistance Completion Report: Capacity Development for Woman and Child Health Development in Uzbekistan. Manila.

6 4 restructuring of the national blood safety program to ensure blood safety and quality assurance; and (iv) identified key areas for improving monitoring by developing a health management information system for the MOH and for meeting the requirements for a database of blood donors. Identifying qualified consultants and coordinating their inputs were not easy, but the TA consultants inputs were valuable to the project, and most of their recommendations were eventually implemented. 15. There were 34 covenants, of which 32 were complied with. Two covenants relating to implementing health sector reforms, which required government orders involving other agencies were in the process of being complied with at project completion. The government was in the process of ensuring that proposed reforms were properly adopted within the Uzbekistan context. II. EVALUATION OF PERFORMANCE AND RATINGS A. Relevance of Design and Formulation 16. The project completion report (PCR) 5 rated the project relevant. It was consistent with the government s health sector priorities and ADB s country strategy and program for Uzbekistan, which highlighted the improvement of human development including health, to reduce poverty and achieve the health-related MDGs. The project was in line with the five strategic directions of the ADB Policy for the Health Sector: (i) PHC for vulnerable groups, (ii) effective interventions, (iii) innovation, (iv) support for health sector reforms, and (v) increase in sector efficiency by improving managerial capacity and collaboration with partner institutions. The PCR also observed that the project was overambitious considering the timelines. While the project focused mainly on developing WCH (as its title indicates), it was spread thinly across several components and risked losing its intended focus. For example, the project supported other health sector reforms whose benefits extended beyond women and children. 17. The project design was formulated using a participatory approach involving stakeholders and other development partners in the health sector. With the government s guidance, ADB prepared the project in close coordination with other health projects of the World Bank and United Nations and bilateral agencies involved in maternal, newborn, and child health. Consequently, the project design ensured the integration of WCH care in PHC, and the project benefited from complementary activities. The project also avoided duplication while supporting the implementation of sector reforms. 18. The project remained relevant at completion. The decision to expand the project s coverage nationwide further strengthened the project s relevance, contributed in reducing disparities among regions, and facilitated the implementation of administrative and technical sector reforms in the country. The complementary activities by other developing partners as mentioned in para. 17 contribute to this validation s rating of relevant. B. Effectiveness in Achieving Project Outcome and Outputs 19. The PCR rated the project effective based on the following: (i) the project attained outcomes in supporting health sector reforms and improved the efficiency of WCH services; (ii) the project expanded the government s WCH reforms nationwide, piloted new finance and 5 ADB Completion Report: Woman and Child Health Development Project in Uzbekistan. Manila.

7 5 services mechanisms, and ensured that it complemented the World Bank health projects; and (iii) the project achieved six of its eight performance targets as follows: (a) more equal allocation of health care resources across provinces (the difference was reduced from 11.6% to 6.6%); (b) 75% increase in the percentage of non-salary recurrent resource allocation for PHC (exceeding the 20% target); (c) increase of 20% in PHC utilization (exceeding the 10% target), and reduction of 10% in hospital referrals and admissions (same as target); (d) reduction of 45% in the incidence of moderate iron deficiency anemia among pregnant women (exceeding the 22% target); (e) contraceptive prevalence among fertile-aged women was 56.6% in 2011; and (f) reduction in the incidence of infectious diseases, particularly tuberculosis a decline from 75.9 per 100,000 in 2004 to 52.9 in The project exceeded many outcomes and improved the health status of women and children, as stated in the project s impact. The project also supported health sector reforms beyond women and children, such as strengthening finance, and information and management systems of primary health care centers. These reforms were necessary to enhance the health of women and children. 20. Most importantly, the project has changed the working approach and behavior of health professionals toward WCH. Midwives and obstetrician-gynecologists (including those in PHC and family nurses in district maternity centers and provincial perinatal centers) have been trained and now use more effective methods and modern equipment. This validation rates the project s effectiveness in achieving its outcomes effective. C. Efficiency of Resource Use in Achieving Outcome and Outputs 21. The PCR rated the project efficient. 6 The project attained its key outcome targets, although there were some delays at project start-up. The project used resources efficiently, for example: (i) the referral system for WCH care and delivering quality care at appropriate levels brought significant cost savings; (ii) the blood safety program consolidated blood safety testing in some blood centers and facilitated quality control and cost savings; (iii) the new MOH guidelines optimized blood transfusion and the use of blood products, which has avoided unnecessary risks and reduced costs; and (iv) the MOH has developed a comprehensive health management information system. 22. As mentioned, the project worked closely with the World Bank and other development partners, thereby avoiding duplications. Project elements, such as improved equipment, clinical guidelines, and training, were jointly implemented. The World Health Organization, the United Nations Children s Fund, and the United Nations Population Fund supported the development of clinical guidelines and training of health professionals at all levels. In addition, integrating WCH in PHC enhanced Uzbekistan s health system and extending project coverage to the whole country ensured maximum project outcome and outputs. Some delays in loan compliance were experienced in the (i) submission of audit reports, and (ii) allocation of government resources for rehabilitating health facilities in the project area. Nevertheless, this validation concurs with the PCR rating and rates the project efficient in achieving its outcome and outputs. 6 No rate of return or cost analysis was included in the PCR, but it used data and information, among others, from in-depth surveys of the project s impact and the project s pilots, and continuous monitoring and evaluation results (PCR, para. 36).

8 6 D. Preliminary Assessment of Sustainability 23. The PCR rated the project likely sustainable. To ensure sustainability, the government significantly increased the allocation of domestic resources for recurrent costs. Also, the government earmarked sufficient funding to cover the operation and maintenance cost of the blood centers. 24. The project strengthened the system of continuous professional education for health professionals, and the new nursing curriculum helped improve the nurses professional qualifications and increased their incentive opportunities. 7 All health professionals use the clinical guidelines and protocols introduced by the project. Graduate and post-graduate students in medical services and institutions nationwide also use the revised curricula and training materials for health professionals. The project introduced a health care monitoring and quality control system in PHC. 25. The project s support for partnership deliveries in which traditional birth attendants partner with health professionals was adopted and became standard procedure for both health professionals and the population. Women can now choose to deliver in the presence of qualified health professionals. Reforms will be further developed and continued under the ongoing World Bank health-iii project. This validation agrees with the project s rating of likely sustainable. E. Impact 26. The PCR rated the project s impact significant. It justified this rating by stating that the project has improved health-related MDG indicators by (i) reducing maternal mortality rate from 31.4 in 2004 to 23.1 in 2011; (ii) decreasing infant mortality rate from 15.2 per 1,000 live births in 2004 to 10.4; (iv) reducing under-five mortality rate from 21 per 1,000 live births in 2004 to 14.8 in 2011; and (v) aiming to halt or begin to reverse by 2015 the spread of infectious diseases such as tuberculosis and HIV/AIDS. Training programs and methodological guidelines have significantly enhanced the service-delivery capacity of the PHC, mother-and-childhealth protection, blood services, and health departments. These achievements are remarkable, especially as they were attained within a relatively short period ( ). However, the health-related MDGs are long-term indicators and their quick restoration under the project could have been assisted by the legacy of WCH services before the 1990s. 27. The project s support for educational programs in cooperation with local social agencies (the Association of Nurses, makhallas, 8 and women s committees in local communities) improved the rural population s awareness in and expectations for reproductive and child health protection. The project had gender equity as a theme, and as such, women were the project s immediate beneficiary. The project ensured strong female participation in training, service delivery, and WHC activities, as discussed in detail in Appendix 9 of the PCR. Overall, this validation rates the project s impact significant. 7 Including the preparation of a master s degree in nursing, expected to be approved in the academic year. 8 A makhalla is a local self-governing community-based organization in Uzbekistan.

9 7 III. OTHER PERFORMANCE ASSESSMENTS A. Performance of the Borrower and Executing Agency 28. The PCR rated the performance of the borrower and the executing agency satisfactory. The MOH as the executing agency showed strong ownership and leadership throughout project implementation, and ensured good coordination with other development partners. The borrower was clear in identifying the type of institutional capacity support needed by government agencies. Resources from the project s attached TA were appropriately used to flesh out policy options, facilitate their implementation, and provide onthe-job training. 29. Initially, the JPIB, which also implemented the World Bank health-ii project, faced some difficulties in managing the complicated administrative processes, but it gained sufficient knowledge by the middle of project implementation. The JPIB was able to adequately manage staff turnover, which contributed to the project s success. Hence, this validation also rates the performance of the borrower and executing agency satisfactory. B. Performance of the Asian Development Bank 30. The PCR rated ADB s performance satisfactory based on ADB s continued support to the project, as it (i) sought solutions by discussing and collaborating with concerned agencies on identified issues; (ii) conducted 12 loan review missions from 2005 to project inception mission, 9 project review missions, 1 midterm review mission, and 1 project completion review mission; (iii) responded flexibly to the needs of the project by (a) using TA resources, (b) approving the government s request to expand project coverage and reallocate loan proceeds, (c) approving the government s request to make changes in the procurement mode and packaging, and (d) extending the loan closing date; (iv) increased the effectiveness of the project by transferring project administration after the midterm review in 2008 to ADB s Uzbekistan Resident Mission; and (v) collaborated well with other development partners, improving project implementation and especially complementing project outputs with the activities of other health donors. This validation rates ADB s performance satisfactory. IV. OVERALL ASSESSMENT, LESSONS, AND RECOMMENDATIONS A. Overall Assessment and Ratings 31. The PCR s overall rating of the project is successful based on the relevant, effective, efficient, and likely sustainable ratings (see table). The project s impact is rated significant. This validation supports the PCR s overall rating. 32. The project covered the entire country compared with six provinces planned at appraisal. It succeeded in getting the community to adopt a new behavior approach toward PHC and WCH care. The services provided by family nurses and SVPs, and the new approach to delivery, focusing on the safety and comfort of the mother and child, were widely used. The project introduced evidence-based medicine and quality assurance mechanisms for WCH care and blood safety, i.e., clinical guidelines and peer reviews, in addition to rehabilitating health facilities, procuring modern equipment, and training health professionals. Referrals for blood transfusions have been modified, thereby reducing the number of transfusions and associated risks.

10 8 Overall Ratings Criteria PCR IED Review Reason for Disagreement and/or Comments Relevance: Relevant Relevant Effectiveness in Effective Effective achieving project outcome and outputs: Efficiency in achieving Efficient Efficient outcome and outputs: Preliminary assessment of sustainability: Likely sustainable Likely sustainable Overall assessment: Successful Successful Borrower and executing Satisfactory Satisfactory agency: Performance of ADB: Satisfactory Satisfactory Impact: Significant Significant Quality of PCR: Satisfactory Refer to para. 38. ADB = Asian Development Bank, IED = Independent Evaluation Department, PCR = project completion report. Source: ADB Independent Evaluation Department. B. Lessons 33. Four lessons were derived from the project. First, the project underscored the important elements for project success such as (i) strong commitment by the government, (ii) close collaboration among development partners, and (iii) prompt response to the client s changing needs during implementation. Second, the co-leadership of the MOH and the project coordinator was key to the project s achievements. Third, the project was not a stand-alone WCH project, but a part of broader health sector reforms. Thus, strong collaboration among partners and responsiveness to actual needs were important contributors to the success of the various programs. This collaboration enabled other partners to address project activities, making it possible to reallocate resources and expand geographic coverage. This approach improved project achievements and increased the possibility of sustainability. This validation agrees with the PCR s lessons. 34. This validation adds another lesson: to ensure that project components are realistically achievable given the proposed time frame and other project resources, needs assessment should have been conducted. C. Recommendations for Follow-Up 35. Project-related. Covenants ADB to monitor the progress of ongoing health sector reforms that have not been complied with, through continued dialogue with the World Bank and other partners. Joint evaluation of primary health care and women s and children s health care the MOH may consider organizing among partners a joint evaluation of the results, lessons, and recommendations of partner-supported health sector projects, including the project. 36. General. Continuous training Health professionals require continuous training to maintain their position as specialists. Training should be further institutionalized for these health professionals to continue their practice, even for general practitioners, midwives, and

11 9 nurses. This validation considers the PCR s recommendations to be sound and need to be rigorously followed up. V. OTHER CONSIDERATIONS AND FOLLOW-UP A. Monitoring and Evaluation Design, Implementation, and Utilization 37. The project monitoring and evaluation (M&E) system and collection of other related data were adequately established. The MOH set up a working group on M&E for the ADB and World Bank projects. This working group developed a project framework that identified the expected joint results. Baseline surveys for evaluating continuous education, safe motherhood, financial reforms, and satisfaction of consumers in WCH facilities were accomplished in In 2009, the pilot projects such as establishing a neonatal intensive care unit in the perinatal center, financing of hospital services, and establishing regional blood centers were conducted. Close to project completion in 2011, a final survey was conducted to evaluate (i) the project s impact on obstetric facilities, (ii) management and information systems in health care facilities, (iii) rationalization and regionalization of blood centers, and (iv) the establishment of a neonatal intensive care unit in perinatal centers. Throughout project implementation, routine data were collected and regular monitoring of the projects M&E indicators were conducted. B. Comments on Project Completion Report Quality 38. Overall, the PCR quality is rated satisfactory. The PCR is in line with PAI and is consistent throughout its text. The PCR provides adequate evidence and analysis in substantiating the project s ratings. It uses data and information collected from the project s extensive surveys. The PCR is clear and succinct in presenting facts, background data, and reasoning. It identifies the exogenous factors that affected the project s achievements. The PCR includes useful lessons derived from the project and sound recommendations for followup actions. C. Data Sources for Validation 39. Data sources for this PCR validation were the (i) PCR, (ii) report and recommendation of the President (RRP), and (iii) selected reports of the loan and TA review missions. D. Recommendation for Independent Evaluation Department Follow-Up 40. If the Independent Evaluation Department deems a project performance evaluation report necessary, the optimal time to conduct it would be in or after By this time, the impact on halting or beginning to reverse the spread of infectious diseases would have been achieved (para. 26); and the two remaining covenants that were being complied with at project completion, would have been fully complied with as expected (para. 15). Also, the joint evaluation among partners, as recommended by the PCR (para. 35), would have been conducted at this time. 9 ADB Project Completion Report for Sovereign Operations. Project Administration Instructions. PAI