IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-95274) ON A CREDIT IN THE AMOUNT OF US$ 2.0 MILLION TO THE LAO PEOPLE S DEMOCRATIC REPUBLIC FOR A

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No: ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-95274) Human Development Sector Unit East Asia and Pacific Region ON A CREDIT IN THE AMOUNT OF US$ 2.0 MILLION TO THE LAO PEOPLE S DEMOCRATIC REPUBLIC FOR A COMMUNITY NUTRITION PROJECT March 26, 2014

2 CURRENCY EQUIVALENTS (Exchange Rate Effective March 26, 2014) Currency Unit = LAK LAK 1.00 = US$ US$ 1.00 = LAK 8039 FISCAL YEAR October 1 September 30 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immuno Deficiency IDA International Development Association Syndrome ANC Ante-natal Care IEC Information, Education and Communication CCT Conditional cash transfer IEG Independent Evaluation Group CED Chronic energy deficiency IMNCI Integrated Management of Neonatal and childhood illnesses CBD Community based distributor ISR Implementation Status and Results Report CBN Community based nutrition IYCF Infant and Young Child Feeding CIEH Center of Information and Lao Lao People s Democratic Republic Education for Health PDR CNP Community Nutrition Project LiST Lives Saved Tool CPS Country Partnership Strategy LWU Lao Women s Union DHHP Department of Hygiene and Health MCH Maternal and child health Promotion DHS Demographic Health Survey MCHN Maternal Child Health and Nutrition DALY Disability adjusted life year MNCH Maternal, Neonatal and Child Health DPT Diphtheria, Pertussis and Tetanus M&E Monitoring and evaluation EGDF Ethnic Group Development MOH Ministry of Health Framework EGDP Ethnic Group Development Plan MOU Memorandum of Understanding ENC Essential Newborn Care ORS Oral Rehydration Solution EU European Union PDO Project development objective FCRRF-TF Food Crisis Rapid Response PMU Project Management Unit Facility Trust Fund FM Financial management PNC Post Natal Care HPA Health Poverty Action SGA Small for Gestational Age FPCR Food Price Crisis Response SUN Scaling Up Nutrition GDP Gross Domestic Product TA Technical assistance GFRP Global Food Response Project TOT Training of Trainers HC Health Center VF Village facilitator HSIP Health Services Improvement WHO World Health Organization Project HSIP AF Health Services Improvement Project Additional Financing Vice President: Country Director: Sector Manager: Project Team Leader: ICR Team Leader/Author: Axel van Trotsenburg Ulrich Zachau Toomas Palu Ajay Tandon Ashi Kohli Kathuria

3 LAO PEOPLE S DEMOCRATIC REPUBLIC Community Nutrition Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 6. Comments of Cofinanciers and Other Partners/Stakeholders Annex 7. List of Supporting Documents MAP 37006

4 A. Basic Information Country: Lao People's Democratic Republic Project Name: Project ID: P L/C/TF Number(s): TF ICR Date: 03/26/2014 ICR Type: Core ICR Lending Instrument: ERL Borrower: Original Total Commitment: Revised Amount: USD 2.00M Environmental Category: C Community Nutrition Project MINISTRY OF FINANCE USD 2.00M Disbursed Amount: USD 1.66M Implementing Agencies: Department of Hygiene and Health Promotion, the Ministry of Health Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2008 Effectiveness: 10/03/ /03/2009 Appraisal: 04/13/2009 Restructuring(s): 09/19/ /25/2012 Approval: 08/27/2009 Mid-term Review: 10/31/ /24/2011 Closing: 09/30/ /30/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Satisfactory Moderate Moderately Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i

5 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project Yes at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes Moderately Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Sector Code (as % of total Bank financing) Health 100 Actual Theme Code (as % of total Bank financing) Global food crisis response 100 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams Country Director: Ulrich Zachau Annette Dixon Sector Manager: Toomas Palu Juan Pablo Uribe Project Team Leader: Ajay Tandon Magnus Lindelow ICR Team Leader: Ajay Tandon ICR Primary Author: Ashi Kohli Kathuria F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To improve coverage of essential maternal and child health services and improve mother and child caring practices among pregnant and lactating women and children less than 2 years old in the seven southern and central provinces. Revised Project Development Objectives (as approved by original approving authority) Not applicable ii

6 (a) PDO Indicator(s) Indicator Indicator 1 : Baseline Value Original Target Values (from approval documents) iii Formally Revised Target Values Actual Value Achieved at Completion or Target Years Percent of women aged years that were attended at least once during pregnancy in the past 12 months by a skilled health personnel Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The project far exceeded the target (125 percent achievement against the target) achievement) Indicator 2 : Percent of women aged with a birth in the last 12 months that delivered at a heath facility Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The target was met (100 percent achievement against target) achievement) Indicator 3 : Percentage of children aged months receiving DPT3 before their first birthday Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) Indicator 4 : With about 88 percent achievement of the parget, the indicator fell short of the target. The shortfall could possibly due to sub-optimal project performance, or vaccine shortage or a combination of both. Percent of women aged with a child aged 0-11 months who attended at least one routine monthly check-up in the past 12 months Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) Achievement of this indicator was 134 percent of the target, indicating that it surpassed expectations. Indicator 5 : Percent of women aged with a live birth in the past 12 month that put the newborn infant to the breast within 1 hour of birth Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013

7 Comments (incl. % The indicator exceeded the target, the percent achievement being 107 achievement) Indicator 6 : Percent of children aged 0-23 months with diarrhea in previous 2 weeks that received oral rehydration salts Value quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The indicator exceeded the target with a percent achievement of 101. achievement) (b) Intermediate Outcome Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) iv Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 1 : Number of health centers that are operating the CCT program Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The target was met (100 percent achievement) achievement) Indicator 2 : Percentage of eligible women who have enrolled Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments The target was met and slightly exceeded (107 percent achievement) as of (incl. % December 31, 2012 (reported in ISR # 6). achievement) Indicator 3 : Percent of enrolled pregnant women who received payment for delivery at the health facility Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) Indicator 4 : Value (quantitative With 78 percent achievement, the indicator fell short of the target. It likely reflects the impact of the initial delays and problems in CCT implementation, largely on account of the low capacity. People with access to a basic package of health, nutrition or reproductive health services (Core Indicator) 0 24,617

8 or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) Indicator 5 : Births (deliveries) attended by skilled health personnel (Core Indicator) Value (quantitative or Qualitative) 0 2,206 Date achieved 06/30/ /25/ /31/2013 Comments The target was exceeded (112 percent achievement). The actual value (incl. % achievement date is December 31, 2012 (noted in ISR # 6) achievement) Indicator 6 : Number of villages in which the 3 facilitators have been trained Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments The target was exceeded (112 percent achievement). The actual value (incl. % achievement date is December 31, 2012 (noted in ISR # 6) achievement) Indicator 7 : Number of villages in which CBN meeting has been held this month Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) 39 percent of the planned target was achieved. Reasons include: multiple modules completed in several villages in one meeting; delays in report collection. Measuring average meetings per quarter rather than 'this month' would have been a better definition. Average number of participants per month in CBN meetings Indicator 8 : Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The indicator achieved more than twice the target (267 percent achievement). achievement) Indicator 9 : Number of Project-supported health centers in which at least one staff has been trained in the 5 module training Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The indicator fell slightly short of target with 95 percent achievement. achievement) Indicator 10 : Number of health centers visited by the monitoring team from central and/or v

9 provincial and/or district level every 3 months (quarterly) Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % The target was met (100 percent achievement against target) achievement) Number of community-based distributors who have received micronutrient Indicator 11 : training Value (quantitative or Qualitative) Date achieved 06/30/ /25/ /31/2013 Comments (incl. % achievement) The indicator achievement exceeded the target (104 percent achievement). The actual value achievement date is November 3, 2012 (reported in ISR # 5) Indicator 12 : Health personnel receiving training (Core Indicator) Value (quantitative or Qualitative) 0 59 Date achieved 06/30/ /31/2013 Comments (incl. % achievement) Indicator 13 : Direct project beneficiaries (Core Indicator) Value (quantitative or Qualitative) 0 24,617 Date achieved 06/30/ /31/2013 Comments (incl. % achievement) Indicator 14 : Female beneficiaries (Core Indicator) Value (quantitative or Qualitative) 0 24,617 Date achieved 06/30/ /31/2013 Comments (incl. % achievement) vi

10 G. Ratings of Project Performance in ISRs No. Actual Date ISR DO IP Disbursements Archived (USD millions) 1 06/09/2010 Satisfactory Satisfactory /28/2010 Moderately Satisfactory Moderately Satisfactory /21/2011 Moderately Satisfactory Moderately Unsatisfactory /24/2011 Moderately Satisfactory Moderately Satisfactory /25/2012 Moderately Moderately Unsatisfactory Unsatisfactory /05/2013 Moderately Moderately Unsatisfactory Unsatisfactory /24/2013 Moderately Satisfactory Moderately Satisfactory /23/2013 Moderately Satisfactory Moderately Satisfactory 1.66 H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO Amount Disbursed at Restructuring in USD millions 09/19/2011 MS MU /25/2012 MS MS 0.56 IP Reason for Restructuring & Key Changes Made Second level Sept 19, 2011 to (a) allow for payment to facilities and financial intermediaries (institutions providing free deliveries to women giving birth in these institutions) under the CCT component; and (b) amend the definition of incremental operating costs to include translation costs and software Second level - July 25, 2012 to (a) grant a one year extension of the closing date; (b) reallocate to increase operating costs to the Ministry of Health from consultant services and Lao Women s Union operating costs; and (c) amend the indicators to ensure they adequately and consistently capture project impact. vii

11 I. Disbursement Profile viii

12 1 Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The Community Nutrition Project (CNP) was prepared during the period when food prices were rising sharply in the Lao PDR, as they were globally. During the period of project appraisal: 2. Undernutrition in the Lao PDR was high, and had shown very little decline notwithstanding improved economic growth. With about 14.5 percent women of reproductive age chronically undernourished (CED) 1, 36.2 percent anemic and over 40 percent of the children under-five stunted, undernutrition in the country was high. The persistent lack of decline in child undernutrition was of particular concern as evident from a prevalence of 40 percent underweight in 1990 and 37.1 percent in 2006, although significant progress in reducing poverty and child mortality had been made since the early 1990s Besides food insecurity, prevailing maternal, infant and young child feeding and caring practices contributed significantly to the undernutrition challenge. While food insecurity was a problem, with food poverty estimated at over 20 percent in 2002, key family behaviors -- such as inappropriate breastfeeding and complementary feeding practices; food fads associated with pregnancy and the postpartum period; high incidence of vector- and food-borne disease; and myriad other factors -- also undermined nutritional improvements. This is indicated by the relatively high rates of malnutrition even in better-off segments of the population Health service utilization rates were exceedingly low. About 71.5 percent of pregnant women received no antenatal care and 85 percent delivered at home 4, typically without skilled attendance. There were also significant gaps in skills and practices of health professionals related to ante-natal, birth and post-natal periods. 5. Household vulnerability to shocks was high, increasing susceptibility of poorer segments to undernutrition. Given the vulnerability of Lao households to natural disasters and macroeconomic shocks, the global food price crisis and financial crises heightened the vulnerabilities, potentially leading to a worsening of undernutrition, maternal and child caring practices and utilization of maternal and child health (MCH) services in the country. 6. The rationale for Bank assistance was sound and the project contributed to the World Bank objectives as well as to the Lao PDR country strategies. The CNP proposed risk-mitigation efforts to protect the health and nutrition of women and children from possible long-lasting effects on their physical and cognitive development, and was designed to test approaches that, if successful, could inform their adoption at scale. These directly supported two of the four pillars of the World Bank s Strengthened and Extended Country Assistance Strategy (CAS) program, , Improving Social Outcomes, and Strategic Approach to Capacity Development and Partnerships, to which health and, within that, nutrition were integral. It also contributed to the objectives of the Lao Government s Strategy and Planning Framework for the 1 CED or Chronic Energy Deficiency is a measure of undernutrition in adults and in women a body mass index of less than 18.5 indicates CED. 2 The incidence of poverty declined from 46 to 38 percent between 1992 and 2002; infant mortality declined from 104 to 70 per 1,000 live births between 1995 and 2005; and under-5 mortality declined from 170 to 98 over the same period. 3 Estimates from Multi-indicator Cluster Survey (2006) suggest that 27 percent of children in urban households and 26 percent of children in the top wealth quintiles are stunted. 4 Lao Reproductive Health Survey (2005). 1

13 Integrated Package of Maternal Neonatal and Child Health Services ( ); the National Nutrition Policy (2008); and the National Nutrition Strategy (2009). 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. The proposed PDO was to improve coverage of essential MCH services and improve mother and child caring practices among pregnant and lactating women and children less than two years old in the seven southern and central provinces. The key indicators as approved originally are presented in Column 1 of Table Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 8. The PDO was not revised. However, several indicators (both PDO and intermediate results) were modified (dropped, revised, or new ones added) as seen in Table 1 below. The rationale for the revision of indicators was to; (a) more precisely define them so as to better align them with standardized indicators from other sources for better data comparability; (b) drop indicators for which data was unavailable or were not directly related to CNP activities; and (c) add new ones to better measure selected processes, e.g., the monitoring exercise. Table 1: List of Original, Revised, Added and Dropped Indicators Original Indicator Percentage of pregnant women with 2 or more antenatal care consultations with a trained health professional Percentage of pregnant women who delivered in health facility Percentage of children under 1 year of age immunized with DPT3 Percentage of children under 2 years of age who attended a well-baby consultation in the last year Percentage of infants under 5 months of age who are fed exclusively with breast milk Percentage of children under 5 years of age with diarrhoea in the past 2 weeks (diarrhea is defined as more than 3 loose stools in 24 hours) Percentage of children between 6 and 59 months who received vitamin A supplement during the last year Percentage of women and children less than 5 years of age who received iron supplements in the last year Percentage of women who practice post-partum food restrictions Percentage of children between 6 and 23 months of age who receive foods from 4 or more food groups Percentage of women participating in community meetings with appropriate hand washing behavior PDO Revised/ New Indicator/Dropped Percent of women aged years that were attended at least once during pregnancy in the past 12 months by a skilled health personnel Percent of women aged with a birth in the last 12 months that delivered at a health facility Percentage of children aged months receiving DPT3 before their first birthday Percent of women aged with a child aged 0-11 months who attended at least one routine monthly check-up in the last 12 months Percentage of women aged with a live birth in the past 12 months that put the new-born infant to the breast within 1 hour of birth Percent of children aged 0-23 months with diarrhea in previous 2 weeks that received oral rehydration salts Dropped Dropped Dropped Dropped Dropped Component One Percentage of women in target areas who received at least Percentage of eligible women who have enrolled one conditional cash payment in the last year Percent of women who received a CCT payment for Percent of enrolled pregnant women who received payment 2

14 Original Indicator delivery who deliver in a health facility Extent of leakage to non-eligible populations Number of communities in which the community nutrition program is operating (facilitators have been trained and meetings are held regularly) Revised/ New Indicator/Dropped for delivery at the health facility Dropped Component Two Number of villages in which the 3 facilitators have been trained Total number of community meetings organized in target Number of villages in which CBN meetings has been held communities this month Average number of participants in community meetings Average number of participants per month in CBN meetings Component Three Availability of annual report that summarizes key lessons Dropped from implementation, drawing on routine supervision and process evaluation Number of project-supported health centers in which staff have been fully trained on maternal, neonatal and children s health course* Number of community-based distributors who have been trained* Number of household visits by community-based distributors in last quarter* Number of Project-supported health centers which have received four (quarterly) supportive supervision visits in the last twelve months* Number of project-supported health centers in which at least one staff has been trained in 5 module training Number of community-based distributors who have received micronutrient training Dropped Number of health centers visited by the monitoring team from central and/or provincial and/or district level every 3 months (quarterly) * These were added to reflect activities undertaken with the additional financing from the previously closed co-financing (TF097071) from the European Union (EU). 1.4 Main Beneficiaries 9. The primary target group for the project was pregnant women and mothers of children under two years of age and their children (less than two years of age). Indirect beneficiaries included the central, provincial, district and health center (HC)/village level functionaries of the Ministry of Health (MOH) and Lao Women s Union (LWU) whose capacities were built as a result of project activities. 1.5 Original Components (as approved) The project had the following components: 10. Component 1, Conditional Cash Transfers (CCTs) to Stimulate Demand for Key Maternal and Child Health Services (US$1.2 million): Key activities under this component included: (a) CCTs to pregnant women and women with young children (under two years old) to stimulate uptake of services, the payments being conditional on uptake of a pre-determined service/ set of services such as, antenatal and post natal consultations, facility-based delivery, and well-baby consultations; and (b) reimbursements of service costs to the health facilities, making the service free for women and children at the point of service. 11. Component 2, Community-based Health and Nutrition Program (CBN) to Stimulate Behavior Change and Mobilize Mutual Support to Improve Nutrition (US$0.45 million): This component included: (a) conducting monthly group meetings convened by community selected local female facilitators to collectively identify key health and nutrition issues and risks, 3

15 identify possible actions to address those risks, and mobilize the community to implement these actions; and (b) training and support to the local female facilitators. 12. Component 3, Project Support and Management (US$0.35 million): This component financed incremental operating costs, as well as local and international technical assistance (TA) to design and implement the CCT and CBN programs, support financial management (FM), including performing internal audits and capacity development support to provinces and districts, as well as costs of an audit firm to perform annual audits of the project. 1.6 Revised Components 13. No component was revised. However, the project was co-financed by the EU Food Crisis Rapid Response Facility Trust Fund (FCRRF TF) of Euros 1.44 million (equivalent to US$2.12 million) which became effective on August 12, 2010 and closed on February 29, EU funds were used to increase the coverage of the CBN component of CNP from 150 to about 500 villages, training of health center (HC) staff, for supportive supervision activities, and for training and support for the community-based distributor (CBD) program. The EU funds also cofinanced TA for the CCT component. 1.7 Other significant changes 14. The project underwent two level-two project restructurings in response to emerging needs and significant implementation start-up delays. The first (Sept 19, 2011) to allow for funding to facilities and financial intermediaries (institutions providing free deliveries to women giving birth in these institutions) under the CCT component, and to amend the definition of incremental operating costs to include translation and software. The second (July 25, 2012) extended the closing date by 1 year to September 30, 2013, reallocated funds from consultant services and incremental operating costs for LWU to incremental operating costs for MOH, and revised the indicators to better align them to measure results of the activities and, in some instances, to harmonize indicator definitions with standard definitions for better comparability. 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 15. Sound background analysis supported project preparation. The problem analysis was holistic; it took cognizance of the complex causes of undernutrition and the full set of its determinants, including those beyond food insecurity. Given that the World Bank was at the time supporting efforts to improve food security in lowlands and livelihood security in the upper highlands, the project rightly focused on protecting and improving maternal and child care practices and use of maternal and child health (MCH) services, interventions known to improve MCH and nutrition (MCHN). 16. The rationale for World Bank involvement in preparing/undertaking the project was sound. Please refer to discussion of this point on page 1, section 1.1, Context at Appraisal. 17. It built on prior CCT experience in the country as well as drew upon experiences from other countries. The CNP design builds upon the successful CCT experience in the country to promote tuberculosis testing and treatment, supported by the Global Fund to Fight AIDS, Tuberculosis and Malaria. For registering beneficiaries, the project strengthened an 4

16 existing family system operating in some HCs. For the CBN component, the project drew upon experiences from Nepal to promote community campaigns to enhance uptake of health and nutrition services. 18. CNP preparation was prolonged due to unanticipated delays. Given that Lao PDR qualified for Global Food Response Project (GFRP) support and for the EU-supported FCRRF TF, the project capitalized on the opportunity offered by these resources to mitigate risks of the food price crisis to vulnerable households. However, despite the emergency nature of the project, several factors contributed to the 13-month preparation period, importantly the protracted discussions with the client to reach agreement on the CCT and CBN approach and on implementation and project management modalities, and the extensive consultations with other stakeholders to get inputs on the two approaches. Furthermore, there was a decision after the CNP concept approval to seek additional resources from FCRRF for the project. This meant developing another set of project documents and activities which while adding to project resources, required additional time. Due to a combination of the above factors, despite the 13- month long preparation phase, the team was compelled to defer the detailed design of the CCT and CBN components to the implementation phase. 19. Project design: The project design was strong and responsive to the situation and context, although somewhat ambitious for the short duration of the project. 20. The PDO is well aligned with the project s theory of change and the results chain. Global evidence suggests that improvements in MCH services and child caring practices contribute to improved maternal and child nutrition. The PDO appropriately addresses the key problem to be tackled by the project. The provinces selected represent some of the poorest parts of the country; therefore, the PDO also reflects responsiveness to a need to mitigate the adverse consequences of the food price increase as well as other macroeconomic shocks amongst those most vulnerable. However, the results framework and the large number of indicators were overambitious relative to timeframe and capacity of the client for implementation of the project. 21. Project components and activities reflect strong causal links with the PDO. The two technical components, CCTs to stimulate demand for key MCH services, and CBN to stimulate behavior change and mobilize mutual support to improve nutrition, directly support the PDO, while the project support and management component ensures the provision of necessary TA and operational support for the technical design and implementation of the Project. 22. The design leveraged as well as complemented the ongoing Health Services Improvement Project (HSIP). By co-locating the CNP in selected HSIP areas, the design of the project leveraged the MCH efforts of HSIP, such as safe deliveries in HCs and hospitals, and complemented as well as strengthened the integrated outreach from HCs, which had been identified as an area for strengthening in the HSIP mid-term review. 23. Wherever possible, the design was kept simple and flexible; it built in adequate technical support. Recognizing the widespread need to improve demand for MCH services and promote behavior change, and that the vast majority of households in the project catchment areas were poor and vulnerable, the project design rightly included all women and children in the project catchment area and did not undertake household level targeting, which would have been very complex. Further, while the design set the broad contours of the CCT and CBN components, it allowed for flexibility to undertake the detailed design during implementation to allow for sufficient consultation and bringing in international experience and expertise through 5

17 the third component. Hindsight suggests that deferring the design to the implementation phase in a low capacity setting caused significant delays and considerably shortened the implementation timeframe. 24. Risk assessment and mitigation measures were satisfactory. Only one safeguard, the indigenous peoples (4.10) was triggered and satisfactorily addressed. An Ethnic Group Development Framework (EGDF) was developed and a round of community consultations was conducted, findings from which were used to update the Ethnic Group Development Plan (EGDP) developed for the HSIP. Fiduciary risk was assessed of the HSIP fiduciary capacity, since, at preparation, the planned fiduciary arrangements were designed to draw upon the Project Management Unit (PMU) of HSIP (with some additional consultants to be appointed). FM and procurement risk was assessed as substantial and adequate mitigation measures were identified with several points included as legal covenants. However, during implementation, the anticipated sharing of HSIP fiduciary staff did not materialize, and alternative arrangement for FM and procurement had to be put in place which also contributed to delays. 2.2 Implementation (a) Factors that contributed to successful implementation of planned activities 25. Strong government commitment for health and nutrition sectors, and for the Project. As mentioned earlier, the MOH was committed to improving maternal, neonatal and child health (MNCH) services and nutrition outcomes as reflected through their then defined strategy documents (see 1.1.6). The MOH commitment to the project is reflected in their full engagement in the pilot, including their strong desire to implement the project through in-house technical team, and the LWU, and not through the consultant staffed PMU of HSIP. 26. Systematic and high-quality implementation support by the World Bank team. Despite implementation delays, proactive support of the task team, including the necessary restructuring of the project, contributed to the completion of all planned activities, and the achievement of intended results. Frequent dialogue and communication, timely, consistent and proactive implementation support by the task team helped resolve problems, overcome constraints, and speed-up implementation. Several core team members were based in-country, which helped. Additional technical support, including nutrition expertise was brought in as necessary, and a process documentation of the CBN component was commissioned to better understand and improve implementation. Project documents reflect that the team clearly recognized implementation issues and appropriately addressed them. 27. Mutual trust and respect between the client and World Bank teams and their shared commitment to resolve issues and meet emerging needs. Several examples illustrate the mutual commitment to monitoring issues, jointly solve problems and respond to the project needs on the ground. For example, agreement to use the additional financing resources (FCRRF) to strengthen supply-side constraints in the Lao health system, including those related to staffing, training, and supervision; developing videos to supplement printed tools for the Village Facilitators (VFs) in villages where language was a barrier to strengthen the CBN component. At mid-term review the teams reached agreement on several critical points, such as revising the results framework as necessary, simplifying the CCT component and harmonize it with the rollout of the MOH free MCH policy, and undertook the necessary restructuring of the project to effect the changes. It was only due to the joint efforts of both teams that the project was able to successfully clear the huge payment backlogs for the CCT component. 6

18 28. Leverage of the ongoing HSIP support. CNP benefitted from the relationships built at the central, provincial and district levels by the HSIP, the full familiarity with the health system including its strengths and weaknesses gained through HSIP experience, and having common team members on the two projects helped cross fertilize experiences between the projects. (b) Factors that constrained implementation 29. Inadequate readiness at entry and a very ambitious timeline. For reasons discussed in section 2.1, despite the 13-month preparation period, several critical design elements including the detailed design of the two technical components, implementation arrangements, funds flow mechanisms, criteria for selection of HCs and villages were not finalized and these tasks were deferred to the implementation phase. The planned timeline of three months during implementation to accomplish all these tasks, especially in a weak capacity setting, was highly ambitious. This, compounded by several other factors mentioned below, resulted in almost twothirds of the planned three year implementation period being taken up by design, HC selection, finalizing implementation arrangements, and funds flows arrangements, among others. 30. Newness of implementation arrangement and lack of experience and expertise, in particular procurement and FM at the central level. CNP represents the first attempt to integrate a World Bank-financed project into a technical department of MOH. The start-up delays were caused primarily by the relatively limited experience of the Department of Hygiene and Health Promotion (DHHP) with regard to setting up the institutional arrangement for procurement and FM-related support. While such an arrangement, whereby project management is integrated into the line Ministry does have a clear and remarkable return in terms of capacity building and sustainability of the investment, the relatively lengthy learning period negatively impacted project implementation. 31. The CCT implementation model for cash transfers to beneficiaries was very demanding on the HC staff. A specific constraint that slowed the pace of implementation was the challenge related to implementation of the CCT component due to MOH s decision not to use third party administrators. A second obstacle was the availability of only a paper-based information system. HCs did not have electronic information systems and all transactions related to the transfer of funds to HCs and to beneficiaries had to be paper-based. Limited availability of financial services (such as banks) in the project catchment areas implied that there was little choice but to integrate these functions under the responsibility of HCs. However, this created a large burden for the HC staff, who also were responsible for delivering health services. 32. Coordination challenges between DHHP and LWU, the two implementing partners. While at the central level there were initial difficulties and delays related to transfer of funds from MOH to LWU, the coordination between the two partners at the provincial and district levels continued to be challenging. While HCs were often the site for VF training and their staff assisted in the training, they were not involved in supervision of VFs, for which the LWU district staff was responsible, and the HC staff was left out of the loop. 33. Weak capacity at provincial, district, HC and village level. The low FM capacity at district and HC level, among other factors, contributed to the huge backlogs for the CCT payments, and the weak capacities of the VFs affected the quality of monthly meetings. Both necessitated additional training and supervision. However, the project had planned for only the initial set of training and refresher training had not been planned and could not be accommodated, especially for the VFs. The project though was responsive to the need and 7

19 strengthened supportive supervision to address some of these weaknesses. Although the LWU s network at the community level is vast, the number of LWU staff at provincial and district levels was small with limited capacity for supportive supervision, thus constraining supervision. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 34. A solid M& E design. The project paper defined a clear results framework with two sets of indicators and data sources one set to be used for project management purposes that derived data from the existing Health Management Information System; and the second set to evaluate the impact of the project with data collected through two rounds of household surveys (at baseline and endline). In order to credibly evaluate the impact of interventions and attribute the results to the project activities, the M&E design included intervention and non-intervention areas, although the areas were not randomized. The impact evaluation of CNP is ongoing and being conducted independently by the World Bank s Independent Evaluation Group (IEG). The endline data collection has been completed, the data are currently being processed and the evaluation is expected to be completed by June SMART 5 indicators, but too many of them and targets overly ambitious for the project duration. The indicators in the results framework meet the SMART indicator definition. However, as many as 11 PDO indicators and seven intermediate results indicators were originally proposed (and four intermediate indicators were further added with the additional financing from the EU) to measure selected processes, outputs and outcomes. The proposed results framework was far too ambitious; it committed to achieving many feeding and care behavior changes that did not appear feasible to attain within the short duration of the project. Global evidence suggests, and as also indicated by the qualitative assessment from the project, behavior change needs more intensive and sustained effort than had been planned for in the project. At the time of project approval, the results framework (being a requirement) had to be defined even though the detailed components and activities were deferred to the implementation phase. Once the components and activities were designed, the results framework was revised to simplify reporting requirements and to better capture the expected impact of the project in the short implementation timeframe. 36. Qualitative information to complement quantitative information. While the initial design focused only on quantitative information, during implementation a qualitative study and review of the CBN component was conducted, to better understand how processes were working (or not working) and what corrective action might be required. This provides valuable lessons for the component on what worked well and areas needing further strengthening. 37. Overall M&E arrangements were well implemented, although there were some data quality weaknesses with the monitoring system. Specific systems were established for monitoring the key aspects of the CCT and CBN components that included data collection, collation and use. The project built upon and integrated project monitoring needs into the existing MOH and LWU management information systems and reporting channels (rather than set up independent project systems), and it contributed to building M&E capacity and strengthening systems for data collection, collation, analysis and use. Some weaknesses in data 5 SMART is specific, measureable, achievable, relevant, and time-bound 8

20 collected through the system were noted, such as delayed and incomplete information, especially from remote and inaccessible locations, and variable quality of data due to weak capacities for reporting and data management. This was no surprise given the weak capacity and low starting base. However, data quality, completeness and timeliness showed improvements over time, suggesting that the project helped strengthen the systems M&E capacity. In addition, the project hired local consultants to travel to HCs and relevant district and provincial health offices to collect and compile quarterly administrative data for key indicators from the government's routine monitoring system. This likely further contributed to increased attention by the system to data collection and collation. For the evaluation, baseline and endline surveys for the impact evaluation were designed by the World Bank team, data collection was contracted to a professional international agency, while data analysis is being undertaken by the World Bank s IEG. 38. Information was well utilized for monitoring purposes. Good use was made of the data for project monitoring purposes, and to take corrective actions. Preliminary analysis of baseline survey information was regularly presented to the client during project supervision missions to enable a greater understanding about the challenges faced by the communities served. Costing and utilization models were informed by the outputs from this baseline survey. With regard to routine project M&E, two keys implementation issues were informed (and alerted) by the M&E system. First, the scale and causes for payment delays for the CCT were informed by the M&E system. Second, issues with reporting and supervision for CBN meetings were highlighted by the reporting system for CBN. 2.4 Safeguard and Fiduciary Compliance 39. Only one safeguard policy, OP/BP 4.10, Indigenous Peoples applied and was handled satisfactorily. The environment category was C, and only one safeguard, the Indigenous Peoples, applied. To ensure that ethnic groups living in project areas had equitable access to project benefits and to respond appropriately to the needs of these communities, the project developed an EGDF based upon the EGDP for the HSIP; the disclosure of which was a legal covenant. In line with the principle of free, prior and informed consultation, the project held consultation in six ethnically distinct villages (selected on purpose to include representation of distinct minorities) in a sample of two provinces from the project areas, output of which was incorporated into the EGDF. Further, the LWU and DHHP were required to carry out free, prior and informed consultations with a selected sample of affected ethnic groups in the project areas, after the selection of villages was finalized. These consultations were carried out satisfactorily by the implementing agencies, using participatory, process-oriented and culturally-sensitive methodology, and outputs were publicly disclosed. 40. Despite the weak fiduciary capacities, the project complied with fiduciary requirements. Recognizing the low fiduciary capacity of the implementing agency and the high need for appropriate FM and oversight, especially given that funds were to be handled and accounted for at the HC level, appropriate legal covenants, such as the appointment of FM and procurement specialist and FM consultants, the adoption of a project operational manual with FM and procurement sections, appointment of a qualified accounting firm for internal audit and to strengthen fiduciary capacity at provincial and district levels, were included in the Grant Agreement. These were complied with. Annual external audits were regular, there were no ineligible expenses and the management letters did not indicate any material weaknesses about the project s internal controls. 9

21 41. During implementation, due to the low procurement and FM capacity, the project faced FM and procurement challenges, including delays in appointment of consultants to develop the operational manual, the non-materialization of initial plans to use HSIP consultants to provide support, and problems with funds flows, among others. Regular support of the task team, including fiduciary specialists, ensured satisfactory resolution of the issues, including meeting the legal covenants mentioned above. For instance, in order to enable project implementation to proceed while contracting qualified fiduciary staff, an interim arrangement to share expertise from another World Bank financed project 6 was put in place. Further, the DHHP staff deputed for FM acquired FM and procurement capacity by working alongside these consultants (whose contracts mandated capacity building), and the World Bank team. 2.5 Post-completion Operation/Next Phase 42. Although the CNP was a small pilot program and its main objective was to test two approaches to improve the utilization of MCH services and promote positive maternal and child care behaviors, efforts were initiated before project closure to transition the CCT component to the ongoing HSIP additional financing (AF) project. A memorandum of understanding (MOU) was signed, and in five of the seven provinces the free deliveries element of the CCTs has been transitioned to HSIP AF; in the remaining two provinces, the government and other donors are planning to finance similar benefits. For the CBN component, a qualitative review undertaken by the project outlines many lessons and indicates the need to strengthen several aspects of outreach prior to scale-up, such as the need for greater capacity-building of the VFs, improving supervision and the need for a more intense and prolonged engagement to bring about behavior change than was possible during the short life of the project. Nonetheless, despite these shortcomings and the need for refinements, the project was able to influence community level behaviors and achieve its intended results. 43. The project interventions have great potential to improve the utilization of MCH services and improve household practices, and must be strengthened and supported further. There is a strong desire on the part of the client for such support as was clearly articulated at the highest levels in the nutrition division, DHHP and the MOH. A similar recommendation was also made by other donors supporting nutrition in the Lao PDR. Therefore, it is recommended that interventions to accelerate undernutrition reductions, particularly outreach efforts to promote behavior change around maternal and infant and young child feeding and care, should be incorporated into any follow-up health or nutrition project, since World Bank engagement in the sector remains critical to support country efforts to improve nutrition. 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 44. The PDO, design and implementation are very relevant in the current Lao PDR context. Not only do these remain relevant, they have assumed even greater relevance to the current priorities of the Government of Lao PDR, global priorities as well as to World Bank s Country Partnership Strategy (CPS) for the country. 45. Undernutrition continues to be a challenge in Lao PDR, despite significant reductions in poverty, infant and child mortality. While food insecurity remains high in many 6 The Avian and Human Influenza Control and Preparedness Project. 10