IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-97456) ON A GRANT IN THE AMOUNT OF US$40 MILLION TO THE REPUBLIC OF LIBERIA FOR A

Transcription

1 Public Disclosure Authorized Document of The World Bank Report No: ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-97456) Public Disclosure Authorized Public Disclosure Authorized ON A GRANT IN THE AMOUNT OF US$40 MILLION TO THE REPUBLIC OF LIBERIA FOR A GLOBAL PARTNERSHIP FOR EDUCATION GRANT FOR BASIC EDUCATION PROJECT May 28, 2017 Public Disclosure Authorized Education Global Practice Africa Region

2 CURRENCY EQUIVALENTS (Exchange Rate Effective [May 27, 2017]) Currency Unit = Liberian Dollar (LRD) LRD 1.00 = US$0.01 FISCAL YEAR January 1 December 31 CDD CF CSR DA DEO DP ECD ECE EFA-FTI EGRA EMIS EPF ERRP ESDC ESMF ESP EU EVD GER GDP GNI GoL GPE GPE BEP ICR IMF LACE LEGAF LPERP LTTP M&E MCSS MFDP MoE NER NGO ODA ABBREVIATIONS AND ACRONYMS Community-Driven Development Catalytic Fund Country Status Report Designated Account District Education Officer Development Partner Early Childhood Development Early Childhood Education Education for All-Fast Track Initiative Early Grade Reading Assessment Education Management Information System Education Pooled Fund Ebola Recovery and Resilience Program Education Sector Development Committee Environmental and Social Management Framework Education Sector Plan European Union Ebola Virus Disease Gross Enrollment Ratio Gross Domestic Product Gross National Income Government of Liberia Global Partnership for Education Global Partnership for Education Grant for Basic Education Project Implementation Completion and Results Report International Monetary Fund Liberia Agency for Community Empowerment Legal Vice Presidency Liberia Primary Education Recovery Program Liberia Teacher Training Program Monitoring and Evaluation Monrovia Consolidated School System Ministry of Finance and Development Planning Ministry of Education Net Enrollment Rate Nongovernmental Organization Overseas Development Assistance

3 PAD PDO PFMU PRSP PST PTA RF RPF RRA SBM SMC SMT TTL UNICEF USAID WASSCE Project Appraisal Document Project Development Objectives Project Financial Management Unit Poverty Reduction Strategy Paper Project Support Team Parent-Teacher Association Results Framework Resettlement Policy Framework Rapid Results Approach School-based Management School Management Committee Senior Management Team Task Team Leader United Nations Children's Fund United States Agency for International Development West African Senior School Certificate Examination Senior Global Practice Director: Jaime Saavedra Sector Manager: Halil Dundar Project Team Leader: Peter Darvas ICR Team Leader: Peter Darvas

4

5 LIBERIA Global Partnership for Education Grant Basic Education Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Grantee/Implementing Agencies/Donors Annex 1. Project Costs and Financing... i Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis... i Annex 4. Grant Preparation and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Annex 9. List of Supporting Documents Annex 10. Indicators under the Project MAP... 78

6

7 Data Sheet A. Basic Information Country: Liberia Project Name: Project ID: P L/C/TF Number(s): TF ICR Date: 05/28/2017 ICR Type: Core ICR Lending Instrument: Original Total Commitment: Revised Amount: Environmental Category: B Implementing Agencies: Ministry of Education Specific Investment Loan Grantee: Global Partnership for Education Grant for Basic Education Project REPUBLIC OF LIBERIA US$40.00 million Disbursed Amount: US$40.00 million US$40.00 million Co-financiers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/30/2009 Effectiveness: 12/29/ /31/2011 Appraisal: 02/26/2010 Restructuring(s): 11/01/ /06/ /28/ /02/ /27/2016 Approval: 07/28/2010 Mid-term Review: 03/19/ /28/2012 Closing: 06/30/ /31/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Grantee Performance: Moderately Satisfactory Substantial Moderately Satisfactory Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: i

8 Overall Bank Performance: Moderately Satisfactory Overall Borrower Performance: Moderately Satisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Performance any) Potential Problem Project at any time (Yes/No): Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: No Yes Moderately Satisfactory Quality at Entry (QEA): Quality of Supervision (QSA): None None Rating D. Sector and Theme Codes Original Actual Major Sector/Sector Public Administration Public administration - Education Education Secondary Education Primary Education Early Childhood Education Major Theme/Theme/Sub Theme Human Development and Gender Education o Access to Education o Education Financing Public Sector Management Public Administration 7 7 Urban and Rural Development Rural Development E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Henry G. R. Kerali Ishac Diwan Practice Manager/Manager: Halil Dundar Christopher J. Thomas Project Team Leader: Peter Darvas Nathalie Lahire ICR Team Leader: Peter Darvas ii

9 ICR Primary Author: Yijun Shen Kabira Namit F. Results Framework Analysis Original Project Development Objectives (from Project Appraisal Document) To increase access to basic education (with a particular focus on poor areas), improve conditions of teaching and learning, and improve school management and accountability. Revised Project Development Objectives (as approved by original approving authority) To improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school-based management. (a) PDO Indicator(s) Indicator Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Indicator 1: Direct project beneficiaries (number) - of which female (%) (Core Indicator) Value (quantitative or 0 470,000 (44%) 430,000 (44%) 483,565 (47%) qualitative) Date achieved 06/29/ /29/ /29/ /29/2016 Comments (including % Target exceeded. Source: EMIS data achievement) Indicator 2: Schools constructed according to standardized school construction guidelines under the project (%) Value (quantitative or 0% 75% 75% 100% qualitative) Date achieved 09/06/ /29/ /29/ /30/2016 Comments (including % Target exceeded. Source: Project Progress Reports and school construction final report. achievement) Procurement of textbooks and teacher guides for grades 5 9 verified under the project Indicator 3: (%) Value quantitative or 0% 100% 100% 100% qualitative) Date achieved 09/06/ /29/ /29/ /29/2016 Comments (including % achievement) Target achieved. Source: Project Progress Report and No-Objection provided by World Bank for the procurement. iii

10 Indicator 4: Distribution of textbooks and teacher guides for grades 5 9 verified under the project (%) Value (quantitative or 0% 85% 85% 98% qualitative) Date achieved 09/06/ /29/ /29/ /30/2016 Comments (including % Target exceeded. Source: Project Progress Report and log of textbook distribution. achievement) Indicator 5: Schools single signatories to own bank account Value quantitative or 0 1,000 2,579 qualitative) Date achieved 09/06/ /29/ /17/2014 Comments (including % achievement) Target exceeded. Source: Project Progress Report and List of Bank Accounts for transfers provided by the MoE. Indicator 6: Basic education completion rate (%) Value (quantitative or 49% 56% qualitative) Date achieved 06/29/ /29/2013 Comments (including % Indicator dropped during Level I restructuring in achievement) Indicator 7: Textbook-Student Ratio at basic education level (defined as number of textbooks aligned with new curriculum to students) Value (quantitative or 0:0 1:2 qualitative) Date achieved 06/29/ /29/2013 Comments (including % Indicator dropped during Level I restructuring in achievement) Indicator 8: Teachers having at least one teaching guide (%) Value (quantitative or 0% 80% qualitative) Date achieved 06/29/ /29/2013 Comments (including % Indicator dropped during Level I restructuring in achievement) Indicator 9: Schools managed by a School Management Committee (SMC) or Parent-Teacher Association (PTA) (%) Value (quantitative or qualitative) 0% 60% iv

11 Date achieved 06/29/ /29/2013 Comments (including % Indicator dropped during Level I restructuring in achievement) Indicator 10: School grants implemented in accordance with agreed upon procedures (%) Value (quantitative or 0% 60% qualitative) Date achieved 06/29/ /29/2013 Comments This indicator was moved to the intermediate results level (see Indicator 6 in the list (including % of intermediate results indicators below). achievement) (b) Intermediate Outcome Indicator(s) Indicator Indicator 1: Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Number of additional classrooms built or rehabilitated at the primary education level resulting from project intervention (number) Value (quantitative or qualitative) Date achieved 06/29/ /29/ /29/ /30/2016 Comments Target exceeded. Source: Project Progress Report and the School Construction Final (including % Reports. achievement) Indicator 2: Annual school construction database updated by annual Joint Sector Review Value (quantitative or qualitative) Yes Yes Yes Yes Date achieved 09/06/ /29/ /29/ /29/2016 Comments (including % achievement) Target achieved. Even though Joint Sector Review was not held in 2014 due to the onset of the Ebola Virus Disease (EVD) epidemic, the school construction database was updated that year. Source: Project Progress Reports. Standardized school construction guidelines for site selection, design, contract Indicator 3: management, and construction quality approved by MoE Senior Management Team (SMT) Value (quantitative or qualitative) No Yes Yes Date achieved 09/06/ /30/ /30/2013 Comments (including % achievement) Indicator 4: Value (quantitative Target achieved. An independent construction management company was hired to develop the standardized school construction guidelines. Source: School construction guidelines document. Teaching and learning materials utilization assessment study conducted and reported No Yes Yes No v

12 or qualitative) Date achieved 09/06/ /29/ /29/ /30/2016 Comments Target not achieved. However, at the time of writing, the fieldwork for this assessment (including % study has been conducted using other funds and the report is being drafted. achievement) Indicator 5: School Grant Guidelines approved by MoE Senior Management Team Value (quantitative or qualitative) No Yes Yes Date achieved 09/06/ /30/ /29/2013 Comments Target achieved. Source: project progress report and School Grant Guidelines (including % document. achievement) Indicator 6: School sub-grants implemented in accordance with School Grant Guidelines (%) Value (quantitative 0 30% 30% 72% or qualitative) Date achieved 09/06/ /29/ /29/ /29/2016 Comments (including % Target exceeded. Source: Monitoring Reports from NGOs and the MoE. achievement) Indicator 7: Social accountability framework for teaching /learning materials and school sub-grants programs developed and approved by MoE Senior Management Team Value (quantitative or qualitative) No Yes Yes Yes Date achieved 09/06/ /30/ /30/ /17/2014 Comments (including % Target achieved. Source: Social accountability framework document. achievement) Indicator 8: Joint Education Sector Review held Value (quantitative or qualitative) Yes Yes Yes Yes Date achieved 09/06/ /29/ /29/ /29/2016 Comments (including % Target was achieved in all years apart from 2014 due to the EVD epidemic. Source: Joint Education Sector Review documents. achievement) Indicator 9: Cost-effective Early Childhood Development model designed and piloted Value (quantitative or qualitative) No Yes Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 10: Children benefiting from deworming interventions Value 0 300,000 vi

13 (quantitative or qualitative) Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 11: Unified Human Resource (HR) database (Civil Service Agency, MoE, Ministry of Finance) developed and main Value (quantitative or qualitative) No Yes Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 12: Development of a pay and grade scale for teachers Value (quantitative or qualitative) No Yes Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 13: Availability of annual sector statistics by May of each year Value (quantitative or qualitative) No Yes Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 14: System for learning assessment at the primary level (rating scale) (Core Indicator) Value (quantitative or qualitative) No Yes Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) Indicator 15: DEOs using the new assessment instruments (%) Value (quantitative 0 75% or qualitative) Date achieved 06/29/ /29/2013 Comments (including % Dropped during Level I restructuring in 2012 as activity was dropped. achievement) G. Ratings of Project Performance in ISRs vii

14 No. Date ISR Actual Disbursements DO IP Archived (US$, millions) 1 03/26/2011 Satisfactory Moderately Satisfactory /25/2011 Satisfactory Moderately Satisfactory /17/2012 Moderately Unsatisfactory Unsatisfactory /08/2012 Moderately Satisfactory Moderately Satisfactory /26/2013 Moderately Satisfactory Moderately Satisfactory /30/2013 Moderately Satisfactory Moderately Satisfactory /30/2014 Moderately Unsatisfactory Moderately Unsatisfactory /26/2014 Moderately Unsatisfactory Moderately Unsatisfactory /29/2015 Moderately Satisfactory Moderately Satisfactory /31/2015 Moderately Satisfactory Moderately Satisfactory /27/2016 Moderately Satisfactory Moderately Satisfactory /28/2016 Moderately Satisfactory Moderately Satisfactory H. Restructuring (if any) Restructuring Date(s) Board Approved PDO Change ISR Ratings at Restructuring DO Amount Disbursed at Restructuring in US$, millions 11/01/2011 N S MS /06/2012 Y MU U 7.26 IP Reason for Restructuring and Key Changes Made As there were no existing buildings that could be used for the purpose of early learning centers, a Level II restructuring was undertaken in November 2011, to include the construction of early learning centers, as opposed to the minor rehabilitation and renovation of existing building for their use as Early Childhood Development (ECD) centers. Project implementation and management capacity were weaker than initially assessed, and a complex project design had posed ongoing challenges. Restructuring included (a) revisions to the PDO, (b) a reduction in the number of project components, and (c) modifications to the Results Framework to better align the indicators with the revised PDO. viii

15 02/28/2013 N MS MS /02/2015 N MU MU /27/2016 N MS MS Based on progress made by the time the Level I restructuring was approved in 2012, it was clear that major activities associated with each of the three major project areas school construction, learning materials, and school sub-grants would not be completed by June 30, The Level II restructuring extended the project's closing date from June 30, 2013, to June 29, Due to the EVD outbreak, many project activities were forced to slow down or came to a complete halt. A Level II restructuring extended the project for a year from June 29, 2015, to June 29, 2016, to allow for the completion of school construction and distribution of learning materials. In addition, US$2.7 million was redirected from the school grants subcomponent to school construction due to cost overrun. In June 2016, a Level II restructuring was approved to further extend the project's closing date from June 29, 2016, to October 31, 2016, to ensure the completion of all school construction activities planned under the project. If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Unsatisfactory Against Formally Revised PDO/Targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory ix

16 I. Disbursement Profile x

17 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country Context 1. In 2010, when the Education for All - Fast Track Initiative (EFA-FTI) (later known as the Global Partnership for Education) Grant for Basic Education Project [GPE BEP] [P117662] was under preparation, Liberia was a country of 3.47 million people. The 14-year civil war ( ) had left the country ravaged, with 75 percent of the population living on less than US$1 a day and 90 percent considered to be food insecure. 1 Nevertheless, the country showed important signs of moving forward. In the years following the civil war, Liberia had made significant political, economic, and social progress. A democratically elected government had come to power in A more stabilized political and economic environment facilitated investments in rebuilding infrastructure and the restoration and development of public services. The labor market was rebounding slowly as skilled and educated citizens from the diaspora returned home. The country s economic performance was broadly satisfactory with a gross domestic product (GDP) growth rate averaging 7 percent between 2006 and Despite this growth, Liberia s postwar recovery was precarious. The country ranked 182nd, in the bottom quintile, on the United Nations Human Development Index, reflecting continued weak basic services. The economy remained especially vulnerable to external shocks. This issue was glaringly evident when the country experienced rapid food and fuel price inflation during the first half of This resulted in GDP growth to dipping to 4.6 percent in 2009, and was the reason for the World Bank extending additional support to Liberia through the Liberia Emergency Food Crisis Response Program. 3. The Government had a vision for improving the Liberian quality of life and thus had developed a Poverty Reduction Strategy Paper (PRSP) focused on rapid pro-poor growth. The PRSP featured education, together with job creation, infrastructure, and health, among the highest priorities. Concurrently, partners such as the United States Agency for International Development (USAID), European Union (EU), the World Bank, and the United Nations Children s Fund (UNICEF) were scaling up their engagement in the country, mobilizing and collectively supporting the Government s vision for the first time in many decades. Sector Context 4. At the time of project preparation and throughout most of this project s implementation phase, the education service delivery was in the process of shifting from a focus on the provision of emergency educational services to the provision of accessible, quality-based services premised on a stronger education system. In 2010, approximately half a million students were studying at the primary level (539,887). The Gross Enrollment Ratio (GER) stood at 94 percent while the Net Enrollment Rate (NER) was 35 percent. Primary challenges confronting the education system included a lack of adequate education infrastructure, and difficulties in implementing sector-wide reforms while concurrently providing education of reasonable quality to citizens. 1 Poverty Reduction Strategy

18 5. In 2010, Liberia s Ministry of Education (MoE) prepared a 10-year Education Sector Plan (ESP) , building on the analysis of the comprehensive Country Status Report (CSR) and extensive consultations with donors and other stakeholders to the education sector. Liberia s ESP laid out a number of critical challenges to be addressed: a) Lack of adequate resources. The Government of Liberia (GoL) had abolished school fees in public primary schools in 2006, which increased accessibility and spurred enrollment in primary education, particularly for girls and children from poor and disadvantaged households. However, increased enrollment had not been accompanied by substantial increase in resource flows to the sector, resulting in strain on the education system, depreciation in quality of services, and poor education outcomes. b) Lack of school infrastructure. At the time of project design, Liberia had an insufficient supply of school infrastructure. In many rural areas, communities lacked access to a school building, or the existing school building had deteriorated to the extent that it was no longer fit for use. Only 22 percent of public and community schools had seats, and only one-third of public and community schools had functioning pit latrines or flush toilets. According to the PRSP, the ratio of public and community school students to classrooms in adequate condition was more than 300:1. c) Insufficient monitoring and supervision. The decentralized offices of the MoE, which were created after the civil war, lacked financial resources and qualified personnel to adequately monitor and report on schools in their districts. Many staff demonstrated low levels of experience in providing pedagogical support, further compromised by a general absence of direction from the center with regard to their supervisory and reporting roles and responsibilities. Additionally, the challenge of ghost workers on the MoE payroll and a weak Education Management Information System (EMIS) with delayed or unutilized data further contributed to a challenging monitoring, planning, and budgeting environment. 6. In line with the country s PRSP, the ESP focused on the rehabilitation and expansion of infrastructure to facilitate pro-poor growth, expanding preprimary education services, effective management of resources at the school level, and other priorities. The ESP outlined short-, medium-, and long-term government strategies to address challenges confronting the education sector, and was accompanied by a three-year implementation plan. In this context, the GPE BEP was envisioned as the vehicle to help implement the ESP s strategy and the Government s agenda for the sector. Project Context 7. In 2007, Liberia s request for funding support from the EFA-FTI to the education sector was not successful. To offset this outcome, donor partners established the Education Pooled Fund (EPF), managed by UNICEF. The EPF set up institutional parameters for donor engagement in the sector and financially supported the Government-led Liberia Primary Education Recovery Program (LPERP). LPERP had a number of demonstrated outcomes, including the construction of 198 additional schools, the renovation of 58 schools, the procurement of 1.2 million textbooks, and distribution of textbooks in the 2009/2010 school year. Also during this time, to enable greater 2

19 understanding of challenges facing the education sector, partners such as USAID, the EU, and the World Bank supported a number of analytical studies, including an Early Grade Reading Assessment (EGRA) in grades 2 and 3 in 180 schools, and a school census capturing data basic enrollment and infrastructure data for Operational and technical lessons from these interventions provided the foundation for implementing the next phase of Liberia s education development and reform agenda. 8. With the aim of moving that agenda forward, the GPE BEP was developed in 2011 as a flagship project to implement the initial stages of the ESP between 2011 and The GPE BEP was intended to respond to the most immediate challenges in basic education and enable longer-term solutions (capacity building). As the first operation in education sector in the postwar period that was financially and operationally managed by the Government (with the World Bank as the grant agency), the GPE BEP was designed to increase access to education especially for poor communities and to tackle shortcomings in teaching and learning while improving management and accountability in the sector. The project set out to achieve these ends by supporting a number of initiatives highlighted in the ESP, including the construction of schools, the provision of additional learning and teaching materials, school grants, inputs to improve school health, the expansion of early childhood development (ECD), and capacity building. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 9. The original PDO according to the Grant Agreement was to increase access to basic education (with a particular focus on poor areas), improve conditions of teaching and learning, and improve school management and accountability. 2 It was to be measured by six PDO indicators, as listed in table 1: Table 1. Original PDO, Baseline, and Targets PDO Indicators Baseline 2008 Target 2012/13 Basic education completion rate (%) Direct project beneficiaries (number) - of which female (%) (Core Indicator) 0 470,000 Textbook-Student Ratio at basic education level (defined as number of textbooks aligned with new curriculum to students) 0:0 1:2 Teachers having at least one teaching guide (%) 0 80 Schools managed by a School Management Committee (SMC) or Parent-Teacher Association (PTA) (%) 0 60 School grants implemented in accordance with agreed upon procedures (%) 0 60 Source: Project Appraisal Document. 1.3 Revised PDO, Key Indicators, and Reasons/Justification 10. A Level I restructuring, approved in September 2012, introduced changes to both the PDO and PDO-level indicators. The PDO was revised to improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement 2 There is no material difference between the PDO in the Project Appraisal Document and in the Grant Agreement. 3

20 and distribution of teaching and learning materials, and school-based management. As a consequence, the PDO-level indicators and targets were modified, as reflected in table 2. Table 2. Revised PDO-level Indicators, Baseline, and Targets Revised PDO Indicators Baseline 2008 Direct project beneficiaries (number) - of which female (%) Schools constructed according to standardized school construction guidelines under the project (%) Procurement of textbooks and teacher guides for grades 5 9 verified by September 2013 under the project (%) Distribution of textbooks and teacher guides for grades 5 9 verified by December 2013 under the project (%) Schools single signatories to own bank account Reasons Driving the Level I restructuring Cumulative Target Value 2011 Y1 Y2 591,000 (44%) 591,000 (44%) Y3 (End Target) 473,000 (44%) % 75% % 100% % , The September 2012 restructuring set out to address the following: a) Education Reform Act of The passage of the new Education Reform Act in August 2011 (eight months into the implementation phase) altered the institutional structure of basic education service delivery by decentralizing many functions. This created a disconnect between the GPE BEP s design and the MoE s strategic priorities. b) Weak implementation capacity. Project implementation and management capacity was weaker than initially assessed at appraisal. As a consequence, activities in the first year following effectiveness focused primarily on building the capacity of the Project Support Team (PST), including the contracting of a project coordinator and financial management and a procurement specialist for project planning and implementation monitoring. Disbursement was low, with only US$1 million expensed (out of the first tranche of US$6.7 million) in the first year. c) Complicated project design. The project was spread across too many thematic areas. While the project consisted of only had three components, each component required the implementation of multiple diverging activities that could be either stand-alone or subcomponents. In many instances the PST, and those responsible for the implementation of subcomponents, were not always aware of their responsibilities. d) Overlapping of initiatives. In late 2011, with support from the Open Society Foundation, the MoE hired a Donor Coordinator who identified substantial overlap in donor-supported projects, including the GPE BEP in the areas of ECD, school health,

21 systems development for monitoring and evaluation (M&E), and personnel/payroll management. While this was detrimental to the project s implementation, it could reflect donors supporting the Government s strategy and aligning their initiatives under the ESP. e) Political transitions. Presidential elections in October 2011 led to changes in political leadership across a range of ministries, including the MoE. This transition, although anticipated, slowed implementation as newly appointed ministry personnel (the Senior Management Team [SMT]) required time to understand and get up to speed with regard to the project. 12. A Level II restructuring in June 2015 introduced further but minor changes to the PDOlevel indicators, described in table 3. Table 3. Revised PDO-level Indicators, Baseline and Targets PDO Indicators Target June 29, Baseline Direct project beneficiaries (number) - of which female (%) 0 430,000 (44%) Schools constructed according to standardized school 0 75% Procurement of textbooks and teacher guides for grades 5 9 verified under the project 0 100% Distribution of textbooks and teacher guides for grades 5 9 verified under the project 0 85% Schools single signatories to own bank account 0 1, Main Beneficiaries 13. The project was originally expected to benefit 470,000 students, most of whom would enroll in schools constructed in targeted areas, as well as students in grades 1 through 9 who would benefit from school grants. Following restructuring in 2012, the beneficiary indicator was revised to 591,000 by 2013, 591,000 by 2014, and 473,000 by 2015, calculating those benefiting from school grants. The projected decline in the number of beneficiaries in 2015 was premised on the notion that as the Government s share of school grant financing increased, more students would be under that umbrella and therefore, would not directly benefit from the project. This beneficiary target number was further reduced to 430,000 during restructuring in 2015 to reflect a fall in enrollment following the Ebola Virus Disease (EVD) crisis. 1.5 Original Components (as approved) 14. The original project included three components. 3 Component 1: Increasing access and equity in rural areas (US$15.5 million) 15. This component s objective was to increase the supply of education in rural areas. It was thus designed to support the construction and/or rehabilitation of classrooms, administration 3 According to the Project Appraisal Document (PAD) and Grant Agreement September Where there are differences, the Grant Agreement is used. 5

22 facilities, latrines, and water points in the selected primary and junior secondary schools. Accounting for special challenges of remote and hard-to-reach areas, the component had two approaches: (a) a delegated management approach whereby a firm would handle the management and supervision of the construction program (up to 24 primary schools and 20 junior high schools); and (b) a community-driven development (CDD) approach focused on communities selecting and managing local contractors (up to 16 primary schools in hard-to-reach areas). Component 2: Improving the conditions of teaching and learning (US$14 million) 16. This component aimed to enhance the conditions of teaching and learning through complementary activities. It was designed to finance: (a) (b) (c) (d) (e) Provision of learning materials to both students and teachers (approximately 1 million textbooks for students in grades 5 to 9 of basic education and approximately 20,000 sets of teacher guides); Provision of sub-grants to schools after a rigorous review of the existing system; Support to ECD, including development of an ECD strategy and design and implementation of a cost-effective ECD model for rural areas, and upgrading of community-based ECD centers in rural markets and of low-cost structures in villages; Preparation of quality standards and guidelines for ECD service delivery, childcentered curriculum, and teacher training and certification frameworks; and School health, including development and adaptation of life-skills student modules for grade 1 9 students, deworming for approximately 300,000 primary students, and training and sensitization materials for teachers and the community in the selected counties and training of master trainers and teachers in the selected counties on deworming. Component 3: Strengthening central and local levels to operate in a decentralized education system by developing management and monitoring and evaluation capacities (US$10.5 million) 17. This component s objective was to strengthen the MoE structures at central and school levels and strengthen personnel and payroll management. Its envisioned activities included: (a) (b) The establishment of a PST to assist with the day-to-day management of the project including a project coordinator and a financial management, and a procurement specialist; Capacity building and training for the Departments of Planning, Administration, and Instruction; school construction management and supervision at the central level; M&E and tracking of school-level data for district education officers (DEOs) and school directors; use of learning assessment instruments as a monitoring tool for early grade reading and mathematics performance by DEOs and teachers; and financial 6

23 management for the Parent-Teacher Associations (PTAs) or School Management Committees (SMCs); (c) (d) (e) Preparation and implementation of a comprehensive M&E system and improved planning tools; Provision of advisory services for the development of a national assessment test for grades 4 and 7, focusing on mathematics and reading comprehension; and Strengthening personnel and payroll management through provision of advisory services to reinforce the personnel and payroll management activities of the MoE and the carrying out studies and surveys to evaluate teacher motivation and working conditions. 1.6 Revised Components 18. The GPE BEP underwent five rounds of restructurings before its revised closing date in October As the landscape of the sector changed, the project shifted focus to improving management capacity and accountability as a means of advancing implementation and ensuring better results. Subcomponents were introduced while some components were dropped to account for increased donor engagement in these areas (for example, UNICEF started working in early childhood education [ECE]). The first and most significant restructuring (Level I) took place in September 2012, when changes were made to the PDO, indicators, and components. Component 1: Improving Management Capacity and Accountability 19. Component 1 activities were modified and subcomponents were introduced. Subcomponent 1.1: Strengthened Institutional Management Capacity and Accountability at the Central and School Levels (US$4.6 million in addition to the US$0.9 million already disbursed on dropped activities) 20. Activities included the following: (a) (b) Establishment of the PST under the MoE s Department of Planning, Research, and Development/Center for Education Management Capacity building and training at the central and school levels for (i) Annual Joint Education Sector Reviews; (ii) Management and supervision of project subcomponents, including information, education, and communication campaigns; and (iii) M&E of project subcomponents. 7

24 (c) Provision of advisory services to the MoE to conduct an organizational development assessment and functional review at the central level 4 Subcomponent 1.2: School Construction in Rural Areas (US$16.2 million) 21. Activities focused on capacity building in the planning, monitoring, and overall management of school construction. The number of ECD centers and primary and lower secondary schools to be built, remained the same. (a) (b) 20 basic education sites (each with a primary and lower secondary school) and four primary sites with six-classroom schools that were to be supervised by a School Construction Management Firm. 16 primary sites with three-classroom schools (eight of which were to have an ECD center) built through a CDD approach (also known as Community Participation in Procurement approach, for the purposes of the Procurement Plan). 22. Standardized school construction guidelines for site selection, design, contract management, and construction quality were to be developed and approved by the MoE SMT. Each school built under the project was to adhere to standards set in school construction guidelines. Capacity-building activities included upstream tasks such as developing a school construction database and post-construction tasks such as establishing a set of school maintenance guidelines. Subcomponent 1.3: Procurement and Distribution of Teaching and Learning Materials (US$11.0 million) 23. Subcomponent 1.3 focused on improving the processes through which learning materials were procured and distributed, and supporting the MoE in monitoring the utilization of learning materials in the classroom. The number of learning materials to be procured and distributed remained the same. (a) Approximately 1 million textbooks in core subjects for grades 5 9 (b) (c) 20,000 teachers guides Supplementary learning materials for grades 1 9 and storage containers Subcomponent 1.4: School-based Management through School Grants (US$7.3 million) 24. As a means of enhancing community participation in school based management, this subcomponent was to finance the development of School Grant Guidelines, including a formula for determining the sub-grant amount and specifics regarding approved uses for the funds. Because schools would face difficulties meeting the sub-grant eligibility criteria (as outlined in the Financing Agreement), the project was to support contracting local nongovernmental organizations (NGOs) that worked with schools to help them meet the eligibility criteria. These NGOs were also expected to carry out support tasks related to the distribution of teaching and 4 According to revisions made to the Grant Agreement in September

25 learning materials under Subcomponent 1.3. Their role was expected to diminish over time, as more schools graduated from their training program to become the sole signatories for the bank accounts. Thus, transfer of skills was a critical design aspect of this subcomponent. The NGOs were required to submit an expenditure statement to the Project Financial Management Unit (PFMU) in the Ministry of Finance. Regular audits were to be conducted on withdrawals and use of the sub-grant funds. Where NGOs were to become cosignatories, their financial management capacity was to be assessed intermittently. 1.7 Other Significant Changes (in chronological order) November 2011 (Level II Restructuring) 25. The project originally proposed the use of existing education infrastructure to house community-based ECD centers. However, in some instances, it was discovered that there were no buildings suitable to house ECD centers and, as a consequence, new infrastructure had to be built. The project was restructured in November 2011 to include the construction of new early childhood learning centers, as opposed to the minor rehabilitation and renovation of existing buildings stipulated in the original PAD. September 2012 (Level I Restructuring) 26. See previous section for full description. February 2013 (Level II Restructuring) 27. Despite the Level I restructuring and streamlining of implementation and accountability, it became clear that some activities school construction, learning materials, and school sub-grants would not be completed by June 30, On November 2, 2012, the GoL requested an extension to the project beyond June 30, In February 2013, a second Level II restructuring was approved, extending the project s closing date by 24 months to June 29, June 2015 (Level II Restructuring) 28. The EVD epidemic that ravaged West African countries in had a devastating impact on Liberia, leading to the imposition of travel curfews and the closing of schools for nearly eight months. As a result of the impact of the crisis, activities such as the national rollout of school grants, the pilot distribution of learning materials, and school construction (132 classrooms built as compared to the target 272) incurred significant delays. Concurrently, restrictions on movement and the impact of quarantine protocols undermined normal business activities and drove up project costs. 29. In June 2015, further Level II restructuring extended the project s closing date by one year to June 29, This additional time was intended to allow for the completion of activities critical to the achievement of the PDO (that is, school construction and distribution of learning materials). 30. Restructuring also reallocated US$2.7 million of monies initially allocated to grant funds from Category 2 (school grants) to Category 1 (school construction). This reallocation addressed 9

26 cost overruns related to the Ebola outbreak and the application of a standardized school design in school construction. June 2016 (Level II Restructuring) 31. Restructuring in June 2016 further extended the project s closing date by an additional four months to October 31, 2016, to allow for the completion of activities including school construction and the distribution of teaching and learning materials. No changes to components or indicators were introduced at this time. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 32. Project preparation. Technical design of the GPE BEP was premised on data and analysis presented in the 2010 Liberia Education CSR a comprehensive overview of key challenges in the sector and incorporated lessons learned through activities related to the EPF. Project priorities were aligned with Government s sector strategy and were expected to affect reforms that would lay the foundation for the education sector. Project preparation included extensive consultation with various stakeholder groups in line with EFA-FTI guidelines and World Bank practices. 33. Government commitment seems to have been strong as the GPE BEP was the first donorfunded project to be implemented by the MoE since the war. Additionally, project was intended to operationalize the country s ESP. The Government had signaled commitment to the project as early as 2009 when it submitted a formal request to the EFA-FTI for funding support. 34. A number of lessons learned from previous interventions in the education sector notably LPERP, EPF, and the World Bank s Community Empowerment Project were incorporated in project design. For example, the EPF disbursed school grants at the county level, allowing for bulk procurement of learning materials and delivery to respective schools. Accountability was weak at the county level leading to delays and anecdotal evidence of leakages. Learning from the experience of administering school grants under the EPF, the GPE BEP allowed for greater decentralized autonomy through the disbursement of school grants directly to schools. 35. The textbook distribution chain used by the EPF led to a long procurement cycle; was undermined by poor management capacity for the storage and distribution of materials; and an absence of accountability mechanisms at the county, district, and school levels. To ensure materials reached the intended beneficiaries, the GPE BEP proposed (and financed) the delivery of learning materials directly to districts or classrooms. 36. Scope of the project. The original design of the project was based on the Education Country Status Report and subsequent ESP. The amount allocated (US$40 million) was significant equivalent to approximately 300 percent of the annual public education budget. It was anticipated that a large infusion of resources, coupled with relevant technical assistance, would lead to significant reform in priority areas targeted by the project. 10

27 37. The project suffered from a lack of selectivity. This may have been a consequence of the Government wanting to set ambitious goals for implementation in the first phase of its ESP. It may also have been a by-product of the EFA-FTI mandated consultative approach, which greatly increases ownership but occasionally leads to stakeholders advocating for a proliferation of initiatives, leaving the Government little room for selectivity. 38. In hindsight, greater selectivity in project components would have reduced risks in a fragile environment characterized by poor human resource capacity. Presentations of lessons from outside Liberia may have assisted the Government and World Bank to push back and deliver a more selective approach. It is not clear, however, that the enabling political environment at the time would have allowed for further selectivity. Liberia had previously been rejected by the GPE and was under pressure to present a catalytic program that would leverage further donor support. 39. From a technical standpoint, the project aimed to intervene in a range of discrete areas (see section 1.5) including school construction, textbook distribution, school grant, deworming, ECD, payroll management, national assessment, and capacity building. Moreover, many activities were insufficiently resourced to reach economies of scale. The GPE-mandated implementation time frame of three years was not sufficient to achieve the targets set during project design. Furthermore, a number of elements that were lacking at the design stage did not materialize as expected once implementation was under way. Examples of this include, inter alia, the effectiveness of the textbook distribution network, Government design standards and guidelines for construction of schools, and managerial and operational capacity. 40. Many activities incorporated into the original project design were found to overlap with, or be duplicated by, interventions supported by other development partners (DPs). This may have been a consequence of donor intentions not communicated or donor agendas not yet finalized at the time of project preparation. 41. Project risks. Substantial risks were identified and mitigation strategies proposed in the PAD and subsequent restructurings: (a) (b) (c) Political risks included the security environment and expected political transition, although these were considered to be largely beyond the scope of the project which limited the capacity for the integration of mitigation measures. In the delivery of grants, substantial leakage risks were identified. The mitigation strategy proposed was to ensure funds would not be transferred to schools until the system of school grants had been strengthened and School Grant Guidelines developed. These measures worked to reduce leakage, but added longer-thananticipated implementation delays due to the requirement for additional human resource and financial management capacity at the MoE. Poor institutional capacity within the MoE was cited as a potential impediment to the project s success. To offset this, activities were included to strengthen MoE capacity in areas specifically related to the project. This approach was further developed during the Level I restructuring in September

28 (d) (e) Procurement and fiduciary risks were appropriately identified at appraisal and proposed measures were incorporated. Procurement management and financial processes were assigned to the PFMU of the Ministry of Finance and Development Planning (MFDP) with the possible transfer to the MoE following a midterm review and assessment by the World Bank. Appropriate environmental and social safeguard management tools were developed and shared in accordance with World Bank guidelines. This included an Environmental and Social Management Framework (ESMF) and a Resettlement Policy Framework (RPF). Though there seems to be little reporting on the subject in the project s documentation, there is also no indication that environmental or social issues arose during project implementation. 42. Quality at entry. Project design was underpinned by a comprehensive analysis of sector needs and priorities, a ten-year ESP, consultation with various partners, applied experience and lessons learned from previous projects, and, for the most part, project design accurately identified relevant risks and incorporated appropriate mitigation measures. At the time, the GPE BEP was among the first projects subject to quality assurance reviews by both the GPE and the World Bank and input on design and processes seems to have been incorporated. Nevertheless, the complexity of project design jeopardized the project s potential success in completing activities envisioned in an initial implementation cycle of three years. Based on this analysis, Quality at Entry is rated Moderately Satisfactory. 2.2 Implementation 43. This section provides descriptive information of key factors influencing the implementation. Political Environment 44. The project benefited from a stable macro-political environment throughout implementation. Political factors that contributed to slowed implementation in the first year included the passage of the Education Reform Act in August 2011 and the transition of the MoE s SMT following the October 2011 elections. Multiple reshuffling of political appointees in government disrupted project implementation. Capacity Constraints 45. The use of existing government structures was a strategic feature of the operation. By leveraging existing mechanisms such as the Unit of Environment Facilities for school construction activity, the Bureau of Curriculum for textbook procurement activity, Grants and Coordination in the Department of Planning for school grant activity, and the MoE Procurement Department for all procurement-related activities, the project fostered strong project ownership within the MoE. This was a shift from previous donor-funded projects in Liberia which did not always work through government systems. 12

29 46. The World Bank helped the Government to carry out Rapid Results Approach (RRA) workshops 5 to improve understanding of the technical and operational chains of the project. In addition, a Level I restructuring was undertaken to reprioritize and narrow the scope of the project, with a focus on building capacity in the MoE to manage key sector activities in the context characterized by decentralization of basic education service delivery. As a result, the project shifted focus to (a) (b) (c) (d) Strengthening institutional capacity in support of improved management and accountability at the central and school level; School construction in rural areas; The procurement and distribution of teaching and learning materials; and School-based management supported by school grants. 47. Service providers or contractors were used where necessary. However, some contracted entities had limited capacity. For example, bids submitted by local construction contractors demonstrated significant variance in quality, and there were insufficient numbers of qualifying bids for awarding the contracts. To address this shortfall, the PST used Direct Contracting and Shopping methods for the tendering of certain construction lots. Moreover, high-performing contractors identified through Phase I construction were rewarded with contracts in Phase II. 48. The anticipated cost of service providers proved to be higher than that calculated at appraisal. In general, the hardship level associated with operating in Liberia was high, which made the country unattractive to international candidates. Filling key positions in the PST, such as procurement specialist and project accountants, proved to be particularly challenging, delaying implementation of related activities for months. Moreover, the use of NGOs for activities that involved traveling through difficult terrain was costly. The project made efforts to alter course where feasible with the aim of reducing costs while maintaining implementation momentum. For example, the MoE initially planned to recruit NGOs to manage the distribution of learning materials from the counties to the schools and to support the school grants program. These plans were abandoned due to the high costs quoted by candidate NGOs. The cost of services provided by the construction supervising company was so high that the MoE took the decision to assume the responsibility of supervising construction sites when only a few sites were left unfinished (in hindsight, this brought down costs but did not guarantee sufficient quality control with regard to construction, particularly in hard-to-reach counties). Government 49. Political commitment to the project on the part of Government was strong throughout implementation, although operational capacity was weakened due changes in leadership. Financially, the Government demonstrated commitment to sustaining project activities such as the school grants according to the restructuring undertaken in 2012, the project would cover The World Bank s RRA helps leaders engage operational teams in government to achieve tangible results in 100 days. In the process, the RRA reveals institutional bottlenecks, and diagnosing and removing these can help make a government more effective. 13

30 percent of school grants costs in Year 1, with the burden shifting to the Government by 25 percent each year thereafter, until full financial responsibility was transferred. The Government took over the responsibility a year ahead of schedule. 6 Implementation Support 50. The project benefited from dedicated and constant World Bank support from both locally based staff 7 and headquarters. An average of four missions took place per year during preparation and implementation, and only slowed down during the height of the Ebola outbreak. 8 Aide Memoires document extensive discussions and agreements on action steps to expedite implementation. The first restructuring (Level II) took place 11 months after effectiveness and an early midterm review was conducted one year into implementation to assess design and inform the Level I restructuring. MoE and the World Bank task teams, in conjunction with DPs, convened extensive technical discussions on the proposed restructuring of project design. Quality Enhancement Review by the World Bank was conducted for the restructured project. It is worth noting that due to the fact that Liberia was not a family station, and that the World Bank did not have a significant portfolio in Liberia, implementation supervision was limited to missions with on the ground supervision constituting only a short portion of the total life of the project. Harmonization with DPs 51. Largely driven by World Bank and GPE guidelines, consultation with DPs was extensive during the preparation phase of the project. However, coordination among DPs during project implementation could have been stronger and could have allowed for a more effective division of tasks across the sector. Project activities that overlapped with other donor-financed initiatives were dropped during restructuring allowing for more effective harmonization among DPs. For example, information on the quality and availability of construction companies and engineers was shared between the GPE BEP and UNICEF, allowing for improved coordination of school construction. The MoE also successfully led annual Joint Education Sector Reviews that facilitated information sharing and coordination between DPs. Progress toward project completion was reported in meetings of the Education Sector Development Committee (ESDC) 9, although it is not clear from documentation how often reporting took place. The EVD Epidemic 52. The EVD epidemic led to significant delays in project implementation. On August 6, 2014, the President declared a three-month state of emergency that included the introduction of a curfew, the closure of schools and some markets, and restrictions on the movement of people and gatherings. It was not until early 2015 that the spread of EVD began to diminish. Critical activities such as the national rollout of school grants, the pilot distribution of learning materials, and school 6 Due to late introduction of school grant under the project, it was expected that the school grant would be given out only in two years, as a result there was not enough time for the phased approach to materialize. The reduction in rounds of school grants also generated surplus in school grant component. Concurrently the school construction experienced cost overrun, thus US$2.7 million was reallocated from the school grant to school construction during the Level II restructuring in June The locally based education specialist was hired only for a portion of the total project life. 8 During the EVD, video and audio conferences were held in replacement of in person mission. 9 EDSC meet monthly. 14

31 construction were slowed due to limitations on movement. However, during this period, some minor progress was made, for example through the use of local laborers for school construction and in the design of a textbook distribution plan. The project fully resumed activities in February 2015, when schools were reopened following six months of closure. In June 2015, the project requested a one-year extension to compensate for the loss of implementation time caused by the EVD. Availability of Data 53. The project encountered challenges linked to the timeous availability of reliable data. The most up-to-date EMIS school data, which had been supported by USAID, was not available to the PST during planning of activities. This complicated the accurate and timely delivery of school grants and learning materials. In many instances, requests for extra textbooks were received from schools that were not in official databases or from schools that reported increased enrollment relative to available data. As a consequence, a second round of textbook distribution was necessary. 2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization Design 54. The original project M&E Results Framework (RF) included six PDO-level outcome indicators and eight intermediate-level output indicators to track implementation and expected outcomes at multiple levels. The quality of the RF was good and the PDO-level indicators were adequately linked to intermediate results indicators. 55. During the Level I restructuring, the majority of the PDO indicators and the intermediate results indicators were amended to reflect the narrowed scope of the project. Some indicators related to discontinued activities were dropped, with new indicators added where appropriate with enhanced measurability. 56. However, links between project activities and the PDO in the revised project could have been more robust. The project correctly identified capacity constraints as the most important challenge and was effectively restructured to focus on management and accountability chains. Thereafter activities were chosen to enhance central capacity. However, measuring improved management capacity and accountability proved to be a challenge. The PDO indicators of the project focused on the completion of activities rather than measuring improvement in management capacity. This was somewhat shortsighted given that measurement could have been achieved using a qualitative survey. M&E Implementation and Utilization 57. Regarding project progress, the PST dutifully monitored project indicators, tracking them through various sources such as a school construction database, EMIS, bank accounts of schools, and NGO reports. Data measuring progress at school construction sites was used to amend and, in one instance, cancel contracts. When sites experienced significant delays, the contractor was issued a nonconformance certificate. Two contractors were placed on probation until they corrected issues of noncompliance and demonstrated progress. One contract was terminated for persistent 15

32 nonperformance. Progress in the procurement and distribution of learning materials was monitored through logs with regular reporting. 58. Active monitoring of construction activities was critical to the successful completion of school construction projects, and there was room for improvement regarding the supervision and monitoring of construction sites under the project. Initially, the project hired an independent construction supervision company, AECOM, to serve as the supervisory entity to oversee all contractors engaged in school construction. However, the oversight role played by AECOM was not sufficient to address the planning and capacity bottlenecks encountered by some of the contractors. Overall AECOM reported issues on sites, however supervision actions were lacking. For example, some of the contractors encountered serious cash flow issues due to infrequent invoicing. 10 In addition, some sites had very little progress due to a lack of materials and food for the staff. These issues could have benefitted from better planning and supervision. Coupled with the high cost of AECOM, its contract was terminated at the end of March The MoE assumed responsibility for supervision and site management till the end of the project. Due to the lack of qualified site supervisors in the MoE, the ministry continued to hire site supervisors trained by AECOM until June 2016 when their contracts expired. Thereafter, no supervisor was on site to monitor construction, with the exception of PST spot visits. While this practice reduced the cost of oversight, it did not sufficiently address quality deficiencies as the PST lacked qualified personnel and time to closely supervise each site. 59. The school grant subcomponent was monitored through the central ministry staff and NGOs because the regional MoE staff were not available; it took the MoE a long time to place the county-level staff on payroll. In the absence of dedicated regional staff, the MoE hired four NGOs to support training, monitoring, and supervision of school grants in 11 counties. The MoE completed the monitoring of the four non-ngo counties for the first tranche of grants to gauge compliance on the part of schools. 60. With the focus of the RF on activities, there did not seem to be sufficient or institutionalized mechanisms for M&E of the project s larger impact. Additionally, baseline information was significantly delayed (until 2013) and data-based reporting was inconsistent. A baseline study was first conducted after the Level I restructuring to update the RF. The baseline study revealed that the original target for beneficiaries (591,000) under the school grants component had been based on enrollment data premised on an extrapolation using the EMIS data, which was higher than what was verified in The discrepancy in the EMIS data may have been a consequence of incentives for schools to inflate enrollment numbers for funding purposes. As a consequence, the end target for project beneficiaries was lowered to 473,000, premised on the verified enrollment data. This target was further decreased to 430,000, to reflect the decrease in enrollment as a consequence of the EVD crisis. 61. No end-line study was conducted to determine the impact of various interventions. A textbook utilization study was designed to examine the effectiveness of textbook use in the classroom. However, details of this and associated teacher training with regard to textbook utilization was not fully developed by the time of project closing, and associated activities had not 10 Unfortunately, the school construction database used by the MoE did not include details about the payments to the contractors, making it difficult to monitor contractors who had cash flow issues. 16

33 been completed in time to be considered in the Implementation Completion and Results Report (ICR). 2.4 Safeguard and Fiduciary Compliance 62. Safeguards. The project triggered OP 4.01 (Environmental Assessment) and OP 4.12 (Involuntary Resettlement) due to the potential impact of school construction on the environment and communities. The impacts were estimated to be minor as the project focused on rural areas where there was ample land. In addition, the majority of schools were built in locations where schools had been built in the past and needed to be rebuilt. The project prepared an ESMF and an RPF which were disclosed in-country and at the World Bank Public Information Center. In addition to describing the environmental and social screening process, the ESMF made recommendations regarding the need for capacity building to ensure its effective implementation, as well as with regard to consultation with potentially affected persons. 63. In December 2010, the task team noted that the Grant Agreement did not allow ECE Center construction (the terminology used is minor rehabilitation and renovation ). Following consultation with the Legal Vice Presidency (LEGAF) and the Environmental and Social Safeguards Teams, the task team determined to (a) (b) (c) Agree to make amendments to the RPF with the Government, Re-disclose the RPF through the InfoShop and in-country, and Amend the Grant Agreement. It was further agreed that no public works would proceed before all three steps had been completed. 64. The MoE obtained the official land deeds (in addition to tribal certificates) for all construction sites. The MoE delivered the probated deeds for all sites to the Land Commission by June 18, The delivery of the deeds to the Land Commission marked the completion of this activity for the purposes of the GPE BEP. 65. In November 2013, the task team s Environmental Safeguards Specialist carried out field visits to nine school construction sites in five counties. His report indicated that construction activities were in compliance with the relevant safeguards policies. No safeguard issue was reported by project closing. 66. Fiduciary. Overall, the Government, through the MFDP s PFMU, was able to fully comply with all financing covenants applicable to the project. Funding was disbursed into a Designated Account (DA) at a commercial bank. The World Bank guidelines and procedures for expenditure process (documents and signatures on commitment, payment orders, and so on) were applied to the project. An in-depth financial management supervision review was conducted in June The aim of this review was to verify the adequacy of the Government s financial management arrangements and compliance with financial management covenants. 11 Quarterly interim unaudited financial reports and external financial audits (final audit conducted by PricewaterhouseCoopers) were submitted to the World Bank on time. The auditors noted in the 11 Articles 4.01 and 4.02 in the Development Credit Agreement. 17

34 post-audit report that the fixed assets register had not been adequately maintained and updated by the project, and that reconciliation items identified in bank account reconciliation statements had not been resolved on time. 67. Allegation of misuse of funds. In October 2014, the MoE alerted the World Bank to an alleged case of fraudulent and corrupt practices involving the PST s junior project accountant relating to approximately US$34,810. The Government returned the money involved to the World Bank. Moreover, the task team leader (TTL) immediately reported the case to the Institutional Integrity Investigative arm of the World Bank, consistent with the World Bank s guidelines for suspected fraudulent and corrupt practices involving projects. Further, the alleged incidence of fraud was reported by the MoE to the Liberia National Police and the case was presented to a Liberian Court of Law. The World Bank was able to support the project and implement mitigating measures to address the apparent breakdown in controls that enabled the alleged fraudulent activity. The staff member was dismissed from the project. At the time of writing, the case is pending judgement by the courts of law. 68. Procurement. Delays in filling key PST positions and the contracting of a management firm to oversee school construction caused delays in the initial implementation of the project. As building of procurement capacity was considered essential for the project, procurement staff in the MoE underwent training provided by a World Bank procurement specialist. The national procurement specialist hired under the project was absorbed by the MoE following the closing of the project, contributing to the enhancement of procurement capacity in the MoE. Overall, procurement related to project-supported activities at the central level were adequately implemented and documented. For example, distribution of textbooks, teachers guides, and supplementary guides at the decentralized levels was adequately recorded and reported. Through monitoring visits, the use of school grants at the school level was confirmed to largely comply with the School Grant Guidelines, despite record keeping not being comprehensive. 2.5 Post-completion Operation/Next Phase 69. At the closing of the operation in October 2016, all targets outlined in the RF had been achieved or surpassed with the exception of one intermediate results indicator relating to the textbook utilization study. Outputs delivered by the project (for example, school construction standardized guidelines, School Grant Guidelines and textbook distribution network) continue to be used by the Government. Two activities that were incomplete at the time of the project s closing were continued using other funds. 70. Textbook distribution and utilization. It was determined that following confirmation of school status, surplus books would be distributed after the completion of the project using funding from the Ebola Recovery and Resilience Project. This second round of textbook distribution was completed by December 2016 (post project). By the time of writing the ICR, the textbook utilization study had been undertaken using the Ebola Emergency Response Project but results were not yet available. 71. Follow-on sector planning and operations. An updated Getting to Best Education Sector Plan (G2B-ESP) has been prepared with extensive consultation with the Government and support from DPs. The G2B-ESP served as the basis for the preparation of an application for a second 18

35 grant from the GPE. Lessons learned during the ICR stage have been considered in preparing the new Project. For instance, the new project will not allocate resources to school construction given its limited scale, but it will focus on school-based management and school grants because these have demonstrated to be efficient in improving learning conditions and school-level accountability. 72. Challenges to sustaining impact. In general, the maintenance of public schools is a serious ongoing challenge in Liberia due to limited human and fiscal capacity within the Government. Budget allocations in support of school construction and maintenance, in particular, have been very low since FY2012/13. The project developed a school maintenance plan. However, without commitment on the part of the Government toward maintaining schools, the sustainability of school construction is subject to risk. 73. Government commitment. Overall, the Government s commitment to the education sector has increased, as reflected in the increased share of education recurrent expenditure as a share of total Government recurrent expenditure from percent in FY2015/16 to projected percent in FY2018/19. As agreed, the Government transferred US$790,000 to the MoE s account to support the second tranche of school grants for the academic year 2014/2015. This amount was equivalent to more than 25 percent of the cost of the program for that academic year. Due to the late introduction of school grant under the project, there was not enough time for the project to pursue a phased financing approach. The tightened fiscal context precipitated by the EVD epidemic prevented the GoL from continuing support for the financing of school grant activities without external financial assistance. As a consequence of these factors, the majority of public schools received only one year of school grants (in two tranches of 80 percent and 20 percent, respectively). 3. Assessment of Outcomes 74. Two rounds of restructuring changed the project PDO and PDO indicators (approved by board). In the first instance, Level I restructuring in September 2012 changed both the PDO and PDO indicators. In the second instance, Level II restructuring in June 2015 maintained the amended PDO but extended the majority of PDO indicators for one year. The ICR team initially engaged in a three-way split approach (dividing project time by the above two restructurings) for the assessment. However, rating differences were not significant to warrant a meaningful split by the Level II restructuring as the Level II restructuring in many ways was simply a no-cost extension of the closing date. As a result, the assessment was conducted by examining the performance in the pre-level I and post-level I restructuring periods. 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives Pre-restructuring Rating: High 75. The objectives of the original project were highly relevant to the Liberian education sector at the time of approval, and continue to be relevant at the time of writing the ICR. The project priorities were closely linked to the key priorities in the ESP and the new G2B-ESP 19

36 ( ) targeting expanded access to education and improvements to the quality of teaching and learning delivered. The project s objectives were aligned with the World Bank/African Development Bank Joint Assistance Strategy , Liberia s PRSP introduced in March 2008, and the World Bank Education Strategy The project objectives contribute to the World Bank s commitment to helping countries achieve the education related Millennium Development Goals. In addition, the project objectives support the second and third pillar of the World Bank Country Partnership Strategy FY13 17 for Liberia: (a) (b) (c) Economic transformation to reduce constraints to rapid, broad-based, and sustained economic growth to create employment; Human development to increase access and quality of basic social services and reducing vulnerability; and Governance and public sector institutions to improve public sector and natural resources governance Post-restructuring Rating: High 76. Post-restructuring, the objective for the project was consistent with the original objective due to the retention of key activities from the original project after restructuring. The restructured project had a narrower scope on the objective, which was to improve management capacity and accountability at the central and school levels. As a consequence, the objective in the postrestructuring period had the same level of relevance as objective prior to restructuring. Relevance of Design and Implementation. Pre-restructuring Rating: Modest 77. The project s components and implementation arrangements were consistent with the project s stated objectives. Project-supported activities aimed to increase access to basic education were, and continue to be, relevant in the contemporary Liberian context. At the time of project preparation, the decision to focus resources on school construction was valid given the high pupilto-classroom ratios and projected increases in demand for education. The school grants approach was strategic given the increased financial burden faced by public schools following the abolition of school fees in primary education. Many lessons learned through implementation of the EPF were incorporated into project design, specifically with regard to school construction, textbook distribution, and school grants. However, in a post-conflict context where priorities are many, selectivity is crucial to the viability of the project. The project design lacked selectivity and as a result, the project encountered significant delays in the initial year of implementation. Post-restructuring Rating: Substantial 78. The project financed activities to enhance central and school-level management capacity and accountability. Project design included the recruitment of an implementation team with skills relevant to the effective management of school construction, and the distribution of learning 20

37 materials and school grants. The design of the project was structured in such a way as to facilitate the inculcation of skills within the MoE through the existing structure and the gradual transfer of responsibility and to hold schools accountable for increased resourcing and monitoring of activities. As mentioned earlier, the link between PDO indicators and the PDO could have been more robust. PDO-level indicators could have measured the improvement in capacity building more directly. 3.2 Achievement of Project Development Objectives Pre-restructuring Rating: Modest 79. The pre-restructuring phase spanned 21 months of the project s lifetime. However, it is worth noting that the Government (with new leadership in the MoE) requested changes only months into implementation, which was the basis of the Level I restructuring report circulated in February/March 2012 (official approval - September 2012). Achievement of the original PDO to increase access to basic education (with a particular focus on poor areas), improve conditions of teaching and learning, and improve school management and accountability appears feasible based on the project s implementation trajectory at the time. However, the effective implementation time to achieve the original PDO was too short and it is likely that some delays were based on the understanding that these specific activities were to be dropped (such as school health, ECE, and so on) and therefore not worth initiating. Without major outputs from the project during this short period, progress toward the PDO is analyzed by linking progress with activities and inputs. Sub-objective 1.1: To increase access to basic education (with a particular focus on poor areas) 80. The MoE conducted fieldwork to identify sites for school construction. The World Bank cleared site selection criteria in March 2011 and fieldwork was conducted in June Further fieldwork was conducted in early September 2011 to finalize the selection of sites. School construction management firm was contracted in September Sub-objective 1.2: To improve conditions of teaching and learning 81. Agreement was reached between the task team and the MoE that the funding to support school grants would only be disbursed following the conclusion of the EPF-supported school grants program and after making an adequate assessment of potential weaknesses in that program s design in April/May Changes were proposed to be made to the county-based grants under EPF, including (a) strengthening the link between grant disbursement and county/school improvement plans, (b) broadening the scope of eligible expenditures, (c) setting a minimum threshold so as to not disadvantage small schools, and (d) reviewing the per student amount. Sub-objective 1.3: To improve school management and accountability 82. The project supported construction of a new MoE building to house all central MoE staff under one roof for the first time. In the long term, the new building enabled the creation of a conducive working environment for MoE staff, with positive implications for improved synergy and efficiency. RRA Cycle 1 (February June 2011) and Cycle 2 (July September 2011) 21

38 workshops were carried out to enhance implementation capacity. Teams responsible for the implementation of each subcomponent were identified, and each team developed an annual implementation plan. After some delay, the Joint Education Sector Review was held in February Post-restructuring Rating: Substantial 83. The PDO was revised at restructuring to improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school-based management. The project provided support to the relevant MoE departments to fulfill their activities relating to school construction, the distribution of textbooks, and school grants, contributing to improvement in management capacity and accountability. 84. As demonstrated in the following paragraphs and in greater detail in annex 2 the project delivered on all of PDO-level indicators and exceeded the targets in some cases. However, as a weakness in the design, the PDO-level indicators were slightly disconnected from the PDO. The assessment of the achievement towards the PDO is thus done using supplementary evidence apart from the PDO-level indicators. 85. The objective of improving management capacity and accountability was meant to be supported by all activities under the auspices of the restructured project and specific capacitybuilding effort was undertaken. As noted earlier, the project was designed to support relevant MoE departments to fulfill their activities relating to school construction and the distribution of textbooks and school grants, with a particular focus on improving financial management and procurement capacity. Subcomponent teams established under the project overseeing school construction, school grants, and textbooks subcomponents were headed by MoE staff from relevant departments. 86. Terms of reference (ToRs) for the recruitment of international specialists for the project included specific reference to capacity building for and knowledge transfer to MoE counterparts. For example, as a condition for the World Bank extending no-objection to the international procurement specialist s contract extension, the specialist was required to prepare training materials and a lesson plan and hold regular training sessions with MoE procurement staff. Training was then followed up through World Bank supervision missions, and the reviewing of training materials and attendance logs. In addition, national procurement staff hired by the PST were required to attend World Bank-sponsored training events and attendance was tracked by the World Bank procurement specialist in the country office. 87. With regard to financial management capacity, a key focus of the project was to enhance the capacity of the MoE s accounting staff, so that the ministry could assume responsibility for finances from the PFMU. Despite efforts on this front, financial management capacity at the MoE was not considered to be adequate for this transition to take place by the time the project closed. 88. PST members were retained initially on consultant contracts with the intention that they would be integrated into the Center for Education Management in alignment with the Education 22

39 Reform Act of Unfortunately, the Center for Education Management was not established due to lack of funding. The absorption of the team was implemented on a case-by-case basis, which provided important human capacity for the MoE. 89. A functional review was carried out to assess strategic alignment, work structure clarification, administration efficiency, and resource utilization. A Capacity Development Coordinator was hired to conduct a survey of gaps in capacity at the MoE and to identify longterm strategies to address capacity deficits in various departments. This exercise was interrupted by the EVD outbreak during which no one-on-one consultation could be held with MoE staff. It was resumed after the crisis and the functional review report was produced, which identified the weakness in the alignment of skills and functions of staff and gap in administration efficiency. 90. The intermediate result identified for the overall capacity building was achieved, that is, Joint Education Sector Review held annually. The MoE successfully held annual Joint Education Sector Reviews apart from in 2014 during the onset of the EVD epidemic. Sub-Objective 1.1: To improve management capacity and accountability at the central and school levels for school construction in rural areas 91. The school construction subcomponent was initially designed to support the construction and furnishing of 48 schools (including preprimary, primary, and lower secondary) on 40 sites in rural areas. The number of schools was later reduced to 41 schools on 35 sites due to grant funds availability and limited implementation time remaining under the project. Even with the reduced number of sites, the project was able to construct more classrooms than originally envisioned (303 classrooms built as compared to the targeted 272). The sites were fully completed by the time of the project closing with furniture delivered to all schools. 92. Construction contracts were awarded to local construction firms. The MoE procurement team was actively engaged in the bidding process, including the preparation of bids and evaluation reports. AECOM, a construction management firm based in South Africa and Britain, was hired to supervise construction across 48 schools in accordance with standardized school designs. AECOM provided supervision training to MoE staff and local contractors. As a result, 20 local engineers and eight technical staff from the MoE received supervision training. 93. The MoE later assumed responsibility for the supervision of the construction work from AECOM in 2015 due to the high cost of retaining the services of AECOM, but it continued to retain the services of existing AECOM site supervisors. The contracts of these site supervisors expired in June 2016 at a time when the construction of some schools was still unfinished. Subsequently, on-site supervision was limited to occasional spot visits by PST staff. During the ICR mission, the Unit of Environmental Facilities engineers recommended that the inclusion of the unit in the school construction subcomponent, and the use of the unit for monitoring sites, could have been strengthened. 94. The CDD approach for school construction and related activities were dropped for both the Phase I and Phase II construction due to unsuccessful negotiations with NGOs regarding financial arrangements. During both phases, anecdotal evidence suggests that the contractors actively hired local labor, fulfilling, to a certain extent, the CDD aspect of the project s design. 23

40 95. The relevant PDO-level target (schools constructed according to standardized school construction guidelines) was surpassed (100 percent as compared to the target that was 75 percent). The related intermediate results indicators number of additional classrooms built or rehabilitated at the primary education level resulting from project intervention was also exceeded, with 303 classrooms constructed by project closing date as compared to 272 classrooms in the target. 96. Another intermediate results indicator annual school construction database updated by annual Joint Sector Review was also achieved. Standardized school construction guidelines for site selection, design, contract management, and construction quality approved by the MoE SMT was achieved as the standardized school construction guidelines were developed by the construction management firm. Sub-objective 1.2: To improve management capacity and accountability at the central and school levels for procurement and distribution of teaching and learning materials 97. The project procured 1 million textbooks, 20,000 teacher guides for grades 5 9, over 1 million supplementary readers for grades 1 4, and a variety of instructional materials for grades 1 9. The project also delivered 98 percent of the materials procured under the project. 98. The MoE s Bureau of Curriculum updated the national curriculum with the support of a learning materials specialist hired by the project. Liberia does not use the standard West African Examination Council exam, with the exception of the West African Senior School Certificate Examination (WASSCE) for senior secondary students. Thus learning materials for grades 7 9 were revised to align with the WASSCE curriculum. The Bureau of Curriculum was also actively involved in the procurement of textbooks, including the reviewing of bidding documents. The project also supported the procurement and updating of equipment for all primary MoE warehouses for the storing of learning materials. 99. The project was able to deliver the majority of items procured for the first round of distribution in County and District Education officers, school principals, and parents were mobilized for textbook distribution. Two methods were adopted: stationary distribution for hardto-reach schools and school-level distribution for all other schools. Books distributed through stationary distributions were wrapped per school at the distribution hubs (usually district capitals). Principals and heads of PTAs from each school arrived at the hubs to pick up the books and carried them back to the schools. For school-level distributions, the sorting and stamping of books took place at regional hubs, after which the materials were delivered to schools. Under both methods, the signatures of the supplier, principal, head of PTA, DEO, and auditor were required to validate the receipt of materials PDO indicators set for the sub-objectives were achieved or exceeded, that is, procurement of textbooks and teacher guides for grades 5 9 verified under the project (100 percent) and distribution of textbooks and teacher guides for grades 5 9 verified under the project (85 percent). An intermediate results indicator for both the Sub-objectives 1.2 and 1.3 was met social accountability framework for teaching /learning materials and school sub-grants programs developed and approved by the MoE SMT. 24

41 101. However, by project closing, the intermediate results indicator (a teaching and learning materials utilization assessment study) had not been achieved. The learning materials team originally decided to conduct an impact evaluation to study the utilization of supplementary readers but determined in May 2014 that there was insufficient time to conduct the study. The World Bank and the MoE agreed to include a more limited textbook utilization study. However, the MoE was slow in determining the scope and details of the utilization study and associated teacher training on textbook utilization, partly because teacher training was anticipated to be difficult by the reduction in donor-supported teacher training programs and partly because textbooks were still being delivered by the closing date for the GPE BEP. At the time of writing the ICR, the study is under way using other funds, and fieldwork for the study has been successfully completed, even though the final result is not yet available. Sub-objective 1.3: To improve management capacity and accountability at the central and school levels for school based management 102. The project supported the piloting of school grants starting in September The PST developed a time line to phase out the county-based grants under the EPF, with the long-term goal of having all schools receive school-level grants based on school improvement plans. The PST worked along with the Grants and Coordination in the Department of Planning to implement this subcomponent with an MoE staff member as the lead person Under the project, the MoE provided training in four pilot counties plus the school district of Monrovia Consolidated School Systems (MCSS). NGOs were contracted to provide training to schools in the remaining eleven counties, which resulted in a total of 2,579 schools developing school grant utilization plans, opening school bank accounts, and successfully receiving two tranches of school grants. Schools were also encouraged and trained by the MoE or NGOs to establish SMCs and PTAs to contribute to improved school management The MoE conducted monitoring of school grants in four counties while four NGOs were in charge of the remaining eleven counties. Together they ensured the distribution of school grants to the schools and monitored the use of associated funds for non-salary related recurrent expenditure. Monitoring reports indicated that 72 percent of the schools used the school grant in compliance with the School Grant Guidelines. The beneficiary assessments performed by the ICR team confirmed that grants had been properly used to improve the learning environment One PDO indicator was defined for the sub-objective schools being single signatories to their own bank account (1,000). The final number achieved under the project was 2,579 schools, far exceeding the target value. Two intermediate results indicators were met, which were School Grant Guidelines approved by the MoE SMT and school sub-grants implemented in accordance with School Grant Guidelines (30 percent). 3.3 Efficiency Rating: Modest 106. Overall efficiency of the project is rated as Modest. This is further elaborated on in annex 3. In summary, the cost of the main activities, namely school construction, textbook procurement and distribution and school grants, compares favorably with other donor projects in Liberia. 25

42 Moreover, percent of the project s US$40 million (US$39,963,547.68) were disbursed during the project period Relatively insignificant disbursement occurred before restructuring, and there is no data to support a calculation of internal/external efficiency or a cost-benefit analysis Unit cost of school construction compares favorably with that for other donor projects in Liberia, though it was higher than that in other fragile countries because the project adopted high quality standardized design. The per-classroom unit cost for a three-classroom primary school under the project was US$37,166. Data from recent ICRs in other low-capacity and fragile contexts (Benin, Burundi, Côte d Ivoire, Northern Uganda, and Tanzania) show per classroom construction costs between US$20,000 and US$25,000. However the project compares favorably with school constructed under the EPF with per classroom unit cost US$40,000. Unit cost for communitybased school construction under the LACE was lower (US$15,000) with a lower quality of material being used for construction The scope of the school construction sub-component did not include the teacher deployment to the newly built schools, which in a way decreased the potential efficiency of the Project. Joint supervision trips (MoE and DPs) to the GPE constructed schools revealed that some schools were not sufficiently equipped with qualified head teachers/teachers due to the lack of qualified teachers in the rural areas and payroll issues. The challenge of insufficiency in qualified teachers needs to be tackled from several fronts with sustained resources. The new GPE-G2B Project will include components on in-service teacher training and payroll management This unit cost for textbooks was higher than that for other countries but lower than the cost of previous textbook procurement and distribution programs in Liberia. GPE-supported textbook distribution activities incorporated lessons learned from the EPF through the distribution of textbooks directly to the school level in alignment with enrollment data drawn from EMIS. This system significantly improved accountability for the effective distribution of textbooks. The project provided a total of 1 million textbooks for grades 5 9 and 1.4 million supplemental instructional materials for grades 1 9 at a unit cost of US$4 per textbook. These costs are in line with the costs indicated in Read and Bontoux (2015) --- unit cost of a primary textbook in Sub- Saharan Africa to be between US$2 and US$4. These costs in Liberia are lower than the costs for the textbook procurement and distribution of the 1.2 million textbooks under the EPF project where the unit cost was US$ Lower quantities were purchased as compared to comparison countries and the individual procurement orders were not more than 200,000 units and so small scale contributed to higher costs. Moreover, the textbooks were for higher grades (grades 5 9) and thus the textbooks were more comprehensive (more pages) as compared to textbooks procured for lower grade (for example, Grades 1 6). In addition, the textbooks were in color and the paper quality was better with a shelf life of four to five years School grants were disbursed directly to the school bank accounts set up under the project to minimize leakages and improve transparency. Under the project, public schools were supported in setting up their own bank accounts by NGOs and MoE. Despite the need for extensive financial 12 Burchell Evaluation of the Distribution of Primary Teachers Funded by the Education Pooled Fund to GoL Primary Schools. 26

43 management and procurement assessments, this decision eliminated the need to transfer school grants to counties, reduced the leakage of funds in the grant and proved to be efficient in improving learning conditions in schools. A total of US$2,433, was allocated toward school grants over the project period at a unit cost of US$5.02 per student. A total of 483,565 students benefited from this grant, with 47 percent of the beneficiaries being female. According to independent verification carried out by NGOs and MoE, 72 percent of expenditure was in accordance with the School Grant Guidelines. The beneficiary assessment conducted at the end of the project supports this finding. Most schools used the funds to purchase teaching and learning materials, including basic supplies such as chalks, in line with the guidelines provided to them. 3.4 Justification of Overall Outcome Rating Objectives High Objectives High Table 4. Overall Outcome Rating Breakdown by Periods Relevance Design and Implementation Modest Substantial Relevance Design and Implementation Substantial Substantial Overall Outcome Rating Pre-restructuring Achievement of PDO (Efficacy) Modest Post-restructuring Achievement of PDO (Efficacy) Substantial Combined Efficiency Modest Efficiency Modest Moderately Satisfactory Overall Rating Moderately Unsatisfactory Overall Rating Moderately Satisfactory 112. The overall rating calculated for the entire project life is 3.84, equal to Moderately Satisfactory. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 113. The project seems to have had a positive impact on poverty reduction, gender equity, and social development. Both the original and restructured project design included a focus on rural areas with new construction of schools targeted only to rural areas. School grants were rolled out nationwide. While specific data are not available to assess the exact impact on poverty reduction, it can be deduced that interventions in these low-income communities had a positive impact in rural areas, as increase in education attainment is associated with higher private returns. The project s core PDO indicator included a target for female beneficiaries which was set at 44 percent of all beneficiaries benefiting from school grants-related activities. The project exceeded this goal, with the share of female beneficiaries reaching 47 percent under school grants. The school grant served as a mechanism to supplement the Government s policy of free basic education (primary and junior secondary), mitigating the potential for financial barriers limiting access to education for disadvantaged populations. 27

44 (b) Institutional Change/Strengthening 114. The project adopted the approach of relying on the existing government structure for key activities, with the objective of improving government management capacity of accountability. The project is assessed to have significantly enhanced the management capacity at the MoE. By the time the project closed, the MoE had high-quality standardized design for school constructions, acquired procurement and distribution capacity for textbooks, developed guidelines for school grants, and engaged extensively in school grant monitoring. In the future, the MoE is assessed to have the capacity to conduct the abovementioned activities without seeking extensive technical assistance. (c) Other Unintended Outcomes and Impacts (positive or negative) 115. Not applicable. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 116. A beneficiary survey of schools (with school principals as the primary respondents) was undertaken, which confirmed the positive impact of the project activities, with majority of the respondents satisfied with the school construction, textbooks, and school grants. The overall findings of the beneficiary assessment were as follows: (a) (b) (c) (d) (e) (f) (g) Ninety-two percent of sampled schools confirmed that they had received school grants. Ninety-six percent of sampled schools reported that they spent grant funds in line with the School Grants Utilization Plan. Schools were largely satisfied with the grant, although there were widespread concerns regarding delayed disbursement and the lack of continuity in school grants. Ninety-two percent of schools confirmed that they had received textbooks. A majority of textbooks were delivered directly to the schools and these delivered textbooks were in good condition and schools were generally satisfied with the textbooks. Principals of schools constructed by the GPE-supported project, who were surveyed, reported being satisfied with the condition of the school when it was handed to them, as well as the current condition of the school. All surveyed principals of GPE-constructed schools reported that the classrooms and toilets constructed by the project were still fully functional. 4. Assessment of Risk to Development Outcome Rating: Significant 28

45 117. The Project benefited from the additional financing provided to the education sector in response to the EVD. By the time the Project closed, activities including second round of textbook distribution and textbook utilization study were carried on using these funds 13. The supervision trip to GPE constructed school has revealed that some schools were not equipped with qualified head teachers/teachers. This is a systematic problem that is caused by lack of qualified teachers, lack of resources to hire qualified teachers and payroll issues. The new GPE Project (GPE G2B) will include components on teacher education and payroll management to help the MoE tackle the challenge. The impact of the GPE BEP will also be potentially enhanced by the new project, particularly by channeling new funds into the school grants and further improving the capacity of the MoE in delivering key education services The most significant risks to the project s development outcome include macroeconomic outlook for Liberia s economy and the fiscal constraints and high level of DPs investment in the education sector At the time of writing, Liberia s macroeconomic performance has not recovered from the twin shocks of the 2014 EVD epidemic and decline in global commodity prices. These events underlined the vulnerability of Liberia s economy to unforeseen shocks. The International Monetary Fund (IMF) forecast a GDP contraction of approximately 0.5 percent in 2016 and modest recovery to 3.2 percent GDP growth in Projections of medium-term annual GDP growth is in the 6 percent range, lower than levels experienced before The macroeconomic situation in Liberia has resulted in significant fiscal constraints for the GoL. Even though the education sector was spared from budget cuts in 2016, the Government was not able to make substantial investments to sustain or support the activities undertaken in the project. As mentioned earlier, the share of education recurrent expenditure in total Government recurrent expenditure has increased to more than 16 percent in FY2016/17; however, it is still below 20 percent, as is recommended as best practice by the GPE There is a high level of donor financing in the education sector. Contributions from DPs continue to account for more than 40 percent of all resources supporting the education sector. DP financing served as a critical means to mitigate the impact of the EVD crisis and associated fiscal challenges. Key activities such as school construction or textbook procurement still heavily rely on donor support, because the budget allocated to these items remains negligible. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 13 Additional financing included the Ebola Recovery and Resilience Program and Ebola Emergency Response Program. 29

46 121. The World Bank s performance in ensuring quality at entry is rated Moderately Satisfactory based on the following: a) The World Bank supported the preparation of a project that was relevant to national development priorities, premised on the Liberia Education CSR that analyzed need and financing gaps of the education sector and on the ESP The project preparation team actively consulted with local DPs and prepared the project based on lessons learned from previous projects implemented in Liberia. b) The project had a good design of the M&E. The project was expected to use rich data sources, including EMIS and qualitative surveys at the school level. However, due to the overall slow progress in the implementation, the baseline study did not commence until after the midterm review and Level I restructuring. c) Given the nature of the GPE project as a vehicle to drive the ESP agenda and the consultative process adopted by the project, the project did not have great selectivity of activities during the design phase. The project was implemented in a high-risk and low-capacity environment, and preparation could have benefited from a technical design more tailored to the fragile context of the country. (b) Quality of Supervision Rating: Moderately Satisfactory 122. The World Bank s performance in ensuring quality at entry is rated Moderately Satisfactory based on the following: a) The World Bank team profile shows appropriate skills mix (education, fiduciary, environmental and safeguards expert, procurement, and architect) that enabled supporting and supervising the project. Missions were carried out regularly (with exception of the EVD crisis period 14 ) and challenges were swiftly identified, discussed and recorded in detailed supervision documentation (for example, Implementation Status and Results Reports and Aide Memories). For example, the World Bank team quickly noticed the struggle of the PST in implementing a complex project in the first year and agreed with the GoL on multiple milestones to expedite implementation during supervision missions and eventually called for an early midterm review to determine a project restructuring. Extensive consultations with DPs were held to identify areas that DPs were engaged in so as to narrow the scope of the restructured project, which was more realistic for the MoE with limited capacity. A Quality Enhancement Review by the World Bank was conducted for the restructured project. The simplified project design and RF focused on a limited number of main activities (school construction, learning materials, and school grants) based on welltested interventions. 14 The task team conducted weekly/bi-weekly video and audio conferences with the PST during this period of the EVD, to compensate for the inability to carry out in-person missions. The World Bank country office facilities were made available to the PST during this period to facilitate the communications. 30

47 b) The World Bank appropriately identified where international technical assistance would be needed to improve planning, reporting, and operating capacity of the MoE, thereby supporting the foundation of competencies required to effectively implement the project. In addition, the World Bank team used the RRA to enhance the PST capacity in implementing the project. The FM, procurement, and safeguard compliance was ensured through a supervision and review mission. c) The World Bank s sustained dialogue with the Government also led to an allocation from the national budget equivalent to 25 percent of funding required to support the first year of school grant, one year ahead of schedule. The local DPs were consulted regularly during the project implementation and they played an important role in providing monitoring of school construction activities. d) The Project could have benefited from an increased continuity in TTLs and a more focused in-country supervision. The Project went through four TTLs from the beginning to closure, though the handovers were mostly smooth. Due to the fact that Liberia was not a family station and the small scale of operations in country, the Project only had supervision staff based in country for a portion of the total project life. (c) Justification of Rating for Overall Bank Performance 123. The rating for the World Bank s performance is Moderately Satisfactory, based on its Moderately Satisfactory performance in ensuring quality at entry and Moderately Satisfactory performance in supporting appropriate supervision of implementation. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 124. The GoL demonstrated commitment to achieving and maintaining PDOs at all stages of preparation and implementation. From the first GoL application to EFA-FTI in 2007 to the second application in 2010, there was strong political support. The Government provided financial resources essential for the project s success. The Government acted on the agreed implementation arrangements to provide 25 percent of the school grant, a year ahead of schedule. The Government also returned the money involved in the fraudulent case (US$34,810) to the World Bank. However, given the country s limited resources and other extraneous factors (including the EVD crisis), it has not substantially increased investment in the education sector as demonstrated by the share of education within the budget and the share of primary education within the education budget Finally, the PFMU within the MFDP complied with legal covenants required of the project; however, this arrangement contributed to some delays in the project implementation. The multistage process for payment approval across the MoE and PFMU led to bottlenecks in the processing of payments to suppliers, consultants, and contractors. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 31

48 126. The MoE was the implementing agency for the project, with day-to-day responsibility delegated to the PST. The MoE demonstrated strong commitment to the project, which was a key factor contributing toward the achievement of project outcomes. Before the Level I restructuring, the MoE appointed a Restructuring Team comprising senior management and technical staff, including the Donor Coordinator to conduct a thorough analysis of GPE BEP components and implementation challenges and to propose the content of restructuring. Commitment was apparent whenever MoE staff were mobilized to conduct field visits under difficult travel conditions, especially during the rainy season. The field visits effectively verified and reported on school-level data related to compliance of school grants. The MoE also conducted annual Joint Education Sector Reviews, which facilitated overall donor coordination However, the weak capacity at the MoE has contributed to delays in the project implementation especially around procurement related activities such as preparing ToRs for key PST staff and construction management firm. In addition, high turnover at the level of deputy minister had a negative impact on the implementation of various project components that required oversight from these offices The project benefited from a strong PST. The PST provided comprehensive status reports on GPE BEP implementation and organized joint supervision missions with the ESDC. The PST managed to meet during the Ebola period and communicate with the World Bank team when the movement of personnel was prohibited. The one area of concern that is raised during stakeholders consultation for the ICR is that the reporting of project progress to the ESDC could have been done in a more structured manner Based on above performance assessment of both MoE and the PST, it is concluded that the implementation agency rating is Moderately Satisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 130. The rating for overall borrower performance is Moderately Satisfactory, premised on the Moderately Satisfactory performance on the part of the Government and the Moderately Satisfactory performance on the part of the implementing agency. 6. Lessons Learned 131. Innovative approach in project implementation can help achieve substantial results. For example, decentralized education officers, school principals, and parents were mobilized for textbook distribution. Stationary distribution for hard-to-reach schools, and school-level distribution for all other schools was developed for textbook distribution with signatures of personnel involved to enhance accountability. It is also the first time that Liberia adopted the direct school grant scheme, reducing the potential leakage in funding The construction component could have benefited from local contractors understanding better design standards and safety practices related to construction of schools. Low capacity within the local Liberian construction industry was a challenge throughout the project with regard to delays and lack of understanding of constructions specifications. To offset this and minimize impact, the MoE school construction team needs to ensure sufficient 32

49 personnel and resources are available, and knowledgeable of the local industry. The safety practice could have been handled by providing related safety guidelines and enforcing them through contract management. They would also benefit from working across Government agencies such as the Ministry of Public Works and LACE, to ensure better contract management and supervision capacity Community engagement early on in construction leads to better results in the postconstruction phase. This includes contract management by the community whose members serve as interlocutors between the MoE and the contractors where constant visits from the MoE are not feasible and addressing minor contractor issues in consultation with the MoE. In addition, school construction projects may benefit from more engagement with communities in the selection of sites, clearing of sites, negotiations for land, and the utilization of local laborers to ensure buy-in and support from the communities during the post-construction phase Timely development and access to comprehensive and transparent school census must be supported. The PST had to rely on outdated EMIS data for the distribution of learning materials and school grants, because updated data was not immediately available by the time the activity was planned. During the delivery process, a number of schools did not receive supplies because they were not listed in the old EMIS data. In addition, due to the discrepancy in enrollment data and actual student number, some schools required additional materials to be delivered in a second round. Access to disaggregated school-level EMIS data may make the task of providing school grants or distributing textbooks more straightforward Decentralized staff should play a more important role in project implementation. After the expiration of the contract for county-level MoE staff members, the MoE and the PST did not have any regional capacity to supervise and coordinate activities for an extended period of time. As a consequence, all activities had to be carried out and monitored by the PST or central MoE staff in instances where no NGO was involved. Moreover, from a financial management perspective, the management of daily subsistence allowance for central staff and individual contracts made compliance more difficult. The MoE may consider working more closely with decentralized agencies and service providers in the future to enhance efficiency and accountability World Bank presence in-country in the education sector ensures greater supervision support. The current scale of education operation in Liberia does not provide enough resources for a full-time staff to be based in Liberia. To have better in-country supervision, a World Bank program (IDA/IBRD) would need to be available as well to allow for leveraging of greater resources. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies Not applicable. (b) Co-financiers/Donors 33

50 137. As detailed in previous sections, donors, including EU and GPE, expressed a desire for greater engagement with the DPs during project implementation and consultations on effective mitigation strategies to address sustainability risk affecting school construction and school grants. (c) Other partners and stakeholders Not applicable. 34

51 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$, million equivalent) - Pre-Level I Restructuring Components Estimate at Appraisal (US$, millions) Actual (US$, millions) Actual as Percentage of Estimate (%) Component 1: Increasing access and equity in rural areas (US$15.5 million) Component 2: Improving the conditions of teaching and learning (US$ million) Component 3: Strengthening central and local levels to operate in a decentralized education system by developing management and monitoring and evaluation capacities (US$10.5 million) Total Project Costs Financing - Pre-Level I Restructuring Source of Funds Type of Cofinancing Appraisal Estimate (US$, millions) Actual/Latest Estimate (US$, millions) Percentage of Appraisal EFA - FTI (b) Project Cost by Component (in US$, million equivalent) - Post-Level I Restructuring (including Investments already Made during the Pre-Level I Restructuring Period) Components Subcomponent 1.1: Strengthened Institutional Management Capacity and Accountability at the Central and School Levels Subcomponent 1.2: School Construction in Rural Areas Subcomponent 1.3: Procurement and Distribution of Teaching and Learning Materials Subcomponent 1.4: School-based Management Through School Grants Estimate at Appraisal (US$, millions) Actual (US$, millions) Actual as Percentage of Estimate (%) Total Project Costs Financing Post-Level I Restructuring Source of Funds Global Partnership for Education Program Grant Type of Cofinancing Estimate at Appraisal (US$, millions) Actual (US$, millions) Actual as Percentage of Estimate (%)

52 Annex 2. Outputs by Component 1. The project went through one Level I restructuring and four Level II restructurings. Details on the outputs, outcomes, and overall achievements under each subcomponent of the project are provided in this annex. 2. The original PDO, as in the Grant Agreement, was as follows: to increase access to basic education (with a particular focus on poor areas), improve conditions of teaching and learning, and improve school management and accountability. 3. The project had a complex design in many priority areas in the ESP. The RF demonstrated the links between the activities and results. 36

53 Figure 2.1. RF of the Original Project Source: PAD. 4. As described in the main text, project implementation began in an unpredictable political environment with elections, the passage of the Education Reform Act 2011, and government reshuffling. The high turnover at the senior management level had consequences for sectoral policy dialogue and project implementation. It was thus essential to put together the PST at the technical levels required to take a lead in project decisions. Due to the hardship condition in Liberia, it was difficult to attract fully qualified candidates for key PST positions. The PST staff lacked the capacity and understanding to implement all activities. The project opted to use the Rapid Results 37

54 Approach Cycle 1 (February June 2011) and Cycle 2 (July September 2011) for the subcomponent teams to identify and develop an annual implementation plan. 5. The MoE processes were lengthy, which contributed to delays in initial implementation, as in the case of ToRs for the construction management firm and the public disclosure of the RPF. The task team discovered that the curriculum revision was late to commence and hired a textbook procurement specialist to help develop a realistic yet expedited time line to ensure that learning materials were distributed. The complicated design of the project exacerbated the already low MoE capacity, coupled with the overlap with other donor activities that fought for the MoE staff s attention. Most importantly, the MoE capacity was overstretched with respect to management of school construction, given that the MoE was also overseeing the new construction of 30 schools and renovation of 60 schools by UNICEF, which was not confirmed at the time that the GPE BEP was prepared. 6. In response to these challenges, the MoE wanted to make significant changes to the school grants and learning materials subcomponents. The MoE appointed a Restructuring Team comprising senior management and technical staff, including the Donor Coordinator. Following a thorough analysis of the GPE BEP components and implementation challenges, the Restructuring Team put together a proposal that reflected a reorientation and narrowing of the project s scope to focus on building capacity in the MoE to manage key sector activities in the context of ongoing efforts to decentralize the basic education system. The World Bank task team also responded to recommendations from a panel that reviewed the Restructuring Package, including representation from the GPE, which expressed strong endorsement for a Level I restructuring of the project, including (a) revision of the PDO, (b) extension of the closing date by two years, (c) focus on highimpact activities in which other donor partners were not engaged, and (d) full alignment of the project with the new decentralized structure of the education system. After comprehensive review of the project and implementation status, the Level I restructuring was approved in September Significant changes were made to the components, especially the dropping of activities that overlapped with other donors. The status of each component is indicated in table 2.1. Table 2.1. Changes Made to the Original Project by Component Original Design Component 1: Increasing access and equity in rural areas Construction and/or rehabilitation of classrooms, administration facilities, latrines, and water points in the selected early childhood centers and primary and junior secondary schools Status Post Level I Restructuring Retained, with revisions, as Subcomponent 1.2. Rehabilitation targets had not been specified in the PAD or the Grant Agreement and rehabilitation was dropped at restructuring. The restriction of school sites for using the CDD approach to hardto-reach areas was removed during restructuring to increase the viability of implementation. Component 2: Improving the conditions of teaching and learning (a) Provision of Learning Materials. Provision of Retained as Subcomponent 1.3 approximately 1,000,000 textbooks for students in grades 5 to 9 of basic education and approximately 20,000 sets of teachers guides (b) Improvement of School Grant Program. Retained, with revisions, as Subcomponent 1.4 (1) Provision of advisory services to strengthen the process and effectiveness of the school sub-grant program, including, in particular, development of guidelines 38

55 Original Design Status Post Level I Restructuring (2) provision of sub-grants to the selected primary and junior secondary schools (c) Support to Early Childhood Development. Activities were dropped from the project and were (1) Development of an ECD strategy and design and to be subsumed under UNICEF s work program. implementation of a cost-effective ECD model for rural areas, including (i) acquisition and supply of equipment, play and learning materials and (ii) training for locally recruited educators (2) preparation of quality standards and guidelines for ECD service delivery, child-centered curriculum and teacher training, and certification frameworks (d) Support to School Health. Activities were dropped from the project and were (1) Development and adaptation of stationaery student to be subsumed under UNICEF s work program. modules for grades 1 9 students, deworming for UNICEF was already financing de-worming in approximately 300,000 primary students, and training and remaining counties, with free de-worming tablets sensitization materials for teachers and the community in the provided by Deworm the World. selected counties (2) training of master trainers and teachers in selected counties on de-worming Component 3: Strengthening central and local levels to operate in a decentralized education system by developing management and evaluation capacities (a) Strengthening MoE Structure through Capacity Building, Retained as Subcomponent 1.1 Performance Management, and Monitoring and Evaluation. (1) Establishment of the PST under the MoE s Department of Planning, Research, and Development. (2) Capacity building and training for (i) the Departments of Activities (i) and (ii) were mainstreamed through Administration, Planning, and Instruction; (ii) school the implementation of Component 1 (such as building management and supervision at the central level; school building management and supervision and (iii) M&E and tracking of school level data for DEOs and financial management for the PTAs/SMCs). school directors; (iv) use of learning assessment instruments Activities (iii) and (iv) were dropped from the as a monitoring tool for early grade reading and mathematics project. The revised project was to focus on central performance for DEOs and teachers; and (v) financial and school-level capacity building, while management for the PTAs or SMCs county and district level was to be subsumed under the USAID/Liberia Teacher Training Program s (LTTP) work program. (3) Preparation and implementation of a comprehensive Dropped from project. Activities were subsumed M&E system and improved planning tools (4) Provision of advisory services for the development of a national assessment test for grades 4 and 7, focusing on mathematics and reading comprehension (b) Strengthening Personnel and Payroll Management. (1) Provision of advisory services to reinforce the personnel and payroll management activities of the MoE, including management of IFMIS by the MoE and skills analysis for setting up a pay and grade scale for teachers (2) Carrying out of studies and surveys to evaluate teachers motivation and working conditions under USAID/LTTP s work program. Dropped from the project Dropped from the project, with the exception of activities already completed. Other activities were to be subsumed under USAID/LTTP s work program. Dropped from the project 7. The revised PDO was to improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school-based management. 39

56 8. The revised project design reduced the number of components from three to one, with a focus on improving education management capacity. Subcomponents focused on the following four key areas: (a) (b) (c) (d) Strengthened institutional capacity for management and accountability at the central and school levels; School construction in rural areas; Procurement and distribution of teaching and learning materials; and School-based management through sub-grants. Component 1: Improving Management Capacity and Accountability Subcomponent 1.1: Strengthened Institutional Management Capacity and Accountability at the Central and School Levels (US$4.6 million plus US$0.9 million already disbursed for activities conducted before restructuring) 138. Significant output has been achieved in alignment with the design of this subcomponent. Throughout the life of the project, the PST supported capacity building and knowledge sharing within the MoE. ToRs for the recruitment of international specialists hired under the project included specific reference to capacity building for, and knowledge transfer to, MoE counterparts. This practice faced challenges during the EVD epidemic, as planned training and study tours had to be placed on hold. However, following the conclusion of the crisis, these activities were resumed. Capacity building for the PST was achieved through the use of Rapid Results Approach workshops. Objective Design Output Table 2.2. Design and Output for Subcomponent 1.1 To strengthen institutional management capacity and accountability at the central and school levels (a) Establishment of the PST under the MoE s Department of Planning, Research, and Development/Center for Education Management (b) Capacity building and training at the central and school levels for (i) annual Joint Education Sector Reviews; (ii) management and supervision of project subcomponents, including information, education, and communication campaigns; and (iii) M&E of project subcomponents (c) Provision of advisory services to the MoE to conduct an organizational development assessment and functional review at the central level (a) An effective PST was established. (b) Training provided for the MoE staff included (i) 16 MoE M&E staff completing M&E training at the Ghana Institute of Management and Public Administration and 13 procurement staff completing procurement training at the same institution; (ii) 3 planning staff receiving Master s Certificates in Education Planning and Management at the UNESCO International Institute for Education Planning in Paris and two staff receiving Master s Certificates in Development Project Management from IDEA Institute and University of Laval, respectively; and (iii) 9 MoE Division of Physical Environment staff completing a training in AutoCAD delivered by the GPE Project Architect. (c) A functional review was carried out to assess strategic alignment, work structure clarification, administration efficiency, and resource utilization. 40

57 (d) The PST developed a consultant and subcomponent activity tracking database to monitor deliverables. (e) The Rapid Results Approach Workshop Cycle 1 3 was conducted to facilitate the PST to better implement the project. (f) Liberia Crusaders for Peace was contracted to conduct the project information and education campaign. 9. One intermediate results indicator was defined for Subcomponent 1.1, that is, Joint Education Sector Review held. The MoE successfully held annual Joint Education Sector Reviews in every year of the project, with the exception of 2014 due to the impact of the EVD epidemic. Subcomponent 1.1 Related Indicators Intermediate results indicator: Joint Education Sector Review held Table 2.3. Indicator(s) for Subcomponent 1.1 Formally Revised in September 2012 Restructuring Note: a. Unless otherwise specified. Same for the rest. Baseline 2012 Target Time June 2015 a New No Yes Yes Revised Target Time June 2016 Actual Value Achieved Yes (except 2014) Subcomponent 1.2: School Construction in Rural Areas (US$16.2 million) 10. The subcomponent supported capacity-building activities in the planning, monitoring, and overall management of school construction, including upstream tasks such as the development of a school construction database, and post-construction tasks such the establishment of a set of school maintenance guidelines. The school construction subcomponent supported the construction and furnishing of 41 schools (including preprimary, primary, and lower secondary) on 35 sites. These figures reflect the cancellation of five sites due to concerns about the availability of grant funds and limited implementation time. Under Phase I of school construction, 14 sites were contracted to local firms, while under Phase II the remaining 21 sites were contracted. The MoE also developed a plan for procuring and distributing furniture to newly constructed schools, which entailed grouping the sites by geography and estimated completion date. The lots were bundled into two large packages. All furniture under both phases was manufactured and delivered to the beneficiary schools. 11. The project adopted an advanced standardized construction design determined by the Government to be used in all school construction (regardless of sources of funding). The design was bigger than that used previously for government-built schools and were costlier than what had been envisaged during Level I restructuring. During the Level II restructuring in June 2015, US$2.7 million of monies originally allocated to support school grants from Category 2 were reallocated to Category 1 to compensate for the cost overrun. The schools were designed to be built in district centers to accommodate a larger student body. The design categories of schools constructed by the project were: basic schools (12 classrooms), six-classroom schools, threeclassroom schools, and ECD centers (ECD + three classrooms). In addition to classrooms, each basic education school constructed in accordance with the new standardized design contained an administrative building, a library with information technology facilities, two toilet blocks, a multipurpose building to be used as a science laboratory, four units of staff quarters, a kitchen and dining hall, connected walkways, two hand pumps, and a football field. The project constructed 41

58 19 basic education schools, 3 six-classroom primary schools, 13 three-classroom primary schools, and 6 ECD centers in rural Liberia. 12. Bids submitted by local construction contractors demonstrated significant variance in quality, and there were insufficient numbers of qualifying bids for awarding the contracts. However, international contractors were very expensive and lacked the local knowledge necessary for successful implementation. To address the shortfall in qualified contractors, the PST used the Direct Contracting and Shopping methods for the tendering of some construction lots. Highperforming contractors, identified through Phase I construction, were rewarded contracts in Phase II. 13. Access to credit was identified as a critical challenge for construction contractors in Liberia. As a consequence, a set of mitigation measures were introduced, including more stringent contractor qualification criteria, an emphasis on supervision, and increasing the frequency of payment. However, the payment needed to be processed by the PFMU, and the processing bottlenecks made more frequent payment difficult. 14. School construction was negatively affected by Liberia s rainy seasons. Due to impassable road conditions, particularly in the southeast of the country, construction had to be halted entirely at some sites during the rainy season. For instance, the bridge to one site was flushed away, making access to the site impossible. Delays resulted in contractors exceeding cost estimates due to increased construction time and increased material costs. As a consequence, the PST negotiated variation orders for sites to reflect additional costs incurred by the contractors. Delays in completing school construction also increased the running costs associated with PST staff. 15. The CDD approach was dropped in both construction phases. It was envisioned that NGOs would serve as cosignatories for the community bank account to facilitate the transfer of funding to communities. However, agreement with NGOs could not be reached. During both construction phases, anecdotal evidence suggests that the contractors actively hired local labor, fulfilling to a certain extent the CDD aspect of the project s design. Objective Design Output Table 2.4. Design and Output for Subcomponent 1.2 Capacity building in the planning, monitoring, and overall management of school construction Under the revised project, the number of ECD centers and primary and lower secondary schools to be built remained the same as the original project: (i) 20 basic education sites (each with a primary and lower secondary school) and 4 primary sites with six-classroom schools to be supervised by a School Construction Management Firm and (ii) 16 primary sites with threeclassroom schools (8 of which will also have an ECD center) built through a CDD approach. Standardized school construction guidelines for site selection, design, contract management, and construction quality were to be developed and then approved by the MoE SMT. Capacitybuilding activities included upstream tasks such as developing a school construction database and post-construction tasks such as putting in place a set of school maintenance guidelines. (a) 303 additional classrooms were constructed and 41 schools were built on 35 sites using local contractors. (b) Many contractors reported using local laborers, indicating that the CDD approach was to some extent applied. (c) Standardized school construction guidelines were developed by an independent supervision company. (d) A school construction database was developed and updated. (e) School maintenance guidelines were developed. 42

59 (f) The MoE developed a plan for procuring and distributing furniture to the constructed schools and furniture was delivered to all schools. 16. All indicators related to Subcomponent 1.2 were achieved. All the constructed schools were completed according to the standardized school construction guidelines and designs (Basic, Primary, and ECD), fulfilling the PDO indicator. Moreover, the final target of constructing 272 classrooms was exceeded, and 303 classrooms were constructed with the support of the project by October Another intermediate results indicator regarding annual school construction database updated by annual Joint Sector Review was also achieved. Standardized school construction guidelines for site selection, design, contract management, and construction quality approved by the MoE SMT was achieved with the standardized school construction guidelines developed by the independent construction management firm. Subcomponent 1.2 Related Indicators PDO Indicator: Schools constructed according to standardized school construction guidelines under the project Intermediate results indicator: Number of additional classrooms built or rehabilitated at the primary education level resulting from project intervention Intermediate results indicator: Annual school construction database updated by annual Joint Sector Review Intermediate results indicator: Standardized school construction guidelines for site selection, design, contract management, and construction quality approved by MoE Senior Table 2.5. Indicator(s) for Subcomponent 1.2 Formally Revised in September 2012 Restructuring Baseline Target Time June 2015 Revised Target Time June 2016 Actual Value Achieved by October 2016 New 0 75% 75% 100% Continued (Core) New No Yes Yes Yes a New No Yes Yes Yes Management Team (SMT) Note: Though the annual Joint Sector Review was not held in 2014 due to the EVD epidemic, the school construction database was updated. 17. Reallocation of proceeds. During the Level II restructuring in June 2015, US$2.7 million of grant proceeds from Category 2 were reallocated to Category 1. The allocation of loan proceeds was revised, as shown in table 2.6.

60 Ln/Cr/TF TF Currency US$ Current Category of Expenditure Goods, works, and consultants services under the Project (except as covered by Category (2) below) Sub-grants under part 2 b 2 Designated Account Table 2.6. Reallocation of Funds Allocation Disbursement % (Type Total) Prerestructuring Postrestructuring Prerestructuring Postrestructuring 33,500,000 36,200, ,500,000 3,800, Total: 40,000,000 40,000,000 Subcomponent 1.3: Procurement and Distribution of Teaching and Learning Materials (US$11.0 million) 18. The MoE incorporated lessons learned from the EPF to improve the accountability of textbook distribution. It procured 1 million textbooks and 20,000 teachers guides for grades 5 9 in the fields of mathematics, science, social studies, and language arts from three suppliers: Pearson Education, Winmat Publications, and Laxmi Publications. The publishers delivered the books to each of the 15 County Education Offices, from which point the MoE assumed responsibility for delivering them to the school level. Approximately 1 million supplementary readers were procured by the MoE from suppliers Pearson Education, We Care, and Evans Bros. These materials were delivered to the MoE s warehouses in Monrovia and subsequently to the County Education Offices. Separately, two batches of EGRA readers (347,440 pieces) were delivered by Hanoi Printing to three county warehouses. The MoE signed contracts for the provision of approximately 1.4 million pieces of instructional materials in the fields of language art, social studies, and science with three suppliers: Pearson Education, Landmark, and Speedo Print. 19. According to the design, the same NGOs were expected to support both the distribution of learning materials and school grants. At least one NGO was to be assigned to each county. However, the NGOs that were retained for the distribution of school grants were too expensive and the project dropped the plan for using NGOs to distribute learning materials. Instead, the project team developed two distribution methods during the first round of distribution Stationary Distribution for hard-to-reach schools and school-level distribution for the rest of the schools. Distribution to schools occurred between June 17 and July , with 98 percent of materials distributed. The MoE validated requests from public schools not on the official list and schools that reported an increase in enrollment and finished the second round of distribution using other resources. 44

61 Objective Design Output Table 2.7. Design and Output for Subcomponent 1.3 To improve the management process through which learning materials were procured and distributed and also provide support to the MoE to monitor the effective utilization of learning materials in the classroom. The number of learning materials to be procured and distributed remained the same as the original project: (i) Textbooks in core subjects for grades 5 9, totaling approximately 1 million textbooks (ii) Teachers guides, approximately 20,000 The project was also designed to provide supplementary learning materials for grades 1 9 and storage containers. The MoE was to contract independent NGOs for the distribution of learning materials from the counties to the schools. The Government s accountability was expected to be strengthened by the supervisory role it undertook under this arrangement. (a) The project procured(i) over 1 million textbooks and 20,000 teachers guides for grades 5 9 in the fields of mathematics, science, social studies, and language arts; (ii) over 1 million grades 1 4 supplementary readers; and (iii) approximately 1.4 million pieces of grade 1 9 instructional materials. (b) Renovation of eight warehouses in Monrovia, Tubmanburg, and Gbaarnga was carried out to store the books before distribution. (c) A distribution plan, which involved two methods that dealt with Liberia s challenges of poor infrastructure, was developed. Stationary Distribution was carried out from a local center (usually the DEO) for hard-to-reach schools and School-Level Distribution was carried out for the rest of the schools. (d) The round of distribution in 2015 covered (i) 979,795 textbooks and 19,000 teachers guides for grades 5 9; (ii) 1,212,140 grade 1 4 supplementary readers; and (iii) 1,344,470 grade 1 9 instructional materials to schools that reported increases in enrollment or had not been captured in the previous education census. 20. Two PDO indicators were set for the sub-objective, that is, procurement of textbooks and teachers guides for grades 5 9 verified under the project and distribution of textbooks and teachers guides for grades 5 9 verified under the project. Both indicators were achieved and the project exceeded the latter target (distribution of materials) by the time the project closed. An intermediate results indicator for both Sub-objectives 1.2 and 1.3 was met social accountability framework for teaching/learning materials and school sub-grants programs developed and approved by the MoE SMT. The textbook utilization study is currently under way using other funds but was not completed by project closing. Subcomponent 1.3 Related Indicators PDO Indicator: Procurement of textbooks and teacher guides for grades 5 9 verified under the project PDO Indicator: Distribution of textbooks and teacher guides for grades 5 9 verified under the project Intermediate results indicator: Teaching and learning materials utilization assessment study conducted and reported Table 2.8. Indicator(s) for Subcomponent 1.3 Formally Revised in September 2012 Restructuring 45 Baseline Target June 2015 Revised Target June, 2016 Actual Value Achieved by October 2016 New 0 100% 100% 100% New 0 85% 85% 98% New No Yes Yes No (underachieved)

62 Subcomponent 1.3 Related Indicators Intermediate results indicator: Social accountability framework for teaching/learning materials and school sub-grants programs developed and approved by MoE Senior Management Team (SMT) Formally Revised in September 2012 Restructuring Baseline Target June 2015 Revised Target June, 2016 New No Yes Yes Yes Actual Value Achieved by October 2016 Subcomponent 1.4: School-based Management through School Grants (US$7.3 million) 21. The PST worked closely with Grants and Coordination in the Department of Planning on school grants activities. The school grants subcomponent began with a pilot program in fall 2013 with the distribution of funds to four counties, followed by the MCSS. In April 2014, a review of the pilot program was conducted to determine the degree of compliance with the School Grant Guidelines and obtain feedback from beneficiaries. The results of the review informed the revision of the school grants guidelines (for example, simplification of documentation and reduction of forms) and aided in the retooling of the training program. The national rollout of the program was anticipated for the 2014/2015 academic year but was impeded by the outbreak of the EVD epidemic and subsequent closure of schools. Despite these hurdles, the MoE proceeded with the recruitment of four NGOs to provide training, monitoring, and supervision of school grants in the 11 counties not included in the pilot. A total of 2,579 schools opened bank accounts and received the first tranche of their grant funds. A compliance review of the school grants program at the national level revealed that the use of the school grant was largely complied with by the schools, according to the School Grant Guidelines. 22. The project design intended for the responsibility for the school grant program to be gradually transferred from the GPE to the Government. The share of financial resources supporting the program were to be 100 percent from GPE support and 0 percent from the Government in the first year, with the transfer of 25 percent of the funding burden each year from the GPE to the Government until school grants were 100 percent funded by the Government in the final year. The Government provided US$790,000 for the second tranche of the school grant, assuming the responsibility a year earlier. Due to the late introduction of school grant under the project and loss of time during the EVD, it was determined that the surplus from school grant component would be reallocated to school construction to compensate for the cost overrun. In addition, there was not enough time under the project for the phased-in financing approach for school grants. Objective Design Output Table 2.9. Design and Output for Subcomponent 1.4 To enhance community participation in school-based management through the provision of school sub-grants. The subcomponent was to finance the development of School Grant Guidelines. As schools were likely to face difficulties meeting the sub-grant eligibility criteria (as outlined in the Financing Agreement), the project sought to support the MoE in signing contracts with local NGOs that were to be tasked with working with schools across all counties to meet the eligibility criteria. Finally, this subcomponent was also designed to fund the school grants. (a) The School Grant Guidelines were finalized and approved, satisfying the grant agreement s legal covenant related to the development of templates to capture expenditure information. (b) The school grants pilot program was launched in four counties and orientation training was conducted in 413 schools. 46

63 (c) The MoE has rolled out the program to all primary and junior secondary schools in the MCSS, serving nearly 20,000 students in the greater Monrovia area. (d) The MoE recruited four NGOs to provide training, monitoring, and supervision to schools for the national rollout. (e) A total of 2,579 schools opened bank accounts, prepared grant utilization plans, and received grant funds. (f) The first tranche, in the total amount of US$1.6 million, was provided under the project. The second tranche was provided by the Government of Liberia in the amount of approximately US$790,000. (g) Monitoring of the compliance of school grants was conducted by 4 NGOs in 11 counties and by the MoE in 4 counties. 23. One PDO indicator was defined for the sub-objective schools single signatories to own bank account (1,000). The final number of schools that opened bank accounts was 2,579, exceeding the target value. Two intermediate results indicators were met, which were School Grant Guidelines approved by the MoE SMT and school sub-grants implemented in accordance with School Grant Guidelines (30 percent). Subcomponent 1.4 Related Indicators PDO Indicator 5: Schools single signatories to own bank account Indicator 5: School Grant Guidelines approved by MoE Senior Management Team Indicator 6: School subgrants implemented in accordance with School Grant Guidelines (%) Table Indicator(s) for Subcomponent 1.4 Formally Revised in September 2012 Restructuring Baseline 2012 Target Time June 2015 Revised Target Time June 2016 Actual Value Achieved by October 2016 New 0 1,000 n.a. 2,577 New No Yes n.a. Yes New 0 30% 30% 72% 24. During the Level II restructuring in June 2015, that extended the project by one more year, most of PDO and intermediate indicators were also extended for a year to June 2016, with minor changes regarding the wording in some cases. Table Changes Made to PDO and Intermediate Indicators During June 2015 Restructuring PDO Indicators Indicator Name Core Unit of Measure Baseline End Target Revised Direct project beneficiaries Number 0.00 September 6, , June 29, 2016 Original a Direct project beneficiaries Number 0.00 September 6, , June 29, 2015 No Change Female beneficiaries Percentage Revised Schools constructed according to standardized school construction guidelines under the project Percentage 0.00 September 6, June 29, 2016 Original Schools constructed according to standardized 47 Percentage 0.00 September 6, June 29, 2015

64 PDO Indicators Indicator Name Core Revised Original Revised Original No Change Intermediate Results indicators Revised Original Revised Original No Change school construction guidelines under the project Procurement of textbooks and teacher guides for grades 5 9 verified under the project Procurement of textbooks and teacher guides for grades 5 9 verified by September 2013 under the project Distribution of textbooks and teacher guides for grades 5 9 verified under the project Distribution of textbooks and teacher guides for grades 5 9 verified by December 2013 under the project Schools single signatories to own bank account Unit of Measure Indicator Name Core Unit of Measure Number of additional classrooms built or rehabilitated at the primary level resulting from project interventions Number of additional classrooms built or rehabilitated at the primary level resulting from project interventions Annual school construction database updated by annual Joint Sector Review Annual school construction database updated by annual Joint Sector Review Standardized school construction guidelines for Baseline Percentage 0.00 September 6, 2012 Percentage 0.00 September 6, 2012 Percentage 0.00 September 6, 2012 Percentage 0.00 September 6, 2012 Number 0.00 September 6, 2012 Baseline Number 0.00 December 29, 2010 Number 0.00 December 29, 2010 Yes/No Yes/No Yes/No Yes September 6, 2012 Yes September 6, 2012 No September 6, 2012 End Target June 29, 2016, reference to September 2013 removed from indicator June 29, June 29, 2016, reference to September 2013 removed from indicator June 29, , June 29, 2015 End Target June 29, June 29, 2015 Yes June 29, 2016 Yes June 29, 2015 Yes June 30,

65 Intermediate Results indicators Revised Original No Change Revised Original Revised Original Revised Original Indicator Name Core Unit of Measure site selection, design, contract management, and construction quality approved by MoE Senior Management Team Teaching and learning materials utilization assessment study conducted and reported Teaching and learning materials utilization assessment study conducted and reported School Grant Guidelines approved by MoE Senior Management Team School sub-grants implemented in accordance with School Grant Guidelines School sub-grants implemented in accordance with School Grant Guidelines Social accountability framework for teaching/learning materials and school sub-grants programs developed and approved by MoE Senior Management Team Social accountability framework for teaching/learning materials and school sub-grants programs developed and approved by MoE Senior Management Team Joint Education Sector Review held Joint Education Sector Review held Note: Post September 2012 restructuring. Same for the rest. Yes/No Yes/No Yes/No Baseline No September 6, 2012 No September 6, 2012 No September 6, 2012 Number 0.00 September 6, 2012 Number 0.00 September 6, 2012 Yes/No Yes/No Yes/No Yes/No No September 6, 2012 No September 6, 2012 Yes September 6, 2012 Yes September 6, 2012 End Target Yes June 29, 2016 Yes June 29, 2015 Yes June 30, June 29, June 29, 2015 Yes June 30, 2014 Yes June 30, 2013 Yes June 29, 2016 Yes June 29,

66 Annex 3. Economic and Financial Analysis 1. The overall findings of the Economic and Financial Analysis are that the cost of the main activities under the project, school construction, textbook procurement and distribution, and school grants compare favorably with other donor projects in Liberia. Of the project s US$40 million, percent (US$39,963,547.68) was disbursed during the project duration and was spent toward project components. When the project closed, the Government had US$1,657 left in the project account. The efficiency and costs of school construction, learning materials, and school grants are examined in greater detail in the subsequent section of the annex. Table 3.1. Expenditure by Subcomponents Subcomponent US$, millions 1.1 Capacity Development School Construction Learning Materials School Grants 2.4 Total 40.0 Subcomponent 1.2: School Construction 2. The project supported the construction of 41 schools at 35 sites. A total of 303 new classrooms were constructed and furniture was delivered to all completed school sites. Furthermore, 72 teacher housing units (two-bedrooms apartments) were constructed, with furnished rooms, including toilets, kitchen, and living and dining areas. A total of 35 kitchens and 100 toilets were also constructed, covering all 35 sites. Table 3.2. Unit Cost of Construction under the Project School Facilities US$ Three-classroom primary school 111, Early childhood center 87, Early childhood center with three-classroom primary school 198, Six-classroom primary school 179, Basic education school 601, A classroom 15, Hand pump 4, Staff housing duplex unit 56, Source: Calculations from government expenditure on school construction report. 3. The project adopted a high-quality design of school construction. School construction supported by the project was originally intended to support the building of low-cost schools using only locally procured materials. However, in light of lessons learned through previous projects, it was determined that the majority of schools, built using local materials, did not meet quality standards and, as a consequence, incurred high maintenance costs. As a consequence, in October 2012 a high-quality standard school design was adopted by the Government that was to be used universally throughout Liberia, regardless of the source of financing (i.e., domestic or development partners). This choice was exercised with the expectation that the use of an architectural prototype with higher-cost and higher-quality specifications would enhance the quality and durability of the schools built and reduce future maintenance costs. The GPE-BEP complied with the requirement and used the standard design, despite its higher cost. 50

67 4. With the high-quality design, unit cost of construction schools under the project was higher than that of other fragile countries, but compared favorably with that of other donor-funded projects in Liberia. The per-classroom unit cost for a three-classroom primary school under the project was US$37,166. Data from recent ICRs in other low-capacity and fragile contexts (Benin, Burundi, Côte d Ivoire, Northern Uganda, and Tanzania) show per classroom construction costs between US$20,000 and US$25,000. However the project compares favorably with school constructed under the EPF with per classroom unit cost US$40,000. Unit cost for communitybased school construction under the LACE was lower (US$15,000) with a lower quality of material being used for construction. 5. These newly constructed schools have been fully incorporated within the public school system. According to EMIS (2015/2016), the 35 schools are staffed with 273 teachers and have nearly 11,000 students enrolled in the schools (4,591 ECE students, 5,349 primary students, and 875 junior high school students). Table 3.3. Current Students in the Newly Constructed/ Rehabilitated GPE Schools Schools Constructed/Rehabilitated Under the GPE Project ECE students 4,591 Primary students 5,349 Junior high school students 875 Total students 10,816 Female teachers 55 Male teachers 218 Total teachers 273 Source: Authors calculations made using Liberia s EMIS (2015/2016). 6. As assumed in the PAD, labor market benefits exceed the costs of construction and recurrent expenditure though the benefits are lower than estimated due to lower rates of enrollment in GPE schools than expected. The PAD assumed that an additional 2,400 students would enter the labor market annually after graduating from primary school as a direct result of school construction. The current graduating primary cohort (P6) from the GPE schools consists of 892 students based on EMIS 2015/2016 data. If it is assumed that the national average repetition rate of 9.2 percent at the P6 level is applicable to GPE schools, there would be approximately 810 new primary graduates annually from GPE schools (though cohort sizes are likely to increase over time). 15 Assuming the same wage benefits from primary education as the PAD does, the revised estimate of US$89,185 would be US$175,068 lower than anticipated for the current fiscal year. Similarly, the additional premium from junior high school education would have to be reduced to US$11,824. However, even at these lower rates of graduation than those assumed in the PAD, the benefits exceed the cost of construction and recurrent expenditure over time. Subcomponent 1.3: Learning Materials 7. Textbooks and supplemental material were distributed directly at the school level based on transparent data from the EMIS. Using the lessons learned from the evaluation of the 15 Students graduating from GPE schools may have completed their education from other schools if GPE schools had not been built or rehabilitated and so caution must be exercised in assuming that the 810 new primary cohort graduates are attributable to the project. 51

68 Liberia s EPF textbook procurement and distribution process, textbooks were distributed directly at the school level. This method helped improve efficiency in the procurement and distribution process, bringing down the unit cost of learning materials and mitigating the risk of illicit sale of procured goods. Moreover, the distribution network explored under the project made it feasible for the Government to regularly replenish school supplies. 8. This unit cost for textbooks was higher than that for other countries but lower than that of previous textbook procurement and distribution programs in Liberia. The project provided a total of 1 million textbooks for grades 5 9 and 1.4 million supplemental instructional materials for grades 1 9 at a unit cost of US$4 per textbook. These costs are in line with the costs indicated in Read and Bontoux (2015) --- unit cost of a primary textbook in Sub-Saharan Africa to be between US$2 and US$4. 16 However, recent projects under World Bank supervision have found lower costs for textbook procurement. For example, the unit cost for textbooks without delivery in the Democratic Republic of Congo for a 2015 education project was US$1.32. Within West Africa, the unit costs in Sierra Leone range from US$0.97 to US$1.66. In Niger, the unit costs range from US$1.10 to US$1.40. The unit cost in Eritrea (US$0.60) and Mauritania (US$0.73) are even lower. However, the higher costs in Liberia are in line with the costs for the textbook procurement and distribution of the 1.2 million textbooks under the EPF project where the unit cost was US$ The costs incurred during the project were higher due to economies of scale (lower quantities were purchased as compared to comparison countries and the individual procurement orders were not more than 200,000). Moreover, the textbooks were for a higher grade (grades 5 9) and thus the textbooks were more comprehensive (many more pages) as compared to textbooks procured for lower grade (for example, grades 1 6). In addition, the textbooks were in color and the paper quality was better with a shelf life of four to five years. 9. The beneficiary assessment attested to the fact that most schools had received the textbooks. During the beneficiary assessment, 88 percent of the random sample of schools surveyed indicated that they had received the textbooks and the number of textbooks that they had received were in line with their expectations. Schools also provided receipts highlighting the number of textbooks that they received. Subcomponent 1.4: School Grants 10. School grants were disbursed directly to the school bank accounts set up under the project to minimize leakages and improve transparency. Under the project, public schools were supported in setting up their own bank accounts. Rather than transferring funds to counties who would then transfer them to districts who would finally transfer them to schools, the direct transfer of funds to schools was swift and reduced the chance of corruption. The process of setting up the bank accounts has led to additional positive externalities future disbursements of school grants 16 Read, T., and V. Bontoux Where Have All the Textbooks Gone? The Affordable and Sustainable Provision of Teaching and Learning Materials in Sub-Saharan Africa: A Guide to Best and Worst Practices. Working Paper, World Bank, Washington, DC. 17 Burchell Evaluation of the Distribution of Primary Teachers Funded by the Education Pooled Fund to GoL Primary Schools. 52

69 through subsequent projects (including the new proposed GPE project) will use this mechanism to transfer funds. 11. Funds were allocated in an open and transparent manner using the latest available EMIS data. The PST allocated the funds using a base grant and per capita component to 2,579 schools using the enrollment figures from the latest available EMIS datasets. On average, a school received US$ The amounts received by the schools were verified by independent NGOs who were hired to conduct monitoring. 12. Over the project period, a total of US$2,433, was allocated toward school grants, at a unit cost of US$5.02 per student. A total of 483,565 students benefited from this grant, with 47 percent of the beneficiaries being female. According to independent verification carried out by NGOs, 72 percent of the expenditure was in accordance with the School Grant Guidelines. The beneficiary assessment supports this finding. Most schools used the funds to purchase teaching and learning materials, including basic supplies like chalks, in line with the guidelines provided to them. Fiscal Sustainability 13. The fiscal sustainability analysis indicate that Liberia has limited fiscal space to make substantial capital and recurrent investments and capitalize on the successes of the project. Nevertheless, the schools constructed and rehabilitated under the project have been mainstreamed and the teachers teaching in those schools are on the Government s payroll. Also, school grants will continue in the five most economically and educationally deprived counties under the new GPE project. However, there is no funding available for additional distribution of textbooks. Thus, the only fiscal implications of the project come from the recurrent expenditure of teachers salaries, who are currently teaching in the newly constructed/rehabilitated GPE schools, which amounts to approximately US$404, Liberia s budget is constrained and extensively donor driven, making it difficult for the Government to make substantial investments in education. In 2010, Liberia received US$1.42 billion in overseas development assistance (ODA), with a gross national income (GNI) of US$805 million. Thus, the ODA to GNI ratio in Liberia was 177 percent. This reduced to 116 percent by 2014, due to declining ODA and GNI, with ODA receipts totaling US$842 million and a corresponding GNI of US$720 million. However, Liberia s ODA to GNI ratio remains the highest in the world and the country has extremely scarce resources, which are primarily invested in infrastructure projects (construction of roads). 15. The MoE s non-salary budget is limited. Compensation of employees accounts for 78 percent of the MoE budget, leaving little room for other crucial activities such as the provision of learning and teaching materials to public schools, school grants, and construction of new schools in localities that lack access to education. 16. School construction. Even within this scarce resource environment, all schools constructed or rehabilitated under the GPE project have been mainstreamed into the public education system. The newly constructed schools have EMIS codes and the teachers teaching in the schools are on the government payroll at a recurrent expenditure of US$404,160. However, there is limited to no 53

70 budgetary allocation for maintenance and rehabilitation of all public schools in Liberia, which means that the schools run the risk of falling into a state of disrepair. 17. School grants. The Government has not been able to independently fund the provision of school grants. In the absence of government funding, the new GPE project GPE-G2B has allocated US$2.6 million to school grants. The grants will resume in the 2018/2019 school year and prioritize ECE schools in five economically and educationally deprived counties for the next three years. The grants are likely to be higher approximately US$10 per ECE student and so each school is projected to receive approximately US$1,000 over a period of three years from 2018/2019 to 2020/2021. As there is no matching component, the Government does not have any fiscal implications from the continuation of the school grants. 18. Learning materials. Like school grants, the Government has no resources available to procure and distribute additional textbooks to schools or provide any earmarked funding for their repair. Thus, there is no fiscal implication of this activity as well. Currently, there are no donor initiatives to fund this activity and no plans for additional procurement and distribution of textbooks. 19. Thus, the fiscal implications from the project are limited to the recurrent expenditure of teachers salaries in the newly constructed/rehabilitated schools to approximately US$404,160. These costs have already been included in the Government s recurrent expenditure on salaries and mainstreamed. 54

71 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Responsibility/Specialty Lending/Grant Preparation Nathalie Lahire Task Team Leader /Education Economist AFTED TTL Samuel Bruce-Smith Financial Management Specialist AFTFM Financial Management Maxwell Dapaah Financial Management Specialist AFTFM Financial Management Daniela A. Braganca Junqueira Legal Counsel LEGAF Legal Counsel Victoria Gyllerup M&E specialist AFTRL M&E Specialist Anthony Mensa-Bonsu Procurement Specialist AFTPC Procurement Specialist Michelle Neuman Early Childhood Development Specialist AFTED Specialist Nyaneba Nhrumah Environmental and Safeguard Specialist AFTEN Environmental Specialist Jonathan David Pavluk Senior Counsel LEGAF Legal Counsel Kristine Schwebach Social Development and Safeguard Specialist AFTCS Safeguards Specialist Atou Seck Sr. Education Economist AFTED Education Specialist Rajiv Sondhi Sr. Finance Officer CTRFC Financial Management Andy Tembon School Health Specialist AFTHE Health Specialist Albert Dupigny Education Specialist AFTED Education Specialist Mohamed I. Diaw Operations Analyst DFPTF Operations Specialist Norosoa Andrianaivo Team Assistant AFTED Team Support Rose-Claire Pakabomba Team Assistant AFTED Team Support David Hope Kanu Consultant AFTME Consultant Priscilla A. Elms Consultant AFTED Consultant Nigel Wakeham Consultant AFTED Consultant Supervision/ICR Peter Darvas Senior Economist GED13 TTL Douglas Sumerfield Senior Operations Officer GEDGE TTL Keiko Inoue Senior Education Specialist GEO03 TTL Charles Taylor Consultant GGO01 Procurement Richard Olowo Lead Procurement Specialist GGO06 Procurement Komana Rejoice Lubinda Senior Procurement Specialist GGO01 Procurement Winter Chinamale Senior Procurement Specialist GGO01 Procurement Luis Schwartz Senior Finance Officer Financial Management Laura McDonald Operations Officer GED13 Operations Officer Bernardo da Cruz Vasconcellos Education Specialist GED13 Education Specialist Saidu Goje Senior Financial Management Specialist GGO31 Financial Specialist Yinan Zhang Operations Analyst DFPTF Operations Analyst Lemu Ella Makain Program Assistant AFMLR Team Support Mugambi Mugisha Mwendia Finance Analyst WFALA Financial Management Janet Adebo Program Assistant GED13 Team Support Sheila Lucrecia Francisc Pene Senior Finance Assistant. WFALA Financial Management Kabira Namit Consultant GED07 Consultant/ICR co-author Yijun Shen Consultant GED13 Primary ICR author 55

72 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, thousands (including travel and No. of Staff Weeks consultant costs) Lending , Total: , Supervision/ICR , Total: ,

73 Annex 5. Beneficiary Survey Results 1. A beneficiary assessment survey was carried out in January 2017 by the MoE to identify successes and issues arising from implementation of the GPE project, as well as to solicit feedback from the beneficiary schools regarding the project. The primary objectives of the GPE beneficiary assessment were to (a) collect recommendations and lessons learned from beneficiary schools and (b) produce recommendations toward sustaining the gains of, and addressing challenges related to, the GPE-supported project going forward. 2. Over the course of two days (January 30 and 31, 2017), principals from 25 schools, which were equivalent to approximately 1 percent of beneficiary public schools, were randomly selected and interviewed. GPE-constructed schools were oversampled in the assessment and 9 of the 25 sampled schools were ones that had been constructed under the GPE BEP. School principals were interviewed by telephone regarding school grants, textbook distribution and utilization, and school construction. School Grants 3. The overall findings of the beneficiary assessment regarding school grants were as follows: (a) (b) (c) (d) Ninety-two percent of sampled schools confirmed that they had received school grants; almost all schools highlighted the fact that the grant amount was not received until the middle of the school year, which created cash flow and planning challenges. Ninety-six percent of sampled schools reported that they spent grant funds in line with the School Grants Utilization Plan. Twenty-four percent of schools reported receiving NGO support during the preparation and implementation of the grant. Schools were largely satisfied with the grant, although there were widespread concerns regarding delayed disbursement and the lack of continuity in school grants. 4. Ninety-two percent of the sampled schools confirmed that they had received the school grant. While the grant money was received, schools complained that the funds were not delivered until the middle of the school year. Moreover, 44 percent of the schools surveyed also reported that the grants that they received were lower than what they had anticipated. 5. Ninety-six percent of the schools surveyed reported that they spent the grant money in line with their School Grant Utilization Plan. The same proportion of schools reported that they found the Utilization Plan helpful, and that it guided their expenditure. Schools were also asked to provide examples of expenditure made using the school grant. Eighty-six percent of sampled schools indicated that they spent their funds on teaching and learning materials, (primarily stationery, chalks, and chalkboard). Other common expenditure line items reported, included minor repairs (primarily to leaking roofs) and recreational materials (football and football jersey). 57

74 Figure 5.1. Use of School Grants Response to the question: What did you use the school grant money for? 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 86% TLMs (including chalk and stationery) Note: TLM = Teaching and Learning Material 64% Minor Repairs 45% Recreational material (primarily footballs) 6. In general, the schools surveyed reported that they were satisfied with the support they received from NGOs and PTAs. Approximately one in every four schools (24 percent) reported receiving NGO support and training for setting up the bank account and planning grant utilization. The schools that did receive NGO support reported being relatively satisfied with the support received. A number of schools reported being particularly appreciative of the support that they received in setting up bank accounts to receive school grants. Of the schools surveyed, 96 percent reported a functioning PTA, and of schools with a functioning PTA, 92 percent reported that school grant funds were utilized in consultation with the PTA. 7. Overall, schools were generally satisfied with the school grants program. However, 82 percent of schools raised concerns regarding delays in disbursement. Schools also expressed concern regarding the lack of a consistent flow of school grants from year to year. Textbooks 8. The beneficiary assessment presented the following findings on textbooks: (a) (b) (c) (d) Of the schools surveyed, 92 percent confirmed that they had received textbooks under the program supported by the GPE project. A majority of textbooks were delivered directly to the schools. The delivered textbooks were in good condition. Schools were generally satisfied with textbooks. 9. Of the schools surveyed for the beneficiary assessment, 92 percent acknowledged receipt of textbooks. A majority of schools indicated that they had not had sufficient numbers of textbooks 58

75 before the delivery of new textbooks through the project. Nearly 70 percent of schools that confirmed receipt of textbooks reported that the textbooks were delivered directly to the school, while the remaining 30 percent indicated that they collected their textbooks from DEOs. Figure 5.2. Method of Delivery of Textbooks Method of delivery of textbooks 80% 70% 70% 60% 50% 40% 30% 30% 20% 10% 0% Delivered to the schools Had to pick up from DEO's office 10. All schools that reported receipt of textbooks reported that the textbooks were received in good condition and in quantities expected. Upon receipt, schools predominantly chose to store the textbooks in the school library. If the recipient school did not have a school library, the textbooks were stored in the principal s office or in a warehouse/storeroom. Figure Storage of Textbooks Storage of Textbooks 70% 60% 59% 50% 40% 30% 20% 10% 23% 18% 0% School Library Principal's office Warehouse/ Store Room 59

76 School Construction 11. Principals of schools constructed by the GPE-supported project, who were surveyed, reported being satisfied with the condition of the school when it was handed to them, as well as the current condition of the school. In this regard, the responses of principals of schools constructed by the project were significantly different from other principals, who universally reported dissatisfaction with school infrastructure, primarily related to nonfunctioning toilets or leaking roofs. 12. All the surveyed principals of GPE-constructed schools reported that the classrooms and toilets constructed by the project were still fully functional. Moreover, 37.5 percent of GPE schools reported receiving support from the community for the maintenance of schools. Recommendations for Subsequent GPE-supported Projects 13. School grants. The primary recommendation forwarded by sampled schools was a desire for the school grant program to continue and for the timely disbursement of funds at an agreed upon time (at the start of the school year). The schools also expressed a wish for the grant money to be increased in future. Finally, respondents were concerned that in instances where a school principal is replaced, the new principal will not be a signatory to the account in which the school grants are deposited. It was requested that a process be implemented in which new principals would automatically become signatories to school bank accounts, with the power to withdraw funds. 14. Textbooks. Schools requested that textbooks to be provided for all grades, that teacher guides be distributed to them regarding the effective use of textbooks, and that the textbooks should be supplied on a regular, rather than a sporadic, basis. Schools also requested additional copies of the national curriculum, ECE textbooks, and that the education sector achieve a textbook to student ratio of 1:1. 60

77 Annex 6. Stakeholder Workshop Report and Results Not applicable. 61

78 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR Purpose of GPE BEP Report 1. This Government summary report seeks to draw to a closure the implementation and evaluation of the GPE BEP in Liberia. The project commenced on August 1, 2011, and ended on October 31, The PCR provides an assessment and understanding of the GPE BEP designs, objectives, and execution experiences and challenges providing a reflection of lessons learned by the Government during the implementation process. Introduction 2. Country context. Liberia is postwar country located on the west coast of Africa. It has an estimated 38,500 square miles bordered by Guinea on the North, Sierra Leone on the west, Côte d Ivoire on the east, and the Atlantic Ocean on the south. According to the 2008 National Population and Housing Census report, Liberia had approximately 3.5 million people during the time of project design with a projected growth rate of 3.8 percent serving as one of the fastest growing populations in the world according to the United Nations Population Fund. More than half of the population is below the age of Moreover, more than half the Liberian population lives in the hinterlands and relies on subsistence farming and hunting for livelihood. Though the country is steadily improving on the indicators of the Human Development Index after the end of the civil war, it is still ranked low in comparison to its peers in the subregion. The United Nations Development Programme 2015 report puts life expectancy at 61 years, thus showing a steady improvement. However, more than twothirds of its citizens experience multidimensional poverty, with about 68 percent living below the income poverty line (United Nations Development Programme 2013). 4. Education sector context. The GoL continues to make efforts in increasing public spending to the education sector. However, it still lags behind its neighbors. For example, Sierra Leone and Côte d Ivoire allocated 18.1 percent (in 2009) and 24.6 percent (in 2008) of their total government expenditure to the education sector. Currently, the education sector makes up only 14.3 percent of the national budget, which is grossly insufficient to meet the demand for education services across the country. More than half of the overall amount to the sector goes to the MoE, with a combined total of about 90 percent accounting for only staff and teachers compensation. 5. The sector continues to make strides in improving accessibility to meet demand and improving learning outcomes. However, it is faced with some obvious challenges, such as low public expenditure to the sector, testing and placement of qualified teachers, cultural barriers affecting girls enrollment and retention, and bad road network in some parts of the country. Background and Objectives of GPE BEP 6. The GPE BEP obtained grant financing from the GPE. The World Bank served as the grant supervising agent and the project was implemented by the MoE-Liberia. The project was initiated on January 1, 2011, and closed on October 31, The GPE BEP sought to enhance the planning, implementation, and monitoring capacity of the current efforts of the GoL in the efficient operationalization of a decentralized education system. Thus, the project was designed to spur the 62

79 supply of education services in disadvantaged and hard-to-reach communities. The PDO aimed to improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school-based management. After the restructuring in 2012, the project had four components which were linked to the ministry s ESP ( ) and the Operational Plan ( ). The completion of milestones leading toward the attainment of the project s objectives was measured by (a) Provision of training and scholarship to staff of central and local offices; (b) The accessibility of primary, basic, and early childhood schools in rural Liberia; (c) Distribution and utilization of textbooks and teaching material to schools across the country, excluding partnership schools; and (d) Opening of school signatory accounts. Project Subcomponents 7. Capacity Building (US$5.5 million). Strengthened Institutional Management Capacity and Accountability at the Central and School Levels. Under this subcomponent, training and scholarships were provided to staff at the central and schools level in areas where capacity was severely lacking to improve the management and supervision of education services within the decentralized systems. Specific training and capacity building of staff were provided in the areas of procurement, finance, project and grant management, and M&E. 8. School Construction in Rural Areas (US$18.8 million). This subcomponent provided the creation of additional access to education, particularly in disadvantaged counties and communities. It allowed the construction of new schools with a standardized design and furniture. A total of 303 classrooms with 100 toilet facilities were constructed at 35 different sites. Furniture was supplied to all completed facilities, including 72 teachers housing units. 9. School Grant (US$2.5 million). Under this subcomponent, the project provided support to the enhancement of management and accountability of governance structures, such as the county schools boards and PTAs. A total of 2,579 schools acquired signatory accounts and received grants according to the guidelines and directly from central level. A compliance analysis conducted showed that more than two-thirds of all public schools met the eligibility criteria for the grant, with over US$2.4 million disbursed. 10. Learning materials. This subcomponent entailed the improvement of the education quality and learning outcomes through the procurement and distributions of teachers guides and textbooks across the country. A little close to 1 million textbooks (966,237) have been distributed for grades 5 9 in core subjects (mathematics, language arts, science, and social studies). Also, a distribution of 1,013,200 supplementary readers for grades 1 4 and 1,216,745 of different instructional materials for grades 1 9 was completed out of the 1. 4 million procured, respectively. A randomized survey of textbook utilization in 8 of the 15 counties remains outstanding and is undertaken using funds from other sources. 63

80 Implementing Agency 11. The GPE BEP was implemented by the MoE through an appointed PST and coordinator. The Department of Planning headed by the Deputy Minster was responsible for the planning, monitoring, supervision, and evaluation of the project s activities and outcomes. Different departments and units were integrated into the management structure to mitigate potential overlapping and lack of coordination to ensure delivery of the PDO. Over the life span of the project, the World Bank served as the grant supervising agent, providing technical support to every aspect of project management. Project Restructuring 12. The GPE BEP underwent restructurings to align to the goals of the ministry s strategic plan for the sector. In November 2011, the project subcomponent was revised to include the construction instead of rehabilitation only of ECD schools. The second restructuring was the revision of the PDO from increase access to basic education to improve management capacity and accountability at central and schools. The Results Framework for M&E was modified to align with the objectives of the revised PDO, with an extension of the project closing date from June 2016 to October 2016 Beneficiaries 13. The key beneficiaries of this project include the Liberian children, many of whom are still struggling to have access to universal quality early childhood development and primary education. The project also benefited teachers, with new housing facilities being built for them and the distribution of guides to make their job in the classrooms easier (Implementation Status and Results Report 2016). Project Development Objective Indicators Direct project beneficiaries (Number, Core) Baseline Actual (Previous) Actual (Current) End Target Value , , , Date September 6, 2012 December 17, 2014 May 1, 2016 June 29, 2016 Female beneficiaries (Percentage, Core Supplement) Baseline Actual (Previous) Actual (Current) End Target Value Date December 17, 2014 May 1, 2016 June 29, 2016 Schools constructed according to standardized school construction guidelines under the project (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value

81 Date September 6, 2012 May 1, 2016 September 30, 2016 September 30, 2016 Procurement of textbooks and teacher guides for grades 5 9 verified by September 2013 under the project (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value Date September 6, 2012 May 1, 2016 October 31, 2016 October 31, 2016 Distribution of textbooks and teacher guides for grades 5 9 verified by December 2013 under the project (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value Date September 6, 2012 December 17, 2014 October 31, 2016 October 31, 2016 Schools single signatories to own bank account (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value , Date September 6, 2012 Source: Project Implementation Status and Results Report. December 17, 2014 May 1, 2016 June 29, 2016 Safeguard and Fiduciary Compliance 14. The project, for the most part, was concentrated on the delivery of education services to disadvantaged communities based mainly in the very remote areas of the country. Mitigating the negative impacts consistent with the requirements of these safeguard policies, the project prepared an ESMF and an RPF which were published on most media outlets across the country and posted at the World Bank Public Information Center. In addition to describing the environmental and social screening process, the ESMF made recommendations regarding the need for capacity building to ensure its effective implementation as well as consultations with potentially affected persons. This step was part of the screening process which took place during the time of construction. 15. The project recruited two procurement specialists for at least 18 to 36 months. These specialists provided training and guidance, as a way of knowledge transfer in enhancing the confidence of the support staff in conducting and managing procurement. The PFMU at the MoF was charged with the responsibility to manage funds under the Catalytic Fund. Data Collection, Analysis and Reporting 16. The GPE BEP employed a two-pronged approach to data collection, analysis, and reporting: 65

82 a) Data collection by project staff and subcomponent teams b) Participation by MoE divisions/units, including M&E, EMIS, and PTA c) Other partners like NGOs Achievement of PDOs 17. The effectiveness of the GPE project in Liberia, according to the evidence assessed, indicates that the activities varied in the degree to which they met their specific stated objectives. However, in most instances the majority of or all objectives were met to a high degree. It can be established that this project has high completion rating and met its targets in the four subcomponents, namely school construction, learning materials, school grants, and capacity building. The school construction component of the GPE project is the first project undertaken at the MoE that exceeded the target. In the project document, the target was to construct 272 classrooms. However, the target was exceeded and 303 classrooms were constructed under the project. School grants have provided hope to schools in the hard-to-reach counties. Minor repairs to most of the schools as well as essential materials such as chairs, tables, clocks, students sporting materials, and educational materials were obtained with the support from the school grant. The result of school grant support showed that students enrollment had increased and teachers absenteeism had also reduced even though some teachers used their compensation issue to stay out of school. Though the project beneficiaries target was only 473,000, the project exceeded the target and served 484,000 beneficiaries. Moreover, the target for schools to have their own bank accounts was also significantly exceeded. 18. The PST procured all the learning materials for the MoE and distributed 98 percent to all public and community schools nationwide. This component was a major challenge for the PST owing to bad road conditions during the rainy season. However, with a well-coordinated and organized distribution plan, the PST was able to accomplish the task. The capacity-building component of the project awarded scholarships to three staff of the MoE through a competitive process to obtain a Master s of Science certificate from UNESCO International Institute for Educational Planning in Paris and another staff obtained a Diploma in Education Planning. Staff were offered opportunities to study various short-term courses in Ghana, Kenya, and Tanzania. As a result of this intervention, the MoE now has capacity in procurement, grant management, education planning, and architecture design. Monitoring and Evaluation (M&E) Design, Implementation and Utilization 19. The M&E Plan of the GPE M&E system was to track the project s delivery of expected results and support Liberia MoE s M&E needs. The overall goal of the M&E system was to provide critical information for decision making and assist in guiding the subcomponent teams to implement project activities. This goal recognizes that specific elements of the project implementation may require adjustment to respond to evolving conditions either within or outside the project. 20. The GPE M&E plan also targeted data collection primarily on activities directly implemented by the subcomponent teams and the impacts of those activities on their beneficiaries. Information that was captured included secondary impacts through field visits and special studies. 66

83 This GPE M&E plan was based on an effect-design that linked program outputs to desired outcomes and impact. 67

84 Overview of Implementation Results Framework and Monitoring PDO: To improve management capacity and accountability at the central and school levels for school construction in rural areas, procurement and distribution of teaching and learning materials, and school-based management PDO Level Results Indicators 1. Direct project beneficiaries (number) - of which female (%) 2. Schools constructed according to standardized school construction guidelines under the project 3. Procurement of textbooks and teacher guides for grades 5 9 verified by April Core X D=Drop C=Cont N= New R=Revise C Unit of Measure Number (%) Baselin e 2008 Cumulative Target Values 2011 YR1 YR2 YR ,000 (44%) 591,000 (44%) 473,000 (44%) Frequency Annual N % Annual N % % 100% Annual Data Source/ Methodology Project supervision report Division of Education Facilities verification reports County Education Offices warehouse reports Responsibility for Data Collection MoE Planning Department MoE Planning Department MoE Administration Department Notes Number of grade 1 9 students benefitting from school grants; government budget to be phased in years 3 and 4, and therefore the decrease in the number of beneficiaries Percent of teacher guides and textbooks received at County 68

85 2013 under the project 4. Distribution of textbooks and teacher guides for grades 5 9 verified by September 2013 under the project N % % Annual 1. Delivery confirmation notes by school 2. Learning materials audit report 5. Schools single signatories to own bank account N Number ,000 Annual NGO report Intermediate Results Indicators 1. Number of additional classrooms built or rehabilitated at the primary education level resulting from project intervention 2. Annual school construction database updated by annual Joint Sector Review Intermediate Result (Component 1): Improved education management capacity X C Number Semiannua l N Yes/No No No Yes Yes Yes Annual Project supervision report List of new entries, revisions, and completed schools in school MoE Planning Department MoE Planning Department MoE Planning Department MoE Planning Department Education Offices Percentage of schools with delivery confirmation notes that verify distribution Schools become single signatories after confirmed financial management capacity 69

86 3. Standardized school construction guidelines for site selection, design, contract management, and construction quality approved by MoE Senior Management Team 4. Teaching and learning materials utilization assessment study conducted and reported 5. School Grant Guidelines approved by MoE Senior Management Team 6. School grants implemented in accordance with School Grant Guidelines (%) N Yes/No No No Yes Yes Yes Annual N Yes/No No No No Yes Yes Annual N Yes/No No No Yes Yes Yes Annual C % 0% 0% 0% 10% 30% Annual construction database Guidelines (document) and approval letter from MOE SMT Assessment study report Guidelines and approval letter from MoE SMT School grants audit report MoE Planning Department MoE Instruction and Planning Departments MoE Planning Department MoE Planning Department Draft guidelines are in progress. Share of schools following School Grant Guidelines, which will be defined in the SGG; consequences for schools that do not adhere to the 70

87 SGG will also be defined. 7. Social accountability framework for teaching/learnin g materials and school grants programs developed and approved by MoE Senior Management Team N Yes/No No No Yes Yes Yes Annual 8. Joint monitoring field N Yes/No No No Yes Yes Yes Annual visit conducted Source: Proposed Project Restructuring Paper, Framework (document) and approval letter from MoE SMT Project supervision report MoE Planning Department MoE Planning Department Social accountability framework to improve feedback from communities, schools, and NGOs to the MoE 71

88 Lessons Learned 21. The project performance goals were not specific enough to measure achievements through qualitative and quantitative assessments (standardized performance appraisals). Training (type/duration/content/sequence) was identified as a solution to a performance constraint but was not tailored to optimize performance outcomes. The package of proposed capacity development interventions that emerges from the assessment was not strongly coherent to ensure a definable combined impact that is beyond improving the skills of individuals in specific critical areas to alleviate organizational performance constraints within and across the departments. 22. The Government faced challenges in dealing with local contractors and lack of community engagement. Many contractors had limited cash flow to finance projects that affected construction duration and materials supply. Adequate community engagement on site selection and use of land was essential. Access to local materials like timber, sand, and crushed rocks was a challenge in some areas where communities were not effectively engaged. Communities did not have the technical and equipment capacity to do site clearing. The contract duration at different levels of school construction was not realistic, given the prolonged rainy season and bad road network. 23. The Government faced significant challenges with textbook distribution. Storage has always been a problem for the ministry, especially at the county level. As a means of mitigating this issue, the project renovated the MoE s major warehouses in the counties and provided forklifts and other warehouse essential materials to aid the distribution exercise. Where there were a large number of teams (for example, 16 county teams in the first round of nationwide distribution), managing their deployment was a huge challenge, considering the limited resources available. It was therefore prudent to ensure a properly planned, well-managed, and scheduled deployment of the teams to allow for last-minute exigencies. 24. Stamping and sorting of materials. The stamping and sorting process was a tedious exercise. There was a need to properly plan for this exercise and allot sufficient time to carry it out. Therefore, it is important to consider this in subsequent and future books/teaching and learning materials distributions. 25. The constant change or transfer of principals and DEOs also had an adverse effect on school grants utilization. To mitigate this, there is a need to professionalize the position of DEOs and school principals by developing a school leadership recruitment, training, deployment, retention, and exit policy. A longer period between disbursement and monitoring of grants in the field could lead to unnoticed irregularities. The MoE can solve this problem by institutionalizing school supervision, with clear roles and mandate for school inspectors to conduct regular supervision. It is difficult to have a proper functioning public school in the face of school fees abolition without grants support. To ensure sustainability of the Government s commitment to school-based support, the Government should continue the current school grants program or introduce a subsidies policy for public schools. The Government should include, in its budget, the school grants program and revise the guidelines to address issues of equity and disadvantaged communities. 72

89 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not applicable. 73

90 Annex 9. List of Supporting Documents Aide Memoires and Project Implementation Status and Results Reports ( ). Grant Agreement and amendments GPE Baseline Study Questionnaires 2013 GPE Final Report GPE Project Update Reports and PowerPoint Presentations ( ) IMF (2016) Liberia Selected Issues paper IMF Country Report No. 16/392, December 2016 Report on Construction of Education for All Restructuring papers, September 2012 Sample of Receipt of Teaching Materials School Construction Final Report School Construction Database School Grant Guidelines NGO School Grant Monitoring Reports by Agency for Economic Development and Empowerment, Child Fund, Liberia Educational Action for Development, and Adventist Development and Relief Agency Liberia MoE School Grant Monitoring Reports, September 2015 Summary of Recommendations and Conclusions of the Quality Enhancement Review Panel for Liberia: Fast Track Initiative Grant for Basic Education Project (March 15, 2012) Final Report on the Distribution of Textbooks and Instructional Materials to Public and Government-owned Community Schools in the Republic of Liberia Burchell Evaluation of the Distribution of Primary Teachers Funded by the Education Pooled Fund to GoL Primary Schools. Read, T., and V. Bontoux Where Have All the Textbooks Gone? The Affordable and Sustainable Provision of Teaching and Learning Materials in Sub-Saharan Africa? A Guide to Best and Worst Practices. Working Paper, World Bank, Washington, DC. 74

91 Annex 10. Indicators under the Project Indicator Direct project beneficiaries (number) - of which female (%) Schools constructed according to standardized school construction guidelines under the project (%) Procurement of textbooks and teacher guides for grades 5 9 verified under the project (%) Baseline Original (2010) Original end target Level I Restructuring (Sep 2012) Achievement noted in restructuring paper Indicator Change Revised Target Level II Restructuring (June 2015) Achievement noted in restructuring paper Indicator Change Revised Target Level II Restructuring (June 2016) Achievement noted in restructuring paper 0 470,000 0 Yes 430,000 No 430, ,565 No 430, ,565 0 New 75% 100% 0 New 100% 100% No 100% 100% No 100% 100% Indicator Change Revised Target Project Closing Achievement Comments Targets were exceeded. Source: EMIS data The target was exceeded by project completion. Source: Constructio n Monitoring Report and Beneficiary Assessment. The target was achieved by the end of the 2014/2015 fiscal year. Source: Project Progress Report 75

92 Distribution of textbooks and teacher guides for grades 5 9 verified under the project (%) Schools single signatories to own bank account School grants implemente d in accordance with agreed upon procedures (%) Basic education completion rate (%) Textbook- Student Ratio at Original (2010) Level I Restructuring (Sep 2012) Level II Restructuring (June 2015) Level II Restructuring (June 2016) 0 New 85% No 85% No 85% 98% 0 New ,579 No ,579 No , Moved to Intermediate Results Framework Dropped 0:0 1:2 Dropped Project Closing The target was exceeded by project closing. Source: Project Progress Report This target was significantly exceeded as 2,579 new bank accounts were created. Source: Project Progress Report. 76

93 basic education level Teachers having at least one teaching guide (%) Schools managed by a School Managemen t Committee (SMC) or Parent- Teacher Association (PTA) (%) Original (2010) Level I Restructuring (Sep 2012) Level II Restructuring (June 2015) 0 80 Dropped 0 60 Dropped Level II Restructuring (June 2016) Project Closing 77

94 MAP 78