L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD Dec ,000, Original Commitment 13,400,

Transcription

1 Public Disclosure Authorized Independent Evaluation Group (IEG) 1. Project Data Report Number : ICRR Public Disclosure Authorized Project ID P Country Kazakhstan Project Name KZ Tech Commercialization Practice Area(Lead) Trade & Competitiveness L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD Dec ,000, Bank Approval Date 15-Jan-2008 Closing Date (Actual) 31-Dec-2015 IBRD/IDA (USD) Grants (USD) Public Disclosure Authorized Public Disclosure Authorized Original Commitment 13,400, Revised Commitment 11,349, Actual 11,349, Sector(s) General industry and trade sector(60%):central government administration(20%):tertiary education(20%) Theme(s) Other Private Sector Development(100%) Prepared by Reviewed by ICR Review Coordinator Group Nestor Ntungwanayo John R. Eriksson Christopher David Nelson IEGFP (Unit 3) 2. Project Objectives and Components a. Objectives The development objective of the project is "to demonstrate significantly improved scientific performance and commercial relevance of research performed by inter-disciplinary teams of scientists selected through transparent competitive process" [Loan Agreement (LA), p.4]. The project appraisal document (PAD) had a similar objective (p.24). b. Were the project objectives/key associated outcome targets revised during implementation?

2 No c. Components The project has three components as delineated below: Component 1: Strengthening Kazakhstan's Science Base. (Appraisal cost of US$46.97 million; Actual cost of US$35.70 million). This component intended to identify and develop leading scientists and talented young researchers so that they can conduct Research and Development (R&D) at international standards through (a) the establishment of an International Science and Commercialization Board (ISCB) and an internationally peer-reviewed journal; (b) a Senior Scientific Groups (SSG) grant program and a Junior Research Group (JRG) grant program; and (c) the establishment of the International Materials and Science Center (IMSC). During the July 2012 restructuring, the IMSC s role was adjusted to (a) serve as a benchmark for other laboratories throughout Kazakhstan with regard to adopting international good laboratory practices (GLPs); (b) improve the access to information on research equipment in national and engineering laboratories; and (c) promote collaboration among these laboratories. The decision was made to build the IMSC using one of the existing laboratories, selected through a competitive tender process. Component 2: Linking Kazakh Science to Markets. (Appraisal cost of US$15.75 million; Actual cost of US$18.16 million). The component aimed to provide technical assistance for the development of expertise required to link science more closely to domestic and international markets through (a) the establishment of a Technology Commercialization Office (TCO) to provide a range of technology commercialization services, including the development of a technology commercialization strategy, and the development of targeted training programs in foreign languages, business planning, and communication; (b) undertaking a technology audit to inform policy makers and scientists about innovation assets and to compile a knowledge map database for the marketing of technology; (c) a review of legal and regulatory provisions, structures, and incentives related to technology commercialization, industrial innovation, venture capital and financing, and intellectual property (IP); and (d) conducting a market assessment to complete design and administration of a range of borrower-financed grant programs. During the July 2012 restructuring, the innovation grant program was redesigned to cover two types of activities: (a) proof of concept and (b) industrial prototype activities. Component 3: Project Management Unit.(Appraisal cost of US$9.55 million; Actual cost of US$8.63 million): This component aimed to support the program management through the establishment of the project management unit (PMU) for project coordination, implementation, and management. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost: This is an IBRD Specific Investment Loan (SIL) for a total cost of US75.0 million at appraisal. Actual cost amounted to US$65.22 million, or 87 percent of the appraised amount. Financing: The project was funded by an IBRD Specific Investment Loan, and resources from the borrower contribution. Out of the appraised amount of US$75.00 million, only 87 percent of the amount was disbursed, with the remaining 13 percent cancelled at project closure. Borrower Contribution: The Kazakh Government contributed to the project funding in the amount of US$53.87 million, or 83 percent of the disbursed amount, with the IBRD loan contributing for the remaining 17 percent of the project funding. Dates: The project was approved on January 15, 2008, and became effective on December 15, It was restructured twice: (i) on July 25, 2012, to adjust the IMSC s role and redesign the innovation grant program, and to extend the project closing date until the December 31, 2015, and (ii) on December 28, 2015, to provide the Borrower with an opportunity to cancel unused and uncommitted funds in the amount of US$1.8 million from the Loan, because the funds could not be utilized prior to project closing on December 31, The project was closed on the new schedule 3. Relevance of Objectives & Design a. Relevance of Objectives The project development objectives were highly relevant, as they were aligned with Kazakhstan s economic development priorities, and were at the core of successive Bank's county strategies. Kazakhstan s leaders have maintained a strategic vision for economic development, based on integration into the global economy through the adoption of international standards for the productive, financial and public sectors. In 2003,

3 Kazakhstan adopted its National Strategy for Industrial Innovation Development (NSIID) for , which aimed at laying the legislative and institutional foundations for economic diversification. In particular, the development programs focused on issues related to competitiveness, such as the investment climate, institutions, human capital, basic infrastructure, and the environment. In turn, the Government's priorities were consistent with the three pillars identified at the time in the country's Bank Country Partnership Strategy (CPS) for Kazakhstan as follows:(i) reducing losses in competitiveness through prudent management of the oil windfall and increased public sector efficiency, (ii) promoting competitiveness by strengthening the government s capacity to identify and reduce barriers to businesses and private investors, and (iii) building the foundation for future competitiveness by investing in human capital and basic infrastructure. The PDO remained relevant until the period of project closure as the first pillar of the Bank s FY12 17 CPS was focused on improving competiveness and fostering job creation through the expansion of non-oil sector exports and employment, and strengthen knowledge for sustained growth. Similarly, the project was still aligned with the May 2015 National Plan announced by the Government, which outlined steps to join the top 30 developed countries by 2050, and one of these steps aimed specifically to reform the scholarly grant and program structure to reflect the needs of the State Program of Accelerated Industrial and Innovative Development. Rating High b. Relevance of Design The PDO was clearly stated, as it underscored the achievement of results toward an improved scientific performance and commercial relevance of research performed by the country's inter-disciplinary teams of scientists. The project supported two categories of activities: the first category aimed to strengthen the science base, while the second one was to link Kazakh science to markets. There was a causality chain between the identified activities and the expected outputs and outcomes. For instance, the set up of institutions, groups and centers devoted to generating and overseeing research activities was the logical approach to improve the scientific performance. Similarly, establishing a technology commercialization office, and developing a technology commercialization strategy were among the needed key steps toward linking science to markets. However, it was unclear how the above expected achievements could be demonstrated credibly. Rating Substantial 4. Achievement of Objectives (Efficacy) PHREVISEDTBL Objective 1 Objective Improved scientific performance of the groups selected through transparent and competitive process Rationale Project efficacy is illustrated by achieved outputs and outcomes under the first sub-objective as summarized below: Outputs: The revised target number of internationally co-authored articles by researchers in Senior and Junior Scientist Research Groups was exceeded by 73 percent, reaching the number of 38 against a target of 22. The target number of articles in national research journals and conferences by researchers in Senior and Junior Scientist Research Groups was exceeded by 28 percent, reaching 142 against a target of 113. The annual revised number of paid graduate students involved in the Senior Scientific and Junior Scientific Groups was exceeded, reaching 16 in 2012, 35 in 2013, 58 in 2014, and 61 in 2015.

4 The number of partnerships established between local researchers and international researchers measured in terms of number of agreements was exceeded, reaching the level of 77, against a target of 28. There is no reliable estimate of actuals for the rate of equipment use in national and engineering laboratories by Senior Scientific and Junior Scientific Research Group scientists (%). Outcomes: The number of internationally co-authored articles by researchers in Senior and Junior Scientist Research Groups, with impact factor, -articles in reputable international journals in the research areas of TCP groups- exceeded the initial target by 400%, from a revised target of 25 to an actual achievement of 85. The new indicator of cumulative number of Patent Cooperation Treaty Agreements approved fell short of the target by 29%, as the actual result was 5 against a target of 7. The revised target of 5 percent, reflecting the share of enterprise sector financing of R&D in Senior Scientist & Junior Research Groups was achieved. Overall, there were 852 international publications from SSGs and JRGs in peer-reviewed journals, 38 internationally coauthored journals, and 145 articles in national research journals, and 77 partnerships established between local researchers and international researchers on diverse topics. Moreover, the ICR assessed that the average number of international peer-review publications per researcher has stood at 0.16 at the national level, while it was 0.25 for the TCP grantees (57 percent higher) for the period Rating Substantial PHREVISEDTBL Objective 2 Objective Improved commercial relevance of research performed by the groups selected through transparent and competitive process Rationale Project efficacy is illustrated by achieved outputs and outcomes under the second sub-objective a summarized below: Outputs: There is no reliable estimate of actuals for the rate of equipment use in national and engineering laboratories and use of Senior Scientific and Junior Scientific Groups' equipment by outside users (%) The target of number of researchers and scientists and entrepreneurs trained by the TCO was exceeded by 267%, reaching the number of 881 against a target of 420. The target of number of partnerships established between researchers and the enterprise sector measured in terms of number of agreements more than quadrupled, reaching 119 against a revised target of 26. Outcomes: The revised target of number of technology-based start-ups created under Senior Scientist & Junior Research Groups and by TCO clients having commercial sales was exceeded, as it reached 40 against a target of 10. The target of cumulative number of license agreements signed sold with the help of the TCO was exceeded, as it reached 11, against a revised target of 4. Progress toward improved commercial relevance of the projects was reflected by the following: (i) 40 of the 65 companies created with the project's support completed their first sales before the project closing date, totaling over KZT 900 million, (ii) the project helped in the

5 approval of 5 Patent Cooperation Treaty (PCT) agreements in 2015, (iii) the number of patents applications registered per 1,000 researchers at the Eurasian level is 11.2 (in national sciences and engineering) while TCP provides a rate of 18.3 for the same scientific fields (TCP outperforms by 64 percent) during 2014 and 2015, and (iv) in 2014, only 3 industrial prototype license agreements were made in Kazakhstan. TCP-funded projects signed 11 license agreements, thus significantly exceeding the national performance before the project, and finally (v) all grantees have reached the commercialization results using the scientific research or proof of concept and prototype development results funded by the TCP. Rating Substantial 5. Efficiency Instead of an economic and financial analysis, the PAD provided a description of recent developments in the areas of research and development (RD), and technology acquisition in Kazakhstan, and the key finding was that the technology development model inherited from the Soviet Union had remained intact, but that model was unsuited for building competitive enterprises that can thrive in the global market. The ICR undertook an economic analysis, which drew on a beneficiary survey and consultations with the members of the PMU. Building on a number of assumptions identified in the ICR related to inflation levels, time horizon of , exclusion of transfers, and comparison to the without-project counterfactual, the economic analysis identified the categories of costs and benefits as delineated below: Costs: General and administrative costs: The analysis considers (a) up-front administrative costs during the stage of design and implementation period; (b) one-off administrative costs during the selection process; and (c) recurrent compliance costs incurred by grantees. Applying relevant nominal average wages in the period , the present value of this category of administrative costs amounts to KZT 12.5 million. One-off administrative costs: This category includes the costs of application for grants and recurrent compliance costs related to Components 1 and 2. Considering the duration of the TCP grant programs, relevant nominal monthly salaries, the present value of this category of administrative costs amounts to KZT million. Direct financial outflows: Another significant group of costs are direct financial outflows in the form of research and commercialization grants and consultancy contracts. In the case of research and commercialization grants, about 40 percent of the grant funds was spent on purchase of equipment and materials from abroad and this analysis takes 40 percent of grant funds as economic costs, while the remaining part of the grant funds is considered either as transfers or even rents. Based on assumptions and project data, the present value of economic costs related to direct financial outflows amounts to KZT 2.4 billion. A major negative externality of the project is the deadweight loss resulting from distorted decisions of economic agents due to collection of taxes to finance the project. The current analysis estimates the present value of distortionary impact of taxes amounts to KZT million. Benefits: Direct benefits. The measurable benefit of the project is the commercialization of research findings and the development of innovative products and services in the period before the project closure. The sales of research groups as of year-end 2015 amounted to KZT million, implying a present value of benefits from commercialization of research activities amounts to KZT 1.16 billion, although this figure does not incorporate increased income and increased income/cost savings in the period beyond the project closure. The second direct benefit of the project was the number of licensing deals where beneficiaries act as the licensor and the number of approved PCT applications. There were 8 such licensing deals and 5 patent applications submitted in the scope of the PCT. Estimation of actual economic value of the licensing deals is approximated by the median economic value of the nearest equivalent of a licensing deal, patents in the European Union, estimated by several recent studies to be around 200,000. Applying assumptions and appropriate exchange rate, the present value of this direct benefit is KZT million. Indirect benefits. Indirect benefits of the TCP stem from the value added from capacity building trainings, brokerage events, conferences, and

6 workshops carried out by the TCO and devoted to technology commercialization training of the scientific community across the country. In particular, during , 881 individuals were issued certificates as a result of 27 workshops devoted to technology commercialization training. Typically, the returns to training are measured with respect to increase in wages and the probability of remaining in employment. For the analysis, it is assumed that an incremental benefit to 881 trainees and 328 members of research groups was a 2 percent wage premium from 2012 to year-end Considering the Statistics Agency s data on the average monthly salaries in the R&D sector during and the key assumptions, the present value of this group of benefits amounts to KZT 89.1 million. Other unquantified principal benefits of the project come from development of innovative research and production in Kazakhstan through the following vehicles: (i) Increased knowledge production, (ii) Increased research capacity, (iii)establishment of strong industry-r&d linkages, and so on. NPV and ERR: According to the ICR, the retrospective cost-benefit analysis shows positive net benefits amounting to KZT million, and an ERR of 23 percent. However, this analysis does not capture (a) the majority of economic benefits that will be realized after the project closure and (b) significant positive externalities generated by the project. Furthermore, the analysis doesn't show how the annual flows of costs and benefits, and how the ERR were calculated. The ICR did not conduct a cost effectiveness analysis, and we have no basis to make an informed assessment of the use of the loan resources. However, 17 percent of the loan resources were not used and had to be canceled. The ICR identified the following shortcomings in assessing efficiency: (i) the cancellation of needed resources was not discussed (ii) there was no table comparing annual streams of costs and benefits until the closure of the project, and (iii) the streams of costs and benefits occurring after the project closure was completely overlooked. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Appraisal 0 ICR Estimate Not Applicable Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of project objectives was high and the relevance of design was substantial. Efficacy was substantial for the two project subobjectives, because improved scientific performance was illustrated by good progress in having the SSG and JRG articles published in international publications and in national research journals, and in establishing partnerships between local researchers and international researchers on diverse topics. Commercial relevance of research improved because the companies created with the project's support completed their first sales before the project closing date, industrial prototype license agreements, and all grantees have reached the commercialization using the scientific research or proof of concept and prototype development results. Efficiency is rated modest, given the lack of sufficient evidence in justifying the expected stream of future benefits and a number of administrative deficiencies. This results in an overall outcome rating of.

7 a. Outcome Rating 7. Rationale for Risk to Development Outcome Rating Progress toward intended outcome was overall positive both in terms of improved scientific performance and commercial relevance of research. Overall, the sustainability of the project achievements will mainly depend on the way Kazakhstan s Government will continue to organize and fund scientific research in the future. The ICR identified key factors (p.22) that favor the sustainability of reported outcome as follows: (a) the follow-up operation launched in 2016 will ensure the scale up and continuity of project results; (b) the pursuit of best international practices brought in by the project such as the establishment of independent agencies for international peer review and competitive grant programs for research and strengthened TCOs in universities, (c) the Government's commitment to a strategy of fostering innovation as a driver of economic growth, (d) the goals and concepts introduced by the project have strong resilience to political changes, and finally (e) the Association of Technology Commercialization Professionals (ATCP) will continue its activities to promote the development of the technology commercialization system, mitigating the risk of loss of local expertise built with the project support. The ICR also identified the following risks: (i) the low oil prices might reduce funding for the R&D sector, (ii) future R&D funding projects might be subject to coordination and communication issues between different stakeholders of the NIS, (iii) the depth of R&D talent with commercialization potential is relatively untested and could prove to be lower than expected, (iv) there is a moderate risk that some of the companies created not survive without further grant funding, and finally (v) since the IMSC has started its operations, there is little evidence to assess the extent to which it will fulfill its mandate. a. Risk to Development Outcome Rating Modest 8. Assessment of Bank Performance a. Quality-at-Entry The ICR was weak in reporting the Bank's role in relation to the quality at entry of the project. However, the PAD indicated that the project design was informed by lessons learned, and that activities were adopted after a rigourous selection, and due consideration of other alternatives to ensure the quality of the project design. The project was informed by lessons learned across the Bank's experience (PAD, p.17), especially its work in Latin America, which illustrated that a competitive selection process, focused on excellence in research produces substantial improvements in research quality and productivity. Other lessons underscored the organization of research along multidisciplinary and problem-oriented lines and ensured that there were close links between teaching and industry. Finally, developing an effective NIS required sustained, focused Government commitment that had to extend far beyond the horizon of any single Bank Project. Furthermore, alternatives to this project were considered and rejected, and an overview of risks was conducted (PAD, p.25). Appropriate implementation arrangements were identified and included central government and decentralized entities and the PIU. Despite a weak assessment of the quality at entry section of the ICR (p.23), based on indications elsewhere in the ICR, quality at entry is rated. Quality-at-Entry Rating b. Quality of supervision The initial Bank s weak capacity on the ground justified a slow project start. Thereafter, the Bank's team engaged more closely the Government to ensure the borrower s ownership of the project objectives. The Bank team also worked with the senior officials and technical staff in line ministries to jointly prepare progress reports and agree on a road map for the project implementation until the extended closing date. For closer supervision, the Bank team instituted a weekly meetings to follow up on progress to assist the PMU and Government entities

8 involved in the project implementation. A midterm review mission was carried out in 2013, during which the Bank team provided adequate guidance to evolve toward achievement of the PDOs. However, the Bank could have been more proactive in addressing implementation issues and delays at the project launch and revise the Results Framework in a timely manner. Finally, there were shortcomings in the Implementation Status Reports, including delays in updating their content to reflect changes in indicators agreed upond during restructuring. Quality of Supervision Rating Overall Bank Performance Rating 9. Assessment of Borrower Performance a. Government Performance Project ownership at the ministerial level played a key role in the advancement of project implementation, especially that annual budget allocations to involved government entities were timely made to ensure adequate funding of borrower s contribution to the project. However, the project launch suffered from implementation delays due to weak technical capacity at the lower levels of the Government. Progressively, changes at the ministerial level contributed to improved performance of the PMU and progress toward project objectives. The borrower s performance significantly improved over the course of the project with steps taken to improve cumbersome institutional arrangements and complex interministerial coordination and approval processes, although some obstacles remained until the end of the project. But the above-mentioned weaknesses contributed to delays and led for instance to the breach of covenants in February Government Performance Rating b. Implementing Agency Performance There were organizational and institutional coordination shortcomings for prolonged periods that affected the the PMU s performance in the financial, procurement, environmental monitoring, and M&E functions. The creation of a more permanent PMU in 2011 and the strengthening of the TCO in 2013 significantly improved the institutional arrangements for the project management. However, the PMU had operational difficulties that affected project implementation despite Government 's efforts to strengthen implementing institutions. Implementing Agency Performance Rating Overall Borrower Performance Rating 10. M&E Design, Implementation, & Utilization a. M&E Design The design of the M&E framework had strengths and weaknesses. While most PDO indicators were specific and measurable, some were unrealistic and not time-bound. Some intermediate indicators were irrelevant to the project outcomes. Proposed targets were set in absolute terms, and this was not the best approach for a pilot project aiming to demonstrate comparative effectiveness in organizing R&D programs. The International Science and Commercialization Board (ISCB) and TCO were tasked to report regularly on a wide set of indicators, including a yearly survey of scientists and entrepreneurs, and annual PMU reports. However, the M&E arrangements did not provide specific indications on

9 techniques to be used in the proposed evaluation reports. b. M&E Implementation The M&E framework was significantly revised in 2013 to unify the results framework and correct the original M&E design, strengthen the PDO indicators, and take advantage of an extended closing date. While the revised Results Framework was significantly improved, some of the indicators of commercial relevance and demonstration effect were still lacking, other indicators suffered from vague or inconsistent definitions and weak relevance to the PDO. Due to limited PMU capacity, M&E data were not collected and reported in a timely manner. The M&E framework was strengthened over time with increased data collection from the grantees and hiring of M&E staff to monitor the quality control function and the accuracy of the collected information. At the time of the ICR, there were still ongoing improvement in the methodology for reporting actual results against the PDO and intermediary indicators. The Bank s ISRs contained gaps in reporting as they did not report end-of-the-project results in the final ISR. Finally an evaluation study in 2015, and a beneficiary survey in 2016 assessed results achieved and analyzed the feedback from the beneficiary respectively. c. M&E Utilization There were examples where the data collected proved to be useful in making strategic decisions to enhance project implementation including (a) redirecting resources toward commercialization objectives in the second half of the project; (b) informing decisions regarding the design of the follow-on operation; and (c) communicating internally and externally with stakeholders. Overall the M&E targets underpinned the project performance and contributed to the realization of PDO goals. M&E Quality Rating Modest 11. Other Issues a. Safeguards At appraisal, an Environmental Assessment identified the project as a category B. Key safeguard policies triggered were (a) a laboratory safety and the safe disposal of wastes and (b) the potential environmental issues related to the rehabilitation of existing laboratory space. An Environmental Management Plan (EMP) was developed and compliance with the EMP was satisfactory throughout the project. Regular reports on the project environmental performance did not indicate any high-risk environment issue. b. Fiduciary Compliance Financial management: The financial management arrangements for the project were overall adequate throughout the project life. However there were several financial management (FM) issues in the early stages of the project, resulting in less than satisfactory FM ratings in 2011 and In the beginning of the project, there were delays in the provision of counterpart funds due to lengthy processes in the line ministries overseeing the project. PMU FM issues were fully addressed by 2014, and the project remained in full compliance with the financial conditions, as established by the audited financial statements. The disbursements of the loan funds started in December 2011, then picked up and by midterm review, the project was fully transferred to e-disbursement that allowed stepped up disbursement. Unaudited interim financial reports and audited financial statements were overall satisfactory. Procurement: There were initial procurement issues, which were addressed through the hiring of a procurement specialist. Procurement performance improved; but procurement capacity remained low, and procurement performance evolved in seesaw, requiring further training c. Unintended impacts (Positive or Negative)

10 None d. Other None 12. Ratings Ratings ICR IEG Outcome --- Risk to Development Outcome Modest Modest --- Bank Performance --- Borrower Performance --- Quality of ICR Substantial --- Reason for Disagreements/Comment Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons IEG agrees with the ICR's findings related to importance of institutional arrangements, innovative M&E framework, and of grant ownership structure and the level and flexibility of grant funding in order to create an environment of science innovation and commercial relevance of research. IEG also endorses the three key lessons identified in the ICR and rephrased below: (i) Innovation projects are most effective when implemented as part of a broader country long-term strategy: To address the divorce between university and industry, this project helped the Kazakh Government and the Bank to work together on a long-term strategy for innovation. The Bank's support was at the core of the national innovation strategy, and helped to strengthen the links between the university research and the market, thus leveraging the use of public resources to maximize the development impact. (ii) International expert advice is essential for the implementation of a successful innovation project: The project funded foreign professional coaching to the country's research groups, and this contribution was critical in promoting a culture of innovation and in connecting academic researchers to markets. The day-to-day coaching advice from commercialization managers produced tangible results in strengthening the effectiveness of the country's technology commercialization. Furthermore, the knowledge transfer to the local staff helped ensure that the commercialization capacity remains in the country, thus enhancing project sustainability. (iii) Effective innovation programs for technology commercialization need to be integrated with the enterprise sector from the outset, and to combine both the supply and demand approaches: To improve the impact of this kind of project, industry partners need to have a more proactive role from the very early stages of the project. Market goals targeting the technological global value chains, should be clearly defined, and a strong focus on demand-driven innovation policies needs to be adopted when appropriate. 14. Assessment Recommended? No 15. Comments on Quality of ICR

11 The ICR provides a fair assessment of the context of the project appraisal, its design and implementation, and the results achieved at project closure. All ratings reflect the storyline developed in the ICR, and the lessons and findings formulated are derived from on the ground experience of the project's implementation. Two areas of the ICR that could have been improved were: (i) the assessment of the Bank's quality at entry lacked depth, and (ii) the efficiency assessment could have been improved by a thorough coverage of the projections of costs and benefits for the entire period of the life project, and an analysis of cost-effectiveness in the use of project ressources. a. Quality of ICR Rating Substantial