POST EVALUATION OF THE PETROCARIBE SANITATION PHASE II

JAMAICA SOCIAL INVESTMENT FUND Investing For Community Development TERMS OF REFERENCE POST EVALUATION OF THE PETROCARIBE SANITATION PHASE II 1. GENERAL INTRODUCTION The Jamaica Social Investment Fund (JSIF) was established in December 1996 as a component of the Government of Jamaica s strategy to reduce and eradicate poverty. Recognizing the limited capacity of existing government institutions to implement small-scale projects at the community level, Cabinet in December 1995 approved the establishment of the JSIF as a key component of the Government s National Poverty Eradication Programme (NPEP). JSIF invests in community-based projects as a means for empowering communities and building social capital. By involving communities fully in identifying, prioritizing, planning, managing and monitoring their own development projects, JSIF helps to build local capacity to sustain and extend development initiatives. The JSIF is an autonomous government company designed to provide investments in community-based projects island wide and is a demand-driven financial intermediary. It works in partnership with communities, the private sector, non-governmental organizations (NGOs), and donor agencies in seeking to channel benefits to the poorest communities across the country. The JSIF promotes, appraises, finances and supervises sub-projects in the following areas: Social Infrastructure - includes the rehabilitation, expansion, construction & equipping of facilities such as schools, health centers, community centers, homes for the elderly and persons with disabilities, infirmaries; it also includes the construction and rehabilitation of public sanitary conveniences, drains, canals and community-based water systems. Economic Infrastructure - includes the rehabilitation and up grading of parochial, feeder and urban access roads and the construction and rehabilitation of community-based agro-processing facilities. Activities to improve market access for micro and small-scale rural agricultural producers and tourism product and service providers. Social Services - includes assistance to programmes offering services in career guidance and job placement, counseling (including conflict resolution & drug abuse), parenting and family life education, and skills training to the un/under employed and persons with disabilities. Organizational Strengthening - includes technical assistance and training for community-based organizations to assist them in developing and managing community-based projects and organizations, to governmental and non-governmental institutions in participatory project cycle management and improving the support they can provide to communities in managing community development initiatives. 1

2. BACKGROUND TO PETRO CARIBE DEVELOPMENT FUND PORTOFOLIO A Memorandum of Understanding (MOU) was signed on August 2, 2011, between the PetroCaribe Development Fund (PDF) and the Jamaica Social Investment Fund (JSIF) for a grant of $200 million (JMD). As per the MOU JSIF used these funds to implement projects that were to; (a) Foster community development and employment (b) Encourage the development of human capital (d) Promote rural development (e) Promote entrepreneurship and capacity building for micro and small businesses and Institutions that serve marginalized populations. These funds were used to support the implementation of the Community Renewal Programme (CRP) designed by the Planning Institute of Jamaica. In addition to the first grant of $200 million (JMD), an additional $200 million (JMD) was provided to the JSIF in July 2012 to provide modern sanitation units to twenty-six (26) rural schools. On May 1, 2012 a supplemental MOU was signed which authorized the JSIF to execute small grants and special Projects on behalf of the PDF; this small grant program is currently being administered. Currently the PDF project components consist of: a. Sanitation Provision of bathroom facilities to both individuals and schools that do not currently have any infrastructure or the existing unit is severely dilapidated. b. Supportive Services This assistance specifically engages the beneficiaries of the sanitation units in five communities to train on group dynamics considering the density of the communities and proper sanitation and hygiene. c. Small Grants/Special Projects These are grant funds provided by the PDF to directly assist grass roots organizations in building organizational capacity and where applicable providing direct assistance to participants. 3. BACKGROUNGD TO ASSIGMMENT The communal sanitation project, PetroCaribe Sanitation Phase II, aims to provide sanitation units to specifically selected locations which did not have sanitation units or have units that are hyper-substandard. One-hundred and fifty-six (156) sanitation units were constructed to alleviate the grave sanitary conditions. The types of units constructed were single, double and multifamily units. 2

In addition to infrastructure maintenance, hygiene and group dynamics training was provided to the seven (7) communities: 1. Central Kingston (Rae Town and Allman Town), 2. Midtown (Tivoli), 3. Denham Town, 4. Fletchers Land, 5. Hannah Town, 6. Jones Town and 7. Rose Town. The project targeted two-thousand one-hundred and ninety-one (2,191) direct beneficiaries across the 7 communities. For the communities of Rose Town and Jones Town the units were constructed directly by Food for the Poor (FFP). The remaining five (5) communities were implemented using the Community Based Contracting Methodology (CBC). CBC methodology empowers communities by allowing them to perform the following activities: (1) Be fully responsible for carrying out project activities (2) Receive funds from JSIF to hire a contractor, a supervisor or purchase material (3) Be accountable for the proper execution of the contracts and the purchases (4) Have control of the resources from the community and from JSIF (5) Form Project Management Committee of 9 members to ensure that the previously listed activities are carried out. In the case of the communal sanitation the project was executed through the Rotary Club of Downtown Kingston (RCDK) (sponsor) and the project management committees (PMC) of the benevolent societies for each community. In some cases beneficiaries were active participants in the construction of the units. Along with the implementation of the sanitation units there was also a social marketing campaign promoting proper hygiene and addressing group dynamics which would lead to the sustainability of the units. This was executed by the People s Action for Community Transformation (PACT). Not including the JSIF, the following are the key stakeholders for the communal sanitation project: a. Food for the Poor (FFP), b. National Water Commission (NWC), c. Rotary Club of Downtown Kingston (RCDK), d. Project Management Committees (PMCs), e. People s Action Community Transformation (PACT), f. General Construction (construction company), g. National Works Agency (NWA), and h. Pastor Neville Wright (Jones Town and Rose Town communities). 3

These entities or persons will be critical during the data collection periods and in answering fundamental queries in regards to the execution of the project. The consultant team should discuss with the JSIF any adjustments to this list. 4. OBJECTIVE OF ASSIGNMENT Sanitation has been identified as one of the most challenging areas for many people in developing communities. The main objective of this consultancy is to conduct an evaluation of the PetroCaribe Sanitation Phase II sanitation project, identify best practices, constraints and lessons learned and recommend improvements to increase effectiveness and results of the project. Specifically, the consultant must: (a) Assess the efficacy of the implementation of the PetroCaribe Sanitation project against the results framework to include: An assessment of the outcomes and outputs to date Investigation of the relevance of the objectives, outcomes and outputs in relation to emerging national development objectives, priorities and goals Programme effectiveness relative to objective and targets (b) Examine the efficiency of the project to include but not limited to the following: Evaluation of the adequacy of the targeting, application, selection and implementation processes; Assessment of the adequacy of monitoring and evaluation framework and systems used to measure results; and Documentation of best practices, bottlenecks/weaknesses and opportunities for improving of the project cycle. (c) Conduct a comparative analysis of the units that were constructed by the community members, with JSIF oversight, using the Community Based Contracting (CBC) methodology and the units that were built by the Food For The Poor The analysis is to include but not limited to an analysis of cost, quality of units, implementation processes, community and other stakeholder participation and beneficiary satisfaction. (d) Undertake cost benefit analysis of infrastructure subproject outputs (e) Determine stakeholder satisfaction with the project to include but not limited to: Relevance of the project in its current design vis-à-vis initial community needs and current situation. Relevance of the project as mechanism for reducing poverty 4

(f) Assess the effectiveness of the WASH training in: Motivating the beneficiaries to adopt and practice positive sanitation and hygiene practices Changing perceptions general as it relates to proper sanitation and hygiene Encouraging good hygiene practices as it relates to sanitation Motivating the beneficiaries to manage and maintain the sanitation units (g) Determine the prospects for long-term sustainability of sub-project outputs and outcomes. (h) Determine lessons which may be learnt, recommend and strategically prioritize any adjustments necessary to assist PetroCaribe in undertaking activities of a similar nature in the future. This assessment should examine project performance from application to implementation, monitoring and evaluation. 5. SCOPE OF THE WORK 5.1 Review the relevant project documents (Project Appraisal Document, Operations Manual Supplement, Monitoring and Evaluation Framework, Capacity Building Reports, Midterm and Pre-end/End of Project Reports etc). A review of relevant sector specific policies, programmes, projects and strategies is necessary. 5.2 Undertake review and document findings of the physical state of the sanitation units in the specified communities. 5.3 Consult and interview stakeholder entities that were involved with key project activities to include but are not limited to: a) Food for the Poor (FFP), b) National Water Commission (NWC), c) Rotary Club of Downtown Kingston (RCDK), d) Project Management Committees (PMCs), e) People s Action Community Transformation (PACT), f) General Construction g) National Works Agency (NWA), and h) Pastor Neville Wright (Jones Town and Rose Town communities). 5.4 Develop relevant instruments which are quantifiable to assess the structural integrity of the community infrastructure. 5.5 Conduct Beneficiary Satisfaction Survey with beneficiaries to include but not limited to: a) Design of unit b) Accessibility of units general and vulnerable sub groups: e.g., disabled, children, elderly, etc. c) Quality of units 5

d) Distribution of units e) Arrangement for management and maintenance f) Targeting g) Implementation processes 5.6 Assess the scope of the overall project impact from the subprojects implemented in areas including but not limited to: a) Relevance of project interventions to community needs. b) Extent of use and efficacy of the community driven development model where JSIF oversaw the implementation of the units(community based contracting) vs the Food For the Poor implementation methodologies. c) Sustainability of project interventions, that is, community capacity for on-going management, maintenance and long-term sustainability of the project outputs. Document if there are any factors (social and otherwise) which will affect beneficiaries ability to benefit from the project. d) Threats to project sustainability internal and external to the community. 5.7 Assess the effectiveness of the WASH training in: a) Motivating the beneficiaries to adopt and practice positive sanitation and hygiene practices b) Changing perceptions general as it relates to proper sanitation and hygiene c) Encouraging good hygiene practices as it relates to sanitation d) Motivating the beneficiaries to manage and maintain the sanitation units 5.8 Assess the impact of the overall project on social capital formation and improvement by way of leadership effectiveness in fostering community cohesion, inclusiveness and participation in all phases of project. 5.9 Consult and interview representatives from entities that were involved with key project activities and these include but are not limited to: a) Food for the Poor (FFP), b) National Water Commission (NWC), c) Rotary Club of Downtown Kingston (RCDK), d) Project Management Committees (PMCs), e) People s Action Community Transformation (PACT), f) General Construction (construction company), g) National Works Agency (NWA), and h) Pastor Neville Wright (Jones Town and Rose Town communities). From these meetings, actual outcomes of the PDF project from the stakeholder entities point of view is to be established, as well as the perceived sustainability of relevant activities implemented. 5.10 Conduct primary research through participatory appraisals, focus group and elite interviews augmented by secondary data sources. 6

5.11 Document the findings clearly stating what has been achieved, best practices implemented, any gaps that exist and recommendations to address any issues that may have arisen in PDF communities. 6. DELIVERABLES The consultancy will require the triangulation of qualitative and quantitative methodologies through primary research to be undertaken within PDF subprojects in order to provide the deliverables below: 6.1. Following an inception meeting with the JSIF team (as per Section 11: Reporting Arrangements), an Inception Report, a maximum of 20 pages (main text, excluding annexes) within 10 calendar days after the contract is signed. It must include a comprehensive research methodology, research questions, data collection templates/instruments and a detailed plan of action with implementation timelines. 6.2. A PowerPoint document and presentation of initial findings, triangulated analysis, recommendations and participant profile (maximum 40 slides), within 75 days after the contract is signed. Within 2 working days of this presentation, the JSIF team and the consultant will hold a meeting to discuss the form of the interim and final reports. The PowerPoint must include, but shall not be limited to: (a) sources consulted and brief description of methodology; (b) participant profile; (c) a synthesized analysis of findings from all communities; (d) disaggregated analysis of findings from each community. 6.3. An interim report (maximum 150 pages of main text, excluding annexes, in Microsoft Word) of findings, triangulated analysis, recommendations and participant profile, within 85 calendar days after the contract is signed. The report must include, but shall not be limited to: (a) sources consulted and brief description of methodology; (b) participant profile; (c) a synthesized analysis of findings from all communities; (d) disaggregated analysis of findings from each community; (e) TOR for the assignment as an annex to the report. 6.4. A PowerPoint document and presentation of findings, triangulated analysis, recommendations and participant profile (maximum 40 slides), within 100 days after the contract is signed. The PowerPoint must include, but shall not be limited to: (a) sources consulted and brief description of methodology; (b) participant profile; (c) a synthesized analysis of findings from all communities; (d) disaggregated analysis of findings from each community. 6.5. A final report (maximum 150 pages of main text, excluding annexes, in Microsoft Word) of findings, triangulated analysis, recommendations and participant profile, within 110 7

calendar days after the contract is signed. The report must include, but shall not be limited to: (a) sources consulted and brief description of methodology; (b) participant profile; (c) a synthesized analysis of findings from all communities; (d) disaggregated analysis of findings from each community; (e) TOR for the assignment as an annex to the report. JSIF shall subsequently have eight (8) working days within which to forward comments or indication of report acceptance to the Consultant. In the case of comments, the final approved deliverable items should be submitted no later than 132 days after the contract is signed. Work Presentation for deliverables 6.1, 6.3, and 6.5 above: a. Three hard copies and one electronic copy of report (on CD or jump drive). b. One copy of audio recordings 1 and transcripts of primary research data collected are to be submitted with relevant report (on CD or jump drive). The transcripts and audio recordings must include the location, date, name of the facilitator and participants/groups. Pseudonyms should be used for focus group participants, to protect their anonymity. Primary data sets created using quantitative SPSS software and qualitative software (NVivo) are also to be submitted. Work Presentation for deliverables 6.2 and 6.4 above: a. One electronic copy of PowerPoint presentation (on CD or jump drive). 7. DURATION OF CONTRACT 140 calendar days (20 weeks). This duration includes 4 weeks of JSIF review time and 1 week for payment processing following the submission of the final report. 8. PROCUREMENT SELECTION METHOD Consultant Qualification Selection. 10. PAYMENT SCHEDULE 15% of contract sum upon submission and approval of Deliverable 6.1 and appropriate invoice. The Inception Report should be a maximum of 20 pages (main text, excluding annexes) and is to be produced within 10 calendar days from the commencement of the contract. In the report, the consultancy firm shall provide a detailed work plan with implementation timelines, a comprehensive outline of quantitative and qualitative research methodologies including group 1 If full anonymity is requested by the respondents and/or is deemed to be necessary by the consultancy firm, the audio will not be required. A justification will however be required. 8

makeup and how the groups will be formed, the research design and dates for the submission of the requisite reports, research questions, and software for analyses (SPSS, NVivo etc.), as well as an outline for the Final Report. The JSIF shall subsequently have five (5) working days within which to forward comments to the Consultancy Firm. 10% of contract sum upon submission and approval of Deliverable 6.2 and appropriate invoice. The PowerPoint document should consist of a maximum of 40 slides. This PowerPoint document shall be submitted and presented no later than 75 calendar days from the commencement of the contract. Within 2 working days of this presentation, the JSIF team and the consultant will hold a meeting to discuss and confirm the format of the interim and final reports. The Consultant should expect the majority of the JSIF s comments on the PowerPoint document and presentation to be given at the presentation and the subsequent meeting, with additional written comments to be submitted to the Consultant by the JSIF within five (5) working days. 25% of contract sum upon submission and approval of Deliverable 6.3 and appropriate invoice. The interim report should consist of a maximum of 150 pages (main text, excluding annexes) of findings, in Microsoft Word. This report shall be submitted no later than 85 calendar days from the commencement of the contract. The JSIF shall subsequently have ten (10) working days within which to forward comments to the Consultancy Firm. 15% of contract sum upon submission and approval of Deliverable 6.4 and appropriate invoice. The PowerPoint document should consist of a maximum of 40 slides. This PowerPoint document shall be submitted no later than 100 calendar days from the commencement of the contract. The Consultant should expect the majority of the JSIF s comments on the submission and presentation to be given at the presentation, with additional written comments to be submitted to the Consultant by the JSIF within five (5) working days. 35% of contract sum upon submission, acceptance and approval of Deliverable 6.5 and appropriate invoice. The final report should consist of a maximum of 150 pages (main text, excluding annexes) of findings, in Microsoft Word. This report shall be submitted no later than 110 calendar days from the commencement of the contract. The JSIF shall subsequently have eight (8) working days within which to forward comments and/or indication of report acceptance to the Consultancy Firm. All final approved deliverable items must be submitted no later than 132 days after the contract is signed, to allow for the processing of payment within the contracted period. Work Presentation for deliverables 6.1, 6.3, and 6.5: a. Three hard copies and one electronic copy of report (on CD or jump drive). b. One copy of audio recordings 2 and transcripts of primary research data collected are to be submitted with relevant report (on CD or jump drive). The transcripts and audio recordings must include the location, date, name of the facilitator and participants/groups. 2 If full anonymity is requested by the respondents and/or is deemed to be necessary by the consultancy firm, the audio will not be required. A justification will however be required. 9

Pseudonyms should be used for focus group participants, to protect their anonymity. Primary data sets created using quantitative SPSS software and qualitative software (NVivo) are also to be submitted. Work Presentation for deliverables 6.2 and 6.4: a. One electronic copy of PowerPoint presentation (on CD or jump drive). 11. REPORTING ARRANGEMENT Prior to the start of this assignment, there will be an initial briefing with the PDF Project Manager, the Social Development Manager, and the M&E Coordinator. The consultant shall report all technical and contractual matters to the PDF Project Manager. The consultant is reminded that he/she should request problem-solving meetings with the PDF Project Manager as soon as there is any indication of a variation in the scope of work, changes to the timeline or additional costs being necessary. No variations are to be made to the agreed time or cost without the prior written approval of the JSIF. 12. GUIDING PRINCIPLES AND VALUES The evaluation team will be required to adhere to, and be guided by, professional and ethical standards while contracted to the JSIF. 13. QUALIFICATIONS The successful firm should possess the following: General Experience A minimum of eight (8) years demonstrated competence in conducting similar consultancies in this field by submitting a list of clients with a brief description of services provided (dates of consultancies must be included) Information should include data on the following: o Inter-agency partnerships at the community level o Use of triangulated qualitative and quantitative research methodologies o Report preparation for Government Agencies or International organizations. o Experience working in peri-urban and urban area, preferably inner-city communities would be an asset 10

Specific Experience of Firm [as per EOI] At least five (5) similar consultancy assignments previously undertaken (briefly state the project objectives and outline the consultancy services provided (dates of consultancies must be included) Information should include data on the following: o Programme/Project development, monitoring and evaluation o PLA and other participatory research methodologies o Partnership with government, international agencies and non-government sectors in peri-urban and urban area, preferably inner-city communities The Consultant team should possess at the minimum the following qualifications a) The Lead Researcher/Consultant should possess: A Masters in related Social Science Discipline such as Community Development, Development Studies, Social Policy, Psychology, Behaviour Change Modification and Sociology or Social Work Knowledge and at least five (5) years experience with the following: o PLA and other participatory research methodologies o Statistical Package for Social Sciences (SPSS) o NVivo. o Microsoft Word, Excel, and Access. At least eight (8) years experience working on similar assignments requiring: o Inter-agency partnerships at the community level o Use of triangulated qualitative and quantitative research methodologies o Report preparation for Government Agencies or International organizations. o Statistical Package for Social Sciences (SPSS) and the qualitative software NVivo. o Proficiency in Microsoft Word, Excel, and Access. At least five (5) years experience working on similar assignments requiring: o Programme/Project development, monitoring and evaluation o PLA and other participatory research methodologies o Partnership with government, international agencies and non-government sectors in periurban and urban area, preferably inner-city communities. b) Team members/consultant should possess: A minimum of a First Degree in related Social Science Discipline such as Community Development, Development Studies, Social Policy, Psychology, Behaviour Change Modification and Sociology 11

Knowledge and at least three (3) years experience with the following: o PLA and other participatory research methodologies o Statistical Package for Social Sciences (SPSS) and the qualitative software NVivo. o Microsoft Word, Excel, and Access. At least eight (5) years experience working on similar assignments requiring: o Inter-agency partnerships at the community level o Use of triangulated qualitative and quantitative research methodologies o Report preparation for Government Agencies or International organizations. o Statistical Package for Social Sciences (SPSS) and the qualitative software NVivo. o Proficiency in Microsoft Word, Excel, and Access. At least five (3) years experience working on similar assignments requiring: o Programme/Project development, monitoring and evaluation o PLA and other participatory research methodologies o Partnership with government, international agencies and non-government sectors in periurban and urban area, preferably inner-city communities. c) The infrastructure consultant should possess at the minimum the following qualifications: o Bachelor s Degree in Civil Engineering o At least five (5) years experience designing and implementing major public sector integrated development projects (water, sewage / sanitation, roads, public buildings) Show at least three (3) years experience working on infrastructure projects that incorporate international and/or national environmental standards in the design o Knowledge of Jamaica s environmental guidelines as it relates to construction of building, road works, sewerage works and implementation of water lines in residential areas would be an asset. o Knowledge of Jamaica s and the International (IBC) building code requirements and familiarity with the National Accessibility Standards for Persons with Disabilities would be asset. o Experience working in the inner city would be an asset 12

ANNEX: 1 Summary (Logical framework) Goal: Sustainable, effective & responsive social & economic infrastructure projects Narrative Indicator Means of verification Assumptions Purpose: Provision of Sanitation units that are in line with international goals and objectives OUTCOME: To improve the quality of sanitation practices in the communities of Tivoli gardens, Hanna Town, Denham Town, Fletchers Land Caribbean Millennium Development Goal Indicator 61 Proportion of urban population with sustainable access to improved sanitation By June 2012 95% of residents who received the new toilets use them in the correct way by final completion. Final Completion Certificate Post Project Evaluation Report Photos Final Completion Certificate Post Project Evaluation Report Consultant Report Project Officer Report Photos No natural or manmade disaster Staff and stakeholders are willing to participate in maintenance training. No natural or manmade disaster Staff and stakeholders are willing to participate in maintenance training. OUTPUT 1: 75 new toilet facilities constructed to Ministry of Health and fire department specifications and allocated as follows: January 2012: (75) Sanitation Units designed and constructed to include the following: Final Completion Certificate Post Project Evaluation Report Consultant Report Project Officer Report Photos No natural or manmade disaster to delay completion of works on project -Tivoli Gardens - 15 units -Central Downtown - 15 units -Hanna Town - 15 units -Denham Town - 15 units -Fletchers Land - 15 units OUTPUT 2: Maintenance and Hygiene Training - (1) Toilet and one (1) shower stall for single units - (2) Toilets and two (2) shower stalls for double units - 1 face basin/ unit - Laundry basin / unit - Painting of Structures - Installation of electrical fixtures and fittings - Connection of households to existing sewer system - At least 1 representative from each household to maintain the new facilities - At least 3 members of Sponsor Body participating in training - General Community Awareness Campaign Carried out Project Officer Report Maintenance Training Register, stakeholders are willing to participate in maintenance training 13