Global Fund evaluation: managing and measuring for impact. Daniel Low-Beer IOM meeting, January 2014

Transcription

1 Global Fund evaluation: managing and measuring for impact Daniel Low-Beer IOM meeting, January 2014

2 Contents 1. Evaluation Approach Measuring Impact Managing for Impact 2. Country platform investing for impact HIV, TB, Malaria, Health evaluations Prioritizing investments 3. Strengths and Weaknesses 2

3 Why impact is important for the Global Fund? Purpose of the Global Fund: a sustainable and significant contribution to the reduction of infections, illness and death Global Fund Strategy Investing for Impact Targets in terms of impact: MDGs, lives saved, infections averted Key part of programming: invest more strategically for impact Management focus of grants Impact upfront: planning for impact to promote prioritisation PBF for impact: identifying and investing in value in the portfolio 3

4 Focus on impact definitions Combines two components, assessing final disease outcomes and impact, and assessing contribution and causation along the results chain Final disease outcomes and impact Has there been a change in disease mortality/morbidity and/or incidence and prevalence, positive or negative? Has there been a change in outcomes and behaviors, positive or negative? Causation Have interventions and other competing explanations or hypotheses contributed to and resulted in these impacts, positive and negative? Investment in rigorous analysis, disaggregation of data by time, person and place and including comparison groups where feasible Avoid narrow attribution of causation to a single program or intervention. Drawing on OECD and World Bank definitions Impact evaluation assesses the overall impact on the burden of cases and deaths due to the three diseases. It will assess causation and the contribution of the Global Fund and other explanations along the results chain from inputs to outcomes 4

5 Approach to Impact Reviews independent but practical and linked to grant and policy process Partner approach (build-on, collaborate and align evaluations with partners, maintaining rigor and objectivity); Periodic (planned, regular, independent reviews rather than one-off evaluations); Plausibility design (provide evidence of program impact, positive and negative, also considering other nonprogram influences along the M&E chain); country Platform (build on national systems including program reviews and strengthening of country M&E capacity) Practical for grants (recommendations for grant management, grant renewal and re-programming). 5

6 Progress Five Year Evaluation to rolling Five Years of Evaluation 1. TERG approved work plan and strategic investments by Board (Dec 2012) 2. All key components underway by end 2013 Country Evaluations (21 program and epi reviews underway, part of the new funding model) Thematic reviews (4 completed, 2 underway, TORs reviewed and launched): Reviews of specific issues or themes which assess past evidence and selected cases, with a primary focus on providing forward looking strategic options Data Quality Assessments (17 and investments agreed with partners and GM in 10 countries (focused investments but continued risks and gaps) 3. Evaluation used for measurement and management New Funding Model: includes epi-stage and program reviews, Myanmar, Zimbabwe, refocused national strategies e.g. Cambodia, Thailand Regional artemisinin resistance grant with specific regional coordination M&E strategic investment fund with key data system areas Assurance Statements: AMFm decision. Fragile states emergency initiative proposed 6

7 2. Country platform investing for impact 21 program reviews (and epi analysis) completed 7 HIV, 6 TB, and 8 malaria Health review incorporating three diseases in preparation for Tanzania Partner process National programs, WHO, UNAIDS, bi-laterals, relevant NGOs, etc. Input to national strategies and grant dialogues Key input for: Epi stage of the New Funding Model and Mid-Term Review of the Global Fund Strategy 8th SIIC Meeting Geneva, July

8 Thailand measuring and managing for impact Epidemiology limited impact Responses refocus investments IDU MSM 20 Male Sex Worker 10 FSW (brothel-based) /11 HIV prevalence consistently high among injection drug users, increased among MSM and male sex workers. Increase coverage by focusing on 27 provinces with 70% new HIV infections; Use network approach to deliver the packaged service with robust monitoring 8

9 Outcome of the impact review agreed actions Recommendation Area 1. Ensure high coverage and geographical focus of activities Progress to Date Actions (immediate and medium term with timing) Follow up responsibility 2. Renew and define the prevention package for key affected populations from information to services 3. Optimise network approach and delivery model for HIV prevention matching services with those at risk 4. Innovative monitoring approaches to support service delivery in HIV prevention services from numbers to end points 5. Sustainability of the response among key groups, delivery models and policies Agreement on actions and missing items to include with Country & Country Team Additional actions for impact Global Fund, National Program, PRs 9

10 Cambodia program reviews: identifying common components of value in programs Malaria: impact 80% decline malaria deaths yet artemisinin resistance is serious risk Supported the grant review, despite financial issues TB: impact Two rounds of surveys, together with surveillance data, 45% decline TB prevalence Supported national strategy development, grant review HIV: recent stagnation of prevention and data limitation Successful high coverage of ART scale up Enhanced epi stage in the program review input to the grant renewal Program review and impact used for country, partner and grants: common value of community workers 8th SIIC Meeting Geneva, July

11 Strengths and weaknesses of the approach Measurement and management for impact Causative framework: time, person, place a powerful framework Challenge of data quality and analysis Needs strong political backing Externally with partners in country Internally to promote focus on impact Improved with investments in country evaluation agenda: trials, studies and operational research Position Impact at the start of programming: prioritise with the end in mind 11

12 Conclusion From Five Year Evaluation to continuing five years of evaluation Key components: country impact reviews, thematic reviews, data quality assessments, periodic syntheses Establish external value (e-valuation) of program - Positive and No Impact: critical step to prioritise investments, identify value in the portfolio and synergies (prevention, MNCH, community workers) Management and Measurement benefits Balance Independence and support with relevance, front line evaluation 12