UNIVERSITY OF NAIROBI MEPI Network M&E Webinar December 10, 2012
PRIME-K M&E Team Prof. James N. Kiarie, PI Dr Dalton Wamalwa, PI-Linked MNCH Francis Njiri, UoN Mara Child, UW Gabrielle O Malley, UW UW PIs Carey Farquhar (Programmatic) Grace John-Stewart (Linked MNCH) UMB PI Robert Redfield
Outline of presentation Introduction to M&E concepts M&E approach for PRIME-K Lessons learned Discussion session
Introduction to M&E Concepts When project monitors come
Program Monitoring is an ongoing, continuous process; requires the collection of data at multiple points throughout the program cycle, including at the beginning to provide a baseline; can be used to determine if activities need adjustment during the intervention to improve desired outcomes.
Program Evaluation Data collection at the start of a program (to provide a baseline) and again at the end, rather than at repeated intervals during program implementation A control or comparison group in order to measure whether the changes in outcomes can be attributed to the program
Indicators Clues, signs or markers that measure one aspect of a program and show how close a program is to its desired path and outcomes. Used to provide benchmarks for demonstrating the achievements of a program. One of the most critical steps in designing an M&E system is selecting appropriate indicators.
Logic Models A logic model, sometimes called an M&E framework, provides a streamlined linear interpretation of a project's planned use of resources and its desired ends.
The PRIME-K M&E approach
Background Mission: To improve health of Kenyans through research and training Objectives 1. Increase the quantity and quality of health professionals trained by faculty training, e-learning, use of Skills Lab and enhancement of libraries 2. Improve retention of health care workers through decentralized training 3. Invest in regionally relevant research by: Providing opportunities to pursue clinical or health systems research Research capacity building
Partnership In Innovative Medical Education For Kenya (PRIME-K) School of Medicine School of Dental Sciences UoN College of Health Sciences School of Nursing Sciences School of Pharmacy School of Public Health Institute of Tropical and Infectious Disease (UNITID) 11
Decentralized Training Sites Year 1 sites: 1. Mbagathi District Hospital 2. Naivasha District Hospital 3. Garissa Provincial General Hospital 4. Coast Provincial General Hospital Year 2 sites: 5. Pumwani Maternity Hospital 6. Kisii District Hospital 7. Busia District Hospital 8. Karatina District Hospital 9. Kitui District Hospital Map of Kenya with the decentralized training sites 7 8 6 2 5 1 3 9 4 12
Developing M&E plan for PRIME-K Initial M&E Plan based on Logic Model for RFA Currently Program Theory of Change approach linking activities to program impact Based on MEPI themes; 3 key implementation strategies per theme 1-2 Evaluation questions derived from each implementation strategy Specific process and impact indicators for each evaluation question
PRIME-K Program Theory of Change Activities Short-term Outcomes Intermediary Outcomes Long-term Program Impact New Training Initiatives Faculty teacher training Multi-disciplinary Skills Lab Medical student training at decentralized sites Development and training of adjunct faculty Mentored seed projects and career development awards Training to improve research capacity MNCH IS Research, Leadership and Management Training Training in research and grants management Support research dissemination activities Staff use innovative teaching methods Integration of Skills Lab into curriculum Increased number of medical education experts at UoN Exposure of undergraduate students to underserved populations Accredited UoN adjunct faculty at decentralized sties Locally relevant research projects Increased motivation to adjunct faculty to conduct research Strengthened research administration and regulation capacity of the UoN Center of Excellence in Medical Education and MNCH at UoN Decentralized training sites hosting students from UoN s CHS Accelerated research activity by UoN faculty and students Ministry of Health staff engaging in teaching and research More and better qualified health care workers Health care workers retained in areas where they are most needed Increased regionally relevant research
Implementation Strategies Theme 1: Increasing quantity, quality and capacity of health care workers Introduce innovative teaching mechanisms Increase learning opportunities for students Establish a center for medical education Theme 2: Retaining graduates and faculty in areas where they are needed most Improve capacity of decentralized training sites to train students Train UoN students at decentralized training sites Theme 3: Increasing the capacity of MEPI institutions to conduct regionally relevant research Implement and promote use of locally relevant research Strengthening research administration and regulation capacity of the UoN
Evaluation Questions (Theme 1) Increasing quantity, quality and capacity of health care workers: Does faculty training improve the quality of medical graduates? Does introduction of new learning opportunities lead to increase in student numbers and improve the quality of training at the College of Health Sciences in the University of Nairobi? Does the UoN have a functional Center for Medical Education?
Evaluation Questions (Theme 2) Retaining graduates and faculty in areas where they are needed most How well has decentralized training capacity been developed? Does the decentralized elective term prepare students to work in rural or underserved areas? Do decentralized training experiences and research opportunities retain medical graduates and faculty members in underserved and rural areas in Kenya?
Evaluation Questions (Theme 3) Increasing the capacity of MEPI institutions to conduct regionally relevant research Have MEPI activities enhanced research activity, dissemination and utilization of research findings? Is there improved research administration at the institution? Has IS and applied research capacity increased? Do multi-disciplinary teams of faculty and postgraduates conduct locally relevant MNCH research?
Indicators Process Trainings Qty, Diversity, Coverage, % # of Trainees (students, faculty) Training sites No. of New sites # of learning activities Curriculum # new, # revised Skills labs Capacity and utilization Research support No. and type/scope of studies Progress Impact Improvement on HCWs Improved quality of care Improved quantity and quality of research output
Data Sources Standardized quantitative tools Qualitative sources (FGDs, Reports, Images etc) University records Library records Health Facility records
Data Collection and Management Program progress is tracked through Microsoft Project Data collected and entered into a passwordprotected Microsoft Access Database which is on LAN Data entry to be transitioned to TrainSmart and REDCap which are online platforms
Data Collection Tools Trainings: Enrollment records, Attendance sheets, Biodata, Pre/Post Training Evaluations Skills Labs: Booking records, Usage records, satisfaction surveys Decentralized sites: Site surveys, student records, research records Surveys: Baseline surveys, midcourse and endline surveys Workshops: Attendance sheets, reports
Lessons Learned Standard M&E approaches are applicable to the MEPI program Some aspects of the program are challenging to conduct M&E activities (e.g. qualitative) Requires feedback and support from various stakeholders at all levels The Theory of Change approach enabled PRIME-K to summarize the our program M&E on a single-page easy to follow diagram
Thank You