Quality Assurance for Rural Transport Services Project Cluster in Tanzania

Transcription

1 Quality Assurance for Rural Transport Services Project Cluster in Tanzania Final Report Annabel Bradbury TRL Limited TAN2015A July 2015

2 The views in this document are those of the authors and they do not necessarily reflect the views of the Africa Community Access Partnership (AFCAP), TRL Limited or Cardno Emerging Markets (UK) Ltd for whom the document was prepared Cover Photo: Annabel Bradbury, Bago Talawanda Road, Tanzania Quality assurance and review table Version Author(s) Reviewer(s) Date 1.1 Annabel Bradbury Paul Starkey 03/07/ Annabel Bradbury Paul Starkey 13/07/ Redacted* Annabel Bradbury Paul Starkey 13/07/15 * This redacted version excludes Annexes that contain various identifiable comments relating to the various research reports reviewed under this contract. AFCAP/ASCAP Project Management Unit Cardno Emerging Market (UK) Ltd Oxford House, Oxford Road Thame OX9 2AH United Kingdom Page 2

3 Abstract This Final Report summarises the review of the six research project clusters undertaken in Tanzania by IFRTD, Amend, HelpAge International and Transaid, and a review of their Final Reports. After a summary of the dissemination workshop which took place in Bagamoyo, Tanzania in April 2015, the report summarises the lessons learned from adopting a cluster approach, including feedback from the contractors themselves. The report also provides recommendations for implementing rural transport service project clusters in other AFCAP countries, in particular focusing on project duration, sample size, statistical significance, disseminating results, and research competency. The conclusion alludes to the strong appetite for rural transport services research and outlines some of the research priorities that will be considered for the next country and regional clusters. Key words Quality assurance Rural Transport Services Motorcycle taxis Project cluster Tanzania AFRICA COMMUNITY ACCESS PARTNERSHIP (AFCAP) Providing solutions for safe and sustainable rural access across Asia AFCAP is a research programme, funded by UK Aid, with the aim of promoting safe and sustainable rural access for all people in Africa. AFCAP supports knowledge sharing between participating countries in order to enhance the uptake of low cost, proven solutions for rural access that maximise the use of local resources. The AFCAP programme is managed by Cardno Emerging Markets (UK) Ltd. See Page 3

4 Acronyms AFCAP ARTReF CBO IFRTD LGA MDA NGO PMO RALG PMU OCR QA RAI RECAP RTS RTSi SME SUMATRA SWOT TSAF Africa Community Access Partnership African Road and Transport Research Forum Community Based Organisation International Forum for Rural Transport and Development Local Government Authority Ministries, Departments and Agencies Non governmental organisation Prime Minister s Office, Regional Administration and Local Government Programme Management Unit Older People Co Researchers Quality Assurance Rural Access Index Research for Community Access Partnership Rural Transport Services Rural Transport Services indicators Small and Medium Enterprises Surface and Marine Transport Regulatory Authority Strengths, Weaknesses, Opportunities and Threats Tanzania Social Action Fund Page 4

5 Contents Abstract 3 Key words 3 Acronyms 4 1 Introduction Report on the Dissemination Workshop Review of the Cluster Final Reports Lessons Learned Planning and methodological approach Project scope and deliverables Dissemination of results and capacity building 13 5 Recommendations for AFCAP Conclusions References Page 5

6 1 Introduction The purpose of this Final Report is to summarise the quality assurance, technical review and project co ordination activities for the Rural Transport Services project cluster in Tanzania, with specific reference to the Final Reports produced by the contractors conducting the cluster research, as well as the dissemination workshop undertaken in April It also summarises the process by which the Final Reports, submitted by the four research organisations (Transaid, Amend, IFRTD and HelpAge International), were reviewed. The suite of Rural Transport Services projects in Tanzania are as follows: 1. Transaid (TAN2015E) Tanzania motorcycle taxi driver training assessment and development of appropriate training curriculum 2. Amend (TAN2015B) Road traffic injury on rural roads in Tanzania: a baseline study to understand the magnitude and characteristics of road traffic injury on low volume rural roads in Kilolo District 3. IFRTD (TAN2015C) A study on first mile transport challenges for smallholder farmers in the hinterland of Iringa Kilolo road in Tanzania 4. IFRTD (TAN2015F) Collecting baseline data for rural transport services indicators in Tanzania 5. HelpAge International (TAN2015D) Learning with older people about their transport and mobility problems in rural Tanzania Amend submitted an additional proposal to AFCAP to identify opportunities to improve safety through regulation of motorcycle taxi associations. This addendum to their main project (TAN2015B) explores the implications of the SUMATRA regulations on boda boda operations in rural Tanzania, identifies road safety best practice among boda boda associations, and provides guidelines on applying best practice in rural areas: 6. Amend (TAN2015G) Road traffic injury on rural roads in Tanzania: Investigations into the opportunities to improve safety through regulation and self regulation of motorcycle taxi associations The activities carried out by the technical reviewer, as set out in the Terms of Reference, are listed below, along with progress on those tasks since the inception report: Liaise with projects to promote good quality research implementation and the timely submission of final reports. Regular communication between the technical reviewer and the project teams ensured that the researchers followed the project timetable and enabled progress with the fieldwork, data collection, data analysis and reporting to be monitored. Attend a workshop in Tanzania where project findings will be presented by the project teams and engage in discussions on follow up initiatives. The dissemination workshop was attended from April 2015 in Bagamoyo, Tanzania. Activities included contributing to break out thematic group discussions, co ordinating a small group on the field visit to the Bago Talawanda road, and facilitating discussion among representatives of countries outside Tanzania, on priorities for undertaking rural transport services research. The workshop report produced by IFRTD was also technically reviewed. Page 6

7 Review the draft final reports and make appropriate recommendations to the AFCAP PMU and the project teams. All the Final Reports were reviewed and comments relating to technical and grammatical issues, as well as formatting, were highlighted and recommendations for future research outlined to the researchers. Liaise with the AFCAP PMU and undertake other relevant tasks relating to quality control assurance for the project cluster in line with the contracted professional and financial resources available. Regular contact with the RTS Technical Manager was made throughout the project and most of the final reports were discussed prior to feedback going to the researchers, with progress reported throughout the period of the assignment. Page 7

8 2 Report on the Dissemination Workshop IFRTD and the Tanzania National Forum Group on Rural Transport organised the Tanzania dissemination workshop from 21 st to 23 rd April 2015 in Bagamoyo, at which members of the project teams reported their research findings. The workshop report (IFRTD, 2015) provides a detailed summary of the workshop proceedings and participants who attended from Tanzania, Ethiopia, Ghana, Kenya, Mozambique, Malawi, Uganda, Zambia and the UK. A selection of government officials from SUMATRA, Prime Minister s Office Regional Administration and Local Government (PMO RALG), Ministry of Works, and Ministry of Agriculture attended, as well as other key stakeholders from the Traffic Police Department, Kilolo District Council and the T 2 Centre. The workshop enabled the project teams to showcase their research, and facilitate discussion on the policy implications for the research outcomes. The workshop objectives were as follows: To present and discuss results of the recent AFCAP supported research cluster relating to rural transport services and rural road safety in Kilolo District, Tanzania; To discuss the policy implications of these studies for Tanzania and other countries; To promote networking and information on sustainable, appropriate and safe rural transport services; To propose and plan for additional transport services research activities, training and information exchange, including potential research clusters in other AFCAP countries. The IFRTD report (2015) gives a detailed summary of the key themes and presentations from the workshop, including the keynote speech and official opening given by Dr Deo Mtasiwa, Deputy Permanent Secretary at PMO RALG, as well as presentations on each of the Tanzania project cluster studies. A field visit to the Bago Talawanda Road and the Makofia Mlandizi Road took place on the second day of the workshop in order to give participants an opportunity to interview motorcycle taxi operators, users and market traders. The Bago Talawanda Road was the site of a previous infrastructure project undertaken in by Roughton (2013) in AFCAP Phase 1, involving trial demonstration sections of various surfacings including: Concrete geocells (Figure 1); Figure 1: Concrete Geocells Hand packed stone (Figure 2); Double surface dressing; Concrete strips (Figure 3); Otta seal with sand cover seal; Double sand seal; Slurry seal, and; Gravel wearing course. Through various discussions with different road users and motorbike taxi operators, who are the only public transport service providers in the area, the concrete geocells were reportedly the preferred surface type (albeit one of the most expensive options). The hand packed stone was considered to be extremely Page 8

9 dangerous for motorcyclists, with reports of three fatalities in recent months on this road section alone. Motorists, cyclists and pedestrians have created a diversion to the side of this section of road which is black cotton soil and is also treacherous to pass during heavy rains. The concrete strips were also found to be unpopular with motorcyclists as they complained of having to leave the strips when they pass a bicycle or motorcycle and being forced off whenever there is oncoming 4 wheeled traffic. The strips have steep angular sides which are not designed for motorcycle or bicycle dismounting, and they also have raised concrete ties connecting the two strips, and so are completely unsuitable for 3 wheeled vehicles. The motorbike taxis also commented that the sections with a double surface seal were slippery when wet. Figure 2: Hand Packed Stone Figure 3: Concrete Strips The field visit was extremely valuable in obtaining a snapshot of the challenges and opportunities for communities living along a typical rural road in Tanzania, and especially the rapidly changing dynamic of the traffic composition along the road. In particular, observations were made about the sudden growth in motorcycle taxi services and how they are improving first mile connectivity between households and their farms, as well as access to the main Bagamoyo Msata Road and the Dar es Salaam Morogoro Road (TanZam Highway) from where conventional bus services are available. The feedback discussions at the workshop were extremely fruitful, and the need for better collaboration on rural accessibility between transport services and engineering practitioners as well as other non transport sectors (agriculture, health, education etc) was fully endorsed by the Director of Rural Roads, Eng. Elina Kayanda. The key topics for discussion at the workshop were: Regulations and training for motorcycle taxis Safety and security of motorcycle taxis Transport for agriculture and small freight Planning, measuring and evaluating appropriate transport services Page 9

10 Access for vulnerable groups in rural areas Improvement of basic rural access (including tracks). The third day of the workshop was dedicated to planning for rural transport services research and policy initiatives, with a view to making recommendations for AFCAP investment in rural transport services projects (including country and regional project clusters). Following feedback from a Tanzania working group and an international working group, thematic and country specific priorities were produced for consideration by the AFCAP Programme Management Unit (PMU), and these can be viewed in the workshop report (IFRTD, 2015). The issues that ranked most highly in these discussions were transport consolidation for agricultural harvests, passengers and freight; and establishing baseline data and performance indicators for rural transport. Subsequently, a workshop session on transport services took place at the 7 th Africa Transportation Technology Transfer (T 2 ) Conference in Bulawayo, Zimbabwe on 14 th May Paul Starkey was the key speaker, and IFRTD, Amend, HelpAge International and Transaid each gave a synopsis of their Tanzania projects. Following the presentations, an extended break out session sought to discuss key issues affecting accessibility in Sub Saharan Africa, the results of the Tanzania research cluster, and recommendations for specific follow up actions by AFCAP. Workshop participants were split into two groups to discuss the following topics: 1. Motorcycle training, safety and regulation 2. Rural transport indicators, first mile connectivity, conventional RTS and load consolidation and significance for AFCAP. The participants emphasised the benefits of a networked research approach, supporting the use of project clusters undertaken principally by local practitioners, with capacity building and dissemination built into the research. Page 10

11 3 Review of the Cluster Final Reports Following the review of the Final Reports for all six projects in the Tanzania cluster, many of the observations and lessons emerging from the technical review of the proposals and Inception Reports still apply. The timescales for delivering the projects and reports were extremely tight, with some of the research contracts only being finalised in late January 2015, leaving only two months for fieldwork, data collection and analysis prior to the AFCAP Transport Services Workshop, which was held in Tanzania from 21 st 23 rd April The majority of the draft Final Reports were submitted for review prior to the Tanzania workshop, but only one Final Report was submitted before the AFCAP steering group meeting took place on 15 th May at the T 2 Conference in Zimbabwe. There was a gross lack of time available for the research teams to collect sufficient empirical research data. The data collection activities were undertaken extremely rapidly, and in some cases the survey questionnaires and checklists were not approved through the QA process before they were administered in the field, and this was a fundamental disadvantage of having such a tight timescale for delivery. Some of the fieldwork began before the Inception Reports were reviewed and approved, and so any recommendations for improving the survey forms and stakeholder checklists could not be implemented. The Technical Reviewer acknowledged that some contracted organisations submitted draft Final Reports that were technically inadequate and contained numerous typographical and grammatical errors. In some respects, they required significant revision before they could be approved. This may have been a consequence of the limited time available between fieldwork ending and submitting the reports. Despite some concerns over the quality of data collection and reporting, the fact that these projects were conceived and delivered in a 3 4 month window is an impressive feat. By their own admission, the researchers who undertook these projects agreed that the programme of work was very ambitious for the timescale allocated to the Tanzania cluster. In spite of the substantial constraints faced by the researchers and their survey teams, they have all managed to produce thoughtprovoking research outcomes that will contribute to the body of knowledge on rural transport services. Most importantly, they have raised the profile of rural transport service challenges and opportunities within Ministries, Departments and Agencies (MDAs) in Tanzania and other AFCAP countries, and have facilitated considerable capacity building and awareness raising among local researchers, District Council staff and the African Road and Transport Research Forum (ARTReF). Page 11

12 4 Lessons Learned There are some lessons to be learned in conducting short term, small scale studies as part of a project cluster. These lessons have been collated by the Technical Reviewer and the researchers themselves, and have been grouped into the following categories: 1. Planning and methodological approach 2. Project scope and deliverables 3. Dissemination of results and capacity building. 4.1 Planning and methodological approach Planning for the research cluster Ideally the teams working in a cluster should attend a planning meeting prior to the projects commencing. This would enable them to do the following: Understand each other s research proposals, objectives and methodologies; Review the research themes and decide on the most suitable and strategic geographical location this applies to both country and regional clusters; Agree on key baseline data that can be collected jointly, and therefore benefit from cost savings brought about by shared data collection and analysis; Identify key cross cutting themes; Agree on a common engagement process with government officials and policy makers at the research locations (both for practical administrative purposes and for influencing policy outcomes). Methodological approaches The planning of the inception phase should consider the methodological approach of each research project, including the sample size, sampling method, empirical data collection methods (qualitative and quantitative) and data analysis methods, including the use of statistical analysis and relevance of statistically significant data. With this in mind, the following should be considered in the inception phase of the projects: Key relevant MDA personnel should be involved in the identification and design of the research projects, and baseline data characteristics to be collected, to ensure that the findings will be of practical use and that there is sufficient buy in to the research and accountability for applying the research outcomes; Timeframes need to be realistic the time available for the Tanzania project cluster was too short a minimum of 6 months for each project is recommended (from contract to final deliverable); A longer inception phase is essential, to allow scheduling and mobilisation of staff. 4.2 Project scope and deliverables The project scope and what is achievable within the programme of delivery is vitally important for projects conducted in a country (or regional) cluster. While the intention of the project cluster is to commission short, multi disciplinary projects that represent value for money, this should not mean compromising the quality of the data collected and analysed, or the research outcomes. To this end, the scope of the project and expectations for deliverables need to be clear at the points of both proposal sign off and methodology sign off, for example: Following reviews of the draft reports, some of the researchers were given the impression that they had been expected to collect more detailed data information that perhaps could Page 12

13 have been expected through larger studies, but not through such short and low budget projects as these; The requirement for a more thorough review process and very concise reporting is different from AFCAP Phase 1, which the researchers had not been expecting, and so not all had budgeted sufficiently for re drafting of project deliverables, in particular the Inception and Final Reports. In their feedback they did admit that it is something they need to adapt to for future AFCAP commissions; The researchers were pleased that the reports were enhanced through technical review, as it will hopefully increase the impact that they have as they will be read by more people. However, they also recognised that getting the reports to a publishable standard on the AFCAP website will require more time from senior staff, which will increase the budget; The teams also commented on the disparity between AFCAP s commitment to involving African researchers in AFCAP funded projects, and the requirement for high level reviews on deliverables, representing a greater level of scrutiny than in AFCAP Phase 1. Many African researchers are sometimes constrained by language and literacy when English is not their first language, and so the quality of their written outputs can be compromised. Lastly, during the course of the research cluster project delivery, it would be useful for the teams to have: A mid term research team meeting to review progress, challenges and ways of overcoming them; Emerging policy issues that can be grouped together and a communication plan for disseminating findings to the MDAs; Joint dissemination plans between the cluster project researchers. 4.3 Dissemination of results and capacity building The research teams, along with the AFCAP Technical Manager and the technical reviewer saw the value of dissemination in the form of a well structured participatory stakeholder s workshop (held in Bagamoyo, Tanzania). There was a suggestion for future research clusters, that after the dissemination workshop, a follow up mechanism could be introduced to establish whether there is any uptake of the project and workshop recommendations by the stakeholders, and especially the MDAs. This would enable AFCAP to be able to monitor and evaluate key outcomes and impacts. In addition to the project workshop, dissemination of the project results should also be specifically targeted towards: The Ministry which assisted in identifying and designing the research, through face to face meetings with the researchers; Local stakeholders, including local government authorities (LGAs) and the communities where the research took place. In addition to collecting baseline information, there is also a need for follow on action research to build evidence of scalable interventions. This follow on research should be identified through the feedback session with the MDAs, and details of this should be provided to the local stakeholders so that they know how their involvement has assisted in the process of delivering real improvements in transport service provision. Finally, feedback from the researchers made reference to the Cardno Emerging Markets contract terms and conditions, which contain clauses (for example regarding insurance requirements) that may not be conducive to local organisations registered in Africa. In order to prevent delays in commissioning future research clusters and contracting African small and medium enterprises (SMEs), the terms and conditions should be reviewed in light of recent experiences. Page 13

14 5 Recommendations for AFCAP In addition to the lessons learned from the recently completed Tanzania research cluster, there are some key recommendations going forward for undertaking networked research in the form of further country based and regional research clusters on rural transport services. Project duration: the initial plan for the AFCAP transport services programme is to continue undertaking a country cluster on a 6 month cycle, starting with Ghana in late 2015, followed by Zambia in 2016 (to be confirmed). Larger scale regional research clusters will also be commissioned to cover a selection of countries within West, East and Southern Africa; and also between Africa and Asia where appropriate. It is absolutely crucial that the minimum duration of these projects is no less than 6 months (from contract signature to final report delivery), in order to facilitate the following: A planning component of the inception phase to accommodate the requirements of MDAs in country; Engagement with local district personnel and mobilisation of local research staff; Agreement on a common methodology that will complement each research project, benefiting from cost savings through shared data collection and analysis; A comprehensive technical review that will feed into the survey design and data collection and analysis methods; Production of quality reporting deliverables to an international published standard. Larger sample sizes: One of the key criticisms of the Tanzania research cluster at the dissemination workshop in Bagamoyo, was the small sample size used by the researchers in all of the projects. HelpAge International had the largest sample of 358 surveys, and IFRTD had the lowest sample of only 45 transport service users in Kilolo District. None of the data collected by the researchers was statistically significant, nor was it intended to be. However, it is clear that the quality of the data collected as part of the research clusters needs to be much higher in order to get the attention of policy and decision makers, and while the data collected does not necessarily need to be statistically significant, a rigorous methodology with a sample size that is representative of the population under survey should be a key requirement. Statistical significance: Where researchers seek to obtain statistical significance in their data, the following questions should be considered when deciding on an appropriate sample size for a survey: 1. How big does the total sample need to be in order to get statistically robust results? 2. How diverse does the sampling need to be in order to be representative of the population? 3. How close is the sample result to a hypothetical population result, had the whole population been surveyed? 4. When comparing the baseline data with impact data, how big a difference will be detected as statistically different? Ideally, factors likely to influence responses to the survey should be identified before the survey and controlled for when selecting the sample to survey. These factors might include occupation, family size and household income. Where possible these factors should be included in the selection of villages chosen to survey, so that a mix of different factors (similar to that seen in the population) is surveyed. Characteristics of the villages sampled should also be used to select control sites based on these factors. Page 14

15 Dissemination and communicating results: results of the research clusters should be fed back to MDAs, local district personnel and research beneficiaries in the rural communities being studied. In keeping with Output 3 of the AFCAP logframe Knowledge exchange amongst institutions involved in rural transport provision in Africa, research outputs should be disseminated through various international conferences, peer reviewed journals and other open access platform providers, as well as the AFCAP website ( and DFID s own Research for Development (R4D) portal ( Research Competency: the Tanzania research cluster has provided an important springboard for young African researchers who are seeking a career in (transport) research, by undertaking empirical fieldwork and data collection through Transaid, Amend, HelpAge International and IFRTD. However, this process has demonstrated that some of these researchers, who may be professionally qualified in other vocations such as medicine or law, have not been specifically trained to undertake field research or write research reports, which themselves require a particular competence that has not necessarily been demonstrated effectively in these projects. AFCAP has a role in developing research skills, particularly in rural transport service provision, and there is a set of research materials produced by TRL in AFCAP Phase 1 that could be utilised for this purpose. The training comprises the following sessions: 1. Transport Services Provision and its Impact on Goods and Services 2. Rural User Needs and Access to Transport Services in Sub Saharan Africa 3. Transport Services and Operators Current Patterns and Constraints 4. The Broader Transport Services Context 5. Planning Transport Services with Stakeholders 6. Research Methods 7. Road Safety and Qualitative/Quantitative Data Analysis. Capacity Building: The procurement and tendering process of AFCAP, ASCAP and the wider Research Community Access Partnership (RECAP) could be more supportive of local experts, Small and medium enterprises (SMEs), Non government organisations (NGOs) and community based organisations (CBOs). More favourable terms and conditions that are tailored to national experts, who may experience restrictions in the contract wording, would build capacity for researchers in Africa and Asia, and encourage them to respond to invitations to tender from the PMU, and submit concept notes for undertaking research. Page 15

16 6 Conclusions In spite of some challenges and constraints in delivering the Tanzania research cluster, it has been an extremely effective model for implementing rural transport services research through a suite of technically aligned research studies in a common geographical location, giving weight to the overall findings from the projects in combination. It has also successfully facilitated the wider dissemination of research outputs and project findings, information exchange between practitioners, and increased opportunities for implementation, and represents a huge accomplishment by the research teams. Going forward, transport services priorities have been identified by participants at the Tanzania and Zimbabwe workshops for other countries as follows: Motorcycle training, safety and regulation Creating/maintaining basic rural access for light vehicles, including low volume rural roads, tracks and paths Planning for rural transport services: establishing baseline data and performance indicators Enabling passenger/freight services, including consolidating demand, proactive route licensing and community involvement Safety awareness for all road users Emergency transport Mobile phones to reduce travel and increase transport efficiency Promotion of intermediate means of transport Sharing innovative technologies and approaches. The appetite for rural transport services research is strong, and has the backing of the African Roads and Transport Research Forum (ARTREF). It is the ideal time to stimulate this enthusiasm through investment in further research clusters with support from key stakeholders that can embed the results into government policy. Page 16

17 7 References IFRTD, Understanding and Improving Rural Transport Services. Report of a Workshop Held April 2015, Bagamoyo, Tanzania. IFRTD 2015 Bagamoyo RTS Workshop Report AFCAP CON2027B SC15014 v rdctd.pdf Roughton, Design and Construction of Demonstration Sites for District Road Improvement in Tanzania: Final Report, AFCAP TAN Tanzania 2013 Demo+Sites+Final+Report AFCAPtan008 v pdf Page 17