Implementation and Formative Evaluation of the Rheumatic Fever Prevention Programme FINAL Report. Prepared for Ministry of Health Manatū Hauora

Transcription

1 Implementation and Formative Evaluation of the Rheumatic Fever Prevention Programme FINAL Report Prepared for Ministry of Health Manatū Hauora 15 February 2013

2 Contents Contents Preface 5 1 Executive summary Background to the Rheumatic Fever Prevention Programme Evaluation overview Evaluation findings Recommendations 10 2 Introduction Rheumatic fever in New Zealand The Rheumatic Fever Prevention Programme Glossary of terms 16 3 Evaluation approach Evaluation aims and objectives Evaluation methodology Evaluation caveats 18 4 RFPP background and context Leading up to Budget Budget 2011 investment in rheumatic fever Allocating Budget 2011 funding across DHBs RFPP and the Better Public Services target 23 5 Strategic issues with national implementation Implementation timelines Efficacy in reducing rheumatic fever Sustainability of continued focus on rheumatic fever prevention 27 6 Local services workstream Contract requirements and contracting models Delivery models Implementation strengths Future considerations and improvements 40 7 Parent/caregiver experiences Family/whānau and household context Parent/caregiver experience with school throat swabbing service Parent perspectives on school-based delivery Parent/caregiver experience of primary care Methods of raising awareness with parents/caregivers Typologies 58 8 Community awareness raising workstream CAR workstream activities Early impact of RFPP on community awareness Potential improvements for CAR 70 9 Conclusions and recommendations Conclusions 72 ii ii

3 Appendices 77 Appendix 1: Description of New Zealand s RFPP 78 Appendix 2: RFPP logic models - Ministry of Health and Litmus 85 Appendix 3: Evaluation team 91 Appendix 4: Evaluation methodology 92 Appendix 5: Evaluation questions and data sources 100 Appendix 6: School intercept survey summary findings 103 Appendix 7: National (HPA-produced) and example local rheumatic fever pamphlets 110 Appendix 8: Evaluation tools 116 References 141 iii

4 It is a challenge we have a four year programme in which to turn around this titanic of a disease, which is not just rheumatic fever. It s poverty and other socioeconomic drivers. The most important thing about this programme is that it is a start We have started on the journey that has highlighted the importance of rheumatic fever. The challenge will be maintaining this. (Evaluation participant) 4

5 Preface This report was prepared for the Ministry of Health by Michele Grigg and Ingrid McDuff from Litmus Limited with input and guidance from Lisa Davies, Kaipuke Consulting; Catherine Poutasi, Integrity Professionals; and Liz Smith, Litmus Limited. We sincerely acknowledge and thank those who provided valuable feedback on their experiences of the Rheumatic Fever Prevention Programme, including providers, schools, key stakeholders, and particularly parents/caregivers and whānau, along with other dedicated individuals working to eradicate rheumatic fever from New Zealand. We also wish to thank our expert advisors, Dr Ricci Harris, Catherine Poutasi, Dr Deborah Read, and Dr Nikki Turner. Their valuable advice and input helped frame and guide the evaluation. Lastly, thanks to the Ministry of Health team for their time, enabling access to information, and participating in workshop discussions. Please contact Liz Smith if you have any questions about this report. 5

6 1 Executive summary 1.1 Background to the Rheumatic Fever Prevention Programme Rheumatic fever is a preventable disease that occurs in some people following an infection of the throat with Group A Streptococcus (GAS). If left untreated the infection can lead to an autoimmune response that may permanently damage the heart through the development of rheumatic heart disease (RHD). Rheumatic fever and RHD is a disease that disproportionately affects Māori and Pacific children, and is a marker of both socioeconomic deprivation and poor access to healthcare. Recognising the social and financial cost of rheumatic fever to the New Zealand population, the Government committed resourcing from both Budget 2011 ($12 million) and Budget 2012 (an additional $12 million) for the Rheumatic Fever Prevention Programme (RFPP). The overarching goals of the RFPP are to achieve equity of incidence and outcomes of rheumatic fever between Māori and Pacific children, and other children, and reduce the overall rates of the disease. The objectives of the RFPP are to establish school-based throat swabbing services in targeted schools in high-risk areas; develop clinical tools and training to support the health sector; develop a health promotion and awareness programme (with high-risk communities, the health sector and the general public); and develop a rheumatic fever surveillance and monitoring programme. RFPP funding was allocated to throat swabbing providers within eight District Health Board (DHBs) areas based on cluster analysis of rheumatic fever incidence, specifically: Northland, Counties Manukau, Waikato, Bay of Plenty, Lakes, Tairāwhiti, Hawke s Bay, and Capital & Coast. As specified in their contract, the overall service objective for these providers is to reduce the rate of rheumatic fever by: Providing throat swabbing and referral services in the school, home or other settings as appropriate for school children aged 5 14 years who present with sore throats in highrisk areas, and eligible whānau/family members living with these children. Increasing awareness of rheumatic fever risk factors among children and their whānau/families in the key geographical areas. Developing and maintaining relationships with other health and social service providers (including Whānau Ora providers) to facilitate referral and support, as appropriate. In 2011 (after the launch of the RFPP), new rheumatic fever targets were announced as part of the Better Public Services (BPS) Action Plan. The RFPP was integrated into this broader inter-agency government work programme from September 2012, acknowledging the need for a multi-faceted approach to reduce the high rates of rheumatic fever in New Zealand. 1.2 Evaluation overview Litmus was commissioned by the Ministry of Health ( the Ministry ) to undertake an implementation and formative evaluation of the first 18 months of the RFPP from 1 July 2011 to 31 December The key aims of the evaluation were to provide an overview of how the RFPP is being been implemented, and recommendations to enhance implementation. 6

7 The evaluation used a mixed-methods approach including: literature and documentation review; key informant and RFPP provider interviews; review of monitoring data; case studies of four RFPP sites (Whāngarei, South Auckland, Rotorua and Porirua); 12 focus groups with parents/caregivers; and a school intercept survey of 407 parents/caregivers. These data sources were analysed collectively to address the evaluation aims and objectives. As this is a formative and implementation evaluation, it is acknowledged that the plans and activities of the Ministry, RFPP providers and other agencies have evolved and progressed since the fieldwork period and the completion of this report. 1.3 Evaluation findings Sore throats are not a priority for parents/caregivers Parents/caregivers interviewed are committed to fostering the health and well-being of their family/whānau. Parents/caregivers seek to be proactive in supporting their children, and in preventing illness and disease. However, they face a number of inter-related environmental, social and economic barriers that undermine their ability to support the wellness of their family/whānau, in particular cold and damp homes. For parents/caregivers interviewed, sore throats are not a priority compared to other health needs of children and family/whānau. Their usual response to a child s sore throat is to keep an eye on their child, and usually they will attend school. Medical care was only sought when the condition got worse or coincided with another more urgent concern. Costs, available appointments and long wait times deter parents/caregivers from taking their child to a doctor with what is perceived as a relatively minor disorder. Awareness of the link between untreated GAS throat infection and rheumatic fever is low. Positive and relevant initiative The RFPP throat swabbing service fills a need for those most at risk of rheumatic fever, addressing many of the barriers to diagnosis, treatment and preventing progression from an untreated sore throat to rheumatic fever. Parents/caregivers are overwhelmingly positive about throat swabbing being delivered in schools as the service does not cost them anything, is convenient, undertaken by trusted providers in a place where children are comfortable, and the results and provision of appropriate treatment is done in a responsive way. The throat swabbing service has been well received by schools, with principals, teachers and administrative staff supportive of RFPP provider visits and the service they are providing for students. Local providers and wider stakeholders are acutely aware of the damage rheumatic fever creates in their communities, and the RFPP is a timely initiative. While DHBs are supportive of initiatives to address rheumatic fever in their regions, they advocate for the RFPP to be linked and integrated with their existing initiatives and wider services. This integration of services is perceived to improve the chances that the RFPP goals will be achieved, and be more sustainable long-term. 7

8 Throat swabbing is occurring amongst high-risk populations As contracted, all RFPP sites have implemented their throat swabbing services. A range of service delivery models for throat swabbing are being used that reflect regional and community needs, and the capacity and capability of the RFPP provider. Providers are at different stages of implementation, reflecting the staggered commencement dates. RFPP providers, across the four case sites, have a highly dedicated frontline workforce, with strong community links with families/whānau, schools and local provider networks, including sub-contracted providers delivering other aspects of the RFPP such as home visits for antibiotics delivery, compliance checks and health promotion. Providers are adopting innovative and culturally appropriate practice to ensure engagement with high-risk populations. During the early implementation stages, local providers have primarily focused on establishing the school throat swabbing component of the RFPP. Community awareness raising and primary care engagement aspects of the RFPP have received less attention. Strengthening RFPP delivery A number of areas were identified that would strengthen the delivery of the RFPP, specifically: Delivering across all RFPP workstreams in particular community awareness raising and engagement with primary care. Focus in these areas is critical in ensuring sustainable outcomes for parents/caregivers after the completion of the RFPP. Parents/caregivers need to be informed on when to act and to be empowered to access appropriate services. - Community awareness raising - Feedback from parents/caregivers highlights the need for communications that are engaging, culturally appropriate, and supplemented by face-to-face opportunities to ask questions, particularly for Māori and Pacific parents/caregivers. For many parents/caregivers communications alone will not support behaviour change due to health literacy and a range of other barriers. For these parents, health systems need to actively facilitate their access to diagnosis and treatment as part of the RFPP. - Engagement with primary care RFPP provider engagement and awareness raising with primary health care providers is minimal across the sites, and usually relies on individual initiative rather than a systematic process. Health professionals in RFPP areas have a key role in reinforcing core messages and behaviours in their interactions with the community. Ensuring effectiveness for tamariki Māori and their whānau While the RFPP has a strong focus for tamariki Māori and their whānau, services could be strengthened. Suggestions include ensuring providers across all regions have the capability to reach those most at-risk from rheumatic fever, and have local credibility for frontline services and home visits. Providers also need CAR resources that have strong relevance for Māori, and do not stigmatise tamariki and their whānau. Ensuring effectiveness for Pacific children and their fanau Currently, there is limited focus on Pacific people at strategic and service delivery levels for the RFPP, despite a high Pacific population in two RFPP sites. Given the considerable burden of the disease on Pacific children and the objectives of the RFPP, this lack of focus is a concern. 8

9 Developing referral pathways with other health and social support services to address the underlying causes of rheumatic fever There is wide variation across RFPP providers in facilitating referral and support of children and their families/whānau with other health and social service providers (including Whānau Ora), where needed. Throat swabbing services are being delivered (in some areas) in a way that does not identify nor follow up child health or social issues. The lack of clear process presents a significant risk for RFPP consistency, and may miss crucial opportunities to address the underlying determinants of rheumatic fever, and improve child and family/whānau health more broadly. Need to enhance the RFPP s focus on quality improvement. Wide variation in approaches to service delivery exists across providers at the local level. A degree of flexibility is important for providers to respond appropriately to meet unique local needs. However, this flexibility needs to be embedded in clear operational frameworks and guidelines for the RFPP service delivery with regard to: consent approaches; the use of standing orders; budgeting and payment for pharmacy, GP and laboratory costs; swab quality and swabbing processes; protocols for community-based/opportunistic swabbing and referral pathways. Efficacy of RFPP in reducing rheumatic fever Experts and providers expressed concern about the efficacy, cost-effectiveness and sustainability of reducing rheumatic fever using the current mix of RFPP activities, particularly as primordial determinants of health and preventing GAS infection are not addressed by current services. Widening the scope of deliverables and strengthening referral pathways to wider health and social services will support providers to build on progress made in the early stages of the RFPP, and potentially address underlying determinants of rheumatic fever. Measuring outcomes and impacts A more systematic approach to impact and outcomes measurement for the RFPP is needed, with assurances of quality data collection. The Ministry is currently completing the monitoring and evaluation framework and streamlining the provider monitoring requirement to ensure quality data. Stakeholders advise that identifying target areas (and allocating funding) based on GAS incidence would be a more reliable indicator of community need. However, currently there are no national measures for GAS, and significant gaps in understanding of the epidemiology of GAS. Planning for sustainable outcomes Providers and stakeholders are advocating for collaborative and future-focused planning ensure RFPP outcomes are sustainable. Providers need adequate lead-in time for contract completion to manage the expectations of schools involved, and ensure parents/caregivers can access diagnosis and treatment for their children when the throat swabbing service withdraws. Awareness rasing for parents/caregivers, effective systems for families/whānau who are under-served by current health services, and engaging with primary care to support rheumatic fever prevention will be key to sustaining RFPP gains long-term. 9

10 1.4 Recommendations The following table details recommendations to enhance the RFPP as drawn from the evaluation. Table 1: Implementation and Formative Evaluation recommendations for the RFPP Area Recommendations 1. RFPP focus Consider collection of GAS incidence data to more reliably inform location of future swabbing services Consider a programme that targets GAS prevention in the community, rather than individual symptomatic throat swabbing in children Continue and widen focus on addressing primordial determinants of health in target areas 2. Pacific Strengthen focus on Pacific from strategic (Ministry) level through to contracts and service specifications (especially for service delivery and CAR in areas with Pacific populations) Ensure Pacific-language resources are made available in a timely manner Consider production of other appropriate CAR resources and materials, including non-written media 3. Māori Seek clarity on requirements of sub-contracted providers around service appropriateness for Māori Consider production of appropriate CAR resources and materials, including non-written media 4. Service integration 5. Primary care engagement 6. National and local CAR 7. Quality frameworks and guidelines Examine options for integrating throat swabbing with other child health services Ensure referral pathway model/protocols developed (with health and wider social support agencies) Involve DHBs in future funding decisions to ensure RFPP planning reflects local knowledge and local strategies Focus effort on building primary care awareness and engagement, with RFPP and wider rheumatic fever messaging Establish a centrally-led primary care awareness raising strategy Review focus and allocation of spend to HPA in out-years Establish strategic national RFPP CAR plan Review respective national (HPA) and local (provider) community awareness raising roles and responsibilities and fund accordingly Ensure national-level CAR learnings from other programmes targeting Māori and Pacific are considered in the development of future resources Ensure timely production of any future resources Ensure resources are appropriate for the local context Provide national guidance and support on: - use of standing orders - the use of an opt out consents approach - referral pathway model/protocols (including health and wider social support) - community-based/opportunistic swabbing guidelines and protocols - use of sub-contracted providers (ie, ensuring appropriateness of service delivery for Māori and Pacific children and families/whānau) Note: some of the above could be delivered through NCS. 10

11 Area Recommendations 8. National Coordination Service Strengthen NCS s role to: - establish quality assurance standards and monitoring for swab taking and ensure consistency in quality within and across sites over time - advise providers on how to implement swabbing approaches and processes (informed by standards developed for school immunisation programmes, for example) - advise on and recommend use of standing orders - create opportunities to share learnings between frontline staff - regularly document and disseminate best practice approaches 9. Sustainability Ensure a transparent exit plan is communicated to providers ahead of contract end date (at least six months) Clarify expectations of relative emphasis on CAR versus throat swabbing with local providers Consider links to other child health programmes with the aim of integrating service delivery and messaging opportunities 10. Future evaluation Review provider data monitoring for content and quality to ensure longerterm RFPP outcomes will be measurable over time Establish a monitoring and evaluation framework to reliably inform effectiveness of RFPP delivery and tracking of RFPP progress 11

12 2 Introduction 2.1 Rheumatic fever in New Zealand Rheumatic fever is a preventable disease that occurs in some people following an infection of the throat with Group A Streptococcus (GAS). If left untreated the infection can lead to an autoimmune response that may permanently damage the heart (Heart Foundation 2009). Recurrent exposure to rheumatic fever can lead to the development of rheumatic heart disease (RHD), which may include valvular disease and cardiac myopathy and sequelae such as heart failure, atrial fibrillation, systemic embolism, stroke, endocarditis and the requirement of cardiac surgery (White et al 2010). Although virtually unknown in other developed countries, rheumatic fever and RHD are of particular concern in New Zealand. Both comprise a substantial and disproportionate burden for Pacific and Māori children living in areas of socioeconomic deprivation, for example Pacific and Māori children account for 95% of rheumatic fever cases (Milne et al 2010). Pacific children have the highest incidence rate of 81.2 per 100,000. Māori have a rate of 40.2 per 100,000 and the total New Zealand rate is 17.2 per 100,000 (Milne et al 2012). A literature review by the New Zealand Guidelines Group (NZGG) concluded that rates in New Zealand are comparable to developing rather than developed countries (New Zealand Guidelines Group 2011). Jaine et al s study (2008) of the incidence of rheumatic fever in New Zealand reported that the disease is intensely concentrated by age group (5 to 14-year-olds), ethnicity (Māori and Pacific people) and geographical area (upper North Island). Between 1993 and 2009, Māori and Pacific rates increased by 79% and 73% respectively, while non-māori/pacific rates declined by 71% (Milne et al 2012), contributing to widening ethnic inequalities. Children living in the most deprived areas (decile 10) in New Zealand have a 150 times greater risk than other children of being admitted to hospital for rheumatic fever by the time they are 15 years old (Milne et al 2012). Diagnosis of rheumatic fever requires fulfilment of a number of clinical criteria and is not always clear-cut. Rheumatic fever typically presents in children aged 5 14 years, while RHD often occurs as a result of repeated, unrecognised rheumatic fever infection, presenting most commonly in those aged years (Lennon et al 2006). In New Zealand, around 80% of rheumatic fever cases have some cardiac involvement. In 20% of those the carditis is moderate to severe, requiring cardiac surgery (Lennon 2007). It is estimated that 20% of those with rheumatic fever in New Zealand have residual severe RHD (Webb et al 2011). RHD remains a significant cause of premature death in New Zealand, accounting for approximately deaths per year (Craig et al 2007). Māori and Pacific RHD mortality rates are substantially higher than those of non-māori/pacific (Milne et al 2012). Periodic assessments show no decline in RHD mortality among Māori over the last 20 years (Pōmare 1980; de Boer and Pōmare 1988; Pōmare et al 1995; Robson and Harris 2007). Rheumatic fever is identified as a marker of socioeconomic inequalities and poor access to healthcare (Percival 2012). Improved living conditions and reduced transmission of the bacteria that causes the initial infection, along with improved access to antibiotics, have reduced rheumatic fever incidence in most countries (WHO 2004). 12

13 As Jackson and Lennon (2009) state: The persistence of acute rheumatic fever and RHD in New Zealand is likely to be due to a combination of factors including poor health knowledge, underrecognition and treatment of GAS pharyngitis in high risk populations, underdiagnosis of rheumatic fever in the acute phase resulting in missed opportunities for secondary prevention and ultimately a diagnosis of RHD being made, barriers to accessing primary health care, and crowded living conditions. Both rheumatic fever and RHD incur significant costs to patients, their families, communities, and to the health system. The Ministry of Health (the Ministry) has estimated that costs of RHD hospitalisations amount to $11.3 million per annum, using conservative estimates. The estimated full economic impact of rheumatic fever may total $40 million per year once individual and family productivity are taken into account (Cabinet Social Policy Committee 2010). There are a number of critical prevention stages for rheumatic fever (particularly in children 5 14 years) and RHD, as summarised by Steer et al (2006): Pre-primary (primordial) prevention involves improvements in socioeconomic conditions, including housing. Primary prevention usually involves identifying and treating a GAS throat infection with antibiotics, although some children may be asymptomatic. Appropriate antibiotic treatment of sore throats in high-risk populations will eradicate GAS in most cases, and prevent individual cases of rheumatic fever and subsequent heart valve damage and chronic RHD. Increased awareness and education about the potential impact of developing rheumatic fever and RHD is important for primary prevention. Secondary prevention usually involves the long-term administration of antibiotics to those with a diagnosis of rheumatic fever or RHD, to prevent further recurrence of rheumatic fever and progression to RHD. The duration of monthly penicillin injections for prophylaxis depends on a number of factors including age, presence and severity of carditis, risk of GAS infection, and time since last acute episode. The New Zealand Guidelines for Rheumatic Fever recommend a minimum of ten years and in some cases, continued treatment until age 30 (Heart Foundation 2009). Another aspect of secondary prevention endorsed in these guidelines involves contact tracing and treatment of GAS carriage for household contacts of rheumatic fever cases. The use of portable echocardiography to screen for RHD is also a possible form of secondary prevention. Tertiary prevention involves those already diagnosed with RHD and requiring additional medication, anticoagulation and/or surgery. Most of the children who have valve repair surgery will eventually need revisional surgery in adulthood. The disease pathway and intervention points are summarised in Figure 1. Evidence supports the importance of addressing the determinants of rheumatic fever, such as improving housing and reducing crowding, improving access to primary care, better detection and treatment of GAS, improved community awareness, training for primary care workers, and timely follow-up and delivery of secondary care (National Heart Foundation and The Cardiac Society of Australia and New Zealand 2006). 13

14 Figure 1: Rheumatic fever and rheumatic heart disease causal pathway and points of intervention Pathway Social and Economic Conditions (primordial factors) Throat infection with Group A Streptococcus Bacteria Acute rheumatic fever Recurrence of acute rheumatic fever Chronic rheumatic heart disease (RHD) rheumatic fever pathway Interventions (Relative complexity and cost of implementing interventions) Improving housing conditions (high complexity high cost) Improving Hygiene (high complexity high cost) Raising public and clinical awareness (low complexity low cost) Improving quality of sore throat management by frontline health services (low complexity low cost) Improving access to primary care (high complexity high cost) Improving surveillance and reporting of Rheumatic Fever (low complexity low cost) Improving patient follow up to ensure antibiotic cover and prevent recurrence (medium complexity low cost) National Rheumatic Fever Register (high complexity high cost) Intervention stage Primordial Prevention Primary Health Care Secondary Health Care Research to look at the effectiveness of interventions, and monitoring and surveillance Note: Tertiary prevention is not included. Source: Ministry of Health 2012a. 2.2 The Rheumatic Fever Prevention Programme In 2011 the Government earmarked $12 million towards rheumatic fever over four years as part of a Rheumatic Fever Prevention Programme (RFPP; Turia 2011). A further $12 million was announced in the May 2012 Budget as additional funding for the RFPP (Turia 2012), and the programme was extended to a total of five years. The goals of the RFPP were to: achieve equity of incidence and outcomes of rheumatic fever between Māori and Pacific children, and other New Zealand children. contribute to the reduction of the age-standardised annual rate of rheumatic fever among Māori and Pacific peoples, other New Zealanders, and all DHB populations to 0.4 per 100,000 by contribute to the reduction of rheumatic fever recurrence in New Zealand to five cases or fewer per annum by 2013 (Ministry of Health 2012b). The objectives of the RFPP were to: establish school-based sore throat services in targeted schools in high risk areas. develop appropriate clinical tools and provide relevant training to support the health sector. develop and implement a rheumatic fever awareness health promotion programme, targeting high risk communities, the health sector, and the public. 14

15 develop and implement a rheumatic fever surveillance and monitoring programme (Ministry of Health 2012a). The following specific RFPP services were planned for implementation as part of the initial $12 million Vote Health allocation to rheumatic fever in Budget : Frontline throat swabbing and referral services in eight key areas (Northland, South Auckland, Waikato, Bay of Plenty, Lakes, Tairāwhiti, Hawke s Bay, and East Porirua) A National Coordination Service (NCS) to coordinate rheumatic fever services to ensure an integrated approach to awareness and prevention A health promotion and education campaign to raise community awareness of sore throats and rheumatic fever A continuous quality improvement (CQI) system and audit tool(s) for general practices Training for health sector staff in the key areas Surveillance and research on rheumatic fever rates Governance and management of the RFPP. Detailed descriptions of early RFPP governance, service plans and budget allocation are in Appendix 1. During the early implementation stage, RFPP activities focussed on providing frontline school-based throat swabbing services, with links to primary care and community awareness raising. The RFPP was intended as a way to link service providers and creating pathways of care for Māori and Pacific populations, along with general capacity building and workforce development. In recognition of the health and disability sector s aim of improving health outcomes and reducing health inequalities for Māori, the Ministry expects that RFPP providers will ensure Māori participation in decision making around service delivery, and involvement in governance, planning and monitoring processes (Ministry of Health 2012b). Services must be responsive to Pacific health needs and are expected to align with Ala Mo ui: Pathways to Pacific Health and Wellbeing (Minister of Health and Minister of Pacific Island Affairs 2010). The Ministry established a Governance Group in 2011 to provide oversight and critical feedback around clinical advice and other expertise for the RFPP, as appropriate. A target for reduced rheumatic fever rates has been set as part of the Better Public Services action plan, aiming to reduce hospitalisation rates (initial episode) by two thirds, from 4.2 to 1.4 cases per 100,000 people by June 2017 (State Services Commission 2012). The identified lead for this target is the Chief Executive Officer of the Ministry of Social Development, supported by the Director General of Health. 1 Most of these activities were implemented as intended in the original plan. However, for a number of reasons and changing priorities, the Ministry has shifted work from some of these areas into the RFPP action plan for

16 2.3 Glossary of terms The following terms have been used in this report: BPS Better Public Services CAR community awareness raising CQI continuous quality improvement DHB District Health Board GAS Group A streptococcus GP general practitioner/doctor HPA Health Promotion Agency Kaiāwhina includes kaiāwhina, community health workers, community support workers, whānau support workers Ministry Ministry of Health NCS National Coordination Service Parents/caregivers includes parents, caregivers, whānau/family PHO Primary Health Organisation PHN Public Health Nurse Provider Budget 2011 local providers contracted to deliver throat swabbing and other services in the eight areas during the evaluation period RFPP Rheumatic Fever Prevention Programme (as specified under Budget 2011 funding and contracts with the Ministry) RHD rheumatic heart disease. 16

17 3 Evaluation approach The Ministry commissioned Litmus to undertake an implementation and formative evaluation of the first 18 months of the RFPP (1 July December 2012). 3.1 Evaluation aims and objectives The key aims of this phase of the RFPP evaluation were to: provide an evidence-based overview of how the RFPP has been implemented and its operation during the first 18 months. provide recommendations to the Ministry to inform any adjustments or improvements to the RFPP based on the evaluation results. The evaluation objectives were to: 1. Understand and refine the overall logic or theory of change underpinning the RFPP 2. Understand the intended implementation processes for the first 18 months of the RFPP 3. Determine the extent to which actual implementation of RFPP workstreams align with intended implementation, and understand reasons for any differences 4. Assess progress towards RFPP short and medium-term outcomes and identify any unintended outcomes 5. Identify key strengths and weaknesses of the first 18 months of RFPP implementation, and identify ways to enhance ongoing implementation of the RFPP. The Ministry plans to undertake subsequent evaluations as the RFPP progresses, focusing on processes, outcomes and impacts. 3.2 Evaluation methodology This evaluation was split into two phases: Phase One The purpose of this phase was to understand the RFPP background by gathering and synthesizing primary and secondary information about the RFPP rationale, intended implementation, and key informant views of early implementation. Phase One data collection included key informant interviews with Ministry, national RFPP Governance Group and national agency staff, local provider interviews conducted by phone, and a desk review of the draft programme logic, relevant literature and provider data for the RFPP. Litmus also reviewed the Ministry s indicative logic for the RFPP alongside key informant expectations of RFPP activities and outcomes and best practice evidence for rheumatic fever prevention. A revised logic model was presented to the Ministry in August 2012, with draft sub-models for each of the workstreams. The model was developed as a live tool, to 17

18 be refined and updated as the RFPP evolves. Iterations of the logic model can be found in Appendix 2. The purpose of the telephone interviews with local providers was to understand implementation progress in each of the eight RFPP sites, and to assess progress (both intended and actual) of RFPP implementation. Interview learnings informed development of evaluation tools and selection of participants and sites to be visited during Phase Two. At completion of Phase One data collection, the evaluation team 2 held an interim findings workshop with key Ministry staff and several expert advisors for the evaluation. Litmus presented early themes from Phase One data collection 3, providing an opportunity to share real-time findings, and enabling knowledge exchange for Ministry decision-making and next steps for the evaluation Phase Two The overall purpose of Phase Two was to collect primary data on local-level experiences and activities of the RFPP, centred around visits to four case sites: Whāngarei, South Auckland, Rotorua and Porirua. These sites were selected as they represent a range of contracting models, service delivery and population sizes across the wider RFPP. In each case, data were collected via interviews with local providers and stakeholders, discussion groups and a school gate intercept survey with parents/caregivers/whānau. Phase Two focused on identifying: local-level service provision and delivery models. what is and is not working, from provider and family/whānau perspectives. models of best practice in service delivery, using provider and family/whānau feedback. suggestions for future improvements to the RFPP. On completion of Phase Two fieldwork, the team held a local learnings workshop with the Ministry and evaluation advisors to share findings from the site visits and parent/caregiver survey, along with a discussion about overall implications for the RFPP. 4 A more detailed methodology for each part of the evaluation is provided in Appendix 4. Appendix 5 contains a table summarising data collection methods against the evaluation questions, which formed the basis of the integrated analysis framework. 3.3 Evaluation caveats The purpose of this evaluation report is to present an overview of the RFPP s early implementation, with key learnings and insights to further enhance the RFPP. Litmus is confident that this report accurately reflects the RFPP context and the views and perceptions of those participants who contributed to this evaluation. Particular care has been taken to critically analyse all data streams together to build a comprehensive understanding of the early implementation phase of the RFPP Appendix 3 lists the evaluation team. Rheumatic Fever Prevention Programme Implementation and Formative Evaluation. Initial learnings workshop with the Ministry of Health, 14 September Internal presentation to the Ministry, Litmus Limited 2012b. Rheumatic Fever Prevention Programme Implementation and Formative Evaluation. Local level learnings workshop with the Ministry of Health, 30 November Internal presentation to the Ministry, Litmus Limited 2012c. 18

19 This evaluation covers the early set-up and implementation of the RFPP from 1 July 2011 to 31 December As this is a formative and implementation evaluation, it is acknowledged that the plans and activities of the Ministry, RFPP providers and other agencies may have evolved during the evaluation period. In considering the findings of this evaluation, a number of limitations are acknowledged. Stakeholders and parents/caregivers from four out of eight RFPP areas were involved in the evaluation case visits. Their experiences may not necessarily reflect those of other sites involved with the RFPP. The case site visits and parent/caregiver discussion groups were conducted between October and November 2012, and findings reflect activities up that time. The qualitative methodology of interviews with key stakeholders, providers and parent/caregivers provides in-depth understanding of their experiences with the RFPP to date. The consistency of themes across participants and analysis of background documentation and progress reporting validates the findings presented. The school intercept survey collected information from parents/caregivers who were present at the school gate before and after school, and does not represent the views and experience of all parent/caregivers in the areas surveyed. Sub-group data should be interpreted with caution due to the small sample numbers involved. Analysis of early monitoring reports from local providers did not allow for identification of short-term outcomes or accurate cross-site comparison due to issues with data quality and consistency. 19

20 4 RFPP background and context This section provides context for the introduction of the RFPP arising from Budget Information was sourced from a combination of key informant interviews, provider telephone interviews, and the RFPP document review. Where relevant, verbatim quotes from interviews illustrate key points and reflect the views and experience of the majority of key informants, local providers and other stakeholders interviewed. 4.1 Leading up to Budget 2011 The announcement of Budget 2011 funding of $12 million followed work already underway within the Ministry to support DHBs, the health sector and communities to reduce the impact of rheumatic fever. The need for a cross-government approach and national coordination was recognised, as existing local initiatives were unlikely to impact on the national burden of rheumatic fever (Cabinet Social Policy Committee 2010). The intention of the interagency work programme was to work with social policy and housing agencies to address underlying causes of rheumatic fever, and to support health improvements made through targeted health interventions (Cabinet Social Policy Committee 2010, p1). The Ministry linked with Education, Social Development, Pacific Island Affairs and Te Puni Kōkiri, along with Housing New Zealand, the Energy Efficiency Conservation Authority and PHARMAC (Cabinet Social Policy Committee 2011). Interagency discussions focussed on prevention and early intervention on the disease pathway, including healthy housing (such as insulation and heating), school hygiene (hand washing and drying facilities, school awareness), and increasing referral to and engagement with primary health care. General awareness raising, community engagement, and training for primary health care professionals were also considered. 4.2 Budget 2011 investment in rheumatic fever The focus of the Budget 2011 (Vote Health) $12 million investment in rheumatic fever was on supporting community and frontline response. This was seen to complement interagency work on social determinants. Funding from the Budget 2011 allocation were weighted towards programme resource development and surveillance, audits and research in the first year, to provide a stronger evidence-base for effective targeting and implementation of interventions (Ministry of Health 2011b, p6). The DHBs that were most vulnerable to rheumatic fever were identified: Northland, Auckland, Counties Manukau, Waikato, Bay of Plenty, Hawke s Bay, and Capital and Coast. Budget documentation estimated that 22,000 children (at a cost of $135 per child per year to implement) would be eligible to receive frontline throat swabbing services (Cabinet Social Policy Committee 2011). Workstream implementation plans, funding decisions and contracting arrangements were developed from July 2011 in the identified RFPP sites. The Whānau Ora approach delivered by Māori and Pacific providers was seen as a way of delivering targeted services to high risk communities (Cabinet Social Policy Committee 2011, p7). To further inform decision making about the Budget 2011 funding allocation, Ministry officials were asked by the Minister of Health to seek feedback from key stakeholders (in 20

21 April 2011) on the components of the rheumatic fever bid. Stakeholders included the National Heart Foundation, the Māori Medical Practitioners Association, the Pasifika Medical Association, the Royal New Zealand College of General Practitioners, PHARMAC, the New Zealand Medical Association, General Practice New Zealand, and the National Hauora Coalition. The following issues were raised by stakeholders during the Ministry s process: Local-level rheumatic fever services need to be integrated with other child health initiatives such as immunisation and dental coordination Local ownership of services and registers are vital to the success of the initiative School-based services should be considered a short-term auxiliary service for very high-risk areas, with a parallel focus on strengthening general practice response Early engagement with Primary Health Organisations (PHOs) and a focus on clinical aspects are required for improving access to, and quality of, general practice care Facilitated meetings among general practice in high risk areas could be used to give collective strength to clinical practice, using peer learning and clinical audit tools Community awareness activities need to prioritise Māori and Pacific people, as should the service interventions National health promotion resources should be produced with the option for local providers to adapt to suit community needs Surveillance and monitoring is important and needs to be action-oriented Connectivity between the Ministry s proposed initiatives and local services is critical (Ministry of Health 2011c). 4.3 Allocating Budget 2011 funding across DHBs RFPP funding was allocated to localities within eight identified DHB areas based on rheumatic fever incidence in each area (using data), the population aged 5 14, and the distribution of cases (clustered or geographically dispersed; Ministry of Health 2012d). A targeted approach was thought to be most appropriate for rheumatic fever, given the geographic and demographic clustering of disease, and the high disease rates among Māori and Pacific children (Jaine et al 2008). Initial cluster analysis identified the target DHBs: Northland, Counties Manukau, Hawke s Bay, and Capital and Coast. Further analysis by the Ministry (based on primary care and school populations ) identified localised clusters in three other DHBs: Waikato, Bay of Plenty, and Lakes. Tairāwhiti DHB was added as the eighth area following consultation with the sector (Ministry of Health 2012d, p1). Areas were not targeted on the basis of GAS prevalence because there is no current measurement of national GAS incidence, and due to gaps in understanding of the epidemiology of GAS infection. A number of contracting approaches are used for the RFPP. Initially, the Ministry considered contracting directly with DHBs, PHOs and/or other agencies, or targeting funds to DHBs in high risk areas (alongside clear monitoring requirements). However, the final contracting agreements were made directly with local providers, and DHBs were not automatically involved in RFPP funding. The Ministry intended that Budget 2011 funding be used as seed money, to which DHBs or other local organisations could contribute further funding for related activities in 21

22 rheumatic fever prevention (Ministry of Health 2012d). In some cases, DHBs have topped up RFPP funds to increase the number of schools or days per week for provision of throat swabbing services. Key informants and stakeholders raised three main issues with funding allocation: 1. GAS incidence data may be a more reliable indicator of areas of need and targeting for the RFPP than rheumatic fever incidence. Variation in relatively low numbers of rheumatic fever makes the disease difficult to monitor and attribute any changes to a specific programme. However, there is currently no national monitoring or baseline measure of GAS incidence. There s also research that needs to be done into things like what s the prevalence of Group A strep in these communities?. At the moment we re targeting on the basis of cases of rheumatic fever. But cases of rheumatic fever are actually pretty rare about 180 a year. Whereas carriage of Group A strep in your throat is common and it s quite likely that Group A strep carriage, although we haven t done the research and we need to, is a sort of marker for the likelihood of children getting rheumatic fever. (Key informant) 2. Inadequate involvement of local providers and stakeholders in active decision-making. While the Ministry consulted with local providers, some felt that local knowledge could have been more actively sought in decision-making about schools to receive RFFP funding and relative distribution of funds to schools. The initial decision of which schools receive funding wasn t equitable some schools with the highest rates missed out. I questioned the allocation at the time; it seemed to be based on out-of-date figures, and this idea that we ll do eight [areas], with no rationale for why that many or consideration for the other high-risk ones. (Local stakeholder) 3. Need for greater involvement of DHBs, at least at the service planning level, and a perception that the RFPP would be more effective with greater communication between the contracted provider and the DHB. Rheumatic fever is a priority in many DHB health plans (particularly for Māori health). However, DHBs noted the challenge of being accountable for BPS target when they have no control over the RFPP service provision. The Ministry has subsequently worked with DHBs to provide planning guidance and support on the new rheumatic fever targets (Ministry of Health 2012f). New data were taken into account when allocating an additional $12 million in funding from Budget 2012, enabling some providers to increase the number of schools receiving throat swabbing services. 5 5 Note that services funded through Budget 2012 are outside of scope for this evaluation. 22

23 4.4 RFPP and the Better Public Services target After the RFPP started in 2011, new rheumatic fever targets were announced as part of the Better Public Services (BPS) Action Plan to support vulnerable children, involving the Ministries of Social Development, Health, Education and Justice, and the Department of Building and Housing 6. The Ministry acknowledges that the BPS target for reducing rheumatic fever hospitalisations will not be met through throat swabbing services activities alone. The broader cross-ministry and interagency rheumatic fever programme is anticipated to support work towards the target (Ministry of Health 2012d). In summary, the Ministry s Rheumatic Fever Work Programme from 2011 included the following key elements: 1. Provision of additional support services (clinical facilitator, kaiāwhina, and sore throat services) in very high-risk communities 2. Development and delivery of clinical support tools, and communications and health promotion to communities at increased risk of rheumatic fever 3. Rheumatic fever system monitoring and surveillance 4. Interagency work to support the health sector in reducing the incidence of rheumatic fever (Ministry of Health 2012a). From September 2012, the RFPP was integrated into the broader government work programme for addressing the high rates of rheumatic fever in New Zealand. 6 Accessed 3 December

24 5 Strategic issues with national implementation This section presents national-level RFPP implementation issues, highlighting key informant and provider feedback about implementation timing, the potential for the RFPP to reduce rheumatic fever, and the sustainability of prevention activities. 5.1 Implementation timelines In the early planning stages, the Governance Group advised that the RFPP roll-out should not be rushed at the expense of a quality implementation process one that was well informed by evidence (Ministry of Health 2011e). The schedule for implementing specific RFPP workstreams and activities varied between sites. Most of the local-level throat swabbing services were under contract with the Ministry by the beginning of the first school term in Providers in the Waikato DHB area, Kaitaia and Hawke s Bay signed their contracts later than others. Feedback from local providers highlighted that the implementation of the RFPP was rushed, which had implications for their ability to set up and deliver throat swabbing services, community awareness raising, and appropriate monitoring and evaluation systems Local throat swabbing services workstream Local-level providers and DHB stakeholders note that the rushed schedule for the roll-out of local throat swabbing services raised concerns about quality, relationship building with schools and DHBs, and insufficient time to prepare frontline staff to deliver services in a systematic and efficient way. Several DHBs felt they were not adequately consulted by the Ministry during both the scoping and contracting stages, and that time was not taken to build relationships across DHBs and other agencies. From an operational point of view, trying to get the schools up and going in the timeframe provided - it s quite unrealistic I think for a start-up programme and potentially will jeopardise the quality of what s delivered. (Local provider) There has been a lot of urgency in setting up some of these throat swabbing programmes because of other imperatives than just the clinical need. They have done as well as they could, given that they were set up rapidly. (Local stakeholder) In some cases, DHBs also commented that they do not necessarily support local providers contracting directly to the Ministry for throat swabbing services, as providers may not align their work with the regional DHB child health strategy. Relationships with DHBs have not been as solid as they could be it was rolled out in a short timeframe and a lot of the groundwork wasn t done. (Key informant) 24

25 5.1.2 Community and health awareness raising workstream The community and primary health sector awareness workstreams for the RFPP have experienced delays. The HPA was funded by the Ministry to develop resources for the RFPP. The HPA s service specification states that the awareness programme should be appropriate for Māori and Pacific communities, nationally consistent, and appropriate to the local context (Ministry of Health 2012e). Development delays were partly due to timing of the HPA consultation process (including a stocktake and needs assessment; see Section A1.3.5), which could not begin until local provider contracts had been signed. In late 2012, the HPA was asked by the Ministry to produce a printed resource based on existing rheumatic fever material, rather than delay production of a new resource any further. A pamphlet and poster were distributed to providers in November However, these resources arrived at least six months after most providers had started their throat swabbing service. These delays meant local service providers had to identify and source alternative rheumatic fever resources to give to parents/caregivers, and many chose to put their own limited funding into developing an interim resource for their community. Locally-produced resources did not always use consistent messaging and approaches to encourage target audiences to get sore throats checked. While consistency of national messaging appears to have been addressed with the production of a national pamphlet and poster (each containing the 0800 Healthline number and a national website address), it is unclear to what extent the HPA has considered resources appropriate to the local context. As of late 2012, only English-language resources had been developed Monitoring, evaluation, surveillance and research activities Many of the RFPP s planned monitoring, evaluation, surveillance and research activities were also delayed 7. It appears that project management resource was diverted to other emerging health priorities within the Ministry s Communicable Diseases Team, including work on measles and influenza outbreaks. RFPP providers are required to report on throat swab volumes and GAS positive results across a number of variables such as school, ethnicity, and age of child. However, it appears little direction has been provided on the collection of other measures that may be important for monitoring RFPP outcomes (for example, antibiotic compliance and follow-up, and referrals to primary care). The Ministry continues to work with providers to streamline RFPP monitoring and reporting requirements. Many stakeholders called for a nationally coordinated list of research priorities and a monitoring and evaluation framework to inform robust measures of effectiveness of frontline service delivery and tracking of RFPP progress across the eight sites. 7 See Appendix 1, Section A1.3.6 for further detail. 25

26 5.2 Efficacy in reducing rheumatic fever Key informants interviewed often cautioned that the mix of RFPP activities would not adequately address the causes of rheumatic fever. In particular, there was concern about the lack of attention given to primordial determinants of health, and prevention of illness. There needs to be a focus on primordial prevention, especially overcrowding, addressing poverty, easy access to primary care, that is, free access these will make a significant difference to the incidence they are needed for the BPS target. (Key informant) The primordial stuff is absolutely key to the long-term prevention of rheumatic fever. Because we ve only got four years of money and what happens then? If we haven t reduced the risk factors that are leading to rheumatic fever we will run out of money in four years time, we won t have reduced rheumatic fever and we won t have addressed the primordial factors. So it ll have been a waste. (Key informant) Questions were also raised by the Governance Group and stakeholders about the relative cost effectiveness of the school-based swabbing approach (Ministry of Health 2011e). In their opinion, a school-based approach is appropriate during disease outbreaks, provided there is a parallel increase in primary care capacity to swab and treat sore throats. The school-based setting was seen as most appropriate for Māori and Pacific families, who may currently experience barriers in accessing primary care. The cost benefit is that you could probably prevent about a maximum of a third of rheumatic fever by throat swabbing everybody. And we re not swabbing everybody, we re probably throat swabbing about a third of the children who are at risk across the country, maybe a half. So the maximum reduction we could get in rheumatic fever is about fifteen percent, using the throat swabbing technique. Now, we re actually being asked to get a 150 percent reduction so we have to reduce it down to 1.4 from 4. (Key informant) Rheumatic fever is driven by more things than just picking up kids with sore throats. There are significant issues around housing, poverty, access in general to medical services that are culturally competent, that is really important. If you have free access to medical care but not culturally competent then that is essentially still a barrier to good effective health care and health outcomes. (Local stakeholder) The BPS target for rheumatic fever has added momentum to RFPP activities. However, there was also concern that the target may raise unrealistic expectations about the RFPP s ability to reduce rheumatic fever as a standalone programme. Many stakeholders felt that the cross-agency work programme needed to gain traction with prevention and wider health determinants of rheumatic fever before it was reasonable to expect a reduction of two-thirds of pre-target incidence. Swabbing in schools is one approach but I am not convinced it is the be-all and end-all. Other innovations should be looked at. What was the overseas experience where the rates went to zero? The difference there is that it was an all of government approach. (Key informant) Clinical leadership for the RFPP was at times fragmented, and senior-level accountability for the RFPP was unclear in the early stages of implementation. The appointment of a Chief 26

27 Advisor, Population Health (in May 2012) with a specific focus on rheumatic fever was seen as a positive step Sustainability of continued focus on rheumatic fever prevention There is concern nationally (and to some extent locally) about the sustainability of results in reducing rheumatic fever, and the sustainability of school or community-based throat swabbing, beyond the five years of allocated funding. The question is not just what rate [of rheumatic fever] we are going to achieve, but also how are we going to sustain this rate? Is it feasible to sustain this rate? It s all very well to achieve a rate reduction but for what purpose if it s going to jump back up again? (Key informant) There are two main areas of concern in relation to RFPP sustainability: 1. The effort providers have put into the school throat swabbing services has not been matched with a parallel effort in addressing rheumatic fever awareness and engagement, particularly in primary health care. Primary care will be a vital component of reducing rheumatic fever beyond the life of the RFPP, and this groundwork is critical before school/community swabbing activity can be withdrawn. School-based services are useful for outbreak control, but in the long term we need to be strengthening primary health care and access to primary care we need to be doing both in this programme outbreak control and establishing structures for ongoing rheumatic fever control. (Key informant) The Governance Group was particularly clear about the need for long-term strengthening of the primary health sector s response to rheumatic fever (Ministry of Health 2011f) to help prevent possible estrangement from the RFPP. The Governance Group also identified that relationships with DHBs were critical to long-term development of the RFPP. 2. Local-level community awareness raising thus far has placed more emphasis on encouraging participation in the school throat swabbing service, rather than taking child(ren) to the doctor if they have a sore throat. Community awareness raising for long-term, sustainable behaviours should include messages to seek treatment for sore throats, particularly in settings or situations where school throat swabbing is not available or offered. The HPA recognises that parents/caregivers need to be given messages that prepare them for when the swabbing service ends, and strategies to prevent reinfection. Local providers interviewed in the case sites were gravely concerned about the RFPP setting up expectations among parents/caregivers that their children will be swabbed and treated in the school setting. In some cases, local provider staff and school contacts were under the impression that school-based throat swabbing was now a permanent service being provided in their community. Providers need ample notice of any intent to withdraw school swabbing or other RFPP services to manage expectations accordingly. 8 The Chief Advisor, Population Health has also been appointed the Clinical Lead for rheumatic fever. In December 2012 a permanent rheumatic fever project team was established under the leadership of the Chief Advisor, and separate from the Ministry s Communicable Diseases Team. 27

28 You set up an expectation [with parents] that this happens [throat swabbing]. If anything like this is going to work it s got to be sustained doing it for a year is just rubbish. The damage it does to people s expectations is huge (Local stakeholder) The DHBs and the Governance Group identified the need for an exit strategy should the RFPP s funding be removed before 2015, or be withdrawn for one or more of the contracted providers. Current RFPP contracts expire in June 2013 for those providers contracted from the Budget 2011 allocation. DHBs and providers request six months notice before the end of their contract term to ensure adequate handover or future planning of alternative service delivery, including transfer to primary care. I think the fact that it s non-sustainable funding that makes it challenging. The fact that it s four years of funding and [area] has undergone this huge change in the way services are delivered and potentially in four years time we re going to have to stop that, is, would be an incredible shame and people don t seem to be talking about that. (Local provider) At the end of four years, what will we have achieved? If we stop investing in schools, then what? Have we strengthened primary care enough? (Key informant) Many key informants agree that the systems, approaches and awareness raising efforts that are established as part of the RFPP need to be embedded into high-risk communities to ensure ongoing sustainability. It was clear that relationships with DHBs will be critical for sustainability. However, provider and DHB working relationships will need to be strengthened (and in some cases, repaired since breakdowns in the set-up phase). 28

29 6 Local services workstream This section presents implementation and formative evaluation findings for the local services workstream for RFPP, in particular the school throat swabbing services which have been allocated a large proportion of Budget 2011 funding. 6.1 Contract requirements and contracting models Most RFPP providers who deliver throat swabbing services in schools are contracted directly to the Ministry and have near-identical service specification requirements. The overall objective of the service is to reduce the rate of rheumatic fever by: providing throat swabbing and referral services in the school, home or other settings as appropriate for school children aged 5 14 years who present with sore throats in highrisk areas, and eligible whānau/family members living with these children. increasing awareness of rheumatic fever risk factors among children and their whānau/families in the key geographical areas. developing and maintaining relationships with other health and social service providers (including Whānau Ora providers) to facilitate referral and support, as appropriate (Ministry of Health 2012b). Specific service requirements linked to these objectives are: Throat swabbing and referral services Increase and improve community awareness of rheumatic fever Network development and service coordination Reporting and monitoring. A number of detailed activities, activity measures and outcome indicators are itemised for each component. Throat swabbing services are to be delivered in accordance with three key principles: 1. Children aged 5 14 years (with sore throats) in high risk areas are able to access throat swabbing and referral services where parental or guardian consent has been given. 2. Household contacts (aged under three years) of eligible children are able to receive throat swabbing and referral services in accordance with eligibility criteria. 3. All children and/or their family/whānau will be provided with information about the importance of treating sore throats, and rheumatic fever prevention (Ministry of Health 2012b). 29

30 Different contracting models have been used with service providers in the eight RFPP areas. In five areas, Māori/iwi providers have been directly contracted by the Ministry (ie, Kaitaia, Whāngarei, Rotorua, Bay of Plenty, and Tairāwhiti). Most of these providers also deliver wider clinical and community-based health services (eg, Korowai Aroha Trust in Rotorua and Turanganui a Kiwa in Gisborne). In two RFPP areas, the Ministry has contracted with the DHB (ie, Hawke s Bay and Capital & Coast). In South Auckland, the contract was awarded to a Primary Health Organisation (PHO), the National Hauora Coalition. In several cases (usually with the larger providers), the contracted provider has sub-contracted one or more components of the service to another local provider. Examples include: Capital & Coast DHB contracts with Regional Public Health for Public Health Nurses (PHN) to undertake the school swabbing, and with Compass PHO to undertake home visits for antibiotic delivery and follow up. Hawke s Bay DHB contracts to Te Taiwhenau o Heretaunga Trust to do swabbing in three schools, and contracts internally to PHNs to provide swabbing services in the other three schools, using the same delivery model. In Tairāwhiti, Te Hauora o Turanganui a Kiwa holds the contract with the Ministry and delivers a swabbing service in the west of the Tairāwhiti district. They sub-contract to Tairāwhiti DHB (to cover Gisborne city) and Ngāti Porou Hauora (to cover the East Coast), each using a different swabbing approach (some school-based, some opportunistic/community-based). In South Auckland, National Hauora Coalition contracts with five providers to deliver swabbing services as part of a wider child health service. In addition to throat swabbing, the service includes a focus on skin infections and other public health issues usually attended to by a PHN. At the time of the visit, Kidz First (part of Counties Manukau DHB) delivered throat swabbing services in six schools, East Tamaki Health Care (a PHO) delivered in seven schools, and three small providers swabbed in one school each (Te Hononga o Tāmaki Me Hoturoa, Health Star Pacific Trust, and Otara Union Health Centre). A summary of contracting models used in each of the eight areas is presented in Table 2. 30

31 Table 2: Contracting model summary for Budget 2011 RFPP providers Area Contract holder Catchment Model Contract period Kaitaia Te Runanga o Te 6 schools School-based Sep 2011 Rarawa 1097 children June 2013 Whāngarei Kia Ora Ngatiwai 6 schools School-based Jan children June 2013 South Auckland Bay of Plenty National Hauora Coalition Tuhoe Matauranga Trust 16 schools N/A 7 schools 637 children Rotorua Te Korowai Aroha 6 schools 1500 children Tairāwhiti Turanganui a Kiwa 20 schools 2269 children Hawke s Bay HBDHB 8 schools 1786 children East Porirua CCDHB 9 schools 1695 children Note: Correct at the time of the evaluation case visits. N/A not available; unable to confirm. ^ Formerly described as opportunistic swabbing. School-based Feb 2012 June 2013 School-based Feb 2012 June 2013 Communitybased^ Communitybased^ Feb 2012 June 2013 Feb 2012 June 2013 School-based Nov 2011 June 2013 School-based Feb 2012 June 2013 Visited for evaluation The services provided in each of the RFPP areas cover some but not all primary and intermediate schools. Consequently, current services do not provide complete coverage of potentially at-risk children in the eight RFPP sites. 6.2 Delivery models The RFPP case sites were selected to cover a range of different service delivery models and stages of implementation for throat swabbing services. Across the wider RFPP, some areas had been operating their throat swabbing service for 12 months (such as Kaitaia and Hawke s Bay), while others had started their throat swabbing service delivery as recently as May June 2012 (for example, South Auckland and parts of Tairāwhiti). Table 3 describes key aspects of the different delivery models used in the four case sites, which are discussed in more detail below Informed consent The Ministry developed a short, two-page parent/caregiver consent form for the RFPP, which is used by some providers. RFPP case sites use two approaches for seeking consent for children to have their throats swabbed: opt in, where parent/caregivers give active and explicit consent for their child to have their throat swabbed as part of the RFPP opt out, where schools inform parents/caregivers that blanket consent is given by the principal for health and wellbeing activities (including the throat swabbing service). The consent covers all children attending the school unless a parent notifies the school otherwise (usually through the use of school notices). 31

32 Whāngarei, Porirua and South Auckland providers use an opt in consenting approach. Some achieve high levels of consent (eg, up to 90% in some areas), while a small number of schools have had a low response (eg, below 70% consent). Innovative incentives for encouraging the return of consent forms have been used by providers one school in Whāngarei provided ice blocks to the first classroom with all consent forms returned (regardless of whether they were consenting or not). Sustaining high levels of informed consent using an opt-in approach needs continual awareness raising activities to ensure rheumatic fever and the throat swabbing service are kept top of mind for parents, and that they fully understand what the service and RFPP means for their child and family/whānau. At the early implementation stage, it is not known what proportion of parents/caregivers do not consent to their child taking part in throat swabbing for specific reasons ( active non-consent), compared to those who do not receive or return the form to the school ( passive non-consent). In cases where response to the consent process has been low to date, providers have invested extra resource during down time from swabbing to follow up on non-returned consent forms (eg, in school holidays, contacting whānau through home visits and/or phone calls to seek consent). Some providers remarked on the difficulties in maintaining high consent throughout the school year, due to high student turnover (up to 60% per year in some schools). Providers in South Auckland and Whāngarei now include consent forms and swabbing service information in new student enrolment packs. [Consent forms] have gone out to every prospective enrolment and every new entrant enrolment. Whenever we have an enrolment enquiry we re giving the information sheet and consent forms out as part of our enrolment package. (Local stakeholder) In Rotorua, an opt out consent approach is used for the school throat swabbing service. Schools in the area report no issues with this approach for throat swabbing and are actively supportive of it. It was reported that a small number of parents/caregivers complete and return the opt out form, thinking they are giving consent for their child s participation in throat swabbing. In these cases, the Rotorua provider makes contact to confirm their choice, and in almost all cases, the parent intended for their child to take part. An opt out consent approach is acceptable to parent/caregivers, schools and providers, and has potential to increase participation in school throat swabbing services in a safe and effective way. It was not clear whether the NCS has given guidance to providers in deciding which consent approach to use for their local throat swabbing service. This could be considered as part of the NCS role for supporting RFPP implementation. As implementation of the service continues across all RFPP sites, it will be helpful to track and explore the reasons some parents/caregivers do not give consent for their child to participate in the swabbing service. 32

33 Table 3: Delivery models employed by the four visited RFPP sites Area Whāngarei South Auckland Rotorua Contract holder Kia Ora Ngatiwai National Hauora Coalition^ Te Korowai Aroha Sub-contracted provider(s) Consent process Student ethnicity* - Opt in Māori (47 100% of school rolls) East Tamaki Health Care KidzFirst Pacific Star Otara Union Health Te Hononga o Tāmaki Me Hoturoa Opt in Pacific (59 95% of school rolls) Māori (5 100% of school rolls) - Opt out Māori (63 100% of school rolls) Days/week swabbing 3 days/week Only positive notified 5 days/week Positive and negative notified As needed Results Meds / notification # Home visits Only positive notified Parents/caregiv ers collect meds from school (deliver if necessary) Home delivery of meds Home delivery of meds Existing rheumatic fever programme - - East Porirua CCDHB Compass PHO RPH Opt in Pacific (40 80% of school rolls) 3 days/week Only positive notified Home delivery of meds Māori (16 99% of school rolls) Notes: Current at the time of the case site visits, and as reported by providers. * - Ethnicity of children at schools participating in the throat swabbing service. # - Results notification sometimes differs within sites. ^ - National Hauora Coalition sub-contracted East Tamaki Health Care to swab in seven schools, Kidz First to swab in six schools, and Pacific Star, Otara Health Union, and Te Hononga o Tāmaki Me Hoturoa to swab in one school each. 33

34 6.2.2 Throat swabbing workforce The frontline workforce delivering school throat swabbing services differs between RFPP areas. Many of the sites visited (Rotorua, Porirua and some in South Auckland) use trained nursing staff such as PHNs or other registered nurses to perform throat swabs. Some sub-contracted providers in South Auckland use trained non-nursing staff to take swabs, with a range of position titles (community support workers, whānau support workers, collectively known as kaiāwhina). The whānau support worker visits schools alongside a PHN to deliver throat swabbing and skin infection services. In Whāngarei, nursing and non-clinical staff are trained to do throat swabbing. The non-clinical project manager helps with swabbing paperwork while on-site at the schools, and is available should the nurse be on leave. Across the different models of service delivery, providers have experienced capacity challenges in delivering throat swabbing services, particularly during seasonal times of high swabbing volumes, such as after winter holidays. As a result, a back-log in swab cases develops when frontline staff are unable to swab all symptomatic students in a single school visit, and have to defer the remainder until the next time they are visiting the school. Some providers felt frontline staff may not have the capacity to deliver services through another winter if the swabbing levels are as high as 2012, particularly as staff regularly fall ill due to high exposure to GAS and other infections. One provider indicated that they were two-thirds of the way through their budget and only halfway through their contract term, due to the higher than predicted demand for swabs. Furthermore, the focus on getting through throat swab volumes means limited opportunity to deliver other components of the operation such as follow-up home visits to assess antibiotic compliance. We are struggling to do the essence of our work. The more efficient you become, the more work we get given. Health promotion used to be a big part of our role but now we are too busy (Local provider) The programme in our region has been hugely onerous on nurse time, and we are trying to find ways to reduce that level of intensity as it has taken them away from their core work. (Local provider) The use of a combination of nursing and non-clinical staff to take throat swabs is acceptable and appropriate for the RFPP, provided suitable training is received before they start, and on an ongoing basis. Different approaches were observed in at least two sites for the collection of swab specimens (eg, in and out swabbing versus side-across-side swabbing). The issue of poor quality swabs was also raised by some of the laboratory staff visited. The NCS has a role in coordinating training for RFPP throat swabbing services, and ensuring quality standards are maintained for RFPP throat swabbing over time. It was not clear whether consistent training and quality assurance processes were in place at a national level, or whether there was a plan for this in later stages of the RFPP. Locally, most RFPP providers have clinical input and leadership in place for their swabbing work. Many providers and local stakeholders emphasised that frontline and clinical staff need to have greater involvement in planning and decision-making for the RFPP, along with management staff from contracted provider organisations. In addition, working with DHBs and existing systems was seen to streamline many administrative processes for throat swabbing service delivery. 34

35 Having a shared oversight group is critical, however we have to make sure there is enough clinical representation. Time after time they set up a group of management from PHOs, who don t understand the reality of frontline work, and what is feasible and most appropriate. (Local stakeholder) When the DHBs are involved, it s been quicker to implement This is a clinical programme. Smaller providers can t access the NHI for example. So DHBs can facilitate that access. (Key informant) Swabbing frequency and approach Three of the four RFPP case site providers have been contracted by the Ministry to provide a school-based throat swabbing service for at least three days a week. Providers visit RFPP schools to swab the throats of children who self-identify as having a sore throat, and for whom consent has been obtained. A range of frequencies and approaches for delivering school-based throat swabbing is used across the RFPP sites visited. The sites with slightly different approaches include: South Auckland, where each sub-contracted RFPP provider delivers swabbing services five days a week, based on advice from the provider s child health and clinical advisors. In addition, this approach was used for an earlier throat swabbing pilot in Wiri Central Primary School (2011), which informed RFPP delivery in the area. Rotorua, where the RFPP provider delivers community-based swabbing (formerly described by the Ministry as opportunistic ), with six designated schools contacting the provider on days when one or more children at school self-identify as having a sore throat. Rotorua is the only case site using this approach, due to fewer cluster areas of rheumatic fever in this area. The Rotorua provider was asked by the Ministry to focus on awareness raising activities rather than offering throat swabbing services in other local schools. Their service has a 0.8 full-time equivalent nurse, and is uncertain whether opportunistic swabbing in the six RFPP schools along with the requested CAR focus in other schools will be sustainable. At present, their service specifications are identical to those of providers delivering a school-based swabbing service, and need to be updated to reflect the actual services provided in the area. Kia Ora Ngatiwai in Whāngarei undertook baseline swabbing before beginning their three-day-a-week swabbing service. Their aim was to measure incidence and provide a baseline against which future swabbing and GAS results can be measured. A secondary outcome was that all of the provider s staff participated in the baseline swabbing exercise, thereby enabling them to engage with the school community and promoting the objectives of the RFPP. Across all case sites, specific challenges with the in-school swabbing location and organising children were observed. Two main approaches were noted: 1. All children with a sore throat were taken to a designated room for swabbing. As a result, children were waiting up to two hours at times, and missing out on classroom time. Challenges with disruption, negative interactions between the children, and fearful reactions to being swabbed (after seeing other children having a swab taken). General crowd control issues were noted, all of which the kaiāwhina and nurse were trying to manage in between delivering the swabbing service. 35

36 2. Children went individually or in pairs to the sick bay or an office to have their throat swabbed before returning to class, a more streamlined process and less fraught for the children. Techniques used by child health services for immunisation may be useful to frontline RFPP providers, such as using a screen to separate those children having a swab taken from those waiting, to reduce anxiety. It did not appear that RFPP providers had received general advice or tools from the NCS to assist in choosing the best swabbing approach for their area. The NCS has a potential role in advising on best practice swabbing processes, which could be informed by standards used for providing immunisation in a school setting Provision of antibiotics In three of the four case sites, antibiotics for GAS positive infections found through the RFPP are provided using a standing order 9 (Whāngarei, South Auckland and Porirua). The main reason for standing orders is to improve patients timely access to medicines (in this case, antibiotics such as amoxicillin and erythromycin), to be delivered to the child s home or collected by parents/caregivers without the need for an additional doctor s visit or prescription. Standing orders are not used in the Rotorua throat swabbing service, Instead, in cases of a GAS positive result, the nurse makes contact with the child s GP to arrange a prescription, collects the script (on the same day or the following day), fills the script and pays the fee at the nearest pharmacy, contacts the child s parents/caregivers to arrange a suitable time, and delivers the antibiotics to the child s home. The provider notes that there can be a delay of up to three days before the antibiotics are able to be delivered to the child s family/whānau. The Māori nurse does the home visits and any follow-up household swabs, and is well known in the local community, which is reported to help in gaining the trust of whānau. There are two main models for providing the antibiotics to children: Children who have a positive GAS test result receive their antibiotics through a home visit from the provider in three of the four case sites (South Auckland, Rotorua and Porirua). We have had really positive feedback on how people have been able to have the antibiotics delivered to their home. It s free of charge. (Local stakeholder) In Porirua, a PHN delivers the antibiotics to the child s parent/caregivers, and a followup home visit is made by a kaiāwhina who is ethnically matched to the family/whānau and is local to the community. It is thought that this gives easier access into the home, enables the kaiāwhina to check in with the family/whānau and give wider support where needed. The provider is mindful of some families/whānau having issues with trust and allowing a home visit from someone perceived as part of the government. Whāngarei uses a unique model of antibiotics provision, where parents/caregivers are contacted and asked to visit the school to collect antibiotics for children with a GAS positive result. This strategy seeks to encourage whānau to engage with the throat swabbing service, and to take responsibility for the health of their tamariki. It also 9 A standing order is a written instruction issued by a medical practitioner that authorises a specified person or class of people (eg, registered nurses) who does not have prescribing rights to administer and/or supply specified medicines and some controlled drugs. A standing order does not allow a person to generate a prescription and provide it to a patient to take to a pharmacy to be dispensed (with the prescription signed later by the issuer of the standing order) (Ministry of Health 2012g). 36

37 gives schools an opportunity to connect with parents/caregivers of their students, and it appears to be an effective strategy only 10 families have been unable to collect their antibiotics (usually due to transport issues) in the last six months. In these cases the nurse has taken the medication to the home. In another example of local provider innovation, incentives are used to encourage completion of the full course of antibiotics (also referred to as compliance ). In Porirua for example, a branded Strong hearts in Porirua sticker chart is provided to mark off each day the antibiotics are taken, and the child is given a thermal t-shirt following completion of the full ten days Links or referrals to other services Across the case sites, wide variation was seen in the extent to which RFPP providers consider or address other health and social issues as a component of delivering throat swabbing services. Service objectives are clear in articulating the Ministry s desire for providers to refer children and family/whānau to other services as needed, and to develop and maintain relationships with health and social service providers in their community. However, the relative level of resource providers are expected to invest in offering and/or arranging referral services is not clearly stated in the service contract. Porirua and South Auckland providers had implemented a process to address wider household issues identified through the RFPP, or were formalising their referral pathway at the time of the visit. Both of these sites were able to draw on experience and wider sector support from existing rheumatic fever prevention initiatives (ie, Porirua Kids Project, and Wiri School pilot in South Auckland). In Porirua, the RFPP nurse/kaiāwhina asks about wider health or social needs in a household when delivering antibiotics or following up on antibiotic compliance. This strengths-based approach where the whānau identifies their own needs is thought to create an empowering experience for the family/whānau. Information is then used by the provider to prepare referral letters and/or arrange other service provision with agencies such as WINZ and Housing New Zealand. Referral agencies spoken to were positive about the RFPP overall, and some had started to actively support the work of local RFPP providers. We are setting up for the project group to come and talk to my tenancy managers who are in and out of our homes regularly, so they understand what strep throat is, what the causes are, and what rheumatic fever is and what damage it can do and how the strep throat and rheumatic fever relate, and what we can do as HNZC to support and help this project succeed and our tenants keep safe and well. (Local stakeholder) In South Auckland, funding for the RFPP throat swabbing service contributes to a broader PHN role in selected schools. This additional service is resourced by Counties Manukau DHB, with a whānau support worker and a registered nurse visiting schools together. The whānau support worker manages the throat swabbing process, while the nurse has oversight of skin infections and other public health work. The same people visit the same schools each week, building an ongoing relationship with staff and tamariki. In this case, the provider referred to their service contract as requiring home assessment and home safety checks. However, this part of the service was still being developed for implementation. In Whāngarei and Rotorua, there were no formalised structures or referral protocols in place for the RFPP to link to other services, possibly due to resourcing issues. In Rotorua, if other issues are identified for the child and whānau, referrals are made in an ad hoc way. Similarly in Whāngarei, while there appear to be no documented protocols around referrals 37

38 and other support, provider nurses took the initiative to provide follow-up suggestions for issues that they had detected while swabbing at school (eg, a case of severe head lice being referred to the school office). In 2011, the Governance Group suggested that identification and treatment of skin infections should be built in alongside RFPP throat swabbing services, and that DHBs could contribute the marginal additional costs for this (Ministry of Health 2011f). Feedback from many stakeholders highlighted the potential for in-school services to address more than one issue, with proper resource and guidelines. The rheumatic fever programme could be the thin edge of the wedge for dealing with other issues, like skin infection. (Key informant) An ethical obligation exists to ensure referral processes are in place in the event nurses and kaiāwhina encounter social or other health issues when working with children or their parents/caregivers as part of the throat swabbing programme (eg, injuries and possible abuse, mental health, nutrition). Currently, frontline staff do their best to handle wider health and social issues within the time and resource available to them. Across the RFPP, formalised pathways and referral processes are required to ensure these ethical obligations and duties of care are met. There s no point asking someone about family violence or poor housing or behaviour problems in kids if you don t then have a pathway. I feel really passionate about that in terms of screening pathways. So we didn t ask any questions on the housing questionnaire unless you could then suggest what would happen. So it wasn t just about going in and collecting information, Oh, you live in a crowded damp mouldy house, that s interesting - that will be why you re sick. It was really about, These are the services that are potentially available, and can we link with MSD and get a housing supplement or can we refer you to an insulation programme That s the vision. (Local provider) Provider reporting requirements for RFPP During the initial phases of their contracts, providers were finding it difficult to prepare monitoring reports. Many described high transaction costs in setting up reporting structures, with some providers needing additional information technology support. The early reporting template provided by the Ministry was described as difficult and time consuming to use. 10 In early stages of RFPP implementation, providers were required to report high level monitoring data monthly, more detailed monitoring data quarterly, and narrative and financial reporting six monthly. I think there are potentially some really useful synergies with the Ministry because they ve obviously got pressures in terms of reporting and I think it would be really useful if what we reported to them was meaningful from an evaluation point of view. And it means that we don t need to do things twice. So I think getting a reporting framework and the evaluation aligned in some way might actually be quite helpful. (Local provider) The forms that they have just sent out, it s just all about swabbing, they are just interested in capturing swabbing, they don t want any narrative, they just want numbers, and it doesn t capture anything else, it s just a numbers game. (Local provider) 10 In September 2012, the Ministry indicated that the reporting template was being modified as part of an iterative process. 38

39 6.3 Implementation strengths A number of early implementation strengths were identified: High levels of provider dedication to the RFPP Providers are very much focused on getting on with the job of throat swabbing and have quickly mobilised in many of the areas (some under pressured circumstances). Providers are pleased to have received direct government funding for rheumatic fever as they recognise the impact of the disease in their communities and are keen to proactively address the issue. There is a local-level tenacity in the way they approach things. They are all very passionate about what they are doing. It s been great to witness all of that. (Key informant) Creating and building strong community links with families/whānau, schools and local provider networks All providers, especially those in smaller centres like Rotorua and Whāngarei, are building on their strong community links nursing and kaiāwhina staff are often long-term residents and are known to many of the schools and whānau. Providers see schools as the best venue for delivering swabbing services. The home visits are particularly helpful for tamariki Māori, Pacific children and those who may have issues in accessing primary care. The nurses are amazing they have earned the right to care for our community, and parents see them in the same way. (Local stakeholder) Providers are proactive in using local networks and resources to support the work of the RFPP. For example, Te Korowai Aroha in Rotorua has formed a strong relationship with staff from the Eastern Bay of Plenty (EBOP) involved in delivering DHB-funded throat swabbing services. EBOP recognised that Te Korowai Aroha was operating in isolation during the early months of their contract, and have been active in offering advice on protocols and resources for throat swabbing services. A number of positive relationships have been developed in many of the RFPP sites, involving goodwill on the part of others involved in the throat swabbing service pathway. In Whāngarei for example, the DHB, pharmacy and laboratory work closely together to ensure a streamlined process for the testing of swabs, the delivery of test results and access to antibiotics. In Porirua, one GP in the area (with back up support) works closely with the PHNs to supervise standing orders. Everyone is sharing what they ve got until such time as there is specific training that the programmes will have to go through. (Key informant) Strong support for RFPP from schools The throat swabbing service has been well received by schools and communities involved in the RFPP. Teachers, principals and Board of Trustees members spoke highly of the RFPP. Schools are enthusiastic about the RFPP, and teachers and administrative staff give support to frontline staff to facilitate the service in their school. Equally, general practices support the work of RFPP providers and recognise the benefits of taking throat swabbing services to children in schools. We will do anything we can to support families, and health needs are so important All school staff [and BOT] were committed to start with, and it s running so efficiently that we are all supportive. (Local stakeholder) 39

40 It s one of those programmes that s come in as a package, everything s been there, there s been minimal additional workload put on to the school, the nurses have just done all of this themselves and it s been great. (Local stakeholder) Anecdotal evidence of high detection of GAS positive infections Providers were confident that their work to date had identified a sizeable proportion of GAS positive cases in the community. One example provided by Rotorua was that of nine swabs taken on one day, six were identified as GAS positive. Reports from providers were anecdotal however, and would need to be verified through the Ministry s ongoing monitoring data Future considerations and improvements Several areas for future consideration and improvement to the RFPP were identified which could be implemented either with newly contracted providers, or introduced with existing providers at contract renewal. In considering these points, the key goal of the RFFP needs to be top-of-mind: to achieve equity of incidence and outcomes of rheumatic fever using a targeted intervention approach for Māori and Pacific children, and other New Zealand children living in high-risk areas (Ministry of Health 2012a). Strategic leadership from the Ministry around evidence-informed models of interaction with Māori and Pacific families/whānau will be important when defining more specific service level agreements with providers (rather than the current generic agreement model) Effectiveness for tamariki Māori and their whānau Local Māori nurses and kaiāwhina are delivering frontline RFPP services in sites such as Rotorua and Whāngarei. Their presence in the community is well-regarded and accepted. In other areas, varied levels of culturally appropriate swabbing service delivery were observed. Porirua has a kaiāwhina making the home visit for antibiotics compliance, while in South Auckland, the majority of swabbing services and follow-up home visits are conducted by mainstream providers (Kidz First and East Tamaki Health Care). The Porirua approach is consistent with evidence from other reviews and evaluations, 12 where having trained Māori health workers leading the home visits may result in better outcomes for tamariki and whānau. Across all sites, it was noted that having familiar and trusted nursing staff going into schools is most important for service acceptability to all stakeholders (including the children themselves) Effectiveness for Pacific children and their fanau The South Auckland and Porirua sites have significant Pacific populations relative to other RFPP areas. However, it was noted that Pacific providers had limited (if any) involvement in service planning or delivery for the RFPP. One Pacific provider is sub-contracted by the Note that at the time of the evaluation reliable Programme monitoring data was not available. Refer Appendix 4, Section A4.3 for further explanation. For example, the Well Child Tamariki Ora Quality Review undertaken by Litmus for the Ministry of Health 2012; and the Barriers to Consumer Utilisation of Primary Maternity Services in Counties Manukau DHB undertaken by Litmus for the Ministry of Health

41 National Hauora Coalition to do RFPP throat swabbing in a single South Auckland school, while two Pacific providers have been contracted for the RFPP in the Waikato area. Targeted planning and implementation for Pacific should be an explicit part of the RFPP, and contract service specifications need to reflect the burden of disease in relation to the local demographic profile. Current contracts make no mention of Pacific models of delivery. Further, it should not be assumed that strategies designed for Māori communities will be suitable or transferable for Pacific communities. The Ministry s future contracting strategies need to explicitly focus on strategies and service delivery for Pacific children and their fanau Capacity to deliver across all RFPP workstreams Providers across all case sites note the limited capacity to deliver throat swabbing services, as well as challenges in making progress in community awareness raising and engagement with primary care. Consequently, there are risks that the quality of the swabbing service may become compromised, and providers may become burnt out, impacting on RFPP sustainability. A formal referral pathway for the throat swabbing service is needed to reflect RFPP service requirements. Currently, if social or other health issues are identified by nurses or kaiāwhina while undertaking throat swabbing, few systems are in place to support possible referral to other services. Direction from the Ministry is required in this area, particularly given the focus on referrals in the contracted service agreements National RFPP standards and guidelines for quality A national-level quality framework and guidelines for RFPP providers needs to be developed to ensure good practice and high quality consent processes, standing orders and swab collection across all RFPP sites. Some providers (eg, Porirua) have developed their own extensive swabbing service protocols. The Ministry could consider prioritising three areas: A national guideline for RFPP standing orders A national standard for the use of opt off throat swabbing consent, rather than opt on consent Quality assurance guidelines for throat swabbing services, particularly for specimen taking and ensuring consistent quality within and across RFPP sites over time. Regular observation of swabbing across the different areas and settings is needed to ensure consistency in practice, and across the wider RFPP. The Ministry may wish to extend the NCS role to include ongoing quality assurance as part of the RFPP. Currently, the NCS is required to promote health worker training and attendance at training opportunities. This role could also provide support to providers facing challenges, offering best practice guidance, training, advice, monitoring and shared learnings, and overseeing the use of consistent quality improvement processes For the programme to work, everything needs to be done extremely well, with high quality processes we have to make sure each step in the RFPP is done 13 In September 2012, the Ministry advised that they were in discussion with Hawke s Bay DHB (who have a strong referral programme in place) to prepare a model of referral approaches, which other providers could use as a guide. 41

42 well, for example, the competency of the nurses, the quality of the throat swabbing, the quality of the lab testing process, and so on. (Key informant) Any national frameworks and guidelines need to retain local flexibility. For example, some providers commented on their limited ability to change the RFPP approach or coverage under their current Ministry contracts (eg, Whāngarei providers would prefer to be swabbing at two different schools where a higher proportion of students are exposed to rheumatic fever risk factors, rather than the current RFPP where GAS positive rates are much lower than anticipated) Exit strategies Concerns were raised by providers and stakeholders about how delivery of the RFPP through schools has raised parent expectations about ongoing delivery in the school setting. There were also strong views that raising parent and school expectations of longterm throat swabbing services is unfair, particularly when funding has not been guaranteed and there is no indication of contracts being renewed beyond June The implications of this timing are now reaching a critical point. Providers request a six-month lead-in time should contracts not be renewed, so they can develop an appropriate exit strategy and ensure plans are in place to either withdraw the service or examine alternative service provision options. This would include strengthening linkages with primary care to ensure ongoing involvement and commitment to rheumatic fever prevention Overview of good practice and challenges in implementation Figure 2 illustrates good practice approaches and challenges identified for the RFPP, aligning with key points along the swabbing pathway. While providers have adopted similar approaches to processing and obtaining throat swabs in general, at the local level they have put in place a range of strategies to respond to frontline challenges and/or to streamline the delivery process in the schools and local area. 42

43 Figure 2: Overview of good practice examples and challenges from early implementation of the RFPP EXAMPLES of GOOD PRACTICE Opt out consent process Incentives (iceblocks) for first class with consents returned BASELINE All children at school swabbed at set times in term/year SELF-IDENTIFIED Teacher asks and identifies children at roll call Swabbing staff obtain list from school office, go to class to call out only those children identified by teacher Children taken out for swab in smaller numbers, individually or in pairs Second staff member talks with kids and reinforces sore throat message Techniques for effective swab taking eg. asking children to 'sing' while swab is being taken Fast turnaround Communicating with parents and whānau Family/whānau offered swab, wider support Reminders / incentives (sticker chart, thermal t-shirt on completing course) Nurse requires child to bring completed sticker chart to school Whānau required to collect antibiotics from school (and engage with service) Consent process Identifying children with sore throats Organising children Taking throat swab Results notification Antibiotics CHALLENGES Opt in consent torms not getting home Low consent rates for opt in Potential disruption to classrooms and learning to identify children with sore throat Disruptive behaviour Logistics of organising large number of children (often in small spaces) Distress at seeing other children being swabbed Inconsistent and poor quality swabbing technique Letter not getting home Parents not being notified/given reassurance of negative results No process to ensure child receives full course of antibiotics Low access for RFPP provider delivery of meds, information for parents and whānau Out of class for long period of time 43

44 7 Parent/caregiver experiences This section presents findings from parent, caregiver and family/whānau discussion groups in each of the four case sites. It focuses on parent/caregiver awareness and knowledge of sore throats and rheumatic fever, and their views on the school throat swabbing service. The second part of this section describes several parent/caregiver typologies identified, and the life situation and message resonance for parents/caregivers across each typology group. While many themes relating to experience with the throat swabbing service were consistent across ethnic groups, views and experiences specific to Māori, Pacific and/or Pākehā parents/caregivers have been noted. 7.1 Family/whānau and household context The parents/caregivers interviewed are committed to fostering the health and well-being of their family/whānau. Parents/caregivers want to be proactive in supporting their children, in particular, and in preventing illness and disease. However, in seeking to achieve this goal, these parents/caregivers face environmental, social and economic barriers. Across all four sites and ethnic groups, there were several common themes relating to household well-being, and being equipped to prevent illness and protect the health of children and the wider family/whānau. These largely relate to housing (and specifically concerns of cold and damp conditions), crowding and sharing limited space with others, and eating well. All of these factors are noted by parents/caregivers as ways to prevent infections in general, and sore throats specifically. Damp and cold housing is an issue which constantly undermines parents/caregivers ability to create an environment of wellness for their family/whānau. Parents/caregivers describe significant costs for heating during winter, and access barriers for those in Housing NZ properties. Those parents/caregivers who have been able to access support to insulate their homes notice an improvement to their family s health. Our house is really cold. Our kids can eat healthy as, but it s still a really cold house. I should really pay for it but I can t afford it. Insulation even would be good. (Māori parent) We had our house insulated and we put the heat pump in and our health improved extremely, so it s definitely to do with housing, all of the kids were sick every year, then when we got the heat pump that helped a bit but then we put in all the insulation and now we hardly go to the doctors (Māori parent) I asked for my house to be insulated, but they haven t got to that part of Porirua. Both my kids were admitted to hospital for bronchitis and pneumonia. It s swampy out the back; my kids can t even play outside. (Māori parent) We have a fireplace but we don t use that because we have to pay for wood and coal and whatever, and that costs more money. We just put on more clothes and run around the house. (Pacific parent) 44

45 7.1.1 Child health, illness and relative importance of sore throats Child health and safety are very important to parents/caregivers, and the needs of their child(ren) are put first before themselves or those of other household members. Parents main priorities for their children are that they are clean and warm, and able to be active and happy. In spite of very challenging circumstances for many of the parents, all are doing their best to look after the health of their tamariki, and to provide stability, consistency and set an example. Parents/caregivers and wider family/whānau felt that other health issues such as meningitis and skin infection are promoted or talked about more widely in their community than rheumatic fever. For most parents, sore throats are not a priority compared to the other health needs of children and family/whānau - they are bottom of the list. They were accustomed to bugs going around and childhood illnesses such as earaches, chicken pox and tonsillitis. Parents/caregivers commented on having greater concern for urgent health issues, such as things they can see (skin infections, sores), and issues that affect the daily life of their children (asthma) Usual response to a child s sore throat Keep them warm, monitor at home For most parents, their usual response to a child s sore throat is to keep an eye on their child, make sure they are warm enough, give them fluids and/or Pamol, and wait and see whether the sore throat gets worse or resolves on its own. In most cases they feel fine with their child going to school with just a sore throat, but if it is very painful or lasted longer than a day or so, parents/caregivers would consider keeping their child at home to monitor them. I do the wait a couple of days to see what happens instead of going to the GP for a waste of money - $20 then I have to go back again, then I have to find a babysitter. I don t drive and I have to walk in. So you just wait it out at home and give them drinks and a lemonade ice block. (Māori parent) Home-made/traditional remedies Māori and Pacific parents/caregivers made frequent reference to treating a child s sore throat first with home remedies (eg, lemon and honey ) or traditional remedies. I would rather use rongoā [traditional Māori medicine] kumara hou; just drink that. Boil the leaves and save the juice. There needs to be more education about alternative medicines. (Māori parent) I think when our children get sick, especially around sore throats; we turn to our natural herb medicine like the lemon drink. (Pacific parent) 14 This feedback from parents is consistent with recent research examining paediatric admissions in Wellington, where fewer parents said they would always or often take their child to the doctor for a sore throat (48%) compared to other health issues such as a skin infection (89%), vomiting or diarrhoea (78%) or for a cough and noisy/wheezy breathing (80%) (Abdulhamid et al 2012). 45

46 Visit the doctor Seeking help from a doctor is not usually parents first course of action when a child had a sore throat, unless it gets worse, lasts a long time, or is associated with a temperature or another symptom. 15 I wouldn t usually take my kid to the doctor cause they had a sore throat unless they were really uncomfortable, like sore ears as well. With the throat you sort of disregard it a little bit: Here s a throat lolly, you ll be alright... (Pākehā parent) The main barriers to parents/caregivers taking their child to the doctor are difficulties in getting an appointment (some parents/caregivers have had to wait several days for an appointment), long waiting times at clinics, and the costs associated with doctors visits, including the costs of after hours services. Parents/caregivers who take their children to free clinics often have to wait long periods (sometimes more than half a day) for their children to be seen by a doctor. Māori and Pacific whānau who often had two or three other young children with them found these barriers particularly challenging Awareness of link between sore/strep throat and rheumatic fever Sore throats are not generally seen by parents/caregivers as a serious or urgent health issue for children. 16 In addition, awareness of the link between untreated GAS throat infection and rheumatic fever was very low. In cases where parents/caregivers are aware of the link, they are not clear about the exact disease pathology, and the single bacterial strain of (untreated) infection that is linked to rheumatic fever. My brother s ex-partner ignored her son s sore throat in the end my parents took him out of school and to the doctor without her knowing and he had rheumatic fever and it has damaged his heart. (Pākehā parent) Know a friend of mine, her partner now had rheumatic fever when he was 15 and he has a pacemaker and where he was from, they didn t have the right medical facilities so they flew him to NZ. He s been here ever since. It s very serious; you can t reverse damage to your heart. (Māori parent) Parents/caregivers are often confused about what rheumatic fever is, describing whooping cough, TB, heart murmurs or a perception that rheumatic fever was congenital, and something that a person is born with. Some parents/caregivers have misconceptions about the cause of rheumatic fever, thinking that it only affects boys, that it was brought on by exertion (such as kapa haka or being run down), or from sharing bath water. My son got it through kapa haka; they pushed him too hard and he lost his voice a few times and I wonder if the strain on his voice caused it. I don t know, I think it was one of the contributions and he was run down. (Māori parent) I ve been told that with males it can stop them from having children it can affect their reproductive ability. (Māori parent) Intercept survey finding: Almost all parents/caregivers (96%) said that they would take their child to a doctor or nurse if they had a sore throat. NB: Some discrepancy was noted between survey results and discussion group findings. The survey results are more likely to reflect parent/caregiver intent, whereas discussion findings focus on the reality of what they do in practice. It may also be due to survey question wording, which asked what would you do if your child had a sore throat, rather than what do you do.. In the intercept survey, just over one in ten parents/caregivers (12%) mentioned sore throats, strep throat, rheumatic fever or heart disease as one of the most important health issues affecting the health of children (unprompted).the other health issues that were more frequently mentioned by parents/caregivers included nutrition/eating, colds/flu/coughs, skin problems/eczema, asthma, and keeping homes warm/healthy home environment. When prompted, almost three-quarters of parents/caregivers in the intercept survey (74%) said they had seen, read or heard something in the last six months about sore throats or strep throat. 46

47 Pacific parents/caregivers generally have limited knowledge and awareness of the connection between a sore throat, GAS infection, and rheumatic fever unless they have personal experience with the disease or if they have a child who has experienced a positive GAS infection or rheumatic fever. [Sore throats are important] because of rheumatic fever and if it s not being checked then the child will suffer for the rest of their life. With my son, I had no idea. He had a sore throat, I thought it was just the flu until his ankle started to get swollen and I took him to the hospital just in time so he s on penicillin for quite a while. So that s why this is important to me, because of my grandchildren and my ten-year-old. My child is 27 now; he was diagnosed with rheumatic fever when he was eight or nine. (Pacific parent) Others who know about the link between sore throats, GAS infection and rheumatic fever are likely to have a professional connection to health services, such as a mother who is a trained nurse, and described the link in simple terms: Once they ve had the bug and it goes into the valve it affects how the heart works. (Māori parent) 7.2 Parent/caregiver experience with school throat swabbing service Parents/caregivers were asked what they knew about the school throat swabbing service, where they had heard about it, and their experience with delivery of the service (including what happens when a swab result is positive). This provides insight into how early stages of RFPP implementation is working for parents/caregivers (and their children), what aspects of the throat swabbing service are most helpful, and where improvements might be made Knowledge of the school throat swabbing service Overall, parental awareness of the school throat swabbing service is variable. Many parents/caregivers feel that there is a lack of information about rheumatic fever and the school throat swabbing service available. It s only cause my daughter, at [school] said they d have to sign a consent form. But I don t know much about rheumatic or the conditions of it until it happens to you. There s not much information out there. (Pacific parent) Those parents/caregivers who know about the school throat swabbing service often found out about it through the school newsletter and when a consent form had come home with their child(ren). The process of signing for consent is seen as easy and straightforward. Although all parents/caregivers had consented to their child participating in the service, some (particularly Māori and Pacific) have questions about the service and exactly what it means for their child. They gave me this, it s an A4, it s got all the information on there. But there are some questions that I don t really understand. (Pacific parent) In some cases, parents/caregivers signed the consent form without reading it in detail, noting that they sign forms left, right and centre. In these cases they only became aware of the consent purpose when their child had a positive result for GAS infection. 47

48 I didn t know they were doing this Then I got a call from [nurse] and she said he had put his hand up and she swabbed him and he had strep throat. (Māori parent) Some parents/caregivers knew of cases where teachers or nurses had followed up with other parents/caregivers to obtain consent forms that were not returned. In areas where the RFPP uses an opt off process, this is not always clear to parents. The letter that got sent out it was a little confusing, it said to sign it as an opt off. Because you are used to getting something to say yes for! So families were signing to say yes please! It was confusing. Some people signed them off and then they moaned as their child wasn t getting swabbed. (Māori parent) Parents/caregivers had found out about the school throat swabbing service through community launch events (in South Auckland, for example), and these events often prompt conversations about sore throats and rheumatic fever within whānau and wider social and community networks. [Community launch] was the first time I had ever heard of rheumatic fever. It was really good. And then I find out down the line that one of my best friend s sons had rheumatic fever and I ve seen her for years but we ve never talked about it. (Pacific parent) Child s experience with throat swabbing Parents/caregivers were asked about their child s experience of throat swabbing at school, and what (if anything) they have said about the swab test itself. 17 They report a range of experiences, with some children being completely comfortable with the swab test (even a bit excited about the swab, thinking it s cool cause their mates are doing it ). My daughter comes home, and goes Mum, guess what, a doctor came and saw me today and I said Was it a lady or a man? and she says A lady and I go Ok, what happened? and she goes She shoved a stick down my throat and I went 'ehhh ' and she went All done. (Pacific parent) My son understands that if he s got a sore throat, to say something otherwise he told me, otherwise you ll end up with heart problems. (Pacific parent) However, some parents/caregivers report that their child found the experience invasive and uncomfortable, due to the physical discomfort and anxiety. Others feel embarrassed or whakamā about being called out of class, or about having the swab test taken. Some parents/caregivers note that their older children can be reluctant to self-identify as having a sore throat. 18 He hates them, but it doesn t matter, we push him, it s for his health I have to push him to go and see the nurse he just doesn t like the throat swabs. Had my son not had the swab done it could have gotten worse, truly grateful that I was there to push him to go and get a swab. (Māori parent) They hate it, cause it s like Open up! and something goes down your throat. And I ve seen it too, she s like gagging and she s in tears and then we find out it s Intercept survey finding: Of those parent/caregivers who were aware of school throat swabbing, almost six out of ten (59%) said that child(ren) from their family/whānau had had their throat swabbed as part of the school service. A similar issue was raised by key informants who noted that older children (particularly in intermediate school) are less likely than younger children (in primary school) to self-identify as having a sore throat. 48

49 not done properly and it s got to be done again. Is there any other way it could be done? (Pacific parent) My little one, sometimes she won t let anyone to do it. So they have to ring me to go in just to get her to sit still and to open her mouth. (Pacific parent) Parents experience of the school throat swabbing service Notification of swab results Parents/caregivers do not have the same experience in finding out the results of their child s throat swab as part of the RFPP. This reflects the variation in processes for the swabbing service between schools, providers and RFPP sites. 19 In many cases, the results notification process for a negative swab was not clearly explained to parents, and several parents/caregivers have followed up on the result. We ve had the swab but nothing was told so we don t know whether it was confirmed or whether they were positive or not. But now we know, only because we came to this group. That if it was they would have come around. So that sort of puts us at ease. But I think if we were told either way it would put us at ease a lot earlier. (Māori parent) I went in and asked the teacher When are we supposed to get a result? and they said if you don t hear anything it s negative (Pākehā parent) Some received a phone call that their child was all clear, others a letter, and some didn t hear anything afterwards (so assumed it was negative, or forgot about it, or never bothered to ask ). Those who did receive notification of a negative swab result appreciated it, and felt a sense of relief and reassurance that their child did not have a throat infection. I was like What was the result? so she [daughter] gave me the bit of paper and she goes She said you re all good. (Pacific parent) Experience of home visits Those parents/caregivers whose child had a GAS positive result and received medication directly from the nurse/kaiāwhina are appreciative of the extra support provided to diagnose and treat their child. These parents/caregivers placed great importance on the rapport that they had with the nurse/kaiāwhina, and a relationship based on trust and support. The nurse came to see me at my house, my son had it [swab] done that day and then the results came back. As soon as she got them she picked up some medicine and came to see me at my house and I didn t have to pay for it. I thought that was a bonus, cause I asked her How much do I owe you? and she goes No, and I was like Cool do you want a coffee? (Māori parent) Within the afternoon she was at home with the antibiotics. She stayed with me for an hour and a half to explain it all I felt so guilty about it all. I didn t know he was sick. (Māori parent) Many parents/caregivers were not aware the RFPP process when/if a child s swab came back as GAS positive. Many Māori and Pacific parents/caregivers thought that a GAS 19 Intercept survey finding: Among those parent/caregivers whose children had a throat swab taken at school, over a quarter (28%) needed antibiotics afterwards. This survey question provides a proxy measure of GAS positivity test rates - when extrapolated out to the full sample, this measures at 10% reflecting the rate seen from provider and laboratory reports (between 7-9% of swabs GAS positive). 49

50 positive result would mean they had to go to (and pay for) the doctor for treatment, and are concerned about what the swabbing service requires of them. Once they heard about the RFPP process of providing free antibiotics and home visits they were very positive about the service s approach. 20 I think if the information went out there was well about the nurses taking the positives, that would also minimise some of the stress these families feel about, you know, going to the doctors and having to fork out for that (Pacific parent) Compliance with antibiotics Some parents/caregivers are proactive about ensuring their child completes the prescribed course of antibiotics, or make a conscientious effort to do so. Parents/caregivers find the tools offered by local providers to remind and encourage completion helpful (eg, sticker charts to show progress, a thermal t-shirt when they had finished the full course of antibiotics). Others used their own methods to remind themselves and their child to take the medication. I always finish the course (Māori parent) It was a matter of that consistency for my daughter to keep taking it. Cause she thinks once a sore throat is gone, it s over and it s only been three days of the medicine. But I m like no, doctor said you have to finish it and they re like No I m not going to shove that in my mouth and I m like No. It s like a child-mother argument. (Pacific parent) For others, there are a number of challenges in ensuring that the full course of antibiotics is completed including: remembering to administer to the child, not seeing it as a priority once children are feeling well, and preferring traditional remedies over Western medicine. Sometimes I feel sorry for them for making them take the antibiotics so I stop after three or four days (Māori parent) To be honest, I probably do it until, yeah, they re well. Maybe a week and a bit. There s still always half a bottle left. I know I should be doing it, giving it to them (Pacific parent) All parents/caregivers who had the nurse deliver their child s antibiotics were given verbal information about the importance of completing the full ten-day course. Often the nurse stayed to chat about other things happening with the family/whānau, and gave more comprehensive information than what is provided in pamphlets or the school letter. This information was given in a way that enabled parents/caregivers to ask questions and feel more confident in ways they could proactively try and prevent the spread of sore throats in their household. A small number of parents/caregivers did not know what was meant by antibiotics during the discussion, and needed some prompting to find out whether their child had been prescribed (and completed) a course. Is that the white medicine? (Māori parent) 20 Towards the end of the discussion group, the evaluators described the local Programme delivery and the process for a positive GAS result (including home visits and antibiotics delivery, where relevant). 50

51 7.3 Parent perspectives on school-based delivery Parents/caregivers from all ethnic groups are overwhelmingly positive about throat swabbing being delivered in the school setting. For parents, the main strengths of schoolbased delivery include: having no associated costs timely delivery of swab result provision of antibiotics for child (and whānau, if necessary) at no cost convenience and not requiring a doctor s visit, especially for parents/caregivers in employment. Had throat swabbing three times, two positives; got given antibiotics and picked up from school; somebody called me back to see how she was going after the ten days. Good feedback, they kept ringing to make sure she was taking medicine. Got given a green pamphlet; I signed the consent form when I enrolled her. School makes it easier on parents especially if you ve got little ones, if you make an appointment and you can t make it you get charged $10 - makes you think about next appointment. (Māori parent) Reasons parents/caregivers support a school-based service The reasons contributing to widespread parental support and appreciation for the service overall are described below. Parents/caregivers thought the school swabbing approach would likely have a higher success rate than leaving it up to people to just take them to the doctor. If it wasn t for the programme they wouldn t be picking up her bacterial throat as fast as I would be they re finding the problems faster than I m finding them. The programme s fantastic. (Māori parent) No need to wait or pay for doctor s visit - School-based throat swabbing services saved parents/caregivers time and money in not having to take their child to the doctor, including taking time off work and transport costs. The swabbing service was particularly helpful for parents/caregivers of children over six are not covered for free primary health care, and visits to after-hours clinics that required payment for services. Doctors are expensive so you re not going to rush and take them there for everything, just for the serious things. Also takes a while to see doctors and its $25 when they hit six years. The schools are great and they bring the medicine to you as well. (Māori parent) At the end of the day they re providing a service, especially for some of my mates who don t have vehicles and they have heaps of kids and if they have to go to the doctors it s a mission. You re either taking one sick one and leaving four behind and you ve got to get a babysitter, or traipsing them all through town. (Māori parent) That s the positive part: that it s based in school. All the kids know the doctors and they re like No, I m not going to the doctors cause they ve got to sit there and wait for a long time and they re like look - there s people coughing, I m going to get germs. (Pacific parent) 51

52 Not missing school - The physical location of throat swabbing services on the school grounds meant child(ren) did not need to be taken out of school. I think the easiest part is knowing they re going to get pulled out of class and taken straight there. They aren t going to miss any school to go to doctors. (Pacific parent) School is the children s place, with a natural focus on well-being of students - This gave a feeling of the school as the heart of the community for their children, and a communal place. At the school it s more friendly and it s their place. It s the kids place not so much of a professional place. And plus a lot of our kids are more comfortable being swabbed at the school They re not alone as well, they ve got their friends with them. (Pacific parent) Children trust and are comfortable with nurse - For all parents, it is important that their children feel comfortable during their throat swab, and the consistency and individual personality traits of the nurses/kaiāwhina are the main drivers of acceptability. The children know and trust frontline staff, which is fundamental to parental buy-in. Parents/caregivers also feel their children are less willing to visit and be treated by providers they see less frequently, such as doctors. [Nurse] is wonderful with the kids; she makes them feel comfortable. They are more than happy. If you could clone [nurse] that would be great she genuinely cares (Māori parent) I think at the school the kids know the nurse. They trust the nurse and they feel comfortable with that. But the doctor, they hardly see the doctor (Pacific parent) Minimises reinfection by diagnosing and treating all children - Parents/caregivers are often frustrated that they can deal with contagious illness within the whānau, but face reinfection when their child returns to school. They feel that throat swabbing all children at the school reduced the risk of exposure of their child to throat infections. Our daughter has had strep throat twice this year. Worse thing is you get your kid sorted but the parents of other kids haven t got their kids sorted. Stop the spread of it, if they catch it early enough it won t spread through the classroom. (Māori parent) Reassurance and know that others are getting it done as well; there s no point in getting your child done when there are carriers in that community that have been done. (Pākehā parent) Reasons some may not support a school-based service A small number of parents/caregivers had initial doubts about whether frontline staff taking swabs are sufficiently qualified, and have a preference for their doctor to swab their child. They had at times chosen to seek a second opinion on test results and treatment, rather than relying on the school service and nurse/kaiāwhina s diagnosis and prescription for antibiotics. 52

53 7.4 Parent/caregiver experience of primary care Parents /caregivers previous experience of primary care and taking their children to the doctor for a sore throat often influenced how they felt about the school throat swabbing service. Some parents/caregivers had no issues with using primary care services when their child had a sore throat. The doctor told me I needed to get the whole whānau swabbed because of close contact. Swabbed the other kids and put them on a course of antibiotics and then if the test came back negative they said you can stop the course of antibiotics. [Daughter] was the only positive, the others were negative. I d rather do it this way, start the course of antibiotics and if they re ok they ll ring up and say they re negative but [daughter] is positive and needs to continue the ten-day course. Everyone came back clear; the adults were swabbed as well. (Māori parent) However, a small group of Māori and Pacific parents/caregivers had less positive experiences of primary care (including after-hours clinics), and appreciated the way the school throat swabbing service was empowering, and more responsive to their needs than primary care had been in the past. We ve already had near deaths and stuff like that so yeah, we need to be more aware. If we go to the doctor for a cough with no appointment you are not going to get seen. I struggle to get into the doctors [name of clinic] so if we ve got every parent with a child with a cough how are they going to deal with every patient. And then that s you [parents] trying and you feel let down so then what s the point. I ring up and nothing 'til next week. (Māori parent) The test takes too long at the doctors and sometimes they don t call you to tell you they ve got the results. But the team at the school, they do the test and then in the next couple of days you ve got the results. And you re like Wow, that was fast. (Pacific parent) Māori and Pacific parents/caregivers in particular commented on feeling disempowered when it came to dealing with health services and are put off by the way that doctors interacted with them in general. Many had experiences of a doctor minimising their child s sore throat and sending them home without a full examination or treatment. I didn t know about the age group it affects. When I got the phone call about the oldest daughter, I didn t know that it doesn t affect under five-year-olds so I was going to take them in to the doctor and they made me feel a bit stupid. The doctor told me they aren t susceptible to rheumatic fever under five. (Māori parent) When mine got a positive result they called and said they d have some antibiotics for her at school. They called me on Thursday and said the antibiotics wouldn t be ready until Monday so I took all my kids to the doctor and got the antibiotics prescribed through the doctor. But he looked in her throat and said he wouldn t bother testing because it doesn t look red I thought, How could you say that? Just give me the antibiotics that the others have prescribed. (Māori parent) I believe the doctors don t really check the sore throats; they just look and say Oh, we re going to give you a Panadol. They ask you Oh, what s wrong with your child? but they don t even check; they just look at your child, no examining and stuff. (Pacific parent) A small number of parents/caregivers had chosen not to attend targeted services (ie, community clinics/iwi/māori health providers) after experiencing long waiting times. 53

54 I was with [Māori provider] and every time I rang I couldn t get an appointment for weeks and was told to go to a walk-in, had to bus all the way in and then couldn t see them. Adds to the stress of it all. (Māori parent) 7.5 Methods of raising awareness with parents/caregivers This section focuses on how information about the RFPP and rheumatic fever has been delivered to parents/caregivers to date, and what styles and media are perceived to be most useful. The themes relating to CAR messages, strategies and communication styles described in this section were raised by both Māori, Pacific and Pākehā parents/caregivers, unless noted as being specifically relevant for a particular group Parent/caregiver views on awareness raising activities to date Parents/caregivers reported that they had been exposed to information about rheumatic fever in a number of ways, from schools, local-level promotion and wider media coverage of the illness. Information received directly from schools In many cases, parents/caregivers feel burdened by info overload with the volume of forms sent out by the school, many of which never make it home, or are found in a child s school bag a week later. In most cases, while the process to sign for consent was straightforward, the supplementary written information or letters provided by schools are not engaging or simple enough for parents/caregivers to read thoroughly. This led to many parents/caregivers not fully understanding what they had agreed to, nor the detail about sore throats, rheumatic fever and/or school throat swabbing. When all the info came home I brushed over it. I go for colours. I read the top part, the bold, the bottom part, then sign. (Māori parent) Parents/caregivers indicated that they regularly read the school newsletter and took note of details about any special initiatives at their child s school. Parents/caregivers reported looking out for information about disease outbreaks, and upcoming events at the school, and the throat swabbing service was one such initiative that they thought could be mentioned more frequently in school newsletters. [What works best for getting information out?] School. We usually have newsletters and even if they put a portion in there about to come on, coming up and be aware of.like they say for winter months, you know, wrap up warm. Pop a few little things with health issues in our letters cause I think [school] has a pretty cool newsletter and I like reading it and I pick up the newsletter to see what s going on. So that would be nice to have in there. Because our lives are so busy we sometimes forget just the little things and it s the little things that escalate into big things. So if we have them in our newsletters then Oh yeah so I can check my boy out and have a chat with him. (Māori parent) Local-level promotion Parents/caregivers in Porirua responded well to a campaign featuring local sportsman Robbie Fruean to raise awareness of rheumatic fever in the community through 54

55 pamphlets, posters, newspaper articles, visits to schools, 21 and felt that his story was worth paying attention to, as a local boy and a Pacific person who has personal experience (and a positive outcome) with rheumatic fever. It s better coming from Robbie [Fruean] cause like, him being Pacific Islander And information and getting cured and stuff like that for our kids so information coming from him is really good (Pacific parent) In Rotorua, a Sore Throats Matter branded car is used by the local provider delivering the school throat swabbing service, and had raised the visibility and profile of the service in the community. There s a car going around, Sore Throats Matter, it s a red car. (Pākehā parent) In the media Some parents/caregivers recall seeing information about rheumatic fever in a documentary, and others referred to national television news stories that had featured children and communities affected by rheumatic fever. [Heard about rheumatic fever] just from the documentary about the boy in Kaitaia (Māori parent) On the news that documentary on Close Up about Poverty in Porirua. (Pacific parent) Parent/caregiver views on key messages Parents/caregivers were asked about key messages that might help them and others when it came to sore throats and the school throat swabbing service. 22 Several themes emerged as important for future awareness raising activities: Strong messages about the consequences of sore throats, the link to rheumatic fever, and the need to finish antibiotics. Given the low priority of sore throats, making the link to rheumatic fever and the seriousness of heart disease is necessary to get parents attention. Parents/caregivers think messages should be blunt, and gave examples such as the graphic warnings on tobacco packaging as a way to put it out there with the images. They feel that parents/caregivers should be advised to take their child to the doctor if they have a sore throat, rather than wait for a school visit from the throat swabbing service. More emphasis needs to be placed on finishing the course. After a few days they start showing wellness and you think Oh, ok, you don t need to worry about the rest of the bottle but you need to finish the rest of the bottle. (Māori parent) Something that will hit them hard, because man, that hit me hard, when I heard it affects the heart. When I heard that, I had to make a change, I had to stop being inconsistent all the time. I had to be on the ball to go. (Pacific parent) See Appendix 7 for examples of locally developed resources, including the pamphlets described in this discussion group. While the discussion groups did not incorporate specific message testing for the RFPP or school throat swabbing service, several themes arose that may be of use to providers and the Ministry. The findings from audience testing of the HPA-developed resources were not available at the time of this evaluation. 55

56 Emphasis on prevention, and what they can do within their household and wider family/whānau to reduce chance of infection and transmission. Many parents/caregivers want additional prevention strategies and information about what they can do after a GAS positive test to further support their child(ren) s and family/whānau s health. Need more education on what should happen next, after the antibiotics have been taken. Need to follow up with yoghurt, Vitamin C, building your immune system back up, not just given a ten-day course and see you later (Māori parent) [We need to know] how to prevent it from happening and also when it does happen there s an aftermath for it so you know how to get back to the clear. Like a proper cycle where they know how to treat it or how to prevent it, something like that with families who haven t experienced it cause they might want a prevention plan. (Pacific parent) Parents/caregivers often find it difficult to act on advice such as hand hygiene, when the reality of what is available in their community does not match recommendations. We encourage the kids to wash their hands, but when I went into my son s toilet [at school] there was no soap (Māori parent) Parent/caregiver recommendations on strategies to raise awareness Parents/caregivers indicated that the best ways to raise community awareness about sore throats and rheumatic fever involve use of different media, and sharing messages and strategies in a way that parent/caregivers and whānau can relate to. Raising awareness through real stories - Some parents/caregivers thought that sharing stories from parents/caregivers and children from the community who have experience of rheumatic fever in their whānau would be a good way to raise awareness, especially when they can identify with those telling their story, and it snowballs through the community. [Referring to mother in group whose son had rheumatic fever] She ll be good for an example, sharing her son s story on the radio for other parents to hear. She ll be a good example to be on the radio. Because she s been through it, she s been a great mum, just been there for her son. Just thank God because he s come a long way and he s now 27. You know, she ll be a good example. (Pacific parent) Kanohi ki te kanohi / face-to-face engagement and communication with health workers, providing opportunities for parents/caregivers to ask questions. Parents/caregivers value face-to-face communication and an opportunity to discuss issues relating to their children s and family/whānau health. Those parents/caregivers who had interactions with a PHN or kaiāwhina and had the chance to ask questions came away with a better understanding of sore throats and rheumatic fever, and had more confidence in preventing the disease in their child and wider family/whānau. Because I wasn t educated on that hygiene or the oral hygiene and stuff like that which we want to learn more of. It s just brochures, we like to sit in front of people who know the whole thing and explain it to us. (Pacific parent) I wanted to know - side effects of medicine, whether he would need to be swabbed again after the meds, if he could go back to school, how long to stay away, if needed to tell the teacher so he could tell the other kids, next 56

57 time he got a sore throat would it be strep throat. [Nurse] answered all of my questions. (Māori parent) Face-to-face communication was also seen as a way to overcome language barriers and low health literacy, particularly for the older Pacific generation (who are often the caregivers of young children). People are not readers, aye. It s the picture that sells it like that guy with his chest open like that. (Pacific parent) You ve got to think about Pacific Island people and English Do they get what that means?... The simplest things you ve got to explain it in another way cause [my mother] won t get it. (Pacific parent) Community-friendly, accessible venues for message delivery - Parents/caregivers identified community venues such as marae for Māori, churches for Pacific, shopping centres and malls for all communities, and events like markets and festivals as appropriate sites for awareness raising activities in a face-to-face and community-led way. Using promotional vehicles or vans to provide a mobile swabbing service (like the UK Embarrassing Bodies van) was also suggested by parents/caregivers, along with media such as community radio and billboards close to schools and community venues. Creek Fest [annual health festival in Cannons Creek] is really good cause everyone knows it s all about health and well-being, and you re more paying attention. It helps when they re giving out free stuff, everyone s more open to information cause it s based around being healthy (Pacific parent) Feedback on CAR pamphlets Toward the end of the discussion groups, facilitators distributed the newly-produced HPA RFPP pamphlet for parents/caregivers to take home with them. 23 Some feedback on the pamphlet is included here, and may be of help for future development of community awareness material. Parents/caregivers did not like the imagery in the pamphlet, and felt that the scenes and faces used were not supportive of the service and awareness raising. Some Māori parents/caregivers in particular were put off as they felt stigmatised by the photos having a focus on Māori children. I m a bit concerned about only seeing Māoris [sic] here. It s not just Māoris getting sick with sore throats. I understand a lot of Māoris are hard to get to the doctors but I always put my kids first. It s almost implying that we don t look after our own kids it put me off straight away. Seems to me that it is pointing the finger. They re our kids. If the pamphlets are given with all the other kids they ll open it up and start shrinking because all the other kids are looking at them. (Māori parent) Don t put it out as a poverty thing, just put it out. Have it all out there so people don t think it won t happen at my school because of this (Māori parent) My daughter s in it [RFPP] but every child should be on it bugs don t discriminate. (Māori parent) 23 The resource was distributed to all four Māori groups, the two Pākehā groups, but only two of the four Pacific discussion groups (the resource was unavailable at the time the other two groups were held). 57

58 Pacific parents/caregivers and caregivers in the groups are generally positive about the pamphlet they specifically liked the flow diagram as it shows the clear pathway between the infection and the disease and how to prevent it. Comment was made however, that Pacific-language resources would be especially useful. There was a positive response to local resources that had been translated into several Pacific languages when shown to the Pacific discussion groups in Porirua. 7.6 Typologies Data from discussion groups with parent/caregivers was used to identify specific patterns in views and life experience, against two axes: baseline knowledge about sore throats and rheumatic fever how well informed parents/caregivers are about the throat swabbing service at their child(ren) s school. Parents/caregivers who participated in the discussion groups were classified into one of five typology groups: 24 Advocates, Confident consenters, Worried consenters, Under-served and Non-consenters (Figure 3). Figure 3: Parent/caregiver/family/whānau typologies for school throat swabbing service The sections below describe each typology group, their life situation and examples of the reality of parenting and their children s health, and strategies that could encourage and support these parents/caregivers in rheumatic fever prevention. 24 Typology: a way to describe different sub-groups of people who have similar knowledge, attitudes and behaviours, with linkages to their daily life and how they might respond to a programme or intervention. 58

59 7.6.1 Advocates Who they are Māori, Pacific, Pākehā. Older parents, 30s 40s, experienced parents, often with a number of children. High awareness and knowledge of school throat swabbing service and rheumatic fever. Their story/reality This group of parents/caregivers are found to be extremely proactive, confident, engaged and empowered in keeping their child(ren) well. Advocates are often professionals. Many had contact with the service directly or experience of rheumatic fever in their own family/whānau or community. These parents/caregivers are likely to be involved with the Board of Trustees, and/or a school parent programme, and have a position or connection to the school/local health services. They are generally better informed about the school throat swabbing service than many other parents, and are often looked up to by other parents. I m a nurse as well, I worked in district nursing for a few years a few years ago, so we were actively engaged in making sure adults who had rheumatic fever as a child were keeping up with monthly medicine. It s antibiotics for the rest of your life basically to keep yourself healthy. (Pākehā parent) I would ask to be one of the volunteer educators. I would tell them to get their kids swabbed, get their throats checked. (Māori parent) A lot of us parents hang outside the school and wait for our kids and whatever newsletter comes out we seem to discuss and debate about it (Pacific parent) What would help/call to action Reinforce behaviour, and to support them as community champions of throat swabbing, rheumatic fever prevention and wider health and wellbeing of tamariki and the community. Encourage this group to help tell the stories in their communities Confident consenters Who they are Māori, Pacific and Pākehā. Wide age range, from 20s 40s, experienced parents. High awareness of the school throat swabbing service, and good awareness of rheumatic fever. Their story/reality This group of parents/caregivers is confident in their understanding of information about the school throat swabbing service and its link to rheumatic fever. Confident consenters are fully aware of what the service means for their child, and trust the school and nurse/kaiāwhina staff in delivering the service. 59

60 I learnt that when my son was going to kōhanga, about strep throat and rheumatic fever they had all these brochures and they had regular check-ups there. (Māori parent) We stay at the school from whatever time we take our kids there till 12 o clock. We get together and talk about a lot of things. It s a good time to get to know other parents and maybe because we ve been in this [discussion group] maybe we can share it with other parents that we know We share information. Like some parents come up and are like oh their kids are sick and they don t know what s wrong. But we re always there to help them (Pacific parent) What would help/call to action Reinforce behaviour, and continue to provide information and swabbing services that are supportive and responsive to parents and children s needs Worried consenters Who they are Māori, Pacific. Younger parents, 20s 30s. Aware of the service and RFPP, but lots of questions about sore throats and rheumatic fever. Their story/reality Worried consenter parents/caregivers are likely to be slightly anxious about the throat swabbing service and their child s risk of rheumatic fever. They are hungry for information. These parents/caregivers often describe feelings of doubt and a certain level of guilt about their knowledge and actions to date, and of not always being a good parent. The main barriers to understanding and awareness of the service and rheumatic fever for this group are related to health literacy and language (particularly for Pacific parents/caregivers for whom English is a second language). What is rheumatic fever anyway?? Is it a tonsil thing? What could cause the bacteria on the throat in the first place? Is it because us as parents aren t doing the well-being for our kids? (Māori parent) I see a lot of mums walk out like. sister, can I talk to you? And we re both standing there going maybe we should have asked more questions, maybe that s where we failed as parents We didn t, you know, ask for clearer answers where we could understand them. (Pacific parent) It s kind of like we don t question the doctor, push for them and then when my child gets sick it makes me feel like I m not a good mum, I didn t do my job very well. So where can you draw the line between asking questions and being a good mum? (Pacific parent) Parents/caregivers in this group differ from under-served (see below) in that they are more likely to comply with antibiotics, for example. A change in life situation could shift this group into under-served, and becoming disconnected from health and other services can lead to low uptake and engagement with the RFPP and wider rheumatic fever prevention strategies. 60

61 What would help/call to action Verbal/non-written information materials, and being able to talk through the RFPP and wider health issues, with a chance to ask questions and discuss. Suggest use of forums such as hui at school, community events, and parent groups in the school setting. Hearing real stories from people they can relate to, such as other parents/whānau with experience of rheumatic fever, and local personalities (such as Robbie Fruean). Build knowledge and confidence, provide reassurance and validation that they are doing the right thing/doing the best they can, thereby empowering them for the future in dealing with other child health issues Under-served Who they are Māori, Pacific. Younger parents (often in late teens) to 30s. Consent to their child taking part in school throat swabbing service, but often signed the form not really knowing what is involved, and have very low knowledge of sore throats and connection to rheumatic fever. Their story/reality Parents/caregivers in the under-served typology group are usually not well connected to primary health care. They do not feel that they are well equipped with parenting skills and the ability to protect their children from infection and illness in general. Many of these parents did not know whether their child had experienced a throat swab as part of the school service, or what the result had been if they did. Others have experience of more than one GAS infection in their family/whānau. Their experience with primary health care in the past has not been positive, and services often fail to meet this group s needs. They have limited knowledge of antibiotics, and are more likely than worried consenters to have not completed their child s full course of antibiotics. Don t know anything, all I know is my son got strep throat twice and was on antibiotics. Then he got it again after that lot of antibiotics. (Māori parent) You don t have enough money so the main thing is pay the rent, pay the food, the power, and that s it; everything else don t worry about your toys if I m not worried about toys, I m not worried about rheumatic fever or sore throats (Pacific parent) A personal relationship with the local provider (particularly public health nurse/kaiāwhina) is crucial for parents in this group, in terms of trust, responsiveness and engagement with health services. This group is often wary about giving their names or personal information to people seen as part of the Government, or being on a list. [Nurse] has been important for my family with the sore throats, because you can ring her, she will come and swab and boom - it s done, there s no questions asked and no bullshit to go through. (Māori parent) 61

62 What would help/call to action The under-served are a high-risk group, and need services that meet their needs to be prioritised. For the RFPP, this should be an active target group for gaining consent for a child s participation in the school throat swabbing service. They are more likely to respond to culturally appropriate home visits to ease distrust of state agencies accessing their homes and information. Strategies to support this group will need to happen at a systems-level, rather than relying on awareness raising or a single message to prompt behaviour change for sore throats/throat swabbing. The majority of information and wisdom about this group is held locally, and linked to wider utilisation/low engagement with health services that do not meet their needs. Establish a strong, supportive RFPP structure to ensure group is not lost to the programme and swabbing services (eg, use opt off, not opt on, consent approach to encourage swabbing participation). Address wider environmental/social determinants, barriers etc. and explore different models of delivery that better serve this group, including referral pathways to other health and social support services. Link with Well Child Tamariki Ora and Family Start activities, and other initiatives for high-risk groups to increase engagement with primary care and child health services. Use awareness raising tactics to increase confidence, and ensure compliance to prevent reinfection Non-consenters Non-consenters were not represented in the discussion groups, reflecting the small numbers of parents/caregivers across the case sites who did not agree to their child participating in the school throat swabbing service Future considerations Across the parent/caregiver typologies described, the goals of the school throat swabbing service (and wider CAR workstream) should be to: Tailor CAR and swabbing service messages and delivery media to work for under-served and worried consenter groups, aiming to shift them towards the top right corner of the axes (Figure 4) Use standardised approaches to service delivery to support and encourage participation (eg, opt off consent) and to create a community environment where prevention processes are normalised Ensure the systems and processes are effective at engaging with under-served parents/caregivers Maintain confident consenters and advocates in their informed and engaged position, and work with them as champions of the service in the community. 62

63 Figure 4: Parent/caregiver/family/whānau typologies for school throat swabbing service 63

64 8 Community awareness raising workstream This section discusses national and local focus on CAR activities throughout early implementation of the RFPP, including parent awareness of these activities. 8.1 CAR workstream activities A key intended short-term outcome of the RFPP is increased health sector and community awareness of rheumatic fever risk factors, the signs and symptoms of GAS infection and rheumatic fever, and how to prevent rheumatic fever (Ministry of Health, 2012b). The expectation for this outcome is that children and their families/whānau, along with staff in general practices, will be targeted by key messages and information about sore throats and rheumatic fever. It is also expected that the general practice workforce will receive training in GAS management, recognising the signs and symptoms of rheumatic fever, and providing information about how to prevent rheumatic fever in their communities. A combination of national and locally-responsive awareness raising activities is integral to ensuring on-going sustainability and consistency of messages around sore throats and rheumatic fever, in both community and primary care settings National activities Health Promotion Agency As discussed in Section 5, the HPA was funded by the Ministry to implement an awareness raising campaign for rheumatic fever, targeting priority populations (ie, Māori and Pacific). The aim of the campaign is to increase awareness amongst school children, their families/whānau, and the wider community of the importance of recognising and seeking early diagnosis and appropriate treatment of sore throats. At the time of reporting, the HPA had produced a Sore throats matter pamphlet and a poster (in English), which were distributed to providers in November These resources arrived at least six months after throat swabbing had begun in each of the eight targeted areas. The HPA plans to produce Māori and Pacific language resources to support the RFPP. However, the timing for distribution is unclear, and findings from audience testing of resource imagery and content had not been made available at the time of reporting. The HPA resources were received with mixed reaction by local providers. Some (especially providers working with Māori children and whānau) view the imagery on the materials as too medical with the depiction of gloves and swabbing tubes. They comment that this may be off-putting for some Māori whānau as the imagery over medicalises the throat swabbing process, while not actually depicting how a swab is actually taken. 26 In contrast, the locally-produced pamphlet used in Northland is viewed by some providers as showing a more realistic scenario, with a child looking relaxed and comfortable with the swabbing process. Providers would prefer to see space on the back of the pamphlet where See Appendix 7, from Accessed 11 December See also Previous research indicates however, that Pacific parents prefer to see imagery depicting health interventions being delivered in clinical settings as it gives them reassurance that the intervention is safe (eg, Litmus Limited Immunisation Audience Research Accessed 18 December 2012). 64

65 they could add their local contact details (as well as the existing Healthline and generic website address Copies of the national HPA pamphlet, along with locally-produced examples from Northland and Porirua, are included in Appendix 7. Some feedback on the pamphlet (while not a detailed focus of this evaluation) indicates that development of the new resource has not necessarily matched well with local need. Providers generally feel that they are not well informed about HPA plans for additional resource production. National CAR direction and priorities Aside from HPA activities, there is no clear strategic direction and priorities for nationallevel CAR activities for RFPP. Local providers were invited by the HPA to provide input into possible resource development. However, national-level learnings from other successful programmes targeting Māori and Pacific populations could be considered as possible approaches for the RFPP. For example, the Ministry s Human Papillomavirus (HPV), MenzB and childhood immunisation programmes place considerable emphasis on CAR, particularly with Māori and Pacific people, and strategies could be used for a more effective and efficient RFPP. Primary care awareness raising To date, RFPP communications targeted at the primary health care sector have not gained significant traction. It is noted that earlier Ministry discussions include recommendations that an online learning tool be developed for health professionals (see Appendix 1, Section A1.3.3). Providers said they had noticed increasing levels of awareness of rheumatic fever among some primary care providers, an objective of the RFPP that is critical for an ongoing sustainable effort to reduce GAS infection and rheumatic fever in the targeted communities. In these things, it s no good educating parents and kids if they go to the GP and they get scoffed at for turning up with a sore throat. It s important to educate the GPs and practice nurses, and give the parents positive feedback so they will come back next time. (Local provider) I think that the National Heart Foundation guideline is a very comprehensive and useful document. I don t think those were socialised well with primary care and I don t think they were implemented, I don t think they went through a process with primary care like other guidelines potentially have been through. (Local provider) Local activities A number of local-level CAR-related activities and associated challenges were identified during this implementation and formative evaluation. CAR requirement of providers All service agreements with the Ministry require providers to develop and deliver CAR. In particular, providers are required to increase awareness of rheumatic fever risk factors among children and their whānau/families (Ministry of Health 2012b). They are contracted to disseminate rheumatic fever prevention resources, and provide health education to individuals and/or groups as appropriate in schools, other settings and the wider community. The Ministry indicates in provider agreements that CAR activities can be delivered through face-to-face advice and support, educational sessions (school and 65

66 community), articles in local newspapers, school newsletters, interviews on Māori radio and TV, information sharing and dissemination, and presentations (hui, fono, at marae, etc). Three health promotion messages are specified in providers service agreements: 1. Sore throats matter. 2. If your child is sick, take them to the doctor. 3. Practice good hand hygiene and cough etiquette. Differences were noted in the focus of messaging with target audiences in some of the eight areas. For example, in Rotorua a decision has been made to focus on the sore throats message, and not on rheumatic fever, to ensure children and parents/caregivers are not alarmed in a community where rheumatic fever is associated with needles and pain. While the messages may reflect the Ministry s health promotion requirements, it is unclear how this strategy reflects the Ministry s intent to raise the profile of the issue of rheumatic fever more generally. Stakeholders and parent/caregivers felt that the seriousness of sore throats and the link to rheumatic fever needs to be highlighted to shift apparent parent complacency about sore throats in children, and prompt action to take their child for diagnosis and treatment of a GAS positive result. At this stage not everyone has a clear and accurate understanding of rheumatic fever. There is awareness of it. Some people are thinking that rheumatic fever can be caused by different things other than a sore throat. Some people are thinking that a cough is going to cause rheumatic fever. But they are talking about it. Over time I think we will refine their knowledge and awareness of it. (Key informant) For many providers, most of the time spent on CAR efforts focused on the RFPP s launch in their community (often attracting media through the involvement of Ministers at the launch and perhaps radio interviews by health experts). More community targeted promotional activities in some areas (for example, promotion at community events) appeared to taper off in some areas due to resource and time constraints, and emphasis on delivering throat swabbing services and meeting school target numbers. CAR vs. throat swabbing The relative importance being ascribed to CAR activities varies between providers despite its apparent focus in service agreements. Some providers do not see it as a key part of their service. In contrast, one provider (in Rotorua) has been asked to focus the majority of their contracted time on awareness raising activity in schools, as opposed to throat swabbing. There are questions about how realistic this is given the small amount of funding this provider receives through the RFPP budget. Probably the most important thing is not what this money is going to buy in terms of school-based throat swabbing, but what it is going to buy in terms of increasing awareness and networking between services, Ministries, providers, and community. (Key informant) 66

67 While current service agreements provide an indication of the types of CAR activities to be undertaken and the key messages of focus, there is little direction provided on the time or resource investment that providers should put towards CAR. A key question is whether messages should focus on encouraging parents/caregivers to get their children swabbed, raising community awareness about sore throats and rheumatic fever more generally, and/or on encouraging parents/caregivers to seek treatment from primary care and/or the school service. On the whole, provider resource is being largely invested in throat swabbing activities at this early stage of the RFPP, leaving little or no resource for CAR activities. There is a risk that the RFPP has no clear plan for ensuring sustainability of knowledge and behaviour change, given the current focus providers are placing on throat swabbing over and above CAR activities. Providers and stakeholders recognise the need to maintain awareness of the sore throats and rheumatic fever message beyond the life of the RFPP. However, based on current levels of CAR activity nationally and locally, this seems unlikely to happen in a sustained way. [PHN] needs to be in every classroom at least a couple of times a term. I can t see how that s possible for 12,000 kids with just [one nurse] doing it. Health promotion with that number of children you would need to be a super-woman to realistically do that! (Local stakeholder) Pacific CAR focus The Ministry s service agreement for providers includes a specific requirement for Māori CAR: systematically promote health and education messages utilising methods, and within settings, that are known to be effective for Māori communities. No specific CAR requirement is included for Pacific communities, nor in provider agreements in areas with a large Pacific population (eg, South Auckland and Porirua). Similarly, it is not clear what expectations lead providers have of sub-contracted providers in terms of Pacific CAR, a particular issue in South Auckland). More broadly, the intended mechanisms for engaging with Pacific families through CAR activities are not clear. Currently, there are no national Pacific language-specific resources available for use by providers. The timeframe and process for producing these was also unknown. In their absence, previously developed locally translated rheumatic fever resources are being shared for use between South Auckland and Porirua. Local CAR activities Many local providers have sought to either develop their own local CAR resources on small budgets (eg, on their own colour printers), use existing rheumatic fever resources, or share resources with other providers who are not necessarily funded through the RFPP. In some cases, providers have developed RFPP resources specific for their community, such as those in several Pacific languages in Porirua, and featuring a local rugby personality with personal experience of rheumatic fever. Production of local CAR resources for the RFPP was dependent on individual providers or staff members taking the initiative and resourcing, and often happened because of an existing prevention programme for rheumatic fever (eg, Porirua Kids Project), and/or because of the delay with the HPA resources. The risk with this approach however, is inconsistency of messaging across the areas, and unnecessary use of budget and time to duplicate resources that are already in use (or development) elsewhere. We have had no access to networking with other throat swabbing initiatives and teams. This would be very helpful. (Local provider) 67

68 It appears that local CAR contract requirements potentially overlap the HPA s CAR service specifications, creating confusion and potential double up of resource development efforts. There is scope for the Ministry to provide clarification of CAR expectations of both local providers and the HPA. Examples of other CAR resources and activities observed at the local level include: Merchandising/incentives for tamariki with a GAS positive result eg, antibiotic reminder sticker charts, thermal T-shirts for completion of antibiotics. Merchandising/incentives for tamariki after throat swabbing eg, wrist bands. Promotional flags outside provider s clinic. Promotional car adorned with sore throats matter messaging. School holiday drop-in for throat swabbing at provider clinic (rather than waiting until the start of school term for swabbing to begin). Chant about rheumatic fever and sore throats taught to children in school setting while waiting for throat swabbing. A skit about germs and sore throats that nurses present to children at school. These demonstrate the innovative way providers have adapted their own CAR activities and message delivery in the local context, to encourage children and whānau to participate in the throat swabbing service. 8.2 Early impact of RFPP on community awareness The school-gate intercept survey with parents/caregivers provides several useful measures of community awareness, both as benchmarks for the RFPP roll-out, and giving insight into early implementation of the CAR workstream. This section describes parent/caregiver awareness of sore throats and rheumatic fever, and the school throat swabbing service. It also outlines parent/caregiver knowledge about related aspects of health promotion such as preventing and treating sore throats in children. Survey findings represent responses from those parents/ caregivers who were present at the school gate at the time of the survey. As such, this may potentially bias the results towards more informed, engaged parents. Additional notes on survey interpretation and more results by site, ethnicity and other sub-groups can be found in Appendix Awareness of sore throats/rheumatic fever as important child health issues At the start of the intercept survey, parents/caregivers were asked to describe what they thought were the most important issues affecting the health of children. This provides a measure of where sore throats rank in terms of priority for parents/caregivers, without being prompted. Slightly more than one in ten (12%) spontaneously mentioned sore throats, strep throat, rheumatic fever or rheumatic heart disease as one of the most important issues affecting the health of children. The other issues for child health that were most commonly mentioned by parents/caregivers included nutrition/eating, colds/flu/coughs, skin problems/eczema, asthma, healthy homes/keeping warm, and non-specific bugs/bacteria/infections/viruses. 68

69 8.2.2 Awareness of sore throats and the throat swabbing service Parents/caregivers were asked whether they had seen, read or heard anything in the last six months about sore throats or something called strep throat. Almost three-quarters of parents/caregivers (74%) reported they had seen, read or heard something in the last six months (Figure 5). Figure 5: Awareness of sore throats and school throat swabbing service, intercept survey Of those who had heard about sore/strep throat, sources of information were the school (68%), practice nurse/gp (37%), advertising or information in the community (23%), or another source (30%, Figure 6). Six out of 10 parents/caregivers (60%) had heard of the throat swabbing service being conducted at the school where the survey was conducted (Figure 5). Figure 6: Sources of information about sore throats, intercept survey 69

70 8.2.3 Usual course of action if child has sore throat When asked what they would do if their child had a sore throat, almost all parents/caregivers said they would take them to a doctor/nurse (96%). 27 Other responses included giving them medicine, Pamol or paracetamol (26%) and/or giving them hot drinks, something to gargle or ice blocks (10%). Other actions taken included keeping the child warm, home from school, and away from other children, and generally monitoring or checking on them regularly Awareness of untreated sore throat Parents/caregivers were asked what they thought could happen if a child s sore throat was not treated, as a proxy measure of awareness of the link to rheumatic fever, and the contagiousness of strep throat. One-third (33%) mentioned that untreated sore throats can lead to heart damage, heart disease or rheumatic fever. More than one in ten (15%) mentioned that a sore throat can spread to others or others can catch it. When asked about ways to stop the spread of sore throats, parents/caregivers gave responses that included not sharing water bottles, cups or toothbrushes (37%), covering mouth when coughing or sneezing (27%), washing hands with soap after coughing or sneezing (27%) or taking medicine (21%). Other behaviours that parents/caregivers gave for stopping sore throats spreading included keeping the child home from school, taking them to the doctor, and more education/awareness. 8.3 Potential improvements for CAR A number of potential improvements for the CAR workstream were identified, including resourcing and enabling local providers with RFPP-specific ideas and tools for CAR, building effective Māori and Pacific CAR activities, and the need for a national strategy for awareness raising with RFPP communities and primary care. Adequately resourcing providers for CAR activities - Clearer expectations are needed on the relative focus and investment providers should be putting into CAR, compared to throat swabbing. The Ministry may wish to consider redirecting some of the investment targeted at national CAR into local provider contracts, with clear expectations, to enable an increase in local-level CAR activity. Effective CAR for Pacific communities - A specific focus on Pacific CAR will be important for building and maintaining the trust of Pacific communities in the RFPP. This is of particular importance given the limited number of Pacific providers involved in delivery of the RFPP. National direction for Pacific CAR strategies will be essential for future delivery of a successful RFPP to Pacific families and children. Effective CAR for Māori communities - CAR activities need to be tested to avoid potential stigma and negative stereotyping. In addition, RFPP providers need to build 27 Some discrepancy was noted between survey results and discussion group findings. Parents in discussion groups said they were more likely to monitor their child s sore throat to see if it resolves on its own, before deciding whether to take their child to the doctor (similar to results from Abdulhamid et al s study 2012). This discrepancy may also be due to wording of the survey question, which asked what would you do if your child had a sore throat, rather than what do you do., and so may reflect parent/caregiver intent rather than action. 70

71 credibility by delivering CAR activities in their community and with families/whānau, using resources and strategies with strong relevance for Māori. Primary care awareness raising - There is opportunity to increase communication and engagement with primary care about the RFPP, the aims of the RFPP, and the role of general practice and PHOs in supporting efforts to reduce the rates of rheumatic fever in the identified priority areas. This communication should be centrally-led, rather than expecting providers to work in raising GP awareness while also delivering throat swabbing and awareness raising activities with parents/caregivers, schools, and others in the community. A centralised approach will ensure a strategic and coordinated focus that aligns with national CAR activities. National CAR strategy and integration Significant investment has already been committed to RFPP resource development from the Ministry through HPA. HPA s role as leading national CAR activities is important as it has the potential to: ensure more consistent evidence-informed rheumatic fever messaging (both in the community and in primary care) ensure messaging and communication approaches are tailored to ethnicity and typology groups as appropriate reduce the burden (both time and cost) on providers to produce their own promotional and merchandising material. To maximise budget efficiencies as the RFPP moves into its second year, it is timely to review and, if necessary, refine national promotional activity approach. In particular, the Ministry needs to clarify expected CAR outputs and timeframes for the RFPP (eg, extend to production of locally-supported merchandise), and ensure timely completion with a clear plan. A clear national CAR plan for the RFPP will enable consistent and sustainable awareness of rheumatic fever across different sites and communities. National coordination and direction must be balanced with a need to maintain local responsiveness. Existing examples of good practice and local experience provides an opportunity to share CAR ideas with other providers, which could be led through the NCS. The HPA can add value through the purchase of bulk merchandising with room for local adaptation, creating efficiencies for providers and the RFPP more generally, while potentially increasing the effectiveness of CAR activities. Another consideration for improving CAR impact could involve horizontal integration of service delivery (eg, throat swabbing and skin infection), particularly where consistently relevant messages are integrated across child/school health programmes, thereby increasing the reach of the messages. 71

72 9 Conclusions and recommendations This section reports conclusions arising from the implementation and formative evaluation and gives recommendations to further enhance the RFPP s operation. 9.1 Conclusions The evaluation found that some of the planned Budget 2011 activities were delayed in their implementation (eg, national health promotion resources, some local swabbing services) and others had not progressed (eg, CQI, health sector awareness raising and training). Positive, well-supported initiative Overall, there has been a positive response to the roll-out of the RFPP. Stakeholders are positive about the funding allocation for rheumatic fever prevention. While they consider that more could be done (especially in addressing primordial determinants), there is recognition that the funding allocated from Budgets 2011 and 2012 is a good start. Dedicated local providers have tackled the task of implementing school throat swabbing services and community awareness raising in short timeframes with enthusiasm. They have been proactive in using local networks, contacts and resources to support their work. Local communities (including schools, primary care, pharmacists, and others) appear supportive of the RFPP. They have assisted local providers in its early implementation and have helped address local RFPP challenges as they arise. Parents/caregivers are overwhelmingly positive about throat swabbing being delivered in schools as the service does not cost them anything, is convenient, undertaken by trusted providers in a place where children are comfortable, and the results and provision of appropriate treatment is done in a responsive way. The following aspects of the RFPP should be considered to maximise the effectiveness, consistency and sustainability of activities across the RFPP sites. Ensure appropriate targeting to achieve RFPP goal At this early stage of implementation, many stakeholders are questioning whether current RFPP activities will have an impact on reducing rates of rheumatic fever, and if school swabbing is the best focus for achieving this. RFPP funding from Budget 2011 has enabled an estimated one-third to one-half of at-risk children to participate in throat swabbing. Clearly, either more investment or a wider scope for diagnosis and prevention is needed if a sustainable impact is to be achieved. In addition, identification of RFPP sites was based on rheumatic fever hospitalisation data, later found to be out-of-date, and also known to fluctuate markedly due to the small numbers involved (especially in some localities). There is a call from stakeholders to identify target areas (and allocate funding) based on GAS incidence, a more reliable and realistic indicator of community need and RFPP impact. This will of course be dependent on the availability of quality data and surveillance information. Local providers and the 72

73 Ministry will therefore need to collect and collate relevant, consistent and quality data to inform decision-making. Integration with DHBs The limited involvement of DHBs in decisions around delivery of rheumatic fever prevention services in their areas means that Budget 2011 contracts were established without sufficient local input, and with limited linkages to existing DHB and local child health strategies. More recently, DHBs have expressed concerns about Ministry expectations for meeting BPS targets, particularly in areas where the DHB is not involved as a contract holder or provider for the RFPP. Ensure appropriate focus and service delivery for Pacific populations There is currently a lack of focus on Pacific children and ensuring Pacific involvement in planning and frontline delivery of the RFPP. Given the considerable burden of the disease on Pacific children, this is a particular concern in areas with a large Pacific population, and limits the potential of the RFPP to be effective in reducing rates of rheumatic fever in Pacific children. Enhance awareness raising community and primary care In the early stages of RFPP implementation providers have been focused on throat swabbing services, with little resource committed to community awareness raising. In addition, there was minimal national support for CAR resources and message consistency, and providers are unsure if or how their local-level activities align with a national CAR strategy or wider DHB initiatives. There is a clear opportunity to focus awareness raising activities to groups who are least likely to access RFPP services (the worried consenters and under-served, Pacific communities, and Māori whānau more generally). Engagement and awareness raising with primary health care providers is minimal across RFPP sites, and usually conducted in an ad-hoc way. This needs to be addressed to ensure that health professionals in RFPP areas reinforce the same messages and behaviours in their interactions with the community. Develop referral pathways to other health and social support services An integral objective of local RFPP service delivery is to facilitate referral and support of children and their families/whānau to other health and social service providers (including Whānau Ora), where needed. Throat swabbing services are being delivered (in some areas) in a way that does not identify nor follow up emerging child health or social issues beyond sore throats and treating GAS infection. The lack of clear process and expectations for local providers presents a significant risk for RFPP consistency, and missed opportunities for improving child and family/whānau health more broadly. There is wide variation in local provider understanding and implementation of the Ministry s expectations in this area. 73

74 Develop RFPP quality frameworks and guidelines Wide variation exits in approaches to service delivery being used by providers at the local level. While a degree of flexibility for the local context is encouraged, there is a strong need for operational frameworks and guidelines across sites, and to foster sharing of strategies and tools in a way that is effective for providers with limited RFPP resource. The scope of the RFPP NCS could be formalised to include guidance and direction for providers on: consent approaches; the use of standing orders; budgeting and payment for pharmacy, GP and laboratory costs; swab quality and swabbing processes; protocols for community-based/opportunistic swabbing and referral processes. In addition, there is limited opportunity at present for providers working at the frontline to share learnings from and experiences of service delivery. There is an opportunity to develop a set of best practice approaches based on local provider experience, and to share these across RFPP sites for a more consistent approach, while maintaining local flexibility. Ensure RFPP sustainability Providers currently holding service contracts for the RFPP are anxious about the term of their agreements, which have an end-date of June Along with other stakeholders, they are calling for transparency from the Ministry regarding an RFPP exit strategy, and sufficient notice if contracts are not renewed. Providers are also concerned about the impact of service termination on local schools that support and have committed to the RFPP, and most importantly, on parents/caregivers and whānau, whose expectations of a school-based throat swabbing service and wider support for their child and family/whānau have now been established. The lack of focus on engaging with primary care also impacts on sustainable change for provider and community activities, and impacts on likelihood of reducing rheumatic fever in the long-term. 74

75 9.2 Recommendations In terms of future RFPP implementation, it is recommended that the Ministry considers the Programme improvements described in Table 4 below. Table 4: Implementation and Formative Evaluation recommendations for the RFPP Workstream Recommendations 1. RFPP focus Consider collection of GAS incidence data to more reliably inform location of future swabbing services Consider a programme that targets GAS prevention in the community, rather than individual symptomatic throat swabbing in children Continue and widen focus on addressing primordial determinants of health in target areas 2. Pacific Strengthen focus on Pacific from strategic (Ministry) level through to contracts and service specifications (especially for service delivery and CAR in areas with Pacific populations) Ensure Pacific-language resources are made available in a timely manner Consider production of other appropriate CAR resources and materials, including non-written media 3. Māori Seek clarity on requirements of sub-contracted providers around service appropriateness for Māori Consider production of appropriate CAR resources and materials, including non-written media 4. Service integration 5. Primary care engagement 6. National and local CAR 7. Quality frameworks and guidelines Examine options for integrating throat swabbing with other child health services Ensure referral pathway model/protocols developed (with health and wider social support agencies) Involve DHBs in future funding decisions to ensure RFPP planning reflects local knowledge and local strategies Focus effort on building primary care awareness and engagement, with RFPP and wider rheumatic fever messaging Establish a centrally-led primary care awareness raising strategy Review focus and allocation of spend to HPA in out-years Establish strategic national RFPP CAR plan Review respective national (HPA) and local (provider) community awareness raising roles and responsibilities and fund accordingly Ensure national-level CAR learnings from other programmes targeting Māori and Pacific are considered in the development of future resources Ensure timely production of any future resources Ensure resources are appropriate for the local context Provide national guidance and support on: - use of standing orders - the use of an opt out consents approach - referral pathway model/protocols (including health and wider social support) - community-based/opportunistic swabbing guidelines and protocols - use of sub-contracted providers (ie, ensuring appropriateness of service delivery for Māori and Pacific children and families/whānau) Note: some of the above could be delivered through NCS. 75

76 Workstream 8. National Coordination Service Recommendations Strengthen NCS s role to: - establish quality assurance standards and monitoring for swab taking and ensure consistency in quality within and across sites over time - advise providers on how to implement swabbing approaches and processes (informed by standards developed for school immunisation programmes, for example) - advise on and recommend use of standing orders - create opportunities to share learnings between frontline staff - regularly document and disseminate best practice approaches 9. Sustainability Ensure a transparent exit plan is communicated to providers ahead of contract end date (at least six months) Clarify expectations of relative emphasis on CAR versus throat swabbing with local providers Consider links to other child health programmes with the aim of integrating service delivery and messaging opportunities 10. Future evaluation Review provider data monitoring for content and quality to ensure longer-term RFPP outcomes will be measurable over time Establish a monitoring and evaluation framework to reliably inform effectiveness of RFPP delivery and tracking of RFPP progress 76

77 Appendices 77

78 Appendix 1: Description of New Zealand s RFPP A1.1 Overall aims The long-term outcome of the Ministry s RFPP is Reduced incidence of Group A Streptococcal infection (GAS) and acute rheumatic fever (rheumatic fever) amongst the New Zealand population (HSC 2012a). The goals of the RFPP are to: Achieve equity of incidence and outcomes of rheumatic fever between Māori and Pacific children, and other New Zealand children. Contribute to the reduction of the age-standardised annual rate of rheumatic fever among Māori and Pacific peoples, other New Zealanders, and all DHB populations to 0.4 per 100,000 by Contribute to the reduction of rheumatic fever recurrence in New Zealand to five cases or fewer per annum by Allocation of the initial Budget 2011 funding for the RFPP is shown in Table A1.1. Table A1.1: Allocation of the Rheumatic Fever funding (Vote Health) allocated in Budget 2011 Workstream activities Total 2011/ / / /15 Local coordination, strengthening of frontline primary health care services, and school-based sore throat services $8,250,000 $1,350,000 $2,300,000 $2,300,000 $2,300,000 Health worker training $450,000 $200,000 $100,000 $100,000 $50,000 Clinical tools $450,000 $250,000 $150,000 $50,000 - National Coordination Service $480,000 $120,000 $120,000 $120,000 $120,000 Development and delivery of health promotion Monitoring, evaluation, surveillance and research $550,000 $200,000 $200,000 $100,000 $50,000 $1,820,000 $750,000 $350,000 $350,000 $370,000 Total $12,000,000 $2,870,000 $3,220,000 $3,020,000 $2,890,000 Source: Ministry of Health 2011a. A1.2 Governance and management A1.2.1 RFPP Governance Group A Governance Group for the RFPP was established in 2011 by the Ministry. The role of the Governance Group was defined in the Terms of Reference as: to provide oversight to the Ministry of Health s Rheumatic Fever programme team to provide critical feedback around clinical advice and other expertise as appropriate the Governance Group is not responsible for sign off of policy papers or advice to the Ministry of Health s senior leadership team or Ministers (Ministry of Health 2011a). 78

79 The Terms of Reference for the Governance Group outlines expected commitments for members, including being available for one meeting each month (alternating between a face-to-face meeting in Wellington, and a meeting to be attended in person by Wellingtonbased group members, and via teleconference for those who are not). Governance Group members include a number of Ministry representatives (up to nine in some meetings) with representation from Communicable Diseases, Child and Adolescent Health, Māori Health, Pacific Health, and Public Health, and five external members: Louisa Ryan and Norman Sharpe from the National Heart Foundation; Lance O Sullivan an independent GP; Diana Lennon from the University of Auckland and Auckland District Health Board (from June 2012); and Helen Herbert from the National Coordination Service (from May 2012). Group meetings are chaired by the Ministry s RFPP Project Manager. Draft minutes of each meeting highlighting action points and significant points of note are circulated following the meetings and agreed at the next meeting. The first meeting was held in July 2011, and the Group has met approximately once each month since that time. Their Terms of Reference was revised in April A1.2.2 Reporting to the Minister The RFPP was established as part of the work programme of the Ministry s Communicable Diseases Team in the Public Health Group (previously the Population Health Directorate) in the Clinical Leadership, Protection and Regulation Business Unit. Monthly reports on RFPP progress are prepared for the Minister and Associate Minister of Health, along with more regular reports on any programme issues arising, as necessary. A1.3 Workstreams and activities The following workstreams form part of the RFFP. Work stream 1: Local coordination and school-based sore throat services Provision of throat swabbing services in the key geographic areas. Community and health sector awareness and education activities. Follow up contact with individuals and their families. Provision of a National Coordination Service. Work stream 2: Raising Awareness Implementation of a rheumatic fever health promotion programme (HPP). Development of an audit tool and CQI system for general practices. Training for health sector staff in the eight key geographic areas. Work stream 3: Monitoring and evaluation, surveillance and research Ongoing surveillance and reporting on rheumatic fever. Rheumatic fever research activities. Ongoing monitoring of the Rheumatic Fever Prevention Programme. Evaluation of the Rheumatic Fever Prevention Programme. Source: Ministry of Health 2012a. 79

80 A1.3.1 Throat swabbing services New Zealand best practice guidelines for rheumatic fever (developed by the New Zealand National Heart Foundation and the Cardiac Society of Australia and New Zealand) recommend that throat swabbing be considered in schools in areas where there is a population incidence of rheumatic fever of at least 50/100,000. Due to limited funding, the threshold used by the RFPP is 75/100,000. The objectives of this component of the RFFP are to reduce the rate of rheumatic fever in key areas by: providing throat swabbing and referral services in the school, home or other settings as appropriate for school children aged 5 14 years that present with sore throats in highrisk areas, and eligible whānau/family members living with these children increasing awareness of rheumatic fever risk factors among children and their whānau/ families in the key geographical areas developing and maintaining relationships with other health and social service providers (including Whānau Ora providers) to facilitate referral and support as appropriate (Ministry of Health 2012b). The Ministry held a series of hui in the key geographic areas identified for targeting by the RFPP during July-September 2011 (Ministry of Health 2011g). In September 2011 they called for tenders to provide throat swabbing and referral services in these areas. Table A1.2 lists the contracts subsequently awarded and the annual value of these. Table A1.2: Contract details for provision of RFPP throat swabbing services 28 Area Contract holder Contracted price (per annum) excl GST Contract duration Kaitaia Te Runanga O Te Rarawa $201,818 Sept 2011 June 2013 Whāngarei Kia Ora Ngatiwai Trust $150,667 Jan 2012 June 2013 South Auckland National Māori Hauora Coalition $972,000 Feb 2012 June 2013 Porirua Capital and Coast DHB $242,118 Feb 2012 June 2013 Bay of Plenty Tuhoe Matauranga Trust $128,471 Feb 2012 June 2013 Hawke s Bay Hawke s Bay DHB $200,938 Nov 2011 June 2013 Rotorua Korowai Aroha Trust $79,765 Feb 2012 June 2013 Tairāwhiti Te Hauora O Turanganui A Kiwa $144,000 Feb 2012 June 2013 Source: Based on Ministry of Health contract agreements with service providers. A1.3.2 National Coordination Service for Rheumatic Fever In September 2011 the Ministry called for tenders for a National Coordination Service (NCS) for Rheumatic Fever, to be undertaken for the duration of the RFPP. The aim of the NCS is to coordinate rheumatic fever services to ensure a nationally integrated approach to awareness and prevention (Ministry of Health 2012h). Delivery of the services is led by a National Coordinator. 28 Note: table excludes details for contracted Waikato providers as these were not in contract during the time that the formative and implementation evaluation activities were undertaken. 80

81 The role of the NCS is to support health care providers and other agencies to implement local-level rheumatic fever programmes, facilitating collaboration and networking amongst providers, promoting training opportunities for health sector staff, distributing health promotion resources, and encouraging the use of clinical tools. The NCS works across the eight high-risk communities of Northland, South Auckland, Waikato, Lakes and Bay of Plenty, Tairāwhiti, Hawkes Bay, and East Porirua. Ngati Hine Health Trust (Ngati Hine) was awarded the NCS contract, commencing 30 January Ngati Hine is a Māori health provider based in Northland. It has delivered a rheumatic fever awareness and prevention programme in Northland, and has contributed to the development and implementation of regional and national rheumatic fever prevention strategies. The value of the contract is $480,000 over four years (Ministry of Health 2012h). The Ministry requires the NCS to: work with providers and agencies to support the provision of culturally appropriate and practical rheumatic fever programmes to meet the diverse needs of Māori and Pacific communities. facilitate collaboration and dissemination of information amongst providers and stakeholders in the areas targeted by the RFPP. promote health worker training, including facilitating awareness of available training, and promoting attendance at training opportunities. coordinate and distribute health promotion materials to the sector, including ensuring that appropriate resources are available, and promoting their effective use. promote the use of clinical tools by health practitioners, such as audit tools for general practice and continuous quality improvement systems. take account of the particular needs within communities served by the RFPP and provide oversight to help ensure there are no barriers for access to communications, and that frontline services are safe for all people (Ministry of Health 2012h, pp2-3 and Ngati Hine Health Trust 2012). Specific activities that the NCS is contracted to undertake are outlined in Annual Service Delivery Plans. The first was for the period 1 January June An annual plan is to be submitted to the Ministry by 30 May of each contract year until 30 June Service monitoring reports are to be provided to the Ministry every six months until the end of the contract. A1.3.3 Raising awareness amongst health professionals Workstream three of the RFPP includes $450,000 for the development of tools and training for health sector staff to raise awareness of GAS and rheumatic fever. Funding was originally allocated as outlined below; however due to delays progressing this work, the 2011/12 funding needed to be carried over (Ministry of Health 2012i). 2011/ / / /15 Total $200,000 $100,000 $100,000 $50,000 $450,000 Ministry consultation with health sector experts identified that, while most health professionals who work with at-risk populations have a degree of knowledge of GAS infection and rheumatic fever, this is not necessarily top of mind when a child presents with a sore throat. Concerns were also expressed around the impact of the PHARMAC 81

82 Wise Use of Antibiotics campaign, 29 which may have led to reluctance to prescribe antibiotics. The Ministry therefore identified that the training should focus on: the signs, symptoms and risk factors of GAS infection and rheumatic fever. increasing awareness and uptake of the Rheumatic Fever algorithm and Sore Throat Guidelines. the importance of prescribing antibiotics. An internal Ministry Memo (Ministry of Health 2012i) recommended that initial scoping work be carried out to develop an online learning module for health professionals on GAS infections and rheumatic fever. A1.3.4 Continuous Quality Improvement System and Audit Tool(s) for rheumatic fever services in general practices In January 2012 the Ministry called for tenders to develop a CQI system and audit tool(s) to support general practices to conduct an audit of their process of responding to sore throats, specifically GAS. The Ministry expected the CQI to take a whole of practice approach reflecting the patient s journey through the system and to come from both a patient and practice perspective. Specifically, the system was expected to: encompass the whole practice, including general practitioners, receptionists, practice nurses, and other staff involved in patient care. be culturally appropriate, with a focus on Māori and Pacific needs and Māori and Pacific involvement in development of the RFPP. engage practice staff to lead the quality improvement process. enable practices to review the quality of their sore throat management. identify areas where the practice is doing well and where it needs to improve. guide the formulation and implementation of plans to address any identified gaps. build, where feasible, on existing CQI processes within the primary care environment (Ministry of Health 2012j). The Ministry indicated a commitment to funding the CQI system and audit tool(s) until 30 June 2014 and it was envisaged that the majority of funding would be invested in system development, with the remainder allocated to system implementation and maintenance. A preferred provider was identified for the CQI system but the provider was not contracted by the Ministry. The introduction of a broader national rheumatic fever programme, focusing on the BPS target, has lead the Ministry to reconsider the scope of a CQI system. As a first step, the Ministry has decided to do a needs assessment to determine what support is required by primary care. As such, communications were sent to a number of GPs, a series of meetings were held with Health Workforce NZ, and a survey was developed to assess the training and development needs of the rheumatic fever workforce (Ministry of Health 2012k). It has subsequently been found that support for primary care should focus on training, rather than audit, and the Ministry is planning a number of interventions including: improved guidance to health professionals about how to diagnose and treat GAS infection. a suite of online training modules to be developed and implemented in This campaign was launched in See Accessed 17 December

83 a training day for the rheumatic fever workforce to be held in early 2013 (Ministry of Health 2012k). A1.3.5 Health promotion programme The Health Sponsorship Council (subsequently part of the newly-formed Health Promotion Agency) was contracted by the Ministry in late 2011 to provide a health promotion programme (HPP) for the RFPP. The objectives of the HPA service are to: implement a rheumatic fever awareness raising campaign targeting priority populations in seven key geographic areas. increase awareness amongst school children, their families/whānau and the wider community of the importance of recognising and seeking early diagnosis and appropriate treatment of sore throats. The HPP service specification states that a successful awareness campaign will consider the most appropriate methods for targeting Māori and Pacific communities including development of culturally appropriate resources for use in the key geographic areas. The Ministry also suggested that the campaign should focus on community engagement, have nationally consistent messages, and be appropriate to the local context (Ministry of Health 2012e). The HPP focuses on the high-risk communities of Northland, South Auckland, Waikato, Lakes, Bay of Plenty, Tairāwhiti, Hawkes Bay, and East Porirua as identified for the throat swabbing services. The primary audiences for the HPP are identified as Māori and Pacific children (5 14 years old), their parents/caregivers and whānau (HSC 2012b). Identified outcomes for the HPP include: key messages and information about risk factors, treatment and management of sore throats and rheumatic fever are readily available and appropriate for target audiences, in the high-risk communities children and their parents/caregivers, and whānau in high-risk communities understand the link between a sore throat and rheumatic fever key people who provide support and advice to children, parents/caregivers and whānau (eg, teachers, community health providers, church leaders, etc) have access to information on risk factors, and treatment channels messages are provided in a culturally appropriate way, suited to the target audiences parents/caregivers and whānau understand the importance of getting sore throats checked either by a GP or school-based sore throat service (HSC 2012b). The Ministry allocated a budget of $550,000 over a four-year period for the HPP project. An annual breakdown of this funding is provided below. Detailed budgets will be developed as part of the HPP service plan development, for each year. 2011/12 (from Feb 2012) 2012/ / /15 Total $200,000 $200,000 $100,000 $50,000 $550,000 In early 2012 (following confirmation of the local service providers) HPA started engaging with contracted local service providers to determine their resource and support needs for rheumatic fever prevention. A stocktake of existing rheumatic fever awareness raising resources was then done in April 2012, also by the HPA. Also in April, the HPA 83

84 commissioned qualitative research amongst the target audiences to determine what facilitates and prevents caregivers living in areas at high risk of rheumatic fever from seeking treatment for sore throats. A1.3.6 Research, surveillance and monitoring Developing a national register/rheumatic fever surveillance Jackson and Lennon (2009) undertook a project to scope the development of a web-based National Rheumatic Fever Register which will support secondary prophylaxis, improve the rheumatic fever recurrence rate, and inform prevention programmes. The project involved consultation with paediatric and public health networks, semi-structured interviews with key informants, and a brief review of the national and international literature. The authors recommended establishing a web-based national rheumatic fever register as part of a comprehensive national rheumatic fever control programme, with the following rationale: Prevention of rheumatic fever and RHD aligns with government health strategy documents There is strong support for a national register from those currently involved regionally in rheumatic fever control (key stakeholders) Register-based programmes are the most effective for delivering secondary prevention; however, there are limitations to the current system of regional registers. Potential gains of a national register were summarised as follows: Updated, remotely accessible, nationally application with improved functionality Improved tracking and management of highly-mobile patients (across DHB boundaries) Improved surveillance particularly of RHD incidence and rheumatic fever recurrence Ability to evaluate effectiveness of primary and secondary prevention programmes Improved rheumatic fever and RHD control with lower morbidity, mortality, and health costs Ability to identify high-risk patients for enhanced follow-up. In April 2012 the Ministry of Health contracted the Wellington School of Medicine, University of Otago to undertake a review of existing rheumatic fever surveillance systems and the performance of existing data sources. An internal audit and review of rheumatic fever registers was also done within the Ministry by a public health medicine registrar working for the Chief Advisor, Child Health. Rapid Antigen Testing (RADT) In early 2012 the Ministry called for tenders to undertake a pilot study of GAS Rapid Antigen Diagnostic Tests in general practice and school-based swabbing service settings, and to assess their accuracy, feasibility and acceptability in these environments. This contract was awarded to the University of Auckland in June/July

85 Appendix 2: RFPP logic models - Ministry of Health and Litmus Draft model (Ministry of Health, 2012l) 85

86 Revised model and workstream sub-models (Litmus Evaluation Team, 2012) 86

87 Source: Litmus Evaluation Team,

88 Source: Litmus Evaluation Team,

89 Source: Litmus Evaluation Team,

90 Updated causal pathway and intervention point model (Ministry of Health, 2013) PM s target: reduce by two-thirds by 2017 Disease pathway Poor environmental and economic conditions Strep A infection Throat infection with Group A Streptococcus bacteria Rheumatic fever (180+ cases/yr) Auto-immune response in 0.3-3% of untreated Strep A infection Recurrence of rheumatic fever Rheumatic heart disease Objectives Work Reduce crowding in housing People know how to reduce their own risks Primordial prevention Housing programmes where eligibility includes RF risk factors Guidance to those contacting at-risk people about when and how to refer them to housing programmes Targeted communications on how people can reduce their risks through healthy communal living habits (safely wash hands, cough/sneeze, sleep) Hygiene facilities & practices in schools Targeted testing Diagnosis and quick action & treatment through any contact with any health service People know how to and when to seek treatment Treat Strep A infections quickly and effectively Throat swabbing in targeted schools and communities Develop alternative delivery models for testing Evaluate and assess different testing and targeting strategies Guidance to health and social professionals who work with at-risk people about how to identify those at risk and what action to take Remove barriers to accessing testing & treatment for those not in target groups Targeted communications about how and when people seek treatment Patients take preventative treatments including monthly antibiotics Better diagnosis & reporting of RF Preventative interventions for their whānau Prevent recurring RF Guidance to health professionals about how to diagnose and treat RF, including prevention for the whānau Use RF case histories to discover risk factors, early warning signs, and assess system performance Active RF patient management systems Targeted communications, incentives and remove barriers to encourage RF patients to take treatments Rheumatic fever and rheumatic heart disease are treated elsewhere in health sector but not as part of this programme Governance Research & evaluation Communications Cross Programme work Ministerial & other reporting Purchasing/commissioning Source: Ministry of Health,

91 Appendix 3: Evaluation team Litmus coordinated the following multi-disciplinary team to undertake this evaluation. Operational Group: Project leader: Michele Grigg, Litmus Principal Consultant Evaluators: Kiri Milne, Litmus Principal Consultant; Ingrid McDuff, Litmus Senior Consultant; Dr Susanna Kelly, Litmus Senior Consultant; Barbara Young, Litmus contractor Māori evaluators: Lisa Davies, Director; Neralee Mahuika, Toni Roberts, contractors; Kaipuke Consulting Pacific evaluator: Catherine Poutasi, Integrity Professionals. Expert Advisors: Project advisor: Liz Smith, Litmus Partner Māori and public health advisor: Dr Ricci Harris, contractor Pacific health advisor: Catherine Poutasi, Integrity Professionals Public health medicine advisor: Dr Deborah Read, contractor Child health, primary care, and disease control advisor: Dr Nikki Turner, Conectus Centre and Immunisation Advisory Centre, University of Auckland. 91

92 Appendix 4: Evaluation methodology A4.1 Document and literature review Litmus undertook a desk review of documentation held by the Ministry relevant to the design, development and planned implementation of the RFPP. The purpose of the document review was to understand the context in which the RFPP was funded and developed, to gain familiarity with key documents that informed programme design and implementation planning, and to seek information on intended implementation of RFPP activity areas and work streams, including timelines, resource, personnel, and performance expectations, and expected structures, processes and delivery models. A brief review of published literature was also undertaken, looking at best practice approaches for rheumatic fever prevention. The brief literature review, in conjunction with feedback from key experts, informed review and validation of the indicative RFPP logic. Relevant documents and literature were identified in consultation with the Ministry during project inception. Other relevant documents were identified through the key informant interviews. A4.2 Key informant interviews Litmus undertook interviews with 17 key informants. These were either strategic nationallevel individuals who are (or were) closely involved in the development, design and early implementation of the RFPP, or experts in the field of rheumatic fever prevention. The purpose of the key informant interviews was to: provide a deeper understanding of the logic underpinning the RFPP and how implementation was intended to take place across the four activity areas. provide an initial, high-level gauge of how implementation is going, including views on progress towards short and medium-term outcomes. seek expert opinion on best practice approaches to rheumatic fever prevention. The interviews, in conjunction with the document and literature review, also informed review and validation of the indicative RFPP logic. Interviews were held with: Ministry personnel (including members of the Communicable Disease Team). representatives from the Ministry s Rheumatic Fever Governance Group (including a general practitioner, the National Heart Foundation and Pacific Heartbeat). RFPP National Coordination Service (Ngati Hine Trust). Health Promotion Agency (previously the Health Sponsorship Council). academics with expertise in rheumatic fever prevention (Table A4.1). 92

93 Table A4.1: Key informant participants Organisation Role(s) Number Ministry of Health Representatives from Rheumatic Fever Governance Group RFPP National Coordination Service Health Promotion Agency University of Otago (Wellington School of Medicine) Manager, Communicable Diseases Team RFPP Project Manager Principal Advisor, Communicable Diseases Senior Advisor, Communicable Diseases Chief Advisor, Population Health Deputy Director, Public Health Chief Advisor, Child and Youth Health Former Senior Advisor, Child and Youth Health Former RFPP Project Manager (employee of Allen and Clarke Ltd) General Practitioner National Heart Foundation Pacific Heartbeat Manager, Ngati Hine Health Trust National Coordination, RFPP Programme Manager Senior Project Manager Rheumatic fever/communicable diseases academic Total Recruitment of key informants was undertaken by Litmus directly with participants, following an introduction of the evaluation by the Ministry s project manager. Participants were provided with an information sheet about the evaluation prior to the interview (all tools are in Appendix 8.1) and gave informed consent for the interview. Interviews were conducted in July/August All but three participants were interviewed face-to-face in Wellington. Representatives from the National Heart Foundation and the RFPP National Coordination Service were interviewed over the phone. Some interviews were conducted with two key informants at the same time. Senior evaluators facilitated the interviews using a semi-structured interview guide which was approved by the Ministry. The guide was designed to gather responses to relevant evaluation questions, while also allowing for flexibility for additional comments of relevance to the project. Interviews were audio-recorded, with participants permission, to assist with analysis and reporting. Discussions lasted for up to 90 minutes. 93

94 A4.3 Data review The purpose of this proposed activity was to provide an assessment of the extent to which children in high-risk communities and their families/whānau are accessing RFPP activities. The intention was to analyse secondary data relevant to the RFPP as a whole, and each regional area using regular monitoring data supplied to the Ministry by local-level providers as part of their contracts for services. An assessment of early supplied data indicated variable quality in reporting standards. As a result, Litmus was limited in its ability to analyse and draw any conclusions from the supplied data. The Ministry continues to work with providers to refine and streamline data collection and reporting requirements. A4.4 Telephone interviews with local RFPP contract holders Semi-structured interviews were conducted by telephone with a small number of stakeholders from the main contract holder in seven of the eight RFPP areas. 30 The aim of the interview was to assist firstly with understanding delivery of the RFPP in the seven areas and secondly, the appropriate selection of case sites to visit during the next phase of the evaluation. The overall purpose of these interviews was to understand the intended regional-level implementation processes for initial implementation of the RFPP and alignment against actual implementation processes. Specifically, the interviews sought to identify and understand: the range of RFPP delivery pathways used (including innovative practices). coordination, management, and contractual arrangements. perceived effectiveness of implementation and service delivery. enablers and barriers to access and uptake. wider environmental influences in implementing the RFPP. training requirements/workforce capacity. monitoring and information management systems. Litmus undertook interviews with 21 local participants (Table A4.2). These included personnel involved in the strategic and operational oversight of the local RFPP. Recruitment of participants was undertaken by Litmus directly with participants, following an introduction of the evaluation by the Ministry s project manager. 30 As noted earlier, Waikato providers were not interviewed as contracts were still being negotiated with the Ministry at the time of the telephone interviews. Another site, Kaitaia, did not wish to participate in the telephone interviews for this stage of the evaluation. 94

95 Table A4.2: Local telephone interview participants Role(s) Number of interviews Chief Executive Officer 3 Clinical Leader/Coordinator/Director 5 Programme Lead/Manager 4 Prevention nurses/kaiāwhina 3 Programme Coordinator/Project Leader 2 Clinical Nurse Manager 1 Clinical and Quality Manager 1 Services Manager 1 Medical Officer of Health 1 Total 21 Participants were provided with an information sheet about the evaluation prior to the interview (all tools are in Appendix 8.2). Informed consent was sought. Interviews were conducted by telephone in August/September Some interviews were conducted in pairs or mini-groups of three. Senior evaluators facilitated the interviews using a semistructured interview guide which was approved by the Ministry. The guide was prepopulated with background information relevant to each locality and provider and also allowed for flexibility in gaining comments of relevance to the project. Interviews were audio-recorded, with participants permission, to assist with analysis and reporting. Discussions lasted for minutes. A Immersion visits The purpose of the 360 immersion visits was to build on the broad understanding of issues generated through earlier data collection, with a more in-depth and focused exploration of implementation at a local level, recognising the potentially different approaches and delivery approaches in the eight Initiative areas. We undertook four 360 immersion visits which involved two to four of the evaluation team visiting each site for up to three days. Two days were spent interviewing stakeholders involved or engaged in the delivery of the RFPP. One day was spent interviewing parents/ caregivers/whānau to determine their experiences of the RFPP. Visits were conducted in October and November Māori and Pākehā evaluators visited each site, and a Pacific evaluator also visited two of the four sites with Pacific communities. Interviews with local stakeholders and parents/caregivers were ethnicitymatched wherever possible. Selection of the four sites was undertaken at the completion of the telephone interviews with local contract holders and in consultation with the Ministry. Sites were selected taking a number of factors into account, including: delivery model (community-based/schoolbased), location (urban/provincial), length of operation, ethnicity mix of community (Pacific focus and/or Māori focus), contracting arrangement (sole contract holder, sub-contracts with multiple providers), and provider type (iwi provider, PHO-led, DHB involvement). Porirua was visited as the evaluation pilot site to test and refine the method and tools before extending the fieldwork to the other sites. Some small revisions were made to the tools following the Porirua site visit. 95

96 A4.5.1 Qualitative local stakeholder interviews Interviews with local stakeholders assisted with understanding the context in which the RFPP is being delivered at the local level, and determining progress towards short and medium-term RFPP outcomes in early implementation stages. The local provider and stakeholder sample was confirmed following discussions with contract managers and key staff during previous data collection, and with the Ministry. A tailored purposive sampling frame was developed for each area, based around local implementation of the RFPP from the local-level provider/funder staff, through to subcontractors, swabbing, health promotion, administration, data collection/monitoring, testing, referral, and follow up of the participating children. In general, interviews and mini-group discussions were held with the follow groups: Local contract holder CEO/Director, clinical lead, nurse/kaiāwhina Local sub-contracted providers PHOs, large GP clinics DHBs Planning and funding portfolio managers, nursing managers, outreach/public Health Nurses (PHNs), ward nurses, health promotion staff, Medical Officers of Health School principals/staff PHO primary care leads General practices GPs, practice nurses Laboratory staff Pharmacists. A total of 65 interviews were held with stakeholders across the four case sites (Table A4.3). Table A4.3: Cases selected for site visits Sites Contract/delivery model Number of interviews Whāngarei South Auckland Direct contract to MOH; 6 schools; 3 days, PHN and community worker swabbing; standing orders; parents collect antibiotics from school PHO contract with MOH, sub-contract to 5 providers; 18 schools; 5 days; range of frontline swabbing staff; standing orders; home visits; existing programme 9 17 Direct contract to MOH; 6 schools; ad hoc swabbing by 18 Rotorua nurse; no standing orders; CAR emphasis DHB contract with MOH, sub-contracts PHO and RPH; 21 Porirua 9 schools, 3 days; PHN swabbing; standing orders; kaiāwhina home visits; existing programme TOTAL 65 Note: Correct at the time of the evaluation case visits. The primary focus of the discussions was on understanding progress with implementation of the RFPP: What is working well with local implementation? What isn t? Lessons learned? How important is the RFPP to the local community? What are the views on ongoing implementation? How are the relevant target groups accessing the RFPP? What awareness raising activities are undertaken and what is the response to these from the community and health professionals? What are the information needs and availability? 96

97 What enhancements are required? Overall, how sustainable is the RFPP? Participants were given an information sheet about the evaluation prior to the interview (all tools are in Appendix 8.3), and gave informed consent. Interviews were conducted during the site visit, and some were held in pairs or mini-groups at participant request. A small number of stakeholders were unavailable during the visit dates, and were interviewed by phone where possible. Senior evaluators facilitated the interviews using a semi-structured interview guide which was approved by the Ministry. The guide ensured that key areas of investigation were addressed, and also allowed for flexibility in gaining comments of relevance to the project. Interviews were audio-recorded, with participants permission, to assist with analysis and reporting. Discussions lasted for minutes. Koha in the form of morning/afternoon tea was provided for many of the group discussions. A4.5.2 Parent/caregiver/whānau discussion groups Discussions with parents/caregivers/whānau who may have been exposed to local RFPP activities provided an additional perspective about local delivery of the RFPP. The evaluation team spoke with a total of 63 parents/caregivers in 12 discussion groups (three in each case site). Participants were parents/caregivers/whānau 31 whose child(ren) had the potential opportunity to participate in some way in the local RFPP (eg, a child in their family aged 5-14 years had been tested and/or referred as part of the RFPP, the school their child(ren) attends is participating in the RFPP, they have been tested as a household contact, etc). An appropriate purposive sampling framework was confirmed following discussions with the Ministry following selection of the four case site areas, to ensure it reflected RFPP delivery in each visited locality. The sample frame therefore differed slightly across each of the four areas to reflect the key target population(s) in each (ie, some localities had small numbers of Pacific residents). The sample frame recognises that Māori and Pacific children are a priority for the RFPP. The final sample frame for the discussion groups is shown in Table A4.4. Table A4.4: Sample frame for parent/caregiver/whānau discussion groups Location Parent/caregiver/family/whānau member ethnicity Māori Pacific Any ethnicity* Whāngarei 2-1 South Auckland Rotorua 2-1 Porirua Total Total participants * These were mixed ethnicity groups. 31 Defined for the purposes of this evaluation as individuals who regularly look after one or more children aged 5-14 years, for significant amounts of time and make decisions about their child s health. This included parents, grandparents, aunties, and wider whānau. 97

98 Litmus recruited parents/caregivers/whānau through a combination of local community connectors (sometimes identified through the local provider) and a professional recruitment agency. The groups were comprised of five participants on average, which enabled interactive discussion between participants and the facilitator. An informed consent process was used and a range of tools was developed for the recruitment process (all tools are in Appendix 8.4). Litmus developed a semi-structured qualitative discussion guide to ensure key areas of investigation were addressed. Guides reflected the evaluation questions as confirmed in the evaluation plan. The groups were conducted in a qualitative manner ie, as a free-flowing discussion, rather than a survey. Discussion groups with Māori parents/caregivers/whānau were conducted by our Māori research partners (Kaipuke Consultants) and groups with Pacific were conducted by our Pacific research partner (Integrity Professionals). The groups lasted 60 minutes and were conducted in local community venues. Participants received a $50 koha in recognition of their time and contribution. A4.6 Survey of parents/caregivers A survey of parents/caregivers of 5-14 year olds enabled collection of a baseline measure of awareness of different aspects of the RFPP from both RFPP participants and nonparticipants. Specifically, it enabled collection of a baseline measure of levels of awareness of the following: Sore throats and rheumatic fever Risk factors for rheumatic fever Prevention and treatment of sore throats The school throat swabbing service, and experience with the service. Interviews with 407 parents/caregivers of 5-14 year olds were undertaken using a school gate intercept method in each of the four sites visited. This method was selected as the most cost-efficient way to find a high probability of eligible parents/caregivers of schoolaged children. The local provider at each site was asked to recommend schools where the throat swabbing service had been implemented. Three schools in each of the four sites were identified and contacted for permission to conduct the survey outside their school gate. The survey was administered at three schools in each of the four sites, over a period of between two and five days. The initial approach was made via a letter from the Ministry of Health (all tools are in Appendix 8.5). This was followed up by a phone call to answer any questions, and provide a suggested information paragraph for the school to use in their newsletter or on noticeboards. At this stage, one of the nine schools approached declined to participate. A replacement school was selected in that area, also in consultation with the local provider. Table A4.5 outlines the achieved sample and the response rates per area. At least 100 interviews were conducted per site. 98

99 Table A4.5: Achieved sample frame for school intercept survey Location Schools Achieved interviews per school Response rates Whāngarei South Auckland Rotorua Porirua Te Kura o Otangarei Hora Hora Primary Whau Valley Primary Rongomai Primary East Tamaki Primary Ferguson Intermediate Western Heights Primary Aorangi Primary Sunset Primary Windley Corinna Primary Russell 21 Total % Participating schools were either decile 1 (n=8) or 2 (n=4) and comprised a majority roll of either Māori and/or Pacific students (75-100%). Survey quotas for each school were set to reflect the school roll size proportionate to the total roll of the three schools in each area. Litmus engaged the services of a professional survey company to conduct the survey. Litmus developed the structured survey questionnaire and pre-tested the questionnaire before this was signed off by the Ministry. The survey was designed to be no longer than five minutes long. Participants received a $5 koha as a thank you for their time, and each participating school received a school koha of $170. Litmus evaluators attended the interviewer briefing. An informed consent process was used with parents/caregivers, following explanation of the survey (but ensuring no bias was introduced to their responses). A range of tools was developed for the survey (see Appendix 8.5). The survey was conducted over ten days between 29 October and 8 November % 70% 80% 58% 99

100 Appendix 5: Evaluation questions and data sources Phase One Data Collection Phase Two Data Collection Phase Three Data Collection Key Evaluation Objectives 1. To understand and refine the overall logic or theory of change underpinning the RFPP - What activities are expected to achieve which outcomes, and how? - How will we know when outcomes have been achieved? (eg, What do strong and sustainable partnerships look like? What makes use of financial resources effective and efficient? How do we know when resources have been equitably distributed?) - Does the indicative logic reflect stakeholder understanding of the RFPP theory of change? - Does the indicative logic reflect international evidence for rheumatic fever prevention programmes? - How can the RFPP logic be enhanced (if at all) to better reflect international evidence of what works, mechanisms for change, and clear and measurable outcome indicators? 2. To understand the intended implementation processes for the first 18 months of the RFPP - How were the four activity areas (partners, resources, management and initiatives) expected to be implemented in the first 18 months? - Who was expected to be involved? In what capacity? Key informant interviews Document review National stakeholder interviews Review of national and provider datasets Survey of DHB contract managers Immersion visits to four case sites Benchmark survey 100

101 Phase One Data Collection Phase Two Data Collection Phase Three Data Collection Key Evaluation Objectives - What resources were required? - What structures, processes and delivery models were intended to be established and used? - What was the timeline? - What was expected progress towards short and medium-term outcomes? 3. To determine the extent to which actual implementation of RFPP work streams have aligned with intended implementation processes in the first 18 months, and understand reasons for any differences - How have the four activity areas been implemented? (personnel, resources, structures, processes, delivery models, timing) - What s different from intended implementation processes? Why? Are these differences perceived to be appropriate? Key informant interviews Document review National stakeholder interviews Review of national and provider datasets Survey of DHB contract managers Immersion visits to four case sites Benchmark survey 4. To assess progress towards RFPP short and medium-term outcomes, and identify any unintended outcomes - How effective are programme governance and management structures? - How effectively and efficiently is the funding being allocated and used? - How effective are programme training /upskilling activities? 101

102 Phase One Data Collection Phase Two Data Collection Phase Three Data Collection Key Evaluation Objectives - How effectively have the various RFPP initiatives been implemented?(local coordination and school-based sore throat services, community and health sector awareness raising activities, monitoring and evaluation, surveillance and research activities) - How effectively are the various RFPP initiatives operating, both separately and as a whole? - To what extent are children in high risk communities and their families/whānau accessing the programme activities? Who is accessing it and who is not? Why? - To what extent has community and health sector awareness of the following been raised: rheumatic fever risk factors; the signs and symptoms of GAS infection and acute rheumatic fever; how to prevent rheumatic fever - To what extent do the community and the health sector have access to rheumatic fever prevention services, tools, and resources on GAS infection and acute rheumatic fever? - Are there any unintended or unexpected effects occurring as a result of the programme? 5. To identify key strengths and weakness of the first 18 months of RFPP implementation, and identify ways to enhance ongoing implementation of the RFPP Key informant interviews Document review National stakeholder interviews Review of national and provider datasets Survey of DHB contract managers Immersion visits to four case sites Benchmark survey 102

103 Appendix 6: School intercept survey summary findings A6.1 Survey methodology and sample description As described in the evaluation methodology section (Appendix 4), the school intercept survey was conducted between 29 October and 8 November Parents/caregivers were asked to complete a short questionnaire (interviewer administered) outside/around the school gate at 12 schools in total (three schools at each of the four case sites). Participants received a $5 koha for their time, and the school received a $170 donation for taking part. The overall response rate for the survey was 71% (Whāngarei 82%, South Auckland 70%, Rotorua 80%, and Porirua 58%). A description of the sample demographics is included in Table A6.1 below. Table A6.1: Sample description, school intercept survey n= % Gender Female % Male % Caregiver type Mother/father % Other caregiver 78 19% Number of children One % Two 95 23% Three or more 97 24% Parent ethnicity - prioritised Māori % Pacific % Pākehā/Other 99 24% Region Whāngarei % South Auckland % Rotorua % Porirua % TOTAL 407 A6.1.1Notes on the results and interpretation Ethnicity respondents were invited to self-identify with as many ethnic groups as they wanted to, and were prioritised during data coding into one of Māori, Pacific, Asian, Other, or NZ European/Pākehā (in that order of prioritisation, using the Statistics New Zealand standard ethnicity question). The latter three categories were then combined during analysis into a single Pākehā/Other category. Ethnicity participants were grouped and analysed as above, using parent ethnicity. The ethnicity of their child(ren) was not collected and results cannot therefore be used to illustrate outcomes for children by ethnicity. 103

104 Differences between proportions was tested to a 95% confidence level, and only significant differences (at the 5% level) have been described between sample subgroups (ie, Māori vs. Pacific, or Māori vs. Pākehā/Other). Questions with an open-ended or other answer category enabled respondents to give more than one response. As such, these are reported as a raw frequency, rather than as a proportion of the total base, and the most common answers are listed in order of the number of times they were mentioned by respondents. Sample bias these results represent the views and experiences of parents who were present at the school to drop off/collect their children on the scheduled survey days, and do not represent all parents/caregivers from the sites included in the survey. A6.2 Survey results A6.2.1 Issues affecting children s health The first survey question was open response, providing an unprompted indicator of awareness and level of priority placed on sore throats and rheumatic fever as child health issues. Over one in ten (12%) mentioned sore throats, strep throat, rheumatic fever or rheumatic heart disease as one of the most important issues affecting the health of children (unprompted). The other health issues that were most commonly mentioned by parents/caregivers included: nutrition/eating (n=130) colds/flu/coughs (n=100) skin problems/eczema (n=69) asthma (n=66) healthy homes/keeping warm (n=50) non-specific bugs/bacteria/infections/viruses (n=48) Other issues of importance that were mentioned included ear infections/hearing, oral health/teeth, head lice, general health, and family relationships/behavioural issues/abuse/neglect/mental health. A6.2.3 Measures to stop sore throats spreading This question was open response, providing an unprompted indicator of awareness of infection control for sore throats. Parents/caregivers were asked what can be done to stop the spread of sore throats. Responses included not sharing water bottles, cups or toothbrushes (37%), covering mouth when coughing or sneezing (27%), washing hands with soap and drying them after coughing or sneezing (27%) or taking medicine (21%). One in twenty mentioned taking a throat swab test (5%), and other ways that parents/caregivers gave to stop sore throats spreading included: keeping the child home from school (n=110) taking them to the doctor (n=58) 104

105 more education/awareness (n=39) basic hygiene (n=32) caring for them or prevention (n=29) keeping them warm (n=22). A6.2.4 Untreated sore throats This question was open response, providing an unprompted indicator of awareness of the link between sore throats and rheumatic fever, and the contagious nature of sore/strep throat. When asked what could happen if children s sore throats weren t treated, a third (33%) mentioned a link to heart damage, heart disease or rheumatic fever. Over one in ten (15%) mentioned that a sore throat can spread to others or others can catch it. Other responses for what can happen if sore throats are not treated included: child gets worse (n=124) child can get another illness, infection or disease (n=88) death (n=22) tonsillitis (n=20) having to go to hospital (n=16). Sub-group analysis Unprompted awareness of a link between untreated sore throats and heart disease/rheumatic fever was significantly higher among those who knew about the school throat swabbing service (38%) than those who did not know about the school throat swabbing service (16%). A6.2.5 Awareness of sore/strep throats This question included specific reference to sore throats, providing a prompted indicator of awareness of sore/strep throat. Almost three-quarters of parents/caregivers (74%) had seen, read or heard something in the last six months about sore throats or something called strep throat. Sub-group analysis Mothers and fathers (77%) reported significantly higher awareness of sore/strep throat than other caregivers (63%). Parents/caregivers in Rotorua, Porirua and Whāngarei reported having seen, read or heard something in the last six months about sore throats or something called strep throat (87, 82 and 78%, respectively), significantly higher than parents/caregivers in South Auckland (50%). Māori and Pākehā/Other parents/caregivers (82%) reported significantly higher awareness of sore/strep throat than Pacific (58%) parents/caregivers who participated in the survey. NB: The ethnic distribution across different sites may confound results, where lower awareness in South Auckland could be due to higher Pacific population in this area (who reported lower awareness), compared to other sites (especially Lakes and Whāngarei). 105

106 A6.2.6 Awareness of throat swabbing service at school This question included specific reference to the school throat swabbing service, providing a prompted indicator of awareness of the service. Six out of ten parents/caregivers (60%) had heard about the throat swabbing service being conducted at the school where the survey was taking place (Figure A7.1). Sub-group analysis Pākehā/Other (72%) and Māori awareness of school throat swabbing service (62%) was significantly higher than Pacific (50%) parent/caregiver awareness of the service (Figure A7.2). Awareness of the school throat swabbing service was significantly higher in Whāngarei (74%) and Porirua (70%) than South Auckland (59%) and Rotorua (55%) (Figure A7.3). A significantly higher proportion of mothers and fathers (63%) knew about the school throat swabbing service than other caregivers (47%). Accordingly, significantly more female respondents (68%) had heard of the school throat swabbing service than male respondents (37%). Figure A6.1: Awareness of sore throats and school throat swabbing service, intercept survey 106

107 Figure A6.2: Awareness of sore throats and school throat swabbing service, by ethnicity, intercept survey NB: Significant difference between Māori and Pacific groups, and between Pākehā/Other and Pacific groups. Figure A6.3: Awareness of school throat swabbing service, by region, intercept survey NB: Significant difference between Whāngarei and South Auckland, and between Whāngarei and Rotorua. Also significant difference between Porirua and South Auckland, and between Porirua and Rotorua. Of those who had heard about sore/strep throat, the sources of information were the school (68%), their practice nurse/gp (37%), advertising or information in the community (23%) or another source (30%) (Figure A6.4). 107

108 Figure A6.4: Sources of information about sore throats, intercept survey A6.2.2 Usual response/action if child has sore throat This question was open response, providing an unprompted indicator of usual actions taken when a child has a sore throat. When asked what they would do if their child(ren) had a sore throat, almost all parents/caregivers said they would take them to a doctor/nurse (96%). Alternative actions included giving the child medicine, Pamol or paracetamol (26%) and/or giving them hot drinks, something to gargle or ice blocks (10%). Other actions that parents/caregivers took if their child had a sore throat included: keeping the child warm, home from school, away from other children (n=26) monitoring or checking on them (n=23) using traditional medicine (n=11) giving the child lots of water and healthy food (n=10). Sub-group analysis No significant differences were found between site, ethnicity or caregiver type sub-group for mentioning that they would take their child to the doctor, or give them medicine if they had a sore throat. A6.2.7 Parent/caregiver reporting of child(ren) s experience of throat swabbing at school This question included specific reference to a child having had their throat swabbed, providing a prompted indicator of child(ren) s experience of throat swabbing at school. It was only asked of those parents who were aware of the service (n=245). 108

109 Of those parents/caregivers who were aware of the service, six out of ten (59%) said that child(ren) from their family/whānau had had their throat swabbed at the school. Sub-group analysis Parent/caregiver reporting of child(ren) s experience of throat swabbing as part of the school service was significantly higher in Whāngarei (81%) than the other three sites (Porirua 66%; South Auckland 55%; Rotorua 25%). A6.2.8 Outcome of school throat swabbing This question included specific reference to a child needing to take antibiotics after their throat swab test, providing a proxy indicator for a GAS positive result. It was only asked of those parents who were aware of the service (n=245). Of those whose child had a throat swab taken, over a quarter (28%) needed antibiotics after the throat swab test (Figure A6.5). Figure A6.5: Experience of children with school throat swabbing service, intercept survey Of those whose parents/caregivers whose children needed antibiotics after having a throat swab test, almost all (95%) said that their child had taken the antibiotics for the full ten days. Sub-group analyses were not appropriate for the final survey questions described above as questionnaire branching meant that base sample sizes were too small. 109

110 Appendix 7: National (HPA-produced) and example local rheumatic fever pamphlets 110

111 111

112 112

113 113

114 Rheumatic fever pamphlet 114

115 115

116 Appendix 8: Evaluation tools A8.1 Key informant interview tools Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Information Sheet Key Informants Thank you for your interest in this evaluation. Please read this information before deciding whether or not to take part in an interview. If you decide to take part, we thank you. If you decide not to take part, thank you for taking the time to consider our request. What is the purpose of the project? Why have you asked me to participate? What s involved? What types of questions will you ask? Do I have to take part? Is the interview confidential? Can I change my mind and withdraw from the project? Who is doing the evaluation? What if I have any questions? Litmus (a research and evaluation company) has been asked by the Ministry of Health to find out how things are going in the early stages of implementing the Rheumatic Fever Prevention Programme. We will be speaking with a number of experts in the field of rheumatic fever prevention, along with individuals who have been closely involved in the development of the Rheumatic Fever Prevention Programme. Later in the evaluation we will be talking with local providers and parents/whānau in the community. We d like to interview you for no more than 60 minutes about your role in implementing the Rheumatic Fever Prevention Programme and your thoughts on how implementation is going. There are no right or wrong answers. We will be asking about what is working well and what isn t working so well, in relation to the RFPP. We will also be asking if you have any documentation around approaches to rheumatic fever prevention, and your thoughts on what the programme should be aiming to achieve. You do not have to answer any questions that you feel uncomfortable with. No, your participation is voluntary. Litmus will seek to ensure your contribution is confidential. Comments made in the report will not be attributed to you. Audio recordings and notes from your interview will be kept securely at the Litmus Office or in secure storage for up to two years, and then securely destroyed. You may stop the interview at any time. You can also withdraw the information you have given in the interview up until the time the analysis begins. You do not need to give a reason to withdraw and there will be no disadvantage to you of any kind. The evaluation is being done by Litmus, a research and evaluation company. For more information about Litmus go to If you have any questions about this project, please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz or Michele Grigg, Principal Consultant, Litmus, Ph michele@litmus.co.nz 116

117 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Consent Form Key Informants I (write name) of (write address)..... agree to take part in the early-stage evaluation of Rheumatic Fever Prevention Programme, as outlined in the information provided to me by Litmus (the evaluation company). I understand that: I do not have to take part in the interview. I can choose not to answer any questions I do not wish to answer (without saying why). I can stop the interview at any time without saying why I no longer want to take part. Litmus will keep my information confidential. That is, I will not be named in the final report. I agree to have some of my comments quoted in the report, as long as I am not identified. The interview will be recorded with my permission, and I can ask for the recording to be stopped at any time during the interview. After the interview, I can ask to have my information removed from the evaluation at any time before the report is written. Recordings, notes, and summaries will be stored securely at Litmus and will not identify me by name. They will be kept for two years and then securely destroyed. I have read the information sheet and this consent form, and have been given the opportunity to ask questions and have them answered. I give my consent to participate in this evaluation. Signature: Date: 117

118 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Interview Guide Key informants This guide is indicative of the relevant subject matter to be covered. It is designed to allow freedom to explore any additional relevant topics that may arise during the interview discussion. The questions asked will be tailored to key informants involvement in the development and implementation of the Rheumatic Fever Prevention Programme. Introduction Introduce self/litmus. Evaluation purpose: to undertake an early-stage implementation and formative evaluation of the Rheumatic Fever Prevention Programme (RFPP) to provide insights into the early implementation stages, and inform adjustments or improvements to the ongoing operation of the RFPP. Interview purpose: As a first stage of the evaluation we are interviewing key stakeholders who have been involved in the planning, design and/or implementation of the RFPP, or who are experts in rheumatic fever prevention. We would like to learn about the thinking behind the RFPP s set up, how implementation is going, and views on progress towards short- and medium-term outcomes. All information is confidential. Participant can choose not to answer any questions without any impact, and without needing to give a reason. Information sheet, informed consent and audio recording. Time: maximum 60 minutes (up to 90 minutes if pair/mini-group). Any questions before we begin? 1. Contextual information To begin, can you please briefly describe your role and involvement in the design and/or implementation of the RFPP? - How long have you been involved with the RFPP? 2. RFPP Programme Logic What is your understanding of what the RFPP is trying to achieve? - Short-term? Medium-term? Long-term? - Partners? Resources? Management? Programme Initiatives (prompt: local services; raising awareness; research, surveillance, evaluation, and monitoring?) What is your understanding of how the RFPP will achieve these outcomes? How will we know when the RFPP outcomes have been achieved? - Short-term? Medium-term? Long-term? - Partners? Resources? Management? Programme Initiatives? (prompt: local services; raising awareness; research, surveillance, evaluation, and monitoring?) In your view, are these the right outcomes for the RFPP to be working towards? - Why do you say that? What changes would you wish to see? In your view, are these the right activities to achieve these outcomes? - Why do you say that? What changes would you wish to see? 118

119 To what extent does the RFPP logic reflect best practice for preventing rheumatic fever? - Why? Why not? (Identify literature where relevant) - What would you change? Any other comments on the logic model? 3. Actual vs expected implementation What is your understanding of implementation progress to-date? - Partners? Resources? Management? Programme Initiatives? (prompt: local services; raising awareness; research, surveillance, evaluation, and monitoring?) How has actual implementation of the RFPP varied from intended? If it has varied, what is your understanding of the reasons for this variation? Is implementation of the RFPP on track/where you would expect it to be at this point? - If no, why is that? 4. Current implementation In your view, what s working well with the implementation of the RFPP? - Governance structures? - Management structures? - Funding allocation and use? - Training? - Programme Initiatives? (prompt: local services; raising awareness; research, surveillance, evaluation, and monitoring?) What are the key challenges? What s not working so well? Any unintended consequences arising from the implementation (benefits or challenges)? In hindsight, could anything have been done differently in rolling out the RFPP? What improvements or enhancements are needed going forward over the next 12 months? Two years? For the remainder of the RFPP? 5. Final comments Any other thoughts/comments? Thanks and close. 119

120 A8.2 Provider telephone interview tools Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Information Sheet Local Providers Thank you for your interest in this evaluation. Please read this information before deciding whether or not to take part in an interview. If you decide to take part, we thank you. If you decide not to take part, thank you for taking the time to consider our request. What is the purpose of the project? Why have you asked me to participate? What s involved? What types of questions will you ask? Do I have to take part? Is the interview confidential? Can I change my mind and withdraw from the project? Who is doing the evaluation? What if I have any questions? Litmus (a research and evaluation company) has been asked by the Ministry of Health to find out how things are going in the early stages of implementing the Rheumatic Fever Prevention Programme. We would like to speak with representatives in each of the local providers who the Ministry has contracted to provide throat swabbing services. Later in the evaluation we will be visiting some of the local areas and also talking with parents/whānau in the community. We d like to interview you on the phone for no more than 45 minutes about your role in implementing the Rheumatic Fever Prevention Programme and your thoughts on how implementation is going. There are no right or wrong answers. We will be asking about what is working well and what isn t working so well, in relation to the RFPP. We will also be asking about your thoughts on why the programme is important to your community. You do not have to answer any questions that you feel uncomfortable with. No, your participation is voluntary. Litmus will seek to ensure your contribution is confidential. Comments made in the report will not be attributed to you. Audio recordings and notes from your interview will be kept securely at the Litmus Office or in secure storage for up to two years, and then securely destroyed. You may stop the interview at any time. You can also withdraw the information you have given in the interview up until the time the analysis begins. You do not need to give a reason to withdraw and there will be no disadvantage to you of any kind. The evaluation is being done by Litmus, a research and evaluation company. For more information about Litmus go to If you have any questions about this project, please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz or Michele Grigg, Principal Consultant, Litmus, Ph michele@litmus.co.nz 120

121 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Consent Form Local Providers I (write name) of (write organisation).. agree to take part in the early-stage evaluation of the Rheumatic Fever Prevention Programme, as outlined in the information provided to me by Litmus (the evaluation company). I understand that: I do not have to take part in the interview. I can choose not to answer any questions I do not wish to answer (without saying why). I can stop the interview at any time without saying why I no longer want to take part. Litmus will keep my information confidential. That is, I will not be named in the final report. I agree to have some of my comments quoted in the report, as long as I am not identified. The interview will be recorded with my permission, and I can ask for the recording to be stopped at any time during the interview. After the interview, I can ask to have my information removed from the evaluation at any time before the report is written. Recordings, notes, and summaries will be stored securely at Litmus and will not identify me by name. They will be kept for two years and then securely destroyed. I have read the information sheet and this consent form, and have been given the opportunity to ask questions and have them answered. I give my consent to participate in this evaluation. Signature: Date: 121

122 Introduction Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Introduce self/litmus. Telephone Interview Guide Local providers Evaluation purpose: to undertake an early-stage implementation and formative evaluation of the Rheumatic Fever Prevention Programme (RFPP) to inform the ongoing operation of the RFPP. Interview purpose: We are interviewing by phone representatives from local providers who the Ministry has contracted to provide throat swabbing services. We would like to learn what you are doing, how implementation is going and your views on ongoing implementation. All information is confidential. You can choose not to answer any questions without any impact, and without needing to give a reason. Information sheet, informed consent and audio recording. Time: minutes. Any questions before we begin? Respondent role/involvement in RFPP: Length of involvement: DHB, COMMUNITY AND PROVIDER INFORMATION Pre populate and check DHB type in Location type in Provider name - type in Provider type (e.g. PHO, Māori health provider) type in Contracted to type in Rheumatic fever rates type in Ethnicity information for community type in Local epidemiology of RF type in Other relevant info type in PROGRAMME DELIVERY Pre Populate where possible and check Start date type in Target populations/groups type in Where/locations type in When/frequency/timing type in How/approach type in probe for different roles eg, doctors, nurses, kaiāwhina, labs, schools etc How many/numbers type in 122

123 PROGRAMME INFRASTRUCTURE Pre populate from contracts and check if any differences from intended Contract period type in Management/coordination, incl any sub-contracting arrangements type in Governance arrangements type in Clinical leadership arrangements - type in Other formal local/provider relationships/partnerships type in RFPP Workforce (incl roles, responsibilities, competencies) type in Information management/monitoring/reporting requirements type in LOCAL CONTEXT Why encouraged to apply for RFPP funding/why get involved? What else happening in the area that relates to the RFPP (pre and post RFPP)? Plans for the future beyond the contract? What next? LOCAL IMPLEMENTATION How are things going with the implementation of the RFPP in your community? How is it working? Probe: coverage of schools/target population, school-based vs opportunistic approaches What s working well with the way the RFPP is being implemented in your community? Probe: school-based vs opportunistic approaches What are the key challenges? What s not working so well? Why? Any unintended consequences arising from the implementation (benefits or challenges)? In hindsight, could anything have been done differently in rolling out the RFPP? What improvements or enhancements are needed over the next 12 months? Two years? For the remainder of the RFPP? Once funding for the RFPP ends, how likely is it that there will be a sustained focus on reducing RF in your community? What could be done to ensure sustainability going forward? ANY OTHER THOUGHTS/COMMENTS? 123

124 A8.3 Immersion visit provider qualitative interview tools Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Information Sheet Immersion Visit Thank you for your interest in this evaluation. Please read this information before deciding whether or not to take part in an interview. If you decide to take part, we thank you. If you decide not to take part, thank you for taking the time to consider our request. What is the purpose of the project? Why have you asked me to participate? What s involved? What types of questions will you ask? Do I have to take part? Is the interview confidential? Can I change my mind and withdraw from the project? Who is doing the evaluation? What if I have any questions? Litmus (a research and evaluation company) has been asked by the Ministry of Health to find out how things are going in the early stages of the Ministryfunded Rheumatic Fever Prevention Programme (the throat swabbing programme ). We would like to speak with representatives of local organisations in four areas who are involved in the throat swabbing programme. In each site we are keen to speak with: programme/project coordinators, nurses/kaiāwhina who undertake swabbing in schools, clinical leaders/advisors, school principals/staff/nurses, laboratory staff, general practice staff, health promoters, and others involved in the programme. We will work with local programme personnel to identify the appropriate people to talk with. We will also be talking with parents/whānau in each community to hear their thoughts. We d like to interview you for minutes about your role in the throat swabbing programme and your thoughts about how things are going. There are no right or wrong answers. We will be asking what is working well and what isn t working so well with the RFPP. We will also be asking your thoughts on why the programme is important to your community. You do not have to answer any questions that you feel uncomfortable with. Your participation is completely voluntary. Litmus will seek to ensure your contribution is confidential. Comments made in the report will not be attributed to you. Audio recordings and notes from your interview will be kept securely at the Litmus Office or in secure storage for up to two years, and then securely destroyed. You may stop the interview at any time. You can also withdraw the information you have given in the interview up until the time the analysis begins. You do not need to give a reason to withdraw and there will be no disadvantage to you of any kind. The evaluation is being done by Litmus, a research and evaluation company. For more information about Litmus go to If you have any questions about this project, please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz or Kiri Milne, Principal Consultant, Litmus, Ph kiri@litmus.co.nz 124

125 \ Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Consent Form Immersion Visit I (write name) of (write organisation).. agree to take part in the early-stage evaluation of the Rheumatic Fever Prevention Programme, as outlined in the information provided to me by Litmus (the evaluation company). I understand that: I do not have to take part in the interview. I can choose not to answer any questions I do not wish to answer (without saying why). I can stop the interview at any time without saying why I no longer want to take part. Litmus will keep my information confidential. That is, I will not be named in the final report. I agree to have some of my comments quoted in the report, as long as I am not identified. The interview will be recorded with my permission, and I can ask for the recording to be stopped at any time during the interview. After the interview, I can ask to have my information removed from the evaluation at any time before the analysis begins. Recordings, notes, and summaries will be stored securely at Litmus and will not identify me by name. They will be kept for two years and then securely destroyed. I have read the information sheet and this consent form, and have been given the opportunity to ask questions and have them answered. I give my consent to participate in this evaluation. Signature: Date: 125

126 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Interview Guide Immersion Visit Introduction Introduce self/litmus/kaipuke/integrity Professionals. Evaluation purpose: to undertake an early-stage implementation and formative evaluation of the Rheumatic Fever Prevention Programme (RFPP) to inform the ongoing operation of the RFPP. Interview purpose: We are interviewing representatives from local providers and organisations who are involved in the local Ministry of Health-funded throat swabbing service. We would like to learn about what s happening in your community with the programme and your views on ongoing implementation. All information is confidential. You can choose not to answer any questions without any impact, and without needing to give a reason. Information sheet, informed consent and audio recording. Time: minutes. Any questions before we begin? 1. Programme implementation What is your involvement in the RFPP? Your role? Your responsibilities? When did you become involved? (Ensure reference to Ministry-funded programme, not DHB-funded) Ask role-specific questions see table Can you please describe how the programme works in your area? (To identify differences in models/approaches) What is the programme aiming to achieve locally? How are things progressing towards this? What s working well with the programme s implementation? What s not working so well? To your knowledge, to what extent are children and families/whānau in high risk communities accessing the programme? Who is accessing it and who is not? Why? 2. Awareness raising (ask relevant respondents only) What are you doing to raise the community s awareness of sore throats and rheumatic fever? o o o What resources are you using? (Probe: new HPA resources, local resources, resources from other RF providers in North Island?) What works, what doesn t? Are parents/whānau more aware now? Are children more aware? What are other organisations in your area doing to raise community awareness? (Probe: eg, DHB s, public health unit, others?) Do parents/families/whānau have access to rheumatic fever and sore throat information? Is the information useful? Appropriate? What improvements are needed to support parents to become more aware of sore throats and to participate in the programme? Is anything else needed? 126

127 What are you doing to raise awareness of sore throat management and rheumatic fever among health professionals, particularly in general practices? o o o What resources are they using? (Probe: new HPA resources, local resources, resources from other RF providers in North Island?) What works, what doesn t? Are health professionals more aware now? Who? What are other organisations in your area doing to raise awareness of health professionals? (Probe: eg, DHB s, CME courses for GPs, others?) Do health professionals have access to rheumatic fever and sore throat information? Is the information useful? Appropriate? What improvements are needed to support health professionals to be more aware of sore throats and rheumatic fever? What other tools are needed? Have you heard from/of the Health Promotion Agency around development of resources? What suggestions do you have for other communities/regions about what works for promotions/resources/ awareness raising? From your experience of living and/or working in the community, have you noticed any change in awareness of sore throats or rheumatic fever? In your view, why may that have happened? 3. Improvements, sustainability, lessons Are there any gaps in the rheumatic fever prevention service from your perspective? (Probe: throat swabbing, referral services, awareness raising, information, tools etc) What enhancements could be made to the programme? How could the programme be improved going forward? Are there any unintended or unexpected effects occurring as a result of the programme? What will happen to the programme after the Ministry s funding ends? 4. Other comments Would you like to make any other comments on the implementation of the Ministry-funded rheumatic fever prevention programme? 127

128 Ask questions below as relevant to role/organisation Local provider Please describe your programme s governance and management structures. How effective are these? Is anything operating differently from what was intended? Why? Are these differences appropriate? How effectively is the funding being allocated and used? How are laboratory costs being paid for? Who pays for these/what is the payment arrangement? Can you give us an overview of the types of communities you are working with? What are the realities for them in relation to rheumatic fever? (Probe: overcrowding, housing conditions, poverty, doctor access). Schools How do you identify schools to include? Prioritisation? Please describe liaison with the schools, principals, BOT. How do you access roll lists? Please describe the consents process. Please describe the relationship with schools. What is key? What is your view on the best way to deliver the programme to the communities you work with? Schools? Primary care? Another approach? Linkages and relationships What linkages do you have with school/public health nurses? What links do you have with other RF programmes in the area? How would you describe your relationship with the DHB/public health nurses? Please describe how you work together. How do you maintain this relationship? In what ways could this relationship be improved? How would you describe your relationship with primary care? How do you maintain this relationship? In what ways could this relationship be improved? Programme delivery and support Please describe the process used for follow up for positive swabs, antibiotics etc. What process is used? Are any improvements needed? What is the response from whānau? How many days per week are you doing school swabbing? Does this differ from the contract? What is the rationale for that? What do you do to follow up on kids who are away sick or absent from school on swabbing days? What happens if there are other factors happening in the home that you identify (eg, poor housing)? Are you able to refer to another service? If yes, how does that work? Is there anyone to refer to? Please describe how you assure the quality of your programme s services. (Probe: clinical oversight, documented policies and procedures) Public Health Nurses (if not directly involved in throat swabbing) How does the throat swabbing programme fit with your role? In what way are you involved with the programme? How would you describe your relationship with (the local provider)? Please describe how you work together. How do you maintain this relationship? In what ways could this relationship be improved? What training do you/public health nurses receive on sore throats and rheumatic fever? Is this sufficient? Could it be improved? What information do you have available for parents/whānau about sore throats and preventing rheumatic fever? Are these resources widely available? What role do you see public health nurses having in the programme in the medium to long term? 128

129 What advice do you have for others about swabbing in schools? What are your tips for getting children involved? (without them seeing it as a chance to wag class) Please tell me about data/information collection and reporting. How much do you have to do with the National Coordination Service (Helen Herbert)? What support do you receive? How useful is that? How else could NCS support you? What training or up-skilling do you receive? How effective are these activities? Please tell me about the information and resources you have available for parents and schools. What information is provided? Is it appropriate? Schools Please describe the general make up of your school? (decile, ethnicity etc) Who from the school is responsible for coordinating the swabbing? How supportive are the BOT/school principal of the programme? Can you please describe how the throat swabbing programme works in your school? What is the role of the school? Of the staff? What do you see as the main benefits of having the swabbing programme in your school? Are there any negative aspects associated with having the programme in your school? How could the process be improved to make it easier for schools? For students? For staff? What information do you have available for parents/whānau about sore throats and preventing rheumatic fever? Are these resources widely available? Practice Nurses/staff, including GPs How does the throat swabbing programme fit with your role? In what way is your practice involved? Can you please describe how the programme works in your practice? How would you describe your relationship with (the local provider)? How do you maintain this relationship? In what ways could this relationship be improved? What level of knowledge do general practice staff have about the risk factors and importance of rheumatic fever? What training do you/general practice staff receive on sore throats and rheumatic fever? Is this sufficient? Could it be improved? What information do you have available for parents/whānau about sore throats and preventing rheumatic fever? Are these resources widely available? What role do you see practices having in the programme in the medium to long term? Have you noticed any difference in patterns of people coming to the clinic with sore throats? Has general community awareness of sore throats increased? 129

130 A8.4 Immersion visit - parent/caregiver/whānau discussion group tools Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Information Sheet Parents/Whānau/Caregivers Discussion Groups Thank you for your interest in this evaluation. Please read this information before deciding whether or not to take part in an interview. If you decide to take part, we thank you. If you decide not to take part, thank you for taking the time to consider our request. What is the purpose of the project? Why have you asked me to take part? What s involved? What types of questions will you ask? Do I have to take part? What are the benefits to me? Is the discussion confidential? Can I change my mind and withdraw from the project? Who is doing the evaluation? What if I have any questions? Litmus (an evaluation company) has been asked by the Ministry of Health to find out how things are going in the early stages of the Rheumatic Fever Prevention Programme (the throat swabbing programme ). We would like to speak with parents, whānau and caregivers of children aged 5 14 years who may have taken part in the local throat swabbing programme. We are visiting four areas where the school throat swabbing programme is running. We would like to hear your thoughts about how the throat swabbing programme is going and how it could be improved. We d like to hold a small group discussion with a few other parents/whānau. This will last about 60 minutes. There are no right or wrong answers. We will be asking what is working well and what isn t working so well with the programme. We will also be asking how it could be improved. You do not have to answer any questions that you feel uncomfortable with. Your participation is completely voluntary. Your involvement will help the Ministry of Health improve the throat swabbing programme in your community, and in other areas. You will receive $50 as a thank you for your time and contribution. Litmus will ensure your contribution is confidential. Comments made in the report will not have your name next to them. Audio recordings and notes from the discussion will be kept securely at Litmus or in secure storage for up to two years, and then destroyed. You can leave the group discussion at any time. You can also withdraw the information you provide up until the time analysis begins. You do not need to give a reason to withdraw and there will be no disadvantage to you. The evaluation is being done by Litmus, a research and evaluation company. For more information about Litmus go to If you have any questions about this project, please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz or Michele Grigg, Principal Consultant, Litmus, Ph michele@litmus.co.nz 130

131 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Consent Form Parents/Whānau/Caregivers Discussion Groups I (write name) of (write address).. agree to take part in the early-stage evaluation of the Rheumatic Fever Prevention Programme, as outlined in the information provided to me by Litmus (the evaluation company). I understand that: I do not have to take part in the group discussion. I can leave the group discussion at any time. I can choose not to answer any question (without saying why). I can withdraw from the evaluation and ask for my information to be destroyed at any time before the analysis begins. The group discussion will be recorded and the recording will be used only for analysis purposes for this evaluation. Notes, audio recordings or reports written by Litmus will not identify me. This signed agreement will be held securely at Litmus office. I have read the information sheet and this consent form, and have been given the opportunity to ask questions. I give my consent to participate in this group discussion. Signature: Date: 131

132 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Discussion Guide Parents/Whānau/Caregivers Groups This guide is indicative of the relevant subject matter to be covered. It is designed to allow freedom to explore any additional relevant topics that may arise during the discussion. Introductions (10 mins) Thanks for coming. Introduce self/litmus/kaipuke/integrity Professionals. Evaluation purpose: to undertake an early-stage evaluation of the Rheumatic Fever Prevention Programme (RFPP) the throat swabbing programme - to inform the ongoing operation of the Programme. Group discussion purpose: We are holding group discussions with a number of parents/whānau/ caregivers to understand parents/whānau perceptions of sore throats and what to do about them. Information sheet, informed consent and audio recording. Time: 60 minutes. Toilets, fire exits, food, etc. Ground rules: give it a go, respect each other, no right or wrong answers. Any questions before we begin? Round of introductions name, number of children/tamariki/mokopuna, number at primary/ intermediate school. Sore throats in the context of child health and wellbeing (10 mins) So we re here to talk about sore throats. Is looking after sore throats important in keeping our children healthy and well? Why? Why not? How important is it to address sore throats in children? What other things are important for keeping our children (aged 5 14 years) healthy and well? Probe: younger children, older children How important is sore throats compared to all these other things? Knowledge and understanding (10 mins) What do we know about sore throats in children aged 5 14? What kinds of things do we do if our children have a sore throat? Why? Probe: do nothing, go to chemist, go to doctor? If nothing, why? What kinds of things can happen sometimes if a sore throat is left untreated? Probe: reoccurs, rheumatic fever, other? Sore throat and rheumatic fever information and channels (15 mins) What have we heard about sore throats? What have we heard about doing something about sore throats? How important is it to get sore throats checked? Probe: could lead to RF, could be in multiple family members What have we heard about rheumatic fever? Probe: what is it, why is it serious?, link to heart failure later in life o Rheumatic fever is caused by an infection of the throat with something called Group A Streptococcus (GAS), which if left untreated, can lead to permanent heart damage. Rheumatic fever usually first happens in childhood (mainly in children aged 5 14) but it can come back as a teenager and into adulthood. Have we heard anything similar to this about how sore throats can cause rheumatic fever? 132

133 Have we heard anything about the causes of sore throats and rheumatic fever? How do we prevent rheumatic fever? Probe: overcrowding, poor housing, spread of bacteria Where do/did we hear this from? Probe: family/whānau, doctor, school, advertising, others? Was the information useful? Was it easy to understand? Was any information missing? What other information is needed? How else could families/whānau learn or hear about sore throats and rheumatic fever? What s the best way of getting the message out there? Throat swabbing in schools (10 mins) Have any of our children had their throat swabbed at school? How, if at all, did we find out about this? Was it all straightforward? What kind of forms did you have to fill in? Probe: consent forms. How did our children find the experience of having their throat swabbed? What, if anything, did they tell you about it? What information did you receive afterwards? Was there enough information? Was it easy to understand what happens? Who did this come from? Probe: letter, phone call etc from kaiāwhina, nurse, doctor, school, others? o o Did anyone receive antibiotics following the throat swab? Did you understand how to use the antibiotics? Did anyone explain it to you? Do you know if your child finished the antibiotics? Did the nurse/kaiāwhina that visited you provide you with advice about things that contribute to sore throats, like insulation and heating? What did they say? What are the benefits of throat swabbing? Probe: rational and underlying motivations Do we have any worries or concerns? If so, what are these? Probe: rational and underlying barriers What makes it easy/difficult to take part in throat swabbing at schools? Explore information, consent process, etc Could the school throat swabbing programme be made better? In what way? Health services through schools (5 mins) For your whānau, what are the positives and negatives of having a health service like the throat swabbing delivered through schools? Future needs (5 mins) Imagine it is our job to encourage parents/caregivers/whānau to learn more about the importance of looking after sore throats in their children aged What information would we give to whānau? What would we say? What things would make it easier for parents to prevent sore throats or get sore throats checked? What if school throat swabbing wasn t available would we go to the doctor instead? Where else would we go? Where would we prefer throat swabbing to happen? Why? Thanks, koha and close Distribute sore throat/rheumatic fever resources as needed. 133

134 A8.5 Parent/caregiver school intercept survey tools Approach letter to selected schools from Ministry of Health Tēnā koe [Principal s name] The Ministry of Health is conducting an evaluation of the implementation of the Rheumatic Fever Prevention Programme (the Programme). Litmus (an independent evaluation company, based in Wellington) has been commissioned by the Ministry to undertake a review of the early stages of the Programme s roll-out. As part of the evaluation Litmus will be visiting four Programme sites to talk about how things are going. It will be important to find out how much parents/caregivers of children at participating schools know about rheumatic fever, sore throats and the Programme. To achieve this, we would like to undertake a face-to-face school-gate survey of parents and caregivers at your school. A professional interviewing company, Consumer Link, will undertake the school-gate survey on behalf of Litmus. Consumer Links interviewers are all security checked, fully trained in fieldwork and experienced at explaining surveys, and seeking informed consent from survey participants. The interviewers also all wear visible company identification. The survey will take place outside school grounds (e.g. with parents/caregivers at child/ren drop off and pick up times) and on the school grounds (e.g. with parents/caregivers waiting outside classrooms). More information about the project is attached for your reference. The school-gate survey of parents and caregivers will take around five minutes to complete and participation will be entirely voluntary. Participating parents/caregivers will receive $5 koha for their time. As a token of our appreciation Litmus will also donate $170 to each school that agrees to have the survey take place outside and/or in their schools grounds. This letter is to inform you of the evaluation (and specifically the school-gate survey of parents and caregivers) and to request permission for Consumer Link s interviewers to approach parents/caregivers outside and/or at your school. This is likely to happen in late October/early November. Litmus will work with you to ensure that the survey is conducted in a way that works best for your school (e.g. having nominated spots for surveying parents/caregivers). Michele Grigg (Principal Consultant) or a colleague from Litmus will call you early in the start of term four to discuss the possibility of conducting the survey at your school. In the meantime, please feel free to get in touch with Litmus earlier if you have any queries about the survey on or michele@litmus.co.nz. Alternatively, if you have any questions about the purpose of the evaluation itself, please contact me, Marnie Carter (Ministry of Health, Project Manager), on or marnie_carter@moh.govt.nz. If you do give permission for the survey to take place outside and/or on your school grounds, Litmus will provide some information about the parent/caregiver survey that you might wish to place in your school newsletter and/or on school notice boards, to inform parents/caregivers about the survey and that they may be approached by the interviewers. Thank you in advance for considering our request and for continuing to support this important Programme. Yours sincerely Marnie Carter Rheumatic Fever Prevention Programme Project Manager Ministry of Health 134

135 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Information Sheet Schools Thank you for your interest in this evaluation. The following is some additional information for you about the project. What is the purpose of the project? How many schools are taking part in the parent/caregiver school gate survey? Why have you asked my school to be involved? What s involved for parents/caregivers? What s involved for my school? What types of questions will you ask parents/caregivers? Are the surveys with parents/caregivers confidential? Do parents/caregivers have to take part? What s in it for parents? Can parents/caregivers change their mind and withdraw from participating in the survey? Who is doing the Litmus (an independent research and evaluation company) has been asked by the Ministry of Health to find out how things are going in the early stages of implementing the Rheumatic Fever Prevention Programme. This will include identifying things that are working well and things that could be improved upon. We anticipate that a total of 12 schools (3 schools in each of the following areas; South Auckland, Rotorua, Porirua and Whāngarei) will take part. From each area, 100 parent/caregivers will be surveyed. Your school was recommended to Litmus by the local Rheumatic Fever Prevention Programme provider in your area because the programme has been running in your school. Parents/caregivers will be asked by interviewers to take part in a brief five minute survey at the school gate and/or on school premises at selected spots (if school permission is granted to do this). Litmus will ask you to inform parent/caregivers about the survey through your school newsletter and/or on school notice boards. Survey interviewers will be present outside school grounds (to survey parents/caregivers at drop off and pick up times) and/or on the school grounds (to survey parents/caregivers waiting outside classrooms) for approximately 1 hour a day over a 2-3 day period. We will donate $170 to your school to thank you for your support. Parents/caregivers will be asked general questions about sore throats in children, awareness of rheumatic fever risk factors, awareness of signs and symptoms of acute rheumatic fever and how to prevent rheumatic fever. Parents/caregivers will also be asked some basic demographic information (e.g. ethnicity, household composition etc). Parents/caregivers do not have to answer any questions that they feel uncomfortable with. Litmus will ensure the parents/caregivers contribution is confidential. All the data collected from parents/caregivers at your school will be grouped together for analysis purposes. No identifying data will be reported. Data from the survey interviews will be kept securely at the Litmus Office or in secure storage for up to two years, and then securely destroyed. No, their participation is voluntary. Parents who take part in the survey will receive a $5 koha and also an information sheet about Rheumatic Fever. Parents/caregivers may stop the interview at any time. They can also withdraw the information they have given in the interview up until the time the analysis begins. Parents/caregivers do not need to give a reason to withdraw and there will be no disadvantage to them (or the school) of any kind. The overall evaluation is being done by Litmus, a research and evaluation 135

136 evaluation? Would our school be able to receive a summary of the project findings? What if I have any questions about my school s involvement? company. For more information about Litmus go to Yes, Litmus will supply a summary of the project findings to the schools who take part in the survey. If you have any questions about this project please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz If you have any questions about the research aspect of the project please contact: Michele Grigg, Principal Consultant, Litmus, Ph michele@litmus.co.nz Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Suggested text for school newsletter/noticeboards Survey of parents/caregivers sore throats and throat swabbing On [date] we will have [xx] interviewers visiting our school at the end of the school day. They will be interviewing parents and caregivers for an evaluation of the rheumatic fever school throat swabbing programme. You may be approached and asked for an interview this will be entirely voluntary. If you take part, you will receive a koha of $5 for your time. The school is receiving a $170 donation from the Ministry of Health. We encourage you to come along a little earlier on [date] and to take part in an interview with one of the friendly interviewers. It will only take 5-10 minutes to answer a few questions. Information from the survey will be important for making improvements to the throat swabbing programme. 136

137 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Health Survey Information Sheet for Parents/Caregivers/Whānau Thank you for your interest in this survey. Below is some additional information for you. What is the purpose of the survey? How many parents and schools are taking part? Why have you asked me? Litmus (an independent research and evaluation company) has been asked by the Ministry of Health to find out parents/caregivers thoughts about child health. We are talking to 400 parents/caregivers in 12 schools in South Auckland, Rotorua, Porirua and Whāngarei. We are randomly speaking to parents/caregivers of 5 14 year old children as they drop off or pick up their children from school. What s involved? You will be asked a small number of questions. This will take less than 5 minutes. We will give you a $5 koha to thank you. What does the school get? What questions will you ask me? We will donate $170 to your school. We will ask you a few questions about child health. We will also ask a small number of questions about yourself. We will not ask for your name or address or any other personal information. You do not have to answer any questions you don t want to. Is the survey confidential? Do I have to take part? Can I change my mind and withdraw from the survey? Who is doing the project? What if I have any questions? Yes. All the information collected will be grouped together for analysis purposes. No identifying data will be reported. Data from the survey will be kept securely at the Litmus Office or in secure storage for up to two years, and then destroyed. No, the survey is voluntary. You can stop the survey at any time. You can also withdraw the information you have given up until the time analysis begins. You do not need to give a reason and there will be no disadvantage to you (or the school). This project is being done by Litmus, a research and evaluation company. For more information about Litmus go to If you have any questions about this project please contact: Marnie Carter, Project Manager, Rheumatic Fever Prevention Programme, Ministry of Health, Ph marnie_carter@moh.govt.nz If you have any questions about the survey please contact: Ingrid McDuff, Senior Consultant, Litmus, Ph ingrid@litmus.co.nz 137

138 Rheumatic Fever Prevention Programme Implementation and Formative Evaluation Survey of Parents/Caregivers/Whānau - Questionnaire Please note: The actual questionnaire used for the survey was interviewer-administered and used a different format. This version is included for reference, to show the question wording, answer options, coding instructions and questionnaire flow. 138