Midwifery ] (]]]]) ]]] ]]] Contents lists available at SciVerse ScienceDirect. Midwifery. journal homepage:

Transcription

1 Midwifery ] (]]]]) ]]] ]]] Cotets lists available at SciVerse ScieceDirect Midwifery joural homepage: Reproductive health services i Malawi: A evaluatio of a quality improvemet itervetio Barbara J. Rawlis, MPH (Seior Moitorig ad Evaluatio Maager, MCHIP) a, Youg-Mi Kim, Ed D (Seior Moitorig ad Evaluatio Advisor) a, Aleisha M. Rozario, MPH (Moitorig ad Evaluatio Advisor, MCHIP) a, Eva Bazat, DrPH, MPH (Seior Moitorig, Evaluatio ad Research Advisor) a, Tambudzai Rashidi, MSc RH/FP (Chief of Party (MCHIP), Coutry Director) a, Sheila N. Badazi, RN, Midwife, MPH, MLitt (Director Nursig Service) b, Faie Kachale, RN, MS (Deputy Director Reproductive Health Uit) b, Harshad Saghvi, MD (Vice Presidet & Medical Director) a, Ji Wo Noh, PhD, MA, USCPA (MPH/MBA studet at Johs Hopkis) c, a Jhpiego, a affiliate of Johs Hopkis Uiversity, USA b Miistry of Health, Malawi c Johs Hopkis Bloomberg School of Public Health, USA article ifo Article history: Received 21 July 2011 Received i revised form Accepted 5 October 2011 Keywords: Performace improvemet Reproductive health Materal ad child health Impact evaluatio abstract Objective: this study was to evaluate the impact of a quality improvemet iitiative i Malawi o reproductive health service quality ad related outcomes. Desig: (1) post-oly quasi-experimetal desig comparig observed service quality at itervetio ad compariso health facilities, ad (2) a time-series aalysis of service statistics. Settig: sixtee of Malawi s 23 district hospitals, half of which had implemeted the Performace ad Quality Improvemet (PQI) itervetio for reproductive health at the time of the study. Participats: a total of 98 reproductive health-care providers (mostly urse midwives) ad 139 patiets seekig family plaig (FP), ateatal care (ANC), labour ad delivery (L&D), or postatal care (PNC) services. : health facility teams implemeted a performace ad quality improvemet (PQI) itervetio over a 3-year period. Followig a exteral observatioal assessmet of service quality at baselie, facility teams aalysed performace gaps, desiged ad implemeted itervetios to address weakesses, ad coducted quarterly iteral assessmets to assess progress. Facilities qualified for atioal recogitio by complyig with at least 80% of reproductive health cliical stadards durig a exteral verificatio assessmet. Measuremets: key measures iclude facility readiess to provide quality care, observed health-care provider adherece to cliical performace stadards durig service delivery, ad treds i service utilisatio. Fidigs: itervetio facilities were more likely tha compariso facilities to have the eeded ifrastructure, equipmet, supplies, ad systems i place to offer reproductive health services. Observed quality of care was sigificatly higher at itervetio tha compariso facilities for PNC ad FP. Compared with other providers, those at itervetio facilities scored sigificatly higher o cliet assessmet ad diagosis i three service areas, o cliical maagemet ad procedures i two service areas, ad o cousellig i oe service area. Service statistics suggest that the PQI itervetio icreased the umber of Caesarea sectios, but showed o impact o other idicators of service utilisatio ad skilled care. Coclusios: the PQI itervetio showed a positive impact o the quality of reproductive health services. The effects of the itervetio o service utilisatio had likely ot yet bee fully realized, sice oe of the facilities had achieved atioal recogitio before the evaluatio. Staff turover eeds to be reduced to maximise the effectiveess of the itervetio. Implicatios for practice: the PQI itervetio evaluated here offers a effective way to improve the quality of health services i low-resource settigs ad should cotiue to be scaled up i Malawi. & 2011 Elsevier Ltd. All rights reserved. Correspodig author. address: joh@jhsph.edu (J.W. Noh) /$ - see frot matter & 2011 Elsevier Ltd. All rights reserved. doi: /j.midw

2 2 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] Itroductio The majority of Malawi s 13 millio people live i rural areas, where access to good quality health care is limited. Mortality amog childre uder age five has bee steadily decliig ad is o a trajectory to meet the Uited Natios Milleium Developmet Goal 4 to reduce child mortality by two-thirds by Yet materal ad eoatal health idicators are ot improvig as quickly. The materal mortality ratio i Malawi remais amog the highest i the world, at 807 per 100,000 live births (NSO ad UNICEF, 2008). I recet years the proportio of births at health facilities where skilled attedace is available has jumped, movig from 55% i 1992 to 72% i 2010 (NSO ad ICF Macro, 2010). However, the quality of emergecy obstetric care remais poor accordig to a atioal eeds assessmet (Leigh et al., 2008). Although cotraceptive use has icreased steadily sice 1992, the rate of uplaed pregacy also remais persistetly high (NSO ad ICF Macro, 2010). Nurses ad urse midwives play a critical role i savig the lives of mothers ad youg childre i sub-sahara Africa. I Malawi, they comprise the largest of health professioals (AHWO, 2009) ad provide the majority of reproductive health services at the district ad sub-district levels (Picazo ad Martieau, 2004). With few physicias available, they form the backboe of basic services prove to reduce materal ad eoatal mortality, such as family plaig, ateatal ad postatal care, ad skilled attedace at birth (Darmstadt et al., 2005; Campbell ad Graham, 2006). However, a severe shortage of midwives, urses ad other health professioals has crippled the ability of the health-care system i Malawi to adequately address materal ad eoatal mortality ad other pressig health issues (MoH, 2004; Palmer, 2006). I 2004 the Miistry of Health (MoH) lauched a Emergecy Huma Resources Programme (EHRP) to help revitalise the health system (MoH, 2004). It offered fiacial icetives to recruit ad retai health workers, expaded the capacity of pre-service traiig istitutios, ad stregtheed huma resources maagemet (Palmer, 2006; O Neil et al., 2010). A recet evaluatio foud that the EHRP has had a cosiderable impact both o the umber of health workers ad also o their commitmet to remaiig i the professio ad i Malawi. From 2004 to 2009, the umber of health workers grew by 53%. There are ow 37 urses (icludig midwives) per 100,000 populatio i Malawi, compared with 29 per 100,000 populatio i 2004 (O Neil et al., 2010). While the EHRP directly addressed chroic uderstaffig, it did less to overcome deep-seated problems i the work eviromet that udermie the quality of health services i Malawi. Job performace depeds ot oly o good pre-service educatio ad adequate staffig, but also o a host of other factors ragig from the availability of supplies ad equipmet to systems providig for cotiuous learig, supervisio, ad recogitio (Rowe et al., 2005). I their aalysis of efforts to reduce materal mortality, Fauveau ad colleagues (2008) have argued that a focus o icreasig the umber of birth attedats has draw attetio away from the eed to improve the quality of care by stregtheig their proficiecy i key midwifery skills. The rapid iflux of ewly traied workers ito Malawi s health system makes performace issues especially pressig. The quality of traiig ad supervisio offered to ursig ad midwifery studets has suffered because the EHRP failed to overcome some key problems, such as a shortage of tutors ad the difficulty of recruitig qualified studets ito traiig programs (Picazo ad Martieau, 2004; O Neil et al., 2010). Pass rates at the 13 traiig istitutios for midwives ad urses i Malawi are decliig steeply, with less tha half of ursig ad midwifery studets i the 2010 cohort expected to graduate, accordig to a report by the Nursig ad Midwives Coucil of Malawi at the 2010 mid-term SWAP review. Give the weakesses i the educatio system, iexperieced ew urses eterig the workforce are icreasigly i eed of close support, supervisio, ad metorig o the job to raise their skill levels. However, a series of studies have described a dysfuctioal work eviromet i Malawi s health system that reduces providers motivatio ad ability to offer good quality care (Picazo ad Martieau, 2004; Muula ad Maseko, 2005; Bradley ad McAuliffe, 2009; Maafa et al., 2009; McAuliffe et al., 2009). Most health workers lack writte job descriptios ad must cope with shortages of drugs, supplies, ad equipmet. There is iadequate metorship, supervisio, recogitio, ad rewards for service providers. Writte stadards, targets, ad timelies for appraisig performace are also largely missig. Performace ad quality improvemet (PQI) i Malawi The MoH has log recogised the eed to improve provider performace ad service delivery, ad its quality improvemet efforts predate the EHRP. I 2001 the MoH lauched a Performace ad Quality Improvemet (PQI) iitiative to improve ifectio prevetio practices i hospitals. I 2006 the MoH ad the ACCESS Program 1 exteded the PQI iitiative to reproductive health (RH) services i hopes of reducig materal ad ewbor morbidity ad mortality. The PQI RH itervetio has sice bee scaled up to all 23 district hospitals i Malawi ad 33 health cetres i four districts. PQI is based o the Stadards-Based Maagemet ad Recogitio (SBM-R) approach to quality improvemet developed by Jhpiego ad used i over 30 developig coutries worldwide. SBM-R sets evidece-based performace stadards ad the empowers health-care maagers ad providers to assess ad address gaps betwee actual ad desired performace at their facility (Necochea ad Bossemeyer, 2005). The approach is well suited to low-resource settigs like Malawi because it is facilitybased ad focuses o practical solutios, requires little additioal mapower or resources, helps trasfer learig ad motivate health workers, ad results i steady improvemets. I Malawi, a Quality Improvemet Support Team (QIST) at each facility leads the PQI RH itervetio. A exteral team workig i collaboratio with the QIST coducts a baselie assessmet of services. The QIST team members ad additioal hospital persoel aalyse the data, determie the causes of performace gaps, desig itervetios to address gaps, ad coduct quarterly iteral assessmets to assess progress. Results from iteral assessmets are shared across facilities i a collaborative approach, ad progress at idividual facilities is bechmarked agaist baselie measures ad the performace of other facilities. Oce a facility scores 80% o a iteral assessmet, it ca request a exteral verificatio assessmet. A score of at least 80% o that assessmet ears atioal recogitio as a cetre of excellece for reproductive health. The exteral verificatio process is repeated aually to moitor adherece ad istitutioalise good practices. Study ratioale ad objectives Despite the widespread applicatio of quality improvemet approaches i health-care applied i low-resource settigs, relatively 1 The ACCESS Program was fuded by Uited States Agecy for Iteratioal Developmet ad led by Jhpiego; it operated globally from 2004 to 2009 ad i Malawi from 2007 to 2009.

3 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] 3 little rigorous evidece is available o their effectiveess i those settigs. Yet systematic evaluatios suggest that stadards-based itervetios ca improve provider performace ad the quality of care i developig coutries, especially whe skill levels are low to start with (Wagaarachchi et al., 2001; Jamtvedt et al., 2006; Bailey et al., 2010). For example, criteria-based audit, which assesses cliical practices agaist best practice guidelies, has bee successfully used to improve the maagemet of postpartum haemorrhage ad woma-friedly materity care at dozes of health cetres i Malawi (Kogyuy et al., 2009a, 2009b). To add to this evidece base ad provide isights for scalig up the PQI process i Malawi, a evaluatio of the PQI RH itervetio was coducted i The objective was to determie the itervetio s impact o the quality of care ad reproductive health outcomes. At the time of the evaluatio, o facilities had yet reached the recogitio stage of the PQI process. Study hypotheses were as follows: 1. Facility readiess for reproductive health services, i terms of supplies, equipmet, ad ifrastructure, is better at itervetio facilities tha compariso facilities. 2. Observed quality of ateatal care (ANC), family plaig (FP), labour ad delivery (L&D), ad postatal care (PNC) is better at itervetio facilities tha compariso facilities. 3. Reproductive health outcomes are better at itervetio facilities tha compariso facilities. Methods Desig The evaluatio employed two desigs. Firstly, a post-oly quasi-experimetal desig compared observed service quality at facilities that had implemeted the PQI RH itervetio (itervetio ) with facilities that had ot yet implemeted the itervetio (compariso ). Secodly, a time-series aalysis of service statistics examied treds i delivery care, such as the provisio of caesarea sectios ad the use of partographs, ad service utilisatio at itervetio ad compariso facilities. Sample Sixtee district hospitals were selected for the evaluatio. Eight itervetio facilities (six i the souther regio, oe i the orther regio, ad oe i the cetral regio) were radomly selected, by beig draw from a hat, out of the 14 govermet district hospitals that had iitiated the PQI RH itervetio i 2007 or earlier. Eight compariso facilities (four i the cetral regio, two i the orther regio, ad two i the souther regio) were radomly selected from amog the ie govermet district hospitals that had ot yet implemeted the itervetio. All facilities i both study s had previously implemeted the PQI itervetio for ifectio prevetio. Data collectio The evaluatio teams icluded 10 cliical experts who served as PQI Master Traiers ad did ot work at the facilities beig evaluated. They had received traiig o both ifectio prevetio ad RH cliical stadards ad had at least two years of experiece i implemetig the PQI RH stadards. Additioal traiig focused o the assessmet tools ad objective scorig to foster iter-rater reliability. Evaluatio teams visited participatig facilities betwee 29 July 2009 ad 10 August They coducted structured cliical observatios of cliet provider iteractios i four RH service areas: ANC, PNC, FP, ad L&D. Withi each service area, each observatio ivolved a differet provider. Evaluators used the same observatio checklists ad performace stadards that QIST teams used for iteral PQI assessmets. From 16 to 20 stadards were assessed i each service area. For example, ANC stadards icluded offerig a cordial receptio ad treatmet, takig a medical history, providig HIV testig ad cousellig, ad maagig pre-eclampsia. For each stadard, assessors oted whether providers performed as may as 29 specific cliical steps, or verificatio criteria. The assessmet teams also coducted a ivetory of ifrastructure, supplies, ad equipmet i each service area; iterviewed RH providers, ad reviewed facility records ad service statistics. The ivetories employed existig PQI checklists, but a provider iterview guide ad record review form were developed especially for this study. The record review icluded service registers, mothly facility summary reports from the Health Maagemet Iformatio System (HMIS), ad charts for materity patiets. I most cases, service statistics datig back to 2004 were examied. Ethical cosideratios The evaluatio study was approved by the Natioal Health Scieces Research Committee i Malawi ad by the Wester Istitutioal Review Board (WIRB) i the Uited States. Followig the approved study protocol, iformed coset was obtaied from all participatig providers ad cliets whose care was observed. Data aalysis Cliical observatio data were used to calculate a performace score for each PQI stadard. The score equals the umber of verificatio criteria performed for each stadard ad ca also be expressed as the percetage of criteria achieved. The uit of aalysis was the cliet provider observatio, ad the sample size for each service area raged from 32 to 38 observatios. A summary score was created for each of the four service areas by totallig the scores for all verificatio criteria for each PQI stadard withi that service area ad calculatig the mea. A further aalysis divided the stadards for each service area ito three types of cliical tasks: cousellig, cliet assessmet ad diagosis, ad cliical maagemet ad procedures. Each set of tasks comprised 2 11 stadards. For each service area, the percetage of verificatio criteria achieved i each skill set was calculated. The evaluatio sought to compare results betwee two s: itervetio ad compariso sites. The stratified cluster desig was used to address the research questios. Selectio of clusters was stratified by itervetio. Data were collected withi a health facility (the primary samplig uit), ad the health facility was treated as a cluster. Amog itervetio hospitals, 8 were selected out of 16 for a probability weight of 2. Amog compariso facilities, all 8 were selected for a probability weight of 1. To address the stratificatio ad clusterig ad to adjust stadard errors, our aalytical approach used complex survey commads ( svyset i Stata 9.0). Mea scores were compared betwee itervetio ad compariso facilities to look for sigificat differeces. p-values were geerated by regressio models that used a t-test; each outcome was regressed o the itervetio status, cotrollig for regio of the coutry (cetral, orther, ad souther). Separately, jackkife variace estimatio was used

4 4 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] for the itervetio variable coefficiet to accout for ay residual clusterig of observatios withi health facilities. To examie treds i service utilisatio, mothly service statistics were aggregated ad graphed by quarter for itervetio ad compariso s. Researchers visually ispected the lie graphs for chages i service utilisatio after the PQI process was lauched, as well as differeces i service utilisatio treds betwee the itervetio ad compariso s. Fidigs Descriptio of participats Iterviews ad observatios were coducted with 98 providers, equally divided betwee itervetio ad compariso facilities. There were o sigificat differeces i sex, age, cadre, ad work experiece betwee the study s (Table 1). At each facility, a miimum of two cliet provider iteractios were observed i each service area; a few additioal observatios of L&D ad PNC services were made at some hospitals. A total of 139 iteractios were observed: 68 at compariso facilities ad 71 at itervetio facilities. Facility readiess to provide reproductive health services ANC service delivery areas were most likely to meet the readiess stadard for cliical records ad least likely to meet the stadard for cousellig ad examiatio areas (Table 2). facilities were more likely tha compariso facilities to have workig toilets, adequate couselig, ad examiatio areas, ad ecessary equipmet ad supplies i the ANC service area. Regardig L&D, most facilities i both study s lacked a system to rapidly assess wome i labour to idetify complicatios ad prioritise admissios. FP service delivery areas were most likely to meet the readiess stadard for the receptio/waitig area ad least likely to meet the stadard for equipmet ad supplies. facilities were more likely tha compariso facilities to have workig toilets ad adequate cousellig ad examiatio areas i the FP service area. Observed quality of reproductive health care After cotrollig for regio, the itervetio had sigificatly higher cliical observatio scores i two service areas, FP (89.0 vs i the cotrol, po.01) ad PNC (144.2 vs i the cotrol, po.01) (Table 3). Results for ANC did ot Table 1 Characteristics of providers who participated i the evaluatio by study. (¼49) (¼49) Total (¼98) Per cet female (%) Mea age (rage) (years) 36.7 (24 55) 36.2 (23 66) 36.5 (23 66) Per cet distributio by cadre Registered urse/midwives (%) Eroled urse/midwives (%) Nursig/midwifery techicias (%) Medical assistats (%) Cliical officers (%) Mea umber of years worked at curret facility (rage) (years) 7.2 (1 26) 9.0 (1 42) 8.6 (1 42) Table 2 Number of facilities achievig readiess stadards i specific service delivery areas, by study. Facility readiess stadards Number of facilities meetig all verificatio criteria for stadard ANC service delivery area (¼8) (¼8) Adequate receptio/waitig area 6 6 Workig toilets for cliets ad providers 7 3 Adequate cousellig area ad examiatio/ 4 3 procedure area Equipmet, supplies, ad materials for ANC 8 2 services Ateatal cliic uses specific cliical records i cojuctio with HMIS 8 8 L&D service delivery area (¼7) (¼7) System to perform a rapid iitial assessmet of wome i labour to idetify complicatios ad prioritise admissios 3 2 FP service delivery area (¼8) (¼8) Adequate receptio/waitig area 6 5 Workig toilets for cliets ad providers 6 3 Adequate cousellig area ad examiatio/ 5 3 procedure area Equipmet, supplies, ad materials for FP services 4 3* No data available for oe facility; ¼7. Table 3 Mea performace score for specific service areas, by study. Service area (umber of verificatio criteria) Mea performace score (95% CI) FP (120 criteria) (51.9, 89.1) PNC (200 criteria) (117.4, 153.1) ANC (215 criteria) (159.3, 186.3) L&D (215 criteria) (145.1, 184.8) 89.0 (71.7, 106.3) (128.6, 159.7) (171.5, 196.1) (148.4, 180.8) po.01; CI¼cofidece iterval. a The adjusted model cotrols for regio (cetral, orther, ad souther). achieve statistical sigificace at.05 i this aalysis. Results were similar ad remaied sigificat whe the effect of the itervetio o overall service scores was estimated usig jackkife variace. This suggests that the effect of the itervetio was robust. A closer examiatio of performace scores for each PQI stadard was udertake to idetify specific stregths ad weakesses i providers performace ad to idetify areas where the itervetio had the greatest impact. Key fidigs are summarised below, although the dataset is too large to preset i its etirety. The results are preseted as the percetage of verificatio criteria achieved, rather tha raw scores, to permit comparisos across stadards ad service areas. Quality of care was geerally good i FP: both study s complied with at least 80% of the verificatio criteria for 9 of the 16 stadards. The itervetio scored sigificatly higher tha the compariso o two stadards: establishig a cordial relatioship with the cliet ad idetifyig her eeds (99% ad 84% respectively, po.05), ad idetifyig the eed for protectio agaist sexually trasmitted ifectios (STIs), icludig HIV (73%

5 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] 5 ad 26%, po.01). Both study s performed less tha half of the verificatio criteria for the four stadards related to isertig implats. The fidigs also show strog performace i ANC: both study s achieved 80% or more of the verificatio criteria for 11 out of the 18 stadards. Scores for 3 of 17 stadards were sigificatly higher i the itervetio tha the compariso : rapid iitial evaluatio, which helps the provider triage ANC cliets who eed urget attetio (63% ad 23%, po.05), cordial receptio ad treatmet (99% ad 84%, po.05), ad coductig the physical ad obstetric exam (89% ad 73%, po.01). Both study s performed less tha half of the verificatio criteria for requestig laboratory tests. I the L&D service area, both study s complied with at least 80% of the verificatio criteria for 7 out of the 16 stadards. I both study s, scores were early perfect (ragig from 96% to 99%) for coductig the obstetric exam ad for performig ifectio prevetio practices durig labour accordig to stadards. There were o sigificat differeces betwee study s. Both study s performed oly about oe-third of verificatio criteria for moitorig postpartum wome durig the 2 hrs after the birth. Quality of care was ot as good for PNC: both study s complied with 80% or more of the verificatio criteria for just 4 of 20 stadards. Scores were especially high for assessig the coditio of the eoate (96% ad 93%, respectively). There were o sigificat differeces betwee study s. A further aalysis subdivided the stadards ito three cliical task sets cousellig, cliet assessmet ad diagosis, ad cliical maagemet ad procedures ad examied the effect of the itervetio o each task set (Table 4). Whe adjusted for regio i regressio aalyses, the itervetio scored sigificatly higher tha the compariso o cousellig i PNC. Results for cousellig i ANC did ot achieve statistical sigificace (p¼.056). The itervetio scored sigificatly higher o cliet assessmet ad diagosis i three service areas: FP, PNC, ad ANC. The itervetio also scored higher i cliical maagemet ad procedures i two service areas: FP ad PNC. Service statistics Five years of service statistics suggest that the PQI itervetio had a effect o deliveries performed by Caesarea sectio. As Fig. 1 shows, the umber of Caesarea sectios icreased i both study s, but more dramatically i the itervetio tha i the compariso. Data were collected o two aspects of routie skilled delivery care: the percetage of deliveries i which a partograph was used ad active maagemet of the third stage of labour. There was o evidece that the itervetio made a impact o either (data ot show), but cocers about the quality of the data suggest the results may ot fully reflect the real situatio. The two study s exhibited similar treds i service utilisatio, such as the umber of retur ANC visits, the umber of deliveries, ad the umber of postpartum care visits withi 2 weeks after birth (data ot show). We also examied service data related to obstetric ad ewbor complicatios diagosed ad treated, icludig severe pre-eclampsia/eclampsia, postpartum haemorrhage (PPH), abortio complicatios, all obstetric complicatios combied, ewbor sepsis, ad all ewbor complicatios combied. No differeces i treds were observed betwee itervetio ad compariso facilities (data ot show). Data quality was also a cocer for these statistics. Discussio Although quality of care was high at compariso as well as itervetio facilities, the evaluatio foud that the PQI itervetio sigificatly improved the maagemet of PNC ad FP cliets. These fidigs are supported by other evaluatio studies showig a positive impact of quality improvemet itervetios o providers compliace with cliical stadards (Hermida ad Robalio, 2002) ad o provider performace i service delivery (Luoma et al., 2000; Bradley et al., 2002; Lade, 2002). facilities were also more likely to have the ecessary ifrastructure, equipmet ad supplies for RH services, which may icrease staff motivatio as well as the quality of care. Although o sigificat chage i the overall quality of L&D services was detected, the itervetio was associated with greater provisio of caesarea sectios. This ca be iterpreted as a improvemet i the quality of care, give that percetage of births delivered by caesarea i Malawi (2.8%) is below the iteratioally recommeded miimum of 5% (Leigh et al., 2008; WHO et al., 2009). Programme effects o L&D services may also have bee delayed by the ature of the PQI process, which ecourages implemeters to focus first o areas that are easier to address, such as FP or ANC, ad oly the to move o to more complex areas, such as L&D, i order to acquire ad stregthe chage maagemet skills. The evidece of impact, while limited, is especially ecouragig give that the itervetio was still i its iitial phases. Iteralisig ad masterig ew stadards takes time ad practice. However, this evaluatio was coducted 2 3 years after the itervetio was lauched, before ay facilities had achieved recogitio for the quality of their services. Public recogitio is a itegral part of the PQI process ad the compoet most likely to have a direct effect o service utilisatio sice it makes potetial cliets aware of improvemets i service quality. Thus, the effects of the itervetio o service utilisatio had probably ot yet bee realized. Table 4 Per cet of stadards performed by cliical task set ad study. Service area (%) Cousellig Cliet assessmet ad diagosis Cliical maagemet ad procedures (%) (%) (%) (%) (%) FP PNC ANC L&D pr.05. pr.01. pr.001. a The adjusted model cotrols for regio (cetral, orther, ad souther).

6 6 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] Cesarea Sectios i Malawi Number of c-sectios RH PQI itervetio bega Cotrol Sites Sites Fig. 1. Treds i caesarea sectios by study, Evets after the evaluatio have demostrated the importace of formal recogitio. Four facilities have sice achieved recogitio for their reproductive health services, ad the atioal recogitio ceremoy, which is atteded by high-level officials, is provig to be a big icetive. Maagers take great pride i showcasig the strides their facilities have made i offerig better quality services. The emphasis o recogitio has also spilled over to the facility level, where maagers have established iteral reward systems for departmets that perform well o iteral PQI assessmets. Wiig departmets may receive trophies or cash icetives that ca be used to supplemet their salaries or to procure additioal equipmet. High turover ad iexperiece amog providers may have bluted the effects of the PQI itervetio, especially i the labour ad delivery ward. High-performig persoel icludig those that have beefited from the PQI itervetio ted to be promoted by the MoH to the cetral level or may be trasferred to other hospital departmets that are ot part of the PQI iitiative. Oe-third of the providers i the study had bee at their curret facility for oe year or less. The assessmet team oted that some L&D observatios ivolved midwifery studets who were doig a cliical practice rotatio, while others ivolved ew providers with less tha six moths experiece i the departmet. I geeral, oe would expect facilities with less staff turover to beefit more from the PQI process, sice it is a effort requirig commitmet to make chages. Low caseloads may also have posed a challege for the itervetio. Providers may forget the skills eeded to carry out specialized procedures, such as vacuum extractio, whe they perform them ifrequetly. Aecdotally, supervisors i Malawi have oted that some providers who lack cofidece i their kowledge ad traiig actively avoid coductig certai procedures, reducig their practice ad performace still further. It should be oted that the EHRP complemeted the PQI itervetio: while the EHRP worked to icrease the umber of providers o staff, PQI worked to stregthe their skills ad motivatio by establishig systems that provide for regular performace appraisal, recogitio, ad rewards. Regular ad detailed moitorig of provider performace through self- ad peer assessmet followed by feedback is built ito the PQI process. While PQI does ot explicitly provide for exteral supervisio, dedicated QIST teams ofte focus o stregtheig metorig ad supervisio osite i order to improve staff performace. However, the cotiuig shortage of skilled providers remais a challege: accordig to a 2010 Natioal EmONC Assessmet, oly 57% of district-level positios for urse/midwife techicias, 39% for registered urses, ad 37% for medical assistats were filled (MoH, 2010). Study stregths ad limitatios The study had two stregths: direct observatios were used to assess performace o the job, ad the observers were highly qualified cliical experts who were exteral to the facility ad hece less likely to be biased. They were also well traied i PQI assessmets. This approach yields a more objective ad more cosistet assessmet of performace tha either provider selfreports or cliet iterviews. Because providers may make a greater effort whe uder observatio (the Hawthore effect), this study may reflect providers best possible performace rather tha their average performace. This does ot matter for the purposes of compariso; however, sice the providers at compariso ad itervetio facilities would likely have reacted i the same way. There are some limitatios. We caot be certai that the facilities selected are comparable o every coditio that might potetially affect the fidigs. I additio, the small umber of observatios for ay oe service area limits the ability to detect differeces betwee s. As described i the methods sectio, efforts were made to accout for some of these problems i the statistical aalysis, for example, by cotrollig for regio. Exposure to the earlier PQI ifectio prevetio itervetio which was implemeted at all compariso as well as itervetio facilities may have affected the evaluatio results. Participatio i the earlier roud of PQI may have stregtheed RH scores because they iclude ifectio prevetio practices. I additio, the earlier roud of PQI may have prompted subtle chages i the way providers approach their jobs that could have spilled over to the rest of their work. However, sice all facilities had received the ifectio prevetio itervetio already, this would have affected both itervetio ad compariso facilities. Service statistics for some idicators (e.g., use of partographs, active maagemet of the third stage of labour, ad eclampsia diagoses) were ot always of good quality. Recordkeepig at some facilities was icomplete. This makes it difficult to iterpret some service statistics with cofidece. Coclusio The PQI process has prove effective i improvig the quality of care of reproductive health services i a low-resource settig. Sice providers at both itervetio ad compariso facilities

7 B.J. Rawlis et al. / Midwifery ] (]]]]) ]]] ]]] 7 may have bee iflueced by the Hawthore effect, observed quality of care may have bee better tha the orm. However it is ulikely that observed differeces betwee the two s i quality of care scores were due to this effect sice both sets of providers would likely have bee affected i the same way. Moreover, if a provider does ot kow how to properly perform a best cliical practice, his/her care will ot improve if someoe is watchig. Nevertheless, efforts should be made to measure quality of care through service statistics ad patiet outcome measuremets as well as observatio of provider cliet iteractios. Give the positive impact of the PQI process, it should cotiue to be scaled up i Malawi. Special attetio eeds to be paid to reducig staff turover i order to maximise the effectiveess of the itervetio. Demad geeratio is also importat to icrease service utilisatio ad esure that providers apply the skills they lear through traiig. The MoH is cotiuig to champio the PQI process ad has edorsed it as a best practice for quality assurace. With fudig from USAID s Materal ad Child Health Itegrated Program (MCHIP), the MoH is scalig up a combied PQI approach for ifectio prevetio ad reproductive health at the health cetre level ad is expadig PMTCT stadards alog the cotiuum of care. The MoH is also applyig PQI to other areas, such as laboratory services. Ackowledgemets The authors would like to thak their colleagues ad parters icludig the Miistry of Health Reproductive Health Uit, Directorate of Nursig, Quality Assurace Techical Workig Group, ad Uited States Agecy for Iteratioal Developmet (USAID), who cotributed to this atioal evaluatio ad who, sice 2001, have facilitated the desig ad atioal roll-out of the PQI RH itervetio i Malawi. We also thak the providers ad cliets who allowed our study team to observe ad iterview them. This evaluatio was sposored by Jhpiego: A Affiliate of Johs Hopkis Uiversity ad USAID. Refereces Africa Health Workforce Observatory (AHWO), Huma Resources for Health Coutry Profile: Malawi. AHWO, Brazzaville, Cogo. Bailey, P.E., Bih, H.T., Bag, H.T., Promotig accoutability i obstetric care: use of criteria-based audit i Viet Nam. Global Public Health 5, Bradley, S., McAuliffe, E., Mid-level providers i emergecy obstetric ad ewbor health care: factors affectig their performace ad retetio withi the Malawia health system. Huma Resources for Health 7, 14. Bradley, J.E., Mayfield, M.V., Mehta, M.P., Rukoge, A., Participatory evaluatio of reproductive health care quality i developig coutries. Social Sciece ad Medicie 55, Campbell, O.M., Graham, W.J., Strategies for reducig materal mortality: gettig o with what works. Lacet 368, Darmstadt, G.L., Bhutta, Z.A., Couses, S., Adam, T., Walker, N., de Beris, L., Evidece-based, cost-effective itervetios: how may ewbor babies ca we save? Lacet 365, Fauveau, V., Sherratt, D.R., de Beris, L., Huma resources for materal health: multi-purpose or specialists? Huma Resources for Health 6, 21. Hermida, J., Robalio, M.E., Icreasig compliace with materal ad child care quality stadards i Ecuador. Iteratioal Joural for Quality i Health Care 14 (Suppl. 1), Jamtvedt, G., Youg, J.M., Kristofferse, D.T., O Brie, M.A., Oxma, A.D., Audit ad feedback: effects o professioal practice ad health care outcomes. Cochrae Database Systematic Review 2, CD Kogyuy, E.J., Mlava, G., va de Broek, N., 2009a. Usig criteria-based audit to improve the maagemet of postpartum haemorrhage i resource limited coutries: a case study of Malawi. Materal ad Child Health Joural 13, Kogyuy, E.J., Mlava, G., va de Broek, N., 2009b. Criteria-based audit to improve wome-friedly care i materity uits i Malawi. Joural of Obstetrics ad Gyaecology Research 35, Lade, R.E., Performace Improvemet. Populatio Reports, Series J, No. 52. Johs Hopkis Uiversity School of Public Health, Populatio Iformatio Program, Baltimore, MD. Leigh, B., Mwale, T.G., Lazaro, D., Luguzi, J., Emergecy obstetric care: how do we stad i Malawi? Iteratioal Joural of Gyecology ad Obstetrics 101, Luoma, M., Jaskiewicz, W., McCaffery, J., Catotti, D., Domiica Republic Performace Improvemet Project Evaluatio. Techical Report 19. INTRAH, Chapel Hill, NC. Maafa, O., McAuliffe, E., Maseko, F., Bowie, C., MacLachla, M., Normad, C., Retetio of health workers i Malawi: perspectives of health workers ad district maagemet. Huma Resources for Health 7, 65. McAuliffe, E., Bowie, C., Maafa, O., et al., Measurig ad maagig the work eviromet of the mid-level provider the eglected huma resource. Huma Resources for Health 7, 13. Miistry of Health (MoH)/Malawi, A Joit Programme of Work for a Health Sector Wide Approach (SWAp), MoH Departmet of Plaig, Lilogwe, Malawi. Miistry of Health (MoH)/Malawi, Natioal EmONC Assessmet. MoH, Lilogwe, Malawi. Muula, A.S., Maseko, F.C., Survival ad Retetio Strategies for Malawia Health Professioals. EQUINET Discussio Paper No. 32Regioal Network for Equity i Health i Souther Africa (EQUINET), Harare, Zimbabwe. Natioal Statistical Office (NSO) of Malawi, ICF Macro, Malawi Demographic ad Health Survey 2010, Prelimiary Report. NSO ad ICF Macro, Calverto, MD. Natioal Statistical Office (NSO) of Malawi, UNICEF, Malawi Multiple Idicator Cluster Survey Fial Report. NSO ad UNICEF, Lilogwe, Malawi. Necochea, E., Bossemeyer, D., Stadards-Based Maagemet ad Recogitio (SBM-R) A Field Guide: A Practical Approach for Improvig the Performace ad Quality of Health Services. Jhpiego, Baltimore, MD. O Neil, M., Jarrah, Z., Nkosi, L., Collis, D., Perry, C., Jackso, J., Evaluatio of Malawi s Emergecy Huma Resources Programme. Departmet for Iteratioal Developmet (DFID), Maagemet Scieces for Health (MSH), ad Maagemet Solutios Cosultig (MSC), Cambridge, MA. Palmer, D., Tacklig Malawi s Huma Resources Crisis. Reproductive Health Matters 14 (27), Picazo, O.P., Martieau, T., Huma Resources ad Fiacig of the Health Sector i Malawi. World Bak ad Miistry of Health, Malawi. Rowe, A.K., de Savigy, D., Laata, C.F., Victora, C.G., How ca we achieve ad maitai high-quality performace of health workers i low-resource settigs? Lacet 366, Wagaarachchi, P.T., Graham, W.J., Peey, G.C., McCaw-Bis, A., Yeboah Atwic, K., Halla, M.H., Holdig up a mirror: chagig obstetric practice through criterio-based cliical audit i developig coutries. Iteratioal Joural of Gyecology ad Obstetrics 4, World Health Orgaizatio (WHO), UNFPA, UNICEF, ad Avertig Materal Death ad Disability (AMDD) Program, Moitorig Emergecy Obstetric Care: A Hadbook. WHO, Geeva.