Joural of Public Health Medicie Vol. 20, No. 1, pp. 6-69 Prited i Great ritai ssessmet of the reproducibility of cliical codig i routiely collected hospital activity data: a study i two hospitals Jeifer Dixo, Coli Saderso, Paul Elliott, Peter Walls, Jeremy Joes ad Mark Petticrew bstract ackgroud The aim of the study was to assess the reproducibility of cliical codig i two Natioal Health Service hospitals withi North West Thames regio. Methods retrospective audit was carried out, of cliical codig i hospital episode statistics, ivolvig compariso of the codes assiged by local staff with those assiged by members of a exteral team uaware of the locally assiged codes. Where local ad exteral coders disagreed, the records were reviewed for a third time by a further idepedet coder. The subjects were a radom sample of 1607 o-materity, o-psychiatric admissios occurrig betwee 1991 ad 199, stratified for year ad type of disease (asthma, diabetes, appedicitis, fractured femur ad 'geeral' - a radom selectio of ay diagoses). The mai outcome measures were the levels of exact agreemet betwee local ad exteral teams over codes for mai diagosis ad procedure, ad the level of approximate agreemet (over the first three characters of the ICD-9 code for diagosis ad the letter ad first two digits of the OPCS- code for procedure). For disagreemets, the outcome measure was the level of agreemet betwee the 'third' coder ad the local ad exteral coders. Results For the mai diagosis i the 'geeral' group at hospital, iteral ad exteral coders agreed exactly i per cet of the admissios examied ad agreed 'approximately 7 i 55 per cet (kappa = 0.5). For hospital the correspodig figures were 60 per cet ad 72 per cet (kappa = 0.72). pproximate agreemet was higher for the specific diseases cosidered, particularly for asthma (: 86 per cet; : 91 per cet) ad fractured femur (: 8 per cet; : 89 per cet). For the mai procedure at hospital, there was exact agreemet for 58 per cet ad approximate agreemet for 70 per cet (kappa = 0.66). For hospital, the correspodig figures were 76 per cet ad 8 per cet (kappa = 0.80). I cases of disagreemet over the first three digits of the ICD-9 code for mai diagosis, the third coder disagreed with both local ad exteral coders i 5 per cet at hospital ad 8 per cet at hospital. greemet was slightly better for discharges i 1992-199 tha i 1991-1992. Coclusios The full cliical codes i NHS hospital episode statistics (HES) data should be treated with cautio. The first three characters of ICD-9 codes for diagoses ad the OPCS- codes for procedures were more reliable. For some specific coditios such as asthma ad fractured femur, reliability of the first three characters is much higher (for example, 86 per cet ad 91 per cet for asthma i the two hospitals), but for the full codes ca be worse. Secodary diagoses or comorbidities may be sigificatly udercoded. higher level of agreemet i 1992-199 tha i 1991-1992 suggests that codig may be improvig. Keywords: hospital episode statistics, quality, codig, survey Itroductio ccurate iformatio o hospital activity i the Natioal Health Service (NHS) is vital to pla ad fud services properly, set ad moitor cotracts, ad for research. Whereas the volume of recorded i-patiet ad day case activity is regularly aalysed, 1 little is kow about the accuracy of cliical codig i routie data, despite the substatial cost to the NHS of collectig them. The few studies i this area have estimated the 'accuracy' of cliical codig to be betwee 60 ad 90 per cet. 2 " 7 Strictly, most of these studies have bee cocered with reproducibility of the codig process rather tha with accuracy of the diagostic codes, the questio at issue beig whether differet coders, give the same set of medical records, register the same set of codes, rather tha ivolvig idepedet cliical review of patiets or test results. ut iterpretatio of these studies is Lodo School of Hygiee ad Tropical Medicie, Keppel St, Lodo WC1E 7HT. Jeifer Dixo, Hoorary Seior Lecturer, Health Services Research Uit Coli Saderso, Seior Lecturer, Health Services Research Uit Peter Walls, Seior Computig Officer, Evirometal Epidemiology Uit Departmet of Epidemiology ad Public Health, Imperial College School of Medicie at St Mary's, Norfolk Place, Lodo W2 1PG. Paul Elliott, Professor Departmet of Epidemiology ad Public Health, Uiversity of Leicester, Leicester. Jeremy Joes, Lecturer NHS Cetre for Reviews ad Dissemiatio, Uiversity of York, York. Mark Petticrew, Research Fellow ddress correspodece to Dr Dixo, Kig's Fud Policy Istitute, Lodo W1M0N. Oxford Uiversity Press 1998
6 JOURNL OF PULIC HELTH MEDICINE difficult because experieced cliical coders have geerally ot bee used to review codes, 6 ' 8 ' 9 ad the reviewers used were ofte ot blid to the origial codes recorded. Furthermore, the studies were of data collected before 1991, whe the icetives for collectig accurate data were differet ad hospital iformatio systems less sophisticated. This study set out to address these methodological issues by usig experieced cliical coders who were uaware of the origial codes recorded, ad by examiig i-patiet data collected after 1991. Methods Selectio of hospitals Fudig was available to coduct the study i two hospitals i the former North West Thames regio. The hospitals selected were large acute NHS Trusts typical of those servig o-metropolita areas ad which had bee icluded i a regio-wide project to aalyse variatios i admissio rates. Selectio of case otes Routie hospital episode statistics were obtaied from the hospitals for patiets discharged betwee pril 1991 ad Jauary 199, the most recet data at the time of the study. 'Fiished cosultat episodes' of care were grouped to represet hospital admissios. However, hospital oly supplied cliical codes relatig to the first episode i each admissio, ad so these oly were used for both hospitals. Hospital admissios with first episodes i the followig categories were excluded i the regio-wide project, ad so were also excluded from this study: psychiatry ad obstetrics; well babies bor i hospital; patiets aged 75 years ad older, real dialysis. From the remaiig first episodes (7 192 at hospital, 9679 at hospital ) five radom samples were selected for each hospital: a 'geeral' group cotaiig a radom selectio of ay diagoses ( = 85), ad four disease-specific groups [asthma ( = 165), diabetes ( = 165), fractured femur (w = 85) ad appedicitis with appedicectomy ( = 85)], givig a total size of 885 for each hospital. I each group, half the sample was draw from 1991-1992 ad half from 1992-199. I each of the four disease-specific groups, half the sample was draw with the diagosis coded as the mai diagosis ad half as a secodary diagosis. I the 'geeral' group approximately 2 per cet of the sample draw i each hospital had o cliical codes assiged. I these cases the local coders were asked to code the admissios as usual. Recodig Four experieced cliical coders from CSPE Healthcare Kowledge Systems (CHKS) were employed as the exteral coders. For each episode, coders could record up to seve diagoses (usig ICD-9 codes 10 ), ad up to four procedures (usig OPCS- codes"). The exteral coders used the same source material (case otes) as the local coders, ad both used the atioal rules for cliical codig. 12 ' 1 To help coders idetify each episode i the case otes, iformatio was provided showig the specialty, ad start ad ed date of each episode. Where local ad exteral coders disagreed, the case otes were reviewed by the seior codig maager at North West Thames Regioal Health uthority. ll coders were uaware of codes recorded by others. alysis greemet betwee exteral ad local coders was idetified at two levels: exact agreemet over the mai diagosis (or mai procedure) ad approximate agreemet (based o the first three digits of the ICD-9 code for diagosis, ad the letter ad first two digits of the OPCS- code for procedure). Kappa statistics were calculated for the 'geeral' groups, but ot for the diseasespecific groups because these did ot provide a full 'square' desig: for example, there were o data o the umbers of cases which were ot coded as asthma by the local coders, but would have bee coded as such by the exteral coders. Percetage agreemet was calculated for all groups, with 95 per cet cofidece itervals for levels of approximate agreemet The sequecig of codes was also compared. The effect of differeces i codig o the 'case mix' of the sample was also estimated because case mix measures are icreasigly beig used i the NHS for cotractig. 1 ' 15 This was doe by assigig to each first episode a 'case mix' score based o the estimated use of resources by a patiet with the diagoses ad procedures listed. The Diagosis Related Groups (DRG) 16 software package developed i the Uited States was used to assig case mix scores ad the ICD-9 codes recorded were coverted to ICD-9CM codes (a variat of ICD-9 used i the Uited States) to eable the DRG software to read them. The case mix score of first episodes coded by local coders was compared with those by the exteral coders. The Healthcare Resource Groups (HRG) software (developed i the Uited Kigdom) was ot used because at the time of the study less tha 70 per cet of episodes could be grouped usig this software. Results Retrieval Of the 885 discharges chose i each hospital, the case otes of 80 were foud i hospital (90.8 per cet) ad 80 (90.7 per cet) i hospital. Diagosis The level of agreemet betwee the exteral ad local coders over the codes for mai diagosis is show i colums (a) ad (b) i Table 1. Colum (a) shows that, i the 'geeral' group,
CLINICL CODING IN HOSPITL DT 65 Table 1 Percetage agreemet by exteral coders with mai diagosis ad procedure as idetified by local coders Local mai code = exteral mai code Local mai code = exteral secodary code Local mai code ot recorded by exterals Hospital Exact* (a) pproximatet (b) Exact* (c) pproximatet (d) Exact* (e) pproximatet (f) Geeral sthma* Diabetes* ppedicitis* Fractured femur* Mai procedure 60 5 5 6 9 51 65 7 0 52 69 55(50.5-61.2) 72 (66.8-76.) 86 (75.-92.) 91 (81.1-95.9) 70 (58.-86.2) 75 (6.8-88 ) 78 (72.7-90.9) 80 (68.8-88 ) 8 (72.7-90.9) 89 (7.6-96.5) 7 (65.9-78.) 82 (75-86.) 5 5 CO CO 1 0 7 7 8 6 25 22 6 1 0 8 52 5 62 81 78 6 9 70 1 27 8 20 10 CO CJl 16 19 12 11 19 1 5 8 77 75 71 75 71 71 7 7 211 205 The 95 per cet cofidece itervals are show i paretheses. *For diagoses: idetical ICD four-digrt codes; for procedures: idetical OF"CS codes. tfor diagoses; first three digits of ICD codes idetical, for procedures: first two characters of OFCS codes idetical. ^Results show i these disease-specific groups are for where the disease was locally coded as the mai diagosis. the exact agreemet over the code for mai diagosis was per cet at hospital ad 60 per cet at hospital. greemet was slightly higher for appedicitis. greemet was very low for diabetes. Colum (b) shows that approximate agreemet was higher for all groups, ad very good for asthma (agreemet of 86 per cet ad 91 per cet i the two hospitals) ad fractured femur. Kappa values for the 'geeral' groups were 0.5 for hospital, ad 0.72 for hospital. These values are very close to the percetage agreemet because, give the large umber of categories used ad reasoably eve spread of observatios across them, the probability of chace agreemet was small. This improvemet i reproducibility was strikig for diabetes; both sets of local coders differed from the exteral coders i their use of the fourth digit of the ICD-9. To idetify how far disagreemets were due to differeces i sequecig, we ivestigated whether the mai diagosis recorded by the local coders had bee recorded by the exteral coders as a secodary diagosis istead. The results are show i Table 1 i colums (c) ad (d). For the 'geeral' group from hospital, the locally coded exact mai diagosis was idetified as a secodary diagosis by the exteral coders i 5 per cet of cases, but i 52 per cet it appeared owhere o the exterally coded list of diagoses (colum (e)). The results were similar for the other disease groups at hospital, except for diabetes, where the proportio recorded as a secodary diagosis by the exteral coders was higher. The results for hospital were similar. The exteral coders teded to code more secodary diagoses or co-morbidities tha the local coders i both hospitals, especially for chroic coditios such as diabetes. Where there was ot eve approximate agreemet over the mai diagosis, the disagreemets fell ito the followig five broad categories: (1) where oe group of coders had recorded a symptom, ad the other had recorded a diagosis related to the symptom; e.g. abdomial pai (ICD-9 code 789.0) ad acute appedicitis (50.9); (2) where both coders had recorded codes for very similar if ot idetical coditios, ofte where oe code was more precise tha the other; e.g. diabetes (250.0) ad diabetes i pregacy (68.0); acute appedix (50.9) ad appedix - uqualified (51), peumoia (86) ad peumococcal peumoia (81); () where there was obvious disagreemet betwee coders over similar coditios; e.g. acute appedicitis (50.9) ad meseteric lymphadeitis (289.2), chroic tosillitis (7.0) ad acute tosillitis (6), emphysema (92) ad asthma (9.9); () where the coders had recorded codes for coditios which were ot similar but obviously related; e.g. diabetes (250.0) ad cellulitis (682.7), phlebitis of the deep vessels of the lower extremities (15.1) ad pulmoary embolus (51.1); (5) where coders had recorded completely differet coditios, e.g. brochopeumoia (85) ad fracture of vault of skull (800.1); (X) where the exteral coders were ot able to reach a decisio ad had ot recorded a mai diagosis. The proportio of disagreemets i each category at each hospital are show i Table 2. This shows that the disagreemets were spread evely across all categories i both
66 JOURNL OF PULIC HELTH MEDICINE Table 2 The proportio of cases i each category* as a percetage of all disagreemets (where exteral ad local coders did ot agree over the first three digits of the ICD-9 code for mai diagosis) ad as a percetage of all cases reviewed Hospital Hospital Category %of disagreemets ( = 295) % of all cases reviewed ( = 80) %of disagreemets </i = 21) % of all cases reviewed ( = 80) 1 2 5 X Total 1.6 17.6 2. 10.5 18. 16.6 100 5.0 6.5 8.6.8 6.7 6.1 6.7 19.1 11.2 29. 15.8 19.5 5.1 100 5.1.0 7.7 2 5.2 1.2 26. The results are show for all disease groups combied. *For a descriptio of categoes, see text hospitals, but the proportio of cases i category X was lower at hospital. For all disease groups together, the level of exact agreemet over the mai diagosis icreased betwee 1991-1992 ad 1992-199 from 8 to 2 per cet at hospital ad from 68 to 79 per cet at hospital. The icrease was statistically sigificat for hospital oly (p < 0.05). Table shows how the exteral coders coded asthma ad diabetes where local coders had recorded these as secodary diagoses. I most cases both sets of coders agreed that the diabetes ad asthma were secodary diagoses. However, for asthma, the exteral coders thought that i 17 per cet (hospital ) ad 18 per cet (hospital ) of cases, this was the mai, rather tha the secodary diagosis. lso, the exteral coders had ot coded asthma at all i 9 per cet (i hospital ) ad 1 per cet (hospital ) of discharges i this group. Procedures Procedure codes were ivestigated for first episodes i the 'geeral' groups. The figures i the last two rows of Table 1 refer to records i which both sets of coders had recorded at least oe procedure i the first episode. Disagreemet about whether a particular procedure was the mai or a secodary oe was relatively ucommo. secod aalysis was doe icludig all records i the 'geeral' groups. I this aalysis, if either coder recorded a procedure, this couted as a agreemet; if oe did ad the other did ot, this was a disagreemet. For hospital, there was exact agreemet for 58 per cet ad approximate agreemet for 70 per cet (kappa = 0.66). For hospital, the correspodig figures were 76 per cet ad 8 per cet (kappa = 0.80). The adjustmet ivolved i the kappa statistic related almost Table Percetage agreemet betwee exteral coders ad local coders over asthma ad diabetes whe these were coded by local coders as secodary diagoses; for disagreemets, the percetage of cases where the exteral coders had recorded asthma or diabetes as a mai diagosis or ot at all greemet Disagreemet Secodary diagosis (-digft ICD-9 code) Mai diagosis (-digit ICD-9 code) Not coded (-dig'rt ICD-9 code) Hospital Disease group sthma Diabetes Hospital Disease group sthma Diabetes 7 (62-8 0) 88(78.1-9.0) 68 (55.9-78.) 92 (8.-96.6) 17(9 7-27.7) 7 (2.7-15.9) 18(0.-29.9) (1.1-11.) 9(.9-18.) 5(1.56-1.) 1(7.-2.7) (1.1-11.) 76 76 72 8 The fractured femur ad appedicitis (with appedicectomy) groups are omitted because of small umbers; 95 per cet cofidece itervals are show i paretheses.
CLINICL CODING IN HOSPITL DT 67 Table The percetage agreemet o the mai diagosis (o the first three digits of the ICD-9 code) betwee the regioal coder ad the local coders ad exteral coders greemet of regioal coder with: Local coders Exteral coders Neither Hospital Hospital 17(1.0-21.9) 27(21.- 6) 0 (2.9-5 6) 8(1.-.9) 5(7 1-58.8) 5 (28.7-1.9) 295 21 The 95 per cet cofidece itervals are show i paretheses. etirely to the possibility of chace agreemets that there had bee o procedure. Review by the regioal coder Table shows that where there was ot eve approximate agreemet over mai diagosis, the third coder disagreed with both local ad exteral coders i a high proportio of cases, but more so with the local tha with the exteral coders. The differeces were statistically sigificat (p < 0.05) at hospital but ot at hospital. Effect o case mix Of 1607firstepisodes reviewed by the coders at both hospitals, it was possible to assig a DRG score to 85.6 per cet. The results for these episodes oly are show i Table 5, which shows that i both hospitals the score assiged to exterally coded episodes was slightly higher (p > 0.0001). Discussio The accuracy of cliical codes depeds upo how closely they reflect the cliical coditio of the relevat patiet (path i Fig. 1). However, this study ivolved compariso of two sets of codes idepedetly abstracted from the case otes (path ), ad is thus a study of the reliability of the codig process. The results will ot allay the widespread suspicios of the full cliical codes i NHS hospital statistics. However, agreemet about the first three characters i diagostic ad procedure codes was relatively good, particularly for certai coditios such as asthma. This, ad the sigificat icrease i agreemet betwee years i hospital, is ecouragig. The patter of results was similar i both hospitals ad reflects the results of earlier studies. 2 " 5 lso, Table 2 idicates that for diagoses, the differig codes chose were for related coditios i a substatial proportio of cases. It is the disagreemets i categories 5 ad X that are most worryig. There may have bee legitimate reasos for disagreemet For example, the exteral coders, who coded -26 moths after discharge, may have had access to iformatio added to the otes after the local codig had bee doe, whereas the local coders may have had the beefit of queryig the diagoses 'o the spot' with cliicias. The 'geeral' groups are most likely torepresetthe overall populatio of hospital discharges. greemet i this group was lowest possibly because of rare, complex ad ill-defied cases which were difficult to code. Exact agreemet was higher for the more acute coditios selected - asthma, appedicitis ad fractured femur. Diabetes seemed to preset particular problems i the way the fourth digit was used. What do the results imply about the quality of codig i the hospitals studied? Iterpretatio depeds partly upo the quality of the exteral coders. ll had coded for more tha four years, ad two had traied NHS regioal ad hospital codig maagers i courses ru by the Departmet of Health. I the past, CHKS coders have bee foud to be reliable ad valid, 5 ad i this study the proportio of disagreemets over mai diagosis ad procedure was cosistet across all four coders. The reasoably high kappa scores idicate that the level Table 5 Mea diagosis related group (DRG) scores for first episodes coded by local ad exteral coders for each hospital DRG score Coder Mea Differece P Hospital Hospital Local Exteral Local Exteral 0.88 0.909 0.812 0.879 0.071 0.067 <0.0001 <0.0001 699 699 677 677
68 JOURNL OF PULIC HELTH MEDICINE Path Path Steps Disease -..Make diagosis. Record diagosis. i case otes I I I Locate relevat documetatio i case otes ssig diagostic (ICD-9) or procedure (OPCS-) codes Resposibility Coder liicia Cliicia Coder Figure 1 simple flow diagram showig the steps ivolved i cliical codig from case otes, ad the perso resposible for each step. of chace agreemet betwee coders was low. However, they caot be regarded as providig a gold stadard. The very experieced regioal 'third' coder agreed more ofte with the exteral tha with the local teams, but ofte disagreed with both. lso, it is ot possible with a study of this kid to assess the extet to which codig differeces result from poor case otes or codig errors. The cliical iformatio i the case otes was frequetly ambiguous, missig, illegible, or poorly orgaized (problems that have bee oted elsewhere 17 ), especially i hospital, where there was more disagreemet betwee all three groups of coders (Table ) ad more disagreemets i category X. lthough diagosis is ofte complex ad geuiely ucertai, it is still the cliicia's resposibility to documet clearly codable coditios 18 ad it is ot for coders to try to work out what the diagoses are. ut eve if cliical iformatio is uambiguous, there may be itrisic ucertaity i the process of cliical codig - i assigig ad sequecig codes - ad some level of disagreemet betwee coders is ievitable. Could similar results be expected elsewhere? Neither hospital had uusual features, the patter of results i each was broadly similar, ad the results mirror those foud i other studies. s such, the results preseted here may well be geeralizable, although there may be greater variatios betwee other acute hospitals i the regio which were ot icluded i this study. However, some admissios excluded from the sample, such as the elderly, may be relatively difficult to code ad so the results may uderestimate the level of agreemet across all admissios i acute hospitals. The case otes of approximately 9 per cet of the sample could ot be foud, ad they too could have bee difficult to code, although they were spread across all disease groups, specialties ad ages ad there was o evidece to suggest they were differet from the admissios reviewed. lso, oly first episodes i the admissios were reviewed - secodary episodes may have bee more difficult or easier to code. However, as typically more tha 90 per cet of admissios o HES data cotai just oe episode, the effect of secodary episodes o the results is likely to have bee very small. How the might cliical codig of hospital activity be improved? To make a start we recommed that a sample of episodes should be audited regularly by idepedet coders who, as i this study, could at least test whether they could reproduce local codes. The results could be shared with local purchasers, who could agree with providers further work ecessary to improve the data to the stadards required at least for purchasig or commissioig. I a quasi-market it is i the iterests of commissioers to kow what they are purchasig ad of health care providers to kow what they are providig. The accuracy of idividual cliical codes for procedure is already of importace to GP fudholders, who may be wrogly billed for a procedure, or health authorities havig cost per case cotracts. Furthermore, this study showed that the case mix scores, ad thus potetially hospital charges i a activitybased chargig system, were uderestimated whe cases were coded locally, largely because the local teams recorded fewer secodary diagoses, which are the source of data o comorbidity i DRGs. Improvig cliical codig will also ecessarily ivolve the co-operatio of cliicias, who up to ow have show little iterest i the valuable resource of HES data. It is the resposibility of cliicias to record iformatio clearly i the case otes, ad this should be a regular focus for cliical audit. ut serious actio by cliicias to improve codig is likely oly if the data are used by them regularly. There are may possible applicatios, such as i epidemiological ad health services research, ad cliical audit. However, the greatest potetial usefuless of HES data i future may be to support the icreasig role of doctors i maagemet. Coclusio The geerally low level of agreemet betwee coders over mai diagosis ad procedure codes foud i this study raises serious questios about the quality of iformatio recorded o hospital episode statistics: Whereas the exact cliical codes should be treated with cautio, the accuracy of the first three digits of ICD-9 codes for diagosis, ad the first two of OPCS- codes
CLINICL CODING IN HOSPITL DT 69 for procedure is likely to be higher. For well-recogized acute coditios three-digit ICD-9 codes gave high agreemet (for example, 86 per cet ad 91 per cet for asthma i the two hospitals). The higher level of agreemet i 1992-199 suggests that codig may be improvig. Purchasers ad providers should ivest more i checkig the quality of codig through regular idepedet audit, especially if commissioig icreasigly depeds upo the accuracy of codig. Cliicias have a key resposibility to improve the poor quality of cliical iformatio i case otes, upo which accurate codig crucially depeds. ckowledgemets May thaks are due to Dr Sheila dam ad Mike Coor at North Thames RH for their support; Meg Eva at Methodologica, CHKS; the codig maagers ad iformatio maagers at both hospitals for all for their support ad efficiet help; ad Nick lack, Marti McKee ad a aoymous reviewer for helpful commets o earlier drafts of this paper. Fudig for this work was provided by North West Thames Regioal Health uthority. Refereces 1 Departmet of Health. Hospital episode statistics 1992-199. Lodo, HMSO, 199. 2 aie JL, Marsh DR. Quality of data i the Machester Orthopaedic Database. r MedJ 1992; 0: 159-162. George M, Maddocks G. ccuracy of diagostic cotet of hospital cotet of hospital activity aalysis i ifectious diseases. r Med J 1979; 1: 12-1. Whates PP, irzgalis R, Irvig M. ccuracy of hospital activity aalysis operatio codes. r Med J 1982; 28: 1855-1858. 5 Cleary R, eard RW, Coles J, et al. Comparative hospital databases: value for maagemet ad quality. Quality Hlth Care 199; : -10. 6 Yeoh C, Davis H. Cliical codig: accuracy ad completeess whe doctors take it o. r Med J 199; 06: 972. 7 Walshe K, Harriso N, Reshaw M. Compariso of the quality of patiet data collected by hospital ad departmet computer systems. Health Treds 199; 25: 105-108. 8 Cleary R, eard RW, Coles J, et al. The quality of routiely collected materity data. r J Obstet Gyaecol 199; 101 (12): 102-107. 9 Martii CJM, Hughes O, Patto V. study of the validity of the Hospital ctivity alysis iformatio. r J Prev Soc Med 1976; 0: 180-186. 10 World Health Orgaizatio. Iteratioal classificatio of diseases, ith ed. Geeva: WHO, 1977. 11 Office for Populatio Cesuses ad Surveys. Tabular list of the classificatio of surgical operatios ad procedures. Fourth revisio. Lodo: HMSO, 1990. 12 NHS Executive. Cliical codig toolbox. Loughborough: Iformatio Maagemet Group, NHS Executive, 1991. 1 NHS Executive. Natioal cliical codig stadards. Loughborough: Iformatio Maagemet Group, NHS Executive, 1995. 1 NHS Maagemet Executive Resource Maagemet Uit. Proposals for Eglish casemix groups. Lodo: NHS Maagemet Executive, 1991. 15 NHS Executive. Comparative cost data: the use of costed HRGs to iform the cotractig process. EL(9)51. Leeds: NHS Executive, 199. 16 Fetter R, Shi Y, Freema JL, verill RF, Thompso ID. Case mix defiitio by diagosis related groups. Med Care 1980; 18(Suppl 2): 1-9. 17 udit Commissio. Settig the records straight: a study of hospital medical records. Lodo: udit Commissio, 1995. 18 Wyatt JC. Hospital iformatio maagemet: the eed for cliical leadership. r Med J 1995; 11: 175-180. ccepted o 2 September 1997