Author's response to reviews Title: Be Careful with Triage in Emergency Departments: Interobserver Agreement on 1,578 Patients in France Authors: Anne-Claire Durand (anne-claire.durand@ap-hm.fr) Stéphanie Gentile (stephanie.gentile@ap-hm.fr) Patrick Gerbeaux (patrick.gerbeaux@ap-hm.fr) Marc Alazia (marclouis.alazia@ap-hm.fr) Pierre Kiegel (pkiegel@ch-aix.fr) Stéphane Luigi (Stephane.luigi@ch-martigues.fr) Eric Lindenmeyer (elindenmeyer@hopital-saint-joseph.fr) Philippe Olivier (polivier@ch-avignon.fr) Marie-Annick Hidoux (Marie-Annick.HIDOUX@chicas-gap.fr) Roland Sambuc (roland.sambuc@ap-hm.fr) Version: 5 Date: 6 September 2011 Author's response to reviews: see over
Corresponding author Anne-Claire Durand Laboratoire de Santé Publique 27 Boulevard Jean Moulin 13005 Marseille France E-Mail : anne-claire.durand@ap-hm.fr Phone : 0033 04 91 32 44 72 BMC Emergency Medicine Marseille, September 6, 2011 Subject: Changes to the manuscript entitled entitled Be Careful with Screening in Emergency Departments: Interobserver Agreement on 1,578 Patients in France written by Anne-Claire Durand, Stéphanie Gentile, Patrick Gerbeaux, Marc Alazia, Pierre Kiegel, Stephane Luigi, Eric Lindenmeyer, Philippe Olivier, Marie-Annick Hidoux, Roland Sambuc. Dear Editor, We refer you to the fifth version of our manuscript. We have completely taken in accordance with the comments you've sent. For each part of the manuscript and for each change made, we took your comments, and explained the answers. Reviewer 1: Cristina Villa-Roel 1. Major Compulsory Revisions The level of training/experience/specialization of the triage nurses and physicians also needs to be described as this factor may influence their decisions and subsequently act as a confounder. Reviewer s comments: The authors described the triage nurses as trained triage nurses. Then, they clarified that the nurses didn t attend a specific training session for 1
this study. I would suggest rephrasing a little bit the first description to avoid confusions. A description like senior nurses or registered nurses (>x years of experience) would be informative. According to this comment, we added a sentence to clarify the level of experience of ED health professionals participating to our study. The principal weaknesses of this study are: 1) The comparison of triage nurse/emergency physicians categorizations at different points of the acute medical care (entry to the ED vs. end of consultation). This was partially addressed by the authors in the first paragraph of the discussion; however the fact that other studies have followed the same approach and that the data of this study were collected from a representative sample (which is not supported by a sample size calculation) doesn t justify this methodological consideration. Physicians point of decision could be related to one of the main study outcomes (hospitalization). 3) The statistical approach (reporting 95% CIs) seems meaningless when there is no description of sample size calculation. Adjusted analyses (e.g., predictive model considering confounder and interaction terms) would be necessary to support the study conclusions. Perhaps a Receiver Operating Characteristics curve would be informative. Reviewer s comments: 2) It is clear that this in an observational study; I would suggest removing the lack of a sample size calculation from the limitations section as you did one! at least based on your response to my review. The interpretation of the magnitude of the Kappa(s) you obtained (and their confidence intervals) could be facilitated by clarifying the factors that were listed in the authors response to reviews: We performed a sample size calculation retrospectively based on the methodology of Flack VF et al [2]. Data were analysed on PASS 2008. In a test for agreement between two raters using Kappa statistic, a sample size of 1,986 subjects achieves 80% power to detect a true Kappa value of 0.43 in test of null hypothesis: Kappa = 0.50 versus alternative hypothesis: Kappa <> 0.50 when there are two categories with frequencies equal to 0.70 and 0.30. This power calculation is based on a significance level of 0.050. Thus, we included 1,578 patients in our study. The last paragraph may also help the authors supporting their approach and discussing its strengths/limitations: We found six similar studies which compared different methods of categorization in the same population [3-8]. These articles showed considerable variability in levels of agreement 2
between the different methods to categorize ED visits into nonurgent or urgent cases, ranging in κ value from 0.20 to 0.74. These studies did not perform a sample size calculation. A more recent article by Sim J & Wright C. (Phys Ther 2005;85(3):257-68) maybe useful for referencing purposes. We agree with this comment. In the section Limitations, we removed the sentence we did not provide a sample size calculation because our study was observational. No similar studies were performed a sample size calculation. According to this comment, we had better explain the lack of a sample size calculation in our study, and that this had no impact for the magnitude of the Kappa. Moreover, we added the proposed reference. 3) My concerns no longer exist as the authors clarified that they met the sample size requirements to interpret a level of significance of 0.05 in their kappa findings. The results of the AUC add to the study findings and it would be worth to mention According to this comment, we added the results of the AUC in the section Results - Variability in the proportions of nonurgent ED visits and overall agreement between triage nurses and ED physicians. The method was explain in the section Methods. Reviewer 2: Michael Ardagh No revision required. Hoping this new version meets the expectations of the reviewer, we remain at your disposal for any further suggestions. For all co-authors, 3
Anne-Claire Durand References 1) Chow SC, Shao J, Wang H: Sample Size Calculations in Clinical Research. Chapman & Hall/CRC; 2007. 2) Flack VF, Afifi AA, Lachenbruch PA, Schouten HJA: Sample size determinations for the two rater kappa statistic. Psychometrika 1987, 53: 321-325. 3) Brillman JC, Doezema D, Tandberg D, Sklar DP, Davis KD, Simms S, Skipper BJ: Triage: limitations in predicting need for emergent care and hospital admission. Ann Emerg Med 1996, 27:493-500. 4) Caterino JM, Holliman CJ, Kunselman AR: Underestimation of case severity by emergency department patients: implications for managed care. Am J Emerg Med 2000, 18:254-6. 5) Frey L, Schmidt J, Derksen DJ, Skipper B: A rural emergency department. West J Med. 1994, 160:38-42. 6) O'Brien GM, Shapiro MJ, Woolard RW, O'Sullivan PS, Stein MD: "Inappropriate" emergency department use: a comparison of three methodologies for identification. Acad Emerg Med 1996, 3:252-7. 7) Lowe RA, Bindman AB: Judging who needs emergency department care: a prerequisite for policy-making. Am J Emerg Med 1997, 15:133-6. 8) Kelly LJ, Birtwhistle R: Is this problem urgent? Attitudes in a community hospital emergency room. Can Fam Physician 1993, 39:1345-52. 9) Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 1977, 33:159-74. 4