Advantages and Limits of Real-Time Epidemiological Surveillance During Military Deployments: The Experience of the French Armed Forces

Transcription

1 MILITARY MEDICINE, 174, 10:1068, 2009 Advantages and Limits of Real-Time Epidemiological Surveillance During Military Deployments: The Experience of the French Armed Forces Lt Col Jean-Baptiste Meynard * ; Lt Col Hervé Chaudet ; Maj Gaetan Texier ; Lt Bruce Dupuy ; Maj Benjamin Queyriaux ; Capt Liliane Pellegrin ; Col Xavier Deparis ; Brigader René Migliani * ; Brigader André Spiegel ; Brigader Jean-Paul Boutin ABSTRACT To perform epidemiological surveillance during deployments, the French military health service has developed a real-time surveillance approach. The objective was to identify the benefits and problems of this approach. A prototype of real-time surveillance has been set up in French Guiana since Its permanent evaluation has allowed identifying strengths and weaknesses. The experience has permitted expansion of the concept to French forces in Djibouti and also development of a global approach for the whole French armed forces. Real-time surveillance has shown its usefulness for early warning during different real and simulated situations. Functional and architectural choices have permitted interoperability with allied nations. However, the information produced was only the first step of the diagnostic epidemiological situation followed by other investigations. This first step of development has highlighted the required complementarity with traditional epidemiological surveillance. INTRODUCTION The French permanent presence abroad is approximately 12,000 soldiers spread across the world with personnel usually rotating every 4 months. Depending on the deployment areas, soldiers are exposed to specific health risks. Epidemiological surveillance of this population is one of the Military Health Services missions. On the basis of longestablished objectives, the requirement for the capability to detect an unusual event to provide an early warning was raised. For this purpose, a prototype of a real-time epidemiological surveillance was set up in October 2004: the système de surveillance spatiale des épidémies au sein des forces armées en Guyane (2SE FAG). This project, carried out by the Institut de médecine tropicale du Service de santé des armées (IMTSSA-Pharo), began in 2002 in a context of international bioterrorism threats 1 and in the aftermath of the North Atlantic Treaty Organization meeting of November 2002 in Prague. During this meeting, 5 main initiatives were endorsed, including the development of military real-time surveillance systems for early warning. The specific purposes of these systems are the early detection of biologic health threats, the evaluation of their potential impact on the forces operational capability, and the providing of information to assist medical responses. Military real-time surveillance systems must take into account medical, technological, human, and organizational aspects that may be very different from the civilian situation. * Ecole du Val de Grâce, 1 place Alphonse Laveran, Paris, France. Institut de Médecine Tropicale du Service de santé des armées, Parc du Pharo, Marseille armées, France. Institut Pasteur de la Guyane, 23 avenue Pasteur, Cayenne, Guyane Française. This manuscript was received for review in December The revised manuscript was accepted for publication in June Reprint & Copyright by Association of Military Surgeons of U.S., The French surveillance system was designed to allow in near real time an epidemiological analysis of geolocated clinical cases occurring within the forces. Its main objectives were to allow the operational study of a real-time surveillance system, the evaluation of the value of such a system compared to traditional surveillance, and the identification of interoperability criteria for allied cooperation. 2 It was developed in collaboration with the Université de la Méditerranée in Marseilles and the Institut Pasteur de la Guyane in Cayenne. French Guiana has been a demonstration place since 2004 because it is an easily accessible country with high incidence rates of tropical diseases and demanding environmental conditions for people and material. 3 The 2SE FAG prototype system covers some 3,000 armed forces in this South American French overseas department (300,000 inhabitants). It has been assessed for different kinds of missions: in permanent health facilities that are the cabinets médicaux d unité (CMU) and during field exercises or operational missions. After demonstration of this system s usefulness, the Joint Staff decided to continue its use with Forces in French Guiana and to set it up in Djibouti under the name 2SE DJIB. In its last and current architecture the whole surveillance system, named Alerte et surveillance en temps réel (ASTER), encompasses several declaration and surveillance networks located in duty areas and a global epidemiologic data analysis network located in France. These systems have provided important insight into the specific aspects and requirements of military real-time surveillance systems and their evaluation. From this experience, the French armed forces developed a new method for evaluating military real-time surveillance systems. 4 After several years of experience in this domain (see timeline flowchart in Figure 1 ), the objective of this article is to present a synthesis of advantages and limitations of real-time 1068 MILITARY MEDICINE, Vol. 174, October 2009

2 FIGURE 1. Timeline flowchart describing the different steps of development and evaluation of ASTER. surveillance during French armed forces deployments. In this article we use the term real-time surveillance, keeping in mind that an epidemiological surveillance can be at best a near realtime surveillance. MATERIALS AND METHODS The French system s architecture is composed of 2 kinds of independent networks working together: a recording network, located in French Guiana, and an analysis network, located in both French Guiana and France ( Fig. 1 ). The recording network allows the input of health-related information into the system by medical officers, nurses, and paramedics. Various input terminals (PC, PDA, GPS, and satellite communication tools) may be used, depending on the situation. The data are then analyzed in real time by an analysis network, the communauté de services pour la surveillance syndromique (CS 3 ), which includes a geographical information system. The result is a system that produces automated control boards displaying health information ready for use by health commanders and military public health practitioners. It automatically defines 3 levels of health conditions using a control chart method derived from the Current Past Experience Graph (CPEG). 5 This method allows comparing the observed number of cases with historical data generated from the past 3 5 years. It gives an answer to the question Knowing the average number of expected events during a period of time, is the current situation unusual? The result of the comparison is coded 0 ( normal situation, green indicators on control boards) if the observed data are inside the historical boundaries, which are 2 standard deviations from the historical value; + if the observed data are outside the historical limits but less than 3 standard deviations ( prealarm, orange indicators); and ++ if they are outside 3 standard deviations ( alarm, red indicators). This analysis can be done globally for all forces or for each unit within the four French armies ( armée de terre, marine nationale, armée de l air, gendarmerie ). Specific feedback updated in real time is permanently available for all stakeholders in the system. This system was initially designed for fever surveillance in an equatorial area affected by many febrile tropical diseases, including dengue fever and malaria. This system is currently involved in the civilian surveillance network for dengue fever in French Guiana. The system surveillance domain has been extended to all symptoms and deaths of military interest since February This system is recognized by the Commission Nationale Informatique et Libertés, the French national commission responsible for security and confidentiality of numerical data. Since the beginning of the system s deployment, an evaluation strategy has been used to assess its efficacy and to improve its functioning before its generalization throughout the French forces. This strategy was based upon the Center for Disease Control and Prevention (CDC) framework for the evaluation of syndromic surveillance systems, 6 which explores a number of different areas. It has been followed by more specific methods for conducting an evaluation. These included Knowledge/ Attitude/Practice surveys (KAP surveys), 7 technical audits, and ergonomic studies ( Fig. 2 ). The timeliness of the system was assessed by the responses of the system to real outbreaks occurring in the duty area and by the simulation of outbreaks of aggressive or natural biological agents (e.g., a NATO exercise with simulated dysentery and anthrax outbreaks that also provided an opportunity for evaluating the interallied interoperability of the system). A statistical assessment was carried out to ascertain the agreement between the results of this real-time surveillance with the traditional surveillance system, surveillance épidémiologique dans les armées (SEA system), when reporting on 3 different categories: all cases of fever, dengue fever, and malaria. For each category, an initial paired Student s t- test was done to assess the presence of a relationship between the 2 sets of data. Then we used the Bland and Altman method of assessing agreement between data sets, 8 setting agreement limits, which give an indication of the discrepancy importance between the 2 sets of data in 95% of cases. A specific evaluation method for military systems was developed around the 3 steps classically used in evaluation: initial, intermediate, and final evaluation ( Fig. 3 ). A specific evaluation was also conducted during a dengue fever outbreak that occurred in French Guiana in 2006 to study the benefits of this kind of surveillance during the outbreak, to highlight issues identified by comparing the results of the military and civilian surveillance systems, and to discuss the interest of real-time surveillance for public health response. The CPEG was also used, with both Poisson and Student laws. 9 The statistical analysis was carried out with EPI_db, version 1.0 (Société Sarvis, Kourou, French Guiana). RESULTS At this step of development, it is possible to draw a first synthesis of the advantages and limitations of surveillance for early warning within the French forces. The main results of this surveillance are summarized in Table I. MILITARY MEDICINE, Vol. 174, October

3 FIGURE 2. network). General organization of the military syndromic surveillance system 2SE FAG (including a recording network on duty area and analytic FIGURE 3. of duty. Schematic representation of the proposed framework for evaluating military surveillance systems for the early detection of outbreaks on tours 1070 MILITARY MEDICINE, Vol. 174, October 2009

4 TABLE I. Main Evaluation Results of the Real-Time Surveillance System Within the French Armed Forces Parameters and Performances Main Results Ref. Pertinence 100.0% (44/44) of the stakeholders agree that real-time surveillance is more appropriate than traditional surveillance to achieve the objective of early warning. Feasibility Demonstrated by implementing the system in French Guiana (since 2004) and Djibouti (since 2006). 3, 4 Acceptability 67.0% (13/19) within the declaration network users in % (16/19) within the declaration network users in Ergonomics 89.5% (17/19) easily used software with desktop or laptop computer % (15/19) never used PDA in the field. Global Coherence Training program, hardware, software, and feedback system adapted to the stakeholders, 6, 7, 10 the field, and the demands of commanders. Interoperability Demonstrated during the 2006 NATO Disease Surveillance System exercise. 4, 10 Permanence 68.0% (26/38) of the users said that the system was not always available when needed. 6, 7 Security Validated by technical audits and intrusion attempts and hacking into the computer system that failed. 4, 10 Current Dynamic Extensive modifications made to the software, the training program, the feedback system. 4, 10 and the indicators supplied to the commanders. Operationality Permanent production of medical dashboards for commanders, updated automatically and in real time. 4, 10 Timeliness minutes between consultation and integration of the data in the dashboards. Validity Better for dengue fever surveillance than for malaria surveillance (in comparison with the 6, 9 mandatory traditional military system). Sensibility Not evaluated. 6, 9 Specificity Not evaluated. 6, 9 Data Quality 10.5% (4/38) of the users said they never recorded data. 57.9% (22/38) of the users said they used the system as it is supposed to be. Reliability 73.7% (28/38) of the users said that the recording software was reliable (clinical form, biological form, discharge form). Usefulness 96.0% (42/44) of stakeholders agree that early warning helps control of epidemics. Portability 83.0% (5/6) of data analysis stakeholders think that the system could be transferred, 10 everywhere. Demonstrated in Djibouti. Stability 85.0% (38/44) of the users said that technical problems occur very often (mainly because of communication problems). Costs Initial development cost: 275,000 euros. 6, 9 Annual costs: between 100,000 and 200,000 euros. Efficacy Demonstrated by detecting several outbreaks few weeks before the 9, 10 mandatory surveillance systems. Efficiency Not evaluated. 6, 9 Impact Improvement of the medical network in the field and strengthening of the medical community. 6, 10 Advantages A first and essential question is: Is real-time surveillance for early warning feasible within the armed forces? The response is positive. The early warning capacity has been demonstrated with a routine delay of 10 minutes maximum between the declaration of a case and its integration within the control boards, 6 dramatically increasing the epidemiologic response timeliness in comparison with traditional epidemiological surveillance based on weekly reporting. Cases reported by realtime surveillance may be declared in the system and detected at any time. This capacity was used during the dengue fever outbreak in French Guiana in 2006 when the 2SE FAG system detected an unusual event several weeks before other military and civilian surveillance systems, 9 regardless of the statistical technique used ( Table II and Fig. 4 ). This detection allowed the issuing of an early warning for this outbreak, enabling a quicker public health response from the armed forces mainly based on the enforcement and improvement of vector control. The civilian surveillance system in French Guiana has since introduced this kind of surveillance as part of its strategy. This early warning capacity was also demonstrated during a simulation exercise in 2005 in French Guiana with reactivity detected at 30 minutes and during the NATO exercise in 2006 with the detection, diagnosis, and countermeasure building for a simulated intentional anthrax outbreak within a 2-hour time interval. 10 Architectural and functional choices have permitted the system to be interoperable in an interallied disease surveillance system setup by NATO, as shown during the 2006 NATO exercise. During this exercise, it was easy for the NATO C3 Agency to have direct access to the required and authorized information in real time for their appropriate treatment. 10 ASTER is a French system, but the standards are mainly open source. The main standards used by the system are XML, SOAP (open industrial), and the medical interoperability standard CEN/TC251 EN (European standard). Moreover, a continuous supervision strategy and an adapted evaluation program allowed permanent enhancements of the recording tools, the training of stakeholders, the feedback system, and the production of control boards easily and directly MILITARY MEDICINE, Vol. 174, October

5 TABLE II. Results of CPEG Tests Obtained from 2SE FAG (Military Syndromic Surveillance), SEA (Military Clinical Surveillance), and CVS (Civilian Biological Surveillance) Systems, with Student and Poisson Statistical Laws, from Observed Incidence Rates of Week 41 of 2005 to Week 25 of SE FAG a (Military-Syndromic) SEAa (Military-Clinical) CVS b (Civilian-Biological) Week Student Poisson Student Poisson Student Poisson W41/ W42/ W43/ W44/ W45/ W46/ W47/ W48/ W49/ W50/ W51/ W52/ W01/ W02/ W03/ W04/ W05/ W06/ W07/ W08/ W09/ W10/ W11/ W12/ W13/ W14/ W15/ W16/ W17/ W18/ W19/ W20/ W21/ W22/ W23/ W24/ W25/ , weekly data included within historical limits; +, weekly data beyond historical limits (more than 2 standard deviations); ++, weekly data beyond historical limits (more than 3 standard deviations). a Military population: 3,000 soldiers. b Civilian population: 300,000 inhabitants. usable by the commanders. This approach provides a permanent development dynamic, strengthening actors acceptability. It was reported as good for 67% in and 84% in Portability and flexibility were positively evaluated during the deployment of the system for the NATO exercise and in Djibouti in Real-time surveillance usefulness having being demonstrated, the Join Staff has officially decided to continue its use within the armed forces in French Guiana and to continue its evaluation within the armed forces in Djibouti. Limits The first identified limitation was the inadequacy of PDAs. Only 20% of actors reported regular use of the declaration network because of limitations with the screen format and size. 8 We decided to stop their use and to replace them with rugged laptops adapted to extreme conditions. Their evaluation is currently in progress within the French armed forces in Djibouti. System reliability was another problem with 68% of the actors highlighting the fact that the system was not permanently available, mainly for technical reasons (informatic issues in particular). 6 Communication tools were also a weak part of the system for technical reasons (bad steadiness and poor quality) but also for human reasons (communication of the results must be activated on an asynchronous mode which is sometimes responsible for delays). These problems had some severe consequences on the acceptability of the recording network s actors. Sixteen percent of actors reported not completing their regular tasks in the 2007 study MILITARY MEDICINE, Vol. 174, October 2009

6 FIGURE 4. Weekly case counts of dengue fever (military clinical surveillance, SEA) and of suspected dengue fever cases (military syndromic surveillance, 2SE FAG) within the armed forces in French Guiana and weekly case counts of biologically confirmed cases (civilian biological surveillance, CVS) within the general population in 2005 and 2006 (period of study between week 41 of 2005 and week 25 of 2006). Validity was studied from the example of 3 pathologies: fevers, dengue fever, and malaria. Validity study showed that the system was less successful for malaria surveillance than for dengue fever surveillance with wider limits of agreement ( 0.36 to 3.26 for malaria in 2006 versus 0.88 to 1.85 for dengue fever the same year, Table III ). Furthermore, the high sensitivity and low specificity of syndromic surveillance observed in the dengue fever results were characteristic of these kinds of systems. The recording of syndromic information generally leads to many false positives. This lack of specificity may lead to costly false alarms. This underlined the necessary complementariness with traditional epidemiological surveillance. 6 With the only prototype located in French Guiana, the surveillance of different duty areas was impossible to study; this is now possible with the implementation of the system in Djibouti. Finally, the information produced constituted only one of the first steps of the epidemiological situation diagnosis that must be completed with other investigations to provide real alerts for which specific and adapted countermeasures must be activated. DISCUSSION Several thousand French soldiers are deployed throughout the world every year, exposing them to natural and aggressive risks. Epidemiological surveillance is a major health activity needed to maintain the operational capacity of these forces. Developing an early warning capacity is one of the priori- TABLE III. Main Statistical Results Comparing Syndromic Surveillance (2SE FAG) and Traditional Surveillance (SEA) for All Cases of Fever, Malaria, and Dengue Fever Within French Armed Forces in French Guiana in 2005 and SEA 2SE FAG SEA 2SE FAG All Cases of Fever No. Cases Reported t-test p Value Limits of Agreement 0.73 to to 3.59 (Ratios) a Dengue Fever No. Cases Reported t-test p Value < Limits of Agreement 0.91 to to 1.85 (Ratios) a Malaria No. Cases Reported b t-test p Value < Limits of Agreement (Ratios) a 0.36 to 3.26 a The Bland and Altman method of assessing agreement between data sets was used. This method allows limits of agreement to be calculated, giving an indication of how wide discrepancies between the two sets of data will be in 95% of cases. These limits are calculated as ratios, if the difference between the ratios is small, the systems are in good agreement, thus if it is large the systems are not. For instance if the limits of agreement are calculated between 0.5 and 3, this would mean that syndromic surveillance reports between 0.5 and 3 times the number of cases of traditional surveillance in 95% of cases would be a poor result as the limits are very wide. b It was not possible to perform this study for malaria in 2005; cases occurred within a single military unit that took part in the system only in MILITARY MEDICINE, Vol. 174, October

7 ties of surveillance to detect as soon as possible an unusual event with possible adverse consequences on the conduct of operations. The first 2SE FAG prototype, because of its strict evaluation, identified the advantages and limits of this kind of surveillance. The main parameters and performances of this new system have been assessed, but not sensibility and specificity because of the lack of a reference method. Its operational use allowed an early warning for a dengue fever outbreak in French Guiana and a quicker public health response by the armed forces than by the civilian authorities. 9 This evaluation emphasized the necessary redundancy with traditional surveillance because real-time surveillance didn t reach all the objectives of epidemiological surveillance. 11 This pilot phase also highlighted the fact that the development of specific tools must be carried out to take into account the current constraints encountered by forces in conditions of engagement. This mostly concerned changes in the geographical location of missions and the possibility of deployment in areas for which no sanitary data are available (no historical data). The high mobility of forces within a theater, the constant turnover of soldiers, stress and difficult work conditions in the field, the multinational character of current missions, the necessity of a permanent operating system, and a long operation time in many different areas, particularly from the deployment start, were also important issues. Other issues included the wide geographical spread of surveillance partners, the compatibility between the surveillance system and the information systems used by commanders, and the high level of security required for both military and medical data. The involvement of commanders is also a main issue. Each development or evaluation process is expensive in money and time and, therefore, represents a significant burden. The surveys cannot be set up without the previous commanders endorsement. This commander must understand the benefits of operational surveillance systems that provide information required to build the best course of action in the duty area. The challenge is to provide nonmedical decision makers with appropriate information in a form that is easy to understand and can be used directly. This has been achieved with the 2SE FAG indicators (with colored indicators for alarm levels, risk maps, and a summary of the main results). Our experience showed the necessity to adapt the different surveillance tools to the different stakeholders to strengthen acceptability of the actors. Although evaluation remains the weakest link in the public health chain, it is necessary to set up an evaluation program of the epidemiological system and to promote its continuous enhancement. The French military evaluation system is of potential interest to the entire public health community as it can present public health information in a format that individuals from outside the field of public health can understand and act on in an informed manner. 3 It could be useful for mobile civilian systems or for national security surveillance systems. The work must be done by multidisciplinary teams to deal with technical and informatic constraints in the quickest amount of time. Also, preliminary signal analysis methods must be reinforced using other statistical methods than the one already used. One or several specific methods must be proposed to allow the deployment of the analysis capacity in new areas for which no historical surveillance data are available. Finally, it should be kept in mind that real-time surveillance systems are simply a source of information, raising the alarm and assisting decision makers in the complex task of managing the health situation. Surveillance is only one of a collection of decision-making aids for medical experts, including medical intelligence, epidemiological investigations, and evolution prediction for the observed phenomena. This is the price to pay to take part everywhere and every time for the preservation of the armed forces operational capacity. ACKNOWLEDGMENTS We thank all the stakeholders for their contribution to the military real-time surveillance systems, in particular health service personnel from French Guiana and Djibouti, the Direction de la Santé et du Développement Social de la Guyane, and the Cellule inter-régionale d épidémiologie Antilles- Guyane. We also thank Lt Col Elizabeth Decker, USAF, for proofreading the transcript. REFERENCES 1. Touze JE, Richard V, Josse R, et al : [New concepts in epidemiological surveillance in French army] Bull Acad Natl Méd 2004 ; 188 (7) : discussion Meynard J-B, Texier G, Sbai Idrissi K, et al : Surveillance épidémiologique en temps réel pour les armées. Med Armees 2004 ; 32 (4) : Chaudet H, Meynard JB, Texier G, et al : Distributed and mobile collaboration for real time epidemiological surveillance during forces deployments. Stud Health Technol Inform 2005 ; 116 : Meynard JB, Chaudet H, Jefferson H, et al : Proposal of a framework for evaluating military surveillance systems for early detection of outbreaks on duty areas. BMC Public Health 2008 ; 8 : Stroup D, Wharton M, Kafadar K, Dean A : Evaluation of a method for detecting aberrations in public health surveillance data. Am J Epidemiol 1993 ; 137 : Jefferson H, Chaudet H, Dupuy B, et al : Evaluation of a syndromic surveillance for the early detection of outbreaks among military personnel in a tropical country. J Public Health 2008 :doi: /pubmed /fdn Daudens E, Langevin S, Pellegrin L, et al : Assessment of a military realtime epidemiological surveillance system by its users in French Guiana. Public Health 2008 ; 122 : Bland JM, Altman DG : Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 ; 1 (8476) : Meynard J-B, Chaudet H, Texier G, et al : Value of syndromic surveillance within the Armed Forces for early warning during a dengue fever outbreak in French Guiana in BMC Med Inform Decis Mak 2008 ; 8 : Meynard JB, Chaudet H, Texier G, Queyriaux B, Deparis X, Boutin JP : [Real time epidemiological surveillance within the armed forces; concepts, realities and prospects in France.] Rev Epidemiol Sante Publique 2008 ; 56 (1) : Spiegel A, Haus R, Berger F, et al : La surveillance épidémiologique des maladies transmissibles dans les armées. Bull Soc Fr Microbiol 2004 ; 19 : MILITARY MEDICINE, Vol. 174, October 2009