Clinical Prediction of Musculoskeletal-Related Medically Not Ready for Combat Duty Statuses Among Active Duty U.S. Army Soldiers

Transcription

1 MILITARY MEDICINE, 178, 12:1365, 2013 Clinical Prediction of Musculoskeletal-Related Medically Not Ready for Combat Duty Statuses Among Active Duty U.S. Army Soldiers Major D. Alan Nelson, SP USA*; Lianne M. Kurina, PhD ABSTRACT No evidence-based mechanism currently exists to inform U.S. Army clinicians of soldiers at risk of being found Medically Not Ready for combat duty. Historically, musculoskeletal conditions represent high-frequency medical problems among Army soldiers. We explored the feasibility of using centrally archived medical and administrative data on Army soldiers in the automated prediction of musculoskeletal-related Medically Not Ready soldiers who did not deploy. We examined 56,443 active duty U.S. Army soldiers who underwent precombat medical screening during March through December 2009 and in March Musculoskeletal problems were associated with 23.0% of nonreadiness cases in the study population. We used multivariable logistic regression in derivation cohorts to compute risk coefficients and cut points. We then applied these coefficients to covariates in validation cohorts, simulating predictions 2 to 3 months before their medical screenings. The analysis yielded c statistics ranging from 83 to 90%. The predictions identified 45 to 73% and 50 to 82% of the individual male and female outcome-positive soldiers, respectively, while obtaining 83 to 95% specificity. Our findings demonstrate the potential of Army data to create evidence-based estimates of nonreadiness risk. These methods could enable earlier patient referrals and improved management, and potentially reduce medically related nondeployment. *Decision Support Center, Office of the Army Surgeon General, Defense Health Headquarters, 7700 Arlington Boulevard, Falls Church, VA Department of Health Studies, The University of Chicago, 5841 South Maryland Avenue MC2007, Chicago, IL doi: /MILMED-D INTRODUCTION U.S. Army soldiers ordered to combat duty undergo medical screening during the 60-day period preceding deployment as part of Soldier Readiness Processing (SRP). There, clinicians examine medical records, view self-reported health information, and interview soldiers to identify those who are Medically Not Ready (MNR) for deployment. 1 While official figures for the proportions of MNR soldiers who do and do not actually deploy have not been readily available, an analysis of the soldiers who underwent SRP in June to October 2009 found that 8.0% were MNR overall. 2 At the October 2011 conference, the Army Surgeon General indicated that the MNR rate was sufficiently high to affect the Army s readiness. A senior Army physician in attendance suggested that the Army could expect MNR-related problems with meeting future personnel needs. 3 Consistent with these concerns, the early identification of medical nonreadiness is the first item in the Army s 2011 to 2016 Soldier Medical Readiness Campaign Plan. 4 A recently implemented policy dictates that prescreening can be conducted up to 120 days preceding deployment. 5 However, even then, there is no evidence-based surveillance mechanism beyond intuition and clinical experience with which to identify patients at elevated MNR risk. We devised evidence-based methodology that could underpin an automated decision support system to assist Army clinicians with this challenge. Outcomes for which other predictive methods have proven clinically useful in risk detection include methicillin-resistant Staphylococcus aureus colonization 6 and myocardial infarction among hospital inpatients. 7 In this analysis, we targeted subset of MNR cases that were associated with musculoskeletal conditions and subsequent nondeployment, an outcome we termed musculoskeletalrelated medical nonreadiness with subsequent nondeployment (MMNR). These are clinically important outcomes because musculoskeletal problems have historically represented top causes of injury and disability among U.S. Army soldiers, 8 10 including during combat duty. 11,12 METHODS Study Design This was a retrospective cohort study designed to determine whether a regression model for musculoskeletal-related nonreadiness among soldiers undergoing SRP (derivation cohorts) could be used to assign risk scores for MNR findings with associated nondeployment to combat among other soldiers at later times (validation cohorts). We answered this question by simulating a prospective risk assessment process in retrospective, longitudinal data on the total U.S. Army population. Our two derivation cohorts consisted of all soldiers who underwent precombat SRP during two 5-month periods (March July 2009 and August December 2009.) The respective validation cohorts for these derivation cohorts consisted of soldiers with SRPs in October 2009 and March We assigned 6-month exposure periods for all subjects in which time-varying, predictive data elements were tabulated. These exposure periods ended 2 months before the SRP month for both the derivation and validation cohort soldiers. This provided an equal, minimum time lag for each group between MILITARY MEDICINE, Vol. 178, December

2 predictions and the earliest medical readiness findings at SRP events among the validation cohorts. Figure 1 demonstrates these temporal arrangements of exposure and SRP in each derivation and validation cohort. The prediction time was the point beyond which no timevarying covariate information was used in prediction (i.e., at least 2 months in advance of the SRP). Regression models using derivation cohort data were thus effectively computed at this past point in time, and derived coefficients were applied to the associated validation cohort covariate values. Dataset and Study Population To conduct this analysis, we used data from the Soldier Outcome Trajectory Assessment database version 1 (SOTA v1), maintained at the University of Chicago. SOTA v1 is a novel, longitudinal dataset generated from information repositories at multiple U.S. Army supporting agencies, with personal identifiers removed. These data included comprehensive details on all SRP and out- and inpatient health care events that occurred at military facilities between July 2008 and June This date range appeared interesting for study because of the ongoing, simultaneous operations in Iraq and Afghanistan at that time. This period was one of high stress on the deployment screening process, potentially providing a rich array of predictors and outcomes. Mid-2008 was also chosen as a start point because over 1 year of electronic health record usage in the Army system had occurred by that time. 13 This suggested that mid-2008 might be a time after which we would realize useful capture of Army-wide care in digital records. To identify remote outcomes stemming from medical predictors, we were also provided data on key administrative events among these soldiers through August These data included return dates from combat and total counts of combat deployments. Data were provided by the Patient Administration Systems and Biostatistics Activity, Fort Sam Houston, Texas. This research was approved by the University of Chicago Institutional Review Board and the Tricare Management Agency Privacy Office, with the direct support of the U.S. Army Medical Command. We conducted all computations using Stata 12 (StataCorp, College Station, Texas). Having undergone precombat SRP Army-wide constituted the study s main eligibility criterion. To decrease the heterogeneity of pre-srp military experience, we restricted the analysis to regular, active duty Army soldiers with at least 6 continuous months of observed active duty as of the simulated prediction month. Soldiers activated for combat duty from the Army National Guard or Army Reserve were not included in this dataset; therefore, the findings may not generalize to these persons. We included soldiers of all races, ethnicities, and ages. After integrating these criteria with the temporal design displayed in Figure 1, the total, eligible study population consisted of 56,443 U.S. Army soldiers who underwent precombat SRP between March 2009 and December 2009 and during March Outcome Variable and Associated Methods The main binary outcome variable among the validation cohort members, MMNR, represented the presence of (1) an MNR designation associated with one or more musculoskeletal conditions and (2) subsequent nondeployment for the entirety of the 12-month tours that were typical for those deploying in the studied timeframe. 14 The binary outcome used for regression analysis in the derivation cohort regression models was termed MMNR2 and met criterion (1) only. No long-term deployment status information was incorporated into MMNR2 FIGURE 1. Temporal relationship between a derivation cohort consisting of subjects in 5 aggregated months of Soldier Readiness Processing (SRP), and its associated validation cohort, when predicting medical readiness outcomes among U.S. Army Soldiers MILITARY MEDICINE, Vol. 178, December 2013

3 to simulate a real-world approach and make risk assessments based on the most complete and up-to-date predictive information available at a given time point. We took this approach because deployment statuses for soldiers found MNR for any reason would have taken up to a subsequent year (the duration of a deployment) to confirm in the real-world, prospective case. In combination with the wait that would also be required to observe for deployments among validation cohort members, this would result in a study taking 2 years or more to conduct. It appeared that using contemporary SRP findings would be more desirable than simulating such a wait in the retrospective data. We, therefore, eliminated the effective wait among derivation cohort members by using MMNR2 as a proxy variable for MMNR. We used the adjusted correlates of MMNR2 among derivation cohort members to predict both nonreadiness statuses and associated nondeployment in the validation cohorts. Both MMNR and MMNR2 required a nonreadiness designation with the identification of at least one musculoskeletal condition. We based the latter on the presence of an orthopedic referral recommendation and/or on statements of musculoskeletal problems in the electronic SRP records. We identified long-term deployment statuses among validation cohort soldiers using SOTA v1 s administrative component, which included total counts of deployments and end of last combat tour dates up to August We established a standard post-srp period in which soldiers were observed then were observed for returns from combat. We allowed for 12 months of combat duty, a possible 2-month delay between SRP and deployment, and a further 3 months for incidental delays in deployment and/or returns home. This totaled 17 months, which we subtracted from the August 2011 dataset end date to establish the last validation cohort s SRP month as March Given the dataset s start date and the requirement of 6 months of exposure time, the earliest possible validation cohort s SRP month was October We applied the same 17-month post-srp observation period for deployment statuses to the October 2009 validation group. The arrangement separated the two validation cohorts by 5 months and placed them in different years and seasons, allowing us to observe predictions at different times. Predictors In determining the covariate content of each regression model, our objective was to support maximally effective automated predictions conducted solely by a computer, rather than short lists of factors to be assessed directly by clinicians. Our intent was therefore not to create an intuitive set of covariates or a parsimonious model. This process yielded a simple risk score for each subject regardless of the number of predictors. Variables were retained in regression models only if predictive performance improved. We devised predictors based on the characteristics of the soldiers in each of the derivation cohorts. Time-varying values were drawn from each associated exposure period. We initially assessed possible covariates by using c 2 tests of association between candidate predictors and MMNR2 to maximize the use of recent, relevant information at the time of each prediction. We retained variables for the multivariable model if there were significant univariate associations at p < 0.05 for at least one category of the covariate. Variables were retained in the final regression models only if they contributed to predictive performance in the derivation cohorts as defined later. Among available demographic covariates, we explored the effect of female sex and age over 40, two factors associated with elevated MNR risk in a prior, unadjusted analysis. 2 We used an interaction term for sex and age as of the prediction month. We also explored race, marital status, Hispanic ethnicity, and the soldier s number of dependents as predictors. Of these, only the number of dependents improved model performance in the derivation cohorts. We also explored the predictive use of the soldier s number of past combat deployments. It was not a useful MMNR2 predictor and was not retained. Other military service-related factors retained in the final models included military rank, the installation where SRP took place, total service time, and military occupational specialty. To identify relevant medical conditions that may have led to later nonreadiness, we used primary International Classification of Disease, Ninth Revision (ICD-9) codes assigned at health care events. These were diagnoses made during the exposure period during health care encounters and admissions at military clinics and hospitals. We based predictor covariates on categorized counts of visits by ICD-9 code during the exposure period. Useful factors for both models included categorized counts of outpatient visits for any medical reason, for physical examinations, and for physical therapy. Other useful covariates represented musculoskeletalspecific outpatient diagnosis counts such as complaints of the spine/pelvis, knee, and shoulder. Counts of depressionrelated visits were present in the initial cross tabulations and proved useful in the models. We also constructed binary covariates for the presence of inpatient care in which an orthopedist was the attending physician, and we controlled for the presence of any inpatient care and its interaction with sex. The model for August to December 2009 further controlled for all main effects of and interactions between sex and shoulder-, depression-, and ankle-related encounters. This model also included a sex/depression visit interaction term and modified categories for dependent counts, military occupation, and physical examination counts. Regression Models and Postregression Analysis of Prediction Performance We computed each derivation cohort s dedicated multiple logistic regression models against the MMNR2 outcome. MILITARY MEDICINE, Vol. 178, December

4 The format and terms of each regression equation were as follows: p ln ¼ b 1 p 0 + b 1 X 1 + b 2 X b N X N where ln( p/1 p) is the natural log of the odds of a binary MNR outcome, p its probability; b 0 the constant/intercept term; b N the coefficient value for each of N covariates; and X N the covariate value for each of N covariates. The first metric used to evaluate model performance was the concordance or c statistic, which quantifies how accurately a predictive model places subjects into a rank order of the risk of the outcome. 15 The potential benefits of the c statistic in prediction include its simplicity, interpretability, independence of parametric distributions, and reliability under a range of conditions. 16 Unlike other measures of regression model performance such as R 2, the c statistic does not depend on the frequency of outcomes. 17 Useful models produce c statistics greater than 0.5 and as close as possible to 1.0, thus higher values indicate better performance. 18 We computed the area under the receiver-operating characteristic (ROC) curve (AUC) to obtain the c statistic. The AUC represents the percentage of possible comparisons of randomly selected pairs of soldiers (one with the outcome, one without) in which the ranking of risk was concordant with the outcome s presence or absence. 19 The AUC is synonymous with the c statistic in the case of a binary outcome variable. 20 We found that this metric is very useful when building a model to test the incremental effect of adding, removing, or reconfiguring covariates. However, we found that the c statistic s single numeric value somewhat limits its use in assessing performance because it does not communicate the relative numbers of true versus false positive and negative findings. We, therefore, also computed the sensitivity and specificity values associated with the predictions for these soldiers. Sensitivity is generally defined as the proportion of tested subjects who experience an outcome (denominator) and are found positive by the test (numerator). In this way, specificity represents the proportion of tested subjects without the outcome designated as outcome free (negative) by the test. Although we were testing for future events, the same basic principles hold for assessing the success of the prediction. In our results, sensitivity denotes the percentage of soldiers later found MMNR who were correctly designated as high risk by the prediction (true positives). Specificity indicates the percentage of non-mmnr soldiers accurately classified as low risk (true negatives). We used cut points to dichotomize the risk scores generated by the derivation group models. Each cut point therefore divided sex-specific derivation and validation cohorts into high- or low-risk groups. Given that large numbers of false-positive findings would limit the clinical use of a screening tool based on this methodology, the statistical goal was to pursue relatively high specificity, reducing false positives and producing manageable lists of high-risk soldiers. The cut points we chose corresponded to specificity values of approximately 85% (cut point 1) and approximately 95% (cut point 2) in each sex-specific derivation group. We generated risk scores among the validation cohort soldiers by multiplying the derivation cohort regression coefficients and validation cohort covariate values and converting the resulting per-soldier log odds values to risk scores. We then assigned high versus low risk statuses on the basis of the derivation model s risk cut points. Then we were able to calculate AUC/ c statistics, sensitivity, and specificity using the known, eventual outcomes. It is important to note that the results realized represent the specific cut-point choices made, and that a range of such cut points might be selected depending on situation-specific needs. RESULTS Table I lists selected demographic and administrative characteristics of the subjects in the total study population. Relative proportions by age and geographic locations were very similar for the two sexes. There were higher proportions of junior officers among women compared to men and warrant officers among men versus women. Among the 56,443 subjects, a higher percentage of women than men were found MNR for any reason. There were 3991 (7.1%) such MNR subjects overall, including 2964 (5.9%) of the men and 1027 (16.2%) of the women. However, musculoskeletal conditions necessitating referrals were identified among a higher percentage of MNR men than MNR women. Musculoskeletal problems were identified in 918 (23.0%) of all MNR cases, 766 (25.8%) of MNR male soldiers, and 152 (14.8%) of female MNR soldiers. The model-building process yielded 18 final predictive factors (such as sex or total outpatient visit counts) for the March to July 2009 derivation cohort. Across these factors, there were 61 distinct nonreference categories that each mathematically constituted a single covariate. For the August to December 2009 cohort, there were 22 such useful predictive factors with 63 nonreference categories. Table II lists selected odds ratios for medical history-related predictors common to both multiple logistic regression models. These are factors a clinician using this methodology might quickly appreciate in reviewing a high-risk soldier s health record. When controlling for the other covariates, the number of health care visits per unit time for any reason was the strongest and most consistent outpatient diagnostic predictor in each derivation cohort. Some covariate categories without statistical significance were retained in the models because their effect sizes, even if modest, were sufficient to produce superior predictive performance compared to models omitting them. Table III displays performance metrics for the predictive models that assigned risk scores to the validation cohort members, stratified by sex and by risk cut point. As is clear, substantial proportions of individual soldiers destined to be 1368 MILITARY MEDICINE, Vol. 178, December 2013

5 TABLE I. Selected Characteristics of U.S. Army Soldiers Eligible for Prediction of Musculoskeletal-Related Medical Readiness Outcomes (N = 56,443) Males Females n = 50,100 n = 6,343 Characteristic No. (% a of n) No. (% a of n) Age, Years b [Mean/Median] [28.6/27] [28.7/27] 21 and Younger 7,538 (15.1) 929 (14.7) ,587 (51.1) 3,265 (51.5) ,332 (26.6) 1,629 (25.7) 41+ 3,643 (7.3) 520 (8.2) Military Rank b Enlisted Grades ,684 (49.3) 2,931 (46.2) Enlisted Grades ,169 (26.3) 1,597 (25.2) Enlisted Grades 7 9 2,980 (6.0) 331 (5.2) Warrant Officer Grades 1 5 1,790 (3.6) 161 (2.5) Commissioned Officer Grades 1 3 5,117 (10.2) 1,000 (15.8) Commissioned Officer Grades 4+ 2,360 (4.7) 323 (5.1) PDHA Location Fort Bliss, Texas 4,598 (9.2) 472 (7.4) Fort Bragg, North Carolina 4,180 (8.3) 528 (8.3) Fort Campbell, Kentucky 4,163 (8.3) 396 (6.2) Fort Carson, Colorado 5,051 (10.1) 529 (8.3) Fort Drum, New York 3,118 (6.2) 263 (4.2) Fort Hood, Texas 2,305 (4.6) 608 (9.6) Fort Lewis, Washington 3,932 (7.9) 397 (6.3) Fort Riley, Kansas 4,513 (9.0) 509 (8.0) Fort Stewart, Georgia 7,057 (14.1) 927 (14.6) Schofield Barracks, Hawaii 1,534 (3.1) 267 (4.2) All Other Sites Combined 9,649 (19.3) 1,447 (22.8) Eligibility criteria: screened for medical readiness at precombat Soldier Readiness Processing (SRP) during March to December 2009 or March 2010; completed 6 months of regular active Army duty as of the prediction month. a Percentages may not equal 100% due to rounding. b These time-varying factors were based on statuses as of the prediction month. found MMNR were identified while preserving high specificity. For example, at cut point 1 (specificity ³ 83%) for outcomes in October 2009, the methodology identified 73% of validation cohort males and 82% of females later found MMNR. At the higher-specificity value (³94%, cut point 2), 45% of males and 55% of females were identified. The limited information conveyed by the c statistic alone was evident in these results. For example, the c statistic was only slightly higher for women than men in the October 2009 prediction (0.90 versus 0.88). However, sensitivity was markedly higher and specificity differed by only 1% for women than men at cut point 2, as noted earlier. Our findings also indicate that the c statistic s use may be limited when comparing populations. The c statistic was substantially lower for women than men in the March 2010 prediction (0.83 versus 0.87). Yet, at cut point 2, sensitivity was higher for women than men (0.50 versus 0.45) at identical specificity. Figure 2 illustrates the ROC curves from which we obtained the validation cohort c statistics. We created these curves by plotting 40 quantiles of risk (every 2.5%) across the range of risk scores in each validation cohort. The diagonal line indicates the curve that would be obtained if the AUC was exactly 0.5, equivalent to a random guess of outcome statuses. The stair-step morphologies of the curves for females were due to their relatively low numbers of true positives compared to males, especially in March 2010, limiting the possible sensitivity values. None of the curves dipped below the chance diagonal, indicating that the methodology provided predictive power across the entire range of sensitivity and specificity values. DISCUSSION The goal of this study was to generate models to support the prediction of musculoskeletal nonreadiness with subsequent nondeployment among active duty U.S. Army soldiers. Based on centrally archived military data, our approach successfully predicted substantial proportions of MMNR validation cohort soldiers. Further, this work may support the new pre-srp screening policy in which soldiers might be assessed up to 120 days preceding combat deployment. Specifically, our findings suggest that evidence-based judgments of MNR risk can indeed be made 2 to 3 months before the SRP. This is up to 5 months before a scheduled deployment and up to a year or more before MNR-related nondeployment becomes evident. Across the predictions for males and females, our computations identified between 45 and 82% of the individual subjects destined for such outcomes while preserving 95% and 87% specificity, respectively. The c statistics obtained in this project ranged from 83 to 90%. These findings were comparable to those obtained in other research on predicting cardiac arrest among inpatients. At 89.9% specificity, the researchers obtained 53.4% sensitivity, a c statistic of 84%. 7 The differences in the optimal covariate content of each model and the performance realized for the two predictions also suggested that periodic revalidation would be required in the real-world use of this methodology. The absence of some theoretically useful data elements represented technical limitations of this study. First, the absence of deployment start dates prohibited us from examining late deployment after an MNR designation. This limitation compelled us to use end of combat dates, and it temporally defined the required span of the longitudinal data. In addition, the database did not include any information on purchased care at nonmilitary facilities or nonactive duty soldiers. Further, comprehensive digital data on duty restrictions or profiles were not available at the time of these data. We plan to address these limitations in data updates that will include more data elements and more sources, including data on care purchased from civilian providers. We experimented with a wide range of model types, variable configurations, and resulting predictions (not reported) to determine the best models. Interestingly, the combination of 6-month exposure windows and derivation cohorts based on 5 months of SRP events produced the best model performance versus shorter or lengthier intervals. This suggested that chronic health care patterns are less predictive of MILITARY MEDICINE, Vol. 178, December

6 TABLE II. Odds Ratios (95% Confidence Intervals) for Selected Health Care Utilization Variables in Multiple Logistic Regression Models Used to Compute the Risk of Musculoskeletal Medical Nonreadiness Among Derivation Cohorts (N = 51,093) Defined by Active Duty U.S. Army Soldier Readiness Processing (SRP) Dates Derivation Cohorts March July 2009 August December 2009 n = 22,132 n = 28,961 Health History Covariates 443 MMNR Cases (2.0%) 398 MMNR Cases (1.4%) Visits for Any Reason a None 1.00 (Reference) 1.00 (Reference) ( )* 1.50 ( ) ( )*** 2.72 ( )*** ( )*** 5.10 ( )*** Visits for Spine or Pelvis b None 1.00 (Reference) 1.00 (Reference) ( )*** 1.55 ( )* ( )*** 2.34 ( )*** Visits for Knee b None 1.00 (Reference) 1.00 (Reference) ( ) 1.97 ( )*** ( )** 2.20 ( )*** Visits for Physical Therapy a None 1.00 (Reference) 1.00 (Reference) ( )** 1.59 ( )** ( )*** 1.53 ( ) Both final models controlled for the number of dependents; main effects of and the interaction between sex and age; military rank; military occupation; years of service; outpatient encounters due to physical examinations, physical therapy, and depression; the presence of inpatient orthopedic care; the presence of any inpatient care and its interaction with sex; and the soldier s location. The model for August to December 2009 further controlled for all main effects of and interactions between sex and shoulder-, depression-, and ankle-related encounters. *p < 0.05; **p < 0.01; ***p < a The variable coded for the number of outpatient visits (OPVs) per month for the reason listed during the 6-month exposure period, rounded to 0.5. b The variable coded for the total number of OPVs for the reason listed during the 6-month exposure period. TABLE III. Performance Metrics for Predictions of Musculoskeletal-Related Medical Nonreadiness with Subsequent Nondeployment (MMNR) among Validation Cohorts of Active Duty U.S. Army Soldiers Organized by Sex and Soldier Readiness Processing (SRP) Dates Validation Cohorts Males Females October 2009 March 2010 October 2009 March 2010 n = 7050 n = 4774 n = 762 n = 576 Metric 55 MMNR Cases (0.8%) 29 MMNR Cases (0.6%) 11 MMNR Cases (1.4%) 4 MMNR Cases (0.7%) c statistic Cut Point a 1 Sensitivity Specificity Cut Point 2 Sensitivity Specificity a Cut points 1 and 2 were predicted values that produced, respectively, 0.85 and 0.95 specificity in the derivation cohort associated with each validation cohort when used to define low versus high risk. MNR than the soldier s health care patterns directly preceding the SRP. Some clinical prediction tools, such as the Framingham cardiovascular event risk calculators 21 and the Ottawa ankle rules, 22 require that providers directly ascertain a relatively low number of easily identified factors. In contrast, our approach used automated risk computation based on a wide array of predictors and eliminated the need to obtain any new information or to enter data during a clinical encounter. Electronic decision support system characteristics that have been shown to improve patient care include support during workflow, availability at the time of decision making, and computer-based support. 23 Our method could easily accommodate each of these parameters by presenting a soldier s precalculated risk level during the health care encounter. This approach automatically draws on a vast array of useful information an Army provider typically would not be able to quantify or easily integrate MILITARY MEDICINE, Vol. 178, December 2013

7 FIGURE 2. Areas under the receiver operating characteristic curve (AUC) for predictions of musculoskeletal-related medical nonreadiness among male and female active duty U.S. Army Soldiers undergoing Soldier Readiness Processing in October 2009 and March The clinical utility of the risk scores can be demonstrated in the prediction of October 2009 outcomes among males when using a derivation cohort specificity of 95%. Over 2 months before a unit s SRP, the methodology would create a highrisk male list numerically equal to about 5% of the males scheduled to undergo SRP. This would equal about 50 male soldiers out of a combat arms battalion of around 1000 soldiers. On average, the list would include about 45% of the male soldiers destined to be found MMNR. This information could be requested on demand from a stand-alone system or integrated into an existing resource available to providers. Among the high-risk soldiers, there would be some with well-known medical statuses and others whose elevated risk might constitute a surprise. The provider could then conduct a chart review or schedule follow-up for the latter group, identifying patients in need of further care and referrals. MILITARY MEDICINE, Vol. 178, December

8 Among false-negative soldiers missed by the instrument, the eventual outcome (a finding of musculoskeletal-related medical nonreadiness at SRP, and associated medical evaluation) would presumably not change. They would receive customary care, thus this methodology would not place patients at risk of nontreatment if used in actual medical screening before the SRP. Newly assigned clinicians at Army units, such as those subject to the Army s Professional Filler System, might find such information especially useful. Professional Filler System providers are assigned to Army units just before major training events and combat deployments. 24 They face the daunting challenge of quickly identifying unit medical needs during the short-time window before deploying. Our methodology would provide them vital information at a point when they lack familiarity with the patient population, but must make critical decisions about it. In conclusion, we find much to recommend this methodology for further research and possible use in pre-srp screening. Risk estimation for female soldiers may benefit from more focused development in light of the reduced statistical power associated with their low proportion of the population. Future research will focus on incorporating more covariates and temporally expanded data to improve predictions for all soldiers. We anticipate that a useful electronic decision support system would support the ability to predict across a range of possible forms of MNR. Because the methodology presented here focuses solely on musculoskeletal MNR, our ongoing research includes applying these methods to other MNR outcomes and all-causes MNR, on which reports will be forthcoming. ACKNOWLEDGMENTS The authors gratefully acknowledge the contributions of the following individuals during the early development of the data resources and methodologies used in this project: Ari Robicsek, Diane Lauderdale, and Ronald Thisted. REFERENCES 1. Department of Defense: Department of Defense Instruction Deployment Health, 2011, pp Available at whs/directives/corres/pdf/649003p.pdf; accessed March 03, McPherson M: Automated soldier medical readiness: prevalence of Medically Not Ready using data from the electronic Soldier Readiness Processing system (esrp). Houston, Texas: University of Texas School of Public Health, ProQuest paper AAI Available at accessed May 19, Sample D: Army wants more soldiers back on deployable status. Army homepage article. Available at accessed March 03, US Army Medical Command: Soldier Medical Readiness Campaign Plan , p 2. Available at news/docs/smr_cp_version_1.2.pdf; accessed February 14, Bilynsky R. ALARACT: Time Period for Completion of the Pre- Deployment Health Assessment (DD Form 2795). In press. To be available at preliminarily disseminated June 4, Robicsek A, Beaumont JL, Wright MO, Thomson RB, Kaul KL, Peterson LR: Electronic prediction rules for methicillin-resistant Staphylococcus aureus colonization. Infect Control Hosp Epidemiol 2011; 32(1): Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP: Derivation of a cardiac arrest prediction model using ward vital signs. Crit Care Med 2012; 40(4): Feuerstein M, Berkowitz SM, Peck C: Musculoskeletal-related disability in US Army personnel: prevalence, gender, and Military Occupational Specialties. J Occup Environ Med 1997; 39(1): Hollander IE, Bell NS: Physically demanding jobs and occupational injury and disability in the U.S. Army. Mil Med 2010; 175(10): Lincoln AE, Smith GS, Amoroso PJ, Bell NS: The natural history and risk factors of musculoskeletal conditions resulting in disability among US Army personnel. Work 2002; 18(2): Zouris JM, Wade AL, Magno CP: Injury and illness casualty distributions among U.S. Army and Marine Corps personnel during Operation Iraqi Freedom. Mil Med 2008; 173(3): Armed Forces Health Surveillance Center (AFHS): Medical evacuations from Operation Iraqi Freedom/Operation New Dawn, active and reserve components, U.S. Armed Forces, MSMR 2012; 19(2): Yoshida S, Bacon B: Contextual history and visual timeline of AHLTA and VISTA/CPRS products. Parsons J Inf Mapp 2008; (10): Lopez T: 12-Month deployments to reduce stress, build depth. U.S. Army homepage article. April 21, Available at article/8665/12-month-deployments-to-reduce-stress-build-depth/; accessed July 3, Ohman M, Granger C, Harrington R, Lee K: Risk stratification and therapeutic decision making in acute coronary syndromes. JAMA 2000; 284(7): Hermansen SW: Evaluating predictive models: computing and interpreting the c statistic. SAS Global Forum 2008, pp 2 4. Available at www2.sas.com/proceedings/forum2008/ pdf; accessed January 30, Steyerberg EW, Harrell FE: Chapter 8: Statistical models for prognostication. Interactive Textbook on Clinical Symptom Research, Available at research/chapter_8/ sec7/cess7pg11.htm; accessed March 01, Steyerberg EW, Harrell FE, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD: Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol, 2001; 54: Vickers AJ, Elkin EB: Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006; 26(6): Haney J, McNeil B: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143(1): Framingham Heart Study. Risk Score Profiles. Available at study.org/risk/index.html; accessed March 03, Stiell IG, McKnight RD, Greeberg GH, et al: Implementation of the Ottawa ankle rules. JAMA 1994; 271(11): Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330(7494): US Army: Army Regulation , Army Medical Department Professional Filler System, 2007, p 4. Available at accessed March 28, MILITARY MEDICINE, Vol. 178, December 2013