ASSESSING HEALTHCARE SURGICAL PERFORMANCE USING DATA ENVELOPMENT ANALYSIS APPROACH

Transcription

1 Indian Journal o Economics & Business, Vol. 13, No. 2, (2014) : ASSESSING HEALTHCARE SURGICAL PERFORMANCE USING DATA ENVELOPMENT ANALYSIS APPROACH BISWADIP GHOSH * AND ABEL A. MORENO Abstract Clinicians and hospital administrators rely on inormation models or healthcare management and decision-making. However, healthcare surgical perormance measurements include both qualitative and quantitative data, oten with conlicting and interdependent variables. As a result many statistical modeling approaches can break down with healthcare data. Other more resilient algorithms, such as Data Envelopment Analysis (DEA) and Fuzzy Composite Programming (FCP) hold promise to address these issues. This paper applies DEA to comprehensively assess the surgical perormance. The results o the DEA model are compared to the results obtained rom a prior uzzy composite programming (FCP) analysis to establish additional validity. Keywords: Healthcare Ddisease Management, Multiple Criteria Decision Making (MCDM), Data Envelopment Analysis (DEA) I. INTRODUCTION The use o inormation systems (IS) in healthcare organizations is on the rise. Such applications include knowledge management systems, decision support systems and reporting systems based on patient record management systems. Among the reasons or this trend are pressures to reduce costs, which have been growing at an unsustainable rate and to improve the quality o healthcare. Also these systems can help healthcare proessionals to cope with inormation overload and to learn about and utilize current research developments into their practice. Reports indicate that several healthcare organizations are proceeding to introduce evidence-based medicine and disease management practices by implementing inormation systems based on this clinical inormation (McGrath, et al., 2008). The recent increase in the use o Electronic Health Record (EHR) systems in health care acilities has resulted in a huge amount o clinical data being collected and available online. Such data is presenting opportunities or creating inormation systems or various * Computer Inormation Systems, Metropolitan State University o Denver, Denver, USA, bghosh@msudenver.edu; morenoa@msudenver.edu

2 208 Biswadip Ghosh and Abel A. Moreno healthcare organizational management and decision making purposes (Figure 1). Personnel at multiple levels in a healthcare organization can rely on such inormation systems to create and deploy analytical models that acilitate decisionmaking. For example, medical chies and hospital directors need to track resource utilization and outcomes o selected treatment and procedures and plan unit based resource allocation and standardized procedures (Epstein, 2006). Healthcare system policy makers also need inormation rom across a healthcare network to make strategic decisions on standardization o treatment protocols and procedures. Clinicians need historical patient outcome inormation to acilitate decisions on elective treatment (e.g. elective surgeries) and judge the suitability o treatment options and medical procedures or a presenting patient. Figure 1: Patient Data Collection, Aggregation and Processing in the Health Inormation System A classiication o inormation systems that acilitate the processes o decision making in an organization are reerred to as Decision Support Systems (DSS). Most DSS oer managers unctionality intended to support all phases o decision making intelligence, design, choice and implementation. DSS technologies support- (1) the general goals o reducing the uncertainty in the decision making process, such as raming the right questions and problem(s) to solve, (2) building a model to evaluate choices and estimating the impact o the choices on one or more objectives and (3) the capability to evaluate changes in assumptions, model inputs and parameter values on a chosen decision. All activities involve the eicient and accurate collection, management, processing and application o data/inormation to the decision making process steps. Data Envelopment Analysis (DEA) is a useul modeling platorm or complex decision making scenarios, as it allows or use o dierent types o data which have large variability in the data set. Real lie situations such as in healthcare organizations are oten dierent because the actual values o the selected measurement criteria may exhibit variability as well have imprecision in the way

3 Assessing Healthcare Surgical Perormance using Data Envelopment they are collected. Statistical data analysis techniques are able to account or variability but may not work well with imprecision, as well as criteria that are not statistically independent (e.g. surgical wait time and complications). By using DEA, an area/volume is used to represent each scenario, instead o a single point (statistical approach) to get a more complete classiication o each scenario under variability. This leads to better decision making in these domains, such as healthcare. II. The goals o this research are as below: 1. Use Data Envelopment Analysis (DEA) to evaluate the perormance o surgical Units in 6 dierent hospitals. 2. Compare the results o DEA analysis with analysis done with Fuzzy Composite Programming (FCP) 3. Demonstrate how DEA analysis can help identiy the actors that can be worked on by the lower perorming units to improve their perormance. MEASURING HEALTH CARE QUALITY Healthcare organizations can vary greatly by size, scope, geographic dispersion, patient mix, treatment policies or medications and patient procedures. However, the end result, in eect o the success or ailure o a healthcare organization is the outcome o the diagnosis and treatment o the patients condition. Outcomes can be inluenced by other patient actors such as age, sex, severity o the disease, liestyle, body weight, blood pressure, etc. For example patient death could be an inevitable outcome in many situations and cannot always be used as an indicator o the ailure o a care process (Lezzoni, 1994). These actors determine a risk actor, which is dierent or each patient, patient group and patient load at a given acility. Hence in decision support systems or comparing healthcare organizations, risk adjusted outcome measures are needed. Thereore the success o a health care system should be measured by patient outcomes, such as treatment compliance, patient satisaction and risk adjusted complication rates. There are important limitations on the sole o use o patient outcomes as indicators o the process o care (Donabedian, 1976). The overall classiication measures must also include the pathways o medical care programs and structures, which are important or the delivery o the care and should be part o any measured o success or ailure o the healthcare institution. Process measures can be collected or resource planning and utilization tracking needs as the delivery o patient care is through these clinical processes (Donabedian, 1976). A care process is a worklow or a set o activities around the delivery o patient care. Care processes are delivered by dierent units in a hospital, such as ICU and pre and post surgical medical units. Institutional measures must include measures o these hospital units. Finally these care processes are highly dependent on the structure or settings in which care takes place and the instrumentalities o which it is a product o.

4 210 Biswadip Ghosh and Abel A. Moreno These structures include the administrative and related inrastructure that support and direct the provision o care, the utilization o the acilities and equipment, the nature o the medical sta, the timely access o the acilities to the patient (Donabedian, 1968). These three dimensions o medical care measurement (Donabedian, 1966) structures, processes and outcomes- are illustrated in Figure 2. Figure 2: Evaluating Medical Care III. DATA ENVELOPMENT ANALYSIS MODEL DEA assess the relative eiciency o perormance units by obtaining the maximum o a ratio o weighted outputs to weighted inputs (Charnes, et.al., 1978; Charnes and Cooper, 1985; Moreno and Lall, 1999). The undamental ormulation or the relative eiciency o a perormance unit is as ollows: 3 max h 0 s r 1 m i l u y u y subject to j n u v r s i m v x r v x i r r 1 : 1; 1,..., r, t 0; 1,..., ; 1,...,. m i 1 i ro lo

5 Assessing Healthcare Surgical Perormance using Data Envelopment where, s indicates the number o outputs, m the number o inputs, n the number o perormance units, yrj the value o the r-th output o the j-th perormance unit, x the value o the i-th input o the j-th perormance unit and ur and vi the variable weights to be determined by the solution. Notice that the ormulation allows or multiple output (s) and multiple input (m) measures, extending the traditional single-input, single-output eiciency ratio analysis to multi-output, multi-input situations. In the DEA model, in general terms, the larger value o an output variable the better, while smaller the value o an input variable the better. However, the eiciency ratio or a perormance unit is the ratio o inputs and outputs. For output variables in the DEA model, i lower values are considered better, or or input variables i higher values are considered better, then the inverse value o that variable is used in the DEA eiciency calculation ormula. The maximum eiciency value o 1 or each perormance unit is limited in value to 1 by the n constraints. A relative eiciency value o 1 or a given perormance unit would indicate that there is no other perormance unit capable o producing better outputs with the same amounts o inputs. In this study, a hospital Unit showing a relative eiciency value o 1 would imply that or the Unit s level o inputs, no better output would be produced by any o the other hospital Units under evaluation (Moreno and Lall, 1999). IV. FUZZY COMPOSITE PROGRAMMING MODEL FCP is one o MCDM techniques, which can handle mixed indicator data (quantitative and qualitative), and also work with conlicting, uncertain and hierarchical criteria. FCP methodology was developed by Bardossy and Duckstein (1992). There have been a lot o successul applications o FCP in the DSS literature (Lee, et al., 1992; Hagemeister, et al., 1996; Ghosh, 2008; Sadip and Veitch, 2002; Prodanovic and Simonovic, 2002). The normalization is done by using the best and worst basic indicator values that are described by the ollowing equation (Lee, et al., 1992): (When is best) Or (When is best) FCP is based on a Fuzzy Composite Index (FCI). The equation is: L n j pj j w i 1 { } 1/ pj Where, Lj is Fuzzy Composite Index or the B+1 level group j o B level indicators;

6 212 Biswadip Ghosh and Abel A. Moreno w is weight o B level indicators in group j; pj is balancing actors among indicators or group j; + is the best value o ith uzzy indicators or group j; - is the worst value o ith uzzy indicators or group j; is the value o ith uzzy indicators or group j. The inal uzzy composite index, which is used or ranking, is obtained by calculating the FCI rom basic level to top level. The weight parameters or indicators at dierent levels (w) are established based on the degree o importance that decision makers eel each indicator has relative to other indicators o the same group (Bardossy and Duckstein, 1992). The balancing actors (p j ) relect the importance o maximal deviations between indicators in the same group, and determine the degree o substitution between indicators o the same group. Low balancing actors (equal to 1) are used or a high level o allowable substitution. High balancing actors (equal to 3) are used or minimal substitution (Bardossy and Duckstein, 1992). The best value ( + ) stands or the maximum possible value o the indicator, and the worst value ( - ) stands or the minimum possible value o indicator. V. RESEARCH MODEL The research model is shown in Figure 3. The ocus o this research is on the development o a uzzy decision making model to rank several hospitals in their surgical perormance. As described in section 2, the hierarchical model contains three irst level indicators o (1) care structures, (2) care processes and (3) care outcomes. Prior healthcare measurement research has proposed varied indicators or each o the above three measures (Dlugacz, 2006). Items that measure care structure could include indicators such as accessibility, utilization and the training and experience o hospital sta. In our research model, care structure is measured using 2 quantitative indicators distance in miles o the patient s reported residence to the hospital and the average wait time or a procedure in weeks. The indicators to measure process o care typically include the patient turnaround time in various departments and activities length o stay in various Units, operating room turnaround time, patient throughput using admit and discharge times, etc. In this model, indicators or the process o care measurement include measures o average overall length o stay in days, the average duration o surgery in hours and percentage o surgeries involving same day discharge. The indicators in the measurement o outcomes were based on using a combination o quantitative and qualitative survey data rom discharged patients. The quantitative measures include the surgical volume o the hospital in number o cases handled per month, the risk adjusted complication rate or a monthly period and the percentage o patients in compliance with post discharge prescribed medication or the monthly period. The qualitative measure include data rom a patient survey or two questions (using a Liekert scale o 1-7) (1) Level o

7 Assessing Healthcare Surgical Perormance using Data Envelopment Figure 3: Research Model satisaction with the treatment provided and the (2) whether the patient elt that the perceived beneits o their treatment outweighed the risks involved. VI. RESULTS The research methodology consists o measurement o each o the indicators over the patient mix in 15 units or 6 hospitals. Each hospital unit provided a complete data set. Table 1 DEA Assessment Results Eiciency Ratios HOSPITAL Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Unit # Number o Perect Eiciency Scores (1) Perormance Rank 3th 6th 5th 4th 1st 2nd

8 214 Biswadip Ghosh and Abel A. Moreno The DEA results rom Table 1, suggest that Hospital 5 has the best perorming Units and hospital 2 has the worst perormance. Hospital 6 is a close second best perormance. Hospitals 1, 3 and 4 are middle perormers. Table 2 DEA Assessment Results Unit #5 in Hospital #2 Variable Hospital 2, DEA Algorithm Reerence Set Unit # 5 Hospital Hospital Hospital Hospital6 VALUE 1 Unit 7 5 Unit 6 6 Unit 3 Unit 9 Input Variables (Lower the better) Same Day Length o Stay Surgical Time Distance Wait Output Variables (Higher the better) Complication Rate Comply Volume PS PSBR The lowest eiciency ratio (0.5605) is exhibited by Unit #5 in Hospital 2. This Unit s data is analyzed urther in Table 2 to ascertain actors that help to improve its eiciency ratio. The DEA algorithm establishes a reerence set o perormance units with perect eiciency scores and then compares the unit under calculation against that reerence set. Table 2 lists the values o the input and output variables o Unit 5 o hospital 2 and the values o those variables or its reerence set o Units (Unit 7 in hospital 1, Unit 6 in hospital 6 and Units 3 and 9 in hospital 6). The comparison indicates that Unit 5 in hospital 2 is deicient in surgical time and wait time areas along with surgical volume and patient satisaction scores. (A) FCP Results at Hospital Level The ranking o the hospitals and the inal FCI values are shown in Table 3 (Ghosh, 2008). From Table 3, we can see the comprehensive assessment results o organization eectiveness or the six hospitals. Among these six hospitals, 5 has the best perormance, while 2 has the worst perormance (Ghosh, 2008).

9 Assessing Healthcare Surgical Perormance using Data Envelopment Table 3 FCP Assessment Results (Ghosh, 2008) HOSPITAL 1 2 FCP Index Rank 5th 6th HOSPITAL 3 4 FCP Index Rank 3rd 4th HOSPITAL 5 6 FCP Index Rank 1st 2nd The inal ranking based on Hospital perormance is close to that based on structures o care (Ghosh, 2008). For example, or units E and F are ranked as irst and second, respectively by both the overall FCI score and the structure score. The overall score and the structure score also correspond on the least eective hospitals, A and B. The above congruence in the scores or the top two and bottom two perorming hospitals indicate that structures o care plays the most important role on assessing hospital perormance in the uzzy model. Table 4 Second Level Indicators (Ghosh, 2008) No Structure Process Outcome Final FCI # FCI # FCI # Rank Other second level indicators (processes o care and outcomes o care) have less impact on measuring the hospital perormance in the uzzy model. For each o those dimensions, there were at least 3 mismatches with the overall ranking (Table 4). Under structures o care, the ranking based on wait time is the closest to that based on the structure indicator and the inal ranking (Table 5). So, wait time plays the most important role in assessing structures o care and hospital perormance in the uzzy model.

10 216 Biswadip Ghosh and Abel A. Moreno Table 5 Third Level Indicators or Structure (Ghosh, 2008) No Volume Distance Wait Time Final Rank VII.CONCLUSIONS This study aimed to build a multi-criteria decision making model using data envelopment analysis and uzzy composite programming to compare the surgical perormance o several Units in six hospitals. By drawing on past epidemiological research, criteria was selected or measuring structures, processes and outcomes o care to build the inal DEA and FCP models. Both quantitative data and qualitative data were used in the hierarchical model. As seen rom this research, both data envelopment analysis (DEA) and Fuzzy Composite Programming (FCP) are appropriate decision making model to work with mixed indicator data (quantitative and qualitative), as well as with conlicting, uncertain and hierarchical criteria. There was agreement among the results obtained rom DEA and FCP. Both algorithms, FCP and DEA ranked hospital 5 as the best perorming hospital, ollowed by Hospital 6. Both algorithms ranked hospital 2 as the worst perorming hospital. DEA allows or inding the variables in which a Unit is perorming poorly, while FCP allows or pin pointing the most important actors that play a role in hospital perormance. By analyzing the second and third level rankings in FCP, structures o care played the most important role in assessing hospital perormance. Other second level indicators (processes o care and outcomes o care) had less eect on the measurement o hospital perormance. Inside structures o care, wait time had the most impact on hospital perormance. Reerences Bardossy, A. and Duckstein, L. (1992), Analysis o a Karstic Aquier Management by Fuzzy Composite Programming, Water Resources Bulletin (28: 1), 1992, pp Charnes, A. C. and Cooper, W. W. (1985), Preace to Topics in Data Envelopment Analysis, Annals o Operations Research (2), Charnes, A. C., Cooper, W. W. and Rhodes, E. (1978), Measuring the Eiciency o Decision Making Units, European Journal o Operational Research (2), Donabedian, A. (1966), Evaluating the Quality o Medical Care, Milbank Memorial Fund Quarterly (44), pp Donabedian, A. (1968), The Evaluation o Medical Care Programs, Bull. N.Y. Academy o Medicine (44:2), pp

11 Assessing Healthcare Surgical Perormance using Data Envelopment Donabedian, A. (1976), Measuring and Evaluating Hospital and Medical Care, Bull. N.Y. Academy o Medicine (52:1), pp Epstein, A. J. (2006). Do Cardiac Surgery Report Cards Reduce Mortality? Assessing the Evidence. Medical Care Research and Review, 63 (4), Ghosh, B. (2008), Assessing Surgical Perormance in Healthcare Institutions using Fuzzy Composite Programming, 2008 IEEE International Conerence on Industrial and Inormation Systems, Kharagpur, India, Dec Hagemeister, M. Jones, D. and Woldt, W. (1996), Hazard Ranking o Landills Using Fuzzy Composite Programming, Journal o Environmental Engineer, April, 1996, pp Lee, Y. L., Dahab M. and Bogardi, I. (1992), Nitrate risk assessment under uncertainty, Journal o Water Resources, Planning and Management (118:2), 1992, pp Lezzoni, L. I. (1994), Risk Adjustment or Measuring Health Outcomes, Health Administration Press, Ann Arbor, MI. McGrath, K., Hendy, J., Klecun, E. and Young, T. (2008), The Vision and Reality o Connecting For Health : Tensions, Opportunities, and Policy Implications o the UK National Programme. Communications o the Association or Inormation Systems, 23 (33). Moreno, A. A. and Lall, V. (1999), Decision Models or Robot Selection: A Data Envelopment Analysis Approach, IJQM (5:2), August, pp Prodanovic, P. and S. Simonovic, S. (2002), Comparison o Fuzzy Set Ranking Methods or Implementation in Water Resources Decision Making, Canadian Journal o Civil Engineering (29), 2002, pp Sadip R. and Veitch, B. (2002), An Integrated Approach to Environmental Decision-making or Oshore Oil and Gas Operations, Canada-Brazil Oil & Gas HSE seminar and Workshop, March 11-12, Simon, H. A., The New Science o Management Decision Prentice-Hall, Englewood Clis, NJ, 1977.

12