Florida Agency for Health Care Administration

FLORIDA ANNUAL PERFORMANCE IMPROVEMENT PROJECT TECHNICAL ASSISTANCE PLAN State Fiscal Year 2006-2007 Issued October 2006 Amended January 2007 Florida Agency for Health Care Administration

CONTENTS 1. Introduction... 1-1 2. Survey Results... 2-1 Florida Agency for Health Care Administration (AHCA) Results... 2-1 Survey Results... 2-1 Health Maintenance Organization (HMO) Results...2-1 Nursing Home Diversion Program (NHDP) Results...2-3 Prepaid Mental Health Plans (PMHP) Results...2-5 Provider Service Network (PSN) Results...2-7 MediPass Results...2-8 3. Technical Assistance Strategies... 3-1 Strategies for Technical Assistance to AHCA... 3-1 Strategies for Technical Assistance to Managed Care Organizations (MCOs)... 3-1 Health Maintenance Organization (HMO) Strategies for Technical Assistance...3-1 Nursing Home Diversion Program (NHDP) Strategies for Technical Assistance...3-1 Prepaid Mental Health Plan (PMHP) Strategies for Technical Assistance...3-2 Provider Service Network (PSN) Strategies for Technical Assistance...3-2 MediPass Strategies for Technical Assistance...3-2 Yet to be Operational Entities...3-2 Appendix A Statement of Intent Cover Letter and PIP Technical Assistance Assessment Survey... A-1 AHCA 2006-2007 Technical Assistance Plan Page i

1. Introduction The Florida Agency for Health Care Administration (AHCA) contracted with Health Services Advisory Group, Inc. (HSAG) to provide external quality review (EQR) services for the State s Medicaid managed care program. One component of the EQR contract is the preparation of an annual technical assistance plan. The purpose of the plan is to describe the approach to technical assistance that will be provided to AHCA and the contracted managed care organizations (MCOs). Throughout this document, the term MCO is used to represent all involved Florida Medicaid MCOs, prepaid inpatient health plans (PIHPs), primary care case managers (PCCMs), health maintenance organizations (HMOs), provider service networks (PSNs), prepaid mental health plans (PMHPs), nursing home diversion programs (NHDPs), and MediPass. The Centers for Medicare & Medicaid Services (CMS) has developed two protocols, one for conducting and one for validating performance improvement projects (PIPs). PIPs are conducted by the MCOs to assess and improve the quality of clinical and nonclinical health care services received by members and are to be evaluated by the external quality review organization (EQRO), HSAG. In compliance with the Balanced Budget Act of 1997 (BBA), the MCOs are required to conduct PIPs that are consistent with the CMS publication, Conducting Performance Improvement Projects, A Protocol for Use in Conducting Medicaid External Quality Review Activities, Final Protocol, Version 1.0, May 1, 2002 (CMS PIP protocol). HSAG s process used for the validation of PIPs is also based on the CMS publication, Validating Performance Improvement Projects, A Protocol for Use in Conducting External Medicaid Review Activities, Final Protocol, Version 1.0, May 1, 2002 (CMS PIP protocol). For the purposes of this report, technical assistance is in response to the MCO s perceived needs related to its knowledge of and education on the CMS protocols used in the conduction and validation of performance improvement projects. Technical assistance is intended to ensure that PIPs are sound, are fully compliant with CMS protocols, and result in sustained improvement in the targeted study area. HSAG determined an approach that would assess the technical assistance needs of both AHCA and the MCOs. HSAG, with AHCA approval, developed a Statement of Intent document (see Appendix A) that was provided to the MCOs. The Statement of Intent was to be completed by each MCO. The Statement of Intent had two components. The first component was developed to elicit information from each MCO as to the number and status of its PIP studies. The second component was developed as a survey for the MCOs to respond to specific technical assistance questions. The survey responses from the second component were used as the basis for this report. On Friday, August 11, 2006, 27 e-mails with a cover letter and Statement of Intent were sent to contacts at each of the MCOs. The response return date was close of business on Friday, August 25. An internal tracking document was developed by HSAG to ensure that all MCOs responded to the Statement of Intent. The MCO survey results are displayed in Section 2 of this document. HSAG conducted a direct discussion with AHCA staff members regarding their perceived technical assistance needs at an on-site meeting Tuesday, June 27th. The results of this discussion are summarized in Section 2 of this document. AHCA 2006-2007 Technical Assistance Plan Page 1-1

2. Survey Results Florida Agency for Health Care Administration (AHCA) Results Verbal communication was exchanged between AHCA staff and HSAG to determine the priority for AHCA in terms of technical assistance. AHCA is interested in having technical assistance in two areas. The first area is best practices. AHCA is interested in defining and identifying best practices. Because of HSAG s presence in eight other states, AHCA s second area of interest for technical assistance is related to MCO contract language used in other states for the validation of PIPs. AHCA is particularly interested in how another state implemented policies and procedures for its MCOs related to PIPs. Survey Results On August 11, 2006, the Statement of Intent was e-mailed to identify MCO staff members responsible for quality. A total of 27 e-mails were sent, from which HSAG received 26 completed Statement of Intent forms. The following tables outline the results received from the MCOs for each question, with the number of results recorded. Health Maintenance Organization (HMO) Results There were a total of 15 HMO entities. Two entities (behavioral health vendors), which provided the behavioral health component to a total of four HMO contracts, responded to the Statement of Intent, leading to a total of 16 responses. The 16 responses represented 14 HMOs (1 did not submit any MCO studies) and two BHO contracts. Fifteen out of 16, or 94 percent, of HMOs reported that they were accredited by an external organization. Fifty-three percent of the HMOs were accredited by the Accreditation Association for Ambulatory Health Care, Inc. (AAAHC), 40 percent were accredited by the National Committee for Quality Assurance (NCQA), and the others held accreditation status through the Utilization Review Accreditation Committee (URAC). General Questions 1. Do the HMOs have internal processes for adopting quality improvement projects? 16 0 2. Are QI staff familiar with developing study indicators? 16 0 All 16 HMOs reported having internal processes for adopting quality improvement (QI) projects and developing study indicators. Topic selection processes included, but were not limited to, meeting federal and State contract requirements; projects that would affect specific populations, such as those AHCA 2006-2007 Florida Technical Assistance Plan Page 2-1

SURVEY RESULTS addressing cultural and linguistic competency; disease prevalence or risk status; HEDIS results; or multidisciplinary quality improvement committee direction or stakeholder participation. Data Collection 1. Do the HMOs have systematic processes for collecting data? 15 1 2. Have the HMOs developed manual data collection tools? 11 4 3. Have the HMOs developed manual data collection tool instructions? 10 5 Data collection is an important aspect of conducting a PIP. Fifteen of the HMOs reported having systematic processes in place for collecting data. For the purpose of PIP studies, data collection was reported as being either administrative or manual. When manual data collection is used for a study, CMS protocols require that a manual data collection tool be used and submitted for review. Eleven HMOs indicated that they have developed manual data collection tools, and of that number, 10 reported that they had developed instructions to accompany the tool. The tool instructions are an important component of the manual data collection process and are required to be submitted with the PIP. With 27 percent of HMOs not having experience developing manual data collection tools, and 33 percent not having experience developing tool instructions, these are two possible areas for technical assistance. Data Analysis 1. Do the HMOs have processes for extracting and reporting data? 15 1 2. Do the HMOs have a data analyst/statistician who does data analysis for the studies? 15 1 3. Do the HMOs have a data analyst/statistician who does statistical testing for the studies? 13 3 4. Do the HMOs have a data analyst/statistician who does sampling for the studies? 14 2 Data analysis and interpretation of the analysis results are key components for producing a successful PIP. Fifteen of the 16 HMOs indicated that there were processes for extracting and reporting data. Data analysis and statistical testing produce the results that determine whether a PIP study has reached true or sustained improvement, which is the goal of a PIP. Fifteen of the 16 HMOs have a data analyst or statistician on staff to conduct the data analysis. Only 13 HMOs reported having a data analyst or statistician who performs statistical significance testing on the study results. Statistical significance testing is a CMS requirement for all PIP submissions. Of the 15 HMOs that reported having a data analyst or statistician, 14 of them employed sampling methodologies for determining the study population and sample size. HEDIS is a registered trademark of the National Committee for Quality Assurance (NCQA). AHCA 2006-2007 Florida Technical Assistance Plan Page 2-2

SURVEY RESULTS Interventions 1. Do the HMOs conduct causal/barrier analysis to determine intervention strategies? 15 1 2. Do the HMOs conduct cost/benefit analysis on study interventions? 9 7 The HSAG PIP review team assesses whether health plans have processes in place to strategically and rationally select and develop interventions. A causal/barrier analysis is a method that health plans can use to assess what interventions would be most successful to achieve the desired outcomes. Fifteen out of 16 of the HMOs reported using a causal/barrier analysis to determine interventions that would lead to improvement. Conducting a cost/benefit analysis on implemented study interventions was performed by only 9 of the HMOs, with 44 percent reporting that they do not do any cost/benefit analysis. Technical Assistance The HMOs expressed a strong interest in fully understanding HSAG s processes for developing a PIP and reporting on the PIP. Other areas of technical assistance needs include: assistance and examples on how to complete the study submission forms; training on causal/barrier analysis; help developing and implementing appropriate interventions; data analysis and statistical significance testing; and resources for HMOs to consult for assistance. Nursing Home Diversion Program (NHDP) Results There are a total of 13 NHDPs in Florida. Only 12 responded to the technical assistance needs assessment survey. Eight out of the 12 NHDPs, or 67 percent, reported that they were accredited by an external organization. Fifty percent of these NHDPs were accredited by AAAHC, 25 percent by the Community Health Accreditation Program (CHAP), and 25 percent by NCQA. General Questions 1. Do the NHDPs have internal processes for adopting quality improvement projects? 12 0 2. Are QI staff familiar with developing study indicators? 11 1 All of the NHDPs have processes to be used when selecting quality improvement projects. The NHDPs identified that the processes for adopting quality improvement projects take place through quality assurance, quality improvement, or ethics committees that consist of multidisciplinary teams; through the review of data analysis results related to claims, grievances, appeals, and utilization; or are based on federal and State contract requirements. Eleven of the 12 programs were familiar with developing study indicators. AHCA 2006-2007 Florida Technical Assistance Plan Page 2-3

SURVEY RESULTS Data Collection 1. Do the NHDPs have systematic processes for collecting data? 12 0 2. Have the NHDPs developed manual data collection tools? 10 2 3. Have the NHDPs developed manual data collection tool instructions? 9 3 All of the responding NHDPs have processes in place for collecting data for PIPs. Ten of the 12 programs have experience developing manual data collection tools and 9 of the 12 have developed data collection tool instructions. Data Analysis 1. Do the NHDPs have processes for extracting and reporting data? 12 0 2. Do the NHDPs have a data analyst/statistician who does data analysis for the studies? 6 6 3. Do the NHDPs have a data analyst/statistician who does statistical testing for the studies? 5 7 4. Do the NHDPs have a data analyst/statistician who does sampling for the studies? 6 6 All of the NHDPs reported that processes were in place for extracting and reporting data. However, only 50 percent of the NHDPs reported having a data analyst or statistician to conduct data analysis and sampling for the PIP studies, and only 42 percent reported having a data analyst or statistician to conduct statistical testing. Data analysis and statistical significance testing is an area where HSAG could offer technical assistance for the PIP studies. Interventions 1. Do the NHDPs conduct causal/barrier analysis to determine intervention strategies? 6 6 2. Do the NHDPs conduct cost/benefit analysis on study interventions? 5 7 Conducting causal/barrier analysis related to developing interventions is a CMS requirement for PIPs. Six of the 12 programs indicated that they were performing this type of analysis as part of the process for developing, implementing, and monitoring interventions. Only five programs conducted cost/benefit analysis on study interventions once they were implemented. With about 50 percent of NHDPs performing causal/barrier analysis and cost/benefit analysis, these are additional areas where HSAG could provide some technical assistance. AHCA 2006-2007 Florida Technical Assistance Plan Page 2-4

SURVEY RESULTS Technical Assistance The NHDPs addressed the need for technical assistance in the following areas: education regarding the development of appropriate studies for the long-term care/nursing home population; developing study indicators; developing and implementing appropriate interventions; causal/barrier analysis; data analysis and statistical significance testing; and cost/benefit analysis. Prepaid Mental Health Plans (PMHP) Results There are four PMHPs that cover 11 service areas. The responses reported in this section are specific to these service areas. The PMHPs reported that they were accredited by an external organization in 7 of the 11 service areas, or 64 percent of the service areas. A majority of the PMHPs, 71 percent, were accredited by URAC. General Questions 1. Do the PMHPs have internal processes for adopting quality improvement projects? 11 0 2. Are QI staff familiar with developing study indicators? 11 0 All of the PMHPs have internal processes for adopting quality improvement projects. These processes include: federal and State contract requirements; results from focused studies; analysis of utilization, encounter, and grievance/complaint data; or requirements based on a plan s quality improvement standards. Data Collection 1. Do the PMHPs have systematic processes for collecting data? 11 0 2. Have the PMHPs developed manual data collection tools? 11 0 3. Have the PMHPs developed manual data collection tool instructions? 11 0 Data collection does not appear to be an area with which any of the PMHPs need technical assistance. All 11 plans have processes in place for collecting data and have experience developing manual data collection tools and instructions. AHCA 2006-2007 Florida Technical Assistance Plan Page 2-5

SURVEY RESULTS Data Analysis 1. Do the PMHPs have processes for extracting and reporting data? 11 0 2. Do the PMHPs have a data analyst/statistician who does data analysis for the studies? 11 0 3. Do the PMHPs have a data analyst/statistician who does statistical testing for the studies? 7 4 4. Do the PMHPs have a data analyst/statistician who does sampling for the studies? 6 5 Extracting and reporting data is not an area with which the PMHPs need technical assistance. All of the plans have data analysts or statisticians to do the data analysis for the PIP studies. Seven of the 11 plans use an analyst or statistician to perform statistical significance testing, and 6 of the 11 plans have an analyst or statistician who conducts sampling for the studies. With 36 percent of PMHPs not performing statistical significance testing and 46 percent not using sampling methodologies, these are potential areas for technical assistance. Interventions 1. Do the PMHPs conduct causal/barrier analysis to determine intervention strategies? 11 0 2. Do the PMHPs conduct cost/benefit analysis on study interventions? 11 0 All of the PMHPs use causal/barrier analysis to assist in the development and implementation of the study interventions, and all of the PMHPs perform cost/benefit analysis on the interventions once they have been implemented. The assessment and monitoring of interventions is not an area with which the PMHPs need assistance. Technical Assistance The PMHPs identified technical assistance needs in several areas. These areas include assistance with data analysis and statistical significance testing, and completing the PIP form for the initial submission. AHCA 2006-2007 Florida Technical Assistance Plan Page 2-6

SURVEY RESULTS Provider Service Network (PSN) Results At the time of HSAG s survey, there was one active PSN. The following outlines the results from the one returned survey. General Questions 1. Does the PSN have internal processes for adopting quality improvement projects? 1 0 2. Are QI staff familiar with developing study indicators? 1 0 The PSN reported that the quality improvement team, through the Quality Improvement Committee, leads all QI activities as directed by the Subnetwork medical directors. Findings from QI studies are presented at QI committee meetings. Results are also shared with the Joint Operations Finance/Executive Committee of the PSN, as well as with hospital system management and executive staff. The study topics for the PSN are HEDIS measures whenever possible. In addition, the PSN uses AHCA methodology when assigned. The PSN reported that, to date, no PIPs have been required or provided. Data Collection 1. Does the PSN have systematic processes for collecting data? 1 0 2. Has the PSN developed manual data collection tools? 1 0 3. Has the PSN developed manual data collection tool instructions? 1 0 The PSN reported that it had processes in place for collecting data. The PSN also reported experience with developing manual data collection tools and instructions. Data Analysis 1. Does the PSN have processes for extracting and reporting data? 1 0 2. Does the PSN have a data analyst/statistician who does data analysis for the studies? 1 0 3. Does the PSN have a data analyst/statistician who does statistical testing for the studies? 1 0 4. Does the PSN have a data analyst/statistician who does sampling for the studies? 0 0 AHCA 2006-2007 Florida Technical Assistance Plan Page 2-7

SURVEY RESULTS The PSN responded that processes were in place for extracting and reporting data. The PSN has a data analyst who conducts data analysis and statistical testing for studies. The PSN reported that the use of sampling methodology by its data analyst was not applicable for its studies. Interventions 1. Does the PSN conduct causal/barrier analysis to determine intervention strategies? 0 1 2. Does the PSN conduct cost/benefit analysis on study interventions? 0 1 The PSN reported that causal/barrier analysis was not used to determine intervention strategies. According to the survey response, cost/benefit analysis was not implemented. It was noted parenthetically for both intervention questions that statistical methods were not being used. Causal/barrier analysis and cost/benefit analysis is an area in which HSAG could provide technical assistance. Technical Assistance The PSN responded that it was not sure what its needs were at the time the survey was submitted. In addition, the PSN reported that it was unsure of its contractual obligations. The PSN did report that it was interested in potentially working on cost/benefit analysis and other statistical assessments to ensure that its methods were comparable to other health plans and PSNs. MediPass Results As a primary care case management (PCCM) model, the structure of MediPass does not support the ability to conduct performance improvement projects as specified in the CMS protocols. In addition, PIPs are not required for PCCM models by CMS. MediPass indicated in its response to the statement of intent that the technical assistance needs do not apply to PIPs; rather, the areas for needed technical assistance were identified as utilization review and medical record review. Technical assistance plans for the MediPass program are addressed in a separate report, entitled the Florida Annual Technical Assistance Plan for MediPass. AHCA 2006-2007 Florida Technical Assistance Plan Page 2-8

3. Technical Assistance Strategies Strategies for Technical Assistance to AHCA Provide a conference call with AHCA staff to discuss HSAG s experience in other states related to best practices. Provide AHCA staff with examples of other state contracts related to PIP requirements and deliverables. Provide AHCA staff with an example of another state s policy and procedure for the conduction of PIPs. Strategies for Technical Assistance to Managed Care Organizations (MCOs) Health Maintenance Organization (HMO) Strategies for Technical Assistance Provide a conference call for all HMOs to address questions related to completing the PIP study form. A sample study could be provided as a guide for the plans to reference. Provide one-on-one conference calls with HMOs that have specific questions regarding their individual PIP studies (e.g., causal/barrier analysis, statistical testing). Provide as a resource, via the Web site, a frequently asked questions (FAQ) document that would address commonly asked questions. Provide an optional e-mail notification process for the HMOs as a resource for obtaining all newly posted PIP information via the Web site. Nursing Home Diversion Program (NHDP) Strategies for Technical Assistance Provide resources for conducting PIPs specific to long-term care/nursing home populations, such as CMS protocols and NCQA quality profiles. Provide a conference call for all NHDPs to address general questions related to developing interventions, causal/barrier analysis, and cost/benefit analysis. Provide one-on-one conference calls with NHDPs that have specific questions regarding their individual PIP studies (e.g., data analysis, statistical testing). Provide as a resource, via the Web site, a FAQ document that would address commonly asked questions. Provide an optional e-mail notification process for the NHDPs as a resource for obtaining all newly posted PIP information via the Web site. ACHA 2006-2007 Florida Technical Assistance Plan Page 3-1

TECHNICAL ASSISTANCE STRATEGIES Prepaid Mental Health Plan (PMHP) Strategies for Technical Assistance Provide a conference call for all PMHPs to address questions related to completing the PIP study form. A sample study could be provided as a guide for the plans to reference. Provide one-on-one conference calls with PMHPs that have specific questions regarding their individual PIP studies (e.g., data analysis, statistical testing). Provide as a resource, via the Web site, a FAQ document that would address commonly asked questions. Provide an optional e-mail notification process for the PMHPs as a resource for obtaining all newly posted PIP information via the Web site. Provider Service Network (PSN) Strategies for Technical Assistance Provide a one-on-one conference call with the PSN to discuss CMS protocols. Provide a one-on-one conference call to address specific questions regarding PIP studies (e.g., causal barrier analysis, statistical testing, cost/benefit analysis). Provide as a resource, via the Web site, a FAQ document to address commonly asked questions. Provide an optional e-mail notification process for the PSN as a resource for obtaining all newly posted PIP information via the Web site. MediPass Strategies for Technical Assistance Strategies for technical assistance for the MediPass program are addressed in a separate report, entitled the Florida Annual Technical Assistance Plan for MediPass. Yet to be Operational Entities AHCA has and is continuing to contract with additional MCOs that will become operational after completion of this report. The new operational entities include Child Welfare Prepaid Mental Health Plan (CWPMHP); HIV/AIDS Prepaid Program (HAPP); Florida Senior Care; and new Reform plans. It should be noted that these strategies may change in the future based on specific needs determined by the entities. The following outlines the proposed technical assistance strategies: The new operational entities will be provided with PIP materials and tools that encompass CMS s protocols, HSAG s evaluation elements, and the scoring methodology. As the entities develop studies, HSAG would be available for questions related to the PIP studies under development. As the plans become operational, HSAG will provide a Technical Assistance Assessment Survey to identify needs and structures for future technical assistance, as needed. ACHA 2006-2007 Florida Technical Assistance Plan Page 3-2

Appendix A. Statement of Intent and PIP Technical Assistance Assessment Survey Introduction This appendix includes the Statement of Intent Cover Letter and PIP Technical Assistance Assessment Survey. AHCA 2006-2007 Technical Assistance Plan Page A-1

August 11, 2006 APPENDIX A <MCO Contact Name> <Title> <MCO> <MCO Address> <City>, <ST> <Zip Code> Re: Statement of Intent and PIP Technical Assistance Assessment Survey Dear <MCO Contact Name>, Validation of Performance Improvement Projects (PIPs) is one of the mandatory Balanced Budget Act (BBA) activities. This letter will outline a component of the PIP validation process which will be completed by Health Services Advisory Group, Inc. (HSAG), as part of the External Quality Review Organization (EQRO) contract with the Florida Agency for Health Care Administration (AHCA). In order to understand the current status of the MCO s PIPs, AHCA in collaboration with HSAG have developed a statement of intent (see attachment) to gather pertinent data related to the PIP studies in progress. This information will assist in determining the extent to which the study has been completed, and the potential for submission for validation in November 2006. In addition, included within the attachment is a PIP Technical Assistance Assessment Survey, which AHCA and HSAG have developed to ascertain the MCO specific educational needs. The information provided will assist AHCA and HSAG in formulating the best educational approach for all of the MCOs. Prior to the submission of your Statement of Intent and the PIP Technical Assistance Assessment Survey to HSAG, all questions should be directed to Cheryl Neel, RN, MPH, CPHQ by e-mail at cneel@hsag.com. E-mail is the preferred method for submission of any questions. Once we receive your questions, if it is necessary to discuss the question(s) directly, we will contact you by telephone. Please submit the completed attachment to cneel@hsag.com. The completed forms are due to HSAG no later than COB on August 25, 2006. Thank you for completing the attachment as it will provide valuable information required for the validation of PIPs. Sincerely, Margaret Ketterer, RN, BSN, CHCA Executive Director, Audits/Private Projects Attachment

PERFORMANCE IMPROVEMENT PROJECT (PIPS) STATEMENT OF INTENT AGENCY FOR HEALTH CARE ADMINISTRATION APPENDIX A MCO Name: Contact Phone Number: Completion Date: MCO Contact: Contact e-mail address: Total Number of Medicaid enrollees in your MCO: In the table below, please list all current studies (PIPs) that are currently underway within your organization. Please complete the following grid for each contract and answer the questions as they apply to each study. If you have different contracts please complete a Statement of Intent for each contract. Study Topics: please enter each study topic in a separate column below 1. Was this PIP a Focused Study? Y/N 2. Is the PIP Clinical or Non-Clinical? C/NC 3. How many eligible enrollees are in the study? 4. Is the PIP based on HEDIS? Y/N 5. Were study question(s) identified? Y/N 6. Were study indicator(s) identified? Y/N 7. Are the indicators/study topic based on Practice Guidelines? Y/N 8. What is the baseline measurement period? 9. Have baseline data been collected? Y/N 10. Has a causal/barrier analysis been performed? Y/N 11. Have interventions been planned? Y/N 12. Have intervention(s) been implemented? Y/N 13. Have remeasurement data collection been performed? Y/N When? 14. Have remeasurement data analysis been performed? Y/N When? 15. Has statistical testing been performed (i.e. t-tests, chi-square)? Y/N 16. Has a cost/benefit analysis been performed? Y/N 1/10/2007 1

PIP Technical Assistance Assessment Survey HSAG is conducting a PIP technical assistance assessment survey so that we can ascertain MCO specific and general educational needs. Please answer the following questions. Thank you. 1. How are your study topics determined? Please explain. 2. Do you have an internal process for adopting quality improvement projects, i.e., QI committee, QI teams? Please explain. 3. Is your QI staff familiar with developing study indicators? Yes No 4. Is your MCO accredited by an external organization? Yes No If yes, by whom? 5. Does your MCO have a systematic process for collecting data? Yes No 6. Does your MCO have a process for extracting and reporting data? Yes No 7. Do you have a data analyst/statistician that does the data analysis for your studies? Yes No 8. Do you have a data analyst/statistician that does statistical testing for your studies? Yes No 9. Do you have a data analyst/statistician that does sampling methodology for your studies, if required? Yes No 10. Has your MCO developed manual data collection tools? Yes No 11. Has your MCO developed manual data collection tool instructions? Yes No 12. Does your MCO conduct causal/barrier analysis to determine intervention strategies? Yes No 13. What process does your MCO use to interpret the findings, i.e., presented and discussed at QI committee meetings, quality staff interpret the findings, other processes? Please explain. 14. What process does the MCO use to operationalize successful interventions? Please explain. 15. Does the MCO conduct cost/benefit analysis on the study interventions? Yes No 1/10/2007 2

What are your technical assistance/educational needs? Please discuss below. 1/10/2007 3