City and County of San Francisco

Transcription

1 City and County of San Francisco DEPARTMENT OF PUBLIC HEALTH Edwin M. Lee Mayor BEHAVIORAL HEALTH SERVICES Quality Improvement Work Plan Evaluation Report FY

2 Table of Contents Introduction. 3 Service Delivery Capacity 3 Access to Care. 9 Beneficiary Satisfaction. 14 Service Delivery and Clinical Issues 20 Performance and Areas for Improvement Continuity and Coordination of Care. 34 Provider Appeals

3 INTRODUCTION This report describes the results of the San Francisco County Behavioral Health Services (BHS) Quality Improvement Work Plan for Fiscal Year Each section provides the objectives, activities, data sources and results for our endeavors in each of the main content areas. This report is divided into the following content areas: I. Service Delivery Capacity II. Access to Care III. Beneficiary Satisfaction IV. Service Delivery and Clinical Issues V. Performance and Areas for Improvement VI. Continuity and Coordination of Care VII. Provider Appeals WORK PLAN REPORT I. SERVICE DELIVERY CAPACITY GOAL I. Ensure that the number, type, geographic distribution and cultural and linguistic competency of behavioral health services is appropriate for the client population. Based on an analysis of service locations, set goals for the number, type, and geographic distribution of services. San Francisco City and County is dedicated to ensuring that services are accessible on multiple levels. In addition to ensuring that services are distributed geographically to meet the needs of San Franciscans, we are committed to providing culturally and linguistically competent behavioral health services to a diverse population. Chinese, Russian, Spanish, Tagalog, and Vietnamese constitute our five threshold languages, although services are available in other languages dependent on clinicians linguistic capacity, or through interpreter services. Objective 1: Behavioral Health Services programs will be located primarily in the neighborhoods in which the majority of our clients reside. Action(s) 1. Describe the number, type, and geographic distribution of county-funded behavioral health service programs. Review geographic location of services and assess appropriateness given client density by June 30, Objective 1 Results: Density maps were produced to illustrate the geographic distribution of clients served and treatment programs during calendar year The darker the blue shading, the greater the density of clients residing in that area; programs are represented by a red dot. Overall the locations of clinics are well positioned in the areas of the city where our clients live. In relative terms, mental health clients living in the Hunters Point and Outer Mission neighborhoods have fewer programs in their neighborhoods, but the distance to programs is very short (within one mile). There are few substance use clinics in the Outer Mission neighborhood as well, but many more in the Hunters Point neighborhood as compared to mental health clinics. 3

4 4

5 5

6 The table below shows the number of mental health programs by modality of service and neighborhood. In previous years the numbers reported have been consolidated by location, meaning that if more than one program existed at a given address with the same modality of care it was only counted once. The decision was made to count all programs, to better reflect not only location but our capacity to provide services within the various levels of care. Since this change was made, it is difficult to compare these numbers to past years. One notable program that was added this year is the Instituto De La Raza Full Service Partnership (FSP) program, our first FSP program for children 0 to 5 years old, which started to bill in August Number of Mental Health Treatment Programs by Neighborhood Neighborhood Crisis Day Treatment ICM Inpatient Outpatient Residential Total Bayview Bernal Heights Castro/Upper Market Chinatown Downtown/Civic Center Excelsior Haight Ashbury Inner Sunset Mission Nob Hill Ocean View Outer Richmond Outer Sunset Pacific Heights Parkside Potrero Hill Presidio Heights Russian Hill South of Market Twin Peaks Visitacion Valley West of Twin Peaks Western Addition Not in SF Totals Objective 2: Clients will report satisfaction with the convenience and cultural appropriateness of behavioral health service programs, as indicated by an average score of 4 or higher on these items in the consumer perception survey, by June 30, Action(s) 1. Conduct system wide consumer perception survey on the schedule determined by DHCS. 2. Assess client satisfaction results for location and cultural and linguistic competence items. 6

7 Objective 2 Results: Mental Health Consumer Perception Survey Results for Culture and Location Consumer Perception Surveys were distributed to all mental health clients who received a face-to-face service during a one-week period in May 2017, including youth, families, adults, and older adults. The surveys were completed in clinic waiting rooms and dropped into secure survey collection boxes. We received over 3,500 surveys from mental health treatment clients. Several questions on our Consumer Perception Survey address client perception of sensitivity to cultural background, as well as convenience of location of services. The table below shows the question, the average response (based on a Likert scale where 1= Strongly Disagree and 5= Strongly Agree) and the number of clients who answered that question. Overall, results show that clients agreed to strongly agreed that service providers were sensitive to their cultural and linguistic needs, and that services were provided in a convenient location. Comparing the current year s results to the prior year, satisfaction within Adult Mental Health programs has increased slightly for clients perception of sensitivity to cultural background and also improved slightly for convenience of location of services. Results from Adult Consumer Perception Survey (MHSIP) Question Average Rating FY N=2602 Average Rating FY N= Staff were sensitive to my cultural background (race, religion, language, etc.) The location of services was convenient for me Comparing the current year's results to the prior year for the Youth Services Survey, satisfaction has increased both for clients perception of sensitivity to cultural/ethnic background and for convenience of location of services. Results from Youth Consumer Perception Survey (YSS) Question Average Rating FY N= 542 Average Rating FY N= Staff were sensitive to my cultural/ethnic background The location of services was convenient for me/us Comparing the current year's results to the prior year for the Youth Services Satisfaction Survey for Family/Caregivers, satisfaction has slightly improved for clients perception of sensitivity to cultural/ethnic background and slightly decreased for convenience of location of services. Results from Family/Caregiver Perception Survey (YSS-F) Question Average Rating FY N= 630 Average Rating FY N= Staff were sensitive to my cultural/ethnic background The location of services was convenient for me/us

8 Substance Use Treatment Satisfaction Results for Culture and Location Treatment Satisfaction Surveys were distributed to all substance use treatment clients who received a face-to-face service during a one-week period in March The surveys were completed in clinic waiting rooms and dropped into secure survey collection boxes. We received nearly 2,500 surveys from Substance Use Treatment clients. Several questions on our Treatment Satisfaction Survey address client perception of sensitivity to cultural background, as well as convenience of location of services. The table below shows the question, the average response (based on a Likert scale where 1= Strongly Disagree and 5= Strongly Agree) and the number of clients who answered that question. Overall, results show that clients agreed to strongly agreed that service providers were sensitive to their cultural and linguistic needs, and that services were provided in a convenient location. Comparing the current year s results to the prior year, satisfaction has remained virtually the same regarding clients perception of sensitivity to cultural background and convenience of location of substance use services. Results from Substance Abuse Treatment Satisfaction Survey Question Average Rating FY N=2431 Average Rating FY N= Staff were sensitive to my cultural background (race, religion, language, etc.). 2. The location of services was convenient Objective 3: By June 30, 2017, enhance existing substance use services in compliance with Drug Medi-Cal (DMC) Organized Delivery System (ODS) waiver for all existing and approximately 3,000 new clients. Action(s) 1. Hire DMC-ODS Project Manager to manage waiver. 2. Hire DMC-ODS Quality Management and System of Care positions. 3. Implement ASAM-based AVATAR residential authorizations. Objective 3 Results: Erik Dubon, MBA, was hired as the new SUD Program Manager to oversee the day-to-day operations of the DMC/ODS Waiver Program. Erik has over twenty years of experience working with non-profit organizations and the SF Department of Public Health. In his various roles, he has worked with all communities, providing direct services for HIV/Prevention Services, Mental Services, and Substance Use populations. He has also worked in the administration part of the contracting process with the SFDPH. Quality Management hired a SUD data analyst, Sherry Lam, to conduct analyses and develop reports for monitoring SUD performance metrics. Several additional SUD Program Manager positions, as well as a dedicated SUD Quality Improvement Coordinator position, are in process after a lengthy HR delay. The ASAM Level of Care form was completed in June 2017 and piloted by one residential substance use program for central authorization in August Some lessons learned from the pilot were the need for a client pre-admission prior to authorization so program could access ASAM Level of Care form in AVATAR and thus the need for client pre-admit status to be changed to admit once client arrived at program site for residential stay. There is a planned roll-out for all programs starting November, 1, One remaining outstanding issue to be resolved is a lack of agreed upon point person or department responsible for this pre-admission enrollment. 8

9 Objective 4: Expand enrollment of Full-Service Partnership (FSP) families with children aged birth to 5 years old with high needs from 5 to 22, by June 30, Action(s) 1. Provide outreach through partnering with local Community Based Organizations, developing a collaboration with in-home day care providers, participating in community events, and going door-todoor to inform eligible families of the new service. Objective 4 Results: Instituto De La Raza's Full Service Partnership (FSP) program, our first FSP program for children 0 to 5 years old, began in August The program, called Strong Parents and Resilient Kids (S.P.A.R.K.), served fourteen families in FY These families all have children aged birth to five years old and either reside in the Sunnydale Housing project or have an open CPS case. Of the fourteen, eight had never received services prior to engagement with the S.P.A.R.K. program and all but one child had been exposed to domestic and/or community violence. The program continues to have capacity for more families, but due to distrust in the community, caregiver engagement has been challenging, with four referred cases extending at least ninety days before the family was ready to start formal treatment, and three additional cases are still in the engagement phase. In order to expand enrollment in this past fiscal year, S.P.A.R.K. developed a strong referral relationship with Mercy Housing Resident Services Coordinators. The S.P.A.R.K. team also provided outreach at over fifteen community events to increase awareness about the program while building relationships with residents in a non-threatening and organic way. In addition to community events, S.P.A.R.K. program staff have met with fifteen community agencies serving their target population, which led to three ongoing intensive collaborative projects such as weekly Reading/Tutoring support for at-risk youth, month-long trauma workshops for Latina Mother Groups, and six-week Preventing Long-term Anger and Aggression in Youth (P.L.A.A.Y.) support groups for African American male teens. In the next fiscal year, S.P.A.R.K. will continue to expand their services to families in need with plans to outreach to specific Early Learning Centers and Family Childcare Homes while also providing activities such as Family Drumming and Infant Massage. II. ACCESS TO CARE GOAL II.a. Ensure timeliness of routine and urgent mental health appointments. Objective 1: At least 90% of individuals requesting behavioral health outpatient services will be offered an appointment within 10 business days of the request by June 30, Monitor time from request for services to first offered appointment quarterly using the Timely Access Log in Avatar, and determine areas for improvement. 2. To monitor compliance with completion of Timely Access Log, provide programs with a biannual analysis of logged requests relative to number of new episodes per program. Objectives 1 Results: BHS Quality Management extracted data from the Timely Access Log in Avatar to report on the timeliness of routine mental health appointments offered during FY As seen in the chart below, the 10 day standard was met 87% of the time, with similar rates for AOA and CYF services. The average number of days to the first offered appointment was approximately 3 days. Based on feedback at our December 2016 EQRO Review, we changed our methodology for calculating this indicator for FY Previously we excluded entries with an attestation, a clinical assessment by a qualified professional that the client could wait longer than 10 days without any untoward 9

10 consequences. The revised method includes all initial requests for services, with or without an attestation. 1 Time to First Offered Appointment Routine Initial Mental Health Appointments Days to 1st Offered Appointment MHP standard or goal Percent of appointments that meet this standard All Services 3.12 days (mean) 0 days (median) 7.01 Std. Dev. 10 business days Adult Services 2.94 days (mean) 0 days (median) 6.81 Std. Dev. 10 business days Children s Services 3.83 days (mean) 2 days (median) 7.93 Std. Dev. 10 business days 87.4% 87.6% 86.7% Time to First Offered Appointment Annual Trends FY FY FY FY FY Average Days to 1st Offered Appointment Percent Offered within 10 days 89% 91% 95% 98% 87% To track compliance with recording initial requests for services on the Timely Access Log, quality management created a Tableau dashboard that shows the number of entries on the log compared to the number of new treatment episodes opened by month, by program. This dashboard was shared and discussed in SOC-QIC meetings. In addition, each program is monitored on a performance objective comparing the number of entries on their Timely Access Log to the number of new episodes opened, with the expectation that the number of requests for service should always be the same or higher than the number of episodes opened. Approximately two-thirds of our civil service and contract providers met this compliance objective in FY Programs that did not meet the objective are required to write a plan of correction for how they will improve their compliance with use of the Timely Access Log. The chart below shows the performance objective results for completion of the timely access log. 1 In order to ascertain the extent to which including entries with an attestation impacted the data we also calculated this metric with those entries excluded. The result with the attestation entries excluded was a mean of 2 days, and almost 98% compliance with the 10 day standard. 10

11 Objective 2: 100% of individuals assessed as having urgent conditions will be served within 24 hours of initial contact. Action(s) 1. On a quarterly basis, monitor number of individuals entered on outpatient Timely Access Log as needing an "urgent" appointment, and whether their episode of care was opened in an urgent care clinic within 24 hours. Objective 2 Results: Data was extracted from our Timely Access Log for entries designated as Crisis. There were a total of 70 Crisis entries, and of those there were 55 entries with sufficient identifying information recorded to match to Avatar. Of the 55 entries we matched to Avatar billings, 53 had a subsequent billed service. Just over 85% of the appointments (45) were offered within one day, meeting our goal. It is possible that some of the 8 clients with longer than one day of wait time received services not billed in Avatar, such as behavioral health services in primary care. Number of Timely Access for Urgent Appointments Requests for Urgent Appt Number of Crisis entry clients from Timely Access Log found (matched) in Avatar 70 Number of those clients with a billed service subsequent to the Timely Access Log date 53 Number of clients with a billed service within 1 day 45 Objective 3: At least 70% of individuals discharged from inpatient psychiatric services will be seen by a prescriber (MD/NP) within 14 business days by June 30, Action(s) 1. On a quarterly basis, monitor time from inpatient hospital discharge to next contact with psychiatrist or nurse practitioner. 2. BHS to participate in monthly hospital discharge meeting at Zuckerberg San Francisco General (ZSFG) Hospital to improve transitions in care through more timely coordination. Objective 3 Results: Time to MD/NP Service after Hospital Discharge We track the time from hospital discharge to prescriber appointment on a run chart to look at trends over time. As seen in the chart below, our mean rate of compliance with our 14 day standard was 74.9% during FY16-17; this is an increase from last year s mean of 69%. 11

12 Improving Transitions between ZSFG and BHS Programs In the fall of 2015 a committee was formed that included staff from psychiatric inpatient services at Zuckerberg San Francisco General Hospital (ZSFG), ZSFG Psychiatric Emergency Services (PES), and Specialty Behavioral Health Services (BHS). The overarching aim of this workgroup was to improve the collaboration between these entities, and to facilitate more efficient transitions of care for consumers, and thereby reduce psychiatric re-admissions. This workgroup has established written protocols and procedures for communication between ZSFG Psychiatric Inpatient and PES, and the programs BHS that specify timeframes for communication from the hospital, return calls from BHS, standard scripts for communication, and the essential content of these communications. The protocol was communicated to all outpatient programs and hospital staff by program directors and hospital administration. The implementation plan was tested in September 2016 and a feedback mechanism was put in place to report and correct any problems in the implementation of these two protocols, as part of their adoption. In the six months since implementation, BHS has received only two notifications of instances when the protocol was not followed, and staff involved were informed of the correct procedures to follow in the future. Objective 4: Reduce psychiatric hospital 30-day readmissions to below the statewide average of 19% by June 30, Action(s) 1. Monitor psychiatric rehospitalization rates on quarterly basis. 2. Continue to monitor program performance objective requiring no more than 15% of psychiatric inpatient hospital discharges occurring during FY15-16 will be followed by a readmission within 30 days. 12

13 Objective 4 Results: BHS Quality Management tracks the rates of 30-day recidivism to the hospital on a monthly basis, based on all hospital discharges recorded in Avatar (see the chart below). The mean for FY16-17 was 17.3%, a negligible increase from the mean recidivism rate of 16.4% during FY While there were minor fluctuations, the rate overall has been steadily declining for several years, as illustrated by the trend line in the chart below. Programs have a performance objective aiming to reduce preventable rehospitalizations, which applies to all clients who were enrolled in the program prior to the initial psychiatric hospitalization, and remained active in the program for at least the next 30 days following hospital discharge. The goal is that less than 15% of clients will be rehospitalized within 30 days. Performance objectives are rated on a scale of 0 5, with a 5 being a perfect score (achieving the target). Results of this performance objective for FY16-17 indicated that 50 out of 63 programs (79%) achieved a perfect score of 5 on the objective; seven programs achieved a score of 4 ; three programs scored 3 points, and the remaining three programs scored less than 3 points. GOAL II.b. All calls to the BHS 24/7 toll-free access line will be answered by live service providers in the language of the caller, and will gather all required information to ensure the caller receives the appropriate information or referral needed. Objective 1: By June 30, 2017, 100% of calls will be triaged to staff who speaks the language of the caller. If a caller speaks a language not spoken by staff, the Language Line will be used. 13

14 Action(s) 1. Monitor the quality and responsiveness of calls to the BHS 24/7 toll-free access line and provide immediate feedback. Objective 2: By June 30, 2017, 100% of calls will be screened for crisis situations and will be referred appropriately. Action(s) 1. Monitor the screening and referral process of crisis calls to the BHS 24/7 toll-free access line. Objective 3: By June 30, 2017, regular test call results for both the business and after-hours 24/7 Access Line will have a 100% success rate. 1. Continue four independent test calls per month, two during business hours and two after hours, conducted by consumer volunteers, clinical interns, and BHS QM staff and provided feedback to Access Coordinator. 2. Continue to meet monthly with Access Coordinator to discuss and document improvements made in response to test call results. Objectives 1-3 Results: The Access Coordinator continued to monitor the quality and responsiveness of all calls, including crisis calls, to the BHS 24/7 Access Line through daily log reviews, weekly Access staff meetings, weekly meetings with the after-hours contract agency, San Francisco Suicide Prevention (SFSP), and monthly meetings with the Quality Improvement Coordinator to identify any areas for improvement. Throughout the fiscal year, both the Behavioral Health Access Center's (BHAC) team covering business hours and SFSP's after-hours team continued to score close to 100% on bi-weekly test calls. Following the Department of Health Care Services' Triennial Medi-Cal Review findings in April 2017, BHS began to specifically assign one test call a month to include a request to file a grievance. From May-July 2017, grievance test calls have consistently scored 100% on both calls and log portions of the analysis. III. BENEFICIARY SATISFACTION GOAL III.a. Monitor beneficiary/family satisfaction at least annually. Objective 1: By June 30, 2017, at least 80% of clients will report being satisfied with their care, as indicated by an average score of 4.0 or higher on the Consumer Perception Survey. 1. Collect and analyze consumer perception results from all mental health and substance abuse treatment programs to determine areas for improvement. 2. Provide individualized feedback to programs regarding client satisfaction. Objective 1 Results: Mental Health Satisfaction Survey Results Satisfaction surveys were distributed to all mental health clients who received a face-to-face service during a one-week period in May 2017, including youth, families, adults, and older adults. Substance Abuse Treatment Programs surveys were collected in March The surveys were completed in clinic waiting rooms and dropped into secure survey collection boxes. Each program received a report of their satisfaction survey results, showing item level results. Results at the program and system level were posted to the DPH website. 14

15 For the Adult and Older Adult mental health programs, 89.5% of clients reported overall satisfaction, with a mean score of 4.29 out of 5.00 (which was virtually the same as Spring 2016 at 89.1% and 4.26 on average). Of the 2566 adult respondents, the three items with the highest satisfaction rating were (1) I like the services that I received here, (2) Services were available at times that were good for me, and (3) I would recommend this agency to a friend or family member. The three items with the lowest satisfaction ratings were (1) Staff told me what side effects to watch out for, (2) I, not staff, decided my treatment goals and (3) I was able to see a psychiatrist when I wanted to. It should be noted that even the lowest rated items still fell into the satisfied range. For the Children, Youth, and Family mental health programs, 92.8% of our youth and families reported overall satisfaction, with the mean score for youth at 4.28 and the mean score for families at These scores have remained same since our last survey. Among the 575 youth respondents, the three items with the highest satisfaction rating were (1) Staff spoke with me in a way that I understood, (2) Staff treated me with respect, and (3) Staff respected my religious/spiritual beliefs. The three items with the lowest satisfaction ratings for youth were (1) I helped to choose my services, (2) I got as much help as I needed, and (3) I helped to choose my treatment goals. It should be noted that the means were in the satisfied range even for the items with the lowest ratings. Among the 710 parents or caregivers responding about their child s services, the three items with the highest satisfaction ratings were (1) Staff spoke with me in a way that I understood, (2) Staff treated me with respect, and (3) Staff were sensitive to my family s religious/spiritual beliefs. Among parents or caregivers, the lowest rated items were (1) My family got the help we wanted for my child, (2) I helped to choose my child s services, and (3) I helped to choose my child s treatment goals. Again, the mean scores even in the lower rated categories were still within the satisfied range. SUD Satisfaction Survey Results For Substance Abuse programs, 92.3% of clients reported overall satisfaction in this fiscal year, with a mean score of 4.48 out of 5 (results are slightly lower than Spring 2016, which was at 93.6% of clients reported overall satisfaction and mean score of 4.49 out of 5.00). The three items with the highest satisfaction ratings were (1) I felt welcomed here, (2) Staff spoke with me in a way I understood, and (3) Staff treated me with respect. The three lowest rated items were (1) I was able to get all the help/services that I needed, (2) The location was convenient (public transportation, distance, parking, etc)," and (3) " Services were available when I needed them. Open-ended Comments Report This year, Quality Management has been able to provide programs with comments that clients wrote into the open-ended section of their surveys. Comments were transcribed, with personal information redacted. Comments were mostly praise and thanks, but also contained some specific feedback for programs that resulted in programmatic change. In addition to displaying the words of the comments as a word cloud, text analysis was used to find the sentiment and emotion content of the comments for each program. Sentiment refers to the positive or negative meaning of words. The emotion content of comments was assessed for the emotions of joy, trust, anticipation, surprise, anger, disgust, fear, and sadness. Objective 2: By June 30, 2017, decrease by 30% the number of mental health providers needing to complete Plan of Action related to client satisfaction survey return rate from 19 to 13 mental health programs. 1. Provide Peer Volunteer to clinics to support completion of client satisfaction survey. 2. Expand training and technical assistance to individual programs. 15

16 Objective 2 Results: This year, only one mental health program required a Plan of Action for having an insufficient survey return rate, which vastly exceeded our targeted reduction. Since the target was reached, this objective will be replaced with a new satisfaction objective specific to improving access and engagement items for FY Quality Management engaged in a number of activities that be believe contributed to this improvement. First, survey administration instructions were modified to include more detail covering the methods and best practices for conducting the twice yearly Consumer Perception Surveys and Substance Abuse Treatment Satisfaction Survey. This included a new Staff Fact Sheet and a Client Fact Sheet in all threshold languages, which explained the purpose of the survey, confidentiality of the information, and how the information would be used to improve services. The instructions also gave more detail on how to print, reproduce, and staple surveys so that they are scannable. Quality Management staff conducted presentations at Adult and Child, Youth and Family Provider meetings to review the survey instructions and reinforce the importance of conducting the surveys. Our survey coordinator also provided individualized training and technical assistance to administrative assistants and managers who called, ed, and/or visited with questions pertaining to the client satisfaction surveys. Quality Management also piloted sending a volunteer to help to support the completion of client satisfaction surveys at a clinic in Nov 2015, May 2016, and Nov 2016 to determine if expanding this service to multiple clinics was viable. The coordination of volunteers proved to be resource and time intensive. Quality Management determined that it did not have the capacity to expand the service to all clinics; therefore, the volunteer coordination has been discontinued. Even without the peer support, clinics were able to demonstrate improvements in survey completion rates. GOAL III.b. Evaluate beneficiary grievances, appeals, and fair hearings at least annually. Objective 1: Continue to review grievances, appeals, and fair hearings and identify system improvement issues. 1. Collect and analyze grievances, appeals, fair hearings, and requests to change persons providing services in order to examine patterns that may inform the need for changes in policy or programming. 2. The Risk Management Committee will analyze trend reports in order to identify any areas of needed improvement. Areas for improvement will be presented to SOC-QIC and/or other management, provider, and consumer forums. Objective 1 Results: Information pertaining to grievances, appeals and State Fair Hearings is regularly collected and analyzed by the Risk Management section of Quality Management (QM). Grievances and appeals are filed in person or by phone directly to the Behavioral Health Access Center s Officer of the Day, who then provides them to the QM Grievance Officer. Grievances and appeals filed by mail are sent directly to our QM Grievance Officer. The protocols governing the processing of grievances and appeals, the informing materials, filing forms, and posters are current and providers are aware of these protocols. An annual risk management report for FY was presented in several quality improvement committee forums, including the Adult/Older Adult QIC, System of Care QIC, the Children, Youth & Family QIC, and the Risk Management Committee. Findings were also presented at our Adult/Older Adult and the Child, Youth & Family Provider meetings. In these meetings, a summary of all grievances/appeals received during FY , broken out by type, were presented and discussed (see table below, Grievances/Appeals by Category). 16

17 BHS Grievances/Appeals by Category July 2016 June 2017 Total Number = 94 (Appeals = 0 Grievances = 94) Grievance/Appeal Category Number Access Service not Accessible 2 Access Timeliness of Service 2 Access Other Access Issues 7 QOC Staff Behaviors 31 QOC Treatment Aspects 14 QOC - Medication 5 QOC Other QOC Issues 4 Change of Provider 6 Other - Finances 2 Other Lost Property 2 Other - Operational 5 Other Patient s Rights 5 Other Peer Behaviors 8 Other Physical Environment 1 Quality Management oversees the problem resolution processes for clients who are seeking a formal avenue to work out any issue or to review any decision pertaining to Behavioral Health Services. These processes involve either a grievance or an appeal process, and upon its completion, clients may request a State Fair Hearing. Quality Management collects and analyzes aggregate information from these processes as well in order to examine patterns or trends that may inform the need for changes in policy or programming. There were no system issues identified in FY During FY , there were no appeals or expedited appeals filed. There was one request for a State Fair Hearing involving a Drug Medi-Cal provider, which was eventually withdrawn by the claimant. Compared to the prior fiscal year, FY had a 25% overall increase in total number of grievances. All grievances were pertaining to services within the adult system of care. There were no grievances filed within the children s system. At the conclusion of the investigation of grievances, 18% were determined to have merit. Seventy percent of the grievances pertained to mental health services, and 30% to substance use services. Fifty-six percent of the mental health grievances pertained to outpatient services, whereas 82% of the substance use grievances pertained to residential treatment programs. Overall, there were slight decreases noted in grievances pertaining to Finances, Patient s Rights, and Quality of Care issues. Compared to the prior FY, there was a significant percentage decrease in grievances relating to Medication (-50%). There were slight increases noted in grievances relating to Access to Care, Lost Property, Operational, and Physical Environment. Compared to the prior FY, there was a significant percentage increase of grievances relating to Change of Provider (+200%), Staff Behaviors (+41%), Treatment Aspects (+75%), and Peer Behaviors (+100%). These four categories of grievances together comprised 63% of all grievances. In regard to grievances relating to Change of Provider in FY 16-17, there were 6 grievances (6.4/%): four were granted and two were denied, both for clinical reasons and need for continuity of care. While a grievance relating to Change of Provider is usually, although not necessarily, about a client s request to change a provider, all six grievances were requesting to change their providers due to dissatisfaction about their current providers. All six grievances were filed by female clients and all occurred at the outpatient mental health level of care. One of the denied requests pertained to a culture-specific issue whereby the client did not want an Asian-focused program nor her monies managed by the program; however, the program had no such focus and is not involved in money management. One of the granted requests also pertained to a culture-specific issue and, while the client did not state this as a reason for requesting a change of provider, she was successfully transferred to an Asian counselor. 17

18 The Risk Management Committee meets on a monthly basis and is comprised of Quality Management staff, the Deputy Medical Director, clinical pharmacists, and service providers. Information about grievances, appeals, and incident reports are entered into a Risk Management database, and then reviewed for patterns that may inform the need for changes in policy or programming. These trend reports are routinely analyzed by the Risk Management Committee and the System of Care Age Directors. The table below reflects incident reports by event category submitted by providers within our Adult & Older Adult System of Care. Quality of Care Report Summary for FY FY FY FY % Diff Adult/Older Adult SOC by Event Category Prior FY Violent Behavior (includes DTO) % Sexual Assault/Misconduct % Medication Related % Suicide Attempt % Unethical Conduct Client Death % Mandatory Reporting % Service Disruption** % Involuntary Hold* (DTS/GD in OTHER) Injury, Accident, Acute Medical Problem % Inapprop Tx, Delay, D/C, Documentation* 6 5 PHI Breach** AWOL/Unauthorized Absence 24 Hr Setting % Other (includes DTS/GD) % Total % * Deleted event category effective 7/1/15 ** Added event category effective 7/1/15 The table below reflects incident reports by event category submitted by providers within our Child, Youth & Family system of care. Quality of Care Report Summary for CYF SOC by Event Category * Deleted event category effective 7/1/15 ** Added event category effective 7/1/15 18 FY FY FY FY % Diff Prior FY Violent Behavior % Sexual Assault/Misconduct Suicide Attempt % Medication Related % Unethical Conduct % Client Death % Mandatory Reporting % Facility Damage* (in SER DIS or VIO) 6 3 Service Disruption** % On-Site Accidents* (in INJURY) 0 1 Injury, Accident, Acute Medical Problem** % AWOL/Unauthorized Absence 24 Hr Setting % Absenteeism* 1 0 Client Records* (in PHI BREACH) 0 1 PHI Breach** % Needlestick* (in INJURY) 0 0 Other % Total %

19 Based upon these trend reports, subsequent recommendations for quality improvement activities were made in a number of forums such as the Medication Use and Improvement Committee, the Adult/Older Adult QIC, the Children, Youth & Family QIC, and the System of Care QIC. FY system issues and/or recommendations identified by the Risk Management Committee include the following: Medication Related Recommend a review of the current prescribing guidelines for ADHD Recommend a system plan for dealing with benzodiazepine dependence The need for provider guidelines to help determine when clients are appropriate (or not appropriate) for medication-only services Difficulty trouble shooting problems with third party payers of medication services Not all outpatient mental health programs have the ability to administer long-acting injectable antipsychotics Require all BHS InfoScriber users prescribing clozapine to use the Clozapine REMS as utilization of the REMS Program would have prevented 50% of the clozapine-related incidents occurring in 2015 to August 2016 Program Capacity and Client Acuity Recognition that service disruptions due to clinic site closures for remodeling can have unforeseen deleterious effects on some patients Consider that selected programs be equipped with automated external defibrillators (AED) To enhance the effectiveness of payee services by including payees as part of a patient s treatment team with plans for money management that are clinically driven and developed in close collaboration with other members of a patient s treatment team Difficulty getting clients admitted to inpatient hospitalization for much needed stabilization despite the direct requests of clients community treatment providers Practice/Service Delivery The dearth of trauma specialized services and structured, non-residential programs (e.g., day treatment, intensive outpatient programs) The possibility that entering needed residential services will cause clients to lose pre-existing housing and become homeless Restriction of two episodes of residential (non-crisis) treatment per year Programs should routinely request ROI when split treatment is provided by multiple behavioral health services and permission is required in order to collaborate and coordinate treatment efforts Development of community care plans (versus program specific treatment plans) which may help integrate care and mitigate the confidentiality barriers which can prevent substance abuse and mental health programs from communicating Not all BHS funded programs are reflected on the MHS 140 report or have access to and/or utilize AVATAR/CCMS database Staff Related Staff capacity issues Patient contact information should be regularly updated in the medical record 19

20 IV. IDENTIFY AND ADDRESS SERVICE DELIVERY AND CLINICAL ISSUES GOAL IV.a Ensure staff are engaging in appropriate prescribing practices. Objective 1: By June 30, 2017, improve diagnosing and prescribing for adult ADHD clients. 1. Form subcommittee to create workflow process, identify screening tool, and non-pharmacologic tools for adult ADHD diagnosis and treatment. 2. Monitor adult ADHD diagnosing and prescribing rates quarterly. Objective 1 Results: A multi-disciplinary subcommittee was created in March 2016 with the goal of improving and standardizing SFDPH s systems approach to the diagnosis and treatment of Adult ADHD. This group met monthly to create guidelines for a workflow process. Please see this link for more information about the guideline workflow identified: The screening tool that was identified is the ASRS-v1.1 ( This tool was chosen because it has been used by the World Health organization to measure attention and focus. Also, it is in the public domain, and available in many threshold languages, making it accessible for most clients in the diverse community of San Francisco s safety net. A functional impairment scale, the Self-report Adult Symptoms and Role Impairment Inventory, was also provided as a supplemental screening tool. Work on finding and using non-pharmacologic tools for adult ADHD diagnosis and treatment continues. A review of these findings show that mindfulness, time management, and organization techniques prove to be effective for people diagnosed with Adult ADHD. Additionally, we had a system-wide training by a national expert in the field by Dr. Craig Surman, an Assistant Professor of Psychiatry at Harvard Medical School and Scientific Coordinator of the Adult ADHD Research Program at Massachusetts General Hospital. The training was focused on accurately diagnosing and appropriately treating Adult ADHD, and was held on October 26, 2016 with 200+ providers and prescribers in attendance. Although the system still views this as an important activity, we abandoned it as a PIP after our 2016 EQRO feedback due to concerns about measurement and not meeting the standards associated with PIPs. Because we are not pursuing it as a PIP, we have not measured these activities quarterly. The following bar graphs show the number of adults diagnosed with ADHD in FY and 16-17: Number of ADHD diagnoses of adult clients FY FY

21 The chart shows that the diagnosis of ADHD has increased from 255 in FY15-16 to 356 in FY Although Whites constitute the majority of clients diagnosed, increases were evident in virtually all gender and ethnicity groups. The following Chart shows the proportion within each ethnic group receiving treatment services who have the diagnosis. Among adults, Whites are substantially more likely to receive the diagnosis than Asian Americans, Latinos, African Americans, Native Americans, or Native Hawaiian/Pacific Islanders. Percent Diagnosed with ADHD Episodes open during FY % 6.0% 6.3% 5.0% 4.0% 3.0% 2.0% 2.2% 1.6% 2.0% 1.9% 1.8% 3.1% 4.2% 1.0% 0.0% African-American /Black (54 / 2353) Asian (37 / 2376) Latino/a (35 / 1734) Multi-ethnic (14 / 223) Native American (3 / 157) Native Hawaiian or Other Pacific Islander (2 / 110) Unknown / No Entry / Other (33/1052) White (183 / 4324) The following Chart documents the number of clients using an ADHD medication. Our criteria for use of the medication is that they have been written prescriptions that extend for 60 days within each 90-day quarter. Number of adult clients on ADHD medications Jul-16 Aug-16 Sep-16 Oct-16 Nov-16 Dec-16 Jan-17 Feb-17 Mar-17 Apr-17 As can be seen, the number of clients using the medications is relatively constant over time. Objective 2: By June 30, 2017, reduce the % of clients screening positive for benzodiazepine in methadone programs. 1. Programs to report baseline positive screen rates and quarterly thereafter. Across all methadone programs in FY 16-17, the range of client positive screens for benzodiazepine was between 10-15%. This relatively low rate has remained stable and is not expected to change, so prescribing improvement efforts will focus on a different priority for the next fiscal year. 21

22 GOAL IV.b. Increase the use of Group Therapy as a modality of care for mental health treatment. Objective 1: By June 30, 2017, implement two cohorts (~16 clients) of evidence-based Supporting Father Involvement (SFI) program. 1. Graduate at least 50% of enrolled clients from SFI program. Objective 1 Results: In FY , the Wellness Center delivered SFI to six fathers, two of whom attended at least half of the sessions, and four of whom attended the graduation. Due to challenges with recruitment and retention, the decision was made to delay starting a second cohort and focus on better understanding the needs of fathers in the Sunnydale community. A focus group with 24 father was held on September 27, We are using the findings from the focus group to guide programming for the Fatherhood Initiative programming at the Wellness Center and ensuring it aligns with the priorities and preferences of the community. Some initial changes are: Providing low-threshold participation events, such as social events, at the Wellness Center to develop trust and buy-in from the community Adding content, either prior to beginning SFI or incorporated into the curriculum, that addresses the effects of trauma and racial discrimination on fathers Reworking the incentive plan to maximize participant motivation Holding a multi-day cross-training between the SFI developer and Wellness Center staff that includes the SFI training components as well as training and conversation about the local context in which SFI is delivered at the Wellness Center. The second SFI series is planned for January 19, 22, and 23, 2018 and the dates are confirmed with the SFI developers. GOAL IV.c. Expand the Trauma-informed Systems (TIS) Initiative. Objective 1: system. By June 30, 2017, implement a workforce training on the principles of a trauma-informed 1. Train a team of at least 2 trainers capable of presenting the training to remaining staff from both ZSFGH and Population Health. 2. Develop workforce healing circles in collaboration with the Black and African American Health Initiative. 3. Pilot a Leadership Learning Community with a multi-department, cross discipline cohort. 4. Disseminate a preliminary Trauma Informed Leadership toolkit of resources. Objective 1 Results: The Trauma Informed Systems (TIS) Initiative has continued to expand through training and piloting programs designed to sustain, support and embed a culture of knowledgeable responses to trauma. To date, one TIS trainer is dedicated to reaching out and training ZSFGH and Population Health staff. The TIS coordinator is working diligently with ZSFGH leadership to identify TIS trainers and a coordinator. The TIS team worked in collaboration with the Black and African American Health Initiative to develop two Healing Circles for DPH staff. The Healing Circles where piloted in the fall of Both were well attended. The Trauma Informed Systems Leadership Learning Community began meeting in January of 2016 and consists of leadership from Children, Youth and Families as well as 4 other child-serving departments in the city. The leadership learning community is partnered with staff Champions from various roles and 22

23 levels within each participating organization to support a collaborative, multi-level model. Participants meet regularly to develop strategies, discuss progress and share resources to promote TIS implementation within the workplace. Leadership from across agencies meets quarterly to discuss progress and share strategies on creating and sustaining TIS policies and practices. Champions meet monthly to participate in and design strategies aimed at organizational change, relevant to Trauma Informed Principles and Competencies. Leadership and Champions collaborated on TIS strategies informed by individualized agency results from an organizational assessment called the Tool for a Trauma Informed Worklife (TTIW) Survey that was administered in May Leadership and Champions continue to review workplace policies and procedures in consideration of TIS principles and practices, and are preparing to re-administer the TTIW and collect relevant data to evaluate progress. A preliminary Leadership Toolkit was disseminated as part of a pilot, consisting of a literature review of existing TIS literature, quality improvement/management tools, and the Trauma Informed Worklife survey and the Trauma Informed Agency Assessment. This toolkit is being further developed in consideration of results and lessons from the pilot. GOAL IV.d. Expand implementation of Wellness and Recovery (W&R) Practices in behavioral health programs. Objective 1: By June 30, 2017, BHS will test a new process for transitioning clients from ICM/FSP to Outpatient services ( step down ) with at least 10 ICM/FSP clients. 1. Convene a series of collaborative working meetings with representatives from the ICM/FSP programs and Outpatient clinics, sponsored by BHS and facilitated by a trained consultant. 2. Identify programmatic and systemic barriers to transitioning clients to OP services, as well as conditions/processes that currently work well. 3. Decide on a set of recovery criteria or conditions that identify clients readiness for referral to lower intensity services (i.e., graduation or step down ). 4. Solicit solutions or pathways to transitioning clients to OP care (brainstorm). 5. Create small tests of change (PDSAs) that explore solutions for transitioning clients and expand testing to multiple sites and implement best learning across BHS. Objective 1 Results: : 1. Convene a series of collaborative working meetings with representatives from the ICM/FSP programs and Outpatient clinics, sponsored by BHS and facilitated by a trained consultant. From April 7, 2017 through June 16, 2017, six meetings were convened with Intensive Case Management (ICM) and Outpatient (OP) providers, consumer advocates and peer employees, facilitators from Learning For Action (LFA), and BHS administrative staff. All six meetings were consistently well attended. Convening # Date The goals of the convenings were to: 1) Build relationships between providers of ICM and Outpatient programs 2) Clarify the problem to address (clients getting lost between ICM and outpatient services) 3) Identify barriers and potential solutions to supporting clients in the referral and linkage to OP 4) Identify a topic for an MHSA Innovation proposal 23 Attendance (sign-ins) 1 April April May May June June 16 23

24 An ICM-OP Flow workgroup consisting of BHS administrators from the System of Care, Quality Management (QM) and MHSA, collaborated with an evaluation consulting firm, LFA, to coordinate the six convenings, define meeting objectives, discussion topics, and arrange logistics. LFA facilitated the discussions and produced detailed notes for the workgroup. 2. Identify programmatic and systemic barriers to transitioning clients to OP services, as well as conditions/processes that currently work. Transitioning clients from ICM to standard outpatient services has been a challenge for many years. Past efforts to identify and understand the causes and conditions that hamper client transitions include 1) the Advancing Recovery Practices Learning Collaborative with Citywide Forensics that focused on client graduations and transitions to South of Market Mental health Services ( ); 2) the San Francisco Health Network QI Academy teams that focused on transitions to primary care; 3) the ICM/FSP Recovery initiative meetings that gave rise to the need to strengthen communication and support for clients transitioning to OP clinics, and 4) the Psychiatric Inpatient transition effort that focused on Gold Card clients or fast-tracking linkage to outpatient care. Causes and conditions that impact ICM client transitions to OP care are captured in the fishbone diagram below. Starred items are those providers identified as within our locus of control to improve. Transitions in Care: Root Cause Analysis PROCESSES / WORKFLOW OP clinicians have very high caseloads ICM and OP clinicians have a lot of documentation requirements No clear protocol defined by BHS No agreed upon set of criteria or conditions agreed upon to assess client readiness No standard designated position at OPs to handle intakes (varies by clinic) Poor communication / lack of communication protocols between ICM and OP providers Benefit program rules (SSI) can discourage full time employment ENVIRONMENT Lack of safe and stable housing for clients Drugs are prevalent and constantly available to clients ICM and OP clinics are usually separate geographically HUMAN FACTORS ICM Case Mgrs worry about clients relapsing Clients may feel attached to their ICM Case Mgr Clients may fear less easy access to support ICM Case Mgr may resist Replacing recovered client w/a new, higher need client OP Case Mgrs may perceive ICM client as high need, more demanding MATERIALS / IT SUPPORT No single Checklist Form in use for BHS Avatar does not have a specific section with transitions data (e.g. Checklist or tracking referrals) Avatar does not have data or reports that identify clients ready for transition Low success in Transitioning Clients to OP care In addition to these potential root causes, there were also several system-level factors that were indentified by the group as contributors to the problem, including a large gap in service and support between ICM and Outpatient levels of care; and the sense that peer employees are undervalued and not yet fully integrated into BHS service delivery. 24

25 3. Decide on a set of recovery criteria or conditions that identify clients readiness for referral to lower intensity of services (i.e. graduation or step down ). A set of criteria has not yet been developed. The six convenings concluded by identifying potential solutions to facilitate referrals and linkage. One set of solutions will focus on improving communications and protocols for referring clients from an ICM to OP services. 4. Solicit solutions or pathways to transitioning clients to OP care (brainstorm). The convenings generated a set of potential solutions for transitioning clients. They fall into two major focus areas: a. Improving communications and protocols between ICM and OP providers to facilitate transitions to OP, and b. Proposing the creation of a clinically supervised, outpatient based peer transition support team. The details will be worked and tested for learning, adaptation and implementation, in volunteer clinics over the next year. The next steps will be to convene the Communications and protocols workgroup (in July) and schedule regular planning and working meetings over the next six months to develop and move forward ideas to improve communication. In particular, the workgroup plans to identify the steps involved in an ICM contacting an OP clinics to discuss a transfer, the OP response process when receiving requests for transfers, protocols for conducting warm hand offs, etc. The Workgroup is drafting an MHSA Innovations proposal, to be completed in September 2017, aimed at funding a peer transition support team to help facilitate transitions from ICM to OP services. This level of care transition support was identified as a system gap that contributes to clients falling through the cracks after discharge from the ICM level of care. Pending approval of BHS leadership, the San Francisco Board of Supervisors and the Mental Health Services Oversight and Accountability Commission (MHSOAC), we anticipate funding will be available in Create small tests of change (PDSAs) that explore solutions for transitioning clients and expand testing to multiple sites and implement best learning. No tests of change were completed on this project, as the planning phase went through the end of the fiscal year (see above). GOAL IV.e. Improve clinical supervision. Objective 1: By June 30, 2017, train 60 Clinical Supervisors across CYF and A/OA SOC in the Clinical Supervision Model. 1. Finalize Best Practice Manual and adoption of standards. 2. Develop curriculum and identify trainers. 3. Develop Best Practice Clinical Supervision Toolkit. 4. Launch 10-month Clinical Supervision Training Institute, including 101, specialty trainings, and learning communities. 5. Complete pre-post test to evaluate program. Objective 1 Results: The Best Practice Manual was completed in June 2016 and will be disseminated to the participants in September when they arrive for their 2-day foundational training. The manual describes the SOC values that frame the material for the training, the model the will be implemented, and the toolkit to support staff in supervising their clinical staff. 25

26 The Academy trainers were identified through an RFQ process. We have a lead trainer for each component of the model: Yale School on Supervision will present the foundational training (BHS 101) in a 2-day format; UCSF Infant Parent Program staff will facilitate the bi-monthly learning communities and present the initial Specialty training Reflective Practice in Clinical Supervision; and Nicholas Ladany, Ph.D. will present the two final Specialty trainings; Supervising to Effective Clinical Care and Creating a Trusting Supervisory Relationship. These three components comprise our 10 month training Academy. The best practice tool kit was developed using resources from our trainers and other experts in the field of clinical supervision. The purpose of the toolkit is to support our staff with concrete resources that can be utilized to structure the supervision process, augment the skills that the supervisor possesses, and as a way to respond to the feedback that was shared during the workgroup process. This toolkit is a living document that will be updated as new resources are identified. We were unable to launch the 10 month Academy in January 2017 because of contracting barriers with our primary trainer for the BHS 101 component. Once the barriers were resolved it was not feasible to launch the model in FY 16/17, so it was postponed until its current launch date in September Drs. Richie Rubio and Farahnaz Farahmand led the design and development of an evaluation plan, which includes collecting data from clinical supervisors and clinicians (the clinical supervisors supervisees) across all of BHS Civil Service clinical staff. This will allow data to be linked by supervisory pairs and includes a comparison group of those who are not attending the Clinical Supervision Training & Learning Collaborative. The evaluation survey includes the following domains: 1) demographics of clinical supervisors and clinicians, 2) supervision practice, 3) supervisory relationships, 4) supervisor / clinician competency, 5) job satisfaction, and 6) burn out. In addition, we incorporated the system lens/priorities of trauma-informed systems and racial humility. The survey was distributed to all BHS staff for completion of a pre-test in January Given delays to the launch of the training and learning academy, the survey was re-administered to all clinical supervisors and clinicians in August 2017 to better capture the baseline of these measures before the launch of the trainings in September, and to capture the current snapshot of supervisory pairs. Survey results are currently being analyzed. Another survey will be distributed in January 2018 (mid-way through the academy) and again in July 2018 at the completion of the academy. GOAL IV.f. Increase use of evidence-based practices. Objective 1: By June 30, 2017, provide Dialectical Behavioral Treatment (DBT) program to 25 clients in 2 CYF behavioral health programs. 1. Provide on-going training and clinical consultation to identified clinics. 2. Monitor use of hospital and crisis utilization of CYF clients, as well as client suicide risk and DBT providers burnout. Objective 1 Results: Training & Consultation Since the start of services in July 1, 2016, our UCSF DBT Consultant has been providing monthly training and clinical case consultation at the DBT clinic site. Intensive training to the two new clinicians was provided by our DBT Clinical Supervisor and our UCSF Consultant. The UCSF consultant will continue monthly training and case consultations services for the team this next fiscal year. Monitoring/Research Data collection of client level evaluation data and clinician burnout data started simultaneously with the beginning of the treatment program in July System level data is still in the process of being gathered and integrated so UCSF staff, when resourced, can analyze the data to examine the impact the DBT clinic is having on crisis utilization and hospitalization, in addition to therapist burn out and skill acquisition. 26

27 DBT clinicians completed a satisfaction measure following the initial DBT trainings and again after 1 year of monthly consultation. The ratings provided are averages of 7 clinicians post-training satisfaction, and 3 clinicians following 1 year of consultation using a five point scale (1=not enjoyable/helpful, 2=somewhat enjoyable/helpful, 3= enjoyable/helpful, 4= very enjoyable/helpful, 5=extremely enjoyable/helpful). Posttraining/preconsultation Average Post-1 year of consultation How much did you enjoy the DBT-A consultation and training? How helpful was the consultation to aid you in treating suicidal patients? Please provide an overall rating of the program with respect to how helpful it was to you in learning how to implement DBT-A V. ASSESS PERFORMANCE AND IDENTIFY AREAS FOR IMPROVEMENT GOAL V.a. Use quantitative measures to assess performance and to identify and prioritize area(s) for improvement. Objective 1: By June 30, 2017, clients will improve on at least 30% of their actionable items on the Adults Needs and Strengths Assessment (ANSA). 1. Develop and disseminate quarterly reports tracking program and client-level outcomes. 2. Facilitate "data reflection" meetings with clinics to explore the meaning of ANSA results relative to clinical practice, and identify areas for improvement. Objective 1 Results: We have continued to produce quarterly reports that display client improvement on the ANSA, both at the program- and the item-levels. The program-level report displays the percent of clients who showed improvement on 30% of their actionable items (those rated 2 or 3 on the time 1 ANSA). The item level report displays the number of clients who had actionable needs at time 1 and the number of those clients who improved on each of those items at the time 2 (most recent in FY 16-17) ANSA. These reports are posted on the DPH public website. The link to the most up-to-date (for the first 3 quarters of FY16-17) summary report is: The BHS program performance objectives state that 40% of clients with actionable needs at the earlier ANSA should improve on 30% of those actionable items. Using the 3 rd quarter results, only 2 programs have not met this benchmark. The summary report is supplemented by an item level report, which shows specifically which needs and strengths are most prevalent in each program, and the extent to which clients improved on each item. The link to the item-level report is: Level_Report_ _wCJCBHS.pdf The first page of the report contains a system-wide summary. It shows that Depression and Anxiety are the most prevalent client needs in the system, affecting 63% and 57% of the cohort of clients, 27

28 respectively. The report also shows that for this fiscal year, about 39% of clients improved on Depression and 37% improved on Anxiety. The final ANSA reports for FY16-17 will be completed and posted by September 15, We have taken findings from qualitative interviews conducted with clinical staff and developed new ANSA reports highlighting what clinicians say is most important to their work: a focus on clinical care. One of these, called the Traffic Light Report, allows for quick comparison of ANSA ratings over time and debuted in October The findings are color-coded: green for improvement, yellow if they stayed the same, and red for deterioration. The report resides within Avatar and can be generated at the click of a mouse for any client with at least two ANSAs. A team of quality management and system of care staff has been visiting BHS clinics to introduce and discuss the Traffic Light Report. Within the fiscal year, 17 meetings took place with staff at a variety of agencies, including those specializing in transitional age youth, older adults, outpatient mental health, hospital settings, residential settings, and intensive case management. Proposed improvement-focused uses of the report include clinical case consultation, supervision, and service utilization review at the clinic level. A subset of clinicians additionally welcomed the use of the ANSA as a communication tool between client and clinician. Clinics have largely welcomed the data reflection initiative, indicating that they are receptive to training and quality improvement activities in service of their clients. Objective 2: By June 30, 2017, clients will improve on at least 50% of their actionable items on the Child and Adolescent Needs and Strengths Assessment (CANS). 1. Develop and disseminate quarterly reports tracking program and client-level outcomes. 3. Facilitate "data reflection" meetings with clinics to explore the meaning of CANS results relative to clinical practice, and identify areas for improvement. Objective 2 Results: Two types of quarterly reports were developed to reflect performance on the CANS objective: programlevel summary report and item-level report. These reports are both posted online quarterly, and are discussed in Executive QI and CYF management meetings, as well as CYF provider meetings. These reports are also the basis for data reflection activities conducted by programs with their clinicians and staff. The program-level summary report is a one-page graph that contains a bar for each CYF-SOC program that represents the percentage of clients in each program who are meeting the objective. This report displays the percent of clients who showed improvement on 50% of their actionable items (those rated as either a '2' or '3', indicating either a moderate or a severe need at the previous or time 1 CANS). The BHS performance objectives state a goal that 40% of clients with actionable needs at the previous CANS should improve on at least 50% of their actionable items. Across all CYF programs, 47% of clients met the benchmark of improving on at least 50% of their actionable items on the CANS. The link to the program-level report is: CalOMS/PerformObjA2a_Q4_FY1617.pdf The item-level report is a more detailed report that illustrates each item on the CANS, the number of clients who had an actionable need at the previous or time 1 CANS, and the number of clients who improved on that item at the current or time 2 CANS. The item-level report allows programs to identify areas where they are doing well, and determine their intervention and training needs. The first four pages of that report contain a system-wide summary. It shows that Anxiety, Depression, and Anger Control (for the 5 to 18 age group); and Adjustment to Trauma, Regulatory problems, and Anxiety (for the 0 to 4 age group) are the most prevalent client needs in the system. In terms of improvement among these prevalent needs, Anger Control (48%) showed the most improvement for the 5 to 18 age group, while Regulatory Problems (62%) showed the most improvement for the 0 to 4 age group. The report 28

29 also includes item-level reports for each program in the CYF system of care. The link to the item-level report is: CalOMS/PerformObjA2_CANSItemLevel_Q4_FY1617.pdf The following data reflection activities were facilitated for the FY 2016 to 2017: (1) Data Reflection Assist Workshop (DRAW). Four 2-hour DRAW workshops were facilitated for clinic/program staff (i.e., clinicians, case workers, clinic managers/directors, QM personnel, data analysts) on September 26, 28, 29, and 30, The DRAW sessions aimed to facilitate data reflection activities and coach staff on how to effectively utilize CANS reports to engage in this reflection process. The workshops provided guidance on how to reflect on client outcomes with the three questions on the Data Reflection Form: (1) How can we understand this data? Please discuss findings; (2) What other information do you use to understand your clients needs, strengths, and progress; and (3) What is your plan moving forward after reflecting on your program s CANS outcomes? Staff who participated were then encouraged to (a) facilitate a data reflection process with their program s management/leadership team and staff, and (b) submit a data reflection form that summarizes the outcome of their data reflection activities. The following flow chart shows the timeline of this data reflection process: (2) Identifying clinical practice improvement opportunities from data reflection outcomes. Twenty-three CYF-SOC programs conducted data reflection with their staff and submitted a data reflection form. These forms were reviewed and scored by QM staff and CYF Management. Qualitative data analysis was also conducted to generate the most prevalent themes from the completed data reflection forms. The themes that emerged were used to facilitate conversations around CYF priorities for clinical practice improvement opportunities. The results of the qualitative analysis were also presented in different venues for further data reflection aimed at seeking recommendations for clinical practice improvement. These venues included the SF-DPH Executive QI meeting, and the CYF-SOC Providers Meetings. The following clinical practice improvement opportunities were identified: Family-Centered system of care, substance abuse integration, reflective practice in data and clinical supervision, school-based interventions, strength-based assessment and intervention, and primary care collaboration. (3) CYF-SOC System-Wide Data Reflection. CANS Item-Level reports are generated by QM on a quarterly basis. These reports include system-wide as well as program-specific data. The system wide data is presented for data reflection at the CYF-SOC providers meeting, which happens on the third Tuesday of each month. The program-specific reports are provided to each clinic/program on a quarterly basis and they are encouraged to conduct data reflection activities with their staff. (4) CYF-SOC Program-Specific Data Reflection. A few programs requested QM and CYF management to facilitate a data reflection meeting with their staff/clinicians: Alternative Family Services (AFS) Outpatient Clinic; Alternative Family Services (AFS) Visitation program; Chinatown Child Development Center (CCDC); Edgewood Center for Children and Families (separate meetings for some of their programs: Outpatient Program, Crisis Stabilization Unit, Residential); Foster Care Mental Health (FCMH); Instituto Familiar de la Raza (IFR); and Mission Family Center (MFC). 29

30 GOAL V.b. Implement Quality Improvement Training Academy. Objective 1: By December , 6 BHS clinics and the Executive Team will complete a year-long Quality Improvement Training Academy. 1. Six clinic teams and the BHS Exec Team will participate in the year-long QI Training Academy (including monthly trainings and twice monthly team meetings) and receive individualized coaching on QI implementation. 2. Teams will apply QI tools to address system priorities. 3. Teams will use data to monitor progress on their QI projects. Objective 1 Results: In November 2016, six multi-disciplinary outpatient clinic teams ((Chinatown North Beach Mental Health (CTNB), Foster Care Mental Health (FCMH), Mission Family Center (MFC), Mission Mental Health (MMH), South of Market Mental Health (SOMMH), and Sunset Mental Health (SMH)) along with a BHS Executive team completed a year-long quality improvement project primarily focused on access-related system priorities within the QI Training Academy structure which consisted of 9 half-day in-person training sessions, two remote webinars, individualized coaching and data support. In addition to the training sessions, clinic teams met twice a month with their assigned coach and data support to discuss potential countermeasures to test, review data to monitor progress, and plan for next steps in the QI process. Teams also met once a month with their assigned Executive Sponsor to address any system barriers that may be hindering the QI progress. Coaches participated in a monthly 2-hour coaching session with two consultant Master Coaches to learn and practice various QI concepts and tools. As a final product of the project, all six teams created a final report which utilized an A3 Problem Solving and Project Documentation tool from the LEAN method. The report included information on clinic background, current conditions, problem statement, target/goal, analysis (e.g., process map, Pareto chart, root cause, etc.), countermeasures, results (e.g., run chart with target/goal data), lessons learned, and next steps. Project posters were displayed at the last training session in November for other clinic teams to observe, ask questions, and provide recommendations/guidance for further QI work if the team chose to continue after the end of the QI Training Academy. Objective 2: By June 30, 2017, three new BHS teams will participate in the 2017 Quality Improvement Training Academy. 1. Beginning in January 2017, 3 new teams will participate in monthly trainings and twice-monthly team meetings to apply QI tools to solve priority clinic or system problems. Each team will be provided a coach and data support person. Objective 2 Results: Two multi-disciplinary clinic teams, one mental health (Special Programs for Youth- SPY) and one substance use (Treatment Access Program-TAP), participated in the QI Training Academy to work on clinic priorities and received individualized coaching and data support. SPY TAP CLINIC PROJECT TOPIC QI TARGET/GOAL COACH Improving Client Increase the percentage of completed Michelle Meier Discharge Process client discharge summaries monthly from 50% to 75%. Implementing ASAM/Level of Care tool Reduce the number of people who wait for entry into residential substance use disorder treatment from 18.2% to less than 10%. 30 Tom Bleecker

31 In this reporting period, there were 5 half-day in-person training sessions, 10 in-clinic QI team meetings, and 3 2-hour coaching sessions. As of June 30, 2016, teams submitted a mid-year progress report which utilized an A3 Problem Solving and Project Documentation tool from the LEAN method. The report included information on clinic background, current conditions, problem statement, target/goal, analysis tool (e.g., process map, Pareto chart, root cause, etc.) and run chart with baseline and one additional data point. Teams will continue to participate in the QI Training Academy program through November of the next fiscal year. GOAL V.c. Improve Clinical Documentation. Objective 1: By June 30, 2017, all mental health clinical staff will have participated in an in-depth clinical documentation training. 1. BHS Clinical Documentation Specialist will organize large system-wide documentation trainings in Fall Brown-bag "drop-in" documentation clinics will be provided for each level of care on a monthly basis. 3. Clinic-level consultation and coaching will be provided to improve clinic specific documentation challenges. 4. Documentation "Super Heroes" will be identified in each clinic, and will have responsibility for being the on-site contact person for documentation resources and questions. Objective 1 Results: In December 2016, Quality Management hired a Clinical Documentation Specialist with an overall goal of developing a Clinical Documentation Improvement Program (CDIP) that includes comprehensive training, individualized clinic coaching, and responding to documentation consultation needs. The CDIP position works closely with the DPH Compliance Officer to provide training and coaching to programs after they have received compliance audits and compliance checks. In FY16-17: Documentation Trainings & Workshops: a comprehensive (8 hour) documentation training curriculum was created and implemented for CYF and A/OA staff. Medical staff received a specific 3.5 hour training. The table below shows the date of training, length and target audience. Dates Time Location Targeted Section 10/28/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF 10/31/2016 9am-3:30pm Koret Auditorium, 100 Larkin St A/OA 11/3/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and Open Training (1p-5p) 11/4/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF 11/7/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA 11/14/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and Open Training (1p-5p) 11/18/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF 11/21/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA 12/5/2016 9am-5pm Koret Auditorium, 100 Larkin St CYF 12/12/2016 9am-5pm Moran Hall, Laguna Honda Hospital A/OA 12/15/2016 9am-5pm Moran Hall, Laguna Honda Hospital Medical (9a-12:30) and Open Training (1p-5p) 12/16/2016 9am-5pm Moran Hall, Laguna Honda Hospital CYF 12/19/2016 9am-5pm Koret Auditorium, 100 Larkin St A/OA In the process of planning the trainings, CDIP, QM and SOC decided to target the trainings to the staff responsible for management, oversight and documentation training at the clinics (i.e., improving organizational infrastructure to provide training) rather than attempting to train every BHS staff member 31

32 within a single year. Over 300 (unduplicated) managers, supervisors, program directors received the comprehensive documentation curriculum in FY In addition, targeted workshops (90 minutes) were conducted that focused on a rotating topic (assessment, client plan, progress notes). In total, 10 workshops held over 6 months (4 workshops on Assessment; 3 Workshops on Treatment Planning; 3 Workshops on Progress Notes) and 90 staff (unduplicated) participated. Coaching & Consultation: staff- and agency-level coaching and consultation was provided to groups whose needs were identified through BHS management (e.g., request from an AOA Program Manager to meet with a clinic), and through DPH Office of Compliance (e.g., after completing a formal audit), as well as through self-referrals (e.g., an agency s with a request for coaching). These coaching and consultation sessions included: Coaching: o SF-FIRST (initial coaching; two trainings in staff meetings) o Mission Mental Health (initial coaching) o Mission ACT (initial coaching; one training in staff meeting) o Westside (one documentation training to staff) o Mobile Crisis (one training to staff) o South of Market Mental Health (initial coaching) o Laguna Honda Hospital (initial coaching; 3 workshop series on documentation) o Chinatown Child Development Center, CCDC (documentation training) o Baker Places (documentation training) o Group Home providers (overview of Medi-Cal system in preparation for CCR) Consultation: o Provider Organization-level consultation (e.g., AIIM Higher Program-consultation on implementing group therapy; CCDC, review clinical documentation and give feedback/guidance to program directors); o Staff-level consultation (e.g., answer questions via , phone and in-person at Provider Meetings) o System-level consultation (e.g., supporting A/OA and CYF Leadership on clinical documentation questions and consultation). In the process of establishing the Superheroes program (i.e., the clinic-level staff member responsible for documentation resources and questions), CDIP, SOC and DPH Compliance observed that programs did not necessarily have staff with sufficient knowledge and expertise to discharge the duty (e.g., Civil Service programs generally do not have any QM staff and in some instances, are mainly staffed by case managers with limited scope of practice). The decision was made to have DPH Compliance lead the task of having programs identify an agency-level Compliance Officer, given the expectations and requirements through the Health and Human Services (US Department of HHS) Office of the Inspector General (OIG). Objective 2: By June 30, 2017, develop a clinic-level structured quality assurance process to proactively identify documentation problems. 1. Train clinics on an internal chart review tool; implement a new clinical documentation manual and a "desk reference" tool. 2. Work with Compliance to develop analytic reports to identify clinics with specific documentation problems. 3. Sample a percentage of charts, selected based on analytic reports, and conduct external clinical documentation improvement review. Provide feedback and coaching to clinics based on results. 32

33 Objective 2 Results: Documentation Tools-Chart Review Tool, Documentation Manual, Desk Reference: Designing, developing and implementing clinical documentation tools was a major improvement area. The mental health chart review tool was updated based on DHCS updated Annual Review Protocol with that update, trainings will be conducted in December 2017 and February 2018 by Compliance. The new Specialty Mental Health Services (SMHS) documentation manual for Outpatient services continued in the FY16-17 year. The process was delayed due to the impending release of the Information Notice for chart documentation. However, the new documentation manual is expected to be published on 10/31/2017. In FY16-17, the decision was made to additionally create a documentation manual for other core constituencies (Psychiatric Emergency Services/PES; Inpatient Hospital/SFGH) those manuals are expected to be finalized in December To round out the suite of documentation support tools for providers, the SMHS Outpatient Desk Reference was published (September 2016) and revised and republished (July 2017). The Clinical Documentation Specialist utilized this tool in ongoing trainings through FY with both Civil Service and CBO mental health providers. During the FY16-17 year, the decision was made to also create mental health Desk Reference tools for specific constituencies (Private Provider Network/PPN, PES, ZSFG) those Desk Reference Tools are also expected to be finalized around December For SUD, a Documentation Manual and Chart Audit Tool were finalized in December 2016 by DPH Compliance. File Review Processes with Analytic Reports: The Clinical Documentation Specialist developed reports to summarize Compliance findings. In May 2017, a Compliance Data Analyst was hired to expand the reports beyond a simple summary (i.e., drill down to identify staff-level problems). From January 2016 through April 2016, CYF-SOC and CDIP collaborated on a pilot project to create a risk-based framework to prioritize programs and the client charts to review (about seven clinics). Cases were sampled based on length of stay (newly opened; six to nine months; more than one year). Following the review, a client-level improvement report was created and provided to the program with required disallowances (if applicable) and recommendations for improvement. As a final step, Program Directors met with CYF-SOC leadership to create a plan to implement improvement recommendations from the report. During the same period, A/OA SOC conducted a project to support Civil Service and CBO programs to improve the use of existing Avatar reports to identify charts at risk for late assessments and treatment plans. AOA Program Managers printed and compiled reports for programs (e.g., performance objective reports on percent of assessments completed), educated providers on the use of the reports and then delegated the chart review and corrections to the programs directly. As part of the Triennial Plan of Correction, BHS will take these pilot activities to the next step in FY17-18 through enhancements to the electronic health record and reports coupled with mandatory structured file review processes at the clinic- and system-levels. Objective 3: By June 30, 2017, ensure that Drug Medi-Cal programs have the appropriate documentation training and are prepared to begin billing Drug-Medi-Cal. 1. Compliance Team will conduct at least 7 pre-billing readiness reviews. Objective 3 Results: Using the newly completed SUD documentation tools, DPH Compliance completed four pre-billing readiness reviews (in June 2017) and expect to complete twelve more in FY

34 VI. CONTINUITY AND COORDINATION OF CARE GOAL VI.a. Ensure that beneficiaries have access to integrated primary and behavioral health care. Objective 1: By June 30, 2017, process and outcome data dashboards will be developed for the remaining Behavioral Health Homes (Sunset Mental Health, Mission Mental Health, and Chinatown North Beach Mental Health) to monitor access to and outcomes for Health Home clients. 1. Hire a QI analyst who will engage in a collaborative process with Health Home staff to build the process and outcome dashboards, and participate in monthly dashboard review meetings. 2. All four Behavioral Health Homes will identify at least one quality improvement goal and engage in a PDSA cycle to improve a clinic process or patient health outcome by the end of June Objective 1 Results: Action 1: A Health Program Coordinator (HPC) was hired in April 2016 to serve as the QI analyst for the Behavioral Health Homes (BHH). The QI analyst works with the BHHs on quality improvement projects, performance metrics, and evaluation. During the fiscal year , the analyst engaged and collaborated with all four Behavioral Health Homes and their leadership and staff to build processes for monitoring access to and outcomes for BHH clients and began the groundwork necessary to develop data dashboards for the BHHs. The analyst participated in monthly meetings with each BHH as well as a monthly joint meeting with BHH leadership and administration. Action 2: Last fiscal year ( ), Quality Management, in collaboration with the BHHs, created a list of metrics that measure access and quality of client care: (1) Process measures include access metrics such as referrals and engagement rate, no-show rate, clinician productivity and time to third next available appointment; and (2) Outcomes measures include rates of health screening (e.g., smoking, hypertension, immunization, cancer) and health improvement measures (e.g., lipids, blood pressure, and HbA1c in good control). During this fiscal year ( ), each BHH reviewed the metrics to be included in a process and outcome data dashboard and confirmed that the chosen metrics are useful and actionable. The BHH at South of Market Mental Health (SOMMH) continues to review a process data dashboard on a monthly basis. However, we have not been able to create an outcomes data dashboard for SOMMH since the beginning of this fiscal year due to issues with the data system (i2itracks) that collects, consolidates, and retrieves population health data from our EHR. Consequently, efforts to develop an outcomes data dashboard for the other BHHs had to be postponed due to these issues. The QI analyst, however, continues to collaborate and confer with the Department s primary care QI team to troubleshoot the problem and identify possible workarounds for obtaining primary care data for the BHHs. The analyst also continues to collaborate with the primary care QI team to gain access for the BHHs to certain measures and reports such as the third next available appointment and no-show rate. The limitations of the i2itracks data system to accurately retrieve primary care data has been a significant barrier this year to meeting our objective. During this time, we have focused on collaborating with the three BHHs (Mission Mental Health, Sunset Mental Health, and Chinatown Mental Health) to develop a process data dashboard. To do so, the analyst conducted site visits with each BHH and worked closely with the staff to determine how the BHH collected process-related data such as number of referrals and new patients, and helped streamline and improve the way the BHHs track data. The analyst has developed a process data dashboard for the Chinatown Mental Health BHH, and is currently working with Sunset Mental Health and Mission Mental Health to organize and streamline their process for collecting and tracking data. The analyst also worked with leadership to develop and administer a staff and client survey to learn about their experience with the BHH and obtain feedback for improving their experience. 34

35 Finally, SOMMH identified a quality improvement goal to decrease the no-show rate for the BHH. This PDSA project is still in process while the analyst works with the primary care QI team to access data necessary for the PDSA. Specifically, the BHH is awaiting data on specific clients with poor show rates. The BHH plans to target these clients with intensive outreach in an attempt to get them to attend their medical appointments, with the overall goal of decreasing the BHH s no-show rate. GOAL VI.b. Improve adequacy and effectiveness of services to youth in Foster Care. Objective 1: By June 30, 2016, 75% of foster care clients will complete three visits with provider organization within 45 calendar days of referral for Specialty Mental Health Services. 1. Monitor percentage of foster care clients referred to Specialty Mental Health provider meeting engagement criteria. Objective 1 Results: Outcome: Objective was not attained (see Figure below). Overall, only 34% (n = 38) of FCMH clients who were referred for child and/or family therapy (N = 113) completed at least three visits within 45 days of their first visit or session. On a monthly basis, this rate ranged from as low as 0% in June 2017 to as high as 75% in January2017. Processes leading to Outcomes: FCMH is still developing procedures and tools for monitoring of service linkage and satisfaction. However, recent improvements in service engagement may be a response to cultural, procedural and staffing changes in the agency. This includes the following processes: 1. Increased, explicit emphasis on care coordination and monitoring responsibilities as part of FCMH s work. 2. Participation in CFT meetings has augmented clients' buy-in and attachment. It allows increased family voice and choice, and allows MH staff to address specific concerns, describe treatment approaches, and provide the family with someone to contact if they have problems with a referral. 3. Pods support collaboration between clinicians and case managers so they can work together to decide among referral options, and to share information about client engagement, agency waiting lists, etc. 4. Improved and increased communication with Human Services Agency Psychiatric Social Workers. 5. Existence of three Health and Wellness coordinators who support the monitoring efforts to track client referrals, follow up with families, address barriers, etc. Challenges Our clients face several barriers to successful service engagement. Additional discussion with our colleagues at other agencies would help us understand which if any of these factors may be limiting child and family participation in treatment. These barriers include the following: Children and youth in foster care are often moved from placement to placement, or between foster care placement and home; geographic changes and communication lapses mean that attendance at services is likely to suffer. Our child and youth clients can be very hard to reach, whether they are in foster care or at home with a parent whose stressors include child welfare involvement and a number of service participation requirements. Service engagement is made more difficult by the very nature of human responses to trauma social withdrawal, mistrust, and avoidance of painful subjects. 35

36 Notes: The height of each bar (with the number on top of it) represents the number of clients who were referred to child and/or family therapy providers. The teal bar (with the number in it) represents the number of clients who engaged in child and/or family therapy. Engagement is operationalized as a client showing up in at least 3 sessions within 45 days of initial appointment. This number is also represented as a percent under each bar. Successes There were several notable successes in our FCMH program this year, as follows: By refining FCMH in-house communication procedures, the speed of service linkage for children identified as meeting medical necessity has improved. The new pod structure made it possible for CANS clinicians and case managers to discuss screening results and determine linkage options immediately after the CANS is completed. Our Health and Wellness Coordinators (HWCs) work with the Human Services Agency (HSA) to conduct quarterly reviews of service needs for all youth identified as members or potential members of the Katie A. subclass. We also developed a mental health screening tool for use by HSA protective service workers and supervisors. The HWCs began (in summer 2017) the second annual round of clinical review of all open child welfare cases, to ensure that each child: o Is assigned to an FCMH pod o Received an initial mental health screening o Was linked to services if needed, and has had regular re-assessments o Receives regular follow-up screenings if the child/parent refused services or the child did not meet medical necessity initially. 36