START (Screening Tools and Referral Training) Oregon Pediatric Society Year 5 Annual Report July 1, 2012 June 30, 2013

START (Screening Tools and Referral Training) Oregon Pediatric Society Year 5 Annual Report July 1, 2012 June 30, 2013 START (Screening Tools and Referral Training) is a statewide Quality Improvement (QI) initiative of the Oregon Pediatric Society promoting highest standard of care and practice change in pediatric primary care. START works in partnership with medical clinics and local community agencies serving children and adolescents and local community agencies. We also interface with higher level state, public and private entities that work on system of care for children and adolescent health and well-being. START trains health care providers in the detection and management of developmental delays, autism, and peripartum mood disorders (PPD). All of the START trainings emphasize the importance of early intervention and prevention, targeting an end goal that all children enter Kindergarten ready to learn. START training incorporates the medical home, team-based care model and includes pediatric primary care providers (physicians, nurse practitioners, and physician s assistants), nurses, and office staff. The entire team learns the science behind standardized screening, mapping of clinic workflow to optimize implementation of universal screening protocols into their practice, and community resources available for children and families. START provides a unique opportunity for primary care providers (PCPs) and community referral resources to enhance collaboration and coordinated care for Oregon s children. Snapshot of all START trainings Year 1 = 18 trainings Year 2 = 16 trainings Year 3 = 20 trainings Year 4 = 11 trainings Year 5 = 29 trainings 5 year START Program 2013 2009-2013 Primary Care Providers trained 199 828 Health Care Workers trained 492 1,882 Snapshot of Year 5 trainings Types of Trainings BASIC Developmental Screening = 15 trainings Maternal Depression (PPD) Screening = 13 trainings Adolescent Depression = 1 pilot Early Childhood Social/Emotional Development = 1 pilot Pediatric clinics: 8 Family Medicine clinics: 23 Obstetrics/Gynecology: 1 NICU Outpatient: 1 NICU: 2 Regional EOCCO Trainings (multi-clinic): 4 Multnomah County FQHC-wide training: 1 Portland Metro Area = 16 trainings Outside Portland Metro = 13 training Counties Served in Year 5 Baker Clackamas Crook Deschutes Grant Harney Jefferson Malheur Marion Morrow Multnomah Umatilla Union Wallawa Washington

START Module Participants, All Types 8/2008-06/2013 93 Trainings, 1882 Participants Primary Care Providers 44% Nursing, Admin, Office support 56% The START Strategy START works to ensure that all of Oregon s children receive the best preventative, evidence-based developmental health care possible. Our Quality Improvement training initiative seeks to increase provider knowledge and confidence and increase standardized screening and appropriate referral for Oregon s children and families. To that end, we collect data to ensure the efficacy of our work, and to improve upon it. Specifically, we look at measuring change at both the provider level (knowledge, skills, confidence) and at the clinic level (MOC Chart Review Data and TA calls). We also collect secondary data from CCOs and Early Intervention/Early Childhood Special Education (EI/ECSE) to help assess the impact of START regionally. Measuring change in provider knowledge, skills and confidence: In-Training Survey We collect a retrospective survey at the conclusion of each training, which seeks to measure individual level data, including intentions to change behavior post-training. The Year 5 cumulative survey results: Survey Results - Basic Developmental Screening Training 76% of participants (n=184) strongly agreed that the training increased my knowledge of local community resources. 72% of participants (n=180) strongly agreed that the training increased my knowledge of developmental screening tools. 70% of participants (n=153) strongly agreed that as a result of the training, they plan to use developmental screening tools, as recommended by the American Academy of Pediatrics (AAP). 82% of participants (n=178) strongly agreed that as a result of the training, they plan to provide parents information about 211Info/Parent Helpline when they visit my office. 63% of participants (n=126) are very committed to making a change in their developmental screening practices. (**several providers were already doing screening) Survey Results - PPD Screening Training 82% of participants (n=102) strongly agreed that the community resource presentation increased my knowledge of local community referral options. 78% of participants (n=113) strongly agreed that the training increased my knowledge of maternal mood disorders screening tools.

83% of participants (n=111) strongly agreed that as a result of the training, they plan to use maternal depression screening tools routinely, as recommended by AAP. 76% of participants (n=68) strongly agreed that they plan to provide parents information about 211Info when they visit my office. 83% of participants (n=98) are very committed to making a change in their maternal depression screening practices. It was wonderful to learn more about these screenings and I am excited to start implementing them at our clinic I liked having us work as a team to go through the map workflow worksheet. It gets everyone thinking and getting excited." (Family Medicine Physician, Eastern Oregon) We needed this to cement the relevance rationale and resources. (Pediatrician, Multnomah County) Measuring Practice Level Change: Chart Review Data from START MOC-IV Quality Improvement Project Participants The START project provides pediatricians an opportunity to earn credit toward the American Board of Pediatrics (ABP) Maintenance of Certification (MOC) Part IV requirements. Part IV of MOC requires evidence of satisfactory performance in practice and involves implementing and collecting data on a quality improvement project. In order to receive START MOC project credit, pediatricians must attend the START Developmental/Autism screening training, implement the ASQ and/or M-CHAT screening tools, and collect their screening rate data using chart reviews or EHR reports. Ninety nine (99) pediatricians have received their Maintenance of Certification (MOC), Part IV through the START Program. These 99 START MOC participants have performed 4,351 well-child visits during which ASQ and MCHAT screens were administered. Figure 1 shows screening rates for recent MOC participants (2011-2013), and Figures 2 and 3 show overall screening rates for all MOC participants over the past 5 years. Data are shown for 9, 18, & 24 month well-child visits at 3, 6, and/or 9 months after implementation of first screening tool in practice. Table 1 depicts a different representation of the same data. Remarkably, START MOC participants consistently achieve an average of 84% screening rate for both the ASQ and M-CHAT at each of the recommended periodicities.

Screening Rates Figure 1: 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Average ASQ Screening Rates at 9 and 18 Month Well-Child Visits (N=~257 for 9 month visits & N=~213 for 18 month visits) data collected 4/11-5/13 83% 80% 0.0% Pre-implementation 3 mo post-implement 6 mo post-implement 9 mo post-implement 95% 84% 18 month visits 9 month visits 97% 91% Figure 2: 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Pre-implementation MCHAT Screening Rates at 18 & 24 Month Well-Child Visits N=~316 and~207 (2009-2013 data) 6 mo post-implement MCHAT Screening - 18 month visits MCHAT Screening - 24 month visits

Average Screening Rates Figure 3: Average ASQ Screening Rates at 9 and 18 Month Well-Child Visits N=~573 visits for ASQ and ~476 for 18 months visits (data collected from 2009-2013) 100.0% 90.0% 80.0% 70.0% 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Preimplementation 3 mo postimplement 6 mo postimplement 9 mo postimplement 9 month visits 0.0% 78.0% 86.0% 90.0% 18 month visits 0.0% 85.0% 81.0% 93.0% In the most recent MOC period (2011-2013), approximately 2000 patient encounters were audited to assess if screening occurred at well visits. Practices were encouraged to begin screening with at least one AAP-recommended time period, and increase as workflows were implemented. The goal of the project is implementation of a validated developmental screening tool for at least one recommended well visit. All providers that submitted for MOC credit demonstrated marked improvement in screening rates. Population data has shown increased referral rates from providers in areas where a significant number of providers increased screening. Through partnership with the Oregon Pediatric Improvement Partnership (OPIP) and directly with our members, we continue to educate Oregon Health Care Providers on the value of screening, referring, and using the 96110 CPT code when performing screening. Accurate use of the 96110 code ensures that regional measures, based on claims data, reflect actual practice screening rates. There are a number of considerations practices need to investigate before billing for 96110 that are reviewed in an OPIP presentation available at: (http://www.oregon-pip.org/resources/opipcoachingstrategiesfor96110.pdf).

Table 1: START MOC Total Participants Screening Rate Data (N=99 physicians) 3 months after implementation 9 MONTH WELL-CHILD VISITS 6 months after implementation 9 months after implementation # well-child visits (ASQ only) 573 618 597 Average ASQ Screening Rate 78% 86% 90% 18 MONTH WELL-CHILD VISITS # well-child visits (ASQ) 476 464 470 # well-child visits (M-CHAT) 316 303 280 Average ASQ Screening Rate 85% 81% 93% Average M-CHAT Screening Rate 76% 82% 87% 24 MONTH WELL-CHILD VISITS # well-child visits (ASQ) 366 379 408 # well-child visits (M-CHAT) 207 239 265 Average ASQ Screening Rate 76% 87% 97% Average M-CHAT Screening Rate 80% 86% 70% DISCUSSION: Adherence to AAP-recommended screening periodicity Though 100% screening rates at the AAP-recommended periodicity are the long term goals, QI principles call for multiple small changes over time. Those practices that did not fully implement standardized screening in the measured time period, as recommended by AAP, cite several factors. Most commonly, providers cite time constraints and the numerous competing demands at well-child visits. Secondly, practices choose a gradual rollout of screening tool implementation into practices. This allows time to optimize workflow and increase reliability of screening as they progress to the full recommended screening schedule. Lastly, and recently, practices statewide are under new and unprecedented changes, with the CCO metrics and the emerging Health Care Transformation in Oregon. This transformation of healthcare delivery in Oregon has incentivized implementation of universal developmental screening in practices, while also imposing increased demands for attention to practice change in areas outside developmental and pediatric care.

Measuring Regional Level Change: Increase in CPT Code 96110 I. CareOregon Increase in the use of standardized developmental screening tools has been a paramount goal of START. Over the years we have measured this by tracking changes in the billing frequency of CPT code 96110. In the first year of the program START identified over a 4000% increase in the number of 96110 claims submitted. These numbers have continued to climb. START is not solely responsible for this increase, as the critical importance of developmental screening is now much more widely accepted within both health care and early education worlds. Developmental Screening is currently one of the 17 incentive metrics of Oregon s new Coordinated Care Organizations (CCOs) and is a Patient-Centered Primary Care Home (PCPCH) requirement. As evidenced by the continued increase in billing code usage over the past 6 years within CareOregon, it is clear that the collaboration among health care, early education and START has had a positive impact. Table 2: Billing Claims for Developmental Screening (96110) Submitted to CareOregon (PRE-START Initiative) 11/1/2007 3/31/2008 (Year 1 START) 11/1/2008 3/31/2009 (Year 5 START) 11/1/2012 3/31/2013 # 96110 CareOregon claims 8 1,604 5,462 II. MODA/Eastern Oregon CCO (EOCC0) START was approached by MODA/Eastern Oregon CCO (EOCCO) to train all pediatric primary care clinics across 12 counties within their rural CCO. The goal was to increase rate of screening for developmental and behavioral delays by using the ASQ3, and connect providers with resource and referral options specific to their communities. START partnered with the EOCCO to identify where most children in this region lived. START then targeted PCPs in areas with clinics that see the most children for well child visits, delivering 6 regional trainings. Altogether, we trained 118 people, including 35 primary care providers, representing a total of 25 practices and clinics. MODA/EOCCO now reports that they have seen a significant increase in use of the billing code for developmental screening. Their baseline, collected in 2011, was 6.7% and they recently reported that the rate is now 50%. This translates into over 1,000 children now being screened for developmental delays. Our MODA partner reports, The only interventions we have done have been in partnership with OPS/START, so I think the increased rate is largely thanks to those [START Training] interventions. As START outreach continues to link PCP s with their local community resources, we ensure Oregon s children are healthy and ready for Kindergarten. Increased screening, billing code usage and referral rates are indications that START is meeting its goals.

Screening Rates Figure 4: EOCCO Developmental Screening Rates (2011 2013) 1 Eastern Oregon CCO (EOCCO) Developmental Screeing Rates at Well-Child Visits ( 0-36 months) 0.8 0.6 0.4 0.2 0 6.7% CCO Baseline (2011) 50% December 2013 (post START trainings) ASQ Screening - 0-36 months Supporting Practice Level Change: Technical Assistance One important aspect of START is the technical assistance (TA) provided to each clinic we train. We contact the clinics at 1,3,6 and 12 months post training to assist with any challenges they are having in implementation of screening, coding or referrals. We also ask for learnings and best practices they are willing to share to help other clinics in the future. During year 5 of the initiative, START made 48 technical assistance calls to clinics we have trained. Of the clinics we spoke to, none had done universal standardized screening using the ASQ and/or the Edinburgh (for those trained in PPD). At the 6 month TA call: 90% reported that all providers were doing screening (at selected periodicity) 50% reported that they planned to implement the full AAP-recommended screening schedule. 70% reported that they were consistently billing for CPT 96110 100% reported that their referral patterns had increased since the training Challenges and barriers that were brought up included: staff resistance to workflow changes; low literacy of patients; parents not completing form during visit; billing code modifiers; questions about scoring the ASQs; complaints about the ASQ activity sheet; communication with Early Intervention, parents not following through with referral; and questions about parent education. START staff provided input and assistance in all instances and called in other clinics and agencies when needed. A Case Example: Richmond Family Medicine OHSU This clinic agreed to collect data on CPT 96110. These data reflect the 12 months before and after the START training: March 1 2011-March 1 2012 = 12 patients START Training April 2012 March 1 2012-March 1 2013 = 102 patients After the START Training, developmental screening became an expectation in our clinic. --Nurse Practitioner, Richmond Family Medicine

Measuring Changes in Referrals and Care Coordination: Secondary Data Ensuring that children with delays are referred for intervention is a key goal of START. We know from local and national research that there are a significant number of patients who are referred, but do not show up for EI/ECSE assessment and services. These are important care coordination concerns to be addressed. Nonetheless, connecting primary care to EI/ECSE has made a great impact, and the number of physician referrals to EI/ECSE has had a steady rise in most cases. Table 3: Number of Referrals to Early Intervention (0-2 years old) from Physicians and Healthcare Clinics by select counties (START has trained in these counties in Year 5) Year 5 START PRE-START Year 3 START (note 9 months data only, County exc. Multnomah) 2007-2008 2010-2011 2012 - Sept. 2013 Baker <6 <6 12 Clackamas 64 116 229 Deschutes 64 73 122 Malheur 25 33 44 Marion 25 224 395 Multnomah 150 486 708 Umatilla 25 46 43 Washington 198 353 518 Table 4: Number of Referrals to Early Childhood Special Education (3 years - Kindergarten) from Physicians & Healthcare Clinics in select counties (START has trained in these counties in Year 5) County PRE-START Year 3 START Year 5 START 2007-2008 2010-2011 2012-2013 Baker 0 <6 3 Clackamas 37 101 123 Deschutes 38 56 59 Malheur 8 17 6 Marion <6 235 222 Multnomah 94 309 258 Umatilla 20 40 22 Washington 262 262 208

Other Highlights from the Year Demand for START trainings has jumped dramatically as the START Program gains increased recognition in the state. In addition, the CCO incentive metric on Developmental Screening has also had an impact. As such, we have had the opportunity to collaborate with and train a number of CCOs. Similarly, several of the emerging Early Learning HUBs have contacted us to discuss Health Care Linkages in Early Education. We look forward to finding new, innovative ways to work statewide to improve child health and well-being outcomes. We continue to reach out beyond Pediatric Clinics, to Family Medicine, NICUs, and even lately OB/GYN clinics statewide. Research and development into other content areas continues as a vehicle for continuous pediatric quality improvement. Collaborative, multi-disciplinary workgroups were convened over a 9 month period to develop curriculum for the three new START modules: ASD 101 for PCPs (Autism Screening and Medical Management), Behavioral Health Integration, and Adolescent Depression. Our Autism 101 training module is now up and running, and we have received CME status. START partnered with Multnomah Project LAUNCH and the Ready for Kindergarten Collaborative to deliver outreach tools to pediatric and family medicine clinics across Multnomah County. We visited 29 clinics to deliver three tools, which included: New START brochure, including new training modules Promotional poster for 211Family Info A handout for parents reminding them to register their children early for Kindergarten. These visits were also an effective way to check in with clinics we have trained in past years. Research and development into other content areas continues as a vehicle for continuous pediatric quality improvement. Collaborative, multi-disciplinary workgroups were convened over a 9 month period to develop curricula for four new START modules: ASD 101 (Autism Screening and Medical Management), Behavioral Health Integration, Adolescent Depression, and Adolescent SBIRT. We look forward to expanding into new content areas to improve child health in new and critical health areas to support the goal of Oregon s children growing into healthy, happy adults.

APPENDIX A START Initiative - Year 5 Date Training Site County Training Module Training Attendees Total # attendees Total # PCPs Marion, Polk, 7/16/2012 The Doctor's Clinic Yamhill Basic 18 11 8/7/2012 Gabriel Park Clinic, OHSU Multnomah Basic 41 8 8/15/2012 Legacy Emanuel Children's Clinic Multnomah PPD 21 5 8/23/2012 Richmond FM, OHSU (part 2) Multnomah Basic 26 6 9/19/2012 SW Family Physicians (part 1) Washington Basic 18 6 Adol. 10/1/2012 Children's Clinic Mult./Clack Dep. 22 22 10/3/2012 Metropolitan Pediatrics Multnomah PPD 22 22 10/17/2012 SW Family Physicians (part 2) Washington Basic 15 4 11/6/2012 Mosaic Medical Prineville Crook PPD 12 5 11/7/2012 Central Oregon Pediatric Assoc. Deschutes PPD 14 10 11/7/2012 Mosaic Medical Bend Deschutes PPD 8 4 11/8/2012 Mosaic Medical Madras Jefferson PPD 9 4 11/12/2012 OHSU NICU Follow Up Team Multnomah PPD 7 1 1/15/2013 Westside Medical Hillsboro Washington Basic 10 4 2/27/2013 Children's Community Clinic Multnomah Basic 8 5 3/18/2013 PMG Milwaukie Clackamas Basic 27 11 3/19/2013 Randall Childrens Hosp NICU Multnomah PPD 9 2 3/20/2013 Mult Co Health Dept Multnomah Basic 37 31 3/26/2013 PMG Southeast Clinic Multnomah Basic 32 13 5/22/2013 PPD Randall NICU Multnomah PPD 16 1 5/23/2013 OHSU South Waterfront Multnomah Basic 44 11 6/17/2013 PPD St Charles NICU @ DCCF Deschutes PPD 11 1 6/17/2013 PPD HighLake Healthcare Deschutes PPD 13 4 6/18/2013 PPD Bend Memorial Clinic Deschutes PPD 8 5 6/18/2013 PPD East Cascade Women's Group Deschutes PPD 16 4 6/25/2013 St Alphonsus Medical Center Malheur Basic 13 9 6/26/2013 Sunridge Inn Baker Basic 10 4 6/27/2013 Center for Human Development Union Basic 12 3 6/28/2013 Good Shepherd Medical Center Umatilla Basic 14 2 TOTAL TRAINED 492 221

APPENDIX B Figure 1: START Developmental/Autism Screening Trainings by Year, Geographic Region, and Training Type Year 1 8/2008 6/2009 Year 2 7/2009 6/2010 Year 3 7/2010 6/2011 Year 4 7/2011 6/2012 Year 5 7/2012 6/2013 Totals Total Developmental/Autism Trainings 18 11 16 7 16 68 Portland Metro Area 12 5 5 5 10 37 Outside Portland Metro 6 6 11 2 6 31 Clinic-Specific Trainings 13 4 9 7 11 44 Community-Wide Trainings 5 6 5 0 5 21 Medical Education Training Programs 0 1 2 0 0 3 Figure 2: START Peripartum Mood Disorders Trainings by Year, Geographic Region, and Type of Training Year 1 11/2009-6/2010 Year 2 7/2010-6/2011 Year 3 7/2011-6/2012 Year 4 7/2012-6/2013 Totals Total PPD Trainings 5 4 8 13 30 Portland Metro Area 3 3 4 5 15 Outside Portland Metro 2 1 4 8 15 Clinic-Specific Trainings 3 2 8 13 26 Community-Wide Trainings 1 1 0 0 2 Medical Education Program Trainings 1 1 0 0 2