The Affiliate Evaluation Final Report. Prepared by:

The Affiliate Evaluation Final Report Prepared by: The Women s & Children s Health Policy Center Johns Hopkins University Bloomberg School of Public Health May 2003

The Affiliate Evaluation Final Report Johns Hopkins Bloomberg School of Public Health Authors: Editor: Contact: Tess Miller, Bernard Guyer, David Bishai, Diane Burkom, Becky Clark, Allison Cosslett, Janice Genevro, Holly Grason,William Hou, Nancy Hughart, Alison Snow Jones, Cynthia Minkovitz, Heather Rutz, Daniel Scharfstein, Heather Stacy, Donna Strobino, Eleanor Szanton, and Chao Tang. Janice Genevro Kamila Mistry Project Director Bloomberg School of Public Health Johns Hopkins University 624 N. Broadway, Room 195 Baltimore, MD 21205 410-955-8694 kmistry@jhsph.edu Healthy Steps for Young Children is a program of The Commonwealth Fund in collaboration with Boston University School of Medicine, and local funders and health care providers across the nation. Healthy Steps is co-sponsored by the American Academy of Pediatrics. The Healthy Steps affiliate evaluation was carried out with grants from The Atlantic Philanthropies, The Robert Wood Johnson Foundation and local funders. The views presented here are those of the authors.

Table of Contents Acknowledgments...i Executive Summary...ii Introduction...1 What is Healthy Steps?...1 How was Healthy Steps Evaluated?...2 Who Participated in the Affiliate Evaluation?...4 Pediatric Practices...4 Families...5 How was Healthy Steps Implemented at Affiliate Sites?...7 Integrating the HS Specialist into the Practice...7 Implementing the Healthy Steps Services...8 What Services did HS Specialists Provide to Affiliate Families?...12 What Services did Affiliate Families Report Receiving?...15 In What Ways Did Healthy Steps Add Value to Standard Pediatric Care at Affiliate Sites?...17 Enhanced Relationships...17 Increased Parent Satisfaction with Care...18 Increased Provider Satisfaction...19 Improved Teamwork and Practice Environment...19 Positive Parenting Practices and Improved Child Outcomes...20 What Did We Learn from the Affiliate Evaluation?...26 What is happening at Affiliate Sites Now?...28 Appendix I Methodology...30 Appendix II Selected Outcomes...47

Acknowledgments We wish to thank Margaret E. Mahoney, Karen Davis, Kathryn Taaffe McLearn, Edward Schor, and members of the Healthy Steps National Advisory Committee for their guidance, advice and support. We also are grateful to Michael Barth at ICF Consulting for his tireless leadership and to his staff, Juliet Konvisser, Philip Rizzi, Cynthia Hansel, Nita Hassan, Samantha Gill and Sara Rogers. The affiliate evaluation has greatly benefited from the close collaboration of Barry Zuckerman, Margot Kaplan-Sanoff, Steven Parker, Andrea Bernard and Tracy Magee at the Boston University School of Medicine. Funding for the Affiliate evaluation was made possible through the generous support of The Atlantic Philanthropies, The Robert Wood Johnson Foundation, and the following local funders: Brown Foundation; The Chicago Community Trust; Children s Trust Fund of Texas; The Duke Endowment; Harris Foundation; Hogg Foundation for Mental Health; The Houston Endowment; John D. and Catherine T. MacArthur Foundation; Kansas Health Foundation; Michael Reese Health Trust; Prince Charitable Trusts; Rockwell Fund, Inc.; San Antonio Metropolitan Health Department; Texas Children s Hospital; W.P. and H.B. White Foundation; Washington Square Health Foundation. The views presented here are those of the authors and not necessarily those of the financial supporters, or their directors, officers or staff. The following members of the evaluation team at Johns Hopkins led by Bernard Guyer assisted in the affiliate evaluation: Mary Benedict; Allison Cosslett; Brandy Fauntleroy; Janice Genevro; Holly Grason; Nancy Hughart; William Hou; Ashraful Huq; Avanti Johnson, Alison Snow Jones; Armenta Jones; Pat Lanocha; Tess Miller; Cynthia Minkovitz; Lexie Motyl; Stephanie Neal, Becky Newcomer; Laura Pagels, Heather Rutz; Daniel Scharfstein; Jane Schlegel; Heather Stacy; Brenda Sterling; Lavonne Sumler; Kristie Susco; Donna Strobino; Eleanor Szanton; Chao Tang; and Marsha Young. In addition, Diane Burkom, Helen Gordon and the interviewing staff of Battelle Centers for Public Health Research and Evaluation conducted the parent telephone interview. Jean Su and her staff at SOSIO, Incorporated provided data entry for the evaluation. Thank you all for your hard work and dedication. We are grateful to the Healthy Steps Specialists, lead physicians, administrators, and staff at each of the sites for their devotion to the program, their commitment to the evaluation, and their hard work to ensure that both succeeded. Their names are listed below. Lead Physicians, Administrators and Others: Anita Berry; Mary Martha Bledsoe Felkner; Julia Bowers-McLain; Mohammad Chaudhary; Glenna Dawson; Jan Drutz; Fernando Guerra; Cynthia Henderson; Molly Jacob; Crystal Mobley; Jane Moss; Jerry Neiderman; Karen Nonhof; Isabelle Patton; Consuelo Sandoval; Penny Schwab; Silvana Shilapochnik; Daniel Treviño; Juan Vargas; Darlene Victorson; and Gail Wilson. Healthy Steps Specialists: Catalina Ariza; JoAnn Allen; Melanie August; Juanita Brown; Jennifer Dubrow; Rosa Fernandez; Patricia Garza; Kris Hawkins; Elaine Nishioka; Janie Ochoa; Esther Oppliger; Sabrina Provine; Veronica Serano; and Claudia Yañez.

Finally, we wish to thank all the families who generously gave of their time. Without them, our work would not have been possible.

Executive Summary This report focuses on the evaluation of the Healthy Steps for Young Children program at six affiliate sites 1. Affiliate sites were selected based on the same criteria as national evaluation sites except they did not have a comparison population. They implemented the Healthy Steps program fully, offering the same program as the national sites. Six sites, comprising seven primary pediatric practices, participated in the affiliate evaluation. On average, mothers participating in the affiliate evaluation tended to be young (more than one half were less than 25 years old), with limited education (45% had not graduated from high school), of Hispanic origin (56% reported they were of Hispanic origin) and poor (for 54% of the families, maternity care was paid for by Medicaid). They differed from the families in the national evaluation in terms of these demographic characteristics. The full Healthy Steps program was implemented at all affiliate sites. All affiliate sites hired two HS Specialists and delivered the package of Healthy Steps services from the time the first family was enrolled into the program. There were some barriers encountered in implementation as well as variability in the programs. Results from both providers and parents indicated that affiliate families received Healthy Steps services in addition to routine pediatric primary care. According to the Healthy Steps Specialists, the average family who participated in the program at least 15 months received a variety of services from their Specialist: 7 office visits 2 home visits 6 telephone calls 2 other contacts such as mailings. The average family did not attend a parent group. Only 20% of families attended at least one parent group during the program. Child development was universally discussed with families, and other important topic areas such as nutrition, child health, injury prevention, family support and maternal health were addressed with a large proportion of families. Families with higher incomes and older, better educated and first-time mothers appeared to receive more Healthy Steps services than their counterparts, but the differences between groups were small. These results are particularly noteworthy because five of the six affiliate sites served lowincome, transient populations at high risk for poor outcomes. Initially, there was doubt whether Healthy Steps could even be implemented at these sites. Not only did these sites 1 Three other affiliate sites participated in Healthy Steps. Two sites implemented variations of the Healthy Steps program and evaluated their programs: one site included a prenatal component; another offered telephone counseling in lieu of enhanced well child visits. Each utilized a randomized, case/control design for their evaluation. A third site participated in a local evaluation.

successfully implement the program, but they also delivered developmental services to populations that are traditionally difficult to reach and to engage in health care programs. Healthy Steps added value to the primary pediatric health care delivered at affiliate sites. It enhanced the relationship between the family and the practice. The key to the program was the relationship that developed between the family and the HS Specialist. This relationship, and the additional services provided, seems to account for increased parents satisfaction with the care they received. According to all the evaluation data, affiliate families whether at high or low risk, new parents or more experienced, young or older mothers were highly satisfied with the program. In addition, based on surveys conducted with health care providers at start-up and 30 months into the program, Healthy Steps appeared to have improved the satisfaction of pediatricians and nurse practitioners with the care they provided. All those in the practice who worked with the HS Specialist acknowledged the benefits that this new professional brought to the practice. Overall teamwork improved over the course of the program. Increased satisfaction occurred among health care professionals that work with at-risk populations. Again, this is particularly noteworthy as these providers are traditionally at higher risk of job dissatisfaction and turnover. The level of positive parenting practices was high among affiliate families. The majority of families reported using safety devices, establishing routines and talking to and playing with their child. There was some evidence to suggest that Healthy Steps improved parents use of the health care system for their children. A greater percentage of HS affiliate children than children who received care at the practice prior to HS were given a Denver Developmental Screening Tool (DDST) by 12 months and made age-appropriate well child visits. At several sites, more affiliate children were fully immunized at 12 months of age than were children who received care at the practice prior to HS. Results from the affiliate evaluation reinforced the results of the national evaluation. Healthy Steps was well implemented. The key to the program seemed to be the relationship that developed between the HS Specialist and families, which in turn strengthened the relationship of the family with their primary care provider and ultimately the practice overall. Healthy Steps improved clinicians and families satisfaction with pediatric care. The program increased the amount of preventive health care children received. Most important, the invaluable contribution of the affiliation evaluation is that it demonstrated that Healthy Steps could be successfully implemented with a low income, high risk population as well as in a high income population.