Alternative Response Research in Missouri, Minnesota, and Virginia

Alternative Response Research in Missouri, Minnesota, and Virginia Findings in Six Areas Institute of Applied Research St. Louis, Missouri www.iarstl.org

1 Three Evaluation Studies Process and Impact Evaluation of The Missouri Family Assessment and Response System 1995-1998 Report: Missouri Family Assessment and Response Demonstration Evaluation Report, January 2000 (Digest) Five-Year Follow-up of the Statewide Missouri Implementation Report: Differential Response in Missouri after Five Years, February 2004 Minnesota Alternative Response Evaluation 2001-2004 Report: Minnesota Alternative Response, Final Report, November 2004 Reports available in PDF format on www.iarstl.org (papers and reports tab) Virginia Report is available at: http://www.dss.state.va.us/files/division/dfs/cps/reports/eval_drs.pdf.

2 Two Track Systems in all Three States Exit System All reports accepted as potential Child Maltreatment Screening Families NOT Appropriate for AR Traditional Investigations Track change possible Families Appropriate for AR Family Assessments Unsubstantiated Substantiated Investigation Accepts Voluntary Services Declines voluntary services or no services needed Formal Cases / Child Removals Agency or community services / Formal or informal cases Exit System Exit System Exit System

3 General Areas to be Considered Screening What proportion of cases become AR and is screening consistent? Child Safety Can child safety be maintained under Alternative Response (AR) at the same levels as in traditional investigations? Family Engagement Does the non-adversarial approach of AR lead to improved participation and satisfaction of families? Services to Families Are services made available to more families under AR and do the types of services change? Recurrence of Maltreatment Does AR lead to reduced abuse and neglect reports and reduced removal of children? Cost Effectiveness Is AR more or less cost effective over time?

4 Screening In Minnesota, Missouri, and Virginia, reports of child abuse and neglect were initially screened for either an Alternative Response (Family Assessment in Missouri and Virginia) or a Traditional Response, that is, a CPS investigation, based on allegation, history or situation. The proportion of reports screened for AR varied significantly from one local office to another in all 3 states: Minnesota average across counties -- 47% Missouri -- 71% Virginia average across counties -- 61%

5 Child Safety (Changes during the Initial Assessment or Case) Based on information on cases provided by assessment workers and investigators: No evidence was found that child safety was compromised under AR either in Missouri or Minnesota. Some evidence was found of relative improvement in child safety under AR. Responses to a Virginia survey showed that 51% of supervisors and 37% of workers believed AR probably or definitely increased child safety. Most of the others believed there was no change.

6 Missouri Safety Change: Based on Reviews of Sample Cases Positive changes (by last contact with family) Positive changes (during first 30 days) Regession or no known change (by last contact with family) Regession or no known change (during first 30 days) Pilot Comparison 1. Safety of children was not compromised by the demonstration. Moreover, there was evidence that child safety was improved in certain circumstances. 2. Children were made safer sooner. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

7 Family Engagement Engagement of families was a factor in improved safety outcomes and increased services to families. Missouri: Cooperation increased; family flight decreased Family satisfaction improved Families sense of participation in decision making increased Minnesota: Family satisfaction improved Families sense of participation in decision making increased Families emotional responses were more positive under AR Workers rated families as more cooperative and less hostile Virginia: Workers and supervisors reported families often more willing to talk about problems and to accept services.

8 Missouri Family Cooperation: A Factor in Child Safety 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Pilot Comparison 1. Cooperation between families and the child welfare agency increased. Family flight decreased. a. because of the more positive and supportive orientation, or b. because of earlier service contacts. Non-Cooperative Cooperative

9 Missouri Family Responses Pilotassessment comparison 14.7% 14.3% 38.5% 32.5% Pilotinvestigation 12.2% 6.9% Comparison pilot 35.5% 45.4% 0% 20% 40% 60% 80% 100% 0% 10% 20% 30% 40% 50% A great deal Somewhat A little Not at all Very satisfied Generally dissatisfied Generally satisfied Very dissatisfied Level of Involvement in Decision Making Satisfaction of Families with Children s Services 1. Families felt they had greater involvement in decision making. 2. Families were more satisfied with the experience. 3. Workers responses reflected these differences.

10 Minnesota Family Responses xperimental 68.2% 16.4% 7.0% 8.4% Experimental 58.0% 33.3% 6.2% 2.5% p< 0.000 p< 0.000 Control 45.1% 22.2% 13.0% 19.7% Control 44.8% 39.1% 9.3% 6.8% 0% 20% 40% 60% 80% 100% A great deal Somewhat A little Not at all 0% 20% 40% 60% 80% 100% Very satisfied Generally satisfied Generally dissatisfied Very dissatisfied Level of Involvement in Decision Making Satisfaction of Families with Treatment by Worker 1. Families felt they had greater involvement in decision making. 2. Families were more satisfied with the way they were treated. 3. Workers responses reflected these differences.

11 Minnesota Family Emotional Response Differences in engagement and alienation were reflected in the reported emotional responses of families in Minnesota. Experimental families reported being relieved, hopeful, satisfied, helped, pleased, reassured and encouraged significantly more often. Control families reported being angry, afraid, irritated, dissatisfied, worried, negative, pessimistic and discouraged significantly more often.

12 Minnesota Worker Assessments of Cooperation As measured by the Minnesota SDM Family Risk Assessment, AR families had significantly better attitudes. Workers rated the primary caregiver as uncooperative for 44 percent of TR control families less then 2 percent of AR experimental families For TR control families, caregivers were rated as less motivated and as viewing the situation that led to the report less seriously than the agency. Workers rated families on cooperation as follows: The average levels of cooperation of families during the first visit and last visit were significantly greater for experimental families. Workers were more likely to report that control parents were hostile throughout the case.

13 Services to Families The Missouri demonstration was cost-neutral. No new funding was provided for services in Missouri. There was a greatly increased emphasis on linking families with community resources. In Minnesota, additional funds were available for the demonstration. Temporary funding was received from the McKnight foundation for the duration of the 20-county demonstration project only. In Virginia, no additional state funds were provided, but agencies sometimes obtained additional funds from local government, or applied for grants, or worked with community organizations to increase services.

14 Services to Families (continued) Services were provided to families earlier. This was evident in Missouri with its emphasis on services by assessment workers but was not measured in Minnesota. Families provided with post-assessment services increased in both states. Linkages of families to funded and unfunded community providers increased in both states. The types of services delivered to families changed in both states, with a shift toward family support services that would address financially-related needs.

15 Missouri Time to First Service 1. Services occurred in a more timely manner under the new approach.

16 Minnesota Proportion of Families with Service Case Control Experimental 15.1% Post-Assessment Services after Initial Report and Assessment 36.2% 1. Over twice as many experimental families had a case-management workgroup opened (the condition for provided paid services) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

17 Missouri Utilization of Community Resources Increased neighborhood organizations schools* alcohol/drug rehabilitation MR/DD providers youth organizations DFS--food stamps* DFS--welfare* health care provider* Job Service/Employment Security* JTPA (employment and training) legal services* support groups* child care/respite care/head start* community action agency domestic violence shelter food pantry church/religious organizations* recreational facilities (e.g. YMCA) neighbors/friends/extended family* pilot comparison *=statistically significant difference; p<.01 Overall there was a greater utilization of community resources in pilot areas. Most were not paid by CPS. 0% 5% 10% 15% 20% 25% 30%

18 Minnesota Utilization of Community Resources Increased school mental health provider support group neighbors/friends/extended family emergency food provider recreational facility (e.g. YMCA) neighborhood organization childcare provider/preschool provider/head Start A similar pattern was found in Minnesota health care provider alcohol/drug rehabilitation agency or program community action agency Experimental Control job service/employment security legal services provider domestic violence shelter youth organization employment and training agency (JTPA, etc) p=. 03 p=. 01 church or religious organization MR/DD provider 0% 5% 10% 15% 20% 25% 30%

19 Missouri Services (Paid and Unpaid) Attempted for Families 1. Basic necessity services 2. Job related services 3. Counseling, instruction or crisis All Services Combined 0.15 0.20 0.53 0.62 1.2 1.28 Pilot Comparison 1.87 2.10 1. More actions were taken in pilot areas to assist families in finding needed services. 2. Differences were small but statistically significant. Basic necessity services increased significantly 0 0.5 1 1.5 2 2.5 Average per family

20 Minnesota (Paid and Unpaid) Services to Families individual counseling marital/family/group counseling parenting classes help with basic household needs mental health/psychiatric services childcare/daycare services help with rent or house payments emergency food medical or dental care domestic violence services housing services assistance with transportation educational services respite care/crisis nursery TANF, SSI or food stamps legal services assistance from support groups recreational services homemaker/home management assistance assistance with employment vocational/skill training family preservation services alcohol abuse treatment drug abuse treatment emergency shelter disability services independent living services Control Experimental 0% 5% 10% 15% 20% 25% In Minnesota the change in types of services delivered was clearer. Family support services related to financial needs increased along with traditional counseling and therapeutic services.

21 Recurrence of Reports of Child Maltreatment Recurrence of child abuse and neglect (CA/N) could be measured indirectly: Families with new CA/N reports after final contact following the initial report. Families with later removal and placement of children. Report recurrence declined in both states under AR. Later removals of children declined in Minnesota but no corresponding evidence could be found in Missouri.

22 Missouri Child Abuse and Neglect Report Recurrence after Five Years (Original Demonstration Families) Mean New Hotlines per Family 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 Pilot Comparison 1. With five complete years of data pilot families continued to have lower rates of recurrence. 2. Difference by risk level dwarfed differences by pilot and comparison area 0.0 Low Moderate High Very High CA/N Risk Level

23 The Effects Service versus Approach on Recurrence of Child Maltreatment The experimental design in Minnesota permitted analysis of the relationship of services to families as well as the family-friendly approach of AR to reduction of later recurrence of reports. Services to families lowered recurrence. This might be expected but is a very difficult thing to prove outside the context of a controlled study. The increase in family support services addressing financially related needs may have been an important component of this effect both in Missouri and Minnesota. The non-adversarial and participatory approach to families reduced recurrence whether or not services were delivered. One possible explanation centers on family engagement, family cooperation, improved communication and participation. The exact mechanisms, however, are unknown and should be studied further.

24 The Costs of Alternative Response The Minnesota evaluation included a cost-effectiveness component. Data are still be collected but interim results have been positive. Costs related to case management and other services during the time the initial case was open were greater for AR than control cases. Costs for case management and other services following the closing of the initial case through the end of the follow-up period were LOWER for AR cases. On balance then, Total costs for case management and other services, both separately and combined, were less for AR cases than control cases. With the result that Effectiveness: The mean cost per family of achieving the goal of recurrence avoidance with AR was $398 less than with the traditional approach.

25 The Costs of Alternative Response Total Cost Control $593 $1,538 $2,13 Period 1 Period 2 Experimental $1,132 $804 $1,93 $0.00 $500.00 $1,000.00 $1,500.00 $2,000.00 $2,500.00 Mean Cost Per Family