ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT

Florida Sheriffs Performing Child Protective Investigations ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT Fiscal Year 2011-2012 Conducted jointly by the Florida Department of Children and Families and The Sheriff Offices of Broward, Hillsborough, Manatee, Pasco, Pinellas and Seminole Counties

TABLE OF CONTENTS EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 EVALUATION PLAN AND DESIGN... 4 QUALITY PERFORMANCE REVIEW... 5 OUTCOME MEASURES AND STANDARDS... 6 CONCLUSIONS... 6 QUALITY PERFORMANCE REVIEW... 6 OUTCOME MEASURES... 7 INTRODUCTION... 8 PROGRAM PERFORMANCE EVALUATION... 8 SHERIFF S INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS IN FLORIDA... 8 ANNUALIZE DATA ON ABUSE REPORTS.. 9 PERFORMANCE EVALUATION PLAN AND DESIGN... 12 PERFORMANCE EVALUATION QUESTIONS... 12 DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN... 12 SAMPLING METHODOLOGY AND SIZE... 12 QUALITY PERFORMANCE REVIEW... 13 PEER REVIEW TEAMS... 13 REVIEW INSTRUMENTS... 13 PROGRAM MANAGEMENT... 14 ASSESSING PERFORMANCE... 14 OUTCOME MEASURES ATTAINMENT... 14 MEASURES AND STANDARDS... 14 SOURCES OF DATA AND ANALYSIS METHODS... 15 QUALITY PERFORMANCE (CASE REVIEWS)... 15 MANATEE COUNTY... 17 PASCO COUNTY... 18 BROWARD COUNTY... 19 HILLSBOROUGHCOUNTY... 20 PINELLAS COUNTY... 21 SEMINOLE COUNTY... 22 OUTCOME MEASURES ATTAINMENT... 23 ANNUAL OUTCOMES FOR COMMENCEMENTS OF REPORTS WITHIN 24 HOURS... 24-32 ANNUAL OUTCOMES FOR VICTIMS SEEN WITHIN 24 HOURS OF CASE RECEIVED... 33-40 ANNUAL OUTCOMES FOR INITIAL SUPERVISORY REVIEWS WITHIN 72 HOURS... 41-48 ADDITIONAL OUTCOME PERFORMANCE...49-53 PROGRAM MANAGEMENT... 54-56 COST EFFICIENCY... 56-58 2

EXECUTIVE SUMMARY INTRODUCTION The13 th annual report regarding the performance of those Sheriff Offices formally charged with protecting children within their communities is submitted at a time when Department of Children and Families (DCF) leadership has actively renewed efforts to examine the design, methods, processes and purposes of the Florida child protection system. In this fiscal year, the intake reports received statewide exceeded any prior year in the history of Florida. Sheriff Offices handled more than 206,000 new intakes requiring a commencement action. During this fiscal year DCF was in year two of a statewide initiative to implement changes in program practice affecting hotline, child protective investigations, and case management. Core tenants of child welfare practice reflecting enhanced practices in assessment of families have been established. Representatives statewide from DCF and Sheriff Office s continue to be involved with DCF program officials in examining business processes, purposes, designs and methods. Sheriff Offices handled 27% of the state s investigations and are an essential element in moving forward with shared values, principles and efforts that DCF seeks for enhancing Florida s child protection system. The longstanding commitment of the Sheriff Offices involved in child protective investigations toward engaging families, protecting children, and working in partnership within their communities remains consistent and strong. Several of these protocols received local, state and national recognition and respect for the best practice,community-based collaborations for their investigative responses. In the mid-1990s, a Sheriff Office pilot program in Manatee County led to legislation in 1998 to expand the model. The expansion in 1999 added Sheriff Offices in Pasco, Pinellas and Broward counties. Legislation transferred full responsibility for child protective investigations to these Sheriffs in SFY 1999. In 2000, the Seminole County Sheriff assumed the role, followed by Hillsborough County in 2005. Citrus County Sheriff assumed the role in 2007. These seven Sheriff Offices encompass metropolitan areas where more than a quarter of the state s population resides. The Sheriffs integration into this child protection role is now virtually seamless and significant in each of their communities. Annual oversight of the Sheriff Offices initially defined under section 415.5071, Florida Statutes, required a committee of seven persons appointed by the Governor to address Sheriffs program performance. That committee met with the respective Sheriffs and developed criteria mutually agreed upon for an annual review. The committee held the responsibility for submitting an annual report regarding quality performance, outcomemeasure attainment and cost efficiency. In 2000, section 39.3065, Florida Statutes, ended the committee and an annual report on program performance by the Sheriff Offices became mandated. The mandated annual review shall use criteria mutually agreed upon by the Sheriffs and DCF. This report completed by a team of Peer Reviewers from the Sheriff Offices with DCF, addresses quality performance, outcome 3

measure attainment, and cost efficiency. This is the 13 th Annual Sheriff Offices Peer Review Report. EVALUATION PLAN AND DESIGN The program performance evaluation questions are based upon language in subsection 39.3065(3)(d), F.S. In summary, these questions are: 1. How does the quality of performance involving the Sheriff Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriff Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriff Offices performing child protective investigations in a cost efficient manner? Representatives from the seven Sheriff Offices with DCF comprised the program evaluation planning team. The Sheriff Representatives were: Broward County Sheriff Captain Andrew Koerick, Program Administrator Joseph Paduano, Child Protective Investigation (CPI) Supervisor Seminole County Sheriff Greg Barnett, Captain Jay Saucer, Quality Assurance Pinellas County Sheriff Brandi Lazaris, Program Administrator Jane Melvy, Supervisor Citrus County Sheriff Dave Wyllie, Lieutenant Richard Patterson, CPI Supervisor 4

Hillsborough County Sheriff Major Robert Bullara, Division Commander Jennifer Hock, Program Administrator Pasco County Sheriff Ken Lilian, Director Lisa Tobin, Training Supervisor Manatee County Sheriff Joyce Edick, Operations Program Specialist In 2011, the planning committee adopted several changes to the Peer Review Tool, including two additional questions. The additional questions related to (1) consideration of the background check information in the assessment of risk and (2) the quality of interactions and observations of the victims and other children. Additionally, separating content information in three existing questions, allowed the review of additional details. The question, Interview of victim and children became two questions: Interview of victims and Interview of other children in home. The question, Observations of all victims and children, became Observations of victims and Observations of all other children. The question, Interviews of adult subjects and other household members, became Interview of adult subjects (Alleged persons responsible, parents, caregivers) and Interviews with all other adult household members. DCF Office of Family Safety accepted the changes related to these enhancement question revisions to the 2011 Sheriff s Peer Review Tool. Each Sheriff s Office scheduled an onsite peer review. Reviewers from each Sheriff s Office and DCF were present for onsite case file reviews and conducted entry and exit presentations. QUALITY PERFORMANCE REVIEW The 2012 Sheriff s Peer Review process included three and a half days of onsite visits for each Sheriff s location. The plan included case file reviews and reviews of performance measures as core components of quality performance. The 2012 process fully addressed the quality of practice standards utilizing the revised standardized review tool. The number of files reviewed at each site was determined using a statistical 90% confidence level, with a +/-10% confidence interval. For all review sites, the sample size was 65 investigative records. Excluded from the sample were duplicate reports, institutional reports, foster care referrals, special condition reports, out-of-town inquiries (OTIs), and no-jurisdiction reports. The sample pulled comprised 50 percent reports with a judicial action and 50 percent non-judicial in disposition. 5

The overall score for each Sheriff s Office includes only the results of the internal case file review and the side-by-side review, and was calculated using the Sheriff s Peer Review Access database with each file receiving equal weight in scoring. OUTCOME MEASURES AND STANDARDS Subsection 39.3065(3)(b), F.S., requires that the Sheriffs performing child protective investigations operate, at a minimum, in accordance with the performance standards and outcome measures established for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. These measures amended the Grant Agreements beginning July 2010. For SFY 2010 2011 the report eliminated the 60-day case closure measureand replaced it with a performance measure tracking the timeliness of victims seen within 24 hours of a report received by the Florida Abuse Hotline. Users enter the data for these performance measures and others into the child welfare information system, Florida Safe Families Network (FSFN). This system produces management reports used for determining statewide performance and outcomes. CONCLUSIONS QUALITY PERFORMANCE REVIEW Sheriff s Office involvement in child protection investigations began in the mid-1990s when calls for reform led to successful pilots that spurred legislative commitment to implement statutory changes. During this time, the Legislature also passed new statutes requiring the outsourcing of foster care and related services statewide. It was the Legislature s intent to encourage communities and other stakeholders interested in the well-being of children to participate in assuring that children were safe and well nurtured in their local community. DCF moved aggressively and successfully outsourced the state s foster care and related services to community-based care lead agencies. Including contracting with Sheriffs Offices, Florida has used the communitybased care philosophy, to safely reduce the number of children in foster care and embrace family-centered practices when working with families. 6

Recommendations from the Subcommittee for Children and Families to Governor-Elect Scott s Health and Human Services Transition Team, dated December 20, 2010, noted in part that DCF under the leadership of former Secretary Bob Butterworth and Secretary George Sheldon was, widely recognized by many Floridians as the best run Department in the state. It further concluded, This leadership has been open to finding innovative solutions such as privatization, using technology, and leveraging existing resources in local communities across the state. Successful leveraging of community law-enforcement personnel and their existing resources for conducting child protective investigations continues to result in quality outcomes for protecting children and supporting families. OUTCOME MEASURES The performance outcomes reported in this review confirm Sheriff Offices continue to achieve the performance outcome measures established by the Legislature. Broward, Manatee, Pasco, Pinellas, Hillsborough and Seminole County Sheriff Offices all achieved passing ratings in the record reviews in this year s Peer Review. 7

INTRODUCTION PROGRAM PERFORMANCE EVALUATION This is the 13th Annual Sheriff Offices Peer Review Report.This annual report compiles outcomes of case file audits completed by a team of Peer Reviewers from the Sheriff Offices, data and performance measure information from DCF; and financial information related to CPI costs with the intent of addressing performance, outcome measure attainment, and cost efficiency. This report was initiated by legislation passed in 1998 that based the initial program performance oversight of Sheriff Offices performing child protective duties in the respective counties from a committee of seven persons appointed by the Governor. In 2000, the law changed regarding the annual review to have criteria mutually agreed upon by the Sheriffs and DCF. In compliance to this statute requirement in section 39.3065, F.S., this annual program performance report is generated. SHERIFFS INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS DCF retained within the Florida Safe Families Network (FSFN) data the same population statistics as for State Fiscal Year (SFY) 2010-2011. This data indicates 27.90% of Florida s children reside within a county where the Sheriff has jurisdiction for child protective investigations. However, the Sheriffs handled 26.54% of all initial, additional and special condition intakes received in the state for SFY 2011-2012. Child Population Total Population % State's Child Population % State's Population total 11/12: Initial, Additional, Special Condtion Reports Reports: % State's Total Broward 407,694 1,751,406 9.86% 9.33% 14,450 7.51% Citrus 21,996 142,202 0.53% 0.76% 1,778 0.85% Hillsborough 298,584 1,203,245 7.22% 6.41% 12,617 6.14% Manatee 65,374 319,293 1.58% 1.70% 4,240 1.86% Pasco 89,970 440,628 2.18% 2.35% 5,961 2.85% Pinellas 171,861 927,994 4.16% 4.94% 10,093 5.14% Seminole 98,034 420,100 2.37% 2.24% 4,352 2.11% Sheriff Totals 1,153,513 5,204,868 27.90% 27.73% 53,491 26.54% DCF Totals 2,981,286 13,567,484 72.10% 72.27% 151,700 73.07% CBC Totals 896 State Total 4,134,799 18,772,352 206,087 Sources: Florida State Office of Economic & Demographic Research / [http://edr.state.fl.us/population.htm] Methodology for Reports % state s totals averaged the monthly percentages from FSFN report: Child Investigations Received by Intake Sequence Type. 8

by Month Initial & Additiona l Reports Special Condition reports Total Reports % of State Reports Received child report rate per 1,000 by Month Initial & Additiona l Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 by Month Initial & Additional Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 by Month Initial & Additional Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 Annualized Data on Abuse Reports: The following data reflects the number of initial, additional, and special condition intake reports accepted by DCF and handled monthly with the Sheriff Offices for SFY 2011-2012. This data lists the percentage of reports received, as well as the alleged reporting rate by victim per 1,000 children. Seminole Sheriff Broward Sheriff Jul-11 296 26 322 2.05% 3.01 Jul-11 982 111 1,093 7.34% 2.40 Aug-11 326 39 365 2.04% 3.32 Aug-11 1069 143 1,212 7.23% 2.62 Sep-11 355 49 404 2.17% 3.62 Sep-11 1,110 26 1,136 6.87% 2.72 Oct-11 313 38 351 1.96% 3.19 Oct-11 1,148 147 1,295 7.45% 2.81 Nov-11 375 39 414 2.34% 3.82 Nov-11 1112 149 1,261 7.48% 2.72 Dec-11 311 19 330 2.08% 3.17 Dec-11 1050 78 1,128 7.83% 2.57 Jan-12 345 18 363 2.16% 3.51 Jan-12 1,110 73 1,183 7.38% 2.72 Feb-12 355 12 367 2.14% 3.62 Feb-12 1,250 82 1,332 8.17% 3.06 Mar-12 384 16 400 2.23% 3.91 Mar-12 1,178 67 1,245 7.80% 2.88 Apr-12 360 26 386 2.18% 3.67 Apr-12 1,149 73 1,222 7.27% 2.81 May-12 373 13 386 2.02% 3.8 May-12 1,250 85 1,335 7.66% 3.06 Jun-12 253 11 264 1.90% 2.58 Jun-12 967 41 1,008 7.62% 2.37 11-12 FY Total 4,046 306 4,352 2.11% 3.44 11-12 FY Total 13,375 1,075 14450 7.51% 2.73 Citrus Sheriff Hillsborough Sheriff Jul-11 129 14 143 0.91% 5.86 Jul-11 859 126 985 6.28% 2.87 Aug-11 168 15 183 1.01% 7.63 Aug-11 925 125 1,050 5.87% 3.09 Sep-11 144 19 163 0.84% 6.54 Sep-11 1,002 106 1,108 5.99% 3.35 Oct-11 142 10 152 0.81% 6.45 Oct-11 971 115 1,086 5.87% 3.25 Nov-11 121 8 129 0.72% 5.5 Nov-11 914 106 1,020 5.71% 3.06 Dec-11 107 8 115 0.72% 4.86 Dec-11 842 44 886 5.91% 2.82 Jan-12 158 4 162 0.93% 7.18 Jan-12 996 45 1,041 6.09% 3.33 Feb-12 136 3 139 0.82% 6.18 Feb-12 1,064 54 1,118 6.53% 3.56 Mar-12 152 8 160 0.87% 6.91 Mar-12 1,075 67 1,142 6.41% 3.6 Apr-12 145 6 151 0.88% 6.59 Apr-12 1,038 48 1,086 6.23% 3.47 May-12 145 10 155 0.81% 6.59 May-12 1,153 44 1,197 6.51% 3.86 Jun-12 125 1 126 0.87% 5.68 Jun-12 845 53 898 6.28% 2.83 11-12 FY Total 1,672 106 1,778 0.85% 6.33 11-12 FY Total 11,684 933 12,617 6.14% 3.26 9

by Month Initial & Additional Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 by Month Initial & Additional Reports Special Condition reports Total Reports Child Report Rate per 1,000 by Month Initial & Additional Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 by Month Initial & Additional Reports Special Condition reports Total Reports % of State Reports Received Child Report Rate per 1,000 Manatee Sheriff Pasco Sheriff Jul-11 286 23 309 2.05% 3.01 Jul-11 427 25 452 2.82% 4.74 Aug-11 333 28 361 2.01% 5.09 Aug-11 473 32 505 2.78% 5.25 Sep-11 367 26 393 2.11% 5.61 Sep-11 482 31 513 2.69% 5.35 Oct-11 365 44 409 2.16% 5.58 Oct-11 484 17 501 2.70% 5.37 Nov-11 353 32 385 2.12% 5.39 Nov-11 461 20 481 2.68% 5.12 Dec-11 280 18 298 1.89% 4.28 Dec-11 466 17 483 3.10% 5.17 Jan-12 325 12 337 1.98% 4.97 Jan-12 443 15 458 2.68% 4.92 Feb-12 349 17 366 2.06% 5.33 Feb-12 428 8 436 2.53% 4.75 Mar-12 375 15 390 2.11% 5.73 Mar-12 580 17 597 3.28% 6.44 Apr-12 332 15 347 1,92% 5.07 Apr-12 519 15 534 2.98% 5.76 May-12 353 10 363 1.90% 5.39 May-12 554 24 578 3.00% 6.15 Jun-12 279 3 282 1.90% 4.26 Jun-12 410 13 423 2.96% 4.55 11-12 FY Total 3,997 243 4,240 1.86% 4.98 11-12 FY Total 5,727 234 5,961 2.85% 5.30 Pinellas Sheriff Statewide Jul-11 737 54 791 4.94% 4.28 Jul-11 13,908 1,646 15,554 3.36 Aug-11 877 78 955 5.34% 5.1 Aug-11 15,692 1,949 17,641 3.79 Sep-11 841 64 905 4.88% 4.89 Sep-11 16,356 2,039 18,395 3.95 Oct-11 824 77 901 5.03% 4.79 Oct-11 15,938 1,988 17,926 3.85 Nov-11 353 32 385 4.75% 4.54 Nov-11 15,593 1,924 17,517 3.77 Dec-11 723 43 766 5.06% 4.2 Dec-11 14,178 1,020 15,198 3.42 Jan-12 825 42 867 5.21% 4.8 Jan-12 15,692 856 16,548 3.79 Feb-12 860 32 892 5.18% 5 Feb-12 16,163 847 17,010 3.9 Mar-12 889 33 922 5.17% 5.17 Mar-12 16,773 931 17,704 4.05 Apr-12 890 46 936 5.32% 5.17 Apr-12 16,581 882 17,463 4.01 May-12 931 56 987 5.32% 5.41 May-12 17,398 908 18,306 4.2 Jun-12 762 24 786 5.48% 4.43 Jun-12 15,187 1,638 16,825 3.22 11-12 FY Total 9,512 581 10,093 5.14% 4.82 11-12 FY Total 189,459 16,628 206,087 3.78 The Florida Abuse Hotline in SFY 2011-2012 accepted 206,087 child intake reports/referrals categorized as initial and additional reports and special condition referrals. The various Sheriff Offices were responsible for 53,491 of those intakes. This was 26.54% of all Florida child intakes received. The total child population within the seven counties where the Sheriff Offices have responsibility for child protective investigations equates to 27.9% of Florida s child population. The community-based care agencies handled 896 of these intakes which were foster care referrals. DCF handled the remaining total of 151,700 or 73.07% of the accepted child intakes. Child reporting rates vary significantly in Florida s 67 counties. DCF maintains monthly data on the rate of initial and additional reports received per 1,000 children population. The state average for SFY 2011-2012 was 3.78 per 1,000 child population. Sheriff average was 3.64 per 1,000 child population. Large urban counties incline to average month to month lower than rural counties. There are some local county specific variables known to effect reporting rates, such as when law enforcement agencies, 10

county school officials and judiciary agents promote specific child abuse training classes, seminars, and/or protocols within their business culture. The Florida Abuse Hotline (Hotline) has three options for reporting. Florida receives the majority of intakes telephonically; however web-based reporting and facsimile reports are the other two options. Florida statute does not specify specifically inter-state requests; however both interstate and intra-state requests are referenced within Florida Administrative Code Rule 65C-29.011 and place the requesting and receiving of these requests at the local county level and not at the Hotline. The majority of inter-state requests do not fall under the regulation provided under Interstate Compact for the Placement of Children (ICPC). The work volume associated with intra-state requests (also known as, out-of-town inquires (OTI s) and inter-state requests varies from county to county but is deemed to account for an additional 10% to 15% of incoming casework assigned to child protective investigators. No present DCF protocol or mechanism within Florida records these interstate requests since they are not channeled through the Hotline nor recorded in FSFN. Through the Hotline Florida accepts five types of child intake reports. In-home report references alleged maltreatments by a child s caregiver. Institutional report references alleged maltreatments of a child by another person responsible outside of their caregiver (i.e., incident at school, childcare facility, etc.). Child-on-child referrals are intakes referencing an allegation of a child 12 years of age or younger displaying inappropriate sexual behavior or an alleged juvenile sexual offense. Human Trafficking maltreatments have been accepted in Florida since 2009 and often do not have the alleged person responsible as a caregiver. Additionally, special condition referrals are accepted if a parent is unavailable or a parent is in need of assistance (PNA). The special condition referrals do not list any specific maltreatment, rather a narrative relating to the special condition issue. 11

PERFORMANCE EVALUATION PLAN AND DESIGN The Citrus County Sheriff s Office did not participate in the SFY 2011-2012 annual Sheriff s case reviews based on communication with DCF of their contract not being renewed in November 2012. The Citrus County Sheriff s Office was under a transition plan in the fall of 2012 for transferring duties back to DCF and this conflicted with the time period of the peer review site visits which were conducted from August 2012 through December 2012. PERFORMANCE EVALUATION QUESTIONS The program performance questions for this evaluation were based upon language in ss. 39.3065(3)(d), F.S. These questions are: 1. How does the quality of performance involving the Sheriffs Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriffs Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriffs Offices performing child protective investigations in a cost efficient manner? DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN For the SFY 2011-2012 Peer Review, representatives from DCF and Sheriff Offices agreed to apply a revised Quality Assurance (QA) review instrument. This automated instrument includes evaluation tools for the functional areas of initial response as well as emergency removal. The resulting detailed case review report for each county totals an average of 130 pages with specifics on each case and summaries of each category for the overall Sheriff s Office. Also agreed upon were sampling methodology and instrument scoring procedures. SAMPLING METHODOLOGY AND SIZE The Peer Review assesses investigative casework on a fiscal year basis. Last year s report changed the reporting period from calendar year to state fiscal year. The work performed from July 1, 2011 through June 30, 2012 formed the basis for investigative casework and performance outcomes data in this report. The methodology for the quality performance component of the evaluation called for a review of closed investigation records. The definition of a closed report as used in this Peer Review is an investigation that has been completed, reviewed, signed by the investigator and supervisor, and closed in the FSFN system. As in prior reviews, 12

reviewers considered the fact that many closed investigations were still potentially active in initial court dependency proceedings involving the investigator, with final judicial disposition outcomes not finalized. The FSFN system selected a random sample size from the list of reports received on or after July 1, 2010, and closed on or before June 30, 2011. The number of reports sampled was determined by using the DCF Sample Size Calculator utilizing a confidence level of 90 percent with an error rate of plus or minus 10 percent. Excluded from the sample were duplicate, institutional and special condition reports. Also excluded were reports where it was determined that there was no jurisdiction to investigate and OTI requests. Fifty percent of the sample consisted of reports that resulted in judicial action and 50 percent were non-judicial in disposition. QUALITY PERFORMANCE REVIEW PEER REVIEW TEAMS Subsection 39.3065(3)(d), F.S., requires that the program performance evaluation be conducted by a team of Peer Reviewers comprised of representatives from the Sheriff Offices with support from DCF. DCF s Quality Assurance program developed the approach to the Peer Review. Modified for this evaluation the case review instruments included a program management instrument. The Peer Review process is similar to those procedures used by national accreditation organizations. The definition of peer, as used in this performance evaluation, means Sheriff and DCF personnel who perform protective investigations and their respective quality assurance personnel, where applicable. The criteria established for Peer Reviewers included experience in child protective investigations; certification or, minimally, completion of child protective investigative training; supervisory level staff or above, or a staff member of DCF s Quality Assurance program. Beginning in SFY 2010, the Peer Review teams included participants from each Sheriff s Office and two representatives from DCF s local District Office. This team composition continued in this year s review. The Peer Review team did not collect or analyze cost data for the cost efficiency component of this evaluation. DCF provided cost data based upon expenditure reports provided by each Sheriff s Office for SFY 2011-2012. REVIEW INSTRUMENTS ABUSE REPORT RECORD REVIEW The Peer Review team conducted a review of the 65 selected files. The abuse report review instrument addresses the statutory requirements for the investigator s initial response to the report of alleged child maltreatment and the emergency removal and placement of children, if this occurred. The instrument contains a number of statements or questions that address indicators used to determine the achievement of essential steps in the investigation process. Indicators cover such areas as thoroughness of 13

background checks, timeliness of investigations, and thoroughness of CSAs. In all, 24 indicators comprise the initial response and emergency removal and placement review form. PROGRAM MANAGEMENT When onsite, the lead representative from the Peer Review Team reviewed management core records listed in this report. Because the scores for the past several years have all been 100% in the management category for these Sheriff Offices, the Peer Review Team decided not to score this category in 2011. ASSESSING PERFORMANCE This year, the rating on the file reviews used a four-step scale with four possible numerical scores. Ratings used were: not achieved, partially achieved, substantially achieved, and achieved. Point values assigned as follows: 0 - Not Achieved 5 - Partially Achieved 7 - Substantially Achieved 9 - Achieved Overall performance was the sum of the indicator scores, divided by the maximum possible score, which produced a percentage. Using the performance categories, the derived percentages translate into the following performance levels: Passing 80-100 % Non-Passing 0-79 % OUTCOME MEASURES ATTAINMENT MEASURES AND STANDARDS Subsection 39.3065(3)(b), F.S., requires that the Sheriffs operate in accordance with the performance standards and outcome measures established by the Legislature for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. 14

Sources of Data and Analysis Methods The data for all three measures come from the FSFN management report, Leader Board for Investigations. The report lists performance for each DCF Region and Sheriff s Office that operates a child protective investigation program. The report period represents SFY 2011-2012. The algorithms for calculating the outcome measures are those established by DCF in consultation with the Governor s Office of Policy and Budget as well as the substantive and appropriations committees of the Legislature having jurisdiction for DCF. The algorithms are as follows: The first performance measure (Investigations commenced within 24 hours): The numerator is the number of reports commenced within 24 hours of receipt of the report. The denominator is the total number of reports closed in the report period. The second performance measure (Victims seen within 24 hours of report received): The numerator is the number of victims listed in recorded reports. The denominator is the total number of victims seen within 24 hours as recorded in the FSFN computer system. This data is retrieved based on closed investigations from July 2011 through June 2012 in a DCF published monthly report known as the Leader Board. The third performance measure (Child Safety Assessments (CSA) reviewed by supervisors in accordance with DCF s timeframes): The numerator is the number of initial CSAs reviewed by the supervisor within 72 hours of submission of the initial CSA for review. The denominator is the total number of reports closed in the report period. Quality Performance Presented in the table below is a summary of the performance findings. The true percentages, if all reports rather than samples had been used, can be assumed with confidence to fall somewhere within plus or minus 10 percent at the 90 percent confidence level. Listed under the following titled categories are core components of questions within the quality assurance (QA) tool: Removal 1. Reasonable Efforts 2. Psychotropic Medication 3. Placement Priority 4. Home Study Initial Response 5. Background Checks 6. Victim Contact 7. Contact with Other Children 15

8. Interviews with Victims 9. Interviews with Other Children 10. Observations of all Victims 11. Observations of all Other Children 12. Quality of Interactions and Observations 13. Interviews with Adult Subjects 14. Interviews with all Other Household Members 15. Indian Child Welfare Act (ICWA) 16. Relevant Collateral Contacts 17. Contact with Reporter 18. Child Protection Team 19. Children s Legal Services Staffing 20. Second Party Review of Safety Assessment 21. Completion of Supervisory Direction 22. Maltreatments 23. Investigative Summary 24. Safety Decision Entered into an electronic format that captures the responses of the reviewer and identifies the reviewer s assessment of each question are the results of the assessment on the above areas. This allows for immediate feedback at the end of the review via a document averaging 150 pages, showing each reviewer s ratings for each question area shown above on cases assigned to the reviewer. The electronic program allows the reviewers to complete an onsite stratification of the data at the conclusion of the review. The electronic program is capable of drilling down in the data to identify specific areas of concern by pinpointing a question and then identifying the unit, supervisor or child protective investigator responsible. This enables the program administrator to take action toward correcting any area of deficiency identified within any unit, or by supervisor or investigator. Completed at each site, exit interviews with reviewers presented trends and information on cases they reviewed for management staff and supervisors. The finalized report fully documented all information discussed at the exit conferences. The review site receives the finalized report prior to the exit conference. 16

Manatee County Sheriff s Office Manatee site visit conducted: November 26 November 29, 2012. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Kristine Fletcher, Pasco Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Jane Melby, Pinellas Sheriff s Office Shawn Creney, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 7.04 3. Placement Priority Average agency score 8.72 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.72 6. Victim Contact Average agency score 8.80 7. Contact with Other Children Average agency score 8.45 8. Interviews with Victims Average agency score 8.70 9. Interviews with Other Children Average agency score 8.24 10. Observations of all Victims Average agency score 8.97 11. Observations of all Other Children Average agency score 9.00 12. Interviews with Adult Subjects Average agency score 8.15 13. Interviews with all Other Household Members Average agency score 8.76 14. Indian Child Welfare Act (ICWA) Average agency score 8.72 15. Relevant Collateral Contacts Average agency score 8.32 16. Contact with Reporter Average agency score 8.96 17. Communication Between the Investigator & Case Manager Average agency score 9.00 18. Child Protection Team Average agency score 8.10 19. Children s Legal Services Staffing Average agency score 9.00 20. Second Party Review of Safety Assessment Average agency score 8.55 21. Completion of Supervisory Direction Average agency score 8.50 22. Maltreatments Average agency score 8.12 23. Investigative Summary Average agency score 7.51 24. Safety Decision Average agency score 8.28 Final Score: 94.37% 17

Pasco County Sheriff s Office Pasco site visit conducted: December 5-8, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Joyce Edick, Manatee Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kimberly Williams, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 8.62 3. Placement Priority Average agency score 8.65 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 7.73 6. Victim Contact Average agency score 8.90 7. Contact with Other Children Average agency score 8.55 8. Interviews with Victims Average agency score 8.51 9. Interviews with OtherChildren Average agency score 8.67 10. Observations of all Victims Average agency score 8.67 11. Observations of all Other Children Average agency score 8.52 12. Interviews with Adult Subjects Average agency score 8.44 13. Interviews with all Other Household Members Average agency score 8.15 14. Indian Child Welfare Act (ICWA) Average agency score 8.86 15. Relevant Collateral Contacts Average agency score 7.93 16. Contact with Reporter Average agency score 8.35 17. Communication Between the Investigator & Case Manager Average agency score 7.71 18. Child Protection Team Average agency score 8.66 19. Children s Legal Services Staffing Average agency score 8.48 20. Second Party Review of Safety Assessment Average agency score 8.60 21. Completion of Supervisory Direction Average agency score 8.23 22. Maltreatments Average agency score 8.04 23. Investigative Summary Average agency score 7.55 24. Safety Decision Average agency score 7.53 Final Score: 92.40% 18

Broward County Sheriff s Office Broward site visit conducted: October 01 04, 2012. The reviewers were: Jay Saucer, Seminole Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kristine Fletcher, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Kat Matthews, Hillsborough Sheriff s Office Mark Holsapfel, DCF Southeast Region Quality Assurance (QA) Sharon Mitchell, DCF Southeast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the rightis the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 9.00 3. Placement Priority Average agency score 9.00 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.60 6. Victim Contact Average agency score 8.94 7. Contact with Other Children Average agency score 8.75 8. Interviews with Victims Average agency score 8.82 9. Interviews with Other Children Average agency score 9.00 10. Observations of all Victims Average agency score 8.97 11. Observations of all Other Children Average agency score 8.87 12. Interviews with Adult Subjects Average agency score 8.50 13. Interviews with all Other Household Members Average agency score 8.56 14. Indian Child Welfare Act (ICWA) Average agency score 8.86 15. Relevant Collateral Contacts Average agency score 8.81 16. Contact with Reporter Average agency score 9.00 17. Communication Between the Investigator & Case Manager Average agency score 9.00 18. Child Protection Team Average agency score 9.00 19. Children s Legal Services Staffing Average agency score 9.00 20. Second Party Review of Safety Assessment Average agency score 8.38 21. Completion of Supervisory Direction Average agency score 8.75 22. Maltreatments Average agency score 8.63 23. Investigative Summary Average agency score 8.23 24. Safety Decision Average agency score 8.66 Final Score: 97.23% 19

Hillsborough County Sheriff s Office Hillsborough site visit conducted: September 17 September 20, 2012. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Joyce Edick, Manatee Sheriff s Office Kristine Fletcher, Pasco Sheriff s Office Katie Favara, Pinellas Sheriff s Office Margaret Gohman, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the rightis the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 9.00 3. Placement Priority Average agency score 9.00 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.23 6. Victim Contact Average agency score 8.77 7. Contact with Other Children Average agency score 8.33 8. Interviews with Victims Average agency score 8.71 9. Interviews with Other Children Average agency score 8.40 10. Observations of all Victims Average agency score 9.00 11. Observations of all Other Children Average agency score 8.32 12. Interviews with Adult Subjects Average agency score 8.59 13. Interviews with all Other Household Members Average agency score 8.35 14. Indian Child Welfare Act (ICWA) Average agency score 9.00 15. Relevant Collateral Contacts Average agency score 8.54 16. Contact with Reporter Average agency score 8.97 17. Communication Between the Investigator & Case Manager Average agency score 9.00 18. Child Protection Team Average agency score 8.19 19. Children s Legal Services Staffing Average agency score 9.00 20. Second Party Review of Safety Assessment Average agency score 8.47 21. Completion of Supervisory Direction Average agency score 8.60 22. Maltreatments Average agency score 8.35 23. Investigative Summary Average agency score 842 24. Safety Decision Average agency score 8.62 Final Score: 96.02% 20

Pinellas County Sheriff s Office Pinellas site visit conducted: September 10 September 13, 2012. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Peggy Niemann, DCF Suncoast Region Quality Assurance (QA) Kathy Newcomb, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 6.58 3. Placement Priority Average agency score 8.65 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.66 6. Victim Contact Average agency score 9.00 7. Contact with Other Children Average agency score 9.00 8. Interviews with Victims Average agency score 8.51 9. Interviews with Other Children Average agency score 9.00 10. Observations of all Victims Average agency score 8.71 11. Observations of all Other Children Average agency score 8.50 12. Interviews with Adult Subjects Average agency score 8.75 13. Interviews with all Other Household Members Average agency score 8.29 14. Indian Child Welfare Act (ICWA) Average agency score 9.00 15. Relevant Collateral Contacts Average agency score 8.57 16. Contact with Reporter Average agency score 9.00 17. Communication Between the Investigator & Case Manager Average agency score 9.00 18. Child Protection Team Average agency score 8.42 19. Children s Legal Services Staffing Average agency score 9.00 20. Second Party Review of Safety Assessment Average agency score 9.00 21. Completion of Supervisory Direction Average agency score 8.66 22. Maltreatments Average agency score 8.63 23. Investigative Summary Average agency score 8.32 24. Safety Decision Average agency score 8.54 Final Score: 96.20% 21

Seminole County Sheriff s Office Seminole site visit conducted: August 27 August 30, 2012. The reviewers were: Joseph Paduano, Broward Sheriff s Office Joyce Edick, Manatee Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Kristine Fletcher, Pasco Sheriff s Office Jane Melby, Pinellas Sheriff s Office Teresa Vella, DCF Central Region Quality Assurance (QA) John Lewis, DCF Central Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotropic Medication Average agency score 9.00 3. Placement Priority Average agency score 8.48 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.72 6. Victim Contact Average agency score 9.00 7. Contact with Other Children Average agency score 8.76 8. Interviews with Victims Average agency score 8.87 9. Interviews with Other Children Average agency score 8.64 10. Observations of all Victims Average agency score 8.91 11. Observations of all Other Children Average agency score 8.60 12. Interviews with Adult Subjects Average agency score 8.69 13. Interviews with all Other Household Members Average agency score 8.69 14. Indian Child Welfare Act (ICWA) Average agency score 8.66 15. Relevant Collateral Contacts Average agency score 8.75 16. Contact with Reporter Average agency score 8.97 17. Communication Between the Investigator & Case Manager Average agency score 9.00 18. Child Protection Team Average agency score 8.75 19. Children s Legal Services Staffing Average agency score 9.00 20. Second Party Review of Safety Assessment Average agency score 9.00 21. Completion of Supervisory Direction Average agency score 8.60 22. Maltreatments Average agency score 8.60 23. Investigative Summary Average agency score 7.92 24. Safety Decision Average agency score 8.38 Final Score: 96.70% 22

Outcome Measures Attainment The performance measures listed within the Sheriffs Grant Agreement determine outcome performance attainment. For SFY 2010 2011 the report eliminated the performance measure of closing 90% of all investigations by the 60 th day and replaced it with a performance measure of seeing 85% of victims within 24 hours. Data came from the Florida Safe Families Network (FSFN) management reports generated monthly and based on closed report information. Noted below are three measures with their statutory or agency basis: I One hundred percent (100%) of investigations commenced within 24 hours FSFN captures this performance measure on reports coded in-home and institutional. The special condition reports and reports closed as duplicate or no-jurisdiction are not applicable to this measure. The performance measure is in the Sheriffs Grant Agreements. Based foremost on Florida Statutes, Florida Administrative Code also references this performance measure. Subsection 39.201(5), Florida Statutes, in part states: If it appears that the immediate safety or well-being of a child is endangered, that the family may flee or the child will be unavailable for purposes of conducting a child protective investigation, or that the facts otherwise so warrant, the department shall commence an investigation immediately, regardless of the time of day or night. In all other child abuse, abandonment, or neglect cases, a child protective investigation shall be commenced within 24 hours after receipt of the report. II Percent of child victims seen within 24 hours [Target goal 85%] FSFN captures this performance measure on reports coded in-home and institutional. The special condition reports and reports closed as duplicate or no-jurisdiction are not applicable to this measure. The performance measure is based on 65C-29.003(1)(b), Florida Administrative Code (F.A.C.), which states: Commencement of the investigation is the first attempt to complete an on-site visit for the purpose of making a face-to-face contact with the child victim of the report within twenty-four hours of acceptance of the report... Subsection 39.302(1), Florida Statutes, references this performance measure. The subsection states in part: the department shall initiate a child protective investigation within the timeframe established under s. 39.201(5)... 23

III One hundred percent (100%) of reports reviewed by supervisors within 72 hours FSFN captures this performance measure on reports coded in-home and institutional. The special condition reports and reports closed as duplicate or no-jurisdiction are not applicable to this measure. This performance measure is also a requirement in the Sheriffs Grant Agreements with DCF. The performance measure is based on Florida Administrative Code 65C-29.003(5)(b) - which states: Supervisors must review all child protective assessments and assure that safety plans are in place when needed, and that the plan appropriately addresses the identified safety threats. This review shall be completed within seventy-two hours from the time the automated assessment tool is submitted to the supervisor for review. Annual Outcomes for Commencements of Reports Within 24 hours This performance outcome is significant, as Florida Statutes require DCF to be capable of receiving and investigating reports of known or suspected child abuse, abandonment, or neglect, 24 hours a day, 7 days a week. In SFY 2011 2012 child protective investigators handled 206,087 (Source: FSFN Number of Reports Received Monthly by County) initial and additional reports as well as special condition referrals. Of this number, 84% (or 173,638) were regular initial abuse reports; 8% (or 16,628) were special condition intakes, and 7.6% (or 15,821) were additional reports. All of these required an initial 24-hour or immediate response, except some parent needs assistance condition cases permitted by DCF protocol and handled outside of the customary 24-hour face-to-face commencement response. Calls received at the Florida Abuse Hotline with supplemental information (with no additional allegations of harm on open investigations), were not referenced or added statistically to the total number of reports received, since supplemental reports do not require additional child protection investigative actions. The DCF FSFN performance reports exclude special condition referrals from being included in the statistical data tracking of this measure. Therefore, this data would exclude: child-on-child sexual abuse referrals, foster care referrals, parent unavailable referrals, and parent needs assistance referrals. The data provided in the statewide Leader Board also excludes those report commencements that are associated with an additional report. Finally, this report also excludes those investigation cases coded closed as being a duplicate or as no-jurisdiction. The computer system, FSFN, therefore selects data only associated with initial commencements, not additional calls and those closed in a traditional fashion. If it appears that the immediate safety or well-being of a child is endangered, the family may flee, the child will be unavailable for purposes of conducting a child protective investigation, or that the facts otherwise so warrant, DCF is required to commence an 24