ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT

Florida Sheriffs Performing Child Protective Investigations ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT Fiscal Year 2010-2011 Conducted jointly by DCF and the Sheriff Offices of Broward, Citrus, Hillsborough, Manatee, Pasco, Pinellas and Seminole Counties

TABLE OF CONTENTS EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 EVALUATION PLAN AND DESIGN... 4 QUALITY PERFORMANCE REVIEW... 5 OUTCOME MEASURES AND STANDARDS... 6 CONCLUSIONS... 6 QUALITY PERFORMANCE REVIEW... 6 OUTCOME MEASURES... 7 RECOMMENDATIONS... 7 INTRODUCTION... 8 PROGRAM PERFORMANCE EVALUATION... 8 SHERIFF S INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS IN FLORIDA... 8 ANNUALIZE DATA ON ABUSE REPORTS.. 9 PERFORMANCE EVALUATION PLAN AND DESIGN... 11 PERFORMANCE EVALUATION QUESTIONS... 11 DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN... 11 SAMPLING METHODOLOGY AND SIZE... 11 QUALITY PERFORMANCE REVIEW... 12 PEER REVIEW TEAMS... 12 REVIEW INSTRUMENTS... 12 PROGRAM MANAGEMENT... 12 ASSESSING PERFORMANCE... 13 OUTCOME MEASURES ATTAINMENT... 13 MEASURES AND STANDARDS... 13 SOURCES OF DATA AND ANALYSIS METHODS... 13 QUALITY PERFORMANCE (CASE REVIEWS)... 14 MANATEE COUNTY... 16 PASCO COUNTY... 17 BROWARD COUNTY... 18 CITRUS COUNTY... 19 HILLSBOROUGH COUNTY... 20 PINELLAS COUNTY... 21 SEMINOLE COUNTY... 22 OUTCOME MEASURES ATTAINMENT... 23 ANNUAL OUTCOMES FOR COMMENCEMENTS OF REPORTS WITHIN 24 HOURS... 24-32 ANNUAL OUTCOMES FOR VICTIMS SEEN WITHIN 24 HOURS OF CASE RECEIVED... 33-40 ANNUAL OUTCOMES FOR INITIAL SUPERVISORY REVIEWS WITHIN 72 HOURS... 41-48 ADDITIONAL OUTCOME PERFORMANCE...49-52 PROGRAM MANAGEMENT... 53-54 COST EFFICIENCY... 55-56 RECOMMENDATIONS... 57 2

EXECUTIVE SUMMARY INTRODUCTION The 12 th annual report regarding the performance of those Sheriff Offices formally charged with protecting children within their communities is submitted at a time when Department of Children and Families (DCF) leadership has actively renewed efforts to examine the design, methods, processes and purposes of the Florida child protection system. In this fiscal year, the intake reports received statewide exceeded any prior year in the history of Florida. Sheriff Offices handled more than 214,000 new intakes requiring a commencement action, which is 12% above the prior fiscal year. Monthly caseload increases of 12% significantly challenged all operations for both DCF and Sheriff Offices. The depressed economic conditions within the state added additional pressures to many families affected by DCF interventions. Despite these challenges statewide, the outcomes for DCF and the Sheriff Offices continue to remain first-rate. DCF prioritized in 2011 a statewide initiative to implement changes in program practice areas affecting hotline and child protective investigations. Representatives statewide from DCF and Sheriff Offices child protective investigation programs have been involved with DCF program officials in examining business processes, purposes, designs and methods. As a result, in 2012 recommendations from many related workgroups will be used both legislatively and administratively to modify and enhance roles, technologies, policy and various practices to enhance the entire framework of Florida s child protection responses. By design, these changes will enhance selection and hiring practices, working conditions, opportunities, and compensation for career child protection professionals employed by both Sheriffs and DCF. Additionally, many enhancements will affect the entire system. Sheriff Offices handled 27% of the state s investigations and are an essential element in moving forward with shared values, principles and efforts that DCF seeks for enhancing Florida s child protection system. The longstanding commitment of the Sheriff Offices involved in child protective investigations toward engaging families, protecting children, and working in partnership within their communities remains consistent and strong. Several of these protocols received local, state and national recognition and respect for the best practice, community-based collaborations for their investigative responses. In the mid-1990s, a Sheriff Office pilot program in Manatee County led to legislation in 1998 to expand the model. The expansion in 1999 added Sheriff Offices in Pasco, Pinellas and Broward counties. Legislation transferred full responsibility for child protective investigations to these Sheriffs in FY1999. In 2000, the Seminole County Sheriff assumed the role, followed by Hillsborough County in 2005. Citrus County Sheriff assumed the role in 2007. These seven Sheriff Offices encompass metropolitan areas where more than a quarter of the state s population resides. The Sheriffs 3

integration into this child protection role is now virtually seamless and significant in each of their communities. Annual oversight of the Sheriff Offices initially defined under Chapter 98 180, Laws of Florida, required a committee of seven persons appointed by the Governor to address Sheriffs program performance. That committee met with the respective Sheriffs and developed criteria mutually agreed upon for an annual review. The committee held the responsibility for submitting an annual report regarding quality performance, outcome measure attainment and cost efficiency. In 2000, Chapter 2000-139, Laws of Florida, the committee ended and an annual report on program performance by the Sheriff Offices became mandated. The mandated annual review shall use criteria mutually agreed upon by the Sheriffs and DCF. This report completed by a team of Peer Reviewers from the Sheriff Offices with support from DCF, addresses quality performance, outcome measure attainment, and cost efficiency. This is the 12 th Annual Sheriff Offices Peer Review Report. EVALUATION PLAN AND DESIGN The program performance evaluation questions are based upon language in subsection 39.3065(3)(d), F.S. In summary, these questions are: 1. How does the quality of performance involving the Sheriff Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriff Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriff Offices performing child protective investigations in a cost efficient manner? Representatives from the seven Sheriff Offices with support from DCF comprised the program evaluation planning team. The Sheriff Representatives were: Broward County Sheriff Capt. Andrew Koerick, Program Administrator Joseph Paduano, Child Protective Investigation (CPI) Supervisor Seminole County Sheriff Capt. Greg Barnett, Captain Jay Saucer, Quality Assurance Pinellas County Sheriff Brandi Lazaris, Program Administrator Jane Melvy, Supervisor Citrus County Sheriff Lt. Dave Wyllie, Lieutenant Richard Patterson, CPI Supervisor 4

Hillsborough County Sheriff Major Robert Bullara, Division Commander Jennifer Hock, Program Administrator Pasco County Sheriff Ken Lilian, Director Lisa Tobin, Training Supervisor Manatee County Sheriff Joyce Edick, Operations Program Specialist. In 2011, the planning committee adopted several changes to the Peer Review Tool, including two additional questions. The additional questions related to (1) consideration of the background check information in the assessment of risk and (2) the quality of interactions and observations of the victims and other children. Additionally, separating content information in three existing questions, allowed the review of additional details. The question, Interview of victim and children became two questions: Interview of victims and Interview of other children in home. The question, Observations of all victims and children, became Observations of victims and Observations of all other children. The question, Interviews of adult subjects and other household members, became Interview of adult subjects (Alleged persons responsible, parents, caregivers) and Interviews with all other adult household members. DCF Office of Family Safety accepted the changes related to these enhancement question revisions to the 2011 Sheriff s Peer Review Tool. Each Sheriff s Office scheduled an onsite review. Reviewers from each Sheriff s Office and DCF were present for onsite case file reviews and conducted entry and exit presentations. QUALITY PERFORMANCE REVIEW The 2011 Sheriff s Peer Review process included three and a half days of onsite visits for each Sheriff s location. The plan developed collaboratively included case file reviews and reviews of performance measures as core components of quality performance. The 2011 process fully addressed the quality of practice standards utilizing the revised standardized review tool. The number of files reviewed at each site was determined using a statistical 90% confidence level, with a +/-10% confidence interval. For all review sites, the sample size was 65 files. Excluded from the sample were duplicate reports, institutional reports, foster care referrals, special condition reports, out of town inquiries (OTIs), and no-jurisdiction reports. The sample pulled comprised 50 percent reports with a judicial action and 50 percent non-judicial in disposition. 5

The overall score for each Sheriff s Office includes only the results of the internal case file review and the side-by-side review, and was calculated using the Sheriff s Peer Review Access database with each file receiving equal weight in scoring. OUTCOME MEASURES AND STANDARDS Subsection 39.3065(3)(b), F.S., requires that the Sheriffs performing child protective investigations operate, at a minimum, in accordance with the performance standards and outcome measures established for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. These measures amended the Grant Agreements beginning July 2010. For FY2010 2011 the report eliminated the 60-day case closure measure and replaced it with a performance measure tracking the timeliness of victims seen within 24 hours of a report received by the Florida Abuse Hotline. Users enter the data for these performance measures and others into the child welfare information system, Florida Safe Families Network (FSFN). This system produces management reports used for determining statewide performance and outcomes. CONCLUSIONS QUALITY PERFORMANCE REVIEW Sheriff s Office involvement in child protection investigations began in the mid-1990s when calls for reform led to successful pilots that spurred legislative commitment to implement statutory changes. During this time, the Legislature also passed new statutes requiring the outsourcing of foster care and related services statewide. It was the Legislature s intent to encourage communities and other stakeholders interested in the well-being of children to participate in assuring that children were safe and well nurtured in their local community. DCF moved aggressively and successfully outsourced the state s foster care and related services to community-based care lead agencies. Including contracting with Sheriffs Offices, Florida has used the communitybased care philosophy, to safely reduce the number of children in foster care and embrace family-centered practices when working with families. 6

Recommendations from the Subcommittee for Children and Families to Governor-Elect Scott s Health and Human Services Transition Team, dated December 20, 2010, noted in part that DCF under the leadership of former Secretary Bob Butterworth and Secretary George Sheldon was, widely recognized by many Floridians as the best run Department in the state. It further concluded, This leadership has been open to finding innovative solutions such as privatization, using technology, and leveraging existing resources in local communities across the state. Successful leveraging of community law-enforcement personnel and their existing resources for conducting child protective investigations continues to result in quality outcomes for protecting children and supporting families. OUTCOME MEASURES The performance outcomes reported in this review confirm Sheriff Offices continue to achieve the performance outcome measures established by the Legislature. Broward, Citrus, Manatee, Pasco, Pinellas, Hillsborough and Seminole County Sheriff Offices all achieved passing ratings in the record reviews in this year s Peer Review. RECOMMENDATIONS 1. The Department of Children and Families should continue its grants (contract for Citrus) to the Sheriff Offices in Manatee, Pasco, Seminole, Broward, Hillsborough, Pinellas, and Citrus counties to conduct child protective investigations. 2. The commendable, longstanding and successful roles that Sheriff Offices have had in Florida for providing child protective investigative services should warrant DCF and the legislature to encourage more sheriff expansions along with corresponding funding for all of their successful operations. 3. The Department of Children and Families should continue to report on the participating Sheriff Offices with regard to achieving the performance measures and targets. 4. The Department of Children and Families should continue involving representatives from the Sheriff Offices in identifying appropriate practice standards and performance measures for child protective investigations. 5. The Department of Children and Families, and the Sheriff Offices should use the results of this year s quality performance review to identify and implement improvements for pre-service and in-service training of investigators and supervisors. 7

INTRODUCTION PROGRAM PERFORMANCE EVALUATION Annual oversight of the Sheriff Offices initially defined under Chapter 98 180, Laws of Florida, required a committee of seven persons appointed by the Governor to address Sheriffs program performance. That committee met with the respective Sheriffs and developed criteria mutually agreed upon for an annual review. The committee held the responsibility for submitting an annual report regarding quality performance, outcome measure attainment and cost efficiency. In 2000, Chapter 2000-139, Laws of Florida, the committee ended and an annual report on program performance by the Sheriff Offices became mandated. The mandated annual review shall use criteria mutually agreed upon by the Sheriffs and DCF. This report completed by a team of Peer Reviewers from the Sheriff Offices with support from DCF, addresses quality performance, outcome measure attainment, and cost efficiency. This is the 12th Annual Sheriff Offices Peer Review Report. SHERIFFS INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS According to Florida Safe Families Network (FSFN) data, some population shifts occurred within Florida over the past fiscal year. Overall, 27.90% of the state s children reside within a county where the Sheriff has jurisdiction for child protective investigations. However, the Sheriffs handled 26.99% of all initial, additional and special condition intakes received in the state for FY2010-2011. Child Population Total Population % state's child's population % state's population total 10/11: All Initial, Additional, Special Condition Reports Reports: % state's total Broward 407,694 1,751,406 9.86% 9.33% 15,461 7.43% Citrus 21,996 142,202 0.53% 0.76% 1,890 0.87% Hillsborough 298,584 1,203,245 7.22% 6.41% 13,635 6.27% Manatee 65,374 319,293 1.58% 1.70% 4,494 2.03% Pasco 89,970 440,628 2.18% 2.35% 6,202 2.83% Pinellas 171,861 927,994 4.16% 4.94% 11,593 5.23% Seminole 98,034 420,100 2.37% 2.24% 4,723 2.15% Sheriff totals 1,153,513 5,204,868 27.90% 27.73% 57,998 26.99% DCF totals 2,981,286 13,567,484 72.10% 72.27% 156,916 73.01% State total 4,134,799 18,772,352 214,914 Sources: Florida State Office of Economic & Demographic Research / [http://edr.state.fl.us/population.htm] 8

Annualized Data on Abuse Reports: The following data reflects the number of initial, additional, and special condition intake reports accepted by DCF and handled monthly with the seven Sheriff Offices for FY2010-2011. This data lists the percentage of reports received, as well as the alleged reporting rate by victim per 1,000 children. by Month Initial & Additional Reports Special Condition reports Seminole Broward (District 7) (District 10) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Jul-10 259 51 1.92% 310 2.6419 942 149 7.14% 1,091 2.3106 Aug-10 343 53 2.33% 396 3.4988 965 154 7.11% 1,119 2.3670 Sep-10 297 67 1.95% 364 3.0296 1,022 181 6.86% 1,203 2.5068 Oct-10 395 67 2.44% 462 4.0292 1,177 177 7.43% 1,354 2.8870 Nov-10 370 68 2.57% 438 3.7742 937 169 6.65% 1,106 2.2983 Dec-10 294 52 2.25% 346 2.9990 950 153 7.57% 1,103 2.3302 Jan-11 345 50 2.14% 395 3.5192 1,121 191 7.51% 1,312 2.7496 Feb-11 329 46 2.12% 375 3.3560 1,194 169 8.01% 1,363 2.9287 Mar-11 358 55 1.89% 413 3.6518 1,398 197 7.81% 1,595 3.4290 Apr-11 406 40 2.19% 446 4.1414 1,282 145 7.44% 1,427 3.1445 May-11 409 43 2.13% 452 4.1720 1,320 187 7.56% 1,507 3.2377 Jun-11 300 26 1.90% 326 3.0602 1,127 154 8.10% 1,281 2.7643 10-11 FYT Total 4,105 618 2.15% 4,723 3.4894 13,435 2,026 7.43% 15,461 2.7461 % of state received Total Reports child report rate per 1,000 by Month Initial & Additional Reports Special Condition reports Citrus (District 13) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Hillsborough (Suncoast) Jul-10 119 17 0.86% 136 5.4101 816 144 5.99% 960 2.7329 Aug-10 161 20 1.12% 181 7.3195 858 153 6.11% 1,011 2.8736 Sep-10 142 19 0.89% 161 6.4557 955 186 6.19% 1,141 3.1984 Oct-10 157 21 0.93% 178 7.1377 1,027 175 6.14% 1,202 3.4396 Nov-10 139 16 0.89% 155 6.3193 958 138 6.40% 1,096 3.2085 Dec-10 121 14 0.89% 135 5.5010 809 138 6.29% 947 2.7095 Jan-11 155 15 0.94% 170 7.0467 1,092 144 6.94% 1,236 3.6573 Feb-11 141 14 0.89% 155 6.4103 1,067 136 6.82% 1,203 3.5735 Mar-11 147 15 0.75% 162 6.6830 1,182 150 6.20% 1,332 3.9587 Apr-11 134 9 0.69% 143 6.0920 1,112 112 6.08% 1,224 3.7242 May-11 156 12 0.79% 168 7.0922 1,122 123 6.01% 1,245 3.7577 Jun-11 134 12 0.83% 146 6.0920 929 109 6.03% 1,038 3.1114 10-11 FYT Total 1,706 184 0.87% 1,890 6.4633 11,927 1,708 6.27% 13,635 3.3288 % of state received Total Reports child report rate per 1,000 9

by Month Initial & Additional Reports Special Condition reports Manatee (Suncoast) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Pasco (Suncoast) Jul-10 299 37 2.07% 336 4.5737 443 43 3.03% 486 4.9239 Aug-10 290 32 1.92% 322 4.4360 467 39 3.08% 506 5.1906 Sep-10 328 29 1.93% 357 5.0173 479 33 2.78% 512 5.3240 Oct-10 398 54 2.33% 452 6.0880 504 28 2.72% 532 5.6019 Nov-10 278 33 1.82% 311 4.2525 462 38 2.84% 500 5.1350 Dec-10 214 42 1.67% 256 3.2735 464 42 3.30% 506 5.1573 Jan-11 332 44 2.02% 376 5.0785 494 19 2.84% 513 5.4907 Feb-11 350 41 2.19% 391 5.3538 426 31 2.51% 457 4.7349 Mar-11 395 40 2.02% 435 6.0422 538 46 2.71% 584 5.9798 Apr-11 380 43 2.07% 423 5.8127 563 23 2.81% 586 6.2576 May-11 457 40 2.34% 497 6.9905 558 21 2.74% 579 6.2021 Jun-11 321 17 1.93% 338 4.9102 425 16 2.63% 441 4.7238 10-11 FYT Total 4042 452 2.03% 4,494 5.1524 5823 379 2.83% 6,202 5.3935 % of state received Total Reports child report rate per 1,000 Pinellas (Suncoast) Statewide by Month Initial & Additional Reports Special Condition reports % of state received Total Reports child report rate per 1,000 Jul-10 719 145 5.38% 864 4.1836 13,501 2,231 100.00% 15,732 3.2652 Aug-10 689 149 4.94% 838 4.0091 14,131 2,270 100.00% 16,401 3.4176 Sep-10 829 176 5.40% 1,005 4.8237 15,343 2,630 100.00% 17,973 3.7107 Oct-10 858 163 5.22% 1,021 4.9924 16,327 2,654 100.00% 18,981 3.9487 Nov-10 795 154 5.46% 949 4.6258 14,624 2,353 100.00% 16,977 3.5368 Dec-10 684 111 5.10% 795 3.9800 12,887 2,148 100.00% 15,035 3.1167 Jan-11 788 147 5.08% 935 4.5851 15,351 2,459 100.00% 17,810 3.7126 Feb-11 838 148 5.36% 986 4.8760 15,388 2,192 100.00% 17,580 3.7216 Mar-11 980 123 5.11% 1,103 5.7023 18,697 2,382 100.00% 21,079 4.5219 Apr-11 957 121 5.33% 1,078 5.5685 18,034 1,926 100.00% 19,960 4.3615 May-11 985 130 5.28% 1,115 5.7314 18,459 2,102 100.00% 20,561 4.4643 Jun-11 822 82 5.15% 904 4.7829 15,187 1,638 100.00% 16,825 3.6730 10-11 FYT Total 9,944 1,649 5.23% 11,593 4.8217 187,929 26,985 100.00% 214,914 3.7876 Initial & Additional Reports Special Condition reports % of state received Total Reports child report rate per 1,000 10

PERFORMANCE EVALUATION PLAN AND DESIGN PERFORMANCE EVALUATION QUESTIONS The program performance questions for this evaluation were based upon language in ss. 39.3065(3)(d), F.S. These questions are: 1. How does the quality of performance involving the Sheriffs Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriffs Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriffs Offices performing child protective investigations in a cost efficient manner? DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN For the FY2010-2011 Peer Review, representatives from DCF and Sheriff Offices agreed to apply a revised Quality Assurance (QA) review instrument. This automated instrument includes evaluation tools for the functional areas of initial response as well as emergency removal. The resulting detailed case review report for each county totals an average of 130 pages with specifics on each case and summaries of each category for the overall Sheriff s Office. Also agreed upon were sampling methodology and instrument scoring procedures. SAMPLING METHODOLOGY AND SIZE The Peer Review assesses investigative casework on a fiscal year basis. Last year s report changed the reporting period from calendar year to state fiscal year. Worked performed from July 1, 2010 through June 30, 2011 formed the basis for Investigative casework and performance outcomes data. The methodology for the quality performance component of the evaluation called for a review of closed investigation records. The definition of a closed report as used in this Peer Review is an investigation that has been completed, reviewed, signed by the investigator and supervisor, and closed in the FSFN system. As in prior reviews, reviewers considered the fact that many closed investigations were still potentially active in initial court dependency proceedings involving the investigator, with final judicial disposition outcomes not finalized. The number of investigations completed during the 12-month period July. The FSFN system selected a random sample size from the list of reports received on or after July 1, 2010, and closed on or before June 30, 2011. The number of reports sampled was determined by using the DCF Sample Size Calculator utilizing a confidence level of 90 percent with an error rate of plus or minus 10 percent. Excluded from the sample were duplicate, institutional and special condition reports. Also excluded were reports where it was determined that there was no jurisdiction to investigate and out-of-town requests. 11

Fifty percent of the sample consisted of reports that resulted in judicial action and 50 percent were non-judicial in disposition. QUALITY PERFORMANCE REVIEW PEER REVIEW TEAMS Subsection 39.3065(3)(d), F.S., requires that the program performance evaluation be conducted by a team of Peer Reviewers comprised of representatives from the Sheriff Offices with support from DCF. DCF s Quality Assurance program developed the approach to the Peer Review. Modified for this evaluation the case review instruments included a program management instrument. The Peer Review process is similar to those procedures used by national accreditation organizations. The definition of peer, as used in this performance evaluation, means Sheriff and DCF personnel who perform protective investigations and their respective quality assurance personnel, where applicable. The criteria established for Peer Reviewers included experience in child protective investigations; certification or, minimally, completion of child protective investigative training; supervisory level staff or above, or a staff member of DCF s Quality Assurance program. In FY2010, the Peer Review teams included participants from each Sheriff s Office and two representatives from DCF s local District Office. The Peer Review team did not collect or analyze cost data for the cost efficiency component of this evaluation. DCF provided cost data based upon expenditure reports provided by each Sheriff s Office for FY2010-2011. REVIEW INSTRUMENTS ABUSE REPORT RECORD REVIEW The Peer Review team conducted a review of the 65 selected files. The abuse report review instrument addresses the statutory requirements for the investigator s initial response to the report of alleged child maltreatment and the emergency removal and placement of children, if this occurred. The instrument contains a number of statements or questions that address indicators used to determine the achievement of essential steps in the investigation process. Indicators cover such areas as thoroughness of background checks, timeliness of investigations, and thoroughness of CSAs. In all, 24 indicators comprise the initial response and emergency removal and placement review form. PROGRAM MANAGEMENT When onsite, the lead representative from the Peer Review Team reviewed management core records listed in this report. Because the scores for the past several years have all been 100% in the management category for these Sheriff Offices, the Peer Review Team decided not to score this category in 2011. 12

ASSESSING PERFORMANCE This year, the rating on the file reviews used a four-step scale with four possible numerical scores. Ratings used were: not achieved, partially achieved, substantially achieved and achieved. Point values assigned as follows: 0 - Not Achieved 5 - Partially Achieved 7 - Substantially Achieved 9 - Achieved Overall performance was the sum of the indicator scores, divided by the maximum possible score, which produced a percentage. Using the performance categories, the derived percentages translate into the following performance levels: Passing 80-100 % Non-Passing 0-79 % OUTCOME MEASURES ATTAINMENT MEASURES AND STANDARDS Subsection 39.3065(3)(b), F.S., requires that the Sheriffs operate in accordance with the performance standards and outcome measures established by the Legislature for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. Sources of Data and Analysis Methods The data for all three measures come from the FSFN management report, Leader Board for Investigations. The report lists performance for each DCF Region and Sheriff s Office that operates a child protective investigation program. The report period represents state FY2010-2011. The algorithms for calculating the outcome measures are those established by DCF in consultation with the Governor s Office of Policy and Budget as well as the substantive 13

and appropriations committees of the Legislature having jurisdiction for DCF. The algorithms are as follows: The first performance measure (Investigations commenced within 24 hours): The numerator is the number of reports commenced within 24 hours of receipt of the report. The denominator is the total number of reports closed in the report period. The second performance measure (Victims seen within 24 hours of report received): The numerator is the number of victims listed in recorded reports. The denominator is the total number of victims seen within 24 hours as recorded in the FSFN computer system. This data is retrieved based on closed investigations from July 2010 through June 2011 in a DCF published monthly report known as the Leader Board. The third performance measure Child Safety Assessments (CSA) reviewed by supervisors in accordance with DCF s timeframes): The numerator is the number of initial CSAs reviewed by the supervisor within 72 hours of submission of the initial CSA for review. The denominator is the total number of reports closed in the report period. Quality Performance Presented in the table below is a summary of the performance findings. The true percentages, if all reports rather than samples had been used, can be assumed with confidence to fall somewhere within plus or minus 10 percent at the 90 percent confidence level. Listed under the following titled categories are core components of questions within the quality assurance (QA) tool: Removal 1. Reasonable Efforts 2. Psychotropic Medication 3. Placement Priority 4. Home Study Initial Response 5. Background Checks 6. Consideration of Background check information to assess risk 7. Victim Contact 8. Contact with other Children 9. Interviews with Victims 10. Interviews with other children 11. Observations of all Victims 12. Observations of all other children 13. Quality of Interactions and Observations 14. Interviews with Adult Subjects 15. Interviews with all other household members 16. Indian Child Welfare Act (ICWA) 14

17. Relevant Collateral Contacts 18. Contact with Reporter 19. Communication Between the Investigator and Case Manager 20. Child Protection Team 21. Children s Legal Services Staffing 22. Second Party Review of Safety Assessment 23. Completion of Supervisory Direction 24. Maltreatments 25. Investigative Summary 26. Safety Decision Entered into an electronic format that captures the responses of the reviewer and identifies the reviewer s assessment of each question are the results of the assessment on the above areas. This allows for immediate feedback at the end of the review via a document averaging 150 pages, showing each reviewer s ratings for each question area shown above on cases assigned to the reviewer. The electronic program allows the reviewers to complete an onsite stratification of the data at the conclusion of the review. The electronic program is capable of drilling down in the data to identify specific areas of concern by pinpointing a question and then identifying the unit, supervisor or child protective investigator responsible. This enables the program administrator to take action toward correcting any area of deficiency identified within any unit, or by supervisor or investigator. Completed at each site, exit interviews with reviewers presented trends and information on cases they reviewed for management staff and supervisors. The finalized report fully documented all information discussed at the exit conferences. The review site receives the finalized report prior to the exit conference. 15

Manatee County Sheriff s Office Manatee site visit conducted: October 10-13, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Michelle Toczylowski, Pasco Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Jane Melby, Pinellas Sheriff s Office Peggy Niermann, DCF Suncoast Region Quality Assurance (QA) Margret Gohman, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 8.50 3. Placement Priority Average agency score 7.90 4. Home Study Average agency score 8.25 Initial Response 5. Background Checks Average agency score 8.56 6. Consideration of Background check information to assess risk Average agency score 8.06 7. Victim Contact Average agency score 8.78 8. Contact with other Children Average agency score 8.08 9. Interviews with Victims Average agency score 8.38 10. Interviews with other children Average agency score 7.58 11. Observations of all Victims Average agency score 8.90 12. Observations of all other children Average agency score 8.18 13. Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score 8.48 15. Interviews with all other household members Average agency score 7.33 16. Indian Child Welfare Act (ICWA) Average agency score 7.73 17. Relevant Collateral Contacts Average agency score 8.26 18. Contact with Reporter Average agency score 7.77 19. Communication Between the Investigator & Case Manager Average agency score 7.87 20. Child Protection Team Average agency score 7.50 21. Children s Legal Services Staffing Average agency score 8.58 22. Second Party Review of Safety Assessment Average agency score 8.58 23. Completion of Supervisory Direction Average agency score 8.61 24. Maltreatments Average agency score 8.15 25. Investigative Summary Average agency score 7.64 26. Safety Decision Average agency score 8.29 Final Score: 91.89% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 16

Pasco County Sheriff s Office Pasco site visit conducted: December 5-8, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Joyce Edick, Manatee Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kimberly Williams, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 8.62 3. Placement Priority Average agency score 8.65 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 7.73 6. Consideration of Background check information to assess risk Average agency score 8.07 7. Victim Contact Average agency score 8.90 8. Contact with other Children Average agency score 8.55 9. Interviews with Victims Average agency score 8.51 10. Interviews with other children Average agency score 8.67 11. Observations of all Victims Average agency score 8.67 12. Observations of all other children Average agency score 8.52 13. Quality of Interactions and Observations Average agency score 9.00* 14. Interviews with Adult Subjects Average agency score 8.44 15. Interviews with all other household members Average agency score 8.15 16. Indian Child Welfare Act (ICWA) Average agency score 8.86 17. Relevant Collateral Contacts Average agency score 7.93 18. Contact with Reporter Average agency score 8.35 19. Communication Between the Investigator & Case Manager Average agency score 7.71 20. Child Protection Team Average agency score 8.66 21. Children s Legal Services Staffing Average agency score 8.48 22. Second Party Review of Safety Assessment Average agency score 8.60 23. Completion of Supervisory Direction Average agency score 8.23 24. Maltreatments Average agency score 8.04 25. Investigative Summary Average agency score 7.55 26. Safety Decision Average agency score 7.53 Final Score: 92.40% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 17

Broward County Sheriff s Office Broward site visit conducted: September 26 29, 2011. The reviewers were: Jay Saucer, Seminole Sheriff s Office Richard Patterson, Citrus Sheriff s Office Jane Melby, Pinellas Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Mark Holsapfel, DCF Southeast Region Quality Assurance (QA) Sharon Mitchell, DCF Southeast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 9.00 3. Placement Priority Average agency score 9.00 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.72 6. Consideration of Background check information to assess risk Average agency score 8.69 7. Victim Contact Average agency score 9.00 8. Contact with other Children Average agency score 8.61 9. Interviews with Victims Average agency score 8.11 10. Interviews with other children Average agency score 7.35 11. Observations of all Victims Average agency score 8.96 12. Observations of all other children Average agency score 8.71 13. Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score 8.60 15. Interviews with all other household members Average agency score 8.58 16. Indian Child Welfare Act (ICWA) Average agency score 9.00 17. Relevant Collateral Contacts Average agency score 8.49 18. Contact with Reporter Average agency score 8.96 19. Communication Between the Investigator & Case Manager Average agency score 9.00 20. Child Protection Team Average agency score 9.00 21. Children s Legal Services Staffing Average agency score 9.00 22. Second Party Review of Safety Assessment Average agency score 9.00 23. Completion of Supervisory Direction Average agency score 8.56 24. Maltreatments Average agency score 8.64 25. Investigative Summary Average agency score 8.34 26. Safety Decision Average agency score 8.66 Final Score: 96.79% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 18

Citrus County Sheriff s Office Citrus site visit conducted: November 7 10, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Joyce Edick, Manatee Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Jane Melby, Pinellas Sheriff s Office Yolanda Ellison, DCF Central Region Quality Assurance (QA) John Lewis, DCF Central Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 9.00 3. Placement Priority Average agency score 8.83 4. Home Study Average agency score 8.71 Initial Response 5. Background Checks Average agency score 8.51 6. Consideration of Background check information to assess risk Average agency score 8.51 7. Victim Contact Average agency score 9.00 8. Contact with other Children Average agency score 9.00 9. Interviews with Victims Average agency score 8.46 10. Interviews with other children Average agency score 9.00 11. Observations of all Victims Average agency score 8.84 12. Observations of all other children Average agency score 9.00 13. Quality of Interactions and Observations Average agency score 9.00* 14. Interviews with Adult Subjects Average agency score 8.54 15. Interviews with all other household members Average agency score 8.50 16. Indian Child Welfare Act (ICWA) Average agency score 8.86 17. Relevant Collateral Contacts Average agency score 8.36 18. Contact with Reporter Average agency score 9.00 19. Communication Between the Investigator & Case Manager Average agency score 9.00 20. Child Protection Team Average agency score 8.37 21. Children s Legal Services Staffing Average agency score 9.00 22. Second Party Review of Safety Assessment Average agency score 9.00 23. Completion of Supervisory Direction Average agency score 8.03 24. Maltreatments Average agency score 8.37 25. Investigative Summary Average agency score 7.93 26. Safety Decision Average agency score 8.40 Final Score: 95.56% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 19

Hillsborough County Sheriff s Office Hillsborough site visit conducted: November 28 December 1, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Joyce Edick, Manatee Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kimberly Williams, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 8.92 2. Psychotherapeutic Medication Average agency score 9.00 3. Placement Priority Average agency score 7.65 4. Home Study Average agency score 8.40 Initial Response 5. Background Checks Average agency score 7.76 6. Consideration of Background check information to assess risk Average agency score 7.96 7. Victim Contact Average agency score 8.73 8. Contact with other Children Average agency score 8.22 9. Interviews with Victims Average agency score 8.54 10. Interviews with other children Average agency score 8.33 11. Observations of all Victims Average agency score 8.93 12. Observations of all other children Average agency score 8.77 13. Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score 8.38 15. Interviews with all other household members Average agency score 7.23 16. Indian Child Welfare Act (ICWA) Average agency score 9.00 17. Relevant Collateral Contacts Average agency score 7.80 18. Contact with Reporter Average agency score 8.75 19. Communication Between the Investigator & Case Manager Average agency score 7.87 20. Child Protection Team Average agency score 6.33 21. Children s Legal Services Staffing Average agency score 9.00 22. Second Party Review of Safety Assessment Average agency score 7.90 23. Completion of Supervisory Direction Average agency score 8.15 24. Maltreatments Average agency score 7.98 25. Investigative Summary Average agency score 8.03 26. Safety Decision Average agency score 8.29 Final Score: 91.99% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 20

Pinellas County Sheriff s Office Pinellas site visit conducted: October 24 27, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Peggy Niermann, DCF Suncoast Region Quality Assurance (QA) Margaret Gohman, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 8.25 3. Placement Priority Average agency score 9.00 4. Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score 8.44 6. Consideration of Background check information to assess risk Average agency score 7.64 7. Victim Contact Average agency score 8.87 8. Contact with other Children Average agency score 8.50 9. Interviews with Victims Average agency score 8.91 10. Interviews with other children Average agency score 8.40 11. Observations of all Victims Average agency score 8.65 12. Observations of all other children Average agency score 8.00 13. Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score 8.75 15. Interviews with all other household members Average agency score 8.89 16. Indian Child Welfare Act (ICWA) Average agency score 8.43 17. Relevant Collateral Contacts Average agency score 8.18 18. Contact with Reporter Average agency score 8.75 19. Communication Between the Investigator & Case Manager Average agency score 9.00 20. Child Protection Team Average agency score 9.00 21. Children s Legal Services Staffing Average agency score 8.76 22. Second Party Review of Safety Assessment Average agency score 8.56 23. Completion of Supervisory Direction Average agency score 8.37 24. Maltreatments Average agency score 8.50 25. Investigative Summary Average agency score 7.21 26. Safety Decision Average agency score 8.24 Final Score: 93.78% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 21

Seminole County Sheriff s Office Seminole site visit conducted: September 12 15, 2011. The reviewers were: Joseph Paduano, Broward Sheriff s Office Joyce Edick, Manatee Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Jane Melby, Pinellas Sheriff s Office Teresa Vella, DCF Central Region Quality Assurance Kristen Pucket, DCF Central Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score 9.00 2. Psychotherapeutic Medication Average agency score 9.00 3. Placement Priority Average agency score 8.79 4. Home Study Average agency score 7.92 Initial Response 5. Background Checks Average agency score 8.50 6. Consideration of Background check information to assess risk Average agency score 8.47 7. Victim Contact Average agency score 9.00 8. Contact with other Children Average agency score 8.56 9. Interviews with Victims Average agency score 8.52 10. Interviews with other children Average agency score 8.60 11. Observations of all Victims Average agency score 8.93 12. Observations of all other children Average agency score 9.00 13. Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score 8.59 15. Interviews with all other household members Average agency score 8.62 16. Indian Child Welfare Act (ICWA) Average agency score 8.86 17. Relevant Collateral Contacts Average agency score 8.38 18. Contact with Reporter Average agency score 8.90 19. Communication Between the Investigator & Case Manager Average agency score 9.00 20. Child Protection Team Average agency score 8.25 21. Children s Legal Services Staffing Average agency score 8.43 22. Second Party Review of Safety Assessment Average agency score 8.42 23. Completion of Supervisory Direction Average agency score 8.65 24. Maltreatments Average agency score 8.56 25. Investigative Summary Average agency score 8.35 26. Safety Decision Average agency score 8.53 Final Score: 95.84% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 22