ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT

Transcription

1 Florida Sheriffs Performing Child Protective Investigations ANNUAL PROGRAM PERFORMANCE EVALUATION REPORT Fiscal Year Conducted jointly by DCF and the Sheriff Offices of Broward, Citrus, Hillsborough, Manatee, Pasco, Pinellas and Seminole Counties

2 TABLE OF CONTENTS EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 EVALUATION PLAN AND DESIGN... 4 QUALITY PERFORMANCE REVIEW... 5 OUTCOME MEASURES AND STANDARDS... 6 CONCLUSIONS... 6 QUALITY PERFORMANCE REVIEW... 6 OUTCOME MEASURES... 7 RECOMMENDATIONS... 7 INTRODUCTION... 8 PROGRAM PERFORMANCE EVALUATION... 8 SHERIFF S INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS IN FLORIDA... 8 ANNUALIZE DATA ON ABUSE REPORTS.. 9 PERFORMANCE EVALUATION PLAN AND DESIGN PERFORMANCE EVALUATION QUESTIONS DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN SAMPLING METHODOLOGY AND SIZE QUALITY PERFORMANCE REVIEW PEER REVIEW TEAMS REVIEW INSTRUMENTS PROGRAM MANAGEMENT ASSESSING PERFORMANCE OUTCOME MEASURES ATTAINMENT MEASURES AND STANDARDS SOURCES OF DATA AND ANALYSIS METHODS QUALITY PERFORMANCE (CASE REVIEWS) MANATEE COUNTY PASCO COUNTY BROWARD COUNTY CITRUS COUNTY HILLSBOROUGH COUNTY PINELLAS COUNTY SEMINOLE COUNTY OUTCOME MEASURES ATTAINMENT ANNUAL OUTCOMES FOR COMMENCEMENTS OF REPORTS WITHIN 24 HOURS ANNUAL OUTCOMES FOR VICTIMS SEEN WITHIN 24 HOURS OF CASE RECEIVED ANNUAL OUTCOMES FOR INITIAL SUPERVISORY REVIEWS WITHIN 72 HOURS ADDITIONAL OUTCOME PERFORMANCE PROGRAM MANAGEMENT COST EFFICIENCY RECOMMENDATIONS

3 EXECUTIVE SUMMARY INTRODUCTION The 12 th annual report regarding the performance of those Sheriff Offices formally charged with protecting children within their communities is submitted at a time when Department of Children and Families (DCF) leadership has actively renewed efforts to examine the design, methods, processes and purposes of the Florida child protection system. In this fiscal year, the intake reports received statewide exceeded any prior year in the history of Florida. Sheriff Offices handled more than 214,000 new intakes requiring a commencement action, which is 12% above the prior fiscal year. Monthly caseload increases of 12% significantly challenged all operations for both DCF and Sheriff Offices. The depressed economic conditions within the state added additional pressures to many families affected by DCF interventions. Despite these challenges statewide, the outcomes for DCF and the Sheriff Offices continue to remain first-rate. DCF prioritized in 2011 a statewide initiative to implement changes in program practice areas affecting hotline and child protective investigations. Representatives statewide from DCF and Sheriff Offices child protective investigation programs have been involved with DCF program officials in examining business processes, purposes, designs and methods. As a result, in 2012 recommendations from many related workgroups will be used both legislatively and administratively to modify and enhance roles, technologies, policy and various practices to enhance the entire framework of Florida s child protection responses. By design, these changes will enhance selection and hiring practices, working conditions, opportunities, and compensation for career child protection professionals employed by both Sheriffs and DCF. Additionally, many enhancements will affect the entire system. Sheriff Offices handled 27% of the state s investigations and are an essential element in moving forward with shared values, principles and efforts that DCF seeks for enhancing Florida s child protection system. The longstanding commitment of the Sheriff Offices involved in child protective investigations toward engaging families, protecting children, and working in partnership within their communities remains consistent and strong. Several of these protocols received local, state and national recognition and respect for the best practice, community-based collaborations for their investigative responses. In the mid-1990s, a Sheriff Office pilot program in Manatee County led to legislation in 1998 to expand the model. The expansion in 1999 added Sheriff Offices in Pasco, Pinellas and Broward counties. Legislation transferred full responsibility for child protective investigations to these Sheriffs in FY1999. In 2000, the Seminole County Sheriff assumed the role, followed by Hillsborough County in Citrus County Sheriff assumed the role in These seven Sheriff Offices encompass metropolitan areas where more than a quarter of the state s population resides. The Sheriffs 3

4 integration into this child protection role is now virtually seamless and significant in each of their communities. Annual oversight of the Sheriff Offices initially defined under Chapter , Laws of Florida, required a committee of seven persons appointed by the Governor to address Sheriffs program performance. That committee met with the respective Sheriffs and developed criteria mutually agreed upon for an annual review. The committee held the responsibility for submitting an annual report regarding quality performance, outcome measure attainment and cost efficiency. In 2000, Chapter , Laws of Florida, the committee ended and an annual report on program performance by the Sheriff Offices became mandated. The mandated annual review shall use criteria mutually agreed upon by the Sheriffs and DCF. This report completed by a team of Peer Reviewers from the Sheriff Offices with support from DCF, addresses quality performance, outcome measure attainment, and cost efficiency. This is the 12 th Annual Sheriff Offices Peer Review Report. EVALUATION PLAN AND DESIGN The program performance evaluation questions are based upon language in subsection (3)(d), F.S. In summary, these questions are: 1. How does the quality of performance involving the Sheriff Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriff Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriff Offices performing child protective investigations in a cost efficient manner? Representatives from the seven Sheriff Offices with support from DCF comprised the program evaluation planning team. The Sheriff Representatives were: Broward County Sheriff Capt. Andrew Koerick, Program Administrator Joseph Paduano, Child Protective Investigation (CPI) Supervisor Seminole County Sheriff Capt. Greg Barnett, Captain Jay Saucer, Quality Assurance Pinellas County Sheriff Brandi Lazaris, Program Administrator Jane Melvy, Supervisor Citrus County Sheriff Lt. Dave Wyllie, Lieutenant Richard Patterson, CPI Supervisor 4

5 Hillsborough County Sheriff Major Robert Bullara, Division Commander Jennifer Hock, Program Administrator Pasco County Sheriff Ken Lilian, Director Lisa Tobin, Training Supervisor Manatee County Sheriff Joyce Edick, Operations Program Specialist. In 2011, the planning committee adopted several changes to the Peer Review Tool, including two additional questions. The additional questions related to (1) consideration of the background check information in the assessment of risk and (2) the quality of interactions and observations of the victims and other children. Additionally, separating content information in three existing questions, allowed the review of additional details. The question, Interview of victim and children became two questions: Interview of victims and Interview of other children in home. The question, Observations of all victims and children, became Observations of victims and Observations of all other children. The question, Interviews of adult subjects and other household members, became Interview of adult subjects (Alleged persons responsible, parents, caregivers) and Interviews with all other adult household members. DCF Office of Family Safety accepted the changes related to these enhancement question revisions to the 2011 Sheriff s Peer Review Tool. Each Sheriff s Office scheduled an onsite review. Reviewers from each Sheriff s Office and DCF were present for onsite case file reviews and conducted entry and exit presentations. QUALITY PERFORMANCE REVIEW The 2011 Sheriff s Peer Review process included three and a half days of onsite visits for each Sheriff s location. The plan developed collaboratively included case file reviews and reviews of performance measures as core components of quality performance. The 2011 process fully addressed the quality of practice standards utilizing the revised standardized review tool. The number of files reviewed at each site was determined using a statistical 90% confidence level, with a +/-10% confidence interval. For all review sites, the sample size was 65 files. Excluded from the sample were duplicate reports, institutional reports, foster care referrals, special condition reports, out of town inquiries (OTIs), and no-jurisdiction reports. The sample pulled comprised 50 percent reports with a judicial action and 50 percent non-judicial in disposition. 5

6 The overall score for each Sheriff s Office includes only the results of the internal case file review and the side-by-side review, and was calculated using the Sheriff s Peer Review Access database with each file receiving equal weight in scoring. OUTCOME MEASURES AND STANDARDS Subsection (3)(b), F.S., requires that the Sheriffs performing child protective investigations operate, at a minimum, in accordance with the performance standards and outcome measures established for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. These measures amended the Grant Agreements beginning July For FY the report eliminated the 60-day case closure measure and replaced it with a performance measure tracking the timeliness of victims seen within 24 hours of a report received by the Florida Abuse Hotline. Users enter the data for these performance measures and others into the child welfare information system, Florida Safe Families Network (FSFN). This system produces management reports used for determining statewide performance and outcomes. CONCLUSIONS QUALITY PERFORMANCE REVIEW Sheriff s Office involvement in child protection investigations began in the mid-1990s when calls for reform led to successful pilots that spurred legislative commitment to implement statutory changes. During this time, the Legislature also passed new statutes requiring the outsourcing of foster care and related services statewide. It was the Legislature s intent to encourage communities and other stakeholders interested in the well-being of children to participate in assuring that children were safe and well nurtured in their local community. DCF moved aggressively and successfully outsourced the state s foster care and related services to community-based care lead agencies. Including contracting with Sheriffs Offices, Florida has used the communitybased care philosophy, to safely reduce the number of children in foster care and embrace family-centered practices when working with families. 6

7 Recommendations from the Subcommittee for Children and Families to Governor-Elect Scott s Health and Human Services Transition Team, dated December 20, 2010, noted in part that DCF under the leadership of former Secretary Bob Butterworth and Secretary George Sheldon was, widely recognized by many Floridians as the best run Department in the state. It further concluded, This leadership has been open to finding innovative solutions such as privatization, using technology, and leveraging existing resources in local communities across the state. Successful leveraging of community law-enforcement personnel and their existing resources for conducting child protective investigations continues to result in quality outcomes for protecting children and supporting families. OUTCOME MEASURES The performance outcomes reported in this review confirm Sheriff Offices continue to achieve the performance outcome measures established by the Legislature. Broward, Citrus, Manatee, Pasco, Pinellas, Hillsborough and Seminole County Sheriff Offices all achieved passing ratings in the record reviews in this year s Peer Review. RECOMMENDATIONS 1. The Department of Children and Families should continue its grants (contract for Citrus) to the Sheriff Offices in Manatee, Pasco, Seminole, Broward, Hillsborough, Pinellas, and Citrus counties to conduct child protective investigations. 2. The commendable, longstanding and successful roles that Sheriff Offices have had in Florida for providing child protective investigative services should warrant DCF and the legislature to encourage more sheriff expansions along with corresponding funding for all of their successful operations. 3. The Department of Children and Families should continue to report on the participating Sheriff Offices with regard to achieving the performance measures and targets. 4. The Department of Children and Families should continue involving representatives from the Sheriff Offices in identifying appropriate practice standards and performance measures for child protective investigations. 5. The Department of Children and Families, and the Sheriff Offices should use the results of this year s quality performance review to identify and implement improvements for pre-service and in-service training of investigators and supervisors. 7

8 INTRODUCTION PROGRAM PERFORMANCE EVALUATION Annual oversight of the Sheriff Offices initially defined under Chapter , Laws of Florida, required a committee of seven persons appointed by the Governor to address Sheriffs program performance. That committee met with the respective Sheriffs and developed criteria mutually agreed upon for an annual review. The committee held the responsibility for submitting an annual report regarding quality performance, outcome measure attainment and cost efficiency. In 2000, Chapter , Laws of Florida, the committee ended and an annual report on program performance by the Sheriff Offices became mandated. The mandated annual review shall use criteria mutually agreed upon by the Sheriffs and DCF. This report completed by a team of Peer Reviewers from the Sheriff Offices with support from DCF, addresses quality performance, outcome measure attainment, and cost efficiency. This is the 12th Annual Sheriff Offices Peer Review Report. SHERIFFS INVOLVEMENT IN CHILD PROTECTIVE INVESTIGATIONS According to Florida Safe Families Network (FSFN) data, some population shifts occurred within Florida over the past fiscal year. Overall, 27.90% of the state s children reside within a county where the Sheriff has jurisdiction for child protective investigations. However, the Sheriffs handled 26.99% of all initial, additional and special condition intakes received in the state for FY Child Population Total Population % state's child's population % state's population total 10/11: All Initial, Additional, Special Condition Reports Reports: % state's total Broward 407,694 1,751, % 9.33% 15, % Citrus 21, , % 0.76% 1, % Hillsborough 298,584 1,203, % 6.41% 13, % Manatee 65, , % 1.70% 4, % Pasco 89, , % 2.35% 6, % Pinellas 171, , % 4.94% 11, % Seminole 98, , % 2.24% 4, % Sheriff totals 1,153,513 5,204, % 27.73% 57, % DCF totals 2,981,286 13,567, % 72.27% 156, % State total 4,134,799 18,772, ,914 Sources: Florida State Office of Economic & Demographic Research / [ 8

9 Annualized Data on Abuse Reports: The following data reflects the number of initial, additional, and special condition intake reports accepted by DCF and handled monthly with the seven Sheriff Offices for FY This data lists the percentage of reports received, as well as the alleged reporting rate by victim per 1,000 children. by Month Initial & Additional Reports Special Condition reports Seminole Broward (District 7) (District 10) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Jul % % 1, Aug % % 1, Sep % , % 1, Oct % , % 1, Nov % % 1, Dec % % 1, Jan % , % 1, Feb % , % 1, Mar % , % 1, Apr % , % 1, May % , % 1, Jun % , % 1, FYT Total 4, % 4, ,435 2, % 15, % of state received Total Reports child report rate per 1,000 by Month Initial & Additional Reports Special Condition reports Citrus (District 13) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Hillsborough (Suncoast) Jul % % Aug % % 1, Sep % % 1, Oct % , % 1, Nov % % 1, Dec % % Jan % , % 1, Feb % , % 1, Mar % , % 1, Apr % , % 1, May % , % 1, Jun % % 1, FYT Total 1, % 1, ,927 1, % 13, % of state received Total Reports child report rate per 1,000 9

10 by Month Initial & Additional Reports Special Condition reports Manatee (Suncoast) % of state received Total Reports child report rate per 1,000 Initial & Additional Reports Special Condition reports Pasco (Suncoast) Jul % % Aug % % Sep % % Oct % % Nov % % Dec % % Jan % % Feb % % Mar % % Apr % % May % % Jun % % FYT Total % 4, % 6, % of state received Total Reports child report rate per 1,000 Pinellas (Suncoast) Statewide by Month Initial & Additional Reports Special Condition reports % of state received Total Reports child report rate per 1,000 Jul % ,501 2, % 15, Aug % ,131 2, % 16, Sep % 1, ,343 2, % 17, Oct % 1, ,327 2, % 18, Nov % ,624 2, % 16, Dec % ,887 2, % 15, Jan % ,351 2, % 17, Feb % ,388 2, % 17, Mar % 1, ,697 2, % 21, Apr % 1, ,034 1, % 19, May % 1, ,459 2, % 20, Jun % ,187 1, % 16, FYT Total 9,944 1, % 11, ,929 26, % 214, Initial & Additional Reports Special Condition reports % of state received Total Reports child report rate per 1,000 10

11 PERFORMANCE EVALUATION PLAN AND DESIGN PERFORMANCE EVALUATION QUESTIONS The program performance questions for this evaluation were based upon language in ss (3)(d), F.S. These questions are: 1. How does the quality of performance involving the Sheriffs Offices conducting child protective investigations comply with the requirements of Chapter 39, F.S.? 2. Have the participating Sheriffs Offices achieved the performance standards and outcome measures specified in their grant agreements? 3. Are the participating Sheriffs Offices performing child protective investigations in a cost efficient manner? DEVELOPMENT OF THE EVALUATION DESIGN AND PLAN For the FY Peer Review, representatives from DCF and Sheriff Offices agreed to apply a revised Quality Assurance (QA) review instrument. This automated instrument includes evaluation tools for the functional areas of initial response as well as emergency removal. The resulting detailed case review report for each county totals an average of 130 pages with specifics on each case and summaries of each category for the overall Sheriff s Office. Also agreed upon were sampling methodology and instrument scoring procedures. SAMPLING METHODOLOGY AND SIZE The Peer Review assesses investigative casework on a fiscal year basis. Last year s report changed the reporting period from calendar year to state fiscal year. Worked performed from July 1, 2010 through June 30, 2011 formed the basis for Investigative casework and performance outcomes data. The methodology for the quality performance component of the evaluation called for a review of closed investigation records. The definition of a closed report as used in this Peer Review is an investigation that has been completed, reviewed, signed by the investigator and supervisor, and closed in the FSFN system. As in prior reviews, reviewers considered the fact that many closed investigations were still potentially active in initial court dependency proceedings involving the investigator, with final judicial disposition outcomes not finalized. The number of investigations completed during the 12-month period July. The FSFN system selected a random sample size from the list of reports received on or after July 1, 2010, and closed on or before June 30, The number of reports sampled was determined by using the DCF Sample Size Calculator utilizing a confidence level of 90 percent with an error rate of plus or minus 10 percent. Excluded from the sample were duplicate, institutional and special condition reports. Also excluded were reports where it was determined that there was no jurisdiction to investigate and out-of-town requests. 11

12 Fifty percent of the sample consisted of reports that resulted in judicial action and 50 percent were non-judicial in disposition. QUALITY PERFORMANCE REVIEW PEER REVIEW TEAMS Subsection (3)(d), F.S., requires that the program performance evaluation be conducted by a team of Peer Reviewers comprised of representatives from the Sheriff Offices with support from DCF. DCF s Quality Assurance program developed the approach to the Peer Review. Modified for this evaluation the case review instruments included a program management instrument. The Peer Review process is similar to those procedures used by national accreditation organizations. The definition of peer, as used in this performance evaluation, means Sheriff and DCF personnel who perform protective investigations and their respective quality assurance personnel, where applicable. The criteria established for Peer Reviewers included experience in child protective investigations; certification or, minimally, completion of child protective investigative training; supervisory level staff or above, or a staff member of DCF s Quality Assurance program. In FY2010, the Peer Review teams included participants from each Sheriff s Office and two representatives from DCF s local District Office. The Peer Review team did not collect or analyze cost data for the cost efficiency component of this evaluation. DCF provided cost data based upon expenditure reports provided by each Sheriff s Office for FY REVIEW INSTRUMENTS ABUSE REPORT RECORD REVIEW The Peer Review team conducted a review of the 65 selected files. The abuse report review instrument addresses the statutory requirements for the investigator s initial response to the report of alleged child maltreatment and the emergency removal and placement of children, if this occurred. The instrument contains a number of statements or questions that address indicators used to determine the achievement of essential steps in the investigation process. Indicators cover such areas as thoroughness of background checks, timeliness of investigations, and thoroughness of CSAs. In all, 24 indicators comprise the initial response and emergency removal and placement review form. PROGRAM MANAGEMENT When onsite, the lead representative from the Peer Review Team reviewed management core records listed in this report. Because the scores for the past several years have all been 100% in the management category for these Sheriff Offices, the Peer Review Team decided not to score this category in

13 ASSESSING PERFORMANCE This year, the rating on the file reviews used a four-step scale with four possible numerical scores. Ratings used were: not achieved, partially achieved, substantially achieved and achieved. Point values assigned as follows: 0 - Not Achieved 5 - Partially Achieved 7 - Substantially Achieved 9 - Achieved Overall performance was the sum of the indicator scores, divided by the maximum possible score, which produced a percentage. Using the performance categories, the derived percentages translate into the following performance levels: Passing % Non-Passing 0-79 % OUTCOME MEASURES ATTAINMENT MEASURES AND STANDARDS Subsection (3)(b), F.S., requires that the Sheriffs operate in accordance with the performance standards and outcome measures established by the Legislature for protective investigations conducted by DCF. The General Appropriations Act sets forth appropriations allocated through multi-year Grant Agreements with the seven Sheriff Offices performing child protective investigations. The Grant Agreements cite three performance measures for the Sheriffs and DCF districts/region: 1. One hundred percent (100%) of investigations commenced within 24 hours, 2. Eighty-five percent (85%) of victims seen within 24 hours of a report received, and 3. One hundred percent (100%) of Child Safety Assessment (CSA) reports reviewed by supervisors are in accordance with DCF s timeframes. Sources of Data and Analysis Methods The data for all three measures come from the FSFN management report, Leader Board for Investigations. The report lists performance for each DCF Region and Sheriff s Office that operates a child protective investigation program. The report period represents state FY The algorithms for calculating the outcome measures are those established by DCF in consultation with the Governor s Office of Policy and Budget as well as the substantive 13

14 and appropriations committees of the Legislature having jurisdiction for DCF. The algorithms are as follows: The first performance measure (Investigations commenced within 24 hours): The numerator is the number of reports commenced within 24 hours of receipt of the report. The denominator is the total number of reports closed in the report period. The second performance measure (Victims seen within 24 hours of report received): The numerator is the number of victims listed in recorded reports. The denominator is the total number of victims seen within 24 hours as recorded in the FSFN computer system. This data is retrieved based on closed investigations from July 2010 through June 2011 in a DCF published monthly report known as the Leader Board. The third performance measure Child Safety Assessments (CSA) reviewed by supervisors in accordance with DCF s timeframes): The numerator is the number of initial CSAs reviewed by the supervisor within 72 hours of submission of the initial CSA for review. The denominator is the total number of reports closed in the report period. Quality Performance Presented in the table below is a summary of the performance findings. The true percentages, if all reports rather than samples had been used, can be assumed with confidence to fall somewhere within plus or minus 10 percent at the 90 percent confidence level. Listed under the following titled categories are core components of questions within the quality assurance (QA) tool: Removal 1. Reasonable Efforts 2. Psychotropic Medication 3. Placement Priority 4. Home Study Initial Response 5. Background Checks 6. Consideration of Background check information to assess risk 7. Victim Contact 8. Contact with other Children 9. Interviews with Victims 10. Interviews with other children 11. Observations of all Victims 12. Observations of all other children 13. Quality of Interactions and Observations 14. Interviews with Adult Subjects 15. Interviews with all other household members 16. Indian Child Welfare Act (ICWA) 14

15 17. Relevant Collateral Contacts 18. Contact with Reporter 19. Communication Between the Investigator and Case Manager 20. Child Protection Team 21. Children s Legal Services Staffing 22. Second Party Review of Safety Assessment 23. Completion of Supervisory Direction 24. Maltreatments 25. Investigative Summary 26. Safety Decision Entered into an electronic format that captures the responses of the reviewer and identifies the reviewer s assessment of each question are the results of the assessment on the above areas. This allows for immediate feedback at the end of the review via a document averaging 150 pages, showing each reviewer s ratings for each question area shown above on cases assigned to the reviewer. The electronic program allows the reviewers to complete an onsite stratification of the data at the conclusion of the review. The electronic program is capable of drilling down in the data to identify specific areas of concern by pinpointing a question and then identifying the unit, supervisor or child protective investigator responsible. This enables the program administrator to take action toward correcting any area of deficiency identified within any unit, or by supervisor or investigator. Completed at each site, exit interviews with reviewers presented trends and information on cases they reviewed for management staff and supervisors. The finalized report fully documented all information discussed at the exit conferences. The review site receives the finalized report prior to the exit conference. 15

16 Manatee County Sheriff s Office Manatee site visit conducted: October 10-13, The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Michelle Toczylowski, Pasco Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Jane Melby, Pinellas Sheriff s Office Peggy Niermann, DCF Suncoast Region Quality Assurance (QA) Margret Gohman, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 8.25 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.29 Final Score: 91.89% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 16

17 Pasco County Sheriff s Office Pasco site visit conducted: December 5-8, The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Joyce Edick, Manatee Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kimberly Williams, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score 9.00* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 7.53 Final Score: 92.40% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 17

18 Broward County Sheriff s Office Broward site visit conducted: September 26 29, The reviewers were: Jay Saucer, Seminole Sheriff s Office Richard Patterson, Citrus Sheriff s Office Jane Melby, Pinellas Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Mark Holsapfel, DCF Southeast Region Quality Assurance (QA) Sharon Mitchell, DCF Southeast Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.66 Final Score: 96.79% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 18

19 Citrus County Sheriff s Office Citrus site visit conducted: November 7 10, The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Joyce Edick, Manatee Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Jane Melby, Pinellas Sheriff s Office Yolanda Ellison, DCF Central Region Quality Assurance (QA) John Lewis, DCF Central Region QA. The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 8.71 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score 9.00* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.40 Final Score: 95.56% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 19

20 Hillsborough County Sheriff s Office Hillsborough site visit conducted: November 28 December 1, The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Joyce Edick, Manatee Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Jane Melby, Pinellas Sheriff s Office Kimberly Williams, DCF Suncoast Region Quality Assurance (QA) Lisa Rivera, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 8.40 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.29 Final Score: 91.99% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 20

21 Pinellas County Sheriff s Office Pinellas site visit conducted: October 24 27, The reviewers were: Joseph Paduano, Broward Sheriff s Office Jay Saucer, Seminole Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Joyce Edick, Manatee Sheriff s Office Peggy Niermann, DCF Suncoast Region Quality Assurance (QA) Margaret Gohman, DCF Suncoast Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 9.00 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.24 Final Score: 93.78% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 21

22 Seminole County Sheriff s Office Seminole site visit conducted: September 12 15, The reviewers were: Joseph Paduano, Broward Sheriff s Office Joyce Edick, Manatee Sheriff s Office Cindy Harrell, Citrus Sheriff s Office Kathleen Mathews, Hillsborough Sheriff s Office Lisa Tobin, Pasco Sheriff s Office Jane Melby, Pinellas Sheriff s Office Teresa Vella, DCF Central Region Quality Assurance Kristen Pucket, DCF Central Region QA The following titled categories contain core components of questions within the QA tool. Listed to the right is the overall average score. Removal 1. Reasonable Efforts Average agency score Psychotherapeutic Medication Average agency score Placement Priority Average agency score Home Study Average agency score 7.92 Initial Response 5. Background Checks Average agency score Consideration of Background check information to assess risk Average agency score Victim Contact Average agency score Contact with other Children Average agency score Interviews with Victims Average agency score Interviews with other children Average agency score Observations of all Victims Average agency score Observations of all other children Average agency score Quality of Interactions and Observations Average agency score N/A* 14. Interviews with Adult Subjects Average agency score Interviews with all other household members Average agency score Indian Child Welfare Act (ICWA) Average agency score Relevant Collateral Contacts Average agency score Contact with Reporter Average agency score Communication Between the Investigator & Case Manager Average agency score Child Protection Team Average agency score Children s Legal Services Staffing Average agency score Second Party Review of Safety Assessment Average agency score Completion of Supervisory Direction Average agency score Maltreatments Average agency score Investigative Summary Average agency score Safety Decision Average agency score 8.53 Final Score: 95.84% * In addition to the agreed upon peer review questions, and the two added questions another question was added (#13) as a pilot question to test the potential of this question. It was not an agreed upon question. That is why all the results are N/A. The reason that Pasco (page 18) and Citrus (page 20) had a score was due to a data error when one of the reviewers did not click NA and entered a score. The instructions were to score and comment only in a comment box. Because the reviewers used an electronic data entry system there was no way to edit a reviewer s earlier entry at the time the computer database produced the final scores. 22