2017 Self-Assessment Report

Transcription

1 2017 Self-Assessment Report Orange County Head Start Key insights from the annual self-assessment which is conducted to evaluate the program s progress toward meeting goals, compliance with regulatory requirements, and the effectiveness of professional development and family engagement systems in promoting school readiness.

2 Contents I. Introduction... 2 Program Description... 2 Context for Self-Assessment... 2 II. Methodology... 3 Pre-Phase - Review of Program Data... 3 Phase One- Design the Process... 5 Phase Two- Engage the Team... 5 Phase Three- Analyze & Dialogue... 5 Phase Four- Recommendations... 6 Phase Five- Prepare the Report... 6 Post Phase- Planning... 7 III. Key Insights... 7 Program Strengths & Successes... 8 Program Improvements to Service Delivery and Systems... 8 Service Delivery... 8 Management Systems... 9 Progress on Program Goals VI. Results of the Self-Assessment School Readiness Health-Dental Treatments Safety Family Engagement Professional Development V. Conclusion

3 I. Introduction Program Description Orange County Head Start, OCHS, a division of the Family Services Department within Orange County Government, is a long standing program, operating since Head Start s inception in The program is center based and geographically dispersed throughout Orange County in twenty-two facilities, of which includes county owned facilities, Orange County Public School campuses, city municipalities, and faith based organizations. The program is funded to serve 1,536 children, of which 756 receive a traditional 6 hour day of instruction and 780 receive an extended 10 hour day of instruction. A formal partnership agreement with the Early Learning Coalition of Orange County enables the program to also provide Voluntary Pre-Kindergarten (VPK) services. Context for Self-Assessment 1. Orange County Head Start conducts a self-assessment once annually in accordance with Head Start Performance Standard (b)(2). The Self-Assessment occurs approximately ¾ of the way through the program term in order to ensure timely completion of the Self-Assessment and development of the Program Improvement Plan which is submitted with the Continuation Grant for each year of the 5 year grant period. The process relies solely on program data to complete an evaluation of the program s progress toward meeting goals, compliance with regulatory requirements, and the effectiveness of professional development and family engagement systems in promoting school readiness. As a result, a full set of program year data is not available for the current program term; however, program data from the previous year is used in conjunction with current program year data in order to ensure a full set of data is reviewed. 2. The Self-Assessment Planning team, comprised of the Quality Assurance Unit, organized and facilitated the 2017 Self-Assessment. The 2017 Self-Assessment Plan was approved by the Policy Council Executive Board on February 10, 2017 and the Board of County Commissioners, BCC, on March 7, The Self-Assessment team 2

4 was comprised of management, staff, Policy Council representatives, parents, and external community members with demonstrated abilities in data analysis and collaborative inquiry. 3. The topics selected for the 2017 Self-Assessment were in alignment with the requirements of Head Start Performance Standard (b)(2) and best practices provided by the National Center on Program Management and Fiscal Operations for areas to focus on that will ensure an effective Self-Assessment process. 4. Summaries of Ongoing Monitoring results from each service area, progress on the program s five year goals and objectives, child assessment data, family engagement data, and professional development data were all used as data sources to identify program strengths, successes, and areas of improvement within the pre-determined focus areas. II. Methodology The Self-Assessment took place in five phases, bookmarked by a pre and post phase. 1. Pre-Phase Review of Program Data 2. Phase 1-Design the Process 3. Phase 2-Engage the Team 4. Phase 3-Analyze & Dialogue 5. Phase 4-Recommendations 6. Phase 5-Prepare the Report 7. Post-Phase-Program Planning Pre-Phase - Review of Program Data Date Action Purpose 4/5 Self-Assessment Planning Review program data and determine Meeting focus areas for Self-Assessment The pre-phase consisted of the review of monitoring results, progress on program goals and objectives, and other program data to identify focus areas for the Self-Assessment team to 3

5 analyze, dialogue, and provide recommendations for program improvement. During the prephase, the management team in conjunction with their service area staff reviewed ongoing monitoring compliance statistics reports to identify areas of strength and concern and had the opportunity to make recommendations for topics for Self-Assessment. The Summary of Ongoing Monitoring Results was submitted to the Self-Assessment Planning team for review and final determination of focus areas. Based on the review of Ongoing Monitoring Results, Program Information Reports, and other relevant program data, the following five focus areas were brought forth for consideration by the Self-Assessment team: Focus Area Concern/Consideration I. School Readiness Evaluate the program s performance in meeting progress toward program and school readiness goals, compliance with Head Start Program Performance Standards, and engaging families in an effort to promote school readiness. II. Health-Dental Treatments Ongoing Monitoring results indicate a high noncompliance in the number of children receiving dental treatment, as needed. Evaluate program data to determine the strengths, successes, and areas of improvement as it relates to meeting children s dental treatment needs. III. Safety-Accident & Incident Reporting The reporting of accidents/incidents is currently being documented by the Education Service Area on a monthly basis. Determine whether additional reporting methods are needed and if the information documented is sufficient to track accidents/incidents that occur within the centers. IV. Family Engagement-Follow Up Determine the program s strengths, successes, and areas for improvement to effectively impact family well being, establish a positive goal-oriented relationship with families, and ensure overall family engagement promotes school readiness. V. Professional Development Evaluate the program s performance in implementing a professional development system that supports ongoing professional development of staff within their roles, 4

6 ensures all staff receive required trainings, and demonstrates progress toward meeting program goals in relation to professional development. Phase One- Design the Process Date Action Purpose 2/10 Presentation of SA Plan to Policy Council Executive Board 3/7 Submission of SA Plan to BCC Review and approve 2017 Self-Assessment Plan and invite representatives to participate on the Self- Assessment team Review and approve 2017 Self-Assessment Plan During phase one, the Policy Council and the BCC approved the Self-Assessment plan. Individuals, both internal and external, were selected to participate on the Self-Assessment team based on their area of expertise and ability to analyze data. The team s composition included key management and service area staff, Policy Council representatives, as well as, community partners. Phase Two- Engage the Team Date Action Purpose 4/21 Self-Assessment Team Orientation Orient team members to the Self-Assessment process and focus areas During phase two, the Self-Assessment team participants were brought together for a half day of training, which oriented them to the program s structure, services provided, the Self- Assessment process and the focus areas to be reviewed. The training was facilitated by the Quality Assurance Unit. Phase Three- Analyze & Dialogue Date Action Purpose 5

7 4/21 Sub-group Work Session Review, discuss and analyze program data For phase three, the Self-Assessment team was divided into sub-groups and assigned a focus area topic. Each member received a packet of information including: a Summary Sheet, pertinent Head Start Program Performance Standards, Orange County Head Start Standard Operating Procedures, and program data pertinent to their topic. After spending time to review the contents of the packets, sub-groups spent the afternoon analyzing the data in order to generate discoveries. Phase Four- Recommendations Date Action Purpose 4/21 Sub-group debriefing Clarification on discoveries and recommendations from each sub-group Phase four required each sub-group to come up with recommendations based on their discoveries. Team leaders for each sub-group were instructed to complete a Summary of Recommendations form. In addition to listing discoveries and recommendations on the form, team leaders were also asked to identify whether the recommendation would help the program achieve progress toward a goal, mitigate a potential systemic issue, and/or pose as an innovation the program may want to consider. Once completed, a debriefing occurred with each sub-group to provide clarity of the discoveries made and recommendations provided. Phase Five- Prepare the Report Date Action Purpose 4/25- Development of Self- 4/28 Assessment Report 5/1 Draft review of Self- Assessment Report Compile the discoveries and recommendations from the Self-Assessment team into a comprehensive report Review report draft for content and understanding of discoveries 6

8 Phase five entailed the writing of the final Self-Assessment report, which included a compilation of all the discoveries and recommendations from the Self-Assessment team, as well as, any identified program strengths within one or more of the focus areas, areas for improvement to address potential risks to compliance, opportunities to enhance management systems and ensure progress on program goals and objectives. Post Phase- Planning Date Action Purpose 5/3 & 5/17 Management team meeting Complete annual planning process and development of the program s improvement plan During the post-phase, the annual planning process begins, which includes the review of recommendations from the Self-Assessment team in order to guide programmatic decisions in support of continuous quality improvement and the development of the Program Improvement Plan. All objectives and strategies developed during this process will be implemented and completed during the program year. Management, staff and Policy Council representatives will also review and discuss long-term goals to determine if revisions are necessary and establish short-term objectives with linkages to expected outcomes. III. Key Insights Discoveries made by the Self-Assessment team highlighted program strengths, areas of improvement, and potential barriers to demonstrating progress on goals and objectives. Further analysis of these discoveries identified opportunities to enhance service delivery, potential threats to compliance, and management system(s) that may need to be reviewed to ensure effectiveness. 7

9 Program Strengths & Successes The Self-Assessment team identified the following program strengths and successes based on the five focus areas reviewed: Health-Dental Treatments 1. The program now has three LPNs and is hiring for a fourth. The additional staff has decreased caseload demonstrating improvements in the amount of follow up on dental service needs by the Health service area. Professional Development 1. The program exceeds the minimum requirements for training hours. 2. The program has implemented intensive coaching practices for teaching staff. To continue to support strengths in this area, the program may want to seek hiring additional staff to assist in intensive coaching practices. 3. The program has successfully implemented programs such as Abriendo Puertas/Opening Doors and Getting Ahead in a Just Getting By World as a twogeneration approach to address the needs of families. Program Improvements to Service Delivery and Systems Service Delivery Many of the discoveries identified present an opportunity for the program to enhance, strengthen and streamline service delivery particularly in Health and Family Engagement. However, a few potential risks to compliance with Head Start Program Performance Standards were also identified. Health-Dental Treatments In accordance with (d)(2), a program must develop a system to track referrals and services provided for oral health and monitor the implementation of a follow-up plan. Ongoing monitoring results from the PFCE and Quality Assurance Units identified a high noncompliance with children receiving dental treatments. Increasing the number of children who received dental treatment that need it is a program goal. Although other program data 8

10 suggests an increase in the program facilitating and ensuring the dental treatment needs of children; there is still a large number of children whose dental treatment needs are not met. Family Engagement There is a potential risk to compliance with (c)(3) due to lack of follow up on family partnership agreements. Ongoing monitoring results by the Parent, Family and Community Engagement Unit identified 85% non-compliance in follow up on PFCE-Partnership Agreements as of the second quarter review for the program year. In addition, as of the second quarter, the completion of Family Needs Assessments was found to be noncompliant 76% of the time. Management Systems Strong management systems are a vital factor in ensuring effective and high-quality delivery of services. With consideration to the discoveries with a potential risk to non-compliance the following management systems are recommended to be reviewed to ensure their effectiveness: 1. Data & Evaluation a. Assess how we collect and use data to inform ongoing monitoring & continuous improvement b. Ensure our data helps us to demonstrate how we are making a difference in the lives of our families c. Continue to demonstrate progress toward Five Year goal #4 in which the program established to support improvement in its capacity to utilize program data to inform continuous quality improvement 2. Technology & Information Systems a. Ensure we have the right technology in place to support our outputs and communication with families b. Ensure the technology used by the program aligns with planning strategies 3. Training & Professional Development 9

11 a. Ensure the program continues to demonstrate progress toward Five Year goal #3 in reference to instituting a professional development program that ensures compliance with regulatory requirements b. Ensure staff have the knowledge needed to meet program goals and objectives 4. Communication a. Ensure we effectively communicate with families in a culturally and linguistically appropriate manner b. Ensure family engagement outcomes are shared with internal and external stakeholders 5. Recordkeeping & Reporting a. Ensure Management is able to obtain timely information vital to program decision making 6. Ongoing Monitoring & Continuous Improvement a. Ensure effective monitoring processes that yield valid and reliable data to utilize in program operations, planning, and continuous improvement Progress on Program Goals Some of the discoveries from the Self-Assessment suggest a review of the following program goals to ensure demonstrated progress will be evident and program objectives are met. 1. Five Year Goal #2-3, & 8 2. Program Improvement Goal # School Readiness Objectives and Strategies #1-#5 The program will spend time reviewing and revising strategies, as necessary, during the planning process. VI. Results of the Self-Assessment Discoveries and recommendations gleaned from the Self-Assessment are delineated under the five focus areas. Management will utilize the information to make decisions about 10

12 program operations that will seek to improve service delivery and close any identified gaps in the program s overall performance. Each recommendation is followed by the category it seeks to address/enhance in parentheses. School Readiness Discovery: User-friendliness of Galileo Parent User Module and low participation from parents o DVD s are not an effective tutorial method for young parents. Develop a 1-2 minute webinar/training tutorial on the purpose of the Galileo Parent User Module and how to use it (Program Improvement Goal #2, Innovation) o Translate tutorial for Creole speaking parents (Communication) o Design a Galileo app (Innovation) Discovery: Evidence of staff implementing skills attained during training o Conduct gain surveys as a follow up to training to determine what staff learned (Five Year Goal #3, Innovation) o Conduct observations that determine if new skills are being implemented and how staff are implementing the new skills (Five Year Goal #3, Innovation) Discovery: Information on the qualifications of successful teachers o When sharing outcomes information provide more information about what attributed to a teacher s success, i.e. credentials/qualifications, experience, hours spent in training/professional development, the amount of dosage or money spent in professional development efforts (Innovation) o Create webinars to be sent to teachers for internal learning (Innovation) o Cross train with partners such as the Early Learning Coalition to gain additional learning abilities (Innovation) Discovery: Additional relevant data to support child outcomes 11

13 o Use ongoing monitoring data to analyze and determine the reasons for noncompliance s (Ongoing Monitoring & Continuous Improvement) o Run more relevant reports that correlate with school readiness goals and objectives (School Readiness Goals) o Revise or establish monitoring indicators to be in alignment with school readiness objectives/outcomes (Program Improvement Goal #1) o Revise or establish monitoring indicators to determine how teachers directly impact child outcomes (Program Improvement Goal #1) Discovery: Objectives for school readiness were written for teachers and children o Revise objectives to focus only on the children and their gains (School Readiness Goals) o Strategies should be quantifiable to demonstrate evidence based outcomes and written to focus on what the teachers will do (School Readiness Goals) Discovery: Monitoring documentation is largely clerical Monitoring should be observational using a reliable tool to assess interactions between teachers and the children (School Readiness Goals, Ongoing Monitoring & Continuous Improvement) Health-Dental Treatments Discovery: Parent Communication Recommendations: o Implement visual cues for parents on sign in/out sheets such as using a highlighting system to notify the parent of when he/she needs to talk go to the Family Service Worker (Innovation) o Use an automated system i.e. text messaging as a means to communicate with parents (with their consent); should be limited to critical issues such as those related to medical/dental (Technology & Information Systems) o Revise procedures to change the responsibility of sending the day expiring/missing dental examinations and treatment notification letters to 12

14 parents to Head Start LPN s (Five Year Goal #2 and Program Improvement Goal #4) Discovery: Oral Health Assessments Recommendations: o To ensure Oral Health Assessments are completed with thoughtful consideration and not just compliance: Increase parent education i.e. how to fill out the assessment and why it s important (Five Year Goal #2) Train staff on how to properly use oral health assessment to educate parents (Training & Professional Development) Translate oral health assessments into other languages primarily spoken by families (Communication) Implement a more user-friendly form (Recordkeeping & Reporting) Complete assessment through electronic data collection tool such as Survey Monkey or Qualtrics for data analysis (Data & Evaluation, Innovation) Discovery: Division liaison to partner/engage with healthcare providers Recommendations: o Improve community collaboration by engaging with local healthcare providers in a face-to-face format i.e. visit their offices to attempt recruitment efforts (Innovation) o Implement and utilize a distribution list of healthcare providers who have seen Head Start children to identify providers who are underutilized (Five Year Goal #2 and Program Improvement Goal #4, Innovation) o Collaborate with other Head Start programs in Florida and eventually other states whose statistics demonstrate a higher level of compliance with dental treatments to gain fresh perspectives (Five Year Goal #2 and Program Improvement Goal #4, Innovation) Discovery: Data entry process Recommendations: 13

15 o Utilize a data field entry method in ChildPlus rather than a text field entry in order to utilize drop down list of options to improve recordkeeping and reporting of contact attempts with parents (Recordkeeping & Reporting) Discovery: Alert system in ChildPlus Recommendations: o Implement flag notification in data management software to alert staff when a child has an expired health event (Recordkeeping & Reporting) Discovery: Access limitations in ChildPlus Recommendations: o Allow LPN s to update healthcare provider contact information (Data & Evaluation) o Increase the capacity to enter dental information in ChildPlus in real time i.e. when staff are completing tasks at the centers (Technology & Information Systems) Discovery: Multi-disciplinary Team approach for Dental Action Plans Recommendations: o Complete initial dental action plans over the phone with parent rather than face to face (Five Year Goal #2 and Program Improvement Goal #4, Innovation) o Reconsider whether dental action plans are needed for every child who has dental treatment needs or whether the use of plans may be more effective if implemented with families who are non-compliant (Five Year Goal #2 and Program Improvement Goal #4, Innovation) o Revise dental action plan to only require signature from parent, LPN, and Family Service Worker (Innovation) Safety Discovery: Accident/Incident report form 14

16 o Implement electronic format in order for necessary parties to review and analyze accident/incident data. (Innovation) o Review the Orange County workers compensation form for additional fields that can be incorporated in the accident/incident form i.e. hours, dates, and type of injury and cause of injury (i.e. child on child, adult on child, child on adult, etc.) (Data & Evaluation, Innovation) o Implement a method similar to DART (Day Away, Restricted, Transfer), a calculation which is used to track injuries reported to OSHA. First, the program must determine a constant in order to calculate frequencies of accident/incident at all centers using historical data for no earlier than 8 years. (Innovation) Discovery: Data analysis of Accident/Incidents o Accidents and incidents should be reported separately. (Recordkeeping & Reporting) o Review SHIRMS (Safety & Health Injury Reporting Management System) for an example of incident reporting processes (Innovation) o Implement a version of the DART (Days Away, Restrictions, and Transfers) calculation to calculate a Dart rate rather than an incident rate of children injuries due to the different number of children enrolled at each center (Innovation) o Preventative measures to reduce risk of accident/injury should be included with form and monitored/followed up on. (Innovation) o Elaborate on procedures for LPNs to review accident/incident reports of children who required medical attention (Data & Evaluation) o Add an Administration Use Only section and signature line to accidents/incidents form for the LPNs to verify nurses review. (Innovation) Discovery: Safety and Injury Prevention at Centers 15

17 o Implement electronic Daily Checklist for Center and Classrooms to provide electronic view access by necessary staff (Innovation) o Work order system should reflect priority levels, which would be used to indicate whether the work order request was the result of an injury. Reference Fire Rescue s system called Track It (Innovation) o Conduct follow up when a work order is needed to repair equipment/furniture (Innovation) Family Engagement Discovery: Follow up on Family Needs Assessment (FNA) and referrals o Establish a system of follow up that addresses family needs, family goals, and referrals in a 3 in 1 approach (Five Year Goal #8) o Revise how referrals are documented in ChildPlus in order for appropriate tracking and follow up to occur (Five Year Goal #8, Data & Evaluation) Discovery: High case load for Family Service Workers o Hire additional staff to support the PFCE Unit (Five Year Goal #8) o Transfer the responsibility for follow up on medical/dental health services from FSWs to the LPNs (Ongoing Monitoring & Continuous Improvement) o Set up automated /text communication to families from the ChildPlus system for when health requirements are due to expire (Innovation) o Utilize electronic data capture in ChildPlus and electronic forms to reduce time spent filing documentation into a paper file (Innovation) Discovery: Progress on program PFCE related goals o Share family engagement outcomes with parents, partners, and staff (Communication) 16

18 o Develop a quarterly/monthly report to demonstrate progress as a result of follow ups on PFCE-Partnership Agreements, needs identified by the family, and referrals (Five Year Goal #8, Recordkeeping & Reporting) Discovery: High non-compliance of follow up with families o Revise ongoing monitoring procedures to establish additional steps that assure compliance with family engagement processes (Ongoing Monitoring & Continuous Improvement) o Increase monitoring oversight by Sr. Family Service Workers (Ongoing Monitoring & Continuous Improvement) o Review PFCE-Partnership Agreements for the quality, content and appropriateness of the goal established and whether they were made in consideration of family needs (Ongoing Monitoring & Continuous Improvement) o Ensure follow up on families with a score indicating crisis or vulnerability in one indicator or more on the Family Needs Assessment is provided with sufficient follow up and have demonstrated progress (Ongoing Monitoring & Continuous Improvement) Professional Development Discovery: Recordkeeping & Reporting of trainings completed by all staff, volunteers, and parents o Assign a staff person to input training and professional development data in the system (Recordkeeping & Reporting) o Provide certificates when trainings are completed (Recordkeeping & Reporting) o Revise procedures to include a specific process to document and track the individual professional development of staff (Recordkeeping & Reporting) Discovery: Tracking of employees individual training hours in ChildPlus 17

19 o Create custom filter to identify who has completed/needs the 15 clock hours of training required per year (Recordkeeping & Reporting) o Develop structured format for titling training topics in order to filter by training topic and use a coding system to track staff attending each training topic (Recordkeeping & Reporting,) o Create an electronic format to identify required trainings needed to meet Performance Standards for staff with/without regular contact with children (Five Year Goal #3, Innovation) Discovery: Tracking short and long term goals for accomplishments o Make Professional Development Plan available in electronic format (Innovation) o Follow Family Needs Assessment model of assessing three times per year to track progress on goals (Five Year Goal #3, Innovation) Analyze Professional Development Plans for trainings that all staff may benefit from (Five Year Goal #3) V. Conclusion Three of the five focus areas for the 2017 Self-Assessment are directly linked to the program s five year goals, which are: Dental Treatments, Professional Development and Family Engagement. During the planning process, the management team will revise objectives and develop strategies on the Five Year Plan for the program year that will be specific to these three areas. A program improvement plan will be developed to ensure immediate attention is given enhancing the monitoring oversight of school readiness goals and strengthen incident and accident reporting, monitoring, and injury prevention procedures. Progress on meeting objectives will be tracked and reviewed quarterly during the program year. 18

20 Orange County Head Start continues to strive for high performance and high-quality service delivery by utilizing program data to guide program decision making and continuous quality improvement efforts. The reliance of program data during the 2017 Self-Assessment led to several discoveries, many of which highlight opportunities for program improvement. The insight received during this process will assist management in making sure appropriate actions take place within the next program year to enhance program performance and ensure achievement of program objectives. 19