Determining Whether Recommendations from Comprehensive Needs Assessment Reports Are Reflected in School Improvement Plans in North Carolina

Transcription

1 3/22/2012 Draft Determining Whether Recommendations from Comprehensive Needs Assessment Reports Are Reflected in School Improvement Plans in North Carolina by Angel Banks A paper submitted to the faculty of The University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree Master of Public Administration Spring 2012 This paper represents work done by a UNC-Chapel Hill Master of Public Administration student. It is not a formal report of the School of Government, nor is it the work of School of Government faculty. Executive Summary: This paper explores whether a sample of forty-five North Carolina public elementary, middle, and high schools that received Comprehensive Needs Assessment (CNA) recommendations from the North Carolina Department of Public Instruction (DPI) CNA team incorporated them into their School Improvement Plans (SIPs). Seventy-one percent of the SIPs reviewed in this study incorporated at least seventy-five percent of the recommendations they received. This finding may suggest that the schools in the sample considered the CNA feedback they received when constructing their SIPs.

2 Background School Improvement Plans (SIPs) are strategic planning documents created by the leadership and staff of public elementary, middle, and high schools. These documents are often mandated for academically low-performing schools by accountability agencies, such as state education departments. Existing research suggests focused and specific school improvement planning improves academic achievement. 1 In North Carolina, SIPs are state-mandated documents for all schools, and are subject to the approval of the local school board of the local administrative unit. (N.C. General Statute 115C-27). Schools create SIPs to reflect priorities for the upcoming academic year. While schools are not required to use a specific template, DPI has published the School Improvement Planning Guide to assist schools with the construction of their SIPs. A SIP that is based on an external evaluation may assist schools in focusing on goals and strategies (McLaughlin and Talbert, 2006). In this analysis, the Comprehensive Needs Assessment (CNA), which is conducted by DPI, is assumed to function as an evaluation of a school s weaknesses and strengths. McLaughlin and Talbert (2006) hypothesized that external evaluations can assist schools in creating SIPs that address long-term needs and structural weaknesses. As a best practice, the U.S. Department of Education encourages CNA teams to include both a school profile with baseline information about the school and a narrative describing the team s observations from the school visit in CNA reports. 2 The baseline assessment and narrative assist the CNA team in providing schools with recommendations for improvement. In North Carolina, CNAs are conducted by DPI s CNA team, which also adheres to the practice of compiling baseline information and writing a narrative in order to offer recommendations to the assessed schools. 3 These two components appear in the form of a CNA report. According to the DPI (2006), a North Carolina CNA provides a framework that: Provides school districts 4 and individual schools with a clear view of their strengths, areas for improvement, challenges, and successes Enables a systematic review of practices, processes, and systems within a school district Assists district and school leadership in determining needs, examining the nature and causes of those needs, and setting priorities for future action Guides the development of a district or school plan and suggests benchmarks for evaluation Offers recommendations to schools based on the indicated areas for improvement DPI has developed a pathways model for school improvement, which encourages low-achieving schools to actively use CNA reports to develop their school improvement efforts. 5 As North Carolina public schools, particularly those that have been designated as low-performing and turnaround, 6 construct strategic plans to improve academic performance, stakeholders may wish to know whether schools are using the external feedback they receive to inform their strategic planning. To that end, this capstone considers the following questions: 1 Mintrop, H. and MacLellan, A. (2002). School Improvement Plans in Elementary and Middle Schools on Probation. The Elementary School Journal.Vol. 102 (4), Department of Education. (1998). Implementing School-wide Programs: Conducting a Comprehensive Needs Assessment. Archived Information. 3 With the exception of Comprehensive Needs Assessments conducted in Charlotte-Mecklenberg, where assessments are conducted by the local education agency (LEA). 4 School districts are legally referred to as school administrative units. 5 See appendix 1.4 for CNA Pathways model, which shows how schools are encouraged to use CNA reports. 6 Low-performing schools are schools that have not met Adequate Yearly Progress (AYP).Turnaround schools are persistently low-performing middle or high schools identified to receive intensive support from the NC Department of Public Instruction.There are currently 118 schools in turnaround status in North Carolina.

3 Have schools that have received Comprehensive Needs Assessments addressed the areas indicated for improvement within their School Improvement Plans? Which areas for improvement are most or least likely to be addressed? As stakeholders, DPI and its CNA team may be assisted by this study in their efforts to evaluate the effectiveness of the message being given to schools through CNA reports. Additionally, this study can also be used to assist schools in constructing SIPs that adequately address the weaknesses DPI has identified. Methodology Thirty SIPs were gathered from individual school websites. The remaining ninety-five schools receiving CNAs during the , , and academic years were contacted, and of those schools, fifteen schools chose to participate in the study by submitting their SIPs. Because the sample was self-selecting, the results of this study are not representative of the population of schools that have received CNAs. The survey response rate was thirty-six percent. To determine the alignment between CNA reports and SIPs, SIPs for the academic year following the year in which the CNAs were conducted were gathered. SIPs were analyzed to determine whether schools incorporated the CNA recommendations. Then, a comparison was performed to determine types of recommendations that were and were not incorporated. Descriptive Statistics of Schools within Sample The sample was slightly more rural than urban, and seventy-three percent of the schools were designated as either low-performing or turnaround schools at the time of the needs assessment. Additionally, over half of the schools in the sample were elementary schools, and almost one-third of the schools in the sample were high schools. Table 1. Distribution of Sample Schools Elementary Middle High Total Sample Rural 50% 89% 25% 53% Urban 50% 11% 75% 47% Low-performing 78% 12% 42% 53% Turnaround *0% 50% 42% 20% No status (L or T) assigned 22% 38% 16% 27% *Only middle and high schools are assigned to turnaround status Findings The sample is evenly distributed, and a correlation between school type or status and the number of recommendations incorporated into the respective SIPs was not apparent. Overall, almost three-quarters of the schools incorporated at least three-quarters of the recommendations they received, as shown in Table 2. Table 2. Schools and Percentage of Recommendations Adopted Percentage Adopted Number of Schools

4 The table in appendix 1.2 lists all thirty-three of the recommendations and the number of schools incorporating each recommendation in their SIPs. The thirty-three recommendations schools in the sample received were grouped into four categories: Curriculum/Instruction Professional Development Student Development School Culture Overall, schools incorporated the majority of recommendations they received from all four categories. 7 There were more recommendations pertaining to curriculum and instruction than any other recommendation type. Table 3 includes recommendations for improvements in curriculum and instruction; and shows that the majority of recommendations in this category were incorporated into SIPs. The recommendations that were incorporated by the smallest number of schools were the recommendations to effectively incorporate instructional technology, prioritize planning for specific grades, and analyze curriculum to ensure alignment with the North Carolina Standard Course of Study (NCSOS). Table 3. Curriculum/and Instruction Recommendations Incorporated into SIPs Curriculum/Instruction Recommendations Schools Incorporating Recommendations Differentiate instruction 27 out of 28 Create options for identifying underserved populations 1 out of 2 Consistent/increased data analysis to improve instruction 21 out of 22 Usage of formative and summative assessment 17 out of 18 Analyze written, taught, and learned curriculum to ensure alignment 4 out 6 with NCSCOS Prioritize planning for specific grades 0 out of 1 Effectively incorporate instructional technology 4 out of 10 Focus teacher attention on specific subgroups 4 out of 5 Increase rigor in lessons; utilize instructional strategies to improve 24 out of 27 student engagement Monitor quality of instruction to provide feedback to teachers 12 out of 13 Examine classroom grading policies 2 out of 2 Table 4 shows the professional development recommendations schools received in CNA reports. The recommendations that were incorporated by the smallest number of schools were the recommendations to create procedures to monitor program effectiveness, clarify assignment expectations to students and parents, and develop a professional development plan. However, the majority of the recommendations in this category were incorporated into SIPs.\ Table 4. Professional Development Recommendations Incorporated into SIPs Professional Development Recommendations Schools Incorporating Recommendations Professional development to improve instructional differentiation 16 out of 19 Create procedures to monitor program effectiveness 5 out of 8 Professional development to increase cultural awareness 1 out of 1 Clarify instruction expectations to teachers 5 out of 7 Clarify assignment expectations to students and parents 3 out of 6 Increase diversity of school staff 1 out of 1 Develop a professional development plan 2 out of 4 Increase teacher collaboration via Professional Learning Communities 3 out of 3 Develop leadership capacity 2 out of 3

5 The remaining two recommendation types were incorporated by the majority of the schools receiving them. Table 5 shows while a smaller number of schools received recommendations that targeted student development, the schools receiving these recommendations incorporated them into their SIPs. Table 6 shows that overall, most schools incorporated recommendations pertaining to school culture into their SIPs. Table 5. Student Development Recommendations Incorporated into SIPs Student Development Improve student attendance on and off-site 1 out of 1 Revise teaching assignments to ensure qualifications 1 out of 2 Adequate and timely support programs for at-risk students 1 out of 1 Expand student access to extracurricular programming 1 out of 1 Focus teacher attention on specific subgroups 3 out of 5 Schools Incorporating Recommendations Table 6. School Culture Recommendations Incorporated into SIPs School Culture Schools Incorporating Recommendations Build/maintain relationships with parents and community 11 out of 12 Allow parents and community to become involved in decision-making process 2 out of 2 Establish a culture of high expectations 13 out of 16 Implications This study shows thirty-seven out of the forty-five schools in the sample explicitly incorporated aspects of the CNA report in their SIPs. This suggests that CNA reports were at least considered in developing the majority of SIPs in the sample. Additionally, because seventy-three percent of schools in the sample were designated as low-performing or turnaround schools at the time of the assessments, the high percentage of SIPs including CNA recommendations is notable. This finding suggests within this sample, the types of schools that are heavily encouraged to use CNA reports in school improvement planning are doing so, and this is reflected in their SIPs. The findings also demonstrate that CNA recommendations of all types were reflected in the SIPs of schools in the sample. However, because participating schools selected themselves into the group, the sample is biased. The remaining eighty schools elected to not participate for unknown reasons, and this lack of participation may have skewed the results of the study. The analysis of SIPs and CNA reports shows that some schools have framed the feedback they have received in a way that is most relevant to their schools' needs. For example, one school s school improvement team addressed the recommendation to increase the rigor of instruction by setting a goal for the team to meet and construct a clear definition of what rigorous curriculum and instruction means for their school, given the backgrounds of their students. While this study did not assess the schools timeframe for implementation or future evaluation efforts, thirty-two schools in the sample included plans to evaluate the implementation of the incorporated recommendations on a quarterly basis. Forty schools allocated resources to the CNA recommendations that were incorporated, such as staff to lead an effort and classrooms for particular enrichment activities. This attention to detail might suggest that CNA recommendations are not only being incorporated into SIPs, but are used to add strategic value to SIPs as well.

6 Future Research This study is limited and does not provide explanations for why some of the recommendations were incorporated into SIPs less frequently. This might indicate an area to be explored in future research. For example, while ten schools received the recommendation to effectively incorporate instructional technology to improve instruction, only four schools included the recommendation in their respective SIPs. This finding could indicate that the remaining six schools had an existing strategy for instructional technology in place, or these schools did not have the necessary resources to make this recommendation a priority in their SIPs. By following up with the schools that did not incorporate particular recommendations into their SIPs, the CNA team could ascertain why these schools chose to not incorporate them. Additionally, following up with schools can help the CNA team understand individual schools decision-making processes when consulting CNA reports, resulting in lessons that can be applied to future assessments. Conclusions These results suggest that schools in the sample used the external feedback they received, which demonstrates that needs assessments can be used to assist schools with school improvement planning efforts. The CNA team can further study the schools that have incorporated all of the recommendations and determine why all of the recommendations were reflected. The team may find that possible reasons for a comprehensive incorporation of all recommendations may range from understanding the feedback they have received to having the ability to tailor the feedback to their school's curriculum and staff-based needs. In addition, an analysis of schools that have not incorporated particular recommendations may uncover ways to make the needs assessment process more helpful to those and similar schools.

7 Works Consulted Department of Education. (1998). Implementing School-wide Programs: Conducting a Comprehensive Needs Assessment. Archived Information. Desimone, L. (2002). How Can Comprehensive School Reform Models Be Successfully Implemented? Review of Educational Research: Vol. 72 (3), Standards-Based Reforms and Accountability, Land, D. (2002). Local School Boards under Review: Their Role and Effectiveness in Relation to Students' Academic Achievement Review of Educational Research: Vol. 72 (2), Mintrop, H. and MacLellan, A. (2002). School Improvement Plans in Elementary and Middle Schools on Probation. The Elementary School Journal.Vol. 102 (4), Mintrop H. and Trujillo, J. (2007). The Practical Relevance of Accountability Systems for School Improvement: A Descriptive Analysis of California Schools. Educational Evaluation and Policy Analysis. Vol. 29 (4), Newmann, F. et. al. (2001). Instructional Program Coherence: What It Is and Why It Should Guide School Improvement Policy. Educational Evaluation and Policy Analysis:Vol. 23 (4), North Carolina Department of Public Instruction. (2006). District and School Transformation: Comprehensive Needs Assessments.

8 Appendix 1.1 Glossary of Terms Comprehensive Needs Assessment: A process by which an internal or external evaluation is conducted to determine a school's strengths and areas for improvement. ( Comprehensive Needs Assessment team: A team of NC DPI staff members, with the task of visiting schools and conducting evaluations of each school s areas for improvement and strengths. North Carolina Department of Public Instruction: The North Carolina Department of Public Instruction is the agency charged with implementing the State's public school laws and the State Board of Education's policies and procedures governing pre-kindergarten through 12th grade public education. ( School Improvement Plan: A document outlining a school's programming efforts to improve school achievement for one academic year. This document is typically mandated by federal and state regulations for schools designated as low-performing (N.C. General Statute 115C-27). School Improvement Plan Team: A committee of school administrators, teachers, and parents convened with the purpose of drafting the School Improvement Plan for the upcoming academic year. This team's membership varies from year to year. The School Improvement Planning Guide provides information on the role of the team and the process (

9 Appendix 1.2 Comprehensive Needs Assessment Recommendations (By Type) Schools Incorporating Recommendations in SIP Curriculum/Instruction Differentiate instruction 27 out of 28 Create options for identifying underserved populations 1 out of 2 Consistent/increased data analysis to improve instruction 21 out of 22 Usage of formative and summative assessment 17 out of 18 Analyze written, taught, and learned curriculum to ensure alignment with NCSOS 4 out of 6 Prioritize planning for specific grades 0 out of 1 Effectively incorporate instructional technology 4 out of 10 Focus teacher attention on specific subgroups 4 out of 5 Increase rigor in lessons; utilize instructional strategies to improve student engagement 24 out of 27 Monitor quality of instruction to provide feedback to teachers 12 out of 13 Examine classroom grading policies 2 out of 2 Integrate cultural arts 0 out of 1 Arrange student schedules to maximize educational opportunities 1 out of 3 Provide remediation through instructional intervention 1 out of 1 Utilize teacher assistants for support 1 out of 1 Student Development Improve student attendance on and off-site 1 out of 1 Revise teaching assignments to ensure qualifications 1 out of 2 Adequate and timely support programs for at-risk students 1 out of 1 Expand student access to extracurricular programming 1 out of 1 Focus teacher attention on specific subgroups 3 out of 5 Professional Development Professional development to improve instructional differentiation 16 out of 19 Create procedures to monitor program effectiveness 5 out of 8 Professional development to increase cultural awareness 1 out of 1 Clarify instruction expectations to teachers 5 out of 7 Clarify assignment expectations to students and parents 3 out of 6 Increase diversity of school staff 1 out of 1 Develop a professional development plan 2 out of 4 Increase teacher collaboration via Professional Learning Communities 3 out of 3 Develop leadership capacity 2 out of 3 School Culture Build/maintain relationships with parents and community 11 out of 12 Allow parents and community to become involved in decision-making process 2 out of 2

10 Establish a culture of high expectations 13 out of 16 Appendix 1.3 Schools Inclusion Percentage for Recommendations Sunset Park Elem (L) 33 (2 out of 6) C.B. Martin Elem (L) 33 (1 out of 3) Dudley High (T) 50 (2 out of 4) Parkview Village Elem (L) 50 (2 out of 4) Southeast Elem (L) 50 (2 out of 4) Eastlawn Elem (L) 57 (4 out of 7) Fairview Heights Elem * 60 (3 out of 5) T. Wingate Andrews High (T) 60 (3 out of 5) W.M. Hampton Elem (L) 60 (3 out of 5) Brawley Middle (T) 60 (3 out of 5) Winston-Salem Prep Acad (L) 60 (3 out of 5) South Edgecombe Middle 62 (5 out of 8) Otis Hairston Middle (T) 70 (7 out of 10) Pembroke Elementary (L) 71 (5 out of 7) Rachel Freeman School of Eng 73 (8 out of 11) Oak Hill Elementary (L) 75 (3 out of 4) Fayetteville Street Elem (L) 75 (3 out of 4) Eastern Guilford High (L) 80 (4 out of 5) Montlieu Ave Academy (L) 80 (4 out of 5)

11 Mary S. Mosley PLC (L) 83 (5 out of 6) Dawson Elem (L) 100 (5 out of 5) Anson High (T) 100 (5 out of 5) Anson Middle (T) 100 (5 out of 5) Williford Elem 100 (6 out of 6) E. Hanford Dole Elem (L) 100 (6 out of 6) Coker-Wimberley Elem (L) 100 (4 out of 4) Middle Fork Elem (L) 100 (4 out of 4) Rochelle Middle (L) 100 (5 out of 5) A.H. Snipes Acad of Arts (L) 100(6 out of 6) Red Springs Middle (L) 100 (2 out of 2) East Elem (L) 100 (3 out of 3) Aulander Elem* 100 (3 out of 3) Hertford County High (T) 100 (3 out of 3) Aurelian Springs Elementary* 100 (5 out of 5) Richmond Senior High* 100 (4 out of 4) H.D. Isenberg Elem 100 (2 out of 2) Union Middle 100 (8 out of 8) Alamance-Burlington Middle College (T) 100 (5 out of 5) Chewning Middle (T) 100 (2 out of 2) Lillian Black Elem (L) 100 (3 out of 3) Westover High (T) 100 (2 out of 2) Fairview Elementary (L) 100 (2 out of 2) Colerain Elem 100 (3 out of 3) Kannapolis Intermediate 100 (3 out of 3)

12 Ashley Chapel Elem (L) 100 (3 out of 3) (L)=Low-performing (T)=Turnaround * The district in which the school is located was in Transformation (low-performing) status at time of CNA Appendix 1.4 CNA Pathways Model Low-performing and turnaround schools receive self-evaluation forms before the two-day CNA visit to their schools. During the CNA process, the CNA team observes teachers and interviews school leadership. A CNA report is issued to each school following the CNA, and the school is encouraged to use this report to construct their SIP, as illustrated in NC DPI s Pathways Model above.