Excerpts from the Baltimore Community Foundation s Neighborhood Small Grants Program Evaluation

Excerpts from the Baltimore Community Foundation s Neighborhood Small Grants Program Evaluation 1

Table of Contents Executive Summary... Projects... 4 Grantees... 5 Technical Assistance (TA)... 5 Grant-Making Review Process... 6 Introduction... Participation in the Evaluation... 11 Organization of the Report... Error! Bookmark not defined. Section 1: Project Outcomes and Impact... Error! Bookmark not defined. Describing Projects... Error! Bookmark not defined. Effort: Resident Engagement... Error! Bookmark not defined. Quality... Error! Bookmark not defined. Outcome... Error! Bookmark not defined. Outcome: Leverage... Error! Bookmark not defined. Outcome: Engaging Residents... Error! Bookmark not defined. Grantee Goals: Impact... Error! Bookmark not defined. Overall Impact of NGP... Error! Bookmark not defined. Section 2: About Grantees... Error! Bookmark not defin Organization size... Error! Bookmark not defined. Organization capacity... Error! Bookmark not defined. Organizations Capacity to Consistently Engage Residents... Error! Bookmark not defined. Geographic Distribution of Grants... Error! Bookmark not defined. Section 3: Technical Assistance (TA)...Error! Bookmark not defined. Describing the TA Process... Error! Bookmark not defined. Technical Assistance: Effort and Quality... Error! Bookmark not defined. Technical Assistance Process: Findings and RecommendationsError! Bookmark not defined. Section 4: Grant Application and Review Process...Error! Bookmark not defined. Grant Process: Effort... Error! Bookmark not defined. Describing Grant Funds... Error! Bookmark not defined. Feedback on the Quality of the Grants Process... Error! Bookmark not defined. Describing the Grant Readers... Error! Bookmark not defined. Feedback on the Quality of Being a Grant Reader... Error! Bookmark not defined. Feedback on the Impact of Being a Grant Reader... Error! Bookmark not defined. Describing the Application Review and Grant Making Process... Error! Bookmark not defined. Recommendations about the Grant Review Process... Error! Bookmark not defined. 2

Executive Summary In September of 2000, the Baltimore Community Foundation (BCF) began the Neighborhood Grants Program (NGP) for neighborhood associations and other resident-driven, community organizations. The primary goals of the NGP are to: Increase effectiveness of community organizations by providing financial resources and other support that will enable them to initiate and complete priority neighborhood projects; Strengthen neighborhoods so that they are supportive environments for families and businesses; and Strengthen neighborhoods so that current and potential residents and businesses are more likely to invest time, effort, and money in their communities. The Foundation seeks to achieve these goals by funding small projects which engage residents and are organized by associations (often volunteer run), by providing technical assistance, and by including diverse stakeholders in the grant review process. The underlying assumption/theory behind this structure is that providing small grants to Baltimore area neighborhood associations for projects will lead to more involved, invested residents which in turn give rise to neighborhoods that are strong, healthy and supportive places to live. (For a diagram showing the evaluation model, see Appendix A.) This evaluation report examines three areas of the NGP the grantees and their projects, the technical assistance program, and the grant-making review process. First, each area of the NGP is described. Second, process data about the effort and quality of each area is presented. Third, outcome and impact data about the grantees and their projects is reviewed. Part of the outcome and impact data seeks to determine if there is evidence to support the assumption that small projects will make a difference in the number of engaged citizens and consequently in the quality of life in the neighborhood. Areas Covered in the Evaluation Descriptive Information Process (effort and quality) information Outcome and impact information Grantees and their projects Included Included Included Technical assistance (TA) Included Included Grant-making review process Included Included The data reviewed in this report was collected from the following sources: Grantee applications Grantee reports Work sessions with grantees Interviews with grantees Interviews with Foundation staff and CPHA technical assistance providers Survey of grant readers The findings, conclusions and recommendations by area covered in the evaluation are summarized below. (For more information on the methodology used in this evaluation including the primary data collection tool, see Appendix B. Please note that only organizations that received grants are 3

included in this evaluation. The Foundation may wish to explore separately gathering satisfaction feedback from organizations that did not receive funding.) Projects The NGP made a difference in the number of engaged citizens in the area. The 27 evaluated projects engaged 221 people as project planners. Of these 58% were African American, 42% were white and 25% were under age 30. Fifty-eight percent of all planners were still engaged in neighborhood activities after the project was completed. NGP projects engaged between 5,000 and 9,000 residents as participants. Leaders estimate that 67% are still engaged in neighborhood projects. The NGP enabled a large range of positive neighborhood projects to take place. Fortyeight percent of the grantees in the evaluation accomplished all of the goals they listed in their applications. Across all evaluation participants at the time of the evaluation, 60% of their project activities were completed corresponding to 58% of their project goals. Beautification projects were the most common type of project with more than 41% of the grant activities being either gardening, planting trees and flowers, or beautifying vacant lots and homes. The evaluation concludes the following cumulative impact was realized from all the grantee projects: NGP projects have led to more engaged citizens in the funded neighborhoods. NGP projects have acted as catalyst for other neighborhood activities either organized by the association or by residents acting on their own. The small NGP projects fueled hope for positive change. Some projects enhanced community cohesion and relationships as well as built or supported a growing sense of community. They did this by providing opportunities for people to meet neighbors they had not met and by providing a common project on which to work which in turn built trust. In some cases, NGP projects helped to develop new leadership or supported existing leadership in the neighborhoods. Following are a sampling of quotes from participants about the difference that NGP projects are having in their communities: The project opened "friendlier" avenues of discussion between generations. The project gave me a much better and lasting relationship with youth and allowed me to get to know other people better. The neighbors really got to know each other. They are reaching out more to one another. The event provided a positive event for people to gather and for neighbors to meet. It got the neighbors out and talking to each other. After the project, there was an increase in energy and imagination and willingness to risk ideas. The overall (community) tone has improved and (the community) has a new outlook in the possibilities of positive change. The project showed monies can be obtained (come back) to Park Heights. The funding has produced a new feeling of pride and interest in community events. People saw that significant changes can be made by volunteers. 4

People had been disappointed and depressed but this helped show them that something could happen. It shows that people care that life is hard, but that people/the human spirit still care about each other and with a little organizing and support are willing to act to make positive change. Participants also recognized how small projects can lead to other activities in their neighborhoods It s so amazing how a small intervention can make such a big difference. For example, a small project, such as a lighting activity, can have a big impact on community safety. As one resident explained, over the last 48 years the trees in the neighborhood have closed out the sky, the streets are very narrow and a large piece of vacant land has turned into a dump. Then there was a murder. That murder galvanized the community. Residents thought with more light they d feel safer and be able to move throughout the community. This led to planning the lighting project NGP funded. One of the ways we knew we had succeeded was when an 80 year old neighbor called to say, now I can sleep at night. Thanks so much! One grantee recounted how their contest to improve the entryways in the neighborhood, sparked new activities in the area. The winners of the contest lived on the same block and they decided to donate their winnings back to the neighborhood to do more activities. Some of the money was used to pay for midnight patrols to address prostitution in the area. Other prize money went to buy supplies to plant pansies. Already fifteen other people on the block are involved and more join the effort weekly. (Other quotes are included throughout this report and in the appendix.) Grantees Grantee organizations were small; 64% had no paid staff when they applied for funds. Grantees generally had a medium to medium-high level of capacity to complete projects without assistance. Grantees capacity to consistently engage residents in projects and activities varied significantly. Grantee organizations were relatively evenly split across Baltimore City using Charles Street as an East-West dividing line and North Avenue as the North-South dividing line. There were more grants distributed in higher poverty areas. Over half of the grants were distributed in neighborhoods where the percentage of families living in poverty was 20% to 50%. Only 9% of grants were given in areas where 5% or less of the families live in poverty. Half of all grants were in neighborhoods where 75% or more of the families are African American. Generally, lead organizers of the project had overwhelmingly positive experiences with over 93% of participants rating their experience as positive or very positive. Many grantees were able to access other funds or resources after being awarded BCF funds. Technical Assistance (TA) Twenty-seven grantees received telephone assistance through CPHA. Approximately eight grantees received more in-depth assistance. 5

According to grantees, when they received assistance, it was very helpful. However, there was some confusion among grantees about how to access help and what that help can include. The Foundation and CPHA staff persons have had an excellent, respectful relationship. These findings suggest the following conclusions: Supportive help and problem-solving is useful and has been helpful for grantees in the NGP. However, there is little clarity on the form that this assistance should take, i.e. formal TA, peer to peer learning, or access to a wide range of assistance. There appears to be an opportunity to increase the use of available assistance by clarifying the types of TA and the procedures on how to access the TA. The process and communication between the Foundation and the TA provider has not been structured. The following recommendations are offered to clarify the TA process and type of TA offered to NGP grantees. It is, however, vital that as the process becomes clearer it does not become more complex. One of the strengths of the program that is important to sustain is its flexibility to respond to grantee needs, to offer assistance, and to offer a process that is appropriate to the level of the grant and activity. Clarify what kind of TA is available and how in-depth TA can be. Assess alternative delivery options for providing TA. Communicate this clearly to grantees. Clarify expectations, reporting requirements and other types of communication protocols between the Foundation and the TA provider. Emphasize the collection of evidence on TA assistance from grantees and TA provider. Grant-Making Review Process The Foundation distributed $318,751 in the four grant rounds between September 2000 and September 2001. Overall, BCF was able to gather a diverse group of grant readers. However, most grant readers live in northern Baltimore City or Baltimore County. In the future the Foundation may seek to increase representation from SE and SW Baltimore. Grant readers were fairly evenly split between the ages of 30 and 60 years old with 33% between 30 and 40; 25% between 40 and 50 and 25% between 50 and 60. Grantee Perspective Generally, grantees were very positive about the NGP, specifically with regards to: the amount of money available, the Foundation s willingness to make changes, the accessibility of the Foundation staff, and the paperwork they had to complete in reporting to the Foundation. Over 60% of respondents rated the NGP as excellent on these facets with less than 7% noting any need for improvement. All participants in the evaluation work sessions would apply to the program again. However, grantees were less positive regarding the paperwork required for the grant application. Less than half of the grantees said it was excellent and 21% said it needs improvement. Further, while most grantees rated the time they spent responding to the Foundation as excellent (or not 6

taking too much of their time), a significant minority, 19%, rated it as needing improvement and indicated that responding to the Foundation took too much time. 7

Grant Reader Perspective Generally the Foundation does an excellent job of facilitating and supporting grant readers. The evaluation team received several unsolicited comments that Foundation staff did a great job with this area of the program. Specific findings include: Twenty-one percent (the largest proportion of readers who rated an aspect of the program as negative) indicated they were unsatisfied with the time they had to discuss the applications with other grant readers. There was also limited dissatisfaction with the process of making the funding decisions, namely deciding who to fund and who not to fund. NGP made changes to the way grant meetings were conducted after the early rounds to provide for more discussion, however, continued negative ratings in the later rounds suggests this continues to be an issue. The large majority of grant readers reported that the time commitment and the amount of information provided and required is about right. A large majority of grant readers reported that the materials they received from the Foundation during the grant review process were of a good quality. Most grant readers were satisfied to very satisfied with the explanation of the NGP, the accessibility of NGP staff and the help that was available to them. Readers were slightly less satisfied with the explanation in advance of what was required as a reader. Generally, grant readers understood why applications were eliminated and agreed that the Foundation goes out of its way to fund projects and distribute all available funds. In conclusion, the grant reader experience did improve the volunteers opinion of neighborhood projects, increased their knowledge of neighborhood activities and leaders, and increased their understanding of BCF operations. As one reader said, It has been rewarding and I do leave the experience with a sense of hope and improvement for communities facing great challenges. In addition, for some grant readers, the experience increased their likelihood to engage themselves and others in neighborhood projects. For example, one reader notes, I talked about it to anyone who would listen; made a modest contribution myself; and brought in two people who also wanted to make a difference (one gave $1,000, the other volunteered.) Recommendations for Improving Grant Review Process Overall, NGP has a strong application review and grant process, however, we suggest the Foundation consider the following enhancements: Develop a report format, 2 pages, to send to grantees just before their project is scheduled to be completed. The report format should provide space for grantees to document project financials, activities, accomplishments, impacts, lessons, receipt of TA, and impact of the TA. This type of report format would address grantee comments that the reporting process and timeline was easy to forget as well as Foundation staff s desire to have outcome and impact data about both the grant project and any assistance the grantee received. Consider further shortening the application form or listing suggested word limits for questions to help grantees spend less time on the application. Clarify the desired end result of the program: is it about building capacity of neighborhood groups or is it about increasing resident participation to get neighbors to do small projects together. If capacity is part of it, program adjustments may be needed. 8

Since the grant review process and the inclusion of diverse readers seems to be an important part of the program, the Foundation should ensure that information about the grant review process is included in public and internal documents about the NGP. Improve the grant review and decision process, including improving orientation to potential readers, providing more time to discuss applications and getting agreement with grant readers about the process for deciding who to fund. Expand diversity among grant readers by increasing recruitment and participation from SE and SW Baltimore. 9

Introduction In September of 2000, the Baltimore Community Foundation (BCF) began the Neighborhood Grants Program (NGP) for neighborhood associations and other resident-driven, community organizations. The primary goals of the NGP are to: Increase effectiveness of community organizations by providing financial resources and other support that will enable them to initiate and complete priority neighborhood projects; Strengthen neighborhoods so that they are supportive environments for families and businesses; and Strengthen neighborhoods so that current and potential residents and businesses are more likely to invest time, effort, and money in their communities. The Foundation seeks to achieve these goals by funding small projects which engage residents and are organized by associations (often volunteer run), by providing technical assistance, and by including diverse stakeholders in the grant review process. The underlying assumption/theory behind this structure is that providing small grants to Baltimore area neighborhood associations for projects will lead to more involved, invested residents, which in turn makes neighborhoods strong, healthy and supportive places to live. (For a diagram showing the evaluation model, see Appendix A.) The NGP includes two different grant levels Mobilization grants of up to $5,000 and Leadership grants of up to either $10,000 or $25,000, depending on the grant round. Both levels of grants are intended for resident-driven community organizations that engage residents in designing and implementing projects. Mobilization grants are intended for small, primarily volunteer organizations or for small projects of well established organizations. If an organization does not have 501(c)(3) status (i.e. is not incorporated as nonprofit organization), then it must have a fiscal representative that is a 501(c)(3) nonprofit to receive and manage the mobilization funds 1. To qualify for a Leadership grant, the organization must be an established 501(c)(3) nonprofit organization with a history of neighborhood leadership and resident involvement and the project should be connected to longer term neighborhood plans. Since the NGP is targeted to neighborhood associations that often have no staff and bring a varying range of experience and capacity for completing projects, the Foundation wanted these groups to have access to assistance with their projects. The Foundation formalized a relationship with Citizen s Planning and Housing Association s (CPHA) Resource Center for Neighborhoods to provide technical assistance to its grantees. One of CPHA s organization goals is to develop and support civic and neighborhood leadership and so is closely linked to the aims of the NGP. Finally, the NGP seeks to engage a diverse group of stakeholders as grant readers for this program in order to expose them to neighborhood activities, engage them with other neighborhood leaders and projects, and educate them about the grants process and the operations of the Foundation. The grant readers are responsible for reviewing all applications and recommending funding levels to the Foundation s board of trustees. 1 Starting in Fall 2002, organizations applying for the Mobilization grants do not have to be incorporated 501c3 organizations or have a 501c3 fiscal representative. 10

Participation in the Evaluation Between September 2000 and December 2001, there were four rounds of grants September 2000, December 2000, March 2001, September 2001. This evaluation examined grants in the first four rounds. The following table summarizes the number of grants, the level of the grant, the number of grantees that had submitted final reports as of September 2002 and the number of grantees that participated directly in the evaluation for each grant period. Grant Round Level of Grant Total Number of Grants Final Reports 2 Represented at Evaluation Work Sessions or were Interviewed Sept 2000 Mobilization 15 10 7 Dec 2000 Leadership 10 5 6 Mar 2001 Mobilization 21 9 9 Sept 2001 Mobilization 14 7 7 60 29 29 55 unique organizations received 60 grants Participation in Evaluation 25 organizations, representing 29 grants, participated in interviews or work sessions 45% Grantees Participated in Evaluation Reviewed reports on an additional 14 organizations 71% Grantees Represented in Evaluation As the chart above shows, this evaluation includes information from 71% of grantees. There were 55 unique organizations that were grantees during the period of the evaluation. 3 Twentyfive organizations representing 29 grantee projects participated in work sessions or interviews and were represented in the evaluation data collection. Organizations were represented by one or more persons and a total of thirty-eight participants and organizers provided information specifically for this evaluation. 2 Some organizations have extended their timeframes and so final reports aren t due. Further, this was the number of reports in September 2002. 3 Five organizations received two grants, making 60 the total number of grants awarded. 11

Another 14 (non overlapping) organizations submitted reports containing information about their project activities and outcomes to the Foundation. In total, this evaluation contains information from 39 of the 55 grantee organizations (71%). 12