Evaluation Report: Year 2 Enhanced Settlement Supports

Transcription

1 Evaluation Report: Year 2 Enhanced Settlement Supports April 2012-March 2013 QUALTRICA ASSOCIATES Submitted by: Reashelle Braiden April 2013 Qualtrica Associates 2013 Page 0

2 Qualtrica Associates would like to acknowledge: Funding and support for the evaluation was provided by: Immigration and Multiculturalism Manitoba Citizenship and Immigration Canada Special thanks to: Family Centre of Winnipeg The ESS Steering Committee Clients of the Enhanced Settlement Supports Services Content and editing assistance by: Esther Blum Jemima Malabanan William Jurens Qualtrica Associates 2013 Page 1

3 TABLE OF CONTENTS EXECUTIVE SUMMARY. 3 INTRODUCTION...5 METHODOLOGY...6 SUMMARY OF FINDINGS...7 CLIENT OUTCOMES 9 THE ENHANCED SETTLEMENT SUPPORTS SERVICE.18 SYSTEM MOBILIZATION 20 ENHANCED SETTLEMENT SUPPORTS MODEL..25 CONCLUDING COMMENTS AND RECOMMENDATIONS.. 29 APPENDIX I.. 30 APPENDIX II...32 Qualtrica Associates 2013 Page 2

4 Executive Summary Qualtrica Associates recently undertook an evaluation of the Enhanced Settlement Supports (ESS) pilot project, which was initiated in April 2011 to provide services to refugees with high settlement needs and to augment existing settlement supports for refugees in Winnipeg. The objective of the evaluation was to provide both an assessment of, and future recommendations for, the ESS Services. This report focuses on the findings from April March A qualitative methodology was employed in order to determine client outcomes, the strengths and limitations of the ESS service, the performance of the advisory structure and the model, and offer suggestions for improvement. Year 2 incorporated data collected from the following: 11 interviews 1 with clients (some individuals, some couples 2 ) 1 focus group interview with ESS case coordinators 1 group interview with ESS management 4 individual and focus group interviews with Steering Committee Collaterals (SCC s) 9 individual and focus group interviews with external collaterals (ECs) 15 survey responses from Steering Committee members (including SCCs) The Enhanced Settlement Supports Service used a client-centred, case management approach based on the holistic needs of each family. Interviews with the ESS team indicated that case coordinators responded to each client s individual needs using needs-assessments and client feedback to determine service provision. In general, the initial ESS Service Model was validated with only modest changes. Both the ECs and clients themselves credited settlement and integration successes to ESS involvement. ESS provided the services needed, including advocacy, goal setting, assistance with day-to-day issues, connections with interpreters and referrals to other organizations. The case coordinators assisted clients with some of the same basic settlement issues as other 1 11 clients comprised 22% of ESS clients served. 2 Couples were counted as one client unit. Qualtrica Associates 2013 Page 3

5 agencies (such as housing, connections to EAL classes and help to understand systems) but the service went far beyond these issues. They built strong relationships through home visits and taking time to assist refugees with many daily problems. One client summed it up: In order for you to serve me well, you need to know me well. Clients who received services through ESS reported they are now: 1. Feeling more supported in their settlement 2. Feeling more trusting of service providers 3. Experiencing decreased isolation 4. Working with the ESS team to set goals and meeting them 5. Experiencing a growth of independence and confidence 6. More confident about where to go for help and information and understand the systems and services available Most clients repeatedly expressed that they did not know how they would have managed without ESS s support. In contrast to client and EC comments, which were highly positive, the final feedback from Steering Committee members varied widely in their assessments of both the model and the Steering Committee functions. To the end of the project, there remained questions for a few SCCs about the role of ESS and reasons for its existence in the pilot form. Nevertheless, client and collateral feedback indicate that ongoing support for refugees with high settlement needs is essential. Continuation of ESS is definitely recommended. Qualtrica Associates 2013 Page 4

6 INTRODUCTION Refugees are increasingly admitted to Canada based on their need to be protected rather than on their ability to settle successfully into Canadian society (Canadian Council for Refugees 2009, p. 3; Citizenship and Immigration Canada, October 2008). In Manitoba, service providers have cited the need to improve supports to refugee clients who have high settlement needs due to issues such as trauma experienced abroad, low levels of formal education, or inexperience living in an urban environment. In response to this situation, the Department of Manitoba Labour and Immigration provided funding to develop a framework for an Enhanced Settlement Service Model for Refugees with High Needs in Manitoba. From November 2009 to June 2010, Qualtrica Associates carried out an environmental scan to identify innovative and promising initiatives in the field of resettlement, particularly for refugees with high needs. Consultations with settlement service providers and government personnel in Winnipeg, other Canadian provinces and abroad were also held. The purpose of the consultation sessions was to map out current settlement services in Manitoba, discuss case studies of refugees with high needs, and determine key features of a made in Manitoba Enhanced Settlement Service model 3. The Enhanced Settlement Supports (ESS) pilot project was initiated in April 2011 to provide services to refugees with high settlement needs and to extend existing settlement supports for refugees in Winnipeg. The twelve key features included in the service model highlight the service philosophy, and components of the service. Also included in the model are ongoing monitoring, evaluation, and training of service workers and collaterals. Qualtrica Associates undertook an ongoing evaluation of the service and training implemented during the pilot project (from October 2011 to March 2013). During the first year, ending March 2012, collaterals (3, 4),Steering Committee members and members of the ESS team were interviewed to identify promising practices, document program trends and challenges and provide recommendations for improvement that would contribute to an effective service. In Year 2, April 2012 to March 2013, clients, external collaterals (3, 4),the ESS team, the Steering Committee members, including Steering Committee collaterals were interviewed 3 Qualtrica Associates, An Enhanced Settlement Service Model for Refugees with High Needs in Manitoba, page 1. Qualtrica Associates 2013 Page 5

7 and/or surveyed to monitor settlement outcomes and determine if the ESS service and conceptual model were appropriate for assisting refugees with high needs. This final report focuses mainly on the Year 2 program evaluation including service delivery, client outcomes after connection with the ESS program, system mobilization and finally, the effectiveness of the ESS service delivery model. The information has been collected from the ESS team, Steering Committee Collaterals (SCC) 4, Steering Committee members, external collateral organizations (EC) 5 and clients. METHODOLOGY To determine client outcomes, the strengths and limitations of the ESS service, the performance of the advisory structure and the model, and suggestions for improvement, the qualitative evaluation of the Enhanced Settlement Supports pilot project for Year 2 incorporated the following four components: 1. Client interviews 2. ESS management and staff individual and focus group interviews and a final survey 3. Interviews with external organizations and steering committee organizations (EC s and SCC s) that partnered with ESS to deliver service, and 4. An open ended survey to Steering Committee members Client interviews As part of the ESS evaluation plan, client interviews were conducted to gain feedback from the clients themselves on their settlement outcomes. In order to introduce the evaluation interviews to the client and to obtain consent to share contact information with Qualtrica Associates, it was determined that it was necessary for the ESS case coordinators to contact the clients. A Qualtrica interviewer then contacted the client, sometimes through an interpreter to further explain and arrange the interview process. 4 Steering Committee Collaterals (SCCs) are refugee and immigrant serving agencies that were part of the Enhanced Settlement Supports Steering Committee. Steering Committee Members also include IAM and CIC staff, the ESS management team and Qualtrica Associates staff. See Terms of Reference (Appendix II) for a full list of participants. 5 External Collaterals (ECs) are wider system agencies that worked with ESS to assist their mutual clients. Some referred clients to ESS, others received referrals and still others both referred and received referrals. Qualtrica Associates 2013 Page 6

8 Eleven client interviews were arranged and conducted by a group consisting of an experienced interviewer, a co-interviewer/ note-taker and an interpreter, when required. The interviews took place during the period of July 2012-February Most were in the clients homes at their request, as they often were not familiar with the city, uncomfortable meeting in a public place or had mobility issues. External collateral interviews The ESS management provided to Qualtrica Associates a list of eleven organizations and contact names who were external to those on the Steering Committee with whom they had worked to serve clients. These were the organizations that ESS considered to be their main partners. All eleven collateral organizations (ECs) were contacted. Nine representatives from eight organizations responded and these workers were interviewed between November 2012 and January Interviews and Follow-up Evaluation Survey with representatives from organizations on the Steering Committee In June and July 2012, interviews were arranged and conducted with Enhanced Settlement Supports collateral service providers from within the Steering Committee. Using structured interview guides their feedback was gathered at the midterm point of the evaluation in order to better suggest changes as the ESS project progressed. In March 2013, a qualitative survey was designed and delivered to twelve settlement workers from SCCs to obtain end of project feedback on experiences, challenges and successes in their work with ESS. Survey with the Steering Committee Members In March 2013, a qualitative survey was designed and delivered to fourteen members of the Steering Committee to obtain their feedback on the usefulness of this committee in assisting ESS, and their ideas for changes that might be needed. Responses to this survey were received and analyzed qualitatively. SUMMARY OF FINDINGS The objective of the service delivery evaluation was to provide an assessment of Enhanced Settlement Support Services and to suggest recommendations for future consideration. Feedback from the ESS staff, collateral organizations (including ECs Qualtrica Associates 2013 Page 7

9 and SCCs), the Steering Committee as well as from ESS clients was necessary to gain an understanding of the program quality. From April 2012-March 2013, data was collected from the following: 11 interviews with clients (some individuals, some couples 6 ) 1 focus group interview with ESS case coordinators 1 group interview with ESS management 4 individual and focus group interviews with SCC s 9 individual and focus group interviews with EC s 15 survey responses from Steering Committee members (including SCCs) Whether in interviews or survey data, clients, collaterals, the ESS team and steering committee members responded to questions about the service, how it was working and continuing and the emerging challenges and successes. The sections that follow outline the interviewed client demographics, feedback received from clients and ECs regarding client outcomes. ESS, SCCs and ECs provided the findings delineated in the ESS service components and the system mobilization sections. Demographics of Clients Interviewed Eleven clients were interviewed from a pool of 50 clients who were served by ESS. This comprised 22% of ESS clients served. These clients were referred to ESS from a variety of sources and all were Government Assisted Refugees (GAR). They had been in Canada between 1 and 3 years and had arrived from five different countries of origin. They represented twelve language groups and education levels ranged from primary school to university completion. Eighty-two percent were studying English while the remaining 18% had intermediate or better English skills therefore, no classes were required. Most had many children (82%) and members of some families had disabilities. Half had lived in refugee camps before arrival to Canada and all had family left in their home country, some left children or spouses. The issues and difficulties that the ESS refugee clients experienced before and after arrival to Canada varied widely. 6 Couples were counted as one client unit Qualtrica Associates 2013 Page 8

10 The following chart identifies client demographics 7 : ESS clients presented with a complex and changing set of needs, unique to each. These varied as their settlement and their comfort with the service progressed. Central issues for ESS clients involved varying combinations of the following: language barriers, in English and low literacy in their first language; depression and mental health challenges; single parenting with many children, disabilities; and navigating the system of available services. Client Interview Results In order for you to serve me well, you need to know me well. ESS client According to the clients, ESS continued to provide a variety of services to refugees with high needs in Winnipeg in Year 2 of the pilot project. These services included advocacy, goal setting, help with many day-to-day issues, connections with interpreters and referrals to helping organizations. 7 Though ESS referral criteria stated that clients must have been in Canada for less than two years at the time of referral. At the time of the interviews some clients had been with ESS for more than a year and therefore in Canada for longer than the two years. Qualtrica Associates 2013 Page 9

11 Through sharing their stories, the clients described their experiences of trying to settle in Winnipeg. They spoke of the day-to-day pressures of being a single parent or being a parent of a child with a disability and having to take the bus in a Winnipeg winter, fulfill and navigate system requirements and find affordable housing. They had to learn English and get training in order to be able to work and earn enough money for the family, yet they struggled to find daycare to enable study or work. They experienced what it was like to have low economic status, feelings of isolation, to try to understand and learn about systems that did not operate in their country of origin, all the while simultaneously dealing with trauma and working to reunite family members. All this was overwhelming since their problems were often inter-related and solutions depended on being able to navigate the service and governmental systems. Having someone to ask for help with their daily issues was critical and for most of those interviewed, this someone was their ESS case coordinator. Without ESS support, it would be very difficult for the types of families they serve to adjust. -Comment from EC In interviews with ECs working closely with ESS workers to assist their mutual clients, the ECs saw successes with the families. The clients themselves also shared some of their successes and difficulties in general, and those specific to ESS involvement. Families that receive services through ESS reported they are now: Feeling more supported in their settlement Feeling more trusting of service providers Experiencing decreased isolation ESS clients are working with the ESS team to set goals and are meeting them Clients are experiencing growth of independence and confidence Clients are confident about where to go for help and information and understand the systems and services available The following sections elaborate on clients views of each of the above areas 8. 8 Quotes are approximate since they were recorded in notes and not taken verbatim. Most of the comments were also translated by an interpreter. Some quotes are merged with two to three client comments as some feedback was similar and we wanted to avoid information that might identify a particular person. Qualtrica Associates 2013 Page 10

12 Feeling more supported in their settlement They really want to help you Because of ESS involvement, clients are feeling supported in their settlement and the transition to living in Canada. Some clients said that they could not have managed without their ESS workers. They reported arriving to Winnipeg with very high expectations and experiencing many difficulties leading to feelings of hopelessness about ever being able to settle. One said that without ESS I would have just tried to go home. At least I knew how things worked there. ESS has helped me with every issue I have asked about. If they cannot help directly, I know they will refer me to someone that can. It is hard because we have to stretch our money so far to have enough for food sometimes that we can t even afford winter clothes. [ESS] even helps me with warm clothes for my kids. [Without English] I felt at times like I could not do anything for myself, I needed extra help and ESS has time for you, they really want to help you. Nobody else does this for me Many refugees interviewed experienced mental health issues arising from a combination of factors such as missing or separated family members, trauma from war and torture, sadness about leaving their home and way of life and such. When I met [my ESS worker], I was really depressed. It was so hard for me to even make it to my English class that I love but I knew that if I didn t go to my class, I could not continue to qualify for assistance and then could not afford to pay my rent or buy any food. This made me feel even more stressed and made it even harder to go out. It was a difficult cycle.but when I felt really down and everything was really hard, ESS checked on me. I could talk with [my ESS worker] and my spirits were lifted. Nobody else does this for me. They helped me with the things I couldn t do on my own. One client described ESS advocacy on their behalf, I needed to attend EAL classes but the start date was before the start date of one of my children for school. I had no childcare so asked if I could attend the class but start later. I was told that I Qualtrica Associates 2013 Page 11

13 would be unable to attend the class until the next time it was offered. [The ESS worker] spoke to them for me and then I was able to start the class later. They helped me with the things I couldn t do on my own. As the clients interviewed had been in Manitoba for between one and three years, their English had improved through attendance at EAL classes. Many were beginning to be able to communicate in English independently. For clients who were not comfortable communicating in English, ESS advocated and/or hired interpreters who were very skilled, compassionate and supportive. The interpreters also built trusting relationships with the clients and were considered an additional source of support. Communication with those in the wider systems was critical for clients progress. Feeling more trusting of service providers Clients applauded ESS for helping them to connect with many services. In particular, the referrals to the respite workers at the Family Centre of Winnipeg was critical for building trust. A small team from this service has been trained to work with ESS clients specifically and is supervised directly by ESS workers. Collaterals reported that when the respite worker was in their clients homes, refugees felt supported and thus could attend to other tasks. (See section on trust for elaboration). I really trust her Many of the clients accessed for interviews described feelings of fear and general distrust of people because of their traumatic experiences. Therefore, they had difficulty leaving their homes or leaving their children with others. Some said that they did not know whom they could trust. The ESS worker was one of the first people they felt they trusted since their arrival to Canada. As trust was built, clients shared more of their concerns with the ESS case coordinators; thus, the ESS services often evolved from initial assistance with basic settlement to include more advocacy and connections to counselling. ECs reported that the ability of the ESS case coordinators to make home visits went a long way towards making the families comfortable with ESS workers and other service providers. Through ESS referrals, whose approach is to facilitate relationship building with the external service provider, clients began to accept the connections to services such as respite and counselling. With the ESS introduction, the external collaterals found the referral process was much smoother as the Qualtrica Associates 2013 Page 12

14 clients trusted them more easily because of their positive connection with the ESS worker. One EC described a smoother introduction process and said, Because they (ESS workers) know the families well, when they introduce us, it makes the process so much easier. ESS developed that deeper relationship with the family, especially through the home visits and they even knew the names of the families children. The positive relationships that exist with the ESS team carried over to the collaterals and to the interviewers for this evaluation as well. Throughout the interview process clients conveyed how their positive experiences with ESS created positive experiences with others. We experienced many difficulties before coming to Canada. We could not trust anyone. When we were accepted to come to Canada, we were so excited. We heard it was safe and clean with a lot of opportunity. When we arrived, we also experienced many difficulties. It was so much harder than we expected. We didn t even want to go outside. We had difficulty with neighbours and some bullies at the school. If we were in my home country, I would know exactly what to do in these situations. Here, I ask [ESS] because I really don t know. They always help me when I ask. [My ESS worker] is the first person I trusted in Canada. Through ESS, I now have [a respite worker] to come into my home and help me. I really trust her. Experiencing decreased isolation Sometimes clients went to ESS to deal with stress or depression and this resulted in the sharing of other issues that the ESS worker was able to help with. I went to ESS because I was depressed and really stressed about money as we were not receiving enough on assistance to survive. Once I started talking to [my ESS worker], I started to feel much better. When they got to know my problems, they discovered something was wrong with some original financial forms and they were able to get it fixed for me. Sometimes they let me know Qualtrica Associates 2013 Page 13

15 about programs for people to join and so I have met more people. I used to sit home and just stress but now I am encouraged and am doing much better. ESS assisted clients to deal with their feelings of isolation, frustration and sadness and provided advice and referrals to counselling services when the situation called for more qualified professional services. Clients expressed gratitude to the ESS workers for their follow-up phone calls and regular home visits to check on them, especially in difficult times. Working with the ESS team to set goals and meeting them Our ESS worker has helped us to begin to meet our goals. The ESS team worked with clients to set regular goals that the clients are meeting and further setting goals on their own. Clients mentioned goals of learning EAL in order to be able to get training and find jobs. They also wanted to find good daycare for their children to facilitate these goals. ESS supported clients in their daycare search and advocated for them to have access to EAL classes that fit their schedules. [My ESS worker] has helped me a lot so I now know what I need to do to reach my goals. I want to get training so that I can get a good job and take care of my family. Because I am on assistance, I am unable to get a private daycare but there are no spots in the city daycares. I do not know how long I will have to wait but now that I am able to work, it is difficult to find a job that can accommodate my children s school hours. I was able to get one job but it started at 7:30am and the bus for my kids doesn t even come until after 8:00am. I had to turn it down. I feel stuck sometimes but ESS helps me with anything they can. Another client talked about some of the family s goals and described how the ESS worker helped them by connecting them to organizations to help find cheaper housing, EAL classes and helped them with understanding their bills. When we arrived to Canada, we expected to quickly find jobs and be able to provide for the family. It did not work that way. After a while, we were on social assistance and had trouble finding affordable housing. Our housing was so expensive that we sometimes had to choose between buying the food we needed and Qualtrica Associates 2013 Page 14

16 paying bills. We wanted to learn English quickly so that we could find jobs and work hard to be able to make enough money. We also wanted to be able to find a cheaper place to live and understand our monthly bills. Our ESS worker has helped us to begin to meet these goals. Clients are experiencing growth of independence and confidence Now I can solve some of these problems on my own Clients are experiencing growth of independence and confidence. Some clients interviewed shared that before connecting to ESS, they often just stayed home and worried. Since receiving support and encouragement from ESS, they learned how to get to the places they needed and go out more often. Many liked the times during the week when they could go shopping and enjoyed going to their EAL classes and meeting the other students there. Client comments included: At times, I felt like I could not get anything done on my own. I didn t understand anything and couldn t speak English well. Once I was connected to ESS, the case coordinator helped me to solve some problems. Once I felt like someone wanted to help me and I learned how they were able to address the issues I was having, I began to feel more confident. Now I can solve some of these things on my own, especially now that my English is better. More confidence about where to go for help and information The system is much easier to understand now. At first, some clients were reluctant to bother people with too many of their questions. One client mentioned that s/he needed so much help that s/he did not want to ask the case coordinator for help with everything. Another did not know what services were available to ask for help with. These clients wished that service providers beyond ESS would give more guidance regarding the kinds of help they could provide. On the whole, they were very appreciative of the assistance they received from ESS. After connection with ESS, clients were more confident about where to go for help and information and understood the systems and services available. Clients told stories in indicating that they knew who to ask for help with specific issues. If they remained connected with the general settlement agency, clients explained the reasons to access that organization. Similarly, if they were connected to disability Qualtrica Associates 2013 Page 15

17 services because of a disability in the family, they expressed the types of issues they would request from there. Regarding other day-to-day issues, some clients said, I did not really know who to ask for help until I met [the ESS worker]. One client described an emergency situation in which she called ESS for help: My child came home from school with a pain. I did not know what the problem was and I could not speak enough English to call any doctor. I didn t know anyone so didn t even know who I could call. I decided to call [my ESS worker] to ask what I should do. She came right away with an interpreter to go with us to the hospital. She helped me when there was nobody else. They knew what to do in this emergency. This was really important for me. Most clients interviewed now had knowledge and understanding of what services were available to them and said that ESS had helped them to understand what was necessary to move forward in many situations. I was trying to get into some training but there were so many obstacles. I needed to show that I had daycare but I was waiting for a spot for my child in daycare. I needed to show that I had assistance in place but the timing for them was wrong. Everything seemed to depend on something else that wasn t working out. ESS helped me by sitting down with me to listen to my issues and to explain some of the processes. ESS offered me some advice on how to work with the system. It is much easier to understand now. All of the clients interviewed were satisfied and thankful for the wide-range of services and connections that ESS provided. They commented that the staff have good training but also compassion and special experience enabling them to understand and want to help, particularly those clients with language barriers and difficult experiences upon arrival to Canada. Refugees experiencing isolation stated that one of the most meaningful and helpful factors in their settlement was the ESS case coordinator. Feeling that someone was accessible and willing to assist them was invaluable. Many clients interviewed had few connections in Manitoba. The ESS case coordinators filled the gaps that they experienced not having family or friends around to help them. A majority of interviewees felt that ESS had been able to respond to all of their requests for assistance. Now that they understood the system better and they would call ESS only when they needed additional Qualtrica Associates 2013 Page 16

18 resources. The best recommendation (without interviewer prompting) was that ESS clients would happily refer their friends to this service. Client comments about settlement experiences outside ESS The previous section on client outcomes focuses on comments about ESS directly. However, in client interviews, they also reflected upon their settlement experiences and on other services not in the control of ESS. These remarks are summarized below. Many clients are having difficulty living on the amount allotted to them for assistance. They mentioned sometimes having to choose between paying bills and buying groceries. Difficulty finding affordable housing in Winnipeg puts further strain on their budget and leaves little for even the basic necessities. Daycare was one of the most difficult barriers for the clients we interviewed, once their most basic needs were met. Many aspects related to settlement, such as learning English, getting training and finding a job, to name just a few, rely on having childcare for young children. Because clients have little money for food, they often rely on food banks for which they are grateful. However, food banks do not always have stocks of food that are culturally appropriate or fit with religious requirements. Some clients thought that better preparation before coming to Canada was necessary. Their expectations upon arrival were very high. They expected safe, welcoming communities, ample job prospects, good schools and opportunities for the children. Upon arrival, they found a much different situation. In some cases, having high expectations made it even more difficult to settle. Clients reported that a very helpful aspect of their settlement was meeting someone from outside the service community who became a friend. Some of the clients had met an individual who introduced them to others from their cultural or religious community and who helped them understand the system. These clients felt the most supported in their settlement and felt that this had a large impact on their experience since arrival. EAL classes were noted by clients as necessary for reaching their goals, and provided an opportunity to make friends with other newcomers. This was Qualtrica Associates 2013 Page 17

19 particularly appreciated by those who had yet to develop strong community connections. Many interviewees had their basic needs met and started to move further along in the settlement process. Although this had different implications for each family, a theme of isolation continued to surface. Clients did not feel safe going out in their own neighbourhoods, had little time for recreation and had few ties in the community. They wanted recreational activities and to establish community networks. Others wanted more information about where they could meet other women and talk about issues. Some clients hoped that in the future an around-the-clock service would be available, as many issues and emergencies arise after business hours. Most of the clients interviewed had access to a computer, possessed at least basic computer skills, including the ability to use communication tools such as Skype. This helped them to feel more connected with family members who lived elsewhere. However, many did not have the language skills to be able to search for local community events or service information online. Service Delivery One goal of the service evaluation was to better understand how the Enhanced Settlement Supports service worked. Interviews with the ESS team yielded themes that included information about the approach, caseload and follow-ups. These areas are addressed in the following section. The Enhanced Settlement Supports service used a client-centred, case management approach based on the holistic needs of each family. Interviews with the ESS team indicated that case coordinators responded to each client s individual needs using needs assessments and client feedback to determine services required. In follow-up surveys and interviews with ECs, SCCs and Steering Committee members ESS was complimented for its service coordination. Their ability to identify barriers and skill in facilitating client progress through referrals and supportive counselling was also noted. Also identified were many specific issues for refugees that could be best handled by the ESS service. These included helping those dealing with barriers compounded by family and parenting issues, low literacy, and health concerns. In fact, any issue or combination thereof could be addressed by ESS. Qualtrica Associates 2013 Page 18

20 Early in the pilot, there was some confusion among Steering Committee Collaterals (SCCs) as to how ESS differed from pre-existing settlement services. Client and external collateral feedback clarified these differences. Client feedback indicated that though ESS assisted its clients with some of the same basic settlement issues as other agencies (such as housing, connections to EAL classes and help to understand systems) the ESS service went far beyond those issues. The ESS approach to addressing basic needs through home visits, accompanying clients to appointments and offering other client-centred services, assisted to build strong relationships with the clients. Once the clients realized that ESS would take the time to help them with their needs, many began to feel comfortable and confident that they could trust ESS staff. Supportive counselling could then be accepted to identify problems, set goals and interventions. The supportive counselling assisted refugees to develop better interactive skills, resolve conflicts and encouraged positive mental health and more successful settlement and integration. Expert referrals to specialized community services helped to continue this process. The ESS clients benefitted from a team of knowledgeable, competent and caring case coordinators and management. The team used a strong collaborative approach and supported one another in their work with refugees having high needs. The clients often knew and trusted both the case coordinator and the program manager. The ESS management thoroughly supported their team. The team reciprocated. Professional development in conjunction with the necessary skills, flexibility and caring enabled the ESS team to meet the diverse and complex settlement needs of refugees in Winnipeg. The case coordinators and management fluency in eleven languages often enabled communication in the client s first language. In addition, ESS also built a team of trusted and reliable interpreters who could work with ESS to assist the clients with day-to-day issues when ESS staff lacked first language capability. Caseload ESS was at its capacity of 50 cases at the end of Year 2 of the pilot. The case coordinator client load varied. New referrals were more time consuming and required frequent contact early on in the process. As goals were met and clients became more independent, contact decreased. Other client situations were complex and required continued involvement of the case coordinator. A system of weighted caseload assignments was implemented and supervised by ESS management. Language fluency was also an additional factor in client assignment. Qualtrica Associates 2013 Page 19

21 To ensure that ESS was able to accommodate new referrals, some of the older cases were closed because they had moved or no longer accessed the ESS services regularly. Before closing the client files, the case coordinator reviewed the goals reached and progress attained with the client if possible. Clients were advised they could return at any time if future assistance was required. The small caseload allowed the ESS workers to conduct home visits, accompany clients to appointments and conduct intensive follow-up as necessary. ECs noted that the smaller ESS caseload and dedicated case coordinators allowed ESS workers to spend time with the families, and to understand and assist them with a widevariety of issues. A client comment confirms this: They helped me with many specific issues. Others did not have time for all this.-ess client Follow-up with clients and with collaterals The findings from earlier research conducted by Qualtrica Associates revealed that newcomers including refugees would have appreciated continued follow-up and outreach by agencies beyond the initial settlement contact (Qualtrica Associates, Newcomers Experiences and Needs Related to Settlement, 2007). ESS provided regular follow-up as needed. Case coordinators and managers reported that they would contact the client more often in times of trouble. Through phone calls, in-person meetings and home visits, they provided information and determined whether additional referrals or advocacy activities were necessary. External collateral service providers appreciated the communication by the ESS team through regular phone and contact. As one EC phrased it, I speak with [the case coordinator at ESS] frequently about what is happening with the family so follow-up just occurs naturally. System Mobilization A key feature of the Enhanced Settlement Supports Service Model included the value of mobilizing various systems to help clients in their settlement process. The themes that emerged from the interviews with collateral agencies were focused on referrals as well as communication and collaboration. The following section elaborates. Qualtrica Associates 2013 Page 20

22 Referrals The evaluation asked questions of SCCs and ECs interviewed to determine how referrals, both to and from ESS were working. In Year 1, referrals addressed incoming clients from the settlement agencies. In Year 2, the referral feature also included outgoing referrals from ESS to the wider systems. Some SCC s concerns about ESS role definition, client consent and first language services contributed to fewer referrals from these organizations in year 1. In response, the ESS service model was adjusted to allow referrals from sources other than the primary settlement organizations. Most referrals in Year 2 came from social workers, hospitals, education staff and others. The maximum caseload of 50 as set out in the ESS key features model was quickly reached, indicating that there were refugees identified by others in the mainstream systems who required additional assistance. The opening up of referrals provided additional opportunities for vulnerable clients to be identified in order to help ensure that the families needing services did not continue to experience barriers to their settlement without assistance. In order to ensure that service was being provided to all refugees with the highest settlement needs, Privately Sponsored Refugees (PSRs) were also included in the eligibility criteria. In interviews with SCCs and in follow-up feedback, those SCCs that referred indicated that they were unable to accommodate the complex and continuing requests and so introduced the family to ESS. However, this was a very small percentage of their caseload, their workload was not significantly reduced by ESS involvement. On the other hand, ECs indicated that ESS helped to reduce time constraints when working with refugees. The following description of the change noticed once a family was connected to ESS summed it all up: When meeting with families, they [the families} sometimes requested me to assist them with day-to-day issues such as help interpreting or filling out forms or accompaniment to appointments. These requests sometimes took focus away from my specific mandate in the client s case, however, once they were connected to ESS, they started to know who to ask for what and ESS began to take care of the day-to-day issues. This allowed me to focus fully on my specific function while ESS took care of the holistic needs of the family. (Blended quote from three ECs) Qualtrica Associates 2013 Page 21

23 Communication and collaboration WITH EXTERNAL COLLATERALS The ESS team assisted families to connect to services as necessary. They built closer ties to existing collateral contacts and accessed many additional services to accommodate the needs of their clients. Through contact with the ESS staff, service providers from areas such as mental health, education, medical, and employment became more familiar with the ESS services and resources. Strong communication and collaboration occurred between ESS and most ECs to assist their mutual clients. ECs reported that: Communication between their organizations was regular and frequent through regular phone and contact. The central location of the Family Centre and openness of ESS staff made it very easy for them to connect with the ESS workers. ESS had the ability to link people with necessary services such as in-home support. For example, respite services helped clients a great deal in their settlement. The referral process was facilitated because the clients trusted the referral. First language ability and access to interpreters was commended. ESS offered close support to their clients. They had the time to deal with social, health, day-to- day issues and helping people to understand how systems work in Manitoba. Once some of the day-to-day issues were addressed, people become more independent. In addition, ESS served an important coordination function as noted in an EC comment below: They [ESS] are very good at the case coordinator role. They orchestrated everyone involved with this case and got them together to talk about what needed to be done and what had already been done this has helped my work and they are so determined to help the families. They get all the organizations together to meet and work to help in any way necessary. Qualtrica Associates 2013 Page 22

24 The work of ESS was described as invaluable. ECs have seen ESS as a great help to their mutual clients, as well as to themselves as service providers. One EC shared a successful experience working with ESS: It was great to have an ESS worker accompany me with the family to a.appointment. They knew [them] well and spoke the language. They got [the issue] resolved and it all worked out very well. They even helped to fill out the necessary forms. WITH STEERING COMMITTEE MEMBERS AND STEERING COMMITTEE COLLATERALS Steering Committee Collaterals were interviewed at the beginning of Year 2. A follow-up evaluation survey was delivered to both Steering Committee members and SCCs in the final month of the pilot project. There were 15 respondents to the Service Evaluation follow-up survey. Of these 15, 9 were Steering Committee members. The Steering Committee members included SCCs 9. Originally, the evaluators were to participate and collect information from working groups within the Steering Committee. Since most meetings were held with the agencies separately or collaboratively on an as needed basis, the evaluators participation and therefore comments, were limited. In the follow-up survey, questions were asked about whether the pilot met most or all of its objectives for service delivery in the areas of developing an enhanced and coordinated service model, designing and implementing the service, establishing a sustainable service delivery and evaluating the service. Seven of the 8 Steering Committee members who responded indicated that it had. [ESS has been] Very successful. Clients are receiving the type and level of support and follow up required to facilitate moving towards independence for refugees facing multiple barriers to successful settlement and integration. For the remainder of the questions about the ESS and the Steering Committee process, the responses varied widely. Comments included: finding the process confusing and bumpy, ESS being a duplication of already existing services; that ESS provided a much needed enhanced service to refugees with high-needs. 9 The follow-up survey responses did not always allow for differentiation of the Steering Committee members and SCCs. Qualtrica Associates 2013 Page 23

25 A lot of time, energy and money has been spent on introducing.a system that was already operating. I find the whole process to be very confusing. (Blend of 2 similar quotes) For the most part, both the ESS team and the settlement agency workers intervened collaboratively in the best interests of the client. In interviews and in the follow-up survey, issues with communication and service coordination were mentioned. Some SCCs commented on problems with role definition, referrals and duplication of funds and services. Throughout the project, some relationships in the Steering Committee remained difficult, despite the considerable effort made during meetings and between meetings to discuss and clarify these issues and roles. Nevertheless, the effects of Steering Committee challenges were not evidenced in client satisfaction of the ESS service. Summary Comments Client and EC feedback indicate that there is definitely a need for ongoing support for refugees with high settlement needs in Winnipeg. As one Steering Committee member described it, Refugees need a trusted contact with whom they can discuss their settlement and integration issues someone to consult with about the.problems they encounter daily, to bridge their cultural understanding, to explain how [systems] work [in Canada] and to help them [find] strategies for coping. By all client accounts, ESS is the only service providing this holistic approach to their issues. Clients are feeling more supported in their settlement, are more confident and are beginning to address issues independently. As clients repeatedly expressed, they do not know what they would have done without ESS s support. Qualtrica Associates 2013 Page 24

26 The ESS model The enhanced settlement supports model, as illustrated in the diagram below, originally included 12 key features. Since these components guided the development of the ESS Service, the evaluation data was examined to determine the effectiveness of this model. Targeted Training Activities Focus on refugees with high-needs Clientcentred philosophy Ongoing Monitoring and Evaluation Collaborative Practice Enhanced Settlement Service Steering Committee 12 Key features of ESS Early Intervention Responsive Referral and Intake System System Mobilization Follow-up with Clients Manageable Case Loads Enhanced Settlement Coordination Unit Qualtrica Associates 2013 Page 25

27 Findings Regarding the ESS Model from the evaluation data: 1. Focus on refugees with high needs The clients of the Enhanced Settlement Supports program were refugees with highsettlement needs such as having a large family, mental health issues, disability of a family member and/or low literacy. The combinations of these issues required the involvement of many service providers. It also required interpreters. Because the clients were new to Winnipeg and had few, if any supports, ESS was necessary to fill this gap. The refugees turned to ESS to understand the system and to resolve their problems. 2. Client-centred philosophy ESS used a client-centred approach to determine how to respond to refugees issues. Home visits, time and individual attention to each client s unique needs were vital to the development of trusting relationships and discovering and preventing future problems. The trust that ESS established carried over to other service providers as well as to the evaluation team. 3. Collaborative Practice ESS strengthened and developed new relationships with organizations in the mainstream service systems along with those serving refugees. ESS staff connected regularly with these organizations, following up through phone or contacts. This has resulted in mutual learning. ESS has expanded their helping networks and the ECs have learned about how best to work with refugees. 4. System Mobilization Collaboration with mainstream organizations facilitated relationships with key change agents when clients encountered difficult situations. This enabled ESS workers to better assist and respond to clients in these circumstances. The data showed that ESS had connected to approximately 170 collateral organizations to help their clients. 5. Enhanced Settlement Coordination Unit One ESS manager, a clinical supervisor, four case coordinators and a strong trusted group of interpreters made up the ESS team. The case coordinators were Qualtrica Associates 2013 Page 26

28 experienced professionals able to implement a social work approach to provide focused services and follow-up as appropriate. They built solid connections with the clients in order to help them develop and reach their goals. The nature of the pilot funding resulted in staff turnover as some workers left for more stable employment. This situation was difficult for both clients and the ESS team. 6. Manageable Caseload In order to best serve refugees with high settlement needs caseloads were maintained at lower than the professional norm. A system of weighted caseload assignments was implemented to maximize efficiency. The ESS team had time to follow-up with clients, conduct home visits, make referrals, accompany clients to appointments and assist them with presenting issues. Administrative activities such as recording and tracking activities for the pilot project in conjunction with regular client file maintenance reduced time available to serve clients. 7. Follow-up with Clients ESS staff maintained regular contact with clients through home visits and phone calls, following up more often if clients were experiencing difficulties. Clients appreciated this additional attention. As the relationship with the case coordinator became more established, clients were able to share concerns before they became crises. This preventative strategy probably reduced the need for intervention by the larger system. 8. Responsive referral and intake system This feature initially referred to incoming referrals from the settlement agencies. In the second year of the pilot, ESS accepted referrals of refugees from a wide range of collateral organizations such as Health, Education, and Employment, thus expanding their networks. ESS staff subsequently could refer clients to wider variety of agencies. 9. Early Intervention Initially clients were referred to the service within one year of arrival. In the model development, it was assumed an earlier intervention to assist refugees with high settlement needs would be advantageous. However, for various reasons, including that some clients have a protracted period of settlement, issues requiring ESS involvement did not arise until later. Therefore, the ESS client intake was extended Qualtrica Associates 2013 Page 27

29 to two years after arrival and it was decided that clients would be referred based on need rather than length of time in Canada. 10. Enhanced Settlement Service Steering Committee The Steering Committee met during the second year of the pilot to update the group on activities and share successes and challenges faced by each organization. The final feedback from Steering Committee members in survey responses varied widely in their assessments of both the model and the Steering Committee. Most respondents felt that the pilot project met its objectives to design and implement an effective service. However, some noted that the relationships among the service providers who were on the Steering Committee were at times strained. This may have influenced the referrals coming to ESS and coordination with the settlement serving agencies but it did not have an impact on client satisfaction. 11. Ongoing Monitoring and Evaluation The ongoing evaluation activities included regular meetings as well as a total of 49 individual and group interviews with clients, Steering Committee collaterals, external collaterals and the ESS Team. Surveys were also distributed to Steering Committee members and Steering Committee collateral agencies to gain final feedback on the successes and challenges of the project. 12. Targeted Training Activities The training in Year 2 successfully delivered sessions to service providers, including some ECs. The training helped increase awareness of refugee issues and provided alternative ways of working with refugees. It improved the way work was carried out with high needs refugees in many service sectors. The Trainer s Report and the Training Evaluation Report covers this material in more detail. Qualtrica Associates 2013 Page 28

30 Concluding Comments and Recommendations Throughout the pilot, lessons were learned and improvements were implemented. There were some recurring problems and evident successes. Recurring problems: Staff turnover was a concern. When staff change, families and organizations must establish new connections. In situations where trust building was an issue for the family, this created a very difficult transition. Administrative tasks added an additional component to the work. Should these be continued at the same intensity as in the pilot, service hours will be less than optimum. Within the Steering Committee, to the end of the project, a few members remained confused about the role of ESS and reasons for its existence. Evident Successes: ESS was flexible and adjusted services as needed. External Collaterals and clients were clear on the need for ESS services and its role. Client feedback was positive and many individuals expressed thanks to ESS for the assistance and support that was provided. The ESS model was validated, with only modest changes. The Service evaluation clearly endorsed the need for enhanced settlement support for refugees with high needs. Continuation of ESS is recommended. Qualtrica Associates 2013 Page 29

31 Appendix I Enhanced Settlement Service Pilot Project for High Needs Refugees Steering Committee Terms of Reference (DRAFT) Role and Scope: The Steering Committee will: assist Manitoba Labour and Immigration in guiding the development and implementation of the Enhanced Settlement Service Pilot Project for High Needs Refugees provide a venue for sharing progress/status reports on project component activities, information sharing and general service updates The pilot project is comprised of 3 components*: 1) Implementation of an enhanced and coordinated service model to support refugees with high needs 2) Training and capacity building for government and community service providers to increase their understanding and capacity for working refugees 3) Pilot Project Evaluation *Note: each component area has a smaller working group to work on specific activities related to that project component (descriptions below) Meeting format: Progress/summary reports on each component (working group reports) Roundtable information sharing from participating programs (e.g. stats, experiences, trends) Participants: Liz Robinson (Chair), Manitoba Labour and Immigration, Integration Services Catherine Cooke, Manitoba Labour and Immigration, Integration Services Amanda Kletke-Neufeld, Manitoba Labour and Immigration, Integration Services Carlos Vialard, Manitoba Interfaith Immigration Council Welcome Place Marta Kalita, Manitoba interfaith Immigration Council Welcome Place Boris Ntambwe, Accueil Francophone Millie Braun, The Family Centre Jennifer Perron, Society for Manitobans with Disabilities Jade Nayler and Sheri Larsen-Celhar, Winnipeg Regional Health Authority, Bridge Care Clinic Connie Jonasson, Citizenship and Immigration Canada Qualtrica Associates Tracy Moore / Irma Nadeau, Family Services and Consumer Affairs (new) Qualtrica Associates 2013 Page 30

32 Consultative Resource Supports: Consultation with representatives of other organizations and stakeholders may take place as needed Individuals may be contacted on an ad hoc basis and invited to attend meetings when agenda item(s) pertain to their areas of expertise and/or when their expertise would support the project Frequency of Meetings: The Steering Committee will meet 3 times annually or at the call of Manitoba Labour and Immigration as required (Spring/Fall/Winter) Project Component Descriptions 1) Implementation of an enhanced and coordinated service model to support refugees with high needs develop and implement service delivery model including needs assessment and referral processes, service approach and service coordination develop capacities and partnerships required for a coordinated approach assist with evaluation of pilot service delivery Members: Family Centre, Welcome Place, Accueil Francophone Society for Manitobans with Disabilities, Manitoba Labour and Immigration 2) Training and capacity building for government and community service providers to increase their understanding and capacity for working with refugees determine priority groups for training conduct training needs assessment and determine priorities for each targeted training group develop and deliver training sessions evaluate training Members: Manitoba Labour and Immigration, Qualtrica Associates, Consultation with partners as needed 3) Evaluation of Pilot Project Complete overall evaluation of the Enhanced Settlement Service Pilot Project Develop and implement program performance measurement and evaluation of enhanced service client outcomes and service coordination Identify promising practices and provide recommendations for service improvement and development Evaluate effectiveness and impact of training Members: Manitoba Labour and Immigration, Qualtrica Associates, Consultation with partners as needed Qualtrica Associates 2013 Page 31

33 APPENDIX II Background ENHANCED SETTLEMENT SUPPORTS SERVICE EVALUATION SURVEY 1. Have you shared work with refugees or referred them to the Enhanced Settlement Supports (ESS) services at the Family Centre of Winnipeg? Yes (SKIP TO QUESTION 3) No (CONTINUE TO QUESTION 2) o Comments 2. What was the reason for not referring to the Enhanced Settlement Supports services? Please select all applicable choices. (SKIP TO QUESTION 12) Our services were sufficient None of my clients fit the referral criteria I did not know the referral criteria Referral to other services was indicated Other o Other (please specify) Referrals 3. How many refugees have you either referred to or collaborated with the Enhanced Settlement Supports program? If known, # of single people # of couples without children # of families with children Additional comments 4. What criteria did you use to guide your referrals to ESS? 5. Earlier evaluation described some barriers to referrals. Have you experienced these? Yes (CONTINUE TO QUESTION 6) No (SKIP TO QUESTION 7) Somewhat (CONTINUE TO QUESTION 6) Qualtrica Associates 2013 Page 32

34 6. Some early barriers reported were as follows: client consent to communication, role definition clarity, and the ESS service was an unknown. Have these barriers been overcome? Describe how. If you experienced any additional barriers, please describe these and comment on possibilities for addressing them. Collaboration 7. Describe some of your successes in collaborating with the Enhanced Settlement Supports team? 8. Describe some of your challenges collaborating with the Enhanced Settlement Supports team? Clients 9. From your experience, describe the benefits for refugees emerging from Enhanced Settlement Supports involvement. 10. From your experience, describe the challenges for refugees emerging from Enhanced Settlement Supports involvement. 11. Do you think that refugees who most need additional support have been able to access ESS? Steering Committee Members 12. Are you a member of the Enhanced Settlement Supports Steering Committee? Yes (CONTINUE TO QUESTION 13) No (SKIP TO QUESTION 17) For ESS Steering Committee 13. As a member of the ESS Steering Committee, do you think that the pilot project met its objectives for service delivery as follows? To develop an enhanced and coordinated service model To design and implement the service To evaluate the pilot service delivery To establish sustainable service delivery Qualtrica Associates 2013 Page 33

35 All Most Some Very little Not at all Don t Know o Please elaborate 14. As a member of the ESS Steering Committee, do you think that the pilot project met its objectives for training and capacity building? To develop and deliver training for service providers in the community and government; To determine training needs and priorities in the community and government services; To deliver applied training using mixed approaches; To evaluate training All: Most: Some: Very little: Not at all: Don t know o Please elaborate 15. On the whole, comment on the successes of the Enhanced Settlement Supports pilot project for: Service Delivery Training and Capacity Building Any additional comments 16. On the whole, comment on any continuing challenges of the Enhanced Settlement Supports pilot project for: Service Delivery Training and Capacity Building Any additional comments 17. What are the issues that refugees bring that can be best handled by ESS? 18. Do you have any other comments or recommendations regarding the Enhanced Settlement Supports program? Qualtrica Associates 2013 Page 34

36 Qualtrica Associates 2013 Page 35

37 Qualtrica Associates 2013 Page 36

38 Qualtrica Associates 2013 Page 37

39 Qualtrica Associates 2013 Page 38