Success Story 1
Improving Opportunities for Street-Involved Youth: Tweaking an Already Strong Program (2007). In Ontario, an increasing number of youth are homeless, living on the streets, and are at risk of never reaching their full potential. EVIDENCE worked with one organization, SKETCH, devoted to assisting street-involved and homeless youth ages 15 to 29, by engaging them in the arts in a cross-disciplinary, studio environment. SKETCH offered a skill development and counselling program focussed on artistic activities to build youth’s technical and employability skills, and provide mentorship and work experience. EVIDENCE involved staff, program directors, board members, the executive director and program participants who worked together to assess how the program was meeting its goals and objectives. It was clear that the program improved the sense of trust among youth, reduced employment barriers and increased their opportunities to find work, further their education and engage in community life. The program also developed youth’s leadership and employability skills. In addition, the evaluation highlighted the unique features of the program such as its use of art as a tool for creative learning, its emphasis on active participation, and its community engagement.
Success Story 2
Proving Impact on a Shoestring: An Evaluation of a Local YWCA (2007). The YWCA has been offering a broad range of women’s and girls’ programs across Canada for over 135 years. Each local YWCA offers a unique blend of programs geared toward its community’s individual needs. EVIDENCE worked with the Community YWCA of Muskoka to verify how successful it had been in meeting its program goals and if it could continue to develop effective programming while building organizational sustainability. One of the unique challenges faced by this relatively young YWCA was that it operated with minimal part-time staff who had little time to devote to evaluation. EVIDENCE’s first step was to determine both the YWCA’s strategic and program objectives and to develop concrete, measurable outcomes based on program goals. EVIDENCE was able to show the YWCA how it had a direct impact on its community through reducing participants’ isolation, increasing their social support and self-awareness and improving their ability to make choices. EVIDENCE was also able to obtain recommendations from program participants and staff, staff from collaborating organizations and members of the Board of Directors to share with the YWCA.
Success Story 3
Contributing to the Knowledge Base: Use of the Brief Child and Family Phone Interview in Working with Youth with a Dual Diagnosis (2007). While in most instances an overall program evaluation is required to uncover and address issues, there are often times when only one aspect of an agency’s procedures needs review. This was the case for the Griffin Centre, an accredited children’s mental health centre in Toronto with a unique focus on providing services to youth with complex needs associated with developmental delays. Staff at the Centre had reported that the Ministry-mandated initial phone interview used during intake was not appropriate to youth with a dual diagnosis. They thought the intake instrument did not provide information that was relevant to their work nor did it provide the complete picture needed to design an effective client service plan for youth with both mental health issues and a developmental disability. Throughout the process, EVIDENCE worked closely with a stakeholder committee made up of staff and clients. The results of the evaluation confirmed that although staff did not find the intake instrument relevant to their work, they nonetheless thought it was useful. They also thought they needed further training, support and opportunities to learn how the instrument could be relevant to their work. The evaluation verified how the instrument required staff to bring their experience to bear on the information it provide about clients. Based on this assessment, EVIDENCE recommended that staff be provided with better information about the intake instrument as part of a staff training that would clarify expectations and improve their understanding of its use during the intake process.
Success Story 4
Flexibility is Key: Evaluation of a Parenting Program for a Marginalized Urban Community (2007). Reaching out to marginalized communities can be challenging when members are isolated due to religious and cultural reasons, and they are also mistrustful of the social service system. When EVIDENCE approached Daryeelka Qoyska: Somali Outreach and Education Project, a unique parenting program for Somali families offered by Yorktown Child and Family Services in Toronto, the staff welcomed the input but was unsure if clients would be willing to offer much information. However, in this situation, flexibility was the key to a successful evaluation that ultimately turned out to have value on a broader scale than expected. After meeting with both clients and staff, EVIDENCE identified the needs of participants, their families, and the cultural community, the effectiveness of the program in meeting those needs and requirements to achieve success. Issues such as isolation, lack of information, lack of youth outreach, and fathers’ absence all created programming challenges. EVIDENCE provided detailed recommendations to overcome these challenges related to Yorktown’s role and that of its community partners, its service delivery and interventions, the target population, and the ecological context in which the program was delivered.
Success Story 5
Maintaining Cultural Identity While Improving Linkages with Existing Mental Health Services (2007). Toronto has many different ethno-racial communities, each one is faced with unique challenges as they struggle to fit while maintaining their identity. EVIDENCE worked with Sabawoon Afghan Family Education and Counselling Center (SAFE), who served Afghan youth experiencing mental health and adjustment issues, and who needed support accessing education and employment services. Given the tight evaluation budget, stakeholders were directly involved in the focus and design of the evaluation as well as in data collection. It became apparent that this group in particular was faced with many issues. Clients and families viewed mental illness as a stigma, clients did not know about their options for education or how to access services, they often experienced culture clash within their families and had diverse mental health issues and needs. These challenges were viewed through the lens of other programs SAFE provided with special consideration to their outcomes. Based on this examination, EVIDENCE developed a set of recommendations jointly with stakeholders that focused on increasing staff training opportunities, improving community partnerships, in particular the connection with medical professionals and school staff, improving intake procedures, defining service parameters and expanding services where possible.
Success Story 6
A Stronger Program Leading to Stable Funding: An Evaluation of a Peer Counselling Service (2007). Although Youth Line had been providing telephone peer counselling and information to lesbian, gay, bisexual and transgendered youth across the province of Ontario for over 10 years, the prospect of an evaluation raised a few concerns. For a few years, funding issues had taken priority even though programming at this youth-led organization had grown in response to different needs expressed by clients. Youth Line needed to step back and better understand the kinds of services they were providing. EVIDENCE’s participatory approach ensured that staff members were full partners in the evaluation process and gave them the opportunity to bring services front and centre. EVIDENCE was able to provide Youth Line with a template that allowed staff to systematically examine its services and ultimately create a foundation that helped stabilize the organization. As a result of the evaluation, staff learned that their services were essential and valued, and acquired the confidence they needed to seek new funding. With solid information about program outcomes based on a thorough and positive evaluation, Youth Line successfully obtained much-needed increased and permanent funding.
Success Story 7
Closing Feedback Loops: An Evaluation of a Local Youth Volunteer Network (2007). There are many levels of feedback that are necessary to evaluate how well a program performs. Designing an evaluation that ensures all available feedback is received can be a complicated process that requires a careful look at the type of program being delivered, the perspective of clients and the perspective of program providers. In the case of one local youth volunteer network, the Youth Volunteer Corps offered by the Employment and Education Centre in Brockville, Ontario, the evaluation helped uncover a missing feedback loop that made a big difference to the success of the program. The Youth Volunteer Corps offered structured volunteer placements for youth aged 14 to 24 providing them with valuable work experience. EVIDENCE worked with participants, collaborating agencies and parents to collect information through case stories, focus groups and surveys. Together, they examined the volunteer activities carried out, the program’s impact on youth and the community, and challenges that were being faced. In this case, the success of the program depended on a unique mix of addressing both participant and community needs. The evaluation provided a clear picture of what was needed in Brockville’s suburban and rural communities and how well the program was meeting these needs. Overall, partnering agencies were receiving good service and the program countered the negative image of youth. However, despite positive outcomes for youth in terms of their increased employability skills, an additional unexpected and valuable message was revealed: youth felt un-noticed and un-appreciated at their placements. By closing this feedback loop, the program was able to better meet youth’s needs.
Success Story 8
Developing Multi-Level Objectives to Improve Academic Opportunities for At-Risk Youth (2006). Many at-risk youth fail academically because they experience poverty, lack of social opportunities, disinterested teachers and community services that are irrelevant to their needs. Ensuring these youth have a fair chance at success is a difficult and demanding goal for organizations that provide them with the encouragement they need to develop, pursue and achieve their academic goals. EVIDENCE evaluated one such educational program developed in partnership between YOUTHLINK and Centennial College in Scarborough, Ontario. The purpose was to understand the processes and activities needed to achieve the goal of providing youth with educational and employment upgrading. EVIDENCE determined that the program had one basic yet extremely challenging goal which was to improve youth’s verbal and written comprehension. Further, the program also had two basic strategic goals which were to attract youth into the program and keep them there. In order to achieve these goals, EVIDENCE identified that the two most important program objectives were to provide both academic and social support to students. EVIDENCE recommended short term-actions for program development as well as mid-term actions that addressed deeper and more challenging issues such as youth’s lack of social support and awareness about how the program could work for them.
Success Story 9
Improving Accountability and Staff Safety at a Women’s Shelter (2006). Women-centred community programs including shelter services had been the long-time focus of one local women’s social service organization in rural Ontario. Over time however, consistent policies and procedures had lapsed at their shelter, resulting in a lack of trust among staff and more immediate concerns about their own safety. EVIDENCE’s approach was to thoroughly examine the shelter’s staff training program which had been recently implemented and provide solid, program-based recommendations. EVIDENCE’s first step was to engage all members of their community in a careful review of the shelter’s services and issues. EVIDENCE discovered that shelter staff lacked confidence in the services they were providing to their clients and were also were concerned about their own personal safety. As a result, many staff members were feeling stressed, isolated and oppressed, and their quality of work was clearly suffering. With no clear benchmarks to evaluate staff performance, management was powerless to address the situation and the volunteer Board of Directors was experiencing a distinct lack of accountability. The most significant recommendation made was that a committee of front-line staff, management and Board members be established to develop clear policies, procedures and program guidelines that would be reviewed and updated on an ongoing basis. In addition, EVIDENCE stressed that the shelter create a learning environment that was supported by front-line staff and management. All recommendations were developed and validated by stakeholders and resulted in improvements to staff safety and morale.
Improving Opportunities for Street-Involved Youth: Tweaking an Already Strong Program (2007). In Ontario, an increasing number of youth are homeless, living on the streets, and are at risk of never reaching their full potential. EVIDENCE worked with one organization, SKETCH, devoted to assisting street-involved and homeless youth ages 15 to 29, by engaging them in the arts in a cross-disciplinary, studio environment. SKETCH offered a skill development and counselling program focussed on artistic activities to build youth’s technical and employability skills, and provide mentorship and work experience. EVIDENCE involved staff, program directors, board members, the executive director and program participants who worked together to assess how the program was meeting its goals and objectives. It was clear that the program improved the sense of trust among youth, reduced employment barriers and increased their opportunities to find work, further their education and engage in community life. The program also developed youth’s leadership and employability skills. In addition, the evaluation highlighted the unique features of the program such as its use of art as a tool for creative learning, its emphasis on active participation, and its community engagement.
Success Story 2
Proving Impact on a Shoestring: An Evaluation of a Local YWCA (2007). The YWCA has been offering a broad range of women’s and girls’ programs across Canada for over 135 years. Each local YWCA offers a unique blend of programs geared toward its community’s individual needs. EVIDENCE worked with the Community YWCA of Muskoka to verify how successful it had been in meeting its program goals and if it could continue to develop effective programming while building organizational sustainability. One of the unique challenges faced by this relatively young YWCA was that it operated with minimal part-time staff who had little time to devote to evaluation. EVIDENCE’s first step was to determine both the YWCA’s strategic and program objectives and to develop concrete, measurable outcomes based on program goals. EVIDENCE was able to show the YWCA how it had a direct impact on its community through reducing participants’ isolation, increasing their social support and self-awareness and improving their ability to make choices. EVIDENCE was also able to obtain recommendations from program participants and staff, staff from collaborating organizations and members of the Board of Directors to share with the YWCA.
Success Story 3
Contributing to the Knowledge Base: Use of the Brief Child and Family Phone Interview in Working with Youth with a Dual Diagnosis (2007). While in most instances an overall program evaluation is required to uncover and address issues, there are often times when only one aspect of an agency’s procedures needs review. This was the case for the Griffin Centre, an accredited children’s mental health centre in Toronto with a unique focus on providing services to youth with complex needs associated with developmental delays. Staff at the Centre had reported that the Ministry-mandated initial phone interview used during intake was not appropriate to youth with a dual diagnosis. They thought the intake instrument did not provide information that was relevant to their work nor did it provide the complete picture needed to design an effective client service plan for youth with both mental health issues and a developmental disability. Throughout the process, EVIDENCE worked closely with a stakeholder committee made up of staff and clients. The results of the evaluation confirmed that although staff did not find the intake instrument relevant to their work, they nonetheless thought it was useful. They also thought they needed further training, support and opportunities to learn how the instrument could be relevant to their work. The evaluation verified how the instrument required staff to bring their experience to bear on the information it provide about clients. Based on this assessment, EVIDENCE recommended that staff be provided with better information about the intake instrument as part of a staff training that would clarify expectations and improve their understanding of its use during the intake process.
Success Story 4
Flexibility is Key: Evaluation of a Parenting Program for a Marginalized Urban Community (2007). Reaching out to marginalized communities can be challenging when members are isolated due to religious and cultural reasons, and they are also mistrustful of the social service system. When EVIDENCE approached Daryeelka Qoyska: Somali Outreach and Education Project, a unique parenting program for Somali families offered by Yorktown Child and Family Services in Toronto, the staff welcomed the input but was unsure if clients would be willing to offer much information. However, in this situation, flexibility was the key to a successful evaluation that ultimately turned out to have value on a broader scale than expected. After meeting with both clients and staff, EVIDENCE identified the needs of participants, their families, and the cultural community, the effectiveness of the program in meeting those needs and requirements to achieve success. Issues such as isolation, lack of information, lack of youth outreach, and fathers’ absence all created programming challenges. EVIDENCE provided detailed recommendations to overcome these challenges related to Yorktown’s role and that of its community partners, its service delivery and interventions, the target population, and the ecological context in which the program was delivered.
Success Story 5
Maintaining Cultural Identity While Improving Linkages with Existing Mental Health Services (2007). Toronto has many different ethno-racial communities, each one is faced with unique challenges as they struggle to fit while maintaining their identity. EVIDENCE worked with Sabawoon Afghan Family Education and Counselling Center (SAFE), who served Afghan youth experiencing mental health and adjustment issues, and who needed support accessing education and employment services. Given the tight evaluation budget, stakeholders were directly involved in the focus and design of the evaluation as well as in data collection. It became apparent that this group in particular was faced with many issues. Clients and families viewed mental illness as a stigma, clients did not know about their options for education or how to access services, they often experienced culture clash within their families and had diverse mental health issues and needs. These challenges were viewed through the lens of other programs SAFE provided with special consideration to their outcomes. Based on this examination, EVIDENCE developed a set of recommendations jointly with stakeholders that focused on increasing staff training opportunities, improving community partnerships, in particular the connection with medical professionals and school staff, improving intake procedures, defining service parameters and expanding services where possible.
Success Story 6
A Stronger Program Leading to Stable Funding: An Evaluation of a Peer Counselling Service (2007). Although Youth Line had been providing telephone peer counselling and information to lesbian, gay, bisexual and transgendered youth across the province of Ontario for over 10 years, the prospect of an evaluation raised a few concerns. For a few years, funding issues had taken priority even though programming at this youth-led organization had grown in response to different needs expressed by clients. Youth Line needed to step back and better understand the kinds of services they were providing. EVIDENCE’s participatory approach ensured that staff members were full partners in the evaluation process and gave them the opportunity to bring services front and centre. EVIDENCE was able to provide Youth Line with a template that allowed staff to systematically examine its services and ultimately create a foundation that helped stabilize the organization. As a result of the evaluation, staff learned that their services were essential and valued, and acquired the confidence they needed to seek new funding. With solid information about program outcomes based on a thorough and positive evaluation, Youth Line successfully obtained much-needed increased and permanent funding.
Success Story 7
Closing Feedback Loops: An Evaluation of a Local Youth Volunteer Network (2007). There are many levels of feedback that are necessary to evaluate how well a program performs. Designing an evaluation that ensures all available feedback is received can be a complicated process that requires a careful look at the type of program being delivered, the perspective of clients and the perspective of program providers. In the case of one local youth volunteer network, the Youth Volunteer Corps offered by the Employment and Education Centre in Brockville, Ontario, the evaluation helped uncover a missing feedback loop that made a big difference to the success of the program. The Youth Volunteer Corps offered structured volunteer placements for youth aged 14 to 24 providing them with valuable work experience. EVIDENCE worked with participants, collaborating agencies and parents to collect information through case stories, focus groups and surveys. Together, they examined the volunteer activities carried out, the program’s impact on youth and the community, and challenges that were being faced. In this case, the success of the program depended on a unique mix of addressing both participant and community needs. The evaluation provided a clear picture of what was needed in Brockville’s suburban and rural communities and how well the program was meeting these needs. Overall, partnering agencies were receiving good service and the program countered the negative image of youth. However, despite positive outcomes for youth in terms of their increased employability skills, an additional unexpected and valuable message was revealed: youth felt un-noticed and un-appreciated at their placements. By closing this feedback loop, the program was able to better meet youth’s needs.
Success Story 8
Developing Multi-Level Objectives to Improve Academic Opportunities for At-Risk Youth (2006). Many at-risk youth fail academically because they experience poverty, lack of social opportunities, disinterested teachers and community services that are irrelevant to their needs. Ensuring these youth have a fair chance at success is a difficult and demanding goal for organizations that provide them with the encouragement they need to develop, pursue and achieve their academic goals. EVIDENCE evaluated one such educational program developed in partnership between YOUTHLINK and Centennial College in Scarborough, Ontario. The purpose was to understand the processes and activities needed to achieve the goal of providing youth with educational and employment upgrading. EVIDENCE determined that the program had one basic yet extremely challenging goal which was to improve youth’s verbal and written comprehension. Further, the program also had two basic strategic goals which were to attract youth into the program and keep them there. In order to achieve these goals, EVIDENCE identified that the two most important program objectives were to provide both academic and social support to students. EVIDENCE recommended short term-actions for program development as well as mid-term actions that addressed deeper and more challenging issues such as youth’s lack of social support and awareness about how the program could work for them.
Success Story 9
Improving Accountability and Staff Safety at a Women’s Shelter (2006). Women-centred community programs including shelter services had been the long-time focus of one local women’s social service organization in rural Ontario. Over time however, consistent policies and procedures had lapsed at their shelter, resulting in a lack of trust among staff and more immediate concerns about their own safety. EVIDENCE’s approach was to thoroughly examine the shelter’s staff training program which had been recently implemented and provide solid, program-based recommendations. EVIDENCE’s first step was to engage all members of their community in a careful review of the shelter’s services and issues. EVIDENCE discovered that shelter staff lacked confidence in the services they were providing to their clients and were also were concerned about their own personal safety. As a result, many staff members were feeling stressed, isolated and oppressed, and their quality of work was clearly suffering. With no clear benchmarks to evaluate staff performance, management was powerless to address the situation and the volunteer Board of Directors was experiencing a distinct lack of accountability. The most significant recommendation made was that a committee of front-line staff, management and Board members be established to develop clear policies, procedures and program guidelines that would be reviewed and updated on an ongoing basis. In addition, EVIDENCE stressed that the shelter create a learning environment that was supported by front-line staff and management. All recommendations were developed and validated by stakeholders and resulted in improvements to staff safety and morale.
Copyright © 2010, First Work: The Ontario Association of Youth Employment Centres. All rights reserved.