YOUTH ON RECORD EVALUATION REPORT AUGUST 2014-JANUARY 2015

Transcription

1 YOUTH ON RECORD EVALUATION REPORT AUGUST 2014-JANUARY

2 PURPOSE The purpose of this biannual evaluation is to present Youth on Record s evaluation findings to the public, our constituents, and our stakeholders in order to document Youth on Record s goal attainment. To present programmatic outcomes, these evaluation findings are based on the identification what Youth on Record (YOR) aims to achieve. Subsequently, these results will be used to adjust programs and services. The end goal is to not only present what Youth on Record has done and is doing, but to determine how evaluation will lead to improved outcomes into short, intermediate and long-term objectives. Evaluation will also inform the fundraising and grant-writing process as seen through needs assessment. The purposes for evaluation include: - Youth on Record s desire to meet a critical need in the communities we serve and will serve - Determining efficiency of Youth on Record s programs - Appealing to different donors and demonstrating how well their funds were expended - Interpreting and using evaluation to improve programs The evaluation measures were developed to meet both the goals and objectives of Youth on Record programming. Youth on Record goals and objectives are to: - Expose at-risk young people to music and provides them with the access to the fundamentals of making music, music production and lyric writing - Support positive youth development, encouraging youth to strive for success and engage in their communities through access to cultural resources, social justice based education, internship opportunities and strong mentors - Support a vibrant music community through increased economic opportunities for artists and access to an expanded audience - Leverage a diverse musical heritage to support the La Mariposa redevelopment effort in La Alma / Lincoln Park by bringing professional musicians to an outdoor concert series, offering free Saturday classes and providing a safe community space where young people can study, meet friends, and youth groups can access meeting space The goals and objectives are then qualified and quantified by identifying programmatic impact. IDENTIFYING IMPACT The following findings are delivered through outcome-based evaluation metrics in order to measure the impact of our creative education programs. Outcome-based evaluation uses questions about programs that relate directly to the needs of the young people Youth on Record serves. These include changes in performance, behavior, actions, and attitudes towards themselves, their peers and their broader communities, as well as changes in what Youth on Record s young people value. Specifically, our evaluations measure: - Student demographic information (gender, ethnicity, age) - Performance (attendance, pass/fail rates/graduation rates/number of high school elective credits earned) - Out of school engagement (not for credit Youth Media Studio programming, community programming, internships) 22

3 - Student well-being (self-value, school/community/world view, empowerment) The following findings are from Quarter 1 and Quarter 2 of the school year. Youth on Record Evaluation Findings will happen twice a year, once after the Fall Semester (March) and once after the Spring Semester (July). The time period evaluated is from August 25, 2014 through January 23, EVALUATION METHODS The evaluation team consists of the Director of Programs, the Operations Manager, the Board of Directors Program Committee, and the Executive Director. All YOR Partner Artists, staff members, students, and partnering agencies contribute to creating and gathering the data and testimonials. Schools and RTC are responsible for: - Providing annual graduation rates - Providing demographic information on YOR students - Providing disciplinary information on students Students are responsible for: - Completing pre and post surveys - Providing testimonials Data is collected through the following methods:. Method Purpose Frequency Client Pre and Post Tests (see Appendix A) To quantify the knowledge attained in the class from a group of students with diverse learning styles and backgrounds through a question survey. More specifically, the tests indicate how the students are learning in the course. Students also report demographics. Tests are taken anonymously. Pre-tests are administered the first day of course instruction and post-tests are administered the last day of course instruction. This is normally the first day of the quarter or semester and the last day of the quarter or semester. Students Reports of student grades, attendance, behavior and performance Written testimonials Evaluate change of a student s academic qualifiers throughout the duration of a YOR class To report qualitative data from constituents served During class meeting times and at the end of the course Twice annually Students Students and principals 23

4 EVALUATION FINDINGS The evaluation findings are divided between the schools and the residential treatment centers that Youth on Record serves. They are further divided between findings from the pre and post tests, student reports from the school and testimonials. The findings were taken from August 25, 2014 through January 23, The following are the pre and post results from West Career Academy, Lincoln Respect Academy, and Venture Prep. All are Denver Public High Schools for at-risk youth, classified by the district as the oldest in the district with the least amount of credits to graduate. The classes taught were Social Solutions, Spoken Word, Basic Audio Production, Music Fundamentals and Emcee School. The classes and the schools they are hosted at are listed below: School West Career Academy Lincoln Respect Academy Venture Prep Classes Spoken Word, Basic Audio Production Spoken Word, Basic Audio Production, Social Solutions Emcee School, Music Fundamentals Results In one term (Fall 2014) of Youth on Record classes, students that agreed or strongly agreed with the following metrics increased from when they took the pre-test to the post-test by the following percentage differences: Metric Pre-test result Post test result I enjoy music, dance, theater, visual arts, and/or poetry 81% 90% I am good at music, dance, theater, visual arts, and/or poetry 64% 79% I understand the way music and art influence society 65% 79% I understand the power of my words 67% 86% I feel good about my future 74% 82% I feel I can make a difference in my community 58% 67% I feel valued in my community 60% 70% 24

5 I feel connected to my own culture 65% 81% I care about people of other cultures 83% 95% I am good at telling others about my ideas and feelings 48% 61% The following are the pre and post test results from Third Way and Devereux Cleo Wallace for Basic Audio Production. The findings are taken from August 25, 2014 through January 23, They are listed as one term of Youth on Record classes. The following table shows how students that agreed or strongly agreed with the following metrics increased from when they took the pre-test to the post-test by the following percentage differences: Metric Pre-test result Post test result I care about people of other cultures 92% 100% I am good at telling others about my ideas and feelings 62% 90% I am good at learning new things 85% 95% I am a creative person 77% 84% I understand the way music and art influence society 79% 69% I feel good about my future 84% 69% The following are metrics taken from specific Youth on Record courses. The table below shows the metric as selected questions according a particular class and how students that agreed or strongly agreed with the metric increased from when they took the pre-test to when they took the post-test. Metric for SPOKEN WORD Pre-test result Post test result I understand the power of my words 72% 87% I understand the way music and art influence society 62% 87% I am good at music, dance, theater, visual arts, and/or poetry 54% 80% I enjoy music, dance, theater, visual arts, and/or poetry 75% 93% I feel I can make a difference in my community 50% 80% Metric for EMCEE SCHOOL Pre-test result Post test result 25

6 I am good at telling others about my ideas and feelings 47% 67% I feel connected to my own culture 60% 78% I care about people of other cultures 83% 95% I understand the power of my words 75% 89% I feel I can make a difference in my community 80% 52% Metric for FUNDAMENTALS Pre-test result Post test result I am good at music, dance, theater, visual arts, and/or poetry 48% 67% I understand the way music and art influence society 47% 67% I care about people of other cultures 90% 100% I understand the power of my words 45% 67% I feel good about my future 68% 83% Metric for SOCIAL SOLUTIONS Pre-test result Post test result I feel connected to my own culture 50% 67% I feel I can make a difference in my community 75% 56% I feel good about my future 75% 78% I understand the power of my words 83% 100% Metric for PRODUCTION (DPS) Pre-test result Post test result I feel connected to my own culture 71% 100% I care about people of other cultures 71% 100% I am interested in community and world problems 53% 100% I am good at telling others about my ideas and feelings 53% 100% I am good at learning new things 71% 100% I like to read and write 41% 100% I am a creative person 47% 100% 26

7 I enjoy music, dance, theater, visual arts, and/or poetry 65% 100% I am good at music, dance, theater, visual arts, and/or poetry 41% 100% I understand the way music and art influence society 65% 100% I understand the power of my words 65% 100% I feel I can make a difference in my community 47% 100% I feel good about my future 65% 100% The pre and post tests were developed from the tests from previous year and will continue to be modified for the academic year, which will include more topics addressing personal, academic, and community challenges Youth on Record s young people face. STUDENT REPORTS The following are reports extracted from Partner Artists, teachers of record and principles of the schools from West Career Academy, Lincoln Respect Academy, Venture Prep, Contemporary Learning Academy and Colorado High School Charter. The findings are taken from August 25, 2014 through January 23, Metric Percentage or amount Number of Youth on Record students who 4 (the large graduation is in May) graduated Percentage of students passing YOR classes 88% Percentage of students taking YOR classes 77% showing increased attendance rates at Lincoln Respect High School Total number of students by class that are 240 currently taking YOR classes in their high schools Total number of Denver Public High school 660 elective credits earned Number of YOR students who participate in 25 internships The following are reports extracted from Partner Artists, teachers of record and the administration of Third Way and Devereux Cleo Wallace. The findings are taken from August 25, 2014 through January 23, Metric Percentage of passing students in Youth on Record Classes Percentage of students taking YOR classes showing increased attendance rates Percentage or amount 100% 100% 27

8 Percentage of students who increased their overall GPA while taking YOR classes 100% The reports from the Fall 2015 academic semester have not yielded all of the results Youth on Record needs to report on change, since it does not include graduation (which is in May and June). Pass/fail rates are also not fully determined halfway through the academic year. The Program Director and Partner Artists at Youth on Record will become less reliant on school reports and utilize internal attendance keeping and grade recording to compare Youth on Record classes to other classes throughout the Spring Semester in order to see growth over the span of the academic year. YOUNG PEOPLE SERVED AT YOUTH MEDIA STUDIO The following findings show the usage of the Youth Media Studio (YMS) by YOR s clients from August 25, 2014 through January 23, Metric Percentage or amount Total Number of Young People Served 850 Total Number of hours of instruction at Youth Media Studio 335 (full-time classroom instruction did not start until January 2015) Total Number of hours of out of school time for 280 DPS students Total Number of Transition Nights 6 Number of outside organizations that utilize YMS 21 Total number of outside community events held 35 at YMS The Academic Year has seen the YMS change dramatically. Every week has brought new development to the space, and with these changes more students, programming, guests and events are being hosted at the Youth Media Studio. In 2015, evaluation findings will also include be based off of the percentage of YMS usage dedicated to DPS course program, out of school activities, and special community events. DEMOGRAPHICS The following pie chart represent the demographics of all of the students served in Youth on Record classes at West Career Academy, Lincoln Respect Academy, Venture Prep, Third Way and Devereux Cleo Wallace between August 25, 2014 through January 23, Grade 28

9 Grade % of Youth on Record students are in the 12 th grade 27.9% of Youth on Record students are in the 11 th grade 26.3% of Youth on Record students are in the 10 th grade 1.2% of Youth on Record students are in the 9 th grade Less than 1% of Youth on Record students report to be in the 8 th grade (RTCs) Age Age and under over 18 29

10 32% of Youth on Record students are 17 20% of Youth on Record students are 18 19% of Youth on Record students are 16 19% of Youth on Record students are 15 and under 9% of Youth on Record students are over 18 Gender Gender Male Female 57% of Youth on Record students are male 43% of Youth on Record students are female 210

11 Ethnicity Ethnicity Latino or Hispanic Black or African American White or Caucasian Bi-racial Other 59.8 % of Youth on Record students are Latino or Hispanic 12.7% of Youth on Record students are Black or African American 11.2% of Youth on Record students are White or Caucasian 9.2% of Youth on Record students are Bi-racial 14% of Youth on Record students identify as other ethnicities UTILIZING THE DATA How do we share these results with new client markets? YOR evaluation results, specifically this report, will be shared with interested new clients to determine if: Their students exhibit similar demographics and needs as YOR s current student population Which courses will likely yield the results that the client hopes to attain Results are shared publically online Results are shared with individual donors via YOR s annual report How do we share these results with interested research parties All results are shared in the evaluations section of local, governmental, private, and national grant application. All results are reported to current funders in an annual grant report (or as required by the specific funding source) Results are shared publically online Results are shared with individual donors via YOR s annual report 211

12 Results are shared with YOR s non-foundation partners bi-annually (Denver Housing Authority, Corporate Funders, YOR s Board of Directors, Partnering schools and RTCs, Colorado Creative Industries) CLIENT TESTIMONIALS The following testimonials were provided by staff of Youth on Record partner schools and residential treatment centers and Youth on Record students between August 25, 2014 through January 23, The partnership between Youth on Record and CHSC is dedicated to securing a high quality music arts education for every young person attending our school. Music is an important component of our curriculum toward the strengthening of creative expression, self-motivation, personal success and 21 st Century skill building of our students. CHSC lacks the staffing, facilities and equipment to incorporate this elective into the school day. We rely on partnership with community-based educational organizations to supply access to up-to-date tools and knowledge. In the beginning of 2015, we began a partnership with Youth on Record to bring the business and industry perspective as well as educational practices for the enrichment of our program offerings. The students work on projects which are realitybased and mirror business situations to challenge students to participate and learn. As our primary partner, Youth on Record has three full-time teaching staff allocated to this project and partnership. This partnership is rooted in community based development and appreciation for music. We are sincerely excited about this partnership and opportunity to expand our music program. - Liz Feldhusen, Vice Principle, Colorado High School Charter Youth on Record has been a great addition to Respect Academy. The classes they offer are engaging and pertinent to our students. We are hoping to continue the partnership for many years to come. Wendy Lanier, Principal, Lincoln Respect Academy I enjoy going to class every morning. It s not something everyone gets to do. It excites me and makes me get up and go to school. I look forward to it unlike other music classes I have taken in the past. The instructors are chill and nice. They help me understand the music and know what they are doing. Paul, 12 th grade DPS student IT s FUN! You get to do a lot of different things. It s not one instrument, it s always changing. It helped me understand music better. It made me break down what music really is. I love it, I really do! - Midori, 9 th grade DPS student It s inspirational. I did not know anything about music before I went to class and I was really nervous. But it did not take long for me to feel like it was home and comfortable. I have learned so much and I want to take the class for as long as I can keep learning new things. - Precious, 10 th grade DPS student My favorite part of this class is making beats. It s exciting to learn how to create beats with different instruments. The teachers are really cool. They have taught me a lot of new things and I have so much fun. Sammy, 11 th grade DPS student I can t believe we get to have class here (the Youth Media Studio) Carlos, 11 th grade DPS student Music is my voice and Youth on Record will help me amplify it Josue, 11 th grade DPS student 212

13 I was acting a fool all the time, and Youth on Record helped me to not act a fool all the time Rainlyn, RTC student Youth on Record has helped me and others to be more tolerant, and express it through music Tyler, RTC student When I was discharge from my RTC, Youth on Record is the only place I knew I had to get to Damon, former RTC student PARTNER ARTIST EVALUATION Throughout this time period, Youth on Record had 12 independently contracted partner artists teaching the classes. A total of 560 partner artist hours of classroom instruction were spent in Denver Public High Schools and 160 partner artist hours of classroom instruction at residential treatment centers. Partner Artists are evaluated by the Director of Programs on the following response scale rubric once a semester in order to quantify their performance both in and out of the classroom in an effort to show where improvement can be made. It is meant to produce critical and constructive feedback from the Director of Programs to the Partner Artists. It is not comprehensive, but serves as a way to document the growth of the Partner Artist in the following categories, with the response scale ranging from 1 to 5, 1 being the lowest (strongly disagree) and 5 being the highest (strongly agree). Specifically, (1/A) Strongly Disagree, (2/B) Disagree, (3/C) Neutral, (4/D) Agree, (5/E) Strongly Agree. Communication with Director of Programs - Attendance - Pre-post tests - Equipment - Able to attend meetings, other functions, in communication Curriculum Development - Develops twice a year - Consistent with YOR mission, values, educational standards Classroom Observation - Communicates with school - Supplements School Culture - Provides instruction - Able to have class covered when not available - Maintain school records, documents classroom, captures classroom Youth on Record Culture Partner Artist evaluations are conducted twice an academic year (once in the Spring and once in the Fall) to measure growth throughout evaluation period. 213

14 APPENDIX A Pre-Post Test Fall 2014 What school do you go to? What grade are you in? How old are you? What is today s date? What Youth on Record class are you in? What is your zip code? What is your gender? Please circle. - Male - Female What best describes you? Please check all that apply. - American Indian, Eskimo or Aleut - Asian/Pacific Islander - Bi-racial/Multi-racial - Black or African American - Latino or Hispanic - White or Caucasian - Other: Please circle the option below that is closest to how you feel. I do well in school. All of the time Most of the time Some of the time Never Don t know I try hard in school. All of the time Most of the time Some of the time Never Don t know I feel valued at my All of the time Most of the time Some of the time Never Don t know school. I feel valued in my All of the time Most of the time Some of the time Never Don t know community. I feel connected to my own culture, All of the time Most of the time Some of the time Never Don t know race, or ethnic group. I care about people of other cultures, All of the time Most of the time Some of the time Never Don t know races, or identities. 214

15 I am interested in community and world All of the time Most of the time Some of the time Never Don t know problems. I am good at telling others about my All of the time Most of the time Some of the time Never Don t know ideas and feelings. I am good at listening All of the time Most of the time Some of the time Never Don t know to other people. I am good at learning All of the time Most of the time Some of the time Never Don t know new things. I like to read and write. All of the time Most of the time Some of the time Never Don t know I am a creative All of the time Most of the time Some of the time Never Don t know person. I enjoy music, dance, theater, visual arts, All of the time Most of the time Some of the time Never Don t know and/or poetry. I am good at music, dance, theater, All of the time Most of the time Some of the time Never Don t know visual arts, and/or poetry. I understand the way music and art All of the time Most of the time Some of the time Never Don t know influence society. I understand the power power of my words. All of the time Most of the time Some of the time Never Don t know I feel good about All of the time Most of the time Some of the time Never Don t know my future. I feel I can make a All of the time Most of the time Some of the time Never Don t know difference in my community. 215