Quality Systems and Evaluation Services Unit http://www.co.multnomah.or.us/dcj/evaluation.shtml Multnomah County Department of Community Justice 501 SE Hawthorne Blvd., Suite 250 Portland, OR 97214 Juvenile Counseling Services Customer Satisfaction Survey Report: 2006 Prepared by: Ellen Churchill Research & Evaluation Analyst February 2007
Executive Summary As a way to measure key performance in 2006, the Department of Community Justice included four statements in the 2006 Juvenile Counseling Services Customer Satisfaction Survey. The first three statements include the categories of timeliness 1, helpfulness and expertise. These three statements were measured using the agreement scale. The fourth statement category measures overall program satisfaction. The majority of probation youth, their parents and GOALS Supervision/Sanction Services youth all indicated the programs they participated performed well in all the key performance categories. However, less then half of GOALS Accountability Services youth indicated the program performed well in any of the key performance categories. Timeliness Statement: I feel the time period between committing my offense/my child s offense and my first appointment/my child s first appointment with the Juvenile Justice Counselor was too long.! About 1/3 of youth [29%] and about 1/3 of parent s [36%] agreed or strongly agreed that the wait time was too long.! A majority of the youth who agreed or strongly agreed that the wait time was too long indicated the wait time should be thirty days or less [65%].! Slightly over half of the parents who agreed or strongly agreed that the wait time was too long indicated the wait time should be thirty days or less [56%] Helpfulness Statement: The program staff helped me/my child better understand the harm done to others and gave me/my child a chance to repair that harm.! Over 75% of probation youth, probation parent s and GOALS/Sanction Services youth agreed or strongly agreed with the helpfulness statement.! Less then half [42%] of GOALS/Accountability Services youth agreed or strongly agreed with the helpfulness statement. Expertise Statement: Since I have participated in the program my behavior/my child s behavior has improved.! Over 75% of probation youth, probation parent s and GOALS/Sanction Services youth agreed or strongly agreed with the expertise statement.! Less then half [44%] of GOALS/Accountability Services youth agreed or strongly agreed with the expertise statement. Overall Customer Satisfaction: Indicate your level of satisfaction with [program] services.! Over 70% of probation youth, probation parent s and GOALS/Sanction Services youth and Skill Groups youth indicated they were satisfied or very satisfied with the program s services.! Less then half [47%] of GOALS/Accountability Services youth indicated they were satisfied or very satisfied with the program s services. 1 The timeliness category was exclusive to probation services. Customer Satisfaction Survey Report: 2006 page 2
Juvenile Counseling Services Customer Satisfaction Survey This report reviews feedback from customers of the Juvenile Services Division [JSD] Counseling Services unit, which includes Probation and the GOALS units. The tabulated results include data from both youth and parents who participated in these services and responded to the survey questions. As a way to measure customer satisfaction, collect data for trend analysis, and explain the results, key performance measures were included in this year s survey. These key performance statements included timelines, helpfulness, expertise, and overall program satisfaction. Data Collection and Methodology Data were collected from mid-september 2006 through mid-october 2006. In most cases, staff from the program unit personally handed the survey to all of the youth and their parents attending the program and asked them to complete it. Upon completion, the youth and parent were asked to drop the survey into a sealed box in the main lobby of the Juvenile Justice Complex. However, survey distribution and collection differed slightly in each program; therefore, the specifics of each program s methodology are outlined below. Probation Services! During the data collection period, each JCC scheduled an appointment with each youth and their parent.! At the end of the scheduled appointment, the JCC handed a survey to the youth and parent and asked them to complete it before they left the office. Upon completion, the youth and parent dropped the survey into the box in the main lobby of the Juvenile Justice Complex, or the collection box located in their field office. GOALS Program! During the data collection period, the GOALS Program Staff handed the survey to all youth completing the program (both successful completions and unsuccessful terminations ) and asked them to fill it out.! The youth dropped the completed survey into the box near the offices of the Accountability Program in the Juvenile Justice Complex. Skill Development! During the data collection period, staff facilitating the various skill development groups distributed the surveys at the last session and asked participants to complete the survey.! Upon completion of the survey, participants handed the survey to the facilitator. Completed surveys for all three units were gathered together and sent to DCJ Research & Evaluation unit for data entry and analysis on November, 14 2006. Customer Satisfaction Survey Report: 2006 page 3
The following figure illustrates graphically the number of completed surveys received by program over the past several studies. Number of Respondents by Program and Survey Year 350 300 286 250 239 243 258 235 200 150 141 100 91 117 100 50 66 43 55 0 2000 2001 2002 2004 2005 2006 Probation Parent & Youth GOALS Program Unit * The survey was not conducted in 2003. The 2005 change in methodology substantially increased total returns for Counseling Services (all programs combined). The total clients surveyed for each year are: [2000 N=371, 2001 N= 339, 2002 N=192, 2004 N=333, 2005 N=416, 2006 N=376 2 ]. The appendix has a tabular breakdown of the responses by location. The tables in this report illustrate results of the survey by program. Youth or parents who did not answer specific questions were not counted in the percentage calculations. The scale used for measuring agreement was: Strongly Agree, Agree, Not Sure, Strongly Disagree and Disagree. For analysis Strongly Agree and Agee were collapsed into the variable Agree and Strongly Disagree and Disagree were collapsed into the variable Disagree. The same is true of the scale used for measuring satisfaction. Very Satisfied and Satisfied were collapsed into Satisfied and Somewhat Satisfied and Dissatisfied were collapsed into the variable Dissatisfied. A subset of the GOALS program is the Skill Groups. The anger management workshop, Save Our Youth, and Family Strength groups facilitated in 2005 were revamped in 2006 in order to combine the curriculum into one workshop titled The Power of Family Unity. During 2006 the skill groups consisted of only two workshops, a youth employment workshop and The Power of Family Unity. 2 This figure does not include 32 surveys for the GOALS Skill Groups. Customer Satisfaction Survey Report: 2006 page 4
GOALS Unit Supervision/Sanction Services The GOALS program staff helped me better understand the harm I have done to others and I was given a chance to repair that harm. 2% N=50 24% 74% N=49 I knew what I had to do to be successful in the GOALS program. 12% I believe the GOALS program staff knows what they are doing. N=49 12% N=49 Since I have participated in the GOALS program, my behavior has improved. 2% 18% 80% N=49 During my GOALS experience, the GOALS program staff DID NOT treat me with respect. 73% 10% 16% The GOALS program staff did a good job in providing a safe environment in the GOALS program. N=49 10% 90% The GOALS program staff always did what they said they were going to do. 2% N=48 8% 90% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Not Sure Agree Positive responses were received by the majority of youth participating in Supervision/Sanction Services for every statement. Customer Satisfaction Survey Report: 2006 page 5
The GOALS Unit Accountability Services The GOALS program staff helped me better understand the harm I have done to others and I was given a chance to repair that harm. N=91 14% 44% 42% N=91 I knew what I had to do to be successful in the GOALS program. 19% 10% 71% N=91 I believe the GOALS program staff knows what they are doing. 9% 27% 64% Since I have participated in the GOALS program, my behavior has improved. N=91 12% 44% 44% N=91 During my GOALS experience, the GOALS program staff DID NOT treat me with respect. 57% 24% 19% The GOALS program staff did a good job in providing a safe environment in the GOALS program. N=91 8% 22% 70% The GOALS program staff always did what they said they were going to do. 3% N=88 24% 73% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Not Sure Agree Over 70% of Accountability youth agreed with the following statements:! I knew what I had to do to be successful in the GOALS program.! The GOALS program staff did a good job in providing a safe environment in the GOALS program.! The GOALS staff always did what they said they were going to do. Less than 45% of youth agreed with the following statements:! The GOALS program staff helped me better understand the harm I have done to others and I was given a chance to repair that harm.! Since I have participated in the GOALS program, my behavior has improved. Customer Satisfaction Survey Report: 2006 page 6
The GOALS Unit Skills Groups N=32 The GOALS program staff helped me to better understand0% the harm I have done to others. 16% 84% The purpose of the group was 0% well explained. I learned a lot in this group. 0% All group leaders made the 0% group interesting. I learned things I use in my life. 0% 16% 84% All group leaders were 0% courteous to me. The program staff always did what they said they were going0% 9% to do. 91% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Not Sure Agree Past surveys show that it is not unusual to receive high agreement percents for positive statements from GOALS skill group participants. Last year agreement for positive statements was over 73% 5. This year s survey shows higher agreement with over 83% agreement for positive statement. 5 Churchill, E. (2005). Juvenile Counseling Services Customer Satisfaction Survey Report, p 9 Customer Satisfaction Survey Report: 2006 page 7
Probation Services Youth Responses The JCC helped me better understand the harm I have done to others and gave me a chance to repair that harm. N=176 5% 14% 81% N=176 My JCC and I worked on my probation plan together. 3% 84% My JCC has included the positive things about me and my family in my probation case plan. N=173 5% 17% 79% N=171 Since I started workiing with my JCC, I have not committed another crime. 17% 70% N=174 Since I started workiing with my JCC, my behavior has gotten better. 5% 12% 83% My JCC was able to help me get into the programs and services I needed. N=175 2% 86% N=177 My JCC does a good job. 2% 6% 92% My JCC helped me to see why my probation requirements were important. N=175 2% 9% 89% N=176 My JCC pays attention to what I say. 1% 6% 93% My JCC cares about my feelings. N=175 11% 3% 86% N=177 My JCC cares about my point of view. 2% 9% 89% N=177 My JCC listens to me and my concerns. 2% 6% 92% N=178 My JCC is respectful to me. 2% 5% 93% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100 % Disagree Not Sure Agree A strong majority [70% and over] of youth participating in Supervision/Sanction Services agreed with every positive statement. Customer Satisfaction Survey Report: 2006 page 8
Probation Parents' Response The JCC has included the postiive things and strengths about my child and my family in his/her probation case plan. The JCC has involved me and my child in my child's probation plan. N=50 2% 10% N=54 7% 7% 85% My child's JCC was able to help him/her get into the programs and services he/she needed. My child's JCC knows what he/she is doing. The juvenile justice system has been helpful to my child. The JCC helped my child to better understand the harm he/she hs done to others and gave him.her a chance to repair that harm. N=52 6% N=51 6% N=50 4% 10% N=53 6% 11% N=52 94% 94% 86% 83% The JCC is NOT respectful to me. 81% 6% My child's JCC helped him/her see why his/her probation requirements were important. N=53 6% 6% N=52 89% The JCC listens to me. 6% 2% 92% N=53 I can tell the JCC cares about my feelings. 6% 4% 91% N=52 I can tell the JCC cares about my point of view. 6% 4% 90% The juvenile justice system could have done more for my child and my family. N=50 N=51 46% 28% 26% My child's behavior has improved since he/she has been working with the JCC. 8% 10% 82% My child's attitude has NOT improved since he/she has been working with the JCC. N=52 73% 6% 21% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Not Sure Agree Parent agreement for positive statements is very high with the exception of The juvenile justice system could have done more for my child and my family. About 1/4 of parents were not sure how to answer the statement. Customer Satisfaction Survey Report: 2006 page 9
Overall Satisfaction "Please indicate your level of satisfaction with the program." N=50 Probation Parent Responses 6% 4% 90% N=171 Probation Youth Responses 11% 14% 75% N=32 GOALS Skills Group 3% 84% N=82 GOALS Program Accountability Services Responses 16% 39% 45% N=33 GOALS Program Supervision/Sanctions Services Responses 6% 94% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Unsatisfied Neutral Satisfied The majority [75% +] of probation parents, probation youth, Goals Skills groups, and GOALS Supervision/Sanctions youth indicated they were satisfied with the program. However, less then half [45%] of GOALS Accountability youth indicated they were satisfied with their program. Customer Satisfaction Survey Report: 2006 page 10
Attendance Issues Percent of parents answering "Yes" to the question, "Has school attendance been a problem for your child?" N=53 34% NO YES About half of these parents [54%] that agreed their child had an attendance problem also indicated their child's attendance had improved since being on probation. 66% Percent of youth answering "Yes" to the question, "Has school attendance been a problem for you?" N=142 NO YES 41% About threequarters of these youth [74%] that had an attendance problem also agreed their attendance had improved since being on probation. 59% Probation services appear to have a positive influence on attendance. The majority of youth that indicated they had attendance problems also agreed their attendance improved since being on probation (74% improved). Customer Satisfaction Survey Report: 2006 page 11
Wait between offense and first appointment with JCC Probation Youth - Wait TimeToo Long "I feel the time period between committing my offense and my first apointment with the JuvenileCourt Counselor was too long." 29% The majority [65%] of these youth who agreed that the wait time was too long indicated they thought the wait time should be 30 days or less. Agreed Disagreed 71% Probation Parents - Wait Time too Long "I feel the time period between my child's arrest and the first appointment with the Juvenile Court Counselor was too long." About half [56%] of these parents who agreed the wait time was too long indicated that they thought the wait time should be 30 days or less Agree 36% Disagree 64% The majority of youth and half of the parents that indicated they felt the wait time between the child s arrest and the first appointment with a JCC was too long thought the wait time should be thirty days or less. Customer Satisfaction Survey Report: 2006 page 12
Appendix Survey Results Table 1 shows the number of youth and parents who completed and submitted a survey for all programs. Due to the 2003 re-organization, some programs/units are no longer the same as they were defined in the previous years. Programs such as Diversion and Adjudication were merged into other units. Skill Development is now handled under the GOALS program. Caution should be exercised when comparing the current year s survey return with results from previous years. In addition, in previous years JCCs were asked to call and schedule appointments with youth randomly selected from the Juvenile Justice Information System [JJIS]. This method resulted in declining response rates. Beginning in 2005 data was gathered for three months rather than one month and an effort was made to obtain data from every youth. Number of Respondents by Program in 2006 Unit Youth Parent/ Guardian GOALS N/A Supervision Sanctioned Services 49 NA Accountability Services 91 NA Total 140 NA Intake Intervention 3 NA Probation 1. Central 19 1 2. East 25 7 3. Family Services 13 6 4. GRIT 17 5 5. Gender 19 9 6. Northeast 12 8 7. Southeast 30 0 8. Sex Offender 22 0 9. Informal 25 17 Total 182 53 Customer Satisfaction Survey Report: 2006 page 13