Client Satisfaction Survey and Measurement

Transcription

1 Client Satisfaction Survey and Measurement June 2004 Project: LAWA/3

2 Client Satisfaction Survey and Measurement June 2004 Client: Legal Aid WA Project: LAWA/3 Consultants: Anna Munday April Rutkay Data Analysis Australia Pty Ltd 97 Broadway Nedlands, Western Australia 6009 (PO Box 3258 Broadway, Nedlands 6009) Website: Phone: (08) Facsimile: (08) A.C.N A.B.N

3 Executive Summary Legal Aid WA provides quality legal services to those in need and helps the community to access justice. A key aspect of Legal Aid WA s work is measured by the quality legal services effectiveness indicator. This indicator is measured in terms of client satisfaction with particular aspects of client service delivery, and is a required aspect of the Annual Report submitted to the Auditor General. The annual satisfaction indicator alternates between Family Law and Criminal Law clients. Family Law clients were surveyed to compile the satisfaction indicator for the 2003/2004 Annual Report. The three Family Law client groups included as part of this year s client satisfaction surveys were Legal Advice, Alternative Dispute Resolution and Grants of Aid. The sample was designed to balance the requirements of optimising the accuracy of the overall satisfaction level whilst having sufficient sample sizes to allow valid statistical analysis to be performed at the individual survey level. Satisfactory response rates were achieved for each of the surveys, which is an important aspect in assessing the validity of any survey. A composite index was designed to measure the overall satisfaction level of Legal Aid WA s Family Law clients. The index estimated the overall satisfaction rating to be 74.0%, with an achieved accuracy level of ±4.3%, meaning that we can be 95% confident that the true rate of satisfaction lies between 69.7% and 78.3%. The Alternative Dispute Resolution clients were the most satisfied group, with a satisfaction rating of 79%. Legal Advice clients had a satisfaction level of 76% and Grants of Aid clients had a rating of 61%. Legal Aid WA was also interested to find out how useful their online and print publications are and how frequently they are accessed by Community Legal Centres. A survey of Community Legal Centres found that the online and print publications achieved an 80% rate of satisfaction. The main points of interest which resulted from running these surveys are: The more assistance which clients receive from Legal Aid, the greater their satisfaction tends to be; The highest levels of satisfaction tends to involve the people at Legal Aid WA, rather than the results that were achieved; and Community Legal Centres are generally satisfied with the Legal Aid online and print publications. These key findings are consistent with the findings in the 2001/2002 survey. LAWA/3 Executive Summary i

4 Table of Contents 1. INTRODUCTION SURVEY METHODOLOGY QUESTIONNAIRE SURVEY POPULATION SAMPLE SIZES SURVEY IMPLEMENTATION ESTIMATION METHODOLOGY SURVEY OF LEGAL ADVICE CLIENTS RESPONSE RATES EXTENT TO WHICH THE CLIENT S LEGAL SITUATION HAS CHANGED ASSISTANCE RECEIVED FROM LAWYER OR ADVISOR COMPARISONS TO THE 2001/2002 SURVEY RESULTS SURVEY OF ALTERNATIVE DISPUTE RESOLUTION CLIENTS RESPONSE RATES ABOUT THE LAWYER ABOUT THE CONFERENCE OVERALL SATISFACTION COMPARISONS TO THE 2001/2002 SURVEY RESULTS SURVEY OF GRANTS OF AID CLIENTS RESPONSE RATES APPLICATION FOR LEGAL AID ABOUT THE LAWYER ABOUT THE SEPARATE REPRESENTATIVE OVERALL SATISFACTION COMPARISONS TO THE 2001/2002 SURVEY RESULTS OVERALL SATISFACTION OF FAMILY LAW CLIENTS METHODOLOGY RESULTS ONLINE AND PRINT PUBLICATIONS USERS CLC SURVEY RESPONSE RATES KNOWLEDGE ABOUT AND USE OF THE ONLINE AND PRINT PUBLICATIONS ABOUT THE LEGAL RESOURCE DATABASE ABOUT THE LEGAL AID WEBSITE ABOUT THE LEGAL AID PRINT PUBLICATIONS OVERALL SATISFACTION OF THE ONLINE AND PRINT PUBLICATIONS COMPARISONS TO THE 2002/2003 SURVEY RESULTS CONCLUSIONS LAWA/3 Table of Contents i

5 APPENDIX A QUESTIONNAIRES APPENDIX B FORMULAE USED APPENDIX C ITEM LEVEL RESPONSES LAWA/3 Table of Contents ii

6 Table of Figures FIGURE 1. HOW THE SITUATION HAS CHANGED, DEPENDING ON WHETHER THE CASE WENT TO COURT FIGURE 2. HOW THE SITUATION HAS CHANGED, DEPENDING ON THE TYPE OF ASSISTANCE RECEIVED FIGURE 3. LEVEL OF SATISFACTION WITH THE LAWYER OR ADVISOR... 7 FIGURE 4. USEFULNESS OF THE ADVICE RECEIVED DEPENDING ON THE TYPE OF ASSISTANCE RECEIVED FIGURE 5. LEVEL OF SATISFACTION ABOUT THE CONFERENCE FIGURE 6. LEVEL OF SATISFACTION WITH THE CONFERENCE CHAIRPERSON FIGURE 7. PERCENTAGES OF CLIENTS WHO ARE SATISFIED WITH THE LEGAL AID PROCESS AND THE ADR PROGRAM FIGURE 8. LEVEL OF SATISFACTION WITH THE LAWYER FIGURE 9. LEVEL OF SATISFACTION WITH THE SEPARATE REPRESENTATIVE FIGURE 10. LEVEL OF SATISFACTION WITH THE LEGAL RESOURCE DATABASE FIGURE 11. LEVEL OF SATISFACTION WITH THE LEGAL AID WEBSITE FIGURE 12. LEVEL OF SATISFACTION WITH LEGAL AID S PRINT PUBLICATIONS LAWA/3 Table of Contents i

7 1. Introduction Legal Aid WA (LAWA) is an independent statutory authority that operates under the Legal Aid Commission Act 1976, and is responsible for providing quality legal services to those in need and to assist the community to access justice. Legal Aid WA provides information and resources to assist the community with their legal concerns, and offers a range of services aimed at target groups or individuals with particular legal problems. It is a requirement under the provisions of Section 66 of the Financial Administration and Audit Act (FAA ACT) 1985, that Legal Aid prepare and submit an annual report, including Performance Indicators, to the Minister. Legal Aid s outcome statement as defined in the organisation s Performance Indicator (PI) framework is: Services are provided through Legal Aid to ensure that community and target groups have access to and are provided quality legal services. The quality legal services effectiveness indicator is measured in terms of client satisfaction with particular aspects of client service delivery. The annual satisfaction surveys alternate between Family Law clients and Criminal Law clients. For the 2003/2004 Annual Report, Family Law clients will be surveyed. The Family Law client groups to be surveyed are Legal Advice, Alternative Dispute Resolution (ADR) and Grants of Aid. Legal Aid WA is also interested to find out how useful their online and print publications are and how frequently they are used. To determine this, Community Legal Centres (CLC s) were surveyed, with the aim of determining their level of satisfaction with the online and print publications. Data Analysis Australia was contracted to undertake the measurement of client satisfaction for the three Family Law client groups and the satisfaction of Community Legal Centres with regards to the Legal Aid online and print publications, for the 2003/2004 Annual Report. 2. Survey Methodology Where possible, the Family Law client surveys were designed to allow meaningful comparisons to be made to the most recent Family Law client satisfaction surveys, which were conducted for inclusion in the 2001/2002 Annual Report. The CLC survey was implemented to make it consistent with the 2002/2003 survey. LAWA/3 Report June

8 2.1 Questionnaire The questionnaires for this year s Grants of Aid, Legal Advice and Alternative Dispute Resolution surveys were heavily based on those used in 2002, with the main changes being the addition of some new questions. All modifications were developed in conjunction with Murray Cribb from Legal Aid WA. This was the third year that the Online and Print Publications Users CLC Survey was run. The questionnaire was not changed from last year. This survey was targeting agencies rather than individuals - the questions reflect this target. A copy of each questionnaire is included in Appendix A. 2.2 Survey population The target population for the Alternative Dispute Resolution Family Conferencing Program survey is those clients who entered the ADR Family Conferencing Program between 1 July 2003 and 30 April 2004, and who have attended a conference. It is possible that a small number of clients have been accepted into the ADR program but have not yet had their conference. These clients are out of scope of the survey. There were 252 clients who were in-scope of the ADR program survey. The target populations for the Legal Advice and Grants of Aid surveys were those clients who received these respective types of assistance from Legal Aid between 1 July 2003 and 30 April The target population sizes were 3,653 for Legal Advice and 406 for Grants of Aid. The Online and Print Publications Users CLC Survey was conducted as a census of all Community Legal Centres in Western Australia. The list of the Community Legal Centres was provided by Legal Aid WA. 2.3 Sample sizes The sample sizes for the three Family Law client groups satisfaction surveys were designed to balance the requirements of obtaining accuracy to within about 5% (to satisfy the requirements of the Office of the Auditor General) and having sufficient sample sizes for each survey group to allow valid statistical analysis at the individual survey level and to make comparisons between the surveys. The requirements of surveying a target of 400 clients in total also had to be satisfied. The sample of 400 was allocated proportional to the square root of the population size, between Legal Advice, Grants of Aid and ADR. Each of the samples was selected randomly. The in-scope clients for each survey were LAWA/3 Report June

9 randomly ordered, and given an index of 1,,N. The first n clients in the list were then selected to be included in the survey. The following table shows the breakdown of the sample into the three survey areas. Client Group Population size (N) Sample size (n) Level of Accuracy Legal Advice 3, ±5.7% Alternative Resolution Dispute ±9.9% Grants of Aid ±9.1% Total 4, ±4.4% Table 1. Sample sizes and resulting levels of accuracy. The formula used to calculate the level of accuracy is included in Appendix B. The expected overall level of accuracy obtained from this design is 4.4%, assuming a satisfaction level of 65%, based on previous results. The accuracy levels for the individual surveys are higher, but still within acceptable error bounds. The achieved overall level was 4.3% and the details of this, as well as the achieved accuracy levels for each client group are in Section 6.2. There were 28 Community Legal Centres to be surveyed in the Online and Print Publications Users CLC Survey. 2.4 Survey implementation Telephone interviewing was adopted to implement the surveys, using a Computer Aided Telephone Interviewing (CATI) system, as it had been used successfully previously, and in order to sustain comparability in methodology to past client satisfaction surveys. The CATI system is scripted to guide the interviewer through the interview so that interviewer bias is minimised and data quality is improved over paper-based alternatives. The interviewers were instructed to make three attempts to contact each client selected in the survey before deeming that client non-contactable and replacing them with the next client in the sampling list. Since the Online and Print Publications User s questionnaire was targeted at agencies instead of individual respondents, pre-contact was made in the form of a letter which included an explanatory page and a copy of the questionnaire, in order to allow them time to consider their answers to the survey and discuss them with colleagues. To target the best person to respond on the CLC s behalf, the explanatory page asked for the name of the most appropriate person to be passed on to Data Analysis Australia. This contact name was then LAWA/3 Report June

10 passed onto the fieldwork company. If a name was not provided, the fieldwork company established the best person to speak to through the phone call itself. Staff were asked not to fill in the questionnaires and return them, but to have the answers ready for when a fieldworker contacted them to conduct a telephone interview. Centres which had not heard of or used the Legal Aid online and print publications were screened out of the survey. A copy of the introductory letter sent to each representative is included in Appendix A. 2.5 Estimation methodology Satisfaction levels were calculated as the sum of the non-missing responses on the positive side of neutral (that is, agree and strongly disagree), expressed as a percentage of the total number of responses excluding not applicables. The not applicables were excluded from the denominator because their inclusion can artificially deflate the observed satisfaction levels. This methodology is consistent with that used in Where multiple responses were allowed, percentages were calculated as the number of clients who gave that response divided by the total number of respondents for that question. Thus, the sum of the percentages could sum to greater than 100%. The denominator in the percentage is not always equal to the sample size. This is because the denominator is calculated as the number of responses to that particular question, and each question could have different numbers of responses due to clients refusing to answer a particular question, or being skipped past the question due to an earlier response. 3. Survey of Legal Advice Clients The Legal Advice service area comprises Legal Advice Bureaus and the Minor Assistance Program. In the Legal Advice Bureaus, solicitors are available to provide more detailed advice on some legal issues, for a small fee. In the Minor Assistance Program, solicitors and advisors are available to assist people to handle their own legal matters. This assistance may include letter writing, negotiations or phone call made on the client s behalf or preparation of court documentation. 3.1 Response rates There were 3,393 clients on the Legal Advice client list who were in-scope of the survey after duplicates, interstate and incomplete phone numbers were removed. To obtain the required sample of 250 clients, attempts were made to contact 628 clients. Of the 378 clients who were attempted but were not included in the survey, 26 were refusals. The remainder either could not be LAWA/3 Report June

11 contacted for reasons such as the phone line being disconnected, no answer being obtained or the phone line being engaged, or were not included in the sample due to language difficulties, being away or being ineligible. 3.2 Extent to which the client s legal situation has changed Over 51% of Legal Advice clients felt that their legal situation had greatly or somewhat benefited as a result of contacting Legal Aid WA (Figure 1). A further 35% felt that their legal situation had stayed the same. The remaining 14% of clients thought that their legal situation had suffered as a result of contacting Legal Aid WA. Clients who went to Court were less likely to say that their situation has stayed the same (26%) than clients who did not go to Court (39%). Of those clients who did go to Court, 55% felt that their legal situation improved, compared to 50% for those who did not go to Court. The clients were more likely to feel that their legal situation deteriorated if they went to Court (19%) compared to those who did not go to Court (11%), suggesting that going to Court is more likely to change the situation, however it does not mean it will change in a beneficial way. Went to court 19% 26% 55% Did not go to court 11% 39% 50% Total 14% 35% 51% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Suffered Stayed the same Benefited Figure 1. How the situation has changed, depending on whether the case went to court. Clients who were provided with advice only (45%) were less likely to feel that their legal situation has benefited from contacting Legal Aid WA than those clients who received minor assistance in addition to advice (Figure 2). Over half (62%) of the clients who received minor assistance felt that their legal situation had benefited and 15% felt that it had suffered. Almost half (42%) of LAWA/3 Report June

12 clients who received only legal advice, felt that their legal situation had stayed the same, 45% felt it had benefited and 13% felt it had suffered. Advice only 13% 42% 45% Minor Assistance 15% 23% 62% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Suffered Stayed the same Benefited Figure 2. How the situation has changed, depending on the type of assistance received. 3.3 Assistance received from lawyer or advisor A number of questions were asked to obtain clients perceptions of the lawyer or advisor who helped them, and how useful the received advice was. Figure 3 shows the level of agreement or disagreement the clients had with a number of statements regarding the assistance the lawyer or advisor provided. LAWA/3 Report June

13 Helped you to understand 12% 9% 79% Listened to you 6% 7% 87% Advice you received w as helpful 16% 9% 75% Clear explaination of w hat to do next 14% 12% 74% Helped you to understand assistance available 11% 12% 77% Would you like to use the same law yer again? 22% 12% 66% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 3. Level of satisfaction with the lawyer or advisor. The satisfaction levels for each question are well over 60%. Most (87%) of clients thought that the their lawyer or advisor listened to them when the client was explaining their situation. Over 75% of the clients felt that their lawyer or advisor helped them to understand the legal situation they were in, and that the advice received was helpful. In total, 75% of the clients who remember receiving advice felt that it was helpful. About 72% of clients who received advice only and 81% of those who also received minor assistance found that the advice provided was useful (of those who rated the usefulness of any advice or assistance received). Similar levels of satisfaction were achieved with the advice and assistance which was provided to assist in the preparation of a court case. Of the clients who went to court and received advice only, 48% thought the advice was useful for their court preparation, and 70% of those clients who received minor assistance felt that it was useful. These comparisons are illustrated in Figure 4. LAWA/3 Report June

14 Advice only (total) 15% 13% 46% 26% Minor assistance(total) 9% 10% 40% 42% Advice only (court) 24% 28% 38% 10% Minor assistance (court) 9% 21% 39% 30% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Not at all useful Not very useful Fairly useful Extremely useful Figure 4. Usefulness of the advice received depending on the type of assistance received. 3.4 Comparisons to the 2001/2002 survey results The percentage of respondents who felt that their legal situation had greatly or somewhat changed as a result of obtaining legal advice from Legal Aid increased from 40% to 51%. An increase was still present when individually considering respondent groups that went to Court, did not go to Court, obtained legal advice only, and those who received minor assistance. More respondents agreed that their lawyer or advisor helped them to understand the legal situation they were in and the assistance that was available, listened to them and gave a clear explanation of what to do next. There was also an increase in respondents who stated that the advice they received was helpful, and that they would use the same lawyer again if they found themselves in a similar situation in the future. 4. Survey of Alternative Dispute Resolution Clients The Alternative Dispute Resolution Family Conferencing Program aims to help people settle their family law dispute and reach an agreement without going to court. This is achieved by means of holding a conference, whereby Legal Aid WA assigns an independent conference chairperson to facilitate the conference, and a lawyer to represent each party at the conference. The aim of the conference is to help people identify the issues which need resolving and develop options and alternatives to settle their dispute. LAWA/3 Report June

15 4.1 Response rates There were 239 clients on the ADR client list who were in-scope of the survey after duplicates, interstate and incomplete phone numbers were removed. To obtain the required sample of 66 clients, attempts were made to contact all 239 clients. Of the 173 clients who were attempted but were not included in the survey, five were refusals. The remainder either could not be contacted for reasons such as the phone line being disconnected, no answer being obtained or the phone line being engaged, or were not included in the sample due to language difficulties, being away or being ineligible. 4.2 About the lawyer In general, clients felt satisfied with the lawyer who represented them. 89% of clients understood well the advice which their lawyer gave them, with the rest (11%) understanding the advice a little. None of the respondents did not understand at all the advice the lawyer gave them. Most clients (88%) felt that their lawyer listened well when they were explaining their situation. A further 11% felt that the lawyer listened a little, and only 1% thought that the lawyer did not listen at all. 4.3 About the conference A high percentage of clients responded positively to questions asking them what they thought of the conference. The following graph shows the satisfaction levels of the clients (Figure 5). The highest level of satisfaction (85%) was in relation to the lawyer looking after the client s interest. During the conference, 79% of people felt confident about getting involved in the negotiations. Although the number of people who felt that the conference helped them to find a solution was quite high (70%), there was also a reasonably high number of people who didn t think the conference helped them to find a solution (24%), with very few respondents (6%) being neutral in their response to this question. LAWA/3 Report June

16 Conference helped us to find a solution 24% 6% 70% Confident about getting involved in the negotiations 15% 6% 79% Confident lawyer looking after my interests 11% 5% 85% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 5. Level of satisfaction about the conference. Over 80% of respondents felt that the Chairperson explained the conference process, identified relevant issues, and made communication easier (Figure 6). Over 77% of clients felt that the Chairperson understood their side of the dispute. People were slightly less satisfied about how the Chairperson helped to generate options and suggestions for settling the dispute (68%) and helping them to reach a good result (65%). These two areas also had the highest levels of dissatisfaction. Chairperson helped to reach a good result 29% 6% 65% Chairperson helped generate options and suggestions 21% 11% 68% Chairperson made communication easier 17% 2% 82% Chairperson clarified each parties interests 14% 8% 79% Chairperson understood the dispute 17% 6% 77% Chairperson identified the relevant issues 6% 6% 88% Chairperson explained the conference process 5% 6% 89% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 6. Level of satisfaction with the conference Chairperson. LAWA/3 Report June

17 Importantly, 97% of respondents felt physically safe while at the Legal Aid Resolution Centre. The majority of conferences resulted in an agreement being reached (88%). A further four disputes (of the eight which were not resolved at the conference), reached an agreement following the conference. Of all the agreements which were made, 67% are still working between the two parties. The following cross tabulation (where the percentages are calculated across the columns) shows that the time the agreement is most likely to stop working is between four and six months after the conference (Table 2). However, the sample sizes for these comparisons are small, so should only be considered indicative of the situation. How long ago was the conference? Is the agreement still working? Yes No Count Percentage Count Percentage One month or less 5 100% 0 0% Over 1 month to 2 months 3 100% 0 0% Over 2 months to 4 months 5 83% 1 17% Over 4 months to 6 months 7 50% 7 50% Over 6 months 21 64% 12 36% Table 2. Percentages of agreements which are still working. 4.4 Overall satisfaction Overall, clients who were assisted by Legal Aid WA in the Alternative Dispute Resolution Family Conferencing Program were satisfied. 88% of clients said that they would recommend the ADR conferencing program to others. Over 91% of those clients who reached an agreement at the conference would recommend it to others, and 63% of those who did not reach an agreement would still recommend ADR. Over three quarters of the clients who were surveyed were satisfied with the Legal Aid process (Figure 7). Overall, 78% of those people who reached an agreement at the conference were satisfied, and 63% of those who did not reach an agreement were also satisfied. LAWA/3 Report June

18 Satisfied w ith the Legal Aid Process 63% 76% 78% Would recommend the ADR family conferencing program 63% 88% 91% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Agreement made No agreement made Total Figure 7. Percentages of clients who are satisfied with the Legal Aid process and the ADR program. 4.5 Comparisons to the 2001/2002 survey results The percentage of respondents who understood the advice given to them by the lawyer who represented them increased from 84% to 89%. The percentage of respondents who felt that their lawyer listened well when they explained their situation stayed constant at 88%. There was an increase in the level of satisfaction concerning all three measures regarding the conference in general that the conference helped them to find a solution, that they were confident about getting involved in the negotiations, and that they were confident the lawyer was looking after their interests. The same percentage of respondents would recommend the ADR conferencing program to others (88%) and reported that they were satisfied with the Legal Aid process (76%). 5. Survey of Grants of Aid Clients Grants of Aid clients are those that receive legal representation by a lawyer. Eligibility guidelines are in place against which applications for grants of aid are assessed. Legal representation is made available to clients in special circumstances when no other solution is considered appropriate. LAWA/3 Report June

19 5.1 Response rates There were 382 clients on the Grants of Aid client list who were in-scope of the survey after duplicates, interstate and incomplete phone numbers were removed. To obtain the required sample of 84 clients, attempts were made to contact 271 clients. Of the 187 clients who were attempted but were not included in the survey, 12 were refusals. The remainder either could not be contacted for reasons such as the phone line being disconnected, no answer being obtained or the phone line being engaged, or were not included in the sample due to language difficulties, being away or being ineligible. 5.2 Application for Legal Aid Over 70% of the clients surveyed were satisfied about their experience in applying for Legal Aid. 75% of the clients thought that it was easy to apply for Legal Aid, 74% thought that Legal Aid responded to their application quickly and promptly, and 79% thought it was easy to understand the information provided by Legal Aid regarding their application. Only about 6% of clients were dissatisfied with all three measured aspects of their experience in applying for Legal Aid. 5.3 About the lawyer Satisfaction levels with the lawyer who represented the clients ranged between 67% and 84% for various aspects of the lawyer s assistance (Figure 8). 84% of the clients were satisfied that the lawyer listened to them while they explained their situation, and 77% agreed that the advice they received from the lawyer was helpful. Approximately 75% of the clients felt that their lawyer helped them to understand the legal situation which they were in, and 72% clearly explained to them what would happen next. About 67% of clients stated that they would like to use the same lawyer again if they had a similar situation in the future, and 25% said that they would not like to use the same lawyer again. This was the statement which received the highest level of negative response. LAWA/3 Report June

20 Helped you to understand the legal situation 16% 8% 75% Listened to you when you explained your situation 8% 8% 84% Advice you received was helpful 15% 8% 77% Lawyer clearly explained what was going to happen next 20% 8% 72% Would you like to use the same lawyer again? 25% 8% 67% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 8. Level of satisfaction with the lawyer. 5.4 About the separate representative Approximately 29% of the clients surveyed remembered having a Separate Representative appointed to represent the children (63% did not have a Separate Representative, and the remainder did not know or could not remember). The satisfaction levels which the clients felt about the Separate Representative were substantially lower than what they felt about their own lawyer. The levels of dissatisfaction were a lot higher than for their own lawyer, with the neutral categories being quite low. Less than 40% of the clients who had a Separate Representative appointed for their children thought that the Separate Representative made communication easier, understood their side of the dispute, and helped the child/children to reach a good result (Figure 9). Between 46% and 52% of the clients thought that the Separate Representative explained their role in the proceedings, identified the relevant issues, generated options and suggestions for settling the dispute, and was looking after their child/children s interests. LAWA/3 Report June

21 Explained their role in my Family Court proceedings 30% 17% 52% Identified the relevant issues 42% 13% 46% Understood my side of the dispute 65% 4% 30% Made communication easier 57% 4% 39% Generate options and suggestions for settling the dispute 43% 4% 52% Helped the child/children to reach a good result 55% 9% 36% Was looking after my childs/childrens interests 39% 13% 48% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 9. Level of satisfaction with the Separate Representative. 5.5 Overall satisfaction The clients were asked to rate their overall satisfaction with the Legal Aid process. Approximately 62% were satisfied with their dealings, and 18% were dissatisfied. 5.6 Comparisons to the 2001/2002 survey results The respondents reported similar levels of satisfaction with their experience in applying for Legal Aid, however in this year s survey, the satisfaction levels were lower for each of the six questions regarding the lawyer who represented them. This year, more clients agreed with the statement: The Separate Representative helped to generate options and suggestions for settling the dispute, than in 2001/2002. The figures for the six other measures of satisfaction with the Separate Representative were equal or lower than for the last Family Law survey. In 2003/2004, 62% of the clients reported that they were satisfied overall with their dealings with Legal Aid, compared to 74% in 2001/ Overall Satisfaction of Family Law Clients A composite score for the overall level of satisfaction of Legal Aid s Family Law clients was derived. Where possible, this index was designed to allow meaningful comparisons to be made with the 2001/2002 value published in the Annual Report. LAWA/3 Report June

22 The index uses the same method as the 2001/2002 but was updated to reflect the additional questions asked in this year s survey. It is similar to the 1999/2000 index, however since the scale changed from a 4 point satisfaction rating to a five point scale in 2001/2002, comparisons to 1999/2000 figures need to be viewed with this in mind. 6.1 Methodology The index of client satisfaction was calculated by taking a weighted average of the overall satisfaction level of each Family Law client group. The respective population to sample ratios (standardised to sum to one) were used as the weights. The overall satisfaction level for each client group was calculated using a simple average of the percentage of satisfaction which resulted for a number of questions relating to client satisfaction. Questions pertaining to the outcome of the client s legal situation were excluded from the overall measure of satisfaction. Respondents who answered not applicable were excluded from the calculation of percentage for that item. All questions included in calculating the index of satisfaction had a five point response scale ranging from strongly disagree to strongly agree. The two highest ratings (agree and strongly agree) were used to calculate the percentage of satisfaction achieved for each item. The following items were included in calculating the percentage of satisfaction for the Legal Advice and Grants of Aid client groups: How well the lawyer or advisor: i. Helped the client to understand the legal situation they were in; ii. iii. iv. Listened to the client when they explained their situation to him/her; Explained what the client needed to do next (Legal Advice) or what was going to happen next (Grants of Aid); Helped the client to understand what type of assistance Legal Aid could give them (Legal Advice only); The extent to which: v. The advice the client received from the lawyer or advisor was helpful; and whether: vi. The client would like to use the same lawyer or advisor again if they were in a similar situation; LAWA/3 Report June

23 vii. viii. ix. It was easy to make an application for Legal Aid (Grants of Aid only); Legal Aid responded to the client s application quickly and promptly (Grants of Aid only); and It was easy to understand the information provided by Legal Aid regarding their application (Grants of Aid only). The following items were included in the calculation of the satisfaction of Alternative Dispute Resolution clients: How well the Conference Chairperson: i. Explained the conference process to the client; ii. iii. iv. Identified the relevant issues; Understood the client s side of the dispute; Clarified each party s interests; v. Made communication easier; vi. vii. Helped generate options and suggestions for settling the dispute; Helped the client to reach a good result; How well the ADR Coordinator, prior to the conference: viii. ix. and whether: Explained information about the conference in a way you understood; Listened to you; x. The client felt confident that their lawyer was looking after their interests; xi. xii. 6.2 Results During the conference the client felt confident about getting involved in the negotiations; and The conference helped the client to find a solution. The Alternative Dispute Resolution clients had the highest level of satisfaction with Legal Aid, followed by the Legal Advice clients and then the Grants of Aid clients. The overall weighted satisfaction level was calculated to be 74.0%. Table 3 gives the satisfaction levels for each client group and their associated levels of accuracy. LAWA/3 Report June

24 Client Group Satisfaction Level of Accuracy Legal Advice 76.5% ±5.6% Alternative Dispute Resolution 79.4% ±8.4% Grants of Aid 60.5% ±9.3% Total 74.0% ±4.3% Table 3. Satisfaction levels and the levels of accuracy achieved. These accuracy levels were calculated to provide a 95% confidence interval. That is, we can be 95% confident that the overall satisfaction level of 74.0% lies somewhere in the interval from 69.7% and 78.3%. 7. Online and Print Publications Users CLC Survey This is the third year that the Online and Print Publications Users CLC Survey was run involving a census of Community Legal Centres (CLCs). The purpose of the survey was to find out the views of the CLCs on the Legal Aid online and print publications. 7.1 Response rates There was an 89% response rate achieved for this census. Of the 28 CLCs on the Legal Aid CLC distribution list, 25 responses were achieved, with one refusal, one wrong number and one CLC not being surveyed because of appropriate staff members being away through the duration of the survey. Of the 25 responses which were obtained, six had not used the online and print publications before the survey (this was the second question asked), and were screened out from the remaining questions, leaving 19 respondents who had used the online and print publications to continue with the interview. 7.2 Knowledge about and use of the online and print publications As was discussed in Section 7.1, 24% of the CLCs had not used the Legal Aid online and print publications before this survey. The remainder of the discussion will only be relating to the 76% of CLCs who had used the online and print publications previously. Of those who have used the online and print publications, 63% use them at least once a week. LAWA/3 Report June

25 7.3 About the Legal Resource Database Of the 19 CLCs who did use Legal Aid s online and print publications, 100% had heard of the Legal Resource Database, which is restricted use only and requires a password and login. Six respondents answered Not Applicable to all of the questions regarding the Legal Resource Database, which suggests that although they are aware of the database, they do not use it or do not have access to it. In calculating the response percentages to the questions concerning the database, those who had not heard of it or responded Not Applicable were not included. Overall, the 13 CLCs that use the Legal Resource Database seem to be very satisfied with it (Figure 10). Of the 13 respondents, 12 found the information on the database useful and the menu choices to be logical, while 11 found the information they required easily. Although 10 of the respondents were able to download the information they needed quickly and easily, one disagreed. Other than the usefulness of a feedback form, this was the only statement to receive a negative response. Three of the CLCs felt that a feedback form would be useful, whilst three did not and seven people felt neutral on the matter. I found the information on the database useful 8% 92% I found the information I required easily 15% 85% The menu choices were logical and made sense I found more information that was useful, apart from what I was originally seeking 8% 15% 92% 85% I was able to download the information I needed quickly and easily 8% 8% 83% The Legal Aid search engine helped me to find information quickly and easily 33% 67% A feedback form on the website would be useful 23% 54% 23% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 10. Level of satisfaction with the Legal Resource Database. 7.4 About the Legal Aid website Three of the respondents answered Not Applicable to all of the questions regarding the Legal Aid Website, suggesting that they do not use it or don t have access to the internet. The responses of the remaining 16 CLCs were LAWA/3 Report June

26 very positive about the website (Figure 11). The only statements that received a negative response were It is a welcoming website and The font, size and colour of the text are easy to read (however this had a high satisfaction level, at 87%). Most (88%) of the CLCs agreed that the website has useful topics that meet the clients needs. Feedback from our clients has been positive 57% 43% It is a welcoming website 13% 31% 56% The content of the website is easy to understand The font, size and colour of the text are easy to read The website has useful topics that meet clients needs 19% 7% 7% 13% 87% 88% 81% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 11. Level of satisfaction with the Legal Aid website. 7.5 About the Legal Aid print publications Only one respondent answered Not Applicable to all of the statements regarding Legal Aid s print publications, suggesting that they are widely used by the CLCs. The responses to the statements concerning the print publications were quite positive, with 78% responding that the publications appear welcoming, 83% feeling that the content is easy to understand and 82% feeling that the font, size and colour of the text is easy to read (Figure 12). A very high percentage (94%) of the CLCs felt that the publications cover useful topics that meet the clients needs, and no one disagreed with this. LAWA/3 Report June

27 The publications cover useful topics that meet client needs 6% 94% The font, size and colour of the text are easy to read 18% 82% The content of the publications are easy to understand 6% 11% 83% The publications appear welcoming 6% 17% 78% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Disagree Neutral Agree Figure 12. Level of satisfaction with Legal Aid s print publications. 7.6 Overall satisfaction of the online and print publications An overall measure of satisfaction of the Legal Aid website was calculated. This was done by calculating the percentage of respondents who were satisfied for each question asking about the usefulness or appearance of the Legal Aid Legal Resource Database, website and print publications, and taking an average. Respondents who answered not applicable were excluded from the calculation for that particular item. In total, 15 items were included in the index (six items were from question 6 on the questionnaire, five from question 7 and the other four items were from question 8). The overall satisfaction with the Legal Aid website was calculated to be 79.8%. The level of accuracy for this estimate is ±6.7%, meaning that we can be 95% certain that the true level of satisfaction with the website lies between 73.1% and 86.5%. The CLCs were asked to provide information on what they consider the best and worst aspects of the Legal Aid online and print publications to be. In general, the comments provided support the quantitative findings from previous questions in the survey. Each of the comments is included (in de-identified format) in Appendix C. These comments are interesting and informative to read to obtain a better understanding of what the users do and do not like about the online and print publications. LAWA/3 Report June

28 7.7 Comparisons to the 2002/2003 survey results This year s CLC survey has exactly the same content as the 2002/2003 survey, making comparison straightforward. The information on the Legal Resource Database was rated as more useful, with more logical menu choices than last year, however it rated lower regarding the ease of finding information. The Legal Aid website had a lower satisfaction rating for each of the five questions asked, with the most noticeable decreases for it is a welcoming website (down from 94% to 56%) and feedback from our clients has been positive (down from 75% to 43%), however the latter had no dissatisfied responses and the remainder were neutral. The satisfaction levels for Legal Aid s print publications were very similar to last year, with the satisfaction levels for the four measures ranging from 78% to 94% in this year s survey and from 85% to 95% in last year s. 8. Conclusions Three Family Law client groups were included as part of this year s client satisfaction surveys: Legal Advice, Alternative Dispute Resolution and Grants of Aid. Satisfactory response rates were achieved for each of the surveys. A census of all 28 Western Australian Community Legal Centres was also carried out, in order to investigate how useful Legal Aid WA s online and print publications are and how frequently they are accessed by the CLC s. The composite satisfaction index for this year s survey estimated the overall satisfaction rating to be 74.0%, with an achieved accuracy level of ±4.3%, meaning that we can be 95% confident that the true rate of satisfaction lies between 69.7% and 78.3%. In 2001/2002, the composite index included similar questions and gave an overall satisfaction rating of 74.3%, with an achieved accuracy level of ±4.2%. In this year s survey, the Alternative Dispute Resolution clients were the most satisfied group, with an overall satisfaction rating of 79% (compared to 72% in 2001/2002). Legal Advice clients had a satisfaction level of 76% (compared to 69% in 2001/2002), and Grants of Aid clients had a rating of 61% (compared to 81% in 2001/2002). LAWA/3 Report June

29 Appendix A Questionnaires Legal Advice Questionnaire Alternative Dispute Resolution Questionnaire Grants of Aid Questionnaire Introductory letter sent to CLCs Online and Print Publications Users CLC Questionnaire LAWA/3 Report June 2004

30 Appendix B Formulae Used Formulae used to calculate the level of accuracy of estimates LAWA/3 Report June 2004

31 Accuracy for an individual client group The following formula was used to calculate the levels of accuracy of results for each individual client group error = ± 1.96 p p n N where n is the sample size, N is the population size, and ( 1 ) 100, p is the satisfaction level expressed as a proportion. Accuracy for the overall sample Since the sampling methodology assigned unequal probabilities of selection to the clients in the different Family Law client groups, this had to be taken into account when calculating the accuracy of the overall customer satisfaction levels. The following formula is appropriate to use in this situation. 3 i= 1 ( ) 100, error = ± w i e i where w i is the weight of client group i, and e i is the error associated with client group i. LAWA/3 Report June 2004

32 Appendix C Item Level Responses Item Level Responses for Legal Advice Survey Item Level Responses for Alternative Dispute Resolution Survey Item Level Responses for Grants of Aid Survey Item Level Responses for Online and Print Publications Users CLC Survey LAWA/3 Report June 2004

33 Item Level Responses from the Legal Advice Survey Q1. Just thinking back to that time, was the lawyer or advisor you spoke to male or female? Count Percentage Male % Female % Can t remember/dk 4 1.6% Total % Q2. Did you go to one of Legal Aid s offices or did you speak to the lawyer over the telephone? Count Percentage Legal Aid office % Over the phone % Other % Total % Q3. How did you find out about the services offered by legal aid? Count Percentage Legal Aid office % Over the phone 6 2.4% Word of mouth % Law Courts % Centrelink % Phone book 9 3.6% Media 4 1.6% Brochures 3 1.2% Internet 3 1.2% Other % Can t remember/don t know % Total % LAWA/3 Report June 2004

34 Q.4 Please rate the assistance the lawyer or advisor provided to you. Helped you to understand the legal situation you were in Listened to you explain your situation to him/her The advice you received was helpful Clearly explained to you what you needed to do next Helped you to understand the type of assistance Legal Aid could give you Would you like to use the same lawyer or advisor again? Strongly disagree Disagree Neutral Agree Strongly agree N/A Total Q5. Which of the following statements is closest to the way you feel about the advice the lawyer gave you about what could be done to help with your problem? I felt very satisfied with the advice the lawyer gave me about what could be done I felt fairly satisfied with the advice the lawyer gave me about what could be done I had some doubts about the advice the lawyer gave me about what could be done, but I went along with it anyway I was not at all satisfied with the advice the lawyer gave me about what could be done to help with my problem Count Percentage % % % % Cant remember/don t know 3 1.2% Total % LAWA/3 Report June 2004

35 Q6. Which of the following categories best describes the assistance you received? (multiple response allowed) Count Percentage Advice only % Advice and letter drafted % Advice and negotiations or phone call made Advice and court documentation prepared % % Cant remember/dk 5 2.0% Total % Q7. How helpful or useful was the advice you received in helping you to progress your legal issue? Count Percentage Extremely useful % Fairly useful % Not very useful % Not at all useful % Cant remember/don t know 1 0.4% Did not give any advice 7 2.8% Total % Q8. How easy was it to get help once you attended the legal aid office (long time etc)? Count Percentage Extremely useful % Fairly useful % Not very useful % Not at all useful % Cant remember/don t know % Did not give any advice % Total % LAWA/3 Report June 2004

36 Q9. Did you go to court about the problem you spoke to the lawyer or advisor about? Count Percentage Yes % No % Can t remember/don t know 2 0.8% Total % Q10. How useful was the advice and other assistance you received in preparing you for your court appearance? Count Percentage Extremely useful % Fairly useful % Not very useful % Not at all useful % Can t remember/don t know 0 0.0% Did not give any advice % Total % Q11. To what extent do you think your situation has changed as a result of contacting Legal Aid? Would you say it has: Count Percentage Greatly benefited % Somewhat benefited % Stayed the same % Somewhat suffered % Greatly suffered % Total % Q12. Which of the following age groups are you in? Count Percentage % % % % % 65 or over 5 2.0% Total % LAWA/3 Report June 2004