Technology Initiative Grant Iowa Case Management & Database Integration Project. Legal Services Corporation of Iowa now known as Iowa Legal Aid

Transcription

1 Technology Initiative Grant Iowa Case Management & Database Integration Project Legal Services Corporation of Iowa now known as Iowa Legal Aid Edward Saunders University of Iowa School of Social Work Program Evaluator June

2 INTRODUCTION The offices of the Legal Services Corporation of Iowa (now known as Iowa Legal Aid) extend across the state of Iowa and include offices in Cedar Rapids, Council Bluffs, Des Moines, Dubuque, Iowa City, Mason City, Sioux City, Ottumwa, Waterloo, and HELP Legal Assistance in Davenport. Prior to January of 2003, all of these separate sites operated using a software application called Turbo Cases. However, two separate legal services offices in Iowa, the Legal Aid Society of Polk County and the Hotline, operated using two different software programs Kemp s Clients for Windows and TIME, respectively. Prior to the merger of the Legal Aid Society of Polk County and the Legal Services Corporation of Iowa in January of 2003, it became apparent that there needed to exist one uniform software program that the merged program could utilize. Therefore, in October of 2002, after extensive research, the Legal Services Corporation of Iowa began operating on a web-based software program called Pika. The objective of this web-based program was to allow for the integration of the databases of all of the statewide offices. This conversion was completed by December 28, Therefore, when the Legal Aid Society of Polk County and the Legal Services Corporation of Iowa merged in January of 2003 and became Iowa Legal Aid, the statewide offices had a uniform software program by which they operated. STUDY METHOD In order to assess the effectiveness of Pika and to compare Pika to programs used previously, it was necessary to conduct a pre- and post-test evaluation of the program. In March of 2002, a survey was distributed to advocates as well as support staff at each of the Legal Services offices in Iowa. In April of 2003, after the conversion to Pika, a second survey was distributed to advocates and support staff in order to determine the effectiveness of the new web-based database program. Support Staff and Advocate Pretest Surveys The pre-test surveys that were given to the support staff and to the advocates in March of 2002 were very similar. The survey distributed to the Support Staff first asked them to record how much time they had spent on a certain activity in a given week. The respondents recorded their estimated times for each activity in a table. In the second section of the survey, the respondents were asked to rate the efficiency and usability of their current software using a five-point rating scale from very poor to very good. The six categories that were rated were (1) Entry of time records, (2) Docketing and Tickler, (3) Reports, (4) Conflict checking, (5) Tracking compliance, and (6) Case closing. The final section of the survey asked respondents to indicate the availability and rate the efficiency of various features of the current office software. The participants were first asked to indicate whether or not that feature was available on their current software and then to rate the efficiency/usability of that particular feature. The five-point rating scale ranged from very poor to very good The four 2

3 features that were included in this section of the survey were (1) used for VLP referrals, (2) Used for Matters reporting, (3) Contains standardized intake questions, and (4) Contains standardized form letters. In addition, the respondents were also allowed to list any other features of the software that they deemed worthy of noting. These responses were qualitatively examined. The Advocate Pre-test asked advocates to give an average rating of their use of the current case management/database software. This was followed up with a series of questions focused on the use and efficiency of the software for timekeeping, looking up client information, LSC Compliance (ie. Retainer, alien, case information disclosure, etc.), and case management (tickler, calendar, etc.). At the end of the survey were two questions that were qualitative in nature. The first of these two qualitative questions asked the advocates to list any other functions or activities that they used with their current software. The second question asked the advocates to list any improvements or additional features that they would like to see in a new case management software program. Support Staff and Advocate Posttest Surveys The posttests that was given to both the advocates and the support staff were identical to the pretest with the exception of a new question at posttest that asked both the support staff and the advocates to rate the quality of the Pika software program as compared to the software program used previously by the various sites (i.e. Turbo Cases, Kemp s Clients, TIME). The respondents rated the compared Pika to the previous software program using a five-point rating scale ranging from very inferior to very superior. In this way, the program evaluators could make a more accurate conclusion regarding the effectiveness of Pika and the respondent s satisfaction with the new program. RESULTS Support Staff Findings A total of 17 support staffers were randomly matched at both pretest and posttest based on their office sites. Although other staff submitted surveys, they were not used in the analysis because there was a pretest but no posttest, or a posttest but no pretest. In order to compare the change in behavior and attitudes from pre to post, both surveys were necessary for the analysis. The analysis is based on the total group of 17 matched responses. Support staff from each of the following offices were matched for analysis in this evaluation. Office Frequency Percent Central Office 5 29 Iowa City 1 6 Hotline 1 6 Davenport 2 12 Waterloo 1 6 North Central

4 Ottumwa 1 6 Northeast 1 6 Southwest 2 12 Cedar Rapids 1 6 Data Entry for Intakes: Pre and Post At pretest, the range of time spent on data entry for intakes among the 17 staff members was 0 to 15 hours for the identified week. The mean was 5.26 hours and the median was 3 hours. The standard deviation was 5.35 hours. At posttest, the range of time spent on data entry for intakes among the 17 staff members was 0 to 16.5 hours for the identified week. The mean was 6.3 hours and the median was 5.25 hours. The standard deviation was 5.8 hours. A paired t-test was performed to determine whether the change in hours spent on intake data entry had changed significantly from pretest to posttest. The results of the analysis showed there was no statistically significant change (t(16)= -.71, p=. 468). As seen by the mean (average) scores, there was actually a small increase in the average weekly amount of time spent by support staff in data entry for intakes at posttest compared to pretest (6.3 hours versus 5.26 hours). Batch Data Entry for Timekeeping: Pre and Post At pretest, the range of time spent on batch data entry for timekeeping among the 17 staff members was 0 to 6.5 hours for the identified week. The mean was 1.91 hours and the median was 1.5 hours. The standard deviation was 2.08 hours. At posttest, the range of time spent on batch data entry for timekeeping among the 17 staff members was 0 to 3.75 hours for the identified week. The mean was.67 hours and the median was 0 hours. The standard deviation was 1.22 hours. A paired t-test was performed to determine whether the change in hours spent on batch data entry had changed significantly from pretest to posttest. The results of the analysis showed there was a marginally statistically significant change (t(16)= 2.05, p=.06). As seen by the mean (average) scores, there was a decrease in the average weekly amount of time spent by support staff in batch data entry for timekeeping at posttest compared to pretest (.67 hours versus 1.91 hours). Running Reports: Pre and Post 4

5 At pretest, the range of time spent on running reports among the 17 staff members was 0 to 4.0 hours for the identified week. The mean was.81 hours and the median was.25 hours. The standard deviation was 1.24 hours. At posttest, the range of time spent on running reports among the 17 staff members was 0 to 6 hours for the identified week. The mean was 1.38 hours and the median was.25 hours. The standard deviation was 1.94 hours. A paired t-test was performed to determine whether the change in hours spent on running reports had changed significantly from pretest to posttest. The results of the analysis showed there was no statistically significant change (t(16)= -1.00, p=. 33). As seen by the mean (average) scores, there was actually an increase in the average weekly amount of time spent by support staff in running reports at posttest compared to pretest (1.38 hours versus.81 hours). Tickler, Calendar and other Case Management Functions: Pre and Post At pretest, the range of time spent on case management functions among the 17 staff members was 0 to 5.5 hours for the identified week. The mean was 1.71 hours and the median was 1 hour. The standard deviation was 1.88 hours. At posttest, the range of time spent on case management functions among the 17 staff members was 0 to 17 hours for the identified week. The mean was 3.28 hours and the median was 1.5 hours. The standard deviation was 4.71 hours. A paired t-test was performed to determine whether the change in hours spent on case management functions had changed significantly from pretest to posttest. The results of the analysis showed there was no statistically significant change (t(16)= -1.50, p=. 15). As seen by the mean (average) scores, there was actually an increase in the average weekly amount of time spent by support staff on case management functions at posttest compared to pretest (3.28 hours versus 1.71 hours). Closing Intakes and Files: Pre and Post At pretest, the range of time spent on closing intakes and files among the 17 staff members was 0 to 3.5 hours for the identified week. The mean was 1.13 hours and the median was 1 hour. The standard deviation was 1.15 hours. 5

6 At posttest, the range of time spent on closing intakes and files among the 17 staff members was 0 to 8 hours for the identified week. The mean was 2.13 hours and the median was.25 hours. The standard deviation was 2.78 hours. A paired t-test was performed to determine whether the change in hours spent on closing intakes and files had changed significantly from pretest to posttest. The results of the analysis showed there was no statistically significant change (t(16)= -1.36, p=. 19). As seen by the mean (average) scores, there was actually an increase in the average weekly amount of time spent by support staff in closing intakes and files at posttest compared to pretest (2.13 hours versus 1.13 hours). Rating efficiency and usability of software At pretest, support staff were asked to rate the efficiency and usability of their current software (ie., Clients, TurboCases, TIME) for several functions: case intakes, entry of time records, docketing and tickler, reports, conflict checking, tracking compliance (retainers, case information disclosure, etc.), and case closing. The possible response categories for each were: very poor, poor, fair, good, and very good. For purposes of data analysis, each of these responses were coded on a continuum from 1 very poor to 5 very good. Consequently, in reading this section of the report, higher scores would indicate more favorable responses, since 5 represents the best possible response very good. At posttest, the same questions and responses were used, although they were now being asked to rate the efficiency and usability of Pika, the new software. Again, the responses are coded 1 to 5, with 5 indicating Very good. Case Intakes: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for Case Intakes among the 17 staff members was 3.7. The median was 4.0. The standard deviation was.47. At posttest, the average rating of efficiency and usability of Pika for Case Intakes among the 17 staff members was 4.4. The median was 4.0. The standard deviation was.51. A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned case intake activity. The results of the analysis showed there was a statistically significant change (t(16)= -4.95, p<. 001). As seen by the mean (average) scores, there was an increase in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (4.4 versus 3.7) for case intake efficiency and usability. 6

7 Entry of Time Records: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for entry of time records among the 17 staff members was 3.7. The median was 4.0. The standard deviation was.61. At posttest, the average rating of efficiency and usability of Pika for entry of time records among the 17 staff members was 4.4. The median was 4.0. The standard deviation was.50. A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned entry of time records. The results of the analysis showed there was a statistically significant change (t(9)= -2.33, p=. 05). As seen by the mean (average) scores, there was an increase in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (4.4 versus 3.7) for entry of time records efficiency and usability. Docketing and Tickler: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for docketing and tickler among the 17 staff members was 3.6. The median was 4.0. The standard deviation was.74. At posttest, the average rating of efficiency and usability of Pika for docketing and tickler among the 17 staff members was The median was 3.0. The standard deviation was.96. A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned docketing and tickler activity. The results of the analysis showed there was no statistically significant change (t(11)= 1.33, p=.21). As seen by the mean (average) scores, there was a small decrease in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (3.38 versus 3.6) for docketing and tickler efficiency and usability. Reports: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for Reports among the 17 staff members was 3.3. The median was 3.0. The standard deviation was.61. At posttest, the average rating of efficiency and usability of Pika for Reports among the 17 staff members was 3.6. The median was 3.5. The standard deviation was A paired t-test was performed to determine whether staff attitudes had changed significantly 7

8 from pretest to posttest, based on the change in software as it concerned Reports. The results of the analysis showed there was no statistically significant change (t(10)= -1.46, p=.17). As seen by the mean (average) scores, there was an increase in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (3.6 versus 3.3) for Reports efficiency and usability. However, this increase was not statistically significant. Conflict Checking: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for Conflict Checking among the 17 staff members was 3.7. The median was 4.0. The standard deviation was.62. At posttest, the average rating of efficiency and usability of Pika for Conflict Checking among the 17 staff members was The median was 4.0. The standard deviation was.71. A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned Conflict Checking. The results of the analysis showed there was a statistically significant change (t(15)= 0, p=1.0). As seen by the mean (average) scores, there was a slight decrease in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (3.65 versus 3.7) for conflict checking efficiency and usability. Tracking Compliance (Retainers, Case info disclosure, etc.): Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for Tracking Compliance among the 17 staff members was The median was 4.0. The standard deviation was.59. At posttest, the average rating of efficiency and usability of Pika for tracking compliance among the 17 staff members was The median was 4.0. The standard deviation was A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned tracking compliance. The results of the analysis showed there was no statistically significant change (t(13)= -1.31, p=.21). As seen by the mean (average) scores, there was an increase in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (4.13 versus 3.73) for tracking compliance efficiency and usability. However, this increase was not statistically significant. 8

9 Case Closing: Pre and Post attitudes toward software At pretest, the average rating of efficiency and usability of their software for Case Closing among the 17 staff members was 4.0. The median was 4.0. The standard deviation was.38. At posttest, the average rating of efficiency and usability of Pika for Case Closing among the 17 staff members was 4.3. The median was 4.0. The standard deviation was.73. A paired t-test was performed to determine whether staff attitudes had changed significantly from pretest to posttest, based on the change in software as it concerned case closing. The results of the analysis showed there was no statistically significant change (t(12)= -1.48, p=.17). As seen by the mean (average) scores, there was an increase in the average satisfaction rating of the Pika software at posttest compared to other software at pretest (4.3 versus 4.0) for case closing efficiency and usability. However, this increase was not statistically significant. Special features of software Support staff were asked, on both the pretest and posttest, to respond to a series of items regarding the efficiency and usability of features of their software for VLP referrals, for Matters reporting, contains standardized intake questions, and contains standardized form letters. However, the responses were so irregular (left largely uncompleted in both surveys) that meaningful pre-post quantitative analysis of this data is inappropriate. Overall rating of Pika At posttest, support staff were asked to rate Pika compared to the software previously used in your office. The response categories were Very inferior, Inferior, About the same, Superior, and Very superior. The following table shows the number and percent of responses for each category. Response category Frequency Percent Very inferior 0 0 Inferior 1 6 About the same 3 18 Superior 9 53 Very superior 4 23 This data suggests that fully 76% of support staff find Pika, overall, to be superior or very superior compared to their previous software. Only one support staff member reported it was inferior to their previous software. 9

10 RESULTS Advocate Findings A total of 32 advocates were randomly matched at both pretest and posttest based on their office sites. Although several other advocates submitted surveys, they were not used in the analysis because there was a pretest but no posttest, or a posttest but no pretest. In order to compare the change in behavior and attitudes from pre to post, both surveys were necessary for the analysis. The analysis is based on the total group of 32 matched responses. Advocates from each of the following offices were matched for analysis in this evaluation. Frequency of Software Use Office Frequency Percent Central Office 4 12 Iowa City 2 6 Hotline 1 3 Davenport 6 19 Waterloo 3 9 North Central 4 12 Ottumwa 1 3 Southwest 2 12 Cedar Rapids 4 12 Northwest 4 12 Program Support 1 3 At both pretest and posttest, advocates were asked the frequency of their use of the office case management/database software. (At posttest, this software was identified as Pika.) Three response categories were used: daily, 2-3 times per week, and less than once a week. The following table shows the usage at pretest and posttest. Frequency of Use Pretest Posttes t Daily times per week 13 3 Less than once a week

11 The data in the table shows that frequency of case management software use had increased, on a daily basis, from 81% to 97%. Rating the Efficiency and Usability of Software Functions Advocates were asked to rate the efficiency and usability of the following functions at both pretest and posttest: timekeeping, looking up client information, LSC compliance (ie., retainer, alien, case information disclosure, etc.), and case management (tickler, calendar, etc.). Each question had two parts. In the first part, advocates were asked if they used the software for the function. In the second part, they were asked to rate its efficiency and usability using a five category response set. The possible responses were: very poor, poor, fair, good, and very good. These were coded with values of 1 to 5, respectively. Consequently, high scores would reflect positive attitudes toward the efficiency and usability of the function. Data for each function are described, in turn, below. Timekeeping: Pre and Post At pretest, 31% of advocates used the software for timekeeping. At posttest, 97% of advocates used Pika for timekeeping. This represents a positive change of 66% from pretest to posttest. At pretest, the mean (average) rating of the software for timekeeping was 3.5. The median was 4.0. The standard deviation was 1.2. At posttest, the mean (average) rating of the software for timekeeping was 4.1. The median was 4.0. The standard deviation was.66. A paired t-test was performed to determine if the change from pretest satisfaction to posttest satisfaction was statistically significant. The results of this test showed there was no statistically significant change: t(9)= -1.3, p=.24). However, the change from pretest to posttest was positive. Looking up client information: Pre and Post At pretest, 97% of advocates used the software for looking up client information. At posttest, 97% of advocates used Pika for looking up client information. There was no change in usage from pretest to posttest. At pretest, the mean (average) rating of the software for looking up client information was 3.6. The median was 4.0. The standard deviation was.87. At posttest, the mean (average) rating of the software for looking up client information was 4.0. The median was 4.0. The standard deviation was

12 A paired t-test was performed to determine if the change from pretest satisfaction to posttest satisfaction was statistically significant. The results of this test showed there was no statistically significant change: t(30)= -1.5, p=.14). However, there was a positive change from pretest to posttest. LSC Compliance (ie., retainer, alien, case information disclosure, etc.): Pre and Post At pretest, 50% of advocates used the software for LSC compliance. At posttest, 78% of advocates used Pika for compliance. This represents a positive change of 28% from pretest to posttest. At pretest, the mean (average) rating of the software for LSC compliance was 3.3. The median was 4.0. The standard deviation was 1.2. At posttest, the mean (average) rating of the software for LSC compliance was 3.9. The median was 4.0. The standard deviation was.96. A paired t-test was performed to determine if the change from pretest satisfaction to posttest satisfaction was statistically significant. The results of this test showed there was a statistically significant change: t(15)= -2.3, p=.04). There was a significant change in attitudes from pretest to posttest. Case Management (tickler, calendar, etc.): Pre and Post At pretest, 45% of advocates used the software for case management. At posttest, 78% of advocates used Pika for case management. This represents a positive change of 33% from pretest to posttest. At pretest, the mean (average) rating of the software for case management was 3.2. The median was 3.0. The standard deviation was 1.0. At posttest, the mean (average) rating of the software for case management was 3.6. The median was 3.5. The standard deviation was.99. A paired t-test was performed to determine if the change from pretest satisfaction to posttest satisfaction was statistically significant. The results of this test showed there was no statistically significant change: t(12)= -1.2, p=.24). However, there was a positive increase in satisfaction from pretest to posttest. 12

13 Other functions: Pretest When asked at pretest if there were any other functions or activities they used their current software to perform, Advocates wrote: Case notes, correspondence, conflict checks, quick-> look up phone #, and run reports. Other functions: Posttest When asked at posttest if there were any other functions or activities they used their Pika software to perform, Advocates wrote: Use it to record notes about casework and store case letters. These functions are fair, better work processing feature; have caselist show/highlight case with associated ticklers; fewer bugs, typing case notes for use by others if I m out of the office, I use most of the Pika reporting functions on a fairly regular basis, conflict checking, client reports for staff meetings; time sheet- use record to complete form, time sheets- keeping track of advocates work, Matter reports, printing out weekly case reports; looking for past intakes/cases on client/client problem, case notes, running reports, and Use Google search engine on home page. Also use Quick Links. Improvements sought in software: Pretest When asked at pretest what additional features they would find useful in a new case management/database software program, Advocates wrote: Ability to export more information opposing parties, name, address, phone, nature of problem (narrative), income info, A place for contact information in respect to adverse counsel, such as name, address, phone, If it clearly indicated a case had been accepted for representation that would be better, Designed for Windows operating system; ability to calculate and total applicant income and non-exempt assets, facilitating real time intakes rather than a manual application followed by computer input of personal and financial data; greater ease in modifying aspects of the software to respond to new reporting requirements; ability to troubleshoot or network with other legal services organizations using the same software; ability to input such current reporting requirements as the Matters form and the Referrals form; and ability to easily interface with WordPerfect in such areas as form letters, more fields for data, increased speed/efficiency, statewide access to data base ability to enter notes from telephone calls, progress notes, etc. and enter time directly rather than writing down and someone else entering later and deciphering very abbreviated notes; allow for more detailed notes of activities, Having OP listed on same page as client name and basic info, Place case notes with time allow directly to database, then anyone can see what is happening on the case, but may want to limit access to each local office, not allow entire system to review. The conflict check is cumbersome- can t check other offices easily; some way to merge database info into letters, pleadings, and other documents, Windows program; Cross reference by all data fields, keeping running client case notes, using formatted advice letters; keeping time sheets info with little entry, mergable forms, tickling, integrate with word processor, being able to input time directly by advocates would save a lot of duplication, allow multiple opposing parties to be entered in single case; also, allow multiple clients, such as husband and wife or co-tenants without having to do multiple intakes. Also, allow 13

14 conflict checks to access this additional info, I would like to be able to store more client info, e.g., having a way to show that the phone number is client s neighbor, rather than clients or add more to case notes, United Way wants us to track how many of UW s top 50 employers have employees using our services. It would be nice if we would get rid of recording time and then support staff having to enter it, Efficiency often depends on the support staff s ability to use or update systems, You should be able to see more details about problem, comments about clients, how resolved. It would prevent having to get the hard copy of all closed files. Run reports on open cases my old job had a system where you got a report with name of client, case # and next action to take not the same as tickler because it was your own need action list and you could go in and edit the action as you ran the report, and increased speed of running reports- very slow now; be able to add case notes directly to computer. Improvements sought in Pika: Posttest When asked at posttest what additional features they would like to see in Pika, Advocates wrote: An easier to read printable intake, Drop down menu item for my open cases on Quick Links; Ability to show calendar items for the whole office at a glance, Be able to bold, highlight, etc. in Summary-Public Notes field, I can t figure out how to tickle future projects- that may be due to my lack of training, Make it easier to enter future calendar entries. I understand I have to retype tickler and client s name to enter into tickler system, I would like to be able to minimize one or more Pika screens and go to another- back and forth between screens without losing information, In the reporting features, it would be nice for advocates to be listed by name rather than by number. Also, reports should list totals; e.g., the Docket Report should list the total # of open cases, the Advanced Activities Report should list total time, etc., Better attachment opening feature- lots of problems/annoyance over it, There have been several times that a client can t be found in Pika; then 2 hours later it works. Doesn t check for conflicts correctly, always. I can see this becoming an ethical issue at some point. We have done several intakes that we later figure out should have been a conflict. Pika is a vast improvement but still needs some little things worked out. It would also be nice if there was someone we could direct compliance questions to re: Pika, Splitting between 2 funding sources is a hassle-would like to enter one time and then have some way to allow Pika to split; conflict screening seems to still be a problem; auto re-tickle- if not get to on the day then auto re-tickle to next day. I didn t think I would like, but it is much better than anticipated. Would like some things to default, like PAI, to no so don t have to click every time. When on search screen, allow us to use 2 letters of name to get person, default to case and casework-other when in time record; have a real tickler system, to-do list weighted by priority; documents pertaining to case could be somehow attached to Pika file (letters, filings, etc.), More info on how to generate reports for evaluations, etc. (Not a one-time training but written materials I can refer to). PS: I think it s probably Mozilla, not Pika, but we still have much difficulty opening non-word Perfect documents that are sent as attachments to . There needs to be better training/communication about this since many other sources use Word, Make it go faster! Should be able to program and save specified reports, reduce the number of blanks/pull-downs on timeslips, e.g., if case name is pulled down, other boxes should automatically default to case ; cumbersome when switching screens, etc. often can t just return but have to start over, e.g., timeslips for someone else s case, when 14

15 saved, won t go back to calendar but goes to case then have to go back. For me, [Pika] is worse/more time consuming because now I m doing the input. [Need] more secure system for data changing information, etc., getting changed improperly, Add TAX/TXC to the Matters report under LSC problem code at Matters Information; Compliance- we used to have the pink sheets where attorneys could indicate compliance info. We have been instructed not to make any changes on the intake- only intake personnel can do that. How should attorneys indicate compliance info? It appears that managing attorneys, office managers and attorneys have problems trying to retrieve data inputs- Matter reports, etc., Sort/filter with other criteria than the criteria on that page. Print out my open intakes and files and import them to a list where I have the next step needed on the case next to their name. I use Word Perfect for this, but have to type in cases manually. It would be nice to have intakes print off a compliance document checklist like the pink sheets contained that we no longer use, e.g., SOF, CID, RET, etc., and improve calendar functions. Overall rating of Pika At posttest, Advocates were asked to rate Pika compared to the software previously used in your office. The response categories were Very inferior, Inferior, About the same, Superior, and Very superior. The following table shows the number and percent of responses for each category. Response category Frequency Percent Very inferior 0 0 Inferior 2 7 About the same 1 3 Superior Very superior This data suggests that fully 90% of Advocates find Pika, overall, to be superior or very superior compared to their previous software. Only two advocates reported it was inferior to their previous software. Support Staff Survey Summary and Discussion A total of 17 support staff in 10 sites/programs responded to questionnaires distributed in March 2002 and April 2003 regarding their use of case management software and their attitudes toward its different functions. These represented matched surveys that were analyzed for changes from pretest to posttest. 15

16 An analysis of time spent on the various tasks performed by support staff was conducted using pretest and posttest data. The analysis found that there was no significant change in amount spent on the following tasks from pretest to posttest: data entry for intakes, running reports, case management functions (tickler and calendar), and closing intakes and files. However, there was a marginally significant change for batch data entry for timekeeping. When support staff were asked to rate the efficiency and usability of their software for these functions at pretest and posttest, there was no significant change reported for the following tasks: docketing and tickler, reports, conflict checking, tracking compliance, and case closing. However, there was a significant change in the efficiency and usability of Pika (compared to the previous software) for: case intakes and entry of time records. Fully 76% of support staff reported that, overall, Pika was superior or very superior to their previous software. Advocate Survey A total of 32 advocates in 11 sites/programs responded to questionnaires distributed in March 2002 and April 2003 regarding their use of case management software and their attitudes toward its different functions. These represented matched surveys that were analyzed for changes from pretest to posttest. The frequency of daily use of office management software increased from 81% at pretest to 97% at posttest. The following table shows the average (mean) efficiency and usability ratings of office management software from pretest to posttest (Pika software). Task Mean- Pre Mean- Post (Pika) Significance level* Timekeeping Looking up client info LSC Compliance Case Management Note: A significance level of.05 is considered statistically significant by researchers. The above table shows that the reported changes from pretest to posttest were all positive, although the changes were only statistically significant for LSC Compliance. Fully 90% of advocates reported that, overall, Pika was superior or very superior to their previous software. When asked how Pika might be improved, advocates suggested a host of ideas focused on 16

17 enhanced calendar functions, increased speed, ability to split cases by funding, enhanced conflict checking capabilities, improved reporting functions, multiple screen capability, sort/filter abilities, and Word Perfect capability (to name a few). Each can be reviewed to determine whether the system can accommodate the changes or whether additional training may be necessary to maximize features that already exist. In conclusion, both support staff and advocates were highly complementary of Pika as a case management software package when they compared it to their previous software. Support staff felt Pika was significantly more efficient and usable for case intakes and entry of time records than their previous software. It was also reportedly more effective for case closings, although not significantly greater. Advocate staff are using the Pika software more frequently than the previous case management software and find it especially useful for LSC compliance requirements. However, they also rated it higher on all other features than the previous software. Suggested improvements to the system were offered to enhance effectiveness. 17