Using Google Analytics and Event Logs to Improve Help Documentation. Steve Lonn & Margaret Wagner University of Michigan

Transcription

1 Using Google Analytics and Event Logs to Improve Help Documentation Steve Lonn & Margaret Wagner University of Michigan 8th Sakai Conference Newport Beach 4-7 December 2007

2 Sakai s Help Documentation Out of the box, pretty generic Covers the tools in the release (not contrib) Requires customization for institutional branding and specific help documents that might need to be added for specific implementations 2

3 About Michigan s Implementation of Sakai: CTools Fall 2007 Stats: Average 18,000 users per day 7,600 peak concurrent use 3,700 course sites 5,300 project sites (so far) 2007 Instructor / Student Survey: 99% of students have used CTools 85% of instructors have used CTools 3

4 Our Help Docs Combination of original documents and modified Sakai documentation (added as new tools go live) LOTS of custom pages for our implementation: getting help, glossary, known issues, etc. Quick Demo 4

5 Why Collect Data on Help? Helps identify tools / functions that users have difficulty with Helps improve training & support Helps direct development / improvement efforts Helps improve Help docs! 5

6 Getting Data on Our Help Docs Started with a simple hit counter Wasn t always accurate Didn t have a lot of information (who, where from, etc) Also had new web-based help form we wanted to track Sakai just started tracking help events in 2.3 Other sources: User surveys, training sessions, help requests, etc. 6

7 Enter Google Analytics Google Analytics is a FREE service up to 5,000,000 page views (hits) per month TONS of data about users, content, browsers, etc. Exportable, , easy web access Coupled with Sakai events, makes it easy to track and pull data about users quickly 7

8 Setting Up Google Analytics Google Analytics uses a short javascript to track <script src=" type="text/javascript"> </script> <script type="text/javascript"> _uacct = "UA "; urchintracker(); </script> Help docs owner puts script at bottom of every page Data starts showing up after about 24 hours 8

9 Main Screen 9

10 Top Content 10

11 Page Detail - Segmenting 11

12 Keywords 12

13 Our Help Form Web-based help form -- available within Help Docs and by clicking Contact Us on bottom of CTools pages 13

14 Tracking Goals Google Analytics is designed for marketing, but you can manipulate the system to track other things... 14

15 Visualizing Goals 15

16 About Your Visitors 16

17 State 17

18 City 18

19 Ann Arbor Network Locations 19

20 New vs. Returning Users 20

21 Selected Language United States English English (not specified) Korean Chinese - Cantonese French Not Set Chinese - Taiwan English - Great Britain Japanese German 21

22 Average Page Views Per Visit 22

23 Return Visits / Depth of Visit 23

24 Browsers 24

25 Internet Explorer Versions 25

26 Operating Systems 26

27 Which OS With Which Browser? 27

28 Screen Colors 28

29 Screen Resolution 29

30 Connection Speed 30

31 Flash Versions / Java 31

32 Compare Any Time Range 32

33 Content Changes 33

34 Other Comparisons 34

35 Google Summary LOTS of data to sift through Can give quick glances at trends, and allow deep drilldowns FREE & relatively easy to set up 35

36 Two events: Sakai Event Logs help.access - Tracks all user clicks to help guide REF is either null for menubar link or has tool ref for? link help.search - Text of keyword searches from within help tool 36

37 One Month for help.access Views Page 9464 Help Link on Menu Bar 415 Discussion 389 Announcements & MOTD October Chat 252 Web Content & Gateway pages 138 Worksite Information on Home Page 132 Resources 117 Assignments 35 Gradebook 17 Syllabus 16 Site Info 14 Schedule 12 Wiki 11 Synoptic Message Center 11 Forums 49 Other 37

38 Tracking help.access 2007: Help link on menu bar 25,000 22,413 20,000 15,000 15,167 10,000 9,464 5,000 5,543 4,336 4,140 6,317 5,262 2,486 1,813 2,352 0 January February March April May June July August September October November 38

39 One month for help.search October 2007 Views Search Term 30 discussion 30 grades 27 wiki 24 upload 23 drop box 20 assignments gradebook 15 chat 13 groups 13 webdav 11 enter search term 10 forum 9 copy 9 delete 9 grade 8 assignment 8 download 8 schedule 39

40 Tracking help.search 2007: upload and January February March April May June July upload August September October November 40

41 Other Data for Help Docs Annual instructor / student survey Ask how many users get help from docs ( most effective ) 12% instructors, 7% students Comments about help guide Feedback from users in Support contact ( , phone), at training sessions, anecdotally 41

42 What Do We DO With All This Data? Margaret Wagner Senior Technical Writer Writes Sakai & CTools documentation Writes Sakai Newsletter 42

43 Docs with the highest usage numbers 3/07 to 11/07 1. Common Questions (viewed 4476 times) 2. Feedback/Contact Consultants Form 3. Discussions 4. Setting Up and Publishing Sites (ranged from 27 to 276 per week) 5. Chat Room 6. Resources 7. Accessibility 8. Announcements 9. Gradebook 10. Performing Common Tasks 11. Known Issues 12. Wiki 13. Setting Up a Site -- Simple Outline 14. Customizing Home Most common search terms 8/07 1. books 2. delete site 3. delete 4. syllabus 5. tabs 6. wiki 7. move 8. publish 9. permissions 10. publish site assignments 13. discussion 14. textbooks 15. contact 15. Web Content (viewed 982 times) 43

44 Online Help Section: Performing Common Tasks with CTools DELETE delete a course site or project site... delete a tab for an official course site... delete a tab for a project or other site... delete an entire site so it no longer exists delete a discussion entry delete a chat entry COPY copy a file in into another resource folder copy a file from CTools to your computer duplicate a file in CTools copy materials from a site to a new site duplicate a site send using CTools receive using CTools set up an archive DOWNLOAD download a file UPLOAD upload a file FOLDER (CREATING) create a folder in CTools PASSWORD change a password create a password PRINT print a CTools page print a resource (file) REMOVE - remove me from a site - remove a course or project tab - remove a course or project site 44

45 Getting Message of the Day topics from common search terms and doc usage numbers Some examples: Control your tabs Users can control the tabs used to access course and project worksites. You can choose to hide particular tabs, and reorder the tabs using the Preferences tool in your My Workspace... Can't find your course in CTools? If you have recently added the course, it takes up to a day for your course elections to be reflected in CTools... Minimum requirements CTools is designed to work fully with the following browsers... 45

46 Recap: Applications for Data on Doc Usage Prioritize doc efforts and make certain docs are clear and thorough Make sure common search terms are in the sections Create new docs that are based on search terms Inform support Head off confusion with tips in the Message of the Day 46

47 Questions / Next Steps Contact Info: Steve Lonn - slonn@umich.edu Margaret Wagner - mwagner@umich.edu Next Steps: Volunteer to participate in sharing your custom help docs and making Sakai Help better! Signup by ing Jon Hays at Berkeley: jonhays@media.berkeley.edu 47