Team A Final Report. Cal Poly Knowledge Base Project

Transcription

1 Team A Final Report Cal Poly Knowledge Base Project PROBLEM DESCRIPTION The Genius Bar is a technical helpdesk that will be located in the Library Learning Commons. We anticipate that this helpdesk will be a very popular section of the library because students often encounter a variety of technical issues. To reduce an overload of common questions, we proposed to create a knowledge base for the Genius Bar. The knowledge base would allow geniuses to write documentation articles for common problems and place them in a place for all students to access. By reading the articles, students can learn to troubleshoot technical problems on their own. Although most helpdesks on campus have their own web pages, there is no central resource where students can find answers to their technical questions. The Cal Poly Knowledge Base will give users a single place to search for help documents and will also serve as a portal to all other technical departments. INITIAL DESIGN To determine what features users would want in a knowledge base, we began by performing usability evaluations of a system that we found to be most similar to our concept of the final product. We found a helpdesk knowledge base site hosted by Princeton University that seemed to provide many functions we believed were important. At this site, users could peruse a database of articles on a myriad of technical subjects. The usability evaluations provided us with a better idea of what users wanted in a Knowledge Base system. CUSTOMER INTERVIEWS In addition to the previously mentioned usability evaluations, we chose to conduct informal interviews with some of the students that staff the ITS helpdesk. These individuals are likely to be the first geniuses at the Genius Bar since they already have experience helping students and faculty with their technical questions. The feedback we received from these students was very informative. We found the ITS helpdesk staff to be keenly aware of the flaws in the existing help systems and excited about the prospect of the new Genius Bar. The ITS helpdesk already has a system for tracking user questions that is built on top of a software product called Remedy. This system allows helpdesk staff members to create problem tickets, assign them to specific staff members and monitor their progress towards resolution. Remedy is very similar to the Bugzilla-style system that we envisioned when we drew up the UML diagram for our knowledge base prototype. The

2 helpdesk people we talked to were generally satisfied with Remedy but there did not appear to be a way for users with open problems to access the information in Remedy directly from the outside. Privacy issues prevent them from giving users full access to the Remedy database, but we so no reason why users shouldn t be able to track the progress of their own open problem tickets online. The helpdesk people we talked really liked the idea of having their own genius profiles. They were already familiar with Facebook and welcomed the opportunity to put up photos of themselves so that they could make the knowledge base more personal and appealing to students. I was also surprised to find that they had no reservations about putting their work schedules online so that users could seek them out individually. There were some ideas we had that helpdesk staff weren t so receptive to. For instance, they thought that listing all the articles geniuses had written at the end of their profiles would foster competitive animosity. DEVELOPMENT PLAN In order to develop a user friendly and easily extensible knowledge base, we had to come up with a good design for both the front and back end of the site. The design of menu buttons and the search field was very important, as they were the primary affordances users had to navigate the website. We opted to design the back end using XML so that we could separate the data from the interface. This allowed the information contained in the actual documents to be accessed in any form desired. XML required the use of XSLT to transform our XML documents into XHTML. We used an XML Schema to enforce a set of rules for the format of articles used in the knowledgebase. This is key because if all articles follow a known set of rules, it is easy to display them in a uniform fashion. For the web front-end, all visual aspects will be developed using CSS (Cascading Style Sheets). We chose CSS because of its flexibility and versatility. By having one CSS for all the pages on the site, it is easy to change aesthetic properties of the page without updating all the pages on the site. One CSS also guarantees a uniform look and feel for all the pages on the site. Using these techniques, we were able to guarantee a uniformity of appearance throughout the site and allow a high degree of reusability among the separate components. USABILITY EVALUATION PLAN In order to test if our system (the Cal Poly Knowledge Base) performs better than the Princeton Knowledge Base in terms of usability, we will perform the same usability evaluation that we used for the Princeton KB. There will be two parts to the usability evaluation. The first part asks the user to perform a set of three tasks. As the tasks are performed, the evaluator will count the number of mouse click, search misses, and false leads. This data will tell us how difficult it is for users to find answers to their questions

3 using our system. The second part will ask the user to rate the Cal Poly Knowledge Base on ease of use, consistency and visual appeal. After obtaining the data, we will be able to directly compare the results with those obtained during the Princeton KB s evaluation. Below are the usability evaluation forms we will use to test our system. There are two forms, one for the participant, and one for the evaluator. Princeton Helpdesk Knowledge Base Evaluation Form Participant Copy Part I: Please use the Princeton Helpdesk Knowledge Base site to perform the following tasks: If at any time, you are unable to find an answer to an article and would like to move onto the next question, just let the evaluator know and move on to the next question. 1. Find an article in the knowledge base that answers the following question: How much space is allotted to students on the OIT Server? Enter the article number where you found the answer here: 2. A student has the following question: How do I sync the calendar on my PDA with the calendars of other students in my group? Find an article in the knowledge base that will help answer the student s question. Enter the article number where you found the answer here: 3. Find an article in the knowledge base that answers the following question: How does Princeton s SPAM filtering work? Enter the article number where you found the answer here: Part II: Now that you ve had a chance to browse the site, how would you rate the site on the following categories: (Place a check mark on under one rating for each category) Category Ease of Use (was it easy to perform the tasks Excellent Good Average Could be Improved Horrible

4 above?) Consistency (were things placed where you expected them to be?) Visual Appeal (does the site s visual interface invite you to visit again?) If you would like to comment on any of the above categories, write your comments below. Be sure to indicate the category you are writing about: Princeton Helpdesk Knowledge Base Evaluation Form Evaluator Copy Part I: For Part I, participants will be asked to perform three tasks. While the user is executing the tasks, please keep track of the following actions: Number of Clicks to Completion Number of Search Misses [Bad Searches Typed Before Item was Found] Number of False Leads [Click Back Button] 1. Find an article in the knowledge base that answers the following question: How much space is allotted to students on the OIT Server? Number of Clicks to Completion: Number of Search Misses: Number of False Leads: Check here if user was unable to complete the task: 2. A student has the following question: How do I sync the calendar on my PDA with the calendars of other students in my group? Number of Clicks to Completion: Number of Search Misses: Number of False Leads: Check here if user was unable to complete the task:

5 3. Find an article in the knowledge base that answers the following question: How does Princeton s SPAM filtering work? Number of Clicks to Completion: Number of Search Misses: Number of False Leads: Check here if user was unable to complete the task: INITIAL RESULTS: Since our prototype currently has only limited functionality, we could not test everything according to evaluation plan described above. The prototype cannot process real text searches because very few real articles have been generated. However, we were able to present the prototype to some of the users who had participated in the Princeton Knowledge Base evaluation before. We asked them to complete Part II of the evaluation plan, adjusting the meaning of the Ease of Use category slightly to compensate for the prototype s limited functionality. Category Ease of Use (how easy is it to access the tools needed to help you find your answers?) Consistency (were things placed where you expected them to be?) Visual Appeal (does the site s visual interface invite you to visit again?) Excellent Good Average Could be Improved Horrible If you would like to comment on any of the above categories, write your comments below. Be sure to indicate the category you are writing about: User 1: I really liked how there were lots of options for searching for answers. User 2: I couldn t understand why the same links appeared on the top and side of the webpage Based on the quick and dirty evaluation of our prototype, we can conclude that our system does improve upon the usability of the Princeton KB. Data from our old evaluations rated Princeton s KB as good under the ease of use category, good to

6 average under the consistency category and average under the visual appeal category. In comparison, the ratings for our prototype did not fall below good.