KNOWLEDGE BASE OPTIMIZATION

Transcription

1 DOCUMENT OVERVIEW Learn specific ways to optimize your RightNow knowledge base to help your customers find the information they need. Review these best practices, including tips on configuration settings, Answer creation, reports and more, based on thousands of RightNow customer implementations. Start improving your RightNow knowledge base search today! HOW TO USE THIS DOCUMENT The contents in this document are presented in a priority order. Those things in the beginning of the document should be addressed before those things at the end of the document. In addition, this document is best approached as an ongoing process. Sections 2, 3 and 4 need regular attention, but in general this entire list should be completed or reviewed on a regular basis to make sure that your knowledge base is in tune with your business. Each section of this document begins with a description of the issue that drives the content. After the issue is a description of the reasons you might need to make changes to your site. Optionally examples follow the description to help clarify the content. Some steps have an Also Consider section pointing out related concepts that might be worth further investigation on your site. Finally a results section is placed at the end of each step inviting you to record the current state of your system and track changes made based off of the information in the step. There could be individual sections or individual parts of sections which are irrelevant due to your processes or procedures. If you have a well established procedure for any of these areas, please consider evaluating your knowledge base relevant to your procedures rather than the specific steps in this document. Many of the analyses requested in this document require your judgment to determine magnitude or quality. These values are often dependent on your business, site traffic, or needs at the time of analysis and can t be set as universals.

2 1. SHORT ANSWER CONTENT/SUMMARY LINES Issue: Usability All visitors navigate the knowledge base via the summary lines. Visitors can have a better, more efficient experience if they are able to quickly scan the entire result list. To improve the visitor experience it is best to use the shortest summary lines possible. As a best practice goal, try for six words or less. The summary lines do not need to be in the form of a question because a statement is a shorter and easier way to write the summary lines. Instead, think of the Answer summary lines as headlines for the content. Why are short summary lines important? In a knowledge base, users are trying to scan info quickly and easily. Longer summary lines are visually difficult to read, and will thus encourage users to scan and potentially miss valuable information. Short summary lines draw users in visually. In fact, after performing a search a user should be able to scan 15 Answers returned in less than 10 seconds. Also important is to use terms consistently throughout the knowledge base. For example, determine if a term such as is used without a hyphen in one Answer, it should be consistent throughout all answers. Example: ORIGINAL SUMMARY LINE: Where can I find the current information about how to install version 8?

3 UPDATED SUMMARY LINE: Installing version 8 After making sure your summary lines are well groomed, consider the different uses of the Question and Answer fields. The Question field should reflect the user s question in their own language. The Question field is an opportunity for the knowledge base to identify those terms that are within the user vocabulary but not necessarily in the normal content of the message. The search engine is then able to better match actual user queries with the relevant content out of the knowledge base. Consider using the Keyword Searches report (discussed in Step 3) to identify the language your customers use. Finally, the Answer field is the solution to the question being asked. Try to keep this field relatively succinct, but at the same time make it complete. Consider identifying when you have summary content versus detailed content on the same topic and separate that information into separate Answers. Often there are multiple degrees of expertise in the user base, each expecting a differing level of information provided. If you have this type of related content, consider using the Manually Related Answers section under the Answer Relationships tab to build in the relationships between these Answers. Item to Review How to Review Results Recommendations Approximate Current % of Short Summaries Average Time to Scan Summaries Randomly scan your end-user answer list and approximate the amount of short summary answers. Approximate the time it takes to scan the first page of results by stopwatch or wall clock with second hand. Consistent Term Usage Final Configuration Changes Go Forward Plan (Monitoring)

4 STEP 2:.REVIEW THE ANSWERS LAST ACCESSED REPORT Issue: Answer relevance Most users navigate a site without ever using search terms. To make it easier for users to browse to content quickly, it is important to make sure that only the most relevant information is kept in the knowledge base at all times. This report shows which date and time each Answer was last viewed by the users. The Score column is also helpful in this report because it shows the Solved Count of each Answer. The Score (Solved Count) goes up when the Answer is used; the Score goes down slowly when the Answer is not used or if the Answer is rated poorly after viewing. If Answers appear in this report with a score of zero, they are the Answers that are used the least and/or the ones that people found least helpful. Any answers with a low score or score of zero should be reviewed to determine if there is an issue with content, if the answer is out of date, etc. Also consider: Answer Feedback It is a good idea to enable Answer feedback and review the user feedback on your Answer content. These comments are a valuable tool for keeping your Answers current and relevant. Also consider: Auto Review

5 Unused Answers will eventually move down the list as their usage decreases. Under Common Configuration System Configuration Settings RightNow Common Configuration Settings Answer Review AR_AUTO_ENABLE you can have the system automatically move an unused Answer into the Review status. This aids workflow for identifying and reworking those answers that need new content or should be removed from the knowledge base to prevent clutter. Importantly, as with other statuses, you can configure the Review status to be either Public or Private depending on whether it is more desirable to flag the content but leave it available or to flag the content and remove it from end user visibility. Should you notice your content is aging too quickly (e.g., too many things are getting Auto Reviewed) make use of the Common Configuration System Configuration Settings Agedatabase Utility Configuration Settings SA_AGE_FREQ. Item to Review Results Recommendations Approximate # of Answers with Old Last Reviewed Date Number of Answers with 0 (zero) Scores Final Configuration Changes Go Forward Plan (Monitoring)

6 STEP 3: REVIEW THE KEYWORD SEARCH REPORT Issue: Topic availability While most visitors don t enter search queries, those that enter searches tell a lot about their needs and experiences with the knowledge base. Numerous adjustments can be made to the search performance of the system by evaluating the searches performed. The primary way to monitor searches is with the Keyword Search report. The Keyword Search report can be found in Service Reports Site Reports Keyword Search. By default, it shows you all the searches performed in the last 30 days, but the date range can be run for any timeframe. Important consideration: When reviewing the Keyword Searches report, keep in mind that the displayed number of Answers for that search query is relevant for the last person to perform the search. If that person had a restricted view of the content or used search filters (such as product or category) then the number could be different than the total available to a user with different access permissions and filter selections. Use the Keyword Searches report to add Answers. Look for zeros in the Answers column. This shows that a search was performed and zero Answers were returned to the end-user. This indicates that a new Answer may be needed, or that an existing Answer needs additional phrases/content added to make sure the right results are found for that search term. After the report is run for the specified timeframe, sort the report sort to put the fewest results at the top. Click the Search button Under Limit and Order in the Order by area, click the hyperlink that shows the current sort In the new window, select Answers -Ascending as the first sort Then add Search Count -Descending as the second sort (using the Then [sort] by functionality) Click on the Answers # to see EXACTLY what the customer typed in when s/he performed the search Then click on the Search String to see which Answers are returned when the search is performed Action Items in the Keyword Search report 1. Look for zero, one or two Answers returned a. Add new Answers to cover this content b. Add aliases to handle common term confusion or misspellings (see Step 4) c. As a last resort, add keywords to existing Answers or change the wording in those Answers (see Step 6) 2. Look at searches returning too many Answers a. Consider adding common words people are using to the Stop Words (see Step 4) b. Use stopindex and startindex tags (see Answer 1985 for more details) c. Clean out unnecessary keywords (see Step 6) d. Shorten summary lines (see Step 1)

7 3. Look at the top searches to make sure the top Answer results are excellent matches for each search a. Review/clean up Aliases or Keywords if needed (see Step 6) b. Shorten summary lines (see Step 1) c. Add new Answers to provide a richer set of results d. Change existing content to match how people are searching Also Consider: Gap Analysis Report The Gap Analysis report is useful for monitoring any information that may be missing from your Answers. It does this by identifying Incidents on similar topics and identifying how closely those Incidents match your existing Answers. The report then prioritizes the identified gap by comparing the size of the Incident group against the distance from the nearest Answer. Since there is some automatic process to the grouping of incidents it is important for the same person to review the Gap Analysis report so they can learn the way that report behaves on your site.

8 Item to Review Results Recommendations Number of 0 (zero) Result Searches Number of Large Result Searches Quality of Results for top 20 Searches Final Configuration Changes Go Forward Plan (Monitoring)

9 STEP 4: IMPROVE ALIASES/EXCLUDE-STOP WORDS FILES Issue: Search efficiency After analyzing the Keyword Searches report, there are two items in particular that are especially important when configuring and tuning the knowledge base: aliases and stop words. When making changes to the aliases, use the aliases.txt file in the Common Configuration System Configuration File Manager wordlist files to create synonyms. Make sure consistent terms are being used in your Answers so you don t need to create too many conflicting aliases. When considering aliases, make sure the terms are always used in an interchangeable fashion. For example, if people search on Bill for your Billing Statement Answers, make sure you don t also have Legislative Bill in your knowledge base before you add the alias that says bill = statement. FORMAT of the aliases.txt file: All capital letters No spaces on either side of the comma The words the customer searched on, then the comma, then the word(s) you use in your Answers Unlike with stop words (discussed later in this section), the first word on the line does not need to be a single word. For aliases you may start the line with two words. Also unlike the stop words, aliases are entered in the unstemmed form. You can have aliases that have billing connect to statement but bill connect to legislation. Any words after the comma are synonyms ONLY for the first word on the line. It goes left to right, but not right to left. Example #1: If you call it " " (all one word), make sure you use that term " " throughout your Answers. If you notice on your Keyword Searches Report that people search for the term " " (notice the hyphen), then your Aliases.txt file would look like this: o , Example #2: The word "job" exists in all the Answers, but searches indicate people are looking for the word career, employment, work for Done correctly, these additions to the Aliases.txt file would look like this: o CAREER,JOB o EMPLOYMENT,JOB o WORK,JOB

10 In contrast, here are incorrect examples of additions to the Aliases.txt file: o Job,career o Warrenty,WARRANTY Editing the Stop Words Stop words are essential for balancing the speed and accuracy of searching. Having more stop words results in faster searching since fewer words are used to compare the search query against the content. However, when more stop words are used the results are less accurate because, again, fewer words are used to compare the search query against the content. While stop words can be edited in the same area of the file manager as the aliases, the best practice is to use the Service Configuration Knowledgebase Configuration Stop Words editors instead. These tools provide a useful user interface to provide automated suggestions to the stop words lists as well as helping to avoid the formatting issues that must be used when editing via the file manager.

11 The best practice recommendation is to maintain a small stop word list for Answers but a larger list for incidents. The content of the stop words lists should be those words that occur so commonly throughout the content that they will never be useful to differentiate amongst search results. A common example is company name. A larger incident stop word list provides for faster searching on the larger amount of incident content while also realizing that the Service Agents have more tools available to them in their searching interface than the end users do. These additional tools can help the Service Agent find information in different ways than simply relying on word matches (for example: date ranges, company names, etc).

12 Item to Review Results Recommendations Number of Aliases in Use Number of Symmetrical Aliases in Use Stopwords List Updated from Default Number of Words in Answer Stopwords Number of Words in Incident Stopwords Final Configuration Changes Go Forward Plan (Monitoring)

13 STEP 5: REVIEW PRODUCT/CATEGORY ASSOCIATION Issue: Topic relevance Not many people know that the names of the Products and the names of the Categories are searchable. It may be noted from the Keyword Searches report that many times a search is performed with way too many Answers returned. Generally, unexpectedly large numbers of results come from the Product and Category names having been indexed for searching and the user s search term happen to include a word from the Product or Category name. As mentioned in the previous step, all indexed words are searchable unless they are stop words. When products and categories are assigned to an Answer, those words are part of what is scanned during searching. Example: An end user searches for: product download Within the knowledge base, there is a Category called Drivers and Downloads as well as a Category called Product Pre-sales info The search will return all the Answers associated to Drivers and Downloads AND all the Answers associated with Product Pre-sales info Also, it will return all the Answers outside of those Categories that have the words product or download in them. A search like this could return hundreds of Answers.

14 Considerations: The good news is that the weight of the Product and Category names can be set to zero, which essentially eliminates the Product and Category names from being searchable. This can be done in Settings Common by modifying the SRCH_PROD_WEIGHT and SRCH_CAT_WEIGHT. As described earlier there is also the option of excluding certain words from being searchable as well in the stop words list. When making changes to the SRCH_PROD_WEIGHT and SRCH_CAT_WEIGHT consider that what is important is the ratio of those values to the other weights rather than the raw values of those weights. Item to Review Results Recommendations Product Weight Default = 50 Category Weight Default = 50 Final Configuration Changes Go Forward Plan (Monitoring)

15 STEP 6: REVIEW THE WEIGHTS OF EACH ANSWER SECTION/KEYWORDS BOX Issue: Search bias If the order of results from searches is occasionally incorrect, use the keywords field or modify the answer content to adjust individual results. However, if the order of results is consistently incorrect, consider adjusting the field biases for the various fields in an Answer. Each section of the Answer is broken down by its importance to the search and carries weight in the search engine to alter the final ranking. As discussed in the previous section the defaults for the Product and Category names carry the same importance as if they were sitting in the keywords box, a score of 50. Other fields have varying levels of bias depending upon their perceived importance as well as the likelihood of word repetition in that section. Default Scores for each section: Product 50 Category 50 Keywords Box 50 Subject(Summary) 45 Question 30 Answer Body 4 File Attachments 4 If, as noted in the precious section, the Product and Category names are not valuable to the search, set the SRCH_PROD_WEIGHT and SRCH_CAT_WEIGHT values to zero 0. This same concept holds for the other searchable fields as well. Review how you use each of the Answer fields to determine if your relative bias settings are correct for each field. When reviewing these bias settings keep in mind not just the importance of each field but also the amount of text you have in each field. The final search score is just a simple calculation based off of the frequency of occurrence of the search term in each field multiplied by the field bias setting. Example: Imagine a search term download which appears in the Category field once ( ) in the Subject field once ( ) in the Question field twice ( ) and in the Answer Body 17 times ( ) the score for that Answer will be calculated to be. The comparative ranking of this Answer will depend on the frequency and location of download in all other Answers. Note that if you change the bias of a field all Answers will readjust their rankings for searches where words match in that field. Thus it is a good idea to approach adjusting biases carefully and only change one bias at a time while repeatedly testing the search results after each change.

16 Other Considerations: Keywords Box Use the keywords box sparingly and conservatively in Answers. If the word is already in the Subject (Summary), Question, or Answer, there is no need to add it to the "Keywords" area in the Answer. These three sections are already indexed by the search engine so therefore they do not need to duplicate words in the keywords box. This is the most common misconception about the RightNow knowledge base search functionality. It is true that keywords can be added to the keywords box to give it extra weight and bring a certain Answer to the top, but this should not be a common practice. Overusing this method can overweight your Answers artificially and consistently return bad results. Similarly, it can be a difficult process to manage simple changes when keywords are replicated in many answers! In general, the best practice is to only populate the keywords field in response to an identified need after review of the keyword searches report. Example: Subject: Installing version 8 Question: How can I install version 8 most effectively? Answer: Download the exe file attached and it will walk you through an install wizard that is easy to use. There is a Help icon in the upper right corner if you get stuck. Keywords: upload,v8 Notice the words upload and v8 are nowhere to be found in the Answer, but need to be searchable in order to ensure this Answer is returned when those search terms are used. In this case, adding them to the keywords box makes sense. Also it is important to be aware that words in the keyword box are stemmed for searching. This means if the word upload is used in the keywords box, a search on uploading will still return the Answer. To the search engine, upload and uploading are the same word.

17 Item to Review Results Recommendations % of Answers Using Keywords Keyword Weight (Default 50) Subject Weight (Default 45) Description Weight (Default 30 Body Weight (Default 4) Attachment Weight (Default 4) Final Configuration Changes Go Forward Plan (Monitoring)

18 STEP 7: REVIEW AND/OR SEARCH Issue: Too many matches Full web search engines such as Google use the AND search because of the near infinite number of documents they could possibly return. In these cases the problem is getting too many search results for any given query and users need to restrict the result set into something smaller. In comparison, search engines working with small repositories use the OR search because it is too easy to get no results by accidentally making a query that is too restrictive. Since people have most of their experience on the full web search engines they sometimes expect the AND search in all their search activities. In RightNow, the default search type is an OR search, but the results are ordered as if it were an AND search. This means that the engine will return answers even if there is not an exact match, but those at the top of the result list will match more of the search words than those lower in the list. However, if the knowledge base seems to return too many results, it is possible to change the search to a Boolean AND search. This can be done in settings RightNow User Interface ANS_AND_SEARCH. By setting this setting to yes, the AND search puts an AND between the words the end user enters into the search box. If the setting is set to No (the default), the OR search is used and puts an OR between the words. Example: With the OR search --If a user searches on download drivers, the Answers that are returned will have either download or driver or both, with those having both words always appearing before those having only one word. With the AND search -- If a user searches on download drivers, the Answers that are returned must have both download and drivers. If there are no Answers with both terms they will receive 0 results. Considerations: The AND search works well when: The AND works well if the Knowledge Base Manager is diligent about keeping up with the keyword searches report and is adding phrases to your Answers that people are actually searching for. The AND works well when the above condition is met and the knowledge base is quite large (e.g. 800 Public Answers or more). The OR search works very well when: Answers have short summary lines Old content has been cleaned out (Answers Last Accessed Report)

19 The weight of the Products and Category names has been reduced (SRCH_PROD_WEIGHT, SRCH_CAT_WEIGHT to zero) or unique Product and Category names have been used The knowledge base is small The product menu and/or category menu is short and simple Preferred Alternative Solution: Consider using the System Configuraiton User Interface SEARCH_RESULT_LIMITING instead of AND searching. This functionality limits the number of results returned for searches by identifying when the result quality decreases in the search algorithm. This feature uses OR searching for full generality but provides the smaller result sets driving the desire to use AND searching without requiring the diligence of monitoring on the keyword searches report. To use this feature the functionality must be enabled via the SEARCH_RESULT_LIMITING configuration setting. The Customer Portal search result limiting widget must be in use (rather than the normal answer widget). Item to Review Results Recommendations Search Type Search Result Limiting Used % of Searches Returning 0 Results Final Configuration Changes Go Forward Plan (Monitoring)

20 STEP 8: REVIEW SMARTASSISTANT SUGGESTED ANSWERS Issue: Irrelevant suggestions SmartAssistant Suggested Answers is a very powerful feature to help reduce the number of incoming Incidents. SmartAssistant reads the text of the potential Incident and searches the Answer knowledge base for potential literal matches using the standard search engine. It then compares the text against conceptual groupings of the Answers. It uses the conceptual matches to filter the literal matches, returning those results that best match both literally and conceptually. These results are displayed to the user to see if their question is answered before they complete the incident submission. Because of this reliance on the search engine, adjustments to SmartAssistant should only be undertaken after the various search tuning exercises described above. This feature is enabled by default, but if it is not performing as expected there are adjustments that can be made to either be more restrictive with the concepts, or to eliminate the use of the concepts altogether. These adjustments are on the System Configuration User Interface SA_NL_MATCH_THRESHOLD setting. In addition, results can be limited to return only those answers that match the user selected product (SA_SUGGEST_LIMIT_PROD_LVL) and/or the same category (SA_SUGGEST_LIMIT_CAT_LVL). More detailed information regarding SmartAssistant is available in our customer care knowledge base located in our customer community. Answer ID 322 and 453 are particularly helpful in explaining the functionality and configuration of this feature.

21 Item to Review Results Recommendations SA_NL_MATCH_THRESHOLD setting Limited to Product Limited to Category Final Configuration Changes Go Forward Plan (Monitoring)

22 STEP 9: LIMIT NUMBER OF ANSWERS PER PAGE Issue: Usability As a final finessing adjustment, it is recommend Answers display per page. Often if the knowledge base returns less results, for example 10, the end user will skim the 10 Answers, think they have not found what they have searched for, and exit the page looking for alternative ways to get their question answered. Very few people will ever browse past the first page of results. Considerations: If the end user has to scroll, try to make the header smaller and clean out extra text at the top. Don t compromise and stay at 10 Answers just because summary lines are too long shorten them, then go to Answers Remove the three lines of text and the Date Updated (new page set defaults) Only the clickable summary lines should be showing Item to Review Results Recommendations Number of Answers per page Date Updated visible? Summaries only Final Configuration Changes Go Forward Plan (Monitoring)

23 10. ADDITIONAL RELEVANT FEATURES Issue: Education A few features relate to the usage of the knowledge base but are not specifically tuning issues. These features should have you focus on your business need and determine if they are right for your situation. Topic Words The topic words feature is an opportunity for presenting a specific Answer or web link in response to a query. This feature allows you to identify common queries and push relevant content to those queries, or to introduce targeted campaigns from within your knowledge base. Think of these query-specific results as similar to web advertising. You identify a specific word or set of words and the landing location for those words. Topic Words are managed via the Service Configuration, Knowledge Base Configuration area. Knowledge Syndication Knowledge Syndication is a way to expose your knowledge base Answer content anywhere on your site. With this functionality you can place your Answer content onto any page on your site. Knowledge Syndication is a small portion of javascript (web) code that you place on your web page. It is available via the developer portal at RightNow: Site Map Site Map is a feature that allows internet search engines to include your content in search results. It is an excellent way to direct customers to your site since most people begin a browsing session at one of the major internet search engines. Make sure this feature is enabled to make your Answer content available to search engines. Answer types If you have Answers that refer to pages outside of the RightNow system, you can create an Answer that is nothing more than a direct link to that web content. The content is indexed for searching by the RightNow system, and when a user selects one of these URL answers they are brought directly to the referenced page. Similarly, you can create File Attachment Answers that are documents attached to an Answer that load directly when the Answer is selected. For example, if you have a spreadsheet that contains all the content you wish to display, you can simply attach that as a File Attachment Answer and when the user selects that Answer from the search results the spreadsheet loads directly.

24 Attachments Answers can have file attachments in addition to the normal Question/Answer fields. These file attachments can also be indexed for searching, but the searching of attachments is not enabled by default. When you add an attachment you need to remember to specify that the attachment should be indexed for searching if you wish the attachment content to be considered. Link Checker When editing an Answer you can access a Link Checker to review the current status of all the links in your Answer. With this tool you can quickly identify if any links are currently inaccessible to those viewing the Answer. Answer Access Levels You have the ability to create content in your Answers that is relevant to specific sub-groups of your customers, such as Gold, Silver and Bronze tiers. If you have made use of the access level functionality be sure to consider how they may affect all of the tuning steps. For example, when using the Keyword Searches report to review the number of Answers returned for a search, the number returned for the reported search query is based on the last person to do the search. If that person had a Bronze tier the report would potentially show fewer results than the same search done by someone with a Gold tier.