Card Sorting, Information Architecture And Usability: Adding in Our Users Perspective to Re-Design the Census Bureau Web Site

Transcription

1 Card Sorting, Information Architecture And Usability: Adding in Our Users Perspective to Re-Design the Census Bureau Web Site Erica Olmsted-Hawala U.S. Census Bureau Abstract 1 Previous research shows that one of the major problems for users of the information rich Census Bureau Web site was in locating or navigating to desired content. This knowledge motivated Bureau staff to begin thinking of how to reorganize the site to attain a more usable information architecture. The team determined that bringing in user-centered design practices into the process could benefit. Thus, in the initial stages of redesigning the Web site, Census Bureau staff in the usability lab conducted two rounds of card sorting. These studies were intended to reveal a better understanding of the content, and the organization of the content that should be on the Census Bureau main page and on the lower-level target pages of the site. This paper looks at how the card sorting studies identified ways that users organize site content and shows how bringing in users to help organize and understand site content gave a better basis for our Web site content development. Keywords: card sorting, information architecture, user-centered design, usability Introduction The Census Bureau is a major producer of economic and demographic data at all levels of geography in the United States. It disseminates the information to the public primarily through the World Wide Web ( The content on the Web site is posted by many different people and without the aid of a corporate wide template. In addition, 1 This report is released to inform interested parties of ongoing research and to encourage discussion of work in progress. The views expressed are those of the author and not necessarily those of the U.S. Census Bureau. many areas of the site are organized by the internal structure of the department rather than by the subject matter. Previous studies have shown that one of the major problems of federal statistical agency Web sites, and the Census Bureaus site in particular, is inadequate navigation.[1],[2] Consequently there has long been a need to redesign the Web site from the ground-up. With the goal of redesigning the Web site to make it more user-centered, the Web Council Team, with members consisting of diverse personnel from throughout the Bureau, asked the usability lab to incorporate user-centered design techniques into the process. The first step was to get a better understanding of the content that should be available on the main page and on the lower level target pages. We were interested in getting a better understanding of the information architecture, or how to structure the site navigation in support of our user s goals. We were particularly interested in the link label names and how typical novice users would group Census data beneath the link labels. To do this we wanted to have our users participate in a card sorting exercise. Card sorting is a technique used to discover how people group concepts (terms) into categories. Typical users categorize and organize terms in a way that is logical to them. The grouping data is analyzed. Cluster analysis offers a visual representation, of the highly correlated groups or categories. This can aid one in interpreting the logical user defined groups.[3, p.550] Often this information is given to the design team and developers of a site so that they will structure the content and navigation of the Web site in a way that is more meaningful and supports user goals. For our site, we chose an iterative card sorting exercise in two rounds to get at this information.

2 In the remaining sections of this paper, we describe the open (Round 1) and closed (Round 2) cardsorting studies in more detail, and we present the results obtained from the studies. We make recommendations based on the card-sorting results and offer suggestions how others can use the same methods to improve their sites information architecture. Methods Round 1 and 2 Card sorting can measure how users categorize, group, label, and organize site content (e.g., their mental model ).[4] While the results for each user may be different, the degree of overlap, as shown in a cluster analysis, indicates hierarchical groupings that most users would agree on. Card sorting results offer information on terminology and suggest ways to organize site content into a user-centered navigation system. Category groupings can be used to reorganize site content and redesign the navigation to make it conform more closely to user expectations and intuition (i.e., their cognitive models).[5] Card sorting is done in two primary methods, open and closed.[5] We conducted two card sorting studies, in the first we used the open method, in the second, we used the closed method. We call these Round 1 and Round 2 as the results of Round 1 informed the methods used in Round 2. In Round 1 we recruited 11 users, and in Round 2 we had 14 users. For both rounds, sessions lasted about an hour. We recruited participants by a general announcement asking for study participants in a free local newspaper. At the beginning of the study, users filled out a pre-questionnaire that measured their familiarity with using computers, the Internet and their general knowledge of Census Bureau terminology and concepts. Users had the following characteristics: All were general or novice users of Census.gov and Census Bureau terminology Most had not used the Web site Census.gov before Many said they were computer savvy Many said they were comfortable searching for information on the Internet. Users were told they could speak at any point about what they were doing or raise any questions they had throughout the process. They were also told that the test administrator might ask questions about the process or their results. The cards Before the sessions began, we worked with the development team to come up with 100 representative terms of the Census.gov Web site. The terms were supposed to be representative of information that users would be looking for when coming to the Census Bureau Web site. Terms were a combination of the following: High-level terms, likely found on our introductory level pages Detailed information likely found on our lower level pages New terminology, such as frequent terms in the web search database or terms used in user queries to our call centers. All terms were typed in 18-point font and taped on 3-by-5 card stock with the definition printed on the back of the card. All terms were presented in a random order to the user. The terminology on the cards was similar between Rounds 1 and 2. In Round 1 the 100 terms were those listed in Attachment A. Some of the terms were removed and others added for Round 2. See Attachment B. In Round 1 we had users sort 100 representative terms into piles that made sense to them. We asked users to suggest high-level category labels for their piles. Finally we asked if there were subcategories that the terms could logically fit into. Since this was an open card sort, there was no limit on the number of piles that users could make, and the number of piles varied by user. In Round 2 the methods were very similar to Round 1 except that we gave users ten pre-set top level categories (obtained from Round 1 results). We asked users to sort 90 cards into the ten pre-set categories. The ten categories were on a different colored card stock so that they could easily be identified by the users. (See Figure 1 below for an example of cards sorted in piles.) We also asked

3 users, if it seemed appropriate, to make subcategories with the cards under the established ten top categories. Because it was a closed sort, we encouraged users to fit all the terms into the pre-set categories. Figure 1. Photo of cards sorted in piles, demoexample Round 2. After a user sorted the cards into piles, the test administrator asked him or her to talk about the piles and answer debriefing questions about the cardsorting process, the terms, definitions and their final category piles. At the conclusion of each session, we took notes on which cards had been placed together as well as any sub-categories that had been made. We then entered the results into IBM s EZ-Sort and analyzed with IBM s EZ-Calc cluster analysis software. The program presents the results in a set of tree diagrams.[6] EZCalc shows the combined groupings in the form of a hierarchical tree structure. The application generates three sets of results, one for each of the combinatorial algorithms used by USort: single, complete, and average. According to the user s manual, the single linkage algorithm puts greater emphasis on rated similarities between pairs of terms as compared to the complete linkage algorithm, which places greater emphasis on rated differences. The average algorithm finds a balance between the single and complete results. For Round 1 we listed out all the high-level group names that users had come up with when doing the sort. We grouped the terms that were the same or similar and identified the number of times a highlevel term was used. In addition we reviewed the hierarchical tree graph to identify clusters of terms that seemed to form a logical group and drew conclusions on the terms that were highly correlated. We looked at the outliers and tried to identify why they didn t fit in any of the other piles. Finally we reviewed the comments that users made on the specific terms, such as terms that were confusing or terms that users said they had no idea what they were or what sort of group the term should be sorted into. For Round 2 we validated the high-level category labels by reviewing the cluster analysis and identifying the terms that consistently fell into the same groups. We again reviewed comments that users made on specific terms, such as terms that were confusing, and so forth, according to the method described for Round 1. For both rounds, usability lab staff met with the development team to look at the cluster analysis and discuss the groupings and outliers that emerged. Results Round 1 We reviewed the list of high-level terms that all the users came up with during their card sorting session. We grouped the similar terms together and identified how many times each was used. For example, for one of the high-level categories, a number of different users came up with the term population, or a variation on that term, as follows. Population 2 users Population Data 2 users Population Stats 1 user Population Estimates 1 user Households/Household 2 users Family and Household 1 user Based on this level of agreement among our users, we created a high-level category labeled Population: People and Households. After reviewing the list of all user created high-level terms, we were able to identify the frequent highlevel terms. The team used this list, as well as our knowledge of the content matter, to come up with ten high-level terms for the closed sort that we would use in Round 2. The ten high-level terms the team identified, based on the results as follows:

4 1. US Federal State and Local Governments 2. Income and Poverty 3. US Business and Industry 4. News and Media 5. Housing 6. Employment 7. Imports and Exports 8. Maps and Geographic Areas 9. Population People and Households 10. International Statistics We gave the development team a summary of problem terms that users had commented on either during or after the card sorting session. An example of a few of the problem terms follows: Census Tract Resources Users said this term makes no sense Gazetteer Many users said it sounds like a newspaper Services Users said this term was too general to place anywhere NAICS Users commented they had never heard of this and wouldn t have known where to put it if they hadn t looked at the definition Facts for Features Users were not clear on what this was, one user after looking at the definition said she would call it Hot Topics Sex A number of users felt it was better to label this Gender Results Round 2 Users did consistently place many of the same cards in the ten high-level categories. These results help validate these categories as being usable link label categories for the Census.gov home page. For each of the ten higher-level categories, we were able to see a list of the terms that tightly clustered into the category as well as to identify terms that should be linked in more than one area. An example follows in Figure 2 below. In the Income and Poverty higher-level category, the cluster analysis showed the following terms grouped tightly: Alternative Poverty Estimates in the US Characteristics of Low Income Population Family Income Median household income for LA county Money Income in the US Small Area Income and Poverty Estimates State Median Family Income Wealth Many of the terms that were placed in more than one category were consistently placed in the same alternate categories. It is likely then that these terms should be located in more than one place on the Web site. For example among the terms that were grouped into the Income and Poverty higher-level category, users also placed some of these terms in other high-level categories. We recommended the following terms be listed in more than one location on the Web site. Wealth (Income and Poverty, also in US Business and Industry) Housing patterns, residential segregation (Income and Poverty, also in Housing) Although a few users placed terms in unexpected categories, most users did consistently place the lower-level terms in anticipated higher-level categories. Many users said they did not know where to put a particular card until they read the definition on the back. Some users said they would not have chosen that particular higher-level category, but because of the definition, they did. The caveat to this finding is that general users of Census data will not easily have access to a definition when searching on the home page of Census.gov, so the terminology that prompted users to read the definition must be modified for general users. In addition, the qualitative results of Round 1 (where users commented on specific terms that they didn t know Figure 1. Cluster analysis of Income and poverty section, all users

5 the meaning of or which category to place them in) showed up fairly consistently in Round 2. This suggests that many of the problem terms (often referred to as Census jargon) from Round 1, still problem terms for users in Round 2, quite solidly need to be addressed. Thus we recommended to the development team that they either modify the confusing/unknown terms by adding synonyms to the Web site or altering the terms so that they would be more understandable for the layperson. Discussion During this redesign the Census Bureau Web council team goal was to create an information architecture such that our users would be able to accurately, easily and efficiently find both the demographic and economic information they need. We chose to begin with card sorting. Any similar project can use these methods to gain a better sense of how to organize content from the perspective of their users which, in the long run, will likely elicit a more favorable outcome. For us, the method proved useful but was only the starting point. We now have a better sense of how to begin organizing our site content and are looking into the next steps. With the data from the first two rounds of card sorting in, we have found that in general the ten high-level categories, identified in Round 1 and validated in Round 2, could potentially be located as link labels on the main Census.gov Web site. With Round 2 we also identified terms that could be double linked on various pages of the Web site. Still we feel it would be worthwhile to conduct another round of card-sorting. This new study, Round 3, would keep the same ten high-level categories used in Round 2 but would have other examples of cards (80 to 90 different terms) that are also representative of content on the Census.gov Web site. (The limit of the software only allows 100 terms and the Census.gov web site has many, many more terms that users will come across.) Using additional terms, not used in either Rounds 1 or 2, is another way to validate whether the ten high-level categories will work for Census.gov users. In addition to the specific work on categories and group labels, during both card-sorting rounds, we identified terms that were confusing or uncommon and thus not usable for the general Census user. Using the methods mentioned above, users, if they felt the need, could read the definition on the back of the card. While it is important during the study to have users work from the same definition, in reality, most Web sites will not easily provide users with definitions. Thus one must remember when working with cards and their corresponding definitions to pay attention to the qualitative information on when users read the definition and how the definition affects their behavior. For our specific situation, the exercise proved useful in identifying jargon terms and motivated us to begin thinking of more common terminology that a layperson would be comfortable with. An additional advantage not foreseen at the beginning of the project was that the card sorting studies would aid us in our communication with Census Bureau Web site content providers. As mentioned earlier, there is, as yet, no template that is used by all content providers. Thus content providers to our Web site have vastly different styles which can be confusing to our current users. Now when meeting with and talking to Census employees who are posting content to the site, we share the information we have on how our users are thinking about and where users are expecting to find our content. This has proven to be information the content providers find informative and persuasive. Thus, the road towards a redesign with a reorganization of the content and use of common templates has become easier as we have convinced more content providers to believe in the process of extracting useful and meaningful categories from users. It is likely that any similar project can get similar buy-in from content providers by using similar data from their users. The information architecture technique of card sorting has helped us get a better understanding on how users conceive of our data and where they may go to find different information. The challenge now is to translate the grouping information we now have into an interface that will direct users towards the information they are seeking. Currently the team is in the process of creating low-fidelity prototype templates. We have recommended to the team to continue user-centered design practices by using the technique of iterative low-fidelity prototyping and continuous user feedback on the emerging design.

6 While this paper focused on how the Census Bureau began to create a more usable information architecture for our Web site, specifically with the method of card sorting, it is not unique to the Bureau and can be used in various environments and circumstances. Domains as diverse as medical, government, political, science, health, etc., all could benefit from learning how their specific user groups think about their content information. Any organization that needs to reorganize Web content or validate current Web navigation structure could easily employ the methods described above to get help on how users think of their information. Running a cluster analysis on the card sorting data, one is able to visualize the commonalities in how their users mentally structure and group the information. Then applying the results into a lowfidelity prototype interface design and showing that to the user for additional feedback, a Web site could well be on its way towards a usable interface. Attachment A Terms on cards Round 1 Advance Monthly Sales for Retail and Food Services Age Alternative Poverty Estimates in the United States Ancestry Annual world population change Births & Deaths Building Permits Census Bureau Training Census Calendar Census Dates for Countries and Areas of the World Census Tract Resources Characteristics of New Housing Characteristics of the Low-Income Children Construction Construction Spending County Business Patterns Current Industrial Reports Disability E-Commerce Statistics Education Estimates Export Codes Facts for Features Families and Living Arrangements Family Income Federal Expenditures Federal, State, and Local Government Organization Fertility Foreign Born Foreign Trade Imports, Exports and Total Trade Gazetteer Geographic Products Release Schedule Genealogy Geography Governments Grandparents Health Insurance Hispanic Origin Immigration HIV/AIDS Surveillance Data Base Homeownership Data Households Housing Affordability Housing Patterns (residential segregation) Housing Units Authorized by Building Permits Housing Vacancy Data Income International Data Base (IDB) International Statistics International trade Labor Force Local Employment Dynamics Manufactured Housing Statistics Manufacturing Manufacturing and Trade Inventories and Sales (MTIS) Maps Marketing Data Media Services Metropolitan and Micropolitan Statistical Areas Migration Mining Money Income in the United States Monthly Retail Sales and Inventories Neighborhood New Residential Construction New Residential Sales News Releases Nonemployer Statistics North American Industry Classification System (NAICS) Occupation Population Population Estimates Population Projections

7 Poverty Profile of U.S. Exporting Companies Public Employee Retirement Systems Public Employment & Payroll Quarterly Financial Report Race Residential Improvements Resources For Teachers Retail trade Services Sex Small Area Income and Poverty Small business State & Local Government Finances State Export Data State Median Family Income Statistical Abstract Statistics of U.S. Businesses Tax Collections Trade Deficit Transportation Urban and Rural Definitions and Data Voting and Registration Wealth Wholesale trade World population information Attachment B Terms removed for Round 2 Census Bureau Training Census Calendar Federal, State and Local Government Organization Geography Imports, Exports, and Total Trade Income International Statistics News Releases Poverty Resources for Teachers Statistical Abstract [2] E. Olmsted and K. Marquis. A usability evaluation of key parts of Census.gov. in Proc. of the American Statistical Association, New York, New York: Quality and Productivity Section [3] D. Stone, C. Jarrett, M. Woodroffe, and S. Minocha. User Interface Design and Evaluation. San Francisco, CA: Morgan Kaufmann, [4] G. Gaffney, (2003). What is cardsorting? [Online]. Available: ign/cardsorting.asp [5] D. Maurer, and T Warfel, (2004). Card sorting: A definitive guide. [Online.] Available: a_definitive_guide [6] J. Dong, S Martin, and P Waldo. A user input and analysis tool for information architecture. [Online]. Available: tpaper.pdf About the Author Erica Olmsted has a Master s degree in Technical Communication, with a focus on Usability, from the University of Central Florida. She has worked at the U. S. Census Bureau s Usability Lab for five years. During this time she has led numerous usability studies of data-dissemination Web sites and of datacollection instruments. She has been instrumental in incorporating user-centered design into Web site redesign projects at the Census Bureau. References [1] I. Ceaparu. (2003, Winter). Finding governmental statistical data on the web: a case study of Fed stats. IT & Society. [Online]. 1(3), pp Available: 1i03.html