Hood or Hypertext: A Comparison of Offline and Online Book Search Sessions

Transcription

1 Hood or Hypertext: A Comparison of Offline and Online Book Search Sessions Maria Gäde and Vivien Petras Berlin School of Library and Information Science Humboldt-Universität zu Berlin Dorotheenstr. 26, Berlin (maria.gaede, vivien.petras)@ibi.hu-berlin.de Abstract. People searching for books make use of various sources and systems. While some users prefer online systems, others tend to visit bookstores in their neighborhood. This study compares book search sessions from the isbs track with observed bookstore search sessions with the aim to investigate similarities and differences in user behavior and preferences. In particular, the focus lies on the observation and comparison of searching, browsing and recommending strategies. The results indicate significant differences in session duration as well as a strong preference for browsing strategies in bookstore sessions. No such strong preference for one strategy could be observed within the online sessions, even though browsing was still preferred. Recommendations are the least preferred interaction type in either environment. 1 Introduction How do we search for books? What makes us select a particular book? Both offline [2] and online [7] information seeking strategies point out the challenges for information systems development. Classical metadata search often does not match observed search tactics. In particular, serendipity cannot be supported by information systems. In the context of digital libraries, several studies investigate the implications for digital libraries and ebook usage [6, 8, 11]. Hinze et al. observed distinctive interaction patterns with printed books. Also, library users tend to judge and evaluate books only after flicking through the actual content. Similar, children often decide to select a book based on haptic and/or visual aspects. However, due to physical constraints bookshelf browsing is limited to a certain section compared to the digital environment providing (in theory) a broader overview of available content. Users discover books in various ways that are not always covered by simple search functionalities. Social book search applications like Goodreads 1 have found that recommendations from online and offline friends play an important role discovering new books. Often, we only search for a particular book 1

2 based on personal recommendations 2. Recommender systems try to adopt these relationship-based access strategies by suggesting relevant objects to a similar user group. However, information systems still often fail in supporting the users in discovering new or unknown content, especially in casual leisure situations. In order to improve these systems, we need to better understand user strategies and preferences and translate them into purposeful features. A common analysis approach is to compare strategies and interactions in the digital environment with those that occur in a similar physical environment. Since the physical environment (in this particular case bookstores) usually precedes the development of digital environments, processes and strategies for interactions in the physical environment have already stabilized and experiences can be translated into patterns for digital information system development. It is a matter of course that digital environments should offer different (usually more efficient or innovative) user experiences than the physical ones. But starting with a familiar user experience and expanding from that with the additional features that the digital environment allows, is a frequent user experience design strategy. This study compares a physical (i.e. neighborhood) and a digital (i.e. hypertext) book discovery environment in order to determine where each environment can learn from the other. In particular, the study analyzes different types of interactions that were already found and discussed in casual search sessions: searching (the user issues a particular request), browsing (the user overviews a variety of resources), and following recommendations (the user follows a previously issued reference)[10]. Within this research context, this analysis aims at addressing two main questions: RQ1 Are searching, browsing and recommendation strategies and preferences (equally) observable in online and offline book search sessions? RQ2 Are there differences between online (digital book discovery environment) and offline (neighborhood bookstore) sessions? In section 2, the Interactive Social Book Search track is described as well as the participants and data gathering approach. Section 3 reports on the bookstore observations. The comparison of online and offline sessions is given in Section 4. The paper concludes with a discussion on the results and the approach. 2 Interactive Social Books Search (ISBS) Experimental Setup. The Interactive Social Book Search track investigates book search sessions in order to find out how users interact with traditional metadata as well as user-generated content in book search applications [3]. Interactive user studies based on user interactions in a ISBS-developed web-based book discovery information system are aggregated across multiple researcher groups. 2

3 Participants could choose to interact with the system in a lab environment or from home. In order to provide a realistic book discovery environment, the underlying book collection should be large and comparable to other book discovery systems such as online book sellers or library resource discovery systems. A monolingual English subset of the INEX Social Book Search s Amazon/LibraryThing book collection is used, consisting of approximately 1.5 million books. Each book contains general metadata (title, author(s), publisher, publication year, etc.), subject metadata (classification codes, subject headings), user-generated content (Amazon user reviews, LibraryThing user tags), and a thumbnail image[1]. In contrast to previous years, this year s observation environment only contained the multistage interface of the ISBS book discovery information system[5, 4]. This interface version consists of three stages supporting search, browsing and exploring strategies[9]. A bookbag allows to store, manage and discover selected and similar books. For an observation experiment, participants are first given a training task that introduces them to the interface. Following, users were asked to complete either an open or closed task, after which they could also work on a second optional task if they so chose. For this analysis, only sessions including the following mandatory open task were considered: Imagine you are waiting to meet a friend in a coffee shop or pub or the airport or your office. While waiting, you come across this website and explore it looking for any book that you find interesting, or engaging or relevant. Explore anything you wish until you are completely and utterly bored. When you find something interesting, add it to the bookbag. Please add a note (in the book-bag) explaining why you selected each of the books. The open tasks sessions are most likely comparable to individual bookstore sessions and have therefore been chosen for further analysis. Participants and Data Gathering. Usage data was gathered through logfiles as well as user background information and responses through questionnaires. A total of 111 users participated in this year s experiment with an almost balanced representation of female (46%) and male (54%) users. The majority of sessions (67%) occurred in a lab and most participants decided to view and conduct an additional task (80%). In total, 55 sessions contained an open task and were extracted for further analysis. Only the first task was considered. 3 Bookstore Observation Natural Setting. For the bookstore observations, a small bookstore in Berlin, Germany was chosen, offering roughly items. The small store is housed in a single room with books grouped by different categories and one central information desk. The observer was placed near the information desk at a position

4 where it is possible to view the customers in the store as well as listening to the customer service dialogues. The observations took place during two days (one weekday and one day in the weekend) in order to capture different user groups and situations. Based on the proposed research question, an observation protocol was designed focusing on duration, strategies and results for each session. The session duration represents the period between entering and leaving the store for each observed person. Interactions were assigned to the search, browsing or recommendation categories and if necessary session details listed. A session outcome could be a purchase or ordering of unavailable books. Additionally, gender information and, if possible, information needs were documented. Participants and Data Gathering. In total, 39 customers in 37 sessions were observed. The vast majority of them were females (74%). Comparing weekday and weekend observations, it is apparent that people tend to visit bookstores alone during the week and with their family and especially with their children during the weekend. However, the session duration did not increase significantly. A large amount of information needs were related to other people the customers where shopping for: either family members or friends. Besides the obvious differences in environments (online and offline) and the number of observed sessions (55 in the online and 37 offline environment), the two set-ups also differed in their collections, their interfaces 3 and the types of available interactions. It is very common that customers approach the information desk to inquire about books that are not available in the stock of the bookstore, but can be ordered to be delivered within a 24 hour time-span. In one interpretation, the physical bookstore added a digital dimension here, which is not reciprocated by the ISBS interface. While the ISBS discovery interface does not immediately reveal the size of its collection, a bookstore customer can quickly get an overview over the number and different categories of books available due to the physical representation. Additionally, the ISBS online interface utilized an automatic algorithm based on the book records in the collection and the book reviews for recommendation purposes, while the bookstore used a personalized, human recommender (the book sellers) who based their recommendations not only on the stock, but also on other experiences. The subsequent analysis will show whether these differences actually lead to different interactions. 4 Comparison of Offline and Online Sessions In this section, the 55 isbs sessions are compared to the 37 bookstore sessions with respect to their session duration and interaction type preferences. 3 In a bookstore, one would probably speak of layout and representation of the stock.

5 Session Duration. Table 1 shows the average duration of sessions for each data set. It shows that online sessions last almost double as long as bookstore sessions. The longest online task completion took 57 minutes, while the longest bookstore session was 34 minutes long. Data Table 1. Average session duration Duration in min. isbs sessions task 1 (open) Bookstore sessions 6.76 One reason for the longer online sessions might be the experimental environment as well as the unknown interface and system, which led to longer task completion times because of unfamiliar complexity. Another juxtaposed reason might be that participants in the ISBS lab study spent more time with the system because they were more attuned to the fact that they were being observed, which led to longer task completion times because of increased carefulness and attention to the task. A third reason might be that task completion in bookstores is faster because of the reduced selection variety, which could be interpreted as both a positive or negative impact factor. Interaction Type Preferences. For this analysis, characteristics indicating search, browsing or recommendation sessions were chosen: Search sessions are either defined by the amount of queries issued as well as the usefulness rating of the ISBS search box by a study participant (i.e. did they use the search box and find it useful or not) or through a specific request by a bookstore customer. Browse sessions are either defined by browsing interactions through topics as well as the ratings of the hierarchical topic explorer offered for browsing by ISBS or through book(shelf) browsing by a bookstore customer. Recommendation sessions are either defined by the usage and rating of similar books provided at the exploratory bookbag stage or through a recommendation offered by the book seller. Table 2 compares search strategies from both environments belonging to either search, browsing or recommendation interactions. With 419 single interactions isbs sessions contain on average 7,6 topic refinements. Following, 166 queries were issued during the open task, averages at least 3 queries per session. Only 32 times in 12 sessions the similar book feature was used. Although almost all sessions included at least one book saving it remains unclear why the majority of users did not use the recommendation possibility at the bookbag stage. Those that made use of this function reported differing evaluations for the presented results (see table 3). Similar, browsing activities are predominately observed in

6 the bookstore. At the second place customers asked a book seller for a specific item, i.e. they performed a search. Only 7 sessions contained an interaction related to recommendations. Mostly, these recommendations were given by the book seller, only in one case a man called his wife and asked for advice while searching for a gift. Table 2. Comparison of isbs and bookstore session strategies Feature isbs Bookstore Searching Browsing Recommendation 32 7 Table 3 provides an overview of participants responses for how useful the above mentioned interface elements were on a scale between 1 (Not at all) and 5 ( Extremely ). Table 3. isbs participant ratings for the usefulness of features Rating Search box Topic explorer Similar books Did not use Not at all Extremely Consistently to the usage data, only 5 participants indicated that they did not use the hierarchical topic explorer at the browsing stage, which indicates that browsing was used by more participants than the other interaction types. In comparison, the search box and similar books were not used by 24% and 42% of the participants. Although a quarter of the participants did not use the search interaction, when used, most participants found the possibility to query for a book extremely useful. 16 participants responded the same for the topic explorer and only 9 for the similar books, indicating that recommendations were not as helpful as search or browsing. Browsing seems to be the preferred interaction type in both environments, however, it is not considered as useful as searching in the online environment. This might be due to the sub-optimal browsing possibilities in the online environment. The instant and - at the same time - complete overview that a bookstore affords, cannot be easily replicated here. Recommendations are the least pre-

7 ferred interaction type in either environment, a surprising finding, considering how much the leading bookseller Amazon focuses on exactly this interaction type. However, this needs to be interpreted carefully. The browsing capabilities of both environments already provide useful indicators for their users, so an additional recommendation might simply not be necessary. Conversely, the recommendations in both study environments could also be not as good as might be expected from a book discovery environment (i.e. as in Amazon) and users learned to not use this as much. These questions once again demonstrate that even carefully designed interactive studies as the ISBS one only lead to more research questions in need to be explored. Success Rates of Sessions. An interesting follow-up analysis of interaction type preferences would be to compare preferences with their impact on the success of an interaction. The vast majority of bookstore sessions were successful with 78 % of all sessions ending with a purchase or ordering of a book. Success in the online sessions could be defined through the saving of a book in the bookbag. In total, 201 books were saved in 55 sessions during the open task. While participants saved 3.7 books on average, individual session results ranged from one to 18 saved books. However, since the experimental tasks required at least one book saving, this is no reliable measure. Also, there is a decided difference between storing books in a digital environment, which has no consequences and the completion of a book buying transaction, which demonstrates a much higher commitment of the buyer. Therefore success rate between online and offline sessions were not taken into account. 5 Discussion This small study of online and offline book search sessions could point out some similarities in the environments, but also some differences. The results indicate significant differences in session duration as well as a strong preference for browsing strategies in bookstore sessions. No such strong preference for one strategy could be observed within the online sessions, even though browsing was still preferred. At the same time, both methods used for data gathering come with limitations. While interactive retrieval experiments provide a more controlled environment and details about the participating users, it remains an experiment that is somehow limited to a certain user group and situation. Unobtrusive observations in the wild (i.e. a bookstore) provide a more realistic picture, but lack context information both about the user and environmental factors. For example, users that look at bookshelves can either want to locate a specific book which would be a classic search or browse through the available content. It remains unclear to which extent individual and environmental differences influences the observed behavior, a research question that remains open for user studies. It makes sense to combine different methods and sources to support the

8 translation of casual leisure situations into information system architectures as the ISBS experiments attempt. For example, the offline study produced a set of information needs that could be used for future experiments. Also, the observation has shown, that bookstore sessions are rather short in duration. However, we do not know how much time people spend in advance to inform themselves about or locate interesting books. It seems that book search is rather a multiple-stage than a single session affair. Especially for personalization features and session time outs this could be an important aspect to consider. Acknowledgments We would like to thank Manuela Wiggert for providing access to her bookstore Albertinen Buchhandlung as well as valuable insight into a physical bookstore environment. References 1. Bellot, P., Bogers, T., Geva, S., Hall, M., Huurdeman, H., Kamps, J., Kazai, G., Koolen, M., Moriceau, V., Mothe, J., et al. Overview of inex In Information Access Evaluation. Multilinguality, Multimodality, and Interaction. Springer, 2014, pp Buchanan, G., and McKay, D. In the bookshop: Examining popular search strategies. In Proceedings of the 11th Annual International ACM/IEEE Joint Conference on Digital Libraries (New York, NY, USA, 2011), JCDL 11, ACM, pp Gäde, M., Hall, M. M., Huurdeman, H., Kamps, J., Koolen, M., Toms, E., Skov, M., and Walsh, D. Overview of the SBS 2015 Interactive Track. In CEUR Workshop Proceedings (2015). 4. Hall, M. M., Katsaris, S., and Toms, E. G. A pluggable work-bench for creating interactive IR interfaces. In Proceedings of the 3rd European Workshop on Human-Computer Interaction and Information Retrieval, Dublin, Ireland, August 1, 2013 (2013), pp Hall, M. M., and Toms, E. Building a common framework for iir evaluation. In Information Access Evaluation. Multilinguality, Multimodality, and Visualization: 4th International Conference of the CLEF Initiative, CLEF 2013, Valencia, Spain, September 23-26, Proceedings (2013), pp Hinze, A., McKay, D., Vanderschantz, N., Timpany, C., and Cunningham, S. J. Book selection behavior in the physical library: implications for ebook collections. In JCDL 12 (2012), pp Kim, J. Y., Feild, H., and Cartright, M. Understanding book search behavior on the web. In CIKM 2012 (2012), pp Reuter, K., and Druin, A. Bringing together children and books: An initial descriptive study of children s book searching and selection behavior in a digital library. Proceedings of the American Society for Information Science and Technology 41, 1 (2004), Vakkari, P. A theory of the task-based information retrieval process: a summary and generalisation of a longitudinal study. Journal of documentation 57, 1 (2001), Villa, R., Clough, P. D., Hall, M. M., and Rutter, S. A. Search or browse? casual information access to a cultural heritage collection. In Proceedings of the 3rd

9 European Workshop on Human-Computer Interaction and Information Retrieval, Dublin, Ireland, August 1, 2013 (2013), pp Zhang, Y., and Kudva, S. E-books versus print books: Readers choices and preferences across contexts. JASIST 65, 8 (2014),