Virtual Reference Service Quality: Critical Components for Adults and the Net-Generation

Transcription

1 Libri, Vol. 60, pp , June 00 - Copyright by Walter de Gruyter -Berlin - New York. DOI 0.55/libr Virtual Reference Service Quality: Critical Components for Adults and the Net-Generation Lynn Silipigni Connaway and Marie L. Radford Dr. Lynn Silipigni Connaway, Senior Research Scientist, OCLC Research, Dublin, Ohio, USA connawal@oclc.org Dr. Marie L. Radford, Associate Professor, Department of Library and Information Science, School of Communication and Information, Rutgers, The State University of New Jersey, New Brunswick, New Jersey, USA mradford@rutgers.edu This paper is based on a presentation at the international conference 'i 3 : Information: Interactions and Impact', organized by the Robert Gordon University's Department of Information Management, and held in Aberdeen, Scotland, -5 June 009. Abstract The project, "Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives," studied the habits and needs of virtual reference services (VRS) users to identify characteristics for informing library system and service development. The results of the online surveys and telephone interviews for users of VRS are the focus of this paper. One hundred thirty-seven VRS users completed web-based online surveys and 76 completed telephone interviews. Both quantitative and qualitative data were collected and analyzed using descriptive statistics, grounded theme analyses, and the Critical Incident Technique. Findings indicate that participants had used other modes of reference service. However, they found chat reference to be the least intimidating mode. The leading factor for choosing VRS was convenience. Anytime/anywhere access to VRS, its immediacy, and efficiency were factors in service selection. Knowledgeable librarians with positive attitudes and good communication skills also were found to be critical. Results for members of the Net Gen were compared to those of older adults. Both Net Gen (Net Generation) and adult participants were likely to be repeat users and had positive reactions to VRS. Net Gens were more likely to use the service if it was recommended to them. It was also more desirable to the younger VRS users to have the ability to develop a personal relationship with the librarian and to interact with a specific, familiar librarian. Users of VRS want librarians to provide specific information quickly, through a variety of formats. This article provides implications and recommendations for practice and library education. Librarians need to provide accurate information in a variety of service modes in a hybrid reference model that provides convenient, authoritative, reliable services to meet an array of diverse needs and communication preferences. Introduction The information-seeking preferences and communication and behavior patterns of library clientele are rapidly changing. In the current economic and research environment it is necessary for libraries to remain relevant and sustainable. Budget cuts are impacting staffing as well as purchasing of both electronic and print resources. As ongoing budget cuts and the downturn in the economic environment are affecting libraries globally, it is critical for librarians to make informed decisions for resource allocations, to sustain and grow responsive services, as well as to choose optimal service delivery models. Research is needed to understand the needs of a diverse user population in order to design services to meet users' needs. The four-phased project, "Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives," [] studied the habits and needs of virtual reference services (VRS) users and non-users to identify characteristics for informing library system and service development. The four phases included focus group interviews, online surveys, and telephone interviews of

2 66 Lynn Silipigni Connaway and Marie L. Radford VRS users, non-users, and librarians, and the analysis of 850 'QuestionPoint' [] transcripts. This multimethod research design was used to identify how and why individuals seek information ensures that the results will be generalisable through large sample sizes, multiple methods of data collection, and triangulation of results. The results of the online surveys and telephone interviews for users of VRS are the focus of this paper. The participants were asked to identify why they decide to use VRS. In an attempt to document how individuals determine if a reference encounter is successful or unsuccessful, the respondents also were asked to identify a successful and an unsuccessful reference encounter and to explain why the encounters were rated in this way. During the focus group interviews the data revealed some distinct perceptions of libraries, librarians, and library reference services by the Net Generation (Net Gens). These perceptions differed from those of older adults who participated in the focus group interviews. To further investigate these differences, Net Gens were targeted as participants of the online surveys and telephone interviews. Study population The Net Gens' distinct communication and information-seeking behaviors are discussed in the literature. Sweeney (006) reports that Net Gens prefer electronic interfaces that offer more choice and selectivity, flexibility, convenience, and personalization and customization options. He characterizes them as experiential learners who demonstrate a resultsoriented approach to communication and searching tasks and an aptitude for multi-tasking (Sweeney 006). [3] Much scholarly and popular literature focuses on the Net Generation. Although the dates of birth for this group vary, most agree they were born between 979 and 99 (Sweeney 006). There also are various names for this generation - Millennials, Digital Generation, or Echo Boomers (Oblinger & Oblinger 005, 006; Hallam & Partridge 006; Sweeney 006). The youngest members (now -9 years old) of this generation have been referred to as 'Screenagers' (Rushkoff 996) because of their affinity for communications technology. Only the Baby Boomer generation (born 96-96) outnumber this group in size. However, Sweeney predicts that the Net Gen eventually will outnumber Boomers (Sweeney 006, ). Those considered part of the Baby Boom Generation were born between 96 and 96 (Gillon 00; Krohn 00; Dempsey 007). Boomers are considered to be better educated, more technology literate and economically more advantaged than any generation before them (Williamson, Bannister, Makin, Johnson, Schauder, & Sullivan 006). Boomers want to maintain mental agility, to remain active in the workforce longer, and to be in touch with technology (Dempsey 007). Remaining in the workforce longer exposes Boomers to new technologies, which they integrate into their daily lives. Joseph (006) reports that Boomers read more and use public libraries more than generations before them; therefore, expecting the libraries to provide the most current resources (Willis 006). Although Boomers and Net Gen members demonstrate different behaviors and characteristics, they do share some similarities. They want a service that is quick, convenient, and easy to use. They expect the correct answer from a librarian who is professional and personable. Online surveys were completed by 37 VRS users and the telephone interviews included 76 VRS users. The online survey and telephone interviews provided both descriptive and quantitative data for analysis. The demographic data, multiple choice questions, and Likert-type questions (included in the online survey) were analyzed using descriptive statistical methods. The open-ended questions in both the online survey and telephone interview used the Critical Incident Technique (CIT) (Flanagan 95; Radford 006b) and provided qualitative data for analysis. Theoretical framework and research questions This research builds on the work of Watzlawick, Beavin and Jackson (967), as applied by Radford to face-to-face (FtF) (993, 999) and live chat (006a, 006c) reference encounters. Watzlawick et al. (967) proposed that all messages have two dimensions: content (information) aspects and relational (affect, interpersonal) aspects. The CIT (Flanagan 95). is a qualitative technique that asks respondents to focus on their most memorable events and

3 Virtual Reference Service Quality 67 experiences, whether positive or negative. CIT allows categories or themes to emerge from the data rather than to be imposed upon the data. Research questions derived from gaps in the literature and application of the Watzlawick et al. (967) and Flanagan (95) perspectives are: * What are VRS users' communication and information-seeking preferences? * What factors lead people to use chat reference? * What concerns do people have about using chat? * What makes for a successful reference encounter? * What is the relative importance of getting the information (or answer) as opposed to how one is treated by the librarian in deciding whether an interaction is successful? * What critical factors influence decisions to use VRS? Data collection and methods of analysis It was difficult to identify and recruit participants. Due to privacy concerns, the names and contact information for VRS users are confidential. For this reason, purposive and snowball sampling methods were used to identify participants. Purposive sampling involves selecting "a sample based entirely on one's knowledge of the population and objectives of the research" (Powell & Connaway 00, 95). Babbie (00, 93) states that "it's appropriate to select a sample on the basis of knowledge of a population, its elements, and the purpose of the research." Although it was not possible to generate random samples, the large sample size, coupled with the broad demographic characteristics of the participants, and the multiple data collection methods and triangulation of results provide sufficient data to generalize the results of the study. Users (those who had used VRS more than one time) were recruited for the online surveys and telephone interviews through links on the QuestionPoint homepage, links in VRS session closing scripts, and through messages sent to users. These messages were sent through a statewide VRS provider (MarylandAskUsNow!) facilitated by one of the Seeking Synchronicity External Advisory Board members. Table : Net Gen User Demographics for Online Surveys & Telephone Interviews (N=70) Gender Female Male Age Ethnicity African American Asian or Pacific Islander Caucasian Hispanic-Latino Other Type of Library Academic Public School Unknown Special Location Urban Rural Suburban Unknown Total Percentage 38 5% 3 6% 9 3% 3% 37 53% % % 66% % 7 0% 6 3% % 3% % 3% 33% 7% 59% % Survey questions were developed from participant responses in the focus group interviews, as well as from the above research questions. Telephone interviews were conducted as the last phase of the project and questions were further informed by online survey results. Interviews were chosen as the final data collection method as they usually produce better response rates than mail or questionnaires and provide opportunities to probe and to correct misunderstandings, as well as to reveal more complex information (Powell & Connaway 00). Users were given a payment of $5 for participating in the

4 68 Lynn Silipigni Connaway and Marie L. Radford Table : Adult User Demographics for Online Surveys & Telephone Interviews (N=) Table 3: Net Gen User Demographics for Online Surveys (N-=9) Total Percent. Total Percentage Gender Gender Female 0 7% Female 5 5% Male 0 8% Male 9% Age Ethnicity African American Asian or Pacific Islander Caucasian Hispanic-Latino Native American Other Type of Library Academic Public School Special Location Urban Rural Suburban % 3% 5% 7% 6% 7% 3% 85% % % 3% 5% 80% % % 8% 8% 6% online survey and $30 for participating in the telephone interviews. Once recruited, all participants were registered through , completed a consent form, and then given the URL for the online survey or a time for the telephone interview. One hundred thirty-seven VRS users completed web-based surveys between June 007 and March telephone interviews were conducted with VRS users between February 008 and March 008. Respondents provided both quantitative and qualitative data which were thoroughly analyzed using Age Ethnicity African American Asian or Pacific Islander Caucasian Hispanic/Latino Other Type of Library Academic Public School Special N/A Location Rural Suburban Urban N/A Educational Level 6' Grade 9d' Grade "' Grade Some College College Degree Some Graduate Work Master's Degree % 39% 7% 0% % 67% % 6% % 9% % % % 0% 59% 9% % % % 0% 3% % 0% descriptive statistics for the quantitative data, and grounded theme analyses (Charmaz 006) and the 6%

5 Virtual Reference Service Quality 69 Table : Adult User Demographics for Online Surveys (N=88) Total Percentage identifies specific strengths and weaknesses, and the features that instill perceptions of success in individual encounters. Gender Female Male % 3% Demographics - Online surveys and telephone interviews Age Ethnicity African American Asian or Pacific Islander % 38% % % 5% 7% 5% There were a total of VRS users [] who participated in the online surveys and telephone interviews. Of these 70 were Net Gens (-8 years) and were adults (9 years or older). The majority of both the Net Gens and adults were female, Caucasian, and used suburban public libraries the majority of the time. The majority of the Net Gens were between the ages of 9 and 8 years old while most of the adults who participated in the online survey were between the ages of 36 and 5 years old. (See Tables and.) Caucasian Hispanic/Latino 7 8% % Quantitative results - Online surveys Other Type of Library Academic Public School Special Location Rural Suburban Urban Level of Education % 7% 77% 5% % 9% 6% 7% Demographics - Online surveys Of the 37 VRS users who completed the online survey, 36% (9) were Net Gens and 6% (88) were adults. All respondents had to have used chat reference services more than once. The majority for both demographic groups were female, Caucasian, and used public libraries in suburban areas. The majority of the Net Gens were between the ages of 5 and 8 years old and had some college education. (See Tables 3 and ). The majority of the Adults were between the ages of 36 and 55 years old and had a Masters' Degree. "' Grade Some College 5% 3% Experience with reference modes College Degree Some Graduate Work Masters' Degree Doctoral Degree % 9% 39% 9% A majority of the users had used (63.5%, 87) for a reference question. A lesser number had used the telephone (8%, 66) or instant messaging (IN) or text messaging (36.50%) for a reference question. CIT. Grounded theme analyses reveal themes which emerge from the participants' own words, in this case their answers to free-response questions. The CIT Frequency of chat reference use A majority were relatively infrequent users of VRS. Although the respondents had to use chat reference

6 70 Lynn Silipigni Connaway and Marie L. Radford Figure : Frequency of Using Chat Reference: User Online Survey (N=37) [ -3 Times Total [E -6 Times Total F0 - Times/Month 03- Times/Month [ 5 or More Times/Month Figure : Chat Least Intimidating: User Online Survey Net Gens (N=9) Adults (N=88) 80%- 70%- 60%- 50%- 0%- 30%- 0%- 0%- 0% 0%,(0 5% 0 F7 5 6% 0% F 0% % % 76 0 / 37 36% Phone FtF Text Chat I otlet Gen VRS Users OAdultVRS Users at least one time to be eligible to participate in the study, the majority had used chat more than once, with 7% (3) using it - times per month (Seventy-eight (57%) of the VRS users had used chat reference -3 times. Thirty (%) had used chat reference -6 times. See Figure.) However, many did have experience using (6%. 87). the telephone (8%, 66), or instant messaging (IW) (37%, 50) to ask reference questions. Chat reference found to be least intimidating mode Both Net Gen and adult users found chat to be the least intimidating form of reference. (See Figure.) One Net Gen stated in the online survey, "Yes. It makes the librarians easier to approach and you can find out while you are still connected whether their advice yields the results you are looking for." (UOS 680) [5]

7 Virtual Reference Service Quality 7 Figure 3: Probability of Using Chat Reference Services Again: User Online Survey Net Gens (N=9) Adults (N=88) 9% 9% 90% 88% 86% 8% 8% 80% 8% 78% 76% Net Gen VRS Users Adult VRS Users Repeat use of chat Both Net Gen and adult VRS users are likely to use the service again. Of the 9 Net Gens and 88 adults who responded to this question, 9% (8) of the adults and 8% (0) of the Net Gens said they would use chat reference again. (See Figure 3.) Both Net Gens and adults included positive comments about using VRS as illustrated below. I am glad I found out about the chat reference, it has been a lifesaver on many occasions. From my experience, virtual chats are the best way to go because it is convenient, reliable and removes barriers such as stereotype that might otherwise impede the receiving of beneficial information. (UOS 6367) Adult I can't wait to use it again. May the service never cease operation. (UOS 66785) Adult I love it, I use it and I wished more people did use this; it's truly a user friendly service and why more people don't use this is beyond me. I would not start a research paper without AskAway. (UOS 0336) Adult Yes, they are great when you can't get to a library or need an answer unfindable or an answer needed right then. I have recommended it and I know I will continue using it through my schooling days. (UOS 637) Net Gen Recommendations important for VRS use by Net Gens Recommendation from a friend, family member, teacher, etc. was important to the Net Gens. They used VRS more than adults because it was recommended to them. Twenty-five percent (5) of the Net Gens said it was important or very important to have a recommendation to use VRS while 0% (8) of the adults said that a recommendation was important or very important. Net Gens also were more apt to recommend VRS to others than were adults. When asked, "Would you recommend chat reference services to someone else? Why or why not?", some Net Gen responses included the following. Definately - they're almost as good as asking the teacher herself/himself. (UOS 58) Net Gen I'd absolutely recommend this service to everyone. (UOS 598) Net Gen Reasons for choosing VRS The leading factor for using VRS was convenience. The fact that the service is available /7 enabling users to work from home and at night and weekends was very important to VRS users. The immediacy of the answers and the efficiency of the service also were very attractive to users. The fact that VRS is a free service was another reason for use. Users who

8 7 Lynn Silipigni Connaway and Marie L. Radford Table 5: Net Gen User Demographics for Telephone Interviews (N=) Gender Female Male Ethnicity African American Asian or Pacific Islander Caucasian Other Total Percentage 6% 38% 5% % 6% 9% Absolutely. It is convenient and always helpful, even more helpful than going physically to the library. And you are on your own computer with all of your information. It is especially helpful when taking an online course because the lessons are online. (UOS 53) Net Gen Yes, the chat format did help my experience to be successful because I was able to multi-task while the librarian did the search. If I was in a FtF situation, I may have gotten frustrated with the amount of time it took for the librarian to find what I needed and would have just said "never mind". (UOS 358) Adult Age % % 67% Yes. I would recommend chat reference services to someone else because it is more convenient than other references. It is available /7 unlike the library. (UOS 9507) Net Gen Type of Libra-y Academic Public School Special Location Urban Rural Suburban % 5% % 5% 3% 0% 57% participated in the online surveys also articulated that chat could provide less intimidating interactions with librarians. Among the factors that respondents report as very important or important to their decision to use chat reference, the most critical are convenience (78% very important, 07; 7% important, 3), lack of cost (75% very important, 03; % important, 9), and needing help late at night or on the weekend (57% very important, 78; 7% important, 3). The following verbatim selected quotes support the above themes. Convenience Absolutely. Quick, easy convenient, reliable! (UOS-6) Net Gen I would use the online references in the future because they seem convenient. (NOS 7606) Net Gen /7 availability I needed information for a school project and it was late at night, on a Sunday, when I could not go to a library. (UOS 39685) Net Gen I really love chat reference service. I think it is very useful especially late at night when public libraries are closed. You can immediately get an answer and find other resources. You can get this type of service from a reference librarian in person but if you can't get to the library or the library is closed, chat reference is very helpful. (UOS 97) Adult Immediacy I would recommend it to someone else because it saves a lot of time and you get information fast. Plus it is very easy to use which ends in a better result. (UOS- 8658) Net Gen The librarian quickly sent me to the website to download the VPN client. Yes - it was quick, and I didn't have to wait on hold for hours at a time. (UOS-9687) Net Gen

9 Virtual Reference Service Quality 73 Less intimidating interactions Yes. The help I have received has been on target and it is easier to engage in live chat the approach a librarian face to face. (UOS-680) Net Gen Yes, definitely! My experiences showed me that it is easy to use, has a level of anonymity which may be attractive to people who are more shy, is available /7, and the librarians have always been very helpful or at least extremely kind and personable. (UOS-5879) Net Gen Efficiency Yes, definitely. I already told the librarians I spoke with that I would recommend this service to anyone. It was so easy to use and very helpful. It was brief enough to answer my questions efficiently and not waste my time. I don't really want to have personal chats with librarians over the chat reference service. I really just wanted to ask my question and get an answer and that is exactly what my experience was. It was great! (UOS- 3087) Net Gen The chat helped me since it was an efficient use of my time. I was able to continue working online while chatting. (UOS-6538) Adult Cost effectiveness Yes - it's free, and convenient. The librarians are available till late at night, and they're very productive. The negative experience I described is the only one I've had, and I must have used the (service name) over 30 times. It's an easy way to get a better understand your subject and get positive answers from a trained specialist - what more could a student ask for? (UOS-3997) Net Gen Yes, quick and convenient. Cost-effective. Most libraries provide free access but parking can be expensive in the city where major libraries are located, not to mention that the closest major library to me is 60 miles away. (UOS-80908) Adult Table 6: Adult User Demographics for Telephone Interviews (N=5) Gender Female Male Ethnicity African America Caucasian Native American Other Age Type of Library Academic Public School Special Location Urban Rural Suburban Total Percentage Quantitative results - Telephone interviews Demographics- Telephone interviews 78% % 7% 87% % % 7% 8% 6% % 7% % 85% % % 30% 6% 65% Of the 76 people who participated in the telephone interviews, (8%) were Net Gens and 5 (7%) were adults. Again, the majority of both the Net Gen and adult telephone interviewees were female, Caucasian, and used suburban public libraries. The majority of the Net Gens were between the ages of 9 and 8 and the adults were between the ages of 36 and 55. (See Tables 5 and 6).

10 7 Lynn Silipigni Connaway and Marie L. Radford Reasons users recommend service When asked during the telephone interviews if the VRS users would recommend the service 89% (68) said they would and 30% () said they already had recommended the service. The reasons that they would recommend the service were the convenience (3%, ) and the speed and efficiency of getting answers and information (3%. ). The responses to this question were very enthusiastic as represented by the quotes below. I have only recommended it to every human being that I know. It is a fantastic service! People should want to know answers to queries and questions. I am already telling everybody, "Do you know what you can do online!?' (UTI 8) Adult [6] Gosh, I've only been, been satisfied, so yeah, I usually tell them, 'Thanks a lot, and I will use this again, and I'm telling my patron about iw and I said I would tell the rest - I don't think a lot of librarians at my library use this, but they don't! I usually tell them, 'Thanks, the patron is here.' I tell them all about it, and I am going to tell others. (UTI 76) Adult Yes I have often because as a paralegal, I'm not allowed to give legal advice. People ask me for legal advice and I have to refer them to a lawyer or refer them to a site where people can ask their questions. I send people to legal chat reference room so they can ask for help. (UTI ) Adult I would tell my friend with good positive results I would say, how about you're working on this project, how bout you go online, chat with a Eibrarian, she'll be able to help you. (UTI 0) Net Gen Reasons for selecting chat reference as first choice Getting quick answers (8%, ) was most often the reason that chat reference was the first choice for getting answers or information. When asked why, once again, convenience (60%, 5) was the response. Some responses to the question of when chat would be their first choice and why are included below. Please note that the respondents were not prompted with reasons for using chat. Their responses were free-response replies in the telephone interviews.... any time I would need to look something up in a hurry it would probably be my first choice. (UT! 0) Adult I work on campus; I can access it from work, so it is the convenience. The first time it was about a book that wasn't on the shelf, it was being restored. Instead of going to the library and running around looking for it, I used chat. To me it would be able to give me an answer quickly. (UTI 78) Net Gen When I really need it right now, then I will chat reference. So it is really expediency. (UTI 66) Adult I've used chat with a whiteboard, that worked pretty well for math questions, but I think that face to face might still be more productive but the convenience is still better online than in person, you don't have to make trips to the library. (UTI ) Net Gen Additional reasons for selecting chat reference The VRS users had used other modes of reference service. However, they found chat reference to be the least intimidating mode of communicating with a librarian for a reference query. Both Net Gen and adult VRS users are likely to be repeat users of the service and had positive reactions to the service. Having the service recommended was very important to Net Gens. They were more likely to use the service if it was recommended to them and were more apt to recommend the service to others than the adult users. The leading factor for using VRS was convenience. The anytime/anywhere access to VRS, the immediacy, and the efficiency were all factors for selecting the service. Knowledgeable librarians with positive attitudes and good communication skills were important to all VRS users. It was important to the younger VRS users to have the ability to develop a personal rela-

11 Virtual Reference Service Quality 75 tionship with the librarian and to ask a specific librarian a question. The Screenager ( to 8 yearolds when the data were collected) VRS users value having a personal connection with the librarian and the opportunity for dialogue with the librarian. However, this was not as important to any of the other respondents. Qualitative data analysis using the Critical Incident Technique (CIT) As previously mentioned, qualitative data collected from the CIT responses for both online surveys and telephone interviews were analyzed according to Flanagan's (95) guidelines. The CIT is a qualitative method which enables researchers to discover the participant's most memorable factors of any experience (Ruben 993). It has been widely used in assessments of programs or services, including analysis of communication behaviors and preferences in libraries (see Ozkaramanli 005; Radford 999, 006a, 006b). CIT questions generally have this format: "Remember a time when you had a successful/unsuccessful (specify activity). Please describe. What was it about (specified activity) that made it successful/unsuccessful?" Flanagan's (95, 37) influential article describes CIT's origins and details guidelines for its utilization. Flanagan defines a critical incident as: "Any observable human activity that is sufficiently complete in itself to permit inferences and predictions to be made about the persons performing the act." This qualitative analysis technique makes it possible for categories or themes to emerge rather than to be imposed by researchers (Ruben 993). CIT is "often used to study effective and ineffective human behavior" (Ozkaramanli 005., ). The online survey and telephone interview instruments included two critical incident questions in which the participants were asked to "think about one experience" in which he/she felt a positive result was achieved and a separate instance in which a negative result was achieved after seeking reference services. Then they were asked to describe each interaction and to identify factors that made the interactions successful or unsuccessful. The critical incidents were carefully read and sorted into content themes in an iterative process by members of the research team. The emerging themes were then expanded and merged into the Critical Incident Coding Scheme developed by Radford (999) for FtF reference encounters. A coding scheme [7] was developed following the constant comparative method for analyzing qualitative data (e.g., see Strauss & Corbin 998; Charmaz 006). The positive and negative experiences contributed by the survey and telephone interview participants aligned with both content and relational themes, as well as combination themes present in the coding scheme. The content themes relate to interactions that focus on the information or materials sought, as well as the degree to which participants perceived that librarians interact knowledgeably about general librarianship and/or specialized subject areas. The relational themes outline interpersonal aspects of the interaction, including attitude, relationship quality, and approachability. As noted above, relational theory, based on the axiom that every message has dual dimensions - both content and relational (Watzlawick, Beavin, & Jackson 967) is used as an analytical framework. Content.dimensions include the information, the "what" of the message, and the information exchange. The relational encompasses the interpersonal communication, the relationship between participants, and how the message is to be taken. Results of Critical Incident Analysis - Online surveys A total of 97 critical incidents was collected from online survey participants. The results of this analysis with representative, verbatim quotes from participants are described below. Positive Critical Incidents (CIs) There were 9 (9%) CIs described as positive. Of these, 8 (37%) of those positive descriptions were from Net Gens. The Net Gens attributed their positive CIs to their interactions with the librarian. Fourteen (9%) attributed the positive experience to getting accurate answers and information. Quick assistance (3%, 6), locating specific resources (9%, 9), and the convenience of the service also were attributed to a successful reference encounter. The Net Gens described a number of content and relational facilitators in their discussion of positive CIs.

12 76 Lynn Silipigni Connaway and Marie L. Radford I asked a pretty specific chemistry question, and the librarian was able to efficiently answer the question by finding a website with the answer. The answer was correct, and I was able to understand it. (UOS-7705) Net Gen I was surprised how simple the process was since I had never used chat before and never really used the library that much either. Yes, I wasn't able to call but I was able to check online and it was a lot more convenient than going to the library in person. I needed an appointment to see the book anyway so I would've wasted a trip if I had gone in person and had to come back again. Instead I got all my answers through the chat conversation. (UOS-3087) Net Gen The librarian was able to guide me through a research problem clearly and thoroughly, assisting me step-by-step. The librarian helped me step-bystep, instead of rushing me through; she was able to work slowly with me. Yes, because I was able to view a transcript of the results at the end of the chat. (UOS 59) Net Gen I was looking for achievements of [the] Celtic civilization. I needed one more category. I came out not with the answer, but the MEANS of finding the answer.... I managed to find a topic on which there was SO much, I could hardly believe I missed it. (UOS 58) Net Gen The chat format did help my experience to be successful because I was able to multi-task while the librarian did the search. If I was in a FtF situation, I may have gotten frustrated with the amount of time it took for the librarian to find what I needed and would have just said 'never mind.' (UOS 358) Adult Negative Critical Incidents There were 68 (50%) CIs described as negative. Thirty (%) of these were from Net Gens. When describing their unsuccessful encounters with librarians the Net Gens said that the librarians impeded the delivery or retrieval of information (67%, 0) or did not answer their questions (30%, 9). Both Net Gens and adults described a number of barriers in their discussion of negative critical incidents as illustrated by the below verbatim examples: Is it illegal for a company to force you to where their clothes, as in their brand of company clothes? Can you be treated differently based on the brand of clothes you wore to work that day? While I was provided with links to the Department that governs the question, the Department of Labor, the librarian was unable to find the information and took the answer no further than for me to call the Department directly. (UOS-60788) Net Gen I do not remember the question, but it was on World War II. I had waited until the last minute and the person seemed like they were not paying attention to me. Finally they just stopped the session without a goodbye. I simply signed in again and received a different person. They were much more help. If anything it was a waste of my time, but when I re-logged in I found someone who did know what I was talking about. (UOS 57) Net Gen I needed information on the West for a book a student was reading. The person did not listen to the question and gave the wrong information. The person did not listen to my needs and did not answer the question. (-UOS 336) Adult Results of Critical Incident Analysis - Telephone interviews Ninety eight critical incidents were collected from telephone interviews. The results of this analysis with representative, verbatim quotes from participants are described below. Positive Critical Incidents There were a total of 6 (8%) positive CIs from the 76 participants who responded to the question. Eighty percent (5) of the users described their successful chat interactions as those when the librarian provided the information. This included assisting with the information delivery and retrieval, providing the accurate answer or information, answering the question quickly, and locating specific resources.

13 Virtual Reference Service Quality 77 Twenty-six percent (7) of the respondents identified the librarian's positive attitude and the quality of the relationship with the librarian (5, 3%) as important factors in a successful chat encounter. The respondents provided comments to support these findings. The person gave me the right answer very quickly... it was exactly what I was looking for. (UTI- ) Net Gen Well when I needed a book for school, for class, and I had left it at school. And they told how many copies of it were available at the library, and that they were in so I knew I could find it there. (UTI-07) Net Gen Yeah it was successful. Yeah, I got an answer pretty quickly they told me who I should call and referred me to the form I should fill out if I wanted to apply for a job in one of the libraries. I guess just the fact that it was really quick was helpful. (UTI-) Net Gen I was looking for information about health insurance appeals in local states, and I had on my own found sort of something that was leading towards what I wanted for Maryland. I will try the chat and the librarian did a great job leading me to websites for what I was looking for in Maryland it was exactly what I needed both the general sites the general data and aggregate data. It was perfect. It was pretty quick. (UTI-0) Adult She was very careful to explain wait time and told me not to leave. That librarian gave me two or three websites. I was not able to get the information I wanted, but that was because it wasn't available. Well, what made it successful was, the second time I logged on, the librarian's helpfulness was very encouraging to me, and expected wait time was very helpful, and the librarian, I don't know how to say this, I don' know how hard they worked, but appeared interested in helping me, there was a human component to it. (UTI-37) Adult... And it didn't take her more than five minutes. I hadn't found it after half an hour of looking online. It was so nice because she found it so quickly. She also gave me the reference source, which I didn't ask for, but she gave it to me anyway. Incredibly polite, useful, helpful... I don't kow if I should go back on to have her give me more adjectives to describe it! And I think the other important thing was that she was so polite. Colleagues laughed at me and called it a dumb question. But number one, it's not a dumb question and number two, it wasn't funny to the librarian. She treated me professionally and in today's world, that is incredibly important. (UTI-8) Adult "The first one was, like I said, I had gotten an article off of one of the databases I had taken all of the bibliography information except for one key part, when I went back to do the bibliography I needed it again, I couldn't find it anywhere that's why I went on there. The person I was talking to couldn't help me. But the next day I got an from a librarian who was able to find it. The first interaction wasn't that successful, but because someone got back to me, showed that they cared. (UTI-8) Net Gen Negative Critical Incidents There was a total of 3 (5%) negative CIs described by 76 of the respondents. Twenty-seven (79%) of the respondents described negative CIs. The majority of the unsuccessful chat interactions were attributed to librarians who impeded the delivery or retrieval of information, did not answer the questions, gave inaccurate answers, or provided wrong information. Twenty-four percent (8) of the respondents attributed the unsuccessful chat interactions to a poor relationship with librarians because of the librarians' poor communication skills. Respondents described the following situations as unsuccessful interaction with librarians. The individual just Googled and produced the same pages I had already gotten to it. Didn't have the information that I was looking for... related, but not actual information. Legal information. Because the individual just used the same Google... there was no reason for me to use that resource because they were not able to find the resources. Very basically I just used Google. (UTI-03) Net Gen

14 78 Lynn Silipigni Connaway and Marie L. Radford I asked a similar question about comparative religion and the person did not take the time to ask me the questions and I guess he seemed a little annoyed that I would use chat reference anyway. I think that he just wrote something like "This is the best I could do", and it really was not much good. Just quickly pulled up something. Yes, I guess the person seemed unconcerned and really not wanting to continue the conversation. (UTI- 038) Adult I was teaching my students how to use it. I spent a lot of time prepping them on proper etiquette. In sessions the person didn't have good etiquette. Several students were brusquely cut off and several of them said "I can't find that now." It was a class situation, the librarians were not aware that it was a class, I think. (UTI-03) Adult... the librarian was a little bit snippy, said he was a librarian and not a linguist, what was I doing anyhow. I occasionally run into librarians who aren't curious. Sometimes the librarian cant find things. Not everyone knows off the top of their head. (UTI-00) Adult Discussion and implications The results of the user online surveys and telephone interviews provide data about how VRS users perceive the service, why they used VRS, and what makes their experience with the service successful or unsuccessful. Findings reveal that both content and relational factors contribute to successful and unsuccessful interactions with librarians. The convenience of chat reference was found to be one of the top reasons for using the service. Recommendations from friends, trusted librarians or teachers were important to users, especially the younger ones who may fear unknown chat environments (see also Radford & Connaway 007). Many of the VRS users also said they would recommend the service to others. Knowing the service is available, authoritative, and safe were incentives for use. Getting a quick, accurate answer and specific as well as relevant information were critical factors for a successful interaction with a librarian. Librarians who follow-up make participants feel as though their questions are important and usually resulted in successful experiences. The users highly valued librarians' subject knowledge and were willing to wait for subject specialists. The users also valued a positive attitude when interacting with a librarian. Personable and friendly librarians were noted in the descriptions of successful encounters. If the librarian had poor communication skills and a poor attitude towards the task, the encounter was described as unsuccessful. Receiving inaccurate answers was attributed to negative interactions with a librarian. If the librarian does not answer the user's question, the user likely will describe the encounter as unsuccessful. Not locating specific resources and a lack of knowledge also were identified as factors contributing to an unsuccessful encounter. Users ended the session abruptly if the librarian asked if the catalog already had been checked. Just as in FtF interactions, users do not like negative closure (Ross & Dewdney 99, 998; Ross, Nilsen, & Radford 009) in which users feel that librarians are trying to end the interaction without actually answering the question. Although accuracy and correct answers and the delivery of specific content were reported as the most important factors of successful reference interactions, users of VRS also value librarians who are knowledgeable about information sources and systems, display a positive attitude, and demonstrate good communication skills. Results from both interviews and surveys reveal that participants consider convenience to be a major factor when choosing how to get their information. Regardless of the preference for FtF reference or VR. library users prefer to interface with friendly librarians and to develop relationships with them. VRS users also want extended service hours, access to electronic information, and interactions and personal relationships with caring librarians. Conclusion Librarians and library service providers must understand their users to provide services and systems to better meet their information needs. The results of this study provide a basis for librarians to make critical decisions to ensure that reference services are sustainable and accessible. Users of VRS want librarians to provide needed and specific information quickly and in a variety of formats and expect courteous service. It is impera-

15 Virtual Reference Service Quality 79 tive that librarians market their services to users and educate librarians to provide accurate information in a variety of service modes in a hybrid reference model that provides convenient, authoritative, reliable services to meet an array of diverse needs and communication preferences. In order to encourage increased library use, librarians need to develop creative marketing of their services and systems to promote the full range of options available. Word of mouth recommendations are important, so librarians should take every opportunity to talk about their services to users and potential users. Librarians also need to reassure young people that VRS is safe and to develop positive relationships whether interacting FtF, on the telephone, or online. It is clear that Adult and Net Gen users tend to be repeat users if their information and interpersonal needs are met. They need to feel welcomed by and fully connected to knowledgeable, tech savvy professionals who take their queries seriously and who provide service that is as hassle free as possible, guiding users to quality information that does not force instruction. Notes. This project was funded by a grant from the Institute of Museum and Library Services (IMLS) and in-kind contributions from Rutgers, The State University of New Jersey, and OCLC Online Computer Library Center, Inc., QuestionPoint "provides libraries with tools to interact with users in multiple ways, using both chat and ." OCLC Web Site htm [Viewed Sep. 3, 009]. 3. For a more in-depth discussion of the characteristics and behaviors of Net Gens, see Radford and Cormaway (007), Connaway, Radford, Dickey, Williams, and Confer (008), and Connaway (008).. The total for Users is actually N=3 as there was one respondent who did not give an age and thus could not be calculated into Adult or Net Gen. However, the Net Gen and Adult total is N=. 5. USO = User Online Survey. 6. UTI = User Telephone Interview. 7. The Online Survey Instruments, Telephone Interview Questions, and the Coding Scheme for Critical Incidents are available in the Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives IMLS Final Performance Report as Appendices (Appendix J: Online Survey Instruments; Appendix K: Coding Scheme for Critical Incidents; Appendix L: Telephone Interview Questions) are available at researcb/activities/synchronicity/reports/ finalappendices.pdf. Acknowledgements The authors thank Erin Hood and Timothy J. Dickey (OCLC Research) and Jocelyn DeAngelis Williams (Rutgers University, School of Communication and Information) for their assistance in collecting and analyzing the data and in developing this paper. References Babbie, E. 00. The practice of social research. th ed. Belmont, CA: Wadsworth/Cengage Learning. Charmaz, K Constructing grounded theory: A practical guide through qualitative analysis. London: Sage. Connaway, LS Make room for the Millennials. NextSpace 0: 8-9. URL - htm [viewed April 00] Connaway, LS., ML. Radford, TJ. Dickey, JD. Williams, and PC. Confer Sense-making and synchronicity: Information-seeking behaviors of Millennials and Baby Boomers. Libri 58: Dempsey, B What boomers want. Library Journal 3 (): Flanagan, JC. 95. The critical incident technique. Psychological Bulletin 5: Gillon, S. 00. Boomer nation: The largest and richest generation ever and how it changed America. New York: Free Press. Hallam. G., and H. Partridge Evidence based library and information practice: Whose responsibility is it anyway? Evidence Based Library and Information Practice 006 (3). URL: article/view/07/5 [viewed 3 November, 006]. Joseph, M Active, engaged, valued: Older people and public libraries in New South Wales. Aplis 9(3): 3-7. Krohn, FB. 00. A generational approach to using emoticons as nonverbal communication. Journal of Technical Writing and Communication 3(): 3-8. Oblinger, DG., and JL. Oblinger, eds Educating the Net Generation. Educause. URL: educatingthenetgen [Viewed 30 April, 00]. Oblinger, DG., and JL. Oblinger Is it age or IT: First steps toward understanding the Net Generation. CSLA Journal 9(): 8-6. Ozkaramanli, E Librarians' perceptions of quality digital reference services by means of critical incidents. PhD dissertation, University of Pittsburgh. Powell, RR._ and LS. Connaway. 00. Basic research methods for librarians. th ed. Westport, CT: Libraries Unlimited. Radford, ML Relational aspects of reference interactions: A qualitative investigation of the perceptions of users and librarians in the academic library. PhD dissertation, Rutgers, The State University of New Jersey. Radford, ML The Reference encounter: Interpersonal communication in the academic library. Chicago: ACRL, A Division of the American Library Association.

16 80 Lynn Silipigni Connaway and Marie L. Radford Radford, ML. 006a. Encountering virtual users: A qualitative investigation of interpersonal communication in chat reference. Journal of the American Society for Ilifonnation Science and Technology 57(8): Radford, ML. 006b. The critical incident technique and the qualitative evaluation of the Connecting Libraries and Schools Project. Libra-y Trends 5(): 6-6. Radford, ML. 006c. Interpersonal communication in chat reference: Encounters with rude and impatient users. In The virtual reference desk: Creating a reference fimtre, ed. R. D. Lankes, E. Abels, M. White, and S. N. Haque New York: Neal-Schuman Publishers. Radford, ML., and LS. Connaway Seeking synchronicity: Evaluating virtual reference services from user. non-user, and librarian perspectives. Funded by the Institute of Museum and Library Services (IMLS). URL: [viewed May 00] Radford, ML., and LS. Connaway "Screenagers" and live chat reference: Living up to the promise. Scan 6(): 3-9. Ross, C., and P. Dewdney. 99. Best practices: an analysis of the best (and worst) in fifty-two public library reference transactions. Public Libraries 33: 6-6. Ross, CS., and P. Dewdney Negative closure strategies and counter-strategies in the reference interview. Reference and User Services Quarterly 38 (): 5-6. Ross, CS.. K. Nilsen, and ML. Radford Conducting the reference interview. " ed.. New York, Neal-Schuman. Ruben. BD What patients remember: A content analysis of critical incidents in health care. Health Communication 5: -6. Rushkoff. D Playing the fiture: How kids' culture can teach its to thrive in an age of chaos. New York: Harper- Collins. Strauss, A., and J. Corbin Basics of qualitative research: Techniques and procedures for developing grounded theory. ed. Thousand Oaks, CA: Sage Publications. Sweeney. R Millennial behaviors and demographics. URL: Article-Millennial-Behaviors.doc [Viewed 3 November, 006]. Watzlawick, P., J. Beavin, and DD. Jackson Pragmatics of human communication. New York: Norton. Williamson, K., M. Bannister, L. Makin, G. Johanson, D. Schauder, and J. Sullivan Wanting it now: Baby Boomers and the public library of the future. Australian Libra,y Journal 55(): 5-7. Willis, SL Technology and learning in current and future generations of elders. Generations 006: -8. Paper received: 0 October 009; accepted 8 April 00

17 COPYRIGHT INFORMATION Author: Title: Virtual Reference Service Quality: Critical Components for Adults and the Net-Generation Source: Libri 60 no Je 00 p ISSN: DOI: 0.55/libr Publisher: Connaway, Lynn Silipigni; Radford, Marie L. Walter de Gruyter GmbH & Co. KG Mies-van-der-Rohe-Str., Munchen, Postfach 0 6 9, Berlin 0785, The magazine publisher is the copyright holder of this article and it is reproduced with permission. Further reproduction of this article in violation of the copyright is prohibited. To contact the publisher: This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sublicensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.