User centred quality health information provision: benefits and challenges

Transcription

1 User centred quality health information provision: benefits and challenges Burstein, F.; Fisher, J.; McKemmish, S, Manaszewicz, R. and Malhotra, P. School of Information Management and Systems Monash University Abstract Recent research indicates people are increasingly looking to the Internet for health information. Equally however, there is increasing frustration with the sheer volume, lack of relevance and at times dubious quality of information retrieved. The Breast Cancer Knowledge Online project sought to build a user sensitive portal to assist women with breast cancer and their families overcome these problems and to facilitate the retrieval of information which would better meet the individual and changing needs of users. The research outcomes discussed in this paper describe the approach taken to building the metadata-driven portal, the outcome of usability testing of the portal, and the limitations of such an ambitious project. 1. Introduction People with serious illness, their families and carers require information in order to better understand, manage and cope with the medical condition that confronts them. When facing a life-threatening illness most people want information on prognosis, drug or treatment side effects, and the latest research news [1-2], as well as facilitative and psychosocial information that will assist them in the management of their condition [3]. They may also want this information in a variety of formats, in varying degrees of detail, in scientific or lay language, and from a variety of viewpoints whether emanating from the health professional, or from the personal perspective of a fellow sufferer. Some may desire this information for informed decision-making; others may need it as a coping tool, or a means of asserting control, or simply to alleviate anxiety and distress [4 5]. The medical, social science and information science literature of the past decade is replete with studies across all medical disciplines reporting on the growing demand for appropriate consumer health information as well as the increasing reliance on the Internet as a major source of such information [6 8]. The development of national health portals, such as HealthInsite, Healthfinder, HealthCanada, and NHSDirect, clearly demonstrates a belief in the efficacy of information provision in improving health outcomes by engendering the expert patient who is capable of actively participating in her own health care and hence lessening the burden on extended and escalating health budgets. Whilst many information providers and system developers recognize the diversity of their target audiences, it appears that little has been done to accommodate individual differences and preferences in consumer health information services, whether this be in traditional print medium, or online resources. To all intents and purposes, the health information consumer remains a homogeneous entity and the information produced assumes that one size fits all. Attempts at tailoring information have generally been confined to specific health intervention programs with the aim of inducing behavioural change [9], or associated with linking generic information with medical records data [10]. Furthermore, the information is largely produced, selected and disseminated according to its evidencebased status and according to what the health professional believes the patient should know [11-12]. The Australian Research Council funded project, A user sensitive portal prototype to breast cancer knowledge online (BCKOnline), embraces the concept of individualized information needs and the primacy of personal values and predilections in the discovery of timely, relevant and reliable information resources. For BCKOnline the user has remained the focus throughout the various phases of the project. This paper will provide an overview of the project including: the role of a metadata schema in facilitating quality information retrieval; the conceptual thinking underpinning the development of the schema and; its pivotal role in linking a repository of user aware resource descriptions to self-selected user profiles. The paper also describes the approach taken to developing the portal interface and the IT infrastructure utilized for its implementation. Finally, a brief report on the usability testing phase is provided as well as an assessment of current limitations and future development proposals. 1

2 2. Current Web based information provision: retrieval and quality Numerous studies document the growing appeal of the Internet as an information resource for patients and those generally interested in health information. In a recent review Eysenbach estimates that on a global level, 278 million Internet searches are being conducted every day, of which approximately 12.5 million searches are health related (p.365) [13]. Yet how efficient are these searches? Pew Internet and the California Health Foundation [14 15] found that for most users, search enginessuch as Google and Alta Vista, were less than optimal tools in locating health information. Spink et al [16] also note that users generally have little comprehension of how search engines work and that their searching techniques and use of lay terminology is only partly successful at locating useful health information where web searching often yields misleading or unrelated information for the lay health consumer. (p.45) In recent years the issue of the quality of online health information has achieved prominence. Numerous commentators across all medical disciplines [17 18] have investigated the accuracy and/or authoritativeness of diverse, disease specific websites often finding that the information is biased, incomplete, and even dangerous [19]. Issues of quality, and its various constituent elements of provenance, authority, currency and accuracy, have thus come to dominate recent discourse on Internet information provision. Public awareness about the usefulness of Internet information needs to be raised and the content of some websites needs to be further evaluated. Major cancer organisations should investigate websites and create and administer a seal of approval for safety and reliability, such as the HON code (p ) [20]. For the current online health information seeker, ignorant of effective free text searching strategies, or the priority rankings of various search engines, successfully navigating the Internet becomes an extremely complex and time consuming task. She may have to sift through countless pages of hits that may or may not be relevant and has no means of evaluating the quality of each retrieved document or record. Nor is the user guaranteed that the retrieved hits will be in the format, depth, or level of complexity she needs. BCKOnline has sought to address these issues of personalised retrieval and the authoritativeness of the information retrieved throughout the various phases of the project as explored below. 3. BCKOnline project This section describes the different phases of the BCKOnline project and the outcomes from each phase. 3.1 Information needs analysis The methodology for this phase of the project included focus groups and individual interviews with 59 women with breast cancer, 7 family members and 11 breast care nurses. Participants were purposively selected from both the private and public health systems operating in Australia and represented a range of demographic and disease stages. The focus groups primarily explored the specific information needs of the illness experience in particular: how participants used the Internet to locate information; their perceptions as to the quality and relevance of both online and supplied information resources; and the nature of the information resources deemed most relevant to their specific and individual needs. The results of this phase are reported in detail elsewhere [21]. Outcome: The focus groups confirmed the previous research findings with participants often commenting on the deluge of retrieved information and frequently expressing their frustration at not being able to easily retrieve the information they required. Other comments focused on the lack of relevance of many of the resources retrieved, and the difficulties encountered when the user was not familiar with medical terms and drug names or how to spell such names. Other issues of note included: the variety of user preferences as to format and presentation style of information; the wide range of needs relating to the level of detail and language used, from detailed scientific information to materials presented in simple, lay terms ; and the way that information needs changed over the disease trajectory or as individual circumstances changed. Generally, the findings from this phase of the research revealed the highly idiosyncratic nature of the information seeking process. Users required materials which they deemed to be relevant to their immediate needs and which they felt they could trust. 2

3 3.2 Development of a metadata schema and metadata repository of user centric resource descriptions The BCKOnline project has addressed the issues of information retrieval and quality, and the individual and dynamic information and decision-support needs of the breast cancer community by prototyping an intelligent portal driven by a metadata repository of user-centric resource descriptions. The development of the metadata schema, which provides the template for the standardised descriptions of breast cancer information resources stored in this repository, or online catalogue, is outlined in this section. Usually online catalogues or metadata repositories contain descriptions of information resources that are resource-centric they describe the resource in terms of its attributes: Author, Title, Subject. Search engines are given access to contextual information about the resource, but not the user. The BCKOnline portal contains a metadata repository, which is a catalogue that describes resources in this conventional way, but also includes user-centric resource descriptors. For example, the metadata stored in this catalogue includes information about target audience, how to access the resource if it is available offline, restrictions associated with its use, and information about quality and reliability. By referring to the descriptions of information resources in its metadata repository, the portal will identify and retrieve resources with greater precision and hence relevance to the individual user. For example, treatments vary according to disease stage, hence the woman faced with advanced disease will require different information to that of an early breast cancer patient. Furthermore, depending on her age, educational level, and personal values, she may desire information, which is solely psychosocial as opposed to medical/scientific. The project adapted and extended the AGLS Metadata Schema [22] to produce user-centric standardised descriptions or cataloguing records of breast cancer information resources. These records are stored in a metadata repository/database and are used by the portal technology to match search queries with descriptions of relevant resources. The nineteen element AGLS Metadata Schema is an Australian National Standard which is based on and extends the Dublin Core set of metadata elements, an international code which was designed to facilitate resource discovery on the Internet. The schema used in the BCKOnline metadata repository adopted fifteen of the AGLS metadata elements, and added a new Quality element and qualifiers. Additional qualifiers and encoding schemes were also defined for a number of other elements. Table 1. A summary of the BCKOnline Metadata Schema, indicating how BCKOnline has extended the AGLS Schema Elements Creator Publisher Contributor Availability Title Subject Date Identifier Rights Source Language Relation Type Format Audience BCKOnline Quality Qualifiers/Encoding Scheme BreastCare Victoria Glossary BCKOnline Disease Trajectory Encoding Scheme BCKOnline key words BCKOnline Category Encoding Scheme medical; supportive; personal Age Group Disease Stage User Type Locality Information Preference Credentials Review Process Evidence-based Purpose Balance Currency References Table 1 presents the elements used in the BCKOnline catalogue of resource descriptions and lists the additional qualifiers for the Audience element and the sector specific encoding schemes developed for the Subject and Type elements. The new Audience qualifiers were defined with reference to user profiling based on the needs of the target audience as identified in the user needs analysis. The user profiles were developed from an analysis of data emanating from the 3

4 focus groups and the current literature. The profiles are intended to meet the characteristics of the majority of potential users. Results indicated that variables such as Age, Disease Stage, Audience Type, Locality and Information Preference were major factors in determining the content, type, and depth of information sought. For example, younger women are more likely to be concerned with issues of fertility and self-esteem as a result of chemotherapy treatment and/or surgery as opposed to women in their sixties or seventies for whom such resources have little relevance. [23]. Nor does the latter group, as a whole, appear to relate to the more in depth scientific material which a younger, educated woman might. The rationale behind the creation of the user profiles was: information needs vary according to disease stage, age, time since diagnosis/treatment [21] the experiential/anecdotal resource is highly valued by certain women, their families and partners [21] women with breast cancer, their families and carers require timely and accessible information at the point of their choosing. users preferences for presentation format, depth and language may also vary between individuals and between content, e.g. a woman may at first require only generalised information, but once she has embarked on a certain treatment, the need for in-depth scientific detail may become a major consideration. The Quality element was added to deal with the breast cancer community s requirements relating to reliability and authoritativeness, while its qualifiers were developed using a conceptual analysis framework derived from examination of existing quality standards and codes, e.g. Health on the Net HonCode, the AMA Guidelines for Medical and Health Information Sites, HiEthics, DISCERN and OMNI [25-29], coupled with an analysis of user needs relating to quality. Schema analysis, metadata modelling, concept mapping, and empirical instantiation (which populates models with examples) were used to develop the semantic structure for the metadata schema [30]. Outcome: Using the template provided by the BCKOnline Metadata Schema, resource descriptions or cataloguing records were developed which provide provenance and identifying information about the resource (Creator, Contributor, Publisher, Title and Date), an Identifier (usually the URL) which enables direct online access, or information about how to access the resource if it is offline (Availability); descriptors that enable retrieval of the resource by subject, resource type and target audience (Subject, Type, Audience), and descriptive information about rights management, related resources, and resource format (Rights, Source, Relation, Format). Finally the cataloguing records include information about the quality of the resource. More details about the extensions to the AGLS Schema relating to Subject, Type, Audience and Quality are provided below. These elements enable the portal to customise information retrieval to user needs and to provide users with information that enables them to assess the quality of the resource they have retrieved. The AGLS Schema references a number of encoding schemes to assist users of the Schema in assigning metadata values to elements. The encoding schemes include specifications of how elements are structured (e.g. the ISO which regulates the presentation or structuring of dates), classification schemes, thesauri and sets of indexing terms. For medical subject matter, AGLS specifies use of MESH, an international standard for the classification of medical knowledge. However, recent studies by Eysenbach [31], Ziebland [4] and Spink et.al [16] demonstrate the importance of the vernacular in the retrieval of web-based documents. MESH s applicability is thus limited for consumer use. The BCKOnline Metadata Schema includes two additional sets of indexing terms, which are specifically geared to breast cancer and the potential searching strategies of lay consumers. Developed by one of the project industry partners, the BreastCare Victoria Glossary is a listing of medical terms dealing specifically with breast cancer diagnosis, treatment and management. Hence it allows a far more precise classification than is currently available through MESH. The BCKOnline Disease Trajectory Encoding Scheme provides the basis for one of the three search strategies available to users of the portal the capacity to search on terms relating to detailed phases in the disease trajectory of breast cancer. It was also used to select resources to be included in the prototype. The additional BCKOnline subject encoding schemes both incorporate terminology that is more likely to be used by a lay audience, e.g. drug names such as Herceptin instead of monoclonal antibodies or trastuzumab, and parochial expressions, idioms and acronyms such as HRT in place of hormone replacement therapy. The BCKOnline Metadata Schema adapted the Type element to enable classification of resources according to the categories Medical, Supportive and Personal. This classification was particularly important given the commitment to provide access to information that was not only of a medical or scientific nature. The information needs analysis clearly identified the need for psychosocial and facilitative information as well as personal anecdotes. The Type element qualifiers are used to enable users of the portal to narrow their search to include only resources of a medical, supportive or personal nature. This categorisation was also used in the identification and selection of resources for inclusion in the prototype. 4

5 User profiles were implemented as an extension of the Audience element of the AGLS Metadata Schema. The user profiles were also used in developing one of the three search strategies presented to the user of the portal. The portal interface enables the user to build a profile based on the attributes defined in the Audience element qualifiers, and then, by searching on the Audience element of the catalogue record in the metadata repository, matches the selected profile with resources that contain information highly relevant to the target audience represented by the profile. User profiles can be built from the following: Age (Under 40: 40-49: 50 69: Over 70); Disease Stage (Early breast cancer: Recurrent breast cancer: Advanced breast cancer); Information Preference (Plain/Brief: Plain/Detailed: Scientific/Brief: Scientific/Detailed) and User (Self: Child: Friend: Partner/Spouse: Parent). For example, if a user selects plain and brief as a presentation format, then her search, for example, will not retrieve medical journal articles. If a young woman is looking for personal stories and accounts the resources retrieved will be of stories from women in her age group. The cataloguing record in the metadata repository also contains information about the quality of resources in the form of qualifiers to the Quality element which specify attributes of the resource such as the credentials of the author and publisher, the purpose and currency of the resource, whether the resource is well referenced, whether the resource has gone through some kind of review process, whether it is evidence-based and the nature of this evidence, and whether it contains information that represents a consensus of opinion or is controversial. This element also enables the portal to provide the user with a narrative report highlighting the essential characteristics of the quality of the individual item its provenance, authoritativeness and reliability Information selection and classification The selection of resources for inclusion in the prototype was determined on the basis of selection criteria derived from the outcomes of the information needs analysis, literature reviews, and expert domain knowledge. The selection itself was undertaken by domain experts. The focus groups provided researchers with data as to: current information gaps; information resources which women valued highly; information concerning preferred information format and presentation as well as an understanding of the importance of both the scientific/medical data as well as the psychosocial, facilitative and anecdotal. Outcome: For the prototype 500 resources were selected and described. The selection criteria specified percentages of records from the Medical, Supportive and Personal categories according to the preferences for these types of resources expressed by the women in the focus groups. Within each of these categories, records relating to different phases of the BCKOnline Disease Trajectory were selected, again proportional in numbers to the interest expressed by the women in information relating to the various phases. The ways the materials were classified into medical, supportive, or personal is illustrated below in Table 2 Table 2 Information categories OUTCOME DEFINITION EXAMPLE MEDICAL SUPPORTIVE PERSONAL Clinical/scientific objective medical data Facilitative information Psychosocial information Subjective views on the experience of breast cancer and its impact *Scientific papers *Clinical Practice Guidelines *Clinical trial information and/or results *Drug reports, etc. *Treatment reviews *Information regarding financial, emotional support, practical assistance *Information on various therapies *Case studies *Women s/families and professional reflections or appraisals of the individual s experience of breast cancer The development of the selection criteria for individual resources also drew on analyses of existing codes and principles such as BIOME, the DISCERN instrument [25-29], and guidelines developed by relevant government agencies. BCKOnline domain experts were also conscious of the need to include resources, which are (1) of potential relevance to the Australian context; (2) in high demand according to the user needs analysis, such as alternative/complementary resources; (3) provide users with alternatives as to format presentation. Finally the quality of the resource was taken into account as part of the selection criteria, as presented below in Table 3. As can be seen different criteria were used to assess the quality of resources from the three different categories. 5

6 Table 3 Quality criteria used in selection RESOURCE Medical: Supportive: Personal: CRITERIA Mandatory Recommended Authorship Review process Currency Referenced Evidence-based (citations are provided) Authorship Currency Purpose Authorship Exemplifying range of views 3.4 Portal development Referenced (citations) Relevance to Australian audience The development of the portal interface was consciously user-centric and attempted to incorporate as many of the findings from the information needs analysis as possible as well as existing principles of web design. Consideration was given to the following factors: users diverse experience in the use of the internet and their varied information needs; the potential need for browsing specific disease trajectory resources when users were either not sure as to appropriate search terms or simply required an overview of available materials; providing users with the potential to select resources according to individual values and preferences; allowing users to directly access materials rather than linking to home pages, which would then require the user to continue searching; providing the user with decision aid tools in the form of an abstract and narrative quality report for each item so that interest, relevance and quality could be gauged immediately. The primary objective of the interface design was that the experience of using BCKOnline should be as intuitive as possible and that choices presented to the user must be self-evident. The current interface consequently provides the user with a home page describing the origin and purpose of the portal as well as direct links to the three search methods: personal profiles; disease trajectory search; and the equivalent of simple search. In representing the user profiles it was decided that cartoons rather than traditional labels and/or photographs would be used. It was felt that users dealing with potentially traumatic situations would find cartoons less confronting and possibly entertaining. Furthermore, consideration was given to the idea that cartoons would be less likely to influence possible user identification with, or rejection of, images portrayed via photographs of actual people. The methodology employed in defining and creating the interface is described below. Johnson and Henderson [32] argue that before the design process for a user interface begins, a conceptual model of that interface should be developed. They define a conceptual model as a high-level description of how a system is organised and operates (p. 26) [21].. The development of a conceptual model enabled the designers of the BCKOnline portal to refine the interface before the final design phase. This was an important step, as, given the nature of the portal, it was essential for the designers to be certain that the users would be able to interpret and use the site in the way intended. The PICTIVE technique (Plastic Interface for Collaborative Technology Initiatives through Video Exploration) was adapted and used to produce the initial conceptual model. Preece et al [33] describe this technique as using low fidelity office items, such as sticky notes and pens, and a collection of design objects to investigate specific screen and window layouts for a system (p. 307). Usually a PICTIVE session is video recorded, but in this case the session was audiotaped, and as the different designs emerged these were photocopied to record progression. The following steps, as suggested by Preece et al, were undertaken. The description below each step indicates how it was implemented in this case. Step 1. Participants are asked to prepare scenarios for using the system. The participants were domain knowledge experts who were part of the research project team. They were also women who had had breast cancer. The scenario they prepared was straightforward: how they would search the system for information relating to breast cancer. Step 2. Developers create a set of system components. The developer brought to the session coloured pieces of A3 paper, pens, different coloured sticky notes and blue tack (to fix to the paper elements of the system that were not represented as sticky notes, for example cut out circles to represent check boxes, prepared labels that had previously been agreed to such as the profile categories). Step 3. Running the design session involves: participants meeting each other, brainstorming ideas using the design objects. During this stage the objects are manipulated and moved around. Finally a walk-through of the final design is undertaken followed by discussions. Each of the participants knew each other. The developer began by explaining to the group the purpose of the session. Time was spent brainstorming ideas about the layout of the screens and how a user would 6

7 move through the system. At the conclusion of the session the conceptual model for the ultimate three main search screens had been decided. Outcome: The conceptual model of the screens was the key outcome of the session. Quotes from the participants illustrate further how this was achieved. Speaking aloud one participant said If I saw this on a screen I would go here and here, indicating by pointing and confirming the workflow. There was discussion on whether to use the headings I want or I am in relation to defining the profiles, with the final decision being I need. Discussion took place around the issues of how to organise the first of the three screens, which contained the profiles, and whether users would be required to enter a profile every time. The comment was: If I was going through the profile options it would be OK but if I wanted to go straight to [the] enter term search then I [would] want to go straight to this without going through this process. I don t want to make this selection every time. This was confirmed by the second participant who said The person down the track might know what it was they were looking for. The final decision was to allow users to go straight to entering a search term on the first screen without selecting a profile if they wished. One of the major issues was keeping the number of screens presented to the users to less than four and after much discussion this was achieved. This model was then used to design the interface, which changed little from what was produced during the session. The only changes made were as a result of technology limitations or fine-tuning (see Figure 1, which illustrates the first input screen) The technology The prototype portal has been developed using a suite of tools produced by the Resource Discovery Unit of the National IT Research and Development Centre DSTC (Distributed Systems Technology Centre). One of the projects of the Centre focuses on the use of metadata to increase the level of precision and recall for Internet search engines [34]. The result of this research is the tool called HotMeta [35]. It consists of a Broker, which provides a metadata repository and search engine, and a Gatherer, which is a document indexer that extracts metadata records from html documents and web crawler. HotMeta works through a web interface. It integrates with MetaEdit, the metadata editor component, which is used to produce metadata or cataloguing records of the selected resources. HotMeta provides interfaces to enable users to browse, find, and search the metadata records stored in the repository. It supports Dublin Core and AGLS metadata but also allows for customized metadata schema. The Metadata Search Engine dynamically generates webpages of the search result. At this stage of the prototype development the search results screen presents the user with a list of records together with the search parameters, eg profile elements, search terms, information type and numbers of records matching the search. Each record is described by the title, short abstract, type of resource (eg Medical, Supportive, Personal), and a narrative quality report. (see Figure 2 for example output screen). Figure 1 The first BCKOnline input screen with profiling characteristics Figure 2 BCKOnline search result output If the user wishes to view the whole metadata description of the resource it is available through a hyperlink situated below each record. 7

8 3.6. Usability evaluation The main objective of the usability evaluation-testing phase is to assess the efficacy of the requirements definition process, design process and the methodology that was employed in the BCKOnline prototype during both its planning and design stages. Further, the intention is to validate the usability of the Breast Cancer Knowledge Online portal as a decision support and information provision system, that could deliver differentiated access to information to meet the diverse and dynamic needs of users. Six women participated in a pilot usability test; four had breast cancer or were currently being treated for breast cancer; one was a medical doctor who had patients with breast cancer; and another woman was the daughter of a woman with breast cancer. All women had some experience with using the Internet. The usability evaluation instrument will be refined as a result of the pilot, and further usability evaluations will be conducted. The results of the first test are reported here. Participants were given 30 minutes to explore the site and were given instructions that they were to search for information relating to breast cancer that was relevant to them. A twenty-minute interview followed. The questions asked were both qualitative and quantitative. Due to space limitations the complete survey is not provided here, however a copy of the full instrument is available at the project website [ ml. The questions addressed the users ability to complete the task and their general reaction to the site, including how well they were able to navigate the site to find information; the usefulness of the retrieved information; who they thought might be interested in the site; whether they would return to the site; and the usefulness of the quality report. Outcome: All the participants were able to complete the task and locate information relevant to them. Most of the users would recommend the site to others but one did comment that users would have to have some experience using the Internet. They all found the site easy to use with ratings of 4 or 5 out of 5. Most users rated their overall experience 3 out of 5 with one user rating it 4. All agreed that they would return to the site and one participant commented: It (the site) [has] got legitimacy. Wealth of information. Easy to navigate once I realised for me [the] best is [the] general search." The information relating to the quality of the resources was appreciated by all of the participants as these comments illustrate: There is no rubbish. Finding out that it was a consumer group that was terrific, that is a real strong point of the site. Once I discovered the quality information, that was also interesting to read, sort of gauge someone else s responses to the information. I tend to question things, so even when I saw the clinician discussing that, I was like OK this was interesting. The participants also appreciated the differentiated access that the portal provided: I think as a young person with breast cancer I really like [the age profiles] because often the information is not relevant to us. So, to see a category where you can choose and look at specific information relevant to you, I think that s really important. I think the early breast cancer and the advanced breast cancer [categories are] useful and it is good to see the advanced breast cancer there. I mean I actually searched on recurrent. Outcome: The testing so far has confirmed that the BCKOnline portal has the potential for better meeting the needs of those looking for information about breast cancer. The testing also confirmed that the research objectives are on the right track in that the women appreciated resources that were more tailored to their needs. A number of minor changes were identified through the usability test such as the language used in some places, the lack of a back button, and the need to make sure that all screens display all their contents without scrolling. Further usability evaluation testing will be undertaken as indicated in the concluding section. 4. Lessons learnt The BCKOnline project aimed at developing ways of better meeting the individual and dynamic information needs of the breast cancer community by building a prototype of a portal driven by a metadata repository or online catalogue of user-centred resource descriptions. The portal can thus support search strategies based on user profiles, and present the user with summary information about relevant resources, including a quality report. The overall result of the project is very encouraging. As reported, the participants of the pilot usability study, indicated that such a resource shows a definite improvement over a conventional free information search, where results are very much dependent on the user s understanding of the way free text search engines work, their ability to formulate appropriate search terms, and their capacity to assess the quality of the resulting information. BCKOnline allows users to readily access resources, which have been selected, described and evaluated on the basis of an 8

9 extensive study of user needs and expert domain knowledge about relevant resources. This feature of the site was identified as a particular strength.. Having demonstrated the value of such an approach, more work is now needed to at least partly automate the ongoing process of identifying and describing resources, and to build more intelligence into the search processes. Some features of the portal will, however, continue to be dependent on the application of expert domain knowledge, for example the assessment of their quality. 5. Conclusions and future work For people with life-threatening illness, their carers and dependants, access to relevant information is vital in order to satisfactorily manage health problems. Information is pivotal in informed decision making, chronic disease management, and helping people to cope with the effects of their illness. Current information provision strategies, however, leave little room for individual autonomy or choice. The predominant model of information transfer generally assumes a generic patient, and consequently supplies generic information. BCKOnline, through its user profiles, innovative metadata repository of user-centric resource descriptions, and extensions to current approaches to describing target audiences and resource quality, attempts to address the diverse and changing information needs of the consumer. The preliminary usability results demonstrated increased satisfaction when users experienced the change from being passive recipients of an avalanche of information to active seekers of highly relevant information resources. This model will be even more successful in an envisaged future when the elements specified in metadata standards are encoded by the information publishers. A larger usability study, particularly for the evaluation of content and optimising information retrieval, is being undertaken to identify future directions for improvements to the portal design and information content refinement. Limitations in the current portal design and implementation, particularly relating to the need to automate metadata capture, and improve the search processes will be addressed by increasing its adaptivity. In particular, the intelligence of the portal will be enhanced by appropriate incorporation of adaptive systems behaviour. For example, the current portal does not support intelligence features like sound-like searches, learning and reasoning. Moreover not all differentiating factors identified in the information needs analysis have been currently addressed, e.g. ethnicity and language. It is also planned to further extend the portal metadata repository and enhance the interface technologies to bring the current prototype to the stage of being ready for testing in an operational setting. Although the outcomes of this study have been directed to supporting the breast cancer community, the study also demonstrates that they are generalizable and can be applied to other areas within the health domain and to online information provision more generally. 6. References [1] Satterlund, M. J., McCaul, Kevin D., Sandgren, Ann K. (2003). "Information gathering over time by breast cancer patients." Journal of Medical Internet Research 5(3). [2] Kirk, P. Kirk, I. Kirstjanson, L.J. (2004). What do patients receiving palliative care for cancer and their families want to be told? A Canadian and Australian qualitative study. British Medical Journal 328 (1343). 3 [3] Fogel, J. (2003). Use of Internet information by women with breast cancer. Health Expectations 6: [4] Ziebland, S. Chapple, A. Dumelow, C. Evans, J. Prinjiha, S., Rozmovits, L. (2004). How the internet affects patients experience of cancer: a qualitative study, British Medical Journal 328 (7439): 564. [5] Pinnock, C. B., Jones, C. (2003). Meeting the information needs of Australian men with prostate cancer by way of the internet, Urology 61 (6): [6] Dolan, G. Iredale, R. Williams, R. Ameen, J. (2004). Consumer use of the internet for health information: a survey of primary care patients. International Journal of Consumer Studies, 28(2): [7] Horrigan JB, Rainie L. Counting on the Internet. Pew Internet and American Life Project. Pew Internet, [8] Harris Interactive (2003). "No significant change in the numbers of 'cyberchondriacs' - those who go online for health care information." Health Care News 3(4 (March 28)): 1-4. [9] Hajak, P. Taylor, T. Z. Mills, P. (2002). Brief intervention during hospital admission to help patients to give up smoking after myocardial infarction and bypass surgery: randomised controlled trial, British Medical Journal 324: [10] Wills CE, Holmes-Rovner M.(2003) Patient comprehension of information for shared treatment decision making: state of the art and future directions. Patient Education and Counseling : 50: [11] Diefenbach, M.A., Butz, B.P. (2004). A multimedia interactive education system for prostate cancer patients: development and preliminary evaluation. Journal of Medical Internet Research 6 (1). [12] Bader, J. L., Strickman-Stein, Mancy (2003). "Evaluation of new multimedia formats for cancer communications." 9

10 Journal of Medical Internet Research 5(3). [13] Eysenbach, G. (2003). The impact of the Internet on Cancer Outcomes, CA: A Cancer Journal for Clinicians, 53 (6): [14] Fox, S., Rainie, L. (2002). Vital Decisions: How Internet users decide what information to trust when they or their loved ones are sick. Washington, D.C, Pew Internet & American Life Project: [15] Cain, M. M., Sarasohn-Kahn, J. Wayne, J. C. (2000). Health e-people: The Online Consumer Experience, California Healthcare Foundation. [16] Spink, A.Yang, Y. Jansen, J., Nykanen, P, Lorence, D. P., Ozmutlu, S. and Ozmutlu C., H. (2004). A study of medical and health queries to web search engines, Health Information and Libraries Journal, 21: [17] Kunst, H., Groot, Diederic, Latthe, Ballayi M., Latthe, Manish, Khan, Khalid S. (2002). "Accuracy of information on apparently credible websites: survey of five common health topics." British Medical Journal 324: [18] Eysenbach, G., Powell, J., Kuss, O, Sa, Eun-Ryoung. (2002). Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review, JAMA: Journal of the American Medical Association, 287: [19] Bessell, T. L., McDonald, S., Silagy, C. A., Anderson, J. N., Hiller, J. E. Sansom, Lloyd N. (2002). Do internet interventions for consumers cause more harm than good? A systematic review. Health Expectations, 5: [20] Schmidt, K and Ernst, E. (2004). Assessing websites on complementary and alternative medicine for cancer, Annals of Oncology, 15: [21] Williamson K. and Manaszewicz R. (2002). Breast cancer information needs and seeking: towards an intelligent user sensitive portal to breast cancer knowledge online. New Review of Information Behaviour Research, 3: [22] National Archives of Australia, mmary.html, 2003, last accessed June [23] Thewes, B., Butow, P., Girigis, A., Pendlebury, S. (2004). The Psychosocial needs of breast cancer survivors: a qualitative study of the shared and unique needs of younger versus older survivors. Psycho-oncology, 13: [24] Eysenbach, G., Kohler, C. (2002). "How do consumers search for and appraise health information on the world wide web? A qualitative study using focus groups, usability tests, and in-depth interviews." British Medical Journal 324: [25] Discern Online (1999). The Discern instrument. instrument.htm last accessed September [26] BIOME Special Advisory Group on Evaluation (2004). Resource evaluation for BIOME. last accessed September [27] Health on the Net Foundation. (2004). HON Code of Conduct for medical and health websites. last accessed September [28] HiEthics: Health Internet Ethics. (2004). Health internet ethics: ethical principles for offering internet health services to consumers. last accessed September [29] American Medical Association. (2000). Guidelines for Medical and Health Information Sites on the Internet. [30] McKemmish S, Acland G, Ward N and Reed B. (1999) Describing records in context in the continuum: the Australian recordkeeping metadata schema. Archivaria 48: [31] Eysenbach, G., Kohler, C. (2002). How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. British Medical Journal, 324: [32] Johnson, J. and A. Henderson, (2002). "Conceptual models: begin by designing what to design". Interactions, February: p [33] Preece, J., Y. Rogers, and H. Sharp. 2002,Interaction Design -- beyond human computer interaction. USA: John Wiley and Sons [34] Renato, I. and Waugh, A. (1997) Metadata: Enabling the Internet, CAUSE97 Conference, Apr. 97, [on line] last accessed June [35]"HotMeta" Metadata Search Engine website last accessed June Acknowledgements: The authors would like to acknowledge the contribution made by the other project team members: Dr.Kirsty Williamson, Ms.June Anderson, Ms. Fiona Ross (all SIMS), Ms. Sue Lockwood (BCAG); Mr. Sergio Viademonte (programmer), Ms. Pooja Maholtra, Ms. Jane Moon, and Ms. Chan Cheah (postgraduate research students). This project was funded by a collaborative Australian Research Council Linkage grant. Industry Partners were BreastCare Victoria, initiative of the Department of Health & Human Services, and the Breast Cancer Action Group (Victoria). 10