Using research evidence in mental health: user-rating and focus group study of clinicians preferences for a new clinical question-answering service

Transcription

1 DOI: /j x Using research evience in mental health: user-rating an focus group stuy of clinicians preferences for a new clinical question-answering service Elizabeth A. Barley*, Joanna Murray & Rachel Churchill*, *Section of Evience-Base Mental Health an Section of Mental Health an Ageing, Health Services an Population Research Department, Institute of Psychiatry, King s College Lonon, Lonon, UK Abstract Backgroun an objectives: Clinicians report ifficulties using research in their practices. The aim of the stuy was to escribe nees an preferences for a mental health clinical question-answering service esigne to assist this process. Metho: Multi-isciplinary clinicians participate in a focus group; users of the service supplie feeback. Results: Fifty-four clinicians receive answers to 84 questions about mental health treatments. User ratings showe that the answers ha multiple uses: informing health care (43), eucation (22), staff evelopment (28) an research (12), an were consiere useful, clear, relevant an helpful. Focus group participants appreciate critically appraise summaries of evience an stresse the time-saving benefit of the service. Clinicians without a meical training were least confient in applying evience. Attitues to research were positive, but concern was expresse about its potential misuse for political purposes. This appeare to arise from an ambiguity aroun the term insufficient evience, which participants felt is wiely misinterprete as evience of no effect. Conclusions: A highly value, responsive service has been evelope. A range of clinicians fin critically appraise summaries of research useful. Eucation about the use of research may help clinicians to be more evience base. Key Messages Implications for Practice Proviers of evience-base information shoul efine carefully the meaning of a fining of insufficient evience as there is confusion over this. Clinical question answering services shoul be tailore to the specific nees of their users as information nees may vary across settings. Methos use to prouce answers to clinical questions shoul be transparent an consier the varying levelsof unerstaningofresearchmethos among cliniciansfrom ifferent professions. Corresponence: Elizabeth A. Barley, Section of Evience-Base Mental Health, Health Services an Population Research Department, PO Box 32, Institute of Psychiatry, King s College Lonon, De Crespigny Park, Lonon, SE5 8AF, UK. elizabeth.barley@iop.kcl.ac.uk 298

2 Using research evience in mental health, Elizabeth A. Barley et al. 299 Implications for Policy Access to critically appraise an summarize research evience, such as that provie by clinical question answering services, shoul be available to clinicians in orer to save them time an help them be more evience base. Training in research methos an critical appraisal is neee for clinicians from the full range of core professions to facilitate evience-base practice. Introuction Clinicians report ifficulties in applying research evience. 1 However, improve access to meical literature an teaching about evience-base practice (EBP) can ai ecision making. 2 Clinical questionanswering services (CQAS) have been evelope to help by proviing summaries of research in response to clinicians questions. A survey of CQASs in the UK 3 foun that the 23 services ientifie varie in the service provie, the type of evience use an the amount of critical appraisal applie. Most i not perform critical appraisal. This is espite previous finings 4,5 that clinicians value information that has been assesse for quality an bias. Research into clinicians information nees has focuse on those working in primary care or general meicine. 2,4,6 However, a favourable attitue towars EBP has been foun among psychiatrists in Scotlan, 7 although, in common with their colleagues from other isciplines, they reporte insufficient time as a barrier. In the same stuy, critically appraise answers to participants clinical questions were provie. The psychiatrists like the answers an thought that they woul have been unable to prouce them themselves, thus inicating that a CQAS woul be useful. A CQAS Best Evience Summaries of Topics in Mental Health Care (BEST in MH) has been evelope to answer the questions of mental health practitioners in South-East Lonon. Limiting the service to local users was esigne to enable the ientification of issues that may interest local policymakers. The service was promote by to service irectorate managers who were aske to isseminate the information to their teams. Any clinician proviing mental health care in South-East Lonon coul use the service. The information prouce is isseminate through a website ( This hosts a atabase of answers, the tools use (e.g. search strategy, critical appraisal tools) anlinksto other EBP information. Answers are prepare by a part-time (22 h per week) researcher with a backgroun in nursing an psychology, base at the Institute of Psychiatry. The service focuses on questions about interventions because research 8 10 has shown therapy questions to be more common than those about iagnosis, prognosis or harms. An online enquiry form, accesse via the website, which prompts for patient/problem, intervention, comparison, outcome (P.I.C.O.) elements 11 was esigne in orer to assist enquirers form an answerable question. This takes the form: In (aults/chilren) with (problem/conition), how effective is (intervention) compare with (comparison/alternative intervention) in (outcome)? Enquirers complete the elements in parentheses; for example, In aults with epression, how effective is cognitive behavioural therapy compare with selective serotonin reuptake inhibitors in improving moo? Methos of preparing the answers are consistent with stanars for CQASs. 3 Following receipt of a question, the researcher conucts a systematic literature search to ientify the best available evience (guielines, systematic reviews an ranomize controlle trials). This is then critically appraise using publishe checklists. 12 A short summary, etailing the evience foun, how consistent an reliable it is an what conclusions can be rawn, is prouce on a front sheet which also contains the question an the ate of preparation. The search strategy is escribe an etails of the inclue evience provie. Answers are limite to 2 3 pages of A4 an returne within 10 ays. The service is free of charge to the user.

3 300 Using research evience in mental health, Elizabeth A. Barley et al. This paper escribes the early usage of BEST in MH an reports user feeback. A focus group iscussion between potential users is also escribe; this sought to etermine expectations of the service an attitues towars using the information provie by it. Methos Characteristics of enquirers an questions were recore. A feeback form (Appenix, available online) sent with each answer aske about its quality, relevance, clarity an usefulness. Enquirers were also aske how they use their answer an how it coul be improve. Forms were returne to the researcher who ha supplie the answer. Focus group iscussion A focus group iscussion was conucte with potential service users. There were 18 participants (seven psychologists, three psychiatrists, four nurses, one social worker, one clinical acaemic, one associate specialist an one mental health liaison practitioner ). The iscussion followe a workshop promoting BEST in MH which provie information on using the service an on EBP generally. Participants ha respone to an forware by their service manager an so were a self-selecte group with an interest in EBP. Because of room size constraints, inclusion was restricte to the first 20 replies (of 24). Two participants faile to atten. All gave written informe consent. The aim of the iscussion was to ientify expectations of the service. As iscussion ais, participants were provie with a BEST in MH answer an answers to the same question prepare by two other proviers of research evience: Clinical Evience 13 an the former National Library for Health (NLH) primary care CQAS. 14 These resources were selecte as they are well known an evience base. However, their approaches iffer from BEST in MH. The NLH service, which recently cease, provie a rapi response; a list of evience was provie but was not systematically appraise or structure; Clinical Evience presents full systematic reviews but is not a CQAS. An evience summary an etails of the trials reporte are provie; interventions are grae as to how beneficial they are likely to be an their potential for harm. The question was: In aults with epression, how effective is exercise as a therapeutic intervention compare with no treatment or any other active treatment? The BEST in MH answer was prepare in the usual manner. The other answers were ownloae from their websites. A topic guie was prepare in avance (Appenix, available online). Two topics were covere: (i) use of research in clinical practice an (ii) presentation of BEST in MH answers. The session was facilitate by one of the authors (JM), who is experience in running focus groups but unconnecte to BEST in MH. The other authors, incluing the author responsible for proucing the answers (EAB), acte as observers. The iscussion laste 1 h an was recore. Tapes were transcribe by an aministrative assistant an fiel notes use to clarify muffle speech. The transcript was rea by two of the authors (EAB an JM) to ientify key themes. Active searching for isconfirming examples was unertaken. The two researchers compare notes an reache consensus on the themes. Finings The service has receive 94 enquiries. Four of these were multiple questions an were reformulate to generate 10 answerable questions. Some coul not be answere by the service: eight were beyon its scope (two iagnosis, two auit relate, one broa info request, two questionnaire valiity, one patient specific); five require clarification but the enquirer i not provie this; one was rugs-relate an was referre to the Meicines Information Service at the Mausley (it coul have been answere, but BEST in MH attempts to utilize existing resources). Finally, two questions ha alreay been answere, copies of these answers were forware. In total, 84 answers have been supplie. Users Fifty-four iniviuals have receive answers to 84 questions; they represent a range of professions (Table 1). The biggest group comprise clinical psychologists.

4 Using research evience in mental health, Elizabeth A. Barley et al. 301 Table 1 Number of users by profession (n ¼ 54) an number of questions (n ¼ 84) aske by each group Job classification No. of users No. of questions aske Psychologist Psychiatrist 7 10 Nurse 5 6 Psychotherapist 4 8 General practitioner (GP) 3 7 Lecturer/acaemic 3 4 Occupational therapist 3 12 Social worker 3 3 Miscellaneous* 6 7 *Profession unknown (i.e. not given or coul not be classifie, e.g. service evelopment manager ). User feeback Feeback forms were returne by 34 (63%) users in respect of 54 (64%) answers. How BEST in MH answers were use Thirty-four users gave us this information for 54 answers (some ha multiple uses). Forty-three were use to provie mental health care, 22 for eucation or training, 28 for professional evelopment an 12 for research. Answers were share with patients (n ¼ 23), carers (n ¼ 10) an colleagues (n ¼ 51). One user i not share their answer. Questions Most questions concerne psychological or psychosocial interventions (n ¼ 63); others concerne pharmacological treatments (n ¼ 10) or both types of treatment (n ¼ 7), the remainer concerne complementary therapies (n ¼ 4). Most questions concerne aults (n ¼ 70) compare with chilren or aolescents (n ¼ 14). The majority concerne patients with mental health conitions (n ¼ 75), but some concerne the effectiveness of mental health treatments for patients with physical problems (n ¼ 7); two concerne the impact of treatments on carers. Time taken to prepare an answer This varie (range 1 18 h) epening on the available evience. The mean completion time was 6 h Figure 1 Users responses to evaluation questions in respect of 54 BEST in MH answers. For each, a high score equals greater satisfaction. Well: Overall, how well i we answer your question? extremely well ¼ 29, very well ¼ 17, well ¼ 1, aequate ¼ 7, poor ¼ 1; Relevant: Was the evience we reporte relevant to your questions? extremely relevant ¼ 34, very relevant ¼ 14, quite relevant ¼ 1, aequate ¼ 3, somewhat irrelevant ¼ 2, missing ¼ 1; Clear: How was our interpretation of the evience? extremely clear ¼ 30, very clear ¼ 19, aequate ¼ 4, poor ¼ 1, missing ¼ 1; Helpful : How helpful was your answer? extremely helpful ¼ 19, very helpful ¼ 27, quite helpful ¼ 4, aequate ¼ 4, very poor ¼ 1 24 min (SD 3 h 18 min). The number of ays taken to answer a question varie consierably (range 2 23 ays, mean 12, SD 5). Thirty-eight (45%) questions were answere within the target 10 ays. Response time was increase when an enquirer aske multiple questions. When response times per user were examine, it was foun that 29 out of 54 (54%) users receive an answer to at least one question within 10 ays. User satisfaction Thirteen users use BEST in MH more than once: nine aske two questions, three aske five questions an one aske 10 questions. Figure 1 shows the users responses to evaluation questions. These were overwhelmingly positive, with most answers rate as extremely or very well answere (83%), relevant (89%), clear (91%) an helpful (83%). Finally, users were aske for improvements. Ten respone: three suggeste no change, two wante help using the service (efinition of trial, help

5 302 Using research evience in mental health, Elizabeth A. Barley et al. formulating questions), one requeste evience from a wier range of stuy esigns, one wante references to all trials inclue in systematic reviews. The remainer wante information not originally specifie; for example, reference to comparative research in other creative therapies. Focus group iscussion Three themes were ientifie: Service Operation, BEST in MH Answer Presentation an Attitues to Using Research in Clinical Practice. Service operation This theme ha four sub-themes. How BEST in MH might improve practice. Participants (P) consiere BEST in MH a useful resource which coul improve access to research an inform their work. Commonly, they talke about how BEST in MH coul save time, often with the implication that they o not currently seek out evience as a result of a lack of time: My ieas woul be to use this for things that really woul take a lot of time, that woul take away from my clinical work an I really wouln t o otherwise. (consultant psychiatrist, P10) The ability of BEST in MH to eal with complex questions. Participants were shown some clinical questions; these were simple examples chosen to emonstrate question structure. Concern was expresse as to whether BEST in MH coul eal with the complex questions that arise in practice: For emonstration purposes, the presentation this morning was very clear, but I wonere also whether they were very simple examples use an, in seconary an tertiary care, complex questions that we re sometimes face with have not yet been emonstrate at this presentation, so I still nee a bit more time to test that out. (psychiatrist, P4) Others note that the service coul cope, but that there may be a lack of evience: Those kins of really complex co-morbi questions, it may well be that there just isn t any evience because research hasn t been one at that level an that specific. Which oesn t necessarily mean that the service isn t useful, it might just mean that there s a lot of isappointing answers once you start formulating questions at that level, but at least you ll get to know. (psychiatrist, P6) Uses for BEST in MH. Participants talke about uses for the information from BEST in MH. Often this was relate to changing services: You coul put the question generally: What is the evience out there? An although it wouln t be focuse out to your subsystem but you woul be able to get that evience an then you can show that to management an say, at local level look, nationally it says that, or whatever. An use that as a tool to progress your work. (nurse, P5) A comment was mae that BEST in MH answers shoul not be use to owngrae services: I mean, I think we shoul say that they shouln t be use as a management tool to owngrae services. It ought to evelop an enhance evience to progress an iniviual s care but it shoul not be collectively use to owngrae services or cut OT services or cut irect exercise classes for groups or yoga classes for iniviuals or the clients or patients we work with. (psychiatrist, P4) Although the same person thought it coul be use to upgrae them: Upgrae iniviual yes, if practice is eviencebase an iniviual kin of interventions improves the quality of life or treatment, then yes. (psychiatrist, P4) Participants thought BEST in MH answers might stimulate research: (If there is a lack of evience) that woul be a goo time to o a bit of service evaluation, a bit of an auit. (nurse, P2) An:

6 Using research evience in mental health, Elizabeth A. Barley et al. 303 What it coul o is just be a very big result in stimulating a small relatively easy research project, which woul be one on the war. Be one by front-line staff, as you say, an then be, an I think, you know, front-line staff getting involve in research.... The projects can be very, very small an still worthwhile. You know they can be one on one war an see whether..., you know. (consultant psychiatrist, P10) The spee of response of BEST in MH. Comments arose in response to the answers from the three services. The consensus was that the spee of response require woul vary accoring to the clinical setting an the purpose for which the answer was require. It was agree that many proviers of mental health care i not nee instant answers, such as provie by NLH, an woul prefer to wait for more etaile information: I think in mental health you o have longer to wait than in primary care, where maybe someboy is coming back tomorrow or later in the ay or you ve ha to go out on a home visit an you know, you see people for 8 min an have a surgery with 20 or 30 people, you know, we on t work at that pace in mental health an I think I rather wait a little bit longer for the more etaile an simplifie answer than to have back something that s much less useful. (clinical acaemic, P7) However, it was acknowlege that in some settings, a rapi response woul be appreciate:... we work in A&E an I woul actually fin that this (NLH) woul be more useful for me, if I just nee quick reference an there s a patient waiting for an answer ownstairs, I woul fin this much easier than waiting for ays for an answer. Because I only see patients as a one-off rather than seeing patients on a regular basis, so this as a one-off woul be goo, but obviously if I nee further information to share with my colleagues then certainly the BEST one woul o for me. (mental health liaison practitioner, P8) BEST in MH answer presentation There were two sub-themes. General presentation. There was agreement that the answers were clear; this concurre with the questionnaire ata. All participants like the summaries provie by BEST in MH an Clinical Evience. They also like having etaile information about the inclue stuies, whether or not they chose to rea it: Having the summary on one page at the front that s fine, because if that s all the time you ve got to rea, you ve got your answer an it s quite clear what the question is an then the etail is there to refer to when you ve got time. (clinical acaemic, P7) Generally, it was felt that structure improve reaability, making specific information easier to fin. The participants were critical of the use of abbreviations, even when they ha been explaine. They wante full references, links to original articles an information about how the answer was prouce inclue in the answer as they were unwilling to spen time referring to the website. Trustworthiness. Participants were alert to the possibility of bias in the preparation of answers. It was thought that having a structure format an etails of the methos use woul help them ientify bias. This was consiere especially important for BEST in MH: Perhaps for a more local service with what is, at the moment, one person running it, those issues about transparency are particularly important compare to, you know, national things that have been prouce by groups of people. (clinical psychologist, P9) Attitues to using research in clinical practice Participants were well ispose to using research in their practice. This was supporte by responses to a pre-workshop questionnaire (15/18 respone) which inicate that unerstaning research was very (n ¼ 9) or quite (n ¼ 6) important to clinical practice. Three sub-themes emerge. Concern about the use of evience. Several participants expresse concern that research evience coul be use inappropriately:

7 304 Using research evience in mental health, Elizabeth A. Barley et al. I think it s quite political, I think there are ways that statistics an information can be use to support political aims. (clinical psychologist, P11) An: I mean, I think we shoul say that they (BEST in MH answers) shouln t be use as a management tool to owngrae services. (psychiatrist, P4) There was also a feeling, particularly from non-meical participants, that over reliance on evience may threaten clinical jugement: As a front-line social worker, I also feel that I think we nee to take a bottom to top approach because we face up with the clients in the front an we know the ifficulties in terms of their ay-to-ay lives an circumstances. So I think there s a great pool of experience in terms of front line. (social worker, P17) An: Where will this all lea to? A sort of mass stanarization an the sort of high street thing, that you know, everything will be the same, an what will happen to clinical jugements?... where to fin an answer to a clinical problem? We just ask the computer. (nurse, P14) Others, however, stresse the importance of reflective practice: I think we, as clinicians an researchers, have a responsibility to also be critical about what we re using an how. (clinical psychologist, P11) Lack of evience. Similarly, there were concerns about the implications for practice when there is insufficient evience to support an intervention: I mean there s stuff that you just wouln t stop oing because there s no evience an, you know, there s years an years of common sense, clinical jugement aroun... (clinical psychologist, P11) There was particular concern that the available evience is skewe against some interventions an populations, such as non-meical interventions, ethnic minorities an complex or rarer conitions. The concern was that this woul lea to biase ecision making: There s only evience that CBT works so that s the only thing we re going to fun. (nurse, P12) Participants felt that people i not always unerstan that lack of evience of effectiveness is not the same as evience of no effect: I think people o nee remining time an time again that lack of evience is not evience that something oesn t work. (clinical acaemic, P7) It was felt that a warning to this effect shoul be inclue in BEST in MH answers. Difficulties in carrying out research for staff members who are not meically traine. The nurses an social worker felt that, compare with meical staff, they lack the time, resources an skills to carry out research. Some also perceive a lack of relevant research, although, this was challenge by other participants. Discussion A CQAS has been evelope an 84 questions concerning mental health treatments from 54 clinicians have been answere. Clinicians from a range of professions have use the service an reporte sharing the information provie with colleagues, patients an carers. Feeback, in the form of questionnaire ratings an from the focus group, was overwhelmingly positive. A range of uses for BEST in MH answers, incluing patient care, eucation, professional evelopment an research, were ientifie. This is the first stuy of EBP in mental health to consier the views of participants from a range of professions. There were insufficient ata to make formal comparisons, but the multi-isciplinary focus group participants escribe similar information nees an there was consensus in their positive evaluation of BEST in MH. Finings from the iscussion were supporte by the questionnaire responses of the service users. In the iscussion, the ability of BEST in MH to save clinicians time emerge as important. This is supporte by other

8 Using research evience in mental health, Elizabeth A. Barley et al. 305 stuies which show lack of time to be a barrier to EBP. 1,7 The most common reason for accessing BEST in MH was to inform patient care. These ata are self-reporte an coul not be externally verifie; however, stuies of other evience-base services in primary care 15 an in Italian physicians 16 have elicite similar reports. The impact of information services is ifficult to stuy. A review of the literature 17 concerning the impact of such services in primary care foun only a small boy of mostly evience. This, nevertheless, suggeste such services were beneficial. Focus group participants were clear about how they like answers to be presente. The evience summaries provie by BEST in MH an Clinical Evience were appreciate. They also like etails of the reference stuies so they ha the option of reaing them. This fining supports that of an evaluation of Clinical Evience, 16 where 64% of the physicians questione reporte reaing only the conclusions. Overall, the participants wante structure, easy-to-rea answers containing all relevant information (methos, references, links to stuies, etc.). Participants felt that BEST in MH provie this. The BEST in MH answer use in the iscussion was prepare for the occasion. However, care was taken, by following the usual BEST in MH stanarize an publishe methos, to ensure that it was representative of actual answers. Questionnaire feeback, base on actual answers, supporte the finings of the iscussion. The responsiveness of BEST in MH was iscusse. In some situations, such as A&E, participants sai they woul require an immeiate response. However, most felt that mental health care proviers iffere from clinicians within other specialities as they saw their patients over a longer perio of time. This meant they coul wait for an answer. Mental health CQASs may therefore have the time to provie more complex information than other CQASs. The ability of BEST in MH to answer complex questions was querie. Previous research 1 has ientifie oubt about the existence of relevant information as an obstacle preventing the pursuit of answers to clinical questions. In our stuy, such oubt may have been increase by the use of simple emonstration questions. This problem has been aresse in subsequent workshops by using questions which exhibit the full range of questions consiere by BEST in MH. Attitues to EBP were positive, which was as expecte for this self-selecte sample. However, concerns were raise that BEST in MH answers coul be use inappropriately to support political ecisions to owngrae services an that clinical jugement woul be overrien. Such fears appeare to be centre on the issue of lack of evience. The participants suggeste that there is more evience available for some populations or treatments than for others. It was felt that over reliance on evience may therefore lea to inappropriate ecision making. This concern appeare stronger for non-meical participants, who were also least confient about their research skills. This may be because psychiatrists have more teaching about EBP than other professions, i.e. they are examine on critical appraisal for the Royal College of Psychiatrists Part II qualification. Other research has also shown ifferences between professionals. Cognitive behavioural therapists, 18 compare with other psychotherapists an psychologists in training, rate evience-base factors, such as manuals an guielines, significantly more highly as influences on practice. Although, this may reflect the larger evience base for CBT compare with other therapies. There is also confusion over what is meant by insufficient evience. It was felt that this is often misinterprete as evience of no effect, i.e. evience that a treatment i not work as oppose to there not being enough evience to etermine whether a treatment was effective. It was agree that this was a problem for many clinicians irrespective of profession. In view of this, those proviing eviencebase information or teaching EBP shoul efine clearly what is meant by a fining of insufficient evience. Conclusion This CQAS has been foun to be well receive by a range of clinicians. Those using the service an those consulte as to their preferences like critically appraise summaries of evience an having more etaile information with the option of reaing. Professionals without meical training were least confient about using research. More teaching

9 306 Using research evience in mental health, Elizabeth A. Barley et al. about EBP an better explanation of what is meant by a fining of insufficient evience may help clinicians to be more evience base. Acknowlegement This project is fune by Guy s an St Thomas Charity. There are no known conflicts of interest. Appenix is available online at interscience. wiley.com/journal/hilj. Supporting Information Aitional Supporting Information may be foun in the online version of this article: Appenix S1. BEST in MH Feeback Form. Please note: Wiley-Blackwell are not responsible for the content or functionality of any supporting materials supplie by the authors. Any queries (other than missing material) shoul be irecte to the corresponing author for the article. References 1 Ely, J. W., Osheroff, J. A., Chambliss, L., Ebell, M. H. & Rosenbaum, M. E. Answering physicians clinical questions: obstacles an potential solutions. Journal of the American Meical Informatics Association 2005, 12, Crowley, S. D., Owens, T. A., Schart, C. M., Warell, S. I., Peterson, J., Garrison, S. & Keitz, S. A. A web-base compenium of clinical questions an meical evience to eucate internal meicine resients. Acaemic Meicine 2003, 78, Bazian Lt. Developing Stanars for Clinical Question-Answering Services: A Report for the NHS National Knowlege Service. Lonon: Bazian Lt., Putnam, W., Twohig, P. L., Burge, F. I., Jackson, L. A. & Cox, J. L. A qualitative stuy of evience in primary care: what the practitioners are saying. Canaian Meical Association Journal 2002, 166, Lawrie, S. M., Scott, A. F. & Sharpe, M. C. Implementing evience-base psychiatry: whose responsibility? British Journal of Psychiatry 2001, 178, Dawes, M. & Sampson, U. Knowlege management in clinical practice: a systematic review of informationseeking behavior in physicians. International Journal of Meical Informatics 2003, 71, Lawrie, S. M., Scott, A. I. & Sharpe, M. C. Evience-base psychiatry: o psychiatrists want it an can they o it? Health Bulletin (Einburgh) 2000, 58, Ely, J. W., Osheroff, J. A., Ebell, M. H., Bergus, G. R., Levy, B. T., Chambliss, M. L. & Evans, E.R. Analysis of questions aske by family octors regaring patient care. British Meical Journal 1999, 319, Smith, R. What clinical information o octors nee? British Meical Journal 1996, 313, Brassey, J., Elwyn, G., Price, C. & Kinnersley, P. Just in time information for clinicians: a questionnaire evaluation of the ATTRACT project. British Meical Journal 2001, 322, Sackett, D. L., Richarson, W. S., Rosenberg, W. & Haynes, R. B. Evience-Base Meicine: How to Practice an Teach EBM. New York: Churchill Livingston, Churchill, R. Critical Appraisal. In: Prince, M., Stewart, R., For, T. & Hotopf, M., (es). Practical Psychiatric Epiemiology. Oxfor: Oxfor University Press, Clinical Evience. Lonon: BMJ Publishing, Available from: 14 National Library for Health. NLH Primary Care Clinical Question Answering Service. Coventry: National Library for Health, Available from: (accesse 20 February 2007). 15 Verhoeven, A. A. & Schuling, J. Effect of an eviencebase answering service on GPs an their patients: a pilot stuy. Health Information an Libraries Journal 2004, 21(Suppl. 2), Formoso, G., Moja, L., Nonino, F., Dri, P., Ais, A., Martini, N. & Liberati, A. Clinical Evience: a useful tool for promoting evience-base practice? BMC Health Services Research 2003, 3, Lacey Bryant, S. & Gray, A. Demonstrating the positive impact of information support on patient care in primary care: a rapi literature review. Health Information an Libraries Journal 2006, 223, Lucock, M. P., Hall, P. & Noble, R. A survey of influences on the practice of psychotherapists an clinical psychologist in training in the UK. Clinical Psychology an Psychotherapy 2006, 13, Receive 22 July 2008; Accepte 25 November 2008