Vol. 40, No. 6 423 Meical Informatics A Training Intervention to Improve Information Management in Primary Care Karen E. Schifferecker, PhD; Virginia A. Ree, PhD; Karen Homa, PhD Backgroun an Objectives: Training programs esigne to improve information management have been implemente but not aequately teste. Three critical components for information management were teste in a ranomize control stuy: (1) knowlege of vali, synthesize summary information, (2) skills to use Web-base resources that provie access to these summaries, an (3) use of Webbase resources in clinical practice. Methos: Twenty-four primary care practices were provie with computers an high-spee Internet access an then matche, with half ranomly assigne to receive training an half to receive training at a later ate. Training was esigne to aress knowlege, skills, an use of Web-base information. Outcomes were assesse by comparing baseline an follow-up questionnaires that focuse on five conceptual omains relate to Web-base resource use for patient care ecisions an patient eucation. Results: Compare to the elaye training group, the initial training group increase their knowlege an skill of Web-base resources an use for patient care ecisions. Some measures of communication with patients about using Web-base resources an of incorporating use of Web-base resources into aily practice increase from baseline to follow-up for all participants. Conclusions: Our finings suggest that training an proviing computers an Internet connections have measurable effects on information management behaviors. (Fam Me 2008;40(6):423-32.) Evience-base meicine (EBM), the integration of best research evience with clinical expertise an patient values, 1 can be ifficult to carry out in ay-toay clinical practice. 2-4 Numerous stuies have shown that proviers have clinical questions that they never answer 5-8 an that there are barriers to the implementation of EBM protocols, 9,10 incluing not having the knowlege, skills, an time to access an process resources for patient care ecisions. 2-4,7,8,10-12 Family meicine has been at the forefront of promoting EBM in clinical practice by focusing on information management skills, 2-4,13-15 efine as the ability to access an use electronically base summaries of evience for patient care ecisions. 4 These skills are relevant not only for proviers but also for patients, who are increasingly accessing health information through the Internet. 16-19 Health care proviers are encourage to From the Department of Community an Family Meicine, Dartmouth Meical School, Hanover, NH (Drs Schifferecker an Ree); an Dartmouth- Hitchcock Meical Center, Lebanon, NH (Dr Homa). assist patients in their use of Web-base resources, 20-24 which is consistent with evience-base patient choice. 25 Stuies suggest, however, that while patients want recommenations of online health resources from their proviers, 26 few receive them. 16,18 The growing availability an use of electronically base resources by patients an proviers thus creates both pressure an opportunity for clinical practices to use these health information resources for patient care ecisions an patient eucation. 27 Proviers may access resources via a number of technologies, incluing personal igital assistants (PDAs), but two recent ranomize stuies foun that computers are use two to four times more often at work than PDAs. 28,29 Stuies also ocument an increase in the use of the Web by health care proviers, 11,27,30 but overall prevalence of use to answer clinical questions is still low compare to other resources (eg, textbooks an colleagues), 7,8 which may not be up to ate on current evience. While one obvious barrier for proviers an practices to use Web-base resources in patient care ecisions an patient eucation is lack of computers an high
424 June 2008 Family Meicine spee Internet access, another important barrier is insufficient knowlege of Web-base resources an skills to efficiently use these resources. 2-4,10,12 Training programs evelope to improve information management skills have focuse primarily on skills, such as critical appraisal, 31,32 or on a single Web-base resource, such as MEDLINE. 33,34 Aitionally, much of the training has been implemente in meical eucation settings. 35-40 Training programs evelope for community-base, primary care settings have not been aequately teste for effectiveness. 37,41,42 One stuy has assesse training for rural physicians using a ranomize esign an foun increases in frequency of use an comfort level in using Web-base resources between intervention an control groups. 43 This paper escribes an evaluates an intervention esigne to promote information management in patient care ecisions an patient eucation via Webbase resources among participants from primary care practices. The intervention aresse three critical components to aopting an maintaining information management behaviors: (1) knowlege of vali, synthesize information, 6 (2) skills to use Web sites an search engines that provie access to these summaries, 44,45 an (3) use of the Web sites an search engines in clinical practice. We hypothesize that the intervention woul increase knowlege, skills, an use of Web-base resources among primary care practice participants. Methos The stuy, conucte since July 2004, involve 24 primary care practices locate in New Hampshire an Vermont. The practices were recruite through a previous stuy, 46 by a hospital computer network expert who knew of practices with few computers or poor Internet access, an through a presentation to a local preceptor boar. 47 The stuy use a ranomize, matche-pairs esign that allowe us to match practices on factors that coul influence training outcomes, such as existing high-spee Internet connections an number of computers with Internet connections in the practice prior to the intervention. Practices were first matche by size, specialty, an location. Then one practice in each pair was ranomly assigne to the initial training protocol (initial intervention group) an the other to the elaye training protocol (elaye intervention group). A crossover esign ensure that both groups woul eventually receive training, as shown in Figure 1. Dartmouth College s Committee for the Protection of Human Subjects approve the stuy protocol. Practices The 24 practices consiste of nine (37%) family meicine practices an five (21%) each of internal meicine, peiatrics, an multi-specialty practices. Practice size inclue five practices (21%) with a single provier, 13 practices (54%) with two to five proviers, an six practices (25%) with six or more proviers (range 6 16 proviers). The number of patients seen per week range from 50 to 800. At baseline, 33% of the recruite practices i not have high-spee connections, 54% ha limite computer resources, an 38% ha electronic meical recor (EMR) systems. To ensure that each of the 24 practices ha at least the basic technological resources neee to access Web-base resources, each practice was provie with two computers, high-spee Internet connection (if not alreay in place), an financial support to maintain Internet connections. The computers an Internet con- Figure 1 Stuy Design
Meical Informatics Vol. 40, No. 6 425 nection were put in place for all practices after baseline ata collection but before ranomization into either the initial intervention or elaye intervention group. Participants The initial intervention group practices (n=12) selecte a team of one to three iniviuals (proviers, clinical, an office staff) for a total of 32, to participate in the initial training protocol. The elaye intervention group practices (n=12), knowing they woul participate in the training later, selecte similar teams, for a total of 32 participants. Intervention The training provie to the initial intervention group participants focuse on two components: knowlege an skills to access an efficiently use a variety of Web-base resources an evelopment of a practice-base plan to incorporate an increase the use of these resources for patient care ecisions an patient eucation in participants own practice settings. Teaching methos were use that have been shown to be successful in changing behaviors an patient care, such as case stuies, hans-on practice, an sequence sessions (learn-work-learn). 48,49 As shown in Table 1, the training, conucte by biomeical librarians an faculty experience in facilitating group learning, consiste of (1) two 6-hour workshops hel 2 months apart, (2) two practice-base exercises using Webbase resources between the workshops, (3) a support phone call 1 month after the first workshop, an (4) a site visit 1 month after the secon workshop to provie aitional support. Measure A questionnaire was evelope to assess knowlege, skills, perceptions, an use of Web-base resources for patient care an eucation. The self-report questionnaire was evelope after a review of the literature relate to use of Web-base resources in clinical practice an consultations with biomeical librarians. The instrument was pilot-teste with 16 proviers an staff not associate with the stuy an moifie prior to use to reflect their feeback. As shown in Table 2, questions selecte for this stuy were chosen to represent five conceptual o- Table 1 Overview of Training Intervention an Resources Provie Activities Accessing/Efficiently Using Resources Practice Improvement Plan Workshop I Search assignment base on a clinical case/exploration of HO, SG several online resources through hans-on exercise SG, LG Report finings Review of resources an search strategies by librarians using Fining EBM Answers Quickly an Efficiently chart as guie (http://www.artmouth.eu/~biome/resources.html/ LG, HO guies/fin_ebm.shtml) Practice search strategies for each resource using secon HO, SG clinical case Give an review hanouts to ai in locating an using resources LG Review between-workshop exercise: ientify an answer one clinical or patient eucation question using Web-base resources covere in workshop LG Discuss experiences using Web-base information in practice for patient care ecisions an patient eucation LG Review baseline ata on current Web-base resource use by proviers (summary) an patients (per practice) TW Create patient care flow iagram TW Brainstorm ieas on how to change the process of care to use Web-base resources for patient care ecisions an patient eucation TW Share finings/ieas LG Review of potential tools available to promote use of Webbase resources (eg, NLM information prescription pas [http:// www.informationrx.org/splash.html]) LG Develop initial improvement plan to increase use of Web-base resources in ay-to-ay practice TW Between-workshop exercises Workshop II Phone call/site visit Complete an sen back clinical or patient eucation question exercise I Discuss/aress search strategy issues ientifie in clinical question exercise LG Review online resources again base on participant LG, HO questions Answer questions about resources Provie aitional materials to ai in use of resources Implement initial improvement plan TW Share experiences from implementation of improvement plans SG,LG Revise/expan improvement plan TW Develop measures to evaluate goals LG,TW Review progress on improvement plan Review of practice goals/successes/plans to improve; offer support/suggestions HO Hans-on Internet searches (each participant ha own computer to work on) LG Large-group presentation/iscussion SG Small-group work (mae up of iniviuals from ifferent practices) TW Work by practice teams, with assistance from workshop faculty I Iniviual work
426 June 2008 Family Meicine Table 2 Composition of Domains Analyze Domain Questionnaire Item(s) Knowlege an skills Score from 1=not at all to 5=extremely How well you can fin information you nee on the Web Score from 1=not at all to 5=extremely How well you can fin eucational materials for your patients on the Web Score=sum of four items score from 1=not How well you can o the following Web tasks: at all to 5=extremely - evelop an effective online search strategy from my patient care questions (α=.93) - narrow search results using limits (eg, English only, age of patients, etc) - use terms such as or an an to wien or narrow my search - use subject heaings (eg, MeSH terms) to evelop a search Score=sum of eight items score from 1=not at all well to 5=extremely well (α=.92) Frequency of resource use Score=sum of seven items score from 1= never or selom to 4=aily or almost aily (α=.80) Score from 1=never or selom to 4=aily or almost aily Communication with patients about using resources for health information Score from 1=never to 5=aily or almost aily Perceptions about the use of resources Score from 1=strongly isagree to 5= strongly agree Incorporating use of resources into aily practice Score from 1=never to 5=very often Score from 1=strongly isagree to 5= strongly agree How well you know how to use the following Web-base resources: - Internet search engine (eg, Yahoo, Google) - MEDLINE (eg, via PubMe or OVID) - online clinical resource (eg, UpToDate, MDConsult) - online evience-base review resource (eg, Cochrane) - online practice guielines (eg, National Guielines Clearinghouse, Institute for Clinical Systems Improvement) - meical professions Web site (eg, American Acaemy of Family Physicians) - meical organization Web site (eg, American Heart Association, Centers for Disease Control an Prevention) - patient-oriente health information Web site (eg, MelinePlus, WebMD) How often in a typical month you use each resource liste to fin information for making specific patient care ecisions (your own ecisions or to help someone else make a ecision): - Internet search engine (eg, Yahoo, Google) - MEDLINE (eg, via PubMe or OVID) - online clinical resource (eg, UpToDate, MDConsult) - online evience-base review resource (eg, Cochrane) - online practice guielines (eg, National Guielines Clearinghouse, Institute for Clinical Systems Improvement) - meical professions Web site (eg, American Acaemy of Family Physicians) - meical organization Web site (eg, American Heart Association, Centers for Disease Control) How often in a typical month you use (eg, show to a patient, give to a patient, tell a patient about) Webbase materials (online an/or printe an kept on file for patients) How often you refer patients to specific Web sites I like it when patients bring in information they fin on the Web I feel that the Web oes or coul help us provie better health eucation materials to patients I feel that the Web oes or coul help us provie better clinical care to patients I use the Web uring my work hours at this practice for patient eucation I use the Web uring my work hours at this practice for patient care ecisions I think there is leaership at this practice that encourages using Web resources for patient eucation I think there is leaership at this practice that encourages using Web resources for patient care ecisions. mains relate to Web-base resource use for patient care ecisions an patient eucation: (1) knowlege of an self-assesse skills for using Web-base resources, (2) frequency of resource use, (3) communication with patients about using resources for health information, (4) perceptions about the use of resources, an (5) incorporating use of Web-base resources into aily practice. The questionnaire was maile to the practices for completion before they receive their computers an high-spee Internet connection (baseline), 1 year later (follow-up I), approximately 3 months after the initial
Meical Informatics Vol. 40, No. 6 427 intervention group complete the training protocol, an 2 years later (follow-up II), approximately 3 months after the elaye intervention group complete the training protocol. While all practice members with patient contact in each of the 24 practices receive questionnaires, for this stuy only participants who participate in the training protocol (initial or elaye) were analyze to evaluate the effectiveness of that training. Analysis To evaluate the effectiveness of the training, we use a linear mixe moel analysis to assess whether the initial intervention an elaye intervention groups were significantly ifferent from each other an whether the measures iffere significantly at baseline an followup I. Thus, there were two fixe factors to consier: group (initial intervention versus elaye intervention) an time (baseline versus follow-up I). While the participant completing the questionnaires was consiere a ranom factor, participants work within practices where they share aspects of the environment unique to the practice. Thus, to avoi violating the statistical assumption of inepenence, a nesting function was use in the moel to ientify which participants were from the same practice, with the moel making the appropriate ajustments in the error term. The two-factor moel create an interaction term that was the main item of interest in this stuy. This interaction term assesse whether the rate of change from baseline to follow-up I for the initial intervention group was significantly ifferent from the rate of change for the elaye intervention group. Significant interaction terms, those reaching a P value less than or equal to.05, were further investigate with paire an inepenent t tests to etermine the nature of the interaction. The interaction term ha four possible pairwise contrasts; since multiple comparisons can increase the likelihoo of Type I errors, the significance levels of these tests were ajuste using the Bonferroni ajustment. 50 Thus, to reach a significance level equivalent to.05 unajuste, the ajuste P value require for significance was.0125. If the interaction term was not significant, then the factor term comparing baseline to follow-up was examine to etermine whether there was a significant effect of time. A significant effect of time inicate that the average score at follow-up I for all participants was significantly ifferent from the average score at baseline for all participants. The crossover esign of the stuy allowe for replication of the training intervention. To assess the equivalency of both arms of the stuy, a linear mixe moel analysis using a single fixe factor, group (initial intervention versus elaye intervention) was use to examine ifferences in the scores of participants from baseline for follow-up II. Again, a nesting function as escribe above was use to account for the fact that participants were clustere within practices. Results Of the 64 participants, 45 (70%) complete both baseline an follow-up I questionnaires: 78% (25/32) in the initial intervention group an 63% (20/32) in the elaye intervention group. Forty participants (63%) complete questionnaires at all three of the survey points: 72% (23/32) in the initial intervention group an 53% (17/32) in the elaye intervention group. As shown in Table 3, there were no significant ifferences between the initial intervention an elaye intervention group participants at baseline. Most of the participants were female, an most were either a provier or clinic support staff member. There were no significant ifferences between those workshop participants who complete all questionnaires an those who i not. There were also no ifferences between initial intervention an elaye intervention group practices on these resources at baseline: (1) high-spee Internet connections (χ 2 =.540, P=.462), (2) aequate computer Table 3 Descriptor Variables at Baseline Delaye Intervention Group Initial Intervention Group Number of participants who complete both surveys 20 25 n (%) n (%) Gener Male 3 (15%) 7 (29%) Female 17 (85%) 17 (71%) Role in practice Provier (MD, DO, PA, NP) 8 (40%) 10 (40%) Clinic staff (RN, LPN, MA, SW) 6 (30%) 7 (28%) Aministrative support 2 (10%) 3 (12%) Other 4 (20%) 5 (20%) Mean (SD) Mean (SD) Age (years) 43.6 (11.1) 46.0 (12.7) Work at practice (years) 6.3 (6.9) 8.8 (8.6) Hours work per week 37.9 (9.7) 41.2 (8.4) A computer with Web access is available at work. Score from 1=never to 5=most of the time 4.5 (1.1) 4.6 (0.9) Computer at work has fast Internet connection. Score from 1=never to 5= most of the time 4.1 (1.4) 4.3 (1.1) Frequency of Web use at work for any reason. Score from 1=never to 6=several times a ay 4.8 (1.4) 4.8 (1.6)
428 June 2008 Family Meicine Table 4 Mean Scores, SDs, an Post-hoc Pairwise Comparisons for Significant Group by Time Interactions Delaye Intervention Group Initial Intervention Group (a) (b) (c) () Measure Baseline Follow-up I Baseline Follow-up I Knowlege an skills Mean (SD) Mean (SD) Mean (SD) Mean (SD) F P Value.05 Ability to fin information on the Web 3.4 (1.0) 3.5 (1.0) Ability to fin eucational materials on the Web for patients 2.4 (1.3) 2.4 (.8) Composite of skills for fining information on the Web for patients 10.6 (5.0) 10.4 (4.8) Skills in using a variety of Web-base resources 13.9 (9.0) 13.9 (7.4) 3.1 (.9) 2.1 (1.0) 7.6 (3.8) 10.0 (6.6) 3.8 (.8) c 5.45.025 3.1 (1.0) c 7.25.012 11.0 (4.2) c 6.11.019 21.2 (6.9) b, c 25.51 <.001 Frequency of resource use How often I use the Web for patient care ecisions 10.1 (3.5) 8.9 (3.2) 7.8 (2.9) 10.0 (3.1) 6.59.017 Perceptions about the use of resources I like it when patients bring in information they fin on the Web 3.4 (1.0) 3.5 (1.0) Incorporating use of resources into aily practice I think there is leaership that encourages using Web resources for patient eucation 2.9 (1.2) 2.9 (1.2) SD stanar eviation 3.2 (.7) 2.6 (1.1) 3.8 (0.5) c 4.19.05 3.5 (0.9) c 4.66.038 Note: Lower case letter enotes that, for a given row, the cell value is significantly ifferent from the values in column enote by the letter. Tests of ifferences for time (column a an b, c an ) use paire-means t test. Tests of intervention ifferences (columns a an c, b an ) were inepenent means t tests. A Bonferroni ajustment was use to control overall Type I error. Table 5 Mean Scores an SDs for Significant Effects of Time for All Participants Delaye Intervention Group Initial Intervention Group Measures Baseline Follow-up I Frequency of resource use Mean (SD) Mean (SD) F P Value How often in a typical month I use Web-base materials for patient eucation 1.6 (0.9) 2.0 (0.8) 6.6.016 Communication with patients about using resources for health information How often I refer patients to specific Web sites 1.8 (0.9) 2.5 (0.8) 17.0 <.001 Incorporating use of resources into aily practice The egree to which I think there is leaership that encourages using Web resources 2.5 (1.2) 3.5 (0.9) 16.1 <.001 for patient care ecisions How often I use the Web uring work hours for patient eucation 1.8 (1.0) 2.3 (1.0) 12.0.002 How often I use the Web uring work hours for patient care ecisions 2.4 (1.2) 2.9 (1.1) 7.0.013 SD stanar eviation
Meical Informatics Vol. 40, No. 6 429 resources (χ 2 =.691, P=.431), or (3) presence of an EMR (χ 2 =.097, P=.756). Table 4 isplays the mean scores an stanar eviations for the variables that ha significant interaction terms an the results of the post-hoc pairwise comparisons. Table 5 isplays the means an stanar eviations for the variables that ha a significant time effect for all participants. Knowlege an Skills There were significant group by time interactions for each of the four variables: (1) fining information on the Web, (2) fining eucational material on the Web for patients, (3) a sum of specific search skills, an (4) skills using a variety of Web-base resources. In each case, pairwise comparisons reveale a significant increase from baseline to follow-up I for the initial intervention group, as shown in Table 4. For the variable, skills using a variety of Web-base resources, there was also a ifference between the groups at follow-up I, with the initial intervention group inicating significantly more skills to use resources than the elaye intervention group. Frequency As shown in Table 4, while there was a significant interaction term for how often the participants reporte using a variety of Web-base resources for patient care ecisions (P=.017), the results of the comparison of baseline to follow-up I for the initial intervention group was not significant when ajuste for multiple comparisons (t 10 = -2.3, P=.041) since this ajustment require a P value of.0125 for significance. As shown in Table 5, all participants reporte a significant increase from baseline to follow-up I in how often they use Web-base materials for patient eucation. Communication All participants reports of how often they referre patients to specific Web sites showe a significant increase from baseline to follow-up I, as shown in Table 5. Perception Analysis of participants liking when patients bring in Web-base information reveale a significant interaction term, as shown in Table 4. Pairwise comparisons showe a significant increase from baseline to followup I for the initial intervention group. There were no significant results for participants reports of feeling the Web coul help them provie better clinical care or better health eucation materials to patients. Incorporation As shown in Table 4, there was a significant interaction term for participants beliefs regaring leaership at the practice that encourages using Web resources for patient eucation. Pairwise comparisons reveale a significant increase from baseline to follow-up I for the initial intervention group. As shown in Table 5, all participants reporte significant increases from baseline to follow-up I for their beliefs regaring leaership at the practice that encourages using Web resources for patient care ecisions an for how often they use the Web at work for patient eucation an for patient care ecisions. Replication of the Training Intervention Given that the participants were ranomize to either the initial intervention or elaye intervention groups, one woul expect there to be no ifferences attributable to early or late training. Differences in the mean scores of participants from baseline to follow-up II are shown in Table 6. Results of the analyses showe a single significant effect of group, for the variable assessing how often in a typical month participants use Web-base materials for patient eucation, with participants in the elaye intervention group reporting a greater gain in use than participants in the initial implementation group (P=.047). These results inicate that the training intervention in both arms of the stuy prouce substantively similar results. Discussion The major finings of this crossover stuy for improving information mastery in primary care settings were reporte increases in knowlege an skills about Web-base resources an frequency of use of these resources for patient care ecisions, all of which were associate with the training intervention. Aitionally, the stuy foun that all participants reporte increases in frequency of Web-base resource use for patient eucation, communication with patients about these resources, an incorporating resource use into aily practice. These increases were associate with all practices receiving aitional computers an high-spee Internet connections through this stuy after baseline ata collection. Although several changes from baseline to followup were statistically significant, the magnitue of the follow-up scores warrants further iscussion. For instance, the use of information management behaviors by participants coul still be consiere quite low (eg, on average, resources were selom use uring work hours or for patient eucation). Other measures i not change, such as participants perceptions about the usefulness of the Web for proviing better health eucation materials or clinical care for patients, although on average they agree that it oes (3.9 on a 5-point scale). In sum, significant room remains for improvement in information management skills, knowlege, an behaviors even after this fairly intensive training an support protocol.
430 June 2008 Family Meicine Table 6 Mean Differences an SDs between Scores at Baseline an Follow-up II Measures Delaye Intervention Group Initial Intervention Group Knowlege an skills Mean (SD) Mean (SD) Ability to fin information on the Web 0.8 (1.0) 0.7 (1.0) Ability to fin eucational materials on the Web for patients 1.2 (1.4) 1.4 (1.1) Composite of skills for fining information on the Web for patients 11.0 (6.9) 11.7 (8.4) Skills in using a variety of Web-base resources 3.6 (2.8) 3.2 (4.8) Frequency of resource use How often in a typical month I use Web-base materials for patient care ecisions 2.6 (3.5) 4.3 (3.3) How often in a typical month I use Web-base materials for patient eucation 1.1 (1.3) 0.2 (0.9) Communication with patients about using resources for health information How often I refer patients to specific Web sites 1.2 (1.2) 0.7 (1.0) Perceptions about the use of resources I like it when patients bring in information they fin on the Web 0.4 (0.7) 0.6 (0.8) I feel that the Web oes or coul help us provie better health eucation materials to patients 0.3 (0.8) 0.2 (0.6) I feel that the Web oes or coul help us provie better clinical care to patients 0.2 (0.8) -0.1 (1.0) Incorporating use of resources into aily practice I use the Web uring my work hours at this practice for patient eucation 1.5 (0.9) 0.9 (1.2) I use the Web uring my work hours at this practice for patient care ecisions 0.9 (1.3) 1.2 (1.2) I think there is leaership at this practice that encourages using Web resources for patient eucation 1.0 (1.2) 0.8 (1.3) I think there is leaership at this practice that encourages using Web resources for patient care ecisions 1.1 (1.5) 1.1 (1.2) SD stanar eviation Given these results, how o we move forwar to improve information mastery in primary care settings? Theoretical approaches, such as the theory of reasone action, 51 woul posit that behavior procees from knowlege, skills, an attitues or perceptions. In this instance, increase knowlege an skills an alreay-positive, existing perceptions (3.9 on a 5-point scale) about the usefulness of Web-base resources have prouce only a slight increase in behavior. This leas us to believe that other factors operating in the organizational environment are not currently reinforcing an/ or attracting proviers an staff members to information mastery behavior, at least in the case of Web-base resources. Factors may inclue perceive lack of time to use resources, 7 the experience of not fining answers using Web-base resources or taking too long to fin the answer, 52 an the habit of establishe behaviors (eg, talking to colleague to fin answers to questions). Training that occurs in primary care practice settings as clinical an patient eucation questions arise, although potentially resource intensive, may be more effective for aressing or improving some aspects of the organizational environment for information management. 53-55 In aition, improvements in the structure an content of existing Web-base resources may go far to increase their use in practice. 7,29,56,57 While a limitation of this stuy is the relatively small sample size that constrains the generalizability, selection bias is not a limitation because the carefully matche crossover esign controlle for the esire to participate. Use of self-report questions as proxies for information mastery measures might be consiere a limitation. While actual observation of participant skills might provie more objective measures, this woul be both impractical an artificial. Although self-report has been thought of as less accurate than assessment by others, Shrauger an Osberg 58 have note that, given the appropriate circumstances, jugments of one s own behavior can be as effective as those mae in other ways. While it is possible that response bias coul have occurre among workshop participants, their responses i not show increases in all measures. Three critical components for improving information management behaviors were teste in a ranomize control stuy. While the finings suggest that both the training protocol an provision of computers an high-spee Internet access have measurable impacts, important questions remain to be explore.
Meical Informatics Vol. 40, No. 6 431 Acknowlegments: This work was supporte by the National Library of Meicine grant 5 G08 LM 008110, the Office of Community-base Eucation an Research at Dartmouth Meical School, an Dartmouth Biomeical Libraries. The authors thank Karen Oato, Pamela Bagley, Tina Foster, an Ruth Burrill for their invaluable contributions to the evelopment an implementation of the training interventions, the practice staff an proviers whose participation mae this stuy possible, an Lynn Foster-Johnson for her statistical expertise. Corresponing Author: Aress corresponence to Dr Schifferecker, Office of Community-base Eucation an Research, Dartmouth Meical School, 46 Centerra Parkway, Suite 102, Lebanon, NH 03766. 603-653-3224. Fax: 603-653-3202. karen.e.schifferecker@artmouth.eu. Re f e r e n c e s 1. Sackett DL, Straus SE, Richarson WS, Rosenberg W, Haynes RB. Evience-base meicine: how to practice an teach EBM, secon eition. Einburgh: Churchill Livingstone, 2000. 2. Guyatt GH, Meae MO, Jaeschke RZ, Cook DJ, Haynes RB. Practitioners of evience-base care. BMJ 2000;320:954-5. 3. McColl A, Smith H, White P, Fiel J. General practitioners perceptions of the route to evience-base meicine: a questionnaire survey. BMJ 1998;316:361-6. 4. Slawson DC, Shaughnessy AF. Teaching evience-base meicine: shoul we be teaching information management instea? Aca Me 2005;80(7):685-9. 5. Ely J, Osheroff J, Gorman P, et al. A taxonomy of generic clinical questions: classification stuy. BMJ 2000;321:429. 6. Ramos K, Linschei R, Schafer S. Real-time information-seeking behavior of resiency physicians. Fam Me 2003;35(4):257-60. 7. Ely JW, Osheroff JA, Chambliss ML, Ebell MH, Rosenbaum ME. Answering physicians clinical questions: obstacles an potential solutions. J Am Me Inform Assoc 2005;12(2):217-24. 8. Gonzalez-Gonzalez AI, Dawes M, Sanchez-Mateos J, et al. Information nees an information-seeking behavior of primary care physicians. Ann Fam Me 2007;5(4):345-52. 9. Green M, Ruff T. Why o resients fail to answer their clinical questions: a qualitative stuy of barriers to practicing evience-base meicine. Aca Me 2005;80(2):176-82. 10. Hollaner S, Lanier D. The physician-patient relationship in an electronic environment: a regional snapshot. Bull Me Libr Assoc 2001;89(4):397-9. 11. Bennett NL, Casebeer LL, Zheng S, Kristofco R. Informationseeking behaviors an reflective practice. J Contin Euc Health Prof 2006;26(2):120-7. 12. Chew F, Grant W, Tote R. Doctors on-line: using iffusion of innovations theory to unerstan Internet use. Fam Me 2004;36(8):645-50. 13. Ebell MH. Point-of-care information that changes practice: it s closer than we think. Fam Me 2003;35(4):261-3. 14. Alper BS. Practical evience-base Internet resources. Fam Pract Manag 2003;10(7):49-52. 15. White B. Making evience-base meicine oable in everyay practice. Fam Pract Manag 2004;11(2):51-8. 16. Diaz JA, Griffith RA, Ng JJ, Reinert SE, Friemann PD, Moulton AW. Patients use of the Internet for meical information. J Gen Intern Me 2002;17:180-5. 17. Fox S. Health information online. May 17, 2005. Available at: http:// www.pewinternet.org/pfs/pip_healthtopics_may05.pf. Accesse January 17, 2006. 18. Fox S. Online health search 2006. October 29, 2006. Available at: http:// www.pewinternet.org/pfs/pip_online_health_2006.pf. Accesse December 15, 2006. 19. Manl KD, Feit S, Pena BM, Kohane IS. Growth an eterminants of access in patient e-mail an Internet use. Arch Peiatr Aolesc Me 2000;154(5):508-11. 20. Gomella LG. The wil, wil Web: resources for counseling patients with prostate cancer in the information age. Semin Urol Oncol 2000;18(3):167-71. 21. Houston TK, Allison JJ. Users of Internet health information: ifferences by health status. J Me Internet Res 2002;4:e7. 22. Ling CA. Guiing patients through the maze of rug information on the Internet. Am J Health Syst Pharm 1999;56(3):212-4. 23. Lorence DP, Greenberg L. The Zeitgeist of online health search: implications for a consumer-centric health system. J Gen Intern Me 2006;21:134-9. 24. Murray E, Lo B, Pollack L, et al. The impact of health information on the Internet on health care an the physician-patient relationship: national US survey among 1,050 US physicians. J Me Internet Res 2003;5:e17. 25. Eysenbach G, Diepgen TL. The role of e-health an consumer health informatics for evience-base patient choice in the 21st century. Clin Dermatol 2001;19(1):11-7. 26. For P. Is the Internet changing the relationship between consumers an practitioners? Journal for Healthcare Quality 2000;22(5):41-3. 27. Ree MC, Grossman JM. Growing availability of clinical information technology in physician practices. June 2006. Data bulletin no. 31. Available at http://www.hschange.org/content/848/848.pf. Accesse July 14, 2006. 28. Carroll AE, Christakis DA. Peiatricians use of an attitues about personal igital assistants. Peiatrics 2004;113(2):238-42. 29. Miller RH, Hillman JM, Given RS. Physician use of IT: results from the Deloitte Research Survey. J Healthc Inf Manag 2004;18(1):72-80. 30. Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians Internet information-seeking behaviors. J Contin Euc Health Prof 2004;24(1):31-8. 31. Crites GE, Chrisagis X, Patel V, Little D, Drehmer T. A locally create EBM course for faculty evelopment. Me Teach 2004;26(1):74-8. 32. Straus SE, Ball C, Balcombe N, Shelon J, McAllister FA. Teaching evience-base meicine skills can change practice in a community hospital. J Gen Intern Me 2004;20:340-3. 33. Schilling K, Wiecha J, Polineni D, Kahalil S. An interactive Web-base curriculum on evience-base meicine: esign an effectiveness. Fam Me 2006;38(2):126-32. 34. Vogel EW, Block KR, Wallingfor KT. Fining the evience: teaching meical resients to search MEDLINE. J Me Libr Assoc 2002;90(3):327-30. 35. Alper BS, Vinson DC. Experiential curriculum improves meical stuents ability to answer clinical questions using the Internet. Fam Me 2005;17(8):565-9. 36. Braley P, Oterholt C, Herrin J, Norheim L, Bjornal A. Comparison of irecte an self-irecte learning in evience-base meicine: a ranomise controlle trial. Me Euc 2005;39:1027-35. 37. Brettle A. Information skills training: a systematic review of the literature. Health Info Libr J 2003;20 suppl 1:3-9. 38. Ross R, Verieck A. Introucing an evience-base meicine curriculum into a family practice resiency is it effective? Aca Me 2003;78(4):412-7. 39. Smith CA, Ganschow PS, Reilly BM, et al. Teaching resients eviencebase meicine skills. J Gen Intern Me 2000;15:710-5. 40. Thom DH, Haugen J, Sommers PS, Lovett P. Description an evaluation of an EBM curriculum using a block rotation. BMC Me Euc 2004;4:19. 41. Balwin C, Niebuhr V, Sullivan B. Meeting the computer technology nees of community faculty: builing new moels for faculty evelopment. Ambulatory Peiatrics 2004;4(1 suppl):113-6. 42. Steiner V, Hartmann J, Ronau T. MeReach: builing an Area Health Eucation Center meical information outreach system for northwest Ohio. J Me Libr Assoc 2002;90(3):317-22. 43. Kronick J, Blake C, Munoz E, Heilbrunn L, Dunikowski L, Milne WK. Improving on-line skills an knowlege. A ranomize trial of teaching rural physicians to use on-line meical information. Can Fam Physician 2003;49:312-7. 44. Alper BS, White DS, Ge B. Physicians answer more clinical questions an change clinical ecisions more often with synthesize evience: a ranomize trial in primary care. Ann Fam Me 2005;3(6):507-13. 45. Ebell M. Information at the point of care: answering clinical questions. J Am Boar Fam Pract 1999;12(3):225-35. 46. Carney P, Poor D, Schifferecker K, Gephart D, Brooks WB, Nierenberg DW. Computer use among community-base primary care physician preceptors. Aca Me 2004;79(6):580-90. 47. Kollisch D, Gephart D, Brooks WB, Gagne R, Allen C, Donahue D. Impact of a preceptor eucation boar an computer network to engage community faculty at Dartmouth Meical School. Aca Me 1999;74(1) (suppl):s70-s74.
432 June 2008 Family Meicine 48. Davis D, O Brien MA, Freemantle N, Wolf FM, Mazmanian PE, Taylor- Vaisey A. Impact of formal continuing meical eucation: o conferences, workshops, rouns, an other traitional continuing eucation activities change physician behavior or health care outcomes? JAMA 1999;292(9):867-74. 49. Khan KS, Coomarasamy A. A hierarchy of effective teaching an learning to acquire competence in evience-base meicine. BMC Me Euc 2006;6:59. 50. Hays W. Statistics, fourth eition. Fort Worth: Harcourt Brace Jovanovich College Publishers, 1988. 51. Ajzen I, Fishbein M. Unerstaning attitues an preicting social behavior. Englewoo Cliffs, NJ: Prentice-Hall, 1980. 52. Casebeer L, Bennett N, Kristofco R, Carillo A, Centor R. Physician Internet meical information seeking an on-line continuing eucation use patterns. J Contin Euc Health Prof 2002;22(1):33-42. 53. Coomarasamy A, Khan KS. What is the evience that postgrauate teaching in evience-base meicine changes anything? A systematic review. BMJ 2004;329(7473):1017. 54. Davis D, Evans M, Jaa A, et al. The case for knowlege translation: shortening the journey from evience to effect. BMJ 2003;327:33-5. 55. Ebell MH, Shaughnessy A. Information mastery: integrating continuing meical eucation with the information nees of clinicians. J Contin Euc Health Prof 2003;23 suppl 1:S53-S62. 56. Jaa AR. Promoting partnerships: challenges for the Internet age. BMJ 1999;319:761-4. 57. Mangrulkar RS. Targeting an structuring information resource use: a path towar informe clinical ecisions. J Contin Euc Health Prof 2004;24 suppl 1:S13-21. 58. Shrauger J, Osberg TM. The relative accuracy of self-preiction an jugments by others in psychological assessment. Psychol Bull 1981;90(2):322-51.