Knowledge Based Public Health Work, Part 1

Transcription

1 Knowledge Based Public Health Work, Part 1 Handbook for Reviews of Published Reviews on Interventions in the Field of Public Health Anna Hedin and Carina Källestål

2 Foreword Sweden s National Institute of Public Health has recently been given a role as centre of competence concerning methods and strategies to foster public health. Reviews of current knowledge are an important part of this work. This paper will guide the reader through the process of compiling the results of previously published reviews on public health topics. The guide is mainly intended for people who work for the National Institute of Public Health at regional research centres or in the central administration and are to integrate current knowledge. It is the first in a series of reports about knowledge-based public health work. The next report will describe the step-by-step process of compiling critical reviews of primary research papers, and will be published in the spring of After that, we plan reports that describe and discuss what knowledge-based public health work comprises, and will present and compile models for systematic planning, implementation and follow-up within the public health sector. We will continuously collate, update and disseminate the scientific literature on the effectiveness of different types of interventions to promote health and prevent disease. This report should be seen as a draft document. The National Institute of Public Health and the authors welcome comments and questions from readers, and will incorporate changes into the next version of this handbook. Stockholm, vember 2002 Gunnar Ågren Director General Christer Hogstedt Head of Research 2

3 Content Summary...4 Chapter 1 Introduction...5 Systematic reviews...5 The aim of this handbook...6 Different work strategies within the public health sector...7 Decisions to make when planning a review of published reviews...8 Description of study designs...10 Chapter 2 Formulating the review question...13 Main components of a review question...13 Avoid unnecessary restriction...14 Timeframe...15 Chapter 3 Searching the literature...16 Creating a search strategy...16 Where to search for information...18 Reference management systems...21 Document your search...21 Selecting reviews...21 Evaluating grey literature...22 Synthesis...22 Timeframe...23 Chapter 4 Writing up and disseminating the results...24 The title and the authors...26 The summary...26 The main body of the text...26 Critical reading of the manuscript...28 Dissemination of results...29 Timeframe...29 References...30 Appendix 1. Quality assessment protocol for review of reviews...33 Appendix 2. Fundamental search terms for articles in English...42 Sample Review of Reviews How can we prevent young people from taking up smoking?...46 Summary...47 Introduction...48 Methods...49 Results...50 Discussion...53 References

4 Summary The first part of this handbook is a step by step guide on how to conduct a systematic review of already existing reviews on public health interventions. The first step is to define the population, the interventions and the outcomes to be included in the review question. This review question is the basis for the strategy to be used in the search for published articles in databases. Databases for existing reviews and for original and review articles are described. A method for selection and quality assessment is described. An appendix to the handbook gives a protocol for the quality assessment of reviews, e.g., to assess how systematic they really are. The results of the review are then summarised in the conclusion section. Finally, the handbook describes how the review findings can be reported and disseminated. The second part of the handbook provides a practical example using an existing systematic review entitled How do we prevent youth from starting smoking? This review used a systematic method of searching and judging literature on interventions used to prevent youth from starting smoking. The review reports that 47 review articles were identified. Eight of these articles provided the basis for the review. The interventions included school based, policy, mass media, and community interventions. The review concludes that the most effective way to prevent youth from starting smoking is by raising the price of cigarettes at the same time as implementing a variety of community interventions, all aimed at youth. 4

5 Chapter 1 Introduction Systematic reviews Modern research generates enormous amounts of knowledge. In this deluge it can be difficult to find time to fish out the information that is relevant to one s own field. Systematic reviews about an intervention should provide information about how effective it is, by identifying, evaluating and collating results from scientific publications and certain other sources. Reviews are important tools when decision-makers and administrators plan programmes and allocate resources. One usually distinguishes between reviews of the literature, research overviews, and systematic reviews. Reviews of the literature are non-systematic compilations of the published literature; they are comprised of a subjective selection of articles, and present equally subjective evaluations. Literature reviews are often compiled by experienced researchers in the field: this is both a strength and a weakness. The strength lies in the authors thorough knowledge of the research field. The weakness is that the authors sometimes have preconceived notions and may overestimate the value of some articles. Systematic reviews are systematic compilations of the scientific literature. They include both articles published in scientific journals and the so-called grey literature. Grey literature includes reports and documents that have not been published in peer reviewed journals or scientific publications. This grey zone includes publications directed toward a broader audience, for instance conference reports. Systematic reviews are compiled in a structured way: this means that material is collected according to a defined scheme in order to avoid bias and prevent distortion of conclusions, and also that the process is described clearly, so it can be traced, evaluated and repeated step by step. Research overviews differ from systematic reviews in that they include only published, peerreviewed scientific papers (no grey literature) within a specific research field. Most of the methods used to compile systematic reviews originate from the Cochrane Collaboration, and from the evidence-based medicine that has sprung forth as a consequence of their work. The Cochrane Collaboration, which started in 1993, is an organisation of scientists and practitioners who have taken upon themselves the task of compiling systematic state-of-the-art reviews, mainly of medical technology and therapy. The organisation was founded on the initiative of the English epidemiologist Archie Cochrane with the aim of compiling and updating systematic reviews of interventions within the health care sector, and to make the information available to physicians, policy-makers and patients. The Cochrane Collaboration is comprised of several working groups that develop methods for systematic reviews in different health fields. One of these working groups, Cochrane Health Promotion and Public Health Field, has been focusing especially on methods for systematic review of health promotion strategies ( The Swedish Council on Technology Assessment in Health Care (SBU) is responsible for compiling the scientific knowledge needed both by the government and by the health authorities to evaluate health care. The working method used by SBU is founded on the systematic review process developed within the Cochrane Collaboration. SBU has published a number of reports since its inception in Prominent among these are the reports 5

6 Preventing cardiovascular disease through population-based programs (1997) and Methods for smoking cessation (1998). The aim of this handbook The first part of this handbook describes a method for collating already published reviews on public health work, i.e., systematic reviews of articles published in scientific journals and books and grey literature. These reviews are meant to address public health work in general, i.e., efforts directed towards entire populations and preferably co-ordinated outside the health care sector, and are meant to be used by public health workers and policy-makers in research and in practice. The handbook is intended primarily as a guide for those who are compiling knowledge for the National Institute of Public Health, both those at regional research centres and central administrators. In working with this handbook we have drawn on the expertise of international groups who work with systematic reviews. Agreement with other handbooks can be seen from the reference list, but some descriptions of methods deserve special mention. The most important are: Center for Disease Control s Guide to Community Preventive Services (Zaza et al. 2000) Methodological Problems in Constructing the Evidence Base in Public Health (Kelly et al. 2002) Undertaking Systematic Reviews of Research on Effectiveness. CRD s guidance for those carrying out or commissioning reviews (Kahn et al. 2001) Cochrane s Reviewers Handbook (Clarke & Oxman 2001) Draft Guidelines, The Campbell Collaboration Center. ( An assessment of the methods and concepts used to synthesize the evidence of effectiveness in Health Promotion: A review of 17 initiatives. (Jackson et al. 2002) Medical Evaluation [in Swedish]. (SBU 2001) The first part of the handbook is arranged in the following way. In this chapter we give a brief introduction to various methods used in public health work and describe aspects of this work. We conclude the chapter with a summary of the most commonly used study designs. In chapter 2 we describe the process of formulating the problem to be addressed in the review. Chapter 3 will guide the reader through the next step in the process, namely how to build a search strategy, how and where to search for previously published reviews, how to assess the quality of these articles, and how to summarise the results presented there. 6

7 Finally, in chapter 4, we suggest ways to organise and synthesise the report. We also discuss ways in which the report can be disseminated. In the appendices we first present a quality-assessment protocol for use in evaluating published reviews (Appendix 1). Then we list search terms used frequently in various public health fields (Appendix 2). The final part of the handbook provides an example of how the first part might be implemented. The authors chose to compile previously published reviews on prevention of tobacco use among young people. Different work strategies within the public health sector Public health work is a general term for actions aimed at improving the health of groups of people, rather than individuals, as one does in clinical health care. Public health work uses an interdisciplinary approach and is based on scientific theories from within several different scientific paradigms. This makes it difficult to define the field, and also leads to differences of opinion concerning the methods used. One obvious difference lies between the two fundamental perspectives on public health: pathogenesis and salutogenesis. The pathogenic perspective predominates clinical health care. In brief, it is focused on disease: how to diagnose it and how to prevent it. The salutogenic perspective is instead focused on health and how to promote it. Disease prevention strategies are founded in the pathogenic perspective, even if they are aimed at primary prevention, i.e., actions taken before disease sets in. Strategies for disease prevention often include identification of risk factors for a disease and attempts to reduce or eliminate risk factors in vulnerable groups. These interventions often involve a specific method and the outcome is often measured in terms of reduced prevalence of the disease or reduced prevalence of factors that contribute to disease development. Health promotion strategies spring from a salutogenic values stance (Judd et al. 2001), where the aim is to empower groups of people or entire societies; this in turn serves to strengthen people s ability to stay healthy. Health promotion work frequently employs combinations of methods targeting different sectors of society. It is evaluated in terms of many different outcomes including not only the ultimate goals of improved health, equity and quality of life, but also the intermediate goals of the intervention itself, such as knowledge, attitudes and motivation. In health promotion work, both the outcome and the process are frequently subject to evaluation. Many different activities based on different approaches can coexist within public health work. Below we present some aspects of public health work that we consider important for identification of the review questions and the focus of a planned systematic review. The rationale for this subdivision is that it simplifies the procedure of evaluating literature on public health interventions. This subdivision into different aspects is not intended as a way of defining public health work in any other context. 7

8 Decisions to make when planning a review of published reviews As mentioned earlier, public health work can be categorised as disease prevention or health promotion; it can entail one intervention, or several; it can aim for one outcome or more. In addition, the main focus of interest when evaluating the intervention can be either the outcome itself or the road toward that goal, i.e., the process. These four main dimensions can form a framework on which to classify questions about work undertaken in the field of public health. 1. The first question to ask is whether the review concerns a strategy for disease prevention or one for health promotion. Disease prevention strategies aim to reduce disease, reduce determinants of disease, or reduce risk factors for disease. Disease prevention strategies are frequently defined as actions that target specific risk groups and risk factors that are associated with different types of risky behaviour. Example: a disease prevention strategy to reduce the risk of lung cancer and heart attack might aim to reduce smoking, for instance by having company nurses give information about the detrimental effects of smoking. The outcome would be described in terms of lower prevalence of disease or reduced consumption of tobacco. The aim of health promotion strategies is often broader than that of disease prevention strategies. According to the Ottawa Charter (1986), health promotion is the process of enabling people to increase control over, and to improve, their health. There are several different viewpoints about what is to be considered a successful result where health promotion is concerned. Planners, economists and policy-makers need to know how costeffective a programme is, whereas practitioners in the field need to know what effects a programme has in real life. The people targeted by the intervention want to feel included, whereas researchers are probably mainly interested in learning what a programme shows about the relation between cause and effect (Nutbeam 2000). Health promotion activities often give results at several different levels. One outcome model, created by Don Nutbeam (2000), is presented in Table 1. In this model, health outcomes and social outcomes represent different end-points for health promotion. They are often described in terms of quality of life and equity, or mortality, morbidity and disability. Intermediate health outcomes include determinants of the ultimate outcomes. Personal habits such as physical activity are part of a healthy life style, whereas a healthy environment is made up of physical, social and economic variables that can influence health directly in addition to contributing to a healthy life style. Access to adequate health care is also a determinant of health. The health promotion outcomes are the individual, social and structural factors that can be manipulated to have an effect on determinants of health. These outcomes are the most immediate effects of health promotion programmes, e.g., knowledge and attitudes. Social action and influence describe the results of efforts to empower groups within society to take charge of their own health. Health policy and organisational practice are the results of efforts to change structural barriers to good health, for instance through legislation, regulation and ordinances. 8

9 Table 1. An outcome model for health promotion Health promotion actions Health promotion outcomes (intervention impact measures) Intermediate health outcomes (modifiable determinants of health) Health and social outcomes Education Health literacy Healthy lifestyles Social outcomes Examples include: patient education, school education, media communication Measures include: health-related knowledge, attitudes, motivation, personal skills, self-efficacy Measures include: tobacco use, food choices, physical activity, alcohol and illicit drug use Measures include: quality of life, independence, equity Social mobilisation Examples include: community development, group facilitation, technical advice Social action and influence Measures include: community participation, community empowerment, social norms, public opinion Effective health service Measure include: provision of preventive services, access to and appropriateness of health services Health outcomes Measures include: reduced mortality disability, morbidity Advocacy Examples include: lobbying, political organisation and activism Healthy public policy and organisational practice Measures include: policy statements, legislation, regulation, resource allocation, organisational practices Healthy environments Measures include: safe physical environment, supportive economic and social conditions good food supply, restricted access to tobacco and alcohol Example: a health promotion intervention can foster a healthy lifestyle by increasing young peoples insight about the strategies tobacco companies use to increase tobacco use among young people. One health promotion intervention within this area might be to involve various forces in the local community, such as sports clubs, and getting them to support smoke-free leisure time or by lobbying for regulations against smoking. 9

10 2. The second question is about the number of interventions. Both disease prevention and health promotion programmes can consist of a single intervention, e.g., giving information in schools about smoking, or of several interventions all aimed at reducing tobacco use. Usually a review will be focused on a specific health effect that can be achieved through several types of interventions. Each intervention must usually be evaluated separately, though similar programmes using the same types of interventions can be compared. If the number of interventions is made clear, the review question can be formulated more precisely and the literature search can be divided up into separate parts. 3. The third question concerns the number of outcomes. As is clear from the description, health promotion interventions are often measured in terms of several outcomes, but a scientific primary publication may focus on evaluating a single outcome. Disease prevention activities generally have a single outcome measure, but here too, several outcomes are conceivable. Outcome measures can also be intermediate for instance insight into the importance of diet for good health, improved eating habits, cholesterol levels in blood or ultimate outcomes, such as cardiovascular disease. If the work with the review is to go smoothly, it is important to define the types and number of outcomes, because it is difficult to evaluate multiple outcomes in the same way. A systematic review can choose to evaluate several outcomes, but they must then be specified. At this stage, the outcome measure(s) should be clarified and established. You should also consider carefully whether the number of outcomes should be reduced, or if material should be written up as more than one review, each covering a particular outcome. Example: outcomes can include morbidity, percentage of smokers, cotinine levels, attitudes towards smoking, knowledge of the effects of smoking, changes of policy such as restrictions on sales or advertising, shifts in public opinion, smoke-free zones, self-efficacy. 4. The fourth question concerns whether the review will analyse the outcome or the process. Both are of importance for policy-makers and practitioners in the public health field. Expectations concerning results are important when an intervention is being planned, but insight into how the results are achieved is equally vital. Outcome analysis measures the results of an intervention quantitatively, e.g., in terms of units of cholesterol, units of blood pressure, number of correct answers on a test, number of participants in an activity. Process analysis studies how the results were achieved: who participated, what attitude did the various participants have towards the intervention, to what extent were they involved? For the latter analysis, qualitative methods such as focus groups and in-depth interviews are often used. Description of study designs Most systematic reviews follow a particular study design. Below we briefly present some of the study designs that are used most frequently. For further detail, we recommend the following literature: 10

11 Rothman, K.J. & Greenland, S. (1998). Modern Epidemiology. Philadelphia: Lippincott- Raven Corp. Green, L.W. & Lewis, F. (1986). Measurement and Evaluation in Health Education and Health Promotion. Mayfield Publishing Company, California. Neal, J. & Liebert, R.M. (1986). Science and Behavior. An Introduction of Methods of Research. Prentice/Hall International. New Jersey. Starrin, B. (1997). Along the Path of Discovery. Studentlitteratur, Sweden. Randomised controlled trials (RCT) are studies in which the investigators compare the effects of an intervention in a study group and a control group. Allocation to study and control group is random, and the study should be double-blind, that is, neither the participants nor the investigators should know who belongs to which group. Moreover, results concerning all participants should assessed, regardless of whether they have participated for the entire duration of the study or not. If the allocation to study and control group is done in a nonrandom fashion, or in a way that is not truly random (for instance if people born on a certain day are assigned to the study group, or if every second patient in a list is assigned to the study group) the study is called a controlled clinical trial (CCT). In a cohort study, a group of people are selected (e.g., on the basis of age or occupation) and the exposure of interest is defined. Then the cohort is followed over time and the outcome is measured at the end of the study. Comparisons of the outcome for exposed and non-exposed members of the cohort can be expressed as relative risks (RR). A cohort study can be prospective (where the cohort is followed into the future from the day the exposure is measured until the end of the study), retrospective (where the exposure is estimated afterwards, when the outcome is known), or ambispective (in which case the study has both prospective and retrospective elements). In before-and-after studies, the same participants are examined before and after an intervention. Here it is often difficult to ascertain whether any differences observed indeed result from the intervention. However, in the absence of RCT or cohort studies, before-andafter studies can provide clues about an effect, for instance when changes in health policy are to be evaluated. A related study design compares two geographical regions. The time-series study (TSS) is another design, where a phenomenon is measured sequentially over time before and after an intervention. This kind of study can give information that is more reliable than that provided by before-and-after studies. Time-series analyses can identify trends and cycles. The correlation between observations within a time-series is a statistically important attribute. When a time-series design is to be used, a stable baseline value or trend must be established before the intervention. Qualitative research is a term that can be used to describe several different study designs (just as all the study designs presented above can be called quantitative). Qualitative research provides information on and insight into how people perceive things what meaning they attribute to various phenomena, and how they communicate their perceptions of various events and of the world around them (Hallberg 2002). The reason these studies are called qualitative is that the results are seldom expressed numerically; nor are they evaluated statistically. Instead, the investigators gather information through focus groups or in-depth interviews, or by participating as observers. 11

12 There are many different qualitative methods for systematic accumulation and compilation of data. They can be divided into three main categories: language-oriented methods, theorygenerating methods, and descriptive methods (Hallberg 2002). An example is Grounded Theory, a theory-generating method that is being used more and more within public health research (e.g., Hallberg & Jansson 1996, Dellve et al. 2001, Thunberg et al. 2001). In qualitative research, the number of informants (i.e., the study population) is often small seldom more than thirty. Data collection and data analysis are often carried out simultaneously. The aim is to give as broad and as varied a picture as possible of the phenomenon under study. For this reason, the investigators try to get as heterogeneous a group of informants as possible. When the investigator finds that no more new information is emerging, the material is considered saturated and data collection ceases. Qualitative and quantitative methods are excellent complements to each other in public health research. They should not be seen as opposites. Hallberg (2002) states that although qualitative and quantitative research use different strategies to find answers to research questions, the investigator s background and perspective will influence which questions are posed, and how the data are collected and analysed, regardless of whether qualitative or quantitative methods are used. Critical evaluation of a study s reliability, accuracy and applicability is necessary in both qualitative and quantitative research. Yet another way to categorise different types of studies is to classify them as having an experimental or non-experimental design. Only RCTs qualify as experimental studies (see above). n-experimental studies include quasi-experimental designs, such as clinical controlled trials, CCT, that do not use truly random sampling. Finally, there is a category called observational studies. These include cohort studies, before-and-after studies and time-series studies. Another type of study design not very frequently used for evaluation of public health interventions is case-referent studies. 12

13 Chapter 2 Formulating the review question To formulate a clearly defined question, you must have precise answers to these questions about what to include: 1. What people (what participants) and from what setting? 2. What interventions? 3. What outcomes? 4. What study designs? Main components of a review question After going through the four questions described in Chapter 1 whether the systematic review will deal with a strategy for disease prevention or one for health promotion, how many interventions (or methods) it will include, how many outcomes it will evaluate, and what type of analysis it will make (process or outcome) you should have a fairly clear picture of the review question, but further refinement is necessary. It is important to judiciously think through the questions posed in the review; clearly framed questions will save much effort. For instance, it is much easier to locate relevant literature if the question is defined precisely. Readers also have use for a clear statement of the research questions: it will help them judge whether they are interested in the review. The following components should be specified in the review question. 1. What people (what participants) are to be included and from what setting? 2. What interventions are to be included? 3. What outcomes are to be included? 4. What study designs are to be included? 1. The first thing to do is decide on a target population and a setting (e.g., 13- to 19-year-olds in schools or middle-aged men in workplaces where most of the employees are women). A question that defines the first main component: Which interventions are effective in preventing children and teenagers in schools from taking up smoking? 2. Which types of interventions to include need only be specified if the main aim of the review is to examine the effects of a specific intervention, e.g., which types of educational initiatives work in a certain setting. If the main focus of the review question is what works (regardless of type of intervention), obviously this should be left open and no interventions should be specified beforehand. 13

14 A question that defines the second main component: Which interventions, alone or in combination, are effective in increasing knowledge about the detrimental effects of smoking? 3. Outcome has also been discussed above. In addition to defining outcome measures, you should decide how to handle combinations of outcomes. Moreover, you should decide whether unintended effects should be included. A question that defines the third main component: Which interventions are effective in increasing motivation not to smoke and fostering positive attitudes towards nonsmoking? 4. In deciding which study designs to include you must consider the review question and the outcome measures. Perhaps the number of study designs needs to be limited? If several different study designs are included, they must be specified and it should be made clear precisely why they have been included. This discussion emphasises the need of competence and consensus concerning the scientific stance among the authors. A question that defines the fourth main component: What findings emerge from before-and-after studies of changes in policy, for example restriction of access to cigarettes or designation of more smoke-free areas? Avoid unnecessary restriction When making a review of previously published reviews, you should not be overly restrictive in defining the review question because important reviews may be missed in the literature search. For instance, if a narrow age group is defined, some reviews may be excluded. To capture all relevant reviews, the question should be broad and general. Once all the components in a review question have been defined, they can be used to formulate a search strategy. When the search strategy is tested, the review question may have to be modified. This is to be expected, but major post-hoc changes in the review question should be avoided. Otherwise there is a risk that interesting articles will give a completely new perspective new, but not pertinent to the problem being addressed. 14

15 Timeframe It is important to invest time in formulating a review question. It is obviously not possible to say how much time you will need to spend. Everyone involved in compiling the review should discuss the formulation of the review question. At this stage it is also worthwhile to set up a reference group for the review. The group should include a panel of experts in the field and representatives of the target group for the review, i.e., people working in the local public health sector. The task of the reference group is to help identify relevant literature especially grey literature and to evaluate and suggest changes in the final review of published reviews. 15

16 Chapter 3 Searching the literature 1. Create a search strategy. Steps in the process of searching the literature 2. Search databases specially compiled for previously published reviews (not primary articles). 3. Search databases comprised mostly of primary articles, but that also include reviews. 4. Search for more reviews in reference lists, in the grey literature, on the Internet, and in registers of ongoing research. 5. At this point, you will have a long list of references as well as abstracts or summaries of reviews. 6. On the basis of your review question, go through the long list and select the reviews that appear most relevant. 7. The reviews you have selected constitute the short list. Order complete copies of all these articles from the libraries. 8. The articles will then be read and evaluated for quality according to a standardised protocol (see Appendix 1). At this stage you may find that some articles do not fit within the framework of the review question, or are not of sufficiently good quality. Such articles are eliminated, but are noted in a table, and the reason for their exclusion is stated. Creating a search strategy You must have a strategy for how you will search for pertinent research within a field a search strategy. The strategy should specify which databases and other sources will be searched, and which search terms and search strings (i.e., search phrases) will be used. In formulating a search strategy you set out from the key components of your review question, e.g., target group, type of intervention, study design, outcome measures (see Chapter 2). The results of the first search (the pilot search) can be useful in helping you improve your search strategy. It is extremely important to make a note of the search terms used, in what combinations they are used, and which databases are searched. Without such records you will soon begin to mix or double your searches. The search strategy you finally decide on will be described in the review. In general, the strategy and the search terms will be influenced by how you have formulated your review question. It is important to define the question (see Chapter 2), as this facilitates the formulation of a search strategy. There are pitfalls to searching only for outcomes. The 16

17 outcome may have been described in many different ways, or it may not be mentioned in the abstract or among the key words (where the search motor looks for them), even if it has been used and measured in the study. Because of this, it is preferable not to search for outcomes. However, this can sometimes be a useful way to delimit a field if a search retrieves an unmanageable number of articles. In the fields of public health and intervention one frequently uses terms such as outcome and process assessment, program effectiveness and program evaluation. Appendix 2 (from Kelly et al. 2002) describes and categorises a number of basic search terms that can be of use when you create a search strategy. When searching specifically for published review articles, you should always include the term review in your search strategy; otherwise you run the risk of retrieving all the relevant original articles as well. It can also be useful to include the terms systematic review and meta analysis. Searching only for a topic is another strategy. This strategy focuses on the subject of interest in the review question. This can, for instance, be a specific type of intervention (are schoolbased tobacco prevention programmes effective?), a specific health topic (what works best for promotion of safe sex?), or a specific outcome (what interventions reduce the risk of bicycle accidents?). For some review questions, a topic search is more likely to retrieve all the relevant reviews. Another advantage is that when you are compiling a review of interventions in public health fields, it can be of importance to locate not only studies that show the effectiveness of an intervention, but also studies that shed light on whether the intervention is suitable for a specific target group, whether special circumstances have influenced the intervention, what factors might have contributed if an intervention did not have the expected effects, what difficulties must be overcome if the study is to be generalised to other populations, etc. If you restrict your search to studies that measure effectiveness, studies that provide other types of relevant information may be missed. The disadvantage is that a search focused only on a topic will probably have very low sensitivity: in other words, only a few of the studies retrieved will be considered relevant enough to include in the review. We recommend a pilot search to give you some idea of how many reviews there are within a certain field. If, for instance, you have decided to include literature in languages other than English and adjust your search strategy accordingly, a pilot search can tell you how much is available in other languages and suggest how to deal with translations. The pilot search should preferably be done in the larger databases, i.e., the databases where you are most likely to find the pertinent reviews. Table 2 lists databases that only include already completed reviews. It will usually be to your advantage to enlist the assistance of a competent librarian. It is also important to restrict your search. If it is not feasible to have articles translated, you should restrict your search to languages you can handle. At the very least, you should search for articles in Swedish, rwegian, Danish and English. You may also want to restrict the time period: how far back do you want to go for information? Some databases (Medline for 17

18 instance) contain studies from the 1960s onward. Psychinfo/LIT did not get started until You should also set a closing date after which you do not search for information. Because the databases are constantly being updated, there is always a chance that new relevant information may come to light. We recommend that you stop looking for more information when you start working with the question of which studies to include in the review. Where to search for information When the search strategy has been defined, you should first search through databases that include reviews, and thereafter, other relevant databases. The Health Development Agency in England provides the list of databases for reviews shown in Table 2 (Kelly et al. 2002). This list has been updated and supplemented with databases that are of use for studies in Sweden. When searching for review articles, it is important not to use too narrow a search strategy. It is often sufficient to search on one subject term each for the intervention and the population of interest. If this does not retrieve enough articles, you can broaden your search to include reviews of closely related fields. Table 2. List of systematic and other databases containing review articles. Reviews of reviews Cochrane Library: Cochrane Database of Systematic Reviews < DARE < Center for Disease Control CDC Community Guide < NICE web page (published appraisals) < HSTAT < Evidence Base 2000, Health Development Agency, London < Evidence for Policy and Practice Information Co-ordinating Centre (EPPI-Centre), University of London < The Combined Health Information Database (CHID), National Institute of Health and Centers for Disease Control and Prevention < SBU The Swedish Council on Technology Assessment in Health Care < Table 3. Ongoing research Databases including reviews in progress CCTR (Cochrane Library) < Controlled Clinical Trials < When you have exhausted these databases, you should go on to other scientific databases. 18

19 Below, we briefly describe the databases that are most useful within the fields of disease prevention and health promotion (Medin & Alexandersson 2000): PsycINFO (access fee). ( This database covers mainly the psychological and behavioural sciences. Many databases overlap to some extent, and this one also contains articles in the fields of medicine, pharmacology and linguistics, as well as some in the nursing sciences. PsycINFO covers over 1500 journals written in many different languages, and is updated monthly. Books and book chapters are also covered, but only those in English. The database was established in 1987 and is produced by APA (The American Psychological Association). Medline/PubMed (free of charge). ( The Medline database covers mainly biomedical literature. Over 4000 of the world s leading journals are included. Medline is updated once a week and can be searched from 1966 onwards. The National Library of Medicine (NLM) in Washington, D.C., USA is responsible for its publication. Social Sciences Citation Index (SSCI, access fee). ( This interdisciplinary database includes articles published in the fields of social and behavioural science since It covers over 3600 journals. It is produced by the Institute for Scientific Information. Only references are available (not full-text articles). At the same time, it is a citation index, i.e., it provides information about who has published articles about a particular topic and which other authors have cited those articles. ERIC (Educational Resources Information Center, free of charge). ( This database mainly covers educational material in the form of scientific articles, books and conference reports. ERIC was established in 1966 and now contains over a million references. The database in updated once a month and is produced by Cambridge Scientific Abstracts. Libris (free of charge). ( Libris focuses mainly on Swedish research and specialist libraries in Sweden. Libris contains over four million references. References to books and journals are updated daily; references to individual articles are updated weekly. Libris is produced by the National Library of Sweden in collaboration with seventy other libraries. Regardless of which database you are searching, always keep the following in mind: read the description of what the database contains and read the help texts find out how the subject matter is described interdisciplinary, specialist, popular science find out what kind of database it is full-text, reference, facts find out what kinds of documents it covers journal articles, conference reports, reports, books time covered, update frequency languages covered When you have searched through all the databases you should proceed to some complementary searches: 19

20 Reference lists. A check through the reference lists of pivotal review articles can turn up additional important references that may have been missed in your search. Handsearching. There is always a risk that a search will miss important publications, e.g., because the search strategy is not optimal or because the database is not fully indexed. If possible, you should try also to look through the journals that are most relevant to your review question in search of articles that were overlooked in the database search and the reference lists. This kind of manual search can also turn up the latest news, reports that have not yet been added to the databases or been cited in other articles. Search the grey literature. Important results are sometimes published in various types of reports that are not documented in any database. Grey literature also includes, e.g., conference proceedings, term papers, case studies, theses and dissertations. Complete identification of all the relevant grey literature in a field is virtually impossible, but some of this grey literature is indexed in databases such as Spriline ( and SIGLE (System for Information on Grey Literature). In this context, your reference group plays a critical role. The experts in the reference group are often well acquainted with the literature. You can also search for grey literature through various research agencies, professional networks and organisations, etc. A search for pertinent dissertations (check the database Dissertation Abstracts ) and term papers can also be fruitful. Conference proceedings often provide valuable information about completed and ongoing research in a field. Here again there are useful databases such as Index to Scientific and Technical Proceedings and Conference Papers Index. Some of the larger libraries have catalogues of conference reports. However, it is important to keep in mind that the summaries presented at conferences do not always agree with the final report on a study. It may therefore become necessary to contact the author directly and ask for a copy of the final report before you decide whether or not to include the information in the review. Registers of ongoing research. There are registers that list ongoing research (not all research gets published). The web site Forskning.se ( provides links to the four major Swedish research councils. Their web sites, in turn, have information on ongoing research projects. The project descriptions are sometimes linked to information from universities, colleges and other research institutions, but the completeness of that information if it is available at all depends on how much the scientists involved are willing to post on the database. The National Research Register ( is an international register of ongoing research. The research councils also maintain registers of all the research they support. On the CRD home page (Centre for Research and Dissemination) ( there are lists of up-to-date research indexes and registers. You can also find research registers on the Internet (for instance metaregister of Controlled Trials, maintained by Current Controlled Trials ). As mentioned above, you should first contact your reference group: they probably know which research registers you should look through. Searching the Internet. An Internet search can provide useful information about ongoing and completed research; in particular, this can give information about studies that have not yet been formally published. But Internet searching can sometimes be a discouraging task. Many of the major search engines are unable to handle a sophisticated multilevel search; one search can give thousands of hits, each of which must be examined. Yet it is possible to search the Internet in a relatively systematic way by using meta-search engines such as Copernic 20

21 and Dogpile. Another useful search engine is called Scirus ( Scirus is a search engine specialised for scientific information. Scirus also has a filter that helps filter out non-scientific information. Reference management systems Reference management programs are exceedingly useful for keeping track of references and ensuring accurate and easy insertion into documents in preparation. With a reference management program, you can download relevant search results electronically, directly from a database to your own reference files and save them either on your hard drive or on a disk. References located manually must be entered into the reference management system by hand. In this situation it is important to ensure that all the relevant bibliographic information is included. Several reference management programs are available: Endte, Reference Manager and GetARef are some of the most common. Document your search As a general rule, the entire process from formulation of the review question to the writing-up of the paper must be clear to the reader. For the reader to be able to evaluate the final product it is necessary for you to describe clearly and in some detail how the search was done and how the studies were selected. Therefore the search process must be documented as you go along, and you should also present the rationale for any changes or additions that are made in the search and selection procedures. Important points on literature searching: 1. It is important that the search for previously published reviews is extensive and systematic, otherwise you risk drawing incorrect conclusions. 2. When you develop a search strategy you should enlist the help of a librarian with experience of electronic searching. 3. For a search to be adequate, it must include not only database searches, but also manual searches of relevant journals, and input from specialists in the field. 4. The search procedure must be documented as you go and search results must be saved. 5. A reference management system will facilitate the work and save time. Selecting reviews When you have completed your literature search, it is time to select which reviews you will read in their entirety. Database searches generally result in long lists of references and 21

22 summaries/abstracts (the long list). This list is comprised of all the references that were retrieved in your searches. It is important to go through these and compare the results of the various searches, because the same reference will often be found in two different databases and the risk of ordering duplicates is substantial. Two or more readers should go through the summaries and decide on the basis of the review question which articles should be included (the short list). If there is any doubt, or if the readers disagree about whether an article should be included, they should discuss it and try to reach a consensus agreement. The readers then divide up the short list and order full-text copies of the articles. When the full article is being read and evaluated, the readers should follow a protocol (Appendix 1). Each reference is read by at least two members of the group, and if they disagree in their evaluation of the article, it should be discussed until consensus is reached. Each review is assigned to one of the three categories listed below after inclusion and exclusion criteria have been defined. Sometimes it is necessary to read the articles in two stages: first to decide whether or not the article should be included, and then once more, at which time the protocol is filled in. 1. Include in its entirety 2. Include parts, or use as background for the review 3. Exclude in which case the rationale is noted and presented in a table as an appendix. All the articles that end up in categories 1 and 2 form together the material on which the review will be based. The next step is to evaluate the reviews that have been selected (categories 1 and 2) to determine whether they can give an exhaustive answer to the review question that has been posed. If not, the group may decide to do a completely new research overview; alternatively, it may be sufficient to supplement the review with more recent literature. Evaluating grey literature Grey literature in the form of reports from studies that are not published in scientific journals can be evaluated in the same way as published reviews of the literature. Other reports should be assessed by the group as a whole, and a decision made on whether the results and findings presented in this type of literature should be included in the review. Synthesis The aim of the synthesis is to summarise data and knowledge gleaned from the reviews. In many cases it is not possible to summarise the results quantitatively in such a way that an average can be calculated (meta analysis) because the material included is so disparate in terms of method or number of studies evaluated. If the final overview is based on a small number of published reviews, it may be possible to synthesise them as a narrative that describes each of the reviews included. Tabular presentation of the effects examined, the number of studies included, the time-span, the method, etc., will make it easier for the reader to comprehend the synthesis. Such a table is also an excellent starting point for creating a synthesis, as it gives a summary of all the results at one glance. The tables should be presented according to type of intervention, so that each intervention has its own table. When these tables are finalised you should go through and 22