Systematic Reviews in JNEB Introduction 3 Associate Editors and 3 Board of Editors members Panel then small workshop tables Evaluation, please Thank you to Elsevier for refreshments! KCN
Systematic Reviews in JNEB What is a systematic review? The purposes of conducting systematic review KCN
Systematic Reviews Learning objectives of the workshop To be able to write a research question for systematic reviews To be able to choose a process To be able to extract data To be able to summarize and conclude To be able to publish in JNEB! KCN
Susan Johnson, PhD Professor of Pediatrics Associate Professor of Community and Behavioral Health, Colorado School of Public Health Director, The Children's Eating Laboratory Associate Editor, Journal of Nutrition Education and Behavior
Systematic Reviews Common Procedures of Systematic Review: 1. Establish a team with appropriate expertise and experience 2. Developing a review question 3. Search, screen, select studies 4. Data extraction & quality assessment of studies 5. Synthesize the evidence 6. Develop conclusion statements & grade studies 7. Develop research recommendations Susan Johnson
1. The Systematic Review Team Selecting the members At least 2 researchers to extract the data If a large body of work, more often divide the work among a number of team members Developing a management strategy Researchers should work independently e.g. one extracts the data one reviews for completeness and accuracy Process for resolution of discrepancies, preferably with a 3 rd party Susan Johnson
Seung-Yeon Lee, PhD Assistant Professor, University of Cincinnati Research interests include nutrition education across the lifespan JNEB Board of Editor s member
2. Developing a review question Review questions should: Address outcomes that are meaningful to people. making decisions about nutrition and health practice. Address issues that are important to consumers, health professionals and policy makers. oto resolve conflicting evidence oto address questions where clinical practice is uncertain oto explore variations in practice oto confirm the appropriateness of current practice oto highlight a need for future research Seung-Yeon Lee
Developing a review question An explicit review question can be composed of PICO: Population: Population(s) of interest, and any relevant subpopulations Intervention: An intervention (an independent variable) or an exposure of interest Comparison: A main alternative to compare with the intervention or exposure Outcomes: A public health or nutrition related outcome of interest (a dependent variable or an outcome) Seung-Yeon Lee
Developing a review question Examples: What is the relationship between whole grain intake and the development of type 2 diabetes in adults aged 18 years or older? Population Intervention Comparison Outcome Adults Whole grain consumption No whole grain consumption Type 2 diabetes What is the effect of nutrition education with parental involvement compared to no parental involvement on children s dietary intake-related behavior? Population Intervention Comparison Outcome Children (0 to 18 years of age) Nutrition education with parental involvement Nutrition education with non-parental involvement Children s dietary intake- behavior
Developing a review question Additional examples which do not follow the PICO format: How has qualitative research advanced understanding of the ways that people interpret healthy eating? How are postsecondary online nutrition courses marketed, managed, designed, and delivered? How is effectiveness measured in studies of online nutrition education credit courses? What theoretical models are used to ensure that postsecondary students are satisfied and have high performance when learning nutrition online? What are the psychometric properties of evaluation instruments that measure mediators of dietary behaviors in school-aged children?
Noel Chavez, PhD, RD, LDN Associate Professor, Community Health Sciences, University of Illinois at Chicago Research interests include: cultural differences in health behaviors, attitudes, health status, and services use, nutritional epidemiology JNEB Board of Editors member
This slide may be deleted if content covered earlier 3. Frameworks - Guidelines Various frameworks available from multiple sources Sample guidelines: PRISMA, Institute of Medicine, Cochrane Reviews-MECIR, USDA Nutrition Evidence Library, CDC Community Preventive Services Many frameworks focused on reporting standards for research/intervention articles can aid in systematic reviews-consort, TREND JNEB requires a framework or set of guidelines to be used, but doesn t specify a particular one
Sample PRISMA Elements to Extract and Report Title Abstract Introduction Methods Results Discussion Funding Identify as systematic review, meta-analysis Structured summary Rationale, objectives of review Protocol, study eligibility, information sources, search strategy, study selection, data collection process, data items, individual study risk of bias, summary measures, results synthesis, risk of bias across studies, additional analyses Study selection, study characteristics, within study risk of bias, individual study results, results synthesis, risk of bias across studies, additional analysis Evidence summary, limitations, conclusions Review funding sources
Systematic Review Methods Develop conceptual framework analytic chain linking intervention to outcome(s) Develop a protocol for the review-the specific methods you will use Specify interventions, study design type(s), population, outcomes, comparison groups, time points Identify criteria to include/exclude studies to be reviewed Search for literature using criteria; hand searches Retrieve and screen article abstracts Read full text of each included article Code the data using a standardized form-form used depends on research question, study design type Describe how reviewer disagreement will be resolved
Things to Remember Document and track searches, keywords used, flow of articles included and excluded Keep records of searches, hand searches, keyword combinations used, data bases/search engines used Be systematic in your approach-rely on your protocol Standardize the quality reviews Quantitative vs. qualitative Study design types: RCTs vs. Observational studies Use/develop a standardized form for coding
Resources Guidelines-Frameworks PRISMA http://www.prisma-statement.org/statement.htm IOM http://www.iom.edu/reports/2011/finding-what-works-in- Health-Care-Standards-for-systematic-Reviews.aspx MECIR (Cochrane) http://www.editorial-unit.cochrane.org/mecir Health Evidence Canada www.healthevidence.org/ USDA Nutrition Evidence Library http://www.cnpp.usda.gov/nel.htm CDC Community Preventive Services Task Force http://www.thecommunityguide.org/about/methods.html
Susan Johnson, PhD Professor of Pediatrics Associate Professor of Community and Behavioral Health, Colorado School of Public Health Director, The Children's Eating Laboratory Associate Editor, Journal of Nutrition Education and Behavior
4. Extracting Data from Paper Depends on the research question, types of data that are available, and whether metaanalysis is appropriate Link publications from the same study Susan Johnson
General Information/ Use a standard data extraction form Researcher(s) performing data extraction Date of extraction ID study features: Unique record #, Author, Article title, Citation, Publication type (journal article, abstract), Country, Funding Source Study Characteristics Aim/objectives of the study Study design Study inclusion and exclusion criteria Recruitment procedures (e.g., randomization, Blinding details) Participant Characteristics Age, Gender Race/ethnicity, SES, Disease characteristics Comorbidities # Participants Intervention Comparison group (# eligible, enrolled, randomized, etc.) Intervention & Setting Setting where intervention delivered Description of intervention(s) and control(s) (dose, duration, provider, how intervention developed, theoretical basis) Description of co-occurring interventions Outcome data & Analysis Unit of assessmt/analysis Statistical techniques used For ea outcome: Reported? Definition, Measurement/ method used, Unit of measurement, Length of followup, # and/or times of followup, Measuremts For intervention group(s) & control group(s): # participants enrolled, Grade
Bret Luick, PhD Professor of Nutrition at the University of Alaska, Fairbanks Research interests include almost every topic you can think of JNEB Board of Editors member
5. Quality of evidence constrains the impact of a review article Quality criteria for articles have been established. Following these criteria ensures: 1. Sound and accepted scientific methods 2. Systematic, objective rating of research articles 3. Consistent inter-rater scoring
Relevance Given a clearly stated objective for the review... 1.Did the intervention show efficacy? 2.Is the article salient? 3.Is the article relevant to the JNEB audience? 4.Were the methods feasible?
Validity, Points 1-5 1.Clear research statement 2.Subject selection bias 3.Comparable treatment/control groups 4.Appropriate management of subject attrition 5.Blinding
Validity, Points 6-10 6. Intervention methods detail 7. Clearly defined outcomes 8. Appropriate statistical analysis 9. Conclusions consistent with results 10. Conflict of interest/source of funding bias
Evaluation Tools Evaluate Relevance and Validity of: Primary research: Human subjects Primary resarch & Review arcticles See: Evidence Analysis Manual, Appendix 8 Academy of Nutrition and Dietetics AMSTAR: Methods quality assessemt tool: Review articles
Mary Murimi, PhD, RD Professor at Texas Tech Associate Editor for JNEB President-elect of SNEB Research interests focus on low income populations
5. Framework for Discussion Discuss the phenomenal of interest based on your major findings and its interpretations Discuss how your study relate or improve practice and advance what readers know Discuss the quality of the studies under review and how that quality affected the outcome including limitations
Review discussion should answer the following questions How did you answer your research question? What is the weight of evidence in answering your research questions? What is the meaning of your findings and how can they be applied by practitioners and readers? What are your findings suggesting and what is the evidence of that suggestion? Does the findings support a particular theory, model, recommendations or practice?
Put results in the context of what is known on the topic Comparing your review with previously-published systematic reviews or current opinions and guidelines How do your findings compare with other similar or related studies? Identify the main findings, including the strength of evidence for each main outcome and their relevance to practitioners Is there enough evidence to arrive at a definite conclusion? If not clarify The level of conclusions reached should not exceed the level of the studies reviewed Provide a general interpretation of the results in the context of other evidence and implications for future research
6.Discuss the Quality and Limitations of the Review Discuss any methodological shortcomings in the systematic review and state how these may affect the interpretation of the results Provide recommendations on how these shortcomings may be rectified in future studies Does the quality of the included studies affect the outcome of your results? Discuss the ethical aspects of the included studies State limitations that might affect the reliability of the results at study and outcome levels incomplete retrieval of identified research, reporting bias, or lack of robustness
Madeleine Sigman-Grant, PhD, RD Extension specialist and professor. Areas of education and research are: prevention of childhood obesity, nutrition and health for pregnant and nursing women, nutrition for teen parents and breastfeeding support. Associate Editor for JNEB
7. Implications for Research and Practice Your discussion should have finished by stating some overall conclusions about the study Concisely state how these findings or major conclusions could be applied to best practices, if they can Madeleine
Implications for Research and Practice Detail additional research that would strengthen the conclusions or extend the results to larger audiences Include any policy implications Madeleine Sigman-Grant
Summary & Conclusions Systematic Reviews are not just literature reviews To be most likely published, conclusion should not be that there is not enough research to support a conclusion These are lots of work, not something to write in a weeknd KCN
Worktables Seung-Yeon: writing the research question Noel: methodology decisions Susan: practice with data extraction Bret Luick: determining paper quality Mary: developing the discussion section Madeleine: developing implications KCN