Secondary Data &Meta-Analysis

Transcription

1 Secondary Data &Meta-Analysis

2 Secondary Data - Overview Secondary data: data that have already been collected for a different purpose, but may be relevant to the research problems at hand. Primary data: survey, observation, or experimental data collected to address the problem currently under investigation.

3 Importance/Usefulness Secondary data are useful for addressing a number of research questions, for example: Estimating market potential/price sensitivity (airlines) Analyzing competitors Sales forecasting Assessing industry trends Determining labor market characteristics/ trends (facilities establishment/acquisition analyses) Providing preliminary information to guide subsequent primary data collection

4 Advantages of Secondary Data Low Cost Less Effort More Timely Some info is available only from secondary data sources; for example: Market share Industry data from trade associations Labor Market Compensation data

5 Sources of Secondary Data There are two key sources of secondary data: Commercial/ Corporate Databases Institutional Databases

6 Commercial/Corporate Databases Commercial/ Syndicated Databases Data available for purchase regarding: Sales Credit rating/history Compensation Examples General: Lexis/Nexis Market Specific: Nielson (consumers), Gartner (IT) Corporate Data Data available regarding: Employee behavior (absence, turnover, compensation, performance) Customer relations Purchasing Financial/Accounting

7 Institutional Sources of Secondary Data Government publications Census, Bureau of Labor Statistics, etc. Trade Associations - Newsletters, special reports, annual state of the industry reports, etc. Other publications - periodicals (e.g., WSJ, Fortune), annual reports, Moody s, Dun & Bradstreet Internet

8 Computer Retrievable Sources of Secondary Data Surfing the web Search Engines Newsgroups on the Internet Internet sites devoted to a specific topic where people can read and post messages. Databases Cox Databases A number of companies offer database packages on DVD for personal computers or by subscription over the internet.

9 Some Useful Numbering Systems SIC & UPC SIC (Standard Industrial Classification) is an uniform numbering system for classifying establishments according to their economic activities. Total economy is divided into 11 divisions (manufacturing/mining etc...) which are further divided into major groups and subgroups. Example: 57 = Home Furniture, Furnishings 571 = Home Furniture and Furnishing Stores 5715 = Furniture Stores Government and several non-govt. industrial data are organized by SIC codes now being replaced by North American Industrial Classification Codes (NAICS) UPC: Numbering system used in consumer goods

10 SIC System for Classifying Retailers

11 Limitations of Secondary Data Lack of Availability For some research questions there are simply no available data. For example, if Nestle wanted to evaluate the taste, texture, and color of three new gourmet brownie mixes, there are no secondary data that would answer these questions. Lack of Depth NLSY has longitudinal data on alcohol consumption, but lacks data on where and when drinking occurs and has limited workrelated data were we to want to look at workrelated consequences.

12 Limitations of Secondary Data Lack of Relevance May be measured in units that cannot be used by the researcher. May relate to a sample other than the intended target. May be outdated. Inaccuracy Always assess the accuracy of the data. There are a number of potential sources of error when a researcher gathers, codes, analyzes, and presents data.

13 What is Meta Analysis? A quantitative approach for systematically combining results of previous research to arrive at conclusions about the body of research.

14 Four Steps of Meta Analysis Identify studies to include Determine eligibility criteria include/exclude according to them. Abstract Data from the studies Analyze data in the studies statistically

15 Identify your studies Be methodical List of popular databases to search Pubmed/Medline Psychlit ABI/Inform Other strategies you may adopt Hand search (go to the library...) Personal references, and s web, eg. Google (

16 How to Search for literature Formulate your question appropriately If you are searching pubmed Use Medical Subject Headings (MeSH) [1] Lookup word in text word, abstract, title [2] Combine [1] with [2] using boolean logic Set up proper filters For Others, use text word, abstract

17 Boolean Logic: AND this is AND zone, covering common area between two ellipses

18 Example: Research Issue Let's say we have a new prophylaxis treatment for heart attacks. We want to know the efficacy of the existing treatment -- streptokinase. How should we set up a search strategy? Search pubmed only

19 Choose your DATABASE here Remember to choose both PUBMED, and MESH for formulating search. Choose PUBMED CENTRAL for free articles!

20 Keep some, throw out others Cannot include all studies Keep the ones with high levels of evidence good quality Randomized Clinical/Controlled Trial Case reports definitely out Selection problems are major problems Patients with prior heart attacks included Anyone over age 60 excluded

21 Plan of Action ARE THE STUDIES ELIGIBLE FOR MA (STEP I)? NO DISCARD YES ABSTRACT THE DATA ENTER INTO A SPECIFIED FORMAT

22 How to Abstract Data: Guidelines Create a spreadsheet by study (row) with the following columns: name of the study name of the author, year published number of participants who received intervention number of participants who were in control arm number who developed outcomes in intervention number who developed outcomes in control

23 Spreadsheet Data for Strepto Study We created seven columns trial: trial identity code trialname: name of trial year: year of the study pop1: study population deaths1: deaths in study pop0: control population deaths0: deaths in control

24 Analyze Data Statistically Combine data to arrive at a summary, 3 measures Effect Size (Odds Ratio) Variance with 95% Confidence Interval Test of heterogeneity Two Graphs Forest Plot Funnel Plot Examine why the studies are heterogeneous, if they are (might indicate need for moderator analysis)

25 Summary Estimates for Strepto Study Mantel Haenszel OR= % Confidence Interval [0.72, 0.83] Test of Heterogeneity: Chi-square (df=21) = 31.5 P-Value = 0.07 The pooled Odds Ratio shows that those receiving streptokinase at AMI are about 77% at risk of death (23% less likely to die) That in 95 out of 100 such meta analyses, the pooled Odds Ratio would lie between 0.72 and 0.83, indicating a statistically significant protective effect That these studies were not significantly heterogeneous

26 Forest Plot The dotted line passes across null, or 1.0 The size of the black square box indicates weight of the study The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the line The diamond is the summary estimate The two ends of the diamond indicate 95% CI

27 Funnel Plot: what and how to read Plots the effect size against the sample size of the study To study a funnel plot, look at its LOWER LEFT corner, that s where negative or null studies are located If EMPTY, this indicates PUBLICATION BIAS Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias

28 Meta Analysis of Correlations Example: Should we select employees based on intelligence? Answer: Let s see what research shows using meta analysis:

29 Issues in meta analysis Choosing a model Fixed effects model or random effects? Bias in meta analysis poor quality of trials publication bias (file drawer problem) Quality control in meta analysis

30 Summary of Key Points Secondary data are information previously gathered for a different purpose that may be relevant to the problem at hand. Secondary data can come from sources internal to the organization or external. The internet has, in many ways, enabled the gathering of secondary data. Secondary data are generally useful, low-cost, rapidly available sources of information. Meta-Analyses offer summary of multiple sources of secondary data.