Karyn Ruiz-Cordell, MA, PhD Shunda Irons-Brown, PhD, MBA, CHCP Tamar Sapir, PhD

Transcription

1

2 Advanced Methodologies in Outcomes & Insights Research Study Design Measuring Knowledge vs. Impact vs. Performance vs. Quality of Care and Everything In Between Karyn Ruiz-Cordell, MA, PhD Shunda Irons-Brown, PhD, MBA, CHCP Tamar Sapir, PhD

3 DISCLOSURES

4 Educational Objectives Statistical Tools - Engaging in analytics from the start Statistical Planning - Hypothesis, research goals, and critical thinking Statistical Techniques - Knowing when to do what with your data Interpreting the Data Applying your findings

5 What does data mean, to you?

6 What we hear

7 What we do Where do you need help?

8 The New Data-Driven World: A Quiet Revolution Then: Data used to answer discrete (predetermined) questions (hypotheses) Now: Data and data systems to drive change and achieve goals

9 What is Data Analysis? Descriptive statistics Quantitatively defining describing the main features of a collection of information Confirmatory data analysis (Inferential) Confirming or falsifying existing hypothesis Exploratory data analysis (Inferential) Discovering new features in the data

10 Considerations, Selection, & Interpretations of Appropriate Statistical Testing When Evaluating the Effectiveness of Education

11 Types of Variables The Terms Dependent Outcome variable (test score) Independent Variable being manipulated in order to observe the effect on the DV Continuous Quantitative variables Interval Continuum, numeric Ratio Zero Categorical Discrete variables Nominal - 2, no order Dichotomous Ordinal - 2, ranked

12 Flow-chart: Test Selection

13 Decisions: Parametric vs. Non-Parametric

14 Examining Differences and Comparing Groups Best case scenarios

15 Case 1: Comparing impact of education among different groups within a cohort Which of the following patient populations with hypercholesterolemia are indicated for the new PCSK9 inhibitor therapies? (Select all that apply) 160% 140% 120% 100% 80% lipidologists % improvement (N = 152) PCP % improvement (N = 252) 80% 149% 80% 77% Considerations Variable type: Continuous Categorical Unpaired Test Independent T-test 60% 40% 20% 0% 0% 9% 3% 0% Failed 1 statin Failed 2 statins Max-tolerated statin 24% 21% HeFH ASCVD *Note Percent change vs. Percent difference

16 Case 2: Comparing impact of educational formats 100% 80% 60% 40% 20% 0% Average test score of participants Correct answer 53% Pre Live (n= 135) 83% Post Live (n= 87) 44% Pre Online (N=589) 69% Post Online (N =468) Considerations Variable type Test/s Continuous Paired t-test comparing pre/post Independent t-test comparing post-test of different formats

17 Non-Parametric Tests (or ) What to do when you don t know what to do

18 Case 1 Please rate your confidence in using advanced methodologies in outcomes and research design? (1 = not all confident; 5 = completely confident) % 5% 5% 5% 14% 14% 18% 28% Pre-Education (N = 337) Post-Education (N = 243) 50% 57% 0% 20% 40% 60% Considerations Ordinal or categorical variables? Ordinal Sample size >5 Paired or unpaired variables? Unpaired Ø What test can be used? Mann-Whitney U

19 SPSS Output If one group (e.g., pre-) tends to have higher values than the other group, that group's scores will have been assigned higher ranks and will have a higher mean rank (and viceversa for the group with lower scores)

20 Case 1 Cont : Now with matched data Please rate your confidence in using advanced methodologies in outcomes and research design? (1 = not all confident; 5 = completely confident) 2% 3% 5% 4% 10% 21% 22% 34% 49% 50% Pre-Education (N = 226) Post-Education (N = 226) 0% 20% 40% 60% Considerations Ordinal or categorical variables? Ordinal Sample size >5 Paired or unpaired variables? Paired/matched Ø What test can be used? Wilcoxon signed-rank test (or the nonparametric T)

21 SPSS Output In other words, it allows you to see how many participants had improved their confidence by post compared to pre-, how many remained the same, and how many had lesser confidence.

22 Case 2 Which of the following patient populations with hypercholesterolemia are indicated for the new PCSK9 inhibitor therapies? (Select all that apply) 80% 60% Pre-Education (N = 567) Post-Education (N = 463) 64% 75% Considerations Question type Recoding: right/ wrong answers Unpaired Sample size (>5) Ø Appropriate test? Chi square (association) 40% % Correct

23 SPSS Output Is there a difference between the actual and expected counts?

24 Case 2 Cont d: Now with matched data Which of the following patient populations with hypercholesterolemia are indicated for the new PCSK9 inhibitor therapies? (Select all that apply) 100% 80% 60% Pre-Education (N = 418) Post-Education (N = 418) 78% 75% 78% 87% Failed 1 statin Failed 2 statins 76% 82% Max-tolerated statin 82% 87% HeFH 80% 85% ASCVD Considerations Categorical (dichotomous) Paired Sample size (>5) Appropriate Test McNemar test (ttest)

25 SPSS Output Of those who didn t select this response at pre- 32 (only 31%) selected it at post-

26 Advanced Analytics Predictive modeling

27 What Does PM Look like?! A source is the raw data that you want to use to create a model. Each row represents an instance or field. A dataset is a structures version of a source where each field has been processed and serialized according to its type (numeric, categorical, text, date-time, etc.) A model represents a set of correlation patterns automatically inferred fro the statistical relationships across the field in your dataset. You can use your model to make predictions. That is, to find the category or expected value of the target (DV) for new instances.

28

29 The software to make it happen

30 Data Analysis Software Excel Statistica SPSS SAS Minitab STATA Systat R NVivo, Atlas, NUDist (QDA)

31 Considerations Use Type of data (dataset vs. database) Skill (analyst, programmer, database architect) Cost (ranging from FREE to $$$$) Visualization (graphing) Programming Language (R, python, etc.) Modeling (algorithms)

32 Free (or almost) & Easy to Use Graphpad PSPP (SPSS) Download: BigML Data modeling Infogram Simple data, infographics Statwing Stats & visuals with plain language interpretations Free trial then $50-$100 per/mo Tableau Public Visualizer Low-cost version ($1k per/yr.) QDA Miner lite (QDA)

33 Full List of Stat Packages Analytica - visual analytics and statistics package Angoss - products KnowledgeSEEKER and KnowledgeSTUDIO incorporate several data mining algorithms ASReml for restricted maximum likelihood analyses BMDP general statistics package Data Applied for building statistical models DB Lytix in-database models EViews for econometric analysis FAME (database) a system for managing time-series databases GAUSS programming language for statistics Genedata software solution for integration and interpretation of experimental data in the life science R&D GenStat general statistics package GLIM (software) early package for fitting generalized linear models GraphPad InStat very simple with lots of guidance and explanations GraphPad Prism biostatistics and nonlinear regression with clear explanations IMSL Numerical Libraries software library with statistical algorithms JMP (application software) visual analysis and statistics package LIMDEP comprehensive statistics and econometrics package LISREL statistics package used in structural equation modeling Maple programming language with statistical features Mathematica a software package with statistical features MATLAB programming language with statistical features MedCalc for biomedical sciences Minitab general statistics package MLwiN multilevel models (free to UK academics) NAG Numerical Library comprehensive math and statistics library Neural Designer commercial deep learning package NCSS general statistics package NLOGIT comprehensive statistics and econometrics package NMath Stats statistical package for.net Framework O-Matrix programming language XploRe

34 Full List of Stat Packages OriginPro statistics and graphing, programming access to NAG library PASS power and sample size software from NCSS Partek general statistics package with specific applications for genomic, HTS, and QSAR data Plotly plotting library and styling interface for analyzing data and creating browser-based graphs. Available for R, Python, MATLAB, Julia, and Perl Primer-E Primer environmental and ecological specific PV-WAVE programming language comprehensive data analysis and visualization with IMSL statistical package Qlucore Omics Explorer - interactive and visual data analysis software Quantum Programming Language part of the SPSS MR product line, mostly for data validation and tabulation in Marketing and Opinion Research RapidMiner machine learning toolbox Regression Analysis of Time Series (RATS) comprehensive econometric analysis package SAS (software) comprehensive statistical package SHAZAM (Econometrics and Statistics Software) comprehensive econometrics and statistics package Simul - econometric tool for multidimensional (multi-sectoral, multi-regional) modeling SigmaStat package for group analysis SmartPLS - statistics package used in partial least squares path modeling (PLS) and PLS-based structural equation modeling SOCR online tools for teaching statistics and probability theory Speakeasy (computational environment) numerical computational environment and programming language with many statistical and econometric analysis features SPSS Modeler comprehensive data mining and text analytics workbench SPSS Statistics comprehensive statistics package that stands for "Statistical Package for the Social Sciences" Stata comprehensive statistics package Statgraphics general statistics package to include cloud computing and Six Sigma for use in business development, process improvement, data imaging and statistical analysis. STATISTICA comprehensive statistics package StatsDirect statistics package designed for biomedical, public health and general health science uses StatXact package for exact nonparametric and parametric statistics Systat general statistics package SuperCROSS - comprehensive statistics package with ad-hoc, cross tabulation analysis S-PLUS general statistics package Unistat general statistics package that can also work as Excel add-in The Unscrambler - free-to-try commercial multivariate analysis software for Windows Wolfram Language[2] - the computer language that evolved from the program Mathematica. It has similar statistical capabilities as Mathematica. World Programming System (WPS) statistical package that supports the SAS language

35 Key Takeaways & Discussion