Resources. A look back 9/21/2015

Size: px

Start display at page:

Download "Resources. A look back 9/21/2015"

Cory Watkins
8 years ago
Views:

1 Module 4: Analyzing Assessment Data: Goals, Purposes, and General Techniques September, 2015 Jeremy Penn, Ph.D. Director Resources Cooper & Shelley, Data Analysis (chapter 6 in the Schuh book) Planning your Data Analyses (handout) Module 4 Rubric (handout) Decision Tree Analysis for Quantitative Studies (handout) A look back In Module 1 we talked about baseball as an example for how anecdotal information is not enough to make important decisions Other teams have caught up, apparently! I hope you still believe assessment can be helpful in informing your work! 1

Director Resources Cooper & Shelley, Data Analysis (chapter 6 in the Schuh book) Planning your Data Analyses (handout) Module 4 Rubric

2 Follow-through 1. Asking a Question (Purpose) Plan 6. Celebrating results (Followthrough and reflection) 2. Designing an Assessment Study SA 3 Modules 5. Using and Communicating Results 3. Assessment Instruments (validity and reliability) Analyze and use 4. Analyzing Results Implement Overview of Today s Session Overview of tools and resources to consider for your data analysis Easy to get lost in data analysis if you don t have a clear plan and a clear goal for your data analyses creating a plan Tools and Resources for Data Analysis 2

Assessment Instruments (validity and reliability) Analyze and use 4.

3 What is the specific goal of your analysis? Different Resources Used for Each Quantitative studies Describe trends Compare groups Relationship between variables Qualitative studies Text analysis Describing information and developing themes Understanding QUAN analysis: Analysis of numbers Statistical analysis, such as t-test comparing groups or correlation Creation of tables, graphs, charts Typically use software QUAL resources: Analysis of text Thematic, descriptive summary Creation of rich, thick description or narrative Can use software but not required QUAN Analysis Software Use the built-in reports available in Campus Labs or Qualtrics Pro: Readily available and easy to use summary Con: Limited customization and stuck with a specific look, some limits on what can be analyzed 3

Understanding QUAN analysis: Analysis of numbers Statistical analysis, such as t-test comparing groups or correlation Creation of tables, graphs, charts Typically use software QUAL resources:

4 QUAN Analysis Software Excel Pro: Powerful tool, can create good visual displays of results, available on nearly all NDSU computers, allows custom analysis, lots of help sites available on the web, many built-in analysis tools (such as pivot tables) Con: Not good with missing data, some other software can be faster depending on the analysis you want to run, does not do matching between data sets very well QUAN Analysis Software SPSS Pro: Extremely powerful, can download files correctly formatted directly from Campus Labs and Qualtrics, excellent at matching between data sets, can run any analysis you want Con: $50 per year per computer but can find in some computer labs on campus, learning curve, output not always visually pleasing Other QUAN Software Tableau Public ( basic version free, can create interesting data visualizations and interactive webpages (although we can t currently use them in NDSU s website); particularly good for location data Infographics (e.g., infogr.am, Visme, etc.) a session on these later this semester! 4

sets very well QUAN Analysis Software SPSS Pro: Extremely powerful, can download files correctly formatted directly from Campus Labs and Qualtrics, excellent at matching between data sets, can run

Other QUAN assistance Office of Student Affairs Assessment NDSU Statistical Consulting Service (https://www.ndsu.

5 Other QUAN assistance Office of Student Affairs Assessment NDSU Statistical Consulting Service ( vices/statistical_consulting/) OIRA ( Viet Doan (pulling student data) QUAL software Smaller data sets you can still analyze by hand Excel and Word also can be used to do some qualitative analyses and bring efficiencies Advanced software, such as Atlas.TI or NVivo, can be helpful for large data sets or analysis of video or photography data but big learning curve and $$$ Other QUAL resources 5

edu/oira/) Viet Doan (pulling student data) QUAL software Smaller data sets you can still analyze by hand Excel and Word also can be used to do

6 Planning for your Data Analysis Key Questions 1. How will you analyze your data? 2. How will your analysis address your assessment questions and suit the purpose of your project? 3. What resources will you need to complete your analyses? What can happen when you don t have a plan 6

How will your analysis address your assessment questions and suit the

Key to Successful Data Analysis: Have a plan! Key questions to consider when creating your data analysis plan (see handout): A. What question will it answer or purpose will it serve? B.

7 Key to Successful Data Analysis: Have a plan! Key questions to consider when creating your data analysis plan (see handout): A. What question will it answer or purpose will it serve? B. What is the specific goal of the analysis? C. What technique will be used? D. What will the results look like? E. Who will do the analysis and when will it be completed? F. What additional resources are needed? What will results look like? Tables? Graphs? Rich, descriptive text? Short text summaries of statistical tests? Pictures, images, videos? Having a clear plan in this area can save substantial time later Visual Displays of Information Can Be Powerful 7

What will the results look like? E. Who will do the analysis and when will it be completed? F. What additional resources are needed? What will results look like?

8 8

9 Excellence in Graphical Displays (From Tufte) Show the data Induce the viewer to think about the substance rather than the methodology or the design Avoid distorting what the data have to say Present many numbers in a small space Make large data sets coherent Encourage the eye to compare different pieces of data Reveal the data at several levels of detail Have a clear purpose Be integrated with the statistical and verbal descriptions of a data set Activity: Planning your Data Analyses Work in your groups to complete the worksheet If you ve already started (or completed) your data analysis, list out what you ve already done and what might be your next step Be prepared to share your plans with the whole group today The focus of your sharing today should be on how your plans for data analysis will address your questions and serve the purpose of your project Remaining Workshops this fall Knowing When and How to End a Program: October 6, 10 AM, Meinecke Creating Assessment Infographics: October 27, 1 PM, Room of Nations 9

the statistical and verbal descriptions of a data set Activity: Planning your Data Analyses Work in your groups to complete the worksheet If you ve already started (or completed) your data analysis,

Analyzing Qualitative Data For Assessment October 15, 2014

Analyzing Qualitative Data For Assessment October 15, 2014 Jeremy Penn, Ph.D. Director Resources Data Analysis, Chapter 6 in Schuh book (pages 158-168) Creswell (2002) or (1998) you can borrow from me