# Student Evaluations and the Assessment of Teaching: What Can We Learn from the Data?

Save this PDF as:

Size: px
Start display at page:

Download "Student Evaluations and the Assessment of Teaching: What Can We Learn from the Data?"

## Transcription

1 Student Evaluations and the Assessment of Teaching: What Can We Learn from the Data? Martin D. D. Evans and Paul D. McNelis Department of Economics, Georgetown University September 2000 This study was commissioned by the University Provost s Office in Fall We wish to thank University Provost Dorothy Brown, University Registrar John Pierce, Vice President Joseph Pettit, and Director of Institutional Research Michael McGuire for making available the data and for their support.

4 Table 2: Cumulative Distribution of A Grades Across Classes Percentage of A/A - Grades in Each Class Percentage of Classes 90.00% 4.27% 80.00% 9.53% 70.00% 18.07% 60.00% 30.49% 50.00% 46.78% 40.00% 69.91% 30.00% 87.91% 20.00% 98.75% 10.00% 99.80% In principle, evaluation forms are filled in by all the students in each class so that the survey aggregates all views on the instructor. In practice there are a large number of classes where only a subset of students actually taking the class fill in the evaluation forms. We refer to this fraction as the survey response rate. We measure the response rate as the number of evaluations actually filled in divided by the number of students enrolled in the class. Table III gives the distribution of the response rates in Spring Table 3: Cumulative Distribution of Response Rates Response Rate Less Than % 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% Percent of All Classes % 57.56% 27.73% 13.27% 5.91% 2.10% 1.18% 0.26% The mean response rate is 85 percent and the median is 88 percent. The table shows that the percentage of classes for which the response rate is less than 70 percent is 13 percent. This is a surprisingly large number when one considers that these response rates reflect attendance of students at class at the end of the term, either on the last day of class or at classes during the final weeks of the term, when the survey forms are handed out. Clearly, the surveys fall well short of aggregating the views of all students in every class. 4

6 Methodology The methodology we use in this study is widely applied in Biostatistics and medical research: It is used to determine the effectiveness of specific drugs or therapies by discriminating between control groups, one of which uses a new regime, and another which makes use of a placebo. Before any decisions are made about the effectiveness of the drug, the results from both groups are adjusted for fixed effects. In medicine, such fixed effects would be the age, the health history, and other medical risk factors. We use the data from the evaluation survey and the grading history of instructors to estimate a statistical model. The model relates the responses in the five categories to question III.5 of the survey to a set of explanatory variables (described below) that identify the fixed effects. The model, known in the literature as an Ordered Multinomial-Choice Model, has been used extensively to study data that take on a finite number of values possessing a natural ordering. 1 Rather than describe the model in technical detail, let us outline some of its key features: The model accounts for the fact that students only have 5 choices when responding to question III.5. We can therefore explicitly deal with the distortions caused by this constraint. The model can accommodate varying response rates across classes. Importantly, its structure allows us to investigate whether low response rates are related to how the students view the instructor. If absenteeism primarily occurs for other independent reasons (like sickness), the model will quantify the degree to which the precision of our statistical analysis is affected. If absenteeism is not independent, the model will allow us to quantify and correct for the biases caused by non-response. The model does not use the average score as the survey measure to be explained. This is important because the average score is a very poor statistical measure of student responses. The average fails to account for the degree of unanimity in the student responses. It also implicitly places uneven weights on individual student responses. (These effects were described in detail in the proposal for this project which is available from the authors.) The explanatory variables included in the model are the following: The student response ratio. The course level, taking on a value of 1 if it is a basic course and 0 otherwise. A small class size indicator, taking on a value of 1 if it is less than 5 and 0 otherwise. 1 We report estimates for an Order-Logit specification of the model below. We have also estimated specifications within normal (i.e., Order-Probit) and extreme value distributions. In each case, the estimates are very similar to those we report. 6

7 A seminar class size indicator, taking on a value of 1 if it is more than 5 and less than 15. An early morning indicator, taking on a value of 1 if the class meets before 10 am, 0 otherwise. Three variables for the grading reputation of the instructor: The percentage of B grades, the percentage of C grades, and the percentage of D/F grades, given by the instructor in the previous year. The model estimates are presented in Table 5. The complex structure of the model makes it hard to judge how large an effect a variable has on the scores from simply looking at the coefficient estimates. However, Table 5 does allow us to identify which variables have significant effects and whether they raise or lower scores. In particular, a positive (negative) coefficient estimate implies that an increase in the associated explanatory variable will raise (lower) the numerical score given by a student in each class. The T-statistics in the righthand column provide a statistical measure of significance. By this measure, all the coefficients except the level of the course are significantly different from zero at the one percent level or less. This means that there is a less than one in a hundred chance of finding the relation we observe in the data between the scores and the explanatory variable if in fact they were unrelated. By this measure, the statistical evidence linking scores to the explanatory variables is extremely strong. Only the level of the course seems unimportant. Teaching a lower or upper level course, everything else being equal, has no systematic effect on the student evaluations. Table 5: Ordered Multinomial-Choice Model Estimates Explanatory Variable Coefficient Std Dev T-stat Response ratio Basic course Less than to Early morning B's C's D's The estimated coefficients indicate that higher response rates increase the instructor s score. One interpretation of this finding is that more effective teachers get better attendance at their classes. However, such an interpretation assumes that the evaluation forms were distributed in a typical class. If the forms were handed out at a pre-announced review session, students who found the instructor particularly effective may have stayed away because they viewed the review as unnecessary. In this case, a lower score would be associated with a lower response rate because the survey under-represents students who found the instructor particularly effective. 7

9 Table 7: Changing Characteristics Initial Characteristics Less response Lower level Less than 5 More than 15 Early morning More B's More C's More D's Lower Response Rate and University Grade Guide Lower Response Rate and Lower than University Grade Guide Response Rate level 1 Less than 5 5 to 15 Early Morning B% C% D/F% Predicted Score The middle panel of Table 7 shows how the predicted score changes when we vary each of the explanatory variables in turn. The effects of changing each variable in isolation are generally quite small. The lower panel of the table shows what happens if the instructor has a response rate of 80 percent, and followed the university grading guidelines in the previous year; giving 32 percent A grades, 54 percent B grades, 13 percent C grades, and 1 percent D/F grades. Here we see that the predicted score falls from 4.5 to While this may not seem particularly striking in numerical terms, it has a dramatic effect on the relative ranking of the instructor. A score of 4.5 places the instructor in the upper 52nd. percentile of scores across the university, i,e., just above the median. A score of 4.22, places the instructor among the lowest 31 percent. Thus, an instructor s decision to abide by the university grading guideline would push the instructor s score from around the university average to one in the lower third. If the instructor adopted a somewhat stiffer grading policy, 9

11 Clearly, opinions may differ as to the appropriate confidence level to use in making a particular decision related to the teaching effectiveness of individual faculty. Opinions may also differ on the confidence level appropriate for different decisions related to an individual s teaching effectiveness. For example, many may argue that the confidence level appropriate for a tenure decision should be higher than for an annual merit review. Similarly, the confidence level appropriate for judging low teaching effectiveness may differ from the level for judging high effectiveness. To make our findings applicable to as wide an audience as possible, Table 8 reports results for confidence levels ranging from 75 to 99 percent. Table 8: Identifying Teaching Effectiveness A: Low Scores Confidence Levels Mean Score Range Number 99% 97.5% 95% 90% 75% B: High Scores Confidence Levels Mean Score Range Number 99% 97.5% 95% 95% 97.5% Table 8 shows how the mean scores from Question III.5 relate to the teaching effectiveness of individual faculty for a range of confidence levels. The left hand columns of 11

12 the table break the mean scores received by each faculty member into different ranges and show how many faculty fall into each range. Panel A reports the fraction of the faculty within a particular mean score range with a significantly low teaching effectiveness. Panel B reports the fraction of the faculty with significantly high teaching effectiveness. The results in Panel A are striking. They quantify exactly how difficult it is to identify ineffective teaching from the mean survey scores with any reasonable level of accuracy. To illustrate this point, consider the first row in the panel. Of the 646 faculty in our study, 19.8 percent displayed low teaching effectiveness at the 99 percent confidence level. While this is close to the 19 percent of the faculty with mean scores of less than 4.0, it does not mean that 4.0 can act as a reliable cutoff point. Since only 88.7 percent of those scoring less that 4.0 display significantly low effectiveness, more than 11 percent of this group would be incorrectly identified as ineffective teachers. This implies an error rate of *0.99 =0.119, which is approximately 12 percent. Matters are even worse if we use the 75 percent confidence level. Here the error rate is over 26 percent. In fact, the error rate associated with the 4.0 cutoff remains above 10 percent for any confidence level. The 4.0 cutoff is also hard to justify on equity grounds. For example, at the 75 percent confidence level, 24.5 percent of faculty scoring above 4.0 display significantly low effectiveness. These individuals appear above the cutoff because their grading reputations and class characteristics counteract their ineffectiveness in the classroom. It is hard to see why this group should be viewed as more effective teachers than individuals scoring less than 4.0. In sum, these results show that a cutoff score of 4.0 for ineffective teaching cannot be justified on either accuracy or equity grounds irrespective of the confidence level one may choose to use. The lower portion of panel A shows the relation between teaching effectiveness and survey scores over narrower ranges. These results provide some guidance how to choose a cutoff point. Consider, for example, a cutoff score of 3.8. From the table we see that 96.4 percent of instructors scoring between 3.6 and 3.8 display low effectiveness at the 99 percent confidence level. The error rate for this group is therefore *0.99 = 0.046, approximately 5 percent. This is lower than the error rate associated with any other confidence level but it is well above the 1 percent error rate that is typically tolerated in important decision-making. To achieve this lower error rate, the cutoff score needs to be 3.6. Identifying high teaching effectiveness from the survey is just as difficult. Panel B of Table 8 reports the faction of faculty with significant effectiveness for different scoring ranges and varying confidence levels. Approximately 20 percent appear highly effective teachers at the 99 percent confidence level. This closely matches the faction that are ineffective. Unsurprisingly, none of the faculty scoring below 4.0 are highly effective at any confidence level. What is more surprising is that so few of the faculty scoring over 4.0 are effective. If we classified everyone with scores above 4.0 as highly effective, the error rates range from 63 to 76 percent. Moreover, the error rates do not decline significantly if we raise the cutoff point. The lowest error rate for effective teaching is still 31 percent and comes from a cutoff score of 4.8. In our view, the results in Table 8 completely undermine the current methods for identifying effective and ineffective teaching. The use of a 4.0 cutoff score to signal 12

13 ineffective teaching results in so many erroneous judgments that it must be view as essential arbitrary. Similarly, one cannot identify particularly effective teaching from high survey scores without making a lot of errors. While there are undoubtedly very able teachers at Georgetown, there is no evidence that these individuals can be reliably identified from the student surveys. Conclusions This empirical analysis of student survey data shows how difficult it is to accurately evaluate teaching effectiveness. But it would be a mistake to draw from this study that the current survey instrument needs to be replaced by another instrument, with more differentiated and probing questions, but with the same numerical scoring device. We are confident that we would be able to replicate similar error rates for assessing teaching from another instrument, which follows a similar modus operandi as the current instrument. One should conclude from this study that teaching evaluation data should only be used cautiously, to separate ineffective teachers from effective teachers with a cut-off at a very low range, of 3.6. Only at this rate can one be reasonably sure that one is not unfairly categorizing good teachers as ineffective. Arbitrary scores of 4.0 or above, as cut-offs for promotion or tenure, are just that. 13

### CALCULATIONS & STATISTICS

CALCULATIONS & STATISTICS CALCULATION OF SCORES Conversion of 1-5 scale to 0-100 scores When you look at your report, you will notice that the scores are reported on a 0-100 scale, even though respondents

### REVIEW OF NEW UNIVERSITY-WIDE TEACHING EVALUATION FORM

Excerpt from the PACTL Report to Senate of May 15, 1998 (S.98-123) Page 1 Provost's Advisory Committee on Teaching and Learning (PACTL) Report to Senate REVIEW OF NEW UNIVERSITY-WIDE TEACHING EVALUATION

### On-Line Course Evaluation Pilot Project at Marquette University: Spring The Marquette Online Course Evaluation System (MOCES)

Online Course Evaluation Pilot 1 On-Line Course Evaluation Pilot Project at Marquette University: Spring 2008 The Marquette Online Course Evaluation System (MOCES) Development of the Evaluation Instrument

### Paul Westrick. Research and Development Center for Higher Education

Research and Development Center for Higher Education Paul Westrick The normal distribution, also known as the bell curve, is a feature of the natural world around us. If we measure the weights of all the

### Chapter III Supporting Data: Collection & Presentation

Chapter III Supporting Data: Collection & Presentation Since the documentation of teaching must be selective to avoid unnecessary bulk in a tenure file and to increase the efficiency of the review process

### MAJOR FIELD TESTS Description of Test Reports

MAJOR FIELD TESTS Description of Test Reports NOTE: Some of the tests do not include Assessment Indicators and some tests do not include Subscores. Departmental Roster Includes scale scores for all students

### Report of the Survey of Writing Intensive Courses Fall 2003 Submitted by: Elizabeth Crepeau, Stephen Hardy, and Victor Benassi

Report of the Survey of Writing Intensive Courses Fall 2003 Submitted by: Elizabeth Crepeau, Stephen Hardy, and Victor Benassi Purpose of the Survey This survey examines student perceptions of the benefit

### More Zeroes of Polynomials. Elementary Functions. The Rational Root Test. The Rational Root Test

More Zeroes of Polynomials In this lecture we look more carefully at zeroes of polynomials. (Recall: a zero of a polynomial is sometimes called a root.) Our goal in the next few presentations is to set

### Answer: C. The strength of a correlation does not change if units change by a linear transformation such as: Fahrenheit = 32 + (5/9) * Centigrade

Statistics Quiz Correlation and Regression -- ANSWERS 1. Temperature and air pollution are known to be correlated. We collect data from two laboratories, in Boston and Montreal. Boston makes their measurements

### Drawing Inferences about Instructors: The Inter-Class Reliability of Student Ratings of Instruction

OEA Report 00-02 Drawing Inferences about Instructors: The Inter-Class Reliability of Student Ratings of Instruction Gerald M. Gillmore February, 2000 OVERVIEW The question addressed in this report is

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### How Not to Count the Poor? A Reply to Reddy and Pogge

How Not to Count the Poor? A Reply to Reddy and Pogge Martin Ravallion 1 Reddy and Pogge (2002) claim that the World Bank has systematically underestimated the extent of poverty in the world in the 1990s,

### Empirical Methods in Applied Economics

Empirical Methods in Applied Economics Jörn-Ste en Pischke LSE October 2005 1 Observational Studies and Regression 1.1 Conditional Randomization Again When we discussed experiments, we discussed already

### Statistics 2014 Scoring Guidelines

AP Statistics 2014 Scoring Guidelines College Board, Advanced Placement Program, AP, AP Central, and the acorn logo are registered trademarks of the College Board. AP Central is the official online home

### Why Hedge Funds? October 29, 2014

Why? October 29, 2014 This report provides an overview of hedge funds and their role within a diversified portfolio, a subject we first wrote about in 2004, the year Cliffwater was founded. 1 Our update

### In this chapter, you will learn to use descriptive statistics to organize, summarize, analyze, and interpret data for contract pricing.

3.0 - Chapter Introduction In this chapter, you will learn to use descriptive statistics to organize, summarize, analyze, and interpret data for contract pricing. Categories of Statistics. Statistics is

Research Report Office of Institutional Research Report on Plus and Minus Grading Fall 2006 This report summarizes the findings of the plus and minus grading trial conducted at Western Kentucky University,

### Age to Age Factor Selection under Changing Development Chris G. Gross, ACAS, MAAA

Age to Age Factor Selection under Changing Development Chris G. Gross, ACAS, MAAA Introduction A common question faced by many actuaries when selecting loss development factors is whether to base the selected

### CHINHOYI UNIVERSITY OF TECHNOLOGY

CHINHOYI UNIVERSITY OF TECHNOLOGY SCHOOL OF NATURAL SCIENCES AND MATHEMATICS DEPARTMENT OF MATHEMATICS MEASURES OF CENTRAL TENDENCY AND DISPERSION INTRODUCTION From the previous unit, the Graphical displays

### Student Response to Instruction (SRTI)

office of Academic Planning & Assessment University of Massachusetts Amherst Martha Stassen Assistant Provost, Assessment and Educational Effectiveness 545-5146 or mstassen@acad.umass.edu Student Response

### Sample Size Issues for Conjoint Analysis

Chapter 7 Sample Size Issues for Conjoint Analysis I m about to conduct a conjoint analysis study. How large a sample size do I need? What will be the margin of error of my estimates if I use a sample

### 11/20/2014. Correlational research is used to describe the relationship between two or more naturally occurring variables.

Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are highly extraverted people less afraid of rejection

### Florida s Plan to Ensure Equitable Access to Excellent Educators. heralded Florida for being number two in the nation for AP participation, a dramatic

Florida s Plan to Ensure Equitable Access to Excellent Educators Introduction Florida s record on educational excellence and equity over the last fifteen years speaks for itself. In the 10 th Annual AP

### German Language Teaching and Teacher Training at Colleges and Universities in the US

German Language Teaching and Teacher Training at Colleges and Universities in the US Summary of Results of Online Survey Conducted Between April 7-24, 2003 By Megan Brenn-White May 2003 The survey was

### Chapter 5: Analysis of The National Education Longitudinal Study (NELS:88)

Chapter 5: Analysis of The National Education Longitudinal Study (NELS:88) Introduction The National Educational Longitudinal Survey (NELS:88) followed students from 8 th grade in 1988 to 10 th grade in

### Credit Card Market Study Interim Report: Annex 4 Switching Analysis

MS14/6.2: Annex 4 Market Study Interim Report: Annex 4 November 2015 This annex describes data analysis we carried out to improve our understanding of switching and shopping around behaviour in the UK

### These two errors are particularly damaging to the perception by students of our program and hurt our recruiting efforts.

Critical Errors in the National Research Council s Ranking of UWM s Economics Program January, 2011 Prepared by Associate Professor Scott Adams, Director of Graduate Studies The National Research Council

### Mathematics 96 (3581) CA (Class Addendum) 3: Inverse Properties Mt. San Jacinto College Menifee Valley Campus Spring 2013

Mathematics 96 (358) CA (Class Addendum) 3: Inverse Properties Mt. San Jacinto College Menifee Valley Campus Spring 203 Name This class handout is worth a maimum of five (5) points. It is due no later

### Random Portfolios for Evaluating Trading Strategies

Random Portfolios for Evaluating Trading Strategies Patrick Burns 13th January 2006 Abstract Random portfolios can provide a statistical test that a trading strategy performs better than chance. Each run

### 7Benchmarks. for Information Technology Investments. Spending benchmarks help campuses evaluate their efforts in supplying IT services

Spending benchmarks help campuses evaluate their efforts in supplying IT services 7Benchmarks for Information Technology Investments Bryan Leister By David Smallen and Karen Leach In 198, Richard Van Horn,

### Normalizing Spectral Counts in Scaffold

Normalizing Spectral Counts in Scaffold When using MS/MS proteomics data to look for differentially expressed proteins, there are several issues that need to be considered: 1) peptides shared between proteins,

### Yoruba CAP Score Interpretation Guide

The Computerized Assessment of Proficiency (CAP) is an online test of proficiency developed by CASLS. In the past, proficiency tests developed at CASLS have been licensed by Avant Assessment through a

### Hypothesis testing. c 2014, Jeffrey S. Simonoff 1

Hypothesis testing So far, we ve talked about inference from the point of estimation. We ve tried to answer questions like What is a good estimate for a typical value? or How much variability is there

### IT Quick Reference Guides Using Scantron (Parscore)

IT Quick Reference Guides Using Scantron (Parscore) Scantron Guides This guide covers the use of Scantron Forms for test sheets. SCANTRON (PARSCORE) TEST SERVICES Scantron (Parscore) test scanning services

### Presentation of Regression Results

I ve put together some information on the industry standards on how to report regression results. Every paper uses a slightly different strategy, depending on author s focus. Please review the earlier

### Descriptive statistics Statistical inference statistical inference, statistical induction and inferential statistics

Descriptive statistics is the discipline of quantitatively describing the main features of a collection of data. Descriptive statistics are distinguished from inferential statistics (or inductive statistics),

### Developmental Education Pilot Report Southeastern Louisiana University Submitted January 2015

1 Developmental Education Pilot Report Southeastern Louisiana University Submitted January 2015 Developmental Education Mathematics Pilot Combined Developmental Math and College Algebra Conducted Fall

### Unit 16 Normal Distributions

Unit 16 Normal Distributions Objectives: To obtain relative frequencies (probabilities) and percentiles with a population having a normal distribution While there are many different types of distributions

### Faculty Satisfaction Survey Report A Comparative Analysis AY

Faculty Satisfaction Survey Report A Comparative Analysis AY 2001-2003 Introduction The Office of Institutional Effectiveness administers a survey to the faculty of Texas Southern University (TSU) annually.

### Developments In Business Simulation & Experiential Exercises, Volume 22, 1995

EVALUATING THE EFFECTIVENESS OF ROLE PLAYING SIMULATION AND OTHER METHODS IN TEACHING MANAGERIAL SKILLS James N. Trapp, Stephen A. Koontz, Derrell S. Peel, and Clement E. Ward Oklahoma State University

### CHAPTER 1 The Item Characteristic Curve

CHAPTER 1 The Item Characteristic Curve Chapter 1: The Item Characteristic Curve 5 CHAPTER 1 The Item Characteristic Curve In many educational and psychological measurement situations, there is an underlying

### HANDOUT #1 INSUFFICIENT P&T LETTER EXAMPLE

HANDOUT #1 INSUFFICIENT P&T LETTER EXAMPLE TO: FROM: RE: Department Chair Department Personnel Committee Department Committee s Recommendation of Dr. Mark Green for Promotion to Associate Professor with

### Size of a study. Chapter 15

Size of a study 15.1 Introduction It is important to ensure at the design stage that the proposed number of subjects to be recruited into any study will be appropriate to answer the main objective(s) of

### Case Study Structured Exercises

Case Study Structured Exercises by Marc C. Marchese, Ph.D. Employee Selection staffing Employee Selection Structured Exercise About This Structured Exercise This exercise introduces undergraduate students

### Test Scoring And Course Evaluation Service

Test Scoring And Course Evaluation Service TABLE OF CONTENTS Introduction... 3 Section 1: Preparing a Test or Questionnaire... 4 Obtaining the Answer Forms... 4 Planning the Test or Course evaluation...

### How Are Interest Rates Affecting Household Consumption and Savings?

Utah State University DigitalCommons@USU All Graduate Plan B and other Reports Graduate Studies 2012 How Are Interest Rates Affecting Household Consumption and Savings? Lacy Christensen Utah State University

### Chapter 14: Production Possibility Frontiers

Chapter 14: Production Possibility Frontiers 14.1: Introduction In chapter 8 we considered the allocation of a given amount of goods in society. We saw that the final allocation depends upon the initial

### Using Education for the Future Questionnaires

ASSESSING PERCEPTIONS Using Education for the Future Questionnaires By Victoria L. Bernhardt Where did the questionnaires come from, what do they tell us, and are they valid and reliable? In 1991, while

### How strong is a linear relationship?

Lesson 19 Interpreting Correlation Student Outcomes Students use technology to determine the value of the correlation coefficient for a given data set. Students interpret the value of the correlation coefficient

### the law of inertia of large numbers, which states that large groups are more stable than small ones. Applied to research, the first law means that a g

T O P I C 7 Data gathering 2 Preview Introduction This topic is a continuation of topic 6 and is concerned with the theory and methodology of sampling. Sampling forms a foundation to methods of data gathering

### Association Between Variables

Contents 11 Association Between Variables 767 11.1 Introduction............................ 767 11.1.1 Measure of Association................. 768 11.1.2 Chapter Summary.................... 769 11.2 Chi

### Unit 1 Number Sense. In this unit, students will study repeating decimals, percents, fractions, decimals, and proportions.

Unit 1 Number Sense In this unit, students will study repeating decimals, percents, fractions, decimals, and proportions. BLM Three Types of Percent Problems (p L-34) is a summary BLM for the material

### Descriptive Statistics and Measurement Scales

Descriptive Statistics 1 Descriptive Statistics and Measurement Scales Descriptive statistics are used to describe the basic features of the data in a study. They provide simple summaries about the sample

### Do Commodity Price Spikes Cause Long-Term Inflation?

No. 11-1 Do Commodity Price Spikes Cause Long-Term Inflation? Geoffrey M.B. Tootell Abstract: This public policy brief examines the relationship between trend inflation and commodity price increases and

### Hypothesis testing S2

Basic medical statistics for clinical and experimental research Hypothesis testing S2 Katarzyna Jóźwiak k.jozwiak@nki.nl 2nd November 2015 1/43 Introduction Point estimation: use a sample statistic to

### GUIDELINES FOR TENURE AND PROMOTION. Kenan-Flagler Business School The University of North Carolina

GUIDELINES FOR TENURE AND PROMOTION Kenan-Flagler Business School The University of North Carolina Adopted April 24, 1985 and amended March 30, 2009, September 12, 2011 All procedures and policies relating

### COURSE AND TEACHER SURVEYS (CATS) AT VILLANOVA UNIVERSITY

COURSE AND TEACHER SURVEYS (CATS) AT VILLANOVA UNIVERSITY A guide for new faculty members There are probably more studies of student ratings than of all of the other data used to evaluate college teaching

### Benchmarking Student Learning Outcomes using Shewhart Control Charts

Benchmarking Student Learning Outcomes using Shewhart Control Charts Steven J. Peterson, MBA, PE Weber State University Ogden, Utah This paper looks at how Shewhart control charts a statistical tool used

### Measurement and Metrics Fundamentals. SE 350 Software Process & Product Quality

Measurement and Metrics Fundamentals Lecture Objectives Provide some basic concepts of metrics Quality attribute metrics and measurements Reliability, validity, error Correlation and causation Discuss

### Households Below Average Income: An analysis of the UK income distribution: 1994/ /15

Households Below Average Income: An analysis of the UK income distribution: 1994/95-2014/15 Annual Financial year 2014/15 Published: 28 June 2016 United Kingdom The Households Below Average Income (HBAI)

### New Estimates of Design Parameters for Clustered Randomization Studies

New Estimates of Design Parameters for Clustered Randomization Studies Findings from North Carolina and Florida Z e y u X u a n d A u s t i n N i c h o l s w o r k i n g p a p e r 4 3 m a y 2 0 1 0 New

### Statistics for those who hate statistics

Statistics for those who hate statistics Dr. Ned Kock Full reference: Kock, N. (2004), Statistics for Those who Hate Statistics, Appendix of Business Process Improvement through E- Collaboration: Knowledge

### Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS)

Mode and Patient-mix Adjustment of the CAHPS Hospital Survey (HCAHPS) April 30, 2008 Abstract A randomized Mode Experiment of 27,229 discharges from 45 hospitals was used to develop adjustments for the

### Constructing a TpB Questionnaire: Conceptual and Methodological Considerations

Constructing a TpB Questionnaire: Conceptual and Methodological Considerations September, 2002 (Revised January, 2006) Icek Ajzen Brief Description of the Theory of Planned Behavior According to the theory

### Use of Pearson s Chi-Square for Testing Equality of Percentile Profiles across Multiple Populations

Open Journal of Statistics, 2015, 5, 412-420 Published Online August 2015 in SciRes. http://www.scirp.org/journal/ojs http://dx.doi.org/10.4236/ojs.2015.55043 Use of Pearson s Chi-Square for Testing Equality

### Chapter 2: Descriptive Statistics

Chapter 2: Descriptive Statistics **This chapter corresponds to chapters 2 ( Means to an End ) and 3 ( Vive la Difference ) of your book. What it is: Descriptive statistics are values that describe the

### Where Class Size Really Matters: Class Size and Student Ratings of Instructor Effectiveness

Where Class Size Really Matters: Class Size and Student Ratings of Instructor Effectiveness Kelly Bedard and Peter Kuhn* Department of Economics University of California, Santa Barbara August 2005 Abstract

### FINAL YEAR PROJECT A HAND BOOK

FINAL YEAR PROJECT A HAND BOOK The Importance of Final Year Projects The final year project is one of the most important aspects of engineering degree. To see why, let s look at a definition of engineering,

### What Is Logic? 1 Arguments

1 What Is Logic? 1 Arguments Symbolic logic is usually described as the study of the difference between valid and invalid arguments, so we begin with an explanation of this terminology. An argument is

### Promotion and Tenure Criteria for the Department of Statistics

Promotion and Tenure Criteria for the Department of Statistics The University of Georgia Guidelines for Appointment, Promotion, and Tenure (Guidelines) specify that review committees [ ] charged with implementing

### A GUIDE TO SAMPLING & STATISTICAL RELIABILITY

July 2009 Technical Note 5 A GUIDE TO SAMPLING & STATISTICAL RELIABILITY The word survey is used most often to describe a method of gathering information from a sample of individuals. This sample is usually

### PRELIMINARY ITEM STATISTICS USING POINT-BISERIAL CORRELATION AND P-VALUES

PRELIMINARY ITEM STATISTICS USING POINT-BISERIAL CORRELATION AND P-VALUES BY SEEMA VARMA, PH.D. EDUCATIONAL DATA SYSTEMS, INC. 15850 CONCORD CIRCLE, SUITE A MORGAN HILL CA 95037 WWW.EDDATA.COM Overview

### A Comparison of Graduation Rates: Transfers and Continuing Students at 4 year University Main Campuses University System of Ohio Institutions

A Comparison of Graduation Rates: Transfers and Continuing Students at 4 year University Main Campuses University System of Ohio Institutions Shoumi Mustafa * and Paula Compton Ohio Articulation and Transfer

### A possible formula to determine the percentage of candidates who should receive the new GCSE grade 9 in each subject. Tom Benton

A possible formula to determine the percentage of candidates who should receive the new GCSE grade 9 in each subject Tom Benton ARD, Research Division Cambridge Assessment Research Report 15 th April 2016

### Chapter 9. Producing Data: Experiments

Chapter 9. Producing Data: Experiments 1 Chapter 9. Producing Data: Experiments Experiments Definition. The individuals studied in an experiment are often called subjects, particularly when they are people.

### Online Student Readiness as a Predictor of Online Student Satisfaction

2011 White Paper Online Student Readiness as a Predictor of Online Student By Dr. Mac Adkins, SmarterServices; and Julie Bryant, Noel-Levitz. Online education continues to be a rapidly expanding trend

### Point and Interval Estimates

Point and Interval Estimates Suppose we want to estimate a parameter, such as p or µ, based on a finite sample of data. There are two main methods: 1. Point estimate: Summarize the sample by a single number

### Online Course Evaluations

Online Course Evaluations Collecting, Consolidating, and Leveraging Data Jackie Charonis Associate University Registrar Stanford University Office of the University Registrar Stanford University Private

### Linear Programming Notes VII Sensitivity Analysis

Linear Programming Notes VII Sensitivity Analysis 1 Introduction When you use a mathematical model to describe reality you must make approximations. The world is more complicated than the kinds of optimization

### AP Statistics 2011 Scoring Guidelines

AP Statistics 2011 Scoring Guidelines The College Board The College Board is a not-for-profit membership association whose mission is to connect students to college success and opportunity. Founded in

### Test-Retest Reliability and The Birkman Method Frank R. Larkey & Jennifer L. Knight, 2002

Test-Retest Reliability and The Birkman Method Frank R. Larkey & Jennifer L. Knight, 2002 Consultants, HR professionals, and decision makers often are asked an important question by the client concerning

### 1.16 Factors, Multiples, Prime Numbers and Divisibility

1.16 Factors, Multiples, Prime Numbers and Divisibility Factor an integer that goes into another integer exactly without any remainder. Need to be able to find them all for a particular integer it s usually

### 0.75 75% ! 3 40% 0.65 65% Percent Cards. This problem gives you the chance to: relate fractions, decimals and percents

Percent Cards This problem gives you the chance to: relate fractions, decimals and percents Mrs. Lopez makes sets of cards for her math class. All the cards in a set have the same value. Set A 3 4 0.75

### Concept Learning. Machine Learning 1

Concept Learning Inducing general functions from specific training examples is a main issue of machine learning. Concept Learning: Acquiring the definition of a general category from given sample positive

### How Are Determined? Salaries? How Are Jobs Evaluated? What Are the Different Types of Salary Increases? BROWN UNIVERSITY. Compensation Office

What Are the Different Types of Salary Increases? How Are Jobs Evaluated? How Are Salaries Determined? BROWN UNIVERSITY How Human Do Resources We Establish Competitive Compensation Office Salaries? Web

### On Victory Point scales in duplicate bridge

On Victory Point scales in duplicate bridge Preamble During 2103 the WBF introduced new VP scales for their events. I first noticed this when Ron Klinger went to press on this subject in the July ABF newsletter.

### Eight things you need to know about interpreting correlations:

Research Skills One, Correlation interpretation, Graham Hole v.1.0. Page 1 Eight things you need to know about interpreting correlations: A correlation coefficient is a single number that represents the

### Chapter 3 Introduction to Predictive Modeling: Predictive Modeling Fundamentals and Decision Trees

Chapter 3 Introduction to Predictive Modeling: Predictive Modeling Fundamentals and Decision Trees 3.1 Creating Training and Validation Data... 3-2 3.2 Constructing a Decision Tree Predictive Model...

### COMMON BELIEF 1 I don t think of my students in terms of their race or ethnicity. I am color blind when it comes to my teaching.

A PROJECT OF THE SOUTHERN POVERTY LAW CENTER.ORG Common Beliefs COMMON BELIEF 1 I don t think of my students in terms of their race or ethnicity. I am color blind when it comes to my teaching. When teachers

### EXCELLENCE IN CHRISTIAN BUSINESS EDUCATION: AN EMPIRICAL EXAMINATION. Abstract

EXCELLENCE IN CHRISTIAN BUSINESS EDUCATION: AN EMPIRICAL EXAMINATION Abstract This article examines two major issues concerning excellence in business education: (1) What are the characteristics of excellence

### WVU STUDENT EVALUATION OF INSTRUCTION REPORT OF RESULTS INTERPRETIVE GUIDE

WVU STUDENT EVALUATION OF INSTRUCTION REPORT OF RESULTS INTERPRETIVE GUIDE The Interpretive Guide is intended to assist instructors in reading and understanding the Student Evaluation of Instruction Report

### Report III: Career Stallers and Stoppers Norms and Analysis

THE LEADERSHIP ARCHITECT 2009 GLOBAL NORMS Report III: Career Stallers and Stoppers Norms and Analysis Technical Report October 2009 Guangrong Dai Kenneth P. De Meuse Table of Contents Section Page Executive

### Measurement Types of Instruments

Measurement Types of Instruments Y520 Strategies for Educational Inquiry Robert S Michael Measurement - Types of Instruments - 1 First Questions What is the stated purpose of the instrument? Is it to:

### Writing Competency Assessment: Summary Report for 2009-2014

Writing Competency Assessment: Summary Report for 2009-2014 In the Competency Area of Writing, Middle Tennessee State University (MTSU) assesses annually the following student learning outcomes: 1. Students

### Necessity vs. Opportunity Entrepreneurs in the Informal sector (Short Note)

World Bank From the SelectedWorks of Mohammad Amin November, 2009 Necessity vs. Opportunity Entrepreneurs in the Informal sector (Short Note) Mohammad Amin Available at: http://works.bepress.com/mohammad_amin/15/

### Teaching Metacognitive Learning Strategies to Individuals or Groups: A Procedure that Works! By: Saundra Y. McGuire, Ph.D. Louisiana State University

Teaching Metacognitive Learning Strategies to Individuals or Groups: A Procedure that Works! By: Saundra Y. McGuire, Ph.D. Louisiana State University Most students who enter colleges and universities have

### Descriptive Statistics: Measures of Central Tendency and Crosstabulation. 789mct_dispersion_asmp.pdf

789mct_dispersion_asmp.pdf Michael Hallstone, Ph.D. hallston@hawaii.edu Lectures 7-9: Measures of Central Tendency, Dispersion, and Assumptions Lecture 7: Descriptive Statistics: Measures of Central Tendency

### Variables and Hypotheses

Variables and Hypotheses When asked what part of marketing research presents them with the most difficulty, marketing students often reply that the statistics do. Ask this same question of professional