# Eliminating Over-Confidence in Software Development Effort Estimates

Save this PDF as:

Size: px
Start display at page:

## Transcription

5 The format of the uncertainty assessment was as follows: We are almost sure that the actual effort is between: Minimum: % of most likely effort Maximum: % of most likely effort The Group B teams received the instructions: a) Estimate the most likely effort of the project, assuming the normal productivity level of your company. b) Recall what you believe is the historical estimation error distribution of similar projects in your company. Use the table below. (The table instructed the estimation team to assess the frequency (in %) of projects with: More then 100% effort overrun, % effort overrun, 25-49% effort overrun, 10-25% effort overrun, +/- 10% effort estimation error, 10-25% too high effort estimates, 26-50% too high effort estimates, and, more than 50% too high effort estimates.) c) Assess the uncertainty of the estimate as % of most likely effort (most likely effort = 100%). The only difference in the instructions was step b) for the Group B teams. The teams spent between 1 and 1.5 hours on the estimation work, i.e., less than in a normal estimation situation. From a statistical point of view, the effect of spending less than the usual amount of time on estimation should be a widening of the minimum-maximum effort intervals, i.e., that the minimum-maximum intervals would indicate a higher uncertainty in the effort estimates. The hypothesis that we tested in this study is that information about the distribution of estimation error for similar projects leads to minimum-maximum effort intervals that more accurately reflect the actual uncertainty of the effort estimates. There is no obvious test of the accuracy of individual minimum-maximum effort intervals. Almost sure does not imply that every minimum-maximum effort interval should include the actual effort. Assume, for example, that the actual effort turns out to be higher than the maximum effort in a project where the estimator was almost sure to include the effort in the minimum-maximum effort interval. The estimator may still have provided a proper uncertainty assessment. It may, for example, be the case that the project experienced an unusual amount of bad luck and that the actual effort in 99 out of 100 similar project executions would have fallen inside the interval. In other words, the optimal evaluation of minimum-maximum intervals would be based on a large set of effort minimum-maximum intervals. This type of evaluation is not possible in the current study. Instead, we had to base our evaluation of the minimum-maximum intervals on the assumption that the actual uncertainty of the team s effort estimates is close to or higher 1 than that reflected in the estimation error for previous, similar projects. For example, if among similar projects there had been a 10% frequency of projects with more than 100% effort overruns, a 90% confidence maximum value should be 200% of the estimated most likely effort. The study [11] described in Section 3, and the results reported in study [14], 1 Reflecting the limitations regarding time spent on the estimation task.

6 suggest that the empirical error distribution is, in many cases, a good indicator of future effort estimation uncertainty. Even if the assumption were not true in our experimental context, the results from the study could be useful for analyzing how the uncertainty assessments are impacted by information about previous estimation error, e.g., to analyze differences in how the error distribution information is applied in the minimum and maximum assessment situations. 4.4 The Results Table 1 shows the estimates of most likely effort, minimum effort, and maximum effort (minimum and maximum in % of most likely effort). Table 2 shows the estimation error distributions provided by the estimation teams in Group B. Table 1. Estimated, Minimum and Maximum Effort Team Group Estimate Minimum (% of Maximum (% of (work-hours) Estimate) Estimate) 1 A A A A A A A A A A Mean values A B B B B B B B B B Mean values B Statistically, a minimum of 110% of most likely effort is meaningless and should lead to an increase of most likely effort. Estimation team 16 defended the minimum value by pointing to the fact that they never had observed any similar project that used less than 110% of the estimated most likely effort. This viewpoint is, to some extent, valid when not separating estimates of most likely effort and planned effort. See the discussion in Jørgensen, M. and D.I.K. Sjøberg, Impact of effort estimates on software project work. Information and Software Technology, 2001, 43(15), p

7 Table 2. Distribution of Estimation Error of Similar Projects Teams (Group B only) Estimation Error Category Mean value >100% overrun % overrun 25-49% overrun % overrun /- 10% of error % too high estimates 24-50% too high estimates >50% too high estimates The large difference in estimates indicates that the estimation teams interpreted, and intended to provide solutions for, the requirements quite differently. This high variation in interpretation of the requirement specification also means that we should analyze the estimates carefully. In principle, we should treat the data as the estimation of 19 different projects, i.e., we should only compare the minimummaximum effort values of one team with the same team s distribution of error for similar projects. However, since the variation of effort estimates is similar in Groups A and B, we assume that the average interpretation in Group A and Group B is similar, i.e., that it is meaningful to compare the mean minimum and maximum values of the Group A and Group B estimation teams. We instructed the estimators to be almost certain to include the actual effort in their minimum-maximum intervals. It is not clear how to translate the concept of being almost certain into a statement of probability. However, we believe it is safe to state that being almost certain should correspond to there being at least an 80% probability that the actual effort is included in a minimum-maximum effort interval. We further assume a symmetric error distribution and base our interpretation of almost certain on a 10% probability of actual effort higher than the maximum effort and a 10% probability of actual effort lower than the minimum effort. These assumptions are open to discussion, but it makes no great difference to the results when changing the almost certain interpretation to, for example, a 5% probability of actual effort higher than the maximum and lower than minimum effort. Table 3 applies the above interpretation of almost certain and shows the minimum and maximum effort values we would expect if the estimation teams had based their uncertainty assessments on the historical distribution of estimation errors for similar projects. An example to illustrate the calculations is this. Estimation team 13 assessed that 10% of similar projects had more than 100% effort overrun. (see Table 2.) We therefore calculated the maximum effort to be 200% of the most likely effort. The wide uncertainty categories of the estimation error distribution, e.g., +/- 10% error, means that it is not always obvious which value to choose as a historybased minimum and maximum. Minor deviations between our interpretation of the

8 history-based and the actual minimum and maximum effort values do not, therefore, necessarily reflect little use of the estimation error distributions. Table 3. History-Based and Estimated Minimum and Maximum Teams (Group B only) Teams History-Based 90% 100% 100% 90% 90% 100% 90% 90% 90% Minimum Estimated 70% 90% 100% 90% 75% 110% 90% 90% 90% Minimum Correspondence Close Close OK OK Close OK OK OK OK minimum? History-Based 200% 200% 200% 200% 200% 175% 200% 140% 200% Maximum Estimated 120% 160% 150% 200% 150% 150% 125% 140% 120% Maximum Correspondence maximum? No No No OK No Close No OK No Table 3 suggests that all the Group B teams achieved a correspondence between the history-based and their actually estimated minimum effort. Comparing the mean minimum effort values of the two groups of estimation teams, see Table 1, shows that the teams in Group B have a minimum that is higher (mean value 88% of most likely effort) than that of the teams in Group A (mean value 71% of most likely effort). The difference in mean values, combined with the results in Table 3, suggests that the minimum values of the estimation teams in Group B were influenced by the error distribution for similar projects. For example, being aware that no similar project ever had been over-estimated by more than 10% (teams 16 and 17) seems to have made it difficult to argue that the minimum effort should be lower than 90% of most likely effort. We interpret this as meaning that the minimum values of the Group B teams were more accurate than those of the Group A teams. Surprisingly, the same impact from the historical error distribution was not present when estimating the maximum effort. Only three of the Group B teams estimated maximum effort values close to the history-based maximum. Comparing the mean maximum effort the two groups of estimation teams shows that the teams in Group B had a mean maximum value similar (mean value of 146%) to that of the teams in Group A (mean value of 150%). In other words, while the estimation teams seemed to have applied the distribution of estimation of similar projects when estimating the minimum effort, only a few of them applied the same information when estimating the maximum effort. We discuss reasons for, and threats to the validity of, these results in Section 5. 5 Discussion of the Results There are several potential reasons explaining the problems of applying historical error data when providing realistic minimum-maximum effort intervals: Conflicting goals: A potentially important reason for resistance towards providing sufficiently wide minimum-maximum effort intervals is reported in [3].

10 focused more on appearing skilled when presenting the results to the other teams and the management. This limitation does, however, not explain the difference in use of the historical distribution of estimation error when deriving the minimum and the maximum effort. Taking into consideration the potential explanations and major limitations of our study, we believe that our most robust finding is the difference in the use of historical information when providing minimum and maximum effort, i.e., that it seems to be more difficult to accept the relevance of historical estimation performance when assessing maximum (worst case) compared with minimum (best case) use of effort. A potential consequence of that finding is described in Section 6. 6 Conclusion and Further Work The experiment described in this paper is a first step towards better processes for effort estimation uncertainty. Our results indicate that in the attempt to ensure history-based maximum effort values, it is not sufficient to be aware of the estimation error distribution of similar projects. Based on the identification of potential reasons for not applying the estimation error distribution we recommend that the uncertainty assessment process be extended with the following two elements: More detailed instructions on how to apply the distribution of previous estimation error to determine minimum and maximum effort for a given confidence level. The presence of an uncertainty assessment process facilitator who ensures that the distribution of previous estimation error is properly applied. The facilitator should be statistically trained in uncertainty distributions and require that deviations from the history-based minimum and maximum are based on sound argumentation. We intend to introduce and evaluate the extended history-based uncertainty assessment process in a software organization. References 1. Connolly, T. and D. Dean, Decomposed versus holistic estimates of effort required for software writing tasks. Management Science, (7): p Jørgensen, M. and K.H. Teigen. Uncertainty Intervals versus Interval Uncertainty: An Alternative Method for Eliciting Effort Prediction Intervals in Software Development Projects. In International conference on Project Management (ProMAC) Singapore: p Jørgensen, M., K.H. Teigen, and K. Moløkken, Better Sure than Safe? Overconfidence in Judgment Based Software Development Effort Prediction Intervals. To appear in: Journal of System and Software, Jørgensen, M., Top-Down and Bottom-Up Expert Estimation of Software Development Effort, (1): p Alpert, M. and H. Raiffa, A progress report on the training of probability assessors, in Judgment under uncertainty: Heuristics and biases, A. Tversky, Editor. 1982, Cambridge University Press: Cambridge. p

11 6. Kahnemann, D., P. Slovic, and A. Tversky, Judgement under uncertainty: Heuristics and biases. 1982: Cambridge University Press. 7. Tversky, A. and D. Kahneman, Judgment under uncertainty: Heuristics and biases. Science, : p Yaniv, I. and D.P. Foster, Precision and accuracy of judgmental estimation. Journal of behavioral decision making, : p Lichtenstein, S. and B. Fischhoff, Do those who know more also know more about how much they know? Organizational Behaviour and Human Decision Processes., (2): p Arkes, H.R., Overconfidence in judgmental forecasting, in Principles of forecasting: A handbook for researchers and practitioners, J.S. Armstrong, Editor. 2001, Kluwer Academic Publishers: Boston. p Kahneman, D. and A. Tversky, Variants of uncertainty, in Judgment under uncertainty: Heuristics and biases, D. Kahneman, P. Slovic, and A. Tversky, Editors. 1982, Cambridge University Press: Cambridge, United Kingdom. p Griffin, D. and R. Buehler, Frequency, probability, and prediction: Easy solutions to cognitive illusions? Cognitive Psychology, (1): p Kahneman, D. and D. Lovallo, Timid choices and bold forecasts: A cognitive perspective on risk taking. Management Science, (1): p Jørgensen, M. and D.I.K. Sjøberg, An effort prediction interval approach based on the empirical distribution of previous estimation accuracy. Journal of Information and Software Technology, (3): p Jørgensen, M. and D.I.K. Sjøberg, Impact of effort estimates on software project work. Information and Software Technology, (15): p Klein, W.M. and Z. Kunda, Exaggerated self-assessments and the preference for controllable risks. Organizational behavior and human decision processes, (3): p

### How should we (not) design empirical studies of software development?

How should we (not) design empirical studies of software development? Magne Jørgensen & Stein Grimstad Simula Research Laboratory Oslo, Norway magnej@simula.no By thinking constantly about it The Software

### A Preliminary Checklist for Software Cost Management

A Preliminary Checklist for Software Cost Management Magne Jørgensen, and Kjetil Moløkken Simula Research Laboratory Abstract: This paper presents a process framework and a preliminary checklist for software

### How Large Are Software Cost Overruns? A Review of the 1994 CHAOS Report

How Large Are Software Cost Overruns? A Review of the 1994 CHAOS Report Magne Jørgensen 1 and Kjetil Moløkken Simula Research Laboratory Abstract: The Standish Group reported in 1994 that the average cost

### Sequence effects in software development effort estimation. Stein Grimstad (steingr@simula.no) Magne Jørgensen (magnej@simula.no)

Sequence effects in software development effort estimation Stein Grimstad (steingr@simula.no) Magne Jørgensen (magnej@simula.no) Accurate estimates of work-effort are essential to enable successful software

### Assessing Uncertainty of Software Development Effort Estimates: Learning from Outcome Feedback

Assessing Uncertainty of Software Development Effort Estimates: Learning from Outcome Feedback Tanja Gruschke 1, Simula Research Laboratory and University of Oslo Magne Jørgensen 2, Simula Research Laboratory

### Expert Estimation of Web-Development Projects: Are Software Professionals in Technical Roles More Optimistic Than Those in Non-Technical Roles?

Expert Estimation of Web-Development Projects: Are Software Professionals in Technical Roles More Optimistic Than Those in Non-Technical Roles? Kjetil Moløkken, kjetilmo@simula.no, +47 67 82 83 14 Magne

### COMMENTARIES 2A (ON CASE 2, THE NEW OXFORD COUNTY COALITION: FOUR MAINE TOWNS TACKLE A PUBLIC HEALTH MYSTERY )

MAX BAZERMAN Harvard Business School Morgan Hall 133, Soldiers Field Road Boston, MA 02163 Tel: 617-495-6429 Email: mbazer@hbs.harvard.edu COMMENTARIES 2A (ON CASE 2, THE NEW OXFORD COUNTY COALITION: FOUR

### Software Process Improvement and Human Judgement Heuristics

Scandinavian Journal of Information Systems Volume 13 Issue 1 Article 2 1-1-2001 Software Process Improvement and Human Judgement Heuristics Magne Jørgensen University of Oslo, Norway, Magnejorgensen@emailaddressnotknown

### A review of studies on expert estimation of software development effort

The Journal of Systems and Software 70 (2004) 37 60 www.elsevier.com/locate/jss A review of studies on expert estimation of software development effort M. Jørgensen * Simula Research Laboratory, P.O. Box

### A Review of Studies on Expert Estimation of Software Development Effort

A Review of Studies on Expert Estimation of Software Development Effort M. JØRGENSEN magne.jorgensen@simula.no Simula Research Laboratory, P.O.Box 134, 1325 Lysaker, Norway Abstract This paper provides

### Forecasting of Software Development Work Effort: Evidence on Expert Judgment and Formal Models

Forecasting of Software Development Work Effort: Evidence on Expert Judgment and Formal Models Magne Jørgensen Simula Research Laboratory P.O.Box 134 NO-1325 LYSAKER, NORWAY Email: magnej@simula.no Fax:

### Describe what is meant by a placebo Contrast the double-blind procedure with the single-blind procedure Review the structure for organizing a memo

Readings: Ha and Ha Textbook - Chapters 1 8 Appendix D & E (online) Plous - Chapters 10, 11, 12 and 14 Chapter 10: The Representativeness Heuristic Chapter 11: The Availability Heuristic Chapter 12: Probability

### IN [1], we found, through a review of surveys on software

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 30, NO. 12, DECEMBER 2004 993 Reasons for Software Effort Estimation Error: Impact of Respondent Role, Information Collection Approach, and Data Analysis

### A Review of Surveys on Software Effort Estimation

A Review of Surveys on Software Effort Estimation Kjetil Moløkken and Magne Jørgensen Simula Research Laboratory P.O. Box 134 NO-1325 Lysaker NORWAY Tel.: +47 67828200) {kjetilmo,magnej}@simula.no Abstract

### Reducing overconfidence in forecasting with repeated judgement elicitation

Reducing overconfidence in forecasting with repeated judgement elicitation Matthew Brian Welsh (matthew.welsh@adelaide.edu.au) Steve. H. Begg (steve.begg@adelaide.edu.au) Australian School of Petroleum,

### The Contextualization of Project Management Practice and Best Practice

The Contextualization of Project Management Practice and Best Practice Claude Besner PhD, University of Quebec at Montreal Brian Hobbs PhD, University of Quebec at Montreal Abstract This research aims

### Anchoring Bias in Forecast Information Sharing in a Supply Chain

Anchoring Bias in Forecast Information Sharing in a Supply Chain N. Karthikram R.K. Rajagopal G. Janarthanan RKAmit Abstract This paper investigates the behavioral aspects of decision making of retailers,

### The Effects of Request Formats on Judgment-based Effort Estimation

The Effects of Request Formats on Judgment-based Effort Estimation Magne Jørgensen Simula Research Laboratory & Department of Informatics, University of Oslo, Norway Torleif Halkjelsvik Department of Psychology,

### The replication of empirical research is a critical

RESEARCH TECHNICAL COMMENT PSYCHOLOGY Comment on Estimating the reproducibility of psychological science Daniel T. Gilbert, 1 * Gary King, 1 Stephen Pettigrew, 1 Timothy D. Wilson 2 A paper from the Open

### Reinsurance Reserving and the Insurance Cycle. Mike Rozema, SVP, Swiss Re CLRS 2011

Reinsurance Reserving and the Insurance Cycle Mike Rozema, SVP, Swiss Re CLRS 2011 Agenda Scope and Introduction The Underwriting Cycle Data from Schedule P The Winner's Curse Cognitive Biases Optimism,

### Market Efficiency and Behavioral Finance. Chapter 12

Market Efficiency and Behavioral Finance Chapter 12 Market Efficiency if stock prices reflect firm performance, should we be able to predict them? if prices were to be predictable, that would create the

### The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: A Randomized Controlled Field Experiment

The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: A Randomized Controlled Field Experiment Magne Jørgensen 1,2, Stein Grimstad 1 1 Simula Research Laboratory

### Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities

Another Look at Sensitivity of Bayesian Networks to Imprecise Probabilities Oscar Kipersztok Mathematics and Computing Technology Phantom Works, The Boeing Company P.O.Box 3707, MC: 7L-44 Seattle, WA 98124

### Better Selection of Software Providers Through Trialsourcing

Better Selection of Software Providers Through Trialsourcing Magne Jørgensen Simula Research Laboratory & University of Oslo magnej@simula.no Abstract: In this article we show that differences between

### Software Effort Estimation: Unstructured Group Discussion as a Method to Reduce Individual Biases

Software Effort Estimation: Unstructured Group Discussion as a Method to Reduce Individual Biases Kjetil Moløkken i, kjetilmo@simula.no Magne Jørgensen ii, magnej@simula.no Abstract: The effort of software

### Inaccurate and Biased: Cost-Benefit Analyses of Transport Infrastructure Projects

Inaccurate and Biased: Cost-Benefit Analyses of Transport Infrastructure Projects Presentation at the workshop Critically Examining the Current Approaches to Transport Appraisal Petter Næss Professor in

### The Independence Referendum: Predicting the Outcome 1. David N.F. Bell

The Independence Referendum: Predicting the Outcome 1 David N.F. Bell Division of Economics Stirling Management School, University of Stirling, IZA, Bonn and ESRC Centre for Population Change 1 Thanks

### Unit effects in software project effort estimation:

Unit effects in software project effort estimation: Work-hours gives lower effort estimates than workdays Magne Jørgensen Simula Research Laboratory magnej@simula.no ABSTRACT Context: Software development

### Illinois Professional Teaching Standards

Illinois Professional Teaching Standards Preamble: We believe that all students have the potential to learn rigorous content and achieve high standards. A well-educated citizenry is essential for maintaining

IS MORE INFORMATION BETTER? THE EFFECT OF TRADERS IRRATIONAL BEHAVIOR ON AN ARTIFICIAL STOCK MARKET Wei T. Yue Alok R. Chaturvedi Shailendra Mehta Krannert Graduate School of Management Purdue University

### Information Security Regulation: Compliance and Non-Compliance. Dr. Tommy Tranvik,, NRCCL, University of Oslo

Information Security Regulation: Compliance and Non-Compliance Dr. Tommy Tranvik,, NRCCL, University of Oslo The Research Project Compliance: the implementation and use of information security regulations

### #55 De Bondt, W.F.M (1993). Betting on trends: Intuitive forecasts of financial risk and. return. A Critical Appraisal.

101347365 1 #55 De Bondt, W.F.M (1993). Betting on trends: Intuitive forecasts of financial risk and return. A Critical Appraisal Emily Giblin University of Newcastle Article sourced from a bibliography

### What is a Problem? Types of Problems. EXP Dr. Stanny Spring Problem Solving, Reasoning, & Decision Making

Problem Solving, Reasoning, & Decision Making Dr. Director Center for University Teaching, Learning, and Assessment Associate Professor School of Psychological and Behavioral Sciences EXP 4507 Spring 2012

### Introduction to Statistics for Computer Science Projects

Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I

### Software Metrics & Software Metrology. Alain Abran. Chapter 4 Quantification and Measurement are Not the Same!

Software Metrics & Software Metrology Alain Abran Chapter 4 Quantification and Measurement are Not the Same! 1 Agenda This chapter covers: The difference between a number & an analysis model. The Measurement

### Central Bank of Ireland Guidelines on Preparing for Solvency II Pre-application for Internal Models

2013 Central Bank of Ireland Guidelines on Preparing for Solvency II Pre-application for Internal Models 1 Contents 1 Context... 1 2 General... 2 3 Guidelines on Pre-application for Internal Models...

### Better forecasting for large capital projects

7 Better forecasting for large capital projects Project proposals often overestimate benefits and underestimate costs. Here s why and what you can do about it. Bent Flyvbjerg, Massimo Garbuio, and Dan

### American Statistical Association

American Statistical Association Promoting the Practice and Profession of Statistics ASA Statement on Using Value-Added Models for Educational Assessment April 8, 2014 Executive Summary Many states and

### Theories of Learning. Ana Savic. VISER, RICUM Belgrade, May

Theories of Learning Ana Savic VISER, RICUM Belgrade, May 24 2012. Objectives Objectives: consider different theories of learning identify several principles of learning see how individual differences

### Simplifying Bayesian Inference

Simplifying Bayesian Inference Stefan Krauß, Laura Martignon & Ulrich Hoffrage Max Planck Institute For Human Development Lentzeallee 94, 14195 Berlin-Dahlem Probability theory can be used to model inference

### Judgmental overconfidence, self-monitoring and trading performance in an experimental financial market

Judgmental overconfidence, self-monitoring and trading performance in an experimental financial market Bruno Biais, Denis Hilton, Karine Mazurier and Sébastien Pouget April 2, 2004 Abstract We measure

### Project Management for Scientists

Project Management for Scientists Cost & Schedule Estimates Christoph U. Keller, c.u.keller@uu.nl Sterrekundig Instituut Utrecht Utrecht University www.astro.uu.nl/~keller Outline Resource Assignments

Linking Risk Management to Business Strategy, Processes, Operations and Reporting Financial Management Institute of Canada February 17 th, 2010 KPMG LLP Agenda 1. Leading Practice Risk Management Principles

### Designing to Motivate: Motivational Techniques to Incorporate in E-Learning Experiences. Charles B. Hodges Virginia Tech. Abstract

The Journal of Interactive Online Learning Volume 2, Number 3, Winter 2004 www.ncolr.org ISSN: 1541-4914 Designing to Motivate: Motivational Techniques to Incorporate in E-Learning Experiences Charles

### A framing effect is usually said to occur when equivalent descriptions of a

FRAMING EFFECTS A framing effect is usually said to occur when equivalent descriptions of a decision problem lead to systematically different decisions. Framing has been a major topic of research in the

### Online Ensembles for Financial Trading

Online Ensembles for Financial Trading Jorge Barbosa 1 and Luis Torgo 2 1 MADSAD/FEP, University of Porto, R. Dr. Roberto Frias, 4200-464 Porto, Portugal jorgebarbosa@iol.pt 2 LIACC-FEP, University of

### Low-Volatility Investing: Expect the Unexpected

WHITE PAPER October 2014 For professional investors Low-Volatility Investing: Expect the Unexpected David Blitz, PhD Pim van Vliet, PhD Low-Volatility Investing: Expect the Unexpected 1 Expect the unexpected

### Banking on a Bad Bet: Probability Matching in Risky Choice Is Linked to Expectation Generation

Research Report Banking on a Bad Bet: Probability Matching in Risky Choice Is Linked to Expectation Generation Psychological Science 22(6) 77 711 The Author(s) 11 Reprints and permission: sagepub.com/journalspermissions.nav

### Abstract. Keywords: controlled experimentation, single-subject experiments

N = 1: an alternative for software engineering research? Warren Harrison Department of Computer Science Portland State University Portland, OR 97207-0751 USA 503-725-3108 warren@cs.pdx.edu Abstract The

### Determining Consensus Values in Interlaboratory Comparisons and Proficiency Testing

Determining Consensus Values in Interlaboratory Comparisons and Proficiency Testing 1. Abstract Speaker/Author: Dr. Henrik S. Nielsen HN Metrology Consulting, Inc. HN Proficiency Testing, Inc. Indianapolis,

### Behavioural Economics: Introduction to Behavioral Game Theory and Game Experiments. University of Oxford, Michaelmas Term 2013.

Behavioural Economics: Introduction to Behavioral Game Theory and Game Experiments University of Oxford, Michaelmas Term 2013 and Asia Summer Institute in Behavioral Economics National University of Singapore,

### Introducing Social Psychology

Introducing Social Psychology Theories and Methods in Social Psychology 27 Feb 2012, Banu Cingöz Ulu What is social psychology? A field within psychology that strives to understand the social dynamics

### WHEN YOU CONSULT A STATISTICIAN... WHAT TO EXPECT

WHEN YOU CONSULT A STATISTICIAN... WHAT TO EXPECT SECTION ON STATISTICAL CONSULTING AMERICAN STATISTICAL ASSOCIATION 2003 When you consult a statistician, you enlist the help of a professional who is particularly

### Can computers help overcome limitations in human decision making?

Can computers help overcome limitations in human decision making? A John Maule Leeds University Business School University of Leeds Leeds LS2 9JT UK ABSTRACT Motivation: The article evaluates computer

Advanced Risk Management to Improve Cost and Schedule Performance on EPC Projects Risk-Management Best Practices from Nuclear Experience Palisade Risk Conference, 2014 Sola Talabi PhD MBA MSc BSc RMP Project

### I.e., the return per dollar from investing in the shares from time 0 to time 1,

XVII. SECURITY PRICING AND SECURITY ANALYSIS IN AN EFFICIENT MARKET Consider the following somewhat simplified description of a typical analyst-investor's actions in making an investment decision. First,

### Chapter 12: Problem solving Chapter 13: Reasoning and Decision Making

File = E:\p355\Goldstein\ch12-13.study.q's.docm 1 John Miyamoto Email: jmiyamot@u.washington.edu http://faculty.washington.edu/jmiyamot/p355/p355-set.htm Psych 355: Introduction to Cognitive Psychology

### A Survey on the Characteristics of Projects with Success in Delivering Client Benefits 1

A Survey on the Characteristics of Projects with Success in Delivering Client Benefits 1 Magne Jørgensen Simula Research Laboratory, Oslo, Norway magnej@simula.no Abstract Context: Not being able to produce

### Chapter 12: Decision Making, Creativity, and Ethics

Chapter 12: Decision Making, Creativity, and Ethics Organizational Behaviour 5 th Canadian Edition Langton / Robbins / Judge Copyright 2010 Pearson Education Canada 12-1 Chapter Outline How Should Decisions

### ARE WE ALL LESS RISKY AND MORE SKILLFUL FELLOW DRIVERS? *

Acta Psychologica 47 (1981) 143-148 0 North-Holland Publishing Company ARE WE ALL LESS RISKY AND MORE SKILLFUL FELLOW DRIVERS? * THAN OUR Ola SVENSON Department of Psychology, University of Stockholm,

### CCPM: TOC Based Project Management Technique

CCPM: TOC Based Project Management Technique Prof. P.M. Chawan, Ganesh P. Gaikwad, Prashant S. Gosavi M. Tech, Computer Engineering, VJTI, Mumbai. Abstract In this paper, we are presenting the drawbacks

### Introduction to Software Engineering. 9. Project Management

Introduction to Software Engineering 9. Project Management Roadmap > Risk management > Scoping and estimation > Planning and scheduling > Dealing with delays > Staffing, directing, teamwork 2 Literature

### Appraisal of Trust Degree among Innovation Team Members Based on Analytic Hierarchy Process

1094 Appraisal of Trust Degree among Innovation Team Members Based on Analytic Hierarchy Process Gao Xia Department of Business Administration, ZhengZhou Institute of Aeronautical Industry Management,

### Journal Article Reporting Standards (JARS)

APPENDIX Journal Article Reporting Standards (JARS), Meta-Analysis Reporting Standards (MARS), and Flow of Participants Through Each Stage of an Experiment or Quasi-Experiment 245 Journal Article Reporting

### Experts, Amateurs, and Real Estate: An Anchoring-and- Adjustment Perspective on Property Pricing Decisions

ORGANIZATIONAL BEHAVIOR AND HUMAN DECISION PROCESSES 39, 84-97 (1987) Experts, Amateurs, and Real Estate: An Anchoring-and- Adjustment Perspective on Property Pricing Decisions GREGORY B. NORTHCFCAFTANDMARGARET

### Behavior of Small Equity Investors in Pakistan s Stock Exchanges

Middle-East Journal of Scientific Research 18 (3): 321-326, 2013 ISSN 1990-9233 IDOSI Publications, 2013 DOI: 10.5829/idosi.mejsr.2013.18.3.11684 Behavior of Small Equity Investors in Pakistan s Stock

### The capstone course as a means for teaching agile software development through project-based learning

World Transactions on Engineering and Technology Education Vol.13, No.3, 2015 2015 WIETE The capstone course as a means for teaching agile software development through project-based learning Viljan Mahnič

### Expert Judgments: Financial Analysts vs. Weather Forecasters

Expert Judgments: Financial Analysts vs. Weather Forecasters Tadeusz Tyszka & Piotr Zielonka Centre for Market Psychology of Leon Kozminski Academy of Entrepreneurship and Management Warszawa (Poland)

### Software effort estimation terminology: The tower of Babel

Information and Software Technology 48 (2006) 302 310 www.elsevier.com/locate/infsof Software effort estimation terminology: The tower of Babel Stein Grimstad*, Magne Jørgensen, Kjetil Moløkken-Østvold

### ACH 1.1 : A Tool for Analyzing Competing Hypotheses Technical Description for Version 1.1

ACH 1.1 : A Tool for Analyzing Competing Hypotheses Technical Description for Version 1.1 By PARC AI 3 Team with Richards Heuer Lance Good, Jeff Shrager, Mark Stefik, Peter Pirolli, & Stuart Card ACH 1.1

### ANNEX 2: Assessment of the 7 points agreed by WATCH as meriting attention (cover paper, paragraph 9, bullet points) by Andy Darnton, HSE

ANNEX 2: Assessment of the 7 points agreed by WATCH as meriting attention (cover paper, paragraph 9, bullet points) by Andy Darnton, HSE The 7 issues to be addressed outlined in paragraph 9 of the cover

### Chapter 13. Prejudice: Causes and Cures

Chapter 13 Prejudice: Causes and Cures Prejudice Prejudice is ubiquitous; it affects all of us -- majority group members as well as minority group members. Prejudice Prejudice is dangerous, fostering negative

### Administrative Support Professionals Competency Framework. The Centre for Learning and Development

Administrative Support Professionals Competency Framework The Centre for Learning and Development Table of Contents 01. Acknowledgements...3 02. Introduction...4 03. Background...5 04. Competency Assessment

### Project : Summary of Results

Project 178692: Summary of Results The results from the project are organized in three scientific papers: We here make brief summaries of the papers: Myopic Investment Management (Eriksen and Kvaløy; forthcoming

### Amajor benefit of Monte-Carlo schedule analysis is to

2005 AACE International Transactions RISK.10 The Benefits of Monte- Carlo Schedule Analysis Mr. Jason Verschoor, P.Eng. Amajor benefit of Monte-Carlo schedule analysis is to expose underlying risks to

### Asymmetric Distributions in Fan Charts: International Experience and an Application to Brazil

Asymmetric Distributions in Fan Charts: International Experience and an Application to Brazil One of the operational consequences for the BCB of the adoption of an inflation-targeting regime in Brazil

### On the Reality of Cognitive Illusions

Psychological Review 1996, Vol. 103, No. 3, 582-591 Copyright 1996 by the American Psychological Association, Inc. 0033-295X/96/S3.00 THEORETICAL NOTES On the Reality of Cognitive Illusions Daniel Kahneman

### National Commission for Academic Accreditation & Assessment. Standards for Quality Assurance and Accreditation of Higher Education Programs

National Commission for Academic Accreditation & Assessment Standards for Quality Assurance and Accreditation of Higher Education Programs November 2009 Standards for Quality Assurance and Accreditation

### Insurance Demand and Prospect Theory. by Ulrich Schmidt

Insurance Demand and Prospect Theory by Ulrich Schmidt No. 1750 January 2012 Kiel Institute for the World Economy, Hindenburgufer 66, 24105 Kiel, Germany Kiel Working Paper No. 1750 January 2012 Insurance

### Chapter 12: Problem solving Chapter 13: Reasoning and Decision Making

File = D:\p355\goldstein\ch12-13.study.q's.docm 1 John Miyamoto Email: jmiyamot@u.washington.edu http://faculty.washington.edu/jmiyamot/p355/p355-set.htm Psych 355: Introduction to Cognitive Psychology

### Project Control with ProTrack

Project Control with ProTrack 1.0 (last update: 11/11/08) 1 Project Control with ProTrack by Mario Vanhoucke Professor at Ghent University and Vlerick Leuven Gent Management School (Belgium) Partner OR-AS

### On the trend recognition and forecasting ability. of professional traders

On the trend recognition and forecasting ability of professional traders Markus Glaser, Thomas Langer, and Martin Weber April 15, 2003 Abstract Empirical research documents that temporary trends in stock

### P3M3 Portfolio Management Self-Assessment

Procurement Programmes & Projects P3M3 v2.1 Self-Assessment Instructions and Questionnaire P3M3 Portfolio Management Self-Assessment P3M3 is a registered trade mark of AXELOS Limited Contents Introduction

### Undergraduate Psychology Major Learning Goals and Outcomes i

Undergraduate Psychology Major Learning Goals and Outcomes i Goal 1: Knowledge Base of Psychology Demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical

### ISAAC LEVI JAAKKO HINTIKKA

ISAAC LEVI JAAKKO HINTIKKA I agree with Jaakko Hintikka that the so-called conjunction fallacy of Kahneman and Tversky is no fallacy. I prefer a different explanation of the mistake made these authors

### RJF ASSET MANAGEMENTINC

EARNINGS HEURISTICS WORKING FOR THE "RIGHT REASONS" RJF ASSET MANAGEMENTINC December 1994 411 Borel, Suite 402 San Mateo, CA 94404 (415) 572-8731 FAX (415) 572-8769 2 EARNINGS HEURISTICS WORKING FOR THE

### Change Management Practitioner Competencies

1 change-management-institute.com Change Management Institute 2008 Reviewed 2010, 2012 Change Management Practitioner Competencies The Change Management Practitioner competency model sets an independent

### James E. Bartlett, II is Assistant Professor, Department of Business Education and Office Administration, Ball State University, Muncie, Indiana.

Organizational Research: Determining Appropriate Sample Size in Survey Research James E. Bartlett, II Joe W. Kotrlik Chadwick C. Higgins The determination of sample size is a common task for many organizational

### Precision and Accuracy of Judgmental Estimation

Journal of Behavioral Decision Making, Vol. 10, 21±32 (1997) Precision and Accuracy of Judgmental Estimation ILAN YANIV Hebrew University, Jerusalem, Israel DEAN P. FOSTER Wharton School, University of

### CONTENTS OF DAY 2. II. Why Random Sampling is Important 9 A myth, an urban legend, and the real reason NOTES FOR SUMMER STATISTICS INSTITUTE COURSE

1 2 CONTENTS OF DAY 2 I. More Precise Definition of Simple Random Sample 3 Connection with independent random variables 3 Problems with small populations 8 II. Why Random Sampling is Important 9 A myth,

### EFFECTIVE STRATEGIC PLANNING IN MODERN INFORMATION AGE ORGANIZATIONS

EFFECTIVE STRATEGIC PLANNING IN MODERN INFORMATION AGE ORGANIZATIONS Cezar Vasilescu and Aura Codreanu Abstract: The field of strategic management has offered a variety of frameworks and concepts during

### NORGES BANK PAPERS. Weaknesses in NIBOR NO. 2 2014

NORGES BANK PAPERS Weaknesses in NIBOR NO. 2 2014 Norges Bank Papers no. 2 2014 Norges Bank Address: Bankplassen 2 Postal address Postboks 1179 Sentrum, 0107 Oslo Phone: +47 22316000 Fax: +47 22413105

"Exploring Solutions to the Fund Assortment Problem" Summary of Research Supported by FINRA Investor Education Foundation Grant # 2005-080 Retirement investment decisions faced by employees are becoming

### Measurement Information Model

mcgarry02.qxd 9/7/01 1:27 PM Page 13 2 Information Model This chapter describes one of the fundamental measurement concepts of Practical Software, the Information Model. The Information Model provides

### ESTIMATES TOO HIGH, LOW? CHECK THE CALENDAR

ESTIMATES TOO HIGH, LOW? CHECK THE CALENDAR FUNDAMENTAL RESEARCH FEBRUARY 11, 2013 ESTIMATES TOO HIGH, LOW? CHECK THE CALENDAR AN ANALYSIS OF THE LIFECYCLE OF QUARTERLY EARNINGS ESTIMATES HIGHLIGHTS The

### Centre for Learning and Development

The Leadership and Management Development Strategy was developed to endorse learning and development opportunities to strengthen the leadership and management capacity of the Newfoundland and Labrador

### Improving the Predictability of the CapEx Portfolio

Improving the Predictability of the CapEx Portfolio Spring 2009 Westney Consulting Group, Inc. www.westney.com If the whipsaw of product price and project cost over the past 12 months has taught anything,

### A CASE STUDY ON SOFTWARE PROJECT MANAGEMENT IN INDUSTRY EXPERIENCES AND CONCLUSIONS

A CASE STUDY ON SOFTWARE PROJECT MANAGEMENT IN INDUSTRY EXPERIENCES AND CONCLUSIONS P. Mandl-Striegnitz 1, H. Lichter 2 1 Software Engineering Group, University of Stuttgart 2 Department of Computer Science,