Individual performance assessment and motivation in agile teams

Individual performance assessment and motivation in agile teams Helen Sharp The Open University, UK helen.sharp@open.ac.uk With particular acknowledgement to Rien Sach, César França and Marian Petre Agile Research Network colleagues, and all our participants Helsinki 27 th May 2015

Overview Background to the talk Individual performance assessment in agile teams What do we know about motivation in agile teams? Motivation in software engineering Feedback in software engineering Motivation and feedback in agile teams What does (might) this tell us? Close

Agile Research Network Background

Agile Research Network (ARN) ARN works with practitioners in their place of work to: Understand the specifics of their organisation Make suggestions based on research to solve problems Discuss how to implement chosen suggestions Assess the effectiveness of solutions ARN produces outputs for: The collaborator organisation (direct feedback) A wider practitioner audience (white paper) Research outputs where possible www.agileresearchnetwork.org

Agile Research Network Background Communicating and assessing value Led to a focus on individual performance assessment Some see individual performance assessment as anti-agile Many years of motivation research: feedback is key Individual assessment is a form of feedback There is some evidence that Agile practitioners have higher motivation than non-agile

Individual performance assessment A focus on individual performance is regarded as anti-agile Focus on team, not individual Traditional approach relies on metrics but it is a reality Treat individual goals as stories and keep a prioritised backlog (Gautam, 2012) Self-evaluation and conversation (Sutherland, 2010) A common team goal, 360 reviews and minimize differences in individual compensation (Coens & Jenkins, 2002)

Individual performance assessment Recognising that there are problems with certain kinds of performance assessment is not new What do you do once an employee has reached the top? Metrics can lead to gaming the system Monetary rewards have only a short-term effect BUT Agile principle: build projects around motivated individuals Feedback on performance is key to motivated individuals So what is the place of individual performance assessment in agile teams?

Motivation of software engineers There have been several studies on the motivation of software engineers (or similar job roles) since 1970s Systematic literature review in 2009 (Beecham et al, 2009): 92 papers identified from over 2000 21 motivators (feedback, task variety, good management ) Outcomes including retention and productivity (performance) Follow-up one in 2011 (França et al, 2011): 53 additional papers 8 further motivators (team quality, creativity, professionalism )

Motivation of software engineers Intrinsic Inherent in software engineering Specific organisational/ job factors Orientate towards Motivators Software Engineer Characteristics Extrinsic Specific organisational/ job factors Influence strength of Mediates Model of Motivation Results in Outcomes Influences effectiveness of Contextual factors Individual personality Satisfaction and motivation are not the same thing Much work in motivation of software engineers has been on satisfaction (França et al, 2014)

Many motivation theories Motivation Psychology has solved the problem of motivation Motivation theories focus on the individual Recent resurgence: Teresa Amabile and Daniel Pink Researching motivation (in software engineering) is hard reliant on self-reporting: questionnaires often used transitory, and very complex everyone experiences motivation and its effects, so what I m going to say is not a surprise to anyone almost whatever I say! Feedback (knowledge of one s own performance) is a key element in four important motivation theories

Four motivation theories that Goal setting theory highlight feedback people are motivated to successfully complete challenging goals feedback is important to tell individuals how well they have completed a goal (knowledge of score) Hygiene Theory (recognition) there is a difference between what provides job satisfaction (intrinsic factors) and what avoids job dissatisfaction (extrinsic factors) feedback to provide praise for a job well done and to provide knowledge of the results of one s actions

Four motivation theories that highlight feedback Achievement Theory (Theory of Needs) key motivational needs that most people strive to satisfy: achievement, power and affiliation an individual expects to receive unambiguous feedback on the outcome of their actions Job Characteristics Theory there is a relationship between five core job characteristics and three psychological states which in turn lead to four personal and work outcomes feedback is a way to provide individuals with knowledge of the outcome of their performance

Job characteristics Psychological states Personal and work outcomes Skill variety Task identity Goal clarity Goal difficulty Job Characteristics Theory Goal acceptance Goal setting participation Task significance è Influence è Experienced meaningfulness of work Autonomy è Influences è Experienced responsibility of outcomes Feedback Feedback from the job Feedback from supervisors è Feedback on goal accomplishment Growth Influences è Experienced knowledge of actual results Need Strength + Social Need Strength High internal work motivation High quality work performance High satisfaction with work General satisfaction Satisfaction with co-worker Satisfaction with supervisors Satisfaction with pay Low absenteeism and turnover

A study of feedback characteristics and impact Multi-method over 18 months: three parts Part 1: Semi-structured interviews to scope the study Part 2: One company SQL tools for database admins Semi-structured interview (18 80 mins; 24 participants) Diary + concurrent observation (15 participants for one week) feedback instances + day summary Seven-scenarios interview (16 participants) to find out what was affected by the feedback Cambridge (UK) 200 people 30 developers

Feedback Instances (diary)

Instances of Feedback (n=76)

Example scenarios You re working with Rick (Software Engineer) on a piece of code he s having trouble with. After helping him, he thanks you and tells you what a life saver you are. You receive an email from Boris (Division Head) telling you what a brilliant job you ve done lately and how he s impressed with your performance. During a one to one meeting with Tom (Project Manager), he tells you how happy he is with your recent performance.

Scenarios were designed to reflect different feedback characteristics

How different feedback characteristics impact developers (n=16) 18 16 16 15 Attitude 14 14 12 11 12 12 12 12 Behaviour 10 10 10 10 9 9 10 8 7 8 7 8 8 Motivation 6 5 6 5 6 6 4 3 4 4 4 Productivity 2 1 1 1 2 2 2 1 0 S1 S2 S3 S4 S5 S6 S7 Job Satisfaction

P8 P7 P6 P32 P31 P30 P28 14% 14% 14% 33% 29% 29% 25% 33% 25% 33% 67% How different scenarios are perceived to impact productivity (n=16) P26 P25 P24 14% 25% 33% 43% 5 All Scenarios Positive Scenarios Negative Scenarios P22 29% 67% P20 P2 P17 67% 86% 10 P16 P15 14% 33% 2 4 6 8 10 % of scenarios reported as affecting productivity for each software engineer

A study of feedback characteristics and impact Part 3: Online scenario-based survey (157 participants) Nine scenarios varying values of source, medium, polarity Post-survey interview for terminology check (14 participants)

How valuable is feedback from different sources? feedback from a senior or experienced software engineer is more valuable than feedback from a senior manager (> 5) feedback from a senior or experienced software engineer is equally or more valuable than from their line manager (8) feedback from the company CEO is as valuable as feedback from a senior manager (55%) feedback from a line manager is as valuable as feedback from a senior manager (> 55%) the least valuable source of feedback is new software engineers the most valuable source of feedback is software engineers with knowledge and experience

How much impact does feedback from valued sources have? Participants reported that feedback from experienced and senior software engineers was more valuable than feedback from their line manager or a senior manager BUT Feedback received from their line manager had more impact than feedback from another software engineer Feedback from a software engineer had more impact than feedback from a senior manager

A model of feedback

Findings Ten reported characteristics of feedback including source, content, medium, goal/intent, setting varying source, medium and polarity changes the effect Six aspects affected including productivity, behaviour, attitude, motivation Validity and Value of feedback Validity of feedback: is it true? Value of feedback: do I care? In the empirical study, feedback was most often related to performance Feedback that is considered the most valuable does not necessarily have the most impact

Findings The value of feedback source: Sources for whom they have technical respect Sources that are above them in the company s hierarchy The value of feedback type: Technical feedback Feedback related to career progress The medium used for feedback: Some preferred to receive feedback via a specific media Receiving feedback via a non-preferred medium would alter the impact of the feedback The setting of received feedback: Some preferred to receive feedback in a specific setting

Overview Background to the talk Individual performance assessment in agile teams What do we know about motivation in agile teams? Motivation in software engineering Feedback in software engineering Motivation and feedback in agile teams What does (might) it all mean? Close

Motivation in agile teams Some evidence that agile development is more motivating Factors that experienced developers expect in a motivating environment are present in agile (Beecham et al, 2007) The ethos of change and improvement means that the environment can be modified if necessary (Tessem and Maurer, 2007) But Also evidence that factors are a little different (Melo et al, 2012) And some evidence that it does not support the need for individual recognition, clear career progression, and may weaken the developer s ability & confidence to work alone

Feedback in agile teams No other research has addressed this issue Our collaborator company practiced agile Surrounded by feedback from clients and end users Representative? This latest patch does resolve the issue that I reported. Great work <name>

What does this tell us? Individual performance assessment in agile teams Agile relies on motivated individuals Feedback (knowledge of your performance) underpins motivation, and so some kind of individual performance assessment is needed So what is appropriate? Practitioners: take account of the characteristics of feedback and how they may impact on the individual Academics: inter-disciplinary studies of motivation and feedback within the agile environment

Thank you Any questions/comments?

References Beecham, S., Baddoo, N., Hall, T., Robinson, H. and Sharp, H. (2008) Motivation in Software Engineering: A Systematic Literature Review, Information and Software Technology, 50, 860-878 Beecham, S., Sharp, H., Baddoo, N., Hall, T. and Robinson, H., (2007) Does the XP environment meet the motivational needs of the Software Developer? An Empirical Study Proceedings of Agile 2007, IEEE Computer Society Press, pp37-48 Coens, T. and M. Jenkins, Abolishing Performance Appraisals: Why They Backfire and What To Do Instead. 2002, San Francisco, CA: Berrett-Koehler França, C., Sharp, H. and Da Silva, F. (2014) Motivated software engineers are engaged and concentrated while satisfied ones are happy in Proceedings of ESEM, Torino Italy Gautam, A. Agile Performance Appraisals. 2012; Available from: https://www.scrumalliance.org/community/articles/2012/march/agile-performanceappraisals Melo, C.O., Santana, C. and Kon, F (2012) Developers motivation in agile teams, 38th Euromicro Conference on Software Engineering and Advanced Applications, 376-383 Sharp, H., Baddoo, N., Beecham, S., Hall, T. and Robinson, H.M. (2009) Models of motivation in software engineering, Information and Software Technology 51, 219-233 Sutherland, J. Agile Performance Reviews. 2006; Available from: http://jeffsutherland.com/2006/11/agile-performance-reviews.html Tessem, B. and Maurer, F. (2007) Job Satisfaction and Motivation in a Large Agile Team in Proceedings of XP 2007

A model of feedback