Improving the productivity of software developers

Transcription

1 Improving the productivity of software developers Lecture 1 - What recommenders can be built? Gail C. Murphy University of British Columbia Tasktop 1 Laser Summer School Lecture 1

2 a recommendation system for SE is a software application that provides information items estimated to be valuable for a software engineering task in a given context Robillard, Walker and Zimmermann, Recommendation systems for software engineering, IEEE Software 27(4), 80-86,

3 a recommendation system for SE is a software application that provides information items estimated to be valuable for a software engineering task in a given context Robillard, Walker and Zimmermann, Recommendation systems for software engineering, IEEE Software 27(4), 80-86,

4 various ways to think about the space of SE recommenders a software application that provides information items estimated to be valuable for a software engineering task in a given context Robillard, Walker and Zimmermann, Recommendation systems for software engineering, IEEE Software 27(4), 80-86,

5 information spaces source code reusable software components (e.g., APIs) project history (e.g., version control) software process information (e.g., issues) interaction information web information 5

6 intent consider the range of recommenders to aid software development tasks that have been investigated consider characteristics of recommenders and their use that will underly remaining lectures representative examples not indicative of all SE recommenders that have been built! For more examples, see: Recommendation Systems in Software Engineering edited by Robillard, Maalej, Walker and Zimmermann, Springer,

7 source code 7

8 examples of recommenders based on source code quick fixes code completion refactoring (e.g., jdeodorant) program transformation some questions to consider for each: 1. how are recommendations generated? 2. how accurate are the recommendations? 3. how hard is it to determine which recommendation to take? 4. how easy is it to back out of a wrong choice? Image from: 8

9 quick fix 9

10 quick fix 10

11 quick fix some questions to consider for each: 1. how are recommendations generated? 2. how accurate are the recommendations? 3. how hard is it to determine which recommendation to take? 4. how easy is it to back out of a wrong choice? 11

12 code completion 12

13 refactoring (e.g., jdeodorant) identify Feature Envy smells in Java code generate movemethod refactorings to reduce smell! [Fokaefs et al 2007] 13

14 program transformation provide examples of similar changes made to source generate edit script (programming by demonstration) recommend similar code apply edit script LASE approach and tool [Meng et al 2013] 14

15 code recommendation summary disclaimer: accuracy are as reported but we aren t giving experimental context here so tread carefully. Selecting and undoing are subjective. recommender generation accuracy selecting undoing quick fix heuristic high but not quantified expertise required limits of undo code completion jdeodorant (refactoring) type information heuristic metric 100% often easy easy?? easy difficult LASE (transformation) AST precision often 100% expert difficult 15

16 reusable software components 16

17 examples of recommenders based on reusable software components FrUiT Strathcona CodeBroker Image from: EEWeb.com 17

18 FrUiT support the usage of frameworks! 1. extract structural relations from applications using an API 2. use association rule mining to identify structural relations that are commonly used 3. recommend all rules that mention any of the source code entities in the current file in the editor 18 [Bruch et al 2006]

19 FrUiT [Bruch et al 2006] 19

20 Strathcona find examples of API use! 1. build a db of structural relations from example code 2. if developer needs help, extract context from developer s existing code 3. query db with context 4. use heuristics to match examples (similarity of structural relations) 5. return examples as UML class diagram fragments [Holmes & Murphy, 2005] 20

21 Strathcona [Holmes & Murphy, 2005] 21

22 Codebroker help develop new methods based on similar existing ones! 1. monitor methods being written and extract words used in comments, signature and types 2. use LSA to match to existing methods modelled similarly 3. filter matches based on similarity of types 4. recommend most similar! [Ye & Fischer 2005] 22

23 Codebroker [Ye & Fischer 2005] 23

24 api recommendation summary disclaimer: accuracy are as reported but we aren t giving experimental context here so tread carefully. Selecting and undoing are subjective. recommender generation accuracy selecting undoing FrUiT associate rule mining ~85% expertise required limits of undo Strathcona heuristic case study eval difficult limits of undo Codebroker LSA high of ~35% easy difficult 24

25 project history 25

26 examples of recommenders based on project history erose expertise recommender Image from: EEWeb.com 26

27 erose recommend program elements likely to change together 1. mine the version control system to form rules of which elements commonly change together 2. when a developer changes element e, find all rules with e and suggest any other elements that usually change with e [Zimmermann et al 2004] 27

28 28

29 expertise recommender use who changed the file to determine who is an expert in the file [McDonald & Ackerman 2000] 29

30 software process information 30

31 examples of recommenders based on software process information who should fix this bug Image from 31

32 who should fix this bug? 32

33 who should fix this bug? during bug triage, a bug needs to be assigned use machine learning to determine which values in the bug fields suggest particular developers to fix the bug [Anvik & Murphy 2006] 33

34 interaction information 34

35 examples of recommenders based on interaction information Eclipse Mylyn command recommendation Image from: EEWeb.com 35

36 examples of recommenders based on interaction information Eclipse Mylyn command recommendation interaction history is! a sequential record of the! commands, artifacts, etc.! that a developer! has interacted with Image from: EEWeb.com 36

37 Eclipse Mylyn track the program elements associated with each task performed to ease recall, sharing, code recommendations, etc. 37 [Kersten & Murphy 2006]

38 38

39 command recommenders recommend new commands in a development environment that a developer is not using yet many peers are! requires interaction data from the crowd! requires interaction data from a user! many algorithms possible, several based on collaborative filtering [Murphy-Hill et al 2012] 39

40 web information 40

41 examples of recommenders based on web information Reverb 41

42 Reverb 23% of all revisits of web pages by developers are related to code Reverb recommends the web pages to revisit [Sawadsky et al 2013] 42

43 summary disclaimer: accuracy are as reported but we aren t giving experimental context here so tread carefully. Selecting and undoing are subjective. (non-code, non-api) recommender generation accuracy selecting undoing erose expertise recommender who should fix this bug Eclipse Mylyn command recommenders association rules heuristic & metrics machine learning degree-ofinterest (SVM) collaborative filtering ~26% moderate easy no data easy not applicable ~40% moderate moderate 100% recall easy easy ~20% easy easy Reverb heuristic ~40% easy easy 43

44 summary wide range of recommenders have been built, lots of room for more to be built techniques used to generate vary substantially accuracy varies greatly (can only be assessed in context of a task) what s next? common technique overview (lectures 2 & 3) how to deliver recommendations (lecture 4) how to evaluate a recommender (lecture 5) 44

45 references Anvik & Murphy. Who should fix this bug? ICSE 2006.! Bruch, Schäfer, & Mezini. FrUiT: IDE support for framework understanding. ETX 2006.! Fokaefs, Tsantalis & Chatzigeorgiou, jdeodorant: identification and removal of feature envy bad smells. ICSM 07.! Holmes & Murphy, Using structural context to recommend source code examples. ICSE 2005.! Kertsen & Murphy. Using a task context to improve programmer productivity. FSE 2006.! Meng, Kim & McKinley. LASE: locating and applying systematic edits by learning from examples. ICSE McDonald & Ackerman. Expertise recommender: a flexible recommendation system and architecture. CSCW 2000.! Murphy-Hill, Jiresal and Murphy. Improving software developers fluency by recommending development environment commands. FSE 2012.! Robillard, Walker and Zimmermann. Recommendation systems for software engineering, IEEE Software 27(4), 80-86, 2010.! Robillard, Maalej, Walker and Zimmerann (editors). Recommendation Systems in Software Engineering, Springer Sawadsky, Jiresal and Murphy. Reverb: Recommending code-related web pages. ICSE 2013.! Ye & Fischer. Reuse-conducive development environments. Automat. SE Int J., Zimmermann, Weissgerber, Diehl and Zeller. Mining version histories to guide changes. ICSE