Does a Programmer s Activity Indicate Knowledge of Code? Software Engineering Seminar 2010

Size: px

Start display at page:

Download "Does a Programmer s Activity Indicate Knowledge of Code? Software Engineering Seminar 2010"

Leslie Fox
8 years ago
Views:

1 Does a Programmer s Activity Indicate Knowledge of Code? Software Engineering Seminar 2010 Who? From? Jan Rüegg 2007 Paper By Thomas Fritz, University of British Columbia Gail C. Murphy, University of British Columbia Emily Hill, University of Delaware When? 16th March

Jan Rüegg 2007 Paper By Thomas Fritz, University of British Columbia

2 Imagine You See Something Like This... What?!? if (fabs (text->letter_spacing) > 0.1) { PangoAttrList *attrs; PangoAttribute *attr; attrs = pango_layout_get_attributes( layout->layout); if (attrs) pango_attr_list_ref (attrs); else attrs = pango_attr_list_new (); attr = pango_attr_letter_spacing_new ( text->letter_spacing * PANGO_SCALE); attr->start_index = 0; attr->end_index = -1;... 2

layout->layout); if (attrs) pango_attr_list_ref (attrs); else attrs = pango_attr_list_new ();

3 To The Rescue, Git Blame!... really? (Martin Nordholts) if (fabs (text->letter_spacing) > 0.1) (Martin Nordholts) { (Michael Natterer) PangoAttrList *attrs; (Martin Nordholts) PangoAttribute *attr; (Martin Nordholts) (Michael Natterer) attrs = pango_layout_get_attributes( (Michael Natterer) layout->layout); (Michael Natterer) if (attrs) (Michael Natterer) pango_attr_list_ref (attrs); (Michael Natterer) else (Michael Natterer) attrs = pango_attr_list_new (); (Michael Natterer) (Martin Nordholts) attr = pango_attr_letter_spacing_new ( (Martin Nordholts) text->letter_spacing * PANGO_SCALE); (Martin Nordholts) (Martin Nordholts) attr->start_index = 0; (Martin Nordholts) attr->end_index = -1; (Martin Nordholts)... 3

pango_layout_get_attributes( (Michael Natterer) layout->layout); (Michael Natterer) if (attrs) (Michael Natterer) pango_attr_list_ref (attrs); (Michael Natterer) else (Michael

4 The Basic Question How can we measure a Programmers Knowledge about source Code? Hypothesis The more Frequently and Recently a programmer has interacted with an element X, the higher is his Knowledge about X. 4

5 The Basic Question How can we measure a Programmers Knowledge about source Code? Hypothesis The more Frequently and Recently a programmer has interacted with an element X, the higher is his Knowledge about X. 4

6 Outline Example, Hypothesis Introduction Study Results Motivation DOI: Degree-Of-Interest Subjects Questionnaires Significance of DOI Problems Improvements 5

7 Outline Example, Hypothesis Introduction Study Results Motivation DOI: Degree-Of-Interest Subjects Questionnaires Significance of DOI Problems Improvements 5

8 Outline Example, Hypothesis Introduction Study Results Motivation DOI: Degree-Of-Interest Subjects Questionnaires Significance of DOI Problems Improvements 5

9 Motivation Who knows What? Important for Expertise Recommendations IDE with Adaptive Interface Recommending group members Mylyn Degree-Of-Interest Model Introduced 2005 by Mik Kersten and Gail C. Murphy Interest level in program elements Original goal: An Adaptive Eclipse Interface 6

Recommending group members Mylyn Degree-Of-Interest Model Introduced

10 Motivation Who knows What? Important for Expertise Recommendations IDE with Adaptive Interface Recommending group members Mylyn Degree-Of-Interest Model Introduced 2005 by Mik Kersten and Gail C. Murphy Interest level in program elements Original goal: An Adaptive Eclipse Interface 6

11 Mylyn DOI Model How Mylyn improves Eclipse... How does it work? Each Java element has its DOI value that Increases with each select/edit/... different operations are differently weighted Decreases over time, when not accessed Idea Elements with higher DOI: Higher up in Eclipse lists... Effect Programmer spends less time searching for important items 7

.. different operations are differently weighted Decreases over time, when not accessed

12 Mylyn DOI Model How Mylyn improves Eclipse... How does it work? Each Java element has its DOI value that Increases with each select/edit/... different operations are differently weighted Decreases over time, when not accessed Idea Elements with higher DOI: Higher up in Eclipse lists... Effect Programmer spends less time searching for important items 7

13 Empirical Study Subjects 19 Industry Java Programmers Two IBM development laboratory locations Experience between one month to twenty years Interactive Monitoring for five weeks Eclipse Mylyn project Modified to the needs of the study Show about one questionnaire per week Evaluation Analysis of the collected data Interviews with programmers about the results 8

weeks Eclipse Mylyn project Modified to the needs of the study Show about one

14 Empirical Study Subjects 19 Industry Java Programmers Two IBM development laboratory locations Experience between one month to twenty years Interactive Monitoring for five weeks Eclipse Mylyn project Modified to the needs of the study Show about one questionnaire per week Evaluation Analysis of the collected data Interviews with programmers about the results 8

15 Empirical Study Subjects 19 Industry Java Programmers Two IBM development laboratory locations Experience between one month to twenty years Interactive Monitoring for five weeks Eclipse Mylyn project Modified to the needs of the study Show about one questionnaire per week Evaluation Analysis of the collected data Interviews with programmers about the results 8

16 Questionnaires Individual Questionnaires are generated After 20000, and interactions Random questions from each of 20% highest DOI 20% lowest DOI 20% in the middle Automatically generated and verified Typical Flow of Events 1. Programmer works as usual with IDE 2. IDE monitors elements for about about a week 3. A questionnaire pops up, asking about high and low DOI elements 4. Programmer answers 9

and verified Typical Flow of Events 1. Programmer works as usual with IDE 2.

17 Questionnaires Individual Questionnaires are generated After 20000, and interactions Random questions from each of 20% highest DOI 20% lowest DOI 20% in the middle Automatically generated and verified Typical Flow of Events 1. Programmer works as usual with IDE 2. IDE monitors elements for about about a week 3. A questionnaire pops up, asking about high and low DOI elements 4. Programmer answers 9

18 Typical Questions What the Subjects were asked... 1 public class Calculator 2 { 3 public Stack<StackItem> stack; 4 5 public Calculator(double operand) 6 { 7 stack = new Stack<StackItem>(); 8 stack.push(new StackNumber(operand)); 9 } public double popoperand() 12 { 13 double result = peekoperand(); 14 stack.pop(); 15 return result; 16 }

Calculator(double operand) 6 { 7 stack = new Stack<StackItem>(); 8 stack.

19 Typical Questions What the Subjects were asked... Q1) Do you know the types of the parameters that are passed to the invocation of constructor Calculator? Q2) Can you recall one method/constructor that calls method Calculator.popOperand()? 11

passed to the invocation of constructor Calculator?

20 Typical Questions What the Subjects were asked... 1 public class Calculator 2 { 3 public Stack<StackItem> stack; 4 5 public Calculator(double operand) 6 { 7 stack = new Stack<StackItem>(); 8 stack.push(new StackNumber(operand)); 9 } public double popoperand() 12 { 13 double result = peekoperand(); 14 stack.pop(); 15 return result; 16 }

21 GUI How the Subjects answered... 13

22 GUI How the Subjects answered... 13

23 GUI How the Subjects answered... 13

24 Results Significance of DOI 14

25 Results Significance of Recency and Frequency 15

26 Results Significance of Recency and Frequency 16

27 Results Significance of Recency and Frequency 17

28 Problems Threads to Validity Number of Subjects Technical Problems with IDE Only Java/Eclipse watched "Nature" of Questions Only IDE monitored 18

29 Improvements What should be Added to the DOI Authorship Leads to more knowledge about authored program elements Code Stability Fewer code changes lead to longer knowledge about program elements Activity Short-term kinds of activity, like debugging, have a skewing effect 19

30 Summary DOI is a good Indicator for structural code Knowledge Improvements are still possible Authorship, Activity,... Outlook Results can be used to create better Expertise Recommendation tools Should be repeated with more Programmers, Languages 20

31 Summary DOI is a good Indicator for structural code Knowledge Improvements are still possible Authorship, Activity,... Outlook Results can be used to create better Expertise Recommendation tools Should be repeated with more Programmers, Languages 20

32 Typical Questions What the Subjects were asked... Q1) Can you recall the name of one class or interface that is directly extended, implemented by TYPE, or can you recall the name of one class that directly extends the type TYPE? Q2) Do you know the types of the parameters that are passed to the invocation of method/constructor METHOD? Q3) Do you know two methods that are called by method/constructor METHOD? Q4) Can you recall one method/constructor that calls method/constructor METHOD? 21

33 Improvements What should be Added to the DOI Authorship More knowledge about authored program elements. Authorship Duration More time spent creating an element more knowledge about the element. Code Stability Fewer code changes longer knowledge about it Code Patterns More code patterns easier to infer knowledge Role Important element more knowledge Task Locality More interaction with other programmers code less DOI-to-knowledge correspondence. Activity Short-term kinds of activity, like debugging, have skewing effect. Work Experience More experience of programmer better DOI-to-knowledge correspondence. 22

34 Mylyn What it looks like

Attacking information overload in software development

Attacking information overload in software development Gail Murphy University of British Columbia Tasktop Technologies This talk contains copyright pictures obtained under license. The license associated