Mining a Change-Based Software Repository

Size: px

Start display at page:

Download "Mining a Change-Based Software Repository"

Mitchell Reeves
8 years ago
Views:

1 Mining a Change-Based Software Repository Romain Robbes Faculty of Informatics University of Lugano, Switzerland 1

2 Introduction The nature of information found in software repositories determines what we can infer from it. Researchers base themselves on popular repositories such as CVS and SubVersion despite their limitations. The number of available case studies is a determinant factor for choosing source code repositories. Versioning systems are designed to be used in a variety of contexts and hence must lower their assumptions about the objects they version. 2

The number of available case studies is a determinant factor for choosing source code repositories.

3 Introduction This paper explores the benefits and drawbacks obtained by breaking the assumption that a popular repository must be used. Proposes a change-based software repository which fetches domainspecific information from an Integrated Development Environment (IDE) In this approach, changes are stored as first-class citizens This model better matches the actual evolution of a system since we reproduce how developers actually change the system. 3

Proposes a change-based software repository which fetches domainspecific information from an Integrated

4 Limitations of Versioning Systems Versioning systems are designed to be used in a variety of contexts and hence must lower their assumptions about the objects they version. Shortcomings: They are file-based and snapshot-based 4

must lower their assumptions about the objects they

5 Limitations of Versioning Systems File-based: Ubiqutous Not Domain-specific Heavy pre-processing must be performed to raise the abstraction level beyond file and lines 5

6 Limitations of Versioning Systems Snapshot-based Stores updates as deltas between two snapshots of the system Dependant on developer commits The larger the delta, the harder it is to distinguish individual changes Small deltas (stable) are not guaranteed Most commits are related to tasks: many changes are aggregated in a single transaction (changes muggled together) 6

is to distinguish individual changes Small deltas (stable) are not guaranteed Most commits

7 Limitations of Versioning Systems Snapshot-based Impossible to determine if a change was performed at once or in a series of incremental steps in the same session. The exact sequence of changes between two versions is hard to recover; A further consequence for evolution research is these two effects reinforce each other with the practice of version sampling. 7

The exact sequence of changes between two versions is hard to recover; A further

8 Change-based Software Repositories History of a software is not viewed as a sequence of versions, but rather a sum of change operations; Change operations can not be deduced with enough precision by differencing two arbitrary program versions; Change operations are recovered by monitoring the IDE usage; 8

operations can not be deduced with enough precision by differencing two

9 Change-based Software Repositories Building a Change-based repository involves: Program representation Change operations Data retrieval 9

10 Program Representation How we represent a software system in a change-based repository to match the problem domain as closely as possible. Focus: object-oriented programs Entities stored: object-oriented constructs (classes, methods..) System representation: evolving abstract syntax trees (AST) of object oriented programs. 10

Focus: object-oriented programs Entities stored: object-oriented constructs

11 11

12 Program Representation A node a is a child of a node b if b contains a (a superclass is not the parent of a subclass, only packages are parents of classes). Nodes have properties. These properties vary with the type of node being considered, such as: Classes: name and superclass; Methods: name, return type and access modifier (public, protected or private, if the language supports them); Variables: name, type and access modifier. This abstract syntax tree represents one state of the program, i.e., one state the program went through during its evolution. 12

These properties vary with the type of node being considered, such as: Classes: name and superclass; Methods: name, return type and access

13 Change Operations Change operations represent the actual evolution of the system under study Change operations enable us to transition from one state of the evolving system to the next. Eg: adding a class to the system, removing a method, moving a class from one package to another, changing the implementation of a method, or performing a refactoring Types of Change operations: Atomic Composite 13

Eg: adding a class to the system, removing a method, moving a class from one package to another,

14 Atomic Change Operations Operations on the program s AST Creation: creates a node n for an entity of a given type t. Addition: adds a node n as a child of a given parent p. Removal: removes a node n from the children of its parent p. Property change: changes the value v of a property p of node n. Atomic change operations are executable. By iterating on the list of changes we can generate all the states the program went through during its evolution. Sufficient to model the evolution of programs. 14

Property change: changes the value v of a property p of node n. Atomic change operations are executable.

15 Composite Change Operations Atomic changes operations groupped into higher-level change operations associated with a more abstract meaning; Specific order of execution; Eg: moving a class from one package to another consists in: first removing it from the old package, and then adding it to the new package. These two change operations can be grouped in a single, higher-level move class change operation 15

package to another consists in: first removing it from the old package, and then adding it to the new

16 Composite Change Operations Developer-level actions: This set of changes constitute a unit from the developer s point of view. Eg: changing the definition of a class, adding or changing a Method; Refactorings: Refactorings are behavior-preserving code transformation. Bug fixes: Changes necessary to fix a given bug. Development sessions: Changes done during a single development session by a developer. Features: Changes necessary to build a given feature of the system. 16

Eg: changing the definition of a class, adding or changing a Method; Refactorings: Refactorings are

17 Data Retrieval The data repository we devised rely on the interaction with an Integrated Development Environment (IDE). Advantages: User activity can be processed as it happens; Time stamps can have an up to the second precision; It is possible to be notified of a variety of events to make the analysis more precise. 17

Advantages: User activity can be processed as it happens; Time stamps can have

18 SpyWare Implemented in an IDE plug-in for the Squeak Smalltalk environment SpyWare monitors the activity of the IDE user and store code modifications in a changebased repositor, alongside other useful information, such as refactorings or navigation paths through the code SpyWare also allows to exploit this information through interactive data reports and visualizations. It also can regenerate the code of the project at any given point in time, and offers dedicated code browsers for this task. 18

paths through the code SpyWare also allows to exploit this information through interactive data reports and visualizations.

19 SpyWare Limitations Developer-level actions, refactorings performed by refactoring tools, and development sessions are detected automatically; We have not yet attempted to recover other changes (manual refactorings, bug fixes, features) in an automated way, but plan to. 19

automatically; We have not yet attempted to recover other changes

20 Case Study Application of Change-Based Mining for Refactoring Detection SpyWare Project X 20

21 Case Study Limitations identified in diferente approaches: The focus is on refactorings which change the interface of classes; work best when the entities are not modified by other changes in the same session Identify refactoring based on manual search, comment logs or release notes for establishing a precision measurement 21

22 Questions What kind of refactorings were used? Are they interface-changing refactorings detected by other approaches? Were several refactorings applied to the same entities in the same session, making them harder to detect? Were refactorings performed alongside other changes, complicating their detection in the process? 22

23 Types of Refactorings Large impact on the interface of the class Little impact on class interfaces No changes to the interface of a class, only to its implementation 23

24 Multiple Refactorings During Sessions 24

25 Refactorings and Method Modification We computed how often a method was involved in several refactorings: For Project X, no method was involved in several refactorings. For SpyWare, we found 28 occurrences Most were affected only two or three times, but a few methods were affected by four or five refactorings. Sessions tended to have several methods associated with several refactorings. Refactorings such as extract method and extract expression to temporary often were applied several times to one method, as well as inlining refactorings: 15 times in total. 25

26 Refactorings and Method Modification 26

27 Results Refactorings are at the interface-level such as in Project X are easier to detect, corroborating the results found in practice by other approaches. Refactorings at the method body level such as in SpyWare are harder to detect, as they tend to come in groups and cause more body-level modifications,. Incidentally, they are not the focus of most refactoring-detection approaches. As for the usability of our repository, its accuracy and change-based model allowed us to query the data in a straightforward way, We think a change-based representation supporting composition is well suited for this type of analysis. 27

28 Change-based versus snapshot-based repositories It is Domain-specific Less information loss Changes are first-class citizens Rare case studies Memory and speed requirements Non-code repositories 28

29 Relations to Software Configuration Management It is NOT a new approach for SCM! 29

30 Conclusions By monitoring IDE usage instead of relying on the checkin/checkout model of versioning systems, a change-based repository can capture more accurate information about an evolving system, and represent its evolution as a sequence of changes rather than several successive versions. 30

31 Mining a Change-Based Software Repository Dúvidas? Obrigado! 31

E-commerce and Software Development: The Great Debate

E-commerce and Software Development: The Great Debate Towards Change-aware Development Tools Romain Robbes Michele Lanza University of Lugano Faculty of Informatics Technical Report No. 2007/06 May 2007 Abstract Software development practice still relies