The Real Challenges of Configuration Management

The Real Challenges of Configuration Management McCabe & Associates

Table of Contents The Real Challenges of CM 3 Introduction 3 Parallel Development 3 Maintaining Multiple Releases 3 Rapid Development with Many Incremental Deliveries 4 How Traditional CM Does Not Address the Challenges 5 Underlying Technology 5 File Deltas 5 Manages File s Not Software Versions 5 Difficult To Remove Historical Changes Without Rewinding 6 Difficult To Migrate Changes Between Parallel Variants Of Files 6 Difficult to Manage Parallel Versions 7 The TRUEchange Difference 8 Change Based CM 8 The TRUEchange Repository 8 Managing Change using ChangeSets 8 Order Independent Change Storage 9 Managing Change using ChangeSets 10 More Powerful Branching 10 Simple Project Loading 10 Lightweight Distribution Model 10 Advanced Security 11 Phased Development Model 12 Other Data File Types 12 ConfigDB 13 Conclusions 14

The Real Challenges of CM Introduction Effective Configuration Management is a critical factor in the process of building and maintaining quality software. All organisations require the ability to manage change in their software projects. The better their CM solution supports the activities of development, change and release, the more able the organisation will be to meet the ever increasing challenges in software development. All CM products are able to do the simple things, including managing file versions and branching/merging streams of development. In this respect there is little to choose between the major CM offerings. Consequently many organisations attempt to differentiate the available CM solutions by looking for fancy integration or neat tricks. However, there are several challenges in software development that are truly difficult. In fact, some things are so difficult that many organisations have become sensitive to the fact that CM cannot easily solve them. Some organisations start their search for a new CM solution with a set of initial requirements, but quickly discover that the requirements cannot be easily met without a heavy overhead of manual intervention. In response they modify their desired process to fit with the tools. This approach of modifying the process can suit some branches of software development quite satisfactorily. On the other hand, some branches of software development have an absolute need to address the more advanced challenges of CM. These difficult challenges include: Parallel Development This is the requirement to manage more than one stream of development where different modifications are made on different branches to the same base software components. At the simplest level, virtually all software development requires parallel working at some time due to the need to modify software to create a new development release, whilst still maintaining a released version which will only be subject to bug fixes and minor enhancements. In this scenario it is frequently important to see if bug fixes in the released version can be applied to the similar code elements being developed for the next version. Version 4.0 - Under D l t Bug Fix Version 3.2 - Under Maintenance In some environments many parallel streams of development are required at the same time due to the need to produce many variants of the same base software. In this circumstance, individual fixes and enhancements will frequently need to be migrated between the parallel streams, without merging all the changes. Maintaining Multiple Releases Although similar to parallel development, maintaining multiple releases represents a distinct challenge. Organisations often have to modify software to customise it to individual client needs, Copyright McCabe & Associates 2004 All rights reserved Page 3

or alternatively the nature of the software product might require or encourage the client to make their own modifications. This means that clients are not always able to upgrade to the latest version of a product without considerable effort to reapply the bespoke changes. The net effect of this is that the software supplier needs to maintain a range of delivered software releases each of which is in use at a subset of the entire client base. Version 4.0 Bug Fix Bug Version 4.1 Bug Fix Version 5.0 Bug Fix Enhancement Version 5.1a Within this release model, the ability to migrate bug fixes and enhancements between the common code elements in parallel versions becomes imperative. The integrity of each release stream is paramount, however, and the technique of merging entire branches would be inappropriate; not all modifications on a branch need to be migrated because they could be specific to that branch only. Rapid Development with Many Incremental Deliveries Some software development programs cannot afford the luxury of a six-monthly delivery schedule where many fixes, upgrades and enhancements are collected together in a single version which is then released to clients. The growth of ecommerce, and the requirement to provide online interactive access to services, has also meant that many software projects consist of many incremental deliveries where perhaps every day equates to a new version of software. Frequently, an enhancement or modification made several releases ago becomes unnecessary or undesirable, and the requirement is to remove this change without removing interim changes to mask the change out from the current version. V2.1.0 V2.1.1 V2.1.2 V2.1.3 V2.2.0 V2.2.1 V2.2.2 V3.0.0 Need to Undo Specific Historical Change Copyright McCabe & Associates 2004 All rights reserved Page 4

How Traditional CM Does Not Address the Challenges Underlying Technology Most of the currently available CM solutions have evolved from historical products. This is probably a function of the high reliance that users have on CM and their requirement for robust, well-tested and mature technology. This has led to a situation where the underlying architecture used at the core of most modern CM solutions is based on a principle identified early in the development of CM, namely that Configuration Management is about managing versions of files. At first glance this may appear to be the only possible way to look at Configuration Management. Perhaps this common approach can be traced to the first (non-mainframe) commercial CM solutions available on UNIX platforms. To understand UNIX it is necessary to accept that everything is a file including programs, data, directories and the operating system itself, and it is therefore a logical step to conclude that the problem of Configuration Management (or Version Control) is about managing files and changes made to files. File Deltas When managing incremental revisions of a single file, it is clear that storing the entire file contents for every revision would be very wasteful because many lines would be similar or identical. This led to the development of file deltas a technique allowing the storage of the difference between incremental revisions of a single file: Delta Delta Delta Delta Delta Delta Delta 1 2 3 4 5 6 7 8 Some CM products use reverse file deltas meaning that the latest revision of a file is used as the baseline and historical revisions can be accessed by applying reverse deltas. This is usually to improve performance which would be consistently degraded if the final revision of a file could only be accessed by applying an ever increasing number of deltas to an increasingly distant original base file. Whether file deltas are used in the forward or reverse direction, they are fundamentally the same. All of the currently available non-mainframe CM products adhere to the same principle that the CM problem is file related and are actually implemented using file deltas. Manages File s Not Software Versions Conceptually, when modifying software to fix bugs or implement an enhancement, people do not make changes to individual files but rather make a set of changes to several files. The file based CM model means that a modification to a software project (made across several files) is actually stored as many individual changes (file deltas) to the altered files. In some CM products there is no mechanism to associate the individual file changes to the high level modification except through a manually entered common textual description for each delta. In others, a technique of change grouping allows the management of the high level modification abstracted from the actual mechanism of storing differences. However, as outlined below, file deltas have a number of inherent limitations meaning that CM products using file deltas are not best suited to solve the difficult challenges within Configuration Copyright McCabe & Associates 2004 All rights reserved Page 5

Management. Difficult To Remove Historical Changes Without Rewinding File deltas actually describe the per line differences between incremental revisions of a file. This means that a file delta is dependent upon the exact revision of a file to which it applies; trying to apply a file delta to a different revision of the same file is not always possible. As a result of this, it is natural to visually represent file deltas as jigsaw pieces that fit one on the other as follows: Final File Baseline File File Deltas Applied If a file delta (excluding the final one) needed to be removed, perhaps because the associated software enhancement or fix had to be undone, the diagram indicates that the file deltas before and after the removed file delta would not fit together: Although illustrated graphically, the analogy is an accurate one for real file deltas within CM products. The only mechanism available to undo a historical delta is to rewind the file state by peeling deltas off, and then reapply the necessary deltas to recreate a new final file revision. In itself this is not a big problem, and most CM systems provide automated functionality to overcome this issue and remove historical file deltas in the case of individual files. We will see however, that this inherent limitation means that CM products reliant on file deltas do not properly address the difficult challenges in CM when scaled up to projects. Difficult To Migrate Changes Between Parallel Variants Of Files Consider now the issue of maintaining parallel versions of the same file. Originally there would be a single file at a particular revision level, from which a branch would be made allowing changes to be applied on each branch independently. The end result would be two variant files with different changes applied, both of which share the same parenthood. The following diagram represents this graphically: Changes Made in Variant 1 File Variant 1 File Variant 2 Changes Made in Variant 2 Common Root Let us consider the requirement to migrate an individual change from one branch to the other, in other words trying to apply a file delta to a different revision of the same file on another branch. Copyright McCabe & Associates 2004 All rights reserved Page 6

Using the graphical analogy this is equivalent to selecting a jigsaw piece from Branch 1 and attempting to join it to the end of Branch 2 as follows: File Variant 1 Changes Made in Variant 1 File Variant 2 Changes Made in Variant 2 Common Root Change Cannot be Applied Because File is Now Different The inherent limitation that file deltas are dependent upon a specific file revision now creates a situation where it is extremely difficult to migrate the change. The only way to resolve this is to try to compare the variant files and do a line by line determination whether a change can be applied. This technique is difficult to automate and can cause contention where the desired outcome cannot be consistently determined except by manual inspection. Difficult to Manage Parallel Versions A software project usually consists of many files, possibly into the hundreds or thousands. Modifications applied to a software project are made up of many changes across many files. The limitations of the file delta architecture when managing single files are magnified when managing complete project versions containing many files. Some basic CM requirements become difficult to address using file deltas when scaled up to full software projects. Functionality not inherently available in file based CM includes: Migrating a project modification between parallel versions of the same project. Removing or unwinding a specific historical modification made to a project. Managing all changes made to a file across all parallel versions in one go (to consolidate parallel activity and ensure that effort is not being duplicated). Despite its limitations, the file based CM model is well suited to managing file modifications, and the fact that all CM products use a file delta model demonstrate support of the major functionality needed to manage versions and implement Configuration Management. The underlying technology behind file deltas, however, means that some tasks are intrinsically difficult to implement. This can lead to the undesirable situation where a client has to modify their process to align more closely with the functionality of the CM system they are using. Copyright McCabe & Associates 2004 All rights reserved Page 7

The TRUEchange Difference McCabe TRUEchange is the only non-mainframe CM solution not to use a file based model or file deltas. The TRUEchange product uses a change based architecture where a modification to a software project is managed as a single distinct entity even though it may consist of changes made across many files, and where the actual file changes are not stored relative to a specific file version. Its unique change based architecture means that TRUEchange is naturally suited to managing highly parallel developments, migrating individual changes between common but variant files in parallel versions and rapid development where changes need to be unwound frequently. TRUEchange is a mature, well-tested and proven product and has been commercially available for over ten years with a large installed user base across many vertical markets throughout the world. Change Based CM TRUEchange does not use file deltas to store changes to files but rather groups the modifications in container objects called ChangeSets. ChangeSets encapsulate the changes made to all the files involved in an enhancement or fix within a software project. Grouping changes in this way is an extremely natural way to manage the change process (although it may be an unfamiliar concept to those only used to using a file based model). In itself, though, this is not unique functionality because several file based CM products provide change grouping linking changes made across more than one file. If change grouping were the only benefit of using ChangeSets then they would provide little additional functionality over a file based model. The major benefit of the TRUEchange architecture is that the changes are not stored relative to a specific file version. In simple terms, this means that changes can be reapplied in any order to achieve the same final result. At first sight this statement appears to be counter-intuitive and possibly unworkable; the following sections provide a basic overview of how the innovative TRUEchange architecture achieves this. The TRUEchange Repository TRUEchange uses a robust high performance repository to store the revision history for components under Configuration Management. The repository uses an object database to store items under change control and manages all aspects of version control. This includes the items in a project, the version tree, and the ChangeSets applied to each version. Managing Change using ChangeSets New files inserted into a TRUEchange repository are stored as a sequence of line entries in a line information table. From the point of initial insertion, all future versions, revisions and variants of the original file refer to the same base entries in the database. The line information table always grows in size and existing entries are never deleted: Copyright McCabe & Associates 2004 All rights reserved Page 8

/* fred.c */ main() { printf( hello world ); exit(0); } Baseline File Check In Order Status Line Contents 1 Active /* fred.c */ 2 Active main() 3 Active { 4 Active printf( hello world ); 5 Active exit(0); 6 Active } TrueChange Line Information Table A ChangeSet manages the changes made to a set of files and each ChangeSet has a unique identifier. When a ChangeSet (comprising many changes across one or more files) is committed to the repository (called checking in) the lines in the modified files are compared one at a time against the previous matching line entries in the line information table. During comparison, lines can be unchanged, deleted, inserted or modified. If lines have been deleted then they are simply marked as deleted by this ChangeSet in the line information table; the line entries for the deleted lines are not removed. If new lines have been inserted then they are appended to the end of the table, marked as inserted by this ChangeSet and the numbering sequence for the lines in the file is modified to describe where they fit. Finally, if lines have been modified then the modified lines are appended to the end of the table, marked as applied only in this ChangeSet, the line sequencing is modified to describe where they fit, and the unmodified original lines are marked as inactive for this ChangeSet. main() { printf( hello everyone ); a = 1; exit(0); } Modified File Check In Order Status Line Contents 1 Inactive CSet1 /* fred.c */ 2 Active main() 3 Active { 4 Inactive Cset1 printf( hello world ); 7 Active exit(0); 8 Active } 5 Active CSet1 printf( hello everyone ); 6 Active CSet1 a = 1; TrueChange Line Information Table Now, by scanning the line information table in line sequence order, and by applying the required ChangeSet, the new file can be read from the table. At the same time, scanning the table using the modified line sequencing (even after the changes made to it) but considering the specific ChangeSet as not applied, the original unmodified file can be read. In this way, a single table representation stores both variants of the file, and different file versions can be read from the table simply by turning flags on and off. The technique scales to any number of parallel branches, changes and files. Order Independent Change Storage Immediately it is clear that all changes are applied against the same line information table and therefore are order independent. In other words if more than one ChangeSet needs to be applied, scanning through the table in the correct line sequence with the desired ChangeSet applied will output the required file. There is no concept of applying the first change followed by the second change. Copyright McCabe & Associates 2004 All rights reserved Page 9

Another major benefit of this approach is the minimal degradation in performance when reading a modified file with any number of changes applied as opposed to the original unmodified file reading any file simply requires a scan of the line information table with ChangeSets applied or ignored. Managing Change using ChangeSets Each TRUEchange user checks out files in order to change them. This process automatically creates a new open ChangeSet, into which the user can then add or delete files as the modification progresses. Each ChangeSet is uniquely identified and has a descriptive field to describe the nature of the modification being made. The ability to add files to or remove files from an open ChangeSet is completely natural, and closely resembles the physical activity of the developer or user making a modification. Once a user is happy that the modification is complete, they can check in the ChangeSet, which will commit the changes to the repository, storing them against the specific project version within which the user has been working. Given that all modifications are applied to the same base table, it now becomes a simple task to apply these modifications to any parallel versions. More Powerful Branching As a result of these fundamental architectural differences, TRUEchange users quickly become confident that they can manage parallel versions very easily creating a branch in TRUEchange is always instantaneous because no files need to be moved, copied or delta-ed, and no extra disk space is required. This encourages a development culture where branching is used more often to manage parallel activity as opposed to trying to manage many parallel activities within a single version a more common approach when using file based CM. One major reason for the relative lack of branching under file based CM is that these products often maintain the baseline file (to which reverse file deltas will be applied) as the latest file version in a branch. Consequently, creating a branch leads to the duplication of every file in the project, immediately requiring double the disk space and possibly taking considerable time to complete. Conflict resolution where the same line has been changed differently in parallel branches is extremely straightforward using ChangeSets as it simply requires the user to select which is the correct line from the list of currently active duplicate lines. Alternatively the user can modify lines or the rest of the file in a new way to resolve the conflict, creating a new conflict resolution ChangeSet to store the changes in the process. By default, conflict resolution is invoked when duplicate changes are detected on the same line, but if necessary it can be modified to apply when a common file has been modified by more than one change (even when line changes do not overlap). Simple Project Loading The ChangeSet architecture allows for the simple loading of files into the TRUEchange repository. All CM products can load the initial version of software, but TRUEchange can scan many sequential versions of a software project to detect difference, and create a version tree containing all versions of all files in a project very quickly. Loading version history into file based CM can be much more time consuming. Lightweight Distribution Model TRUEchange utilises a lightweight TCP/IP based distribution model. This means that dedicated and expensive centralised hardware is not necessary to support a TRUEchange installation. TRUEchange servers (repositories) can be placed on any supported Copyright McCabe & Associates 2004 All rights reserved Page 10

server platform (Windows NT/2000, UNIX, VMS etc.) and immediately become available to all TRUEchange clients. This enables TRUEchange repositories to be created within a project development area and to be managed from a central location. TRUEchange repositories are platform independent, and can be accessed from a TRUEchange client running on any supported client platform (Windows 95/98/2000/NT, UNIX, VMS etc.) regardless of the server platform. Code is supplied to the client in a platform independent format ensuring that files are readable on all clients. This means that the server platform and the client platform are fully independent, and that servers and clients can be distributed according to user requirements. TRUEchange uses a secure TCP/IP protocol managed as part of the security package, but the protocol is similar to the standard FTP protocol and is equally fast. TRUEchange is provided with standalone graphical interfaces on Windows and UNIX clients as well as being integrated to several popular development environments using the Microsoft SCM API. It also provides a command line interface allowing for the easy access of CM features from the UNIX or Windows shell. This enables the easy scripting of regular tasks or activities that need to be automated. Advanced Security TRUEchange provides powerful and configurable security to control access to and modification of code under Configuration Management. Every TRUEchange user is required to login, and can optionally be forced to supply a login password. Four standard levels of security are provided which are Developer, Project Leader, Administrator and Owner, but any number of new security levels can be created according to requirements. Each of the four standard security levels grants or denies access to a selection of 48 standard commands available in TRUEchange, and the selection can be modified to include or exclude commands from any level upon demand. New security levels can be created granting or denying access to a custom set of commands as required. Browse Checkin Checkout Branch Merge Migrate Changes Promote Phase Create project Checkpoint Remove Changes Etc. Developer Project Leader Administrator Author Example Matrix of Security Levels and TRUEchange Commands Security levels are assigned to each user dependent upon the repository being accessed, the individual project within a repository or even the current development phase (see below for a description of phased development). Consequently, security levels are dynamically assigned to users as they switch between TRUEchange repositories, projects or phases. Copyright McCabe & Associates 2004 All rights reserved Page 11

Phased Development Model TRUEchange allows up to six customisable phases within a project version tree; phases are typically used to signal that a software project has moved from the jurisdiction of one group into another. An example of phased development would be to create code, test and release as the three phases for a software project. The software production process would start in the code phase, during which members of the development team can access and modify source code. Specific individuals in the development team would then be allowed to promote phase from the code to the test phase when a specific product line needs to be passed to the testing team, and the code would now only be accessible or modifiable by the testing team. Other individuals in the test team would then be allowed to promote phase from the test phase to the release phase once testing changes have been completed, and the code would now be under the jurisdiction of the release/maintenance team. Code_v2.1.0 Promote Test_v2.1.0 Promote Release_v2.1.0 At any time, if a new code stream needs to be created to allow developers (who can only access code in the code phase) to modify and continue to develop the next version of software, then a new branch could be created and reassigned to the code phase. Code_v2.1.0 Promote Test_v2.1.0 Promote Release_v2.1.0 Branch Code_v2.1.1 Project versions can be promoted to any phase provided the security level allows it (perhaps bypassing intermediate phases), but a specific promotion model can be enforced to ensure a standard development process. Phase based security is a unique feature of McCabe TRUEchange. Other Data File Types TRUEchange supports all file types, both ASCII character based files and binary file types. Line based ChangeSet application is of course only applicable for ASCII files; binary files are stored in the repository as is and can be extracted in the standard way. Copyright McCabe & Associates 2004 All rights reserved Page 12

ConfigDB A TRUEchange installation could comprise several distributed repositories providing Configuration Management for different development groups or projects. It is often necessary, however, to create a software release in a central area made up of components managed in the distributed repositories. The TRUEchange ConfigDB provides this functionality by creating a version tree of software releases. The ConfigDB stores information about which specific versions of which projects from which repositories comprise a specific software release, and where that release needs to be placed. The ConfigDB supports the concept of a version hierarchy of versions, storing specific release configurations, while allowing new configurations to inherit from existing ones. An example of this would be a software product1 release 2.1 comprising project1 version 1.27, project2 version 3.1 and the latest version of project3. The ConfigDB would store this information and allow the recreation of the specific release at any later date. Copyright McCabe & Associates 2004 All rights reserved Page 13

Conclusions The TRUEchange change based architecture provides superior support for parallel development activities. Creating branches is simpler and quicker than in a file based model. Individual changes made on one branch can easily be applied to other parallel branches by selecting the appropriate ChangeSets and making them active. Conflict resolution is straightforward. A TRUEchange user working with a file on one branch can easily see all the parallel changes made to the same file across all branches (without having to apply changes to his local copy of the file), enabling the changes made elsewhere to be understood and duplication removed. The TRUEchange phased development model in addition to the phase dependent security ensures that a project development process can be easily modelled and enforced within TRUEchange. The TRUEchange distributed, platform independent architecture allows for a lightweight yet robust deployment model, and the TRUEchange ConfigDB allows for the production of software releases from distributed TRUEchange repositories. Copyright McCabe & Associates 2004 All rights reserved Page 14