Managing Software Debt Continued Delivery of High Values as Systems Age
Speaker - Chris Sterling Certified Scrum Trainer Managing Consultant, Agile Coach, and Architect at SolutionsIQ Consults on enterprise architecture and Agile software development practices across a spectrum of industries Founder of the International Association of Software Architects (IASA) Puget Sound chapter Supports Seattle Scrum Users Group with Kane Mar UW Extension Agile Developer Certification Program board member Email: csterling@solutionsiq.com Blog: http://chrissterling.gettingagile.com Delivering the best in Agile development, consulting, training and coaching services, SolutionsIQ offers a full range of technology services from Agile training and coaching to software delivery and talent acquisition. 2
The Problem Software gets more difficult to add features to as it ages Business expectations do not lessen as software ages Software must remain maintainable and changeable to meet needs of business over time Lack of emphasis on software quality attributes
Software Debt Creeps into our systems and leaves our organizations with software asset liabilities
Software Asset Liabilities Like-to-like migrations Limited experts capable to work on a system Long release stabilization periods Increasing duration of costly regression test phases Higher costs for people to work on legacy software
Like-to-Like Migrations It will be easy since we worked on the original version - although we understand the domain we will be fighting with new features, technology, tools, and processes We don t have any other options - Refactoring and test automation are potential alternatives to like-to-like migrations.
Limited Expertise Risk increase as expertise becomes more limited for an aging software system.
Long Stabilization Periods Release 1 Release 2 Release 3 30% 70% 45% 55% 60% 40% Feature Development - Business Value Stabilization Period - Cost of Delay
Costly Regression Test Phases Cost of Feature Implementation $5,000 $3,750 $2,500 $1,250 $0 1 2 3 4 5 6 7 8 9 10 11 12 13 Weeks Test-Driven Development and Continuous Integration Manual Regression Test Execution
Higher Costs for People Legacy software expenses increase with time COBOL programmers more expensive due to less experienced people to meet demand Specialized tools have more of a cult following with higher bill rates
Software Debt Creeps In Function Area Component 1 Component 2 Component 3 F1 0 2 0 F2 0 0 0 F3 1 1 0 F4 2 1 1 F5 0 0 0
Software Debt Creeps In Function Area Component 1 Component 2 Component 3 Component 4 F1 0 2 0 2 F2 1 0 1 4 F3 2 1 0 1 F4 2 1 1 0 F5 0 2 0 2 F6 3 4 0 4 F7 4 0 1 0 F8 2 1 0 3 F9 1 1 1 5 F10 0 4 3 1
Software Debt Creeps In Function Area Component 1 Component 2 Component 3 Component 4 Component 5 F1 1 4 0 3 0 F2 3 0 2 4 1 F3 2 1 2 1 4 F4 4 1 1 0 4 F5 0 2 0 2 0 F6 5 4 0 4 3 F7 7 0 1 0 4 F8 2 1 0 5 2 F9 3 6 1 5 2 F10 2 4 4 1 0 F11 4 2 1 2 2 F12 2 2 3 4 4
Types of Software Debt Technical Debt Quality Debt Configuration Management Debt Design Debt Platform Experience Debt
Technical Debt Issues in software implementation that will impede future development if left unresolved
* Technical Debt Technical Debt includes those internal things that you choose not to do now, but which will impede future development if left undone. This includes deferred refactoring. Technical Debt doesn t include deferred functionality, except possibly in edge cases where delivered functionality is good enough for the customer, but doesn t satisfy some standard (e.g., a UI element that isn t fully compliant with some UI standard). * Ward Cunningham - Technical Debt - http://c2.com/cgi/wiki?technicaldebt
Quality Debt A lack of quality, either technical or functional, will minimize value per feature added increasingly over time
$40,000 $30,000 $20,000 $10,000 $0 Release 1 Release 2 Total Debt Accrued Accrual of Software Debt Debt compounds over time
70.0 52.5 35.0 17.5 0 Sprints Bugs Fixed Debt Accrued (Bugs) Break/Fix Only Prolongs It Longer cycle times to fix bugs increases debt in surrounding code
A Fit Case Study Cost reduction using Fit for test automation and data conversion
Background Testing was taking 75 person hours during 2 full test runs consisting of: Comprehensive manual regression testing Data conversion and validation Cost for testing was $17,000 each iteration
Introducing Fit into Testing Process After 8 iterations team had introduced healthy amount of Fit fixtures and automated tests Reduced 70+ hour test runtime down to 6 hours which now included: Fit automated regression testing Data conversion and validation automated with Fit fixtures Reduced cost of testing each iteration from $17,000 to $7,000
Bug or Enhancement? In Bug Tracking System In Product Backlog
The Situation: Bug Came from important customer and must be fixed and deployed within 2 weeks
The Situation: Enhancement Must be in next release which is 1 month away and takes multiple user stories currently estimated as 3 weeks
The Situation: Should we work on bugs or enhancements first?
Maintain One List of Work Put all work into Product Backlog Make tradeoff decisions based on reality of the situation
Configuration Management Debt Debt in this area can be unpredictable, prone to error, and cause poor decision-making on quality attributes of a system
Traditional Source Control Management
Traditional Source Control Management Main Branch
Traditional Source Control Management Version 1 Branch Code Complete Integrate for Version 2 Main Branch
Traditional Source Control Management Version 1 Branch Code Complete Integrate for Version 2 Debt Main Branch Death March
Traditional Source Control Management Version 1 Branch Code Complete Integrate for Version 2 Debt Main Branch Death March { Debt accrues quickly within stabilization periods
Flexible Source Control Management
Flexible Source Control Management Main Branch
Flexible Source Control Management Version 1 Main Branch
Flexible Source Control Management Version 1 Version 2 Main Branch
Flexible Source Control Management Version 1 Version 2 Main Branch { Not Easy! Must have proper infrastructure to do this.
Continuous Integration Each Team Member Checks in Product Artifacts to Source Control Every Day
Scaling to Multiple Integrations
Design Debt Design decays when not attended to so put effort into maintaining your software by designing all the time
Merciless Refactoring Refactoring: a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.* Merciless: having or showing no [mercy - showing great kindness toward the distressed] Relieve your distressed code through kindness and disciplined restructuring * From http://www.refactoring.com/
Automate Testing to Support Refactoring Refactoring cannot be done effectively without automated tests surrounding code Start by creating automated test which fails If difficult to create at unit level look at automated acceptance tests from functional perspective Over time look for ways to create automated unit tests
Proper Architecture Design Provides Value Software is a business asset and our job is to produce the greatest Return on Investment (ROI) by delivering high value quickly
* Architecture Risk Themes Observed that twice as many risk themes are risks of omission as are risks of commission. That is, risk themes identify decisions or investigations that were never made rather than those that were made and could lead to undesirable consequences. No discernible relationship between articulated business and mission goals of a system and risk themes from an ATAM evaluation of that system No discernible relationship between domain of system being evaluated and risk themes associated with development of that system * From Risk Themes Discovered Through Architecture Evaluations by Len Bass, Robert Nord, William Wood, and David Zubrow
* Abuse User Stories Implement Security for User Information * From User Stories Applied presented by Mike Cohn Agile 2006
* Abuse User Stories Implement Security for User Information As a Malicious Hacker I want to steal credit card information so that I can make fraudulent charges * From User Stories Applied presented by Mike Cohn Agile 2006
Platform Experience Debt How many of you are writing Legacy Code?
How to Combat Platform Experience Debt Ignore it (I do not suggest this!) Make interface to underlying system Transfer learning of platform to more people Rewrite system on more current platform Slice functionality to more current platform over time
Architecture Team Patterns Virtual Architect Pattern Integration Team Pattern Component Shepherd Pattern Team Architect Pattern
Enterprise Planning Virtual Architect Pattern
Virtual Architect Pattern Pros Share architecture ideas and needs across teams Based on verbal communication Cons Usually singles out special Team Member role Could lead to top-down architecture decisions IT may gain extensive influence and begin to run Product Backlog prioritization for architecture needs
Integrate Features All features are implemented and integrated Feature Development Integration Team Pattern
Integration Team Pattern Pros Reduces complexity on Feature Teams Forces delivery from Integration Team instead of interface and deployment designs Cons Perpetuates specialized roles Don t always work on highest value Product Backlog items
Component Shepherd Pattern
Component Shepherd Pattern Pros Share more knowledge within organization to minimize platform experience debt Work on highest value Product Backlog items Cons Not always optimal as using individual knowledge Difficult to learn multiple systems across Teams
Team Architect Pattern Whole Team leads architecture decisions
Team Architect Pattern Pros Team owns architecture decisions Decisions are made close to implementation concerns Cons May not have appropriate experience on Team Team could get stuck on architecture decisions
Lets wrap this up...
Principles for Managing Software Debt Maintain one list of work Emphasize quality Evolve tools and infrastructure continually Improve system design always Share knowledge across the organization And most importantly, hire great people to work on your software!
Principles of Agile Software Quality The system always runs No code is written without a failing test Zero post-iteration bugs * From Flawless Iterations presented by Alex Pukinskis at Agile 2005
Continuous Integration Each Team Member Checks in Product Artifacts to Source Control Every Day
Thank you Questions and Answers
ScrumNoir.com Current Issue: Mad Dog Mary Episode 1 Previous Issues