Agile, Rails, and the Cloud Why your customer should care about Agile, Rails and the Cloud Ian McFarland, VP Technology ian@pivotallabs.com
Or... Why companies can t afford to ignore the efficiencies of modern development approaches Ian McFarland, VP Technology ian@pivotallabs.com
You
Most or all of you are Rails Developers, Agile Developers, Craftsman Developers, and run a rails shop You Most or all of you have current deployments On hardware not owned by your company...managed by someone else like Engine Yard? On EC2? On other Clouds?
What were you doing before? Java Developer C/C++ Developer PHP Developer COBOL Developer Nothing: Rails/Ruby is my first development platform Goat Farmer? Stock Broker? Shoe Salesman?
Pivotal Labs We make Pivotal Tracker We ve been doing agile for 10 years, first in SmallTalk, then in Java, now Rails and JavaScript We ve grown from 20 to 70 since starting the Rails Practice 4 years ago We do enablement, but mostly implementation
Why do we build software Because it s cool Because it s fun Because it s exciting Because we love the challenge Because it beats the crap out of coal mining Because employment is a nice thing to have
Why do Companies build software To make money To save money To manage risk To satisfy business and government requirements
Building Business Value The revolution in Software is about one thing: Building business value as cheaply and efficiently as possible
So what s expensive about software Developers Defects Deployment/Operations Code Maintenance Change
Three Trends Agile Rails The Cloud
What is it? What s in it for Us What s in it for Them Agile Rails Cloud Cleaner, more flexible code; more fun coding; code is more maintainable More powerful, more expressive, less grunt work, more gets done with less effort, more fun. Easy Scaling, capacity planning, setup, no pagers Fewer defects, more predictable delivery, more transparency, business determines what s built Less effort = lower cost Less effort = shorter time to market Less effort = lower TCO Lower TCO, No initial investment, lower operating costs, shorter deployment cycle, no sunk cost
It s the Economy, Stupid Budgets are smaller The stakes are higher Departments have to do more with less money Failure, though always an option, is more catastrophic
So what s this Agile stuff?
Manifesto for Agile Software Development We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Title Individuals and interactions over processes and tools Goes Here documentation Bulleted Working software Text over comprehensive Customer collaboration over Here contract negotiation Text Goes Bulleted Responding to change over following a plan That is, while there is value in the items on Bulleted Text Goes right, the we value the items on Here the left more. Kent Beck James Grenning Robert C. Martin Mike Beedle Jim Highsmith Steve Mellor Arie van Bennekum Andrew Hunt Ken Schwaber Alistair Cockburn Ron Jeffries Jeff Sutherland Ward Cunningham Jon Kern Dave Thomas Martin Fowler Brian Marick 2001, the above authors this declaration may be freely copied in any form, but only in its entirety through this notice.
That s nice... How do we do that? Business Driven: Requirements come from business stakeholders Iterative Development, with Short Iterations Test/Behavior Driven Development Continuous Integration, Continuous Releasability Pair Programming Productive Work Environment
Business Driven Requirements come from business stakeholders One designated Customer is empowered to make decisions Priorities are set by that Customer The Customer can change priorities on anything unstarted The Customer accepts the work in fine-grained increments The Customer is intimately aware of progress, and projected completion dates Closing the feedback loop is critical Accept Reject
TDD/BDD Good tests tell us when we ve met the customer requirements They tell us when we ve broken behavior that used to work They tell us when we haven t, so we can refactor with impunity Writing tests first keeps us from overdesigning/doing things we don t need to do Writing tests first forces cleaner API design, because we have to call into our own code in order to write it It leads to looser coupling and encourages higher cohesion Good developer testing keeps the cost of change constant
Iterative Development Because the customer is seeing the work on a daily basis, the feedback cycle is short This keeps the cost of change low, by preventing unnecessary work It allows for new insights to be gained from the work we ve already completed, and for those insights to be incorporated into our new code Iterations are as short as we can make them
Continuous Integration, Continuous Releasability Knowing when things break is critical to reducing the cost of fixing defects. Keep the build status visible, so you can fix it quickly A broken build is a stop the line event Continuous releasability does not mean you release every day. It just means you can. Releases can be distracting, so weigh the cost of a release against the value it adds to the business.
Pair Programming Do we really have to pair? Isn t Pairing Slower? I don t like pairing. My teammates smell funny. I don t want to look stupid.
Do we really have to pair? Yes, you do....but only if you want to be efficient This is one of the least-used practices, and one of the most important. And stop whining! You do it already when you get stuck on something.
What do developers really do all day? Coding Reading web pages about coding Stuck on some problem, unsure of: The right approach What the API for that object was How SQL indexes are selected How bind(this) works in JavaScript Checking email Checking news, stock price, staring blankly into space
How does pairing help? 80/20 rule: You don t get stuck, so you spend your time on the most interesting part of the code. As you eliminate the grunt work (thanks Rails) more of the work requires real thinking, and design You talk through design, and refine before you code. You learn from your pair, everything from design and testing techniques to SQL, CSS, and JavaScript tips. Focus matters: Your pair keeps you paying attention, and can smooth over disruptions
How does pairing help? More developers in a smaller space How many truly independent fronts are there in your codebase on which you can make progress? New team members: You re really productive the first hour, not marginally productive starting two weeks in They have a local sherpa to tell them how the code they re working on actually works. Knowledge Silos: Your bus number approaches
Productive Workspace Open Workspace Colocated Developers and Customer Consistent Pairing Stations One big screen, 2 keyboards (we use 24 imacs) No laptops on the floor Visible build monitors Everyone can see the backlog in Tracker Breakfast, snacks and beverages on hand Space for interruptions away from the workspace
Title
Title
Title
Title
Title
Title
Title
Title
Why Sustainability Matters Predictable delivery is at a premium Tired developers introduce bugs Developer retention is still important Good developers are never easy to come by Ramp-up is still expensive Team changes expose companies to risk
Why Developer Happiness is Important to the Business Leading Indicator: Developer Happiness strongly correlated to Developer Productivity Grunt Work = Money Wasted Retention Matters Happy workers are more focused
Rails: Why should businesses care? Convention over Configuration Fewer lines of code More developer comprehensibility Much greater developer productivity More extensibility (DSLs & metaprogramming made easy) Agile baked into the libraries and the culture...oh and by the way, it s Web based!
Rails compared to Java: From an anecdotal perspective When we started doing Rails, we couldn t hire Rails developers, because there weren t any So we converted Java developers into Rails developers The same developers were about 2x as productive in business terms after one month Once they actually got good at Rails, they were about 4x as productive
Rails compared to Java: From an anecdotal perspective Some people report as much as 5-10x I suspect that s because for them Rails includes Agile Large client: 10 Pivots, 10 client, vs. 50 offshore Pivotal Labs had already been doing Agile for 10 years, so we already got the productivity benefit.
The Cloud The Cloud means you don t have to buy hardware anymore... ever. And scale is available on demand... if you architect (or refactor) for it. Lets you stay focused on where you provide differentiating value. (Unless you re in the Ops business or have a sick fascination with pagers, you shouldn t do your own ops.) [See also: Rails, Agile] But... enterprise business rules don t always allow for this, at least for some systems, at least not yet.
How the Cloud changes the Game Obvious Quantitative Wins: Almost zero provisioning time Scalable on demand No risk, no capex, low opex specialization breeds efficiency
How the Cloud changes the Game Less Obvious Qualitative Wins: Spin up test environments that match production Realistic load testing environments much simpler Clone production data when debugging Snapshot production to isolate production issues
So how did we do? Developers Defects Deployment/Ops Code Maintenance Change
The ARC Tool Chain Pivotal Tracker RubyMine or TextMate Engine Yard Cloud RSpec/TestUnit, Selenium Cruise.rb GitHub New Relic RPM Scout HopToad Zendesk Lighthouse
Differentiating Value
Differentiating Value: Questions to ask yourself Does the work I m doing materially relate to my company s core line of business? Does it provide competitive advantage? Does it make users happier or more productive? Does it reduce operating cost, or improve efficiency? Does it eliminate drudge work, so people can concentrate on higher-value work? Could it be done better by an existing library or product? If I didn t do this work, would it matter?
Some things that don t pass that test Writing accessors (unless you need to change their behavior) Writing CRUD Writing Inner Classes Writing features customers don t really need anymore Writing provisioning, monitoring and backup scripts Writing your own libraries, when existing ones will do...unless you re in that line of business.
Interesting Problems ARC is about getting to where developers can work on the interesting, differentiating problems, at each layer
If giants are around, by all means, stand on their shoulders!
Classic economic theory, based as it is on an inadequate theory of human motivation, could be revolutionized by accepting the reality of higher human needs, including the impulse to self actualization and the love for the highest values. Abraham Maslow
Risk and how to manage it Transparency exposes risk earlier Agile planning is reality-based. Tools like Tracker keep you honest. (With your customer and yourself.) Cloud-based projects get you live faster, without sunk cost, and respond to growth with linear cost.
The Dirty Secrets Ruby and Rails still are big resource hogs The Rails developer community (more than the software) isn t quite Enterprise Ready The Rails Ecosystem has a lot of niches that need filling The Enterprise isn t really ready for the Cloud
Rails Performance Not as good as Java or C++ For the most part, good enough. especially for the enterprise Slow is usually because of design and architecture, not execution speed And this will change with market pressure
Rails... Title Ready for the Enterprise?
This is more what they had in mind... Title
Is the Enterprise Cloud-Ready In general no... Lots of CIOs really want everything running on the hardware they spent all this money on. There are (sometimes) legitimate concerns about security There are sometimes legislative restrictions But vertical providers are showing the way People run CRM in the cloud with tools like SalesForce People run HR in the cloud with tools like Workday People run financials in the cloud with tools like Intacct
Is the Cloud Enterprise-Ready It s getting there Many entire businesses are running on the cloud today The obstacles are being overcome Tools are improving around managing, provisioning, security, monitoring, etc. As the business need is articulated, solutions are built The economics will push it the rest of the way
Déjà-Vu all over again In case you don t remember... These are exactly the same reasons they said Java would never work in the Enterprise.
Economics always wins The benefits already outweigh the costs One proof is that many Fortune 500 companies are seeing benefits in Agile, Rails, and Cloud deployments The problems are shallow We ve been down this road before
Enterprise is Easy Internet Scale is the Hard thing Enterprise is more fussy than hard Data tends to be deeper instead of wider Concurrency is in the 10s, not millions You re integrating with lots of fussy endpoints Enterprise is more about orchestrating services than about building massive throughput Hmm... This calls for a scripting language
Rewrites are scary... Don t do them lightly. Keep what s already working well, and iterate from there. Replace systems from the orchestration layer down Test legacy systems from the edges in Replace verticals when integrating with them is more expensive Less Baby, More Bathwater! Refactor your way to happiness, one system at a time.
It s not about the technology Businesses don t care about technology Technology is a means to an end
Think like a Business Think about what s motivating your company s technology decisions. It s probably not about doing something innovative, but more about making sure things get done cheaply, quickly, and well.
Which leaves us to ask... How can companies afford not to be doing Agile, Rails and Cloud
Q&A More Resources http://pivotallabs.com/talks Contact Info @imf or ian@pivotallabs.com Photo and Text Credits Agile Manifesto: http://agilemanifesto.org/ Flickr Photos used under Creative Commons: Cloud by akakumo, Risk Factory by kyz, Laundry by T.M.O.F., Tank engine by Corvair Owner