Agile Development c/w Continuous Integration and Testing.... JOHN PAYNE... SENIOR DATA AND SERVICES DEVELOPER UNIVERSITY OF ESSEX... DevCon1 12 APRIL 2013
Overview of today s talk Brief background: the UK Data Archive and the UK Data Service Historical and Current Structure of ADM Why use Agile? Why use CIT? How we use Agile and CIT together Where we are going with Agile and CIT? Questions?
The UK Data Archive and the UK Data Service based at the University of Essex since 1967 curator of the largest collection of digital data in the social sciences and humanities in the UK see data-archive.ac.uk for more details makes these available via the new UK Data Service UK Data Service also provides value-added services for UK Census data, government surveys and beyond UK Data Service includes Universities of Essex, Manchester (Mimas, CCSR), Leeds, Southampton, Edinburgh (Edina) and University College London See ukdataservice.ac.uk for more details
Historical Structure of ADM Until two years ago, Application Development and Maintainance (ADM) used the waterfall method of application development. One developer assigned to work on a product from inception to implementation across all layers of development. This had lead to some of the following all too common issues Repetition and duplication of code across projects Isolated effort Stove Pipes Repetition of data across systems but not quite! Poor quality code and lack of consistency Unmaintainable code Poor scheduling and long lead times Unfocused meandering development cycles Building some tables!= designing a database
Current Structure of ADM Application Development and Maintenance (ADM) now consists of two teams which opens up Agile possibilities: Applications Front end development Umbraco/MVC/Webforms Java/Solr /Sharepoint.. Services Back end development C# Web services/tools/cit/databases/data feeds Maintenance functions can fall within either team if appropriate and sensible.
Why use Agile The introduction of Agile techniques has had many benefits: Focused project management and decision making during a sprint Reduced development time Better utilisation of resources and skills Technology specialists rather than application specialists Improved communication within ADM itself Stand up Meetings Whiteboards Improved communication outside of ADM All interested parties encouraged to attend stand ups not just the team and the project manager.
Why use CIT To avoid the mistakes and pitfalls of the past and improve turnaround, quality and reuse. With reuse comes dependency not always visible to a developer Small changes can lead to big problems library reuse Errors are spotted early Resource changes should not lead to unmaintainable code Enforces stylistic rules for readability Force developers to produce documentation Improve quality and reliability Resharper encourages refactoring BUT handle with care! Builds (DLLs) are automatically versioned. Essential for managing deployments
How we use Agile Our main tools for Agile are Requirements User Stories Given that when then. JIRA Issue Tracking software Time logging and sprint management Greenhopper Plugin to JIRA Scrum Kanban Scrumban! Prioritising Burndown graphs Sprints One or Two Weeks Daily standups 15 minutes at 9.30am
Sprint Planning in JIRA using Greenhopper Sprint Commitment Sprint Backlog
KanBan Board within JIRA Swimlanes are configurable
Jira Project reporting showing sprint activity Sprint 2 Planning Sprint 1 review Go Live! Sprint 1 Planning Sprint 2 activity Sprint 1 activity
How we use CIT Our main tools for CIT are User Stories Requirements JIRA Issue Tracking software SVN Source Control Repository Jenkins CIT Server Resharper Visual Studio Tool Stylecop Code style ruleset Nunit (Rhino Mocks and MOQ) Selenium UI Testing Dogygen API documentation Robocopy Depoloyment Sonar Monitors Coverage
How does Jenkins fit in? Jenkins Developer PC Monitor builds SVN Build Script Stylecop Tools Source Code Test Code Shared Libraries Resharper Stylecop Check in/out Source Code Test Code Shared Libraries Build occurs Workspace Builds, Test results Documentation
Jenkins Screenshots Job Status Build History
Complete CIT flow Monitor builds and failures Email notifications sent Jenkins Developer PC Code development Test development SVN Jenkins SVN Trigger fires remote build Pre-Build DLL Version Build Unit Tests Selenium Tests Code Coverage API Documentation Deployment script using Robocopy Test Server Success! Triggers further lift and shift job from Jenkins s Workspace. Updates JIRA issues Successful build triggers further jenkins job and updates JIRA Jenkins (Manual) Manually triggered lift and shift job Staging Server Jenkins (Manual) Manually triggered lift and shift job Live Server
Agile and CIT Issues Because of the team structure within ADM, any application sprint requires a pre-sprint by the services team. How far to go with testing? Code Coverage and Mocking Functional Tests and Unit Tests Interfaces! Estimating Time logging is good R&D during a sprint is bad. Creep (requirements churn) UI developer can end up building to an outdated requirement. Communication becomes paramount. Speak up at a stand up! JIRA email blindness
Where we are going with Agile and CIT Database and deployment tools Redgate Developer Tools All new developments involving 3 or more people to be developed as sprints. Thorough testing and continual upskilling Test-Driven Development (TDD) Behaviour-Driven Development (BDD) Improved planning and estimation Jira time tracking Improved products
Useful Links Jira http://www.atlassian.com/software/jira/overview Greenhopper http://www.atlassian.com/software/greenhopper/overview SVN http://subversion.apache.org/ Jenkins http://jenkins-ci.org/ Selenium http://docs.seleniumhq.org/ Stylecop http://stylecop.codeplex.com/ Resharper http://www.jetbrains.com/resharper/ Redgate http://www.red-gate.com/products/sql-development/sqldeveloper-bundle/ Doxygen http://www.stack.nl/~dimitri/doxygen/index.html Sonar http://www.sonarsource.org/
Thanks for listening Any Questions?
CONTACT UNIVERSIY OF ESSEX WIVENHOE PARK COLCHESTER ESSEX CO4 3SQ T +44 (0)1206 872001 E info@data-archive.ac.uk W data-archive.ac.uk