Agile Testing of Business Intelligence Cinderella 2.0 Armando Dörsek (Verified) & Iris Groenewoudt (Ordina) Nordic Testing Days 6/6/2013
Programme About Us The Customer Background Information Business Intelligence Agile vs Waterfall Projects Customer Organization Issues & Lessons Learned 6
About us Armando Dörsek Verified Testing Services Sr Test manager 15 years experience (finance, retail, government) Iris Groenewoudt Ordina Sr BI consultant 15 years experience (finance, travel industry, care) 7 7
About the case Recent customer Dutch Top 3 Health Insurance Co. 3 brand names, 2.1 mio customers (2011) In : EUR 4.998.132.000 Out : EUR 4.684.321.000 Strategic Goal: from Insurance Co. to Health Organization Corporate wide DWH & BI Project 8 8
Background Information DWH & BI 9
Definition of DW/BI A data warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management's decision making process. Business intelligence usually refers to the information that is available for the enterprise to make decisions on. 10
Voorbeeld Note: no actual production data. 11
DW/BI Pyramid Reports, Dashboards, Analyses Semantic Layer Domain Dept. Data Marts Enterprise Datawarehouse Staging Area Source Systems CRM Call C. Backoff. Etc. 12
Customer Situation Integrated ERP, coming with a solution for Business Intelligence. Extended with bespoke DWH to enable answering the necessary questions Report Report Sources OHI BO Proprietary OHI BI DWH DB Schemas Universe Universe SQL: SQL: Select * from Select * from Report Report Files
DW & BI Testing Validating ETL processes with Test Cases, checking totals etc. Testing end user products: right, in time, usable etc. Report Report Sources OHI BO OHI BI DWH DB Schemas Universe Universe SQL: SQL: Select * from Select * from Files Auditing Files
Background Information Waterfall vs Agile 15
Waterfall vs Agile Waterfall Project Agile Project 17
Agile Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions Working software Customer collaboration Responding to change over processes and tools comprehensive documentation contract negotiation following a plan That is, while there is value in the items on the right, we value the items on the left more. 18
Scrum Three Roles Four Ceremonies Three Artifacts 19
User Stories "As a <role>, I want <goal/desire> so that <benefit>" 26
Planning Poker 27
Board Story Todo Thinking Doing Verify Done 28
Burndown Chart Bron: http://www.brightgreenprojects.com/ 32
Agile Testing in DW & BI The Mango Project 33
Agile (Scrum) Datawarehousing Project Challenge #1 1001 Cinderellas want to go to the ball 34
Agile (Scrum) Datawarehousing Project Challenge #1 1001 Cinderellas want to go to the ball How to prioritize the Product Backlog with so many clients? PIT 37
And then 38
Day after day, sprint after sprint 39
DW/BI Pyramid Reports, Dashboards, Analyses Semantic Layer Domain Dept. Data Marts Enterprise Datawarehouse Staging Area Source Systems CRM Call C. Backoff. Etc. 40
Right, in time and complete?
Product Risk Analysis (PRA) Waterfall Projects Agile Projects 42
The Team is Responsible 46
Everyone responsible for everything? 47
Distributing Testing Activities ETL & Database Semantic Layer Reports UT, ST & UAT Approach/Intensity 48
Testing: Roles, Tasks, Activities Testing is a Role Defining the Testing Strategy Compare Architect and Project Management Roles System Testers Typical Testers Activities Professional Testers for Test Specification and Execution Testing the Integrated Product Acceptance Testing Bridging the Gap, together with Information Analists Testing is a Task, too: Everyone tests Whoever can help out, given the priority Testing Activities are Distributed According to Matrix: Unittest through Acceptancetest (proof) Regression Testing Need for clear Unit Testing instructions 51
The Verify Column 52
Issues 53
Issues Heartbeats Timing issues because of Surroundings (release schedules of source systems) Development User Stories in Teams Developing vs Testing 54
Issues (continued) Complexity of DWH Dependencies regarding Meta Model, EDW Data Model Delivering Value vs Preparing for the Future Not unique for Agile BI? 55
Issues (continued) Test Base Documentation Created during the Sprint (not beforehand as with Waterfall) Misinterpretation of Working Software over Comprehensive Documentation is easily done Creating Testcases proves Difficult without HQ Test Base Definition of Done User Stories Not unique for Agile BI? 56
Issues (continued) Acceptance Absence of End Users in the Sprint Team Absence of Maintenance Team in the Sprint Team 57
Issue: Functional vs Technical Functional Stories Direct End User Orientation Technical Stories Technical Debt Architecture 59
Issues (continued) Test Data and -Environment Slow, partially due to using Production Volumes Short Time for Integration Tests High Frequency of Regression Tests Who s in Charge (when it comes to it?) Testing End User Functionality SMART Criteria needed Can sometimes tested late in the Sprint only Data Models > Universes > Reports Demo s with Production-Like Data are deemed essential to End Users 60
Lessons Learned 66
Lessons Learned Agile Scrum: Delivers Most urgent products first, not state-of-art Meanwhile: think of the near future Sr End User vs PIT Business Intelligence = Company Wide All Customer Groups need to be involved and decide who s first Final Acceptance can become an issue End users need to be involved throughout the sprint This can be tough (day to day vs project activities) 67
Lessons Learned Functional Maintenance Can become an issue in several ways Functional maintenance team needs to be involved This can become an issue (day to day vs project) Maintenance by Sprint Team needs to be avoided It s about People Need for pros, experienced people Need for agility It s about standards Agility vs Rigidity Experienced people need standards to adhere to 68
Lessons Learned Technology Learn to use tools in use by the team to build in quality Regression testing using tools Start early, or else Test Data Management Start early: it pays off for all 69
Questions? 70
armando.dorsek@verified.nl +31623054054 @adorsek iris.groenewoudt@ordina.nl