Agile Testing of Business Intelligence. Cinderella 2.0

Size: px

Start display at page:

Download "Agile Testing of Business Intelligence. Cinderella 2.0"

Arthur Stephens
8 years ago
Views:

1 Agile Testing of Business Intelligence Cinderella 2.0 Armando Dörsek (Verified) & Iris Groenewoudt (Ordina) Nordic Testing Days 6/6/2013

2 Programme About Us The Customer Background Information Business Intelligence Agile vs Waterfall Projects Customer Organization Issues & Lessons Learned 6

3 About us Armando Dörsek Verified Testing Services Sr Test manager 15 years experience (finance, retail, government) Iris Groenewoudt Ordina Sr BI consultant 15 years experience (finance, travel industry, care) 7 7

government) Iris Groenewoudt Ordina Sr BI consultant

4 About the case Recent customer Dutch Top 3 Health Insurance Co. 3 brand names, 2.1 mio customers (2011) In : EUR Out : EUR Strategic Goal: from Insurance Co. to Health Organization Corporate wide DWH & BI Project 8 8

5 Background Information DWH & BI 9

6 Definition of DW/BI A data warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management's decision making process. Business intelligence usually refers to the information that is available for the enterprise to make decisions on. 10

7 Voorbeeld Note: no actual production data. 11

8 DW/BI Pyramid Reports, Dashboards, Analyses Semantic Layer Domain Dept. Data Marts Enterprise Datawarehouse Staging Area Source Systems CRM Call C. Backoff. Etc. 12

9 Customer Situation Integrated ERP, coming with a solution for Business Intelligence. Extended with bespoke DWH to enable answering the necessary questions Report Report Sources OHI BO Proprietary OHI BI DWH DB Schemas Universe Universe SQL: SQL: Select * from Select * from Report Report Files

Extended with bespoke DWH to enable answering the necessary questions

10 DW & BI Testing Validating ETL processes with Test Cases, checking totals etc. Testing end user products: right, in time, usable etc. Report Report Sources OHI BO OHI BI DWH DB Schemas Universe Universe SQL: SQL: Select * from Select * from Files Auditing Files

11 Background Information Waterfall vs Agile 15

12 Waterfall vs Agile Waterfall Project Agile Project 17

13 Agile Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions Working software Customer collaboration Responding to change over processes and tools comprehensive documentation contract negotiation following a plan That is, while there is value in the items on the right, we value the items on the left more. 18

collaboration Responding to change over processes and tools comprehensive documentation contract

14 Scrum Three Roles Four Ceremonies Three Artifacts 19

15 User Stories "As a <role>, I want <goal/desire> so that <benefit>" 26

16 Planning Poker 27

17 Board Story Todo Thinking Doing Verify Done 28

18 Burndown Chart Bron: 32

19 Agile Testing in DW & BI The Mango Project 33

20 Agile (Scrum) Datawarehousing Project Challenge # Cinderellas want to go to the ball 34

21 Agile (Scrum) Datawarehousing Project Challenge # Cinderellas want to go to the ball How to prioritize the Product Backlog with so many clients? PIT 37

22 And then 38

23 Day after day, sprint after sprint 39

24 DW/BI Pyramid Reports, Dashboards, Analyses Semantic Layer Domain Dept. Data Marts Enterprise Datawarehouse Staging Area Source Systems CRM Call C. Backoff. Etc. 40

25 Right, in time and complete?

26 Product Risk Analysis (PRA) Waterfall Projects Agile Projects 42

27 The Team is Responsible 46

28 Everyone responsible for everything? 47

29 Distributing Testing Activities ETL & Database Semantic Layer Reports UT, ST & UAT Approach/Intensity 48

30 Testing: Roles, Tasks, Activities Testing is a Role Defining the Testing Strategy Compare Architect and Project Management Roles System Testers Typical Testers Activities Professional Testers for Test Specification and Execution Testing the Integrated Product Acceptance Testing Bridging the Gap, together with Information Analists Testing is a Task, too: Everyone tests Whoever can help out, given the priority Testing Activities are Distributed According to Matrix: Unittest through Acceptancetest (proof) Regression Testing Need for clear Unit Testing instructions 51

31 The Verify Column 52

32 Issues 53

33 Issues Heartbeats Timing issues because of Surroundings (release schedules of source systems) Development User Stories in Teams Developing vs Testing 54

34 Issues (continued) Complexity of DWH Dependencies regarding Meta Model, EDW Data Model Delivering Value vs Preparing for the Future Not unique for Agile BI? 55

35 Issues (continued) Test Base Documentation Created during the Sprint (not beforehand as with Waterfall) Misinterpretation of Working Software over Comprehensive Documentation is easily done Creating Testcases proves Difficult without HQ Test Base Definition of Done User Stories Not unique for Agile BI? 56

36 Issues (continued) Acceptance Absence of End Users in the Sprint Team Absence of Maintenance Team in the Sprint Team 57

37 Issue: Functional vs Technical Functional Stories Direct End User Orientation Technical Stories Technical Debt Architecture 59

38 Issues (continued) Test Data and -Environment Slow, partially due to using Production Volumes Short Time for Integration Tests High Frequency of Regression Tests Who s in Charge (when it comes to it?) Testing End User Functionality SMART Criteria needed Can sometimes tested late in the Sprint only Data Models > Universes > Reports Demo s with Production-Like Data are deemed essential to End Users 60

39 Lessons Learned 66

40 Lessons Learned Agile Scrum: Delivers Most urgent products first, not state-of-art Meanwhile: think of the near future Sr End User vs PIT Business Intelligence = Company Wide All Customer Groups need to be involved and decide who s first Final Acceptance can become an issue End users need to be involved throughout the sprint This can be tough (day to day vs project activities) 67

41 Lessons Learned Functional Maintenance Can become an issue in several ways Functional maintenance team needs to be involved This can become an issue (day to day vs project) Maintenance by Sprint Team needs to be avoided It s about People Need for pros, experienced people Need for agility It s about standards Agility vs Rigidity Experienced people need standards to adhere to 68

42 Lessons Learned Technology Learn to use tools in use by the team to build in quality Regression testing using tools Start early, or else Test Data Management Start early: it pays off for all 69

43 Questions? 70

Waterfall to Agile. DFI Case Study By Nick Van, PMP

Waterfall to Agile. DFI Case Study By Nick Van, PMP Waterfall to Agile DFI Case Study By Nick Van, PMP DFI Case Study Waterfall Agile DFI and Waterfall Choosing Agile Managing Change Lessons Learned, Sprints Summary Q and A Waterfall Waterfall Waterfall