BUSINESS INTELLIGENCE ANALYTICS QUALITY ASSURANCE- A NEW FRAMEWORK FOR FASTER DELIVERY STC 2013 Anshika Goyal 1 and Suman Gopalan 2 Accenture Services Pvt Ltd. Bldg.No.1A and 1B Raheja Mind Space, Hitech City, Madhapur Hyderabad - 500086 Author s Title: 1 Analyst Programmer and 2 Senior Systems Analyst Conference Area: Emerging Areas Business Intelligence Solution Testing ABSTRACT Oracle Business Intelligence Analytics is one of the leading Business Intelligence platforms in the current Industry. Managing hundreds of Oracle Business Analytics Reports through Adhoc Business Requirements and Change Release Management Cycles is a challenge for Biz.IT Operations. We bring a new framework fresh from Accenture Innovation Centre that reduces regression testing cycle time for Oracle Business Analytics Implementation up to 50% or more. The Framework provides an optimized, scalable and generic solution which is used to validate the business functionality, GUI, textual and metric data of OBIEE Dashboards and Adhoc Analyses including security verification. The proposition enumerates the benefits of the proposed automation framework along with example of real time scenario with sample test run conducted at a large scale Implementations.In addition to it we will discuss how this framework is scalable to any Oracle Business Analytics Product that utilizes Oracle Business Intelligence Enterprise Edition (OBIEE) v10g/11g as the platform for various industry applications like Financial/HR Analytics etc. WHAT IS OBIEE
Oracle Business Intelligence Suite Enterprise Edition(OBIEE) is a comprehensive suite of enterprise BI products, delivering the full range of BI capabilities including interactive dashboards; full ad hoc, proactive intelligence and enterprise reporting.in addition to providing the full range of BI functionality. Oracle Business Intelligence provides the clients with high end interactive Dashboards and Adhoc Analyses tools to build KPI s, Reports and Dashboards that enable them to make better decisions with appropriate information and accurate metrics. NEED OF AUTOMATION TESTING FRAMEWORK Considering the significance of OBIEE Reports and Dashboards, we realize the importance for Quality Assurance of all deployed reports. Also due to the varying nature of Business requirements of end users, there is a significant need of regression testing required when there are consecutive deployments done with a very less duration in between releases. Automation Testing Framework would help OBIEE (Oracle Business Intelligence Enterprise Edition) Implementations improves the turnaround time for these Quality Assurance cycles of Oracle Business Intelligence Applications. SOLUTION DETAIL We have used Quick Test Professional (QTP) which is an automation tool from HP for creating automation test scripts and HP Quality Center(QC) is used to store the Re- Usable/Custom scripts, Test Data and for executing the scripts on a web shared environment or automated test scripts can also be stored on local machine. OBIEE Report testing can be majorly categorized in following categories: GUI Testing (Dashboard links/ Prompts/ Alignment/ Formats) Functionality Testing (Links and Drill Downs/ Filters/ Graphs) Data validation (Data validation for prompts/ graphs/ reports/ Aggregation/ SQL Data Vs Report Data) Security Testing (Security Testing for OBIEE Dashboard & Report Objects & Data) Adhoc Report testing (Subject areas/tables/columns/adhoc report generation and verification) For Data Validation, database connection script is written to connect to the OBIEE data source system and data on the report is verified against the data in source system acquired by firing SQLs prepared by QA personnel. Figure 1 shows the Data flow in an OBIEE Environment and depicts the various Application involved in the Data Validation process
Figure 1: Solution Architecture QTP Data Table behaves like an excel sheet within QTP where column headers are labeled as meaningful names as per the objects on OBIEE application and dynamically changing url/names/values on the application are fed by the user into the Data Table before running the automated test scripts.qtp Data Table can be imported from an external excel sheet and also exported to it. The below list of actions will be used to understand the flow of testing a OBIEE Dashboard through automation test cases I. Dashboard Functional and GUI Validation 1. Login to OBIEE application 2. Navigate to the specified Dashboard Group, Dashboard and then to specified Page 3. Verify Dashboard Page Name along with its font, text size and alignment 4. Validation of Prompts a. Verify Prompts with its name and type multi select, drop box, check box, radio button, and textbox b. Change the Prompts value and click on Go button and verify report is getting refreshed c. Deselect Values from Multi select Filters 5. Validation of Reports a. Verify Report Heading and Column headers b. Verify Grand total on the Report 6. Verify Drill down on the report table 7. Verify Back and Return Functionality on the report 8. Verify any Hyperlink present on the report 9. Verify Guided Navigation 10. Verify Summary to Detail Report Drill Down 11. Verify the download options 12. Verify the footer along with its font, text size and alignment 13. Verify the Logout II. Dashboard Data Validation
1. Data Validation of Report a. Establish database connection b. Verify data on the report tables c. Execute the SQL Query and return the results d. Compare the data on the report with database 2. Data Validation of Prompt Values a. Establish database connection b. Verify data on the prompts c. Execute the SQL Query and return the results d. Compare the data on the report with database III. Dashboard and Answers Security Validation 1. Verify Security at Dashboard Group Level 2. Verify Security at Dashboard Level 3. Verify Security at Page Level 4. Verify Security at Report Level 5. Verify Data Security 6. Verify Security on Answers IV. Answers Validation 1. Verify all the objects present in Answers Page 2. Verify the application of filters in Answers Page 3. Verify the generation Adhoc Report in Answers Page 4. Verify the Data of Adhoc Report in Answers Page a. Establish database connection b. Verify data on the report table c. Execute the SQL Query and return the results d. Compare the data on the report with database BENEFITS OF AUTOMATION TESTING FRAMEWORK The Proposed Framework intents to benefit the QA process of SDLC cycle for SI/AO Implementations. Below listed are the salient features of the proposed tool. The proposed Frameworks can be utilized for User Interface Testing, Functionality Testing and Data validation of OBIEE Dashboards and Reports. Reduces Effort required for System Testing and Regression Cycle of Medium to Large Scale Implementations by 50% or more. Increase in Quality of delivered product as all the textual data and Metric data displayed on Dashboards are verified in totality. Optimized Automation Script developed by using modules implementing maximum Re-usable Actions. OBIEE ADOPTION IN THE GLOBAL BI MARKET In the Current BI Market, Oracle Business Intelligence Tool has been the most adopted BI Tool for most of the large Scale BI Implementations across the globe. The implementation Opportunity of the BI Automated Testing tool is directly proportional to the global Adoption of OBIEE in the Global BI Market. Gartner has identified more than 50 vendors with more than $10 million in BI-software related revenue; the market is highly concentrated in the top five, which own close to three quarters of the market. Oracle BI has 16% Market Share as of 2011 and has been consistently growing at 15% or more over the past 3 years.
Figure 2: BI Market Share. Figure 3: BI Market Share Growth EFFORT SAVINGS DETAILS: SAMPLE TEST RUN Case Study: Large Scale IT Infrastructure Company Below are the details about OBIEE automation for one of the subject areas implemented for an existing Client on OBIEE v10g: Total no. of Manual scripts: 108 No. of Scripts automated: 97 Manual Execution Effort for 97 scripts: 204 hrs. Automation Execution Effort for 97 scripts: 98 hrs. Effort saving = 106 hrs. for one cycle.
After Automation an effort saving of 106 hours was registered out of which 50-60 hrs. of time was utilized as part of Data retrieval time for Source Data SQL s and Report Retrieval time. Therefore the Automation process registered an effort saving of 66% in above scenario. PROPOSED ADOPTION OF THE TOOL FROM IDC CLIENTS. The developed tool has been proposed to be part of future releases for various clients at Accenture India Delivery Center. The implementation team has been discussing on a deployment strategy with the below mentioned clients using the Automation Testing Framework. Large Scale Steel Manufacturing Client. Global Coffee Company and Largest Coffeehouse company. Dutch Landline and Mobile Telecommunication company. Large American Broadband and Telecommunication company. AUTOMATION TOOL SCREENSHOTS: Below displayed are the screenshots of the proposed tool in progress which depicts the status of the various steps involved in the Test Script while testing an OBIEE Dashboard. Oracle BI Capability Figure 4: QTP Screenshot
Figure 5: QTP Test Results HOW IT CAN BE EXTENDED TO OBIA This framework can be extended to all the modules of Oracle Business Intelligence Applications (OBIA) like Finance, HR. The reusable actions mentioned in the above section are created to overdo scripting of same actions individually for every report. It helps us to save effort during building test scripts for each module. Hence optimizing the automation scripts by creating reusable actions. We have fully automated the OBIA Finance module which is ready for deployment to a banking client. CONCLUSION Based on the results of the test runs of the Automation Framework we do recognize a significant saving of 50% or more efforts when automating the QA cycle of OBIEE Implementations. This would provide a significant savings of time and cost to Clients. With the vast potential of OBIEE Application ranging across a wide array of Functional Modules the Automation Testing Framework is definitely an impeccable asset which will enable faster deployments of OBIEE/OBIA Applications. REFERENCES: 1. http://www.gartner.com/ 2. http://www.slideshare.net/haydenmccall/istart-business-intelligence 3. http://docs.oracle.com/cd/e28280_01/doc.1111/e14860/new_bi.htm AUTHOR BIOGRAPHY
Anshika Goyal: Senior Software Engineer from Oracle Capability team India delivery Centre. She is having 4+ years of hands on experience in different phases of SDLC and STLC. She has worked on OBIEE, BRIO, ETL testing and QTP Automation. ASDA, HP QC, HP QTP and OBIEE 11g Certified Professional. White paper published in QAI Software Testing Conference 2010 on -OBIEE TESTING - EFFECTIVE AND EFFICIENT APPROACH. Actively involved in Oracle BI Capability on Assets and Initiatives.Currently working for a client project on OBIA, DAC and Informatica. Suman Gopalan:Associate Manager from Oracle Capability team India delivery Centre. He is an Oracle Certified Professional (Oracle Business Intelligence Server Architect) and PMI certified Project Management Professional (PMP ). He has more than 9 yrs of experience including extensive Business Consulting Experience in implementing End-to- End Global OBIEE/CRM Solutions involving Multiple Country Business Users and Multi-Lingual Applications in all phases of SDLC. Actively involved in providing support to Pre-Sales Business Development/ Capability Initiatives.Currently working from onsite for a client project on OBIEE end to end implementation.