Oracle Essbase Aggregate Storage Option Benchmark on Oracle Sun Enterprise M5000 Server. An Oracle White Paper April 2010

Transcription

1 Oracle Essbase Aggregate Storage Option Benchmark on Oracle Sun Enterprise M5000 Server An Oracle White Paper April 2010

2 Oracle Essbase Aggregate Storage Option Benchmark on Oracle Sun Enterprise M5000 Server Executive Summary... 3 Introduction... 3 Benchmark Objectives... 3 Product Overview... 4 Benchmark Scenario Overview... 5 Key Benchmark Metrics... 6 Benchmark Details... 6 Benchmark Infrastructure Profile... 6 Storage Configuration... 7 Test Application Characteristics and Configuration... 8 Essbase Aggregate Storage Database Configuration... 9 Test Methodology Workload Specification Benchmark Results Summary Conclusion APPENDIX Oracle Essbase Aggregate Storage Benchmark Page 2

3 Oracle Essbase Aggregate Storage Option Benchmark on Oracle Sun Enterprise M5000 Server EXECUTIVE SUMMARY Oracle Essbase is the industry-leading OLAP (online analytical processing) server. Oracle Essbase provides an environment for rapidly developing custom analytic and enterprise performance management applications. With a rapid application development environment that is managed by the business, Oracle Essbase enables the business to quickly model complex business scenarios. Introduction Business Intelligence (BI) systems are becoming more and more pervasive as stakeholders across organizations demand better transparency and visibility. To succeed in today s fast-paced, extremely competitive business environment, organizations have to constantly re-invent themselves. Success in today depends on an organization's ability to convert the vast amounts of data in various operational systems, data marts, and data warehouses into information and then derive intelligence from it. The key to addressing these business challenges is establishing a business intelligence infrastructure that delivers valuable information to employees, shareholders, customers and suppliers for timely, informed decisionmaking. Oracle Essbase delivers key foundational technology to the business intelligence infrastructure for today s dynamic fast paced organizations. Oracle Essbase is the leading online analytical processing (OLAP) server available today. Oracle Essbase provides an environment tailored for business users to rapidly develop custom analytic and enterprise performance management (EPM) applications. Engineered for scalability, security, and rapid-response, Oracle Essbase brings analytics to the business user and enables the business to quickly model complex business scenarios. Through an intuitive interface, business users can manipulate large data sets to model complex scenarios, forecast outcomes, and perform what-if analyses to identify trends and optimize business results. Together with other modules of Oracle s EPM system, Oracle Essbase offers broad support for demanding business intelligence analytics requirements. Benchmark Objectives The objective of the benchmark is to showcase the ability for Oracle Essbase to scale in terms of user population and data volume for large enterprise deployments. Typical administrative and end-user operations for OLAP applications were simulated on Sun Server and Storage products to produce benchmark results. These benchmark test results demonstrate the following: Oracle Essbase Aggregate Storage Benchmark Page 3

4 Oracle Essbase can support a named user population of 200,000 users with the benchmark configuration. This assumes a 10 percent concurrency rate for the 20,000 active concurrent users in the benchmark. Oracle Essbase running as a 64-bit application deployed on UltraSPARC64 VII powered Oracle Sun SPARC Enterprise M5000 server running Oracle Solaris 10 can handle a large scale enterprise sales analytics application with a database containing upwards of 1 billion records, 15 dimensions and dimensions with millions of members. Oracle Essbase can easily scale to meet the needs of any demanding business enterprise as demonstrated by the sub second query response time with 20,000 concurrent users and a large enterprise size Essbase application. Product Overview Oracle Essbase is a multi-threaded OLAP database server that takes advantage of symmetric multiprocessing hardware platforms. All Essbase application components, including database outlines and calculation scripts, application control, and multidimensional database information, reside on a server. Oracle Essbase provides two storage kernels for multidimensional databases, Aggregate Storage Option (ASO) and Block Storage Option (BSO). Aggregate Storage databases typically address read-only, rack and stack applications that have large dimensionality. Using Aggregate Storage, Essbase serves a wide range of analytic needs financial analysis, planning, budgeting, sales analysis, marketing analysis, supply-chain analysis, and profitability analytics all from a single analytic infrastructure. Block storage databases provide robust support for write-back and complex calculation operations and are well suited for modeling and forecasting applications. Essbase partitioning capabilities allow customers to take advantage of both storage technologies in a single application. Oracle Essbase products incorporate powerful architectural features to handle a wide range of analytic applications across large multi-user environments. Figure 1 provides a high-level view of the information flow between the three tiers of the Essbase architecture. The client tier includes Essbase Server clients, such as Oracle Hyperion Smart View for Office, Fusion Edition, Administration Services Console and Oracle Business Intelligence Enterprise Edition Plus. The middle tier includes services, such as Oracle Hyperion Provider Services and Oracle Essbase Administration Services. Communication between the client and middle tiers is through HTTP and communication between the client and database tiers can be either through TCP/IP or HTTP. Figure 1 High-level Information Flow between Product Components Oracle Essbase Aggregate Storage Benchmark Page 4

5 . Benchmark Scenario Overview The goal of this benchmark is to highlight how Oracle Essbase can support pervasive deployments in large enterprises on Oracle Sun servers and storage, by simulating an organization that needs to support a large Essbase Aggregate Storage database with over one billion data items, large dimension with 14 million members and 20,000 active concurrent users, each operating in mixed mode: adhoc reporting and report viewing. The Oracle Essbase Aggregate Storage application employed for this benchmark was designed to model a scaled out version of a real customer financial business intelligence application. The benchmark simulated typical administrative and end-user operations in an OLAP application environment. Administrative operations include: dimension build, data load, and database aggregation. Administrative operations were simulated using scripts through administrative interfaces available with Oracle Essbase. End-user testing modeled a total user population of 200,000 with 10 percent of the users actively retrieving data from Essbase. Scripts were used to simulate end-user reporting tasks. Think-time and time between iterations were built into end-user simulations. The tests were performed on a single Sun SPARC Enterprise M5000 server running Oracle Solaris 10 Operating system and Oracle's Open Storage array. These tests demonstrate scalability of Essbase and also provide a framework to enable customers to help reduce the risk, time, and complexity in deploying an OLAP solution. Oracle Essbase Aggregate Storage Benchmark Page 5

6 Oracle Essbase supports extremely fast query response times for vast numbers of users, for large data sets, and for complex business models. The Oracle Essbase Aggregate Storage option demonstrates sub-second response times for a simulated environment with active concurrent users reporting against a financial business intelligence application. Key Benchmark Metrics This section summarizes the key metrics captured during this benchmark exercise on the Essbase Aggregate Storage database. Test Name Test Description Test Results Data Retrieval Active concurrent users In this test, user retrieval is simulated with MDX query scripts against a tuned Essbase database Average query response time: <1 second Dimension Build - Loading 13 million members to the Customer dimension Data Load into - 1 Billion Data Lines using 6 Load Buffers and 4 Data Load Threads. Database calculation through aggregation based on tracked queries In this test, dimensions and members are loaded to an Essbase database outline by using a data source text files and rules file. In this test, data values are loaded to an Essbase database from a text files based data source and using rules file in parallel. Parallel data load was achieved with the help of multiple load buffers. In this test, the data retrieval queries are tracked and resulting set of proposed aggregate views are materialized. Total time to perform Dimension Build: 41 Minutes Overall Load Time: 269 Minutes Elapsed time to perform the database aggregation based on tracked queries 115 Minutes BENCHMARK DETAILS The following sections describe the test infrastructure, test methodologies, and detailed results for the benchmarking exercise. Benchmark Infrastructure Profile This section outlines the infrastructure profile employed in the benchmark tests. Oracle Essbase Aggregate Storage Benchmark Page 6

7 Component Oracle Essbase Server Oracle Essbase Administration Server Load Generation LoadRunner Servers Essbase Storage Server specification Oracle Sun SPARC Enterprise M5000 Server 4 X UltraSPARC64 VII, 2.53GHz (Quad Core) 2 threads/core Solaris 10 10/09 (Update 8) 64 GB 4x 1Gb Ethernet ports Each core with own FPU 4 X Local SAS drives with and RPM Windows Client - 8 server s 8 CPU 24 GB Memory Oracle Sun Storage F5100 Flash Array Total 1 TB of disk storage 40 Flash Modules Sun SPARC Enterprise M5000 server attached to the Flash Array via SAS HBA with 2 SAS channels Oracle Solaris ZFS Data Storage Scheme Striped RAID 0 2 Oracle Solaris ZFS file systems created from a single Zpool of 1 TB The storage employed for this benchmark is the Sun Storage F5100 Flash Array, the world's fastest solid-state Flash Array. It provides unprecedented performance & eco-efficiency as well as delivers the most efficient way to accelerate database applications that doesn't require expensive tuning, power or space. It also decreases the latency and improves I/O response times for greater productivity and operational efficiency The table below shows the product versions used in benchmark tests. Product / Component Name Oracle Essbase Oracle Essbase Administration Services Oracle WebLogic HP LoadRunner 9.5 Version bit bit 9.2 Maintenance Pack 64 bit JVM 1.5 update 19 Storage Configuration Sun Storage F5100 Flash Array Sun Storage F5100 Flash Array is attached directly to Sun SPARC M5000 server through SAS HBA. For this benchmark, 40 flash modules were employed with each flash module having a capacity of 22 GB and using these flash modules a single Zpool was created. Two ZFileystems were created from a single Zpool. Oracle Essbase Aggregate Storage Benchmark Page 7

8 Sun Fire M5000 Flash F5100 LoadRunner client machines Test Application Characteristics and Configuration The Oracle Essbase Aggregate Storage application employed for this benchmark was based on a real customer application in financial industry with 13 Million members in the customer dimension. This test application uses storage and dynamic hierarchy with up to 4-5 levels. All dimensions were structured as stored hierarchy with the exception of measures dimension which is structured as dynamic hierarchy and Account dimension is tagged as compression dimension. Application details and outline: Oracle Essbase Aggregate Storage Benchmark Page 8

9 The Oracle Essbase Aggregate Storage database was loaded with one billion lines of data from 40 different load files using different rule files to load into different intersections. The size of each load file was 1.2 Gigabytes containing 10 million data lines. Essbase Aggregate Storage Database Configuration This section outlines the different configurations applied to the Essbase environment in this benchmark exercise. Aggregate Storage Cache o Aggregate Storage cache facilitates memory usage during data loads, aggregations, and retrievals o Set to 32 GB (half the total available memory on the Essbase Server) CALCPARALLEL set to 8, to enable parallel processing during view materialization. MAXLOGINS was set to 25000, to allow more than user sessions allowed to connect to the Essbase Server at any one time. Location of Tablespace and Source Files: o Default tablespace: Default tablespace was placed on ZFilesystem DefaultTS which use the Zpool ASODB o Temp tablespace: Temp tablespace was placed on ZFilesystem TempTS which use the Zpool ASODB o Source Files: Source files were placed on the M5000 internal drives (SAS drives with 10K rpm, striped with a stripe length of 128K). Oracle Essbase Aggregate Storage Benchmark Page 9

10 Test Methodology This section describes the test methodology employed to perform benchmark tests on the Essbase Aggregate Storage database. The Essbase Aggregate Storage database administrative functions were tested using custom developed MaxL scripts. Data sources for Dimension Build and Data load were flat files. The Data retrieval tests were performed using industry standard load test tool HP LoadRunner. In this test, proprietary test harness was used to submit queries directly to Essbase via the Essbase API. Workload Specification This section describes the workload employed in this benchmark testing exercise. Aggregate Storage Database Administrative Functions: Dimension build: This test involves, loading 13 Million members to the customer dimension using a flat files as a data source and rule file. Data load: Data was loaded into the Oracle Essbase Aggregate Storage database from 40 different load files using multiple rule files. The size of each of the load file was 1.2 Gigabytes containing 10 million data lines. This test was performed using a custom MaxL (Data Definition langrage for Essbase) script. Database calculation: Database is calculated by aggregating and storing the values. Two options for aggregation were tested: Default Aggregation: Typically when retrieval request is submitted, Essbase calculates the data values by consolidating the values loaded into level 0 members these calculated values are not stored. In this test, the data values are pre-calculated and stored in the database by the aggregation process. Usage Based Aggregation: Fine tuning retrieval performance was performed with usage based aggregation. In this test, the proposed aggregate views which are created as a result of tracked retrieval queries were materialized. Oracle Essbase Aggregate Storage Benchmark Page 10

11 Aggregate Storage Database User functions Data Retrieval: This section outlines the query profile employed for the Data Retrieval tests. User requests were simulated using both Report Writer and MDX scripts. Report Writer is a text-based script language that you can use to report on data in multidimensional databases. Following is the Report Writer request used in enduser testing. {WIDTH 12} {OUTALTNAMES} <PAGE (Measures, Customer, Shipper) QtyShipped <param1> <Param2> <COLUMN (Shipment_Date) <ICHILD <Param 3> <ROW (Product) <ICHILD <Param4>! MDX is a language to develop scripts for applications to query and report against data and metadata in Essbase databases. Following are MDX queries used in testing. Query 2: MDX Query 1: SELECT {[<Param 1>].children, [<Param 1>]} on axis(0), {[psp_04].children} on rows, TopCount ([customer].levels(4).members,5,[measures].[orderlinediscount]) ON AXIS(2),{[Shipment_term].levels(1).members} on axis(3) from <appname>.<dbname> where ([QtyShipped],[<Param 2>],[<Param 3>]) Query 3: MDX Query 2: SELECT {[<param 1].children, [param 1]} on axis(0), {[psp_04].children} on rows, {[psp_07].children} on axis(2),topcount ([customer].levels(3).members,10,[measures].[orderlinediscount]) ON AXIS(3),{[Shipment_term].levels(1).members} on axis(4) from mill13m.mill13m where ([QtyShipped],[<param 3>],[< param 4>]) Query 3 is executed once every ten iterations. The different parameter values for the queries were supplied via data files and randomized during the test. Data Retrieval test - step by step instructions: 1. User logs in to Essbase 2. User selects the application 3. Think time 1 second 4. User submits Report writer request 5. Think time 1 second 6. User submits MDX query 1 7. Think time 1 second 8. User submits MDX query 2 9. Logout Steps 4 to 6 (both inclusive) were performed in a loop for the test duration of 1 hour. Oracle Essbase Aggregate Storage Benchmark Page 11

12 Load Runner configuration: Random think time 50 to 150% Pacing Random - 60 to 90 seconds after each iteration Benchmark Results Summary This section describes the results of the benchmark tests performed with various configurations Test 1 Custom Dimension Load In this test 13 million members were loaded into custom dimension in the Aggregate Storage database outline using rule files. This test was executed using a custom developed MaxL script. Loading custom Dimensions involves building dimensions and performing a restructure. The elapsed time taken for these activities are show Activity Building dimensions Restructure of the outline Total Elapsed Time ( in Minutes) 37 Minutes 4.2 Minutes Test 2 Data Load using Rules In this test, 1 billion lines of data were loaded from 40 different load files using rules. Each load file was of size 1.2 Gigabytes containing 10 million data lines. For optimal performance, data values were loaded to a temporary data load buffer, and then written to storage after all data sources has been read. The data load tasks involve the sub-tasks parsing of load file, loading the contents of load file to data load buffer and committing the contents of buffer to the database. In this test, the Sun Storage F5100 Flash Array was employed. The data load performance can be influenced by the following factors: Number of Data Load Threads: This configuration (DLTHREADSPREPARE) is typically set at the server level for all or specific Databases/application on the server. This setting enables parallel processing of data load files during the data load preparation stage which may improve data load performance. As a part of Data Load test, various executions were performed with different DLTHREADSPREPARE values. Oracle Essbase Aggregate Storage Benchmark Page 12

13 Number of Load Buffers: The data load performance can be improved by loading values to a temporary buffer load buffer, followed by a final write to storage after all the data load files are read. As a part of this Data Load test, various executions were performed with different number of data load buffers. The overall data load performance may be improved by having multiple load buffers and loading data into those buffers in parallel. Aggregate Storage Cache: Aggregate Storage Cache aids in data load and other Aggregate Storage Database tasks. In this benchmark test, cache 32 GB was used to determine the impact on performance. Location of Data load files, Default and Temp Tablespace: Tablespaces in Aggregate Storage Database help to optimize the data and work file storage and retrieval. Configuring the location of database (default tablespace) and work (temp tablespace) files on different physical drives helps to improve the performance. The data load files should also be placed on different drives to improve performance however it doesn t matter if it s on the same location as default tablespace. Impact of Data Load Threads and Load Buffers on Data Load Performance The impact of data load threads (DLTHREADSPREPARE) and the number of Load Buffers on data load performance were accessed by scheduling various test executions with different number of data load threads and Load buffers. Storage Technology - location of tablespace & Data load file: Sun Storage F5100 Flash Array ZPool: Default Tablespace: placed on the ZFS Filessytem DefaultTS which uses a single Zpool ASODB of 1 TB. Temp Tablespace: placed on the ZFS Filessytem - TempTS which uses a single Zpool ASODB of 1 TB. Data Load Files Internal Drives of the Essbase Server running on the Sun SPARC Enterprise M5000 server The metrics below demonstrates optimum data load performance can be achieved with 4 Data Load Threads and 6 Load Buffers. Please note: This optimum number of Data load threads and Load buffers depends on factors such as amount of physical memory and number of cores on the Essbase Server. Oracle Essbase Aggregate Storage Benchmark Page 13

14 Number of Load Buffers and Data Load Threads # Load Buffers: 8 # Data Load Threads: 4 Data Load Elapsed Time ( in minutes) Overall Load Time: Minutes Time to parse and write the contents of load file to buffer: Minutes Time to write the contents of buffer to Database: Minutes # Load Buffers: 6 # Data Load Threads: 4 Overall Load Time: Minutes Time to parse and write the contents of load file to buffer : Minutes Time to write the contents of buffer to Database: Minutes Resource Utilisation: Processor and IO Utilisation The figures below show the processor and IO utilisation during data load test. Execution 1: # Load Buffers: 6 # Data Load Threads: 4 Figure 1: Processor Utilisation Figure 2: IO Utilisation Over all system activity (in Execution 2: # Load Buffers: 8 # Data Load Threads: 4 Figure 3: Processor utilisation Oracle Essbase Aggregate Storage Benchmark Page 14

15 Figure 2: IO Utilisation Over all system activity (in Test 3 Default Aggregation In this test Aggregate Storage database is calculated by aggregating and storing values in the database. Aggregations are consolidations based on outline hierarchy and has two phases Aggregate view selection Calculation and storage of values for the selected aggregate views. The factors influencing aggregation materialization performance are Aggregate Storage Cache and Number of Threads for parallel view materialization (CALCPARALLEL). In this test for optimal performance, 8 threads for parallel calculation and 32 GB of Aggregate Storage cache were used. Note: The number of threads for parallel calculation and amount of Aggregate Storage cache depends on the available hardware resources. Storage Configuration Sun Storage F5100 Flash Array Default Tablespace placed on the ZFS Filesystem DefaultTS which uses a single Zpool ASODB of 1 TB. Temp Tablespace placed on the ZFS Filesystem TempTS which uses a single Zpool ASODB of 1 TB Default Aggregation ( Elapsed Time) in Minutes 526 Minutes ( Total of 20 aggregate views built) Resource Utilization: Processor and IO The figures below show the processor utilisation of Essbase Server and IO. Oracle Essbase Aggregate Storage Benchmark Page 15

16 Figure1: Processor Utilization Figure 2: This figure shows the over all IO activity Test 4 Usage Based Aggregation In this test, the aggregate views proposed as a result of tracked retrieval queries were materialized, to gain better performance for some retrieval requests. Storage Configuration Sun Storage F5100 Flash Array Default Tablespace placed on the ZFS Filesystem DefaultTS which uses a single Zpool ASODB of 1 TB. Temp Tablespace placed on the ZFS Filesystem - TempTS which uses a single Zpool ASODB of 1 TB Usage Based Aggregation ( Elapsed Time) in minutes 115 Minutes ( Total of 6 aggregate views built) Resource Utilization Processor and IO The sections below show the resource utilisation Processor and IO. The figure below shows the processor utilisation of Essbase server during the test. Figure 1: Essbase Server Processor Utilisation Oracle Essbase Aggregate Storage Benchmark Page 16

17 Figure 2: This figure show the overall IO activity during the test. Test 5 Multiuser- Data Retrieval Tests In this test, concurrent users submitted MDX (language-based data analysis mechanism to Essbase databases) queries to Essbase VIA API calls. In order to achieve concurrency industry standard load test tool HP LoadRunner was employed. In this test scenario, test executions before and after usage based aggregation was performed. Impact of usage based aggregation on retrieval performance The metrics below clearly shows the performance improvements in data retrieval transactions after usage based aggregation. The queries submitted to Essbase were tracked using the Query Tracking option and the proposed aggregate views were materialized (usage based aggregation) for better retrieval performance. Table 1 Table below shows the transaction response time for Data Retrieval test executed before usage based aggregation with Query Tracking enabled. In this test, 3.38 queries per second and 3 reports per second were served during this 500 user retrieval test. Transaction Name Min ( in Average (in Max ( in SD (in 90 Percent ( in Number of Transactions Load Application Load Database Login to Essbase MDX Query ,620 Report Writer ,620 MDX Query ,421 Oracle Essbase Aggregate Storage Benchmark Page 17

18 Table 2 Table below shows the transaction response time for Data Retrieval test executed after usage based aggregation. In this test, 4.9 queries per second and 4.43 reports per second were served during this 500 user retrieval test. Transaction Name Min (in Average ( in Max ( in SD ( in 90 Percent ( in Number of Transactions Load Application Load Database Login MDX Query ,249 Report Writer ,249 MDX Query The rows marked bold are the response time for the queries submitted to Essbase. Further data retrieval tests with concurrent users after usage based aggregation also showed similar performance with no signs of degradation in query timings. Table 3 shows the response time of the transactions with concurrent users. Transaction Name Min ( in Average ( in Max ( in SD 90 Percent Number of transactions Load Application ,000 Load Database ,000 Login ,000 MDX Query ,392,461 Report Writer ,392,461 MDX Query ,932,253 Oracle Essbase Aggregate Storage Benchmark Page 18

19 CONCLUSION The Oracle Essbase Aggregate Storage benchmark demonstrates the ability of Oracle Essbase to support business intelligence applications with requirements for large data volumes and enterprise-scale user populations. The combination of Essbase, Sun SPARC Enterprise M-series servers and Sun Storage F5100 Flash Array offer industry- leading -performance for end-user and administrative operations in OLAP applications. Oracle Essbase Aggregate Storage Benchmark Page 19

20 APPENDIX Essbase - Aggregate Storage Database - Statistics The section shows the Aggregate Storage statistics gathered at various stages of benchmark tests. Essbase Database Statistics after Data Load Essbase Statistics Value Number of Input Cells Input level data size (GB) Current Cache Size(GB) Disk space used by data (GB) 32 GB 52 GB Essbase Database Statistics after Default Aggregation Essbase Statistics Value Number of Input Cells Input level data size (GB) Current Cache Size(GB) Disk space used by data (GB) 32 GB 104 GB Number of aggregate views 20 Number of aggregate cells Aggregate data size (KB) Essbase Database Statistics after Usage based Aggregation Essbase Statistics Value Number of Input Cells Input level data size (KB) Current Cache Size(GB) Disk space used by data (GB) 32 GB GB Number of aggregate views 20 Number of aggregate cells Aggregate data size (KB) Oracle Essbase Aggregate Storage Benchmark Page 20

21 Essbase ASO Benchmark April 2010 Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA U.S.A. Worldwide Inquiries: Phone: Fax: oracle.com Copyright 2010, Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle, JD Edwards, PeopleSoft, Siebel, Hyperion, and Sun are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.