Integrating MicroStrategy Analytics Platform with Oracle

Transcription

1 MicroStrategy, Inc. Integrating MicroStrategy Analytics Platform with Oracle This document explains how MicroStrategy integrates with Oracle and lists best practices MicroStrategy Incorporated, 2015 Page 1 of 38

2 Contents Introduction... 4 About MicroStrategy... 4 About Oracle Database... 4 Overview of MicroStrategy Architecture... 5 MicroStrategy Technology Philosophy: Leverage the Power of the RDBMS... 5 Model-based Dynamic SQL Generation... 5 Schema Abstraction... 5 Aggregate Awareness... 6 Multi-pass SQL... 6 VLDB Driver... 6 Integration Best Practices for Performance... 7 System Considerations... 7 Hyper Threaded CPU... 7 Huge Page Memory Bit OS on > 4G Memory... 8 Database Block Size... 8 Node Clustering... 9 Parallelism... 9 Optimal Query Generation Intermediate Tables Table Creation Type Sub Query Type SQL Global Optimization Set Operator Optimization Join Type Full Outer Join Analyzing Statistics on Intermediate Tables Optimizer Hints Planning a pre-aggregation strategy and using Materialized Views Oracle XML Features Using Bitmapped Indexing MicroStrategy Incorporated, 2015 Page 2 of 38

3 CUBE, ROLLUP, and Grouping Sets Integration with Oracle OLAP Option and Analytic Workspaces Additional VLDB Settings Extended Analytics Using Oracle Functions Analytic Functions: Functions for OLAP Custom Group Banding via the DECODE statement Other Performance Considerations Collect Statistics MicroStrategy Certification of Oracle Status Oracle Database 12c Oracle Database 11g Release Oracle Database 11g Oracle Database 10g Release Oracle Database 10g Oracle Database 9i and 9i Release Appendix Default VLDB Settings for Oracle Database 11g R Data type Mapping Oracle Database Functions Used by MicroStrategy MicroStrategy Incorporated, 2015 Page 3 of 38

4 Introduction This paper, explains how the MicroStrategy BI platform integrates with the technology advances and business intelligence functionality of Oracle database. Furthermore, it details best practices allowing to take full advantage of both technologies. MicroStrategy offers an unsurpassed range of analytical functionality, which is further optimized when integrated with Oracle s broad list of database features that improve data warehousing performance. About MicroStrategy MicroStrategy is a full featured BI platform, delivering all 5 styles of BI in a fully integrated architecture. The MicroStrategy platform provides VLDB drivers for all supported RDBMS platforms to generate optimized SQL that takes advantage of database specific functionality. The full set of VLDB properties is documented in the MicroStrategy System Administration Guide. Settings that are most relevant to Vertica are discussed below. To learn more about MicroStrategy, visit and follow us on Facebook ( and Twitter ( About Oracle Database Oracle Database is an object relational database management system produced and marked by Oracle Corporation, it provides a comprehensive feature set to easily manage the most demanding transaction processing, business intelligence, and content management applications. The Oracle database is packaged in various editions. The features available by edition are listed here: MicroStrategy Incorporated, 2015 Page 4 of 38

5 Overview of MicroStrategy Architecture MicroStrategy Technology Philosophy: Leverage the Power of the RDBMS The MicroStrategy architecture has its roots in the principles of Relational OLAP (ROLAP). A ROLAP architecture provides OLAP functionality to the end user (e.g. multidimensional framework, slice-anddice interaction, drilling, etc.), but uses a relational database to resolve queries and perform calculations, rather than using a specialized proprietary multidimensional database. While the virtues of ROLAP are fully extolled elsewhere, this is an important distinction for this article for two reasons. First, MicroStrategy applications have been able to achieve industry-leading data scale because of the reliance on the processing power and data management capabilities of the underlying RDBMS. The system design principle of partnering with the RDBMS, through features discussed in this document, is the cornerstone of achieving such scalability. Second, this focus on integration with the RDBMS allows a business intelligence system to tap into a powerful dynamic over time: technical advances in RDBMS technology are seamlessly accessible to a MicroStrategy-based BI system. The net benefit to the customer is a BI system that increases in value when either MicroStrategy or the RDBMS adds features to their products. Model-based Dynamic SQL Generation When a user runs a report or dashboard in MicroStrategy will push down the analytics to the RBDMS in form of optimized SQL queries and visualize the data of the result sets according to the report and dashboard specifications. Hence, to the RDBMS, a MicroStrategy application is an SQL-based application, in many ways like any other SQL application accessing the Oracle Database. Schema Abstraction The SQL Engine component of MicroStrategy will generate the queries dynamically at runtime. The SQL Engine performs its work based on a metadata model defined to the system. Note, that the MicroStrategy metadata is not used to store joins or schema-type information, such as star or snowflake. Instead, the metadata model stores content information for each table indicating that it contains a set of particular facts and a set of particular attributes. When a report request is submitted, the Engine breaks the report down into the individual components (i.e. attributes and facts), then begins searching the model to determine which combination of tables will be necessary and efficient in resolving the request. Schema abstraction of the database columns (into MicroStrategy attributes and facts) provides the flexibility necessary to allow applications to be created quickly without having to change the structure of the data model. MicroStrategy is able to support virtually any type of star, snowflake, or hybrid physical design, including transactional schemas. The business model defined in MicroStrategy is easily able to span multiple stars/snowflakes in a single application and even a single query. MicroStrategy supports dimensional models well, but does not require a dimensional model. MicroStrategy Incorporated, 2015 Page 5 of 38

6 Aggregate Awareness Query performance in many data warehouses is enhanced through the use of aggregate tables. Aggregate tables, also called summary tables, store pre-computed results of data allowing users to query from a summarized set of data rather than the detail level data that would be stored in the fact table. In many cases, use of aggregate tables will improve query performance by orders of magnitude. MicroStrategy s SQL Engine is aggregate-aware and determines the use of aggregate tables transparently at query time. MicroStrategy has allowed transparent navigation of aggregate tables, directing queries to summary tables when they exist without the user having to specify to use the table. For more information see Planning a pre-aggregation strategy and using Materialized Views. Multi-pass SQL One of the key elements to providing analytical sophistication in analytical applications is MicroStrategy s ability to generate multi-pass SQL. Multi-pass SQL is required to answer analytical questions that cannot be answered with a single SQL query block. Examples of questions / scenarios that require multi-pass SQL include: Set qualification: Show me sales by region over the last six months, but only for customers who purchased one of the 5 most popular products. Split metrics: query returns sales data from a sales star schema and inventory data from an inventory star schema Metrics calculated at different levels of aggregation Metrics calculated with different filtering criteria Simulating outer joins on RDBMS platforms that do not support them natively Querying multiple tables due to application-level partitioning Support for these scenarios, especially when combined together, provide a framework for significant analytic questions and value to the users of the system. One of the main optimizations the MicroStrategy SQL Engine makes is to generate SQL that performs these multi-pass queries as efficiently as possible. VLDB Driver MicroStrategy SQL engine contains a list of very large database (VLDB) drivers containing a significant number of settings that allow minute adjustments to the generated SQL. These are primarily used for MicroStrategy to optimally support RDBMS platforms, such as Oracle. MicroStrategy Incorporated, 2015 Page 6 of 38

7 Integration Best Practices for Performance This section will focus on the areas where MicroStrategy and Oracle have integration points of interest. The intent of this document is to provide clarification on how the two products work together and the necessary steps to ensure they are optimized. This document focuses mostly on configuration rather than installation steps. The goal is to highlight concepts that refer to detailed how -to steps in the relevant parts of the respective product documentation. MicroStrategy proposes ROLAP architecture and hence pushes as much processing to the underlying data warehousing platform as possible. Thus, performance of data warehouse platform is key to the performance of MicroStrategy analytical queries. MicroStrategy analytical queries are OLAP in nature and typically, process large set of data rows. Tuning a data warehouse ecosystem requires tweaking at multiple levels i.e. 1. Database system (Hardware, OS, and Database configuration) 2. Physical schema design 3. Optimal query writing 4. Clients connecting to data warehouse (E.g. MicroStrategy) This section discusses performance features of Oracle at database system and optimal query writing levels and other performance considerations. System Considerations This section discusses the performance features and considerations for hardware, operating system and database software. These features affect the whole database system and hence, care should be taken to implement them as per recommended best practices from the inception of system. Note that the considerations for these features may change with new releases of hardware, operating system and database software thus; revisions must be made during upgrade of any of the 3. Hyper Threaded CPU With hyper threading enabled on nodes, each physical CPU core is split into multiple (typically 2) logical cores. This means OS can now allocate multiple processes to a physical core. This enables efficient use of physical core cycles which otherwise is wasted in waiting for data read. The efficiency of this CPU feature however, is dependent on OS and database software being aware and written for hyper threading. Hyper threading is a recommended best practice for Oracle. As a thumb rule enable CPU hyperthreading for your workload if, adverse effects are seen revert back to no hyper threading. This feature is completely transparent to MicroStrategy hence, enabling or disabling hyper threading requires no change on MicroStrategy. Hyper threading may result in increased concurrency on data warehouse system and if so, MicroStrategy WLM can be tweaked to leverage the concurrency gain. MicroStrategy Incorporated, 2015 Page 7 of 38

8 Huge Page Memory Page is a smallest unit of memory I/O for operating system. A typical OS page size is 4K or 8K however; with huge pages enabled page size can be set to higher values e.g. 2MB for Linux. This decreases size of page table maintained by OS and also increases TLB hit ratio, leading to performance benefits. Huge pages are typically beneficial in data warehousing environments which have nodes with high memory capacity. Oracle recommends usage of memory with huge pages. However, huge pages are not compatible with certain Oracle feature e.g. Automatic Memory Management. Detail for Oracle huge page recommendation can be found here. OS huge pages are transparent to MicroStrategy and their implementation requires no change on MicroStrategy. 32 Bit OS on > 4G Memory 32 Bit OS and applications have limit of 4G to total addressable memory. Certain database vendors have parameters that can enable usage of more than 4GB memory on 32 bit OS; e.g. Oracle. Addressing >4G would enable database environment to use total memory available on the hardware. Oracle's Very Large Memory (VLM) configurations allow a 32-bit Oracle Database to access more than 4GB RAM that is traditionally available to Linux applications. Enable this option for the hardware systems with more than 4G of available memory. Detail for Oracle VLM recommendation can be found here. This feature is transparent to MicroStrategy and its implementation requires no change on MicroStrategy. Database Block Size A block is smallest unit of disk I/O for database software file system. Database software are written to store a table data in contiguous data blocks, whenever possible. Larger data blocks mean fewer I/O calls from database for a full table scan. E.g. database with 4K block size requires 1024 I/O calls to read a 4MB table whereas with 1M block size requires only 4 I/O calls. Analytical queries generally scan full or close to full table thus, having larger database block size would have clear performance benefit in data warehouse environment. Block Size can be set for an Oracle table space. Default table space block size is set from DB_BLOCK_SIZE parameter which is equal to Operating system block size; typically either 4K or 8K. DB_BLOCK_SIZE initialization parameter specifies the standard block size for the database. For MicroStrategy workloads, consider creating a separate table space with increased block size (non-standard) for fact tables and large dimension tables typically, 16K or 32K block size serves the purpose. Ensure that the database block size is a multiple of the operating system block size. This feature is transparent to MicroStrategy and its implementation requires no change on MicroStrategy. MicroStrategy Incorporated, 2015 Page 8 of 38

9 Node Clustering Nodes (computers) are connected together to form a single system known as cluster. Multiple or all nodes in a cluster can work together to process large dataset for an analytical query thus, enabling faster query response time. In cluster setups where all nodes are not used; free nodes can process concurrent query without affecting the response time; thus providing workload balance and increased concurrency on the system. Multiple commodity hardware nodes are clustered using Oracle cluster software such configuration is commonly known as Oracle RAC (Real application Cluster). Typically configured in shared disk, Oracle RAC nodes are clustered and connected to network attached array of storage. Oracle RAC can be configured to span a MicroStrategy analytical query across all or multiple nodes thus, taking advantage of their processing power. Multiple nodes of an Oracle cluster can also be setup to divide workload of concurrent queries between them. More nodes can be added to the Oracle RAC to scale up the performance as data or number of queries grows. A premium offering is Oracle s Exadata appliance which can be used in lieu of commodity hardware. The major difference is Exadata cells are used for storage array. Unlike conventional storage arrays used in Oracle RAC, Exadata storage cells can perform SQL operations like project, filter, etc on locally residing data (a.k.a. Smart Scan). Thus, offloads these SQL operations from nodes CPUs to storage cells and also reduces data to be moved to database server nodes over network. Exadata appliance also uses faster InfiniBand fabric (40GB/sec) to transfer processed data from exadata storage cells to database server nodes. Smart Flash storage in Exadata appliance automatically caches frequently accessed data into fast flash storage. A node failure can bring the cluster down since; working of a cluster is dependent on working of all participating nodes. To avoid such occurrences, some nodes in a cluster may stand by. In case of a node failure, these standby nodes take over operations of failed node thus, providing node failure tolerance to the cluster. This topic is related to high availability of database system and lies out of scope for this document. A cluster to MicroStrategy appears as a single big machine and hence underlying setup is transparent. However, no of queries submitted by MicroStrategy, to the data warehouse cluster, should be tweaked based on the concurrent queries can be handled by underlying cluster setup. This can be done using MicroStrategy's workload management feature (WLM). Parallelism SQL processing is split into multiple threads which can be executed in parallel independent of each other. Such parallel processing can dramatically enhance performance for heavy SQL operations like full table scan of large fact table, which is a typical characteristic of analytical queries. MicroStrategy Incorporated, 2015 Page 9 of 38

10 Multiple parallel threads are opened for each query running on the system which can quickly exhaust available system resources. Hence, no. of parallel threads created for each query inversely impacts query concurrency that can be handled by a system. Oracle, by default, opens fewer parallel threads per query thus, is capable for handling high concurrency. However, Oracle provides flexibility to increase parallelism for queries when heavy processing is needed. A typical data warehouse workload has heavy processing queries with low query concurrency and hence, it is recommended to increase no of parallel threads equal to no of cores on the system. For highly concurrent workloads, reduce the number of parallel threads that are opened on the database. Resource utilization logs provide good indicators for resource exhaustion or underutilization; tweak parallelism and concurrency parameters accordingly. In Oracle database, degree of parallelism (DOP) settings defines the no of parallel threads to be created for the SQL. DOP can be explicitly set by specifying the no of threads in PARALLEL clause e.g. PARALLEL(8). Degree of Parallelism can be set at query level using PARALLEL hint, at session level using ALTER SESSION, and at TABLE level by defining parallelism in CREATE TABLE statement. Additionally, Oracle can automatically determine parallelism and DOP at runtime for each query; the feature is known as Auto DOP. It is recommended to use Auto degree of parallelism and can be set by PARALLEL_DEGREE_POLICY = AUTO parameter. Read URL for details on Oracle s degree of parallelism feature. When the report SQLs get overly complicated, the DOP calculated by the Optimizer (with Auto DOP) may not be the most ideal. In such cases, the recommendation is to run the query/report by capping the max DOP using parallel_degree_limit, and run multiple tests by changing the max DOP to figure out the ideal DOP for the workload/report. This is because that certain complicated SQLs are just too hard to parallelise, and beyond a certain DOP the parallelization benefits being to wear off. Parallelism impacts no of queries that can be submitted by MicroStrategy, to the data warehouse cluster hence, it should be tweaked based on the concurrent queries that can be handled by underlying cluster setup. This can be done using MicroStrategy's workload management feature (WLM). MicroStrategy Incorporated, 2015 Page 10 of 38

11 Optimal Query Generation MicroStrategy differentiates two kinds of tables accessed as part of its analytical workload. The first consists of a set of base tables. These generally contain the fact and attribute data as required to implement the desired logical data model for the analytical application. These tables are created, populated, and maintained outside of MicroStrategy. The second is a set of intermediate tables are required to implement multi-pass SQL. They are temporary in nature and hold intermediate results. They are created, populated and dropped by MicroStrategy. MicroStrategy implements a list of strategies to generate intermediate tables the choice of which can have a significant impact on query performance. MicroStrategy s SQL Engine is capable of generating different SQL syntax in order to provide optimal support for any given RBDMS. The following sections discuss the default choices for the Oracle Database and potential alternatives. Intermediate Tables By default any RBDMS contains a lot of functionality that ensure ACID compliance when creating tables. In the context of creating temporary result sets for answering an interactive user query a lot of this functionality can be considered overhead as the goal is primarily a read-only operation against base tables. The Oracle Database implements a range of features that help reduce the overhead involved when processing intermediate results for multi-pass queries. NOLOGGING Tables One way to implement multi-pass SQL is to execute each pass (i.e. each query block) in a separate table. Oracle NOLOGGING tables (called unrecoverable tables before Oracle 8) do not incur writing to the redo logs during creation and population, so there is low overhead cost for creating them. NOLOGGING tables are catalogued, so there is some slight overhead due to system catalog access. MicroStrategy by default uses the Nologging table feature when generating intermediate tables. create table ZZSP00 nologging as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_MNTH_SLS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id create table ZZSP01 nologging as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.units_received) WJXBFS1 from INVENTORY_ORDERS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, MicroStrategy Incorporated, 2015 Page 11 of 38

12 a12.subcat_id select pa1.subcat_id SUBCAT_ID, a11.subcat_desc SUBCAT_DESC, pa1.year_id YEAR_ID, pa1.wjxbfs1 WJXBFS1, pa2.wjxbfs1 WJXBFS2 from ZZSP00 pa1, ZZSP01 pa2, LU_SUBCATEG a11 where pa1.subcat_id = pa2.subcat_id and pa1.year_id = pa2.year_id and pa1.subcat_id = a11.subcat_id drop table ZZSP00 drop table ZZSP01 Global Temporary Tables Global Temporary Tables are another variation on creating temporary tables for multi-pass SQL available in Oracle. Like NOLOGGING tables, Global Temporary Tables do not incur logging to the redo logs for creation and population. In addition, Global Temporary Tables do not need to access the system catalog as they are not catalogued. However, parallel DML (such as insert statements) is not supported for these tables. The level of benefit gained from full parallelism compared with no catalog access should be considered in choosing this property value. To use Global Temporary Tables, the following VLDB settings should be modified: VLDB Category VLDB Property Setting Value Tables Intermediate Table Type Permanent table Tables Table Qualifier global temporary Tables Table Space on commit preserve rows Tables Drop Temp Table Method Truncate table, then drop after final pass create global temporary table ZZSP00 on commit preserve rows as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_MNTH_SLS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id create global temporary table ZZSP01 on commit preserve rows as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.units_received) WJXBFS1 from INVENTORY_ORDERS a11, LU_ITEM a12, LU_MONTH a13 MicroStrategy Incorporated, 2015 Page 12 of 38

13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id select pa1.subcat_id SUBCAT_ID, a11.subcat_desc SUBCAT_DESC, pa1.year_id YEAR_ID, pa1.wjxbfs1 WJXBFS1, pa2.wjxbfs1 WJXBFS2 from ZZSP00 pa1, ZZSP01 pa2, LU_SUBCATEG a11 where pa1.subcat_id = pa2.subcat_id and pa1.year_id = pa2.year_id and pa1.subcat_id = a11.subcat_id truncate table ZZSP00 drop table ZZSP00 truncate table ZZSP01 drop table ZZSP01 Comparison of NOLOGGING tables and GLOBAL TEMPORARY TABLES highlighting features that are relevant to MicroStrategy s usage: NOLOGGING Tables No logging to redo logs during creation or population Created in user s default table space Supports indexes Supports collection of statistics Supports parallel DML GLOBAL TEMPORARY Tables No logging to redo logs during creation or population Created in user s default temporary table space Supports indexes Supports collection of statistics Does not support parallel DML Derived Tables (Inline Views) Rather than implement each pass in a separate table, Derived Table syntax allows the SQL Engine to issue additional passes as query blocks in the FROM clause. Instead of issuing multiple SQL passes that create intermediate tables, the SQL engine generates a single large pass of SQL. This can allow queries to run faster since there are no CREATE TABLE or DROP TABLE statements to catalog, no corresponding locks on the system tables, and no logging of records inserted into a physical table. In theory, derived table syntax should result in query performance at least as good as when using temporary tables. Customers may want to experiment with this option to determine if it is beneficial in their specific environment. MicroStrategy Incorporated, 2015 Page 13 of 38

14 VLDB Category VLDB Property Setting Value Tables Intermediate Table Type Derived table select pa11.subcat_id SUBCAT_ID, a13.subcat_desc SUBCAT_DESC, pa11.year_id YEAR_ID, pa11.wjxbfs1 WJXBFS1, pa12.wjxbfs1 WJXBFS2 from (select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_MNTH_SLS a11 join LU_ITEM a12 on (a11.item_id = a12.item_id) join LU_MONTH a13 on (a11.month_id = a13.month_id) group by a13.year_id, a12.subcat_id ) pa11 join (select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.units_received) WJXBFS1 from INVENTORY_ORDERS a11 join LU_ITEM a12 on (a11.item_id = a12.item_id) join LU_MONTH a13 on (a11.month_id = a13.month_id) group by a13.year_id, a12.subcat_id ) pa12 on (pa11.subcat_id = pa12.subcat_id and pa11.year_id = pa12.year_id) join LU_SUBCATEG a13 on (pa11.subcat_id = a13.subcat_id) Note that not all reports are able to use derived tables. There are two primary scenarios in which temporary tables must be used instead of derived tables: When a report uses a function supported in the MicroStrategy analytical engine that is not supported in Oracle (e.g. many of the functions in the financial and statistical function packages). If these functions are used in intermediate calculations, the MicroStrategy analytical engine will perform calculations and then insert records back into the RDBMS for further processing. Inserting records back into the database requires a temporary table. When a report uses the MicroStrategy partitioning feature. When using partitioning, the SQL Engine executes a portion of the query in order to determine which partitions to use. The results are then used to construct the rest of the query. Because the full structure of the query is not known prior to execution, the SQL engine must use temporary tables to execute the query in multiple steps. MicroStrategy Incorporated, 2015 Page 14 of 38

15 These situations do not cover 100% of the cases in which temporary tables must be used. The rest of the cases are relatively obscure combinations of VLDB settings, such as certain combinations of Sub Query Type plus outer join settings on metrics plus non-aggregatable metrics. If the Intermediate Table Type is set to Derived tables, then either True temporary tables should be specified as the fallback table type should the specific report requirements not support the use of derived table expressions. For each report, the SQL Engine will follow an all or nothing policy in determining whether to use derived table syntax. If the entire report cannot be resolved in a single statement with derived tables, the SQL Engine will automatically revert to the Fallback Table Type syntax and not use derived table syntax at all for the report. When using derived tables, the UNION Multiple INSERT setting should be setting to Use Union. This allows the engine to use UNION statements within inline views instead of multiple INSERT INTO statements. This setting is relevant for reports that use partitioning and consolidations. Query Factoring Clause (Common Table Expressions) Rather than implement each pass in a separate table, Common Table Expression syntax allows the SQL Engine to issue additional passes as query blocks in the WITH clause. Oracle calls this syntax the query factoring clause. The query factoring clause defines a name for a query block that can be referenced as a table name in any FROM clause of the full SELECT statement that follows. Common table expressions provide a way to perform many operations within a single SQL statement. As with derived table syntax, there are no CREATE TABLE or DROP TABLE statements to catalog, no corresponding locks on the system tables, and no logging of records inserted into a physical table. In theory, common table expression syntax should result in query performance at least as good as when using temporary tables or when using derived table syntax. Customers may want to experiment with this option to determine if it is beneficial in their specific environment. VLDB Category VLDB Property Setting Value Tables Intermediate Table Type Common Table Expressions with pa1 as (select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_MNTH_SLS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id ), pa2 as (select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.units_received) WJXBFS1 MicroStrategy Incorporated, 2015 Page 15 of 38

16 from INVENTORY_ORDERS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id ) select pa1.subcat_id SUBCAT_ID, a11.subcat_desc SUBCAT_DESC, pa1.year_id YEAR_ID, pa1.wjxbfs1 WJXBFS1, pa2.wjxbfs1 WJXBFS2 from pa1, pa2, LU_SUBCATEG a11 where pa1.subcat_id = pa2.subcat_id and pa1.year_id = pa2.year_id and pa1.subcat_id = a11.subcat_id Note that not all reports can use common table expressions; the same restrictions apply as with derived table syntax. In the event that the SQL engine cannot construct the query using common table expressions, it will fall back to either True temporary tables or Permanent tables as indicated in the Fallback Table Type setting. Table Creation Type When creating intermediate tables in Oracle, performance is better when using implicit table creation than when using explicit table creation, the reason is that in the implicit table syntax the nologging option can be used, with this option you can create a table based on a query without generating redo log entries, the performance of the create table command will improve because less work is being done; the larger the table, the greater the impact. Implicit table syntax create table ZZSP01 nologging as select... Explicit table syntax create table ZZSP01( <column_expressions> ); insert into ZZSP01... Sub Query Type There are many cases in which the SQL Engine will generate subqueries (i.e. query blocks in the WHERE clause): Reports that use Relationship Filters Reports that use NOT IN set qualification, e.g. AND NOT <metric_qualification> or AND NOT <relationship_filter> Reports that use Attribute qualification with M-M relationships, e.g. show Revenue by Category, filter on Catalog MicroStrategy Incorporated, 2015 Page 16 of 38

17 Reports that raise the level of a filter, e.g. dimensional metric at Region level, but qualify on Store Reports that use non-aggregatable metrics, e.g. inventory metrics Reports that use Dimensional extensions Reports that use Attribute to attribute comparison in the filter The default setting for Sub Query Type for Oracle 11gR2 is Option 3 Where (col1, col2) in (Select s1.col1, s1.col2 ). This setting instructs the SQL Engine to generate noncorrelated IN subqueries instead of correlated EXISTS subqueries. The IN subquery allows the Oracle optimizer to use index access on tables within the subquery. select a11.item_id ITEM_ID, a12.item_name ITEM_NAME, sum(a11.tot_dollar_sales) WJXBFS1 from ITEM_MNTH_SLS a11, LU_ITEM a12 where a11.item_id = a12.item_id and in (((a11.item_id) (select r11.item_id from ITEM_MNTH_SLS r11 where r11.month_id in (200012))) and a11.month_id in (200012)) group by a11.item_id, a12.item_name Some reports may perform better with Option 6 Use temporary table, falling back to IN for correlated subquery. Reports that include a filter with an AND NOT set qualification (e.g. AND NOT relationship filter) will likely benefit from using temp tables to resolve the subquery. However, such reports will probably benefit more from using the Set Operator Optimization discussed below. The other settings are not likely to be advantageous with Oracle. Option 0 WHERE EXISTS (select * ) IN performs better than EXISTS. This setting is useful for RDBMS platforms that do not support any of the other syntax. Option 1 WHERE EXISTS (select col1, col2 ) IN performs better than EXISTS. This setting is useful for RDBMS platforms that do not support other syntax and for which selecting column names performs better than select *. Option 2 WHERE col1 IN (select s1.col1), falling back to [Option 0] for multiple columns Oracle supports multiple columns in an IN subquery. This is setting is useful for RDBMS platforms that do not support multiple columns in an IN subquery. Option 4 Use temporary table falling back to [Option 0] for correlated subqueries IN performs better than EXISTS for. Option 6 is better than Option 4 for Oracle. Option 5 WHERE col1 IN (select s1.col1), falling back to [Option 1] for multiple columns Oracle supports multiple columns in an IN subquery. This is setting is useful for RDBMS platforms that do not support multiple columns in an IN subquery. MicroStrategy Incorporated, 2015 Page 17 of 38

18 SQL Global Optimization This setting can substantially reduce the number of SQL passes generated by MicroStrategy. In MicroStrategy 9, SQL Global Optimization reduces the total number of SQL passes with the following optimizations: o o o o Eliminates unused SQL passes, e.g. a temp table is created but not referenced in a later pass Reuses redundant SQL passes o E.g. exact same temp table is created multiple times single temp table is created Combines SQL passes where the SELECT list is different o E.g. two temp tables have same FROM clause, same JOINs, same WHERE clause, same GROUP BY SELECT lists are combined into single SELECT statement Combines SQL passes where the WHERE clause is different o E.g. two temp tables have same SELECT list, same FROM clause, same JOINs, same GROUP BY predicates from the WHERE clause are moved into CASE statements in the SELECT list. See the System Administration Guide for a complete description of the cases covered by this setting. The default setting for Oracle 11gR2 is to enable SQL Global Optimization at its highest level. If your Database Instance is configured as an earlier version of Oracle, you may have to enable this setting manually. Set Operator Optimization This setting is used to combine multiple subqueries into a single subquery using set operators (i.e. UNION, INTERSECT, MINUS). The default setting for Oracle 11gR2 is to enable Set Operator Optimization. If the MicroStrategy Database Instance was initially configured for an Oracle version of 8 or earlier, you will have to enable this setting manually. select a13.category_id BIG_CATEGORY_ID, a14.category_desc CATEGORY_DESC, a14.category_desc CATEGORY_DESC0, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_EMP_SLS a11 join LU_ITEM a12 on (a11.item_id = a12.item_id) join LU_SUBCATEG a13 on (a12.subcat_id = a13.subcat_id) join LU_CATEGORY a14 on (a13.category_id = a14.category_id) where (a11.emp_id) in (((select r11.emp_id from ITEM_EMP_SLS r11 where r11.item_id = 37) minus (select r11.emp_id from ITEM_EMP_SLS r11 where r11.item_id = 217))) group by a13.category_id, a14.category_desc, a14.category_desc MicroStrategy Incorporated, 2015 Page 18 of 38

19 Join Type Oracle s traditional convention for join syntax is to indicate the join column within the WHERE clause. This convention is referred to as Join 89 Syntax in the Join Type VLDB setting. VLDB Category VLDB Property Setting Value Joins Join Type Join 89 From STORE_DIVISION a21, LOOKUP_STORE a22 where a21.store_nbr = a22.store_nbr Prior to Oracle 10g, the SQL Engine used Join 89 syntax by default. With Oracle 10gR2 and higher, the default is to use the ANSI-92 join syntax. Oracle recommends this syntax, particularly for queries using outer joins. VLDB Category VLDB Property Setting Value Joins Join Type Join 92 From STORE_DIVISION a21 join LOOKUP_STORE a22 on (a21.store_nbr = a22.store_nbr) Full Outer Join Oracle 11g introduced support for a more optimized ANSI-compliant full outer join. To take advantage of this MicroStrategy v9.3 has the Full Outer Join VLDB setting set to Support for Oracle 11g and 11gR2. Oracle supports FULL OUTER JOIN syntax between tables in the FROM clause. By default, MicroStrategy will use FULL OUTER JOIN. create table TEMP1 as select a22.year_id YEAR_ID, sum(a21.reg_sls_dlr) REG_SLS_DLR from REGION_DEPARTMENT a21 join LOOKUP_DAY a22 on (a21.cur_trn_dt = a22.cur_trn_dt) group by a22.year_id create table TEMP2 as select a22.year_id YEAR_ID, sum(a21.tot_sls_dlr) TOT_SLS_DLR from MARKET_DIVISION a21 join LOOKUP_DAY a22 on (a21.cur_trn_dt = a22.cur_trn_dt) group by a22.year_id select distinct a23.year_id YEAR_ID, a23.year_desc YEAR_DESC, a21.reg_sls_dlr REG_SLS_DLR, a22.tot_sls_dlr TOT_SLS_DLR from TEMP1 a21 full outer join TEMP2 a22 MicroStrategy Incorporated, 2015 Page 19 of 38

20 on (a21.year_id = a22.year_id) join LOOKUP_YEAR a23 on (coalesce(a21.year_id, a22.year_id) = a23.year_id) A typical report with multiple full outer joins contains several metrics with the Metric Join Type set to Outer; the SQL may look something like the following (SQL for the intermediate passes is omitted for brevity): create table ZZMD00 nologging as select... create table ZZMD01 nologging as select... create table ZZMD02 nologging as select... select coalesce(pa11.customer_id, pa12.customer_id, pa13.customer_id) CUSTOMER_ID, a14.cust_last_name CUST_LAST_NAME, a14.cust_first_name CUST_FIRST_NAME, coalesce(pa11.month_id, pa12.month_id, pa13.month_id) MONTH_ID, a15.month_desc MONTH_DESC, pa11.wjxbfs1 WJXBFS1, pa12.wjxbfs1 WJXBFS2, pa13.wjxbfs1 WJXBFS3 from ZZMD00 pa11 full outer join ZZMD01 pa12 on (pa11.customer_id = pa12.customer_id and pa11.month_id = pa12.month_id) full outer join ZZMD02 pa13 on (coalesce(pa11.customer_id, pa12.customer_id) = pa13.customer_id and coalesce(pa11.month_id, pa12.month_id) = pa13.month_id) join LU_CUSTOMER a14 on (coalesce(pa11.customer_id, pa12.customer_id, pa13.customer_id) = a14.customer_id) join LU_MONTH a15 on (coalesce(pa11.month_id, pa12.month_id, pa13.month_id) = a15.month_id) Note for Oracle 10gR2 As part of Oracle 10gR2 patch set 3 ( ), Oracle back ported their support for a more optimized ANSI-compliant full outer join. (refer to Oracle Metalink: Poor Performance Of Native Full Outer Join [ID ]) To take advantage of this feature and optimize reports containing a full outer join in Oracle 10gR2, set the following: VLDB Category VLDB Property Setting Value Pre/Post Statements Report Pre Statement 1 alter session set optimizer_native_full_outer_join = force; Analyzing Statistics on Intermediate Tables Using VLDB settings, the MicroStrategy SQL engine can create indexes for intermediate tables. The Table Post Statement VLDB setting can be used to collect optimizer statistics on tables and indexes. In some cases, collecting statistics will help improve query performance. MicroStrategy Incorporated, 2015 Page 20 of 38

21 VLDB Category VLDB Property Setting Value Pre/Post Statements Table Post Statement DBMS_STATS.GATHER_TABLE_STATS( schema,??? ) create table ZZSP00 nologging as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.tot_unit_sales) WJXBFS1 from ITEM_MNTH_SLS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id call DBMS_STATS.GATHER_TABLE_STATS('TUTORIAL_WH', 'ZZSP00') create table ZZSP01 nologging as select a13.year_id YEAR_ID, a12.subcat_id SUBCAT_ID, sum(a11.units_received) WJXBFS1 from INVENTORY_ORDERS a11, LU_ITEM a12, LU_MONTH a13 where a11.item_id = a12.item_id and a11.month_id = a13.month_id group by a13.year_id, a12.subcat_id call DBMS_STATS.GATHER_TABLE_STATS('TUTORIAL_WH', 'ZZSP01') select pa1.subcat_id SUBCAT_ID, a11.subcat_desc SUBCAT_DESC, pa1.year_id YEAR_ID, pa1.wjxbfs1 WJXBFS1, pa2.wjxbfs1 WJXBFS2 from ZZSP00 pa1, ZZSP01 pa2, LU_SUBCATEG a11 where pa1.subcat_id = pa2.subcat_id and pa1.year_id = pa2.year_id and pa1.subcat_id = a11.subcat_id drop table ZZSP00 drop table ZZSP01 Note: ANALYZE ESTIMATE syntax is being deprecated by Oracle. The DBMS_STATS PL/SQL package should be used instead. Optimizer Hints In theory, a SQL application should require no knowledge of the access plan used to resolve SQL queries; it is the responsibility of the database optimizer to select the optimal plan. Indeed, the optimizer is in the best position to select an access plan given its knowledge of indexes, cardinality and physical distribution of data, and other relevant statistics. However, under certain circumstances in practical implementations, the database optimizer may choose a sub-optimal plan. Oracle supports the use of hints to influence the optimizer in selecting an access MicroStrategy Incorporated, 2015 Page 21 of 38

22 plan. A hint is a comment that passes instructions to the optimizer on choosing an execution plan for the statement. The SQL Hint VLDB property allows the placement of a general hint within a MicroStrategy issued SQL statement. For example, when set to --+ STAR, the following SQL is generated. create table ZZSP00 nologging as select --+ STAR a22.year_id YEAR_ID,... Note that MicroStrategy metadata does not contain information on table indexes, thus it is not possible to issue a specific hint that would dynamically choose an index from the selected fact table. Use of this property should be considered carefully and should generally be applied on a report by report basis. Planning a pre-aggregation strategy and using Materialized Views Pre-aggregating data is a common and powerful way to improve end user query response times. Rather than aggregating many, many rows at query time, pre-summarization allows the database to perform row access and aggregation ahead of time, satisfying the query at request time much faster. Data warehouse practitioners have long used physical summary tables to pre-aggregate data in a ROLAP environment. MicroStrategy is an aggregate aware application that allows end users to query physical summary tables without needing to specify which table to use to satisfy the query. Oracle 12c supports Materialized Views, a database server feature that provides for the creation of, maintenance of, and automatic navigation to aggregate tables. This makes the use of aggregate tables entirely transparent to the end user and to the MicroStrategy developer. With Materialized Views, Oracle will rewrite the query to use the summary table, provided that it determines its cost to be less than the original query. Query rewrite in Oracle is based on the SQL text itself or on rewrite conditions explicitly determined by defining dimensions in Oracle. Any MicroStrategy generated SQL could make use of query rewrite and materialized views. There is no special syntax to invoke. Provided that the proper conditions (Dimensions, Levels, Hierarchies, query rewrite, etc.) are set up on Oracle, MicroStrategy will transparently take advantage of Materialized Views. Materialized views can provide some key advantages over physical summary tables. Oracle provides refresh features that may make it easier to set up materialized views than physical aggregate tables. Additionally, Oracle provides Advisor tools that can suggest materialized views to build based on characteristics of the data (e.g. sparsity). Note however, that there are some cases in which physical tables will be used more often than materialized views. MicroStrategy can take advantage of its knowledge of hierarchical relationships to select from physical summary tables; with materialized views in Oracle, this information must be redefined via hierarchical links. For example, within a Time hierarchy having Day, Week, Month, and Year attribute levels; direct mappings between Year to Month, Year to Week, Year to Day, Month to Week, Month to Day, and Week to Day need to be made. The Oracle optimizer cannot infer a relationship between Year to Week from relationships between Year to Month and Month to Week. Thus materialized view links need to be created for every possible combination of pre-aggregated material view level to levels where a physical table exists. MicroStrategy Incorporated, 2015 Page 22 of 38

23 However, physical aggregate tables may have advantage of their own. Depending on the organization s ETL capabilities, it may be more manageable to build and refresh real aggregate tables as part of the ETL process. Additionally, there are some cases where Oracle query rewrite will not access materialized views and will miss opportunities to access pre-aggregated data where MicroStrategy aggregate navigation of real summary tables would not miss the opportunity. For example, suppose a base table exists at the Day level and a user submits a query at the Year level. If a physical table is created at the Month level, the MicroStrategy application will access that table to fulfill the Year level query; Oracle does not inherently know the relationship between Year and Month. This can be overcome by redefining the dimensional model in Oracle via DIMENSIONS and HIERARCHIES, but it does represent additional work to set up. There may be additional cases where Oracle query rewrite will miss a summary where the SQL Engine would not miss it. Whether using materialized views or real aggregate tables, an equally challenging aspect of defining an aggregate strategy is selecting which aggregations to pre-summarize. The optimal aggregation strategy needs to consider the query profile, storage space constraints, build and refresh time constraints, and data characteristics such as sparsity. In general, MicroStrategy is neutral to the use of materialized views. If you think it is easier to build and refresh materialized views than real aggregate tables, go ahead and use them. If you use them, you should either define the materialized views to MicroStrategy as available summary tables or define the dimensional model to Oracle so it can be used during query rewrite. Oracle XML Features Oracle 12c supports storing, managing, and querying XML data. Oracle has a built-in data type for XML called XMLType so that users can store XML documents in columns in relational tables. Oracle also provides functions on the XML data type that allow users to manipulate and query XML data from within SQL statements. MicroStrategy can take advantage of these methods to access XML data stored in Oracle. The following examples use XML documents like the one below to represent information about Books. Instead of storing Title, Author, etc. in separate relational columns, the Book information is stored in an XMLType column. <book pages="110"> <title>history of Cats</title> <author> <fn>john</fn> <ln>doe</ln> </author> <author> <fn>jane</fn> <mi>e.</mi> <ln>doe</ln> </author> <chapter pages="10">beginning</chapter> <chapter pages="80">middle</chapter> <chapter pages="20">end</chapter> </book> MicroStrategy Incorporated, 2015 Page 23 of 38

24 The examples are built on a simple MicroStrategy project using the tables below. LU_BOOK LU_MONTH LU_STORE BOOK_SALES BOOK_ID integer MONTH_ID integer STORE_ID integer BOOK_ID integer BOOK_XML xmltype YEAR_ID integer STORE_NAME varchar(50) STORE_ID integer REGION_ID integer MONTH_ID integer REGION_NAME varchar(50) DOLLARS integer UNITS integer The following schema objects are defined: Attributes o = = BOOK_XML o = YEAR_ID o = MONTH_ID; Parent = Year o = = REGION_NAME o = = STORE_NAME; Parent = Region Facts o Dollars = DOLLARS o Units = UNITS Note that the Book attribute does not have a Desc form, but rather has a form called XML that maps to the XMLType column in the LU_BOOK table. The XML form should not be used as a default display form. In fact, the XML form should not be displayed in the grid at all. When importing XMLType columns via the Warehouse Catalog Editor, the Table Structure dialog will show the XMLType data type will be shown as Unknown. This is not a problem in of itself, but there are a few implications: Data of an XMLType column is likely to be displayed incorrectly or garbled in a grid. Attribute forms or facts mapped to XMLType columns should be wrapped in appropriate functions (e.g. CAST) that return strings or other standard data types. The XMLType data type will not be created by Engine-generated SQL when creating a data mart table or temp table that requires the XMLType column. However, as long as an XMLType column is not used as the ID form of an attribute or directly as a metric column, there should be no need to create a temp table using the XMLType data type. Filtering on XML data using EXISTSNODE MicroStrategy SQL can generate the EXISTSNODE syntax via a Filter using ApplyComparison. ApplyComparison( existsnode(#0, /book/author[ln="doe"] ) = 1, Book@XML) select a11.book_id BOOK_ID, sum(a11.dollars) WJXBFS1 from BOOK_SALES a11 join LU_BOOK a12 on (a11.book_id = a12.book_id) where existsnode(a12.book_xml, '/book/author[ln="doe"]') = 1 group by a11.book_id Displaying elements from XML data using EXTRACT and EXTRACTVALUE The EXTRACT method can be called in Attribute Form, Fact, or Metric expressions using ApplySimple. ApplySimple( cast(extract(#0, /book/title ) as varchar(100)), Book@XML) MicroStrategy Incorporated, 2015 Page 24 of 38

25 EXTRACT returns XML, while EXTRACTVALUE returns a specified SQL data type. Wrap calls to EXTRACT with CAST to convert XML to character string select a11.book_id BOOK_ID, cast(extract(a11.book_xml, '/book/title') as varchar(100)) WJXBFS1 from LU_BOOK a11 Metrics Title Book 1 <title>history of Dogs</title> 2 <title>history of Cats</title> 3 <title>counting to Three</title> The EXTRACTVALUE method can be called in Attribute Form, Fact, or Metric expressions using ApplySimple. ApplySimple( extractvalue(#0, /book[1]/title ), Book@XML) The first argument to EXTRACTVALUE must be a single element; if the XQuery expression results in multiple elements, an error will result. EXTRACTVALUE can automatically map the XQuery return values to the appropriate SQL data type. select a11.book_id BOOK_ID, extractvalue(a11.book_xml, '/book/title') WJXBFS1 from LU_BOOK a11 Metrics Title Book 1 History of Dogs 2 History of Cats 3 Counting to Three Modeling XML data as a Logical Table using XMLTABLE Oracle s XMLTABLE function returns a result table from the evaluation of XQuery expressions against data in an XMLType column. We can use MicroStrategy logical views to make a column of XML look like a table to the SQL Engine. The data can then be accessed like any other table modeled in the MicroStrategy schema. For example, define a new Logical Table called LV_CHAPTER using the SQL statement below. select b.book_id, t.title, t.authorfn, t.authormi, t.authorln, t.chapter, t.chapterlength from LU_BOOK b, xmltable('for $c in /book/chapter return <book>{$c/../title}{$c/../author}{$c}</book>' passing b.book_xml columns Title varchar(50) path '/book/title', AuthorFN varchar(50) path '/book/author[1]/fn', AuthorMI varchar(50) path '/book/author[1]/mi', AuthorLN varchar(50) path '/book/author[1]/ln', Chapter varchar(50) path 'chapter', ChapterLength varchar(50) path 'chapter/@pages') t MicroStrategy Incorporated, 2015 Page 25 of 38

26 The XMLTABLE table function takes the XMLType column LU_BOOK.BOOK_XML and returns a row for every element matching /book/chapter (i.e. one row for every chapter). Each row also contains book, title, and author elements so that they can be mapped to columns. The COLUMNS phrase contains a set of XQuery expressions that map XML elements to columns in the resulting table. The user must be aware of a few rules for these mappings: If the path expression returns no elements, then the column value is NULL for the current row. If the path expression returns exactly one element, then that element will be mapped to the data type specified. If the path expression returns more than one element, the function will return an error. The SELECT statement above returns results like the following. BOOK_ID TITLE AUTHORFN AUTHORMI AUTHORLN CHAPTER CHAPTER LENGTH 1 History of Dogs John NULL Doe Introduction NULL 1 History of Dogs John NULL Doe Body NULL 1 History of Dogs John NULL Doe Conclusion NULL 2 History of Cats John NULL Doe Beginning 10 2 History of Cats John NULL Doe Middle 80 2 History of Cats John NULL Doe End 20 3 Counting to Three John Q Public One 10 3 Counting to Three John Q Public Two Counting to Three John Q Public Three 15 Since each chapter is associated with a book that has only one title element, the /book/title path returns one element per row. Since each book may have multiple authors, we have to use the /book/author[1] path to specify the first author element. Without the [1] syntax, the XML table function would return an error since multiple author elements would be returned for a single row. Finally, since the <mi> element is optional, NULLs are returned for some rows for the /book/author[1]/mi path. Using the new Logical Table as a source table, define a new attribute and a new fact: Attribute = Chapter, Parent = Book Fact ChapterLength: ChapterLength Also, ensure that LV_CHAPTER is a source table for Book@ID. The logical view syntax will be included in SQL generated by the Engine. Report: Attributes: Chapter Metrics o Book Units Sold: sum(units) {Book*, ~+} o Number of Pages: sum(chapterlength) {~+} create table ZZMD00 nologging as select a11.chapter Chapter, a11.chapterlength WJXBFS1 MicroStrategy Incorporated, 2015 Page 26 of 38

27 from (select b.book_id, t.title, t.authorfn, t.authormi, t.authorln, t.chapter, t.chapterlength from LU_BOOK b, xmltable('for $c in /book/chapter return <book>{$c/../title}{$c/../author}{$c}</book>' passing b.book_xml columns Title varchar(50) path '/book/title', AuthorFN varchar(50) path '/book/author[1]/fn', AuthorMI varchar(50) path '/book/author[1]/mi', AuthorLN varchar(50) path '/book/author[1]/ln', Chapter varchar(50) path 'chapter', ChapterLength varchar(50) path t ) a11 create table ZZMD01 nologging as select a11.book_id BOOK_ID, sum(a11.units) WJXBFS1 from BOOK_SALES a11 group by a11.book_id select pa11.chapter Chapter, pa11.wjxbfs1 WJXBFS1, pa13.wjxbfs1 WJXBFS2 from ZZMD00 pa11 join (select b.book_id, t.title, t.authorfn, t.authormi, t.authorln, t.chapter, t.chapterlength from LU_BOOK b, xmltable('for $c in /book/chapter return <book>{$c/../title}{$c/../author}{$c}</book>' passing b.book_xml columns Title varchar(50) path '/book/title', AuthorFN varchar(50) path '/book/author[1]/fn', AuthorMI varchar(50) path '/book/author[1]/mi', AuthorLN varchar(50) path '/book/author[1]/ln', Chapter varchar(50) path 'chapter', ChapterLength varchar(50) path 'chapter/@pages') t ) a12 on (pa11.chapter = a12.chapter) join ZZMD01 pa13 on (a12.book_id = pa13.book_id) Using Bitmapped Indexing Bitmapped indexing is an alternative indexing scheme that provides substantial performance benefits and storage savings. Bitmapped indexes are particularly appropriate for data warehousing environments where ad-hoc queries are more common and data is updated less frequently. When used in conjunction with conventional indexing schemes, such as B-tree indexes and parallel processing techniques, bitmapped indexes provide significant performance benefits to a data warehouse. The queries that benefit the most from bitmapped indexes have the following characteristics: The WHERE-clause contains multiple predicates on low-cardinality columns. The individual predicates on these low-cardinality columns select a large number of rows. The bitmapped indexes have been created on some or all of these low-cardinality columns. The tables being queried contain many rows. Bitmapped indexes, like any index available to the Oracle optimizer, are automatically leveraged by MicroStrategy SQL. CUBE, ROLLUP, and Grouping Sets MicroStrategy does not actually make use of CUBE, ROLLUP, or GROUPING SETS. These extensions are useful for populating multidimensional databases that require multiple levels of aggregation. Since MicroStrategy Incorporated, 2015 Page 27 of 38

28 MicroStrategy does not pre-store multiple levels of aggregation, these functions have less value for us. MicroStrategy does generate multiple levels of aggregation when calculating subtotals in the Analytical Engine, Integration with Oracle OLAP Option and Analytic Workspaces MicroStrategy can access data stored in Oracle OLAP cubes (i.e. analytic workspaces). Since Oracle 11gR2, analytic workspaces can be exposed as materialized views accessible by any SQL application. MicroStrategy-generated SQL can access those materialized views. Recommendations and tradeoffs are discussed more completely in the MicroStrategy TN MicroStrategy Integration with Oracle OLAP. Additional VLDB Settings Many of the VLDB properties control string syntax used in SQL queries generated by the MicroStrategy SQL engine. MicroStrategy application developers can further optimize SQL for their specific Oracle environment using these string insertion settings. Possible locations for VLDB optimizations in the query structure are listed below. VLDB String Insertion Settings when using Temporary Tables [Report Pre Statements] [Table Pre Statements] create table <table_name> nologging as select [SQL Hint] <column_expressions> from <tables_and_joins> where <filter_expressions> group by <column_expressions> having <column_expressions>[insert Post String] create [Index Qualifier] index [Index Prefix]<index_name> on <table_name> (<column_expressions>) [Index Post String] [Table Post Statements] select [SQL Hint] <column_expressions> from <tables_and_joins> where <filter_expressions> group by <column_expressions> having <column_expressions>[select Post String][Select Statement Post String] [Report Post Statements] drop table <table_name> purge [Cleanup Post Statements] VLDB String Insertion Settings when using Permanent Tables When Intermediate Table Type is set to Permanent Tables, some additional string-valued settings are enabled so that the user can customize the syntax of CREATE TABLE statement. The SQL below shows the position of these VLDB settings. MicroStrategy Incorporated, 2015 Page 28 of 38

29 [Report Pre Statements] [Table Pre Statements] create [Table Qualifier] table [Table Descriptor][Table Prefix]<table_name> [Table Option] [Table Space] as select [SQL Hint] <column_expressions> from <tables_and_joins> where <filter_expressions> group by <column_expressions> having <column_expressions>[insert Post String] create [Index Qualifier] index [Index Prefix]<index_name> on [Table Prefix]<table_name> (<column_expressions>) [Index Post String] [Table Post Statements] select [SQL Hint] <column_expressions> from <tables_and_joins> where <filter_expressions> group by <column_expressions> having <column_expressions>[select Post String][Select Statement Post String] [Report Post Statements] drop table [Table Prefix]<table_name> purge [Cleanup Post Statements] Extended Analytics Using Oracle Functions The MicroStrategy platform includes libraries of mathematical, statistical, and financial functions that can be applied to any report or customized application. More than 350 analytical functions are available out-of-the-box, and additional libraries can be added by plugging in custom functions or proprietary algorithms. The MicroStrategy platform provides the advantage of determining whether a given function can be performed in Oracle or whether it should be performed in the MicroStrategy engine. This collaboration between MicroStrategy and Oracle allows calculations to be performed in the most appropriate location. In general, the MicroStrategy engine will push as much processing as possible to Oracle, in order to take advantage of the processing power of the database server and database parallelism. Additionally, using RDBMS functions avoids transport of data between the database server and the MicroStrategy Intelligence Server, further improving scalability and system performance. MicroStrategy Incorporated, 2015 Page 29 of 38

30 Analytic Functions: Functions for OLAP OLAP functions are a powerful family of functions for business intelligence and data warehousing applications. These functions are collectively called analytic functions in Oracle and they provide significant analytical capabilities for many business analysis queries. SQL Syntax for OLAP Functions <analytic function> ::= <function_name> OVER ([<partition clause>] [<order by clause> [<windowing clause>]]) MicroStrategy s OLAP functions will be calculated via SQL in Oracle rather than in the MicroStrategy analytical engine. There is no VLDB setting associated with this option. Moving average example select a11.day_date DAY_DATE, a12.region_id REGION_ID, a13.region_name REGION_NAME, sum(a11.tot_dollar_sales) TOT_DOLLAR_SALES, avg(sum(a11.tot_dollar_sales)) over(partition by a12.region_id order by sum(a11.tot_dollar_sales) asc rows 2 preceding) WJXBFS1 from DAY_CTR_SLS a11, LU_CALL_CTR a12, LU_REGION a13 where a11.call_ctr_id = a12.call_ctr_id and a12.region_id = a13.region_id group by a11.day_date, a12.region_id, a13.region_name Ranking The RANK function is closely related to other OLAP functions. Key distinctions of the RANK function are that it requires an order by clause and does not support a window clause. The MicroStrategy Engine is aware that Oracle supports the Rank function, so the ranking will by done by the database rather than the MicroStrategy Analytical Engine. Rank example select a13.region_id REGION_ID, a14.region_name REGION_NAME, a11.emp_id EMP_ID, a12.emp_last_name EMP_LAST_NAME, a12.emp_first_name EMP_FIRST_NAME, sum(a11.tot_dollar_sales) TOT_DOLLAR_SALES, rank () over( order by sum(a11.tot_dollar_sales) desc nulls last) WJXBFS1 from ITEM_EMP_SLS a11, LU_EMPLOYEE a12, LU_CALL_CTR a13, LU_REGION a14 where a11.emp_id = a12.emp_id and a12.call_ctr_id = a13.call_ctr_id and a13.region_id = a14.region_id group by a13.region_id, a14.region_name, MicroStrategy Incorporated, 2015 Page 30 of 38

31 a11.emp_id, a12.emp_last_name, a12.emp_first_name Custom Group Banding via the DECODE statement Custom Group Banding is a powerful technique for generating histograms and other complex analyses. The Engine uses Oracle s DECODE statement to implement custom group banding reports. create table ZZMQ00 nologging as select a11.customer_id CUSTOMER_ID, Decode(sign(a11.TOT_DOLLAR_SALES- 1.0)+sign(a11.TOT_DOLLAR_SALES ), 1, 100.0, 0, decode(1.0, , 1, floor(1.0*(a11.tot_dollar_sales- 1.0)/( )* )), -1, 1, 0) DA58 from CUSTOMER_SLS a11 where (a11.tot_dollar_sales between 1.0 and ) select a12.da58 DA58, sum(a11.tot_dollar_sales) TOT_DOLLAR_SALES from CUSTOMER_SLS a11, ZZMQ00 a12, LU_CUSTOMER a13, LU_CUST_CITY a14 where a11.customer_id = a12.customer_id and a11.customer_id = a13.customer_id and a13.cust_city_id = a14.cust_city_id and a14.cust_state_id in (5) group by a12.da58 drop table ZZMQ00 Other Performance Considerations Collect Statistics One of the most important sources of information the Oracle optimizer uses for choosing access plans is the set of statistics Oracle collects about the system and the data in it. Database statistics include data demographics such as the number of rows in each table, the number of unique values within each column, the skew and distribution of values within each column, etc. The optimizer uses this information when selecting the best access plan from among all the possible access plans that could satisfy a given query. It is important for the statistics to be updated as frequently as practical. Whenever new data is loaded or new indexes are built or any other significant change occurs to data, the statistics should be updated to reflect the change. Note that out-of-date statistics can be worse than not collecting statistics at all. Oracle statistics are fairly transparent to a MicroStrategy application, since the optimizer is their main consumer. However, keeping statistics up to date is a critical factor in overall system performance, i.e. MicroStrategy queries will be affected by out of date statistics. In addition, some MicroStrategy reports create intermediate tables and insert records into them, so it can be important to collect statistics on these intermediate tables as well. See the VLDB Optimizations section for details on how to do this. MicroStrategy Incorporated, 2015 Page 31 of 38

32 MicroStrategy Certification of Oracle Status Listed below is the current status of certification of Oracle by MicroStrategy. Oracle Database 12c OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Oracle Database 11g Release 2 OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Oracle Database 11g OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Oracle Database 10g Release 2 OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Oracle Database 10g OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST MicroStrategy Incorporated, 2015 Page 32 of 38

33 Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Certified: MD, WH, ST Oracle Database 9i and 9i Release 2 OS platform Recommended ODBC Driver Status Windows 32-bit and 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Supported: MD, WH, ST, NC AIX 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Supported: MD, WH, ST Solaris 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Supported: MD, WH, ST RedHat and SUSE 64-bit MicroStrategy ODBC Driver for Oracle Wire Protocol Supported: MD, WH, ST HP-UX 64 bit Itanium MicroStrategy ODBC Driver for Oracle Wire Protocol Supported: MD, WH, ST MicroStrategy Incorporated, 2015 Page 33 of 38

34 Appendix Default VLDB Settings for Oracle Database 11g R2 <Please provide a full list of VLDB settings with their default values> MicroStrategy Incorporated, 2015 Page 34 of 38

35 Data type Mapping MicroStrategy supports the following Oracle data types: MicroStrategy does not currently support the TIMESTAMP with time zone types, the INTERVAL types, or the row id types. MicroStrategy Incorporated, 2015 Page 35 of 38

36 Oracle Database Functions Used by MicroStrategy MicroStrategy makes use of Oracle functions listed below through built-in functions. MicroStrategy Incorporated, 2015 Page 36 of 38

37 MicroStrategy Incorporated, 2015 Page 37 of 38

38 MicroStrategy Incorporated, 2015 Page 38 of 38