BUSINESS INTELLIGENCE

Transcription

1 BUSINESS INTELLIGENCE Understanding Oracle BI Components and Repository Modeling Basics by Abhinav Banerjee

2 2 INTRODUCTION

3 The importance of Business Intelligence (BI) is rising by the day. BI systems, which help organizations make better and more informed decisions, are becoming crucial for success. One of the most common reasons for unsuccessful or delayed BI implementations is an improperly modeled repository not adhering to basic dimensional modeling principles. This book discusses this subject and describes the intricacies related to repository modeling and the associated concepts KPI Partners, Inc.

4 Introduction In an Oracle Business Intelligence (OBI) implementation, the repository plays the most important role as the heart of any BI environment. The entire BI implementation can go wrong because of a repository that is not well designed. Repository (RPD) designing and modeling is one of the most complex processes in an OBI implementation. RPD success is dependant on knowledge of a few principles, which include dimensional modeling and data modeling. In any implementation, we need to ensure our data and dimensional models are welldesigned because the model plays such a significant role. Once these data and dimensional models are in place, we need to ensure that the physical and the business models are also properly designed and developed based on the operational or analytical reporting requirements. Dimensional models are level-based measures, aggregates, multiple facts, multiple logical sources, conforming dimensions, slowly changing dimensions designed to optimize performance for process reporting. This is a different approach from traditional data-relational models which are optimized to process transactions. The complexity of dimensional modeling increases when requirements call for the inclusion of very large data volumes KPI Partners, Inc.

5 Table of Contents CHAPTER 1 Dimensional Modeling 7 Dimensional Modeling (DM) 8 Star Schema 9 Snowflake Schema CHAPTER 2 Oracle BI Architecture 11 Oracle BI Architecture 14 Oracle BI Server 14 Oracle BI Presentation Services 15 Oracle BI Repository 15 Actions Services 16 Security Service 17 Cluster Controller Severs 17 Oracle BI Administration Tool CHAPTER 3 The Oracle BI Development Cycle 19 Oracle BI Development Cycle CHAPTER 4 Building the Oracle BI Model 21 Build Oracle BI Model 22 Import Objects 24 Build Physical Model 27 Physical Layer Best Practices 28 Build BMM Model 34 BMM Layer Best Practices 38 Build the Presentation Layer 40 Presentation Layer Best Practices KPI Partners, Inc.

6 About Author Abhinav Banerjee Abhinav Banerjee is a Consulting Manager working with KPI Partners. He has more than eight years of business intelligence and data integration experience with more than four years in OBIEE (custom and packaged analytics). He has worked with several global clients in various domains that include telecommunications, high tech, manufacturing, energy, education, and oil and gas. He is also a frequent speaker at various Oracle conferences such as COLLABORATE and Oracle OpenWorld. Abhinav specializes in OBIA as well as custom OBIEE implementations. He can be reached at abhinav.banerjee@ kpipartners.com KPI Partners, Inc.

7 About KPI KPI Partners provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications and big data. KPI works with both corporate technology departments and corporate business units to develop value-added decision support solutions, not just new technology deployments KPI Partners, Inc.

8 CHAPTER 1 DIMENSIONAL MODELING 8

9 Dimensional Modeling (DM) DM refers to the methodology used to design data warehouses optimized for performance while querying and reporting. DM uses the concept of facts and dimensions. Facts, or measures, refer to measurable items or numeric values. These include sales quantity, sales amount, time taken, etc. Dimensions are the descriptors, or the relative terms, for the measures. Facts are relative to the dimensions. Some of the most common dimensions include account, customer, product, and date. Dimensional modeling requires the design of star or snowflake schema KPI Partners, Inc.

10 Star Schema The star schema architecture constitutes a central fact table with multiple dimension tables surrounding it. It will have one-tomany relationships between the dimensions and the fact table. The dimensions typically have the relative descriptive attributes that describe business entities. In case of a star schema, no two dimensions will be joined directly; rather, all the joins between the dimensions will be through the central fact table. Joins are completed via a foreign key relationship, with the dimension having the primary key and the fact having the foreign keys to join to the dimension KPI Partners, Inc.

11 Snowflake Schema The snowflake schema architecture also has a central fact table with multiple dimension tables and one to many relationships between the dimension and the fact table, but it also will have one-to-many relationships between dimensions. The dimensions are further normalized into multiple related tables. In this case, multiple dimension tables will exist related to the main dimension table. Normally, we have one-to-many relationships between the dimensions. A primary key-foreign key relationship exists between the dimension and the fact tables as well as between dimensions KPI Partners, Inc.

12 CHAPTER 2 ORACLE BI ARCHITECTURE 12

13 Oracle BI Architecture In order to understand the importance of the repository, we will need to have a look at the Oracle Business Intelligence Enterprise Edition (OBIEE) architecture. The OBIEE repository directly corresponds with the Oracle BI server which, in turn, talks to the database, presentation services, and the security service. OBIEE is a state-of-the-art, next-generation BI platform that provides optimized intelligence to take advantage of relational/ multidimensional database technologies KPI Partners, Inc.

14 It leverages common industry techniques based on data warehousing and dimensional modeling. The OBIEE engine dynamically generates the required SQL based on the user s inputs and designed repository to fetch data for reports from the databases. The various clients include Catalog Manager, BI Administration Tool, Scheduler Tool, Scheduler Job Manager, BI Answers, and Interactive Dashboards. The next section takes a closer look at some of the major components within OBI 11g. The various components of an OBI environment 11g, as shown in Fig. 1, include Java EE Server (WebLogic), Oracle BI Server, Oracle BI Presentation Services, Cluster Controller Services, Oracle BI Scheduler, Oracle Presentation Catalog, Oracle BI Repository, Security Service and BI Java Host KPI Partners, Inc.

15 Figure 1: OBIEE Enterprise Architecture KPI Partners, Inc.

16 Oracle BI Server The Oracle BI server is the core behind the OBIEE platform. It receives analytical requests created by presentation services and efficiently accesses data required by the user s inputs using the defined metadata (RPD). The BI server then generates dynamic SQL to query data in the physical data sources and provides data to the presentation services based on the request received. The BI Sever works with the help of definitions set in configuration files and the metadata (RPD) KPI Partners, Inc.

17 Oracle BI Presentation Services Presentation Services are implemented as an extension to a web server. Presentation Services are deployed on the default OC4J web server. Oracle, depending on the nature and scale of deployment, supports additional web servers, such as WebLogic, WebSphere, and IIS. Presentation Services are responsible for processing the views made available to the user and the data received from the BI server in an appropriate, user-friendly, format to the requesting clients. There also is an associated Oracle BI Java Host service that is responsible for proper display of the charts and graphs. The Presentation Services use a web catalog to store the saved content KPI Partners, Inc.

18 Oracle BI Repository The ORACLE BI Repository holds all the business logic. It is configured through the Oracle BI Administration Tool. The Repository helps build the business model and organize the metadata properly for presentation to users. The repository is comprised of three layers: physical, business model and mapping, and the presentation layer. Each layer appears in a separate pane when opened with the administration tool. Actions Services Actions Services provides the dedicated web services required by the action framework. The action framework enables users to invoke business process based on values of certain defined key indicators. Actions Services exist as action links in the presentation catalog KPI Partners, Inc.

19 Security Service There is a paradigm shift in the security architecture in OBIEE 11g. It implements the same common security architecture as the Fusion Middleware Stack, which leverages the Oracle platform security service (OPSS) and WebLogic authenticators. The various security controls that are available include: Identity Store - an embedded LDAP server in WebLogic to store users and groups. Policy Store - a file to store the permission grants. Credential Store - a file to store user and system credentials for interprocess communication KPI Partners, Inc.

20 Cluster Controller Servers There are two cluster controller servers in OBI 11g: a primary and secondary cluster controller. By default, they get installed in a clustered environment. This provides a proper fallback environment in case of a single installation. The environment consists of a cluster controller and the cluster manager. Oracle BI Administration Tool The Oracle BI administration tool is the thick client used to configure the OBIEE repository. It allows for viewing the repository in three separate layers: physical, business model and mapping, and presentation. The first step of the development process involves creating the physical layer KPI Partners, Inc.

21 CHAPTER 3 THE ORACLE BI DEVELOPMENT CYCLE 21

22 KPI Partners, Inc. Oracle BI Development Cycle The development process begins with the creation of initial business requirements. 1. You should have as many sessions with the business as possible to gather and confirm all the requirements. 2. Try to look at the sample reports, spreadsheets, etc. 3. Analyze the existing transactional system and the reporting system if any exist. 4. Analyze the existing schema for the reporting system and the transaction system. 5. Based on the requirements and the transaction schema, try to define the dimension model. There might be multiple iterations to the above steps.

23 CHAPTER 4 BUILDING THE ORACLE BI MODEL 23

24 Build Oracle BI Model We can now look at how to build the Oracle BI model in the repository. Before we begin, the dimension model will need to be designed to meet the business requirements. In this chapter, I will explain the entire process of building the Oracle BI model. There are multiple parts to this process: Import the objects if they don t already exist in the physical layer. Build the physical layer. Build the logical-business model and mapping layer. Build the presentation layer. Build the reports and dashboards based on the presentation-layer objects KPI Partners, Inc.

25 Import Objects The first step involves creating a new RPD using the BI administration tool and saving it. Next, we must import the objects into this repository to start building the model as shown in Fig. 2. You will need to define the connection type and other credentials for connecting to the database. To import the tables, select the tables or just click on the schema name to bring them in the schema. Figure 2: Sample Schema for Import KPI Partners, Inc.

26 Next, the connection pool needs to be defined in repository. Details on how to connect to the database are stored in the OBIEE repository as shown in Fig. 3. Once complete, the physical layer will have the imported objects. It populates the connection pool with default values on import. Figure 3: Connection Pool Details KPI Partners, Inc.

27 Build Physical Model The next step is to build the physical model with the help of imported tables. It is here that we will define the objects and their relationships. To build the model, we need to create joins between the physical tables. At the physical layer we need to create foreign key joins - a sample is shown in Fig. 4. Figure 4: Physical Model We should know the join criteria between the various tables. We need to maintain a one-to-many relationship between the fact and the dimensions, which can be done by selecting the dimension first and then joining it to the fact KPI Partners, Inc.

28 While creating the join, if the fact and the dimensions have the same keys, by default, they will appear in the Expression Builder. The expression shows the join criteria; Fig. 5 shows a sample. Figure 5: Foreign Key Join KPI Partners, Inc.

29 Figure 4: Physical Model There is also a feature to have database hints that tell the database query optimizer to use the most efficient way to execute the statement. We need to be very careful with this feature and use it only after proper evaluation as it may have adverse impact in certain scenarios. This creates the join between the two selected tables. Similarly, we need to create joins between all the other dimensions and the fact. In the end, the physical model should look like Fig. 4. Next, we need to run a consistency check on the physical layer to ensure there are no errors related to syntax or best practices. If there are no consistency errors, we will see the consistency check manager screen with no error messages KPI Partners, Inc.

30 Physical Layer Best Practices Here are some best practices that I have observed are important to follow to help your project be successful: You should have a separate set of connection pool for the initialization blocks and for the regular queries generated for the reports. This ensures a better utilization of the connection pools and ultimately results in performance improvements. Ensure that Connection Pooling, Multithreaded Connections, Shared Logon and the appropriate call interface is selected. You should not have any connection pools that cannot connect to the databases; this might lead to a BI server crash due to continuous pooling of the connection. It is recommended to have a dedicated database connection - and preferably a system account - for OBI with access to all the required schemas. Always ensure that a proper call interface is being used in the connection pool definition. In the case of an Oracle database, it s better to use an OCI connection instead of an ODBC connection KPI Partners, Inc.

31 Use the aliases of the tables instead of the main tables; this will avoid circular joins and caching-related issues. Follow a proper, consistent naming convention to identify the aliases, tables and columns. These may include W_XXX_F (for the fact tables), W_XXX_D (for the dimension tables), Dim_W_LOCATION_D_PLANT (for dimension alias tables) and Fact_W_REVN_F_ROLLUP (for fact alias tables). Always have cache persistence time set to ensure the data gets refreshed as required in case caching is enabled. Build the Business Model and Mapping (BMM) Layer The next step is to build the Business Model and Mapping (BMM) Layer. If you are in the Development mode, then this is the second step after creating the physical model. If you are in the Designing mode, then it is the first step normally done before designing the physical model. Planning the physical model is usually the first step in developing a usable data model for decision support. A successful model allows the query process to become a natural process by allowing analysts to structure queries in the same intuitive fashion as they would ask business questions. This model must be one that business analysts will inherently understand and one that will answer meaningful questions correctly KPI Partners, Inc.

32 To begin, we need to give a name to the business model. Right-clicking in the BMM section of the window opens the following window, which allows the assignment of a name to the business model. Figure 6: Logical Model The next step is to create the container for the business model. The easiest way to build the BMM layer is to either import in the tables from the physical layer or bring in the tables one by one as per requirements and then create the joins between them. In a complex environment, we normally do it one by one, as there might be multiple logical table sources, calculations, and other customizations involved KPI Partners, Inc.

33 Now we must look at the business model. To do that, we right-click on the HR model and select business model diagram. That will display the BMM diagram as shown in Fig. 6. The model is similar to the physical model. Figure 6: Logical Model KPI Partners, Inc.

34 The major difference that exists is in terms of the join criteria. We do not specify any joining columns in the logical layer; we only specify the cardinality and the type of join in this layer, as shown in Fig. 7. Figure 7: Logical / Complex Join KPI Partners, Inc.

35 The administration tool considers a table to be a logical fact table if it is at the many end of all logical joins that connect it to other logical tables or if it s not joined to any of the tables and the facts are displayed in yellow. As previously shown in Fig. 7, there are no expressions, so the tool picks up the base joins from the physical layer itself. Here, in the logical layer, we can configure the type of the join (inner, left outer, right outer, full outer) or the driving (fact or the dimension) and the cardinality. Cardinality defines how rows in one table are related to rows in the table to which it is joined. A one-to-many cardinality means that for every row in the first logical dimension table, there are possibly 0, 1, or many rows in the second logical table. Setting up the driving cardinality is an optional step; generally, it is set to none and left to the OBI server to process it. You should note that this option should be used with extreme caution; an improper configuration can lead to severe performance degradation. Driving tables are used in optimizing the manner in which the Oracle BI server processes cross-database joins when one table is very small and the other table is very large KPI Partners, Inc.

36 Specifying driving tables leads to query optimization only when the number of rows being selected from the driving table is much smaller than the number of rows in the table to which it is being joined. Driving tables are not used for full outer joins. Also important to note here are the two entries in the features tab of database object that control and tune driving table performance: MAX_PARAMETERS_PER_ DRIVE_JOIN and MAX_QUERIES_PER_ DRIVE_ JOIN. The BMM layer allows you to create measures with custom calculations. You can build dimensional hierarchy by right-clicking on the dimension and selecting Create Dimension. Dimensional hierarchy is created for entities having two or more logical levels, a very common example being year, quarter, month and day. Once the customizations are finished, we need to do a consistency check before the business model can be made available for queries. The BMM object will have a red symbol until it passes the consistency check. If the connection is not working or objects have been deleted in the database, this utility will not report these errors. We can use the consistency check to test for errors, warnings, and best practices violations. In certain implementations, the number of consistency failures increases after an upgrade due to a lot of reasons KPI Partners, Inc.

37 BMM Layer Best Practices To get the most from your OBI solution, each layer must be optimized. The following tips are some of my best practices that will help in the BM: Always use complex joins for joining the logical tables. Never use foreign key joins at the logical level as it might restrict the OBIEE server from using the most optimized path. Use inner joins wherever possible Minimize the usage of outer joins, as they normally impact the performance. An easier solution for the problem of using outer joins is to build multiple logical table sources and, depending on the requirement, the appropriate logical table source is accessed. There should be a hierarchy defined for every logical dimension even if it only consists of a grand total and a detail level. If there is possibility of a hierarchy, then it s always good to have a dimension hierarchy defined, as it helps to improve user experience KPI Partners, Inc.

38 Ensure each level of the hierarchy has an appropriate number of elements and the level key. The lowest level of the hierarchy should be same as the lowest grain of the dimension table. The lowest level of a dimension hierarchy must match the primary key of its corresponding dimension logical tables. Always arrange dimensional sources in order of aggregation from lowest level to highest level KPI Partners, Inc.

39 Figure 8: Custom Calculation Give business-meaningful names in the BMM layer itself instead of making the changes in the presentation layer. Use aggregates if required and enable the aggregate rule for all measures. Aggregation should always be performed from a fact logical table and not from a dimension logical table. Columns that cannot be aggregated should be expressed in a dimension logical table and not in a fact logical table KPI Partners, Inc.

40 Nonaggregated columns can exist in a logical fact table if they are mapped to a source that is at the lowest level of aggregation for all other dimensions. The content/levels should be configured properly for all the sources to ensure that OBI generates optimized SQL queries. Create separate logical table sources for the dimension extensions KPI Partners, Inc.

41 Build the Presentation Layer Once you are done with the physical and the BMM models, it is time to create the presentation layer. To begin, drag and drop the model from the BMM layer to the presentation layer. This approach can only be used when we have fairly simple models or are building a new model. Figure 9: Consistency Check Next, we will need to run another consistency check to ensure that the presentation layer and the repository are correct in terms of syntax and best practices KPI Partners, Inc.

42 Figure 10: Repository Before completing the development cycle, we will need to take a few steps to clean the repository. We can remove all the columns not required for analysis, but we must keep in mind to not remove the keys from the logical dimensions, as the business model will not be valid. Ensure that there are no extra objects in the repository; it helps with the maintenance and also keeps the repository light. Once done, the presentation layer will look as it does in Fig KPI Partners, Inc.

43 Presentation Layer Best Practices The final layer of the OBI solution is the presentation layer. The best practices that follow have improved the implementation of reporting: Ensure proper order of the objects so that it allows easy access to the required entities. Have business friendly/relevant names. Give a small description to serve as a tool tip for the users. Avoid designing the dashboard with large data sets. The requests should be quick and simple. Avoid using too many columns and use appropriate color combinations. Never combine tables and columns from mutually incompatible logical fact and dimension tables. Avoid naming catalog folders the same as presentation tables. Detailed presentation catalogs should have measures from a single fact table only as a general rule. The detailed dimensions (e.g., degenerate facts) are nonconforming with other fact tables. Never use special characters for naming convention in the presentation layer and dashboards KPI Partners, Inc.

44 This completes the configuration of the repository. To use it, we will need to ensure that the BI server recognizes that this is the correct repository. That will require configuring the NQSConfig.ini and configuring the instanceconfig.xml to create a new presentation catalog and open your reporting solution to the end users for a robust and reliable experience. Figure 11: Usage of NQSConfig.ini KPI Partners, Inc.

45

46