Oracle BI EE: Overview and Architecture

Transcription

1 Oracle BI EE: Overview and Architecture Chapter 1 Chapter 1 - Page 1

2 Chapter 1 - Page 2

3 Oracle Business Intelligence Suite EE Overview and Architecture Section Objectives At the end of this section, you should be able to: Discuss BI EE Architecture Chapter 1 - Page 3

4 Oracle Business Intelligence Suite EE Overview and Architecture Chapter 1 - Page 4

5 BI Challenges Today Chapter 1 - Page 5

6 The Vision - Enabling the Insight-Driven Enterprise Chapter 1 - Page 6

7 Oracle BI Suite Enterprise Edition Plus Unified Business Intelligence Infrastructure Chapter 1 - Page 7

8 Hot Pluggable Open, Integrate with Existing Investments Chapter 1 - Page 8

9 Oracle BI Server Next-generation BI and Analytics Server The heart of the BI enterprise edition is the Oracle BI Server. The BI Server is able to semantically integrate information from a variety of different sources. It is a scalable, sophisticated data access, aggregation, and calculation engine for real-time intelligence across heterogeneous data sources. Through a combination of a distributed query engine and a logical, semantic representation of the data, the BI Server enables you to represent multiple physical data sources as a single, simplified data structure to end user tools. This simplified view can then be queried by all end user tools through a simple SQL-based interface. So to end users the BI server looks like just another SQL data source. The BI Server includes its own advanced calculation engine, so regardless of the calculation capabilities of the underlying data sources users can get the intelligence they need, without needing to worry or know anything about the physical data or the capabilities of those systems. Calculations can be defined that cut across data from multiple sources. For example, you may have a calculation % of quota that compares closed revenue from a financial system with quota assignments in an Access database, and runs a calculation. This calculation would be defined in the BI server metadata and made available as a column that end users can select and embed into their analysis. The BI server is capable of complex calculations, like time series calculations, and these calcs can then be made available to any user to select for analysis. The BI Server was built for delivering intelligence to tens of thousands of Web users, so it is designed with the highest scalability in mind, including native SQL for a multitude of different sources including Oracle, MS SQL Server, IBM DB2, Teradata and others. Chapter 1 - Page 9

10 Common Enterprise Information Model Model once, deploy everywhere This also enables point and click access to information in an ondemand way. Here in Answers you see service metrics, revenue metrics, and inventory metrics all being combined for the user in one place. The user just points and clicks, creates charts, reports, dashboards, etc without needing to worry about where that information is coming from or how different metrics are being calculated. Here you can see a Dashboard, a formatted report, and a Delivers alert with some inventory information all driven off the same business model and same measures. The same consistency would be there whether accessing through any of the Oracle BI EE end user options, or through Web Services, or through Excel. Chapter 1 - Page 10

11 Intelligent Federation and Navigation Chapter 1 - Page 11

18 Guided Analytics Leading Users from Discovery to Action Oracle BI EE is designed with pervasive deployment in mind. With guided analytics users can be led along a path to discovery and action, based on their role. This is enabled by providing intelligent links and analytic alerts and workflows that lead a user to the next logical step or action in an analysis. For example in this case say a VP of Sales is looking at the overall opportunities pipeline. The guided analytics alerts him that the projected sales will be less than last year with a link to find out the reasons why and the actions that can be taken to address this. This enables the VP of Sales to not just look at the data but step in to see what can be done to change this. Organizations can encapsulate the knowledge and expertise of their best employees in the usage of information and use guided analytics to bring all others to this high level of skill and ability. In this example it is like learning from the expertise of the best salesperson and enabling everyone in the organization. Chapter 1 - Page 18

19 Sense and Respond Real-Time Proactive Monitoring and Response Another aspect of delivering more pervasive BI is moving to a more condition-based approach, where the intelligence finds you only when it s necessary or relevant, or drives multi-step intelligent workflow. It s not just about bursting reports anymore, it s about providing intelligence on top of a business process. For example say one of my customers, Best Value, places a large order. We want the system to automatically identify large orders like this and sends a notification to the account manager in charge of Best Value so it s first interacting with the CRM system. Then we want the BI system to automatically query the Inventory application to make sure there is sufficient inventory to satisfy this large order. If there isn t enough inventory, the system needs to automatically respond with an alert to the mobile phone of a distribution center manager to stock up for the large order that came in. And since this is a large order, I also want to trigger additional analysis related to this customer s lifetime value, and drive additional steps and process around that specific customer based on projected value. To do this, I need real-time access to information from multiple sources in this case both a CRM and inventory systems and need to be able to set up proactive monitoring, workflow, and information delivery to enable the organization to sense and respond to changes in real-time. This enables organizations to optimize business workflows and add value to each interaction. Chapter 1 - Page 19

20 Oracle Disconnected Analytics Full-Featured Disconnected Client For the mobile professional, Oracle BI provides full analytical functionality including fully interactive dashboards and ad hoc analysis while disconnected from the corporate network. It provides the same intuitive interface for users whether they are working in a connected or disconnected mode. Leveraging Oracle's advanced data synchronization capabilities, Disconnected Analytics allows for full and incremental synchronization of data with enterprise data sources. Data is personalized for each user, maintaining all role-based security and visibility, and is compressed during synchronization, resulting in minimal data set size and fast sync times. Chapter 1 - Page 20

21 Oracle Briefing Books Pack & Go The content of Oracle BI Dashboard pages can be saved into a "Briefing Book" viewable by anyone with a Briefing Book reader. The Briefing Book provides a way to create snapshots of dashboard pages, view them offline, or share them with others. Multi-page Briefing Books have paging controls and are well-suited for presenting information to others. Briefing Books also provide a way to archive the information in a dashboard. They can be saved locally (on a desktop, for example) and can be updated with a single click whenever the user chooses. Chapter 1 - Page 21

22 Oracle Reporting and Publishing Lastly, Oracle BI Reporting and Publishing products provide developers with precision control over report format, layout, and output enabling the creation and distribution of "pixel-perfect" reports, regardless of graphical complexity. The Reporting and Publishing products are fully integrated with the Oracle Business Intelligence Suite EE platform products and share common dashboarding, metadata, security, calculation, caching, and intelligent request generation services. Chapter 1 - Page 22

23 Oracle BI Publisher One Solution for all Business Documents Many customers today have very strong business requirements in the reporting area around the e-business Suite. These include: High Fidelity Reports These reports are rich in images, charts, multiple fonts and color are a very desirable feature for reports when they are to be distributed in and outside the company. Partner Reports These include Invoices, Purchase Orders and Checks, which require rich content and are currently highly customized. Financial Statements Financial statements need to be presented to shareholders and upper management in a high fidelity format; your accounts would rather see them in Microsoft Excel. Government Forms Many government agencies now demand that you interact with them in a specific format e.g. tax forms. Marketing Materials The marketing materials generated by companies require rich personalized content. Chapter 1 - Page 23

24 Contracts Interacting with your customers and suppliers often requires written contracts that need to be presented in a high fidelity format. Checks The checks you send to employees and suppliers require a specific format and even ink so that their banks are able to process them. Labels Any company generating product will need to label it often including a barcode. XML B2B interaction requires XML as the format and this may require a transformation of one xml format to another. EFT/EDI Electronic File Transfer (EFT) and (EDI) are formats used to communicate with your banks and partners. Multiple Destinations Of course all of these documents then need to be delivered to a report consumer be that via , fax or printer. Chapter 1 - Page 24

25 BI Publisher Oracle Business Intelligence Answers Start Page Select a subject area from the list to begin creating a request. The selection pane contains requests and other objects saved in the BI Catalog. The BI Catalog contains personal and shared requests, personal and shared filters, and briefing books (if your organization licensed this feature). Briefing books are static snapshots of dashboards or individual Answers requests that can be stored and viewed offline. Chapter 1 - Page 25

26 Oracle BI Suite Enterprise Edition Common BI Infrastructure All of these products share a common architecture, common shared services, and a seamlessly integrated and consistent user experience. Chapter 1 - Page 26

27 Service Oriented Business Intelligence The fusion of Business intelligence and SOA Chapter 1 - Page 27

28 Action From Insight Example: Invoking a business process from BI Dashboard Chapter 1 - Page 28

29 Architecture Now that you have an overview of the BI EE products and end user functionality, I d like to dive a level deeper to explain more about how the BI server is architected and describe in some detail the inner workings and key capabilities of the BI Server to enable more pervasive BI. Chapter 1 - Page 29

30 Oracle BI EE Architecture One of the most important and valuable services the BI EE Server provides is a single, enterprise information model that we previously mentioned. This model provides several benefits: It provides a simplified business view of information available to all BI tools and all users in the organization regardless of physical data locations or structures. In this way, you can almost think of the business model as a sort of firewall between the users of the information and all the mechanics of the physical data access layer. Data is separate from the model. So let s say that you have data on inventory in a certain operational data store that is mapped in, and then someone decides that system is going to be retired, so you have to map to a different source to get that inventory data. You can simply remap to the new schema, and it won t impact all the end user reports because there is a separation or abstraction layer between the physical data layer and the logical business model layer. This allows a high degree of flexibility for IT to make changes to data structure without breaking thousands of end user reports, dashboards, etc. It provides a place where business metrics and calculations are defined, enabling more consistency and alignment in the organization. For example, let s say the organization has a particular way in which they define customer value, or product profitability. These calculations and metrics can be defined in the metadata and made available to users define once, deploy everywhere. Chapter 1 - Page 30

31 The enterprise information model presents itself as a single logically modeled database of information. Complex calculations and time series analysis like % Change vs. Year Ago Revenue does not need to be calculated by end users, but simply selected for analysis just like any other column (dimension or measure) in the model. Chapter 1 - Page 31

32 Oracle BI EE Architecture Chapter 1 - Page 32

33 Enterprise Business Model Administration The basic function of the Oracle BI Server administration tool is to create the mapping of physical data schemas (tables & joins) to a logical business model. The logical business model stores a simplified mapping to one or more physical data sources. It also contains a shared library of default drill paths, complex derived calculations, aggregate navigation, etc. Most administrative tasks are edited/modified through the Siebel Analytics Administration Tool. The Oracle BI Server Admin Tool is broken out into three separate layers - Physical, Business Model & Presentation. The Physical layer is an exact depiction of the target physical database(s). To create a new Physical layer the administrator simply selects the File Menu Import feature, allowing the administrator to read in a system catalog from a relational, XML or Host-based systems or all three. The Oracle BI EE platform is 100% data agnostic and has no preconceived notion of how the data is modeled. The Key Point here is that the Analytics platform should not dictate the data design. Some BI tools require the data to be modeled in a particular way, such as a star or snowflake schema. The Oracle BI EE tool maps in the data the exact same way in which the customer designed that data to support their business processes. No data is moved. Oracle BI EE simply references the way the data looks from the database system catalog. Function Shipping: The Oracle BI EE platform was designed with the notion that not all databases were created equally. For example, some databases are efficient at certain processes (like aggregations) while terrible at calculations (like time-series analysis). The Oracle BI EE database adapters are intelligent enough to dynamically determine whether or not a specific database function is Chapter 1 - Page 33

34 supported. If the database supports a function the BI Server will push the processing of that operation down to the database where it will usually perform more efficiently. If the database does not support the specific function the BI Server will compensate for the database s lack of functionality and pull back appropriate data to the BI Server for post-processing before returning a result-set to a user. Connection Pooling: Oracle BI EE also has sophisticated Connection Pooling that manages database connectivity, call interfaces, asynchronous query execution any pertinent database specific parameters. Chapter 1 - Page 34

35 Enterprise Business Model Administration The Business Model is a layer of abstraction that sits on top of the physical data. The goal of the business model is to simplify, potentially very complex database schemas (Normalized Structures, 3 rd -Normal Form, Header/Detail, etc.). The Business Model allows administrators to collapse common database tables into a single logical table. For example, if the target database is a snowflake schema (i.e. customer attributes stored within many look-up tables), the administrator may want to simplify the schema by moving all common attributes into a single unified logical table called Customer. The same holds true for any other attribute (Products, Geography, Account), as well as additive items (Measures). The Business Model is also where the administrator creates the default Drill-Paths (Year-Qtr- Month-Week-Day or NA-Region-District-City-Zip Code), as well as the Derived Business Measures or Calculations. Oracle BI EE supports a robust set of functions not necessarily supported by today s database vendors (SQL). Included are: support for time-series analysis, moving averages, weighted averages, rolling sums, cumulative calculations, % of column/row or level, etc. The Business Model also contains some of the rules for how the databases should be interrogated. If the application contains Aggregate or Partitioned database tables used to improve query response times, these rules would be defined within the Logical Table Sources within the Business Model. Chapter 1 - Page 35

36 Enterprise Business Model Administration The Presentation layer is a second layer of abstraction that sit-on-top of Business Model. Administrators simply drag-and-drop logical applications or tables from the Business Model into the Presentation layer. The Presentation layer is presented to end-users in a complete WYSIWYG framework. Administrators can Sort, Reorder, and combine attributes and calculations in virtually anyway they like. The goal is to make the navigation of the Subject Area as easy as possible for the end-users to enable self-service analyses. Another benefit of having a Presentation layer that is an abstraction of the Business Model layer is the administrator can create as many views into the Business Model as they like. You can create an simplified Presentation for Executives or a very robust Presentation for Power-Users (many complex calculations, etc.). Each view is associated with the users role and responsibility. NOTE: The two layers of abstraction on-top of the physical data allow administrators to focus on building robust applications without the fear of re-work when the underlying physical data changes. Data Warehousing is an organic process, not like a race with a beginning and an end. Applications and database systems change from time-to-time, much like how Oracle Applications databases are enhanced release-over-release. As systems are tuned (i.e. Aggregates or Fragments added) these changes need to be accounted for. Legacy BI (Business Objects, Cognos, etc.) tools often force administrators to hard-code SQL into each report. Therefore, when the underlying physical data changes all saved metadata & reports need to be updated or they will not work properly. In the Oracle BI EE paradigm, administrators Chapter 1 - Page 36

37 only have to re-map the deltas in the Business Model to reflect the changes in the physical structures. The Presentation layer is completely masked from changes in the physical data. Therefore there is no need to re-build reports each and every time the physical data changes. Put this in context when you have a couple thousand reports within an application. The amount of re-work is mind-boggling. Chapter 1 - Page 37

38 Oracle BI EE Architecture Next I d like to zoom in and discuss in detail the BI Server, which is the heart of the BI EE platform. The BI Server performs several important functions, but boiling it down to the simplest terms what the BI Server does it process inbound logical SQL requests for information, build and execute optimized queries for one or many physical data sources, and return the result set to the end users. To an end user tool, the BI Server presents itself like a single database, with subject areas of information available as single tables that can be queried through a SQLbased API. This simplified view of information could be coming from data that is quite complex, stored in different physical structures including relational, multi-dimensional, flat files, etc. Let s walk through the various functional components of the BI Server to get an understanding of how the platform works. Chapter 1 - Page 38

39 Oracle BI Server Common Enterprise Information Model One of the most important and valuable services the BI EE Server provides is a single, enterprise information model that we previously mentioned. This model provides several benefits: It provides a simplified business view of information available to all BI tools and all users in the organization regardless of physical data locations or structures. In this way, you can almost think of the business model as a sort of firewall between the users of the information and all the mechanics of the physical data access layer. Data is separate from the model. So let s say that you have data on inventory in a certain operational data store that is mapped in, and then someone decides that system is going to be retired, so you have to map to a different source to get that inventory data. You can simply remap to the new schema, and it won t impact all the end user reports because there is a separation or abstraction layer between the physical data layer and the logical business model layer. This allows a high degree of flexibility for IT to make changes to data structure without breaking thousands of end user reports, dashboards, etc. It provides a place where business metrics and calculations are defined, enabling more consistency and alignment in the organization. For example, let s say the organization has a particular way in which they define customer value, or product profitability. These calculations and metrics can be defined in the metadata and made available to users define once, deploy everywhere. Chapter 1 - Page 39

40 The enterprise information model presents itself as a single logically modeled database of information. Complex calculations and time series analysis like % Change vs. Year Ago Revenue does not need to be calculated by end users, but simply selected for analysis just like any other column (dimension or measure) in the model. Chapter 1 - Page 40

41 A Day in the Life of a Query Next, let s discuss the Intelligent Request Generation function and examine the various steps that go into processing an inbound logical SQL request. The first step in the process is the generation of the logical SQL query itself. Logical queries are defined in the Oracle Answers product, shown on the right here. Users select columns from a simple business model. These queries, once defined, are saved and can be embedded in other BI EE end user products such as in Interactive Dashboards. A logical SQL query could also come from another BI tool, SQL query tool, or from Excel. Chapter 1 - Page 41

42 A Day in the Life of a Query In this case we re looking at a query based on Brand, Closed Revenue from the General Ledger, Service Requests from the CRM system, and Revenue Share, which is a multi-pass query. The first thing that happens is the request comes into this Logical Request Generation engine, and it determines how to process the request Chapter 1 - Page 42

43 A Day in the Life of a Query The first thing the server does is check cache. The cache can satisfy requests that are aggregates of other requests, subsets of other requests, or derivations, and it is secure and personalized for every user. So it again acts as sort of a firewall to the database where you are not needing to hit that iron every time a user has a request. And the nice thing is that as more users get onto the system, the system gets faster. The cache is a runtime concept verses a premeditated concept, so this does not take away from the importance of good data modeling and good aggregates that are in place in the database. So as users are on the system, it gets faster. You can also pre-meditatively populate, or seed, the cache. So for example, you might say, OK, these are the long running queries or dashboards, so I want to use the Delivers scheduling capability to run queries periodically to seed the cache with the answers to those longer runnig queries. You might also want to seed the cache each morning so that when workers start their day the first users have the same performance as uses will have once the system has been at load for a period of time. Additional background / questions: When is cache purged? Answer: Cache can be purged on either a time basis, so it could be every 2 hours or every 2 minutes, or every 5 seconds or it could be on an event basis, so every time the data warehouse is updated cache is purged for example. In addition, this can be done by data element, so some data could be purged every 2 seconds and some could be purged once per day. Chapter 1 - Page 43

44 A Day in the Life of a Query The next thing is the navigator. Once the BI server figures out that the request can t be satisfied through cache, the BI server now starts to figure out what is the most optimal way to satisfy this request. Chapter 1 - Page 44

45 A Day in the Life of a Query The first thing it looks at is what is the complexity of this analysis? Is this a multi-pass analysis? Is there a sub-request involved? In this case we re talking about Revenue Share, so it decomposes that logical request to figure out what is difficult and what is easy from a logical server perspective. Chapter 1 - Page 45

46 A Day in the Life of a Query The second thing is fragmentation optimization. In this case, closed revenue is in one data silo, and service requests are in another data silo. Fragmentation optimization figures out the best way to get this data. Optional Additional Background: Fragmentation Optimization The BI Server Fragmentation Optimization Engine is also a piece of middleware that sits between the requesting client and the database. The Fragmentation Optimization Engine intercepts the client's SQL and, wherever possible, transforms base-level SQL into fragment-aware SQL. The Fragmentation Optimization Engine understands how to transform base-level SQL into fragment-aware SQL, because the engine uses special business rules within the BI metadata that describes how the database is partitioned across large common fact tables. The fragment-aware SQL can dynamically determine the most efficient table or tables to fetch the data from. Typically, large fact tables (billion rows) are fragmented into smaller more efficient tables partitioned by time (Q Frag, Q Frag, Q Frag, etc.). Chapter 1 - Page 46

47 A Day in the Life of a Query Next, the system determines if the request can be satisfied through an aggregate of some kind. So rather than going down to the one million customer SKU rows of detailed data, the system checks to see if there an aggregate, or an analytical workspace, or an XMLA cube that can satisfy this request? Let s navigate there as opposed to going down to the detail data. In this example you are looking for Brand summaries, so perhaps that could be satisfied by looking at regional aggregates and adding those up by Brand. So to summarize the Navigator, what the Navigator does is handle this whole decision tree. How complex is the request? What data sources need to be put into play? What is the best way to satisfy this request and are there aggregates that can be used? Then it generates a query plan based on what it believes is the best plan. Optional Additional Background: Aggregate Navigator The BI Server Aggregate Navigator is a piece of middleware that sits between the requesting client and the database. The Aggregate Navigator intercepts the client's SQL and, wherever possible, transforms base-level SQL into aggregate-aware SQL. The Aggregate Navigator understands how to transform base-level SQL into aggregate-aware SQL, because the navigator uses special business rules within the metadata that describes the databases aggregate portfolio. The aggregate-aware SQL can dynamically determine the most Chapter 1 - Page 47

48 efficient table to fetch the data from (i.e. the table with the least number of records to successfully fulfill a request). Chapter 1 - Page 48

49 A Day in the Life of a Query The server then takes this query plan and does re-writes to optimize for the underlying databases. It figures out what can be function shipped down to the underlying databases? Does the database support a RANK or does it not support a RANK? Does it use sophisticated analytical functions or not? Is it an XML source, a text file, or is it in Oracle? And then it will parallelize that and ship that down to the underlying sources as appropriate, using native connectivity. Chapter 1 - Page 49

50 Oracle BI Server Exploiting Native Database Functionality This is a picture of the administration tool adapter or data source dialogue. To the left you can see a list of various databases that are available, and to your right are the database features that are being driven off of that selection. So I know that in this case this is a Teradata database, and these are the features that Teradata supports. The ones that are not checked are the ones that the BI Server is going to have to function supplement for that data source. Again, the goal is that you have that firewall common enterprise model that provides a consistent level of features regardless of the type of database and the functions supported by that database underneath, taking that problem away from the user. Again, the goal is to take problems away from the business user. Chapter 1 - Page 50

51 Oracle BI Server Optimized Native SQL Rewrites Here s an example of database specific SQL generated for the same request. You can see that here we re shipping down a rank function to Oracle, but since SQL Server doesn t support Ranks we re not pushing that down. For SQL Server, that function will be compensated and performed by the BI Server at the middle tier. Optional Example Go back one slide so here s a question, what do you think it would take to migrate an application from a Teradata warehouse to an Oracle database using this capability? Answer: Easy, all you would do is just change the dropdown for the data source, and the change would cascade throughout all the queries in the application. That s huge. That s another advantage or requirement for being an analytic applications development platform you can separate all of the end user logic and reports from the data access issues under the covers. Chapter 1 - Page 51

52 Yet Best for Oracle DBMS Uniquely Exploits Oracle DBMS Analytics Chapter 1 - Page 52

53 A Day in the Life of a Query Last but not least, the server takes that result and puts it back in the cache for the next user. So that a day in the life of a query. Chapter 1 - Page 53

54 Oracle BI Server System Services In designing the BI platform, there s a recognition that it needs to be a first class citizen in terms of the corporate network. There is native load balancing within the BI server. You ve got session management, as well as query governance so different types of users can be restricted in terms of the queries they can send. It s got to be able to fail over gracefully, and distribute load as appropriate. Likewise, companies have made investments in other products and so the idea is to leverage those investments. You can leverage all of your existing security models, whether that be for personalization from an authorization perspective, or for authentication via LDAP or ADSI. Chapter 1 - Page 54

55 BI EE Answers Oracle Business Intelligence Answers Start Page Select a subject area from the list to begin creating a request. The selection pane contains requests and other objects saved in the BI Catalog. The BI Catalog contains personal and shared requests, personal and shared filters, and briefing books (if your organization licensed this feature). Briefing books are static snapshots of dashboards or individual Answers requests that can be stored and viewed offline. Chapter 1 - Page 55

56 BI EE Interactive Dashboards Interactive Dashboards Oracle Interactive Dashboards provide personalized views of corporate and external information. Based on your permissions, you can view preconfigured dashboards or create your own. Users with administrative privileges can create shared dashboards for groups of users with common responsibilities or job functions. Personalized views can be created based on a user s permission. You can view your personalized views by clicking My Dashboard. This is also the default dashboard. Preconfigured views are linked across the top of the screen and can be created and shared with groups of users with common responsibilities or job functions by users with administrative privileges. Chapter 1 - Page 56

57 BI EE Web Administration BI Web Administration Page To open the BI Web Administration page, click the Settings link and select Administration in the drop-down list. Chapter 1 - Page 57

58 For More Information Chapter 1 - Page 58