Using and Extending Fusion Analytics. Written by Tom Connolly President, BizTech

Transcription

1 Using and Extending Fusion Analytics Written by Tom Connolly President, BizTech

2 Note: The following information is intended to outline a general solution offering. It is intended for information purposes only, and may not be incorporated into a contract. It is not a commitment to deliver any material or functionality and should not be relied upon when making purchasing decisions.

3 Using and Extending Fusion Analytics Contents Introduction...2 Functional Overview...3 Dashboards...8 Technical Architecture...10 Follow the Data Trail...10 Extending Pre-Built Applications...14 Conclusion...17 About BizTech...18 Witepaper Using and Extending Fusion Analytics 1

4 Introduction Sanity check. Why would an organization purchase Oracle s pre-built Fusion Analytics and then spend many months and many thousands of dollars to implement the software? What is the motivation? Is it to allow the organization to collect large amounts of information? Or perhaps it is the sexy dashboards that allow analysts to show off cool new charts and colorful reports? No. The reason why companies invest in the Fusion Analytics is, simply, to improve enterprise performance. And how do the Fusions Analytics enable improved performance? By allowing an organization to gain a better understanding of complex business activities. If the activities weren t complex, then we wouldn t need an industrial application to help us understand them. If the understanding we gain did not allow for improved performance, then we wouldn t waste the time and money on such a system. Oracle's Fusion Analytics (aka Seibel Analytics) can provide a powerful foundation for enterprise-wide reporting and analysis. This white paper starts with a functional overview of the Fusion Analytic Applications and then delves into the technical architecture with an end-to-end example of how data moves from source system into the warehouse and onto a reporting dashboard. The final section provides an example of how to extend the pre-built analytics to include custom data sources and reports. Warning This paper is specifically focused on pre-built Business Intelligence applications. This is not a paper on building custom dashboards! Furthermore, while the Fusion Analytics are source-systemagnostic, this paper is not, and does in fact include a bias towards the Oracle EBS applications. This is not to exclude other source systems (even if they are inferior), but simply reflects the fact that the author happens to know Oracle EBS. Witepaper Using and Extending Fusion Analytics 2

5 Functional Overview Oracle s Fusion Analytics is essentially an integrated set of pre-built Business Intelligence, or BI, applications. But, what is Business Intelligence? The simple perspective is that BI is a reporting and analysis application that enables senior managers and executives to see operational information. A more compelling definition would hold that BI is an integrated, out-of-the-box, reporting and analysis application that enables all properly authenticated business process members to see accurate and timely operational and performance management information using a self-service framework. Integrated tied directly to the transactional systems that run your entire enterprise Out-of-the-box leveraging pre-built components, allows for rapid, cost-effective deployment Properly Authenticated Business Process Members much wider access to key enterprise information to allow business users at all levels to make informed decisions Accurate / Timely because it is directly accessing operational data Performance Management Pre-defined Key Performance Indicators (KPI s) across a wide range of enterprise areas Self-Service direct management/executive access (no need for IT involvement in report distribution) Sounds great, right? Well it is great, but, it s not perfect. To start with, there is some confusion (just a tad) around Oracle s many Business Intelligence products, a natural outcome of the growth and acquisition strategy employed over the past five years. With an overlapping product set and an increasingly aggressive sales environment, customers are often confused, and at times mis-lead, in regards to the simple task of identifying the product(s) that would be most appropriate for them. A quick visit to Oracle s Business Intelligence home page, and you will see over thirty products listed across five product areas, and this is only listing the products that Oracle is actively pushing when in fact there are probably at least another two dozen products which are available and may in fact be suitable for many customers with legacy software that is still best served with some of the older BI products. But I digress. Point being that today s software landscape is getting increasingly complex and the Oracle Business Intelligence Applications Fusion Edition, OBIA for short, is a powerful application but is not necessarily the best solution for every organization. This paper will point out many of the strengths and a few of the weaknesses of OBIA and, in so doing, hopefully shed some light on where/how the product is best utilized. With this in mind, the functional definition presented here will highlight several of the most common product misconceptions within the Oracle BI landscape. Witepaper Using and Extending Fusion Analytics 3

6 Fundamentally, the Oracle Business Intelligence Applications are a suite of software modules which provide the BI foundation as defined above (i.e. integrated, out-of-the-box, secure, accurate, timely, performance-based and delivered in a self-service framework). But, the OBIA product is not the only BI application suite that is sold. A bit of history will help explain why this is so. In the beginning, Oracle created the relational database. Shortly thereafter, development tools were introduced to allow customers to create custom applications to use the all-powerful database. However, when Oracle realized that customers were not very good at developing applications (and they saw how much money SAP was making) they set out to develop their own suite of ERP applications. This was pretty much the focus of the 1990 s and, in all fairness to Oracle, SAP and the other enterprise software companies, these ERP applications (and a little thing that Al Gore came up called the internet) were critical to the growth of large, multi-national, complex companies that were driving the world economy in ways that had never been imagined before. Then, the bubble burst and the euphoria wore off. It was true that the large ERP systems provided the means for tremendous growth in productivity and support for complex business models. It was also true that the ERP applications had matured year after year such that even the small businesses could now take advantage of the same competitive technology that was used by the big boys. However, with the dot-com craze subsiding, companies realized that they could not continue to introduce newer (and wackier) business models every quarter. There was a collective realization that companies had to actually make a profit in order to increase shareholder value. The days of reckless investment in new software technology were over. In fact, as the financial hangover of the Y2K and dot-com bubble began to clear, CEO s and CFO s (and some shrewd investors) began to take a critical eye toward their multi-million dollar ERP investments. They began to question how they could best utilize these investments and, with the help of their technology brethren, some of them began to realize that there was interesting side-effect of all of these large transaction processing systems. There was a large and rapidly growing mass of data (which is of course why EMC s stock continued to rise). Software vendors noticed this as well and set out to build products to mine this data, turning it into actionable information and, with that, the Business Intelligence industry was born. The first several years witnessed a number of innovative players in the industry developing new technologies and applications. Oracle, while by no means leading the BI pack, did have their own BI products that were developed in the early days of BI. First came the Discoverer tool, which allowed multi-dimensional analysis the core of all good BI systems. Soon after, Oracle developed their own set of BI applications to complement their ERP package and called this, Daily Business Intelligence (DBI). Along the way, Oracle picked up a product called Oracle Express and built a couple of applications using this product called Oracle Financial Analyzer and Oracle Sales Analyzer. And, the Oracle faithful purchased, implemented and cursed these products for many a year. Then one day, Oracle s CEO came down from on high and decided that it was time for the software industry to consolidate. And yes, he was right. With a wall street analyst at his side to do his bidding, Larry Ellison started to gobble up companies like cheese steak sandwiches at an Eagles game. Many niche companies, too numerous to mention, but the most significant purchases were PeopleSoft, Siebel and Hyperion. With PeopleSoft, Oracle picked up not one but three additional ERP application products (PeopleSoft applications and two ERP platforms from JD Edwards). Interestingly, PeopleSoft had it s own BI applications called EPM which, surprise-surprise, were not based on Oracle technology but used Witepaper Using and Extending Fusion Analytics 4

7 Cognos for the analytic reports. Next came Siebel, the leading CRM software company in the world and, with it, one of the leading BI tools and applications as well. Oracle may tell you otherwise, but the plain truth is that Oracle didn t realize the true value of the Siebel Analytics until many months after the purchase was completed. In any event, when Oracle realized the value of their purchase, the Siebel Analytics toolset and applications quickly became the flagship Oracle BI products and were branded Oracle Business Intelligence Enterprise Edition (OBI-EE) with the older Discoverer tool now named Standard Edition (OBI-SE). Of course, there were/are some Discoverer die-hards (aka lunatics) who resisted the change, but the vast majority of Oracle users quickly breathed a sigh of relief as they came to appreciate the improved user interface and found out that this tool set could actually access data that was not in an Oracle database! One more interesting factoid, shortly after the Siebel acquisition the Oracle BI gurus came up with the brilliant idea that they could place an OBI-EE cover on top of the DBI foundation for EBS customers or, as Sarah Palin would suggest, putting lipstick on the pig. This nifty new product with the sexy facelift was branded Fusion Intelligence (in much the same way that George Foreman named all three of his sons George Junior, Oracle started branding many of their products with the Fusion moniker). Of course, this is the part of the presentation where some readers gasp in dismay as they realize that they have purchased and implemented the Fusion Intelligence product and not the true Fusion Analytics! Not to worry, Fusion Intelligence (or FI) is not really that bad and does more than just cover up a bungled warehouse design with a pretty front-end as you now have the OBI-EE foundation and can extend this in all the right ways. The last (or latest) piece of the puzzle fell into place, sort of, when Oracle acquired Hyperion. Although initially a source of great consternation as the Oracle faithful who had finally started to get used to the OBI-EE name and tools were concerned that Oracle would do a quick change to adopt the Hyperion tools and applications as the golden child. However, within a few months, the message from on high was that the OBI-EE and Hyperion tools would co-exist under the brilliantly named brand of OBI-EE+. There are, of course, many other side notes in the Oracle BI story, such as the Essbase vs. Oracle RDBMS debate and the HFM vs. OBIA Financials conundrum, however, we will leave these for another day. Now that we have an appreciation for, or at least a general awareness of, the Oracle BI products, we can safely provide a proper description for the flagship Oracle Business Intelligence Applications Fusion Edition, or OBIA for short. OBIA is comprised of a number of integrated, or interdependent, modules which are most commonly described as a set of web-based reporting dashboards. In actuality, each module is comprised of a number of technical and functional components that go well beyond the dashboards, however, the dashboards serve as a useful foundation for describing the functional boundaries for each module. OBIA currently contains ten modules grouped into two product families as follows CRM Analytics (Sales, Service, Contact Center, Marketing, Pricing and Partner Analytics) and ERP Analytics (Supply Chain & Order Management, Financials, Procurement & Spend and Human Resources). Each of the ten modules typically includes about five to ten individual dashboards each comprised of five to ten pages with multiple reports on each page. Witepaper Using and Extending Fusion Analytics 5

8 In addition to these ten baseline, or horizontal, BI applications, the OBIA products include a number of industry-specific vertical applications across the following industries Telecom, Financial Services, Insurance, Life Sciences, Consumer Goods and Public Sector. Most of the industry applications are largely CRM-focused, which makes sense given the fact that these were developed by Siebel. Looking under the hood, OBIA includes four primary assets. First, with OBIA you get a number of predefined multi-dimensional data structures. In the OLAP world, data is stored in groups of tables called star schemas with a Fact table at the center and a number of Dimension tables surrounding the central Fact table. Since the ground-breaking work of Ralph Kimbal in the mid 80 s, this structure has long been recognized as ideal for analytic reporting (i.e. reporting on a measure by one or more dimensions). The typical analyst might be interested in sales (the measure) by region, year, product line, and customer classification (the dimensions). This star schema structure not only supports multi-dimensional analysis but also allows for quick re-alignment of the dimensions (adding/removing or re-ordering them) as well as on-the-fly derivation of new measures (percent increase or Year-over-year measures for example). The commonly held wisdom is that analyzing your General Ledger or Payables data is pretty much consistent across all companies so the underlying data structure should be consistent as well. Given this, why would any organization want to have their $60,000 / year programmer defining the data structure when they could simply adopt what has already been invented and thoroughly tested by hundreds of companies before them? The second asset included with OBIA is probably the most technically challenging and therefore risky area of a custom data warehouse or Business Intelligence system the ETL programs which move data from the transactional source system into the reporting warehouse. Without going into the technical details, which will come later, suffice it to say that a lot can wrong when your programmers start pulling data from hundreds of ERP tables that they know little about. Witepaper Using and Extending Fusion Analytics 6

9 Third, OBIA comes with what is called a metadata library which is where a BI system captures the analytic smarts needed to power the reports. Facts, dimensions and the relationships between them are defined here as are drillable hierarchies, aggregations, data-dependent access controls and complex business calculations. As with the data model, conventional wisdom suggests that these metadata definitions are pretty similar from one company to the next in such common areas as financials, order management and CRM. The fourth asset, as noted previously, represents the most visible and most commonly understood component within the OBIA products and is called the Presentation Catalog. This best practice library is where the dashboards organization and detailed reports are defined. And yes, once again, the thinking is that one company s Payables Trial balance is pretty much the same as the next. Witepaper Using and Extending Fusion Analytics 7

10 Putting all of this together, OBIA provides a complete data warehouse and Business Intelligence solution. At this point, the reader is probably starting to realize that OBIA is a pretty HUGE system. With thousands of tables, thousands of ETL programs, thousands of meta data objects and thousands of reports, OBIA is certainly a very large and complex system. A system of this size and complexity is not for all organizations, however, in much the same way that the large ERP systems have revolutionized the supply chain, manufacturing and financial management activities of thousands of companies around the world, OBIA (and similarly complex competitive products) are changing the way companies analyze and improve their operations in far reaching ways. It is now time to succumb to the inevitable and much anticipated first look at the sexy part of OBIA, the dashboards. While it is not possible to gain a full appreciation for the dashboard content in a written document, a few screen shots will help to stretch the imagination. Shown in the next section are the Overview pages from the General Ledger, Profitability and Payables dashboards. Dashboards Witepaper Using and Extending Fusion Analytics 8

11 In each dashboard, the logical organization is evident in the tab names for each page. The General Ledger dashboard, for example, includes pages for Expense Analysis, Balance Sheet, Cash Flow, Budget vs. Actual, Asset Usage, Liquidity, Financial Structure and Ledger Balances. Each page in turn includes several reports which present the pertinent analytic data in the very powerful multi-dimensional format. Moreover, most reports within the dashboard allow for multi-level drill-down to related detailed transactional data all the way down to the source transactions in many situations. Witepaper Using and Extending Fusion Analytics 9

12 Technical Architecture For those with a technical bent this next section, and the rest of the paper, will endeavor to explain the technical workings of the Fusion Analytics. As noted above, the pre-built analytics include four primary assets, or components the tables which comprise the Enterprise Data Warehouse (EDW), the Integration or ETL programs which move data from source to warehouse, the Meta Data Repository and the Presentation Catalog. Recognizing that a picture is worth a thousand words (and also that the submission deadline is looming large), the following system diagram with these four components numerically identified is provided to help illustrate this architecture. In addition to the four primary components previously discussed, this diagram shows the source systems on the left (i.e. your EBS or PeopleSoft applications) and two additional components which are included within the OBIA product. The Data Warehouse Administration Console (DAC) is a powerful extension to the Informatica ETL source programs which provide the ability to organize, execute, monitor and manage the ETL process. The OBI Admin is the software tool which is used to view and/or update the meta data repository on the OBI server. Follow the Data Trail At the risk of stating the obvious, the data flows from left to right in what is commonly described as the three stages in the Data Life Cycle. 1. Source system to Data Warehouse 2. Data Warehouse to Logical Reporting Mart (aka Subject Area) 3. Reporting Mart to Business User Witepaper Using and Extending Fusion Analytics 10

13 In the first stage of the Data Life Cycle, ETL programs move data from the source system, where it is stored in a highly normalized OLTP format, to the data warehouse, where it is in the preferred OLAP or star schema structure. These programs, also called mappings, were developed using the Informatica ETL toolset and, yes, the customer must purchase an Informatica license when purchasing the OBIA product (although, it is a discounted OEM license so the cost is pretty reasonable ). An Informatica designer screen is show below. Pre-built mappings are available for Oracle EBS, PeopleSoft, Siebel and even SAP. Mappings are organized into one of three different areas. Source Dependent (SDE) mappings are designed with Witepaper Using and Extending Fusion Analytics 11

14 specific source system physical table knowledge and are used to extract the raw transactional data and place this in a series of staging tables in the warehouse. Then the Source Independent Load (SIL) mappings pull from the staging tables, perform any necessary transformations and load the actual warehouse reporting tables. Lastly, there are a number of Post Load Processing (PLP) mappings which are designed to perform any ancillary processing such as data aggregations for summary reporting. As noted above, the Informatica mappings are controlled through the DAC, which organizes a set of related mappings into a Subject Area and then allows for execution of one or more subject areas. For any old-timers in the audience who have spent significant man-months developing detailed execution and control scripts, the DAC is a huge benefit and is very simple to use. A DAC screen is show below. The climactic conclusion of an ETL execution is a populated data warehouse. This is probably 75% of the battle for most organizations. Now comes the (relatively) easy stuff. With data in our warehouse, if this was a custom BI project, we would set out to create the meta data, the layer which describes the facts, dimensions, hierarchies, business rules, access controls, etc. But, since we purchased the OBIA products, we get a host of meta data already defined! A curious developer can quickly open the OBI Admin tool and peruse the thousands of meta data objects that are installed out-of-the-box. An adventurous developer can then take the next step and start to make changes to the meta data, but I am getting ahead of myself, so for now let s just peruse. Witepaper Using and Extending Fusion Analytics 12

15 As displayed above, the meta data is organized into three distinct layers. On the right is the Physical layer, which is where the physical table definitions are brought in to the design. On the left is the Presentation layer, which is where the objects are organized in a way that is most closely aligned with the needs of the business users (i.e. with names that make sense to an AP clerk for example). And in the middle is the Business Model and Mapping layer (or Business layer for short), which is where everything else (i.e. all the interesting stuff noted above) is defined. The mapping of physical data, through the business layer and then organized into the presentation folders, or subject areas, within the Presentation layer represents the second stage in the data life cycle. Of course, in this stage no data is physically moved, however, the logical organization of the subject areas is drastically different from the underlying physical data model which is why it is considered a separate stage (and besides, I am writing the paper so I can call it what I like). The third and final stage takes place within the OBI-EE front-end development toolset. In this stage, reports are created using a tool called Answers and these reports are then organized into pages on a dashboard by a tool that is simply called Dashboards. Witepaper Using and Extending Fusion Analytics 13

16 Reports and dashboards are stored within the OBI-EE presentation catalog and organized by subject area. Each subject area typically contains several dozen pre-built reports which provide a wealth of information right out of the box and also serve as a foundation for more extensive analytics. Extending Pre-Built Applications Having spent much of the past fifteen years implementing package software, I have learned that there is no single way to get to the finish line. However, I have picked up a few common sense guidelines for extending Oracle s pre-built analytic applications, which I will summarize below. But first, let s outline what we mean by extending the pre-built apps. The most common, easiest and most valuable extension to the pre-built apps is to extend the front-end dashboards/reports. This extension builds on the pre-built warehouse and allows customers to get more analytic output without doing any of the heavy lifting which is involved with modification to the back-end components. I would also consider Ad Hoc reporting within this type of extension and, again, this is the type of high-value, low cost benefit that companies can and should take full advantage of. This type of extension is handled via the front-end tools Answers, Dashboard and, in some cases, the Admin tool is used to make changes to the metadata for presentation or business logic purposes (obviously, meta data changes to the business layer can be more technical in nature). Witepaper Using and Extending Fusion Analytics 14

17 The other type of change involves a modification to the underlying data structures. In my experience, data extensions typically come in one of three varieties, Related columns within the source system (i.e. EBS flex fields) Supplemental or custom tables, not included in the pre-built data model or ETL adapters External systems/applications data (two approaches) Using universal adapter if data can be mapped to pre-built data model Custom ETL into custom tables I have described the front-end extensions first as this type of extension should be considered first for the reasons already noted (i.e. low cost, high value). However, in describing the 5-step process for extending OBIA, I will begin with the back-end components as this is the proper sequence that must be followed. For front-end only extensions, the first three steps, and sometimes the fourth step, can be skipped entirely. 1. In the Oracle warehouse, create or extend target tables Use X_ for custom columns Use WC_ for custom tables and include required system columns INTEGRATION_ID, DATASOURCE_NUM_ID, ROW_WID and ETL_PROC_WID 2. In Informatica, Copy or create new mappings in custom folder and extend the mapping to include additional columns or tables Follow the safe path through the existing mappings Copy or create workflows in custom folder and modify as necessary 3. Update the Data Administration Console (DAC) Change object reference to custom folder Optionally, create additional subject areas, tasks, or execution plans 4. Update the Meta Data Definition via the Admin tool Bring extended objects into the physical layer Update data model in the business layer and Extend format in the presentation layer 5. Create or modify Presentation Catalog Reports, Dashboards, Alerts, Briefing Books Witepaper Using and Extending Fusion Analytics 15

18 This process is summarized in the following diagram as well. With over a dozen implementations completed, I have identified a relatively short list of implementation tips, summarized below. If at all possible, implement plain vanilla first and then map out your extensions Even if you do not want to use the plain vanilla dashboards, best to work out any issues with the standard components before you customize Keep a plain vanilla dashboard as a reference, place customized pages into a custom dashboard Make sure you have someone with a solid understanding of the source system data More important that having an OBI-EE superstar Avoid Version Lock Do not modify the base ETL objects, copy to custom folder, follow the safe path through the pre-built data flow, use custom naming standards Place all custom metadata in custom folders, use custom naming standards, tag with a custom icon Create all custom presentation objects with a custom naming standard/prefix Full vs. Incremental The DAC includes separate tasks for Full and Incremental loads Incremental tasks use the underlying ETL base mapping (no SQL over-ride), which means that all ETL base programs are coded for incremental loads Full load tasks use a SQL over-ride in the Informatica session, which must be manually modified whenever the underlying base ETL is modified Developers need to understand this or you will have inconsistent coding which will cause significant problems Large project teams should employ multiple development schemas. For example, a development team with three parallel development threads, Primary: OLAPDW, PM_REPO, DAC_REPO Secondary: OLAPDW2, PM_REPO2, DAC_REPO2 Witepaper Using and Extending Fusion Analytics 16

19 Tertiary: OLAPDW3, PM_REPO3, DAC_REPO3 Define all data warehouse tables and indexes in the DAC repository This allows the DAC to perform standard ETL tasks on those objects (i.e. drop and recreate indexes) and also allows for automated creation of new schemas. Be sure to index all foreign keys in the FACT tables and primary keys in the Dimension tables. Beware multi-user repository management Coordination between multiple users is inherently challenging. The OBI Admin tool works fine but there will be conflicts which require human interpretation during the check-in and reconciliation process. The out-of-the-box metadata repository includes a large amount of extraneous content which can be removed This will speed development If possible, create an APPS-RO account (i.e. a read-only account with access to the tables required for the ETL process. May need to modify the directory location for Informatica Lookup files and Parameter files (depends on Apps version and OS) When migrating from one environment to another (i.e. Dev to Test to Prod), do not change the Informatica repository name Backup/restore will complete without error but will fail when an ETL is run Income Statement lines are driven by the account segment Frequently extend this to include cost center and/or sub-account Multi-currency support applies conversion rate in the ETL process and allows up to 3 different reporting currencies Most reports default to Global Currency (Global amount 1) but can be modified to access the other currencies (local amount or document amount) For multi-byte character set (i.e. UTF8), be sure to set the OS-level environment variables for NLS Conclusion OBIA enables rapid benefit with pre-built content Warehouse tables, ETL, meta data and presentation catalog True Enterprise Data Warehouse architecture Leveraging multi-dimensional model (OLAP) Provides solid foundation for developing custom BI content Requires disciplined professionals to implement, manage and extend It Works! Witepaper Using and Extending Fusion Analytics 17

20 About BizTech BizTech is recognized throughout the Mid-Atlantic and New York Metro regions as a leading IT Services firm. For more than a decade we've focused on Oracle applications, technology and consulting services for small, midsize and Fortune 500 companies. As an Oracle Platinum Partner with over 300 successful implementations, we are committed to Oracle solutions and the service we provide to our clients. BizTech offers a unique combination of proven industry methods, and a tailored selection of experienced Oracle experts to lead your organization through changes required to manage, build, and implement business driven solutions. We offer a flexible approach enabling you to select the scope of services that will meet your business goals. Using and Extending The Fusion Analytics May 2011 Author: Tom Connolly Business Technology Services (BizTech) 1150 First Ave Suite 320 King of Prussia, PA Phone: BIZTECH Copyright 2011, Business Technology Services Inc, All rights reserved. Trademarks: Products mentioned herin are trademarks or registered marks of their respective owners. This document is provided for infromation purposes only and the contents hereof are subject to change without notice. Witepaper Using and Extending Fusion Analytics 18