<Insert Picture Here> Analytic World and Oracle s Approach Tolga YILDIRIM - BI Presales Manager (EE-CIS)
Despite pressure on IT spending Gartner EXP Worldwide Survey of More than 1,600 CIOs Shows IT Budgets in 2010 to be at 2005 Levels Gartner Press Release - January 19, 2010 Business Intelligence remains a top priority Top 3 Business Priorities for 2010 1. Business process improvement 2. Reducing Enterprise Costs 3. Increasing the use of Information/Analytics Copyright 2010 Oracle Confidential 2 OCDM Technical Training 2
Information Management Challenges Q: Why can t I get access to the data I need for decision making? Q: Why are my applications referring to last week s numbers? Q: Why do I have so many duplicate copies of data? Much of it is inaccurate! Accessible Information Available Secure Reliable Up-to-Date Information Fast access Multiple sources Actionable Trusted Information Accurate Consistent Quality 3
Today s Reality Not planned, not consistent, not secure, difficult to change, etc.. Custom Reporting Analytics Packaged Applications Business Intelligence Performance Management Data Silos Data Data Data Migration Replication Warehousing Data Marts Data Hubs Data Federation Data Access Batch Scripts Custom SQL Java OLTP & ODS Systems Data Warehouse Data Mart SAP, Oracle PeopleSoft, Siebel, Custom Apps Files Excel XML OLAP 44
SOLUTION IS; Enterprise Data Warehouse environment instead of having departmental reporting silos Single BI tool to satisfy all reporting needs 5
WHAT IS DATA WAREHOUSE? A data warehouse (DW) is a database used for reporting. The data is offloaded from the operational systems for reporting. The main source of the data is cleaned, transformed, catalogued and made available for use by managers and other business professionals for data mining, online analytical processing, market research and decision support. 6
Challenge: Much More Data to Analyze Data Warehouse Size and Growth Source: TDWI Next Generation Data Warehouse Platforms Report, 2009 7
Challenge: User Requirements Not Met High Churn in Data Warehouse Platforms Source: TDWI Next Generation Data Warehouse Platforms Report, 2009 8
Reference Architecture Major Components 9
Reference Architecture Major Components Data Sources 10
Reference Architecture Major Components Information Management 11
Reference Architecture Major Components Information Access 12
Reference Architecture Load Process from sources 13
Reference Architecture Load Process to Data Warehouse 14
Reference Architecture Information Provisioning 15
Moving toward a Clean Architecture EXADATA EXADATA- ORACLE - ORACLE OLAP Sales Analysis Forecasting Inventory Analysis Data Mining Market Basket Analysis Loss Analysis Customer Loyalty Analysis GoldenGate Oracle Data Integrator OBIEE Foundation Interactive Dashboards Ad-hoc Analysis Reporting & Publishing Detect & Alert Scorecards Office Integration ESSBASE 16
Moving toward a Clean Architecture EXALYTICS EXADATA EXADATA- ORACLE - ORACLE OLAP Sales Analysis Forecasting Inventory Analysis OBI Foundation Data Mining Market Basket Analysis Loss Analysis Customer Loyalty Analysis GoldenGate Oracle Data Integrator TimesTen for Exalytics Memory Optimized Essbase 17
Oracle Golden Gate 18
Oracle GoldenGate : One Line Summary The Solution for Enterprise-Wide Real-Time Data Needs Database and applications, Mixed sources, distributed systems, legacy, OLTP, OLAP Real-time information Real-time Access Mission Critical Applications & Data, Business Intelligence, Reporting for Customers, Partners & Employees Oracle GoldenGate delivers real-time access of real-time information, enabling companies to dramatically improve the availability, reliability, and performance of critical data across enterprise systems. 19
Exadata as Ideal Platform for DW 20
Oracle Exadata Database Machine The Ideal Data Warehousing Platform Improve query performance by 10x Better insight into customer requirements Expand revenue opportunities Consolidate OLTP and analytic workloads Lower admin and maintenance costs Reduce points of failure Integrate analytics and data mining Complex and predictive analytics Lower risk Streamline deployment One support contact 21
Why is Exadata Fast? Exadata off-loads data intensive processing to the storage Row filtering based on where predicate Column filtering Join filtering Incremental backup filtering Storage Indexing Scans on encrypted data Data Mining model scoring Database Machine delivers a high speed IO subsystem Exadata delivers smart flash cache for all workloads 22
Traditional Query What Were Yesterday s Sales? Select sum(sales) where salesdate= 22-Jan-2010 Return entire Sales table Discard most of sales table Sum Data is pushed to database server for processing I/O rates are limited by speed and number of disk drives Network bandwidth is strained, limiting performance and concurrency 23
Exadata Smart Scan Improve Query Performance by 10x or More What Were Yesterday s Sales? Select sum(sales) where salesdate= 22-Jan-2010 Return Sales for Jan 22 2010 Sum Off-load data intensive processing to Exadata Storage Server Exadata Storage Server only returns relevant rows and columns Wide Infiniband connections eliminate network bottlenecks 24
Exadata Storage Index Transparent I/O Elimination with No Overhead Exadata Storage Indexes maintains summary information about table data in memory Stores MIN and MAX values of filter columns Typically one index entry for every MB of disk Eliminates disk I/Os if MIN and MAX can never match where clause of a query Negative index Completely automatic and transparent Order_date Ship_date Cust _ID Prod _ID Amount 03-SEP-2009 19-SEP-2009 10075 32932 10,000.00 03-SEP-2009 05-SEP-2009 20098 20098 20,000.00 03-SEP-2009 07-OCT-2009 10089 20010 15,000.00 03-SEP-2009 01-OCT-2009 20100 10000 35,000.00 03-SEP-2009 19-OCT-2009 80300 30000 10,000.00 03-SEP-2009 03-NOV-2009 10000 2030 40,000.00 MIN ship_date = 19-SEP-2009 MAX ship_date = 07-OCT-2009 MIN ship_date = 01-OCT-2009 MAX ship_date = 03-NOV-2009 Select * from orders where ship_date < 31-SEP-2009 Only first set of rows can match 25 25
Exadata Compression Reduce Disk Space Requirements 1.4x 2.5 x Oracle Advanced Compression 3x Oracle Hybrid Columnar Compression 10x 15x Uncompressed Data Data Warehouse Appliances OLTP Data DW Data Archive Data Oracle 26
Advanced Analytics 27
Built-in Analytics Secure, Scalable Platform for Advanced Analytics Oracle OLAP Analyze and summarize Oracle Data Mining Uncover and predict Complex and predictive analytics embedded into Oracle Database 11g Reduce cost of additional hardware, management resources Improve performance by eliminating data movement and duplication 28
Summary Management Improve Response Time with Materialized Views Region SQL Query Date Sales by Region Sales by Date Query Rewrite Sales by Product Sales by Channel Products Relational Star Schema Channel Materialized Views Pre-summarized information stored within Oracle Database 11g Separate database object, transparent to queries Supports sophisticated transparent query rewrite Fast incremental refresh of changed data 29
Cube Organized Materialized Views Region SQL Query Date Summaries Query Rewrite Automatic Refresh Products Channel Exposes Oracle OLAP cubes as relational materialized views Provides SQL access to data stored in an OLAP cubes Any BI tool or SQL application can leverage OLAP cubes 30
Oracle Data Mining Find Hidden Patterns, Make Predictions Retail Customer Segmentation Response Modeling Communications Customer churn Network intrusion Healthcare Patient outcome prediction Fraud detection Financial Services Credit Scoring Possibility of default Utilities Product bundling Predict power line failure Public Sector Tax fraud Crime analysis Collection of data mining algorithms that solve business problems Simplifies development of predictive BI applications Embedded in Oracle Database instance and storage 31
Oracle #1 for Data Warehousing Source: IDC, July 2009 Worldwide Data Warehouse Management Tools 2008 Vendor Shares 32
Oracle Data Integrator 33
Oracle Data Integrator Data Movement and Transformation from Multiple Sources to Heterogeneous Targets B E N E F I T D I F F E R E N T I A T O R 1 Best Performance Heterogeneous E-LT 2 Productivity Declarative Design 3 Real-time Integration Declarative CDC 4 Hot-Pluggable Knowledge Modules 5 Native SOA SOA-friendly Architecture 34
Differentiator: E-LT Architecture High Performance Conventional ETL Architecture Extract Transform Load Transform in Separate ETL Server Poor Performance High Costs Proprietary Engine Proprietary Language Next Generation Architecture E-LT Transform in Existing RDBMS High Performance Easier to Manage & Lower Cost Transform Extract Load Transform Leverage RDBMS Native SQL 35
Differentiator: Declarative Design Productivity Specify ETL Data Flow Graph Developer must define every step of Complex ETL Flow Logic Traditional approach requires specialized ETL skills And significant development and maintenance efforts Conventional ETL Design Declarative Set-based Design Simplifies the number of steps Automatically generates the Data Flow whatever the sources and target DB Benefits Significantly reduce the learning curve Shorter implementation times Streamline access to non-it pros ODI Declarative Design 1 2 Define What You Want Automatically Generate Dataflow Define How: Built-in Templates 36
Differentiator: Declarative Design Productivity 1 Define What You Want Targets Sources Transforms Oracle Data Integrator Interface Declarative Design 37
Differentiator: Declarative Design Productivity 1 Define What You Want Targets Sources Transforms Oracle Data Integrator Interface Declarative Design 38
Differentiator: Declarative Design Productivity 1 Define What You Want Targets Sources Transforms Oracle Data Integrator Interface Declarative Design 2 Define How to Do It: Select Template Bulk Load Changed Data Capture Incremental Update Slowly Changing Dimension 39
Differentiator: Declarative Design Productivity 1 Define What You Want Targets Sources Transforms 3 Automatically Generate Data flows Oracle Data Integrator Interface Declarative Design 2 Define How to Do It: Select Template Bulk Load Changed Data Capture Incremental Update Slowly Changing Dimension 40
Differentiator: Declarative CDC Real-Time Business Intelligence CDC High efficiency of integration by reducing the volume of source data processed in the flow Reduced implementation time Tracking and moving just the changed data Declarative set-based mapping Ensured Data Consistency 41
Differentiator: Knowledge Modules Hot-Pluggable: Modular, Flexible, Extensible DESIGN: Declarative Design IMPLEMENTATION: Extensible Knowledge Module Templates Reverse W S W S W S Sources CDC Journalize Load Staging Tables Check Error Tables Integrate Target Tables Services 42
Heterogeneous Sources & Targets Generic SQL DB Oracle DB 9i Oracle DB 10g Oracle DB 10g XE IBM DB2/400 IBM DB2/UDB IBM Informix SE IBM LDAP Server MS SQL Server 2000 MS SQL Server 2005 MS SQL Server 2005 SE MS Office Access 2000 MS Office Excel 2000 MS Active Directory Sybase ASA 8.x & 9.x Sybase IQ 12.x Sonic MQ v7.0 Teradata V2R5.x Teradata V2R6.x Netezza Performance Server 2.2.1 Hyperion Essbase PostgresSQL 8.1 MySQL 4.0 MySQL 5.0 Oracle BI Suite 10g Oracle BAM 10g Oracle Internet Directory 9i OpenLDAP 2.3 Siebel CRM 7.8 JD Edwards PeopleSoft SAP R/3 Oracle EBusiness Suite Oracle AQ 10g Oracle SOA Suite Oracle ESB 10g SalesForce.com App Exchange Any JMS Standard Implementation 43
Differentiator: SOA-friendly Architecture Native SOA Bulk Transf. Services Data Access Transform Business SOA Infrastructure ESB Business Processes Data Flow Services 1. Expose transformations as Web Services 2. Orchestrate data flows 3. Publish data flows as web services in your SOA infrastructure Services Data Access Transform Business SOA Infrastructure ESB Business Processes Data Access Services 1. Generate and share data access services 2. Generate and deploy data services 3. Test data services 4. Leverage data services in your SOA infrastructure 44
Oracle Business Intelligence 11g Most complete. Most integrated. 45
Lowest TCO Among Mega-vendors According to Gartner 46
Unified End User Experience Complete. Consistent. Accurate. Many Channels Mobile Scorecards Reports Office Integration Interactive Dashboards Applications & Portals Geospatial Visualization Search Ad-hoc Queries Collaboration 47
Open. Oracle Business Intelligence 11g Data Integration Scorecards Interactive Dashboards Reporting & Publishing Ad-hoc Analysis Office Integration Search Detect & Alert Collaborate Mobile Embedded Common Business Intelligence Foundation OLTP & ODS Systems Data Warehouse Data Mart Exadata OLAP Sources Packaged Applications (Oracle, SAP, Others) Unstructured & Semi-Structured Excel XML/Office Business Process 48
Integrated. Oracle Business Intelligence 11g Data Integration Scorecards Interactive Dashboards Reporting & Publishing Ad-hoc Analysis Office Integration Search Detect & Alert Collaborate Mobile Embedded Common Enterprise Information Model Common Metadata Foundation across all Data Sources Common Security, Access Control, Authorization, Auditing Common Request Generation and Optimized Data Access Services Common Clustering, Workload Management, & Deployment Common Systems & Operational Lifecycle Management OLTP & ODS Systems Data Warehouse Data Mart Exadata OLAP Sources Packaged Applications (Oracle, SAP, Others) Unstructured & Semi-Structured Excel XML/Office Business Process 49
Complete. Oracle Business Intelligence 11g Data Integration Scorecards Interactive Dashboards Reporting & Publishing Ad-hoc Analysis Office Integration Search Detect & Alert Collaborate Mobile Embedded Common Business Intelligence Foundation OLTP & ODS Systems Data Warehouse Data Mart Exadata OLAP Sources Packaged Applications (Oracle, SAP, Others) Unstructured & Semi-Structured Excel XML/Office Business Process 50
Interactivity and Visualization Comprehensively Improved Ease of Use Extensive user interface improvements Find what you want quickly Do want you want easily 51
New Interactive Visualizations Interactive, Accessible and Intuitive Animated transitions Range and Paging sliders Legend-based interactions Master-Detail linking Extensive and extended charts Consistent, hi-fidelity charting across Oracle product line 52
Interactive Analytics Powerful Analysis interface for every class of user Powerful dashboards Visually Appealing Intuitive 100% Thin Client Across all styles of analysis R-OLAP, M-OLAP, Scorecards, Reporting, Collaboration, Actions Across all Data Sources Simplified model for users Federated data access On-the-fly calculations, even with complex share and time series Custom members & groups Share, collaborate, & publish Consistency & alignment 53
Location Intelligence New Depth and Breadth of Analysis Most BI data already contains geographic dimension of analysis Deliver deeper analytical insights through Spatial visualisation and data Increase the ROI of BI & GIS systems In effect, Oracle is "flipping the switch" and turning spatial into a product feature when such a capability is needed. It will be daunting to compete against this company. Microsoft, beware we haven't heard much about geospatial and SQL Server integration lately you are in danger of becoming a non-player. And IBM continues to play with ESRI only, and resists striking out on its own in what could be a missed opportunity. 54
Location Intelligence Where does it really matter? Vertical Sectors Agriculture Security and Defence Healthcare Utilities Travel & Transportation Telco Banks & Insurances Postal services Local & Central Government Environment... Functional Domains Sales Analysis Fraud detection Impact Analysis Fleet Management Behavioural Analysis Network Monitoring Resource allocation planning Customer Satisfaction Crime Analysis... Every entity providing services, or selling products, through an infrastructure deployed on the territory. Public Sector and Utilities are especially hot 55
Oracle Business Intelligence Mobile An option / feature of Oracle BI 11g Enables ipads and iphones to become another 1 st class BI channel Delivers mobile BI instantly, leveraging existing BI Foundation 56
First Closed-Loop BI Solution Action Framework THE OLD WAY THE NEW WAY Disconnected. Open Ended. Integrated. Actionable. 57
11g Provides Creates Closed Loop Intelligent Business Processes Business Events Business Conditions Key Performance Indicators Reporting and Alerting Initiate Business Processes & workflows Track and Monitor Progress ACT DETECT SINGLE ENTERPRISE INFORMATION MODEL ANALYZE Ad-hoc OLAP query Interactive visualization Role-based dashboards Guided Navigation 58
Intelligent Business Process Sales Rep in Consumer Electronics Manufacturer Rep receives out of stock alert on iphone. Unable to fulfill order for key account on hot new product. DETECT Initiates change request in order management system. Updates discount terms to reflect greater discount. ACT SINGLE ENTERPRISE INFORMATION MODEL ANALYZE Reviews all orders for customer with hot products and analyzes available to promise for replacement. 59
Actionable Business Intelligence Empower Users to take Appropriate Action Define Actionable response to insight Associate with key business conditions Navigate guide users to relevant analysis, web content and transactional systems Take Action initiate workflows & business processes, scripts and Java programs Extensible, Secure, Action Framework for User Driven Insight to Action 60
Insight Driven Business Processes Align Insight with Execution 1. BI on the Process Cycle time for order fulfillment has increased in the last hour Field Service visits are running 90 minutes behind schedule 2. BI within the Process Should I offer this customer a discount? Should I extend this customer s credit? 3. BI initiating the Process DSO for this customer is too high for 60-90 days. Issue a credit hold. DPO for our #1 supplier is too high, initiate a payment action. BI Analyzes Process Metrics Oracle BI & BAM Process Calls into BI Oracle BI Invoke a Process from BI Oracle BI 61
Scorecards and KPIs New in Oracle Business Intelligence 11g Integrated BI Component KPIs as Core Metadata Thresholds, Owners, History Auto-Generated Interactive Analyses Linked objectives & Initiatives Automatic Detection KPI alerts triggered by thresholds Strategy Visualization Strategy Maps and trees Cause and Effects Watchlists Annotations & Override All methodologies Balanced scorecard, 6 sigma, Baldrige Strategy Trees KPI Watchlists Cause and Effect 62
Oracle Exalytics 63
Oracle Exalytics Super fast R-OLAP and M-OLAP Super fast Financial & Operational Planning Optimized Hardware Memory: 1 Terabyte DRAM Processors: 40 Intel Cores Software Breakthroughs Adaptive In-Memory Caching In-Memory Columnar Compression Optimized Storage Block Access Free Form Data Exploration High Density Visualizations Optimized for Exadata Best Cost/Performance 64
Oracle Exalytics Under the Hood TimesTen for Exalytics Memory Optimized Essbase 1 TB RAM 40 Processing Cores High Speed Networking Adaptive In-Memory Tools Optimized Oracle Business Intelligence Foundation Suite In-Memory Analytics Software In-Memory Analytics Hardware 65
Exalytics Hardware Architecture RAM Machine Optimized to Run BI Foundation Suite Memory 1 TB RAM, 1033 MHz Compute 4 Intel Xeon E7-4870, 40 cores total Networking 40 Gbps InfiniBand 2 ports 10 Gbps Ethernet 2 ports 1 Gbps Ethernet 4 ports Storage 3.6 TB HDD Capacity 66
Oracle Exalytics Customer Results Largest mortgage provider in Denmark, major private bond issuer in Europe Need to deliver outstanding performance for summary and transaction grain analysis 35X to 70X faster with Exadata + Exalytics Supplies automotive industry with market intelligence PolkInsight Need highly interactive dashboards and visualizations for global analyst community > 10X faster on average and up to 100X faster in specific cases Large oilfield services company with about ~860 rigs deployed around the world Need to drive usage of packaged BI Applications across the organization 5X shorter time to develop; 50X faster than a custom report (without tuning) Large cloud infrastructure services company Need highly interactive visualizations for large numbers of individual analyst data sets Consistent Sub-second interactivity on par with desktop tools down from ~30 secs A Global CPG Company Global consumer pre-packaged foods company Need more frequent planning and budgeting cycles for 2000+ users 6X faster cycle time - 4 hours down from more than 24 hours 67
References 68
BI Türkiye Referansları İletişim Finans/Bankacılık Sigortacılık Kamu 69
BI Türkiye Referansları Üretim / Perakende Otomotiv Sağlık Enerji / Maden Servis ETĐ MADEN ĐŞLETMELERĐ 70
71