Oracle Data Integrator 11g New Features & OBIEE Integration Presented by: Arun K. Chaturvedi Business Intelligence Consultant/Architect
Agenda 01. Overview & The Architecture 02. New Features Productivity, Capabilities and more 03. New Features Knowledge Modules 04. ODI and OBIEE Together 05. What s there in ODI for BI? 06. Extras ODI, Essbase, SOA and more 07. Summary & Helpful Resources Q & A 2
Overview Why Oracle Data Integrator 11g? Best of breed data integration platform Ahead of rest of the industry New architecture and features BENEFIT KEY FEATURE Performance: Heterogeneous E LT Flexibility: Active Integration Platform Productivity : Declarative Design Hot Pluggable: Knowledge Modules 3
Overview Oracle Fusion Middleware Architecture Oracle s #1 application infrastructure foundation Fully integrated within broader Fusion Middle platform Key integration component in this Oracle Technology stack Provides it s run time component as Java EE Application Fully leverages the capabilities of WebLogic Server ODI 11g Component Architecture. 4
The Architecture Oracle Fusion Middleware Architecture ODI Repositories Master Repository Topology of resources, security, version management. Work Repository Data model, projects and their use. ODI 11g Component Architecture. 5
The Architecture Oracle Fusion Middleware Architecture ODI Studio Designer Operator Topology Security ODI 11g Component Architecture. 6
New Features High Availability and Scalability Via Clustered deployments for Java EE Components Uses WebLogic Server capabilities WLSCluster JDBC Connection Pooling, Load Balancing Runtime Agent supports Connection Retry Mechanism Security Integration Passwords can be externalize in Credential Stores Users can mapped to Identity Stores LDAP Oracle Internet Directory Active Directory Authentication delegation Single Sign On High Availability Configuration for ODI JEE Components. 7
New Features Enterprise Scale E LT Architecture Shift from ETL to Heterogeneous E LT Use of lightweight run agent to run in strategic location (DW Server) Transform in Separate ETL Server Dedicated/Proprietary Resources Poor Performance High Costs Traditional Approach Transform in Existing RDBMS Leverage Existing Resources High Performance Efficient Benefits Optimal Performance & Scalability Easier to Manage & Lower Cost 8
New Features Simplified Deployment Capabilities Java EE Components deploy in WLS using Predefined Templates (Metadata in topology) Create Src/Trg Data Server Definitions (WLS) Creating/Extending a domain made easy Unified Administration & Monitoring Manage/monitor components New plug in Enterprise Manager Fusion Middleware Control Single Screen Console Monitor status, metrics, notifications ODI in Oracle Enterprise Manager. Wizards creating Agent Template in WLS. 9
New Features Enhanced Monitor/Control Capabilities Oracle Data Integrator Console Extreme tool for monitoring large scale production deployments Rewritten using ADF Faces Ajax Framework Seamless Integration Oracle Fusion Middleware Control Console Allows drill down to detailed ODI design time/run time objects 10
New Features Enhanced Control Capabilities Improved Session Control New Mode Stop Immediate New Stale Session Review Running State Error State Improved Error Management Precise Information Component Life cycle (agent startup, shutdown, schedule refresh etc) Session Life Cycle (incorrect session version, agent not available etc) Gives info. for lowest level of session (steps/tasks) Automatic Notification to Enterprise Manager Fusion Middleware Control 11
New Features Design Time Productivity New JDeveloper based Integrated Development Environment ODI Studio Designer, Topology, Operator, Security Manager State of the art Features for productivity (Enhanced Windows, Docking ) Single IDE with Multiple Navigators Redesigned Interface Editors Auto Fixing Quick Edit Tab (interface diagram) Window Management Document Management Usability Features 12
New Features Design Time Productivity New fresh look & Feel. Design comparison with previous version 10g 13
New Features Enhanced Error Management Prevent Errors at design time Auto Fixing Validation Report Code Simulation (run in design time) Troubleshoot at runtime Abort session immediate mode Precise Error Information KM row by row Debugging 14
New Features ODI Knowledge Modules/Adapters Plug N Play Components Connect Extract Transform Check Integrate Enhanced KM Adapters. Oracle EBS, PeopleSoft, Oracle CDC, Golden Gate CDC,Hyperion Planning etc. New KM Adapters. ADF View in OBIEE Server, Oracle DB Multi Table Insert, Teradata Multi Statements. 15
New Features ODI Knowledge Modules/Adapters The Declarative Design approach focuses on designing declarative rules separately from the flow into which these rules are executed. Oracle Data Integrator generates the flow using these rules and code templates named Knowledge Modules. This approach is opposed to the Flow based Design that focuses on designing the execution flow logic. 16
New Featurves ODI Knowledge Modules/Adapters/Code Template Dedicated Code templates for individual task Code generated/executed derived from declarative rules/designer metadata Reusability across several interfaces/modules Can not be executed stand alone (they require metadata) Six Categories Service Knowledge Modules (SKM) 17
ODI and OBIEE Together Extract Prepare Load Model BI 18
ODI and OBIEE Together ODI 11g enhances OBIEE 11g OBIEE ODI Hides the complexity of underlying data to users and presents information in business terminology Intelligent Request Generation Optimized Data Access Populates Enterprise Data Warehouse Optimized Performance for Load & Transform Extensible Pre Packaged E LT Content Answers Oracle BI Suite EE Interactive Dashboards Publisher Delivers Oracle BI Presentation Server Oracle BI Server Oracle BI Enterprise Data Warehouse Bulk E LT Oracle Data Integrator E LT Agent E LT Metadata Other Sources SAP/R 3 PeopleSo ft Oracle EBS Siebel CRM 19
What is here for OBIEE? ODI 11g 11.1.1.5 Patch Level Load Plans New ODI Object High Data Throughput in narrow batch window Advanced Restartability Enhanced Exception Handling Capabilities Organizes Scenario Executions Parallel/Sequential/Conditional 20
What is here for OBIEE? ODI 11g 11.1.1.5 Patch Level Load Plans Can be managed from Cmd, Studio, Console, Web Service Interface Not a substitute for package/scenario Packages/Scenarios can be Orchestrated Ultimate tool for optimization of ETL Examples: Global Orchestration of various scenarios developed at project level by developers Handle Parallel in order to use best hardware resources Handle Sequential in order to avoid overloading resources Handle Conditions to restart only when it needed, in case of failure 21
What is here for OBIEE? ODI 11g 11.1.1.5 Patch Level Data Lineage Provides Origin data source details to OBIEE users ODI Lineage for OBIEE feature for consolidation Expose metadata in OBIEE dashboard Report To Source Data Lineage OBIEE Users can click to view the Lineage of data from their BI dashboard. 22
What is here for OBIEE? ODI 11g 11.1.1.5 Patch Level Data Lineage Important capability to view detailed metadata lineage info directly within OBIEE Answer dashboards Understand your data flow OBIEE Users can view the Lineage details directly in their BI dashboard. 23
What is here for OBIEE? OBIEE Concept maps the ODI Concept ODI seamlessly integrates data from OBIEE ODI Provides specific methods for reverse engineering/extracting data from ADF View Objects via OBIEE physical layer (using integration interfaces) An Oracle BI Server correspond to a data server in ODI In this server, a catalog/owner pair maps to ODI physical schema ODI connects to this server to access via bypass connection pool (that supports ADF View Objects). ODI uses JDBC to connect an Oracle BI Server 24
What is here for OBIEE? How to work for OBIEE within ODI 11g? Install/Configure No technology specific requirements for using OBIEE in ODI 11g Connectivity JDBC Drivers, Bypass Connection Pool (that supports ADF View Objects) Setup the Topology Creating an Oracle BI Server Creating Oracle BI Physical Schema Setup an Integration Project (Using Oracle BI Server) Create and Reverse Engineer an Oracle Business Model Design an Interface (Using BI related KM) Limitation Oracle BI can not be used as target or staging area. It is not possible to integrate data into Oracle BI using KM. Data integrity check is not supported in Oracle BI Server. You may check extracted data in staging area using other technology. 25
Extras ODI, SOA and more ODI s More New Features Java APIs for Task Automation. Create/Manage both design and run time operations. Enhanced Declarative Design Approach Dramatically reduces the effort at design time E LT provides maximum performance at run time New Datasets and set based operators (in Interfaces) 26
Extras ODI, SOA and more ODI s More New Features Lookups New wizard in interface editor (to lookup driving source data store) Execute Left Outer Join in FROM, Expression in SELECT 27
Extras ODI, SOA and more ODI s More New Features (Other E LT Features) Partitioning Data Server Partitions can be reverse engineered using RKM Support for Native Sequence Directly Maps to database defined sequences Support for Natural Joins Defined at topology level, Handled by native engine Automatic Temp. Index Mgmt. Optimization of execution of joins/filters at source (by KM) ODI s More New Features (Other Studio Usability Features) Scenario Naming Convention Object name, folder path, project name etc Long Object Names DB Object Names (128 Chars), Repository Object Names (400 Chars) Enhanced OdiPurgeLog Tool Purge a log while retaining a number of sessions in it Import Report Import becomes fully traceable operation, All changes can be identified & archived SOA ODI 11g is only integration tool which can run entirely within a Service Oriented Architecture 28
Summary With New release ODI 11g and 11.1.1.5 patch level Oracle has consolidated numerous one off patches, and introduced new enhancements for customers using its strategic data integration platform in various contexts BI, Data warehouse, Application and Service Oriented Integration while preserving the key product differentiators: Declarative Design, Knowledge Modules, Hot Plugability E LT. Resources/References E12642 01: Upgrade Guide for ODI 11g E12645 02: KM Developer s Guide for ODI 11g Oracle Data Integrator Documentation URL (OTN) OBE Oracle By Example (OTN) White Papers, Blogs and more But, Never forget your best friend 29
Questions Contact Information Arun K. Chaturvedi Business Intelligence Consultant/Architect Email: achaturv@astcorporation.com Phone: 630 778 1180 Presentation Copywww.astcorporation.com News & Events Presentations 30