Oracle Data Integration Real Time Access to Real Time Information Tony Velardo DIS Sales Manager Valarie Bedard Principle Sales Consultant Oracle Data Integration Moving Data to Transform Business 1
IT Obstacles to Unifying Information Avoiding the Accidental Architecture Custom Reporting Analytics Packaged Applications Business Intelligence SOA Applications Data Silos Data Migration Batch Scripts Data Marts Data Replication Custom Data Hubs Data Warehousing SQL Data Federation Java Data Access OLTP & ODS Systems Data Warehouse, Data Mart Oracle PeopleSoft, Siebel, SAP Custom Apps Files Excel XML OLAP Fragmented Data Silos Slow Performance Out of sync Lack of Trusted Data What s the cost? 2
Choosing the Right Data Integration Platform Key Criteria for Maximum ROI from Cloud Enterprise-Class Comprehensive set of capabilities Flexible to address multiple use cases Support for heterogeneous platforms High Performance High performance to handle spikes Easy and cost-efficient to scale Fast Time to Value Easy to deploy and manage Pre-integrated, certified solutions Low cost of ownership 3
Oracle Data Integration Complete Offering for Enterprise Data Integration Legacy Applications OLTP Unstructured Oracle Data Integrator Oracle GoldenGate Oracle Enterprise Data Quality Active Data Guard Complete and best-of-breed approach for enterprise data integration Maximum performance with lower TCO, ease of use and reliability Certified for leading technologies to deliver fast time to value Oracle Customers Report: 80% lower TCO Five times higher performance 70% reduction in development costs Modernization Custom MDM BI Big Data Synchronization SOA 4
Data Integration Solutions High Availability Do you need 24/7 system availability? Is your standby database sitting idle? Is planned downtime at least as costly as unplanned? Need to synchronize data across geographies? Consolidation / Migration Want to get onto lower cost standard infrastructure? Can t afford downtime? Does your data cost $$ or make $$? Why not fix data assets as well as HW/SW assets during an infrastructure move? Business Intelligence Need fresh data 24X7 to optimize operations? Mission-critical OLTP? Minimize impact on source systems? Need operational visibility in real time? Growing data volumes and shrinking batch windows? Oracle Active Data Guard ensures business continuity and resource utilization Oracle GoldenGate reduces planned downtime and enables Active-Active Oracle GoldenGate enables zero downtime migrations and upgrades Oracle Data Quality for ODI moves and improves data so it can be trusted Oracle GoldenGate provides realtime data for analysis Oracle Data Integrator (ODI) populates DW with high performance and low TCO 5
Oracle GoldenGate Real-time Data Integration Legacy Applications OLTP Unstructured Oracle GoldenGate Log-based Change Data Capture and Delivery Bidirectional Replication Heterogeneous Sources and Targets Reliability and Transaction Integrity CEP Integration, Real-time Events High-performance, lowimpact real-time data integration Timely data for improved business insight Continuous availability for 24/7 operations 6
Who s Using GoldenGate? Leader in real-time data integration solutions Over 3,000 customers including Fortune 500 companies, representing Financial Services, Communications, Healthcare, Public Sector, Retail, Utilities industries 4 of top 5 largest commercial banks 3 of top 3 busiest ATM networks 5 of top 5 financial data services companies 5 of top 5 telecommunications providers 3 of top 5 largest food & drug stores 7
Oracle GoldenGate Under the Covers That s the Magic of Macy s Shopping at the worlds largest retails and your credit card transactions are moved in real-time to data centers around the world, before being approved, using Oracle GoldenGate (Retail Decision) Good TV. Better TV. DirecTV. If you call DirecTV customer service, your call data will move to their data warehouse via Oracle GoldenGate, so the rep can provide you with personalized recommendations and promotions. Say it with Flowers Ordering from 1-800-Flowers, Oracle GoldenGate will move your order to their reporting system to help process and track your order seamlessly No Charge ATM Fees 3 of the 3 busiest ATM Networks use Oracle GoldenGate for Continuous Availability. Get Sick in Pittsburgh If you get a health check at UPMC, the clinical applications that doctors use are supported by Oracle GoldenGate to enable high-availability 8
GoldenGate for Banking / Financial Services Bank of America High availability for ATM network Eliminate downtime during database, hardware & Siebel migrations High availability for app. that assesses online access risk State Street Bank Offload reporting from hosted applications High availability for private Exa cloud Integrate relational data with enterprise service bus JP Morgan Chase Eliminate planned downtime for retail banking app. Eliminate planned downtime for treasury app. Transform production data and synchronize with reporting database UBS Standardize to reduce complexity & cost Distribute data among internal applications High availability for planned & unplanned downtime Fidelity Eliminate planned downtime for Siebel app. High availability for tax accounting app. High availability for trade confirmation system SunTrust Fraud detection for online banking app. Integrate relational data with enterprise service bus Reduce or eliminate batch extracts Capital One High availability for enterprise imaging app. Eliminate downtime during database upgrades Off-load reporting for car loan origination app. Chicago Mercantile Exchange, Deutsche Bank, US Bank, Swedebank, Visa, Prosa, FNBO, First Data, Retail Decisions, Zagreb Bank, ADP, American Express, VISA, Desjardins Bank, Hyundai Securities, Bombay Stock Exchange, Nomura, RBS, Bankart 9
Oracle GoldenGate Across the Enterprise Databases Oracle GoldenGate Capture: Oracle DB2 (all platforms) Microsoft SQL Server Sybase ASE MySQL Teradata Enscribe SQL/MP SQL/MX JMS message queues Oracle GoldenGate Delivery: All listed above, plus: PostgreSQL TimesTen, Coherence Flat Files for Netezza, Greenplum, ETL products O/S and Platforms Linux Sun Solaris Windows 2000, 2003, XP HP NonStop HP-UX HP OpenVMS IBM AIX IBM z Series IBM i Series zlinux 10
Oracle GoldenGate Use Cases Low-Impact Real-Time Data Integration & Transactional Replication New DB/HW/OS/APP Zero Downtime Upgrade & Migration Mainframe Database Log-based, changed data ODS Data Integrator Fully Active Distributed DB Reporting Database Data Warehouse Highly Available / Disaster Recovery Query Offloading, Disaster Recovery Real-time BI, Operational Reporting, MDM Data Synchronization across the Enterprise Message Bus Global Data Centers Message Bus Event Driven Architecture, SOA 11
Sample GoldenGate Mainframe Customers Chicago Mercantile Exchange (DB2/zOS to/from Oracle) Migration off mainframe AARP (DB2/zOS to Oracle and JMS) Integrate mainframe data with downstream applications and SOA/EDA Liberty Mutual (DB2/zOS & DB2/LUW to Teradata) Real-time feed to data warehouse for single view of the business Macy s (DB2/zOS to Exadata) Real-time integration of inventory for faster shipping & replenishment Discover (DB2/zOS to Oracle) Mainframe data integration with downstream applications ACS-Xerox (DB2/zOS to Oracle) Report off-loading to reduce cost of reports AutoZone (DB2/zOS to DB2/LUW DPF & GreenPlum) Report off-loading to save mainframe cycles and speed reporting Citigroup (DB2/zOS to Oracle) PeopleSoft application migration off mainframe Zagreb Bank (DB2/zOS to Exadata via ODI & DB2/zOS to DB2/zOS) Data center consolidation, Data warehouse feed Sears (DB2/zOS to GreenPlum) Report off-loading Intercontinental Hotels Group (DB2/zOS to Oracle) Mainframe data migration and sharing CIBC (DB2/zOS to HP NSK) Data integration with mainframe and HP Non-stop 12
Across the Enterprise Market Transparency with Real-Time Data Access Business Challenges: High rate of analysis required to ensure the market is functioning properly The freshest data is required to track pricing, bidding and purchasing so the correct buy and sell price is displayed Trades must be monitored in real-time to ensure market visibility and integrity Performing analysis against the production system negatively impacts system performance Alternatively, relying on nightly batch loads caused delays in uncovering potential issues GoldenGate Solutions: Live Reporting from Comprehensive Bid Management System to an ODS Real-time Data Integration to monitor market performance Zero-downtime database upgrade from Oracle 9i to Oracle 11g (not depicted) Results: Real-time transaction management, issue resolution and automated analysis Better marketplace performance and lower cost of ownership Live Reporting & Real-Time Data Integration Real Time Pricing, Load, Demand (OISR) ODS for Reporting & Analytics Market Data (MIS) Market Monitoring Performance Database DW Historical Data GoldenGate has provided the data integration solution we were looking for. We look forward to rolling this product out for other projects across the organization. -John Bubb, Product Manager, Business Intelligence Products The New York Independent System Operator 13
Across the Enterprise Site License Multiple Implementations of GoldenGate Pervasive Use Today Initially had handful of uses in 2008. Has grown to over 50 implementations since then. Combination of reducing barriers to use technologies and spread of GoldenGate knowledge. CHALLENGES / OPPORTUNITIES Globally distributed bank with needs to integrate data and ensure business resiliency Multiple types of databases requiring common data sharing in all facets of bank Experienced online verification outage that increased risk to bank and impacted customer satisfaction Some of the GoldenGate Implementations Supports sign-on authentication optimizing performance, risk avoidance, and supporting always there policy Multi-master global wire transfer application Zero downtime DB migration to new version using phased approach (e.g. Exadata consolidation project) Consolidation of wholesale credit from 50+ disparate data sources Always available, zero data loss reference architecture ATM application fault tolerance Real-time feed to data warehouse and business intelligence systems Payments application integration with messaging technology for mass broadcast 14
Oracle GoldenGate Runtime Architecture Capture: committed transactions are captured (and can be filtered) as they occur by reading the transaction logs. Trail: stages and queues data for routing. Pump: distributes data for routing to targets. Route: data is compressed, encrypted for routing to targets. Delivery: applies data with transaction integrity, transforming the data as required. Manager Manager Source Oracle & Non-Oracle Database(s) Bi-directional Target Oracle & Non-Oracle Database(s) 15
Oracle GoldenGate Adapters ANY OGG CAPTURE: captures data from any source and creates an OGG trail for normal routing and delivery OGG for FLAT FILE DELIVERY: generates data and control files from trails for batch consumption by third party systems Oracle GoldenGate for Flat File CAPTURE FILE - FILE - UDF UDF FILE LAN / WAN / INTERNET OVER TCP/IP FILE - FILE - UDF UDF FILE FILE DELIVERY CONTROL FILE DATA FILES Source System Target System OGG for JAVA JMS CAPTURE: captures data from JMS text messages and creates an OGG trail for normal routing and delivery OGG for JAVA DELIVERY: delivers change data from any OGG Capture to Java APIs or directly to JMS as formatted messages Oracle GoldenGate for Java JMS DELIVERY JMS CAPTURE FILE FILE - - UDF UDF FILE LAN / WAN / INTERNET OVER TCP/IP FILE FILE - - UDF UDF FILE JAVA DELIVERY Source System Target System 16 16 Copyright 2011, Oracle and/or its affiliates. All rights reserved.
Oracle GoldenGate Flexible Deployment Models Unidirectional Query Offloading Zero-Downtime Migration Bi-Directional Hot Standby or Active-Active for HA Peer-to-Peer Load Balancing Multi-Master Broadcast Data Distribution Integration/Consolidation Data Warehouse Data Distribution via Messaging BPM BAM CEP 17
Oracle GoldenGate Deliver Many Integration Example GoldenGate s flexibility allows for multiple integration schemes The Capture process is always the same Delivery can be through Staging tables Intermediate flat files Messaging Capture Trail Integration via Staging Tables Delivery Integration via Files Data Pump + UE Integration via Messaging data files data files data files ETL Tool ETL Tool Data Pump + UE JMS / MoM Queue or Topic ETL Tool 18
Management Pack for Oracle GoldenGate Administration & Monitoring Oracle GoldenGate Monitor & OEM Centralized management of GoldenGate modules Advanced monitoring, alerting (SNMP support), lag graphs, and historical repository. Browser based access to error logs and process reports Enterprise Manager Plug-in 19
Oracle GoldenGate Veridata Data Comparison Without Downtime Oracle GoldenGate Veridata identifies and reports data discrepancies between heterogeneous databases Reduce financial/legal risk exposure Speed and simplify IT work in comparing data sources No disruption to business systems Improved failover/migration Confident decision-making and reporting 20
Streams Statement of Direction Oracle Database Upgrade Guide, Version 12c http://docs.oracle.com/cd/e16655_01/server.121/e17642/deprecated.htm#babeajje Oracle Streams is deprecated in Oracle Database 12c and may be desupported and unavailable in a later Oracle Database release. Oracle s stated direction is to use Oracle GoldenGate for replication implementations in place of Oracle Streams. Oracle Database Advanced Queuing is independent of Oracle Streams and continues to be enhanced. Oracle Database Advanced Replication is deprecated in Oracle Database 12c. Use Oracle GoldenGate to replace all features of Advanced Replication, including multi-master replication, updatable materialized views, hierarchical materialized views, and deployment templates. 21
Join the Data Integration Community Twitter or Facebook twitter.com/orclgoldengate facebook.com/oracledataintegration Oracle GoldenGate YouTube channel youtube.com/oraclegoldengate MOS Knowledgebase http://support.oracle.com/ Oracle Technology Network LinkedIn Oracle Data Integration Oracle s Data Integration blog blogs.oracle.com/dataintegration OTN: GoldenGate http://www.oracle.com/technetwork/middle ware/goldengate/overview/index.html Oracle Learning Library http://apex.oracle.com/pls/apex/f?p =44785:1:0::NO Oracle Technology Network Oracle s Public Help Forum http://forums.oracle.com/forums/forum.j spa?forumid=860&start=0 Oracle.com/goto/dataintegration 22
23