Oracle Data Integration Solutions GoldenGate New Features Summary

Transcription

1 Oracle Data Integration Solutions GoldenGate New Features Summary Valarie Bedard Principle Sales Consultant Tony Velardo DIS Account Manager Oracle Data Integration Moving Data to Transform Business 1

2 Note to Reader - March 2014 This deck represents highlights of features in OGG 11.2 and OGG 12.x It does not represent ALL features in either of these releases It is recommended that the reader review the Release Notes for either version of OGG to get the complete picture of new features per release It is recommended that the reader review the OGG Install Guide, Admin Guide, and Reference Guides for either version to understand dependencies, usage and syntax for any functionality or feature mentioned in this deck. 2

3 Quick List of OGG Database Integration points Ability to ignore triggers when applying data (OGG V 12 default) Ability to ignore constraints (like cascade delete) when applying data Optimized ASM capture - DBLOGREADER Access to Oracle s Encryption Methods Transparent Data Encryption Tablespace Encryption RMAN Integration log retention Automatic Schema level supplemental logging Integrated Capture - Capture compressed data including EHCC Database Vault access Oracle s AES encryption algorithms for trail files OEM Grid Control Plug-in for Monitoring, control and metrics Exadata DBFS optimizations for GoldenGate 3

4 Oracle GoldenGate 11g Release2 Integrated Extract Conflict Detection and Resolution Manageability and Monitoring Security and Performance Globalization Event Marker Enhancement Coherence Integration 4

5 Integrated Extract Leverages Proprietary Oracle Database LCR API Source database and Integrated Extract process are in the same machine User/Application changes Record changes Redo Logs Oracle Database Capture changes LCR1 LCR2 LCR3... Logical Change Records Capture Integrated Extract Trail File.. Source Machine 5

6 Integrated Extract Benefits - Summary Category New Features Comments Exadata Capture support for EHCC compression Enable capture from Exadata sources Compressions Capture support for OLTP, Segment compression Distributed Transactions XA-RAC, PDML Support capture from XA transactions originated from multiple nodes of RAC RAC Simplified RAC management No need to modify parameter files, if an instance is not available New Datatypes LOB XML OR, XML Binary Full and Partial read from REDO log REDO Processing Multithreaded support Performance gain from parallel processing Deployment Source and Downstream capture support Helps to offload some of the extract work from the source database; DDL Others Captures tables created with column level password specification Support IOT with MAPPING table option 6

7 Integrated Extract Expanded Deployment Options Source database and Integrated Extract process are in different machine User/Application changes Record changes Oracle Database Source Machine Redo Logs Continuous REDO Shipping Capture Capture changes LCR1 LCR2 LCR3... Standby Logical Change Redo Logs Records Oracle Database Downstream Machine Capture Integrated Extract Trail File.. 7

8 Conflict Detection and Resolution Conflicts occur when the timing of simultaneous changes results in one of these out-of-sync conditions: a replicated insert attempts to add a row that already exists in the target. the before image of a replicated update does not match the current row in the target. a replicated delete attempts to remove a row that does not exist in the target. 8

9 Conflict Detection and Resolution Insert INSERTROWEXISTS Overwrite Ignore Discard USEMIN,USEMAX Update UPDATEROWMISSING UPDATEROWEXISTS Overwrite Ignore Discard Overwrite Ignore Discard USEMIN,USEMAX USEDELTA DELETEROWMISSING Ignore Discard Delete DELETEROWEXISTS Overwrite Ignore Discard 9

10 Performance - Streaming Prior to Oracle GoldenGate 11gR2 Synchronous data communication between data pump extract and remote collector process Data sent by data pump extract to remote collector process must be followed with ACK from the remote collector. Extract can only send out new data after ACK. Oracle GoldenGate 11gR2 Acknowledgement message can be skipped for every data sent Remote collector does not send an acknowledgement to the data pump extract unless a packet contains a flag requesting a response, typically when the data pump extract must checkpoint or determine a write position. 10

11 Security Enhancements: Encryption GGSCI Interactive Interface Manager Manager GGSCI Interactive Interface LAN / WAN / INTERNET Over TCP/IP Source Database PASSWORDS IPv6 support PASSWORDS Target Database Parameter Files Parameter Files TCP/IP BLOWFISH, AES128, AES192,AES256 11

12 Globalization Database Object Name Enhancement Non US-ASCII Character Support (both DDL/DML) Support database object name with any characters including European accent, multi-byte, white space and symbols as long as database supports Database Like Object Level Case Sensitivity Support Oracle, DB2 and SQL/MX case sensitive object name Case sensitive from/to case insensitive mapping Partial Character Set Conversion Implicit Character Set Conversion Support by replicat Between two different character set CHAR/VARCHAR/CLOB columns CHAR/VARCHAR/CLOB from/to NCHAR/NVARCHAR/NCLOB columns 12

13 Event Marker System 11gR2 ACTIONS TRACE LOG CHECKPOINTBEFORE IGNORE DISCARD ROLLOVER User/Application changes REPORT ABORT CHECKPOINTAFTER FORCESTOP/STOP Event record Event record SHELL with parameters SUSPEND/RESUME Event record DDL LAN / WAN / INTERNET Over TCP/IP Source Database Transaction Logs Target Database 13

14 Coherence HotCache Coherence/GoldenGate Integration Capture changes from DB Pump the data to coherence node, using Java Toplink maps relational changes to the objects Updates the cache User will see the latest data in their cache in Real-Time after the database is updated 14

15 Oracle GoldenGate 12c Highlights Optimized for Oracle Database 12c Non-consolidated, Multitenant and Cloud-based Real-Time Replication Three part naming convention, New data types (Large VARCHAR) Integrated Delivery for the Oracle Database Leveraging high-speed, lightweight Streaming API built Exclusively for OGG to take advantage of database parallel apply server for auto dependency aware parallel apply Coordinated Apply for Non-Oracle Databases Orchestrates the High-Speed Apply Processes & Simplifies Setup and Management Improved Ease of Use Automatic Discard File, Enhanced Debugging, Schema Wildcarding, and Improved Security Expanded Heterogeneity 12c Brings Support for New Databases and Enhancements to Existing Supported Platforms Enhanced High Availability for Zero Data Loss Integration with Data Guard and FSFO for Automated & Transparent failover of Components Expanded Oracle Application and Technology Support Active/Active ATG, Low Downtime E-Business Suite Migrations and Coherence Integration 15

16 Oracle GoldenGate New Functionality New with OGG 12.x Oracle Installer Oracle database Version 12 CDB support 3 part object name support Automatic Discard Improved initial load with added data type support for remote task Improved SHOWSYNTAX Schema Wildcard Support Credential store for userid/(encrypted)password >> Alias Source time zone in trail, replicat sets its session to this Extract start at CSN ( Oracle SCN) Oracle dbms_goldengate_auth.grant-admin_priviledge Package 16 16

17 Integrated Replicat Overview Integrated Replicat for Oracle target databases only and Leverages database parallel apply servers for automatic dependency aware parallel apply Minimal changes to replicat configuration Single replicat parameter file for all tables Single Replicat, no need to or THREAD or other splitting 17 17

18 Integrated Replicat Architecture Diagram Oracle Database Replicat Lightweight Streaming API Inbound Server Receiver Preparer Coordinator Applier 1 Applier N Replicat Reads the trail file Constructs logical change records (LCRs) Transmits LCRs to Oracle Database via the Lightweight Streaming API Inbound Server (Database Apply Process) Receiver: Reads LCRs Preparer: Computes the dependencies between the transactions (primary key, unique indexes, foreign key), grouping transactions and sorting in dependency order. Coordinator: Coordinates transactions, maintains the order between applier processes. Applier: Performs changes for assigned transactions, including conflict detection and error handling. 18

19 Integrated Replicat - Dependency Aware Apply Source Database Supplemental Logging Supplemental logging of scheduling columns (PK, UI, FK) needed for dependency aware apply processing at target New default of ADD TRANDATA is to log all unique indexes and foreign key columns. If Primary Key, Unique Index, Foreign Key columns are same at both source and target, ADD TRANDATA provides requisite information. If target has different scheduling columns, then ADD TRANDATA [table] COLS format should be used to identify any additional columns For old behavior specify NOSCHEDULINGCOLS option to only log primary key ADD TRANDATA [table], NOSCHEDULINGCOLS ADD SCHEMATRANDATA [schema], NOSCHEDULINGCOLS Scheduling columns must be present in trail file Extract parameter LOGALLSUPCOLS Includes any supplementally logged column in trail file Automatically enables GETUPDATEBEFORES and NOCOMPRESSDELETES 19 19

20 Converting to Integrated Replicat Convert to integrated mode from existing nonintegrated replicat GGSCI> STOP REPLICAT repdb GGSCI> ALTER REPLICAT repdb, INTEGRATED Updates checkpoint file to indicate INTEGRATED DB checkpoint table is not maintained when using integrated replicat Consolidate the parameter files into a single file Start replicat with proper parameter file GGSCI> START REPLICAT repdb Registers replicat repdb1 database Starts and attaches to inbound server INFO REPLICAT repdb will show INTEGRATED in output 20 20

21 Coordinated Replicat Applicable to all databases Full barrier coordination across events requiring ordering DDL (Oracle only) Primary Key Update EMI SQLEXEC Ease of Use Simplify parameter file manageability (fewer param files) Simplify error recovery 21

22 Unified Parameter File (THREAD) MAP sales.acct1, TARGET sales.acct1; MAP sales.acct2, TARGET sales.acct2; MAP sales.acct1, TARGET sales.acct1, THREAD(1); MAP sales.acct2, TARGET sales.acct2, THREAD(2); MAP sales.acct,3 TARGET sales.acct3, THREAD(3); MAP sales.acct3, TARGET sales.acct3; 22

23 Unified Parameter File (THREADRANGE) MAP sales.acct, TARGET sales.acct, FILTER (1, 3, ID)); MAP sales.acct, TARGET sales.acct, FILTER (2, 3, ID)); MAP sales.acct, TARGET sales.acct, THREADRANGE(1-3, ID)); MAP sales.acct, TARGET sales.acct, FILTER (3, 3, ID)); 23

24 Key Concepts Coordinated vs. Integrated Replicat Coordinated Replicat Integrated Replicat User Specified Partitioning Automatic Scheduling Split transaction Semantics All databases Continues to employ SQL generation and execution outside the database. Does not split transactions Oracle specific with version requirements Apply is in the database server. Tight integration with Oracle database

25 Integrated Extract Overview Integrated Extract was introduced in Oracle GoldenGate 11.2 Oracle GoldenGate enhances the feature. With Oracle GoldenGate By default, all newly created extracts will be integrated extract Upgrading from old IE you can use TRANLOGOPTION _LCRCAPTUREPROTOCOL V2 _LCRCAPTUREPROTOCOL V2 provides better performance by Streamlined for improved performance (V2 as fast or faster than Classic Extract) Drastic improvement in LOB performance Reduced overhead on database Available when the mining server is BP 14 and higher or Oracle 12c 25 25

26 Integrated Extract DDL Trigger Removal Requirements: Database must be running Oracle Database 12c ( ), and later versions. DDL Trigger Removal is required for CDB. Behavior Integrated Extract running against Oracle Database 12c or will automatically run in this mode No need to run DDL setup scripts DDL trigger can remain enabled Integrated Extract running against Oracle Database 11g Release will get metadata gathered from DDL trigger Classic Extract will continue to require existence of DDL trigger 26

27 Integrated Extract and Replicat Review Oracle Database Only Support Integrated Extract Oracle GoldenGate : the source database can be 11.1 or 11.2 or above.... the MINING database has to be version newer than the source database, but must be BP 6 or higher. Same as OGG If the source database is Oracle DB 12.1, then OGG must be used. For Oracle Database 12c with CDB setup, integrated extract is required. Integrated Replicat Target database has to be Oracle Database and The source database (Extract) can be other databases but have to include required supplemental logging information for dependency calculation. Oracle GoldenGate and versions or non Oracle source databases, use 2 parameters: GETUPDATEBEFORES and NOCOMPRESSDELETES, these are required. For Oracle GoldenGate use LOGALLSUPCOLS and UPDATERECORDFORMAT COMPACT. This works for classic and integrated extract only. 27

28 Oracle GoldenGate 12c Database Support Oracle Database Oracle Database 12c Multitenant Container Database (CDB) More Datatypes: Large VARCHAR2 Other Databases MySQL 5.6 & NDB 7.x Clusters Sybase 15.7 DB2/LUW

29 Join the Data Integration Community Twitter or Facebook twitter.com/orclgoldengate facebook.com/oracledataintegration LinkedIn Oracle Data Integration Oracle GoldenGate YouTube channel youtube.com/oraclegoldengate MOS Knowledgebase Oracle Technology Network Oracle s Data Integration blog blogs.oracle.com/dataintegration OTN: GoldenGate ware/goldengate/overview/index.html Oracle Learning Library =44785:1:0::NO Oracle Technology Network Oracle s Public Help Forum spa?forumid=860&start=0 Oracle.com/goto/dataintegration 29

30 30