SOUG-SIG Data Replication With Oracle GoldenGate Looking Behind The Scenes Robert Bialek Principal Consultant Partner BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
Who Am I Principal Consultant, Partner and Trainer at Trivadis GmbH in Munich robert.bialek@trivadis.com Focus: Oracle Database High Availability Database Architecture/Internals Backup/Recovery Troubleshooting/Performance Tuning Linux Administration Trainer for the following Trivadis courses Oracle Grid Infrastructure (O-GRINF) Oracle Real Application Cluster (O-RAC) Oracle Data Guard (O-DG) 2 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Our company. Trivadis is a market leader in IT consulting, system integration, solution engineering and the provision of IT services focusing on and technologies in Switzerland, Germany, Austria and Denmark. We offer our services in the following strategic business fields: O P E R A T I O N Trivadis Services takes over the interactive operation of your IT systems. 3 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
With over 600 specialists and IT experts in your region. COPENHAGEN 14 Trivadis branches and more than 600 employees HAMBURG 200 Service Level Agreements Over 4,000 training participants Research and development budget: CHF 5.0 / EUR 4 million DÜSSELDORF FRANKFURT Financially self-supporting and sustainably profitable Experience from more than 1,900 projects per year at over 800 customers BASEL STUTTGART FREIBURG BRUGG ZURICH MUNICH VIENNA BERN LAUSANNE GENEVA 4 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Technology on its own won't help you. You need to know how to use it properly. 5 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Agenda 1. Introduction 2. Oracle GoldenGate Architecture 3. Oracle GoldenGate Topologies 4. Demo Integrated Extract/Replicat 5. Core Messages 6 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Introduction 7 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
What is Oracle GoldenGate? Oracle strategic Middleware software product used for data extraction and replication Use-cases: data/service high-availabillity, zero-downtime upgrades/migrations, easy fallback (minimize risk) Moves only committed transactions across TCP/IP network without Oracle Net Services In contrast to Oracle Streams Supports asynchronous data transfer (DML, DDL) heterogeneous sources and targets (databases, operating systems) different extraction, propagation and apply methods different topologies, mappings, transformations, filtering, error handling, 8 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Why Oracle GoldenGate? Oracle Streams is deprecated in the Oracle 12c version (but still supported!) https://docs.oracle.com/database/121/upgrd/deprecated.htm According to the Oracle GoldenGate Statement of Direction http://www.oracle.com/technetwork/middleware/goldengate/overview/statement-ofdirection-gg-132120.pdf Oracle Streams continues to be supported but will not be enhanced to support any new database features in future releases. This means that Streams will continue to support exactly what it supports today (11.2) in future releases. It also means that any new database features, data types, or capabilities will not be supported by Streams. Beginning with the version 12.1.0.2, the non-cdb architecture is also deprecated As of now, Streams is not supported with the new CDB architecture 9 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Architecture 10 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Optional Oracle GoldenGate Architecture: Big Picture Oracle GoldenGate Instance Oracle GoldenGate Instance Source DB Optional Primary (Passive) Extract Manager (VIP) MONITOR Manager (VIP) MONITOR Optional Primary (Alias) Extract Target DB Primary (Regular) Extract Collector Replicat Local Trail Secondary Extract Data Pump Remote Trail Static Configuration Runtime Configuration Runtime Configuration Static Configuration 11 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Primary Extract Group: Capture Source DB Primary Extract can run in either Classic Capture mode (all database versions) Integrated Capture mode (full support: 11.2.0.3+) Primary (Regular) Extract Primary (Passive) Extract Extracts and optionally filters/transforms the transactions from the source database Only for configured objects Many Extracts per source database can operate at the same time Only committed transactions (CSN) will be propagated to the target system Optionally written to a local trail (recommended) Without a local trail, the primary Extract send the data to the target system Not committed transactions will be discarded from memory or dirtmp directory 12 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Primary Extract Group: Classic Capture Classic Capture mode Traditional mode, reads directly Oracle database redo logs Able to mine redo logs in ASM (two methods) Redo Logs All database versions supported Source DB With some restrictions. For example no support for: BASIC, OLTP, Exadata Hybrid Columnar compression PDML on RAC database CDB architecture! DDL, TDE support no out-of-the box support Primary Extract OS System Calls Local Trail Secondary Extract Data Pump Not transparent to RAC configuration changes (#threads) 13 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Primary Extract Group: Integrated Capture Integrated Capture mode Interacts with the database logmining server Required to capture data from a CDB architecture Fully transparent with RAC, TDE, ASM Full support for databases 11.2.0.3+ Source database 11.2.0.4+ does not require supplementary objects for DDL support Supports more complex data types than classic capture Integrated capture deployments Local or a downstream deployment The best choice for most deployments Logmaining Server LCR LCR Primary Extract Local Trail Redo Logs Secondary Extract Data Pump Source DB 14 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Secondary Extract Group (Data Pump): Propagation Optional but recommended component Reads the local trail and propagates the data to the remote trail (via target collector process) Data can be transferred as is (PASSTHROUGH) Local Trail Encryption Secondary Extract Data Pump Optionally filters/transforms the transactions from the source database (NOPASSTHROUGH) Isolates the primary extract from TCP/IP timeouts The network transfer can be optionally encrypted Not necessary, in case the trail is already encrypted 15 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Replicat Process: Apply The Replicat process can run in either Classic Replicat mode (nonintegrated) - all database versions Coordinated Replicat mode (nonintegrated) - all database versions Integrated Replicat mode database versions 11.2.0.4+ Target DB Replicat The Replicat process reads the remote trail, reconstructs the SQL and applies them to the target database Optionally filters/transforms the data Remote Trail The application of the data can be optionally delayed (DEFERAPPLYINTERVAL parameter) 16 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Replicat Group: Classic/Coordinated Mode Classic Replicat Constructs SQL statements from the trails and applies the SQL through OCI Supported for all database versions Transactions are applied sequentially (single-threaded) Replicat SQL OCI Remote Trail Target DB Coordinated Replicat Transactions are applied in parallel (multi-threaded) There is 1 coordinator plus many apply threads Supported for all database versions GoldenGate 12c required Remote Trail Coordinator Thread 1 Thread 2 SQL OCI Target DB Thread n 17 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Replicat Group: Integrated Mode Integrated Replicat Supported for target database versions 11.2.0.4+ Replicat LCR Database Inbound Server Target DB Constructs LCRs and transmits DMLs to the database inbound server Remote Trail SQL Some changes (e.g. DDL) are not supported directly by the inbound server In those cases the Replicat process disconnects from the inbound server and executes the transaction in direct apply mode through OCI With many transactions applied in direct apply mode it might be necessary to switch to nonintegrated Replicat 18 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Capture/Apply: CDB & Pluggable Databases Extract must operate in integrated mode Extract must connect to the CDB$ROOT container as a common user (c##) One Extract can capture from one or many PDBs to a single trail common_user_prefix (12.1.0.2) Replicat can connect and apply only to one PDB (one Apply/PDB, connect using service) Source DB CDB CDB$ROOT PDB1 PDB2 Target DB CDB CDB$ROOT PDB1 PDB2 Primary Extracts INTEGRATED Data Pumps Collectors Replicats 19 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Capture & Apply: Different Combinations Possible OS System Calls Redo Logs Logmaining Server Source DB Replicat(s) LCR Integr ated Database Inbound Server Target DB Classic Primary Extract(s) LCR Integr ated LCR SQL OCI Classic Coordi nated Different combinations are possible Classic Extract with integrated or classic/coordinated Replicat Integrated Extract with integrated or classic/coordinated Replicat You can switch between the modes with some configuration changes 20 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Topologies 21 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Topologies Active-Passive Data replication only in one direction Only one set of primary Extract, Data Pump and Replicat groups Active/Standby Additional inactive primary Extract, Data Pump and Replicat groups Switchover: Stop one set of process Start the inactive one 22 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate Topologies Active-Active Data replication in both directions Two active sets of primary Extract, Data Pump and Replicat groups With some restrictions: e.g.: DDL-support restricted to two databases with identical metadata Additionally broadcasting, cascading, consolidation is possible 23 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Demo Integrated Extract/Replicat 24 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate: Setup Flow 1 2 3 Architecture Software Installation/ Design Patches Database Preparation 6 5 Primary Extract, Data 4 Register/Start Primary Pump, Replicat Extract, Data Pump Configuration Files Manager Configuration 7 8 9 Target Database Register/Start Replicat Instantiation Cluster Integration / INIT Scripts n Tuning/Customizing/ Maintenance/ 25 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Oracle GoldenGate: Demo Environment Source DB: 2-nodes RAC 11.2.0.4 Schema: APP1 GoldenGate version: 12.1.2.1.2 Active/Passive data replication Target DB: CDB 2-nodes RAC 12.1.0.2 PDB: APP1, Schema: APP1 GoldenGate version: 12.1.2.1.2 Active/Passive data replication CDB$ROOT APP1 APP2 APP3 Schema: APP1 Primary Extract INTEGRATED Local Trail RemoteTrail Replicat INTEGRATED Secondary Extract Data Pump Collector 26 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Core Messages 27 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Core Messages GoldenGate is a very powerful replication software framework For most deployments the integrated Extract/Replicat is the best choice Supplemental logging generates more undo and redo In some cases you need even to supplementally log all columns! Script all the configuration steps Very useful if you need to re-setup the whole environment You don t need to replace your current Streams installation ASAP by Oracle GoldenGate, but Streams is deprecated in the version 12c and will not be enhanced in future releases There is a framework to ease the migration from Streams to GoldenGate (streams2ogg) As of now Streams is not supported with CDB architecture For new projects it might be a better choice (licensing)! 28 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes
Questions and answers Robert Bialek Principal Consultant Partner Tel. +49 89 99275930 robert.bialek@trivadis.com 29 11.06.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes