SOUG-SIG Data Replication With Oracle GoldenGate Looking Behind The Scenes Robert Bialek Principal Consultant Partner

Transcription

1 SOUG-SIG Data Replication With Oracle GoldenGate Looking Behind The Scenes Robert Bialek Principal Consultant Partner BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

2 Who Am I Principal Consultant, Partner and Trainer at Trivadis GmbH in Munich robert.bialek@trivadis.com Focus: Oracle Database High Availability Database Architecture/Internals Backup/Recovery Troubleshooting/Performance Tuning Linux Administration Trainer for the following Trivadis courses Oracle Grid Infrastructure (O-GRINF) Oracle Real Application Cluster (O-RAC) Oracle Data Guard (O-DG) Data Replication With Oracle GoldenGate - Looking Behind The Scenes

3 Our company. Trivadis is a market leader in IT consulting, system integration, solution engineering and the provision of IT services focusing on and technologies in Switzerland, Germany, Austria and Denmark. We offer our services in the following strategic business fields: O P E R A T I O N Trivadis Services takes over the interactive operation of your IT systems Data Replication With Oracle GoldenGate - Looking Behind The Scenes

4 With over 600 specialists and IT experts in your region. COPENHAGEN 14 Trivadis branches and more than 600 employees HAMBURG 200 Service Level Agreements Over 4,000 training participants Research and development budget: CHF 5.0 / EUR 4 million DÜSSELDORF FRANKFURT Financially self-supporting and sustainably profitable Experience from more than 1,900 projects per year at over 800 customers BASEL STUTTGART FREIBURG BRUGG ZURICH MUNICH VIENNA BERN LAUSANNE GENEVA Data Replication With Oracle GoldenGate - Looking Behind The Scenes

5 Technology on its own won't help you. You need to know how to use it properly Data Replication With Oracle GoldenGate - Looking Behind The Scenes

6 Agenda 1. Introduction 2. Oracle GoldenGate Architecture 3. Oracle GoldenGate Topologies 4. Demo Integrated Extract/Replicat 5. Core Messages Data Replication With Oracle GoldenGate - Looking Behind The Scenes

7 Introduction Data Replication With Oracle GoldenGate - Looking Behind The Scenes

8 What is Oracle GoldenGate? Oracle strategic Middleware software product used for data extraction and replication Use-cases: data/service high-availabillity, zero-downtime upgrades/migrations, easy fallback (minimize risk) Moves only committed transactions across TCP/IP network without Oracle Net Services In contrast to Oracle Streams Supports asynchronous data transfer (DML, DDL) heterogeneous sources and targets (databases, operating systems) different extraction, propagation and apply methods different topologies, mappings, transformations, filtering, error handling, Data Replication With Oracle GoldenGate - Looking Behind The Scenes

9 Why Oracle GoldenGate? Oracle Streams is deprecated in the Oracle 12c version (but still supported!) According to the Oracle GoldenGate Statement of Direction Oracle Streams continues to be supported but will not be enhanced to support any new database features in future releases. This means that Streams will continue to support exactly what it supports today (11.2) in future releases. It also means that any new database features, data types, or capabilities will not be supported by Streams. Beginning with the version , the non-cdb architecture is also deprecated As of now, Streams is not supported with the new CDB architecture Data Replication With Oracle GoldenGate - Looking Behind The Scenes

10 Oracle GoldenGate Architecture Data Replication With Oracle GoldenGate - Looking Behind The Scenes

11 Optional Oracle GoldenGate Architecture: Big Picture Oracle GoldenGate Instance Oracle GoldenGate Instance Source DB Optional Primary (Passive) Extract Manager (VIP) MONITOR Manager (VIP) MONITOR Optional Primary (Alias) Extract Target DB Primary (Regular) Extract Collector Replicat Local Trail Secondary Extract Data Pump Remote Trail Static Configuration Runtime Configuration Runtime Configuration Static Configuration Data Replication With Oracle GoldenGate - Looking Behind The Scenes

12 Primary Extract Group: Capture Source DB Primary Extract can run in either Classic Capture mode (all database versions) Integrated Capture mode (full support: ) Primary (Regular) Extract Primary (Passive) Extract Extracts and optionally filters/transforms the transactions from the source database Only for configured objects Many Extracts per source database can operate at the same time Only committed transactions (CSN) will be propagated to the target system Optionally written to a local trail (recommended) Without a local trail, the primary Extract send the data to the target system Not committed transactions will be discarded from memory or dirtmp directory Data Replication With Oracle GoldenGate - Looking Behind The Scenes

13 Primary Extract Group: Classic Capture Classic Capture mode Traditional mode, reads directly Oracle database redo logs Able to mine redo logs in ASM (two methods) Redo Logs All database versions supported Source DB With some restrictions. For example no support for: BASIC, OLTP, Exadata Hybrid Columnar compression PDML on RAC database CDB architecture! DDL, TDE support no out-of-the box support Primary Extract OS System Calls Local Trail Secondary Extract Data Pump Not transparent to RAC configuration changes (#threads) Data Replication With Oracle GoldenGate - Looking Behind The Scenes

14 Primary Extract Group: Integrated Capture Integrated Capture mode Interacts with the database logmining server Required to capture data from a CDB architecture Fully transparent with RAC, TDE, ASM Full support for databases Source database does not require supplementary objects for DDL support Supports more complex data types than classic capture Integrated capture deployments Local or a downstream deployment The best choice for most deployments Logmaining Server LCR LCR Primary Extract Local Trail Redo Logs Secondary Extract Data Pump Source DB Data Replication With Oracle GoldenGate - Looking Behind The Scenes

15 Secondary Extract Group (Data Pump): Propagation Optional but recommended component Reads the local trail and propagates the data to the remote trail (via target collector process) Data can be transferred as is (PASSTHROUGH) Local Trail Encryption Secondary Extract Data Pump Optionally filters/transforms the transactions from the source database (NOPASSTHROUGH) Isolates the primary extract from TCP/IP timeouts The network transfer can be optionally encrypted Not necessary, in case the trail is already encrypted Data Replication With Oracle GoldenGate - Looking Behind The Scenes

16 Replicat Process: Apply The Replicat process can run in either Classic Replicat mode (nonintegrated) - all database versions Coordinated Replicat mode (nonintegrated) - all database versions Integrated Replicat mode database versions Target DB Replicat The Replicat process reads the remote trail, reconstructs the SQL and applies them to the target database Optionally filters/transforms the data Remote Trail The application of the data can be optionally delayed (DEFERAPPLYINTERVAL parameter) Data Replication With Oracle GoldenGate - Looking Behind The Scenes

17 Replicat Group: Classic/Coordinated Mode Classic Replicat Constructs SQL statements from the trails and applies the SQL through OCI Supported for all database versions Transactions are applied sequentially (single-threaded) Replicat SQL OCI Remote Trail Target DB Coordinated Replicat Transactions are applied in parallel (multi-threaded) There is 1 coordinator plus many apply threads Supported for all database versions GoldenGate 12c required Remote Trail Coordinator Thread 1 Thread 2 SQL OCI Target DB Thread n Data Replication With Oracle GoldenGate - Looking Behind The Scenes

18 Replicat Group: Integrated Mode Integrated Replicat Supported for target database versions Replicat LCR Database Inbound Server Target DB Constructs LCRs and transmits DMLs to the database inbound server Remote Trail SQL Some changes (e.g. DDL) are not supported directly by the inbound server In those cases the Replicat process disconnects from the inbound server and executes the transaction in direct apply mode through OCI With many transactions applied in direct apply mode it might be necessary to switch to nonintegrated Replicat Data Replication With Oracle GoldenGate - Looking Behind The Scenes

19 Capture/Apply: CDB & Pluggable Databases Extract must operate in integrated mode Extract must connect to the CDB$ROOT container as a common user (c##) One Extract can capture from one or many PDBs to a single trail common_user_prefix ( ) Replicat can connect and apply only to one PDB (one Apply/PDB, connect using service) Source DB CDB CDB$ROOT PDB1 PDB2 Target DB CDB CDB$ROOT PDB1 PDB2 Primary Extracts INTEGRATED Data Pumps Collectors Replicats Data Replication With Oracle GoldenGate - Looking Behind The Scenes

20 Capture & Apply: Different Combinations Possible OS System Calls Redo Logs Logmaining Server Source DB Replicat(s) LCR Integr ated Database Inbound Server Target DB Classic Primary Extract(s) LCR Integr ated LCR SQL OCI Classic Coordi nated Different combinations are possible Classic Extract with integrated or classic/coordinated Replicat Integrated Extract with integrated or classic/coordinated Replicat You can switch between the modes with some configuration changes Data Replication With Oracle GoldenGate - Looking Behind The Scenes

21 Oracle GoldenGate Topologies Data Replication With Oracle GoldenGate - Looking Behind The Scenes

22 Oracle GoldenGate Topologies Active-Passive Data replication only in one direction Only one set of primary Extract, Data Pump and Replicat groups Active/Standby Additional inactive primary Extract, Data Pump and Replicat groups Switchover: Stop one set of process Start the inactive one Data Replication With Oracle GoldenGate - Looking Behind The Scenes

23 Oracle GoldenGate Topologies Active-Active Data replication in both directions Two active sets of primary Extract, Data Pump and Replicat groups With some restrictions: e.g.: DDL-support restricted to two databases with identical metadata Additionally broadcasting, cascading, consolidation is possible Data Replication With Oracle GoldenGate - Looking Behind The Scenes

24 Demo Integrated Extract/Replicat Data Replication With Oracle GoldenGate - Looking Behind The Scenes

25 Oracle GoldenGate: Setup Flow Architecture Software Installation/ Design Patches Database Preparation 6 5 Primary Extract, Data 4 Register/Start Primary Pump, Replicat Extract, Data Pump Configuration Files Manager Configuration Target Database Register/Start Replicat Instantiation Cluster Integration / INIT Scripts n Tuning/Customizing/ Maintenance/ Data Replication With Oracle GoldenGate - Looking Behind The Scenes

26 Oracle GoldenGate: Demo Environment Source DB: 2-nodes RAC Schema: APP1 GoldenGate version: Active/Passive data replication Target DB: CDB 2-nodes RAC PDB: APP1, Schema: APP1 GoldenGate version: Active/Passive data replication CDB$ROOT APP1 APP2 APP3 Schema: APP1 Primary Extract INTEGRATED Local Trail RemoteTrail Replicat INTEGRATED Secondary Extract Data Pump Collector Data Replication With Oracle GoldenGate - Looking Behind The Scenes

27 Core Messages Data Replication With Oracle GoldenGate - Looking Behind The Scenes

28 Core Messages GoldenGate is a very powerful replication software framework For most deployments the integrated Extract/Replicat is the best choice Supplemental logging generates more undo and redo In some cases you need even to supplementally log all columns! Script all the configuration steps Very useful if you need to re-setup the whole environment You don t need to replace your current Streams installation ASAP by Oracle GoldenGate, but Streams is deprecated in the version 12c and will not be enhanced in future releases There is a framework to ease the migration from Streams to GoldenGate (streams2ogg) As of now Streams is not supported with CDB architecture For new projects it might be a better choice (licensing)! Data Replication With Oracle GoldenGate - Looking Behind The Scenes

29 Questions and answers Robert Bialek Principal Consultant Partner Tel Data Replication With Oracle GoldenGate - Looking Behind The Scenes