Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL

Transcription

1 Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL

2 High Availability with MySQL Higher Availability Shared nothing distributed cluster with MySQL Cluster Storage snapshots for disaster recoverygeographical Replication for disaster recovery Virtualised Environments Active/Passive Clusters + Local Asynchronous ReplicationActive/Passive Clusters through shared storage DRBD + Local Asynchronous Replication Synchronous Replication through DRBDLocal Semi-synchronous ReplicationLocal Asynchronous Replication 2

3 Local Asynchronous Replication Data written on the master is written into the binary log The I/O thread on the slave collects logs from the master binary log and writes a relay log on the slave The SQL thread on the slave reads the relay log and apply the writes on the slave Slave writes are optionally added to the binary log on the slave It can be statement-based (SBR), row-based (RBR) or mixed (MBR) In case of fault: Read-Write The master server is taken down The slave server is updated up to the last position in the relay log The clients point at the Read-Only Read-Only designated slave server binlog The designated slave server becomes the master server 99 3 relaylog relaylog relaylog relaylog

4 Local Asynchronous Replication 1. A write transaction is sent to the Master INSERT INTO binlog relaylog relaylog relaylog relaylog 4

5 Local Asynchronous Replication 2. The transaction is stored into the binlog INSERT INTO binlog relaylog relaylog relaylog relaylog 5

6 Local Asynchronous Replication 3. The client receives the ACK from the server TXN OK! binlog relaylog relaylog relaylog relaylog 6

7 Local Asynchronous Replication 4. The I/O thread on the slave(s) pulls the transaction and it stores it in the relaylog binlog INSERT INTO INSERT INTO INSERT INTO INSERT INTO relaylog relaylog relaylog relaylog 7

8 Local Asynchronous Replication 5. The SQL thread reads the transaction from the relay log and applies the transaction on the slave(s) binlog INSERT INTO INSERT INTO INSERT INTO INSERT INTO relaylog relaylog relaylog relaylog 8

9 Local Asynchronous Replication Typical Usage High availability with manual failover Easy to implement and maintain, provides a good level of availability Non-intrusive backup Backups can be executed on a slave server The binlog and provides incremental backup and possible point-in-time recovery Scalability for read-intensive applications Read-only transactions can be executed on the slave servers Systems and applications upgrades Chains of Master/Slaves reduce to virtually 0 the downtime for upgrades 9

10 Local Asynchronous Replication Other Features Out-of-the box with MySQL Server Platform independent Master and Slaves can be heterogeneous in OS and version Storage engine independent Master and Slaves can be heterogeneous in storage engines Multiple topologies Works with transactional and non-transactional engines Advanced settings can define a selection of data to replicate 10

11 Local Asynchronous Replication Other Features Out-of-the box with MySQL Server Platform independent Master and Slaves can be heterogeneous in OS and version Storage engine independent Master and Slaves can be heterogeneous in storage engines Multiple topologies Works with transactional and non-transactional engines Advanced settings can define a selection of data to replicate 11

12 Local Semi-Synchronous Replication One or more slaves are defined as semi-synchronous servers the Master server waits until the I/O thread on one of the semi-sync slave(s) has flushed the transaction to disk, or until it receives a timeout, then it returns the ACK to the application. In case of fault: The master server is taken down The slave server is updated up to the last position in the relaylog (the update will be fast) The clients point at the designated slave server The designated slave server becomes the master server binlog Read-Write Read-Only Read-Only 12 relaylog relaylog relaylog relaylog

13 Local Semi-Synchronous Replication 1. A write transaction is sent to the Master INSERT INTO binlog relaylog relaylog relaylog relaylog 13

14 Local Asynchronous Replication 2. The transaction is stored into the binlog INSERT INTO binlog relaylog relaylog relaylog relaylog 14

15 Local Asynchronous Replication 3. The I/O thread on the slave(s) pulls the transaction and it stores it in the relaylog binlog INSERT INTO INSERT INTO INSERT INTO INSERT INTO relaylog relaylog relaylog relaylog 15

16 Local Asynchronous Replication 4. The client receives the ACK from the server TXN OK! binlog relaylog relaylog relaylog relaylog 16

17 Local Asynchronous Replication 5. The SQL thread reads the transaction from the relay log and applies the transaction on the slave(s) binlog INSERT INTO INSERT INTO INSERT INTO INSERT INTO relaylog relaylog relaylog relaylog 17

18 Synchronous Replication through DRBD One server is designated as active and another server as passive or stand-by The active server owns a virtual IP address. Applications refer to the virtual IP address to connect to the database server. Data is synchronously replicated at block level from the active to the stand-by server In case of fault: The master server is taken down The block device on the stand-by server is mounted The mysqld on the stand-by server starts The virtual IP address moves to the stand-by server The stand-by server becomes active Block Device Active/Hot Server Block Device Passive/Std-by Server 18

19 Synchronous Replication through DRBD 1. A write transaction is sent to the active Server INSERT INTO Active/Hot Server Passive/Std-by Server Block Device Block Device 19

20 Synchronous Replication through DRBD 2. The transaction is written to the block device Active/Hot Server Passive/Std-by Server INSERT INTO Block Device Block Device 20

21 Synchronous Replication through DRBD 3. The blocks modified by the transaction are replicated to the stand-by server Active/Hot Server Passive/Std-by Server Block Device Block Device 21

22 Synchronous Replication through DRBD 4. The client receives the ACK from the server TXN OK! Active/Hot Server Passive/Std-by Server Block Device Block Device 22

23 Synchronous Replication through DRBD Typical Usage High availability with automatic failover Fully redundant solution with no loss of data and no single points of failure Point-in-time snapshots One or more servers can be updated at regular intervals to provide historical analysis 23

24 Synchronous Replication through DRBD Other Features Fully certified, Linux-only environment Works with Pacemaker and Heartbeat Works with InnoDB only Inexpensive solution 24

25 DRBD + MySQL Replication The combination of DRBD and MySQL Replication provides more availability for scheduled and unscheduled downtime In case of fault of the master server, DRBD switches over to the stand-by server During the switchover, clients can still read from the slaves For planned downtime and upgrades, DBAs can apply rolling upgrades Block Device binlog relaylog relaylog relaylog relaylog 25

26 Active/Passive Clusters through Shared Storage Similar to DRBD, but data is not replicated through the network. Redundancy and replication must be guaranteed by the shared storage Shared storage is usually Storage Area Networks (SAN) or Network Attached Storage (NAS) InnoDB only, available with Linux, Windows or Solaris Used with FS like GFS or OCFS2 In case of fault: The master server is taken down The mysqld on the stand-by server starts The virtual IP address moves to the stand-by server The stand-by server becomes active Active/Hot Server Shared Storage Passive/Std-by Server 26

27 Active/Passive Clusters through Shared Storage Large Deployments VIP0 1VIP0 5 VIP0 VIP0 VIP0 2VIP0 3VIP0 4VIP in01 in02 in03 in04 in05 in06 in07 in Shared Storage 27

28 Active/Passive Clusters through Shared Storage Failover in Large Deployments VIP0 1VIP0 2VIP0 6 VIP0 VIP0 5 VIP0 3VIP0 4VIP0 7 8 in02 in03 in04 in06 in07 in08 in01 in Shared Storage 28

29 Active/Passive Clusters plus Replication The combination of active/passive cluster + MySQL Replication provides more availability for scheduled and unscheduled downtime In case of fault, the master, active server switches over to the stand-by server Slaves will pull data from the new slave Replication can also be used for planned downtime and rolling upgrades Replicated data may be stored on shared storage Passive/Std-by Server Active/Hot Server Shared Storage relaylog relaylog relaylog relaylog 29

30 Active/Passive Clusters plus Replication through Shared Storage Large Deployments VIP0 1VIP0 5 VIP0 VIP0 VIP0 2VIP0 3VIP0 4VIP M01 S02 S03 M04 S05 S06 M07 S08 M01 > S02, S03 M04 > S05, S Shared Storage 30

31 Virtualised Environments MySQL Servers - masters or slaves - are located on any available physical server High availability and load balancing are provided by the virtualised software Data storage is usually managed by the virtualised software but it is masked as local storage In case of fault, the virtualised software restarts on any other physical server InnoDB only Shared Storage

32 Geographical Replication for Disaster Recovery Active Data Centre Master-Master Asynchronous Replication is used to update the backup data centre In case of fault, the network traffic is redirected to the backup data centre. Failback must be executed manually Cross-platform and crossengine Backup Data Centre 32

33 Storage Snapshots for Disaster Recovery Active Data Centre Snapshots are managed by the NAS and SAN firmware. There is usually a short read-only freeze Snapshots can be used as run-time backup InnoDB only, NetApp NASs and firmware are certified using Snapshot and Snapmirror Backup Data Centre 33

34 MySQL Cluster Shared-nothing, fully transactional and distributed architecture used for high volume and small transactions. MySQL Cluster is based on the NDB (Network DataBase) Storage Engine Data is distributed for scalability and performance, and it is replicated for redundancy on multiple data nodes. Nodes in a cluster: SQL Nodes: provide the SQL interface to NDB API Nodes: provide the native NDB API Data Nodes: store and retrieve data, manage transactions Management Nodes: manage the Cluster Load balanced Memory or disk-based Geographically replicated for disaster recovery Full online operation for maintenance and administration Application Nodes NDB API, ClusterJ/JPA Data Nodes SQL Nodes Management Nodes 34

35 Thank You! Joffrey MICHAÏE Consultant MySQL 35