MySQL High-Availability and Scale-Out architectures

Size: px

Start display at page:

Download "MySQL High-Availability and Scale-Out architectures"

Maud Goodman
10 years ago
Views:

1 MySQL High-Availability and Scale-Out architectures Oli Sennhauser Senior Consultant 1

2 Introduction Who we are? What we want? 2

3 Table of Contents Scale-Up vs. Scale-Out MySQL Replication MySQL HA solution MySQL Cluster Application architectural solutions Other architectural stuff It's your turn... Let us build a replication 3

4 When do we think about architecture? Performance problems HA requirements 4

5 Scale-up vs. Scale-Out Scale-up Scale-out t Relaxation of constraints 5

6 MySQL Replication The MySQL answer: Master-Slave-Replication: Application r t w 2 types of sessions!!! Master Slave 1 Slave 2 Slave 3... LB ro Even Oracle uses the expression scale-out now! 6

7 MySQL Replication in detail Appl r t w M ro asynchronous! S IO thread SQL thread... bin-log bin-log relay-log relay-log wide distance possible! thin line possible! 7

8 MySQL Replication varieties Cascaded replication: M S I S I S I... black hole SE! S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8... for example Yahoo! 8

9 MySQL Replication varieties HA with hot standby Slave: Appl VIP HA solution S backup M S standby S 1 S 2 S 3 active active fail over! 9

10 HA with MySQL: MySQL HA solution Appl VIP heartbeat M M' DRBD S 1 S 2 S % HA (four nine) active passive fail over! 10

11 MySQL HA solution HA with MySQL and SAN: Appl VIP heartbeat M M' S 1 S 2 S 3 SAN 11

12 Multi-Master Replication actually Master-Master Replication Appl Appl VIP/LB M 1 M 2 S 11 S 12 S 13 S 21 S 22 S 23 This architecture does NOT solve your write problems!!! 12

13 Multi-Master Replication HA Multi-Master Replication Appl Appl VIP VIP/LB VIP M 1 ' M 1 M 2 M 2 ' DRBD DRBD S 11 S 12 S 13 S 21 S 22 S 23 13

14 Shared disk cluster DB 1 DB 2 ro! shared disk DB 2 This is in 99.9% of the cases NOT your solution!!! It looks like Oracle RAC but MySQL does NOT (yet) have the instruments needed! Can be useful in some special kinds of reporting. 14

15 Where are we now? The architectures above solved: your read problems. some HA requirements. But they did not solve: your write problems! And what when we need: higher HA? synchronous replication? more write speed? 15

16 MySQL Cluster Shared-nothing architecture Synchronous replication (2-Phase commit) Fast automatic fail over High performance (also writing) High transactional throughput No special components required In-Memory database (in 5.1 also disk support) Scalable, 1000's of transactions per second % HA (five nine) On-line upgrade path (at least on GA within same version) 16

components required In-Memory database (in 5.

17 MySQL Cluster architecture Application Application Application Application SQL Nodes MySQL Server MySQL Server MySQL Server my.cnf NDB Cluster (Data nodes) config.ini Data Nodes DB DB MGM Server DB DB MGM client Mgmt Nodes 17

18 MySQL Cluster HA features MySQL Server X MySQL Server X MySQL Server X MySQL Server X MySQL Server NDB API Data Node NDB Storage Engine Data Node Management Server X Data Node X Data Node X Management Server MySQL Cluster 18

Node NDB Storage Engine Data Node Management Server X

19 Cluster with Replication for read scale-out (Reporting): Appl rw mysqld mysqld mysqld S 1 S 2 S 3 Reporting 19

20 Cluster with Replication Cluster-Cluster replication for disaster fail over (MySQL 5.0): Appl rw fail over mysqld mysqld mysqld mysqld mysqld mysqld 20

21 Cluster with Replication Cluster-Cluster replication for disaster fail over (MySQL 5.1): Appl rw fail over mysqld mysqld mysqld mysqld mysqld mysqld 21

22 Cluster with Replication circular Cluster-Cluster replication (>= MySQL ?): 22

23 Session handling Telecom (Mobile) VoIP RSS-Feed aggregation Mail On-line Games Cluster examples Use cluster where you need HA you have high write load you do little Joins and Grouping 23

24 How to go on? Read is a caching problem! More RAM or scale-out. Write is a batching problem! Batch your load, Buy stronger I/O system. Use MySQL Cluster. SSD! What then? 24

25 SSD disruption I/O system without any movable parts! SSD = Solid State disk (Flash memory, NAND, NOR chips, RAM-SAN). During the last year(s) Price from 15'000 USD / 160 Gbyte -> 100 USD / 32 Gbyte times faster than mechanical I/O systems 1 Mio write cycles dead :-( (special FS!) RAM: Huge amount of memory is cheap! This will disrupt the database world! 25

26 What then? With or without SSD we will reach a physical limit. What then? Application partitioning: Split applications OLTP vs. OLAP all in one Segment your application 26

27 Application partitioning OLTP vs. OLAP hot business data (trx) vs. old reporting data All in one Sessions, user tracking, ads, chat, booking Segment split by for example 1 Mio users (split by user_id). 27

28 Architecture examples 28

29 Architecture examples 29

30 Architecture examples 30

31 Architecture examples 31

32 Some more architecture stuff? Databases are slow!!! Whenever possible try to avoid databases! Use memcached for example. The SP trap. Stored Procedures are a lock in! Try to avoid SP! Use Materialized Views (MV) and/or shadow tables. VM/SAN is nice for consolidation but not for performance/scale-out! Backup and Staging MySQL Proxy 32

33 Your problems? Now it's your turn... Let us build a replication... 33

Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL

Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL 04.20111 High Availability with MySQL Higher Availability Shared nothing distributed cluster with MySQL Cluster Storage snapshots for disaster