High Availability and Scalability for Online Applications with MySQL

Transcription

1 High Availability and Scalability for Online Applications with MySQL Part 1I - Advanced Replication Ivan Zoratti Sales Engineering Manager EMEA ivan@mysql.com April 2007

2 Agenda Welcome back! and Welcome to the new attendees! Introduction to MySQL - Usual 2 slides! Understanding MySQL Replication Hands-on Replication Tips and Tricks Replication Examples Q & A 2

3 High Availability and Scalability for Online Applications with MySQL 27 th March - High Availability and Scalability Architectures architectures php th April - Advanced Scalability Solutions Hands on Replication - Single, Multiple, Circular, Multimaster 3. 2 nd May - MySQL Enterprise To Control Mission Critical Online Services rd May % High Availability solutions 5 solutions to explore - Hands on MySQL Cluster th June - MySQL Enterprise performance and benchmarking th June - Advanced HA solutions 3

4 Overview of MySQL AB Founded in 1995 Operations in 26 countries 10 million product installations 50,000 downloads each day Over 9 million downloads of 5.0 to date Dramatically reduces TCO of database management Bundled by hundreds of SW and HW companies Sold by partners such as HP, Dell, Novell, & others 4

5 Chosen by Successful, Modern Companies Embedded Batteries included database in software applications Network elements High Volume Web Sites Dynamic content ecommerce Gaming & entertainment Scale Out Enterprise Data Warehousing High-Volume Transactions Departmental Intranet/Extranet Scale Out 5

6 Pluggable Storage Engine Architecture 6

7 % Replication Better response time by splitting the load for processing queries between the master and slave SELECT queries may be sent to the slave to reduce the query processing load of the master Enabled for all storage engines Achieve higher availability with a master/ slave setup In the event of problems with the master, you can switch to the slave as a backup Replicate data from one MySQL server/ storage engine combination to a different MySQL Server/storage engine combination Perform backups using a slave server and continue to process updates on the master while the backup is being made on the slave Extremely simple configuration to enable very robust replication Multiple replication topologies supported Application Master Slave 7

8 Replication - How it works Data written on the master is also written into the binary log The I/O thread on the slave collects logs from the master binary log and writes a relay log on the slave The SQL thread on the slave reads teh relay log and apply the writes on the slave Slave writes are optionally added to the binary log on the slave 8

9 Replication Topologies Single Chain Circular Multiple Multi - Master Multi - Circular 9

10 Why Replication? Mirroring & High Availability Same Storage Engines Backup Different Storage Engines Data Warehouse/Data Mart Web Search/Surfing Intensive Read/Scale-out Application Integration Web Logging/Blogging Intensive Write 10

11 Replication - How to On the Master Turn on the binary log Set the server-id Create a new user with REPLICATION SLAVE privilege On the slave Set the server-id Load the backup from the master Set up the master coordinates Start Replication 11

12 Advanced Replication Circular and Multi-master apply log-slave-update to add replicated statements to the binary log Application logic, different storage engines and delayed writes may increase write performance Multi-master on a single table PKs using AUTOINCREMENT Define offsets for each server 12

13 Hands on Replication 13

14 High Availability with Replication Manual Fail-over The availability of the master must be checked by the client Timeouts are somewhat unreliable Switchover is handled at a connection level within the application or manually The master must be isolated in order to avoid inconsistencies Master Applications Replication Slave 14

15 High Availability with Replication Automatic Fail-over Third party products such as Linux-HA Heartbeat Applications refer to the VIP to access the Master server Switchover is handled at Cluster level The HA Slave server can be used as an ordinary slave during the normal operations Scale-out slaves are used for readonly operations Master Virtual IP Replication Applications HA Slave Scale-out Slave Scale-out Slave Scale-out Slave Replication 15

16 Replication in 5.1 Row Based Replication (RBR) Effective row values are sent to the slave Statement Based Replication (SBR) is still available SQL Statements are sent to the slave Mixed Based Replication (MBR) SBR is used by default, but the system switches automatically to RBR when: When a DML statement updates an NDB table When a function contains UUID() When 2 or more tables with AUTO_INCREMENT are updated When any INSERT DELAYED is executed When a call to a UDF is involved 16

17 Row Based vs. Statement Based Row Based The master logs rows. It is the safest replication to avoid inconsistencies Some DDL operations are automatically converted to SBR Write operations apply fewer locks RBR is open to multi-threading extensions Log files are generally larger compared to SBR Rollback of transactions that affect a large number of rows will be logged anyway The binary log cannot be examined for auditing Bulk writes are considered as ordinary writes on the slave, bringing to a different locking behaviour between master and slave Statement Based The master logs statements. Since a statement can affect multiple rows, SBR log is usually smaller than RBR log Logs can be used for auditing Logs can be used for point in time recovery Master and slave tables can have a different row structure Some non-deterministic operations are automatically handled RAND(), NOW(), timestamps etc. Non-deterministic writes cannot be replicated 17

18 Controlling Replication with Replication Advisors Slave Has Been Stopped Slave Has Login Accounts With Inappropriate Privileges Slave Has Stopped Replicating Slave I/O Thread Not Running Slave Not Configured As Read Only Slave SQL Thread Not Running Slave Too Far Behind Master Slave Without REPLICATION SLAVE Accounts Replication Delay Graphs MySQL Enterprise 18

19 Replication - Points to Consider Asynchronous Replication Good for geographical redundancy and write performance Less good for high availability Platform and Storage Engine independence Different storage engines can be applied on master and slaves BLACKHOLE can be used to discard unnecessary tables and save space on the slave HA considerations I/O Thread and SQL Thread improve performance and High Availability Use sync_binlog = 1 for maximum safety 19

20 Replication - Points to Consider Read performance Linear and unlimited scalability provided by multiple slaves Multilevel configurations reduce network traffic on the master INSERT DELAYED can help when the master uses InnoDB and the slaves use MyISAM (must be SBR) Write performance DELAY_KEY_WRITE and innodb_flush_log_at_trx_commit = 0 improve performance on slaves Concurrency on MyISAM tables on slaves has less impact than on masters 20

21 Replication - Points to Consider Read performance Write Linear and unlimited scalability provided by multiple Read Applications slaves Multilevel configurations reduce network traffic on the master INSERT DELAYED can help when the master uses InnoDB and the slaves use MyISAM (must be SBR) Write performance DELAY_KEY_WRITE and Slave/Master innodb_flush_log_at_trx_commit = 0 Scale-out Slave improve performance on slaves Concurrency on MyISAM tables on slaves has less impact than on masters Master Slave/Master Slave/Master Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave 20

22 Replication - Points to Consider Multimaster and Circular Replication It provides positive effects on performance only if a write delay is acceptable The Application must have the logic to identify the proper connections for reading and writing Multimaster on different tables schemas is more effective Reduces concurrency Updates on the same table from different masters may bring to unresolved conflicts Use two phase commit with FEDERATED engine in 5.1 or XA Transactions in 5.0 to guarantee integrity on multiple masters The use of the FEDERATED on an InnoDB on a slave server can produce the multi-source effect 21

23 Replication - Points to Consider Multimaster and Circular Replication It provides positive effects on performance only if a Applications write delay is acceptable The Application must have the logic to identify the proper connections for reading and writing Slave Slave Slave Multimaster on different tables schemas is more effective Reduces concurrency Updates on the same table from different masters may bring Master FEDERATED to unresolved conflicts Use two phase commit with FEDERATED engine in 5.1 or XA Transactions in 5.0 to guarantee integrity on Scale-out Slave multiple masters The use of the FEDERATED on an InnoDB on a slave server can produce the multi-source effect Master Master Master Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Write Read 21

24 Application-side Aspects Master Connections and Slave Connections Define Master Connections for write operations Use Virtual IP for automatic failover Use multiple connections for multimaster replication Define Slave Connections for read operations Load balance vs Sticky vs Round Robin Use Sticky (perhaps with HTTP load balance) if possible Preferred and alternative servers Connector/J provides Replication-ready features jdbc:mysql://[host][,failoverhost...][:port]/[database] [?propertyname1][=propertyvalue1][&propertyname2] [=propertyvalue2]... autoreconnect roundrobinloadbalance Connection.setReadOnly( false ) for masters and Connection.setReadOnly( true ) for slaves 22

25 Application-side Aspects Master Connections and Slave Connections Define Master Connections for write operations Use Virtual IP for automatic failover Use multiple connections for multimaster replication Define Slave Connections for read operations Load balance vs Sticky vs Round Robin Use Sticky (perhaps with HTTP load balance) if possible Preferred and alternative servers Connector/J provides Replication-ready features jdbc:mysql://[host][,failoverhost...][:port]/[database] [?propertyname1][=propertyvalue1][&propertyname2] [=propertyvalue2]... autoreconnect roundrobinloadbalance Connection.setReadOnly( false ) for masters and Connection.setReadOnly( true ) for slaves 22

26 Hardware Considerations Use twin servers for HA Same HW, same storage engines Use the mirror machine as slave for read during normal operations Apply rolling upgrades of schemas, applications and base software on the mirrored slave Do not apply failback whenever possible Similar slave servers can help with balancing Storage Use RAID 10 for Masters and RAID 0 for Slaves 15k SCSI disks have a significant impact on performance Place data and logs (binary and relay logs) on different drives 23

27 And when things get really tough... Apply Application partitioning Partition Masters and Slaves accordingly Build queries handling multiple connections to groups of masters and servers 24

28 Replication - Heavy Web Site Example Web/Application Servers Write Read Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Scale-out Slave Virtual IP Master HA Slave Virtual IP Master HA Slave Virtual IP Master HA Slave 25

29 *Ideas* for the Future Multi-source Replication Conflict Resolution Synchronous Replication Multithreaded Replication 26

30 Yahoo Application Personalized content and search options. Chatrooms, free , clubs, and pager - #1 in Alexa More than 50 MySQL Servers LAMP stack MySQL Key Points MySQL used for many services (more than 200 applications), such as Yahoo! Finance "Our needs vary considerably, and that is one reason that MySQL works well with us. It handles the high-volume, 260 million record tables just as well as the small, low-volume ones. One of our servers handled over a quarter of a billion queries in a month-and-ahalf, and it still has capacity to spare." Jeremy Zawodny Database Expert Yahoo! Finance 27

31 Wikipedia Application An online collaborative encyclopaedia - #11 in Alexa LAMP stack RAID 0 disks 12 DB servers Language based server splitting (application partioning) Performance 3,000 pageviews per second 8,000+ http requests per second 25,000+ SQL requests per second MySQL Key Points 90% of DB Server Memory is InnoDB buffer pool General availability and scalability using replication 28

32 Flickr Application Picture galleries available with social networking, chat, groups, and photo ratings LAMP stack PHP, Java and XML w/xpath InnoDB Storage Engine Language based server splitting (application partioning) Performance 1,000 pageviews per second 25,000+ SQL requests per second MySQL Key Points Great session manager, to handle 100% stateless PHP application on client side Replication and vertical partitioning Fulltext indexing within MySQL Partial replication on multiple servers 29

33 Mixi.jp Problems with level 2 member tables NODE 1 member tables Fortunately, on mixi, Application Largest & fastest growing social networking site in the Japan.majority are small data sets More than 3M users, with 15k new users every day. NODE 2 member tables NODE 3 Ranked 30th on Alexa worldwide them all by Cache LAMP stack application using distributed memory caching Key Business Benefit community tables NODE 1 community tables NODE 2 Too many connections to different DBs By using more than 100 servers (with 10 new We rarely hit the DB servers added every month) and horizontal partitioning strategy, combined with replication, Average page mixi.jp can provide 2bnload pageviews/day with a page load < 0.02sec time is about 0.02 sec* * depending on data sets average load time may vary 30

34 More Resources Available Consulting MySQL Architecture & Design MySQL Performance tuning MySQL HA Jumpstart Training MySQL 5.0 Performance Tuning MySQL 5.0 for DBAs MySQL Cluster for High Availability White Papers A Guide to Database HA What s new in Cluster and many more 31

35 Special Offers! Valid until 30th June 2007 Purchase any performance tuning consulting offering and attend a 4 day performance tuning class for only 795 EUR (1,595 EUR value)! Register for one of the following classes : MySQL 5.0 for Developers MySQL 5.0 for DBAs And win an ipod shuffle! 32

36 High Availability and Scalability for Online applications with MySQL Thank You! Don t Miss Part III: MySQL Enterprise in Mission Critical Applications Wednesday, 2nd May Ivan Zoratti Sales Engineering Manager EMEA ivan@mysql.com 33