Running MySQL on CentOS Linux. Peter Zaitsev April 14, 2014

Transcription

1 Running MySQL on CentOS Linux Peter Zaitsev April 14, 2014

2 About the Presenta9on 2 Cover what you need to run MySQL on CentOS Linux successfully Distribu7on Hardware OS Configura7on MySQL Installa7on MySQL Configura7on

3 Why Linux? 3 Almost 90% of MySQL users use Linux for produc7on Almost 80% for Development hnp://bit.ly/qsdooc CentOS the Most Popular Linux Distribu7on for MySQL hnp://bit.ly/1rliunv Great default choice Unless you have a very good reason not to, choose Linux for MySQL

4 Reasons for CentOS 4 Server Grade Distribu7on Long support, stability Completely Open Source Free Modern but not bleeding Edge Great Community support You can get help when you need to

5 Hardware 5 Hardware can go a long way! Really! Over 200K simple queries/ sec on modern hardware With 100 queries/page = 120K page views/minute Some 80M page views/day (considering daily spike)

6 Hardware 6 CPU Go for fast cores. All cores are rarely used these days Cache and fast memory bus is important Memory Ocen most important for performance Your working set must fit in memory well Less memory = more pressure on IO

7 Storage 7 Directly anached Flash is best - PCI- E cards (FusionIO, Virident) are fastest SATA SSD are very cost effec7ve Mind vendors and monitor for uniform performance Where performance might be limi7ng (hnp://bit.ly/1f2u919) Ensure you have a RAID w BBU on conven7onal drives RAID10 is best for heavy loads

8 Know your Goals 8 Are you bound by Reads? Writes? Random? Sequen7al? Do you need Sustained writes? Handle spikes? What concurrency do you operate? Many devices reach peak IOPs at unreachable concurrency Think about latency/ response 7me SAN/NAS is a frequent cause of surprise

9 Performance vs. Memory 9

10 Compression of Performance Gains 10

11 Alignment 11 Par77on alignment File system alignment op7ons Can cause significant performance difference hnp://bit.ly.ilfnkt hnp://bit.ly.ks9trm

12 Alignment Benchmarks 12

13 More on SSD/Flash 13 Think about Write Endurance Behavior on Power Loss Data Reten7on Firmware Linux RAID Compa7bility

14 Network 14 Latency is king Minimize number of hops between Database and Web Server Ensure running 1Gbit link speed at least (10Gb is gaining popularity) Monitor for packet loss and latency Network problems are ocen blamed on database

15 Network Tuning 15 Might be needed some7mes echo 8192 > /proc/sys/net/ipv4/tcp_max_syn_backlog Mysql: back_log=1000 net.ipv4.ip_local_port_range=" net.core.somaxconn = 1024 net.core.rmem_max = net.core.wmem_max = net.ipv4.tcp_rmem = net.ipv4.tcp_wmem = net.ipv4.max_tw_buckets= net.core.netdev_max_backlog = 2500 ifconfig eth0 txqueuelen 1000 Persistent connec7ons with MySQL Thread_pool (Percona Server, MariaDB, MySQL Enterprise) Proxy

16 Virtualiza9on and Cloud 16 Virtualiza7on has cost Cloud rarely provides highest performance hardware (Gerng bener) There is a lot of MySQL use in the Cloud and Virtualized Environments

17 17 Linux Configura9on

18 In General 18 For most workloads Linux Runs MySQL surprisingly well with no addi7onal tuning

19 General Configura9on 19 Disable SELinux Unless you really need it on your system In /etc/selinux/config SELINUX=disabled Same with AppArmor Reduce Tendency to swap vm.swappiness=0 vm.dirty_ra7o=5

20 General Configura9on 20 Reduce the File Cache vm.dirty_ra7o=5 NUMA can cause problems hnp://bit.ly/i0gst3 Interleaving with numactl Out there in Percona Server startup scripts

21 Storage Configura9on 21 Separate OS and MySQL Par77on Re- Configure MySQL without OS Reinstall Configure IO Scheduler CFQ can cause problems Deadline or Noop are ocen bener choice elevator=deadline on kernel boot Queue length (especially MyISAM) # echo > /sys/block/sdx/queue/nr_requests Consider using LVM Virtually no overhead when not in snapshot mode Reserve some space for snapshots

22 22 Which one do you use? FILESYSTEMS

23 Filesystems 23 XFS Old High Performance Favorite EXT4 Works very good for some workloads Avoid: Ext2 Ext3 ReiserFS ZFS (may be good for development) Btrfs

24 FS Tuning 24 rela7me (noa7me) ext3: tune2fs O dir_index - c l i 0 e remount- ro xfs: nobarrier Assuming RAID w BBU or Flash

25 25 Why are you running? MYSQL VARIANTS

26 Main Choices 26 MySQL Percona Server MariaDB

27 MySQL Versions 27 Newer versions scale bener MySQL 5.6 is most scalable Percona Server 5.6 brings even more improvements J Use 5.6 for New Development Upgrade to MySQL 5.6 if you need it now

28 Check out Percona XtraDB Cluster 28 Based on Galera Technology Innodb Storage Engine you already know Synchronous Replica7on Parallel Apply Automa7c Provisioning No Stale Reads Not for all workloads

29 How to Install 29 Use Repositories: Best hnp://dev.mysql.com/downloads/repo/ hnp:// server/5.6/ installa7on.html Download RPM Need older Version Use Tar.gz package Good for tes7ng with MySQL Sandbox Build our own Do you really have good reason to do it? Many problems are caused by bad builds

30 MySQL Configura9on 30 You need to tune MySQL configura7on Even newer MySQL 5.6 defaults are unlikely to be op7mal Check out presenta7on which goes in depth into configura7on tuning Typically gerng 5 variables right responsible for 90% performance gain hnp://bit.ly/1fup0sz Avoid obsessive tuning disorder

31 Top Variables 31 Innodb_buffer_pool_size Set 80% of memory some7mes more Innodb_flush_method=O_DIRECT Best serng in most cases Innodb_log_file_size Innodb_flush_log_at_trx_commit=? Set 256MB or more Larger logs = longer recovery 7me Truly ACID: 1 Can Afford Data Loss: 2

32 Advanced Ideas 32 Running mul7ple MySQL instances? Memory Alloca7on Hostspot? LVM Snapshot before maintenance or upgrade taskset - pc 0,12,2,14 `cat /var/ lib/mysql/mysqld1.pid` Jemalloc Tcmalloc old favorite Roll back easily if things go wrong

33 Beware of the Scripts 33 Start/ Stop Scripts can be nasty Timeouts might be not enough for safe database shutdown The automated upgrade/check

34 Automa9on 34 Manual approach is a no Does not Scale And is Error Prone Automate installa7on, upgrades, configura7on Puppet, Chef Ansible Keep Configura7on under version control At very least leave comments on what you chance and why

35 Monitoring and Trending 35 Many tools to choose from! Just make sure you re doing it Nagios and CACTI Check out PMP hnp:// monitoring- plugins Graphite is a great tool for advanced analyses hnp://graphite.wikidot.com/ Looking for Cloud Based Solu7ons Percona Cloud Tools (FREE Beta) hnps://cloud.percona.com/

36 Linux OOM Killer 36 What are your priori7es? SSH MySQL Server Various jobs Backup Percona Toolkit Background Batch jobs Configuring echo - 17 > /proc/2592/oom_adj do not kill ever echo 10 > /proc/2592/oom_adj More like to be killed

37 Tools you need to know 37 General Linux Top Vmstat Iostat Strace gdb MySQL Percona Toolkit Innotop Percona Xtrabackup MyDumper

38 Learn more 38 Percona Training hnp:// Percona Webinars hnp:// MySQL Performance Blog hnp://

39 Aaend The Conference 39 4 More Days of great Learning and Networking Expo Hall, Keynote Free for CentOS Dojo ANendees Single Day Passes Available Use GoLinux Discount Code for 15% off

40 Thank You! Peter Zaitsev 40