Deploying MySQL with the Oracle ZFS Storage Appliance

Deploying MySQL with the Oracle ZFS Storage Appliance Paul Johnson Principal Software Engineer Robert Cummins Principal Software Engineer John Zabriskie Software Engineer John Baton Software Engineer

Table of Contents Introduction 1 Provisioning the Oracle ZFS Storage Appliance 2 Log in to the Browser User Interface 2 Creating a Project 3 Creating Shares 5 Modify Database Record Size 7 Provisioning Oracle Linux 7 9 Terminal 9 Create a Directory Structure 9 Edit /etc/fstab for the New Shares 9 Mount the Shares 9 MySQL Configuration 10 my.cnf Configuration 10 Initialize MySQL and Create a Database 11 Populate and Stress the Database with SysBench 11 Monitor Progress and Performance Using ZFS Storage Analytics 13 Navigate to the Analytics section 13 Add a Statistic 13 Backup MySQL Using ZFS Snapshots and Clones 16 Open a New Terminal 16 Create a Project Snapshot 16 Clone the Project Shares 17 Additional Links 22 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Introduction This in-depth hands-on lab demonstrates how to install and manage MySQL with Oracle ZFS Storage Appliance. You ll learn how to correctly tweak MySQL's database configuration to maximize performance over protocols such as Fibre Channel, iscsi, and NFS. In addition, you ll learn how easy it is to use the Oracle ZFS Storage Appliance software to configure services, create and use storage shares, and monitor database performance with Oracle ZFS Storage Appliance analytics. 1 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Provisioning the Oracle ZFS Storage Appliance Log in to the Browser User Interface 1. Open a web browser by double-clicking on the Firefox icon on the desktop. 2. Enter https://192.168.56.3:215 into web browser window. 3. In the web browser, enter oow as Username and oow as Password, then press the LOGIN button. 2 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

4. The Oracle ZFS Storage Appliance Status Dashboard will be displayed. Creating a Project A project is a collection of filesystems and LUNs that share the same common settings. By grouping all shares into the same project, the management of the storage becomes simplified. 1. Begin by clicking on Shares on the top toolbar. 2. Click on PROJECTS on the right side then click on + on the left side to create a new project. 3 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

3. Name the project HOL9760. 4. Once the project has been created, edit the HOL9760 project by selecting the pencil icon on the left side of the screen. 5. With the HOL9760 project opened, select General at the top to view the project settings. 6. The following changes will need to be made: a. Mountpoint should be /export/hol9760 b. Update access time on read should be deselected c. User should be 12345 d. Group should be 12345 e. Select RWX permissions for Group and Other to turn the icons orange 4 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

7. Click Apply at the top right. Creating Shares The MySQL tablespace will be stored on the Oracle ZFS Storage Appliance Simulator. There will need to be two shares created to store the data and log files. 5 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

1. Click on Shares next to General to begin creation. Do not click Shares on the top toolbar as this will deselect the current project. 2. Click the + button to create a new file-system. 3. Create a share called innodb-data and click Apply. 6 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

4. Create a share called innodb-log and click Apply. You should now have two shares created: innodb-data and innodb-log. Modify Database Record Size 1. Edit the innodb-data share by clicking on the pencil icon. 2. Under Properties, locate Database record size. Click on the lock icon to unlock and select 16K. 7 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

3. Click Apply at the top right. 8 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Provisioning Oracle Linux 7 The binaries for MySQL have already been preinstalled on the Oracle Linux 7 VM. Terminal Open a new terminal session by double-clicking on the Terminal icon on the desktop: Create a Directory Structure Each of the shares created on the storage will need a local directory to contain the mounts on the Oracle Linux 7 VM. Please use the following locations: $ sudo mkdir -p /zfssa/hol9760/innodb-data $ sudo mkdir -p /zfssa/hol9760/innodb-log Edit /etc/fstab for the New Shares The entries have already been populated inside of /etc/fstab. $ cat /etc/fstab # MySQL Lab 192.168.56.3:/export/HOL9760/innodb-data /zfssa/hol9760/innodb-data nfs rw,bg,hard,nointr,rsize=1048576,wsize=1048576,tcp,vers=3,timeo=600,noac,nolock 0 0 192.168.56.3:/export/HOL9760/innodb-log /zfssa/hol9760/innodb-log nfs rw,bg,hard,nointr,rsize=1048576,wsize=1048576,tcp,vers=3,timeo=600,noac,nolock 0 0 Mount the Shares $ sudo mount -a 9 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

MySQL Configuration my.cnf Configuration The following configuration is used for this hands-on lab. Certain buffer and log sizes have been scaled back to meet the limited resources of this virtual environment. $ cat /etc/my.cnf [client] port = 3306 socket = /var/lib/mysql/mysql.sock [mysqld] user = mysql group_concat_max_len = 8192 max_allowed_packet = 64M key_buffer = 24M sort_buffer_size = 2M read_buffer_size = 2M read_rnd_buffer_size = 8M myisam_sort_buffer_size = 64M thread_cache_size = 8 query_cache_size = 0 thread_concurrency = 16 max_connections = 2048 tmp_table_size = 24M binlog_cache_size = 10M expire-logs-days = 5 skip_name_resolve innodb_file_per_table innodb_table_locks = 0 innodb_open_files = 4096 innodb_log_files_in_group = 2 innodb_thread_concurrency = 0 innodb_log_file_size = 12M innodb_buffer_pool_size = 24M innodb_additional_mem_pool_size = 64M innodb_lock_wait_timeout = 5 innodb_flush_log_at_trx_commit = 2 # ZFSSA innodb_doublewrite = 0 innodb_flush_method = O_DIRECT innodb_data_home_dir = /zfssa/hol9760/innodb-data innodb_log_group_home_dir = /zfssa/hol9760/innodb-log innodb_data_file_path = ibdatafile:24m:autoextend The following options are the most important for successful configuration with the Oracle ZFS Storage Appliance: 1. innodb_doublewrite = 0 A double write buffer is necessary in the event of partial page writes. However, the transactional nature of ZFS guarantees that partial writes will never occur. This can be safely disabled. 2. innodb_flush_method = O_DIRECT Ensures that InnoDB calls directio() instead of fcntl() for the data files. This allows the data to be accessed without OS level buffering and read-ahead. 10 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

3. innodb_data_home_dir = /zfssa/hol9760/innodb-data The data filesystem for InnoDB should be located on its own share or LUN on the Oracle ZFS Storage Appliance. 4. innodb_log_group_home_dir = /zfssa/hol9760/innodb-log The log filesystem for InnoDB should be located on its own share or LUN on the Oracle ZFS Storage Appliance. 5. innodb_data_file_path = ibdatafile:24m:autoextend This configures a single large tablespace for InnoDB. The ZFS controller is then responsible for managing the growth of new data. This eliminates the complexity needed for controlling multiple tablespaces. Initialize MySQL and Create a Database 1. Switch the current user to mysql. Use the password oow. $ su mysql 2. Use mysql_install_db to initialize the MySQL data directory and populate the system tables. $ mysql_install_db 3. Enable the MySQL service: $ service mysql start 4. Log in to the MySQL console. There is no password. Press return to bypass the password prompt. $ mysql u root p Enter password: 5. Create a database called HOL9760: mysql> create database HOL9760; Query OK, 1 row affected (0.00 sec) mysql> quit Populate and Stress the Database with SysBench 1. Change the directory to the SysBench scripts. $ cd /home/oow/mysql 2. Populate the database using the following script: $ cat filldb.sh #!/bin/sh sysbench \ --test=oltp \ --oltp-table-size=10000 \ --mysql-db=hol9760 \ --mysql-user=root \ --mysql-password= \ prepare $./filldb.sh 3. Run an OLTP workload using the following script: $ cat stressdb.sh #!/bin/sh 11 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

sysbench \ --test=oltp \ --oltp-table-size=10000 \ --oltp-test-mode=complex \ --oltp-read-only=off \ --num-threads=8 \ --max-time=3600 \ --max-requests=0 \ --mysql-db=hol9760 \ --mysql-user=root \ --mysql-password= \ --mysql-table-engine=innodb \ run $./stressdb.sh 12 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Monitor Progress and Performance Using ZFS Storage Analytics Analytics is an advanced facility that you can use to graph a variety of statistics in real-time and record this data for later viewing. It has been designed for both long term monitoring and short term analysis. When needed, it makes use of DTrace to dynamically create custom statistics, which allows different layers of the operating system stack to be analyzed in detail. Navigate to the Analytics section While the database workload is running, click on the Analytics tab on the right side of the upper taskbar. Add a Statistic On the left side of the browser, click on the + icon next to Add statistic to bring up the statistic menu. The most relevant statistics for database performance analysis can be found in the NFSv3 category. 13 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Analytics has been designed around an effective performance analysis technique called drill-down analysis. This involves checking high level statistics first, and to focus on finer details based on findings so far. This allows you to quickly narrow the focus to the most likely areas. 14 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

15 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Backup MySQL Using ZFS Snapshots and Clones The snapshot and clone features of the Oracle ZFS Storage Appliance provides a convenient mechanism to backup, restore, and fork a MySQL database. A ZFS snapshot is a read-only copy of a file-system that is created instantaneously while initially occupying zero additional space. As the data of a file-system changes, the differences are tracked inside the snapshot. A ZFS clone is a writeable snapshot that can be used to branch off an existing file-system without modifying the original contents. Open a New Terminal 1. Open a new terminal session by double-clicking on the Terminal icon on the desktop: 2. Change the directory to mysql: $ cd /home/oow/mysql 3. Change your user to mysql: $ su mysql Create a Project Snapshot The MySQL database must have all its open tables closed with a read lock prior to a snapshot being created. After the snapshot has completed, the tables can be unlocked. The following python script accomplishes these MySQL tasks and creates a project level snapshot using the Oracle ZFS Storage Appliance REST interface. $ cat mysql_snapshot.py #!/usr/bin/python import MySQLdb import datetime import json import urllib2 def zfs_snapshot(): user = "oow" password = "oow" url = "192.168.56.3" path = "/api/storage/v1/pools/oow/projects/hol9760/snapshots" url = "https:// " + zfs + ":215" + path properties = {"name":"hol9760-snapshot"} post_data = json.dumps(properties) request = urllib2.request(url, post_data) request.add_header("content-type", "application/json") request.add_header("x-auth-user", user) request.add_header("x-auth-key", password) response = urllib2.urlopen(request) 16 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

def main(): mysql_server = "localhost" mysql_user = "root" mysql_pass = "" try: connection = MySQLdb.connect(host=mysql_server, user=mysql_user,passwd=mysql_pass) except MySQLdb.OperationalError: print "Could not connect to the MySQL server" sys.exit(-5) print "Connected to the MySQL server" backup = connection.cursor() start_time = datetime.datetime.now().replace(hour=0) backup.execute("set autocommit=0;") backup.execute("flush tables with read lock;") print "Creating project snapshot \"HOL9760-snapshot\"" zfs_snapshot() backup.execute("unlock tables;") finish_time = datetime.datetime.now().replace(hour=0) total_time = finish_time - start_time print "Completed in ", total_time if name == " main ": main() Run the mysql_snapshot.py script. $./mysql_snapshot.py Clone the Project Shares With the successful creation of a snapshot, you can now rollback the database to a previous state. Additionally, you can use the cloning feature to generate a newly forked database. The following steps outline this process: 1. Open the Firefox web browser. Use the username oow and password oow if the login screen for the Oracle ZFS Storage Appliance appears. 2. Click on Shares. 3. Click on Projects. 17 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

4. Double-click on the HOL9760 project. 5. Edit the innodb-data project by clicking on the pencil icon. 6. Click on Snapshots. 7. Under NAME, click on the + icon for the share innodb-data to create a clone. 18 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

8. Set the Name to innodb-data-clone and click Apply. 9. Click on Projects. 19 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

10. Double click on the HOL9760 project. 11. Edit the innodb-log share by clicking on the pencil icon. 12. Click on Snapshots. 13. Under NAME, click on the + icon for the share innodb-log to create a clone. 20 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

14. Set the Name to innodb-log-clone and click Apply. 15. On the Oracle Linux VM, create a directory structure to hold the new clones: $ sudo mkdir /zfssa/hol9760/innodb-data-clone $ sudo mkdir /zfssa/hol9760/innodb-log-clone 16. Mount the clones: $ sudo mount o rsize=1048576,wsize=1048576,tcp,vers=3,timeo=600,noac,nolock 192.168.56.3:/export/HOL9760/innodb-data-clone /zfssa/hol9760/innodb-dataclone $ sudo mount o rsize=1048576,wsize=1048576,tcp,vers=3,timeo=600,noac,nolock 192.168.56.3:/export/HOL9760/innodb-log-clone /zfssa/hol9760/innodb-log-clone The clones can now be used for a separate MySQL instance. The configuration file my.cnf can be updated to use the innodb-data-clone and innodb-log-clone for database test and development purposes. 21 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Additional Links 1. VirtualBox http://www.virtualbox.org/ 2. Oracle ZFS Storage Simulator http://www.oracle.com/technetwork/server-storage/sun-unified-storage/downloads/sun-simulator- 1368816.html 3. Oracle Linux 7 http://edelivery.oracle.com/linux 4. HOL9760 Lab Information http://blogs.oracle.com/paulie 22 HOL9760 - DEPLOYING MYSQL WITH THE ORACLE ZFS STORAGE APPLIANCE

Oracle Corporation, World Headquarters Worldwide Inquiries 500 Oracle Parkway Phone: +1.650.506.7000 Redwood Shores, CA 94065, USA Fax: +1.650.506.7200 C O N N E C T W I T H U S blogs.oracle.com/oracle facebook.com/oracle twitter.com/oracle oracle.com Copyright 2014, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0914