MySQL Backups: From strategy to Implementation

Transcription

1 MySQL Backups: From strategy to Implementation Mike Frank Senior Product Manager 1

2 Program Agenda Introduction The 5 Key Steps Advanced Options References 2

3 Backups are a DBAs Top Priority Be Prepared Automate Test and Practice Be Ready to recover when a data disaster strikes 3

4 Backups As a DBA you must have a backup strategy Data loss will happen OS crashes Power Failures Data center outages Are you in a cloud? File system crashes Human error Application Error Just a reminder of course - you know all that. 4

5 MySQL Enterprise Backup Performs Hot Backup Highly efficient and Consistent Performs the backup while the database is running Necessary when The database can t be taken offline Those are Cold Backups The database is large backups can take a long time Export backups are too slow, lock tables, Also lets you perform Warm Backups for non-innodb tables. Table level locks 5

6 What we ll cover today There s more than we can cover this session The goal today is to get you running scheduled backups regularly. The first key step to protecting your data Later you can explore more powerful work flows and features Partial data moment Compression Task synchronization Hot and Cold support 6

7 The 5 Easy Steps 7

8 The 5 Steps 1. Download and Install 2. Setup Backup Directories 3. Creating Full and Incremental Backups 4. Scheduling 5. Recovery Prepare and Restore 8

9 Step 1 Download and Install 9

10 Download Login Pick MySQL Database and then OS platform Hit Go 10

11 Find the download package Your looking for MySQL Enterprise Backup Half way down the page select Specific OS/Version and Format - either TAR or RPM Note for Windows - MySQL Installer for Windows includes MySQL Enterprise Backup 11

12 Install Uncompress the downloaded file - then ZIPs and Tars Basically you can unzip, untar MEB where ever you want RPM sudo rpm -i package_name.rpm Deb packages sudo dpkg i package_name.rpm Windows Run the MSI installs the product in C:\Program Files\MySQL\MySQL Enterprise Backup

13 Linux from tar 13

14 Windows After running MSI installer 14

15 Once installed set path or note location Default Windows from MSI C:\Program Files\MySQL\MySQL Enterprise Backup 3.8\mysqlbackup.exe Used in 5 step example Default Linux from RPM / DEB /opt/mysql/meb-3.8 If tar or zip will be where you unpacked the files For linux 5 step example used /home/mfrank/meb linux2.6-x86-64bit/ 15

16 Step 2 Setup Backup Directories 16

17 Where do you want the backups to go Make sure you have plenty of space Monitor that space Linux mkdir /MySQLBackupHome chown mysql R /MySQLBackupHome chgrp mysql R /MySQLBackupHome chmod 600 R /MySQLBackupHome Windows mkdir C:\mydatabasebackups\ Make sure only the user that has rights to mysql file can read and write to the directory Right Click properties and Edit permissions for only that user (mysql) Permissions - Modify, Read & Execute, List Folders, Read, Write 17

18 Step 3 Backup 18

19 Backup Types Full all the database data Note but not the mysql programs and control files Incremental only the database changes Prerequisite a full backup Can be more than one incremental backups The changes between last full or a prior incremental Partial just part of the database Binary Logs contains each event/transaction

20 Keeping it simple You ll want to run backups on a schedule For example here we will show 1 full once week An incremental daily 20

21 Commands and Scripts You need to have scripts that you schedule In this case a minimum of 2 FULL INCREMENTAL 21

22 Create Backup User For this example the backup user is mysqlbackup mysql -u root p Mysql> CREATE USER 'mysqlbackup'@'localhost' IDENTIFIED BY '<password>'; mysql> GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost'; mysql> GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost'; mysql> GRANT CREATE, INSERT, DROP ON mysql.ibbackup_binlog_marker TO 'mysqlbackup'@'localhost'; mysql> GRANT CREATE, INSERT, DROP ON mysql.backup_progress TO 'mysqlbackup'@'localhost'; mysql> GRANT CREATE, INSERT, DROP ON mysql.backup_history TO 'mysqlbackup'@'localhost'; mysql> GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost'; mysql> GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost'; mysql> GRANT CREATE TEMPORARY TABLES ON mysql.* TO 'mysqlbackup'@'localhost'; mysql> FLUSH PRIVILEGES; 22

23 Step 3 a Full Backup 23

24 FULL Script will Create a time stamped subdirectory in the home backup area Create the backup command Run the backup command 24

25 Linux FULL Bash Script #!/bin/bash dtstring=`date +%F` new_dir="/mysqlbackuphome/mysqlfullweeklybackup_"$dtstring mkdir $new_dir cd /MySQLBackupHome /home/mfrank/meb linux2.6-x86-64bit/bin/mysqlbackup --defaults-file=mysqlback.cnf --backup-dir=$new_dir/backup backup &> $new_dir/backup.log echo $? 25

26 Linux Configuration File - mysqlback.cnf [client] port = 3306 protocol = TCP user = <your backup user> password = <your backup user password> [mysqld] datadir = /var/lib/mysql [mysqlbackup] datadir = /var/lib/mysql innodb_data_home_dir = incremental_base = history:last_backup comments = My main backup - Full once a week, Incrementals daily Note: MEB supports MySQL External Authentication 26

27 Windows FULL VBScript Set shell = WScript.CreateObject("WScript.Shell") Dim fso Set fso = CreateObject("Scripting.FileSystemObject") new_dir = "C:\MySQLBackupHome\mysqlfullweeklybackup_" & replace(formatdatetime(now,2),"/","") If Not fso.folderexists(new_dir) Then Set f = fso.createfolder( new_dir ) End If backup_command = "cmd.exe /c ""C:\Program Files\MySQL\MySQL Enterprise Backup 3.8\mysqlbackup.exe"" --defaults-file=mysqlback.cnf --backup-dir=" & new_dir & "\backup backup > " & new_dir & "\backup.log 2>&1" Set oexec = shell.exec(backup_command) Do While oexec.status = 0 Loop WScript.Sleep

28 Windows Configuration File c:\mysqlbackuphome\mysqlback.cnf [client] port = 3306 socket = MySQL user = <your backup user> password = <your backup user password> [mysqld] datadir = C:\ProgramData\MySQL\MySQL Server 5.6\data\ [mysqlbackup] datadir = C:\ProgramData\MySQL\MySQL Server 5.6\data\ innodb_data_home_dir = Note: MEB supports MySQL External Authentication (Native Windows User) 28

29 Step 3 a Incremental Backup 29

30 Linux Incremental Bash Script #!/bin/bash dtstring=`date +%F` new_dir="/mysqlbackuphome/mysqlincrementaldailybackup_"$dtstring rm -Rf $new_dir mkdir $new_dir cd /MySQLBackupHome /home/mfrank/meb linux2.6-x86-64bit/bin/mysqlbackup --defaults-file=mysqlback.cnf --incremental --backup-dir=$new_dir/backup --incremental-backupdir=$new_dir/backup/inc backup &> $new_dir/backup.log echo $? 30

31 Windows Incremental VBScript Set shell = WScript.CreateObject("WScript.Shell") Dim fso Set fso = CreateObject("Scripting.FileSystemObject") new_dir = "C:\MySQLBackupHome\mysqlincrementaldailybackup_" & replace(formatdatetime(now,2),"/","") If Not fso.folderexists(new_dir) Then Set f = fso.createfolder( new_dir ) End If Dim backup_command backup_command = "cmd.exe /c ""C:\Program Files\MySQL\MySQL Enterprise Backup 3.8\mysqlbackup.exe"" --defaults-file=mysqlback.cnf --incremental --backup-dir=" & new_dir & "\backup --incremental-backup-dir=" & new_dir & "\backup\inc backup > " & new_dir & "\backup.log 2>&1 Set oexec = shell.exec(backup_command) Do While oexec.status = 0 Loop WScript.Sleep

32 Step 4 Scheduling 32

33 Step 4 a Linux Scheduling 33

34 Linux CRON Weekly Full Sun at 2:00 am crontab e 0 2 * * 0 /MySQLBackupHome/MyFullBackup.sh Daily Incremental Mon-Sat at 2:00 am crontab e 0 2 * * 1-6 /MySQLBackupHome/MyIncrementalBackup.sh 34

35 Step 4 b Windows Scheduling 35

36 Windows Scheduler Create a Basic Task 36

37 Weekly for Full Daily for Incremental (6 days not including the day the full runs) 37

38 Select Time, Specify Start a Program 38

39 Path to your full or incremental script 39

40 When scripts run MEB details to this log 40

41 Step 5 Recovery 41

42 Recovery - Preparing Determine time to recover to If the time is for an incremental Applying an Incremental Backup changes reflected in incremental backup files bring the backup up-to-date Apply those changes for each - oldest to newest For example mysqlbackup --backup-dir=/mysqlbackup/mysqlfullweeklyfullbackup_ /backup --incremental-backup-dir =/MySQLBackup/mysqlfulldailyincrementalbackup_ /backup/inc apply-incremental-backup 42

43 Recovery Restore the data Shutdown the database to be restored Stop the service or run a mysqladmin shutdown Run Copy-Back Copies the datafiles, logs, and other backed up files into place For example mysqlbackup --defaults-file=/etc/my.cnf --backup-dir =/MySQLBackup/mysqlfullweeklyfullbackup_ /backup copy-back 43

44 Advanced - Backup Getting finer grained for Point in Time Schedule the mysqlbinlog utility to dump all the SQL activity that happened after the binlog position of the last backup Direct the output of the mysqlbinlog command to an output file $ mysqlbinlog --start-position= /sqldata/binlog >partial_binlog $ mysqlbinlog /sqldata/binlog >> partial_binlog $ mysqlbinlog /sqldata/binlog >> partial_binlog... $ mysqlbinlog /sqldata/binlog.00000n >> partial_binlog 44

45 Advanced - Recovery Getting finer grained for an exact Point in Time To recover the database to the latest possible time, Pipe the output from mysqlbinlog --start-position=... Directly to mysql This replays all the SQL statements after the last backup. Or to get to a point in time say be before an oops happened Edit the bin log and remove the oops and all statements after it. Then $ mysql < partial_binlog 45

46 Advanced Single File Recommended Has been seen to be faster than directory backup Gives you an additional facility to verify the backup Does take a little longer to restore BUT - Can be prepared for restore Extract Apply log Compress back again 46

47 Advanced Backup Direct to Tape Meb Is a complete tape solution - Direct to Tape using SBT SBT is Oracle s API for backing up and restoring data via media management products use SBT for integration with Oracle Secure Backup use SBT for integration with Symantec Net Backup use SBT for integration with Tivoli Storage Manager Other products that support SBT 47

48 Advanced Using SBT 48

49 Advanced - Compression The compression value needs to be determined 1 is normally a good value Unless you have an idle CPU And a very strict disk limitation Compression will save disk but eat up more CPU and will take longer The restore will also be longer 49

50 Advanced Internal Table backup_history table MySQL Enterprise backup writes status to internal backup table Useful to see detailed history of backups Provide an audit trail to managers and auditors 50

51 Finally This is just the most basic of scripts You also need to decide on Retention What to save and for how long Schedule and automate this process Its often nice to keep recent backups close for fast recovery Archiving Whether to go to tape MEB also supports direct to tape backups Move files to archive storage tape, HDFS, Cloud - S3. 51

52 Resources Product Page Latest Release User Guide Team Blog Questions, Ideas, Phone: USA= ; Outside USA = mike.frank@oracle.com 52