SkySQL Data Suite A New Open Source Approach to MySQL Distributed Systems Serge Frezefond V1212.01
Agenda SkySQL Cloud Data Suite Architecture SkySQL Cloud Data Suite on Amazon EC2 Components for automated provisioning in AWS Components for high availability in EC2 The Administration Console Automated Backup/Restore with S3 2
Intro to SkySQL Data Suite 4
SkySQL Cloud Data Suite Replication Replication High High Availability Availability Online Online Apps. Apps. Galera Galera Full-Text Full-Text DRBD DRBD Shared Shared Storage Storage SkySQL SkySQL Data Data Suite Suite Analytics Analytics Carrier Carrier Grade Grade 6
Cloud Data Suite vs Enterprise Data Suit Cloud Cloud Data Data Suite Suite Deployed Deployed in in Amazon Amazon AWS AWS Automatic Automatic configuration configuration and and provisioning provisioning MariaDB MariaDB 5.5 5.5 MySQL MySQL Replication Replication with with Pacemaker/Corosync Pacemaker/Corosync Administration Administration Console Console with with Backup/Restore Backup/Restore SkySQL SkySQL Enterprise Enterprise Monitoring Monitoring aka aka MONyog MONyog -- Trial Trial PhpMyAdmin PhpMyAdmin Community Community Support Support Development Development Release Release Fully Fully OSS OSS (with (with MONyog MONyog Trial) Trial) Enteprise Enteprise Data Data Suite Suite Deployed Deployed in in Clouds, Clouds, virtualised virtualised env. env. and and bare bare OS OS Standard Standard download download packages packages MariaDB, MariaDB, MySQL, MySQL, Percona Percona Servers Servers MySQL MySQL Replication Replication with with MHA, MHA, DRBD, DRBD, Galera Galera or or shared shared storage storage and and Pacemaker/Corosync Pacemaker/Corosync ZManda ZManda Recovery Recovery Manager Manager SkySQL SkySQL Enterprise Enterprise Monitory Monitory aka aka MONyog MONyog SQLyog SQLyog Enterprise Enterprise Technical Technical Support Support Production Production Release Release Mix Mix of of OSS OSS and and Commercial Commercial 7
SkySQL Cloud Data Suite Sep 2012 1.Configure your DB system on 11 config.skysql.com or cloud.skysql.com 2.Deploy the DB system: Online Configuration pull from AWS Manually 33 22 1.Connect the DB system to your application 8
SkySQL Cloud Data Suite Dec 2012 Rel. 1.1 with bug fixing New Administration Console with: rpm package and installation on Linux OS Administration and monitoring of replicated and standalone servers Customisable graphs Customisable commands 9
SkySQL Cloud Data Suite Architecture 10
SkySQL Cloud Data Suite Configurator Deployment Deployment triggered triggered by by the the first first boot boot of of each each instance instance User Account Configuration Engine Rules Repository Provisioning Queue Provisioning Engine Deployment Engine use rda ta.s h I M A Software Repository Deployment Queue 11
SkySQL Cloud Data Suite Terminal Admin Console PhpMyAdmin MONyog 12
SkySQL Data Suite Building Block Admin Console Data Store Replicator 13
SkySQL Cloud Data Suite Sep 2012 Technologies and Software FRONT FRONT END END Apache Apache Tomcat Tomcat Data Store Admin Console Linux Linux 3.2 3.2 (Amazon (Amazon Distro) Distro) BACK BACK END END SQLite SQLite MONyog MONyog Trial Trial Replicator MariaDB MariaDB PHPMyAdmin PHPMyAdmin Pacemaker Pacemaker Corosync Corosync Xtrabackup Xtrabackup GlusterFS GlusterFS MySQL MySQL Replication Replication 14
SkySQL Cloud Data Suite Sep 2012 Data Store Public Public SkySQL SkySQL AMI AMI Data Store MariaDB MariaDB 5.5 5.5 MyISAM, MyISAM, InnoDB, InnoDB, XtraDB, XtraDB, Enhanced Enhanced Optimizer, Optimizer, Virtual Virtual Columns Columns Pacemaker Pacemaker controls controls the the automatic automatic failover failover System System and and Data Data file file systems systems From From standalone standalone to to 11 Master Master ++ 33 Slaves Slaves 15
SkySQL Cloud Data Suite Sep 2012 Administration Console GUI GUI to to monitor monitor the the status status of of the the Cluster Cluster Admin Console Extended Extended Monitoring Monitoring from from MONyog MONyog as as aa trial trial PhPMyAdmin PhPMyAdmin as as graphical graphical user user tool tool Backup/Restore Backup/Restore to/from to/from S3 S3 using using Xtrabackup Xtrabackup Start/Stop, Start/Stop, Isolate, Isolate, Promote Promote nodes nodes mysql mysql command command line line interface interface Automatic Automatic failover failover for for the the admin admin and and monitoring monitoring servers servers 16
SkySQL Cloud Data Suite Sep 2012 Replicator Replicator Standard Standard MySQL MySQL Replication Replication in in asynchronous asynchronous and and semi-synchronous semi-synchronous mode mode GlusterFS GlusterFS on on ext4 ext4 to to replicate replicate the the data data collection collection and and monitoring monitoring Automatic Automatic failover failover of of database database servers servers and and console console components components 17
SkySQL Data Suite Provisioning Elastic Elastic IP IP on on the the Master Master Server Server Adaptive, Adaptive, distributed distributed setup setup Currently, Currently, the the Administration Administration Console Console is is on on the the Master Master server server Variable Variable definition definition -- they they will will be be used used later later when when other other scripts scripts are are called called On-the-fly On-the-fly configuration configuration files, files, such such as as my.cnf my.cnf Execute Execute predefined predefined scripts scripts already already in in the the AMI AMI User Account use rda ta.s h I M A 18
SkySQL Data Suite Deployment 19
SkySQL Data Suite Deployment 20
Automated Provisioning in AWS 21
Starting Here! You You can can try try itit for for FREE! FREE! Even Even without without an an AWS AWS account account 22
Simplified Interface Immediate Immediate Help Help Real Real Time Time Check Check 23
Advanced Configuration 24
Cost Estimation 25
Delivery Mechanism 26
Online Deployment The The keys keys will will be be discarded discarded immediately immediately after after the the provisioning provisioning 27
Extra AWS & Contact Info 28
Welcome Email mysql mysql command command line line Administration Administration Console Console ssh ssh connection connection How How to to remove remove SCDS SCDS An An inventory inventory of of the the AWS AWS components components IMPORTANT! IMPORTANT! Your Your pem pem key key 29
Job Done! 30
SkySQL Cloud Data Suite in Amazon EC2 31
Security Identity Access Manager Roles EIP Address Associate/Disassociate, Describe Instances Start, Stop, Describe Attributes, Status, Tags, Net IF Create, Describe S3 Buckets List, Create, Delete Objects Get, Put, List, Delete 32
Security Security Groups ssh http MySQL MONyog 33
Storage 34
Components for High Availability in EC2 35
Automatic Failover with Pacemaker CRM Configuration Admin Console Data Store Replicator 36
Automatic Failover with Pacemaker CRM Configuration Elastic Elastic IP IP [root@node2 [root@node2 init.d]# init.d]# crm crm status status============last ============Last updated: updated: Sat Sat Dec Dec 11 12:42:03 12:42:03 2012Last 2012Last change: change: Fri Fri Nov Nov 30 30 12:43:27 12:43:27 2012 2012 via via crm_attribute crm_attribute on on node2stack: node2stack: openaiscurrent openaiscurrent DC: DC: node2 node2 -- partition partition with with quorumversion: quorumversion: 1.1.7-6.el61.1.7-6.el6148fccfd5985c5590cc601123c6c16e966b85d144 148fccfd5985c5590cc601123c6c16e966b85d144 Nodes Nodes configured, configured, 44 expected expected votes14 votes14 Resources Resources configured.============online: configured.============online: [[ node2 node2 node3 node3 node1 node1 node4 node4 ]] Master/Slave Master/Slave Set: Set: ms_resmysql ms_resmysql [resmysql] [resmysql] Masters: Slaves: Masters: [[ node2 node2 ]] Slaves: [[ node1 node1 node3 node3 node4 node4 ]] AmazonEIP AmazonEIP (lsb:amazon_master_eip): Started (lsb:amazon_master_eip): Started node2 node2 Clone Clone Set: Set: apache-phpapache-phpmyadmin Started: myadmin [ApachePhpMyAdmin] [ApachePhpMyAdmin] Started: [[ node1 node1 node2 node2 node3 node3 node4 node4 ]] Resource monyog-fs Resource Group: Group: monyog monyog monyog-fs (ocf::custom:filesystem): (ocf::custom:filesystem): Started monyog-service Started Started node2 node2 monyog-service (lsb:monyogd): (lsb:monyogd): Started node2 console-fs node2 Resource Resource Group: Group: admin-console admin-console console-fs (ocf::custom:filesystem): Started SkySQL-monitor (ocf::custom:filesystem): Started node2 node2 SkySQL-monitor (lsb:skysql_monitor): Tomcat7 (lsb:skysql_monitor): Started Started node2 node2 Tomcat7 (ocf::heartbeat:tomcat): Started (ocf::heartbeat:tomcat): Started node2 node2 admin-console admin-console and and MONyog MONyog group group Node Node 22 Public Public IPs IPs Admin Console Data Store Replicator 37
The Administration Console 38
Admin Console Dec 2012 39
Admin Console Dec 2012 Master Master Server Server Access Access to to MONyog MONyog Context Context Sensitive Sensitive Customisable Customisable Graphs Graphs 40
Admin Console Dec 2012 Backup/Restore Backup/Restore Access Access to to PhpMyAdmin PhpMyAdmin Customisable Customisable Commands Commands 41
Administration Console Dec 2012 Customizable Graphs Monitors Name SQL Icon Delta Avg Chart 42
Administration Console Dec 2012 Customizable Commands Commands Desc Icon Order CommandStep Steps Desc Icon Script 43
Automated Backup/Restore with S3 44
Automated Backup Using Xtrabackup from the Admin Console 45
Automated Backup Using Xtrabackup from the Admin Console 46
Automated Backup Using Xtrabackup from the Admin Console 47
Why is the SkySQL Cloud Data Suite important for you? You can test MariaDB/MySQL on AWS for FREE You can use and fully control a ready made highly available solution AWS You can compare MySQL on EC2 and RDS You can study/review security and availability aspects in AWS You can use some of the SCDS components in your environment 48
SkySQL Cloud Data Suite & OpenStack? Nova, glance, swift, keystone, horizon, quantum ceph Position for public/private/hybrid cloud Rackspace,HPcloud,VMware... CloudWatt, Numergy Our solution will be there 49
Is not Amazon RDS enough for my needs? It is fully managed! Provisioning Patching Backup/ PointInTIme recovery Replicas HA / Failover monitoring 50
Or maybe Google Cloud SQL? New offer Similar to AWS RDS Rely on storage feature for HA Fully managed 51
Private/Hybrid cloud Eucalyptus / VMware Vcloud They own a huge market share Eucalyptus is tied to Amazon / open core VMware is tied to VMware and fully closed Excellent solutions but... 52
For more information... Try the Configurator and How-To Video: http://cloud.skysql.com The Admin Console: http://www.skysql.com/downloads/community/skysql-console Forums: https://www.skysql.com/forums/skysql-cloud-data-suite Web: https://www.skysql.com/cloud Bugs: http://bugs.skysql.com Try it! http://config.skysql.com Free SkySQL account available for testing Test on your account 53