SkySQL Data Suite. A New Open Source Approach to MySQL Distributed Systems. Serge Frezefond V

Transcription

1 SkySQL Data Suite A New Open Source Approach to MySQL Distributed Systems Serge Frezefond V

2 Agenda SkySQL Cloud Data Suite Architecture SkySQL Cloud Data Suite on Amazon EC2 Components for automated provisioning in AWS Components for high availability in EC2 The Administration Console Automated Backup/Restore with S3 2

3 Intro to SkySQL Data Suite 4

4 SkySQL Cloud Data Suite Replication Replication High High Availability Availability Online Online Apps. Apps. Galera Galera Full-Text Full-Text DRBD DRBD Shared Shared Storage Storage SkySQL SkySQL Data Data Suite Suite Analytics Analytics Carrier Carrier Grade Grade 6

5 Cloud Data Suite vs Enterprise Data Suit Cloud Cloud Data Data Suite Suite Deployed Deployed in in Amazon Amazon AWS AWS Automatic Automatic configuration configuration and and provisioning provisioning MariaDB MariaDB MySQL MySQL Replication Replication with with Pacemaker/Corosync Pacemaker/Corosync Administration Administration Console Console with with Backup/Restore Backup/Restore SkySQL SkySQL Enterprise Enterprise Monitoring Monitoring aka aka MONyog MONyog -- Trial Trial PhpMyAdmin PhpMyAdmin Community Community Support Support Development Development Release Release Fully Fully OSS OSS (with (with MONyog MONyog Trial) Trial) Enteprise Enteprise Data Data Suite Suite Deployed Deployed in in Clouds, Clouds, virtualised virtualised env. env. and and bare bare OS OS Standard Standard download download packages packages MariaDB, MariaDB, MySQL, MySQL, Percona Percona Servers Servers MySQL MySQL Replication Replication with with MHA, MHA, DRBD, DRBD, Galera Galera or or shared shared storage storage and and Pacemaker/Corosync Pacemaker/Corosync ZManda ZManda Recovery Recovery Manager Manager SkySQL SkySQL Enterprise Enterprise Monitory Monitory aka aka MONyog MONyog SQLyog SQLyog Enterprise Enterprise Technical Technical Support Support Production Production Release Release Mix Mix of of OSS OSS and and Commercial Commercial 7

6 SkySQL Cloud Data Suite Sep Configure your DB system on 11 config.skysql.com or cloud.skysql.com 2.Deploy the DB system: Online Configuration pull from AWS Manually Connect the DB system to your application 8

7 SkySQL Cloud Data Suite Dec 2012 Rel. 1.1 with bug fixing New Administration Console with: rpm package and installation on Linux OS Administration and monitoring of replicated and standalone servers Customisable graphs Customisable commands 9

8 SkySQL Cloud Data Suite Architecture 10

9 SkySQL Cloud Data Suite Configurator Deployment Deployment triggered triggered by by the the first first boot boot of of each each instance instance User Account Configuration Engine Rules Repository Provisioning Queue Provisioning Engine Deployment Engine use rda ta.s h I M A Software Repository Deployment Queue 11

10 SkySQL Cloud Data Suite Terminal Admin Console PhpMyAdmin MONyog 12

11 SkySQL Data Suite Building Block Admin Console Data Store Replicator 13

12 SkySQL Cloud Data Suite Sep 2012 Technologies and Software FRONT FRONT END END Apache Apache Tomcat Tomcat Data Store Admin Console Linux Linux (Amazon (Amazon Distro) Distro) BACK BACK END END SQLite SQLite MONyog MONyog Trial Trial Replicator MariaDB MariaDB PHPMyAdmin PHPMyAdmin Pacemaker Pacemaker Corosync Corosync Xtrabackup Xtrabackup GlusterFS GlusterFS MySQL MySQL Replication Replication 14

13 SkySQL Cloud Data Suite Sep 2012 Data Store Public Public SkySQL SkySQL AMI AMI Data Store MariaDB MariaDB MyISAM, MyISAM, InnoDB, InnoDB, XtraDB, XtraDB, Enhanced Enhanced Optimizer, Optimizer, Virtual Virtual Columns Columns Pacemaker Pacemaker controls controls the the automatic automatic failover failover System System and and Data Data file file systems systems From From standalone standalone to to 11 Master Master Slaves Slaves 15

14 SkySQL Cloud Data Suite Sep 2012 Administration Console GUI GUI to to monitor monitor the the status status of of the the Cluster Cluster Admin Console Extended Extended Monitoring Monitoring from from MONyog MONyog as as aa trial trial PhPMyAdmin PhPMyAdmin as as graphical graphical user user tool tool Backup/Restore Backup/Restore to/from to/from S3 S3 using using Xtrabackup Xtrabackup Start/Stop, Start/Stop, Isolate, Isolate, Promote Promote nodes nodes mysql mysql command command line line interface interface Automatic Automatic failover failover for for the the admin admin and and monitoring monitoring servers servers 16

15 SkySQL Cloud Data Suite Sep 2012 Replicator Replicator Standard Standard MySQL MySQL Replication Replication in in asynchronous asynchronous and and semi-synchronous semi-synchronous mode mode GlusterFS GlusterFS on on ext4 ext4 to to replicate replicate the the data data collection collection and and monitoring monitoring Automatic Automatic failover failover of of database database servers servers and and console console components components 17

16 SkySQL Data Suite Provisioning Elastic Elastic IP IP on on the the Master Master Server Server Adaptive, Adaptive, distributed distributed setup setup Currently, Currently, the the Administration Administration Console Console is is on on the the Master Master server server Variable Variable definition definition -- they they will will be be used used later later when when other other scripts scripts are are called called On-the-fly On-the-fly configuration configuration files, files, such such as as my.cnf my.cnf Execute Execute predefined predefined scripts scripts already already in in the the AMI AMI User Account use rda ta.s h I M A 18

17 SkySQL Data Suite Deployment 19

18 SkySQL Data Suite Deployment 20

19 Automated Provisioning in AWS 21

20 Starting Here! You You can can try try itit for for FREE! FREE! Even Even without without an an AWS AWS account account 22

21 Simplified Interface Immediate Immediate Help Help Real Real Time Time Check Check 23

22 Advanced Configuration 24

23 Cost Estimation 25

24 Delivery Mechanism 26

25 Online Deployment The The keys keys will will be be discarded discarded immediately immediately after after the the provisioning provisioning 27

26 Extra AWS & Contact Info 28

27 Welcome mysql mysql command command line line Administration Administration Console Console ssh ssh connection connection How How to to remove remove SCDS SCDS An An inventory inventory of of the the AWS AWS components components IMPORTANT! IMPORTANT! Your Your pem pem key key 29

28 Job Done! 30

29 SkySQL Cloud Data Suite in Amazon EC2 31

30 Security Identity Access Manager Roles EIP Address Associate/Disassociate, Describe Instances Start, Stop, Describe Attributes, Status, Tags, Net IF Create, Describe S3 Buckets List, Create, Delete Objects Get, Put, List, Delete 32

31 Security Security Groups ssh http MySQL MONyog 33

32 Storage 34

33 Components for High Availability in EC2 35

34 Automatic Failover with Pacemaker CRM Configuration Admin Console Data Store Replicator 36

35 Automatic Failover with Pacemaker CRM Configuration Elastic Elastic IP IP init.d]# init.d]# crm crm status status============last ============Last updated: updated: Sat Sat Dec Dec 11 12:42:03 12:42: Last 2012Last change: change: Fri Fri Nov Nov :43:27 12:43: via via crm_attribute crm_attribute on on node2stack: node2stack: openaiscurrent openaiscurrent DC: DC: node2 node2 -- partition partition with with quorumversion: quorumversion: el el6148fccfd5985c5590cc601123c6c16e966b85d fccfd5985c5590cc601123c6c16e966b85d144 Nodes Nodes configured, configured, 44 expected expected votes14 votes14 Resources Resources configured.============online: configured.============online: [[ node2 node2 node3 node3 node1 node1 node4 node4 ]] Master/Slave Master/Slave Set: Set: ms_resmysql ms_resmysql [resmysql] [resmysql] Masters: Slaves: Masters: [[ node2 node2 ]] Slaves: [[ node1 node1 node3 node3 node4 node4 ]] AmazonEIP AmazonEIP (lsb:amazon_master_eip): Started (lsb:amazon_master_eip): Started node2 node2 Clone Clone Set: Set: apache-phpapache-phpmyadmin Started: myadmin [ApachePhpMyAdmin] [ApachePhpMyAdmin] Started: [[ node1 node1 node2 node2 node3 node3 node4 node4 ]] Resource monyog-fs Resource Group: Group: monyog monyog monyog-fs (ocf::custom:filesystem): (ocf::custom:filesystem): Started monyog-service Started Started node2 node2 monyog-service (lsb:monyogd): (lsb:monyogd): Started node2 console-fs node2 Resource Resource Group: Group: admin-console admin-console console-fs (ocf::custom:filesystem): Started SkySQL-monitor (ocf::custom:filesystem): Started node2 node2 SkySQL-monitor (lsb:skysql_monitor): Tomcat7 (lsb:skysql_monitor): Started Started node2 node2 Tomcat7 (ocf::heartbeat:tomcat): Started (ocf::heartbeat:tomcat): Started node2 node2 admin-console admin-console and and MONyog MONyog group group Node Node 22 Public Public IPs IPs Admin Console Data Store Replicator 37

36 The Administration Console 38

37 Admin Console Dec

38 Admin Console Dec 2012 Master Master Server Server Access Access to to MONyog MONyog Context Context Sensitive Sensitive Customisable Customisable Graphs Graphs 40

39 Admin Console Dec 2012 Backup/Restore Backup/Restore Access Access to to PhpMyAdmin PhpMyAdmin Customisable Customisable Commands Commands 41

40 Administration Console Dec 2012 Customizable Graphs Monitors Name SQL Icon Delta Avg Chart 42

41 Administration Console Dec 2012 Customizable Commands Commands Desc Icon Order CommandStep Steps Desc Icon Script 43

42 Automated Backup/Restore with S3 44

43 Automated Backup Using Xtrabackup from the Admin Console 45

46 Why is the SkySQL Cloud Data Suite important for you? You can test MariaDB/MySQL on AWS for FREE You can use and fully control a ready made highly available solution AWS You can compare MySQL on EC2 and RDS You can study/review security and availability aspects in AWS You can use some of the SCDS components in your environment 48

47 SkySQL Cloud Data Suite & OpenStack? Nova, glance, swift, keystone, horizon, quantum ceph Position for public/private/hybrid cloud Rackspace,HPcloud,VMware... CloudWatt, Numergy Our solution will be there 49

48 Is not Amazon RDS enough for my needs? It is fully managed! Provisioning Patching Backup/ PointInTIme recovery Replicas HA / Failover monitoring 50

49 Or maybe Google Cloud SQL? New offer Similar to AWS RDS Rely on storage feature for HA Fully managed 51

50 Private/Hybrid cloud Eucalyptus / VMware Vcloud They own a huge market share Eucalyptus is tied to Amazon / open core VMware is tied to VMware and fully closed Excellent solutions but... 52

51 For more information... Try the Configurator and How-To Video: The Admin Console: Forums: Web: Bugs: Try it! Free SkySQL account available for testing Test on your account 53