2000 databases later. Kristian Köhntopp Mittwoch, 17. April 13

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "2000 databases later. Kristian Köhntopp http://google.com/+kristiankohntopp. Mittwoch, 17. April 13"

Transcription

1 2000 databases later Kristian Köhntopp

2 TL;DR about 2000 MySQL instances in production Mostly running MySQL 5.5, some 5.6 beta since May 2012 made gamble that we would be all-out on 5.6 by now, lost that gamble reason are interesting new features in the optimizer, most likely 2

3 Database Environment in numbers

4 14 TB of unique data

5 73 TB archived unique data

6 445 TB space used by MySQL databases

7 2000 instances of MySQL running

8 7 The number of Database administrators

9 Environment CentOS Apache Perl with Plack MySQL 5.5, moving to 5.6 A lot of local custom tools 9

10 Servers and Versions 10

11 Replication chains 11

12 Cold standby Master/Standby: shared storage (on filer, previously DRBD) migratory VIP failover manual (scripted) automated solution less reliable than manual 12

13 Master hardware HP DL 380 G7 12 phy cores (24 HT) 96/192 GB Memory 6 drives RAID-10 (or filer) 13

14 Unreliable slaves HP BL:460c G7 12 phy cores (24 HT) 96/192 GB Memory 2 drives RAID-1 (or 1 SSD) 14

15 Situation May 2012: Need 5.6

16 Use Case that breaks it Availability Data Normalized data transformed into flattened, read-optimized view Massively parallel writes 16

17 MySQL 5.5 does not cut it InnoDB internal locks stalling with 24 cores Especially writes Redo Log limit: 4GB max Again writes 17

18 Fix: MySQL 5.6 Less global locks 32 GB Redo-Log 200 concurrent writes typically 24GB Redo-Log busy SSD-Optimized flushing strategy 18

19 Problems Using MySQL (beta) initial problems with I_S crash bug core dumping 192 GB memory to SSD takes 15 minutes 800 GB SSD total size uploading coredump to Oracle for analysis :-) 19

20 Fixes Workaround for crash bugs found: I_S tables: 2 concurrent reads + r/o txns = boom Stop monitoring = good Fixed in later 5.6 releases Second showstopper also fixed in new version 20

21 September 2012: Situation good

22 MySQL 5.6, autumn 2012 Excellent dev work, Oracle! Most stable beta experience, ever! Without 5.6 we would not have made it through the summer peak MySQL 5.6 beta - at the core of the business 22

23 Typical MyQL release New features have new bugs Previous features greatly improved 23

24 Late 2012: A bold plan

25 Ambitious migration goal Be able to move all production to 5.6 in less than 30 days (20 work days) Minimize network traffic Fully orchestrate the automation No human involved after 'go' decision 25

26 Planning migration 5.1 to 5.5 migration was a pain mysql_upgrade did not work for us. Create initial 5.5 slave to 5.1 master. Test with this, clone from this. Took ages to complete. With current data volume and networking situation not possible to complete in time. 26

27 Beyond puppet: Orchestration 27 Goal: To control all subsystems in a service to allow mass operations without affecting availability and QoS.

28 Beyond puppet: Orchestration 28 Goal: To control all subsystems in a service to allow mass operations without affecting availability and QoS.

29 Parts needed

30 Orchestration Central controller Service group definitions and dependencies Flexible repointing A load balancer OR dynamic repointing Faster data distribution faster than rsync, that is 30

31 Parts we have serverdb Asset and configuration management Django Application, MySQL backend defines groups drives puppet, nagios, graphite drives my.cnf generation through puppet 31

32

33 Parts we are not happy with Clone Script (/bin/bash) maintain inheritance data for debugging shovel data from clone source to destination driven by serverdb state machine uses rsync (and ssh) need not be fast, because unattended 33

34 Moving to Python Painful decision: keep legacy bash scripts, but no more bash development 'dba' command as a shell FE to.py scripts hierarchy of Python classes 34

35 Moving to Python Better reuse Better error reporting and robustness Better security Less processes, less connections Threading, Concurrency control 35

36 Fast Data Distribution

37 Shoveling data faster use "nc" python based unencrypted socket used in cloning and backups works at network speed does not scale use BitTornado modified Torrent tracker take network topology into account used in mass cloning long setup time scales just fine 37

38 Fast Data Distribution 'rsync' still the default method 'secure' default 'nc', 'torrent' provide scaling options use less CPU use less network bandwidth 38

39 Repointing

40 Old cell structure Puppet generates "db.conf" "db.conf" points to fixed databases per "handle" 40

41 Old cell structure Slow to change: puppet run required Hard to control: puppet runs eventually Error backpropagation: db failure fe error lb drops entire cell Capacity loss: error in a single db loss of complete cell 41

42 Old cell structure Goal: DBA controlled, local change temp hack, not using db.conf "Solution": iptables! 42

43 Repointing def reflect(self, destination, port): if not self._forwarding_enabled: self._enableforwarding() ret, out, err = self._run('sudo', get_os_command('iptables'), '-t', 'nat', '-A', '-p', 'tcp', '--dport', str(port), '-j', 'DNAT', '--to-destination', destination logging.info("traffic redirect status: %s, %s, %s" % ( ret, out, err )) return ret 43

44 Repointing Works. That's about the only redeeming feature it has. 44

45 Repointing (done right)

46 DB level load balancing Who is doing it? Query through all channels came back empty. What are typical implementation pitfalls?? 46

47 Solutions considered haproxy (rejected) single threaded routing traffic through haproxy process SPOF, Bottleneck 47

48 Solutions considered LVS Direct Routing Mode (rejected) Bulk traffic not going through LB Ugly ARP hack Not useable due to network topology constraints (broadcast domain) 48

49 Solutions considered haproxy redirector + client lib Modify Bookings::Db to ask haproxy for server to contact ('redirect') No traffic through LB Make connection stick to last target if haproxy is down Did run as interim solution, success. 49

50 Solutions considered haproxy as a DNS server modify haproxy to answer DNS queries FE resolves virtual db name through haproxy controlled DNS server no Bookings::Db change required also works with straight connections currently in production. 50

51 Solutions considered New 'cell free' design 51

52 Benefits Easier maintenance Automated rolling restarts possible Wrapper script as POC Enhanced capacity Single queue, multiple consumers 52

53 Towards Orchestration

54 DBA command insert commands into job queue Django + MySQL Backend (Job table) Task runner fetches jobs, executes them repoint, motd, check idle, take MySQL down, perform job start mysql, warmup, undo motd, unpoint 54

55 DBA command # dba downtime 2h 'Kris is testing' # dba clone_slave_from_dedicated_source bc43bprdb-01 OR # dba torrent_get_share --path='/mnt/clone_snapshot' # dba retrieve_root_mycnf # dba warmup 55

56 Orchestration Central queue Central error reporting LB stats from haproxy Central realtime service quality status information 56

57 Orchestration Central service quality information Group definitions from serverdb Centralized orchestration controller 57

58 Monitoring Graphite + Merlin reimplemented as etsy dashboard Icinga/Check_MK Merlin (deprecated) DBAng dashboard Controlrooms (business level monitoring) Show_errors page 58

59

60

61

62 Management POV

63 Strategy We are an agile organization. We value velocity (development speed) over all other factors. The task of operations is to enable development. 63

64 Operations vs. Incident We made database upgrades part of regular operational scripting at a scale. An upgrade is no longer an abnormal thing that is to be avoided. 64

65 Giving Management Choice Upgrade decition based on features and benefits, not on cost and manpower. "We found to be useful and stable, and would like to have it available across all of production next month." 65

66 March 2013: Still not using MySQL 5.6

67 CPU Comparison 67

68 Testing in production Take 3x the normal number of FE blades Increase LB weight in small steps Wait until load stabilizes Check "siege" latency, check error log 68

69 Load Test Results 69

70 We Are Growing! We Are Hiring!

Scaling Graphite Installations

Scaling Graphite Installations Scaling Graphite Installations Graphite basics Graphite is a web based Graphing program for time series data series plots. Written in Python Consists of multiple separate daemons Has it's own storage backend

More information

High Availability Solutions for the MariaDB and MySQL Database

High Availability Solutions for the MariaDB and MySQL Database High Availability Solutions for the MariaDB and MySQL Database 1 Introduction This paper introduces recommendations and some of the solutions used to create an availability or high availability environment

More information

MAGENTO HOSTING Progressive Server Performance Improvements

MAGENTO HOSTING Progressive Server Performance Improvements MAGENTO HOSTING Progressive Server Performance Improvements Simple Helix, LLC 4092 Memorial Parkway Ste 202 Huntsville, AL 35802 sales@simplehelix.com 1.866.963.0424 www.simplehelix.com 2 Table of Contents

More information

MySQL Enterprise Backup

MySQL Enterprise Backup MySQL Enterprise Backup Fast, Consistent, Online Backups A MySQL White Paper February, 2011 2011, Oracle Corporation and/or its affiliates Table of Contents Introduction... 3! Database Backup Terms...

More information

Managing MySQL Scale Through Consolidation

Managing MySQL Scale Through Consolidation Hello Managing MySQL Scale Through Consolidation Percona Live 04/15/15 Chris Merz, @merzdba DB Systems Architect, SolidFire Enterprise Scale MySQL Challenges Many MySQL instances (10s-100s-1000s) Often

More information

Monitoring MySQL. Kristian Köhntopp

Monitoring MySQL. Kristian Köhntopp Monitoring MySQL Kristian Köhntopp I am... Kristian Köhntopp Database architecture at a small travel agency in Amsterdam In previous lives: MySQL, web.de, NetUSE, MMC Kiel, PHP, PHPLIB, various FAQs and

More information

Performance Monitoring AlwaysOn Availability Groups. Anthony E. Nocentino aen@centinosystems.com

Performance Monitoring AlwaysOn Availability Groups. Anthony E. Nocentino aen@centinosystems.com Performance Monitoring AlwaysOn Availability Groups Anthony E. Nocentino aen@centinosystems.com Anthony E. Nocentino Consultant and Trainer Founder and President of Centino Systems Specialize in system

More information

Release Notes LS Retail Data Director 3.01.04 August 2011

Release Notes LS Retail Data Director 3.01.04 August 2011 Release Notes LS Retail Data Director 3.01.04 August 2011 Copyright 2010-2011, LS Retail. All rights reserved. All trademarks belong to their respective holders. Contents 1 Introduction... 1 1.1 What s

More information

ENTERPRISE-CLASS MONITORING SOLUTION FOR EVERYONE ALL-IN-ONE OPEN-SOURCE DISTRIBUTED MONITORING

ENTERPRISE-CLASS MONITORING SOLUTION FOR EVERYONE ALL-IN-ONE OPEN-SOURCE DISTRIBUTED MONITORING ENTERPRISE-CLASS MONITORING SOLUTION FOR EVERYONE ALL-IN-ONE OPEN-SOURCE DISTRIBUTED MONITORING 1 CONTENTS About Zabbix Software... 2 Main Functions... 3 Architecture... 4 Installation Requirements...

More information

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Use case Figure 1: Company C Architecture (Before Migration) Company C is an automobile insurance claim processing company with

More information

Multi-Datacenter Replication

Multi-Datacenter Replication www.basho.com Multi-Datacenter Replication A Technical Overview & Use Cases Table of Contents Table of Contents... 1 Introduction... 1 How It Works... 1 Default Mode...1 Advanced Mode...2 Architectural

More information

Tushar Joshi Turtle Networks Ltd

Tushar Joshi Turtle Networks Ltd MySQL Database for High Availability Web Applications Tushar Joshi Turtle Networks Ltd www.turtle.net Overview What is High Availability? Web/Network Architecture Applications MySQL Replication MySQL Clustering

More information

Why Engine Yard is better than Do it yourself

Why Engine Yard is better than Do it yourself Why Engine Yard is better than Do it yourself Comparisons between Platform as a Service and self-administered infrastructure Engine Yard www.engineyard.com sales@engineyard.com 1-866-518-9273 1-415-624-8380

More information

WebGUI Load Balancing

WebGUI Load Balancing WebGUI Load Balancing WebGUI User Conference October 5, 2005 Presented by: Len Kranendonk len@primaat.com Course Contents Introduction Example: The Royal Netherlands Football Association Scaling WebGUI

More information

1:1 NAT in ZeroShell. Requirements. Overview. Network Setup

1:1 NAT in ZeroShell. Requirements. Overview. Network Setup 1:1 NAT in ZeroShell Requirements The version of ZeroShell used for writing this document is Release 1.0.beta11. This document does not describe installing ZeroShell, it is assumed that the user already

More information

MySQL Reference Architectures for Massively Scalable Web Infrastructure

MySQL Reference Architectures for Massively Scalable Web Infrastructure MySQL Reference Architectures for Massively Scalable Web Infrastructure MySQL Best Practices for Innovating on the Web A MySQL Strategy White Paper April 2011 Table of Contents Executive Summary... 3!

More information

Using Redis as a Cache Backend in Magento

Using Redis as a Cache Backend in Magento Using Redis as a Cache Backend in Magento Written by: Alexey Samorukov Aleksandr Lozhechnik Kirill Morozov Table of Contents PROBLEMS WITH THE TWOLEVELS CACHE BACKEND CONFIRMING THE ISSUE SOLVING THE ISSUE

More information

Load Balancing McAfee Web Gateway. Deployment Guide

Load Balancing McAfee Web Gateway. Deployment Guide Load Balancing McAfee Web Gateway Deployment Guide rev. 1.1.4 Copyright 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 3 Loadbalancer.org Appliances Supported...3 Loadbalancer.org

More information

Database Resilience at ISPs. High-Availability. White Paper

Database Resilience at ISPs. High-Availability. White Paper Database Resilience at ISPs High-Availability White Paper Internet Service Providers (ISPs) generally do their job very well. The commercial hosting market is segmented in a number of different ways but

More information

University of Edinburgh. Performance audit. Date: 01-07-2015. Niels van Klaveren Kasper van der Leeden Yvette Vermeer

University of Edinburgh. Performance audit. Date: 01-07-2015. Niels van Klaveren Kasper van der Leeden Yvette Vermeer University of Edinburgh Performance audit Date: 01-07-2015 By: Niels van Klaveren Kasper van der Leeden Yvette Vermeer Contents Summary... 3 Background... 4 Why... 4 Who... 4 When... 4 What... 4 How...

More information

ClusterLoad ESX Virtual Appliance quick start guide v6.3

ClusterLoad ESX Virtual Appliance quick start guide v6.3 ClusterLoad ESX Virtual Appliance quick start guide v6.3 ClusterLoad terminology...2 What are your objectives?...3 What is the difference between a one-arm and a two-arm configuration?...3 What are the

More information

Assignment 3 Firewalls

Assignment 3 Firewalls LEIC/MEIC - IST Alameda ONLY For ALAMEDA LAB equipment Network and Computer Security 2013/2014 Assignment 3 Firewalls Goal: Configure a firewall using iptables and fwbuilder. 1 Introduction This lab assignment

More information

Critical Database. Oracle Enterprise Manager Oracle Open World 2010 Presented dby Venkat Tekkalur. Prem Venkatasamy. Principal Technical Architect

Critical Database. Oracle Enterprise Manager Oracle Open World 2010 Presented dby Venkat Tekkalur. Prem Venkatasamy. Principal Technical Architect Real World Mission Critical Database Monitoring i at AT&T with Oracle Enterprise Manager Oracle Open World 2010 Presented dby Venkat Tekkalur Principal Technical Architect Prem Venkatasamy Director IT

More information

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com

Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com Using MySQL for Big Data Advantage Integrate for Insight Sastry Vedantam sastry.vedantam@oracle.com Agenda The rise of Big Data & Hadoop MySQL in the Big Data Lifecycle MySQL Solutions for Big Data Q&A

More information

WEBAPP PATTERN FOR APACHE TOMCAT - USER GUIDE

WEBAPP PATTERN FOR APACHE TOMCAT - USER GUIDE WEBAPP PATTERN FOR APACHE TOMCAT - USER GUIDE Contents 1. Pattern Overview... 3 Features 3 Getting started with the Web Application Pattern... 3 Accepting the Web Application Pattern license agreement...

More information

Load Balancing Web Proxies Load Balancing Web Filters Load Balancing Web Gateways. Deployment Guide

Load Balancing Web Proxies Load Balancing Web Filters Load Balancing Web Gateways. Deployment Guide Load Balancing Web Proxies Load Balancing Web Filters Load Balancing Web Gateways Deployment Guide rev. 1.4.9 Copyright 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 3 Appliances

More information

VPS Cloud Hosting. Call (02) 9934 0594

VPS Cloud Hosting. Call (02) 9934 0594 Hosting With a clustered hosting platform, auto-failover network, built on IBM Blade servers and completely managed by VMware - Netregistry s hosting is a great solution for resellers, creative agencies

More information

Cloud Based Application Architectures using Smart Computing

Cloud Based Application Architectures using Smart Computing Cloud Based Application Architectures using Smart Computing How to Use this Guide Joyent Smart Technology represents a sophisticated evolution in cloud computing infrastructure. Most cloud computing products

More information

Booking.com: Evolution of MySQL System Design. Nicolai Plum <nicolai.plum@booking.com>

Booking.com: Evolution of MySQL System Design. Nicolai Plum <nicolai.plum@booking.com> Booking.com: Evolution of MySQL System Design Nicolai Plum Booking.com 2 Founded in 1996 as bookingsportal.nl Early days Purchased by Priceline.com Inc in 2005 Became Booking.com

More information

Dave Stokes MySQL Community Manager

Dave Stokes MySQL Community Manager The Proper Care and Feeding of a MySQL Server for Busy Linux Admins Dave Stokes MySQL Community Manager Email: David.Stokes@Oracle.com Twiter: @Stoker Slides: slideshare.net/davidmstokes Safe Harbor Agreement

More information

Administration of Soft IOCs under Linux

Administration of Soft IOCs under Linux Administration of Soft IOCs under Linux Ralph Lange (BESSY) Objectives BESSY II is still on R 3.13.9 The MLS Control System (based on 3.14) uses soft (host based) IOCs in production for the first time

More information

Kickstart & Puppet @ Booking. Kristian Köhntopp, booking.com

Kickstart & Puppet @ Booking. Kristian Köhntopp, booking.com Kickstart & Puppet @ Booking Kristian Köhntopp, booking.com What Booking does Facilitates Hotel Room Bookings between Travelers and Hotels. Just that. Booking Data Hotel Base Data, Brochures, Reviews &

More information

TFE listener architecture. Matt Klein, Staff Software Engineer Twitter Front End

TFE listener architecture. Matt Klein, Staff Software Engineer Twitter Front End TFE listener architecture Matt Klein, Staff Software Engineer Twitter Front End Agenda TFE architecture overview TSA architecture overview TSA hot restart Future plans Q&A TFE architecture overview Listener:

More information

Storage and Disaster Recovery

Storage and Disaster Recovery Storage and Disaster Recovery Matt Tavis Principal Solutions Architect The Business Continuity Continuum High Data Backup Disaster Recovery High, Storage Backup and Disaster Recovery form a continuum of

More information

Scaling Analysis Services in the Cloud

Scaling Analysis Services in the Cloud Our Sponsors Scaling Analysis Services in the Cloud by Gerhard Brückl gerhard@gbrueckl.at blog.gbrueckl.at About me Gerhard Brückl Working with Microsoft BI since 2006 Windows Azure / Cloud since 2013

More information

High Performance MySQL Choices in Amazon Web Services: Beyond RDS. Andrew Shieh, SmugMug Operations shandrew @ smugmug.

High Performance MySQL Choices in Amazon Web Services: Beyond RDS. Andrew Shieh, SmugMug Operations shandrew @ smugmug. High Performance MySQL Choices in Amazon Web Services: Beyond RDS Andrew Shieh, SmugMug Operations shandrew @ smugmug.com April 15, 2015 Agenda 2 All about AWS Current RDS alternatives Cloud failures ->

More information

Ensuring scalability and performance with Drupal as your audience grows

Ensuring scalability and performance with Drupal as your audience grows Drupal performance and scalability Ensuring scalability and performance with Drupal as your audience grows Presented by Jon Anthony Bounty.com Northern and Shell (OK! Magazine etc) Drupal.org/project/

More information

Maintaining Non-Stop Services with Multi Layer Monitoring

Maintaining Non-Stop Services with Multi Layer Monitoring Maintaining Non-Stop Services with Multi Layer Monitoring Lahav Savir System Architect and CEO of Emind Systems lahavs@emindsys.com www.emindsys.com The approach Non-stop applications can t leave on their

More information

Load Balancing VMware Horizon View. Deployment Guide

Load Balancing VMware Horizon View. Deployment Guide Load Balancing VMware Horizon View Deployment Guide v1.1.0 Copyright 2014 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 4 Appliances Supported... 4 VMware Horizon View Versions Supported...4

More information

A Quick Start Guide to Backup Technologies

A Quick Start Guide to Backup Technologies A Quick Start Guide to Backup Technologies Objective Of This Guide... 3 Backup Types... 4 Logical Backups... 4 Physical Backups... 4 Snapshot Backups... 5 Logical (Data dump) vs. Physical (Raw) backups...

More information

Hosting Solutions Made Simple. Managed Services - Overview and Pricing

Hosting Solutions Made Simple. Managed Services - Overview and Pricing Hosting Solutions Made Simple Managed Services - Overview and Pricing NETRACKservers Internet Security Package: NETRACKservers's Internet Security Package is an ideal security service for business that

More information

HP OO 10.X - SiteScope Monitoring Templates

HP OO 10.X - SiteScope Monitoring Templates HP OO Community Guides HP OO 10.X - SiteScope Monitoring Templates As with any application continuous automated monitoring is key. Monitoring is important in order to quickly identify potential issues,

More information

Mark Bennett. Search and the Virtual Machine

Mark Bennett. Search and the Virtual Machine Mark Bennett Search and the Virtual Machine Agenda Intro / Business Drivers What to do with Search + Virtual What Makes Search Fast (or Slow!) Virtual Platforms Test Results Trends / Wrap Up / Q & A Business

More information

What is included in the ATRC server support

What is included in the ATRC server support Linux Server Support Services What is included in the ATRC server support Installation Installation of any ATRC Supported distribution Compatibility with client hardware. Hardware Configuration Recommendations

More information

Real Time Replication in the Real World

Real Time Replication in the Real World Real Time Replication in the Real World Richard E. Baum and C. Thomas Tyler Perforce Software, Inc. Doc Revision 1.1 Table of Contents 1 Overview... 1 2 Definitions... 1 3 Solutions Involving Replication...

More information

SLIDE 1 www.bitmicro.com. Previous Next Exit

SLIDE 1 www.bitmicro.com. Previous Next Exit SLIDE 1 MAXio All Flash Storage Array Popular Applications MAXio N1A6 SLIDE 2 MAXio All Flash Storage Array Use Cases High speed centralized storage for IO intensive applications email, OLTP, databases

More information

Scalability of web applications. CSCI 470: Web Science Keith Vertanen

Scalability of web applications. CSCI 470: Web Science Keith Vertanen Scalability of web applications CSCI 470: Web Science Keith Vertanen Scalability questions Overview What's important in order to build scalable web sites? High availability vs. load balancing Approaches

More information

Alfresco Enterprise on AWS: Reference Architecture

Alfresco Enterprise on AWS: Reference Architecture Alfresco Enterprise on AWS: Reference Architecture October 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 13 Abstract Amazon Web Services (AWS)

More information

Load Balancing Barracuda Web Filter. Deployment Guide

Load Balancing Barracuda Web Filter. Deployment Guide Load Balancing Barracuda Web Filter Deployment Guide rev. 1.1.4 Copyright 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 3 Loadbalancer.org Appliances Supported...3 Loadbalancer.org

More information

High Availability Essentials

High Availability Essentials High Availability Essentials Introduction Ascent Capture s High Availability Support feature consists of a number of independent components that, when deployed in a highly available computer system, result

More information

What s new in Hyper-V 2012 R2

What s new in Hyper-V 2012 R2 What s new in Hyper-V 2012 R2 Carsten Rachfahl MVP Virtual Machine Rachfahl IT-Solutions GmbH & Co KG www.hyper-v-server.de Thomas Maurer Cloud Architect & MVP itnetx gmbh www.thomasmaurer.ch Before Windows

More information

Load Balancing VMware Horizon View. Deployment Guide

Load Balancing VMware Horizon View. Deployment Guide Load Balancing VMware Horizon View Deployment Guide rev. 1.2.6 Copyright 2002 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide...4 Loadbalancer.org Appliances Supported...4 Loadbalancer.org

More information

Techniques for implementing & running robust and reliable DB-centric Grid Applications

Techniques for implementing & running robust and reliable DB-centric Grid Applications Techniques for implementing & running robust and reliable DB-centric Grid Applications International Symposium on Grid Computing 2008 11 April 2008 Miguel Anjo, CERN - Physics Databases Outline Robust

More information

OTM in the Cloud. Ryan Haney

OTM in the Cloud. Ryan Haney OTM in the Cloud Ryan Haney The Cloud The Cloud is a set of services and technologies that delivers real-time and ondemand computing resources Software as a Service (SaaS) delivers preconfigured applications,

More information

Loadbalancer.org Appliance Setup v5.9

Loadbalancer.org Appliance Setup v5.9 Loadbalancer.org Appliance Setup v5.9 This document covers the basic steps required to setup the Loadbalancer.org appliances. Please pay careful attention to the section on the ARP problem for your real

More information

The Future of PostgreSQL High Availability Robert Hodges - Continuent, Inc. Simon Riggs - 2ndQuadrant

The Future of PostgreSQL High Availability Robert Hodges - Continuent, Inc. Simon Riggs - 2ndQuadrant The Future of PostgreSQL High Availability Robert Hodges - Continuent, Inc. Simon Riggs - 2ndQuadrant Agenda / Introductions / Framing the High Availability (HA) Problem / Hot Standby + Log Streaming /

More information

ZEN LOAD BALANCER EE v3.04 DATASHEET The Load Balancing made easy

ZEN LOAD BALANCER EE v3.04 DATASHEET The Load Balancing made easy ZEN LOAD BALANCER EE v3.04 DATASHEET The Load Balancing made easy OVERVIEW The global communication and the continuous growth of services provided through the Internet or local infrastructure require to

More information

Module 14: Scalability and High Availability

Module 14: Scalability and High Availability Module 14: Scalability and High Availability Overview Key high availability features available in Oracle and SQL Server Key scalability features available in Oracle and SQL Server High Availability High

More information

WHO IS CONSTANT CONTACT? A snapshot view of an incredible company

WHO IS CONSTANT CONTACT? A snapshot view of an incredible company Zero Downtime Upgrades Using Q-REP at Constant Contact, Inc. 2027A Dan Berry, Constant Contact Sankar Padhi, Constant Contact 0 AGENDA Who is Constant Contact, Inc Legacy Architecture Database Transactions

More information

Rapid Access Cloud: Se1ng up a Proxy Host

Rapid Access Cloud: Se1ng up a Proxy Host Rapid Access Cloud: Se1ng up a Proxy Host Rapid Access Cloud: Se1ng up a Proxy Host Prerequisites Set up security groups The Proxy Security Group The Internal Security Group Launch your internal instances

More information

ZEN LOAD BALANCER EE v3.02 DATASHEET The Load Balancing made easy

ZEN LOAD BALANCER EE v3.02 DATASHEET The Load Balancing made easy ZEN LOAD BALANCER EE v3.02 DATASHEET The Load Balancing made easy OVERVIEW The global communication and the continuous growth of services provided through the Internet or local infrastructure require to

More information

Windows Server Performance Monitoring

Windows Server Performance Monitoring Spot server problems before they are noticed The system s really slow today! How often have you heard that? Finding the solution isn t so easy. The obvious questions to ask are why is it running slowly

More information

Smoothwall Web Filter Deployment Guide

Smoothwall Web Filter Deployment Guide Smoothwall Web Filter Deployment Guide v1.0.7 Copyright 2013 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 3 Loadbalancer.org Appliances Supported...3 Loadbalancer.org Software Versions

More information

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011

SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications. Jürgen Primsch, SAP AG July 2011 SAP HANA - Main Memory Technology: A Challenge for Development of Business Applications Jürgen Primsch, SAP AG July 2011 Why In-Memory? Information at the Speed of Thought Imagine access to business data,

More information

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014

How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 How to Deploy OpenStack on TH-2 Supercomputer Yusong Tan, Bao Li National Supercomputing Center in Guangzhou April 10, 2014 2014 年 云 计 算 效 率 与 能 耗 暨 第 一 届 国 际 云 计 算 咨 询 委 员 会 中 国 高 峰 论 坛 Contents Background

More information

Load Balancing Microsoft Terminal Services. Deployment Guide

Load Balancing Microsoft Terminal Services. Deployment Guide Load Balancing Microsoft Terminal Services Deployment Guide rev. 1.5.7 Copyright 2002 2016 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 4 Loadbalancer.org Appliances Supported... 4 Loadbalancer.org

More information

Table of Contents. Overview... 1 Introduction... 2 Common Architectures... 3. Technical Challenges with Magento... 6. ChinaNetCloud's Experience...

Table of Contents. Overview... 1 Introduction... 2 Common Architectures... 3. Technical Challenges with Magento... 6. ChinaNetCloud's Experience... Table of Contents Overview... 1 Introduction... 2 Common Architectures... 3 Simple System... 3 Highly Available System... 4 Large Scale High-Performance System... 5 Technical Challenges with Magento...

More information

Load Balancing Bloxx Web Filter. Deployment Guide

Load Balancing Bloxx Web Filter. Deployment Guide Load Balancing Bloxx Web Filter Deployment Guide rev. 1.1.8 Copyright 2002 2016 Loadbalancer.org, Inc. 1 Table of Contents About this Guide...4 Loadbalancer.org Appliances Supported...4 Loadbalancer.org

More information

Load Balancing Trend Micro InterScan Web Gateway

Load Balancing Trend Micro InterScan Web Gateway Load Balancing Trend Micro InterScan Web Gateway Deployment Guide rev. 1.1.7 Copyright 2002 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide... 3 Loadbalancer.org Appliances Supported...

More information

Red Hat Cluster Suite

Red Hat Cluster Suite Red Hat Cluster Suite HP User Society / DECUS 17. Mai 2006 Joachim Schröder Red Hat GmbH Two Key Industry Trends Clustering (scale-out) is happening 20% of all servers shipped will be clustered by 2006.

More information

1 Basic Configuration of Cisco 2600 Router. Basic Configuration Cisco 2600 Router

1 Basic Configuration of Cisco 2600 Router. Basic Configuration Cisco 2600 Router 1 Basic Configuration of Cisco 2600 Router Basic Configuration Cisco 2600 Router I decided to incorporate the Cisco 2600 into my previously designed network. This would give me two seperate broadcast domains

More information

Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL

Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL Architectures Haute-Dispo Joffrey MICHAÏE Consultant MySQL 04.20111 High Availability with MySQL Higher Availability Shared nothing distributed cluster with MySQL Cluster Storage snapshots for disaster

More information

Feature Comparison. Windows Server 2008 R2 Hyper-V and Windows Server 2012 Hyper-V

Feature Comparison. Windows Server 2008 R2 Hyper-V and Windows Server 2012 Hyper-V Comparison and Contents Introduction... 4 More Secure Multitenancy... 5 Flexible Infrastructure... 9 Scale, Performance, and Density... 13 High Availability... 18 Processor and Memory Support... 24 Network...

More information

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module June, 2015 WHITE PAPER Contents Advantages of IBM SoftLayer and RackWare Together... 4 Relationship between

More information

MySQL Enterprise Monitor

MySQL Enterprise Monitor MySQL Enterprise Monitor Lynn Ferrante Principal Sales Consultant 1 Program Agenda MySQL Enterprise Monitor Overview Architecture Roles Demo 2 Overview 3 MySQL Enterprise Edition Highest Levels of Security,

More information

Introduction to Replication. Brian Bowman Sr. Solutions Engineer

Introduction to Replication. Brian Bowman Sr. Solutions Engineer Introduction to Replication Brian Bowman Sr. Solutions Engineer bowman@progress.com Introduction to Replication Replication Overview OpenEdge Replication Process Failover, Secondary Replication and Failback

More information

Why Not Oracle Standard Edition? A Dbvisit White Paper By Anton Els

Why Not Oracle Standard Edition? A Dbvisit White Paper By Anton Els Why Not Oracle Standard Edition? A Dbvisit White Paper By Anton Els Copyright 2011-2013 Dbvisit Software Limited. All Rights Reserved Nov 2013 Executive Summary... 3 Target Audience... 3 Introduction...

More information

CDL Database Administration Framework v. 1.1 August 2009

CDL Database Administration Framework v. 1.1 August 2009 CDL Database Administration Framework v. 1.1 August 2009 Contents Purpose of the framework 1 Terminology 1 Database administration conventions 1 Summary Table 1 Database design, location, and naming 2

More information

PostgreSQL Backup Strategies

PostgreSQL Backup Strategies PostgreSQL Backup Strategies Austin PGDay 2012 Austin, TX Magnus Hagander magnus@hagander.net PRODUCTS CONSULTING APPLICATION MANAGEMENT IT OPERATIONS SUPPORT TRAINING Replication! But I have replication!

More information

Real-time reporting at 10,000 inserts per second. Wesley Biggs CTO 25 October 2011 Percona Live

Real-time reporting at 10,000 inserts per second. Wesley Biggs CTO 25 October 2011 Percona Live Real-time reporting at 10,000 inserts per second Wesley Biggs CTO 25 October 2011 Percona Live Agenda 1. Who we are, what we do, and (maybe) why we do it 2. Solution architecture and evolution 3. Top 5

More information

The syslog-ng Store Box 3 F2

The syslog-ng Store Box 3 F2 The syslog-ng Store Box 3 F2 PRODUCT DESCRIPTION Copyright 2000-2014 BalaBit IT Security All rights reserved. www.balabit.com Introduction The syslog-ng Store Box (SSB) is a high-reliability and high-performance

More information

MySQL backups: strategy, tools, recovery scenarios. Akshay Suryawanshi Roman Vynar

MySQL backups: strategy, tools, recovery scenarios. Akshay Suryawanshi Roman Vynar MySQL backups: strategy, tools, recovery scenarios Akshay Suryawanshi Roman Vynar April 15, 2015 Introduction 2 This is a brief talk on Backups for MySQL, where we will cover basics of backing up MySQL,

More information

Web Load Balancing on a Budget

Web Load Balancing on a Budget Web Load Balancing on a Budget Pain Hosting 60+ websites Single web server Redundant subsystems (disk, power) SPOF Inconvenient maintenance windows Clients MY TEAM! Scope Simple. Availability. Minimize/mitigate

More information

MySQL: Cloud vs Bare Metal, Performance and Reliability

MySQL: Cloud vs Bare Metal, Performance and Reliability MySQL: Cloud vs Bare Metal, Performance and Reliability Los Angeles MySQL Meetup Vladimir Fedorkov, March 31, 2014 Let s meet each other Performance geek All kinds MySQL and some Sphinx Working for Blackbird

More information

Load Balancing Sophos Web Gateway. Deployment Guide

Load Balancing Sophos Web Gateway. Deployment Guide Load Balancing Sophos Web Gateway Deployment Guide rev. 1.0.9 Copyright 2002 2015 Loadbalancer.org, Inc. 1 Table of Contents About this Guide...3 Loadbalancer.org Appliances Supported...3 Loadbalancer.org

More information

Snapt Redundancy Manual

Snapt Redundancy Manual Snapt Redundancy Manual Version 2.0 p. 1 Contents Chapter 1: Introduction... 3 Installation... 3 Chapter 2: Settings... 4 Chapter 3: Server Management... 6 Chapter 4: Virtual IP Management... 7 Chapter

More information

HP ProLiant BL660c Gen9 and Microsoft SQL Server 2014 technical brief

HP ProLiant BL660c Gen9 and Microsoft SQL Server 2014 technical brief Technical white paper HP ProLiant BL660c Gen9 and Microsoft SQL Server 2014 technical brief Scale-up your Microsoft SQL Server environment to new heights Table of contents Executive summary... 2 Introduction...

More information

InterWorx Clustering Guide. by InterWorx LLC

InterWorx Clustering Guide. by InterWorx LLC InterWorx Clustering Guide by InterWorx LLC Contents 1 What Is Clustering? 3 1.1 What Does Clustering Do? What Doesn t It Do?............................ 3 1.2 Why Cluster?...............................................

More information

Large Scale file storage with MogileFS. Stuart Teasdale Lead System Administrator we7 Ltd

Large Scale file storage with MogileFS. Stuart Teasdale Lead System Administrator we7 Ltd Large Scale file storage with MogileFS Stuart Teasdale Lead System Administrator we7 Ltd About We7 A web based streaming music service 6.5 million tracks 192kbps and 320kbps mp3s Sending over a gigabit

More information

PLUMgrid Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure

PLUMgrid Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure Toolbox: Tools to Install, Operate and Monitor Your Virtual Network Infrastructure Introduction The concept of Virtual Networking Infrastructure (VNI) is disrupting the networking space and is enabling

More information

Analyzing Big Data with Splunk A Cost Effective Storage Architecture and Solution

Analyzing Big Data with Splunk A Cost Effective Storage Architecture and Solution Analyzing Big Data with Splunk A Cost Effective Storage Architecture and Solution Jonathan Halstuch, COO, RackTop Systems JHalstuch@racktopsystems.com Big Data Invasion We hear so much on Big Data and

More information

BASICS OF SCALING: LOAD BALANCERS

BASICS OF SCALING: LOAD BALANCERS BASICS OF SCALING: LOAD BALANCERS Lately, I ve been doing a lot of work on systems that require a high degree of scalability to handle large traffic spikes. This has led to a lot of questions from friends

More information

Siemens SAP Cloud - Infrastructure as a Service

Siemens SAP Cloud - Infrastructure as a Service Siemens SAP Cloud - Infrastructure as a Service Siemens Value Proposition April 2010 Agenda 1 Challenges with traditional infrastructure setup 2 Siemens SAP Cloud solution 3 Benefits of Siemens SAP Cloud

More information

Load balancing MySQL with HaProxy. Peter Boros Consultant @ Percona 4/23/13 Santa Clara, CA

Load balancing MySQL with HaProxy. Peter Boros Consultant @ Percona 4/23/13 Santa Clara, CA Load balancing MySQL with HaProxy Peter Boros Consultant @ Percona 4/23/13 Santa Clara, CA Agenda What is HaProxy HaProxy configuration Load balancing topologies Checks Load balancing Percona XtraDB Cluster

More information

Avoiding Pain Running MySQL in the Cloud

Avoiding Pain Running MySQL in the Cloud ! Avoiding Pain Running MySQL in the Cloud Neil Armitage whoami DBA Oracle/Mainframes/MySQL (25 Years) Deployment Engineer @ Continuent 1 or 2 Customer Deployments/Week On Premise or Cloud deployments

More information

Linux Server Support by Applied Technology Research Center. Proxy Server Configuration

Linux Server Support by Applied Technology Research Center. Proxy Server Configuration Linux Server Support by Applied Technology Research Center Proxy Server Configuration We configure squid for your LAN. Including transparent for HTTP and proxy for HTTPS. We also provide basic training

More information

Jive and High-Availability

Jive and High-Availability Jive and High-Availability TOC 2 Contents Jive and High-Availability... 3 Supported High-Availability Jive Configurations...3 Designing a Single Data Center HA Configuration... 3 Designing a Multiple Data

More information

MySQL Administration and Management Essentials

MySQL Administration and Management Essentials MySQL Administration and Management Essentials Craig Sylvester MySQL Sales Consultant 1 Safe Harbor Statement The following is intended to outline our general product direction. It

More information

Agenda. Enterprise Application Performance Factors. Current form of Enterprise Applications. Factors to Application Performance.

Agenda. Enterprise Application Performance Factors. Current form of Enterprise Applications. Factors to Application Performance. Agenda Enterprise Performance Factors Overall Enterprise Performance Factors Best Practice for generic Enterprise Best Practice for 3-tiers Enterprise Hardware Load Balancer Basic Unix Tuning Performance

More information

How to create a load testing environment for your web apps using open source tools by Sukrit Dhandhania

How to create a load testing environment for your web apps using open source tools by Sukrit Dhandhania How to create a load testing environment for your web apps using open source tools by Sukrit Dhandhania Open source load testing for web putting demand on an application and measuring its response see

More information