Linas Virbalas Continuent, Inc.

Transcription

1 Linas Virbalas Continuent, Inc.

2 / Introductions / What is Tungsten? / Architecture of a Rule Based Management Framework for Database Clusters / Demo of Business Rules in Operation / Business Rules in Source Code / Questions and Comments

3

4 / Our Mission: Continuent is the data reliability company that keeps business in business / Our Solution: Continuent Tungsten (Master/Slave Database Replication) / Our Value: Enterprise class data management Open source cost Integrated solution / Our Technical Expertise Database replication Database cluster management Application connectivity Software-as-a-Service (SaaS)

5

6 What Is Tungsten? / Tungsten implements master/slave clusters to: Protect data Maintain high availability Improve resource utilization Raise performance / Install and set up in a few minutes / Integrated backup/restore and data integrity checks / Efficient failover operations / Distributed, rule-driven management / No/minimal application changes / Highly pluggable / No specialized hardware requirements

7 What s Inside Tungsten? / Replication - Making copies Tungsten Replicator -- Database-neutral, platform independent master/slave replication / Connectivity -- Finding databases Tungsten Connector -- Fast MySQL/PostgreSQL client to JDBC proxying Tungsten SQL Router --JDBC wrapper for high-performance and transparent failover, load-balancing, and partitioning (no proxy required) / Management -- Administering the database Tungsten Manager -- Distributed administration with autonomic, rule-based configuration and no single point of failure Tungsten Monitor -- Track resource status and database liveness

8 Application Server Application Server SQL Router/ Connector Manager SQL Router/ Connector Manager Replicator Replicator Monitor Monitor Manager Manager Management Client Management Client

9 Multiple Routes to Databases Java App Server Tungsten SQL Router PostgreSQL JDBC Driver PHP Application libpq.a Tungsten Connector Tungsten Cluster

10 Tungsten on Open Source Databases / MySQL 5.0/5.1 Read and replicate from MySQL binlog Works with Oracle, MySQL and look-alikes (Percona, MariaDB) / PostgreSQL < 9 Warm Standby + WAL Shipping Good basic availability/fast failover Slaves open up for reads only after failover / PostgreSQL >= 9 Hot Standby + Streaming Replication Slaves opened up for reads = Tungsten scaling facilities work Add Streaming Replication = minimal delay in replicating data / Sometime in the future: Read PG logs directly!

11 Tungsten Added Value / 15 minute cluster installation / Single commands to: View cluster status Provision a new standby Confirm liveness of replication Switch servers safely for maintenance Failover a dead server to most current replica / Automatic discovery of new replicas / Automatic failover when databases fail / Simple procedures for provisioning / Transparent application routing / Easy scaling

12

13 Distributed Rule-Based Management Admin Client Manager Manager (Coordinator) Admin Client Admin Client Manager

14 Group Communications in Clustering / Tungsten Manager processes connect to each other via Group Communications protocol / Group Communications provide essential features for running multi-node environments: Membership detection Notification about joined/left/crashed members Reliable message broadcast (lost messages retransmitted) Ordered message broadcast (all members receive messages in the same order) / Different transport protocols can be used IP Multicast for open local networks Peer to Peer TCP for firewalled local networks Centralized Gossip Server for Cloud environments

15 What is a Rule? / Trivial Rule: / Example (pseudo code):

16 What is a Rule Engine?

17 How does the Rule Engine Work? Application with a Rule Engine (Tungsten Manager)

18 How does the Rule Engine get its Facts? Tungsten Monitor Tungsten Replicator Tungsten Manager

19 Who is The Coordinator? / Only one Coordinator for a group (cluster) Only one Manager must act on important events like failures Having more than one Coordinator would lead to race conditions and eventually chaotic cluster behavior / Tungsten chooses Coordinator from the Group Communications Coordinator is the oldest member of the group New members handshake with the Coordinator when joining Upon Coordinator death, GC automatically promotes a new one

20

21

22 The Rules are Layered / Lower level rules react on events from Database servers Replicators Managers / Outcome: the state change of a concept data source / Higher level rules react on Data source state changes / Outcome: actual operations to the cluster components (eg. promote slave to a master)

23 Scenario: Database Crashes / Failover is handled by a controlled chain reaction of rules rather than a single rule: 1. "DETECT AND FENCE FAILED DATASERVER" rule detects state transition of the master DB from ONLINE to STOPPED It then 'fences' the failure by setting the data source of the stopped DB server to FAILED 2. "AUTOMATIC FAILOVER" rule fires, because DataSource(state == ResourceState.FAILED && role == "master ) It then calls a method in the ClusterManagementHelper to do the actual failover, after which it refreshes the data source states i If the data source was of a slave, it would stay in the FAILED state and no further action would take place

24

25

26 Conclusion / Business Rules are a very powerful and flexible tool for automating otherwise complex tasks / They are especially useful in implementing automated DB clusters / A robust top layer connecting all the components underneath into a seamless experience

27

28

29 HQ and Americas 560 S. Winchester Blvd., Suite 500 San Jose, CA Tel (866) Fax (408) EMEA and APAC Lars Sonckin kaari Espoo, Finland Tel Fax sales@continuent.com, linas.virbalas@continuent.com Continuent Web Site: