Database Scalability {Patterns} / Robert Treat

Size: px

Start display at page:

Download "Database Scalability {Patterns} / Robert Treat"

Alaina Harper
8 years ago
Views:

1 Database Scalability {Patterns} / Robert Treat

2 robert treat omniti postgres oracle - mysql mssql - sqlite - nosql

3 What are Database Scalability Patterns?

4 Part Design Patterns Part Application Life-Cycle

5 Part Design Patterns Part Application Life-Cycle

6 Part Design Patterns Part Application Life-Cycle

7 Part Design Patterns Part Application Life-Cycle

8 MyFirstDatabase

9 Vertical Partitioning

10 Vertical Scaling

11 Vertical Scaling

12 Vertical Scaling

13 Federated Data Storage sharding Horizontal Partitioning Read Slaves Multi-Master Horizontal Scaling

14 Read Slaves / Master - Slave Scale Read Load

15 Read Slaves / Master - Slave Scale Read Load writes go here!

16 Read Slaves / Master - Slave Scale Read Load writes go here! reads go here (or here) (or here)

17 Read Slaves / Master - Slave Scale Read Load database writes data to slaves writes go here! slave db slave db slave db reads go here (or here) (or here)

18 Read Slaves / Master - Slave Scale Read Load app writes data everywhere writes go here! memcached memcached memcached reads go here (or here) (or here)

19 Typically Full Copy of Data On Each Node Asynchronous Read Slaves / Master - Slave Scale Read Load

20 Read Slaves / Master - Slave Scale Read Load Typically Full Copy of Data On Each Node Asynchronous Consider Partial Copy Synchronous Don t use a RDBMS?

21 Read Slaves / Master - Slave Scale Read Load Typically Full Copy of Data On Each Node Asynchronous Consider Partial Copy Synchronous Don t use a RDBMS? Requires Application Changes easy

22 Scaling Writes not easy

23 Multi-Master many different ways to implement this, few that actually work in production

24 Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data

25 Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn t reduce i/o

26 Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn t reduce i/o failover solution not a scalability solution

27 Horizontal Partitioning sharding SOA

28 Horizontal Partitioning sharding SOA

29 Horizontal Partitioning Divide schema by job operations Move each piece to own server Duplicate some data as needed

30 Horizontal Partitioning Divide schema by job operations Move each piece to own server Duplicate some data as needed items forums users

31 Horizontal Partitioning Divide schema by job operations Move each piece to own server Duplicate some data as needed items You must separate dependencies in the app code first! users forums

32 Horizontal Partitioning Divide schema by job operations Move each piece to own server Duplicate some data as needed items You must separate dependencies in the app code first! users Each node is a new instance of vertical scaling forums

33 Horizontal Scaling data split across servers based on algorithm data dropped into buckets (multiple?) app magic hash algorithm

34 Horizontal Scaling data split across servers based on algorithm data dropped into buckets (multiple?) magic hash algorithm app

35 Horizontal Scaling data split across servers based on algorithm data dropped into buckets (multiple?) someone must keep track of data, and provide lookup services magic hash algorithm app

36 Universal Truths of Scaling Databases Vertical Scalability is Helpful for Every Pattern Even in a horizontally scaled, fully distributed database, the number of nodes needed is affected by vertical scalability

37 Universal Truths of Scaling Databases New Nodes Are Never Free Add points of failure Add management costs Add complexity to architecture Add complexity to your app code

38 MyFirstDB V Vertical Partitioning V Vertical Scaling V Read Slaves V Horizontal Partitioning

39 tips plan for layered data sources read / write connections in code use schemas to separate services

40 THANKS! more: omniti.com/surge (scalability conference)

F1: A Distributed SQL Database That Scales. Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013

F1: A Distributed SQL Database That Scales Presentation by: Alex Degtiar (adegtiar@cmu.edu) 15-799 10/21/2013 What is F1? Distributed relational database Built to replace sharded MySQL back-end of AdWords