Products for the registry databases and preparation for the disaster recovery

Size: px

Start display at page:

Download "Products for the registry databases and preparation for the disaster recovery"

Felicia Atkinson
8 years ago
Views:

2 Introduction To introduce products and preparation for Disaster Recovery To measure performance of products in lab To evaluate results for Disaster Recovery Copyright 2013 Japan Registry Services Co., Ltd. 2

products in lab To evaluate results for Disaster

3 Our primary and secondary sites Secondary site Primary site Properties about network: WAN (Leased line) RTT around 15ms (actual measurement) Bandwidth 100Mbps over 500km distance Copyright 2013 Japan Registry Services Co., Ltd. 3

around 15ms (actual measurement) Bandwidth 100Mbps over

4 Important things for Disaster Recovery We're considering so far: High throughput from many clients Propagation delay time in replication through databases: less than 1 sec Time to complete failover to each slave database: less than 1 min Copyright 2013 Japan Registry Services Co., Ltd. 4

through databases: less than 1 sec Time to complete failover to each

5 Primary site Architecture supporting failover and replication Secondary site Clients Clients R/W R/O pgpool-ii Load balancing pgpool-ii / Handling failover R/W R/O R/O R/O Master 1 Synchronous Replication WAL (Write Ahead Log) 2 Asynchronous Replication 3 Copyright 2013 Japan Registry Services Co., Ltd. 5

failover R/W R/O R/O R/O Master 1 Synchronous Replication WAL (Write Ahead

6 Used products for the disaster recovery PostgreSQL for registry databases Open source SQL database Streaming replication for data replication Built-in replication feature of PostgreSQL pgpool-ii for handling failover Open source middleware for PostgreSQL Handling detection of failure on Master and failover to Copyright 2013 Japan Registry Services Co., Ltd. 6

PostgreSQL pgpool-ii for handling failover Open source middleware for PostgreSQL Handling

8 Environment in lab R/W Client (pgbench) pgpool-ii R/O Transactions loading R/O pgpool-ii R/O Master 1 WAN Emulator (Netem) delaying packet transfer 2 3 Copyright 2013 Japan Registry Services Co., Ltd. 8

9 Specification of lab test 4 virtual machines (KVM) 1 CPU, 8GB Memory Host's CPU: Intel Xeon 1.80GHz 4 cores Cent OS 6.3 PostgreSQL pgpool-ii Network between 1 and 2 RTT around 10 ms (manipulated by Netem) Bandwidth 100 Mbps Copyright 2013 Japan Registry Services Co., Ltd. 9

10 Throughput in TPS Measurements measured for each SQL with clients: 1 to 128, total transactions: 51,200 Propagation delay time since updating on Master until applying WAL on 3 measured for each write SQL calculated from PostgreSQL log files Failover time since a failure on master until failing over to each Copyright 2013 Japan Registry Services Co., Ltd. 10

on 3 measured for each write SQL calculated from PostgreSQL log files Failover time since

13 Thoughts on throughput result "SELECT" transactions scaled. Load balanced using pgpool-ii "UPDATE" and "INSERT" not scaled. Same behaviors as single database "DELETE" resulted in under 100 TPS. No index on table utilized using pgbench as: DELETE.. WHERE name = 'DOMAIN' to_char( :num, '00000' ); How can we improve DELETE? Over 500 TPS Copyright 2013 Japan Registry Services Co., Ltd. 13

Same behaviors as single database "DELETE" resulted in under 100 TPS.

14 Propagation delay time WAL Master 1 Netem 2 3 updating time Propagation delay time applying time INSERT: ms UPDATE: ms DELETE: 7.40 ms* less than 1 sec * too fast, packets not delayed by Netem Copyright 2013 Japan Registry Services Co., Ltd. 14

15 Failover handling time 1)fallen down Case1: 2) detect pgpool-ii 3) fail over 1)fallen down Case2: 2) detect pgpool-ii 3) fail over Master 1 Netem 2 3 Case1: Master => 1 Case2: 2 => 3 less than 1 min Copyright 2013 Japan Registry Services Co., Ltd. 15

pgpool-ii 3) fail over Master 1 Netem 2 3 Case1: Master => 1

16 Configuration of pgpool-ii Script to promote to (new) Master: Remote login to by SSH Changing configuration file of PostgreSQL Invoking "pg_ctl" to restart Example of pgpool.conf : failover_command = '/path/to/failover.sh %M %m %H %D' %M: Old Master Number %H: New Master Hostname %m: New Master Number %D: Data Dir Copyright 2013 Japan Registry Services Co., Ltd. 16

conf : failover_command = '/path/to/failover.

17 Conclusion and Future Work Satisfied: SELECT, INSERT, and UPDATE Throughput: over 500 TPS Propagation delay time: less than 1 sec Failover handling in less than 1 min Not satisfied: DELETE Throughput: under 100 TPS Propagation delay time: too fast (< 10 ms) Future Work DELETE: improving and re-measuring Copyright 2013 Japan Registry Services Co., Ltd. 17

satisfied: DELETE Throughput: under 100 TPS Propagation delay time: too fast (< 10 ms)

18 What about sharing good ideas? pgbench How can it utilize indexes when DELETing? PostgreSQL + pgpool-ii Please share good ideas and good practices, e.g. best configuration on pgpool-ii Disaster Recovery products Do you have any other good products? How to switch over to another site Manually? Automatically? Copyright 2013 Japan Registry Services Co., Ltd. 18

Remote Copy Technology of ETERNUS6000 and ETERNUS3000 Disk Arrays

Remote Copy Technology of ETERNUS6000 and ETERNUS3000 Disk Arrays V Tsutomu Akasaka (Manuscript received July 5, 2005) This paper gives an overview of a storage-system remote copy function and the implementation