Benchmarking the Availability and Fault Tolerance of Cassandra Marten Rosselli, Raik Niemann, Todor Ivanov, Karsten Tolle, Roberto V. Zicari Goethe-University Frankfurt, Germany Frankfurt Big Data Lab www.bigdata.uni-frankfurt.de 6 th Workshop on Big Data Benchmarking 2015 June 16 th 17 th, Toronto, Canada
Contact Marten Rosselli Goethe-University Frankfurt, Germany Frankfurt Big Data Lab rosselli@dbis.cs.uni-frankfurt.de www.bigdata.uni-frankfurt.de Accenture, Germany Accenture Digital marten.rosselli@accenture.com www.accenture.com - 2 -
Agenda Motivation Approach Background Cassandra and YCSB Setup and Configuration Experimental Results Conclusions and Outlook - 3 -
Motivation With each additional machine in a cluster the likelihood for a hardware failure increases. How is a Cassandra cluster impacted by a machine failure? Cluster remains stable? Impact on the response time for single user? Impact on system throughput? Influence of different operation types on the performance? Cassandra a good choice for high availability use cases? - 4 -
Approach Prepare Cassandra/OS: Drop page cache, remove old datasets, (re-)load dataset, rebalance cluster YCSB 1 -Workload Execution (Ramp-Up time): 100 seconds YCSB-Workload Execution (Pre-Failure): 80 seconds Shutdown of a Cassandra node (after 3 minutes) Repeated 3 times for each workload (throughput and latency monitored, average values taken) YCSB-Workload Execution (Post-Failure): 120 seconds 1 Yahoo! Cloud Serving Benchmark - 5 -
Cassandra (version 2.0.8.39 used) Peer-to-peer ring architecture Wide-column NoSQL store SQL-like interface (CQL v3 used) Source: DataStax OpsCenter (Screenshot) - 6 -
Yahoo! Cloud Serving Benchmark (YCSB) Cooper et al., Benchmarking Cloud Serving Systems with YCSB, SOCC 10-7 -
Setup Component 7 Cassandra Nodes YCSB Client Data Center Fujitsu BX620S3 Blade Center CPU AMD Operton 870 (2.0 GHz) AMD Opteron 890 (2.8 GHz) Main Memory 16 GByte DDR-2 reg. 32 GByte DDR-R reg. Hard Drives 2x 146 GByte (RAID-0) 2x 300 GByte (RAID-0) NIC Operating System Cassandra Broadcom NetXtreme BCM5704S, 1 GBit/s transfer speed Ubuntu Server 12.04 64bit DataStax Enterprise Server v4.5.1 (Apache Cassandra v2.0.8.39) YCSB v0.1.4 with a CQL-based Cassandra binding 1 JRE Oracle Java Runtime Environment v1.7.60 1 YCSB Cassandra binding based on CQL: https://github.com/jbellis/ycsb - 8 -
Setup (cont.) Seven DataStax Cassandra nodes Setup using the DataStax installation routines Source: DataStax OpsCenter (Screenshot) - 9 -
Cassandra Configuration Parameters to tune the consistency level: Replication Factor = 3 Write Consistency = 1 Read Consistency = 1-10 -
Dataset used 600 million data record (1 KB record size) 200 million records replication factor 3 Single record consists of ten fields plus PK Data uniformly distributed Replication Strategy: Replicas placed clockwise ( SimpleStrategy ) - 11 -
Data Loading - 12 -
Workloads used Read Workload: 10 million read operations (benchmark configured to read all record fields) Update Workload: 10 million update operations (an update replaces a single field on an existing record) Mixed Workload: 5 million read and 5 million update operations. Defined by Cooper et al. 1 to simulate a session store recording recent user actions. 1 Cooper et al., Benchmarking Cloud Serving Systems with YCSB, SOCC 10-13 -
Throughput Read Workload Throughput -7.5 % - 14 -
Latency Read Workload Latency +8.6 % - 15 -
Throughput Update Workload Throughput -10.2 % (for reads it was -7.5 %) - 16 -
Latency Update Workload Latency +11.9 % (for reads it was +8.6 %) - 17 -
Throughput Mixed Workload Throughput -8.7 % (close to the average based on the results of the read/update workloads) - 18 -
Latency Mixed Workload Read Latency +3.6 % (read-only was +8.6 %) Update Latency +27.1 % (update-only was 11.9 %) - 19 -
Conclusions The cluster remained stable during the node failure. Updates were more negatively affected (throughput and latency) by a node outage compared to reads. For the mixed workload, the throughput decrease was close to the average based on the results of the read/update workloads. Updates were slowed down by concurrent reads. Read operations had a negative impact on concurrent update operations in terms of latency. - 20 -
Recommendations and Outlook Recommendations (for comparable setups): Cassandra is well suited for applications that require high availability and stable read response times. For applications with mixed workloads, the latency of the updates drop significantly after failure. Reducing the number of concurrent reads helps to avoid this behaviour. Outlook: Simulation of multiple node failures Evaluation of the impact of the cluster size Inclusion of additional use case oriented workloads Benchmarking of additional database systems - 21 -