Performance test StorPool vs. Ceph. November StorPool. All rights reserved. 1 of 9

Size: px

Start display at page:

Aleesha Preston
7 years ago
Views:

2 Executive Summary StorPool is a distributed storage system running on standard server hardware. It uses minimal system resources to achieve outstanding performance. Even on a small system with 3 nodes and 12 hard drives, StorPool outperforms Ceph by a large margin. This difference in performance and efficiency translates to a large economy in the number and type of servers needed to provide the high performance block storage service, underpinning clouds. We ran both Ceph and StorPool on 3 nodes with 12 HDDs and 3 SSDs. The network is 10Gigabit Ethernet. We ran the test workload(fio) on a separate node. Performance Test Ceph HDD (baseline) Ceph HDD + Journal StorPool HDD StorPool Hybrid Sequential reads 908 MB/s 1.3x * 1.3x * 1.3x * Sequential writes 204 MB/s 1.7x 3.0x 3.9x Random reads 4k block size IOPS 1.1x ** 1.2x ** 71x Random writes 4k block size IOPS 1.4x 6.5x 7.8x * - constrained by 10GE interface ** - constrained by HDD random IOPS Server CPU Test Ceph HDD (baseline) Ceph HDD + Journal StorPool HDD StorPool Hybrid Sequential reads 4.1 % 0.6x better 1.1x better 1.6x better Sequential writes 7.2 % 0.6x better 1.5x better 1.6x better Random reads 4k block size 2.2 % 1.0x better 1.9x better 0.3x better * Random writes 4k block size 25 % 0.6x better 6.4x better 6.4x better ** Note: Percentages are average out of 24 CPU threads total. Because of hyperthreading, 40% CPU usage (on Ceph HDD+Journal) really means near full load. * - 4x higher CPU usage, while delivering 71x higher IOPS ** - 6.4x lower CPU usage, while delivering 7.8x higher IOPS 2014 StorPool. All rights reserved. 2 of 9

3 Introduction StorPool is distributed storage software. It pools the attached storage (hard disks or SSDs) of commodity servers to create a single pool of shared storage. The StorPool software is installed on each server in the cluster and combines the capacity and performance of all drives attached to the servers into one global namespace. This document presents results from performance tests run in StorPool's test lab. No test result can be an exact replica of actual workloads, thus these results could be used only as an indication of expected performance of StorPool. Customers are advised to perform their own tests. Server Configuration Name CPU RAM RAID/HBA Drives s11 Xeon E5-1620V2 4 cores 3.7GHz 32 GB Intel C600 AHCI Client s12 Xeon E5-1620V2 1x SSD 32 GB Intel C600 AHCI 4 cores 3.7GHz 4x HDD s13 Xeon E5-1620V2 1x SSD 32 GB Intel C600 AHCI 4 cores 3.7GHz 4x HDD s14 Xeon E5-1620V2 1x SSD 32 GB Intel C600 AHCI 4 cores 3.7GHz 4x HDD Hard drive model: HGST Deskstar 500GB (HDS721050CLA360) SSD models: Intel DC S GB (SSDSC2BB240G4) Total: 1 client 3 servers 3 SSDs 12 HDDs Network configuration 2x Mellanox MCX312A-XCBT, 1x Solarflare SFN5162F (sfc), 1x Intel 82599ES (ixgbe) Single 10GE link per server Switch: Dell S8024F 24-port 10GE SFP+ switch 9000 bytes MTU (Jumbo frames) Flow control enabled Software configuration and testing methodology The tests are performed on 3 servers and 1 client. All servers are installed with a CentOS 6.5 operating system StorPool. All rights reserved. 3 of 9

4 Each test run consists of: 1. configuring and starting a StorPool cluster or Ceph cluster 2. creating one 200GB volume 3. filling the volumes with uncompressible data 4. performing all test cases by running FIO on the client The following table summarizes the test parameters common to all tests. Operating system CentOS 6.5 Performance-testing software fio and StorPool test runner scripts Number of servers used for storage 3 nodes Number of servers used for storage 1 clients Number of volumes 1 Volume size 200GB Replication level (number of copies) 2 Number of hard drives used for test 12 HDDs 12 HDDs + 3 SSDs Kernel version Linux (storpool build) StorPool-specific parameters Distributed storage software version StorPool Integrity provided by system End-to-end data integrity. Protects data throughout its lifetime. Object Size 32MB Stripe Size 1MB Caching, Buffering, Data consistency Read caching with 4GB cache per storage node Write-through caching shared with read cache. No write-back cache Ceph-specific parameters Distributed storage software version Integrity provided by system Object Size Stripe Size Caching, Buffering, Data consistency Ceph el6 None 4MB 4MB Read caching in Linux buffer cache. No write-back cache. Journal on SSD or on HDD. Test cases Test name Read/write Block size Queue depth Duration IOPS tests randwrite Random Writes 4 KB 4, 16, 64, 256 1m randrw Random Reads + Random Writes 4 KB 4, 16, 64, 256 1m 2014 StorPool. All rights reserved. 4 of 9

5 50/50 randread Random Reads 4 KB 4, 16, 64, 256 1m Sequential tests seqwrite Sequential Writes 1 MB seqread Sequential Read 1 MB 1, 4, 16, 64, 256 1, 4, 16, 64, 256 1m 1m Latency tests writelatency Random Writes 4 KB 1 1m readlatency Random Reads 4 KB 1 1m All tests use FIO with libaio, direct, sync, norandommap and randrepeat=0. There were 10-minute pauses between tests. Test runs We tested the following scenarios Ceph HDD with 12 HDDs Ceph HDD+Journal 12 HDDs + 3 SSDs StorPool HDD 12 HDDs StorPool Hybrid 12 HDDs + 3 SSDs For each scenario we ran all tests in sequence, after filling the 200GB volumes with uncompressible data. Test results Random ops (IOPS) 2014 StorPool. All rights reserved. 5 of 9

Numbers in reddish highlight denote workloads limited by the random IO

Numbers in reddish highlight denote workloads limited by the

9 Latency (ms) StorPool write latency is governed by latency of the underlying hard drives. Write latency can be mitigated by using safe battery-backed write-back cache in a RAID controller under StorPool. Conclusion StorPool has exceptional performance, while at the same time providing end-to-end data integrity and shared storage capabilities. Additionally, StorPool maintains high performance even with many competing workloads. Even with SSD journal, Ceph just matches StorPool s performance on hard disks only. When StorPool also uses SSDs it has staggering performance advantages, peaking at 71 times (!) the performance of Ceph (for random reads 4k block size). Furthermore, Ceph s CPU usage for random writes is very high, making it unsuitable for converged architectures - running storage+compute on the same servers. StorPool can run on the same node or on a standalone node. However, we recommend running the compute nodes which lowers TCO and provides a single building block for the datacenter. Contacts If you would like to learn more or test StorPool, contact us: info@storpool.com 2014 StorPool. All rights reserved. 9 of 9

Intel Solid- State Drive Data Center P3700 Series NVMe Hybrid Storage Performance

Intel Solid- State Drive Data Center P3700 Series NVMe Hybrid Storage Performance Hybrid Storage Performance Gains for IOPS and Bandwidth Utilizing Colfax Servers and Enmotus FuzeDrive Software NVMe Hybrid