Sussex University Lustre Benchmarking Report

Transcription

1 Sussex University Lustre Benchmarking Report Alces Order number: DELL04 Table of Contents Sussex University...1 Lustre Benchmarking Report...1 IOZONE benchmark test configuration and method...2 IOZONE benchmark summary...3 IOZONE benchmark results for ALL OSS...4 IOZONE benchmark results for OSS1...6 IOZONE benchmark results for OSS2...7 IOZONE benchmark results for OSS3...8 Copyright 2010 Alces Software Ltd. Page 1 of 8

2 IOZONE benchmark test configuration and method Each benchmark run was performed under the following conditions: Tests launched from inactive metadata server (mds2) Seven cluster nodes used as clients running the following software: Bright 5.0/Scientific Linux 5.4 Linux kernel el5 Qlogic OFED stack Lustre patch-less client IOZONE version Lustre filesystem unmounted between successive tests to clear caches Clients balanced across available OSS/OSTs for optimum performance Results shown in this report were obtained from a number of benchmark runs and condensed in a single output to allow caches to be cleared between individual tests All tests were performed with the metadata service running on server mdt1: Dell PowerEdge R610 with dual E Ghz processors and 12GB RAM Dual mirrored 146GB SAS6 system disks MD3200 with 8 x 300GB 15,000RPM SAS disks configured in RAID0/1 Qlogic 7340 QDR Infiniband card Object storage server configuration for servers oss1, oss2 and oss3: Dell PowerEdge R510 with dual E Ghz processors and 12GB RAM Dual mirrored 146GB SAS6 system disks 12 x 1TB 7,200RPM SATA disks configured in RAID6 on PERC H700 (ost1) MD1200 with 12 x 1TB 7,200RPM SAS disks configured in RAID6 on PERC H800 (ost2) MD1200 with 12 x 1TB 7,200RPM SAS disks configured in RAID6 on PERC H800 (ost3) Qlogic 7340 QDR Infiniband card Qlogic QDR Infiniband switch Dell 6248 out-of-band gigabit Ethernet management switch Copyright 2010 Alces Software Ltd. Page 2 of 8

3 IOZONE benchmark summary The table below summaries the benchmark results obtained: Servers All-OSS OSS1 OSS2 OSS3 Clients metric KB/s IOPS KB/s KB/s KB/s write 3,854,980 60,235 1,273,054 1,272,119 1,273,239 rewrite 3,005,168 46,956 1,039,489 1,117,662 1,020,844 read 2,982,143 46,596 1,143,591 1,164,641 1,165,076 reread 18,411, ,680 9,127,979 8,918,498 7,724,083 random read 151,069 2,361 random write 261,675 4,089 reverse read 251,587 3,932 stride read 303,852 4,748 mixed 280,632 4,385 Points to note: Performance is approximately equivalent for all three OSS machines Aggregate performance is approximately equivalent to 3 x single OSS performance Re-read performance demonstrates affect of OSS and client side cache All tests performed with two clients per OST Copyright 2010 Alces Software Ltd. Page 3 of 8

4 IOZONE benchmark results for ALL OSS iozone]#./iozone -i 0 -w -e -s10g -r64k -t 18 -+m /mnt/lustre/iozone/alces/iozone/clientlist Iozone: Performance Test of File I/O Version $Revision: $ Compiled for 64 bit mode. Build: linux-amd64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root. Run began: Thu Sep 9 17:43: Setting no_unlink Include fsync in write timing File size set to KB Record Size 64 KB Network distribution mode enabled. Command line used:./iozone -i 0 -w -e -s10g -r64k -t 18 -+m /mnt/lustre/iozone/alces/iozone/clientlist Output is in Kbytes/sec Time Resolution = seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Throughput test with 18 processes Each process writes a Kbyte file in 64 Kbyte records Children see throughput for 18 initial writers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 18 rewriters Children see throughput for 18 readers Children see throughput for 18 re-readers = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 18 random readers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 18 random writers = KB/sec = KB/sec = KB/sec = KB/sec = KB Copyright 2010 Alces Software Ltd. Page 4 of 8

5 Children see throughput for 18 reverse readers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 18 stride readers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 18 mixed workload = KB/sec = KB/sec = KB/sec = KB/sec = KB Copyright 2010 Alces Software Ltd. Page 5 of 8

6 IOZONE benchmark results for OSS1 iozone]#./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Iozone: Performance Test of File I/O Version $Revision: $ Compiled for 64 bit mode. Build: linux-amd64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root. Run began: Thu Sep 9 18:56: Setting no_unlink Include fsync in write timing File size set to KB Record Size 64 KB Network distribution mode enabled. Command line used:./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Output is in Kbytes/sec Time Resolution = seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Throughput test with 6 processes Each process writes a Kbyte file in 64 Kbyte records Children see throughput for 6 initial writers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 6 rewriters Children see throughput for 6 readers Children see throughput for 6 re-readers = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB iozone test complete. Copyright 2010 Alces Software Ltd. Page 6 of 8

7 IOZONE benchmark results for OSS2 iozone]#./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Iozone: Performance Test of File I/O Version $Revision: $ Compiled for 64 bit mode. Build: linux-amd64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root. Run began: Thu Sep 9 18:54: Setting no_unlink Include fsync in write timing File size set to KB Record Size 64 KB Network distribution mode enabled. Command line used:./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Output is in Kbytes/sec Time Resolution = seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Throughput test with 6 processes Each process writes a Kbyte file in 64 Kbyte records Children see throughput for 6 initial writers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 6 rewriters Children see throughput for 6 readers Children see throughput for 6 re-readers = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB iozone test complete. Copyright 2010 Alces Software Ltd. Page 7 of 8

8 IOZONE benchmark results for OSS3 iozone]#./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Iozone: Performance Test of File I/O Version $Revision: $ Compiled for 64 bit mode. Build: linux-amd64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root. Run began: Thu Sep 9 18:48: Setting no_unlink Include fsync in write timing File size set to KB Record Size 64 KB Network distribution mode enabled. Command line used:./iozone -i 0 -w -e -s10g -r64k -t 6 -+m Output is in Kbytes/sec Time Resolution = seconds. Processor cache size set to 1024 Kbytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. Throughput test with 6 processes Each process writes a Kbyte file in 64 Kbyte records Children see throughput for 6 initial writers = KB/sec = KB/sec = KB/sec = KB/sec = KB Children see throughput for 6 rewriters Children see throughput for 6 readers Children see throughput for 6 re-readers = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB = KB/sec = KB/sec = KB/sec = KB/sec = KB iozone test complete. Copyright 2010 Alces Software Ltd. Page 8 of 8