Performance Analysis and Testing of Storage Area Network

Transcription

1 Performance Analysis and Testing of Storage Area Network Yao-Long Zhu, Shu-Yu Zhu and Hui Xiong Data Storage Institute, Singapore

2 Motivations What should we do to optimize the storage system directly connected to storage network instead of directly connected to server? How to compare IP storage, Fibre Channel, and InfinBand? Do we need new algorithms to replace the RAID technology which introduced in 198s? How to evaluate and analyze the performance of the Storage Area Network easily and quickly? Modeling and simulation is a faster way to study these questions than implementation and testing Storage system Block level FC/DWDM FC protocol FC/DWDM Storage system FC-SAN FCIP FCIP FC-SAN Server NAS iscsi WAN iscsi NAS Server Clients LAN Router Router LAN Clients File level IP protocol IP storage system IP storage system

3 Key Points Build up a Queuing Network Model for total SAN performance analysis from perspective of networking Analyze the effects of the I/O workload on the SAN performance Analyze the effects of disk cache and Fork/Join model Analyze the s scheduling algorithms Comparison of the theoretical and experimental results

4 Modeling of SAN Disk Client IP switch Server FC-switch RAID controller Disk Client Server Disk

5 Queuing Network Model Disk Network s view points Client Client IP switch Server Server FC-switch RAID controller Disk Disk I/O response time Throughput (MB/s) IOPs λ host1 λ disk λ hda s λ fc-sw λ da λ fc-al Disk Controller & Cache center λ host2 FC-SW Network Disk Array Controller and Cache Network Disk Array Disk Unit IO Queue

6 SAN Performance for Big Sequential I/O SAN performance (MB/s) System response time varies with system throughput for sequential 1MB I/O size. I/O workload: read System configuration: 1 server+ 1 DACC + 5 disks : Head-Disk Assembly (mechanical part) : Disk array controller and cache : FC fabric Switch Utilizations FC = Bottleneck! Utilizations of service nodes vary with system request rate for sequential 1Mb I/O size Request rate (IOPs)

7 SAN Performance for Big Random I/O Response time (ms) SAN performance (MB/s) response time is the largest portion (4%~6%) in the whole response time distribution. performance of 5 hard disks : 66MB/s 15 Hard disks : 79MB/s 25 Hard disks : 82MB/s Utilizations System response time varies with system throughput for random 1MB I/O size. Utilizations of service nodes vary with system request rate for random 1MB I/O size Physical Drive = Bottleneck! request rate(iops)

8 SAN Performance for Small Sequential I/O response time (ms) System performance (MB/s) System response time varies with the throughput for sequential 4KB I/O (single user) Response time (ms) SAN performance (MB/s) System response time varies with the throughput for sequential 4KB I/O (2 users). Utilizations Utilizations Server = Bottleneck! Request rate (IOPs) Utilizations vary with system request rate for sequential 4KB I/O (single user). (45 IOPs) Disk Array Controller = Bottleneck! Request rate (IOPs) Utilizations varies with the system request rate for sequential 4KB I/O (2 users). (56 IOPs)

9 SAN Performance for Small Random I/O response time (ms) Response time (ms) performance(mb/s) System response time varies with the throughput for random 4KB I/O (5 HDDs) SAN performance (MB/s) System response time varies with the throughput for random 4KB I/O (15 HDDs). Utilizations Request rate(iops) Utilizations vary with the system request rate for random 4KB I/O (5 HDDs). Physical Drive = Bottleneck! Performance of 5 hard disks : 42 IOPs 15 hard disks : 126 IOPs 25 hard disks : 21 IOPs

10 Empirical Model Comparison Throughput (MB/s) theoretical experimental SAN performance varies with I/O size for sequential I/O workload I/O size (KB) The tests are based on the multiple Pentium 733MHz hosts with 64bits and 66MHz PCI bus, HBA with Qlogic QLA 22A, 1G FC switch with Brocade Silkwarm 24, and a self-developed virtual FC Disk. IOMeter is used as the benchmark tool. Both of the testing and theoretic results show that the FC network is the system bottleneck for big I/O size (>32KB), and the storage system controller overhead is the system limitation for small I/O size (<16KB).

11 Lesson #1: Disk Cache / Controller Performance DCC Small I/O : DCC overhead larger than Physical Drive overhead Large I/O : Physical Drive overhead dominates λ disk Controller and Cache center λ hda center Hard Disk model Bandwidths of the disk unit, DCC and vary with IO size for single hard disk I/O size (KB) 45 esponse time varies with I/O size hen the request rate is 2 IOPs r hard disk. Single Disk Cache MAY be bottleneck Throughput( MB/s) Disk Array Cache depends largely on Disk Stripe Size IO size (KB) Disk Cache

12 Lesson #2: Disk Array and Fork/Join Model λ da Disk Array Controller and Cache equest j λ fc-al Fork/Join model Request j for disk 1 Network λ disk λ disk λ disk λ disk Disks Request j out Ratio access time waiting time Disk number Fork/Join access time and queue waiting time ratio vary with disk numbers. Request j for disk k δ * µ TR ( λ) = TS () + k k 2 disk * ρdisk * E[ TS µ λ disk disk 2 disk ] Response time ratio varies with disk numbers and system utilization.

13 Lesson #3: Algorithms Data waiting time (ms) FAA RCF FLF CF utilization Cmd waiting time (ms) FAA RCF FLF CF utilization Data average waiting time varies with utilization. Command average waiting time varies with utilization FAA: Fairness Access Algorithm RCF: Read Command First FLF: FL-port First CF: Commands First

14 Lastly. The queuing network model has been shown to be a useful tool for analyzing the overall SAN system performance The model was also used to analyze our advanced SAN storage technology, DA 2 We missed the proceedings publication. However, this paper will be made available on the conference website! More Questions? Visit our Poster Session Thank you for your attention!