Rapid Remote File System Benchmark on Long Fat Network and User ID Mapping Function Naoyuki FUJITA, Hirofumi OOKAWA

Transcription

1 Rapid Remote File System Benchmark on Long Fat Network and User ID Mapping Function Naoyuki FUJITA, Hirofumi OOKAWA 11 th -14 th April 2005 MSST2005 P.1

2 Table of Contents Application and System Configuration Technical Targets Former Works Benchmark on Long Fat Network User ID Mapping Summary and Future Works 11 th -14 th April 2005 MSST2005 P.2

3 Application 300 [MByte/s] >1 [TB] 11 th -14 th April 2005 MSST2005 P.3

4 System Configuration Oct : Installed Computing Storage Storage Devices: [Disk : 50TB] RAID5:9D+1P(FujitsuPW-D500B1) 80[drives] [Tape : 600TB] LTO(IBM3580) 40[drives] Visualization Local File System: SAM-QFS 6Pipes Crossbar Network File System:SRFS GSN-Lib I/O System 11 th -14 th April 2005 MSST2005 P.4

5 Technical Targets File Share over IP-network, especially the Internet Data coherency under a file-sharing environment High-speed without TCP/IP tuning Unified user administration Security (Working with firewall, Encryption, Intrusion protection) 11 th -14 th April 2005 MSST2005 P.5

6 Former Works File Share over IP-network Added Ethernet communication control module Data coherency under a file-sharing Guaranteed by original Crossbar-SRFS High-speed without TCP/IP tuning 8 stripes data stream Switch to UDP when I/O length is small 55 [Mbps] on 100Mbps-WAN 610 [Mbps] on GbE-LAN 11 th -14 th April 2005 MSST2005 P.6

7 Technical Targets File Share over IP-network, especially the Internet Data coherency under a file-sharing environment High-speed without TCP/IP tuning Unified user administration Security (Working with firewall, Encryption, Intrusion protection) 11 th -14 th April 2005 MSST2005 P.7

8 1.7[GB/s] Previous Benchmark Results 6[GB/s] 550[MB/s] 80[MB/s] Compute Node Compute Node 3D Visualization System Workstation??? [MB/s] FORTRAN X-bar SRFS X-bar SRFS Original Library SRFS on Ether The Internet Crossbar interconnect network 4[GB/s] x 2 GSN 800[MB/s] Ethernet Original Library SRFS on Ether Crossbar SRFS Local File System: SAM-QFS Operating System IO node Disk FC-RAID5 Tape x 80 x 40 LTO One file disk I/O: 80[MB/s] 1 Terabyte file I/O: 3.5[GB/s] DISK: 80[MB/s] TAPE: 14[MB/s] 11 th -14 th April 2005 MSST2005 P.8

9 Benchmark on Long Fat Network SuperSINET JGN-II Research Center JAXA chofu APAN WAN SINET-US TransPAC JGN-II TransPAC SC2004 Exhibit Hall Abilene 1Gbps MAX: 0.93Gbps MEAN: Mbps SCinet 1G-SX L3 Switch Catalyst Base-SX TCP8001 TCP8010 L2 Switch Summit48i 1000Base-T PP HPC2500 JAXA chofu /24 UDP8010 TCP9998 UDP9998 RPC TCP1024+ UDP V440 ITBL (Booth# 2748) /27 11 th -14 th April 2005 MSST2005 P.9

10 Benchmark on Long Fat Network (High Speed Function) Throughput [Mbyte/s] 転 送 速 度 (Mbyte/s) SRFS tuning (1)Socket Buffer Size = 32KB~16MB (2)BB HPC Socket Buffer Size = 32KB (3)SRFS IO 長 = 256MB~1GB TCP tuning (1)tcp_time_wait_interval = a: Single stream bandwidth w: TCP window size=16[kb] r: Round Trip Time=200[ms] w 16[KB] a=----= =80[kb/s] r 200[ms] # of stripes : 1563[streams] Measurement Result in Lab. Throughput [Mbps] Time [sec.] 11 th -14 th April 2005 MSST2005 P.10

11 User ID Mapping Unified user administration over multi-organization SRFS Client Node 1 User ID Mapping UID A SRFS Server UID B UID B Node 2 UID C Filter UID C Network 1 Node 3 Node 4 Any UID Filter UID D SRFS Server : Node n Network 2 Filter X 11 th -14 th April 2005 MSST2005 P.11

12 Summary Benchmark on Long Fat Network Pacific Ocean 1Gbps Link MAX: 0.93Gbps MEAN: Mbps Striping data stream is not good for High-speed I/O on Long Fat Network User ID Mapping Simple User ID Mapping between client and server 11 th -14 th April 2005 MSST2005 P.12

13 Future Works Security (Working with firewall) PP HPC2500 TCP8001 TCP8010 UDP8010 TCP9998 UDP9998 -Too much ports -Dynamic allocation TCP1024+ UDP V440 SERVER RPC CLIENT Linux Client Deployment Server Solaris 8 & 9 Client Solaris 8 & 9,Linux (Red Hat Enterprise) Coming Soon! 2005 May Contact fujita@chofu.jaxa.jp 11 th -14 th April 2005 MSST2005 P.13

14 Thank you 11 th -14 th April 2005 MSST2005 P.14

15 FORTRAN I/O 1.7[GB/s] Previous Crossbar Benchmark GSN-Lib LAN Results (Ethernet) 6[GB/s] 550[MB/s] 80[MB/s] Compute Node Compute Node 3D Visualization System Workstation FORTRAN X-bar SRFS X-bar SRFS Original Library SRFS on Ether The Internet Crossbar interconnect network 4[GB/s] x 2 GSN 800[MB/s] Ethernet Long Fat Network Original Library SRFS on Ether Crossbar SRFS Local File System: SAM-QFS Operating System IO node Disk FC-RAID5 Tape x 80 x 40 LTO Local One file disk I/O: 80[MB/s] 1 Terabyte file I/O: 3.5[GB/s] Storage device characteristics DISK: 80[MB/s] TAPE: 14[MB/s] 11 th -14 th April 2005 MSST2005 P.15