Secure File Sharing SRFS on Ether with host-to-host IPSec connection over the Pacific Ocean link

Transcription

1 Secure File Sharing on Ether with host-to-host IPSec connection over the Pacific Ocean link Hirofumi OHKAWA, Naoyuki FUJITA Japan Aerospace exploration Agency SC06 Nov. 15 th 2006, TAMPA CONVENTIONAL CENTER

2 Background HPC-System becomes huge system 1P 1968 HITAC 5020F 1975 FACOM HITAC DATATRON FACOM AP 1983 FACOM M FACOM VP FACOM VP NWT 1996 NWT 2002 NS III 9.3TFLOPS 1T 1G 1M 1K Processor Speed [FLOPS] History of our HPC-systems

3 Background Recent CFD simulation codes generate TB-class result file CFD : Computational Fluid Dynamics

4 IO Speed Background Real-time visualization is necessary for CFD simulation Understanding the complex phenomena Monitoring to debug large-scale job Steering the simulation

5 ex. Huge turbulence computation with real-time visualization Grid points : a = 1024 x 1024 x 1024 [points] Information quantity per grid : b = 5 [variables/point] Length of variable : c = 8 [Bytes/variable] Calculation time : d = 500 [hours]= [sec] Time calculation steps : e = 1.2 x 106 [steps] Visualization frequency : f 100 time calc. steps Number of visualization : g = 1000 File Size = a x b x c x g = 39.1 [TB] IO Speed = (a x b x c) / (d / e x f) = 273 [MB/sec]

6 Solution on Ether Remote File System for HPC-users especially Shared Rapid File + Ethernet Interface System jointly developed by JAXA and Fujitsu

7 SC Performance evaluation on Real Long Fat Network Solaris8 Server 1G NIC IPv4 1G NIC MAX 800(Mbps) Solaris8 Solaris8 Server to with 1G-NIC SC05 - Multi-to-Single machine connection over IPv4 / IPv6 Solari9 Solari9 Solaris9 1G Server Solaris9 1G 1G Server NIC Solari9 Server 1G 1G NIC NIC Solaris9 Server Server 1G NIC ServersNIC NIC IPv4/v6 10G NIC Linux 6 Solaris9 Servers with 1G-NIC to 1 Linux with 10G-NIC IPv4 5.4(Gbps), IPv6 4.3(Gbps)

8 Demonstration in SC06 BWC on Ether with host-to-host IPSec over the Pacific Ocean link Tokyo JGN-II Abilene Chicago Tampa

9 System Server for Linux Fujitsu PRIMERGE RXI300 Intel Itanium2 1.6GHz Mem:8GB RHEL4 U2(IPF) for Linux Dell PowerEdge 2850 Intel Xeon 2.8GHz x 2 Mem:6GB RHEL4 U3(EM64T) IPSec board Cavium NITROX-II 10G NIC Chelsio T210-LR IPSec IPSec board Cavium NITROX-II 10G NIC Chelsio T210-LR

10 ioperf read(2)/write(2) IO _IO _IO Double buffering BBH Max 1024 thread TCP/IP (=TCP Window Size) rep2 rep1 TCP ack Internet TCP/IP BBH Server _IO _IO Direct IO FS/disk driver read(2)/write(2) Server

11 Thank You for your attention! If you have any questions Please visit JAXA booth #851

12 ***Support of science mission*** PIV : Particle Image Velocimetry CIFS Folder on Ether Folder High Speed Backup Server

13 ***Support of science mission*** ITBL : IT Based Laboratory We demonstrated this issue at SC05 Bandwidth Challenge AVS/Express read() Pressure Particle Surface Uviewer3D() Server on Ether Remote Interactive Visualization

14 ***Use of production network at our home institution*** Network Log Trap on Ether MIB Admin Log High Speed Archive Server Trap MIB

15 ***Use of production network at our home institution*** JSS:JAXA Supercomputer System (Under construction) Kakuda Tsukuba 1GbE Console VPN Sagamihara Remote Server VPN 1GbE Console Remote Server The Internet VPN Server HPC Server 1GbE Console Remote Server VPN Chofu on Ether 1GbE Console

16 ***Innovative use of protocols*** Protocol Switching on on Ether Server TCP,UDP Server UID Mapping Host ACL User Application Selects appropriate Transmission layer Protocol Another Technical Issues TCP,UDP User Application