Shared Parallel File System

Shared Parallel File System Fangbin Liu fliu@science.uva.nl System and Network Engineering University of Amsterdam

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

Introduction of the project Parallel file system Industrial standard commodity PCs NFS (Network File System) Many new shared file systems available: PVFS2, GFS, The goal of the project Test PVFS and GFS file system on the test nodes and generate a recommendation report

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

The PVFS2 parallel file system What is PVFS2 file system Parallel Virtual File System 2 A remote storage server management system Internal organization Servers and clients Data storage organization Two types of interfaces

The PVFS2 parallel file system

The PVFS2 parallel file system Features supplied by PVFS2 file system Few dependencies Multiple interfaces supported Modular design Distributed data storage Deployment of PVFS2 file system Generate the configuration file Start up the servers

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

The GFS cluster file system Introduction of GFS file system Managed by Red Hat Cluster Suite Supplies a uniform view among cluster nodes Load distribution and balancing Installed directly on SAN or by GNBD devices Built up with Logical Volumes created by Cluster Logical Volume Manager

The GFS cluster file system

The GFS cluster file system Deployment of GFS file system 1. Forming a cluster domain through cluster configuration tools 2. Forming a fencing domain through fence daemon 3. Export/Import GNBD device 4. Starting Cluster LVM daemon 5. Mount the GFS file system

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

Comparison of PVFS2 and GFS Work theory Servers and Clients management Storage management System consistency supports Robust and scalability

Comparison of PVFS2 and GFS Test of performance Test conditions: 4 test nodes: CPU: Intel Xeon CPU 3.40 GHz Cache: 1024 KB Memory: 2075884 KB Organizations: 2 servers - 2 clients 3 servers 1 clients Network connection: Gigabit Ethernet

Comparison of PVFS2 and GFS Test of performance Test sets: Write, ReWrite, Read, ReRead Throughput test: multiple processes concurrent operations Test programme: IOZONE With options: -A Automatic, all the file size (64 KB, ) and block size (4KB, ) -t # Throughput test with # process for alternative operations (write, read,, random write, )

Comparison of PVFS2 and GFS

Comparison of PVFS2 and GFS

Comparison of PVFS2 and GFS

Comparison of PVFS2 and GFS

Comparison of PVFS2 and GFS kb file

Comparison of PVFS2 and GFS

Comparison of PVFS2 and GFS GFS node 12 1 client 3 servers Write performance 800 700 600 500 700-800 600-700 MB/sec 400 300 500-600 400-500 300-400 200 200-300 100 0 64 256 1024 4096 KB File 16384 65536 262144 1048576 4194304 4 1024 64 16384 KB Record 100-200 0-100

Comparison of PVFS2 and GFS GFS node 12 1 client 3 servers Read performance 3000 2500 2000 2500-3000 MB/sec 1500 1000 2000-2500 1500-2000 1000-1500 500 16384 500-1000 0-500 0 64 256 1024 4096 KB File 16384 65536 262144 1048576 4194304 4 1024 64 KB Record

Shared Parallel File System Introduction of the project The PVFS2 parallel file system The GFS cluster file system Comparison of PVFS2 and GFS Project conclusion and future outlook

Project conclusion and future outlook The PVFS2 file system A popular parallel file system Stable performance Multiple possibilities User level stripping Ease installation The GFS file system A reliable package Robust and scalability Multiple configuration forms Future outlook PVFS2 more chances

Shared Parallel File System Questions?

Shared Parallel File System Thank you so much for your help!