CS4500/5500 Opera-ng Systems Distributed and Parallel File Systems

Transcription

1 Opera-ng Systems Distributed and Parallel File Systems Jia Ra Department f Cmputer Science hcp://

2 Recap f Previus Classes File systems prvide an abstrac0n f permanently stred data Namespace: files and directries Translate paths t crrespnding lca0ns n disks Space management and p0miza0ns Free blcks Caching and prefetching Reliability and cnsistency

3 Distributed and Parallel File Systems Prvide similar abstrac0ns f data n mul$ple machines Namespace: path name à machine ID: disk blck address Management: placement f files n machines Replica0n Striping Designed fr perfrmance and availability

4 Distributed v.s. Parallel File Systems Design bjec0ves Fault- tlerance v.s. Cncurrent perfrmance Data distribu0n En0re file n a single nde v.s. striping ver mul0 ndes Symmetry Strage c- lcated with apps v.s. strage separated frm apps Fault- tlerance Designed fr fault- tlerance v.s. relying n enterprise strage Wrklad The bundary is blurring Lsely cupled, distributed apps v.s. crdinated HPC apps

5 Examples Distributed File Systems NFS, GFS (Ggle File System), HDFS (Hadp Distributed File System), GlusterFS Parallel File Systems PVFS (Parallel Virtual File System), Lustre, OCFS2, GPFS

6 Design Issues (1) Nameserver maps file names t bjects (files, directries, blcks) Implementa0n p0ns Single name Server Simple implementa0n, reliability and perfrmance issues Several Name Servers (n different hsts) Each server respnsible fr a dmain

7 Design Issues (2) Caching Caching at the client: Main memry vs. Disk Cache cnsistency Server ini0ated Server infrms cache managers when data in client caches is stale Client cache managers invalidate stale data r retrieve new data Disadvantage: extensive cmmunica0n Client ini0ated Cache managers at the clients validate data with server befre returning it t clients Disadvantage: extensive cmmunica0n Prhibit file caching when cncurrent- wri0ng Several clients pen a file, at least ne f them fr wri0ng Server infrms all clients t purge that cached file Lck files when cncurrent- write sharing (at least ne client pens fr write)

8 Design Issues (3) Update (write) plicy Once a client writes int a file (and the lcal cache), when shuld the mdified cache be sent t the server? Write- thrugh: all writes at the clients, immediately transferred t the servers Advantage: reliability Disadvantage: perfrmance, it des nt take advantage f the cache Delayed wri0ng: delay transfer t servers Advantages: Many writes take place (including intermediate results) befre a transfer Sme data may be lst Disadvantage: reliability Delayed wri0ng un0l file is clsed at client Fr shrt pen intervals, same as delayed wri0ng Fr lng intervals, reliability prblems

9 Design Issues (4) Availability What is the level f availability f files in a distributed file system? Use replica0n t increase availability, i.e. many cpies (replicas) f files are maintained at different sites/servers Replica0n issues: Hw t keep replicas cnsistent Hw t detect incnsistency amng replicas

10 Design Issues (5) Scalability Deal with a grwing system? Issues Nde jin and leave (fail) Cache cnsistency Nameserver Slu0ns Replica0n Design cache cnsistency prtcl fr scalability Mul0ple name (meta) servers Take advantage f mul0- thread and mul0- cre

11 Example - GlusterFS (DFS) Client- 1 Client- 2 Client- N IP netwrk Gluster Glbal Namespace (Gluster Na-ve) Gluster Virtual Strage Pl (built n dnated par00ns n each machine)

12 Example GlusterFS (2) Three ways t place files Distribute: place en0re files n different servers Prs: gd scalability, efficient disk space usage Cns: pr reliability Replicate: place iden0cal cpies f files n different servers Prs: reliability Cns: wasted disk space, mderate scalability Stripe: place nly part f a file n ne server Prs: gd perfrmance fr cncurrent and randm access Cns: pr scalability and reliability

13 Example PVFS(PFS)

14 Example PVFS (PFS) Significant imprvement in thrughput What culd be the issues? 1. Sever crdina0n affects efficiency 2. Client QS?

15 GFS and PFS in the Clud (1) Bth appraches prvide cheap, reliable and high- perfrmance clud strage slu0ns Use case- 1

16 GFS and PFS in the Clud (2) Use case- 2

17 Sme Real Results Hst a 8- VM Hadp cluster n 8 DELL machines Perfrmed micr and real I/O intensive wrklads Tw strage slu0ns: PVFS and lcal ext3 PVFS Gridmix websrt 20GB data 2391 secnd Lcal ext secnd PVFS 16k 32k 64k 256k 1M Sequen0al randm Netwrk bandwidth bcleneck Lcal ext3 16k 32k 64k 256k 1M Sequen0al randm