Designing an NFS-based Mobile Distributed File System for Ephemeral Sharing in Proximity Networks

Transcription

1 Designing an NFS-based Mobile Distributed File System for Ephemeral Sharing in Proximity Networks Nikolaos Michalakis Computer Science Department New York University, New York, NY Dimitris Kalofonos Pervasive Computing Group (PCG) Nokia Research Center, Boston, MA 1 ASWN 04, August 11, 2004

2 Outline Ephemeral vs. Persistent file sharing Motivation for using NFS as a basis for Ephemeral M-DFS Related Approaches NFS Performance in Proximity Networks Problems with Mobility Experimental and Simulation Setup Performance Results Designing an M-DFS for Ephemeral Sharing M-DFS Design Topics and Suggestions An Implementation Proposal Future Directions 2 ASWN 04, August 11, 2004

3 Ephemeral vs. Persistent File Sharing >> cd /sophia/pictures >>./netscape /tom/cool_page.html >> /sophia/mplayer /mary/music/cool_song.mp3 >> /tom/xview /jack/cool_pic.jpg >>./excel /office/not_so_cool_spreadsheet.xls >> ls /home/my_files/ JOHN HERMES PHONE HOME TOM HERMES HERMES INTERNET PROXIMITY NETWORK Example of persistent sharing OFFICE SOPHIA JACK Example of ephemeral sharing HERMES 3 ASWN 04, August 11, 2004

4 Ephemeral vs. Persistent File Sharing (cont.) Ephemeral file sharing: a remote file system is discovered and mounted to enable short-lived collaboration among users Intended mainly for read-only file sharing Client caching takes place mainly to improve performance (e.g. in low bandwidth-links). It does not guarantee disconnected operation Deals with short disconnections (order of seconds-minutes) due to intermittent connectivity at the link level. No state is expected to survive long (as defined by the user) disconnections If files are to remain accessible in the long-term, user saves local copies Persistent file sharing: enables users to keep a personal distributed file repository among their devices (e.g. phones, PC s) Intended mainly for read-write file sharing Client performs (Coda-like) hoarding when fully connected, can operate on files when disconnected, reintegration-conflict resolution when reconnected Disconnections can be arbitrarily long (order of hours-days) Files are accessible in the long-term without saving local copies 4 ASWN 04, August 11, 2004

5 Why Use NFS as Basis for Ephemeral M-DFS? NFS perhaps the most successful DFS to date. Widely used, well tested Native support for NFS in Linux kernel. Appealing for Linuxbased mobile devices and experimentation NFS interface has made it attractive as a basis to build DFS with additional features. Tools available (e.g. SFS toolkit) for user level implementation. Makes portability across platforms possible Light mounting and unmounting process allows discovering exported files systems and mounting on demand Allows RPC-based access to remote files, without requiring the transfer of whole files Not as complex as other popular DFS for mobility (e.g. CODA). More suitable as a basis to build M-DFS for resourceconstraint devices 5 ASWN 04, August 11, 2004

6 Related Approaches CODA/Intermezzo Supports true disconnected operation Client performs aggressive caching (hoarding) when connected Optimizations available for weak connectivity Intended for Persistent sharing Some design aspects applicable to Ephemeral sharing. However, too complex to be used as is. Removing functionality tricky NFS/M NFS modified to support (CODA-like) disconnected operation Requires Kernel modifications Intended for Persistent sharing. Redundant functionality for us SFS/LBFS User-level DFS on top of NFS to add security, low BW enhancements SFS toolkit available that deals with issues of loopback NFS 6 ASWN 04, August 11, 2004

7 NFS Performance in Proximity Networks: Experimental Setup NFS v3 client and server on two laptops running Linux Mobility: IP address changes by attaching to different AP Intermittent connectivity: disconnections by releasing IP address b measurements Transfers with both client and server on same AP Data rate selection: 1Mbps, 2Mbps, 11Mbps Bluetooth measurements BlueZ stack v2.2 with PAN profile 1.0 Bluetooth v1.1 USB adapters Simple piconet with a Master (NFS Server) and a Slave (NFS Client) Increasing distance increased the RTT because of link-level re-tx Data rate selection: DH1, DH3, DH5 7 ASWN 04, August 11, 2004

8 NFS Performance in Proximity Networks: Simulation Client1 Routing Host Link11 Server1 Hub1 eth0 Link12 Link21 Link22 Hub2 eth1 Client2 Server2 Network simulator using the Click modular router 8 ASWN 04, August 11, 2004

9 NFS Performance in Proximity Networks: Simulation (cont.) Client to server path Delay Module Bandwidth Module Disconnection Module Loss Module eth0 Control Socket eth1 Loss Module Disconnection Module Bandwidth Module Delay Module Server to client path The configurable link 9 ASWN 04, August 11, 2004

10 NFS Problems with Mobility and Disconnections Client has mounted Client not mounted (server already started) Client IP address change: Server IP address change: Client network interface down: Server network interface down: Client disconnects and reconnects with old IP address: Server disconnects and reconnects with old IP address: Client receives a stale filehandle error. The client is using a filehandle that is exported to another client address. Client is trying to contact old address. If hard mounted then it blocks forever. If soft mounted it timeouts and fails (I/O error). Network unreachable error, no matter whether data cached or not at the client. If hard mounted client blocks forever. If soft mounted it timeouts and fails (I/O error). As long as the reassignment does not take longer than the timeout, the client only experiences a delay. As long as the reassignment does not take longer than the timeout, the client only experiences a delay. Client receives an access denied error. The server has already created a list of client addresses that can access the exported file systems. No errors. The mounted directory is empty, i.e. not mounted to anything. Client cannot mount. Mount process gets can't get address for server error. No errors. No errors. 10 ASWN 04, August 11, 2004

11 NFS in Proximity Networks: Performance Results Effect of RTT vs Bandwidth (measurements) BTH WLAN 10K file 50K file 100K file time to read file (s) bandwidth (KB/s) T N ( RTT + rpc ) bw 11 ASWN 04, August 11, 2004

12 FS in Proximity Networks: Performance Results (cont.) 70 Effect of RTT vs Bandwidth (simulation model) Time to read 50K file (s) Bandwidth (KB/s) log scale T N ( RTT + RTT=200ms RTT=100ms RTT=20ms rpc ) bw 12 ASWN 04, August 11, 2004

13 FS in Proximity Networks: Performance Results (cont.) BTH dark zone (measurements) BTH dark zone (measurements) 140 UDP, timeout =10s UDP, timeout =1s 120 TCP 100 UDP, timeout =0.5s UDP, timeout =0.1s timeout =0.1s 40 timeout =0.5s timeout =1s 20 timeout =10s TCP Time (s) time to reply (s) timeout =0.1s timeout =0.5s timeout =1s timeout =10s TCP RPC req. no 13 ASWN 04, August 11, 2004

14 FS in Proximity Networks: Performance Results (cont.) Effect of disconnections (measurements) RPC seq. no cold no discon. UDP finish when no time finish when no time wasted (warm) wasted (cold) warm no discon. UDP cold with discon. UDP warm with discon. UDP cold with discon. TCP warm with discon. TCP wasted time (cold) 5.5s 5.5s 5.5s wasted time (warm) Time (s) 14 ASWN 04, August 11, 2004

15 FS in Proximity Networks: Performance Results (cont.) 350 Effect of packet loss rate (simulation model) time to read 100K file (s) timeo=0.1s timeo=0.5s timeo=1s timeo=10s TCP packet loss rate (%) 15 ASWN 04, August 11, 2004

16 M-DFS Design Topics and Suggestions More aggressive caching to support temporary disconnections In the case of RO ephemeral sharing caching can be simplified, e.g. callbacks with leases: close-to-open consistency, reduced traffic In addition to file and directory contents, client caches metadata such as picture thumbnails or mp3 samples Automatic session recovery to deal with errors because of client/server mobility and loss of connectivity Session support mechanism: client and server have active session (server exported and client mounted the file system), data in transit. Need to decide whether to resume or drop pending sessions upon reconnection Session recovery mechanism: the server has started a session and is waiting for client to join (server only exported file system). Need to preserve the file system state related to mounting Techniques for traffic reduction data compression techniques, application specific data reduction techniques using metadata (e.g. thumbnails, mp3 samples) 16 ASWN 04, August 11, 2004

17 M-DFS Design Topics and Suggestions (cont.) Enhanced metadata to help deal with weak connectivity networks Application specific metadata (e.g. thumbnails, mp3 samples) are opaque to the M-DFS. The M-DFS allocates extra space and an API for applications to use them M-DFS specific metadata are seen only by the FS and the OS. Can be used to improve performance, provide enhanced capabilities and make the system more adaptive to mobile conditions. Example: estimated file transmission time Access control and distributed authentication ACL provide better security than Unix access control Use ephemeral ACLs, i.e. ACLs that are created and updated by the server owner on the fly. Remove stale entries in ACLs to avoid long ACLs with few active user. Distributed client/server authentication an issue without trusted authorities. Simplest solution would be to use offline means, e.g. userto-user communication Naming and file system removal A global name space not required. Server repository may be identified explicitly by the device name that exports it. Uniform name space desirable 17 ASWN 04, August 11, 2004

18 An Implementation Proposal M-DFS client M-DFS server App RO M-DFS Client (loopback server) Cache Manager (loopback client) User-level RO M-DFS Server (loopback client) VFS Kernel-level VFS NFS client NFS server NFS server local FS file_op vnode file_op inode NFS RPC M-DFS RPC local FS 18 ASWN 04, August 11, 2004

19 Future Directions Conclude the proposed implementation and experiment with the design suggestions. Assess performance of different design choices Experiment with the M-DFS for ephemeral sharing over ad-hoc testbed using Bluetooth Explore alternatives to NFS to base design of M-DFS, e.g. HTTP-based approach 19 ASWN 04, August 11, 2004