Distributed File System Choices: Red Hat Storage, GFS2 & pnfs

Transcription

1

2 Distributed File System Choices: Red Hat Storage, GFS2 & pnfs Ric Wheeler Architect & Senior Manager, Red Hat June 27, 2012

3 Overview Distributed file system basics Red Hat distributed file systems Performance profiles How to choose Future work

4 What Are Distributed & Shared Disk File Systems? Shared disk architectures Each cluster member shares the same SAN storage No external clients Applications run on cluster members Client/Server architectures Servers provide data for multiple clients Clients access storage only via the server Applications runs on the clients

5 Shared Disk Architecture Server 1 SAN Server 2 SAN Server 16 SAN

6 Shared Disk File System Access Protocols Looks just like a local file system Provides full VFS semantics to applications Many applications developed on non-clustered systems can run without tuning Complexity of design makes performance tuning tricky Can layer either NFS or Samba on top to provide traditional client/server access Many high end, commercial NAS appliances are shared disk internally

7 Client/Server Architecture Client 1 Client 2 LAN Client 500

8 Client/Server File System Access Protocols NFS: Network File System IETF standard evolved in the UNIX world Implemented on a variety of platforms Newest version in the works is 4.2 CIFS: Common Internet File System Evolved out of the Microsoft SMB protocol Implemented by Microsoft server natively and by Samba on other platforms Newest version in the works is SMB3.0

9 Scale Out Servers Client 1 Server 1 Client 2 Server 2 LAN Client 1000 Server 50

10 Scale Out Storage Scale out breaks out of the big storage model Most NFS enterprise clients use large NAS appliances All shared disk file systems use SAN attached arrays Scale out is designed to run on commodity hardware with local storage Normally accessed by client/server protocols like NFS or CIFS Often supports object access like Amazon S3

11 Red Hat Distributed & Shared Disk File Systems

12 RHEL Resilient Storage - GFS2 Layered Product Supported in RHEL5.5 and later Part of Resilient Storage add on Supported Configurations Scales up to 100TB Supports from 2 up to 16 servers in a GFS2 cluster X86_64 architecture only RHEL6.3 adds support for running Samba on top of GFS2

13 RHEL NFS Client Support Standard part of RHEL Robust support for NFS client and servers RHEL6.2 added tech preview support for parallel NFS Segregates the duties of the meta-data server and data path Client queries for layout and then can stream data from multiple sources in parallel File layout only in 6.2 Close cooperation and performance tuning with enterprise vendors

14 Red Hat Storage Overview Red Hat Storage is a stand alone product Red Hat Storage supported servers have RAID HBA & locally attached storage A number of XFS file systems Dedicated RHEL6.2 servers and gluster software Client access Native gluster access with client software installed NFS & CIFS (aka SMB) support Much lower costs than traditional enterprise gear Multiple talks during summit for details

15 Performance

16 RHEL NFS Performance Enterprise servers have multiple performance features Large, non-volatile write cache to mask disk latency Internally tiered storage Offloads snapshots, remote replication, dedup, etc Can construct a RHEL NFS/Samba server Best workload Well suited to most workloads Can run transactional DB's using O_DIRECT Worst workload would be random reads

17 RHEL GFS2 Performance GFS2 performance depends on many factors Server class used as a member of the GFS2 cluster Shared storage type and SAN topology Especially sensitive to file access pattern! Best workload Uses distinct sets of the files & directories per node Streaming read or write to existing files Worst Workload Actions that bounce locks between nodes

18 Red Hat Storage Performance Performance wins Lots of disks & servers can work in parallel Support for high speed fabrics and storage Performance challenges No gigantic, battery backed write cache to hide disk latency User space file systems add a little extra latency as they transition from kernel to user space Best workload: large, streaming read/writes Worst workload: transactional databases/random IO

19 Choosing!

20 GFS2 Best Fit GFS2 is focused on highly available, tightly coherent applications Scales up to 100TB in size per file system Shared by no more than 16 nodes Designed to run applications directly on servers GFS2 runs in production at many mission critical customers Needs careful review of configuration and SAN backend

21 RHEL NFS Client NFS performance is pretty general purpose Maximum size is vendor specific Can scale up to a fair number of clients Choice of multiple vendors across the board Including building your own single node NFS server on RHEL! Can get costly at large capacities

22 Red Hat Storage High performance if the workload is appropriate Not a fit for transactional workloads Scales to hundreds of clients & several PB Can add capacity dynamically Relatively easy to setup and test with commodity hardware Can mitigate some performance issues by server component choice Affordable cost

23 Combining Technologies File systems can be combined Add Samba or an NFS server to GFS2 to make a NAS appliance Run glusterfs on GFS2 servers to export them Lots of possible combinations Red Hat Storage selects, tests and tunes the selected combination Focus is on affordable, easy to use scale out storage!

24 Future Work

25 RHEL7 and Upstream NFS Work Experience with NFS 4.0 and 4.1 growing Upstream code has support for all three pnfs layouts NFS servers Work to resolve lock recovery deficiencies with cluster backend Work to add NFS 4.2 features FedFS, labeled NFS and server offload Go to the NFS campground Thursday at 4:50PM NFS protocol: NFSv4, NFSv4.1, pnfs, Secure NFS

26 CIFS and SMB3.0 Microsoft is moving rapidly to SMB3.0 Specification will be finalized when Windows 8 Server ships Learns from NFS lessons and refreshes SMB Good support for clustered servers Linux support Samba support for SMB2.1, SMB3.0 dev underway CIFS Linux client currently supports only SMB1, SMB2.1 support aiming for upstream 3.7 kernel

27 Red Hat Storage See the various summit talks for details Upcoming focus will be on Increased scalability of algorithms Enhanced support for virtual guests Tighter integration with Samba for Windows guests Will benefit directly from the NFS team's work Looking at using the in kernel NFS server to gain support for NFSv4 and later

28 GFS2 Upstream Work Upcoming focus will be on Improved logging to reduce journal overhead Block reservations to decrease fragmentation Multipage writes to improve IO performance Improved resilience and performance metrics Streamlined user space tools replace GFS2 tools Work on resource groups to boost statfs performance Will land first upstream Might land in RHEL6 as a backport

29 Stay connected through the Red Hat Customer Portal Red Hat Storage- Overview of GlusterFS Technology Watch video Installing & Configuring the Red Hat Storage Software Appliance (RHSSA) Read Ref Arch access.redhat.com

30 Resources Visit the Red Hat Customer Portal for content Visit storage alley and meet the core architects! Talks Wed 10:40 - A Deep Dive into Red Hat Storage Wed 2:30 Distributed File System Choices Wed 4:50 & Thurs 1:20 GlusterFS Overview Thurs 10:40 Introduction to Red Hat Storage Thurs 2:30 The Future of NFS Thurs 4:30 - NFS protocol (Campground) Thurs 4:50 Red Hat Storage Roadmap & Futures Fri 9:45 Red Hat Storage Performance