SerNet. Clustered Samba. Nürnberg April 29, Volker Lendecke SerNet Samba Team. Network Service in a Service Network

Transcription

1 Clustered Samba Nürnberg April 29, 2009 Volker Lendecke SerNet Samba Team 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 1

2 Volker Lendecke Co-founder SerNet - Service Network GmbH Free Software as a successful business model Network Security for the industry and the public sector Samba-Support/Development in Germany For 20 years concerned with Free Software First patches to Samba in 1994 Consultant for industry in IT questions Co-founder emlix GmbH (Embedded Systems) 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 2

3 What is Samba? Interoperability between Windows and Unix systems Most protocols Windows speaks today SMB (File Sharing), Printing, Browsing, Authentication Samba makes unix machines show up in Network Neighborhood Samba runs on most Unixes these days Main development platform is Linux Solaris, AIX, HP/UX, Stratus V/OS, Tru64, etc... 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 3

4 Samba 3 Stable version of Samba, production code Solid file and print server NT and Active Directory domain member NT4-compatible Domain Controller Security and access control model based on Unix Flexible posix-based VFS modules to change file access semantics Current version: 3.2.8, /2009, Volker Lendecke, SerNet Service Network GmbH, Seite 4

5 Samba 4 Started in 2003 by Andrew Tridgell as a new VFS system with the goal to support cluster file systems Complete re-write of Samba Target: 100% semantics of Windows Main feature today: Active DirectoryDomain controller Most useful component for development: smbtorture, a very comprehensive SMB test suite 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 5

6 Why Samba4? In 2003 the code base of Samba3 was quite convoluted Clustering requires major internal changes, Tridge has seen it as too much work to adapt the historic code base Sven Oehme, IBM, put me on exactly that task Huge amounts of internal cleanup since In July 2006 I could present an early prototype of clustered Samba3 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 6

7 What is SMB/CIFS? Originally a development of Barry A. Feigenbaum, IBM to support redirecting drive D: to a NetBIOS based network server Initial name was BAF-Protocol, later renamed to Server Message Block or SMB In 1996 the Internet was hyped, so SMB was renamed to Common Internet File System Compatibility to ancient applications is the primary goal 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 7

8 Clustering CIFS The easy part: Active/Passive clusters The holy grail: Share the same clustered file system via different Samba nodes, and scale linearly NFS is relatively easy: No locking around GFS/GPFS do it for Posix semantics Samba needs to fake SMB semantics, which are particularly hard to get right and fast I'm not aware that any correct CIFS cluster is available on the market except Samba/ctdb 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 8

9 Posix vs CIFS semantics Unix historically (VERY early) did not have proper IPC Applications had to cope with very few atomic operations like creating a file Locking until today is very rudimentary SMB/CIFS was designed for compatibility Existing single tasking applications needed to be protected against each other Share Modes provide exclusive access to open files 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 9

10 Share Modes On Posix, opening a file is an isolated operation that no other cluster node needs to know about Easy to get fast in a cluster file system Modifying a directory (create, rename, unlink) requires exclusive access, this gets slow For CIFS, every file open needs to be communicated Does another node have a conflicting file open mode? 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 10

11 The ctdb trick Samba uses a trivial database to hold file open information Ctdb is clustered tdb Traditional clustered databases provide very high consistency guarantees -> several orders of magnitude too slow for Samba CTDB can loose data! If a node dies, the files held by that node are closed by definition 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 11

12 CTDB architecture 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 12

13 Other CTDB tasks Complete HA solution, based on GPFS at this time Cluster membership detection via a byte range lock on a shared file (no OCFS2) IP Address takeover, Service control TCP tickle acks Necessary for extremely fast recovery of Windows clients, they might be stuck in a state where they wait for data from a dead node 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 13

14 Persistent databases Not all tdb files are as highly dynamic as locking db's are. secrets.tdb: Domain membership registry.tdb: Holds Samba version of registry Persistent tdb's are handled differently: Everyone has an up-to-date copy, writes are broadcast With configuration in the registry, keeping all nodes the same becomes easy 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 14

15 Registry configuration Parsing and writing smb.conf files with GUI tools is awkward at best Samba 3 has to implement a registry, clients expect to find certain keys to determine the server type Registry data model matches exactly smb.conf format, it was designed as a.ini file replacement HKLM\Software\Samba\smbconf Enabled only if config backend = registry is enabled in the smb.conf text file 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 15

16 NTFS ACL support Samba 3 is a good Unix process Access Controls are done by the kernel, Samba does su - <authenticated user> Posix ACLs are very limited Only rwx, NFSv4 modeled after the 13 NTFS rights Posix can't delegate right to set ACLs Samba 3.3 contains acl_xattr and acl_tdb modules Optional full NTFS Acls in user space 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 16

17 Current status Clustered Samba is in production at many sites SambaXP has 5 talks on clustered Samba, only two of them will be by IBM :-) Not only run on GPFS, working installation on GFS OCFS2 won't still do it, it's missing coherent locking Working on optimizations to work around syscall latencies in clustered file systems (unlink, ftruncate and even open can take very long) 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 17

18 Work to do Integrate into other HA solutions Modularize Samba better to adapt to file systems with special semantics beyond posix Change the CIFS file server world :-) 04/2009, Volker Lendecke, SerNet Service Network GmbH, Seite 18

19 Questions/comments? Volker Lendecke, SerNet Service Network GmbH Bahnhofsallee 1b Göttingen Tel: Fax: /2009, Volker Lendecke, SerNet Service Network GmbH, Seite 19