NDS2 Secure storage, sharing and publishing of data in the NDS

Transcription

1 NDS2 Secure storage, sharing and publishing of data in the NDS Maciej Brzeźniak, Supercomputing Dept. of PSNC, TF-Storage Sep., 26-27th 2012 Project funded by: NCBiR for under KMD2 project (no. NR /2011) Project partners 10 Polish universities and supercomputing centres:

2 NDS2 -presentation plan Background and project status NDS and BADSS NDS2 NDS2: Secure storage and sharing Secure storage clients: ndscryptofs (Win/Linux) Java GUI, CLI, library and Android Appliance virtual/physical for institutions Client-side encryption & integrity control Concept and some details Performance Secure sharing inside NDS2 concept and keys management Secure publishing general information 2

3 NDS ( ) Background: NDS & BADSS (1) R&D project: distributed, replicated data storage Virtual Filesystem in user space implemented using FUSE library Standard user interfaces: SFTP (SSHd), WebDAV, Web application, GridFTP Replication: Automatic, system-side, synchronous and asynchronous Performed using NFS (local replicas) and GridFTP (remote ones) protocols Funded from national sources BADSS ( ) Deployment of NDS for academic community 10 sites in Poland Tapes: 12,5 PB in 5 sites Disks: 2 PB Funded from EU structural sources PLATON project

4 Background: NDS & BADSS (2) Assumptions for NDS and experience from NDS deployment: No needfor dedicatedaccesstools OK for users, BUT No encryption of the data supported by system: Data encrypted only during transfer (SSL) and on tape media (LTO5 encryption) + disks de-magnetization Users may encrypt data on their side: Manually impractical with large data Automatically with external tools, that supports on-the-fly encryption POSIX-like access to data: Linux: SSHfs works for most use-cases => OK Windows: Problems with native Webdav client in some versions of Windows To have a stable solutons for accessing big files extra (paid) clients are needed => possibly it s best to provide your own client (however it s not easy)

5 NDS2 project status NDS2 ( ) extension of NDS project( ) NDS2 = NDS reliable, replicated and distributed storage + secure storage & sharing & publising + versioning + ACLs support + user management de-centralisation Progress: Some prototypes worked out already: nds2cryptofs 4 Linux and Windows Android client without encryption Some are under development: Java-based GUI application Appliance for institutions Android client with encryption Project partners 10 Polish universities and computing centres Funded by: NCBiR for under KMD2 project (no. NR /2011)

6 Clients for NDS2 Assumptions: We address most popular platforms (Windows, Linux) with native client providing POSIX-like access to data Linux user Windows user Appliance for institutions GUI and Android user JAVA GUI can be used for remaining plaforms Android client as a proofof-concept for mobile users (currently no plan for IOs) Clients being developed: SSHFS + extensions Commercial FUSE-like and SFTP libraries SSHFS + extensions Java crpto libraries nds2cryptofs 4 Linux nds2cryptofs 4 Windows Java-based GUI application Appliance for institutions NDS filesystem + supportfor encryption keys mgmt Android client 6

7 NDS2: Client-side cryptography Linux: SSHFS + extensions FUSE-based project We patched the SSHFS code: it calls cryptographic functions (encryption & digests) while serving read and write operations of VFS layer Prototype exists! Ready for testing. Linux user SSHFS + extensions WAN

8 NDS2: Client-side cryptography Windows: Commercial Virtual FS library(fuse-like) and commercial SFTP client library Why we use paid libraries?: Portability among diferent versions of Windows Wanted a quick win and the working solution ASAP We focus on cryptography and feautres on top of the filesystem (secure storage, sharing, ACLs ) Virtual FS library: We considered DOKAN but the project looks not to be well maintained SFTP library: Open source libraries have serious performance limitations Windows user Commercial FUSE-like and SFTP libraries WAN Client prototype exist! Ready for testing.

9 NDS2: Client-side cryptography GUI application (1) Allows storage & retrieval of files and provides filesystem structure view: Put, get, move, delete etc. Drag & drop support Sharing management: Initialisation and control of sharing SHARE DIRECTORY creation Assigning the directory with the sharing keypairs Access control lists management (ACLs) Advanced, user-level metadata access and management: (Automated) annotation, tagging control etc. Meta-data based search (free form/structured) (on the roadmap) Implementation: Java library (used for CLI, GUI and Android app.) Shell integration for Windows and Linux (on the roadmap) Data and filesystem meta-data Operating system supporting JAVA Java crpto libraries WAN User meta-data & control

10 NDS2: Client-side cryptography GUI application (2) screenshot of the prototype (Polish version)

11 Client-side cryptography Appliance for institutions idea: REMOTE STORAGE SPACE: storage space in NDS2 system VFS with transparent, on-the fly encryption and digests LOCAL STORAGE SPACE: local storage for local usage people need it anyway; e.g. workspace for users within the small organisation exported to LAN by SMB protocol; LOCAL and REMOTE spaces synchronized (scheduled or on-demand) Appliance administration basic web console: Defining storage, shares and backup/synchronization schedules Managing user accounts User accounts: Appliance is the user of NDS2 system Internal accounts may be taken from LDAP or defined manually Status: concept is still evolving: e.g. should the intenal disk be persistent storage or cache only? Local disk space NDS filesystem + support for encryption keys mgmt Appliance for institutions LAN SMB server SSHFS + extensions WAN Remote space cryptofs

12 Client-side cryptography Appliance for institutions possible implementations: Box for small groups/ instiutions Small (19,5x70x18,6cm) and silent, green(fits below the desk): CPU with AES-NI support (not a problem these days) 2 x 2,5 HDDs or 2x green SSDsinside (up to ~ 2 TB of RAW internal storage) Must be cheap! e.g. ~600 EUR/box (not more than PC) Rack server for bigger institutions Rackserver: CPUs with AES-NI on board Low voltage! (being green, costs) 4x 3,5 or 8x 2,5 SSD (up to 12 TB of RAW storage) Reasonable costs - ~2500EUR with 12TB of capacity Some fancy hardware for users: Smart cards + readers (expresscard or USB) Psychological trick (works for some users) VMware vappliance Virtual machine: E.g. vapp easy to run on vmware cluster or another VM image No assumptions on hardware just needs LUN for local storage and account in NDS2 for backups and sync s

13 Client-side cryptography Appliance for institutions discussion: RISK analysis Hardware = cost must be included in the service delivery model Hardware = failures too much hassle? outsourcing? but it costs Hardware problem = data loss? Disk failures:» RAIDs helps in case of single disk failure» Frequent backups/sync s protects in case of total crash Server failures:» Data not available at local storage for a while, but NOT LOST» Access to data still possible using software client (keys needed)» Certificate for authorisation securely stored on smart card (SC)» MASTER keys for encryption on SC (future) or other media (e.g. SD)» Appliance configuration data on SD card and/or in the remote storage» Hardware may be easily exchanged Experimental work we will build some prototype and check users buy-in

14 NDS2: Client-side cryptography Android application: Challenge 1: User-friendly, intuitive interface Core functionality only simplicity: Data storage and retrieval Remote filesystem s view and file access» Local caching of files e.g if user reads PDF file Device memory/storage view» Click to upload to NDS2 Interface integration:» e.g. send to NDS2 function in file browser No sharing, user-level metadata mgmt etc. At least not in 1st approach Challenge 2: Encryption / digests performance / battery: Benchmarks for ARM CPUs promising comparing to WLAN bandwidth AES support planned for ARMv8 architecture Encryption may exhaust battery However, note that typically Android will be used for small files (PDFs, DOCs, photos etc.) Again, an experimental work: Proof-of-concept for Android V1 screenshot Android OS Data and filesystem meta-data Java crpto libraries WAN

15 Secure data storage (1) Confidentiality thanks to encryption Integrity control thanks to digests Assumptions and decisions: We need performance > symmetric encryption (AES 256 counter) We don t want to keep per-file keys on the user side let s store them in the system (encrypted) MASTER KEY asymm. + symmetric FILE KEYs Converged encryption? NO, we generate key for each file Strong digests (i.e. preventig us from collisions) SHA512 calculated per block (64k) File format overhead: 512bits=64Bytes / 64kBytes = 1/1024 = 0, % Issues: Only the user has a MASTER KEY: User is responsible for proper key protection -> may be a problem for less aware users Some tools needed, e.g. backup to USB dongle or SD card feature of GUI application User education needed!

16 Secure data storage (2) File format: Header: File s symmetric key encrypted with user s public key In default setup (no sharing), only the owner may know the secret symmetric key While sharing, additional headers stored in the system; they contain the file key encrypted with sharing users public keys N x data block, each block has: Data part: encrypted with the unique symmetric FILE KEY generated per file Control part: contains digest, encrypted by the FILE KEY

17 Secure data storage (3) Performance: Encryption: AES256 counter: Regular PC reading data + encrypting(no writes): Intel Core 2 Duo E GHz, RAM: 4 x 1024MB DDR2, Fedora bit, kernel or Win 7 Pro SP1 64 bit» C + openssl, Java + Bouncy Castle ~46 MB/s» C# + Bouncy Castle 20 MB/s» C# + Commercial Library 47 MB/s in-ram operations faster:» C + openssl ~100 MB/s More if you exploit AES-NI Intel Core i7 3.4GHz 8GB RAM Ubuntu11.10 SFTP (AES-256): C + openssl 150 MB/s In-RAM encryption ~ 1 GB/s (or more) Most libraries (and Java VM) can exploit AES-NI! SHA512 digests: Regular PC: 2 x Six-Core AMD Opteron GHz (2 cores used by VM), 2 GB RAM, Centos 5.6 gcc: 4.1.2, OpenSSL: 0.9.8e, Boost: , ~ 1,2 GB/s

18 Secure data storage (4) comparison with other solutions discussed during meeting Feature Trusted Cloud Drive SpiderOak NDS2 Encryption keys for files Symmetric Symmetric Symmetric Whoknowsthe secretkey Namespaceencryption? Trusted Cloud Drive provider (on-premises) Namespace (meta-data) stored on-premises Only end-user Yes Only-end user Secure sharing Planned Planned Planned Dedicated client needed No Yes Yes Filesystem-like access? (drive letter/ local mount) Mobileclient Files cut to fragments while stored in the system? Win: BitKinex / other Linux: DAVfs WebDav file manager, other WebDAV clients No No Custom Yes, files stored in blocks in the system No, considered for 2nd approach Yes: ndscryptofs4 Windows and Linux Custom (Android in 1st approach) Only logically, one file on the storage Assummed storage back-end Disks / clouds Disks HSMs and/or disks

19 Safedata sharing& publishing(1) Secure data sharing with internal NDS2 clients ACLs for physical access control to files Encryption + appropriate keys handling for security; assumptions: Users don t trust the service provider Some users from the same institution (it may be large see universities) don t want to use the same encryption keys as their colleagues Current concept (still evolving): File encryption keys protected by per-directory or per-user-group key pair (called PROXY KEYS), known to the sharing group Keys exchange automatic, however initiated by share owner using GUI: The user (owner) creates SHARE directory and defines members of sharing group (determined by setting ACLs to individuals or for a system group) PROXY keypair is associated to this SHARE, then securely exchanged with allowed users using their public keys PROXY private key is needed to access the FILE KEYs for the shared files PROXY public key is needed to create new files in shared folder Will work for all interfaces: ndscryptofs for Linux, Windows, GUI and Android

20 Safe data sharing & publishing (2) Sharing data with external World Secure sharing: FileSender-like use-case: Temporary storage of data Import/export voucher Multiple files per voucher Encryption & digests supported Planned deployment: SANDBOXed environment: Servers/infrastructure separate from NDS2 for security No interactions from sandbox->nds2; all actions initiatied from NDS2 side Possible integration with FileSender (open topic) Secure publishing: Data put to the public www servers for permanent storage Read-only access from World Publication link generated by the system to be shared with other users, put to CMS or to website Planned deployment: dedicated, publication SANDBOX: Infrastructure separate from NDS2 for safety reasons Multiple www servers and storage systems + load balancing + client proximity for performance reasons No interactions from sandbox->nds2; all actions initiatied from NDS2 side

21 More information: NDS: National Data Storage: nds.psnc.pl BADSS: Backup and Archival Data Storage Service for Polish Science: storage.pionier.net.pl PLATON project: