Distributed Data Storage Based on Web Access and IBP Infrastructure Lukáš Hejtmánek Faculty of Informatics Masaryk University Brno, The Czech Republic
Summary New web based distributed data storage infrastructure built on the top of an IBP (Internet Backplane Protocol) Model overview distributed I-Node concept and directory services Security aspects Prototype implementation overview Performance tests Pilot groups
Pilot groups Multimedia applications read performance read and write access, no rewriting
Model Overview I-Nodes IBP byte array XML serialization (exnode) End-to-End services (encryption, checksums, compression) Stored on AFS volume one AFS server shared by all IBP depots
Model Overview Directory services Collection of I-Nodes ACL based security CVS-like versioning (no rewrite in IBP)
Model Overview Web based directory service upload download remove rename attributes Metadata permission XML Meta data FILE Data Authorization Service AFS storage IBP depots Web browser lors tools user
Internet Backplane Protocol developed at University of Tennessee Best effort block storage service Time limited storage Supported operations: read, append, manage Security based on capabilities User space solution
Network Storage Stack Applications LoRS Logistical Runtime System L-Bone exnode IBP Internet Backplane Protocol exnode Metadata collection of capabilities of allocated IBP byte arrays essential for file access IBP I-Nodes
exnode capabilities <exnode:metadata name="exnode_offset" type="integer">0</exnode:metadata> <exnode:metadata name="logical_length" type="integer">1347975</exnode:metadata> <exnode:read>ibp://test5.fi.muni.cz:6714/ibp-896306328... 7028/798203447/READ</exnode:read> <exnode:write>ibp://test5.fi.muni.cz:6714/ibp-896306322... 7028/679207605/WRITE</exnode:write> <exnode:manage>ibp://test5.fi.muni.cz:6714/ibp-896306320... 8870/1089394976/MANAGE</exnode:manage>
exnode End-to-End services <exnode:function name="aes_encrypt"> <exnode:argument name="key" type="string">a0209b203920e9206e20e820c5208920 </exnode:argument> <exnode:argument name="blocksize" type="integer">1048592 </exnode:argument> </exnode:function>
Security Aspects Evil Internet assumption Insecure low level media Different from local storage HTTPS connections into Web infrastructure IPsec tunnels among Web servers and IBP depots AES/DES encryption of IBP byte arrays Security model available to all users
Prototype Implementation Overview User AFS Web Servers lors tools IBP IBP IBP I Nodes Directory Service L Bone IBP depot pool
Prototype Implementation Overview User interface Web based file manipulation Upload and Download Permission changes IBP related changes duration, replication level
Prototype Implementation Overview Software implementation IBP depots modified L-Bone server (load balancing) Web server with mod backhand IPsec tunnels lors tools
IPsec HTTPS User Web Servers IPsec lors tools IPsec AFS IBP IBP IBP L Bone IBP depot pool
Prototype Implementation Overview IBP access library libxio provides standard UNIX I/O interface open(2) read(2) write(2) lseek(2) stat(2), fstat(2), lstat(2) close(2)
Crash scenarios AFS server crash no files available through web access all files available through lors tools if user has XML metadata L-Bone server crash all files available for read, writing not possible IBP server crash only particular files unavailable for read, writing possible
Performance tests Testbed 3 IBP depots and Web servers running on Intel Pentium 4 with 1GB RAM internal hardware RAID 5 SCSI disk array internal hardware RAID 5 PATA disk array external hardware RAID 5 PATA disk array 5TB disk capacity 1Gb ethernet connection up to 32 CPUs from Meta Center cluster as clients focus on read performance
Brno Cl. #2 Cl. #1 Cl. #3 Cl. #4 1Gb 1Gb 1Gb 1Gb router 400Mb Gb Switch 1Gb 1Gb WAN 1Gb Gb Switch 1Gb Prague
800 700 Aggregate bandwidth [Mbps] 600 500 400 300 200 100 0 2 4 8 16 32 64 128 Number of clients
Pilot groups Lecture recording at Faculty of Informatics, Masaryk University National Library in Prague storage of large electronic archive of books, maps, etc. National Library in Prague storage of MrSID files Offer storage capacities to all the Czech academic community
Future work Integrating web access with Linux filesystem via LUFS project Generic server interface (i.e. processing video file formats) Enhance reduced semantic to be close to distributed filesystem like AFS
Conclusions Scalable distributed infrastructure based on web access and IBP Suitable for number of clients Large capacity Easy access