Storage Research in the UCSC Storage Systems Research Center (SSRC)

Transcription

1 Storage Research in the UCSC Storage Systems Research Center (SSRC) Scott A. Brandt Computer Science Department Storage Systems Research Center Jack Baskin School of Engineering University of California, Santa Cruz UC Santa Cruz

2 SSRC Overview Systems oriented storage research center Supported by low level research in materials, devices and interconnects Funded by DOE and NSF, and industry sponsors 10 faculty members Others involved as associates External researchers as affiliates Faculty growth in databases & security Significant educational component Undergraduate and graduate storage and related systems courses graduate and undergraduate students Close cooperation with industry research sponsors HP, IBM, Microsoft, Intel, Agile/OnStor Actively pursuing others CITRIS SSRC Talk 2

3 Primary SSRC Faculty Darrell Long, Director High performance storage systems, distributed systems Patrick Mantey Sensor networks, data acquisition, and multimedia systems Scott Brandt High performance storage systems, distributed systems, soft real time systems Ethan Miller Scale & security in file systems, next generation file system design CITRIS SSRC Talk 3

4 Other SSRC Faculty Alexandre Brandwajn (Performance Analysis) Performance modeling and analysis of computer systems Katia Obraczka (Networking) Network architectures for large scale storage Hamid Sadjapour (Coding Theory) Storage system reliability and availability Raymie Stata (Web Archaeology) Mechanisms to preserve and mine multi terabyte data sets Claire Gu (Optical Storage) Phokion Kolaitis (Logic and Databases) CITRIS SSRC Talk 4

5 SSRC Storage Research Challenges Huge Capacity and Scalability Internet Archive Genomic databases Performance Ever increasing gap between CPU and secondary storage Security Networked storage introduces many new security issues Portability Power management, disconnected operation, new devices New storage technologies MEMS, MRAM, Flash, etc. Large scale information management CITRIS SSRC Talk 5

6 SSRC Research Thrusts 1. Object based Storage Scalable high performance distributed storage 2. Archival Storage Large scale on line disk based storage systems 3. New Storage Technologies Systems level research in new storage technologies 4. Predictive/Adaptive Techniques Machine learning based techniques for increasing performance and reducing I/O latency and traffic 5. Secure Storage Secure file systems techniques and systems CITRIS SSRC Talk 6

7 1. Object based Storage Scalable High Performance Object based Storage from Commodity Components 2 Petabytes 100 GB/sec aggregate throughput Parallel accesses from up to 10,000 clients Possibly to the same file Files: bytes to terabytes 1 10,000+ files/directory 50 µsec access times Mid performance local access by visualization workstations Wide area access Client Client Client High speed networks Metadata Server Cluster Metadata Server Metadata access Metadata Server Metadata update Storage area network Direct data access Metadata Server High speed networks Object Based Storage Device Object Based Storage Device Object Based Storage Device Disk array Disk array Disk array Disk array Disk array Disk array CITRIS SSRC Talk 7

8 1. Object based Storage: OBFS Object based Storage Manager Design Principles: Flat object name space Variable block size with fixed maximum (common) Object ID Hash Table Onode ID Region Head Free Onode Bitmap Co located Data Blocks and Onodes Large Block Region Disk Boot Sector Region 1 Region Head Free Block Bitmap Onode Table Free Onode Bitmap Data Blocks Small Block Region Region 2 Region n OBFS outperforms Ext2/3 and meets or exceeds the performance of XFS with 1/25 the code CITRIS SSRC Talk 8

9 1. Object based Storage: Lazy Hybrid Metadata Management Efficient, flexible, scalable metadata cluster management Filename hashing Efficient Avoids hot spots Directory hierarchy Provides standard hierarchical directory semantics Lazy policies Efficient metadata operations Dual entry Access Control List (DACL) Server side permission caching Update Logging CITRIS SSRC Talk 9

10 1. Object based Storage: Reliability in Large Scale Storage More disks reliability problem Disk failures Non recoverable bit errors 1 in ~10 15 bits Large disks long rebuild time Capacity outpaces data transfer rate RAID alone cannot solve the problem Solution for Disk failures Configuration for a redundancy set 2 way or 3 way mirroring RAID5+1 Fast Mirroring Copy Lazy Parity Backup Solution for Bit errors Signature scheme Mean Time To Data Loss of a 2PB storage system by using three configurations and fast recovery mechanisms. The upper lines are for a system built from disks with 10 6 hour MTTF, and lower lines are for a system built from disks with 10 5 hour MTTF. CITRIS SSRC Talk 10

11 1. Object based Storage: Robust Data Distribution OBSDs are added to the system in groups Allocation/Reallocation Objects are placed in the new group with a probability equal to the fraction of the total number of OBSDs in the new group Lookup If an object isn t found in the newest group, the next newest group is checked as if the newest group does not exist In memory very fast This figure shows placement of an object into a system with three groups. In this case, the object didn t get placed in Group 3, or Group 2, so it will be placed in Group 1. CITRIS SSRC Talk 11

12 2. Archival Storage: Deep Store u Efficient On line Deep Store Reference File Differentially Compress Data (100:1) Disk based for on line performance u u u u Search for similar files Compress against existing data Organize similar files using data clusters Scale to billions of files Delta Encoded (Compressed) File = + = = + Reconstructed File Differentially Compressing Files Breakpoint FD 34 FF FE 1D 1D 7F 7F 4A 12 0 mod n Similarity Data Clusters 0 mod n Token (breakpoint bounded) Content Based Hash Table = 0 mod n Rabin Fingerprint File/Fragment Storage Organizing Similar File Data Shingling (overlapping fingerprints) CITRIS SSRC Talk 12 =

13 3. New Storage Technologies: MEMS based Storage MEMS based storage: very dense nonrotating orthogonal magnetic or physical recording 2D seeks Large # of active read/write tips Device Modeling Power management Aggressive spin down, sequential request merging, subsector accesses 50% lower power consumption with no performance penalty Storage Subsystem Architectures MEMS metadata storage and MEMS disk write buffer Performance MEMS alone Request scheduling Zone based Shortest Positioning Time First SPTF like response times and C SCAN like variability Storage Allocation Zone based allocation (in progress) CITRIS SSRC Talk 13

14 3. New Storage Technologies: HeRMES MRAM based Storage Magnetic RAM: Fast nonvolatile DRAM like storage How to use MRAM in a file system? Combined disk/mram file system File systems for mobile devices MRAM metadata storage and data caching Online metadata and file compression Bytes per Inode Ext2 Native Conquest Like All/Nothing Average Inode Types All/Nothing BEST All/Nothing WORST Bytes % Huffman Best Huffman Worst 100.0% 90.0% 80.0% 70.0% % Improvement over Ext2 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% CITRIS SSRC Talk 14

15 4. Predictive/Adaptive Techniques Dynamic techniques improve application and system performance Much better than static parameter tuning Formal Machine Learning based approach Profs. Manfred Warmuth and David Helmbold Problems identified/examined File access pattern prediction for prefetching and grouping Cache management algorithm selection Disk spin down timeout selection File lifetime prediction Network congestion control CITRIS SSRC Talk 15

16 4. Predictive/Adaptive Techniques: Predictive Prefetching/Data Grouping Recency based models track file access patterns Successor information maintained in metadata Prefetch related files or groups Reduces storage latency and increases cache effectiveness Predictors: Finite Multi Order Context (Kroger) Noah aggregating cache (Amer) Program based Successor (Yeh) Current research Hoarding for mobility CITRIS SSRC Talk 16

17 4. Predictive/Adaptive Techniques: Adaptive Caching Best cache management policy changes over time Workloads change Filtering occurs Cache relationships change Solution: Dynamically choose best policy Machine Learning: Fixed Share to Uniform Past Refetching helps CITRIS SSRC Talk 17

18 4. Predictive/Adaptive Techniques: Adaptive Caching Results 28% fewer cache misses than LRU 8% fewer than BestFixed 4 24% reduction in I/O traffic CITRIS SSRC Talk 18

19 5. Secure Storage: Secure Network Attached Storage For each file block on disk, keep sufficient information to 1. Decode the data (at the client) 2. Validate the sender of the data 3. Ensure data integrity Use encryption to keep data secret on disk and in transit Decryption occurs at the client Information to decrypt available only at client! Prevent compromise of data Impossible to protect against denial of service Loss of data may occur make sure it s noticed! Three similar security schemes Trade off resistance to intrusion for speed Speed (MB/s) Read performance Block size (KB) Seq (raw) Seq (S1) Seq (S2) Seq (S3) Random (raw) Random (S1) Random (S2) Random (S3) CITRIS SSRC Talk 19

20 5. Secure Storage: Intra file Security (IFS) IFS end to end file system encryption technology Encrypts independent file extents Flexible encryption region size Files may contain one or more isolated or overlapping secure regions Transparent to the user Supports strong encryption L L L L L Insecure Secure 1 Secure 2 CITRIS SSRC Talk 20

21 Summary The UCSC Storage Systems Research Center is becoming a nationally recognized storage systems research group Darrell Long recently founded the Conference on File and Storage Technology (FAST), already the premier storage systems research conference We are actively recruiting faculty and students to participate in SSRC research activities We are actively soliciting corporate sponsorships and research relationships CITRIS SSRC Talk 21