Data Compression and Deduplication. LOC Cisco Systems, Inc. All rights reserved.

Transcription

1 Data Compression and Deduplication LOC Systems, Inc. All rights reserved. 1

2 Data Redundancy Elimination Landscape VMWARE DeDE IBM DDE for Tank Solaris ZFS Hosts (Inline and Offline) MDS + Network (Inline) Data Domain Netapp s A-SIS Array Filer (Inline and Offline) LOC Systems, Inc. All rights reserved. 2

3 s Focus Areas for Redundancy Elimination Cloud Data Center Cloud Deployments Data Center to Data Center Connectivity Branch Office Connectivity Disaster Recovery using long distance replication Data Center LOC Systems, Inc. All rights reserved. 3

4 Challenges in Deduplication CPU Intensive Chunking Operations CPU Intensive Fingerprint Calculation such as SHA-1 Managing Large Indexes in Memory as well as in Persistent Reducing Disk I/O for Index Lookup Load Balancing and Scalability in shared SAN environments Inter Host communication Host based deduplication for Shared File Systems LOC Systems, Inc. All rights reserved. 4

5 s Network Centric Redundancy Elimination Fully Integrated System for lowering WAN OPEX Deduplication Compression TCP optimization Encryption Hardware Assistance for Compression Scope for Hardware Assists in Deduplication Coordination less approach for Shared SAN file systems Application/Host/Array Agnostic LOC Systems, Inc. All rights reserved. 5

6 Interconnecting Data Centers using MDS 9000 Primary Data Center Secondary Data Center DWDM Optical MAN MDS IP WAN MDS SAN Extension WAN Optimization Redundnacy Elimination Security FCIP protocol for SAN extension TCP Optimizations Hardware Compression Encryption LOC Systems, Inc. All rights reserved. 6

7 for Interconnecting Data Centers Data Redundancy Elimination includes Data Deduplication: application-agnostic technique eliminates redundant data from TCP streams providing up to 100:1 reduction Persistent LZ Compression: session-based compression provides up to an additional 10:1 compression even for messages that have been optimized by deduplication LZ WAN LZ Dedupe Dedupe Synchronized Compression History LOC Systems, Inc. All rights reserved. 7

8 DC-DC Replication Using DATA CENTER (Primary) DATA CENTER (Backup) arrays GigE WAN GigE arrays Catalyst 6500 Catalyst 6500 Ease of deployment Integration with existing network topologies LOC Systems, Inc. All rights reserved. 8

9 Replication in WCCP Mode DATA CENTER (Primary) DATA CENTER (Backup) arrays WCCP GigE WAN GigE WCCP arrays Catalyst 6500 Catalyst 6500 Higher availability and scalability through N+1 clustering LOC Systems, Inc. All rights reserved. 9

10 Replication using MDS and DATA CENTER (Primary) DATA CENTER (Backup) arrays MDS 9000 with IP Service Modules MDS 9000 with IP Service Modules arrays FC FCIP Catalyst 6500 Catalyst 6500 FCIP FC GigE WAN GigE Improve end-to-end network resilience (against port failure) Optimize data replication for legacy FC storage Support multiple heterogeneous FC storage arrays LOC Systems, Inc. All rights reserved. 10

11 Trends in Deduplication Use of Solid State Disks to overcome Index Lookup and retrieval latency Price Performance Trade off Design Choices to overcome poor random write throughput of SSDs Random write IOPS is still about 350 Adaptive deduplication that combines Inline and Offline methods Data Agnostic Deduplication Integrated approach for File and Block Deduplication LOC Systems, Inc. All rights reserved. 11

12 LOC Systems, Inc. All rights reserved. 12