EMC Backup/Restore and Data Deduplication

Transcription

1 Kompetera SolutionsDay Brøndby Stadion, September 2012 EMC Backup/Restore and Data Deduplication Michael Hemmingsen Technology Consultant EMC Backup Recovery Systems (BRS) Division 1

2 Our Division Approximately 3,000 Employees 10 R&D Locations Market Leadership #1 In Deduplication #1 In Purpose Built Backup Appliances #1 In Combined Software & Storage Rapid Market Adoption Data Domain & Avamar Exited 2011 At A Revenue Run Rate Over $2 Billion Global Sales, Support, & Services Approximately 6,000 Channel Partners 2

3 EMC And BRS at a Glance EMCis a global leader in enabling businesses and service providers to transform their operations and deliver IT as a service. Revenues (2010): $17B (2011) Projected Revenue: $19.8 Fortune 500 Rank: 166 Employees: 50,000 Countries with EMC operations: 83 R&D investment (2010): $2B Founded: 1979 EMC Backup Recovery Systems (BRS) is a division of EMC focused on backup and recovery solutions Division HQ: Santa Clara, CA Global sales, support, and services 10 R&D locations, 2,000 employees, 6,000 channel partners 3

4 Our Journey to the Cloud/Big Data EMC cumulative 8 year technology investment R&D $10.5B 2011 CLOUD & BIG DATA JOURNEY TO THE CLOUD 2003 AND BEFORE 2005 Virtualization ENTERPRISE STORAGE 4

5 IT Trends Impacting Backup and Recovery Zettabytes Zettabytes DATA DELUGE BUDGET DILEMMA Transformation INFRASTRUCTURE SHIFT 5

6 Is Growing Infrastructure the Answer??? 18 Cabinets of IBM Tape 6

7 Deduplication Is Accelerating the Transition More efficient Reduced storage Less bandwidth 7

8 Deduplication Is Accelerating the Transition Deduplication times less data stored versus fulls + incrementals with typical retention policies 30 Data Stored Weeks in Use Deduplication storage Traditional storage 8

9 EMC Continues to Evolve Backup Purpose Build Backup Appliance (PBBU) - Software and storage designed to work together Unlocks the power of disk Software INTEGRATED ARCHITECTURE Storage Improves Performance:Up to 50 percent faster than traditional solutions Simplicity: One user interface common policy management Predictability:Proven best practices more than 20,000 installations 9

10 EMC Data Domain Leading the Deduplication Market In-use rating 3-times over nearest competitor Reduce: Backup windows by up to 90 percent Data stored by up to 30 times* Replication bandwidth by up to 99 percent *Versus full and incremental with typical retention Source: Wave 15 Storage Study Q2 2011, published 5/16/11, large-enterprise sample, n=31,theinfopro ( 10

11 Disk Based Backup w/data Deduplication ReplacesTape EMC Purpose Build Backup Appliance Data Domain Avamar 11

12 Purpose-Built Backup Appliances Fastest growing segment in backup and recovery market 2010 Total Market $1.69B EMC 64.2% EMC IBM HP Oracle Quantum Sepaton FalconStor Dell Others Source: Worldwide Purpose-Built Backup Appliance Forecast and 2010 Vendor Shares, IDC Doc #228091, May

13 TraditionalBackup Infrastructure 13

14 NextGeneration Data Protection 14

15 NextGeneration Data Protection 15

16 Backup Operations Hassle 20% slow or offline clients 20% post-processes failures 30% media-related failures 10% performance and job-queues 10% network interruptions 10% plugin and other Expand disk pools Inject additional media Library inventory, media-to-pool Help-desk tickets escalated Prepare test & dev environments Prep tape media for pickup Retrieve reports from consoles Retrieve reports scripts Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success/Failures Long-running Jobs Media Errors Restart failed jobs 16

17 Backup Operations Hassle 20% slow or offline clients 20% post-processes failures 30% media-related failures 10% performance and job-queues 10% network interruptions 10% plugin and other Expand disk pools Inject additional media Library inventory, media-to-pool Help-desk tickets escalated Prepare test & dev environments Prep tape media for pickup Retrieve reports from consoles Retrieve reports scripts Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success No tape End-to-end phone home support Dedupe speeds job completion 17

18 Backup Operations Hassle 20% slow or offline clients 20% post-processes failures 30% media-related failures 10% performance and job-queues 10% network interruptions 10% plugin and other Expand disk pools Inject additional media Library inventory, media-to-pool Help-desk tickets escalated Prepare test & dev environments No tape Centralized mgmt & reporting Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success No tape End-to-end phone home support Dedupe speeds job completion 18

19 Backup Operations Hassle 20% slow or offline clients 20% relieved -> No post processes 30% relieved -> No tape 10% relieved -> Performance & flexibility 10% network interruptions 10% plugin and other Expand disk pools Inject additional media Library inventory, media-to-pool Help-desk tickets escalated Prepare test & dev environments No tape Centralized mgmt & reporting Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success No tape End-to-end phone home support Dedupe speeds job completion 19

20 Backup Operations Hassle 20% slow or offline clients 20% relieved -> No post processes 30% relieved -> No tape 10% relieved -> Performance & flexibility 10% network interruptions 10% plugin and other Single pool of storage Help-desk tickets escalated Prepare test & dev environments No tape Centralized mgmt & reporting Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success No tape End-to-end phone home support Dedupe speeds job completion 20

21 Backup Operations Hassle 20% slow or offline clients 20% relieved -> No post processes 30% relieved -> No tape 10% relieved -> Performance & flexibility 10% network interruptions 10% plugin and other Single pool of storage Yes, spend more time doing this Application admins create test & dev No tape Centralized mgmt & reporting Client software/configuration Backup jobs, scheduling adjustments Media servers changes Media purchases SAN/Network changes Success No tape End-to-end phone home support Dedupe speeds job completion 21

22 Backup Operations Hassle 20% slow or offline clients 20% relieved -> No post processes 30% relieved -> No tape 10% relieved -> Performance & flexibility 10% network interruptions 10% plugin and other Single pool of storage Yes, spend more time doing this Application admins create test & dev No tape Centralized mgmt & reporting Less changes Less Risk Better SLAs Success No tape End-to-end phone home support Dedupe speeds job completion 22

23 Failure to Restore is Costly Operational Restore, OK-ish. Disaster Recovery Situation How long to retriveand ready tape clones? Backup is the Life Line when other systems has failed! Backup administrators desktop 23

24 EMC DATA DOMAIN OVERVIEW 24

25 Deduplication Fundamentals 25

26 Data Deduplication: Technology Overview Store more backups in a smaller footprint Friday Full Backup A B C D A E F G Backup Estimated Data Logical Reduction Physical Mon Incremental A B H Tues Incremental C B I Weds Incremental E G J Thurs Incremental A C K Second Friday Full Backup B C D E F L G H A B C D E F G H I J K L FRIDAY FULL 1 TB 2 4x 250 GB Monday Incremental 100 GB 7 10x 10 GB Tuesday Incremental 100 GB 7 10x 10 GB Wednesday Incremental 100 GB 7 10x 10 GB Thursday Incremental 100 GB 7 10x 10 GB Second FRIDAY FULL 1 TB 50 60x 18 GB TOTAL 2.4 TB 7.8x 308 GB 26

27 Retain: Store More for Longer with Less Over one year of retention in 3U of Data Domain deduplication storage Backup Cumulative Estimated Physical Data Logical Reduction First Full 1 TB 4x 250 GB Week 1 Week 2 Week 3 Month 1 Month 2 Month 3 April TB 8x 308 GB April TB 10x 366 GB April TB 12x 424 GB April TB 14x 482 GB May TB 17x 714 GB June TB 19x 946 GB Month 4 July TB 20x 1,178 GB TOTAL 23.4 TB 20x 1,178 GB 27

28 Variable Length: Not Fixed Segment Size Intelligent, self-tuning algorithms automatically adjust to data changes Call me Ishmael. Some years ago -never mind how long precisely -having little Fixed size wouldn t handle inserts/deletes well: Call me Ish. Some years ago -never mind how long precisely -having little Call me Izzy. Some years ago -never mind how long precisely -having little Variably sized segments maximize redundancy: Call me Izzy. Some years ago -never mind how long precisely -having little Advantages: Application independent Protocol independent File pathname independent Block address independent Key: - Consumes new capacity - Deduplicated 28

29 Data Domain Basics 29

30 Data Domain Basics Easy integration with existing environment Control Tier Target Tier Disaster Recovery Tier Backup and Archive Applications CIFS, NFS, NDMP, DD Boost EMC Symantec CommVault IBM HP Veeam Quest NEW: Oracle RMAN Ethernet Virtual Tape Library (VTL) over Fibre Channel DD890 appliance Replication DD890 appliance 2U 2 to 14 ports 10 and 1 GigabitEthernet; 8 Gb/s Fibre Channel RAID 6 Up to 285 TB usable capacity with shelves 2 TB or 1 TB 7.2K rpm SATA HDD in shelf File system NVRAM N+1 fans and redundant, hot-plug power supplies 30

31 Methodology: Inline vs. Post-Process Deduplication INLINE Deduplication Before Storing POST-PROCESS Deduplication After Storing Deduplication Store Deduplication 3x disk accesses to shared store Other activities unimpeded Predictable Simpler The more processes, the more resource contention Copy to tape: Too slow to stream tape Recovery: Service level agreement predictability Replication: Poor time-to-disaster-recovery Deduplication: If interleaved with backup or restore More administration to fight these issues 31

32 Stream Informed Segment Layout 99% of duplicate segments identified in RAM All related segments are stored in close proximity on disk for optimal reads Process: New write comes in i.e. segment Function generates the offsets and compared within the summary vector Summary Vector cannot conclude redundancy It can have false positives Role of the Summary Vector is to figure out if the segment has been seen before or not, If SV test says segment may not be unique, move to next step If new write to container compressed when container full write to disk If not new verify again Fingerprint lookup in cache (rarely) load FPC from disk Key results System capacity is very scalable No RAM constraint on index Good performance with few spindles Very few disk accesses during write 32

33 SISL Scaling Architecture Data Domain answer: SISL Stream-informed Segment Layout includes: Summary Vectorin RAM says if segment is new Segment Localities minimize seeks if answer is on disk Check uniqueness with Summary Vector Check in-memory fingerprint cache Key results See: 33

34 Performance: CPU-Centric vs. Spindle-Bound 6,000 Data Domain Throughput MB/s Fibre Channel Most deduplication vendors SATA Number of Disk Spindles 34

35 Segment Localities Metadata Segment data abcd A B C D efgh E F G H ijkl I J K L... stuv S T U V DDFS log structure Localities Stream-informed storage units Neighboring unique segments stored together Fingerprints and segments stored together with other metadata One seek can retrieve hundreds into RAM Fast caching for fingerprint lookup Fast reads during recovery or copy to tape 35

36 Data Integrity: Data Invulnerability Architecture End-to-end data verification Checksum Deduplication, write to disk Verify Self-healing file system Cleaning Expired data Defrag Verify Other RAID 6 NVRAM Snapshots Generate Checksum File System Deduplication Local Compression RAID Verify Data End-to-end data verification Verify the file system metadata integrity Verify user data integrity Verify stripe integrity 36

37 Fault Avoidance and Containment Localities Data Domain Log-structured File System Architecture New Data Never Overwrites Good Data Previous backups are not at risk Protect against software bugs Fewer Complex Data Structures Mean Fewer Bugs No bitmaps and link counts to corrupt NVRAM for Fast, Safe Restart Snapshot protection option DD-RAID does no partial-stripe writes No RAIDvulnerability to power loss or NVRAM failure 37

38 Continuous Fault Detection and Healing DD-RAID 6 Protection against Two disk failures Disk read errors during reconstruction Operator pulling the wrong disk Verifies data integrity and stripe coherency after writes On-the-fly Error Detection and Correction All on-disk structures covered by strong checksums Data correctness verified on every disk read Data errors corrected automatically from data redundancy Continuous Scrub - Finds and fixes errors before they become a problem 38

39 DD Boost Software DD Boost Distributes parts of deduplication process to backup server or application clients Licensable software works across Data Domain portfolio Supports majority of backup software market EMC Avamar and NetWorker Symantec NetBackup and Backup Exec Speeds backups by up to 50 percent Process more backups with existing resources 20 40% less overall impact to backup server 80 99% less LAN bandwidth Enables Data Domain replication management from the backup application 39

40 Data Domain Portfolio 40

41 New Data Domain DD160 Appliance For small enterprise data centers and remote offices Protect up to 4 TB user data Back up 4 TB in less than four hours Supports all Data Domain software options DD160 41

42 Industry s Most Scalable Inline Deduplication Systems Data Domain Software Options DD Boost DD Encryption DD Extended Retention DD Replicator DD Retention Lock DD Virtual Tape Library Large Enterprise Small Enter./ ROBO Midsize Enterprise DD160 DD620 DD640 DD670 DD860 DD890 DD990 Speed (DD Boost) 1.1 TB/hr 2.4 TB/hr 3.4 TB/hr 5.4 TB/hr 9.8 TB/hr 14.7 TB/hr 31.0 TB/hr Speed (other) 667 GB/hr 1.1 TB/hr 2.3 TB/hr 3.6 TB/hr 5.1 TB/hr 8.1 TB/hr 15.0 TB/hr Logical capacity TB TB PB PB PB PB PB PB PB 1 Usable capacity Up to 3.98 TB Up to 8.3 TB Up to 32.2 TB Up to 55.9 TB Up to 142 TB Up to 570 TB 1 Up to 285 TB Up to 570 TB Up to 1.3 PB 1 1 With DD Extended Retention software option 42

43 EMC Backup Assessments Backup Daily Server by What s in an assessment? Data collection Analysis Recommendations Learn about... Metrics on backup capacity and the backup jobs Details about the data change rate Reliability of your backup infrastructure And more 43

44 Technology Deployment Services Operational Assurance Services Post-implementation service to enable operational selfsufficiency Provides practical and thorough on-going operational bestpractices guidance Allows you to fully realize backup and recovery capabilities with best practice knowledge transfer, operational best practices, performance/tuning optimization guidance, process recommendations for improvement, and training recommendations QuickStart version for environments that require an easy to order fixed scope of work and price Custom version for more complex environments outside the scope of fixed scope/price Professional Services Excellence in Enabling Customer Success 44

45 With Data Domain Deduplication Storage Systems, You Can WAN Retain longer Keep backups onsite longer with less disk for fast, reliable restores, and eliminate the use of tape for operational recovery Replicate smarter Move only deduplicated data over existing networks with up to 99% bandwidth efficiency for cost-effective disaster recovery Recover reliably Continuous fault detection and self-healing ensure data recoverability to meet service level agreements 45

46 TAK Michael Hemmingsen Technology Consultant EMC Backup Recovery Systems (BRS) Division 46

47