The next step in Software-Defined Storage with Virtual SAN

Transcription

1 The next step in Software-Defined Storage with Virtual SAN VMware vforum, 2014 Lee Dilworth, principal 2014 VMware Inc. All rights reserved.

2 The Software-Defined Data Center Expand virtual compute to all applications Virtualize the network for speed and efficiency Transform storage by aligning it with app demands Management tools give way to automation 2

3 The Software-Defined Data Center Transform storage by aligning it with app demands 3

4 Today s Challenge: Massive Increase in Storage Demand & Complexity 120M 100M 80M 60M Storage Growth Terabytes Sold 41% YoY Meeting SLA Troublesho oting Data Migrations Most Pressing Storage Challenges 31% 28% 42% 40M Time/budge t 28% 20M M Terabytes Sold Provisioning Manageme nt Complexity 26% 24% Source: IDC, Yezhkova, Worldwide Enterprise Storage Systems Forecast, November 2013, # Source: IDC, Storage Predictions 2014, January 2014, General Storage QuickPoll, #243511, n=307 4

5 Storage Market in Midst of Disruption Server Storage Shared Storage years ago years ago New Forms Today Key Drivers Server flash Falling storage prices Abundant CPU cycles Hypervisor-converged infrastructure Cloud economics 5

6 The Hypervisor Opens Up New Opportunities The virtualization platform: vsphere Knows the needs of all apps in real time Sits directly in the I/O path x86 Servers SAN / NAS Cloud Storage Global view of underlying infrastructure Hardware agnostic 6

7 Leveraging The Hypervisor We Can Transform Storage Today Software-defined Storage LUN LUN LUN Replicati on Hypervisor Converged Pool Snapshot s SAN/NAS Pool Object-based Pool Automate SLAs via VM-centric policies (Policy-based Control Plane) VM level Data services (Virtual Data Services) LUN Array A LUN Array B vsphere Abstract and pool (Virtualized Data Plane) x86 Servers SAN / NAS Cloud Object Storage 7

8 VMware Virtual SAN Radically Simple Hypervisor-Converged Storage

9 VMware Virtual SAN Hypervisor-Converged storage platform vsphere + Virtual SAN SSD Hard disks SSD SSD Hard disks Hard disks Virtual SAN Shared Datastore Storage scale out architecture built into the hypervisor Simple cluster level feature tick to enable! Aggregates locally attached storage from each ESXi host in a cluster Dynamic capacity and performance scalability Flash optimized storage solution Fully integrated with vsphere and interoperable: vmotion, DRS, HA, VDP, VR VM-centric data operations 9

10 Unprecedented Customer Interest And Validation 12,000+ Virtual SAN Beta Participants 95% Beta customers Recommend VSAN 90% Believe VSAN will impact Storage like vsphere did to Compute 10

11 Virtual SAN Simplifies And Automates Storage Management Per VM Storage Service Levels From a Single Self-tuning Datastore Per VM Storage Policies Policies Set Based on Application Needs Software Automates Control of Service Levels Capacity Storage Policy-Based Management Performance vsphere + Virtual SAN SLAs Availability Virtual SAN Shared Datastore No more LUNs/Volumes! 11

12 Virtual SAN Puts The App In Charge Simpler and Automated Storage Management Through Applicationcentric Approach Today Today Virtual SAN 1. Pre-define storage configurations 2. Pre-allocate static bins 3. Expose pre-allocated bins 4. Select appropriate bin 5. Consume from pre-allocated bin Overprovisioning (Better safe than sorry!) Wasted resources, wasted time Frequent data migrations Virtual SAN Shared Datastore 1. Define storage policy 2. Apply policy at VM creation Resource and data service are automatically provisioned and maintained. No overprovisioning Less resources, less time Easy to change 12

13 Virtual SAN Delivers Enterprise-Grade Scale Maximum Scalability per Virtual SAN Cluster 32 Hosts 2M IOPS 3,20 0 VMs 4.4 Petabytes Virtual SAN allows us to build out scalable heterogeneous storage infrastructure like the Facebooks and Googles of the world. Virtual SAN allows us to add scale, add resources, while being able to service high performance workloads. Dave Burns VP of Tech Ops, Cincinnati Bell 13

14 Virtual SAN Enables Elastic Linear Scaling of Performance and Capacity No More Complex Forecasting & Large Upfront Investments Scale OUT Add more nodes Scale UP Add more Disks Elastic Grow or shrink on demand Granular Add single nodes or disks 4.4 PB 400 TB 40 TB Capacity IOPS Non-disruptive No app downtime Virtual SAN enables us to scale our storage infrastructure and while providing the necessary redundancy. This allows us to be more agile and bring our solutions to market faster. Frans Van Rooyen, Cloud Architect, Adobe 14

15 Virtual SAN Use Cases Use Cases for Virtual SAN 5.5 Virtual Desktop (VDI) Tier 2 / Tier 3 / Staging DR Target Site A Site B Handle peak performance requirements (boot, login, read/write storms) Granularly scale from POC to production without huge upfront investments Support high VDI density Rapid storage provisioning and complete automation Ideal price/performance Enables Cloud Architect to easily provision storage Integrated with vsphere Replication and VMware SRM Reduces cost of storage Minimizes data center footprint 15

16 Virtual SAN: Deeply Integrated with VMware Stack Ideal for VMware Environments vsphere Data Protection Virtual Desktop vmotion vsphere HA DRS Storage vmotion Snapshots Linked Clones VDP Advanced vsphere Replication VMware View Cloud Ops and Automation vcenter Operations Mgr vcloud Automation Center Disaster Recovery Site Recovery Manager Storage Policy-Based Management IaaS Site A Site B 16

17 Virtual SAN Implementation Requirements Virtual SAN requires: Minimum of 3 hosts in a cluster configuration All 3 host MUST!!! contribute storage vsphere 5.5 U1 or later Maximum of 32 hosts vsphere 5.5 U1 Cluster local storage local storage local storage HDD HDD HDD Locally attached disks cluster Magnetic disks (HDD) Flash-based devices (SSD) esxi-01 esxi-02 esxi-03 Network connectivity 1GB Ethernet 10GB Ethernet (preferred) 17

18 Two Ways to Build a Virtual SAN Node Completely Hardware Independent 1. Virtual SAN Ready Node Preconfigured server ready to use Virtual SAN 2. Build Your Own Choose individual components Any Server on vsphere Hardware Compatibility List SSD or PCIe SAS/NL-SAS/ SATA HDDs HBA/RAID Controller with multiple options available at GA + 30 using the Virtual SAN Compatibility Guide* Note: For additional details, please refer to Virtual SAN VMware Compatibility Guide Page Components for Virtual SAN must be chosen from Virtual SAN HCL, using any other components is unsupported 18

19 Virtual SAN Disk Groups Virtual SAN uses the concept of disk groups to pool together flash devices and magnetic disks as single management constructs Disk groups are composed of at least 1 flash device and 1-7 magnetic disks Flash devices are use for performance (Read cache + Write buffer) Magnetic disks are used for storage capacity Disk groups cannot be created without a flash device Each host: 5 disk groups max. Each disk group: 1 SSD HDDs disk group disk group disk group disk group disk group 19

20 Virtual SAN Datastore Virtual SAN is an object store solution that is presented to vsphere as a file system The object store mounts the VMFS volumes from all hosts in a cluster and presents them as a single shared datastore Only members of the cluster can access the Virtual SAN datastore Not all hosts need to contribute storage, but its recommended vsandatastore Single VSAN datastore per cluster disk group disk group disk group disk group esxi-01 esxi-02 esxi-03 esxi-04 20

21 Virtual SAN I/O flow Write Acknowledgement esxi-01 esxi-02 esxi-03 esxi-04 raid-1 vsan network vmdk vmdk witness Destaging to HDD is done independently between hosts. VSAN mirrors write IOs to all active mirrors, these are acknowledged when they hit the flash buffer!

22 VSAN IO flow Reads esxi-01 esxi-02 esxi-03 Read block 1 and 2. Block 1 is owned by esxi-01 and block 2 by esxi-03. vsan network Read Cache 1MB (1) Write Buffer Read Cache Write Buffer Magnetic Disks vmdk witness Magnetic Disks vmdk 1MB (2) VSAN track of where IO resides and reads from where located

23 Virtual SAN Network New Virtual SAN traffic VMkernel interface. Dedicated for Virtual SAN intra-cluster communication and data replication. Supports both Standard and Distributes vswitches Leverage NIOC for QoS in shared scenarios NIC teaming used for availability and not for bandwidth aggregation. Layer 2 Multicast must be enabled on physical switches. Much easier to manage and implement than Layer 3 Multicast uplink1 uplink2 vmk0 vmk1 vmk2 Management Virtual Machines vmotion Virtual SAN 20 shares 30 shares 50 shares 100 shares Distributed Switch 19

24 Virtual SAN Is Highly Resilient Against Any Hardware Failure Virtual SAN is Designed to Ensure Data is Never Lost in Case of Failures esxi-01 esxi-02 esxi-03 esxi-04 Simple to set up via policy raid-1 vsan network vsphere + Virtual SAN Delivered on per VM basis Zero data loss in case of disk, network or host failures Interoperable with vsphere HA and Maintenance Mode vmdk vmdk ~50% of I/O ~50% of I/O witness Virtual SAN Policy: Number of failures to tolerate = 1 24

25 Virtual SAN Key Benefits Radically Simple High Performance Lower TCO Installs in two clicks Managed from vsphere Client Policy-based management Self-tuning and elastic Deep integration with VMware stack Embedded in vsphere kernel Flash-accelerated Matches the VDI density of all flash array Best price/performance 100 kazillion IOps Eliminates large upfront investments (CAPEX) Grow-as-you-go (OPEX) Flexible choice of industry standard hardware Does not require specialized skills 25

26 Thank You