White Paper. NexentaConnect TM Technology Review

Transcription

1 White Paper NexentaConnect TM Technology Review By Alex Aizman and Murat Karslioglu, February 2014

2 Table of Contents Background... 3 Storage Deployment Options, vmotion and DRS... 4 Local Storage... 5 External Shared Storage rd Party NAS... 7 NexentaConnect as a Storage Accelerator... 9 Compression... 9 Read Caching... 9 Write Logging and Deduplication Block Diagram: Local Storage External SAN Desktop Provisioning Login VSI Benchmark Maintenance and Backup Backup Snapshots Disk Replacement Replicated High Availability (RHA) Getting Started Conclusion Nexenta is a registered trademark of Nexenta Systems Inc., in the United States and other countries. All other trademarks, service marks and company names mentioned in this document are properties of their respective owners Nexenta 2

3 Background NexentaConnect provides: 1) Powerful I/O acceleration for a full range of storage backend options from local direct attached storage (DAS), to storage attached network (SAN), to external shared network attached storage (NAS). 2) Graphical User Interface (GUI)- based Wizard- driven automation from raw VMware ESXi clusters to deployed and calibrated, performance optimized and benchmarked virtual desktop infrastructure (VDI) pools In addition, it can be used in any VMware 5.x environment from 5.0 to the latest 5.5. NexentaConnect uses a combination of available: VMware ESXi host RAM Write commit log Inline compression and deduplication Optionally local solid state drives (SSDs) These capabilities dramatically reduce read and write I/O latency thereby increasing the responsiveness of Microsoft Windows to provide an excellent user experience. This increases the productive density of virtual machines on VMware vsphere. NexentaConnect achieves this entirely from software without requiring any specific hardware; it works exclusively with VMware Horizon View. NexentaConnect can be deployed using either ESXi direct attached storage or shared storage. When deployed on shared storage, NexentaConnect is fully compatible with vsphere vmotion and vsphere distributed resources scheduler (DRS). The product includes a simple UI that removes the complexity of installation and configuration in a large scale; in part the product completely hides storage with all its related discovery, provisioning, and management aspects that constitute a separate, vendor dependent, skillset and a large portion of the total cost of ownership (TCO). NexentaConnect provides extensive performance data enabling a quick determination of the overall storage acceleration effectiveness and the performance of the accompanying storage subsystem. 3

4 Storage Deployment Options, vmotion and DRS NexentaConnect supports a full variety of storage options including: 1) Local DAS for each host in a given ESXi cluster 2) NexentaStor - based or 3rd party storage arrays (SAN) attached to the ESXi hosts via Internet Small Computer System Interface (iscsi), Fibre Channel (FC) and/or Fibre Channel over Ethernet (FCoE) 3) Bare- metal installed NexentaStor NAS deployed with two sub- options: o Existing network file system (NFS) shares o Unused ( raw ) disks (NexentaConnect will discover them and deploy optimally) 4) And any 3rd party NAS providing NFSv3 shares For the first two of these options, NexentaConnect provides virtual storage an automatically created and deployed virtual storage appliance (VSA) that is specifically designed and designated to provide storage to each virtual desktop of its own (the VSA s) host. NexentaConnect VSAs provides NFS datastores accessible to all hosts in a given ESXi cluster. Figure 1: NexentaConnect datastores NexentaConnect will ensure that the virtual desktops and the VSA(s) that serves their storage are co- located (VMware Blogs: Horizon Tech Blog, By John Dodge). To achieve this co- location, NexentaConnect uses VMware vmotion when provisioning new virtual desktops via standard VMware interfaces. Figure 2: VMware vmotion conceptual diagram 4

5 In the VDI world, VMware vmotion is used to perform hardware maintenance without scheduling downtime or otherwise disrupting business operations. By VMware specification, vmotion (not to be confused with Storage vmotion or SVMotion) is not available to virtual machines who s VMDKs (virtual disks) are on local storage. A migrating virtual machine must be able to access its virtual disks from both the source and destination hosts. As a side note on multiple VSA based NFS shares and vmotion VMware vmotion requires shared storage (NFS datastore or SAN) that is, storage simultaneously accessible by both source and target ESXi hosts. It may be less obvious that vmotion still works perfectly with multiple NexentaConnect VSAs with each VSA providing NFS share based on local ESXi storage. Here s how it works: Each NexentaConnect VSA provides shared NFS storage to the entire ESXi cluster note that by default NexentaConnect will create a separate (from the management) NFS (data) network based on local subnet visible only from inside the cluster VMware Horizon View supports multiple NexentaConnect VSAs NFS shares in combination to provide one aggregated pool of desktops NexentaConnect built- in re- balance function (different from View s datastore re- balance) makes sure in the background that each View virtual machine accesses an NexentaConnect VSA - NFS that is located on the same ESXi host Let s now consider NexentaConnect provided storage options in greater detail. Local Storage Without NexentaConnect, View s datastores would effectively be on a several SAS/SATA HDDs or SSDs attached to each clustered ESXi host. Performance is limited to the aggregated available disk I/O of each host, and the VDI datastore is local to each clustered ESXi and not shared across multiple hosts. In this case, VMware vmotion is not supported. With NexentaConnect, the same local drives are used to create NexentaConnect NFS shares; the View desktop pool is stored on cached, compressed (or de- duplicated) and accelerated NFS provided by the hosted NexentaConnect VSAs of each clustered ESXi. 5

6 Technically, the storage is shared across multiple hosts. Although vmotion is supported, moving a VM (separately from the VSA that stores it) to another host isn t recommended since the I/O from the View desktop to its corresponding VSA NFS volume will be forwarded across the host boundary and over an external network. NexentaConnect s built- in rebalance capability is precisely designed to address this scenario and is used internally to move VMs to a local host; the host is where this VM s own storage is located. As a side note, VMware DRS at present does not take into account storage locality, as far as virtual clustered desktops. For more on the local storage performance, detailed extended benchmarks and discussion, refer to NexentaConnect Branch Office white paper, results in this whitepaper were independently validated by Cisco UCS E- series (Branch Office) engineers. External Shared Storage Figure 3 illustrates two VDI deployment options, given external SAN storage array: with NexentaConnect on the left and without NexentaConnect on the right. Figure 3: NexentaConnect to accelerate SAN With NexentaConnect (left on Figure 3) remote SAN logical unit number(s) (LUN) are presented to the hosted VSAs in a way that is identical to the local direct- attached storage as a group of ESXi- local drives. Thus, similar to the local case above, NexentaConnect discovers and deploys the corresponding virtualized disks to create ESXi- local NFS shares, and ultimately VDI pools upon those respective NFS datastores. In comparison with the VDI deployed directly on SAN (Figure 3, on the right), NexentaConnect introduces local caching and write I/O aggregation, inline compression and de- duplication. Combination of all of the above results in dramatic improvements: 38x (38 times) physical SAN data retrieval 6

7 reduction and 2.5x (2.5 times) SAN write I/O reduction was observed in our experiments with EMC VNX (Nexenta/VMware NexentaConnect SAN acceleration joint white paper). This performance boost in turn provided for an estimated 72% (seventy two percent) increase in VM density per ESXi host. The Nexenta/VMware joint white paper referred above, details SAN acceleration performance analysis and results. Note: Virtual desktop images are hosted on the SAN. NexentaConnect VSAs are also stored on the external shared storage which means VMware vmotion and DRS will work out of the box. On DRS specifically, recall the previous point on storage locality, NexentaConnect VSAs are run by the same hypervisor that runs the virtual machines stored on those respective VSAs. In most cases, locality of storage vis- à- vis virtual desktops will be the overriding factor when weighed against all the resource- related factors that VMware DRS (automatically) takes into account (this is why NexentaConnect automatically sets VMware DRS to Manual mode, see Figure 4). Figure 4: By default, the vsphere DRS setting is enabled and set to Manual, the Automatic mode can be re- enabled at any time. 3rd Party NAS NexentaConnect supports 3rd party NFSv3 storage; it provides all of the aforementioned benefits but sacrifices some value- added management capabilities. These capabilities stem from the fact that NexentaStor extends NexentaConnect management functionality, for example, it provides real- time DTrace based statistics on all storage appliance subsystems. The following conceptual diagram illustrates the point. Figure 5: NexentaConnect management interfaces 7

8 This shows that NexentaConnect deployed with 3rd party NAS will provide its entire suite of functionality except those functional pieces that require NexentaStor appliance s interface #3 (Figure 5). The latter includes: Storage auto- discovery and deployment from raw physical disks DTrace based analytics Appliance s fault management (including disk faults) Wizard- guided disk replacement Copy- on- write (CoW) snapshotting and backup/restore via CoW snapshots 8

9 NexentaConnect as a Storage Accelerator VDI workloads generate bursts of small- block random reads and writes. Part of the NexentaConnect provided performance optimization is attributed to: Compression NexentaConnect uses Lempel- Ziv family LZJB lossless data compression to improve I/O throughput, the improvement averages 2x for linked clones, and up to 4x for full clones. Figure 6: Compression ratio Compression of the VDI persistent data varies of course depending on the original desktop replica, installed software, application- generated (application- specific) local data. With a freshly deployed (linked- clone) pool of 55 (fifty five) desktops we have registered compression ratios ranging from 1.9x to 2.5x. In other experiments, the compression ratio reached up to 4x for full- clone non- deduplicated desktops (NexentaConnect by default disable compression when deduplication is enabled the latter is, again, by default enabled for full- clone desktops). In general, NexentaConnect compression proved to be extremely efficient for all VSA- based deployment options including both local ESXi DAS (Section Local Storage above) and SAN (Section External SAN and Figure 3 above). Read Caching The underlying NexentaConnect technology for read caching is called Adaptive Replacement Cache or ARC: refer to Wikipedia: Adaptive Replacement Cache (ARC). Figure 7 illustrates in part that NexentaConnect ARC can be extremely efficient in the VDI environments (99% ARC hit). 9

10 Figure 7: VSA ARC Statistics For read caching NexentaConnect utilizes RAM based ARC and, optionally, low- latency disks (typically SSDs) for Level 2 ARC (L2ARC). Level 2 caching, if present in a given NexentaConnect- deployed storage system, gets populated by the read data evicted from memory. The eviction is driven by a combination of the most recently used (MRU) and the most frequently used (MFU) policies. Write Logging and Deduplication NexentaConnect writes (syncs) new and modified data to disk at so called transaction group commits every few (configurable) seconds. The product provides write logging functionality, also referred to as ZFS Intent Log (ZIL), to process synchronous writes in accordance with SCSI and NFS protocols/standards. Write optimized low- latency SSD drives are often deployed; NexentaConnect auto- provisions write log by default upon auto- discovery of free write- optimized SSDs. In addition, NexentaConnect deploys low- latency media not only for the ZFS conventional write log and cache but it also provides a list of NexentaConnect VSA performance optimizations that includes inline deduplication. Internally NexentaConnect utilizes auto- provisioned low- latency storage, typically solid- state drives (SSDs) if available, to optimize data deduplication for both full and linked- clone desktops. Further NexentaConnect: Auto- provisioning logic will place both the write log and deduplication metadata on the same low- latency write- optimized drive(s) Uses VMware VAAI- NAS (included with the product) to rapidly provision new desktops to reduce deployment times; the result we ve seen in the lab averages 19 seconds per new desktop, from start to finish including desktop customization 10

11 Block Diagram: Local Storage Figure 8 illustrates NexentaConnect multi- pool deployment with storage being just a group of local ESXi drives. On the diagram below, each of the two desktop pools have their own per- pool VSA inside each of the two depicted ESXi hosts. For true multi- tenancy a decision to provide (per- host, per- pool) VSA for a given clustered pool of virtual desktops was driven by three major requirements/motivations: 1) Isolation - resource, application and user security, and I/O performance 2) Management separation ability to manage and monitor each desktop pool separately from the other pools of the same ESXi cluster, including: migrate, snapshot, replicate, disable, etc 3) Performance - via per- pool dedicated VSA(s) Figure 8: NexentaConnect deployed on local storage External SAN As far as NexentaConnect is concerned, external SAN LUNs attached to their respective ESXi hosts over FC, FCoE or iscsi all perform the same exact function as local (DAS) drives (discussed in the previous section). Similar to the local storage, NexentaConnect - provided data compression is applied locally on the NexentaConnect VSA before the data is sent through the FC/FCoE/iSCSI interconnect to its ultimate SAN destination; two levels of read caching are utilized (ARC and L2ARC.) The VSA write log is used to provide immediate stable storage for synchronous writes, as NFS semantics require. Both L2ARC and ZIL will benefit from local (not SAN attached) SSD drives, when available. 11

12 To test NexentaConnect SAN I/O offload capability, we conducted a test provisioning of 80 linked- clone virtual machines on a single host. We then compared I/O levels passed down to the LUN on both the bare- metal and NexentaConnect configurations. Desktop Provisioning Figure 9 illustrates disk level read/write throughput during the View Composer (linked- clone) desktop provisioning. Highlighted values in Figure 9 represent read/write rate averages over a one- hour period; since the actual provisioning cycle only took 30 minutes, the numbers had been adjusted accordingly. The read throughput of 47MBps* and write rate of 19MBps represents adjusted bandwidth levels hitting the physical SAN during the actual provisioning activities. The test was run multiple times with VMware Content Based Read Cache (CBRC) enabled and disabled. We observed that CBRC helped to dramatically reduce read I/O by up to 50% (fifty percent) on the bare- metal configuration but offered minimal incremental value when used in tandem with NexentaConnect (for detailed results, refer to NexentaConnect SAN acceleration joint VMware/Nexenta white paper). Figure 9: disk level read/write throughput during desktop provisioning operations Figure 9 tells the rest of the story. In contrast to the read I/O rates of 47MBps on the bare- metal configuration, here NexentaConnect shows 38x offload passing 1.2MBps of actual traffic down to the physical LUNs. These results demonstrate superior read caching by the NexentaConnect virtual appliance. Compared to the CBRC- enabled bare- metal configuration, NexentaConnect still provides 19x offload from the physical SAN. NexentaConnect uses a variant of ARC technology that allows it to utilize all available RAM (from 2GB in the most restrictive environments to 256GB on the high- end) to aggressively cache data, including but 12

13 not limited to, shared desktop replica. Level 2 cache called L2ARC is provided and is typically used via dedicated low- latency media (SSDs) serving effectively as a direct extension of memory based cache. NexentaConnect read caching proves to be extremely efficient in VDI environments. Figure 10 illustrates that ARC is heavily utilized, using 12GB or 80% (eighty percent) of RAM. The diagram below also shows a very high hit ratio of 85% (eighty five percent), implying that over three- fourths of our data requests are already in memory. Additionally, 50% (fifty percent) of the data is pre- fetched or already in the cache and the Data Demand results show that there is a 96% (ninety six percent) chance that requests will hit the data already read. Figure 10: ARC statistics for (virtual storage + SAN backend) For write I/O performance, the same desktop- provisioning benchmark with NexentaConnect showed 5.7MBps write traffic reaching physical disks vs. 14.3MBps without NexentaConnect. That constitutes 2.5x write I/O offload from the physical SAN or about 60% reduction. Several factors contribute to the NexentaConnect write I/O performance optimization: Inline compression Write caching that takes place between ZFS transaction group commits (refer to the above section Write Logging and Deduplication for more information) Write I/O aggregation (also known as batching) effectively delivers bigger than the average 4KB I/O size that s seen in VDI environments. The latter results in proportionally fewer seek operations by the physical rotating media For compression, NexentaConnect uses Lempel- Ziv family LZJB lossless data compression. This replaces repeated occurrences of data with references to a single copy and improves I/O throughput. On the 13

14 same setup, deployment of 80 linked- clone desktops registered compression ratio of 1.86x, refer to the section above on Compression for more information. Login VSI Benchmark The table below summarizes I/O throughputs and IOPS obtained during Login VSI tests. Table 1: I/O throughputs obtained during Login VSI tests with and without NexentaConnect Bare- Metal LUN column in Table 1 above shows load levels on the physical disk during testing without NexentaConnect acceleration. In the tests multiple spikes up and down were observed with I/O latency reaching close to 1s. As a comparison, the NexentaConnect LUN column highlights load levels on the physical disk during the same Login VSI benchmark run with NexentaConnect. Table 1 also clearly reflects write I/O efficiency in the reduced write latency: 629ms vs. 32ms with NexentaConnect. NexentaConnect executes 28% less write operations (188 IOPS vs. 259 IOPS) while delivering 2x more write bandwidth. This is a testament to the improved I/O latency and I/O aggregation and write optimization discussed above. NexentaConnect NFS datastore column above records I/O throughout and IOPS for the NexentaConnect VSA- provided NFS datastore during these same benchmarks. The numbers in this column represent I/O statistics measured by the ESXi NFS client. Notice that NexentaConnect NFS provides aggregated 7.3MBps read I/O to the virtual desktops while only passing about 3MBps down to the underlying virtual LUN. This further reinforces ARC/L2ARC efficiency. The bare- metal (without NexentaConnect) Login VSI benchmark simultaneously recorded much higher CPU load levels on the ESXi host causing multi- second delays at times when retrieving VMware vsphere performance statistics. However, NexentaConnect demonstrated stable no- spikes execution. With NexentaConnect, the overall CPU utilization on the host was reduced which may seem counter- intuitive considering NexentaConnect VSA itself is consuming ESXi CPU and memory. As a result of the reduced I/O latency, read caching, write I/O compression and aggregation, virtual machines spend less time waiting for I/O. This leads to fewer guest OS cycles spent waiting for the storage stack to respond. The result is an improved user experience with increased desktop density on the same hardware. 14

15 Maintenance and Backup Backup The desktop pool can be backed- up to a remote NexentaStor appliance with virtual or bare- metal installations. Frequency of backups is set via NexentaConnect GUI based on a user s recovery point objectives. Snapshots Copy- on- write instantaneous snapshots can be scheduled with the capability to roll back to a given snapshot or a given time in history for an individual desktop or an entire pool of virtual desktops. This feature is available at any time. Disk Replacement Degraded or failed drive(s) can be replaced with new (unused) drive(s) of the local ESXi. NexentaConnect automates the whole process of creating a VMware Virtual Machine File System (VMFS) datastore attaching the VMware Virtual Machine Disk (VMDK) to the VSA and replacing LUN in degraded volume. Replicated High Availability (RHA) Replicated High Availability (RHA) is another schedulable NexentaConnect service that combines periodic incremental backups of a given VDI datastore and provides high availability (HA)- type failover/failback capabilities. The latter is currently only offered as a manual capability; automated monitored HA failover is planned for a future release. RHA is only meant for system maintenance and not 24/7 availability. RHA manual failover, as the name implies, will switch the storage of a given pool of virtual desktops to the replicated destination, whereas manual failback will do just the opposite. NexentaConnect executes all the appropriate VMware Horizon View APIs (for more information, refer to Figure 5 above). 15

16 Getting Started Figure 11 below gives a quick overview of the NexentaConnect Dashboard and its main GUI elements; this is the screen that users will see when they run the product the first time. In this example there are no deployed VDI pools. This single- pane- of- glass dashboard is designed to see and manage (calibrate, benchmark, reconfigure) all the NexentaConnect deployed pools and their respective clustered resources. Figure 11: NexentaConnect Dashboard 16

17 Conclusion NexentaConnect provides for I/O performance acceleration for the full range of storage deployment options. NexentaConnect VSAs works effectively as a storage upgrade or as an accelerator. VMware vmotion and Distributed Resource Scheduler (DRS), CBRC, other vsphere 5.x capabilities are available for all/any NexentaConnect - deployed VDI pools NexentaConnect will assist in making your VDI pools replicated and highly available NexentaConnect provides full automation for your VDI provisioning and subsequent management, thus reducing overall TCO These points, as well as management- automation and performance- acceleration capabilities, were independently validated by the Enterprise Strategy Group Lab Report: NexentaConnect which concludes with the following observations: The installation and deployment of an NexentaConnect was both quick and easy through the use of a fully automated, VDI deployment wizard Management and monitoring was made easy through an intuitive interface that seamlessly integrated with vcenter Using the NexentaConnect s monitoring capabilities, ESG Lab was able to watch the performance metrics of a VDI deployment as Iometer workloads were run on the virtual desktops. Metrics like CPU, memory, and IOPS were clearly shown, and adjustments were able to be made in real time as more or fewer resources were required to meet the needs of simulated business applications on a per- desktop basis Impressive performance results were measured when utilizing a highly efficient NexentaConnect. In a virtual desktop environment featuring 55 fully cloned VMs, IOPS improved by more than 48 times, while latency was reduced by more than 11 times when comparing the use of Nexenta s technology to the same deployment that utilized direct- attached storage without a NexentaConnect Using the LoginVSI tool, ESG Lab witnessed a virtual desktop density improvement of 72% (seventy two percent) when utilizing an NexentaConnect over a bare- metal third- party SAN storage array. While application simulations were running, the bare- metal configuration scored a VSImax of 76, while the NexentaConnect scored a VSImax of 131. * = Capital B in the MBps notation stands for Bytes (not bits), throughout this paper. 17