SPECIAL ISSUE for DataCore Software provided by nt4admins.de Editorial This month's topic is storage virtualization with a focus on Virtual SAN This year VMware launched a vsphere-based virtual SAN for converged server and storage systems, Nutanix has released version 4.0 of its pre-packaged hardware and software appliance offering and DataCore released SANsymphony-V10 storage virtualization software and with it, a new virtual SAN capability designed to work with any hypervisor or storage. We therefore decided to compare these different approaches and concepts. Please note, we utilized relevant information from a related article that appeared within our last E Paper, portions of that material, including the concept behind VMware s VSAN has been reused here to provide background and add context. Rainer Huttenloher 1
Virtual SAN DataCore versus VMware DataCore s Virtual SAN or VMware s Virtual SAN this type of comparison and question easily comes to mind whenever the use of storage virtualization is being evaluated. Other important issues to consider is the ease of deployment and ability to scale these systems in terms of performance and capacity, a key point highlighted by converged system suppliers such as Nutanix for example. With this backdrop and the introduction of DataCore s SANsymphony-V10, and the recent release of VMware s Virtual SAN, we decided there was a need to offer a comparative analysis overviewing the virtual SANs and their essential differences. A major upheaval is taking place in the storage arena the growing relevance of software and the agility it brings are clearly gaining momentum. The current buzzword of note in the industry is software-defined storage (SDS) and it is causing a fundamental change on how we implement storage infrastructures. To highlight the point, storage specialist EMC last year introduced its VIPR technology, a software-based approach that it claimed was designed to address and integrate lower cost "commodity hardware". develop their solutions for the storage market and therefore they now have to acquire or write their own software to manage and integrate these new high-speed solid state storage devices into their solution offering. They need more than hardware, they need to deliver a complete storage software stack to effectively compete. This is due to the fact that even with flash-based storage systems, the user wants to be abstracted from the details and demands functionality such as snapshot and the ability to tier storage However, before less expensive shared storage approaches are made fit for use in mission-critical applications particularly in relation to high-availability there are a number of developments that need to be taken into account. VMware is attempting to take advantage of this trend and therefore introduced a virtual SAN (VSAN) that has been designed to rely on key vsphere virtualization platform services to provide core functionality especially in terms of ensuring higher availability. This concept was announced last year and at that time a beta phase was launched that lasted over a year. Earlier this year, version 1.0 of VMware s VSAN was finally released. Interestingly, with VSAN, VMware has not set the bar particularly high: VMware claims VSAN can do 2 million IOPS and a maximum 4.4 petabytes of capacity in a 32 node VSAN cluster. They also clearly positioned the solution for smaller business and less demanding application use cases. Picture 1. Scalability is one benefit of SANsymphony-V version 10; Source: DataCore This endorsement of a software based approach running on commodity hardware has led many users to rethink their storage approach. Following the maxim: If the leading storage vendor EMC says that storage is a software subject, then it must be ok to start using software solutions that operate on low-cost standardized hardware. This along with all the momentum around SDS over the last year has caused a major disruption in the mindset on how to approach storage going forward. With the general acceptance of flash-based storage and within application host servers, a further issue comes into play. All the major flash memory manufacturers are trying to between flash and existing disk based storage investments. Data mirroring to protect data must also be supported. Likewise, it is expected that there will be solutions for metro-clusters and remote distance replication all these functions are essential and now expected. In addition, integration with new and existing virtualization platforms presents providers of flash memory devices with a number of challenges. Moreover, the incorporation of flash storage alongside the use of local hard disks within virtualized host systems running applications on commodity servers is proving to be a very cost-effective option for providing high-performance storage services. VMware VSAN Overview With VMware, VSAN wanted to open the door for small and medium-sized (SME) enterprises. VMware plans to enable these SME customers to virtualize their critical applications and to have access to highlyavailable low-cost local storage. The VSAN system requires flash and magnetic disks to be installed directly within the servers. The magnetic disks provide the storage capacity while the required flash devices (i.e. solid state disks) are responsible for caching, hence yielding higher performance. In principle, the VSAN software combines the storage running on various ESXi hosts i.e. the virtualization hosts into a distributed pool of shared storage. The flash drives are there only to assume the role and function of read 2
Picture 2. Autotiering provides up to 16 tiers for data. Source: DataCore and write cache memory accelerators and do not contribute to the overall storage capacity of the cluster. When a new virtual machine (VM) is deployed, the virtual disks (also known as VDMKs) are assembled from the common pool. The virtual disks are thus provided with different performance and availability service levels, depending on the relevant profiles assigned to the VMs. If a vsphere cluster uses hosts which are deployed only as compute nodes (i.e. they do not contribute storage capacity to the cluster), then these systems can utilize storage from the distributed pool (this is also known as a datastore). VMware s VSAN depends heavily on its use and integration with other vsphere functions and services. The VSAN software mirrors the VMDKs onto one or more physical servers and in this way it avoids single points of failure scenarios. The VMs in a cluster can be moved from one physical host to another to achieve this, VSAN interacts with the vmotion and distributed resource scheduling (DRS) functions within vcenter. The system does not require its own external storage arrays or dedicated file servers. Cache contents may also be migrated to ensure minimal performance impact if the VMs continue to work at the new location. With SANsymphony-V, DataCore s Virtual SAN one of the earliest established software vendors in the field of storage virtualization has launched an approach which is suitable not only for virtualized environments like vsphere, but also for other hypervisors such as Microsoft Hyper-V, Citrix XenServer, or any other hypervisor capable of running a Windows virtual machine. As with VMware's VSAN, DataCore s SANsymphony-V10 (the 10th version) fully utilizes and transforms local storage resources from the application servers (SAS, SATA, SSD, etc.) into shared storage pools. However, unlike VMware VSAN, DataCore virtual SANs can also manage and integrate the resources from both dedicated traditional SANs and directly attached storage (DAS) systems, or even cloud-based storage solutions. In SANsymphony-V10, a number of I/O performance tuning-related features have also been added. The number of nodes in a SANsymphony-V10 server group has been doubled to 32 nodes. Additional network technologies such as 40/56 GigE iscsi and 16 Gbit/s fibre channel are now also supported. The iscsi NIC teaming feature is supported. The result of these enhancements is drastically improved storage performance. SANsymphony-V10 can be used to build virtual SAN environments which achieve throughputs of 50 million IOPS and support up to 32 petabytes of capacity. With the help of special tools such as heat maps for performance visualization, the system administrator can quickly see the realtime placement of hot and cold data on the corresponding storage pool disks. Additionally, automatic migration and distribution of data on the appropriate storage devices (with frequently used data going to the faster drives) known as automated tiering has been further improved, to allow better exploitation of more expensive resources such as flash storage. The redistribution of data, for example, if individual hard drives need to be removed from the storage pools, is automatic. No downtime is necessary. DataCore has also improved its caching algorithms to obtain even higher performance out of the latest server technology. This is designed to allow better use of flash; especially in terms of write access which is key in many database and transactional workloads (i.e. ERP, SAP, Oracle, Microsoft SQL, Exchange and SharePoint). DataCore s virtual SAN provides a comprehensive feature set. However, when there are many features available, installation can become generally quite complex. SANsymphony-V10 has made major improvements in this area. A Smart Deployment Wizard offers templates for various deployment options, such as virtual SANs, highly available SANs, or NAS file servers. The desired configuration can then be started with a single click of the template. The templates give system administrators a clearly defined starting point and guide them through deployment scenarios, from which they then can add further functions when they are needed. DataCore s Virtual SAN Overview Picture 3. DataCore easily integrates Flash based memory and supports caching algorithms to optimize and accelerate IO performance. Source: DataCore 3
VMware VSAN and DataCore Virtual SAN Differences When comparing DataCore s SANsymphony-V10 Virtual SAN with VMware s VSAN, one aspect is particularly striking: VSAN is suitable only for ESXi-based hosts which have a directly connected storage device hard drives and SSDs/flash memory devices. By contrast, SANsymphony-V is suitable both for physical environments and virtualized infrastructures. It also supports other Hypervisor environments, such as Hyper-V, Xenserver, etc. This means that storage functions from "normal" systems and existing storage investments are also available and usable i.e. non-virtualized environments that run on Windows, Linux, Unix or MacOS operating systems. A further advantage of DataCore s Virtual SAN as compared to VMware s VSAN is the ability to take advantage of locally connected storage, and also external storage systems and traditional SANs that are connected to the server via the standard interfaces (ie Ethernet, Fibre channel). This ability to utilize universal storage services and features that span the entire infrastructure makes managing the overall storage environment really simple for the users. In SANsymphony-V10 all actions provisioning, allocating, protecting and managing storage can be carried out from a single console. This means that the administrator does not have to use different vendor tools for various Flash, storage arrays and disk devices which minimizes the learning curve especially in very heterogeneous multivendor storage environments. Also worth noting, the VMware vcenter, VAAI and Microsoft System Center plug-ins are made available to simplify storage management and hypervisor integration. Performance. The issue of data throughput is a major differentiator. In VSAN the specified read and write speeds are achieved by the use of SSD storage as a buffer. This is different for SANsymphony-V10, where the buffers are held in working memory i.e. in the DRAM. Moreover, access to the DRAM is many times faster than access to flash-based memory. Therefore, with SANsymphony-V, the use of flash-based storage systems are not absolutely necessary to be used in order to achieve a high data throughput (achieving over 1 million IOPS per node is common and practical). The realized performance of DataCore s Virtual SAN is significantly higher than the throughput of VMware s VSAN. As a result, the solution is less expensive than VSAN while also delivering greater performance. In VSAN, replicated write accesses to flash or SSD memory must be "committed" on different hosts before confirmation of the transaction is delivered back to the applications. This is done differently with SANsymphony-V10: here a faster confirmation is obtained, since it is based on "memory to memory mirroring" technology. In addition, IT professionals should certainly allow for the possibility of a cache hit. If such a hit occurs during read access, SANsymphony- V10 fetches the data blocks directly from the DRAM. In this way, the system does not have to wait for the flash memory or even the hard drives. Caching of the data in the working memory (DRAM) has another positive effect: there will be fewer read-write cycles that need to be performed directly on the SSD media. This both improves the service life of the expensive flash memory devices and there are fewer of the delays which typically occur in flash memory when older blocks need to be overwritten. As far as high availability (HA) is concerned, SANsymphony-V10 works according to the "N+1" principle. This means that the software can be applied even on a single node but then there is no failure resilience. High availability is only available with an additional node. Picture 4. SANsymphony-V combines virtual and physical SAN environments. Source: DataCore 4
In contrast to DataCore, all the "converged storage" approaches, including VMware s VSAN, need at least two nodes plus one as a "witness" node (also sometimes known as a quorum node). This means at least one additional node is required, in order to ensure high availability which adds cost and complexity. This is obviously not as efficient as the "N+1" approach where the information can be distributed over all nodes (no special purpose witness nodes are needed) in SANsymphony-V10 for example this only requires 2 nodes and can scale to a maximum of 32 nodes currently. Another advantage of DataCore s HA approach in direct comparison to the converged approach, is that DataCore offers a "metro-wide mirroring" facility. The software can run on a VM or on a server and the 2 instances can be located in 2 different racks, 2 different rooms or across town at 2 different sites. This brings further benefits, since a complete location can be mirrored and over greater distances. If multiple nodes are used in SANsymphony- V10, the write access procedure is as follows: the system writes to the cache memory (DRAM) on one node then writes to cache located on a second node. In high-availability configurations, these two nodes are usually located at two different locations, then a write-back to the memory takes place, a confirmation is made and everything is completely confirmed, this insures that the information is located in 2 locations and avoids the single point of failure scenario. This is the most efficient design and represents the design used in all the high-performance enterpriseclass controllers and arrays in the storage industry. This basic approach also called a 2 stage commit is what EMC, IBM, Hitachi and others have implemented and it well proven in the real world. Another design concept in modern storage architectures remains to be considered. In VMware's Virtual SAN, storage virtualization Comparison with Virtual SAN With the VSAN concept, VMware is targeting a very specific clientele. These are the companies which do not want to use expensive, stand-alone storage arrays, but would prefer to combine multiple servers with directly attached storage devices to form a convergent system. Many small and medium-sized enterprises don't want to incur the costs associated with purchasing and operating dedicated storage arrays or complex traditional SANs. Each storage system uses its own proprietary tools to operate storage and to manage functionality within the array. If one owns different models or brands then each has specialized training and support requirements as well, the management of these systems quite quickly becomes very complicated. Generally speaking, it therefore seems that VMware with its current offering has inspired but avoided engaging with larger-scale competition and with major storage manufacturers such as EMC, Netapp, HP, IBM or Fujitsu. Note: There is currently a maximum of 32 ESXibased hosts allowed in a cluster delivering relatively modest performance of 2.million IOPs In general at least three hosts (each with at least one hard drive and a SSD) are required in a cluster. Each host that wants to add its rotating disks to the VSAN pool must also have the added expense of a flash memory / SSD device. The Flash or SSD memory devices are only used as read-write cache memory. Only devices connected locally via the SATA, SAS or PCIe interfaces are supported. Only ESXi-based hosts within the VSAN cluster can use the storage thus created. The maximum size of a VMDK (i.e. a virtual hard disk in a VSAN datastore) is limited to being not more than 2 TByte. With ESXi, a maximum size of 62 TByte for a VMDK is currently supported, but only on a "non- VSAN Storage device". Only one VMware datastore with VSAN storage can be created. One VSAN datastore is created per VSAN cluster; this can then be "sub-divided" into up to five disk groups in order to optimize the failure resilience; each disk group consists of one SSD and at least one or up to 7 hard disks. The exact "per VM availability" is defined by the policy management (VM-centric). Within a vcenter, however, multiple clusters and hence multiple VSAN clusters can be created. Migration of VMs between these VSAN datastores is possible. If an ESXi host needs to be transferred to Maintenance Mode, this will take quite a long time to set up with VSAN. VMware recommends that you perform a "full migration" of the VSAN data onto the remaining ESXi hosts within a VSAN cluster before switching to "Maintenance Mode". There are also other, faster options, but with these, administrators risk some data loss. There are three scenarios for the maintenance case. Firstly: The host is only briefly switched off or restarted in this case the "Ensure accessibility" variant should be used. This will ensure that the VMs, which access data in the VSAN datastore or run on the host, will remain available during the reboot. A partial data migration is performed automatically and this is the fastest method of ensuring that this happens. However, it is recommended that this should really only be considered as a temporary action, because while the availability of the VMs is indeed ensured, there is no longer any full protection against hardware failure. If a host is to be removed from the cluster, then the "Full Data Migration" variant should be used. This may take some time, depending on how many VMs are operated on the host with a VSAN datastore. The third variant is the "No Data Migration": This means that a reboot or shutdown of a vsphere host is carried out without further demand and without VM migration, and thus cannot result in unavailability or data loss while machines are running. This method is not recommended by VMware though! The licensing for VMware's VSAN is based on the scheme already being used by vsphere: Sockets i.e. without limitation of cores. An example of this is: 1 server host with 2 CPUs with 8 cores each therefore needs 2 vsphere (regardless of the edition) or VSOM or vcloud Suite licenses, as well as 2 VSAN licenses. As mentioned previously, for high-availability configurations a minimum of 3 nodes would be needed due to requirement for a witness node. DataCore SANsymphony-V offers a simpler per server model and only requires two nodes for high-availability. In addition, the licensing is based on systems (i.e. per server node) and not on the number of processors in the system. Rainer Huttenloher 5
Picture 5. DataCore supports high availability via metro-wide mirroring and remote replication for even longer distance disaster recovery. Source: DataCore forms a component of the Hypervisor. The argument for this approach runs as follows: the more closely this function is linked to the Hypervisor, the higher the system throughput. But in the first version at least, VSAN does not exploit this very well. VMware quotes performance figures of only 2 million IOPS in a network with 32 nodes. Another disadvantage of this methodology is seen in the portability aspect. If the functionality is located in the Hypervisor, then it will only work for this Hypervisor and possibly even only for a specific version. Nutanix Nutanix has chosen another approach for its storage. The Nutanix virtual computing platform is a converged infrastructure solution that consolidates the compute (server) tier and the storage tier into a single, integrated 2U hardware-based appliance. While it positions itself as software-defined storage, in reality the entire appliance has to be purchased and deployed as a whole and while modular, there is a considerable difference in flexibility as compared to pure software based approaches like VMware or DataCore that allow hardware interchangeability. With Nutanix, the storage functionality is provided in a converged virtual system which is pre-packaged and, which works with the applications and the associated guest operating systems on the same level as the "normal" VMs, so to speak. The company can therefore also adapt such a VM more easily for other hypervisors since Nutanix supports not only vsphere environments, but also, for example, configurations based on Hyper-V. However, the greatest portability though is offered by an approach like the one followed by DataCore where there is no dependence on the hardware nor the hypervisor: the storage software runs on any standard hardware (x86-based servers) and supports any storage and there is no limitation to use a specific Hypervisor environments. For more information, please visit the following sites: VMware VSAN: http://www.vmware.com/products/ virtual-san Nutanix Converged Platform: http://www.nutanix.com/ DataCore Virtual SAN: http://www.data core.com/products/ features/virtual-san NT4Admins the German source for technical content www.nt4admins.de addresses professionals (system operators, administrators, help desk staff) and provides independent, technical oriented information about the use of Windows-based platforms. Subscribers even receive scripts developed and provided by the nt4admin team in order to facilitate administrator s life. Rainer Huttenloher has been active in IT journalism for over 30 years. The graduate Electrical Engineer worked for major German IT publications like Markt & Technik, Design & Elektronik, Unix Magazin, Elektronik, Systeme, etc. and served as editor in chief for NT Administrator, LANline and the IT weekly Computerzeitung. Since 2009, Rainer Huttenloher works independent and cooperated with the online publications speicherguide.de and virtualisierungsguide.de before he developed and realized www.nt4admins.de in 2009 where he takes the role of a publisher and editor in chief. 6