A Virtual Filer for VMware s Virtual SAN A Maginatics and VMware Joint Partner Brief

Transcription

1 A Virtual Filer for VMware s Virtual SAN A Maginatics and VMware Joint Partner Brief With the massive growth of unstructured data in today s enterprise environments, storage IT administrators are constantly seeking new ways to solve the performance and scalability challenges they face in their day-to-day operations with monolithic and incumbent storage appliances. To resolve these challenges, they need a solution that will allow them to scale their data storage infrastructure reliably and elastically through a softwaredefined architecture. VMware Virtual SAN running on their existing VMware infrastructure coupled with the Maginatics Cloud Storage Platform (MCSP) is the ideal solution to meet these pressing storage IT demands. Layering VMware Virtual SAN on top of existing VMware infrastructure will allow IT administrators to create simple, reliable and cost effective virtual environments enabling the distribution of VMs across different nodes. By deploying MCSP with its industry-leading Maginatics Virtual Filer over the VMware infrastructure, a scalable data storage solution can be created.

2 Maginatics Cloud Storage Platform Overview The Maginatics Cloud Storage Platform (MCSP) is a new breed of storage software technology that has been natively designed for software-defined enterprise data centers and cloud providers, providing enterprises with an unparalleled solution to manage their growing unstructured data while enabling agile IT and business growth. At the core of MCSP is the Maginatics Virtual Filer (MVF). The MVF is an elastic cloud-optimized distributed file system that combines a global namespace, strong data consistency, WAN optimization, end-to-end security, application compatibility, edge and mobile connectivity and other technologies into a single integrated software fabric. Figure 1: Maginatics Cloud Storage Platform

3 There are three foundational components that comprise MCSP (see figure 2). Figure 2: Maginatics Cloud Storage Platform foundational components 1. Maginatics Virtual Filer: The point of control for the MCSP is the Maginatics Virtual Filer, which is deployed as a virtual appliance and hosted where the customer wishes: (1) in their own data center, (2) hosted by a trusted service provider or (3) in the cloud. The MVF provides, among other things, the control plane for the MCSP. It is responsible for mediating all data and metadata operations, as well as for access control (via the customer s existing Active Directory or another identity management system), data integrity, cache consistency and management of de-duplication maps. The MVF does not host data and therefore does not present the dataflow bottlenecks seen with monolithic appliances. Instead, the Maginatics Virtual Filer hosts metadata (including all encryption keys) and acts as a gatekeeper to the data, which is hosted in the cloud. Today customers are able to cluster up to ten Virtual Machines (VMs) enabling them to address up to 14.4 PB of unique data. Within a cluster, any VM can be promoted to the role of the Maginatics Virtual Filer Management Console allowing the IT administrator to create, edit, delete and monitor shares and the VMs that power them from a single pane of glass. 2. Maginatics MagFS Agent: Maginatics provides native agents for popular operating systems including Windows, Mac OS X, Linux, ios and Android and supports endpoint devices that range from mobile clients and laptops to servers and virtual machines. The

4 agent takes full advantage of the power and functionality of modern devices and does much of the heavy lifting in the system (in concert with the Maginatics Virtual Filer), including WAN optimization, fine-grained end-to-end encryption and file segmentation and re-assembly. The use of endpoint agents allows MCSP to deliver security, scalability and control throughout the entire system: in the cloud, on the endpoint devices, and in-flight. By allowing endpoint devices to communicate directly with the cloud (as well as with the Maginatics Virtual Filer) over their own secure connection, MCSP delivers a distributed architecture that removes the bottlenecks associated with technologies that are based on monolithic appliances. 3. Object Storage: Object storage is highly cost-effective, simple to deploy and scalable, supporting billions of objects per namespace. This scalability was one of the critical driving factors behind the decision to select objectbased storage as the backend for MCSP. With Maginatics, customers have the freedom to select the object storage vendor or service provider of choice.

5 VMware Virtual SAN Overview Virtual SAN is a new software-defined storage tier that extends the hypervisor to pool both compute and direct attached storage. By clustering server direct attached hard disk (HDD) and solid state drives (SSD), Virtual SAN creates a distributed scale-out shared datastore designed and optimized for virtual machines. It implements a distributed architecture that leverages SSDs for high performance read/write caching and HDDs for cost effective data persistence. Virtual SAN is based on a highly available architecture with no single point of failure. It can withstand failures at the disk, server and network level with no data loss thanks to built-in redundancy mechanisms that transparently store multiple copies of the data across disks and hosts. Figure 3: VMware Virtual SAN

6 In addition, storage is instantly provisioned and automatically configured according to the assigned policies. Each VM maintains its unique policy regardless of its physical location in the cluster. Virtual SAN dynamically self-tunes and load balances to meet the policies of each individual VM adapting to ongoing workload condition changes. Virtual SAN is an object-based storage system designed to provide virtual machine centric storage services. Leveraging Storage Policy Based Management (SPBM) it implements a policy based approach to storage management that allows specifying storage attributes such as capacity, performance, availability, etc. - in the form of simple policies associated with individual VMs or virtual disks. Virtual SAN is deeply integrated with vsphere and the entire VMware stack including VMware vsphere High Availability (vsphere HA), VMware Data Protection Advanced (vdpa), VMware Site Recovery Manager (SRM) and VMware Horizon View. Virtual SAN provides high availability and scale-out storage functionality to make storage for VMs simple and easy. Virtual SAN is fully managed through the vsphere Web Client for ease of use. Figure 4: Policy-based Management with Virtual SAN

7 Integration of MCSP and Virtual SAN Maginatics has integrated the Maginatics Virtual Filer, a highly distributed software-defined filer, with the VMware Virtual SAN software platform. The MVF coupled with the VMware Virtual SAN offers a NAS consolidation solution on top of a highly redundant VMware infrastructure and object storage infrastructure. With this architecture, IT administrators can deploy filer-as-a-service capabilities, setting up and tearing down virtual filers to remain in perfect lock step with elastic demand. In addition, MCSP s foundational end-to-end security and WAN optimization capabilities ensure distributed, centrally controlled access to data for users in branch offices, remote locations or on mobile devices. The combination of MCSP and VMware Virtual SAN brings a powerful and tightly integrated offering to meet the needs of enterprise organizations that require high availability, security, and performance for network shares. Figure 5: The power of VMware Virtual SAN and the Maginatics Cloud Storage Platform combined

8 How does the integrated solution work? The MVF coupled with the VMware Virtual SAN offers a NAS consolidation solution on top of a highly redundant VMware infrastructure and object storage infrastructure. With this architecture, IT administrators can deploy filer-as-a-service capabilities, setting up and tearing down virtual filers to remain in perfect lock-step with elastic demand. In addition, MCSP s foundational end-to-end security and WAN optimization capabilities ensure distributed, centrally-controlled access to data for users in branch offices, remote locations or on mobile devices. The MVF stores file system metadata on the VMware Virtual Machine (VM) disks. The availability of this metadata is critical for the functioning of the virtual filer. With Virtual SAN the metadata is automatically replicated across multiple ESX storage nodes. MVF does not have to worry about maintaining multiple copies of the MVF metadata. In the case of disk failures, Virtual SAN will transparently fetch the data from one of the other copies and MVF will continue to operate without service disruption. File system metadata is accessed on several critical read/write paths during operations. During these accesses, it is important that read/ write functionality of the metadata from disks be as rapid as possible. VMware Virtual SAN automatically keeps the most recently accessed data on fast SSDs, which enhances MVF response times. In particular, the file system metadata is kept on SSDs in a tiered approach that enables cold metadata to be moved to lower-cost spinning disks. Key Attributes on the combined solution include: 1. Redundancy: VMware Virtual SAN does not require a local RAID set, only a set of local disks. By setting appropriate policies, the number of failures a host can tolerate can be defined. The Maginatics Virtual Filer running

9 on this fault tolerant hypervisor platform provides a ubiquitous namespace for enterprises to consume object storage without being concerned about the underlying storage or the need to integrate with a RESTful API. 2. Security: The Maginatics Virtual Filer integrates natively with existing enterprise identity management systems such as Active Directory, enabling IT administrators to control access to data stored in the cloud exactly as they would with data stored in a traditional filer. In addition, VMware Virtual SAN s ability to be segmented on a layer 2 network adds to the overall security profile of the combined solution. Connectivity between endpoints and the MVF is secured using either VPN or a reverse proxy, while cryptographically signed, time-limited tokens ensure that only authorized clients can access data stored in the system. Upon entry into the system, every file is broken into many small, variable sized chunks, each of which is individually encrypted with its own independent AES-256 key. Data is encrypted in flight and at rest. In addition, all data and metadata transfers take place over SSL/TLS. The file system objects in a MVF deployment are individually protected by access control lists (ACLs), identical to those used in a NTFS or CIFS file system. When the MagFS Agent accesses a file, the MVF checks the ACL on the file and compares it with the Security Identifier (SID) and groups in the current user session. Access control can additionally be enforced at the share level. 3. Data Integrity: Data integrity is a critical requirement for any storage architecture, especially in today s distributed enterprise environment where untrusted and unmanaged endpoint devices outside the control of central IT must be able to safely write to the system, and do so over the public Internet.

10 MVF ensures that all data entering the system is stored accurately, reliably and consistently. In this environment, a misbehaving client may mark a failed write as successful or a misbehaving HTTP proxy may provide incorrect information about write operations. More ominously, a malicious client may provide incorrect or simply random hash values (blob-references) that bear no connection to the uploaded objects. Without appropriate protection, any of these conditions may lead to data corruption issues. MCSP guards against these and all related risks with a robust, out-of-band data verification process. Maginatics guarantees the reliability and consistency of all data written to the system and ensures that buggy, misbehaving or malicious clients cannot affect data integrity. Additionally, MCSP offers efficient methods in order to alleviate concerns around data protection: Snapshots and Backups. Snapshots provide for user-accessible access to a view of the namespace at a point-in-time. Backups, a solution controlled by the administrator and typically used in a Disaster Recovery scenario, are pushed to the highly reliable object store for resiliency. Recovery can range from a single share to the loss of an entire site with the MVF restored from the latest backups at the local site or remote site. 4. Performance: VMware Virtual SAN supports the ability to utilize active-active physical network adapters connecting to Link Aggregation Control Protocol (LACP) port channels. The Maginatics Virtual Filer cluster can utilize the enhanced network performance capabilities of VMware s Virtual SAN to provide additional benefits. To further enhance the performance of the system, the MCSP architecture avoids bottlenecks by leveraging intelligent endpoint agents that handle encryption and decryption of objects, variablesize chunking for deduplication, and object transfers without having to pass those objects through the Maginatics Virtual Filer. MCSP s optimizations also include global de-duplication, aggressive encrypted local caching, read-ahead and write-behind.

11 In addition, MCSP has multiple caching layers to greatly reduce the impact of network latency and bandwidth limitations. MCSP can place software-defined caches at various tiers in a network architecture ranging from endpoint agents to Site Caches and Content Distribution Networks (CDNs). MagFS Agents cache unique chunks of data, providing a more powerful caching mechanism rather than simply caching entire files. Mapping single chunks to many different files helps reduce the amount of cache required locally. In addition, MCSP can be deployed in conjunction with a Site Cache. The Maginatics Site Cache, also deployed as a VM, allows end users to leverage a local shared cache resource. When dealing with large enterprises that have limited bandwidth to some of their sites, the MCSP Site Cache, deployed locally onsite, can prove to be an ideal solution to enhance performance at those sites by directly lowering bandwidth utilization. The MagFS Agents intelligently select the appropriate Site Cache at all times without requiring end-user manipulation. In addition, each object store is designed differently, and MCSP understands the capabilities and unique scaling characteristics of these systems and applies system-specific optimizations to improve performance and robustness. 5. Unified Global Namespace: As VMware s Virtual SAN scales out utilizing multiple ESX hosts by leveraging the local disk attached to each host, MCSP scales with it through a Distributed File System (DFS) implementation. DFS allows multiple namespaces to be stitched together into a single namespace. Once these namespaces are stitched together, the MVF allows for petabytes of data to be stored in the filer and accessed through a single mount point, helping alleviate inefficient siloed storage environments and deployment models.

12 How will the combined solution benefit customers? The power of the combined solution delivers key benefits to customers including: Leverage VMware Virtual SAN s data protection and availability Massive scale-out in the cloud with an industry first policy-driven storage backend Built-in failure tolerance Multi-platform clients (e.g., Linux, Windows, Mac, ios, Android Interoperability with the VMware stack Simplified operations and storage provisioning to enable agile IT Improved productivity via remote access to centralized data Seamless, secure, mobile access to data Significant reduction in TCO Conclusion The combination of the VMware Virtual SAN and the Maginatics Cloud Storage Platform delivers a powerful architecture that enables organizations to improve network share performance and scalability, while reducing the cost and complexity of deployment and day-to-day operations. Together, VMware and Maginatics ensure that users always have quick and easy access to network attached storage no matter what device they are using, or the location from which they are accessing the data. More Information Maginatics Virtual Filer Free Trial VMware Virtual SAN maginatics.com/product/maginatics-virtual-filer testdrive.maginatics.com/controller/customer_registration_form.php