White Paper EMC VNXe3200 UFS64 FILE SYSTEM A DETAILED REVIEW Abstract This white paper explains the UFS64 File System architecture, functionality, and features available in the EMC VNXe3200 storage system. February, 2015
Copyright 2015 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other trademarks used herein are the property of their respective owners. Part Number H13770.2 2
Table of Contents Executive Summary... 4 Introduction... 4 Audience... 4 Terminology... 4 Overview... 6 What is the UFS64 File System?... 6 Limitations... 7 Value... 7 Scalability... 8 Availability and Recoverability... 8 Storage Efficiency... 8 Virtualization... 9 Management... 10 UFS64 File System Functionality... 11 User-Managed Operations... 11 Extending UFS64 Datastores... 11 Shrinking UFS64 Datastores... 12 Converting UFS64 Datastores from Thick-to-Thin... 15 Converting UFS64 Datastores from Thin-to-Thick... 16 System-Managed Operations... 17 Auto-Extend for Thin Provisioned UFS64 Datastores... 17 Auto-Shrink for Thin Provisioned UFS64 Datastores... 18 Conclusion... 18 References... 18 3
Executive Summary EMC VNXe3200 storage systems introduce the UFS64 File System, a new 64-bit file system architecture that provides a variety of new features and functionalities. UFS64 File Systems can scale in size while providing capacity efficiencies, and provide availability and recoverability in the event of an error. In addition, the VNXe3200 storage system is designed for simplicity and ease-of-use. This allows administrators to easily implement the 64-bit file system into their storage environments while providing the ability to effortlessly manage their file-level data through the Unisphere GUI or CLI. Introduction This white paper discusses the UFS64 File System features and functionality available in EMC s VNXe3200 storage systems running VNXe Operating Environment 3.1.1 and later. It describes the different ways a user can manage their UFS64 File Systems from their storage system. More information can be found in the Unisphere Online Help and the Unisphere CLI (UEMCLI) User Guide. Audience This white paper is intended for EMC customers, partners, and employees who are considering the use of UFS64 file systems with their VNXe3200 storage systems. It is assumed that the reader is at least an IT generalist who has experience as a system or network administrator. Terminology Allocated Space The size of the storage resource (such as a file system, LUN, or VMware datastore) that is provisioned from the storage pool. For thick provisioned storage resources, the allocated space is equal to the requested capacity. For thin provisioned storage resources, the allocated space relates to the capacity that is currently provisioned from the storage pool which could be less than the requested capacity of the storage resource. NAS Server A VNXe storage server that uses the CIFS or NFS protocol to catalog, organize, and transfer files within designated file system shares. A NAS Server must be available before you can create file-level storage resources such as CIFS or NFS file systems, or VMware NFS datastores. Network File System (NFS) An access protocol that enables users to access files and folders from Linux/UNIX hosts located on a network. 4
Oversubscription A storage provisioning method that allows administrators to provision more capacity than may be physically available in a particular storage pool. When thin provisioned storage resources are associated with a common storage pool, they can potentially request (or subscribe to) more storage capacity than the storage pool contains. Administrators can then add more drives to the system or assign more drives to the storage pool as needed. Hosts connected to thin provisioned storage resources are unaware of the pool oversubscription. They see the subscribed (or maximum) size for each thin provisioned storage resource, not the current allocated size. Storage Pool A collection of disk drives configured with a particular storage profile. The storage profile defines the type of disks used to provide storage and the type of RAID configured on the disks. The storage pool s configuration defines the number of disks and quantity of storage associated with the pool. Storage Processor A hardware component that performs VNXe storage operations such as creating, managing, and monitoring storage resources. Thick Provisioned Storage Resource A storage resource (such as a file system, LUN, or VMware datastore) that is fully allocated from the storage pool. Thin Provisioned Storage Resource A storage resource (such as a file system, LUN, or VMware datastore) that is not fully allocated from the storage pool. The client can see the full size of the storage resource even though only a portion of the storage resource is allocated from the storage pool. Unisphere CLI (UEMCLI) The command-line interface for managing VNXe storage systems. Unisphere for VNXe The web-based user interface for managing VNXe storage systems. VMware vsphere Storage APIs Array Integration (VAAI) A set of APIs to enable communication between VMware vsphere ESXi hosts and storage devices. The APIs define a set of storage primitives that enable the ESXi host to offload certain storage operations to the array. This reduces resource overhead on the ESXi hosts and can significantly improve performance for storage-intensive operations such as storage cloning, zeroing, and so on. The goal of VAAI is to help storage vendors provide hardware assistance to speed up VMware I/O operations that are more efficiently accomplished in the storage hardware. 5
Overview EMC File & Unified storage arrays have been available for many years and provide a variety of features and functionalities for your storage environment. These storage arrays include Celerra systems, VNX1 systems 1, VNXe systems 2, VNX2 systems 3, and VNXe3200 systems. All of these storage systems currently provide file-level functionality through the use of 32-bit file systems and will continue to do so in the future. Figure 1 File System Architecture in EMC s File & Unified storage arrays In order to provide more file-level features and functionality in your storage environment, the ability to create 64-bit file systems is now available using EMC s VNXe3200 storage system (with VNXe Operating Environment 3.1.1 and later). What is the UFS64 File System? The UFS64 file system is based on a general purpose 64-bit architecture that provides a number of benefits including increased scalability, better resiliency and availability, improved storage efficiency, and virtualization support. 1 VNX1 systems include: VNX5100, VNX5300, VNX5500, VNX5700, VNX7500 2 VNXe systems include: VNXe3100, VNXe3150, VNXe3300 3 VNX2 systems include: VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, VNX8000 6
Limitations The use of UFS64 File Systems in the VNXe3200 system using VNXe Operating Environment 3.1.1 (and later) includes the following limitations: Restricted to VMware NFS datastores only, where the maximum datastore size is 64TB NFSv3 only Unified Snapshots as well as file-level compression and deduplication are not supported NOTE: Any attempted use of UFS64 File Systems outside of these limitations is not supported. Figure 2 VMware Datastores page in Unisphere Value The implementation of the UFS64 File System provides value across several different areas that will benefit your entire storage environment, including: Scalability Availability and Recoverability Storage Efficiency Virtualization NOTE: While the UFS64 File System provides great value across these different areas, keep in mind the limitations in place with the VNXe Operating Environment 3.1.1 release. Please refer to the Limitations section for more details. 7
Scalability When provisioning UFS64 File Systems, you will notice several changes have been made in terms of scalability. You still will be able to create thick provisioned or thin provisioned 4 UFS64 File Systems, however the maximum configurable size of a file system is 64TB. Table 1 shows a few of the scalability attributes of UFS64 File Systems. Table 1 UFS64 File System Attributes File System Attribute Maximum file system size Subdirectories per directory Files per file system Filenames per directory Availability and Recoverability UFS64 64TB 10 million 4 billion 10 million The UFS64 File System allows for recovery of any faulted object, including measures to avoid panics of the NAS Server. To achieve this, the affected UFS64 File System will be taken offline when it detects any inconsistencies in the file system metadata. Once offline, the file system recovery process can be used to repair the problem and bring the file system back online. Storage Efficiency The UFS64 File System architecture includes efficiency features such as Storage Reclamation Technology, which provides the ability to evacuate portions of the file system block address space. This allows the system to repurpose or release the underlying storage of the file system back to the storage pool using the shrink operation. More details regarding the storage reclamation technology functionality can be found in the Shrinking UFS64 Datastores and Auto-Shrink for Thin Provisioned UFS64 Datastores sections. Figure 3 Shrink Operation of UFS64 File System 4 While Thin Provisioning provides many benefits, your Storage Pools may become overprovisioned. Overprovisioned Storage Pools can potentially cause out of space errors as storage resources consume space from the Storage Pools. 8
Virtualization The UFS64 File System allows the ability to leverage VMware vsphere Storage APIs Array Integration (VAAI) to create more levels of VMDK snapshots, also known as fast file clones. Using VAAI with UFS64 Datastores, you can create up to 256 levels of snapshots. In order to leverage VAAI with NFS Datastores, you will need to install the VMware VAAI Plug-in for EMC VNX for File, which is available on EMC Online Support. The hierarchical snapshot functionality of UFS64 File Systems can also be leveraged in Virtual Desktop Infrastructure (VDI) environments using VMware s vcloud Director and View Composer software 5. Figure 4 shows different levels of snapshots used to create VMs in an example VDI environment. In the Base VMs section, we have a VM which is used as a source or golden image that the other VMs in this level are created from. We can take a snapshot of any of these Base VMs (1) and use it as a source or golden image, providing a First-Level Snapshot based off that particular Base VM. We can then use this snapshot as the source for other VMs in this level. This process can be repeated to create additional levels of snapshots as needed. Figure 4 UFS64 Hierarchical Snapshot Functionality for VDI Deployments This hierarchical snapshot functionality can be leveraged in many different scenarios using UFS64 File Systems. For example, an administrator could leverage this in a test or development environment, where they need to test incremental software changes made to operating systems or applications prior to deployment into their production environment. For example, the Base VMs could be their default VM with a base operating system installed. In the First-Level Snapshots, the VMs could have some updates or patches for the operating system that need to be tested. The Second-Level Snapshots could include applications installed on top of the updated operating 5 For more information regarding VMware s vcloud Director and View Composer software, please refer to VMware s Technical documentation site: https://www.vmware.com/support/pubs/ 9
systems. Third-Level Snapshots might include updates or patches for the applications that are installed, and so on. Management UFS64 File Systems can be managed on your VNXe3200 system using the Unisphere GUI or Unisphere CLI. For more information regarding the Unisphere GUI or Unisphere CLI, please refer to the EMC Unisphere for the VNXe3200 white paper and Unisphere CLI User Guide available on EMC Online Support. Figure 5 VMware Datastores Page in Unisphere 10
UFS64 File System Functionality The following sections provide additional details regarding the available operations that can be performed on UFS64 File Systems that are built on your VNXe3200 system, including: User-Managed Operations o Extending UFS64 Datastores o Shrinking UFS64 Datastores o Converting UFS64 Datastores from Thick-to-Thin o Converting UFS64 Datastores from Thin-to-Thick System-Managed Operations o Auto-Extend for Thin Provisioned UFS64 Datastores o Auto-Shrink for Thin Provisioned UFS64 Datastores UFS64 File Systems are only available on VNXe3200 systems running VNXe Operating Environment 3.1.1 and later, and are only supported for VMware NFS datastores. For additional information regarding VMware NFS storage and your VNXe3200 system, please refer to the Using a VNXe System with VMware NFS or VMware VMFS document available on EMC Online Support. User-Managed Operations In the following sections, you will find details regarding the operations that you can initiate on the UFS64 File Systems. This includes extending and shrinking the sizes of the file systems, and converting the file systems to thick or thin depending on your needs. These sections prominently showcase the operations and their functionality using the Unisphere GUI, however the same functionality can be achieved through Unisphere CLI. For more information regarding Unisphere CLI, please refer to the Unisphere CLI User Guide available on EMC Online Support. Extending UFS64 Datastores In the event that you need additional capacity in your UFS64 File Systems, you can simply extend the size of the storage resource. It is important to note that extending the size of the storage resource equates to increasing the maximum size of the UFS64 datastore; this results in different behavior for thick provisioned UFS64 datastores and thin provisioned UFS64 datastores. For thick provisioned UFS64 datastores, extending the maximum size increases the amount of guaranteed space available for use by the storage resource. This will consume the additional space from the storage pool immediately. For thin provisioned UFS64 datastores, extending the maximum size increases the amount of space that can be available for use by the storage resource. This will not consume additional space from the storage pool at this time, but instead will rely on Thin Provisioning to allocate additional storage from the storage pool as it is needed by the storage resource. 11
Shrinking UFS64 Datastores In the event that you need to reduce capacity in your UFS64 File Systems, you can simply shrink the size of the storage resource, which will initiate a background operation to decrease the maximum size of the UFS64 datastore. NOTE: You cannot shrink below the Used space value of the storage resource. While the UFS64 datastore is undergoing the shrink operation, the VNXe3200 system may need to relocate portions of the file system block address space to lower addresses in order to free space for repurposing by the file system or to return to the storage pool for use elsewhere in your system. This process is only engaged when shrinking a storage resource below its Allocated value. Figure 6 depicts a thin provisioned UFS64 datastore that will shrink from 1TB to 300GB. The View in Unisphere is what you would see, however, the actual layout of data inside the file system is not always contiguous, shown as the Layout in File System in the image. In order to complete this shrink operation, a red-highlighted block of data in the Layout in File System view must be relocated to a lower address space so its current location can be freed and returned back to the storage pool. Once the relocation process completes, the shrink operation can proceed as planned to reduce the maximum size of the selected storage resource. Figure 6 Example of UFS64 File System Shrink Operation into Allocated Space 12
For thick provisioned UFS64 datastores, shrinking the maximum size decreases the amount of guaranteed space available for use by the storage resource. This will reduce the storage resource s consumed space and return it to the storage pool immediately, shown in Figure 7. Figure 7 Shrinking a Thick Provisioned UFS64 Datastore For thin provisioned UFS64 datastores, shrinking the maximum size has two possible scenarios: (1) Shrinking the maximum size above the Allocated value, or (2) Shrinking the maximum size below the Allocated value. In scenario 1, shrinking the maximum size of a thin provisioned UFS64 datastore above the Allocated value will decrease the amount of space that is available for use by the UFS64 datastore. This will not reduce the amount of consumed space on the storage pool. Figure 8 illustrates this point. We are currently attempting to shrink a thin provisioned UFS64 datastore from 2TB to 1TB, which is above the Allocated value. The confirmation window that appears notifies us that we will return 0GB of space back to the storage pool. Figure 8 Shrinking a Thin Provisioned UFS64 Datastore above the Allocated Value 13
After the shrink operation completes, shown in Figure 9, you can see that our maximum size has been reduced to 1TB, our Free space value has been reduced, but our Storage Pool free space value and Pool Size Used value remain the same. Figure 9 Shrinking a Thin Provisioned UFS64 Datastore above the Allocated Value In scenario 2, shrinking the maximum size of a thin provisioned UFS64 datastore below the Allocated value decreases the amount of space available for use by the storage resource. This will also reduce the amount of consumed space on the storage pool. Once this shrink operation completes, your UFS64 datastore will resemble a fullyallocated thin provisioned storage resource. Figure 10 is used to illustrate this point. We are currently attempting to shrink a thin provisioned UFS64 datastore from 1TB to 60GB, which is below the Allocated value. The confirmation window that appears notifies us that we will return approximately 17GB of space back to the storage pool. Figure 10 Shrinking a Thin Provisioned UFS64 Datastore below the Allocated Value 14
After the shrink operation completes, shown in Figure 11, you can see that our maximum size has been reduced to 60GB, our Free space value has been reduced, our Storage Pool free space value has increased, and our Pool Size Used value has decreased. Figure 11 Shrinking a Thin Provisioned UFS64 Datastore below the Allocated Value You can also see the thin provisioned UFS64 datastore is now fully allocated, however, you may see the Allocated value decrease if the Auto-Shrink functionality is engaged by the system. More information regarding the Auto-Shrink functionality can be found in the Auto-Shrink for Thin Provisioned UFS64 Datastores section. Converting UFS64 Datastores from Thick-to-Thin Converting UFS64 datastores from thick to thin provisioning is simple and easy to do. In Figure 12, you can view the current Capacity attributes of the selected UFS64 datastore including its thin attribute (highlighted in red) which is currently disabled. To convert this UFS64 datastore to a thin provisioned datastore, you just need to enable the Thin checkbox and click the Apply button (not shown) to initiate the background operation to modify the storage resource. Figure 12 Converting UFS64 Datastore from Thick to Thin: Screen 1 15
Once the conversion operation completes, shown in Figure 13, you will see a new value in the legend on the right the Allocated value. Because you are converting a thick provisioned storage resource to a thin provisioned storage resource, the Allocated value after the operation completes will show as fully allocated. You may see the Allocated value decrease over time when the Auto-Shrink functionality is engaged by the system. More information regarding the Auto-Shrink functionality can be found in the Auto-Shrink for Thin Provisioned UFS64 Datastores section. Figure 13 Converting UFS64 Datastore from Thick to Thin: Screen 2 Converting UFS64 Datastores from Thin-to-Thick Converting UFS64 datastores from thin to thick provisioning is also simple and easy to do. In Figure 14, you can view the current Capacity attributes of the selected UFS64 datastore including its Thin attribute (highlighted in red) which is currently enabled. To convert this UFS64 datastore to a thick provisioned datastore, you just need to disable the Thin checkbox and click the Apply button (not shown) to initiate the background operation to modify the storage resource. Figure 14 Converting UFS64 Datastores from Thin to Thick: Screen 1 16
Once the conversion operation completes, shown in Figure 15, you will no longer see the Allocated value in the legend on the right. Because you are converting a thin provisioned storage resource to a thick provisioned storage resource, the maximum size of the UFS64 datastore will be fully allocated from the storage pool. NOTE: When using thin provisioned storage resources, be aware of overprovisioning of the storage pool. This is especially important when converting a thin provisioned storage resource to thick provisioned because the system will allocate the entire size of the storage resource from the storage pool. Figure 15 Converting UFS64 Datastores from Thin to Thick: Screen 2 System-Managed Operations In the following sections, you will find details regarding the operations that the VNXe3200 system will initiate and manage on any thin provisioned UFS64 File Systems. General use of the thin provisioned UFS64 Datastores will cause these operations to engage. There are no user-configurable settings for these operations, however, the VNXe3200 system will prioritize all host I/O operations higher than the system-managed operations, which are detailed below. Auto-Extend for Thin Provisioned UFS64 Datastores The Auto-Extend functionality looks at the ratio of Used space compared to Allocated space of a thin provisioned UFS64 datastore and will allocate additional capacity from the storage pool if needed. The VNXe3200 system will actively track the Used/Allocated ratio of the UFS64 datastore, and whenever this ratio exceeds 75%, the system will look to allocate additional capacity from the storage pool to bring this ratio less than or equal to 75%. This process will allocate additional capacity as needed to satisfy this ratio up to the maximum size of the UFS64 datastore. This means the Auto-Extend process will not extend the maximum size of the UFS64 datastore. If you need to extend the maximum size of the UFS64 datastore, you will need to manually initiate that operation. Please refer to the Extending UFS64 Datastores section for more information. Because this process will allocate additional capacity from the storage pool for the storage resource, it is important to monitor the available (or free) space on the storage pool, especially if it is oversubscribed. 17
Auto-Shrink for Thin Provisioned UFS64 Datastores The Auto-Shrink functionality looks at the ratio of Used space compared to Allocated space of a thin provisioned UFS64 datastore and will reclaim capacity from the storage resource and return it to the storage pool if needed. The VNXe3200 system will analyze the UFS64 datastore s Used/Allocated ratio in the background without any user intervention. If this ratio stays less than 70% for several hours, the system will look to reclaim capacity from the storage resource and return it to the storage pool. The VNXe3200 system will shrink the UFS64 datastore as necessary to bring the Used/Allocated ratio back to 70-75%. This process will reclaim capacity as needed to satisfy this ratio for the UFS64 datastore. This means the Auto-Shrink process will not shrink the maximum size of the UFS64 datastore. If you need to shrink the maximum size of the UFS64 datastore, you will need to manually initiate that operation. Please refer to the Shrinking UFS64 Datastores section for more information. Conclusion By leveraging UFS64 File Systems in EMC s VNXe3200 storage systems, administrators have the ability to leverage a new file system architecture in their storage environment that is more scalable, provides better capacity efficiencies, and is hardened to deliver better availability and recoverability for their critical file data. This new file system architecture pushes the limits well beyond what is available today, providing the firepower needed to fulfill file-level workloads and requirements today while providing the scalability to fulfill future needs. References For additional information regarding any of the topics presented in this white paper, please refer to the following documentation available on EMC Online Support: EMC Unisphere for the VNXe3200 white paper Unisphere CLI User Guide Using a VNXe System with VMware NFS or VMware VMFS VNXe Unisphere Online Help 18