Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000

Transcription

1 Post-production Video Editing Solution Guide with Microsoft SMB 3 File Serving AssuredSAN 4000 Dot Hill Systems introduction 1

2 INTRODUCTION Dot Hill Systems offers high performance network storage products that are ideally suited for environments where distributed access to data is desired, and high speed streaming access to the data is an absolute must. In particular, the AssuredSAN 4000 series storage arrays are specifically designed for networked access, % availability and streaming performance of over 5000 MB/s. An emerging data streaming environment is the Video Post-production Editing studio using Microsoft Windows workstations and a professional editing suite such as Adobe Premiere. However, video editing is one of the most data intensive applications on a computer. This means that the requirements of the system demand fast access to large amounts of data, usually in a sequential (streaming) fashion. At the same time, modern workflows require that the video files be available at any one of many Workstations, since the same video may be edited in different ways by different individuals at different Workstations. These two pervasive characteristics of an efficient Video Editing studio tend to be at odds with each other: 1) High speed access usually means using local storage, or high speed access to networked storage. 2) Distributed access means using File Serving in order to access a file from any Workstation. Local storage isn t an option here. However, most file serving solutions are far too slow for the streaming needs of an Editing Workstation. These two requirements may be addressed with the new SMB 3 File Serving Protocol, from Microsoft. The distinctive characteristics of this solution is the distributed nature of file serving (files can be accessed from any Workstation), and the high speed access to the storage. When matched with the AssuredSAN 4000 storage array, all the pieces are in place for a professional video editing environment. Unfortunately, the task of identifying all the necessary pieces and putting them together can be confusing and intimidating to the un-initiated. And storage vendors don t help matters by offering a confounding mix of competing and overlapping protocols, form factors, drive types and interfaces. It is clear that simple guidelines are needed to point the way. This whitepaper will outline the elements of a Microsoft Windows-centric Video Post-production Editing environment that includes high speed access to all data files from every Workstation. It will include recommendations for equipment, connectivity, configurations, and options, along with best practice guidelines to assist customers in optimizing the performance and reliability of their investment. The intent is to guide the customer or the integrator along the path to success in deploying a distributed storage subsystem for Windows Workstations. Dot Hill Systems introduction 2

3 THE VIDEO POST-PRODUCTION EDITING SOLUTION More and more, Video Post-production editing studios are deploying Microsoft Windows Workstations to perform a wide range of video processing and editing tasks. In this environment, multiple editing Workstations are used for manipulating one or more video streams, and then rendering and saving the results to disk. Following are the critical needs of a Video Post-production environment: Capacity Cost Speed Availability Video has an insatiable appetite for Megabytes. Value is an important aspect of the solution. High speed access to data with low latency allows for streams to be uninterrupted (no dropped frames) during editing sessions. This means that the video files should be accessible from any Workstation at any time. This also may refer to the robustness of the solution. A highly available solution can sustain the failure of any single component without losing access to any data. Since the editor needs to have access to any video file from any editing Workstation, it is necessary to provide a networked file system for this purpose. An optimized network file system, such as Microsoft SMB 3, can provide the fast, streaming access to data that is necessary for Video Post-production studios that utilize Windows Workstations. At the same time, the file system is designed to share the resources of the file volumes among multiple client systems. When combined with Dot Hill Systems 4000 Storage Arrays, and SAS connectivity to the File Serving host, this video editing solution can deliver more than 5000 MB/s to the Windows 8 Workstations used for video editing. In video terms, that is more than 20 uncompressed High Definition video streams being rendered at the same time. The key to constructing a high performance video editing studio is in the selection of components, the proper network configuration, and the proper storage configuration. The remaining sections of this solution brief deliver an overview of recommended equipment, configuration, and deployment best practices. Dot Hill Systems The Video Post-production Editing Solution 3

4 SOLUTION COMPONENTS Storage Array. The Dot Hill AssuredSAN 4500 is the RAID array storage subsystem for this solution. It features dual, active-active RAID controllers with eight 4Lane 6Gbit SAS interfaces. The array supports SAS disk drives in the 2.5 and 3.5 form factors. File Server. It is absolutely necessary to have at least one file server in the environment. We recommend the use of a late model Intel server that supports the PCIe Gen3 bus architecture, and provides sufficient HBA slots for the necessary SAS and 10Gbit iscsi interfaces. One good example is the HP ProLiant DL380 Gen8 server. If high availability is a requirement, then two servers must be deployed. The File Servers must be running the Windows Server 2012 Operating System Client Workstations. The Workstations play the role of the Video Editing Workstation. A Video Post-production environment will have one or more of these Workstations deployed, all working from the same storage subsystem. A popular Workstation that is commonly used for video editing is the HP Z820. The Workstation must be running the Microsoft Windows 8 Operating System. SAS HBA. The File Servers must have a SAS Host Bus Adapter (HBA) installed for connectivity to the Storage Array. The recommended SAS HBA for this solution is the LSI SAS e. Ethernet HBAs. The File Servers and the Workstations must have at least one 10Gbit Ethernet Host Bus Adapter (HBA) installed for File Serving connectivity. To provide sufficient throughput, one dual port HBA is recommended for each Workstation. Each File Server will require two dual port HBAs (four ports total) in order to accommodate the available bandwidth of the storage. A recommended 10Gbit HBA that supports the advanced features of SMB 3 is the Mellanox ConnectX-3 EN Dual-Port 10Gbit Ethernet HBA. Dot Hill Systems Solution Components 4

5 Ethernet Switch. Connectivity between the File Servers and the Workstations requires an Ethernet switches that support 10Gbit interfaces. High Availability configurations require the deployment of two switches. Connections. All of the 10Gbit Ethernet components are connected with fiber optic cables. In the terminology of cables and connectors, the type you will need is LC-LC. You can purchase these cables in a variety of lengths. These cables will connect directly into the ports provided by the 10Gbit Ethernet HBAs, the AssuredSAN 4000 storage system and the 10Gbit Ethernet Switch. In figure 1, each orange line represents one fiber optic cable. IMPORTANT NOTE: These cables plug into a module called a transceiver on the port. Most components already provide the transceivers pre-installed on all the ports. However, some components (HBAs, switches) do not. You may have to purchase transceivers separately for the ports on the devices you purchase, along with a license to use those ports. Connectivity between the File Servers and the Storage Array is accomplished with standard SAS cables. The connector type required is SFF-8088 on each end. Dot Hill Systems Solution Components 5

6 SOLUTION CONFIGURATION Refer to Figure 1 for an example of how to deploy and connect Workstations, servers and storage in a Microsoft Windows SMB 3 File Serving environment. The Workstations are deployed as Video Editing stations. Figure 1 Video Post-production Editing Solution with Microsoft SMB 3 File Serving Fully Redundant Configuration Dot Hill Systems Solution Configuration 6

7 SOLUTION BEST PRACTICES When configuring the storage subsystem for a Video Post-production Editing environment, the following Best Practices will aide in configuring the solution for high performance and reliability. 1) Consider the need for High Availability Redundant components in a storage network are necessary if continuous access to storage is of paramount importance. The Dot Hill AssuredSAN storage system comes pre-configured with redundant controllers and redundant power supplies. The loss of any one of these components will not compromise access to storage. A fully redundant storage network will also include two File Servers, combined together with Microsoft Failover Clustering. The interface requirements of each server would be a single dual port SAS HBA, and two dual port 10Gbit Ethernet HBAs. In addition, two Ethernet switches are required to eliminate the switch as a single point of failure. This configuration allows for any single component in the system to fail, and still provide un-interrupted access to all storage. Beyond the deployment of redundant components, another important aspect of redundancy in a storage network is in the cabling. Proper cabling provides redundancy, so that the failure of any one component or path does not compromise access to any of the storage. At the same time, this redundancy should also provide a mechanism for aggregating throughput, delivering higher performance to the host systems. Figure 1 provides an example of proper cabling for throughput and redundancy. 2) Use Nearline (7K RPM) SAS drives. These drives provide the best MB/$ value, the highest storage density, and the reliability of an enterprise class disk drive. Don t let the term nearline fool you. 7K SAS drives can saturate the throughput of the system almost as easily as 10K SAS drives can. 3) One Volume (LUN) per RAID group is best In a configuration where sequential throughput (i.e. video streaming) is important, then configuring the physical disk drives for minimal seek operations is a good idea. If multiple Volumes are created on a RAID group (VDisk), then the physical disk will require seek operations as it goes from one Volume to the other to perform I/O. To prevent unnecessary seek operations; the configuration should be designed with only one Volume per RAID group (VDisk). 4) Configuration of RAID groups is important In Video Post-production environments, fast sequential (streaming) access to storage is important. Also important is optimized use of disk space. These two factors combine to suggest that RAID 5 or RAID 6 is the best option for data volumes in a File Serving environment. These RAID levels provide redundancy protection against drive failure, yet also provide high speed sequential access to data. Dot Hill Systems Solution Best Practices 7

8 5) Configure MPIO on File Servers To optimize the features of SMB 3 File Serving on Microsoft Windows, all volumes (LUNs) should be presented to all the hosts via all the SAS ports (default presentation). Using this methodology, there will be more than one path to each Volume. Not only that, but some paths may be more optimized or preferred, meaning that they will operate faster than other paths. To account for this, the Server must have the Microsoft Multipathing (MPIO) feature installed, and the SPC-3 compliant SAS DSM must be discovered and added to the list of MPIO devices. The driver will balance the load ONLY across the preferred paths. The non-preferred paths will be held in standby in case the preferred paths fail. FREQUENTLY ASKED QUESTIONS 1) Isn t SMB a relatively slow, ordinary File Serving protocol? Video streaming demands tremendous throughput and no dropped frames. Traditionally, the SMB protocol (and its cousin, NFS) has been deployed as a simple file sharing mechanism for office workers. The SMB 3 protocol, introduced on Windows 2012 and Windows 8, represents a complete re-write of the entire software stack, with the intent of providing a high speed, low latency solution for demanding applications. In addition, SMB 3 includes new features that take advantage of newer hardware configurations, such as multiple processors, multiple NIC interfaces, and RDMA. Testing has demonstrated that SMB 3 is a serious solution for streaming applications. 2) Why all the 10Gbit Ethernet ports on the File Server? Shouldn t two ports provide enough redundancy? The AssuredSAN 4000 storage array is rated to deliver over 5000 MB/s in sequential throughput. To provide this throughput through the File Server, enough bandwidth must be provided by the SAS interfaces and the Ethernet interfaces, so as not to introduce a performance bottleneck. The four Ethernet ports specified on the File Server represent over 4700 MB/s of throughput capability. Even this amount of bandwidth would represent a potential bottleneck if the load was not shared with a second File Server. 3) Are there other network connectivity options to consider besides 10Gbit Ethernet? Yes. The 40Gbit Ethernet and 100Gbit Ethernet standards are defined and products are now available. These interfaces would be ideal between the File Server and the Ethernet Switch. Alternatively, Infiniband interfaces may be used between the File Server and the Workstations (SMB 3 works great over Infiniband). While all of these options provide much more throughput potential with fewer ports, they are still relatively expensive to deploy. 4) Can I use other Workstations besides Windows 8? How about Windows 7, or a Mac Pro? Dot Hill Systems Frequently Asked Questions 8

9 The SMB 3 protocol is currently only deployed and supported on Microsoft Windows 8 and Microsoft Windows Server To gain the advantages of this File Serving protocol, the video editing Workstations must be running Windows 8, and the File Server must be running Windows Server ) Doesn t a File Server represent a bottleneck in the data path? How can this be better than a solution that connects the Workstations directly to the Storage Array? Traditional File Server deployments were perceived as a performance bottleneck due to the latency and inefficiencies introduced by the File System and the File Serving protocol. With SMB 3, the protocol stack has been optimized to serve I/O and throughput nearly as fast as the underlying storage system. This means that the bottleneck is no longer in the software stack, but in the limitations of the hardware or the interfaces. On new servers with PCIe 3 bus architectures, sufficient resources are available to handle the I/O and the throughput of an AssuredSAN array. 6) Is SMB expensive? Is it difficult to configure and manage? SMB 3 is pre-installed with every Windows 8 and Windows Server 2012 Operating System. The only licensing required is the standard Client Access Licenses (CAL) for Windows. There are no agents to install, and nothing to download from the Internet. Configuration is no more complicated than the traditional Windows File Server setup. There is nothing special to do in order to gain the performance advantages of SMB 3. CONCLUSION The Dot Hill AssuredSAN 4000 storage array is an ideal storage solution in Microsoft SMB 3 File Serving environments. The AssuredSAN has been demonstrated to work seamlessly with Microsoft Windows 2012 Server and supported SAS HBA s. In addition to the inherent benefits of SAN based storage, the AssuredSAN storage array delivers: High Performance throughput of over 5000 MB/s Reliability redundant components and paths High Availability Clustered controllers with failover technology Flexibility a variety of drive options and configurations The Microsoft SMB 3 File Serving configuration with AssuredSAN 4000 storage is of special interest to the Video Post-production Editing market. This solution offers distributed access to storage, a necessary element of a shared editing environment. The solution utilizes 6Gbit SAS interfaces for block access to the array, 10Gbit Ethernet for File access to the Server, along with path optimization and load balancing to deliver tremendous throughput for intensive streaming and rendering applications. Dot Hill Systems Conclusion 9

10 2013 Copyright Dot Hill Systems Corporation. All rights reserved. Dot Hill Systems Corp., Dot Hill, the Dot Hill logo, and AssuredSAN are trademarks or registered trademarks of Dot Hill Systems. All other trademarks are the property of their respective companies in the United States and/or other countries Dot Hill Systems Conclusion 10