BEST PRACTICES FOR USING SMB3 MULTICHANNEL FOR 4K VIDEO PLAYBACK

Transcription

1 BEST PRACTICES FOR USING SMB3 MULTICHANNEL FOR 4K VIDEO PLAYBACK Uncompressed 4K image sequence playback with EMC Isilon, SMB3 Multichannel, and the Assimilate SCRATCH play application ABSTRACT This white paper explains the configuration requirements for playback of uncompressed 4K image sequences over SMB3 multichannel. This document details the EMC Isilon OneFS Operating System or greater configuration requirements as well as the architectural requirements for greater than 10GbE connectivity. The Assimilate SCRATCH Play application is used as the reference platform for 4K playback in the document. September, 2014 EMC WHITE PAPER

2 To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller, visit or explore and compare products in the EMC Store Copyright 2014 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. All other trademarks used herein are the property of their respective owners. Part Number H

3 TABLE OF CONTENTS EXECUTIVE SUMMARY... 4 AUDIENCE... 4 EMC ISILON ONEFS IMPROVEMENTS... 4 SMB MULTICHANNEL SUPPORT... 5 SMB 3 MULTICHANNEL BEST PRACTICES... 6 Client Driver Tuning... 6 Network Considerations... 7 ISilon requirements... 7 Network validation... 7 FILE NAME PREFETCH... 8 ASSIMILATE SCRATCH PLAY... 9 CONCLUSION

4 EXECUTIVE SUMMARY There's no denying the inevitability of 4K. 4K support is either the 800 pound gorilla you're media company is currently wrestling or it's at least a checkbox item on technologies you need to fit into your workflow in the coming years. It won't be long before 4K TVs are the only option at your local big box electronics store. Almost every video camera manufacturer has a 4K camera out now, including low-cost consumer cameras. Without a doubt, the time to switch to 4K acquisition and production workflows is NOW! HDTV and 2K content in the looming 4K world will be of little or no value from a quality standpoint. Isilon customers have been able to support streaming data rates up to ~800 MB/s per client since the introduction of the X400 platform. 800 MB/s per client is sufficient for most media formats, including 4K formats ranging from multiple streams of the compressed Sony XAVC codec up to Ultra HD resolutions. Media and Entertainment businesses currently working with 4K digital intermediate workflows are unable to take advantage of advanced 4K compression technologies like the Sony XAVC 4K codec. Uncompressed 4K media presents a number of challenges: File size - an hour of uncompressed 4096X bit RGB content at 24 frames per second requires over 4 TB of storage capacity. File format - uncompressed 4K DPX and EXR image sequences are comprised of a single ~50 MB file for every frame of video. Reading and processing 24 or more 50 MB files per second with low latency can be a challenge for most storage and workstation configurations. Storage throughput - your storage system must be able to sustain 1,200 MB per second in order to stream a 4096X bit RGB image sequence. The 1,200 MB per second storage throughput requirement for uncompressed 4K has historically required a high performance SAN or local attached storage. With the introduction of SMB 3.0 multi-channel support, EMC Isilon can now handle single client streams that exceed the bandwidth limitation of a single 10GbE connection! Applications such as Assimilate SCRATCH that are compatible with the Windows 8 SMB 3.0 multi-channel implementation may now take advantage of SAN performance over NAS by automatically load-balancing a single SMB stream over dual 10GbE network connections. AUDIENCE This white paper is intended for media professionals working with uncompressed 4K digital intermediate, color correction, restoration, or visual effects workflows. This paper focuses on 4K image sequence playback using Assimilate SCRATCH play, however, the Isilon tuning parameters detailed herein may be applied to any workflow utilizing high performance, dual 10 Gigabit Ethernet (10GbE) network interface cards (NICs) and applications optimized for network attached storage (NAS). EMC ISILON ONEFS IMPROVEMENTS Starting with EMC Isilon OneFS 7.1.1, the OneFS operating system includes a number of performance improvements: SMB Multichannel support OneFS supports the Multichannel feature of SMB 3.0, which establishes a single SMB session over multiple network connections. SMB Multichannel enables increased throughput, connection failure tolerance, and automatic discovery. To take advantage of this new feature, client computers must be configured with Microsoft Windows 8 or later, or Microsoft Windows Server 2012 or later with supported network interface cards (NICs). For more information, see the SMB Multichannel section of the OneFS Web Administration Guide and OneFS CLI Administration Guide. SmartFlash caching In OneFS, level 1 (L1) cache uses random access memory (RAM) to store copies of system metadata and files requested from front-end networks. Level 2 (L2) cache uses RAM to store copies of file system metadata for files that are stored on the node that owns the data. SmartFlash, or level 3 (L3) cache, uses solid-state drives (SSDs) to hold file data and metadata released from L2 cache, increasing the total size of cache memory available in a cluster as well as the speed that data is retrieved. In OneFS 7.1.1, SmartFlash is enabled by default for new node pools. NDMP backup performance improvements OneFS uses multiple threads to restore files, making data transfer occur as fast as the tape backup device can deliver it. Additional operational enhancements improve throughput when transferring small files. 4

5 SyncIQ performance enhancements To allow multiple SyncIQ workers to replicate a single file simultaneously, SyncIQ now allows for file splitting, where a large file is split into segments, each of which is processed in parallel by a different thread. Together, all of the performance improvements provide substantial increase in performance for media applications. However, one feature SMB Multichannel support is especially important, especially with 4K workloads and applications that can take advantage of it. Notablly, the SCRATCH Play universal media player combines with SMB Multichannel support to optimize streaming 4K playback over SMB3. SMB MULTICHANNEL SUPPORT SMB Multichannel supports establishing a single SMB session over multiple network connections. SMB Multichannel is a feature of the SMB 3.0 protocol that provides the following capabilities: Increased throughput OneFS can transmit more data to a client through multiple connections over a high speed network adapters or over multiple network adapters. Connection failure tolerance When an SMB Multichannel session is established over multiple network connections, the session is not lost if one of the connections has a network fault, which enables the client to continue to work. Automatic discovery SMB Multichannel automatically discovers supported hardware configurations on the client that have multiple available network paths and then negotiates and establishes a session over multiple network connections. You are not required to install components, roles, role services, or features. You must meet software and NIC configuration requirements to support SMB Multichannel on the EMC Isilon cluster. OneFS can only support SMB Multichannel when the following software requirements are met: Windows Server 2012, 2012r2 or Windows 8, 8.1 clients SMB Multichannel must be enabled on both the EMC Isilon cluster and the Windows client computer. It is enabled on the Isilon cluster by default. SMB Multichannel establishes a single SMB session over multiple network connections only on supported network interface card configurations. SMB Multichannel requires at least one of the following NIC configurations on the client computer: Two or more network interface cards. One or more network interface cards that support Receive Side Scaling (RSS). One or more network interface cards configured with link aggregation. Link aggregation enables you to combine the bandwidth of multiple NICs on a node into a single logical interface. SMB Multichannel automatically discovers supported hardware configurations on the client that have multiple available network paths. Each node on the EMC Isilon cluster has at least one RSS-capable network interface card. Your client-side NIC configuration determines how SMB Multichannel establishes simultaneous network connections per SMB session. 5

6 Table 1 Possible Client-Side NIC configurations SINGLE RSS-CAPABLE NIC MULTIPLE NICS SMB Multichannel establishes a maximum of four network connections to the Isilon cluster over the NIC. The connections are more likely to be spread across multiple CPU cores, which reduces the likelihood of performance bottleneck issues and achieves the maximum speed capability of the NIC. If the NICs are RSS-capable, SMB Multichannel establishes a maximum of four network connections to the Isilon cluster over each NIC. If the NICs onthe client are not RSS-capable, SMB Multichannel establishes a single network connection to the Isilon cluster over each NIC. Both configurations allow SMB Multichannel to leverage the combined bandwidth of multiple NICs and provides connection fault tolerance if a connection or a NIC fails. Note SMB Multichannel cannot establish more than eight simultaneous network connections per session. In a multiple NIC configuration, this might limit the number connections allowed per NIC. For example, if the configuration contains three RSS-capable NICs, SMB Multichannel might establish three connections over the first NIC, three connections over the second NIC and two connections over the third NIC. AGGREGATED NICS SMB Multichannel establishes multiple network connections to the Isilon cluster over aggregated NICs, which results in balanced connections across CPU cores, effective consumption of combined bandwidth, and connection fault tolerance. Note The aggregated NIC configuration inherently provides NIC fault tolerance that is not dependent upon SMB. SMB 3 MULTICHANNEL BEST PRACTICES For best performance playing back uncompressed 4K image sequences, configure your Windows client with a high performance, dual-port 10GbE NIC like the Intel X520 and a high-performance non-blocking 10GbE switch like the Arista 7150S series. Client connectivity is simple just attach 2 SFP+ short range optical or compatible Twinax cables from the client to the switch. No complicated Link Aggregation Control Protocol (LACP) configuration is required or desired, SMB3 multichannel will automatically detect the IP addresses of both 10GbE interfaces on the client and load balance across each of the two interfaces on the dual-ported NIC. CLIENT DRIVER TUNING In the Advanced properties tab of the NIC driver, the Jumbo Packet setting should be set to the maximum value of 9000 MTU. Later in this document, we ll discuss the need for ensuring that all clients on the network segment dedicated to high performance NAS connectivity are set to an MTU of In the NIC setting for Maximum Number of RSS Queues, the value should be set to at least match the physical core count for the client s CPU. Also, ensure that the Receive Side Scaling setting is set to Enabled. The Receive Buffers setting should be set to 4096 The Transmit Buffers setting should be set to If using Windows Server 2012, make sure the Enable Virtual Machine Queues setting under Virtualization is set to disabled. If using Windows 8.x, you may need to set the interrupt moderation rate setting in the advanced Intel NIC driver properties to disabled in order to achieve optimal performance. The NIC should be installed in at least an 8X PCIe Gen 2 slot in the 4K workstation. Additionally, the workstation must have sufficient RAM, CPU, and GPU resources to support the playback of a high-data rate 4K stream. In the tests conducted to create this document, a Lenovo D30 workstation with dual 12 core Xeon CPU, 32 GB of RAM, and an NVIDIA Quadro K5000 GPU were employed. 6

7 NETWORK CONSIDERATIONS In the configuration tested for this document, each client and each node in the Isilon cluster is dual-homed for Ethernet connectivity. A standard 1 GbE Ethernet connection with a gateway, DNS configuration, and standard 1500 MTU was used for all client and Isilon node access to the Internet, Active Directory authentication, and so on. A Dedicated 10GbE network is employed for all media playback requirements. By employing a dedicated media network, much in the same way one would deploy a dedicated highperformance SAN network, 3 performance issues are easily mitigated: 1. Traffic interrupts and chatter from applications unrelated to the 4K workflow do not degrade client network performance. 2. All high-performance streaming media playback is isolated to a non-routed network segment with no latency from additional network hops. 3. The requirement for all 10GbE clients on the network to use MTU of 9000 rather than a standard MTU of 1500 are easily met only clients requiring high-performance 10GbE connectivity need to connect to the dedicated MTU 9000 LAN segment and each client on the MTU 9000 LAN segment is dual-homed with a primary 1 GbE network interface for all non-streaming traffic. ISILON REQUIREMENTS The X410 nodes should have dual 10GbE network interfaces configured for MTU 9000 and connected to the same LAN segment as the MTU GbE interfaces of the 4K playback workstations. Each Isilon 10GbE interface is given it s own static IP address in the same subnet do not bond the 10GbE connections using LACP. On the Isilon server side, you will need a cluster with at least 3 X410 nodes for each client that will handle a 4K stream, assuming you are doing a full frame 4K stream at 24 fps. An additional 4K stream can be supported for every 3 rd Isilon node a three node cluster supports one stream, a five node cluster supports two streams, and so on. Table 2 shows the bit rate for a full frame 4096X3112, 24 fps stream in contrast to some other 4K formats. Table 2 Bitrates for Various Uncompressed 4K Formats FRAME SIZE COLOR DEPTH FRAMES PER SECOND MEGABITS PER SECOND 4096X bit RGB X bit RGB X bit RGB X bit RGB X bit RGB X bit RGB 24 1, X bit RGB 24 1,200 NETWORK VALIDATION In order to validate that you have maximum 10GbE Connectivity between each 10GbE interface on your client and each 10GbE interface on the Isilon cluster, run a quick validation of network integrity using the open source iperf tool. On a client with iperf installed (client_shell) and a single Isilon node (Isilon_node), run the following test: isilon_node# iperf -s -w 2M -l 1M -N client_shell# iperf -c <node 10G IP> -w 2M -l 1M -N -t 30 (this measures "write speed" using a single iperf thread, and would represent the most that any single thread from an application could achieve) Then do: isilon_node# iperf -c <client 10G IP> -w 2M -l 1M -N -t 30 client_shell# iperf -s -w 2M -l 1M -N (this measures "read speed") 7

8 This test will ensure that you are getting >9 Gigabit per second throughput between your client NIC and the Isilon NIC without performance degradation due to a network or client configuration issue. FILE NAME PREFETCH To improve image sequence playback performance even further, Isilon has added a feature to pre-fetch sequentially named files such as DPX, Cineon, and EXR sequences into RAM cache as soon as the first file in the sequence is read by a client. To enable this sequential file name prefetch feature on a node in your cluster, you will need to add the following sysctl settings to the /etc/mcp/override/sysctl.conf file on the node that will handle client connections for sequential file playback. Please consult EMC Isilon customer support before implementing any of the changes detailed below: isi.access.custom1.write.prealloc=1 isi.access.custom1.write.writealloc_blocks=0 isi.access.custom1.write.realloc_blocks=0 isi.access.custom1.fnprefetch.size_limit= isi.access.custom1.fnprefetch.l1_all=1 isi.access.custom1.fnprefetch.lookahead=9 isi.access.custom1.fnprefetch.fn_enabled=1 isi.access.custom1.coalescer.coal_falloc=1 isi.access.custom1.coalescer.cregion_max_forced=16 isi.access.custom1.coalescer.cregion_min_size= isi.access.custom1.prefetch.meta_pgs=8 isi.access.custom1.prefetch.l2_ssd_disable=0 isi.access.custom1.prefetch.l2_cluster_blocks=512 isi.access.custom1.prefetch.l2_window_blocks=8192 isi.access.custom1.prefetch.l1_cluster_blocks=0 isi.access.custom1.prefetch.l1_window_blocks=1024 isi.access.custom1.prefetch.offload=1 isi.access.custom1.prefetch.aread=0 isi.access.custom1.prefetch.read_cluster_limit=256 isi.access.custom1.read_batch=0 Once the changes to /etc/mcp/override/sysctl.conf are saved, run the following command to the directory containing the image sequences, assuming the path is /ifs/data/dpx in this example: isi set -R -l streaming /ifs/data/dpx This will set the data access pattern for the folder (recursively) to Streaming mode, increasing the number of spindles used to serve the stripes of streaming media files on OneFS. Once this command completes, run the following following command: isi set -R -a custom1 /ifs/data/dpx This will set the custom1 prefetch configuration defined in /etc/mcp/override/sysctl.conf for the directory in question. For more information about the modifying sysctl settings, please see EMC Isilon KB article The custom1 prefetch configuration must be set on a per-node basis to any node that will serve 4K image sequences. Please note that it is best to dedicate specific nodes for 4K streaming, as you will not want to have competing client traffic attached to the node in question. Further, the stress in L1 cache created by the file name prefetch configuration could be disruptive to other workflows using L1 cache on the node in question. 8

9 ASSIMILATE SCRATCH PLAY EMC Isilon partner Assimilate have provided a free version of the SCRATCH Play universal media player on their website at No special configuration is required to optimize SCRATCH play for streaming 4K playback over SMB3. Simply install the application, load a CONstruct, and import media from an Isilon SMB share via a node with the file name prefetch configuration. In order to verify playback of 4K content without dropping frames, simply press ctrl + F1 to bring up the playback statistics display. CONCLUSION 4K workflows are inevitable. The 1,200 MB per second storage throughput requirement for uncompressed 4K has historically required a high performance SAN or local attached storage. With the introduction of SMB 3.0 multi-channel support, EMC Isilon can now handle single client streams that exceed the bandwidth limitation of a single 10GbE connection. Applications such as Assimilate SCRATCH that are compatible with the Windows 8 SMB 3.0 multi-channel implementation may now take advantage of SAN performance over NAS by automatically load-balancing a single SMB stream over dual 10GbE network connections. 9