Virtualized Video Infrastructure with Harmonic Media Processing Applications on HP Moonshot Systems Technical Guide September 2015 1
Table of Contents Abstract... 3 Introduction... 3 The Harmonic-HP VVI... 4 Harmonic Media Processing Software Applications for HP Moonshot... 5 WFS... 5 ProMedia Carbon... 6 ProMedia Xpress... 6 VOS... 7 Transcoding-on-the-Fly... 7 The HP Moonshot System... 8 HP Moonshot 1500 Enclosure... 8 HP ProLiant m710p Server Cartridge... 11 HP Moonshot Bill of Materials... 12 Proof of Concept... 12 The Moonshot Discovery Lab... 12 Conclusion... 13 Footnotes... 13 2
Abstract This document describes the benefits and features of a Virtualized Video Infrastructure (VVI) with the Harmonic WFS file-based workflow engine and VOS virtualized media processing platform operating on the HP Moonshot integrated server system. The key components and system architecture for the Moonshot server optimized for Harmonic media processing software applications are highlighted. Introduction Video service providers are increasingly pursuing virtualized solutions based in data centers rather than in traditional broadcast facilities. The objective of these VVI solutions is to achieve a higher-efficiency, lower-cost video service with application mobility, better quality of service (QoS) and superior video quality (VQ). Figure 1. Video service solution objectives Real-time video encoding and transcoding involves huge amounts of media traffic and intensive computation. It demands real-time performance but is delay-sensitive. Video compression must also deliver the highest video quality for content delivered by various live video services. In conjunction with these realities, the need for file-based video transcoding is growing faster than traditional hardware-based infrastructure can sustain. Between the years 2013 and 2018, IP traffic is expected to grow 300%, and IP video is predicted to be 79% or more of that activity by 2018 1. The main reason for this trend, of course, is that the amount of consumers around the world demanding the ability to view high-resolution video content from anywhere, on any device, at any time, continues to skyrocket. While video on demand (VOD) is currently the leading type of content consumed over Internet-connected devices, live video delivered to any screen is a growing capability of many service providers. Simultaneously, the expected adoption of Ultra HD television in coming years will further strain existing media processing infrastructures. 3
The Harmonic-HP Moonshot VVI solution is specifically designed to address the evolving ways people consume video content. Operating on the Moonshot server system with ProLiant m710p server cartridges, the architecture provides a framework for service providers to remain competitive by accelerating the production rate of their media processing. The Harmonic software applications optimized for the Moonshot platform include: WFS, an enterprise-class workflow engine for controlling multiple ProMedia Carbon and ProMedia Xpress transcoding nodes VOS, a software-based, fully virtualized platform that unifies the entire media processing chain, from ingest to delivery, and enables powerful real-time transcoding-on-the-fly capabilities Utilizing Harmonic media processing application on the Moonshot platform, IT managers can extend the value of their content processing workflow and delivery to a broader set of partners and end-users, as well as to a wider array of devices. The Harmonic-HP VVI Hardware-based media processing and delivery solutions are standard tools for most content operators and service providers. However, this type of approach requires expensive and complex infrastructure that is both difficult and time-consuming to manage, and doesn t scale well for increased capacity. A VVI solution addresses these challenges with the following features: Eliminates the need for expensive and variant custom hardware by using a COTS hardware platform and cartridge-based servers. Employs the same common infrastructure for video applications with different functions implemented in the application software. Simplifies the ability to add new features and functions to the application software; no new hardware is required. Decreases the need for data center rack space by integrating power modules, fabric connection I/O modules, servers and management modules. Reduces the cost of power and cooling in the data center by maximizing power efficiency in the platform enclosure. Dramatically reduces switch port count, along with cabling and wiring complexity and cost. Simplifies and unifies infrastructure management. VVI uses only a common infrastructure platform manager, common virtualization layer manager and video application manager. Involves only the common infrastructure platform, cartridge servers, application software, virtualization management and tools; all of these are shared for different applications in the same inventory pool. Uses a common physical platform with cartridge-based servers for different applications, including new products and features, reducing the approval testing cycle. Accelerates the ability to launch new revenue-generating services by reducing the time needed to deploy new functions and features. Scales easily to increase capacity with approved basic VVI building blocks. Supports dynamic services on demand; quickly switches the software functions and features. With the above listed advantages, a VVI solution provides lower TCO and greater agility over traditional custom, discrete appliance-based or fixed-function solutions. 4
The Harmonic-HP Moonshot VVI solution applies the ideal set of compute, I/O and storage resources to deliver these results. With complementary technology at the hardware and software levels, this solution enables customers to boost productivity by consolidating and simplifying their infrastructure, as well as gain greater space and power efficiency. This pre-engineered infrastructure includes integrated compute, storage and networking, and can linearly scale out from tens to thousands of concurrent encode and transcode streams. Applying the HP ProLiant m710p compute engine within the Moonshot ecosystem provides a pre-engineered, high-density, scalable, low-cost and flexible-to-deploy architecture. The solution can function on its own or in conjunction with multiple geographically dispersed headends for OTT multiscreen delivery. Harmonic Media Processing Software Applications for HP Moonshot Harmonic is the worldwide leader in video delivery infrastructure for emerging television and video services. By providing market-leading innovation, high-quality service and compelling TCO, Harmonic enables customers to produce, deliver and monetize amazing video experiences with unequalled business agility and operational efficiency. The Harmonic software applications optimized for the Moonshot system are the WFS file-based workflow engine and VOS virtualized media processing platform. WFS WFS is an enterprise-class workflow engine that provides powerful, intuitive control of carrier-grade transcoding farms for broadcast and OTT video production and delivery. The application is used to control multiple ProMedia Carbon and ProMedia Xpress transcoding nodes, and provides full visibility into the entire transcoding process. Open and extensible, WFS possesses a distributed nature that grants reliability and scalability while allowing the automated processing of transcoding tasks, failover support, job distribution management, job prioritization, load balancing, file transfer, status monitoring and job notification. Features of WFS include: Advanced Architecture: WFS separates the workflow and automation processes from the transcoder, allowing a much more robust and scalable architecture than previously available. An open API allows traditional outside integration, as well as the ability for third-party applications to integrate into WFS. Integration & Management: WFS is a true enterprise-class workflow engine. It allows for a set of tasks to be executed in any order with full visibility into the entire process. WFS can be used for the simplest tasks, such as retrieving files from varied locations, to complicated workflows that include multiple transcodes and condition checks, file transfers and the triggering of external scripts and notifications. Simplified File-Based Workflows: WFS simplifies the acquisition, transformation and delivery of video files. It can manage the movement of files over FTP, HTTP and CIFS, and via the Amazon S3 cloud file-storage service. Running on the Moonshot platform simplifies the ability for WFS users to manage thousands of simultaneous transcoding streams. 5
Figure 3.Harmonic s WFS workflow engine interface ProMedia Carbon ProMedia Carbon is a file-based transcoding solution that enables the conversion of media to a massive array of acquisition, editing, broadcast, web and mobile formats. Controlled by WFS, the application provides highperformance, scalable and cost-effective transcoding for a broad range of video production and distribution environments, from specialized studios to enterprise-scale installations. With ProMedia Carbon, users gain the ability to transcode from virtually any media format to every media standard in use today. Supporting all major SD and HD formats, the application is continually updated to accommodate new formats and ensure the user s ability to meet evolving content production and distribution requirements. ProMedia Xpress ProMedia Xpress provides faster-than-real-time transcoding and packaging of SD, HD and UHD assets for a broad range of multiscreen applications with superior video quality. Managed by WFS, ProMedia Xpress allows content owners and service providers to significantly increase content hours delivered while adhering to processing timetables and video quality targets in a cost-effective and scalable manner. The system employs Harmonic s MicroGrid parallel-computing technology to achieve a dramatic improvement in transcoding performance over stand-alone processing platforms. The inclusion of packaging technology and seamless integration with the Harmonic ProMedia X Origin multiscreen video server enables a complete file-based, scalable workflow that provides the operational integrity required for demanding 24/7 VOD environments. 6
VOS VOS brings together the technologies that have made Harmonic the leader in broadcast playout, pay-tv encoding, IPTV headends and file-based transcoding. It leverages the computing power of the Moonshot platform to host a robust set of dynamically deployable application modules that can be selected individually, or in combination with any of the others, to deliver the broadcast industry s first fully virtualized media processing architecture. Integrated system capabilities include ingest, compression, graphics and branding, transport stream playout, and multiscreen packaging and delivery. Encoding and transcoding within the VOS environment are performed by the software-based Harmonic PURE Compression Engine, which supports SD, HD and UHD content formats for broadcast, cable, satellite, IPTV and OTT delivery via MPEG-2, MPEG-4 AVC and HEVC codecs. Figure 4. Harmonic s VOS platform Modular in nature, VOS can be transformed on demand across the full range of media processing functions, which makes it unparalleled in its elasticity and flexibility. And as software operating across virtual machines in one or multiple data centers, it delivers significant savings in rack space, ports, power and cooling. By unifying the entire media processing chain, from ingest to delivery, VOS gives video content and service providers the ability to simplify workflows, maximize flexibility and gain unprecedented operational efficiency. Transcoding-on-the-Fly Transcoding-on-the-fly (TOTF) is a capability of the PURE Compression Engine that alleviates transcoding bottlenecks in multiscreen workflows. 2 Enabled by the power and density of VOS running on Moonshot, the Harmonic TOTF solution stores only the segments of the most common streaming profile; other profiles are generated upon request for individual devices. To increase transcoding throughput, video segments are transcoded ahead of time and stored, so that future requests are served from a fast cache. Transcoding segments are pushed to an external storage system, so that playout servers can grab them directly from storage. 7
Figure 5. A multiscreen workflow featuring TOTF capability The HP Moonshot System The HP Moonshot system is a highly efficient, flexible, software-defined server. Ideal for Harmonic VVI applications, Moonshot allows users to get more out of their infrastructure with less cost, power and space. This section details the basics of the HP Moonshot system and the ProLiant m710p server cartridges that are used for the Harmonic virtualized media processing solution. HP Moonshot 1500 Enclosure The HP Moonshot 1500 chassis is a highly integrated ecosystem with great flexibility, designed for redundancy and fault tolerance, with superior cooling and power capabilities. The Moonshot system offers the following universal benefits to all workloads: 1. Targeted high resource utilization high efficiency 2. Performance with low energy consumption 3. High compute density less space 4. Integrated infrastructure storage, networking, fabrics, management, power, cooling 5. Simplified and reduced cabling 6. Ease of new technology testing and integration 7. Simple to repair in place high availability Figure 6 and Table 1 provide an overview of the components in the HP Moonshot system. For further system information, refer to hp.com/go/moonshot. 8
Figure 6. HP Moonshot system at a glance 9
Form Factor Compute Network Switches Networking Uplink Modules Power Cooling Configurations Chassis Management 4.3 RU, 900 mm deep Up to 45 hot-pluggable cartridges top-loaded and serviceable in rack. Cartridge options include single and quad processor offerings with various SoC selections. Each cartridge offers dual port NICs per SoC device. Dual integrated network switches with options of 1G or 10G downlinks to the cartridge NICs. 45-port or 180-port options are available depending upon cartridge selection. Flexibility to use one switch in a non-redundant mode, two switches in redundant mode, or two switches teamed for High Availability. For further details on the network switches, software stacks and specifications, refer to the Networking cookbook for Moonshot switches: hp.com/go/moonshot/networking. Each network switch is paired with a corresponding uplink module to interface to the external networks. The chassis can support up to 160G bandwidth capability from each uplink module. Options include 4x 40G QSFP+, 16x 10G SFP+, 6x 10G SFP+. For further details on the network switches and specifications, refer to the Networking cookbook for Moonshot switches: hp.com/go/moonshot/networking. Up to four platinum plus hot-pluggable power supplies (1200 W or 1500 W options). Configurable in N, N+1 or 2N, depending upon desired configuration. Five hot-pluggable dual-rotor fans with N+1 rotor redundancy. Anywhere from 1-45 cartridges may be mixed in chassis, allowing for flexibility of design and custom high-density solutions. The HP Moonshot 1500 Chassis Management Module allows you to configure, update and operate the HP Moonshot system via a Command Line Interface (CLI), a Graphical User Interface (GUI), Intelligent Platform Management Interface (IPMI) or remote serial console access. Benefits include: Quickly monitor Moonshot chassis and server cartridge health and power Easily customize workflows and scripts with a fully programmable, industry-standard interface (RESTful API) Receive automatic notifications on issues via SNMP Traps Table 1. HP Moonshot system features For further information and specifications, refer to hp.com/go/moonshot. 10
HP ProLiant m710p Server Cartridge The HP ProLiant m710p server cartridge for the Moonshot chassis is a workload-optimized and right-sized solution based on the Intel Xeon E3-1284L v4 quad-core processor with Intel Iris Pro Graphics P6300. The on-die GPU provides for hardware-based graphics acceleration, which results in an ideal compute workhorse for various media processing applications. The server attributes are outlined in Table 2. Figure 7. HP ProLiant m710p server cartridges CPU Memory Network Storage Power OS 47 W Intel Xeon E3-1284L v4 quad-core 2.9 GHz (3.8 GHz single core Turbo); support for hyper-threading 6 MB of L3 cache 128 MB of L4 cache with peak bandwidth of 50 GBps Intel Iris Pro Graphics P6300, on-die GPU using L3 and L4 caches for data transfer between the CPU cores and the GPU execution units 32 GB (4) DIMMs 1600 MHz, DDR3 SO-DIMM ECC (8 GB) Low-latency and peak bandwidth of 25 GBps (8GiB per CPU core) Internal dual-port 10 GbE Mellanox CX3 with RoCE capability (RDMA over Converged Ethernet) Direct attach, optional local solid-state storage (M.2): 120 G, 240 G, 480 G, 960 G 72 W peak per cartridge, including system overhead burden and network switches 45 W typical operating power per cartridge in a full system including system overhead burden Consult the HP power calculator tools at hp.com for typical power measurements for the Moonshot full system by workload. Windows Server 2012 and 2012 R2, Windows 7 x64, 8.1 Red Hat Enterprise Linux 6.7 and 7.1, CentOS 6.7 and 7.2 Canonical Ubuntu 15.04 and 14.04.3 LTS SuSE Linux Enterprise 12 and 11 SP4 Table 2. HP ProLiant m710p server cartridge features For further information and specifications, refer to hp.com/go/moonshot. 11
HP Moonshot Bill of Materials A suggested bill of materials (BOM) is outlined in Table 3. Please consult your HP and Harmonic sales representatives or authorized reseller for additional details on service options, rack/power options and confirmation of part numbers as those are subject to change. Cartridge 808915-B21 HP ProLiant m710p Server Cartridge 45* Infrastructure 755372-B21 HP Moonshot 1500 Chassis Opt OS 1 704654-B21 HP Moonshot-45XGc Switch Kit** 1 704652-B21 HP Moonshot 4QSFP Uplink Kit** 1 681254-B21 HP 4.3 RU Rail Kit 1 681260-B21 HP 0.66 RU Spacer Blank Kit 1 Commodities 684532-B21 HP 1,500 W CS Plat PL Hot Plug Power Supply Kit 3 Network Cable Options 765479-B21 HP Moonshot 120 G SATA VE M.2 2280FIO Kit 45 JG326A JG330A HP X240 40G QSFP+ QSFP+ 1m DAC Cable*** HP X240 QSFP+ 4x10G SFP+ 3m DAC Cable*** * Up to 45 cartridges may be configured. ** If switch redundancy is desired, add one additional 704652-B21 and 704654-B21. *** For network cables and adaptors, refer to the Networking cookbook for Moonshot switches: hp.com/go/moonshot/networking. Table 3. System bill of materials Proof of Concept As a matter of best practice for all deployments, HP encourages implementing a proof-of-concept using a test Moonshot environment that matches the planned production environment as closely as possible. In this way, appropriate performance and scalability characterizations can be obtained. For help with a proof-of-concept on the Harmonic-Moonshot VVI solution, consult your Harmonic and/or HP sales representative or authorized reseller. The Moonshot Discovery Lab The HP Moonshot Discovery Lab is available for remote access and testing of your Harmonic applications on Moonshot from anywhere in the world. The lab allows customers and partners unfettered access to an HP Moonshot system to experiment, test and benchmark applications in a secure and confidential environment. Labs are located in Houston, Texas; Grenoble, France; and Singapore. You can also gain access to a Discovery Lab through the HP VPN from anywhere in the world. Visit the HP Discovery Lab site: hp.com/go/discoverylab. 12
Conclusion The world of media and video processing is growing faster than traditional compute can sustain. The flux of the industry due to continual technical changes and increased consumer expectations challenge real-time media processing and file-based transcoding workflows, and this growth and constant churn puts extreme strain on current compute solutions. These trends are unsustainable without a real change. The solution rests in collaboration between industry leaders to address such problems. In response, Harmonic and HP offer a highly scalable VVI solution that combines Harmonic WFS and VOS software applications with HP s Moonshot server system. The HP Moonshot 1500 enclosure and HP ProLiant m710p server cartridges form the nucleus of this solution, enabling a cost-efficient, fully virtualized solution that satisfies the requirement of programmers, service providers, and their discriminating customers, who demand the ability to view an ever-growing amount of video content anywhere, on any device at any time. Footnotes 1. Cisco Visual Networking Index: Forecast and Methodology, 2014-2019 White Paper 2. Check Harmonic sales for availability. 2015 Harmonic Inc. All rights reserved. Harmonic, the Harmonic logo, WFS, VOS and ProMedia are trademarks, registered trademarks or service marks of Harmonic Inc. in the United States and other countries. HP, Moonshot and Proliant are trademarks, registered trademarks or service marks of Hewlett-Packard Development Company, LP. Other company, product and service names mentioned herein may be trademarks or service marks of their respective owners. All product and application features and specifications are subject to change at Harmonic's sole discretion at any time and without notice. 09.2.15 13