March 2010 Webcasting: Dealing with significant audiences behind the corporate firewall Ed Van Petten CIO / Vice President, Network Operations ON24, Inc.
Introduction Webcasts sometimes involve significant numbers a single corporation s employees in supporting the events (manning booths and the like) and/or visiting the events as guests. In some cases, the Webcast is intended to accommodate a corporate meeting such as a sales meeting, where virtually all the attendees are from a single corporation, many attending from a network inside the corporate firewall. In these situations, it is essential to assess the demands that such events place upon the corporate network, and to provide methods to minimize any negative impacts on both the networks and the Webcast. 1
Loads: What you should be worried about When a Webcast occurs, there are several loads: The Interface mostly graphic elements and code that presents the show experience Chat traffic, if applicable Webcasts - Most webcasts are comprised of audio or video streams along with slides and various modes of interactivity. The media streams all arise from servers outside the corporate firewall. The first two of these loads are not significantly different than the normal browser traffic that a corporate network and its gateways are designed to handle routinely. Chat traffic is supported by Jabber servers, which use XMPP as their protocol for communication. ON24 s implementation tunnels the XMPP traffic using BOSH, which is a polled protocol using HTTP (http://xmpp.org/extensions/xep-0124.html). The practical effect of using BOSH is to have each client make continuing HTTP request every 5 seconds. The XMPP traffic is then passed in both directions in the 1 per 5 second package. There is no traffic other than port 80 traffic, and the packets individually are small, consistent with chat activity. The webcast traffic, however, can have characteristics that will load the network gateways significantly perhaps to the extent of congesting the gateway and interfering with other traffic. The traffic of interest is that which is simultaneous (i.e., a large number of participants initiate a stream all at once) and demanding (i.e., audio streams at 20K are not especially demanding, while video streams at 300K are more demanding). Webcast Types: On Demand Webcasts usually elements of content available for selection by the show participants at their option. These events lack the characteristic of simultaneity. While they may have 300K video streams, they are simply not accessed by large numbers of people at the same times. Live webcasts these are frequently keynotes or other addresses of significant interest. The speakers may be well known and capable of drawing a crowd. This is almost always a subject of concern if the media is video. Simulive webcasts these are webcasts which are actually pre-recorded but presented as though they were live. The attendees cannot distinguish that the presentations are not live. These events share the characteristics of live events in terms of the potential for network load. So, it s clear that loads to be concerned about include live and video audio streams are lightweight enough to ignore and on-demand webcasts do not generate the simultaneity required to cause a problem. There is another significant area of concern VPN access. It is common to have corporate VPN access set up such that the default route for all the network traffic points to the corporate VPN gateway. Even web traffic that is entirely unrelated to the corporate network flows through the corporate VPN gateway inbound and then out the corporate gateway to the internet, returning by the same path. This is in contrast to the arrangement where only traffic destined for the corporate network gets routed to the VPN, while other traffic goes directly to the local ISP. If all traffic flows through the corporate VPN, then media traffic may overwhelm the VPN gateway, even if the corporate gateway is capable of supporting the indicated volume of traffic. It also seems obvious that a corporate network where all the traffic flows through one or a few internet gateways is vulnerable in a different way than a network which is organized with each site having its own gateway to the internet. Of course the size and utilization of the gateway determines the scope of the vulnerability 2
Sensitivity Analysis It should be clear, then, that the IT staff needs to address the following points: With respect to the live and simulive webcasts, in Video format, what is the expected attendance per internet gateway location and/or for VPN users who route media streams through the VPN gateway 3
Measures to Address Vulnerable Networks The analysis suggested above should yield a list of sites which are vulnerable given the expected audience. If one or more sites are problematic, there are a options available to address the situation. They boil down to limiting the number of internal stations playing the event or re-distributing the stream inside the network to minimize stress on the gateway. Option 1: Limiting the number of internal stations have participants gather in a conference room, providing one projected stream rather than one per viewer. Block access to the stream through the corporate firewall which protects the network but prevents access to the stream Password protect access to the event, and limit the distribution of the valid password Limit the distribution of the access URL to a small group of personnel Option 2: Redistributing the stream inside the network Plainly, it is possible to locate a source for the stream inside the corporate network. Several options exist: Multicasting - In the event that the network is multicast enabled, and the multicast is reliably known to be functional in all locations, it is possible to use it to distribute the streams inside the network. Using multicasting, a stream is fetched from the head end media server and replicated to a multicast-capable media server internal to the customer network. This customer multicast server actually delivers the stream to the end users. Unicast Replication - It is also possible to configure unicast replication. In this scenario, Windows Media servers are deployed in the client network, and publishing points set up to enable to replication of streams from the head end server. End users connect to the local streaming servers, fetching the streams from a local network location rather than through the internet gateway. This relieves the capacity strain on the inbound gateway. Unicast Replication and Multicasting are further documented in the Advanced Signal Services PDF, which is available upon request. Peer-to-peer stream propagation Multicasting typically requires a considerable lead time to get active and tested in all locations; unicast replication requires the deployment and maintenance of physical servers. Many companies elect to deploy software which receives a live stream and redistributes the stream in the local network. This peer-to-peer distribution strategy works well and avoids the issues which stand in the way of the multicasting and unicast replication strategies. It does rest on a client which downloads to the client desktop, which presents an obstacle in some networks. ON24 partners with Octoshape for its peer-to-peer implementation; further information is available upon request. Launch Mechanics When an event is launched on the ON24 system, the launch sequence (see below) results in a playlist being provided to the end users; this playlist provides access to the media stream. In the situation described here, where different clients need to access the media through different servers or technologies, logic has to be provided which inspects the user network location and generates a playlist specific to that situation. For example, External users get sourced from the external CDNs Internal Users if the client is from 10.2.1.0/24, select and the Dallas media server, for example. Playlists and the criteria to select them are manually built by ON24 Signal Acquisition. 4
Usual Launch Sequence 1 Lobby Page Client Audience Console Audience URL Launch Media Player Slides 2 3 4 Decide which playlist to deploy based on network location of client Registration Page (Optional, and bypassed after first access, if used) Playlist provides list of media assets to be accessed in sequence Usually, it is not possible or worth the effort to address every possible network location; there are generally a list of sites where you d like the media play attempt to fail, and the client access to be provided by a non-network constrained approach. ON24 provides events that support media streams for the AV distribution and simultaneously allow the audio to be accessed by means of conference calls (non-streaming events). It is usual to configure 80-90% or the corporation s users to access via streaming, but to support the small or remote network locations by provisioning non-streaming capability providing conference call access to the signal. Importance of Failover Mechanics The playlist controls what happens when a server can t be accessed by a particular user. Consider these two scenarios: Assume we ve got media being distributed through a set of unicast replicas to an audience in Cincinnati. 5
Dallas New York X 1 Internet Gateway and Firewall Head End Los Angeles Cincinnati Little Rock Miami Scenario 1. Cincinnati Server Working Properly In this scenario, the bandwidths to the unicast replica servers are all at 1 times the stream speed. This is the typical set up. Consider the impact if the Cincinnati media server goes down, and the playlist New York listed below Cincinnati. New York to Cinci Link is loaded at 3X Dallas New York 0 X Stream Internet Gateway and Firewall Head End Los Angeles Cincinnati Little Rock Miami Scenario 2. Cincinnati Server Out of Service It seems clear that selecting a sequence of servers in the playlist which avoids the risk of overloading a link in case of server failure is important. ON24 will assist in identifying and evaluating failover scenarios. VPN behavior presents an additional risk. Generally, the VPN infrastructure is comprised of one or more VPN servers which are supported by gateway capacity (bandwidth facing the internet-located remote users). This gateway capacity may be limited. Also, since the VPN gateways are encrypting and shipping the bits from the local network resources to the remote users, handling media streams can place demands on the devices which they are not configured to support. VPN configurations generally provide routing information to their remote clients which differentiates between traffic to be sent to the corporate network and traffic to be sent to the ISP supporting the remote user. In Cisco terms, this is referred to as split tunneling = some traffic is sent to the VPN tunnel (i.e., the corporate site) and other is split out of the tunnel and handled locally. Many corporations elect to route all traffic through the VPN tunnel, including the media server traffic for webcasts. This concentration of traffic through the VPN facilities and associated gateways presents a risk of congestion. A simple resolution is to route the media server traffic directly to external media servers. If this is unacceptable, the risk can be mitigated to some extent by locating a media server adjacent to the VPN servers. Complex failover behavior can be configured to reflect the various network scenarios in place. 6