Video streaming and playback

Video streaming and playback January 2006 Sony Ericsson mobile phones

Preface Purpose of this document This Developers Guideline contains descriptions of the streaming video standards supported in Sony Ericsson mobile phones. The document also mentions some important issues to take into consideration when setting up a streaming media server and preparing media for streaming to a mobile phone. Readers who will benefit from this document include: Business decision-makers Corporate buyers Support engineers Software developers IT professionals These Developers Guidelines are published by: Sony Ericsson Mobile Communications AB, SE-221 88 Lund, Sweden Phone: +46 46 19 40 00 Fax: +46 46 19 41 00 www.sonyericsson.com/ Sony Ericsson Mobile Communications AB, 2006. All rights reserved. You are hereby granted a license to download and/or print a copy of this document. Any rights not expressly granted herein are reserved Sixth edition (January 2006) Publication number: EN/LZT 108 605, R6A This document is published by Sony Ericsson Mobile Communications AB, without any warranty*. Improvements and changes to this text necessitated by typographical errors, inaccuracies of current information or improvements to programs and/or equipment, may be made by Sony Ericsson Mobile Communications AB at any time and without notice. Such changes will, however, be incorporated into new editions of this document. Printed versions are to be regarded as temporary reference copies only. *All implied warranties, including without limitation the implied warranties of merchantability or fitness for a particular purpose, are excluded. In no event shall Sony Ericsson or its licensors be liable for incidental or consequential damages of any nature, including but not limited to lost profits or commercial loss, arising out of the use of the information in this document. 2 January 2006

Sony Ericsson Developer World On www.sonyericsson.com/developer, developers will find documentation and tools such as phone White Papers, Developers Guidelines for different technologies, SDKs and relevant APIs. The website also contains discussion forums monitored by the Sony Ericsson Developer Support team, an extensive Knowledge Base, Tips & Tricks, example code and news. Sony Ericsson also offers technical support services to professional developers. For more information about these professional services, visit the Sony Ericsson Developer World website. Document conventions Products Sony Ericsson mobile phones are referred to in this document using generic names as follows: Generic names Series Z1010 K700 S700 K500 Z500 V800 K300 J300 K750 W800 Z800 K600 V600 Z520 W600 W550 Sony Ericsson mobile phones Z1010 K700i, K700c S700i, S700c, S710a K500i, K506c, K508i, K508c, F500i Z500a V800, Vodafone 802SE K300i, K300c, K300a J300i, J300c, J300a K750i, K750c, D750i W800i, W800c Z800i K600i, K608i V600i Z520i, Z520c, Z520a W600i W550i, W550c 3 January 2006

W900 W810 W900i W810i, W810c, W810a Abbreviations and acronyms 3GPP AAC AMR EDGE GPRS HTTP MPEG NAT PSS QCIF RTSP UDP UMTS URL 3rd Generation Partnership Project Advanced Audio Coding Adaptive Multirate Enhanced Data rates for GSM Evolution General Packet Radio System Hypertext Transfer Protocol Motion Picture Expert Group Network Address Translator Packet Switched Streaming Quarter Common Intermediate Format Real Time Streaming Protocol User Datagram Protocol Universal Mobile Telecommunications System Uniform Resource Locator Trademarks and acknowledgements The Bluetooth word mark and logos are owned by the Bluetooth SIG, Inc. and any use of such marks by Sony Ericsson is under license. RealAudio and RealVideo are trademarks or registered trademarks of RealNetworks, Inc. Other product and company names mentioned herein may be the trademarks of their respective owners. 4 January 2006

Document history Change history 2004-06-30 Version R1A First edition 2004-08-16 Version R1B Minor revision 2004-10-15 Version R2A Second edition. Information about K500, Z500 and V800 series added. 2004-10-26 Version R2B Minor editorial changes 2005-04-28 Version R3A Information about K300, J300, K750, W800, and Z800 series added 2005-07-15 Version R4A Information about K600, V600, Z520, and W600 series added 2005-08-08 Version R4B Information about W550 series added 2005-09-26 Version R4C Real8 streaming support in K600 and V600 series removed. 2005-11-30 Version R5A Information about W900 series added 2006-01-04 Version R6A Information about W810 series added 5 January 2006

Contents Overview...7 Applications...8 User scenarios...8 Standards, architecture and protocol...9 Streaming support...9 Supported standards...10 The MPEG-4 standard...12 MPEG-4 video compression...12 Audio compression...13 File format...13 Content recommendations...14 Server considerations...16 6 January 2006

Overview The video player in Sony Ericsson mobile phones plays video content that is locally stored or streamed. It can also play audio-only material, encapsulated in a file or stream. 7 January 2006

What is streaming? Streaming is a method for making audio, video and other multimedia available in near real-time, in different types of networks. Streaming to computers has been used for a number of years. With the introduction of faster mobile networks such as GPRS, EDGE and UMTS, streaming can also be utilized in mobile phones. The name streaming refers to the technique it is based on. Previously it was necessary to download an entire file to a computer hard disk or a mobile phone memory, and then play it. Through streaming, the user can begin to watch or hear the content of a requested media file after only a short delay. The data in the file is split into small packets that are sent in a continuous flow, a stream, to the end-user's computer or mobile phone. The user can begin viewing the contents in the first packets, while the rest of them are still being transferred. The short delay at the start is due to time it takes for the client to buffer a small amount of data. The data buffer makes it possible for the client to playback the video or sound without interruption, even if the rate of received data varies slightly. Applications Applications for streaming services can be divided into on-demand and live information delivery applications. Examples of the first category are music and video, news-on-demand applications, and ondemand instructional material. Receiving of radio and television broadcasting are examples of the second category. User scenarios Streaming of music on demand Streaming of news (video, audio) on demand Streaming of movie trailers on demand Streaming and download of video on demand Live streaming of music/video (broadcast). 8 January 2006

Standards, architecture and protocol Sony Ericsson supports the architecture, protocols and codecs of PSS (Packet Switched Streaming) within the 3GPP system, as well as supporting all ongoing standardization activities within 3GPP. Sony Ericsson works constantly to follow standards and to ensure interoperability between business solutions, while at the same time endeavouring to meet additional market requirements within this area. The relevant 3GPP specification is TS 26.234, Transparent end-to-end packet switch streaming service (PSS). The PSS includes media codecs for video, still images, bitmap graphics, text, audio, and speech. Streaming support The video player in Sony Ericsson mobile phones supports local playback and streaming of 3GPP media, mainly for Video On Demand (VOD) and live video applications. 9 January 2006

Supported standards Streaming and playback specifications for Sony Ericsson mobile phones are based on the 3GPP PSS standards. 10 January 2006

The 3GPP PSS standard The video player application supports multimedia streaming as specified in the 3GPP Packet-Switched Streaming Services specification (TS 26.234). The following codecs are supported for streaming: H.263 Baseline, profile 3, level 10 MPEG-4 Simple Visual Profile, level 0 (Not supported in K300 and J300 series) RealVideo (ver. 8) (W900 series only) AMR narrowband AMR wideband (V800, Z800, K600, V600, and W900 series only) AAC-LC Note: For the Z1010, V800, Z800, K750, W800, Z520, K600, V600, W550, W600, W900, and W810 video player some parameters of the specifications above can be exceeded. However, Sony Ericsson does not recommend higher video bit rates than 128 Kbps. Note: AAC stereo is not supported in the K300 and J300 series The following streaming session initiation options are supported in WAP applications: Plain rtsp:// URL entered in the Bookmarks or Enter address dialogs http:// URL pointing to a.sdp file In addition, streaming sessions may be started from outside the video player via file association. For example, the user may click on a link to an SDP file from inside the Web browser, or on a local.sdp file from the File Manager. In either case, the video player application will be invoked to start the streaming session. The user can play locally stored video clips by selecting a.3gp or.mp4 file in the video player application. 11 January 2006

MIME and file type support MIME type Extension Notes video/mp4.mp4 Not supported in K300 and J300 series audio/mp4.mp4 MPEG 4 Audio only, AAC or AMR audio/amr-wb.3gp AMR wideband, V800, Z800, K600 V600, and W900 series only video/3gpp.3gp audio/3gpp.3gp All 3GPP treated as video application/sdp.sdp audio/x-pn-realaudio.ra RealAudio ver. 8, K600, V600, and W900 series only video/x-pn-realvideo.rm RealVideo ver. 8, K600, V600 and W900 series only Note: Sony Ericsson mobile phones support version 4.2 of the SDP format. When SDP files are sent to the mobile phone via Bluetooth wireless technology or the infrared port, the files are saved in the Other folder, in the mobile phone s File Manager. The MPEG-4 standard MPEG-4 was developed in 1998 by the Motion Pictures Expert Group, and has been incorporated into the 3GPP specifications for mobile multimedia. The earlier standards, MPEG-1 and MPEG-2, are widely used for multimedia CD-ROMS and digital broadcast television. MPEG-4 has functionality for support of both small mobile terminals and larger stationary devices within the standard. MPEG-4 has the following advantages: Flexible range of bit rates supported, from 9.6 Kbps to 6Mbps (compared to 1.5 to 12 Mbps for MPEG-2) High error resiliency Variable frame rate, enabling optimization based on the transmission path and the overall load on the server. MPEG-4 video compression Note: MPEG-4 video decoding is not supported in the K300 and J300 series. 12 January 2006

The video compression component of the standard is called MPEG-4 Visual and covers a range of bit rates and functionalities. Profiles are used to describe functionality packages. The Simple Visual profile provides efficient and error-resilient coding of video content, and is supported by the mobile phone s video player. MPEG-4 Visual is also broken down into levels, describing such things as frame size, bit rate and buffer capacity. Level 3 is targeted at mobile devices and provides for frame sizes up to QCIF (176 x 144 pixels) at a maximum bit rate of 64 Kbps at 15 frames per second. Note: The K750, W800, Z520, K600, V600, W550, W600, W900, and W810 series support a maximum bit rate of 128 Kbps. Audio compression AAC (Advanced Audio Coding) is one of the audio compression formats defined within the MPEG-4 standard. Compared to MP3 it has more advanced features and is more efficient. The 3GPP standard uses AMR for audio coding, though this is actually outside the MPEG-4 standard. This is because AMR is highly optimized for the mobile environment, requiring as little as 4.75 Kbps bandwidth. File format The file format defined by MPEG-4 has extension MP4 or 3GP. It is applicable for both streaming and local storage/playback. MP4 uses a structured yet flexible method to describe and encapsulate multimedia material. 13 January 2006

Content recommendations The following content parameters are recommended as a baseline for testing. They should provide excellent quality for the screen size and processing power of the Sony Ericsson mobile phones. Supported frame sizes are QCIF (176x144 pixels) and SQCIF (128x96 pixels). Note: MPEG-4 video decoding is not supported in the K300 and J300 series. Streaming: 32 Kbps, Video MPEG-4 or H.263, QCIF or SQCIF, 15 fps, Audio AMR 12.2 Kbps (for the V800, Z800, K600, V600, and W900 series also Audio AMR-WB over 12.2 Kbps) Audio only: AAC, 42 Kbps (AAC stereo is not supported in K300 and J300 series) Download / local playback: 64 Kbps, Video MPEG-4 or H.263, QCIF or SQCIF, 15 fps, Audio AAC 64 Kbps mono Short clips (<30 seconds): 64 Kbps total (video with Variable Bit Rate, VBR, with or without audio) 14 January 2006

Network limitations When streaming content is sent over a wireless network, the network s capabilities might affect the practical parameter limits of the content. The K700, S700, K500, Z500, K300, J300, K750, W800, Z520, W550, W600, and W810 series are designed for GPRS class 8, 4:1 (4 download and 1 upload time-slots). Normally the Channel Coding Scheme CS-2 is used for the GPRS network. This scheme specifies a data rate of 13.4 Kbps, which with 4 time-slots, gives the maximum data speed 53.6 Kbps. If the CS-1 coding scheme is used, the maximum data speed with 4 time-slots is 4 x 9.05 = 36.2 Kbps. In addition to GPRS, the Z500a, W600i, S710a phone models, and the W810 series support EDGE 4+2, Multislot Class 10. Modulation Coding Schemes MCS1- MCS9 are supported, with typical throughput rate of 50 120 Kbps 15 January 2006

Server considerations The server for streaming should be compliant with 3GPP Packet-Switched Streaming protocols. The content should consist of 3GPP-compliant.3GP files generated by some compliant authoring tool. Issues to take into consideration when setting up and testing a server for streaming media: Streaming standards supported by the mobile phone clients, see above Content recommendations, see above Proper configuration of Firewall/NAT To minimize interoperability problems, it is strongly suggested to select a server that has been through PSS-AG testing. Information about PSS-AG interoperability testing may be found on http://www.imtc.org/act_pss.htm 16 January 2006

Configuration for firewall/nat Firewalls on both the client and server sides introduce complexities into end-to-end video streaming system deployments that are not present in a direct client-server link environment. Firewall components have evolved from simple packet filters to complex stateful inspection firewalls, and will continue to evolve in the future. It is the carrier's responsibility that any firewall/nat present in the network be properly configured to allow UDP media streams to pass correctly between the server and the client. The first required action is to be sure that any Firewall has opened the ports needed for communication between the mobile phone s video player application and the server. On the player side, the following port range must be opened in the firewall: Mobile phone video player application: UDP ports 3456 to 3459 In most cases, the video player application will use one port-pair per media stream. So for a simple audio/ video presentation, it would use all four ports; 3456-3459, assuming these are not in use by another application. On the server side, the port range to open on the firewall will depend on the particular server used and how it is configured. Port 554 is used for RTSP by default. A second required action is to be sure that any firewall or NAT is RTSP-aware and thus is transparent to the streaming session. A typical problem seen in carrier networks is that the introduction of non-rtspaware port remapping prevents the UDP packets from reaching the client. The problem is described in detail in the following IETF Internet draft: http://www.ietf.org/internet-drafts/draft-ietf-mmusic-rtsp-nat-03.txt. If the firewall or NAT is RTSP-aware, then it understands that the port information is sent in RTSP and may thus adjust the port remapping so that it is transparent to both the client and the server. 17 January 2006