Contents. About AirPlay 5. Preparing Your Media and Server for AirPlay 8. Opting Into or Out of AirPlay 11

Transcription

1 AirPlay Overview

2 Contents About AirPlay 5 At a Glance 5 Choose the Best Option for Media Playback 5 You Can Opt Into or Out of AirPlay for Video 6 Provide a Great AirPlay User Experience in Your App 6 Protect Your Media Using Encryption and Authentication 7 See Also 7 Preparing Your Media and Server for AirPlay 8 Preparing Your Media 8 Providing AC3 Audio and High-Definition Video 9 Configuring Your Server 9 Opting Into or Out of AirPlay 11 Enabling or Disabling AirPlay on Your Website 11 Enabling or Disabling AirPlay in Your App 12 If Your App Uses AVPlayer 12 If Your App Uses MPMoviePlayerController 13 If Your App Uses UIWebView 13 Enriching the AirPlay Experience in Your App 14 Provide an AirPlay Picker 15 Send Audio Where It Belongs 16 Provide Audio Metadata 16 Respond to Remote Events 17 Encryption and Authentication 19 Encryption for Media Downloads 19 Authentication for Encryption Key Downloads 19 Document Revision History 21 Swift 4 2

3 Figures, Tables, and Listings Preparing Your Media and Server for AirPlay 8 Table 1-1 File extensions and MIME types 10 Opting Into or Out of AirPlay 11 Listing 2-1 Allowing AirPlay to use your video 11 Listing 2-2 Disallowing AirPlay to use your video 12 Enriching the AirPlay Experience in Your App 14 Figure 3-1 System AirPlay picker 15 3

4 SwiftObjective-C 4

5 About AirPlay If you provide audio or video content from a website, or display audio or video content within your ios app, you should learn about AirPlay. AirPlay lets users stream your audio and video to high-definition displays and high-fidelity audio systems. Using AirPlay, users redirect audio and video from itunes or an ios-based device to either an Apple TV (and from there to a home theater system) or to an AirPlay-enabled sound system. AirPlay can stream media coming live from the Internet, media already stored in itunes, or media stored on an ios-based device. AirPlay can stream Internet-based media when it is playing in ios apps, in the Safari browser on ios-based devices, or in itunes on any platform. AirPlay is supported on ios 4.3 or later and itunes 10.2 or later on OS X and Windows. At a Glance Using AirPlay, users redirect audio and video to either an Apple TV or to an AirPlay-enabled sound system or remote speakers. AirPlay is user controlled. As a content provider, your primary role is to ensure that your media is AirPlay compatible and that your app or website works correctly with AirPlay. You might also want to provide high-definition video or surround-sound audio when your content is being redirected to a home theater system. As an app developer, you can enhance the user s AirPlay experience in several ways: Provide an AirPlay output device picker. Provide audio metadata that the AirPlay-enabled output device can display. Listen for and respond to remote events from the AirPlay output device, such as play/pause. Use the ios-based device s built-in display independently when a second display is active. Choose the Best Option for Media Playback You can deliver AirPlay-compatible audio and video to ios-based devices in four distinct ways: Embed audio or video in your website using the HTML5 <audio> and <video> tags. Media delivered this way works on ios-based devices through the Safari browser or in apps that use a web view. Stream audio and video directly to your app, playing the media using AV Foundation, the MPMoviePlayerController class, or the UIWebView class. 5

6 About AirPlay At a Glance Store the media locally on an ios-based device. Make your content available through itunes, either commercially or as a podcast. AirPlay is compatible with media delivered by progressive downloading or HTTP Live Streaming, but HTTP Live Streaming is preferred. To prepare your media for playback through AirPlay, make sure that it s in a supported format for example, H.264 video and AAC or MP3 audio. No special server configuration is required, apart from associating the correct MIME types with the file extensions used to send the media. To enrich the user s AirPlay experience, you can also provide alternative audio and video streams that take advantage of a home theater system s capabilities. Relevant chapter: Preparing Your Media and Server for AirPlay (page 8) You Can Opt Into or Out of AirPlay for Video AirPlay is user initiated and, generally speaking, user controlled. Audio content (other than system sounds) can always be redirected by the user to an AirPlay-enabled sound system, and so can itunes content, whether audio or video. If you provide video from your website or display it from within your app, however, you can choose to opt into AirPlay, enabling user redirection, or opt out of AirPlay, restricting your video to the device on which it is received. In ios 5.0 and later (including Safari in ios 5.0), AirPlay is enabled by default and you must explicitly opt out of AirPlay if you want to prevent users from redirecting your video to Apple TV. Relevant chapter: Opting Into or Out of AirPlay (page 11) Provide a Great AirPlay User Experience in Your App You can include an AirPlay picker in your app s user interface, allowing the user to select an AirPlay device for output without leaving your app. If your app s audio or video is being redirected to an AirPlay-enabled device that has its own controls, you can provide metadata such as a song title and album artwork to enhance the user s experience. In addition your app should respond to remote events, such as play/pause or next-track requests. 6

7 About AirPlay See Also Relevant chapter: Enriching the AirPlay Experience in Your App (page 14) Protect Your Media Using Encryption and Authentication If you deliver your audio and video using HTTP Live Streaming, there is built-in support for encryption, with automatic generation of encryption keys and initialization vectors on a periodic basis (see HTTP Live Streaming Overview for details). If you deliver the keys over HTTPS and use authentication, however, your app is responsible for the initial authentication handshake. Relevant chapter: Encryption and Authentication (page 19) See Also Multiple Display Programming Guide for ios describes how to take advantage of an external display by mirroring the content of an ios-based device or displaying different content on each device. HTTP Live Streaming Overview explains how to set up and use HTTP Live Streaming to deliver live and on-demand audio and video over HTTP from an ordinary web server. Safari HTML5 Audio and Video Guide explains how to embed audio and video in websites. AVPlayer Class Reference describes the AVPlayer class, including AirPlay-specific properties. MPMoviePlayerController Class Reference describes the MPMoviePlayerController class, including AirPlay-specific properties. UIWebView Class Reference describes the UIWebView class, including AirPlay-specific properties. 7

8 Preparing Your Media and Server for AirPlay For your media to play over AirPlay, it needs to be playable on an ios-based device, such as an iphone, ipod touch, or ipad. For your server to work with AirPlay, it needs to be configured to send audio and video over HTTP. Preparing Your Media When preparing your media for AirPlay, you should do the following: Encode your audio using mono or stereo AAC or MP3 compression. Encode your video using H.264 compression. Use Baseline Profile 3.0 if your video is intended for playback on iphone 3G or earlier. For playback on iphone 4 and later, ipod touch, ipad, and Apple TV, use Baseline Profile 3.1. Use Main Profile 3.1 if your video is intended for playback exclusively on ipad, OS X, and Apple TV. Use a QuickTime reference movie or an alternate-stream playlist to provide an alternative version of audio/video content with an AC3 surround-sound track and high-definition video to be used when your media is being played through a home theater system. If you are using HTTP Live Streaming to provide video at multiple bandwidths, see HTTP Live Streaming Overview for recommended bit rates and encoder settings. You can package your media in.mp3,.aac,.m4a,.m4v,.mp4, and.mov files. Alternatively, you can pass your media directly to the HTTP Live Streaming media segmenter as an MPEG-2 transport stream containing AAC-compressed audio and H.264-compressed video (normally the case for live broadcast). You can also create.m3u8 playlists, but these are typically generated automatically by the HTTP Live Streaming server software, which also generates.ts files from the media. 8

9 Preparing Your Media and Server for AirPlay Providing AC3 Audio and High-Definition Video Providing AC3 Audio and High-Definition Video The AirPlay-enabled sound system that is playing your audio media might be able to play AC3 surround-sound audio. You can provide an alternate stream playlist that has an AC3 selection to give your app richer sound when the output device supports it. Similarly, the AirPlay output device might be able to display high-definition video; provide a 1280 x 720 video stream alternate to give the user the richest display possible. Alternate stream playlists are a feature of HTTP Live Streaming. Alternate streams are normally specified by bit rate, but you can also specify streams by screen resolution and required codecs, so that the AC3 audio and high-definition video streams are chosen only when the output device supports them. You can specify the ability to play AC3 audio as a requirement for a stream alternate by setting the CODECS parameter in your master playlist. The string for the AC3 audio codec is "ac-3". Similarly, you can use the RESOLUTION parameter to set the minimum video resolution required for a video stream. As an example, the following variant playlist specifies a low bandwidth stream, a high bandwidth stream, and an AirPlay stream that requires a 1280 x 720 display, AC3 audio capability, and a 1.5 Mbit/s Internet connection. #EXTM3U #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH= #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH= #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH= , RESOLUTION=1280x720,CODECS="avc1.42e01e,ac-3" For more information on alternate stream playlists, see HTTP Live Streaming Overview. Configuring Your Server Your server needs to be configured for sending audio and video over HTTP. The most important step in configuring your server is to associate the correct MIME type for each supported file extension. Table 1-1 summarizes the MIME types and file extensions. 9

10 Preparing Your Media and Server for AirPlay Configuring Your Server Table 1-1 File extensions and MIME types File extension.m3u8 MIME type application/x-mpegurl or application/vnd.apple.mpegurl.ts.mov.mp3.aac.m4a.m4v,.mp4 video/mp2t video/quicktime audio/mpeg3 audio/aac audio/mpeg4 video/mpeg4 Tip: If you are using HTTP Live Streaming to deliver your media, you can increase transmission efficiency by enabling on-the-fly gzip compression of.m3u8 files from your server. 10

11 Opting Into or Out of AirPlay SwiftObjective-C Audio content (except for system sounds) and media played via itunes or the Music app are always available to AirPlay. Video playing in your app or from your website can be enabled for AirPlay or not, at your discretion. Before ios 5.0, you needed to opt into AirPlay to enable your media to be played over Apple TV. In apps compiled with the base SDK set to ios 5.0 and later, AirPlay is enabled by default; if you do not want ios-based devices to be able to play your video over Apple TV, you must disable AirPlay explicitly. In ios 5.0 and later, Safari opts in for web content by default. Enabling or Disabling AirPlay on Your Website To enable video on your website to be viewable via AirPlay, embed it in your webpage using the HTML5 <video> tag. See Safari HTML5 Audio and Video Guide for details. To explicitly opt in to AirPlay, set the x-webkit-airplay attribute for the video tag or the airplay attribute for the embed tag to "allow", as shown in Listing 2-1. Listing 2-1 Allowing AirPlay to use your video <video src="myplaylist.m3u8" height="300" width="400" x-webkit-airplay="allow" > <embed airplay= allow src="movie.mov" width=400 height=300 mime-type="video/quicktime"> </embed> </video> 11

12 Opting Into or Out of AirPlay Enabling or Disabling AirPlay in Your App To explicitly opt out of AirPlay, set x-webkit-airplay attribute for the video tag or the airplay attribute for the embed tag to "deny", as shown in Listing 2-2. Listing 2-2 Disallowing AirPlay to use your video <video src="myplaylist.m3u8" height="768" width="1024" x-webkit-airplay="deny" > </video> <!-- or --> <embed airplay="deny" src="movie.mov" width=320 height=240 mime-type="video/quicktime"> </embed> Enabling or Disabling AirPlay in Your App AirPlay is automatically enabled for video when your app uses AV Foundation, the MPMoviePlayerController class, or the UIWebView class to display video. If necessary, you can explicitly disable AirPlay for your app s video. The exact method depends on the API you use for video playback. If Your App Uses AVPlayer If your app uses the AVPlayer class to display video, you can explicitly enable AirPlay by setting the allowsairplayvideo property to YES. To disable AirPlay, set the allowsairplayvideo property to NO. To determine whether your video is playing over AirPlay, test the state of the airplayvideoactive property. 12

13 Opting Into or Out of AirPlay Enabling or Disabling AirPlay in Your App Note: The airplayvideoactive property supports key-value observing. To configure an object in your app to be notified of changes to this property, register the object as an observer. If Your App Uses MPMoviePlayerController If your app uses the MPMoviePlayerController class to display video, you can explicitly enable AirPlay by setting the allowsairplay property to YES. To disable AirPlay, set the allowsairplay property to NO. To determine whether your video is playing over AirPlay, test the state of the airplayvideoactive property. Note: You can listen for changes to the airplayvideoactive property by registering for the MPMoviePlayerIsAirPlayVideoActiveDidChangeNotification notification. If Your App Uses UIWebView If your app uses the UIWebView API to display video, you can explicitly enable AirPlay by setting the mediaplaybackallowsairplay property to YES. In addition, the content you are displaying in the web view must not disallow AirPlay. To disable AirPlay, set the mediaplaybackallowsairplay property to NO. 13

14 Enriching the AirPlay Experience in Your App There are several ways you can provide a richer experience for your users when they are using AirPlay with your app: Provide an AirPlay picker in your app. Make sure that all of your app audio goes to the AirPlay output device, and that user interface sounds stay on the host device. Provide audio metadata that can be displayed on the AirPlay output device. Respond to remote events from the AirPlay output device. Take advantage of an external display by providing separate content for the ios-based device display and the external display (to learn how to do this, see Multiple Display Programming Guide for ios ). Provide alternate data streams with high-definition video and AC3 audio (for more information, see Providing AC3 Audio and High-Definition Video (page 9)). 14

15 Enriching the AirPlay Experience in Your App Provide an AirPlay Picker Provide an AirPlay Picker The user can go to the system AirPlay picker in the ios multitasking interface to select an AirPlay output, as illustrated in Figure 3-1. If an AirPlay output device is available, an AirPlay button appears next to the volume slider; tapping the button displays a list of available devices. Figure 3-1 System AirPlay picker If your app plays media, you might want to provide an AirPlay output picker within your app so that the user doesn t have to change focus. You can add an AirPlay picker to your media playback controls by using MPVolumeView, as shown in the following code snippet: MPVolumeView *volumeview = [ [MPVolumeView alloc] init] ; [view addsubview:volumeview]; If you have a custom media controller and don t want to use the standard volume controller, you can add just the AirPlay picker using the following code snippet: 15

16 Enriching the AirPlay Experience in Your App Send Audio Where It Belongs MPVolumeView *volumeview = [ [MPVolumeView alloc] init] ; [volumeview setshowsvolumeslider:no]; [volumeview sizetofit]; [view addsubview:volumeview]; The picker is visible only when there is an AirPlay output device available. Send Audio Where It Belongs Apps typically use two types of sound app audio (such as ambient sounds, background music, and incidental noises) and system sounds (such as key clicks and alert sounds). AirPlay attempts to deliver the app audio to a remote sound system while keeping system sounds on the host, so that feedback sounds remain local to the input device. If your app uses system sound APIs for app audio, AirPlay does not redirect them to the AirPlay-enabled sound system. The result is a less-than-optimum user experience. It is important to use system sound APIs only for system sounds. For app audio, use APIs such as AVAudioPlayer. Provide Audio Metadata Your audio might be playing on a big-screen home theater system or on a sound system with an LCD display. Your app gives a better user experience if you provide metadata that can be shown on the AirPlay device s display, such as the artist name, song title, and album art. Add metadata by passing a dictionary into the setnowplayinginfo method of MPNowPlayingInfoCenter. The MPNowPlayingInfoCenter class is part of the MediaPlayer framework, but works with all playback frameworks, including MediaPlayer, AVFoundation, and AudioQueue. In addition to providing the usual song information strings, you should also pass in the playback rate, elapsed time, and media item duration. The playback device can use the duration and playback rate to create a progress bar. Update the elapsed time and playback rate whenever the playback rate changes. 16

17 Enriching the AirPlay Experience in Your App Respond to Remote Events Respond to Remote Events When AirPlay is in use, your media might be playing in another room from your host device. The AirPlay output device might have its own controls or respond to an Apple remote control. For the best user experience, your app should listen for and respond to remote events, such as play, pause, and fast-forward requests. Enabling remote events also allows your app to respond to the controls on headphones or earbuds that are plugged into the host device. Use the following code snippet to receive remote events: - (BOOL) canbecomefirstresponder {return YES;} - (void) viewdidappear: (BOOL) animated { [super viewdidappear:animated]; [ [UIApplication sharedapplication] beginreceivingremotecontrolevents]; [self becomefirstresponder]; } When your app is finished playing media, let the system know that you are no longer the receiver for remote events by using the following code snippet: - (void) viewwilldisappear: (BOOL) animated { [super viewwilldisappear:animated]; [ [UIApplication sharedapplication] endreceivingremotecontrolevents]; [self resignfirstresponder]; } Respond to events of the type UIEventTypeRemoteControl and the subtypes appropriate for your app. The following code snippet gives an example: - (void) remotecontrolreceivedwithevent: (UIEvent *) receivedevent { if (receivedevent.type == UIEventTypeRemoteControl) { switch (receivedevent.subtype) { case UIEventSubtypeRemoteControlTogglePlayPause: [self playpausetoggle: nil] break; case UIEventSubtypeRemoteControlNextTrack: [self nexttrack: nil] break; 17

18 Enriching the AirPlay Experience in Your App Respond to Remote Events... See UIEvent.h for an enumeration of event subtypes. 18

19 Encryption and Authentication If you use HTTP Live Streaming, ios provides built-in support for media encryption, and beginning in ios 5.0, it also provides support for authentication. If you serve the encryption key from a protected domain, your app is responsible for handling the initial authentication handshake with your server. Encryption for Media Downloads Using HTTP Live Streaming, you can instruct the media segmenter software to encrypt your media. The segmenter generates encryption keys and initialization vectors for those keys at specified intervals. The keys are used to encrypt all video for a given period. The client software retrieves the keys and uses them to decrypt the video. An initialization vector is required to use a key. You tell the segmenter how often to change the key and how often to change the initialization vector for the current key. You also give the segmenter a base URL to use for the keys and the initialization vectors. The segmenter includes the URLs of the keys and initialization vectors in the playlist files it generates. The keys are normally changed infrequently, and are served over HTTPS for security. The initialization vectors are normally changed frequently, but are served over HTTP for speedy delivery. The use of HTTP does not reduce security, because the initialization vectors are useless without the keys. For more information, see HTTP Live Streaming Overview. Authentication for Encryption Key Downloads To protect your encryption keys, your server should require video clients to authenticate themselves. The method of authentication varies with the app. Using AirPlay with encrypted streaming content is supported only in ios 5.0 and later. For Web-based video playing in Safari (in ios), you need to have the user authenticate manually before playing video encrypted with a key served from a secure domain. The authentication can be provided in JavaScript, but not without the risk of exposing the login information. Although manual authentication works well for pay-per-view or subscription video, it is normally undesirable for advertisement-supported video because it requires a login for viewing. 19

20 Encryption and Authentication Authentication for Encryption Key Downloads Bear in mind that if the video is playing over AirPlay, the viewer is watching on a television with no way to enter a password. In such cases, the viewer must authenticate on the device that is relaying the video via AirPlay, which might be in another room. This situation has other implications, including how long cookies or other authentication tokens should last before expiration. You should configure your server accordingly. For video playing in an app, there are three recommended means of authenticating key delivery: Serve keys from a protected HTTPS realm. Before playback begins, your app can use NSURLConnection to authenticate itself, providing credentials that are kept hidden. Use cookies over HTTPS. Your app can make a connection to an HTTPS server and authenticate the app in an app-defined way. Your server can then issue a cookie that applies to the key URLs. You should set the cookie to expire long after playback is complete. The server must then require the presence of a valid session cookie in future GET requests for the keys. For maximum reliability, if the expiration date is in the near future, the server should update the cookie s expiration date in its response to future GET requests. Specify the keys in the.m3u8 files using an app-defined URL scheme. The app should register a custom NSURLProtocol to handle requests for those URLs. The player then calls back into your app when it needs to load a key URL; your app can then obtain the key using a secure side channel and can provide it to the player. For more information, see HTTP Live Streaming Overview. 20

21 Document Revision History This table describes the changes to AirPlay Overview. Date Notes Moved information about using an external display to Multiple Display Programming Guide for ios and made minor edits New document that explains how to prepare video content, websites, and apps to work with AirPlay. 21

22 Apple Inc. Copyright 2012 Apple Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, mechanical, electronic, photocopying, recording, or otherwise, without prior written permission of Apple Inc., with the following exceptions: Any person is hereby authorized to store documentation on a single computer or device for personal use only and to print copies of documentation for personal use provided that the documentation contains Apple s copyright notice. No licenses, express or implied, are granted with respect to any of the technology described in this document. Apple retains all intellectual property rights associated with the technology described in this document. This document is intended to assist application developers to develop applications only for Apple-branded products. Apple Inc. 1 Infinite Loop Cupertino, CA Apple, the Apple logo, AirPlay, Apple TV, ipad, iphone, ipod, ipod touch, itunes, OS X, QuickTime, and Safari are trademarks of Apple Inc., registered in the U.S. and other countries. iad is a service mark of Apple Inc., registered in the U.S. and other countries. DEC is a trademark of Digital Equipment Corporation. IOS is a trademark or registered trademark of Cisco in the U.S. and other countries and is used under license. Java is a registered trademark of Oracle and/or its affiliates. APPLE MAKES NO WARRANTY OR REPRESENTATION, EITHER EXPRESS OR IMPLIED, WITH RESPECT TO THIS DOCUMENT, ITS QUALITY, ACCURACY, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. AS A RESULT, THIS DOCUMENT IS PROVIDED AS IS, AND YOU, THE READER, ARE ASSUMING THE ENTIRE RISK AS TO ITS QUALITY AND ACCURACY. IN NO EVENT WILL APPLE BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT, ERROR OR INACCURACY IN THIS DOCUMENT, even if advised of the possibility of such damages. Some jurisdictions do not allow the exclusion of implied warranties or liability, so the above exclusion may not apply to you.