CS3051 Digital Content Management Lecture: File Transfer and Media Streaming

Transcription

1 CS3051 Digital Content Management Lecture: File Transfer and Media Streaming Adrian O Riordan, Computer Science, Western Gateway Building, University College Cork, Cork, Ireland

2 File sharing File sharing is the practice of distributing or providing access to digital media, such as text, computer programs, multimedia (audio, images and video), documents or electronic books. Various methods exist:- file transfer (FTP) Web (HTTP) Web syndication (RSS) IM cloud media streaming peer-to-peer (e.g. BitTorrent) Best approach depends on requirements: type of data size of data security requirements... 2

3 File sharing in CMSes CMSes will usually use Web (HTTP) but also facilitate file sharing in multiple other ways Drupal supports upload, download and content sharing using HTTP with caching; and other methods of file sharing through APIs and modules: Various core modules for processing FileTransfer class for FTP SMS Gateway Hub API to send SMS messages Streaming CCK (Content Construction Kit) field to facilitate streaming Syndication Modules for RSS 3

4 File Transfer Enables sending and receiving of (text or binary) files over the Internet Requires an application program on the client computer and a server program on a server Well known protocol is FTP (File Transfer Protocol) note: FTP was one of first ARPANET/Internet applications; RFC FTP has official URI scheme ftp: FTP uses TCP as the transport protocol Other file transfer protocols on Internet e.g. Trivial File Transfer Protocol (uses UDP; simpler than FTP); 4

5 How FTP Works Client initiates connection user commands such as GET are translated into FTP commands such as RETR separate control and data connections two most common transfer modes are ASCII mode and binary mode Protocol Interpreter (PI) implements the FTP protocol itself (port 21), while Data Transfer Process (DTP) performs data transfer using separate TCP session (port 20) control connection open for the duration of the session server responds on the control connection with three digit status codes data connection for transfer FTP sites can requires an account name and password 5

6 Web (HTTP) HTTP protocol is based on request/response In general case (RFC 2616): Client establishes a connection with a server and sends a request to the server in the form of a request method, URI, and protocol version, followed by a MIME-like message containing request modifiers, client information, and possible body content. Server responds with a status line, including the message's protocol version and a success or error code, followed by a MIME-like message containing server information, entity meta-information, and possible body content. 6

7 How the Web Works Clicking on a hyperlink or typing a URL into a browser starts a requestresponse cycle A request-response cycle: include multiple steps since web pages often contain embedded files, such as graphics, each requiring a separate response. 7

8 Web Proxy A browser may connect directly to a Web server or instead use a Web proxy Web proxy is an intermediary that handles requests from clients, evaluates the request before connecting to the relevant server The primary purposes of a Web proxy are security, caching and possibly content filtering proxy can also anonymize clients so server would not be able to distinguish access patterns of any particular clients proxy can transform requests and responses, e.g. tailor response to particular browser proxy can act as gateway to non-http services, e.g. FTP, if browser does not support this Common techniques for content filtering include URL or DNS blacklists, MIME filtering, and keyword matching. 8

9 Caching Web caching can improve performance, server load but not all responses are usefully cacheable Basic operation: If the cache contains the referenced URL it is checked for freshness by comparing with the "Expires:" date field of the content. Fresh objects are delivered to the client as a cache hit. If the cache does not have a local copy of the URL, or the object is stale, this is a cache miss. In this case the proxy acts as an agent for the client, opens its own session to the server named in the URL, and attempts a direct transfer to the proxy. (summary of part of "Web Caching", Cisco documentation) Wide variety of architectures and configurations of caches and proxies currently being experimented with or deployed across the Web e.g. national hierarchies of proxy caches to save transoceanic bandwidth 9

10 Web Syndication Many periodically updated Web sites have facility to syndication content called feeds orange symbol signifying an RSS feed XML based feed formats allow users to get updated site content in their favorite reader software (web-based or desktop-based) Used for portals, news, blogs, podcasting (streaming audio) and community-based sites Easy to parse, publish, filter, share or aggregate content from multiple sources

11 Syndication Technologies RSS was born in 1997 Userland s "Really Simple Syndication" for blog updates, and Netscape's "RDF Site Summary" RSS is a family of XML feed formats used to publish frequently updated content file format.xml or.rss different formats: RSS 2.0, RSS 1.0, RSS 0.90, RSS 0.91 (two) there are incompatibilities between the different versions some versions support multimedia and podcasting Atom is main alternative to RSS designed to overview shortcoming; also XML based

12 Atom feed Example single entry <?xml version="1.0" encoding="utf-8"?> <feed xmlns=" <title>example Feed</title> <link href=" <updated> t18:30:02z</updated> <author> <name>john Doe</name> </author> <id>urn:uuid:60a76c80-d399-11d9-b93c e0af6</id> <entry> <title>atom-powered Robots Run Amok</title> <link href=" <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id> <updated> t18:30:02z</updated> <summary>some text.</summary> </entry> </feed>

13 Cloud-based sharing service Cloud-based file syncing and sharing services allow users to create special folders (data is generally stored on remote servers), which the service then synchronizes so that it appears to be the same folder regardless of which computer is used to view it. content could be accessible on desktop or through website or mobile app commonly used for backups and to facilitate multiple modes of access Automatic process prevents copying already identical files and thus can be faster and save much time versus a manual copy, and less error prone Popular consumer products include Dropbox and Google Drive. 13

14 Handling media files Issues to consider: Very large file sizes Some information loss may be tolerated Requirement for live broadcast? Is Digital Rights Management (DRM) required? 14

15 Media Streaming Streaming is a method of transmitting or receiving data (especially video and audio) over a computer network as a steady, continuous flow, allowing playback to proceed while subsequent data is being received. with a sufficiently fast Internet connection, you can stream live audio or video to your computer. Streaming server delivers the data over the network Client media player usually begins playing the data (such as a movie) before the entire file has been transmitted (after part is buffered). Media are generally compressed for both storage and streaming. 15

16 Basic operation 16

17 Streaming media bandwidth & storage size The storage size of media on the streaming server is calculated from the streaming bandwidth and length of the media using the following formula (for a single user and file): storage size (in GB) = length (in seconds) bit rate (in bit/s) / (8 x 10 9 ) Example: One hour of video encoded at 4 Mbit/s (sufficient for HD video of 1280 x 720 pixels) will require around (3,600 s 4,000,000 bit/s) / ( ) = 1.8 GB of storage. For 1,000 users (unicast) the bandwidth requirement is then 4 Mbit/s 1,000 = 4 Gbit/s of bandwidth Services such as Netflix, Amazon Prime, etc. can determine the speed that a video is streaming and will adjust the video quality to match that speed. 17

18 Media and Streaming Protocols Audio streams are compressed using audio codecs such as MP3, Vorbis or AAC. Video streams are compressed using video codecs such as MPEG-4 Part 2 and H.264. Encoded audio and video streams are assembled in a container bitstream such as MP4, Ogg, and Flash (FLV/F4V). The bitstream is delivered from a streaming server to a client using protocols, such as Microsoft Media Server (MMS), Real-time Streaming Protocol (RTSP), Real-time Transport Protocol (RTP) or Real Time Messaging Protocol (RTMP) or with newer technologies that use HTTP such as HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH). 18

19 HTTP Downloading vs. Streaming HTTP can be slower than dedicated streaming protocols With streaming, client doesn t need to store whole file Streaming will have a buffer (e.g. keeps video playing in the event of an interruption of the internet connection). In downloading, DRM depends on file format used Streaming server may allow provision of content is different qualities (for slow or fast connections) HTTP Progressive download has characteristics of both 19

20 Delivery Unicast protocols send a separate copy of the media stream from the server to each recipient. unicast is the norm for most Internet connections, but does not scale well Multicast protocols send a single stream from the source to a group of recipients. Content delivery network or content distribution network (CDN) is a large distributed interconnected system of servers that use geographical proximity as a criteria for delivering content. with high availability and high performance. 20