CS 455/555 / Spring 2014 Intro to Networks and Communications Application Layer, Client/Server Computing and Socket Programming (II) Dr. Tamer Nadeem http://www.cs.odu.edu/~nadeem/classes/cs455-s14/
The Web & HTTP Outline } Terminology (KR 2.2.1) } HTTP protocol } message format (KR 2.23) } non-persistent and persistent connections (KR 2.2.2) } pipelining } Authentication } Cookies (KR 2.2.4) } Web caches (KR 2.2.5-2.2.6) } Security (KR 8.1-8.3) 2 CS 455/555 - Spring 2014 - Nadeem
HTTP Protocol Design Non-persistent connections } The default browser/server behavior in HTTP/1.0 is for the connection to be closed after the completion of the request } Server parses request, responds, and closes TCP connection } The Connection: keep-alive header allows for persistent connections u With non-persistent connections at least 2 RTTs are required to fetch every object» 1 RTT for TCP handshake» 1 RTT for request/response Browser Web Server 3 CS 455/555 - Spring 2014 - Nadeem
HTTP Protocol Design Persistent v. non-persistent connections } Non-persistent } HTTP/1.0 } Server parses request, responds, and closes TCP connection } At least 2 RTTs to fetch every object } Persistent } Default for HTTP/1.1 (negotiable in 1.0) } Client sends requests for multiple objects on one TCP connection } Server, parses request, responds, parses next request, responds... } Fewer RTTs 4 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent vs. Persistent Connections Performance Example } A base page with five embedded images located on the same web server } How many TCP connections to download the page with non-persistent connections? } How many round-trip times (RTTs)? } How many TCP connections to download the page with persistent connections? } How many RTTs? 5 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent vs. Persistent Connections Performance Example u A base page with five embedded images located on the same web server u How many TCP connections to download the page with non-persistent connections? } 1 TCP connection to download the base page } 1 TCP connection to download each of the 5 embedded images } Total: 6 TCP connections u How many round-trip times (RTTs)? } Each TCP connection requires a handshake: 1 RTT } Once connection is setup, it takes 1 RTT to download an object (send HTTP request, receive HTTP response) } Can only download one object per connection (non-persistent), so 2 RTTs per connection } Total: 2 RTTs * 6 connections = 12 RTTs 6 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent vs. Persistent Connections Performance Example } A base page with five embedded images located on the same web server } How many TCP connections to download the page with persistent connections? } 1 TCP connection to download the base page } Since all 5 embedded images are at the same web server as the base page, no more TCP connections are needed } Total: 1 TCP connection } How many round-trip times (RTTs)? } Each TCP connection requires a handshake: 1 RTT } Once connection is setup, it takes 1 RTT to download an object (send HTTP request, receive HTTP response) } There are 6 objects (base page + 5 images) to download } Total: 1 RTT (TCP handshake) + 6 RTTs (download all objects) = 7 RTTs 7 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent Connections Performance A transmission propagation B nodal processing queueing u Example: A 1 KByte base page with five 1.5 KByte embedded images coming from the West coast on an OC-48 link (R=2.486 Gbps)» 1 RTT for TCP handshake = 50 ms» 1 RTT for request/response = 50 ms u Page download time with non-persistent connections? u Page download time with a persistent connection? 8 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent vs. Persistent Connections Your Turn } A base page with 3 embedded images located on the same web server } RTT from client to web server is 50 ms } slowest link on the path is 2 Mbps } other links are high-speed, so transmission delay is negligible } base page is 1000 bytes } each image is 3000 bytes } What is the total time (propagation + transmission delays) needed to download the entire web page with non-persistent HTTP connections? 9 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent vs. Persistent Connections Your Turn } A base page with 3 embedded images located on the same web server } RTT from client to web server is 50 ms } slowest link on the path is 2 Mbps } other links are high-speed, so transmission delay is negligible } base page is 1000 bytes } each image is 3000 bytes } What is the total time (propagation + transmission delays) needed to download the entire web page with persistent HTTP connections? 10 CS 455/555 - Spring 2014 - Nadeem
Non-Persistent Connections Parallel connections } To improve performance a browser can issue multiple requests in parallel to a server (or servers) } Server parses request, responds, and closes TCP connection Web Server Web Server Browser 11 CS 455/555 - Spring 2014 - Nadeem
Persistent Connections Persistent connections with pipelining Persistent without pipelining: } Client issues new request only when previous response has been received } At least one RTT for each embedded object Persistent with pipelining: } Default in HTTP/1.1 } Client sends requests as soon as it encounters a embedded object } As little as one RTT for all the embedded objects 12 CS 455/555 - Spring 2014 - Nadeem
Persistent Connections Without Pipelining } Client issues new request only when previous response has been received Client HTTP request msg base HTTP response msg HTTP request msg (1st embedded object) Server } At least one RTT for each embedded object HTTP response msg (1st embedded object) HTTP request msg (2nd embedded object) HTTP response msg (2nd embedded object) Time 13 CS 455/555 - Spring 2014 - Nadeem
Persistent Connections With Pipelining } Default in HTTP/1.1 } Client sends requests as soon as it encounters an embedded object } As little as one RTT for all the embedded objects Client HTTP request msg base HTTP response msg HTTP request msg (1st HTTP embedded request object) msg (2nd embedded object) HTTP response msg (1st HTTP embedded response object) msg (2nd embedded object) Server Time 14 CS 455/555 - Spring 2014 - Nadeem
The Web & HTTP Outline } Terminology (KR 2.2.1) } HTTP protocol } message format (KR 2.23) } non-persistent and persistent connections (KR 2.2.2) } pipelining } Authentication } Cookies (KR 2.2.4) } Web caches (KR 2.2.5-2.2.6) } Security (KR 8.1-8.3) 15 CS 455/555 - Spring 2014 - Nadeem
HTTP User-Server Interaction Authentication } Problem: How to limit access to server documents? } Servers provide a means to require users to authenticate themselves } HTTP includes a header tag for user to specify name and password (on a GET request) } If no authorization presented, server refuses access, sends WWW authenticate: header line in response } Stateless: client must send authorization for each request } A stateless design } (But browser may cache credentials) Client 16 CS 455/555 - Spring 2014 - Nadeem usual HTTP request msg 401: authorization WWW authenticate: usual HTTP request msg + authorization: usual HTTP response msg usual HTTP request msg + authorization: usual HTTP response msg Server Time
HTTP User-Server Interaction Cookies } Server sends "cookie" to browser in response message Set-cookie:<value> } Browser presents cookie in later requests to same server cookie: <value> } Server matches cookie with server-stored information } Provides authentication } Client-side state maintenance (remembering user preferences, previous choices, ) Client usual HTTP request msg usual HTTP response + Set-cookie: S1 usual HTTP request msg cookie: S1 usual HTTP response msg usual HTTP request msg cookie: S1 usual HTTP response + Set-cookie: S2 Server cookiespecific action cookiespecific action 17 CS 455/555 - Spring 2014 - Nadeem
The Web & HTTP Outline } Terminology (KR 2.2.1) } HTTP protocol } message format (KR 2.23) } non-persistent and persistent connections (KR 2.2.2) } pipelining } Authentication } Cookies (KR 2.2.4) } Web caches (KR 2.2.5-2.2.6) } Security (KR 8.1-8.3) 18 CS 455/555 - Spring 2014 - Nadeem
Caching on the Web Web caches (Proxy servers) u Web caches are used to satisfy client requests without contacting the origin server } Users configure browsers to send all requests through a shared proxy server } Proxy server is a large cache of web objects u Browsers send all HTTP requests to proxy» If object in cache, proxy returns object in HTTP response» Else proxy requests object from origin server, then returns it in HTTP response to browser client client Proxy server Origin server Open research question: How does the proxy hit ratio change with the population of users sharing it? 19 CS 455/555 - Spring 2014 - Nadeem
Why do Proxy Caching? The performance implications of caching } Consider a cache that is "close" to client } E.g., on the same LAN public Internet origin servers } Nearby caches mean: } Smaller response times } Decreased traffic on egress link to institutional ISP (often the primary bottleneck) To improve Web response times should one buy a 10 Mbps access link or a proxy server? 1.5 Mbps access link 10 Mbps LAN proxy server campus network 20 CS 455/555 - Spring 2014 - Nadeem
Why Do Proxy Caching? Delay in packet-switched networks (review) A transmission propagation B nodal processing queueing u Packets experience variable delays along the path from source to destination u Four sources of delay at each hop» Queuing delay depends on the load ("traffic intensity") on the network 21 CS 455/555 - Spring 2014 - Nadeem
Why Do Proxy Caching? Traffic Intensity, or Utilization L = packet length (bits/packet) R = link speed (bps) a = average packet arrival rate (packets/second) La = average bit arrival rate (bits/second) La/R = traffic intensity = number of bits arriving per second / number of bits that can be transmitted per second What happens when La/R > 1? 22 CS 455/555 - Spring 2014 - Nadeem
Why do Proxy Caching? The performance implications of caching } Web performance without caching: } Mean object size (L) = 50 Kbits } Mean request (object) rate = 29/sec } Mean origin server access time = 1 sec Average response time =?? public Internet origin servers u Average response time is» average access time over access link + average access time over public Internet 1.5 Mbps access link 10 Mbps LAN campus network u Average access time over access link is... 23 CS 455/555 - Spring 2014 - Nadeem
Why do Proxy Caching? The performance implications of caching origin servers } Web performance without caching: } Mean object size (L) = 50 Kbits } Mean request (object) rate (a) = 29/sec } Mean origin server access time = 1 sec Average response time =?? u Traffic intensity on the access link: La = (50,000 bits/obj) (29 obj/sec) = 1,450,000 b/sec R = 1.5 Mbps La/R = 1,450,000 bits 1 sec X" 1 sec 1,500,000 bits = 0.96 public Internet 1.5 Mbps access link 10 Mbps LAN campus network 24 CS 455/555 - Spring 2014 - Nadeem
Why do Proxy Caching? The performance implications of caching origin servers } Web performance without caching: } Mean object size = 50 Kbits } Mean request (object) rate (a) = 29/sec } Mean origin server access time = 1 sec Average response time =?? u Average response time is» average access time over access link + average access time over Internet» a long time (w/traffic intensity = 0.96) + 1000 ms public Internet 1.5 Mbps access link 10 Mbps LAN campus network Rule of thumb: If traffic intensity < 60%, ignore queuing delay. If traffic intensity >= 60%, assume queuing delay is large. 25 CS 455/555 - Spring 2014 - Nadeem
Why Do Proxy Caching? Delay in packet-switched networks (review) } Understand queuing delay in terms of traffic intensity La/R } R = link transmission speed (bps) } L = packet length (bits/packet) } a = average packet arrival rate (packets/second) u If La/R ~ 0: Average queuing delay small u As La/R 1: Delays become large u If La/R > 1: Work arrives faster than it can be serviced» Average delay goes to infinity! 26 CS 455/555 - Spring 2014 - Nadeem Average Queuing Delay! 1! La/R
Why Do Proxy Caching? The performance implications of caching origin servers } Web performance without caching: } Mean object size = 50 Kbits } Mean request (object) rate = 29/sec } Mean origin server access time = 1 sec } Average response time =?? } What is traffic intensity on LAN? public Internet 1.5 Mbps access link 10 Mbps LAN campus network 27 CS 455/555 - Spring 2014 - Nadeem
Why Do Proxy Caching? The performance implications of caching origin servers } Upgrade the access link to 10 Mbps } Response time =?? } Queuing is negligible hence response time ~ 1 s } Leave access link at 1.5 Mbps but add a proxy cache with 40% hit ratio and 10 ms access time } Cache access time includes time to send request to proxy, time to search cache, and time to send response back to client } Response time =?? } Traffic intensity on access link = Response time = 0.4 x 10 ms + 0.6 x 1,010 ms = 610 ms 0.6 x 0.97 = 0.58 public Internet 1.5 Mbps access link 10 Mbps LAN campus network A proxy cache lowers response time, lowers access link utilization, and saves money! 28 CS 455/555 - Spring 2014 - Nadeem
Cache Performance for HTTP Requests What determines the hit ratio? } Cache size } Locality of references } How often the same web object is requested } How long objects remain "fresh" (unchanged) } Object references that can't be cached at all } Dynamically generated content } Protected content } Content purchased for each use } Content that must always be up-to-date } Advertisements ("pay-per-click" issues) 29 CS 455/555 - Spring 2014 - Nadeem
Why Do Proxy Caching? The case for proxy caching } Lower latency for user's web requests } Reduced traffic at all network levels } Reduced load on servers } Some level of fault tolerance (network, servers) } Reduced costs to ISPs, content providers, etc., as web usage continues to grow exponentially } More rapid distribution of content 1.5 Mbps access link 10 Mbps LAN proxy server public Internet origin servers campus network 30 CS 455/555 - Spring 2014 - Nadeem
HTTP User-Server Interaction The conditional GET } Goal: don t send object if cache has up-to-date cached version } no object transmission delay } lower link utilization } Proxy specifies the date of cached copy in HTTP request If-modified-since:<date> Proxy HTTP request If-modified-since: <date> HTTP response HTTP/1.0 304 Not Modified Server object not modified } Server's response contains the object only if it has been changed since the cached date } Otherwise server returns: HTTP/1.0 304 Not Modified HTTP request If-modified-since: <date> HTTP response HTTP/1.0 200 OK <data> object modified 31 CS 455/555 - Spring 2014 - Nadeem
HTTP Message Format Telnet example Connect to HTTP server port Telnet output Type GET command plus blank line HTTP response status line HTTP response headers plus blank line Object content Telnet output % telnet www.cs.odu.edu 80 Trying 128.82.4.2... Connected to xenon.cs.odu.edu. Escape character is '^]'. GET /~nadeem/files/foo.txt HTTP/1.0 HTTP/1.1 200 OK Date: Wed, 30 Jan 2013 01:44:23 GMT Server: Apache/2.2.17 (Unix) PHP/5.3.5... Last-Modified: Thu, 19 May 2011 19:23:43 GMT ETag: "5c-4a3a5f178cdd0" Accept-Ranges: bytes Content-Length: 92 Connection: close Content-Type: text/plain This test file is stored in the UNIX file system at /home/nadeem/public_html/files/foo.txt Connection closed by foreign host. 32 CS 455/555 - Spring 2014 - Nadeem
The Web & HTTP Outline } Terminology (KR 2.2.1) } HTTP protocol } message format (KR 2.23) } non-persistent and persistent connections (KR 2.2.2) } pipelining } Authentication } Cookies (KR 2.2.4) } Web caches (KR 2.2.5-2.2.6) } Security (KR 8.1-8.3) 33 CS 455/555 - Spring 2014 - Nadeem
Security HTTPS } HTTP over Secure Socket Layer } Secure version of HTTP } Encrypts the session data } Using either the SSL (Secure Socket Layer) protocol or the TLS (Transport Layer Security) protocol } SSL and TLS work above TCP but below application protocols (HTTP, SMTP, etc.) } Transferred using HTTP, encrypted } with default TCP/IP port 443 } For Web pages, the URL begins with https:// } Provides server authentication and encrypted communication 34 CS 455/555 - Spring 2014 - Nadeem
Security Authentication vs. Encryption } Authentication } to confirm the sender is who they say they are } using a digital certificate issued by a trusted third party } Encryption } to prevent others from reading the message } using a cipher (encryption-decryption algorithm) } with symmetric or asymmetric (related public and private) keys 35 CS 455/555 - Spring 2014 - Nadeem
Security Encryption and Decryption } Symmetric cryptography } Same key for encryption and decryption } would be efficient as long as the key is pre-agreed and secure } Not suitable for the Web } Asymmetric-key cryptography (also called Public-key cryptography) } Using a pair of related public and private keys. Encryption and decryption are asymmetric. } Used in HTTPS 36 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography The language of cryptography "crypto" - secret "graphy" - writing symmetric key crypto: sender, receiver keys identical public-key crypto: encryption key public, decryption key secret 37 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography Symmetric key cryptography Substitution cipher: substituting one thing for another } Caesar cipher: substitute one letter for another by shifting alphabet k letters plaintext: abcdefghijklmnopqrstuvwxyz ciphertext: lmnopqrstuvwxyzabcdefghijk E.g.: Plaintext: bob. i love you. alice ciphertext: mzm. t wzgp jzf. lwtnp 38 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography Symmetric key cryptography Substitution cipher: substituting one thing for another } monoalphabetic cipher: substitute one letter for another plaintext: abcdefghijklmnopqrstuvwxyz ciphertext: mnbvcxzasdfghjklpoiuytrewq E.g.: Plaintext: bob. i love you. alice ciphertext: nkn. s gktc wky. mgsbc 39 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography Symmetric key cryptography K A-B K A-B plaintext message, m encryption algorithm ciphertext K A-B (m) decryption algorithm plaintext m = K ( K (m)) A-B A-B Symmetric key crypto: Bob and Alice know same key, K A-B E.g.: Key is knowing substitution pattern in monoalphabetic substitution cipher 40 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography Public Key Cryptography } Symmetric key cryptography } requires sender and receiver to know shared secret key } Public key cryptography } sender and receiver do not share secret key } public encryption key known to all } private decryption key known only to receiver 41 CS 455/555 - Spring 2014 - Nadeem
Principles of Cryptography Public Key Cryptography 42 CS 455/555 - Spring 2014 - Nadeem
Public Key Cryptography Algorithms Requirements Need K + B (m) and K- B (m) such that 1. K - B (K+ B (m)) = m 2. Given public key K + B, it should be impossible to compute private key K - B RSA: Rivest, Shamir, Adelson algorithm 43 CS 455/555 - Spring 2014 - Nadeem
RSA Another Important Property The following property will be very useful later: - + K (K (m)) = m B B = + - B B K (K (m)) use public key first, followed by private key use private key first, followed by public key Result is the same! 44 CS 455/555 - Spring 2014 - Nadeem
Public Key Cryptography How To Bind a Public Key to Its User? } Public-key infrastructure (PKI) } Provide a trusted third party (Certifying Authority) to use identity certificates to bind public keys to users } PKI may refer to the software that manages certificates in a large-scale setting } A certificate may be revoked } check the certificate revocation list (CRL) 45 CS 455/555 - Spring 2014 - Nadeem
Public Key Cryptography How to Generate a Certificate? } Generate a public-private key pair from a large random number } Keep the private key, send the public key and identifying information to a Certificate Authority (CA) } Pay a fee to the CA } The CA verifies the identity } The CA creates a certificate (including all ID information and the URL of the web site) } The CA signs the certificate (encoded with its own private key) and sends the signed certificate to you 46 CS 455/555 - Spring 2014 - Nadeem
Key Distribution and Certification Certification Authorities } Binds public key to particular entity, E. private } E registers its public key with CA. } E provides "proof of identity" to CA. } CA creates certificate binding E to its public key. } Certificate containing E's public key digitally signed by CA CA says "this is E's public key" Certification Authority (CA) 47 CS 455/555 - Spring 2014 - Nadeem
Key Distribution and Certification Certification Authorities When Alice wants Bob's public key: } get Bob's certificate (Bob or elsewhere). } apply CA's public key to Bob's certificate, get Bob's public key } web browsers and other apps are pre-loaded with public keys for various CAs digital signature (decrypt) Bob's public key, K B + K CA - (K B+ ) 48 CS 455/555 - Spring 2014 - Nadeem
HTTPS Putting Everything Together } Server has obtained a certificate validating that it is who it says it is. } certificate contains: CA - (S +, ID information) } Server sends: m, S - (m), CA - (S +, ID information) } Client uses CA's public key to open certificate, then gets server's public key. } Client uses server's public key S + to open encrypted message S - (m) } Client compares message to unencrypted message m } Once authenticated, client can encrypt its messages with the server's public key, S +. 49 CS 455/555 - Spring 2014 - Nadeem
The Web & HTTP Outline } Terminology (KR 2.2.1) } HTTP protocol } message format (KR 2.23) } non-persistent and persistent connections (KR 2.2.2) } pipelining } Authentication } Cookies (KR 2.2.4) } Web caches (KR 2.2.5-2.2.6) } Security (KR 8.1-8.3) 50 CS 455/555 - Spring 2014 - Nadeem
Application-Layer Protocols Outline } The architecture of distributed systems } Client/Server computing } P2P computing } Hybrid (Client/Server and P2P) systems } The programming model used in constructing distributed systems } Socket programming } Example client/server systems and their application-layer protocols } The World-Wide Web (HTTP) } Reliable file transfer (FTP) } E-mail (SMTP & POP) } Internet Domain Name System (DNS) 51 CS 455/555 - Spring 2014 - Nadeem application application transport network link physical local ISP company network regional ISP
Application-Layer Protocols FTP: The Internet file transfer protocol (RFC 959) User at host FTP! user! interface! FTP! client! File Transfer FTP! server! Local file system Remote file system } FTP is used to transfer a file to/from remote host } FTP uses a client/server model } Client: side that initiates transfer (either to/from remote) } Server: remote host } FTP server listens for connections on port 21 52 CS 455/555 - Spring 2014 - Nadeem
FTP Protocol Design Control and data sockets FTP client TCP control connection" (port 21)" TCP data connection" FTP server } FTP client contacts FTP server on port 21, using TCP as the transport protocol } Two parallel TCP connections opened: } A single control connection for exchanging commands, responses ("out of band control") } n data connections for transferring file data to/from server } FTP server maintains "state" } Remembers current directory, earlier authentication 53 CS 455/555 - Spring 2014 - Nadeem
FTP Protocol Design Active vs. Passive FTP } Traditional use of FTP is "active" } client contacts server on control port } server uses data port to contact client and transfer data } Use of firewalls has made "passive" more common } client contacts server on control port } client and server agree on a new server port for data } client contacts server on data port for data transfer 54 CS 455/555 - Spring 2014 - Nadeem
FTP Protocol Design Active FTP } The client connects from a random unprivileged port (N > 1023) to the FTP server's command port, port 21 } The client starts listening to port N+1 and sends the FTP command PORT N+1 to the FTP server } The server will then connect back to the client's specified data port from its local data port, which is port 20. 55 CS 455/555 - Spring 2014 - Nadeem
FTP Protocol Design Passive FTP } The client initiates both connections to the server } When opening an FTP connection, the client opens two random unprivileged ports locally (N > 1023 and N+1). } The first port is used to contact the server on port 21 } The client then issues the PASV command. } The server then opens a random unprivileged port (P > 1023) and sends the PORT P command back to the client. } The client then initiates the connection from port N+1 to port P on the server to transfer data. 56 CS 455/555 - Spring 2014 - Nadeem
FTP Protocol Design FTP commands, responses } Sample commands: } Sent as ASCII text on control socket USER <username> PASS <password> LIST Return list of file in current directory RETR <filename> Retrieves (gets) file STOR <filename> Stores (puts) file onto remote host } Sample return codes } Status code and phrase (as in HTTP) 331 Username OK, password required 125 Data connection already open; transfer starting 425 Can't open data connection 452 Error writing file 57 CS 455/555 - Spring 2014 - Nadeem
Application-Layer Protocols Outline } The architecture of distributed systems } Client/Server computing } P2P and Hybrid computing } The programming model used in constructing distributed systems } Socket programming } Example client/server systems and their application-layer protocols } The World-Wide Web (HTTP) } Reliable file transfer (FTP) } E-mail (SMTP & POP) } Internet Domain Name System (DNS) 58 CS 455/555 - Spring 2014 - Nadeem application application transport network link physical local ISP company network regional ISP
Application-Layer Protocols Electronic mail } Major components: } User agents } Mail servers } Mailboxes } Protocols: } Simple Mail Transfer Protocol (SMTP) delivers mail to servers } From clients to local mail server } Inter-mail server delivery Post Office Protocol (POP) for user access to delivered email mail server SMTP" mail server SMTP" user agent SMTP" SMTP" SMTP" mail server user agent outgoing message queue POP" SMTP" POP" user mailbox user agent SMTP" user agent 59 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail Mail servers } Servers maintain: } A message queue of outgoing email messages } A mailbox containing incoming messages for each user outgoing message queue user mailbox POP" user agent } SMTP protocol is run between mail agents and servers to send email messages } Client the sending mail server or agent } Server the receiving mail server mail server SMTP" SMTP" mail server user agent SMTP" SMTP" POP" user agent 60 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail The email delivery process Protocol flow Email flow user agent mail server mail server SMTP SMTP POP IMAP user agent Mail Sender Local Mail Server Remote Mail Server } User's mail agent contacts its local mail server u Local mail server contacts the destination mail server(s) Mail Recipient u Destination mail server places the mail into the appropriate user's mailbox u User retrieves mail via a mail access protocol 61 CS 455/555 - Spring 2014 - Nadeem
The Email Delivery Process SMTP [RFC 2821] } SMTP uses a TCP socket on port 25 to transfer email reliably from client to server } Email is temporarily stored on the local server and eventually transferred directly to receiving server } Intermediate relay is a special case } Three phases of the protocol: } Handshaking ("greeting") } Transfer of messages } Closure } Client/server interaction follows a command/response paradigm } Commands are plain ASCII text } Responses are a status code and an optional phrase } Command and response lines terminated with CRLF 62 CS 455/555 - Spring 2014 - Nadeem
The Email Delivery Process Sample SMTP interaction u SMTP client establishes TCP connection to server hamburger.edu at port 25» (SMTP is non-standard in that the server "talks first") Server: 220 hamburger.edu Client: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <alice@crepes.fr> S: 250 alice@crepes.fr... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: Do you like ketchup? C: How about pickles? Line with single '.' is C:. the message delimiter S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection 63 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail Mail message format (RFC 2822) } Header lines, e.g., } From: } To: } Subject: these are different from SMTP commands! header body blank line } Body } The "message", ASCII characters only 64 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail Mail message format example Server: 220 hamburger.edu Client: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM: <alice@crepes.fr> S: 250 alice@crepes.fr... Sender ok C: RCPT TO: <bob@hamburger.edu> S: 250 bob@hamburger.edu... Recipient ok C: DATA S: 354 Enter mail, end with "." on a line by itself C: From: alice@crepes.fr C: To: bob@hamburger.edu C: Subject: food C: C: Do you like ketchup? C: How about pickles? C:. S: 250 Message accepted for delivery C: QUIT S: 221 hamburger.edu closing connection 65 CS 455/555 - Spring 2014 - Nadeem
Mail Message Format MIME Multimedia mail extensions (RFC 2045, 2056) } SMTP requires all data to be 7-bit ASCII characters } All non-ascii data must be encoded as ASCII strings } Additional lines in the message header declare MIME content type MIME version Method used to encode data Multimedia data type, subtype, parameter declaration ASCII encoded data From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data.........base64 encoded data 66 CS 455/555 - Spring 2014 - Nadeem
MIME Multimedia Mail Extensions MIME types Content-Type: <type>/<subtype>[; <parameters>] Content-Type: text/plain; charset=us-ascii (optional, type-dependent) Content-Type: application/pdf; filename=foo.pdf } Text } Subtypes: plain, html } Image } Subtypes: jpeg, gif } Audio } Subtypes: basic (8-bit µ-law encoded), 32kadpcm (32 kbps ADPCM) } Video } Subtypes: mpeg, quicktime } Application } Other data that must be processed by reader before it is "viewable" } Subtypes: msword, octet-stream 67 CS 455/555 - Spring 2014 - Nadeem
MIME Types Multipart Type MIME version" Indicates multiple, mixed types" 1 part " 1 part " From: alice@crepes.fr To: bob@hamburger.edu Subject: Picture of yummy crepe. MIME-Version: 1.0 Content-Type: multipart/mixed; boundary=98766789 --98766789 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain Dear Bob, Please find a picture of a crepe. --98766789 Content-Transfer-Encoding: base64 Content-Type: image/jpeg base64 encoded data.........base64 encoded data --98766789-- Header/Body separators" Defines parts separator " Encoding & type" specified for each part" 68 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail SMTP notes } SMTP uses persistent connections } SMTP is a "push" protocol } SMTP requires that message (header & body) be in 7-bit ASCII } All binary objects must be ASCII encoded } Certain character strings are not permitted in a message } Message has to be encoded if these strings are used } With MIME extensions, multiple objects can be sent in a single multipart message } SMTP server uses CRLF.CRLF to determine end of message 69 CS 455/555 - Spring 2014 - Nadeem
Application-Layer Protocols HTTP v. SMTP } HTTP is a "pull" protocol, SMTP is a "push" protocol } Persistence: } SMTP uses persistent connections } HTTP may or may not } Message/object content: } Both have ASCII command/response interaction and status codes } SMTP requires that messages be in 7-bit ASCII multiple objects message sent in a multipart message } HTTP can transfer anything each object is encapsulated in its own response headers 70 CS 455/555 - Spring 2014 - Nadeem
Electronic Mail Mail access protocols user agent SMTP mail server SMTP mail server POP IMAP user agent HTTP Mail Sender" Local Mail Server" Remote Mail Server" Mail Recipient" } SMTP: Delivery to receiver's server } Mail access protocol: Retrieval from server by a user } POP [RFC 1939] Authorization and download } IMAP (Internet Mail Access Protocol) [RFC 2060] } More features (more complex) } Manipulation of stored messages on server } HTTP: Gmail, Yahoo! Mail, etc. 71 CS 455/555 - Spring 2014 - Nadeem
Mail Access Protocols The POP-3 protocol } Authorization phase } Client commands: } user: declare username } pass: password } Server responses } +OK } -ERR } Transaction phase } list: list message numbers and sizes } retr: retrieve message by number } dele: delete } quit 72 CS 455/555 - Spring 2014 - Nadeem S: +OK POP3 server ready C: user alice S: +OK C: pass hungry S: +OK user logged on C: list S: 1 498 S: 2 912 S:. C: retr 1 S: <message 1 contents> S:. C: dele 1 C: retr 2 S: <message 2 contents> S:. C: dele 2 C: quit S: +OK POP server sign off
Mail Access Protocols POP-3 and IMAP More about POP3 } Previous example uses "download and delete" mode. } Bob cannot re-read e-mail if he changes client } "Download-and-keep": copies of messages on different clients } POP3 is stateless across sessions IMAP } Keep all messages in one place: the server } Allows user to organize messages in folders } IMAP keeps user state across sessions: } names of folders and mappings between message IDs and folder name 73 CS 455/555 - Spring 2014 - Nadeem
Mail Access Protocols Web-Based Email } User agent is a web browser } User communicates with remote mailbox via HTTP } When recipient wants to access a message } email message is sent from mail server to browser using HTTP } When sender wants to send a message } email is sent from browser to mail server using HTTP } mail server still uses SMTP to communicate with other mail servers 74 CS 455/555 - Spring 2014 - Nadeem
Application-Layer Protocols Outline } The architecture of distributed systems } Client/Server computing } P2P and Hybrid computing } The programming model used in constructing distributed systems } Socket programming } Example client/server systems and their application-layer protocols } The World-Wide Web (HTTP) } Reliable file transfer (FTP) } E-mail (SMTP & POP) } Internet Domain Name System (DNS) 75 CS 455/555 - Spring 2014 - Nadeem application application transport network link physical local ISP company network regional ISP
Application-Layer Protocols The Domain Name System (DNS) } Computers (hosts, routers) connected to the Internet have two forms of names: } IP address a 32 bit identifier used for addressing hosts and routing data to them } Hostname an ASCII string used by applications } The DNS is an Internet-wide service that provides mappings between IP addresses and hostnames } The DNS is a distributed database implemented in a hierarchy of name servers } The DNS is also an application-layer protocol } Hosts and routers use name servers to resolve names (address/name translation) } Name resolution is an essential Internet function implemented as applicationlayer protocol 76 CS 455/555 - Spring 2014 - Nadeem
The Domain Name System Name Hierarchy in DNS com" edu" net" org" npr" gov" yahoo" amazon" odu" cornell" sprint" bellsouth" srv" ads" } hostname = "dot" separated concatenation of domain names along path toward the root } odu.edu } cs.odu.edu cs" ece" cs"» atria.cs.odu.edu 77 CS 455/555 - Spring 2014 - Nadeem
Name Hierarchy in the DNS Top level domains com" edu" net" org" gov" } Generic domains: }.com,.org,.net,.edu,.gov,.mil,.int }.biz,.info,.name,.pro } Special sponsored names }.aero,.coop,.museum } Country code domains }.uk,.de,.jp,.us, etc. 78 CS 455/555 - Spring 2014 - Nadeem
DNS Overview } } } Applications need IP address to open connection Use DNS to find the IP address given a hostname Steps: 1. Application invokes DNS (gethostbyname() in C) 2. DNS application in host sends query into network (UDP port 53) 3. DNS application in host receives reply with IP address (after some delay) 4. IP address passed up to the application DNS is a black box as far as the application is concerned. 79 CS 455/555 - Spring 2014 - Nadeem
The Domain Name System Designing a distributed service } Why not centralize the DNS } A server process on a big, well connected supercomputer? } Centralized systems do not scale! } Poor reliability: centralized = single point of failure } Poor performance: centralized = "remote access" for most users } Difficult to manage: centralized = all traffic goes to one location, a large staff has to be present to handle registrations } A centralized system is not politically feasible in an international network 80 CS 455/555 - Spring 2014 - Nadeem
DNS Name Servers Distributed, Hierarchical database Root DNS Servers com DNS servers org DNS servers edu DNS servers yahoo.com DNS servers amazon.com DNS servers pbs.org DNS servers poly.edu umass.edu DNS servers DNS servers client wants IP for www.amazon.com; 1 st approx: } client queries root server to find com DNS server } client queries.com DNS server to get amazon.com DNS server } client queries amazon.com DNS server to get IP address for www.amazon.com 81 CS 455/555 - Spring 2014 - Nadeem
Designing a Distributed Service DNS Name Servers } No server has every hostname-to-ip address mapping } Authoritative name server: } Every host is registered with at least one authoritative server that stores that host's IP address and name } The authoritative name server can perform name/address translation for that host's name/address } Local authoritative name servers: } Each ISP, university, company, has a local (default) name server authoritative for its own hosts } Resolvers always query a name server local to it to resolve any host name Local authoritative server cruzan.cs.odu.edu Name resolution: Query and Reply Local host atria.cs.odu.edu What if the name is not a local host (e.g., www.yahoo.com)? 82 CS 455/555 - Spring 2014 - Nadeem
DNS Name Servers Root name servers e NASA Mt View, CA f Internet Software C. Palo Alto, CA (and 36 other locations) a Verisign, Dulles, VA c Cogent, Herndon, VA (also LA) d U Maryland College Park, MD g US DoD Vienna, VA h ARL Aberdeen, MD j Verisign, ( 21 locations) k RIPE London (also 16 other locations) i Autonomica, Stockholm (plus 28 other locations) m WIDE Tokyo (also Seoul, Paris, SF) b USC-ISI Marina del Rey, CA l ICANN Los Angeles, CA } A root name server is contacted when a local name server can't resolve a name } The root server either resolves the name or provides pointers to authoritative servers at lower level of name hierarchy } There are 13 root name servers worldwide http://root-servers.org/ } a.root-servers.org m.root-servers.org 83 CS 455/555 - Spring 2014 - Nadeem
DNS Name Servers Using a server hierarchy for resolving names } Host atria.cs.odu.edu wants to know the IP address of www.yahoo.com } atria contacts its local DNS server cruzan.cs.odu.edu } To resolve a non-local name, the local name server queries the root server } The root server responds with the TLD for.com } The local DNS server contacts the TLD server } The local DNS server contacts the authoritative server dns.yahoo.com } Results feed back to atria } atria can now use the IP address of www.yahoo.com to make a connection 84 CS 455/555 - Spring 2014 - Nadeem Root DNS server a.root-servers.net 2 TLD DNS server Authoritative DNS server dns.yahoo.com 3 4 1 8 5 6 Requesting host atria.cs.odu.edu 7 Target host www.yahoo.com Local DNS server cruzan.cs.odu.edu
DNS Name Servers Recursive vs. Iterative Queries u The DNS supports two paradigms of queries:» Recursive queries» Iterative queries u Recursive queries place the burden of name resolution (recursively) on the contacted server u In an iterated query the contacted server simply replies with the name of the server to contact» "I don't know; trying asking X" 2 Root DNS server a.root-servers.net 7 3 6 1 8 TLD DNS server 5 4 Authortative DNS server dns.cs.odu.edu Local DNS server dns.yahoo.com 85 CS 455/555 - Spring 2014 - Nadeem Requesting host estore.yahoo.com
DNS Name Servers Recursive vs. Iterative Queries } Any query can be recursive or iterative } Iterative and recursive queries can be combined } Typically, the query from the requesting host to the local DNS server is recursive and the remaining queries are iterative Root DNS server a.root-servers.net 2 TLD DNS server Authoritative DNS server dns.yahoo.com 3 4 1 8 5 6 7 Target host www.yahoo.com Local DNS server cruzan.cs.odu.edu 86 CS 455/555 - Spring 2014 - Nadeem Requesting host atria.cs.odu.edu
DNS Name Servers Caching and updating DNS entries u Every server caches all the mappings it learns» Cache entries are "soft state"» They timeout (are deleted) after some time period u DNS cache update/notify mechanisms under design by the IETF» See RFC 2136 Root DNS server a.root-servers.net 2 TLD DNS server Authoritative DNS server dns.yahoo.com 3 4 1 8 5 6 7 Target host www.yahoo.com Local DNS server cruzan.cs.odu.edu 87 CS 455/555 - Spring 2014 - Nadeem Requesting host atria.cs.odu.edu
DNS Name Servers DNS resource records } The DNS is a distributed database storing resource records (RRs) u Type = NS } } } RR format: <name, value, type, time_to_live> u Type = A» name is a hostname» value is hostname's IP address u Type = CNAME» name is an alias name for some "canonical" (the real) name» (relay1.west-coast.yahoo.com, 145.137.93.126, A)» value is canonical name name is a domain value is name of authoritative name server for this domain (yahoo.com, dns.yahoo.com, NS)» (yahoo.com, relay1.west-coast.yahoo.com, CNAME) u Type = MX» value is name of mail server host associated with name» (yahoo.com, mail.yahoo.com, MX) 88 CS 455/555 - Spring 2014 - Nadeem
DNS Inserting Records into DNS } Example: new startup "Network Utopia" } Register name networkuptopia.com at DNS registrar (e.g., Network Solutions) } provide names, IP addresses of authoritative name server (primary and secondary) } registrar inserts two RRs into com TLD server: (networkutopia.com, dns1.networkutopia.com, NS) (dns1.networkutopia.com, 212.212.212.1, A) } Create on authoritative server Type A record for www.networkuptopia.com; Type MX record for networkutopia.com } How do people get IP address of your Web site? 89 CS 455/555 - Spring 2014 - Nadeem
DNS Protocol DNS query and reply messages } DNS query and reply messages both have the same message format u Messages have a fixed length message header» Identification 16 bit query/reply identifier used to match relies to queries» Flags: v Query/Reply bit v "Reply is authoritative" bit v "Recursion desired" bit v... Identification Number of questions Number of authority RRs Flags Number of answer RRs Number of additional RRs Questions (variable number) Answers (variable number of records) Authority (variable number of records) Additional Information (variable number of records) 12 bytes" variable" 90 CS 455/555 - Spring 2014 - Nadeem
DNS Protocol DNS query and reply messages } Messages have a variable-length "question & answer" body u Questions:» The name and type fields (type A or MX) for a query hotmail.com MX u Answers:» One RR for each IP address answering query u Authority:» Resource records of other authoritative servers Identification Number of questions Number of authority RRs Flags Number of answer RRs Number of additional RRs Questions (variable number) Answers (variable number of records) Authority (variable number of records) Additional Information (variable number of records) 12 bytes" variable" 91 CS 455/555 - Spring 2014 - Nadeem
DNS Resource Records dig query/reply message example atria:[~]% /usr/bin/dig www.google.com ; <<>> DiG 9.8.1-P1 <<>> www.google.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65309 ;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.google.com. IN A ;; ANSWER SECTION: www.google.com. 33 IN A 173.194.73.105 www.google.com. 33 IN A 173.194.73.147 www.google.com. 33 IN A 173.194.73.106 www.google.com. 33 IN A 173.194.73.103 www.google.com. 33 IN A 173.194.73.104 www.google.com. 33 IN A 173.194.73.99 92 CS 455/555 - Spring 2014 - Nadeem
dig /usr/bin/dig on CS Linux machines } man dig } at Unix command line } or see http://linux.die.net/man/1/dig } +norecurse } run an iterative query } @servername } ask the specified server (instead of your local authoritative server) } +trace } make iterative queries and show results each time } MX, CNAME, NS } indicate type of query (default is A) 93 CS 455/555 - Spring 2014 - Nadeem
DNS Example Empty Cache } Resolve the hostname in http://www.cnn.com QUERY:! www.cnn.com! QUERY: www.cnn.com! QUERY: www.cnn.com REPLY: com (NS) REPLY: cnn.com (NS)! com root edu net REPLY:! 207.25.71.28! Local Server QUERY: www.cnn.com! REPLY: 207.25.71.28(A)! cnn com NS A.GTLD-SERVERS.NET 192.5.6.30 cnn.com NS DNS.CNN.COM 128.2.232.1 www.cnn.com A 207.25.71.28 94 CS 455/555 - Spring 2014 - Nadeem
DNS Example Empty Cache - Steps } Client sends query www.cnn.com to local server } Local server looks in cache, cache is empty } Local server contacts root server } root server responds with NS of.com domain } Local server contacts.com NS }.com NS responds with NS of cnn.com domain } Local server contacts cnn.com NS } cnn.com NS responds with A of www.cnn.com } At every step, the information sent to the local server is stored in its cache 95 CS 455/555 - Spring 2014 - Nadeem
DNS Example Warm Cache } Resolve the hostname www2.cnn.com with a warm cache root QUERY:! www2.cnn.com! com edu net REPLY:! 66.40.26.232! Local Server QUERY: www2.cnn.com! REPLY: 66.40.26.232(A)! cnn com NS A.GTLD-SERVERS.NET 192.5.6.30 cnn.com NS DNS.CNN.COM 128.2.232.1 www.cnn.com A 207.25.71.28 www2.cnn.com A 66.40.26.232 96 CS 455/555 - Spring 2014 - Nadeem
DNS Example Warm Cache - Steps } Client sends query www2.cnn.com to local server } Local server looks in cache } doesn't find www2.cnn.com, but does find cnn.com NS } Local server contacts cnn.com NS } cnn.com NS responds with A of www2.cnn.com } At every step, the information sent to the local server is stored in its cache 97 CS 455/555 - Spring 2014 - Nadeem
The Domain Name System Summary u F gets 270,000,000+ hits per day } Other servers have comparable load u The Verisign TLD servers answer 5,000,000,000 queries per day u Clearly the DNS would collapse without: } Hierarchy } Distributed processing } Caching u If DNS fails, Internet services stop working! 98 CS 455/555 - Spring 2014 - Nadeem
The Domain Name System For More Info } DNS Specification } RFC 1034, RFC 1035 } DNS Caching } RFC 2136 } DNS Attacks } CAIDA, Nameserver DoS Attack October 2002 analysis http://www.caida.org/projects/dns/dns-root-gtld/oct02dos.xml 99 CS 455/555 - Spring 2014 - Nadeem
Application-Layer Protocols Outline } The architecture of distributed systems } } } Client/Server computing P2P computing Hybrid (Client/Server and P2P) systems } The programming model used in constructing distributed systems } Socket programming } Example client/server systems and their application-level protocols } } } } The World-Wide Web (HTTP) Reliable file transfer (FTP) E-mail (SMTP & POP) Internet Domain Name System (DNS) 100 CS 455/555 - Spring 2014 - Nadeem application transport network link physical local ISP company network regional ISP