Communication Networks Introduction & Socket Programming Yuval Rochman
Administration
Staff Lecturer: Prof. Hanoch Levy hanoch AT cs tau Office hours: by appointment Teaching Assistant: Yuval Rochman yuvalroc AT gmail.com Office hours: by appointment (feel free to schedule!) HW Grader: Alon Grinshpoon comnet2015a AT gmail.com 3
Assignments(1) 2-3 practical assignments C/C++ 20% of final grade 4-5 theoretical assignments Might include guided-reading 20% of final grade 4
Assignments(2) Submission is mandatory HW- submitted in pairs Moodle forum- to ask questions on HW Final Exam 60% of final grade 5
Course website: Moodle website Main textbook: References Computer Networking: A Top-down Approach, by J. F. Kurose and K. W. Ross (from 3 rd edition) Other references: Can be seen in appendix 6
Introduction
Why communication networks? Interface between theory & practice Understanding design principles of complex system Industry-relevant knowledge Research is practical! Fundamental Basics Information security Cloud computing SDN & OpenFlow 8
What is a network? (1) 9
What is a network? (2) Formally- system connecting Network Devices Computers Servers Routers Phones 10
Network communication(1) How is network communication done? A: Using protocols! Protocol- a set of rules for data exchange Sending correspond messages Example: automated attendant Pressing the right key (message) to get the right department 11
Network communication(2) Problem: Network communication is complex! Different protocols to multiple tasks! Optimal routing Handle data corruptions Loss of information 12
Solution- layering Layering Dividing protocols into layers Each layer tailored for different scope of tasks In our course- we use TCP/IP layering 5 layers 13
Physical layer Overview of TCP/IP(1) Transmits raw bits Data link layer Deliver frames (set of bits) Between nearby connected network devices Ethernet is a family of data link protocols 14
Network layer Overview of TCP/IP(2) Delivers packets across the network I.e how to forward the packets Between far network devices IPv4 and IPv6 is a forwarding network protocol 15
Transport layer Delivers data between end hosts Overview of TCP/IP(3) Define service of transportation Two protocols TCP: reliable, connectionoriented UDP: unreliable, connectionless End terminal End terminal 16
Application layer Overview of TCP/IP(4) Used to Implement an application over the network E.g FTP (used to transfer files) HTTP(used to browsing the internet) SMTP (used to transfer emails) 17
Packet encapsulation(1) Protocol level i sends message via protocol level i-1 TCP (level 4) can use IPv4 (level 3) UDP can use IPv6 Every protocol adds its header before passing to the next protocol Data link, in addition, adds a tail 18
Packet encapsulation(2) Application A A Application Transport T h A T h A Transport Network N h T h A N h T h A Network Data link D h N h T h A D t D h N h T h A D t Data link Physical layer Glossary: A: Application message T h : Transport header N h : Network header D h : Data link header D t : Data link tail 19
Socket Programming Main reference- Beej's Guide to Network Programming http://beej.us/guide/bgnet/
Why socket (network) programming? 21
IP address/ domain name Uniquely identifies a host on the network Not really, we ll get to that later in the course An 32\128-bit number (for IPv4\IPv6) (IPv4) Represented 4 numbers in the range 0-255 E.g 132.67.192.133 Domain names 132.67.192.133 = nova.cs.tau.ac.il 22
Port(1) A 16-bit number (i.e., 0-65535) Identifies a service on the host Again, not quite, we ll get to that later, blah-blah. For instance: HTTP = 80, SMTP = 25. A socket address is a combination of IP + port 132.67.192.133 : 80 23
Port(2) The server listens on a certain port The client randomly chooses a port For instance 94.127.73.5 : 1902 132.67.192.133 : 80 Client Server 24
Relevant Headers #include <sys/socket.h> Sockets #include <netinet/in.h> Internet addresses #include <arpa/inet.h> Working with Internet addresses #include <netdb.h> Domain Name Service (DNS) #include <errno.h> Working with errno to report errors 25
Address Representation struct sockaddr { u_short sa_family; /* type of address (ipv4/ipv6) */ char sa_data[14]; /* 14 bytes of address }; General address representation used in socket programming functions 26
Address Representation IPv4 struct sockaddr_in { short sin_family; /* = AF_INET */ u_short sin_port; /* port number */ struct in_addr sin_addr; /* 32-bit address */ char sin_zero[8]; /* unused */ }; struct in_addr { } uint32_t s_addr; Except for sin_family, all contents are in network order 27
Big Endian / Little Endian Memory representation of multi-byte numbers: 288240001810 = ABCDEF12 16 Big Endian: 0xAB CD EF 12 (IBM z) Little Endian: 0x 12 EF CD AB (intel x86,arm) Hosts on the web must use same order Traditionally, Network order=big endian htonl () / ntohl() / htons() / ntohs() host to network long network to host short network to host long host to network short 28
TCP vs UDP TCP Reliable transport Connection oriented Keep state SOCK_STREAM UDP Unreliable transport connectionless stateless SOCK_DGRAM 29
Session overview(tcp) Client TCP Server socket() bind() socket() listen() connect() session start accept() send() data transfer recv() recv() data transfer send() close() terminate session close() 30
Socket creation socket() int socket(int domain, int type, int protocol); domain: PF_INET for IPv4 type: SOCK_STREAM (tcp) or SOCK_DGRAM (udp) protocol: set to 0 (default protocol) Returns new socket descriptor or -1 on error In error- errno is set. socket descriptor is used in subsequent calls Don t forget to close the socket when you re done with it Server socket() bind() listen() accept() recv() send() close() 31
Bind socket to IP and port bind() int bind(int sockfd, const struct sockaddr *my_addr, socklen_t addrlen); sockfd : socket descriptor my_addr: address to associate with the socket IP portion often set to INADDR_ANY INADDR_ANY bind to all IP address associate with machine addrlen: set to sizeof(my_addr) Returns 0 on success, or -1 on error errno is set accordingly in error. Server socket() bind() listen() accept() recv() send() close() 32
Wait for an incoming call listen() int listen(int sockfd, int backlog); sockfd : socket descriptor backlog: number of pending clients. Before starting to refuse connections. Returns 0 on success, or -1 on error Errno is set accordingly in error. Server socket() bind() listen() accept() recv() send() close() 33
Accept an incoming call accept() int accept(int sockfd, struct sockaddr *addr, socklen_t *addrlen); sockfd : socket descriptor addr: contains the address of client which connects server. addrlen: contains the sizeof() the structure returned in the addr parameter Returns the newly connected socket descriptor, or -1 on error, with errno set appropriately. Don t forget to close the returned socket when done with it Server socket() bind() listen() accept() recv() send() close() 34
Server-side example sock = socket(pf_inet, SOCK_STREAM, 0); myaddr.sin_family = AF_INET; myaddr.sin_port = htons( 80 ); myaddr.sin_addr = htonl( INADDR_ANY ); bind(sock, &myaddr, sizeof(myaddr)); listen(sock, 5); sin_size = sizeof(struct sockaddr_in); new_sock = accept(sock, (struct sockaddr*) &their_addr, &sin_size); Server socket() bind() listen() accept() recv() send() close() In real-life code, don t forget to check for errors 35
Session overview(tcp) Client TCP Server socket() bind() socket() listen() connect() session start accept() send() data transfer recv() recv() data transfer send() close() terminate session close() 36
Connect to a listening socket connect() int connect(int sockfd, const struct sockaddr *serv_addr, socklen_t addrlen); sockfd : socket descriptor serv_addr: the address you re connecting to. addrlen: set to sizeof(serv_addr) Returns 0 on success. or -1 on error (and errno is set accordingly). Most of the times, no bind() is required on the client side If bind() wasn t called local IP address and random high port are used. Client socket() connect() send() recv() close() 37
Client-side example sock = socket(pf_inet, SOCK_STREAM, 0); dest_addr.sin_family = AF_INET; dest_addr.sin_port = htons( 80 ); dest_addr.sin_addr = htonl(0x8443fc64); connect(sock, (struct sockaddr*) &dest_addr, sizeof(struct sockaddr)); In real-life, the server s IP is not hard-coded In real-life code, don t forget to check for errors Client socket() connect() send() recv() close() 38
Session overview Once session is initiated, both parties can Send and receive data Both can decide it s time to close the connection As long as the listening socket is open: it can accept new incoming clients by calling accept() Client socket() connect() send() recv() close() Server socket() bind() listen() accept() recv() send() close() accept() 39
Closing a connection close() int close(int sockfd); sockfd : socket descriptor Returns 0 on success, or -1 on error (and errno is set accordingly) After we close a socket: If the remote side calls recv(), it will return 0. If the remote side calls send(), it return -1 & errno will be set to EPIPE. In this case SIGPIPE signal is received. If errno!=epipe- you have a system error in send() 40
Session overview (UDP) Client TCP Server socket() socket() bind() sendto() data transfer recvfrom() recvfrom() data transfer sendto() close() terminate session close() 41
Sending data (TCP + UDP) TCP: ssize_t send(int socket, const void *buffer, size_t length, int flags); UDP: ssize_t sendto(int socket, const void *buffer, size_t length, int flags, const struct sockaddr *dest_addr, socklen_t dest_len); buffer: buffer of the data to send length: number of bytes to send. flags: send options. Refer to the man pages. Use 0 for no options. In unconnected sockets (UDP) you specify the destination in each sendto() 42
Partial send send() sendto() returns # of bytes actually sent or -1 on error (and errno is set accordingly). If error=-1 and errno set to EPIPE- remote side has closed its connection! The # of bytes actually sent might be less than the # you asked it to send. 43
A code considering partial sending int sendall(int s, char *buf, int *len) { int total = 0; /* how many bytes we've sent */ int bytesleft = *len; /* how many we have left to send */ int n; } while(total < *len) { n = send(s, buf+total, bytesleft, 0); if (n == -1) { break; } total += n; bytesleft -= n; } *len = total; /* return number actually sent here */ return n == -1? -1:0; /*-1 on failure, 0 on success */ 44
Receiving data (TCP + UDP) TCP: ssize_t recv(int socket, void *buffer, size_t length, int flags); UDP: ssize_t recvfrom(int socket, void *buffer, size_t length, int flags, struct sockaddr *from_addr, socklen_t * from_len); buffer : allocated space for the received data, length: buffer size ( = max data received by call ) flags: receive options. Refer to the man pages. Use 0 for no options. 45
Receiving data (TCP + UDP) recv() & recvfrom() returns # bytes received, or -1 if error occurred (and errno is set accordingly). Need to write a similiar recvall() function In TCP sockets, 0 is returned if remote host closed its connection. In UDP from_addr is the source address of the received message. from_len should point to sizeof(from_addr) 46
Translating a host name to an IP address int getaddrinfo(const char *hostname, const char *servname, const struct addrinfo *hints, struct addrinfo **res); Supports many options and thus seems complex, but basic use is simple. Refer to Beej s guide for more info & example http://beej.us/guide/bgnet/output/html/multipage/getaddrinf oman.html Don t forget to use freeaddrinfo() to release memory when you re done with getaddrinfo s result. 47
inet_pton() Other Useful Functions Convert IP addresses from human readable text (i.e 94.127.73.5 ) to integer (uint32_ type) inet_ntop() Convert IP addresses from binary to human readable text getpeername() Return address info about the remote side of the connection. Used after calling accept() (server) or connect() (client) 48
How to define a protocol Tips for defining a protocol
Protocol design Two ways to define protocols Binary protocol (as DNS) Textual protocols (as HTTP) 50
Binary protocol(example) DNS response for query www.icann.org: 91 73 81 80 00 01 00 01 00 00 00 00 03 77 77 77 05 69 63 61 6e 6e 03 6f 72 67 00 00 01 00 01 c0 0c 00 01 00 01 00 00 02 58 00 04 c0 00 20 07 Text view:.s...www.icann.org... X.... 51
Textual protocol(example) HTTP request for the page http://www.ietf.org/rfc/rfc3514.txt GET /rfc/rfc3514.txt HTTP/1.1 Host: www.ietf.org Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive 52
Binary vs Textual Binary messages are small compare to textual messages Binary Messages are difficult to read and debug. 53
Message representation(1) We transfers a stream of data send() and recv() do not necessarily match message boundaries! Can receive multiple messages together / parts of messages. The application protocol must define a way to separate messages within the stream. 54
Message representation(2) Bad: messages have always constant size hello = 0x 68 65 6C 6C 6F 00 00 00 00 Better: use a length field hello =0x 05 68 65 6C 6C 6F Another method: Define a delimiter between different messages hello + hello == 0x 68 65 6C 6C 6F 00 68 65 6C 6C 6F Must bound every message by a constant size 55
Portability In our assignments- it is not require Code is not portable 32-bit vs 64-bit computers. Different endianity machines Windows vs Linux computers Different compilers. To make your code to be portable- see appendix 56
Word of caution packing(1) Assume you have a struct represents protocol header struct ProtocolHeader { unsigned short datagramlength; unsigned short datagramtype; unsigned char flag; //... }; 57
Word of caution packing (2) Compiler may add padding Sending the struct asis is: Expensive in space Not portable Not follows application specification Output: S s size is: 24 #include <stdio.h> #include <stddef.h> struct S { short i; /* 2 bytes */ int j; /* 4 bytes */ char k; /* 1 byte */ double l; /* 8 bytes */ }; int main() { printf("s's size is: %ld\r\n\r\n", sizeof(s) ); } 58
Word of caution packing (3) Possible solution: use #pragma pack & #pragma pop Compiler portability issues Is OK with GNU,Microsoft Visual Studio, Borland Output: T s size is: 15 #include <stdio.h> #include <stddef.h> #pragma pack(push, 1) struct T { short i; /* 2 bytes */ int j; /* 4 bytes */ char k; /* 1 byte */ double l; /* 8 bytes */ }; #pragma pack(pop) int main() { printf("t's size is: %ld\r\n\r\n", sizeof(t) ); } 59
Packing Disadvantage Using packing reduces system performance Accesses to members of packed struct is slower Use packing only on structs representing protocols. 60
Handling blocking calls
Blocking function calls Many functions blocking until a certain event: accept: until client initiates session connect: until connection is (half) established recv, recvfrom: until data is received send, sendto: until data is pushed into socket s buffer What should we do in complex programs? multiple connections simultaneous sends and receives simultaneously doing non-networking processing 62
How do we handle blocking? There are several ways: Initiate multiple threads\processes Do not allow blocking by using fcntl() Call a function only when it s guaranteed not to block select(), pselect(), poll(), ppoll() In the next slides will explain select() 63
Select(1) select() receives a set of fd s (socket ids) Returns which of them are: Read-ready: recv() or accept() will not block Write-ready: send() or connect() will not block Expect-ready Not relevant to our course (rarely used in practice) 64
Select(2) int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout); nfds: highest-numbered file descriptor in any of the three sets, plus 1. readfds, writefds, exceptfds: sets of fd s to see if they re read-ready, write-ready or except-ready. Any set can be replaced with NULL Set exceptfds to be NULL timeout: timeout parameter 65
Select(3) Returns when one of the fd s becomes ready, or when the timeout expires Returns total number of ready fd s in all the sets. Returns 0 if timeout expired Returns -1 on error (and errno is set accordingly). The sets are changed to indicate which fd s are ready. 66
Working with fd_set void FD_ZERO (fd_set *set) Initializes to an empty set void FD_SET (int fd, fd_set *set) Adds fd to the set int FD_ISSET (int fd, fd_set *set) Returns non-zero value if fd is in the set, 0 otherwise void FD_CLR (int fd, fd_set *set) Removes fd from the set stdin, stdout, stderr are associated with fd s 0, 1, 2 67
struct timeval { }; select s timeout argument long tv_sec; /* seconds */ long tv_usec; /* microseconds, always less than 10^6 */ Pass (0,0) to return immediately Pass NULL pointer to wait indefinitely Until one of the fd s is ready Some OS s decrease the time elapsed Linux does 68
fd_set read_fds; select example for(;;) {/* main loop of the program */ FD_ZERO(&read_fds); /* reset fd set */ FD_SET(listening_sock, &read_fds); for(/* for each active client with fd = client_sock */) { FD_SET(client_sock, &read_fds); } fdmax = /* the highest fd in read_fds */ select(fdmax + 1, &read_fds, NULL, NULL, NULL); if (FD_ISSET(listening_sock, &read_fds)) { /* listening socket is read-ready: a new client is available. */ /* new_client_sock = accept(listening_sock, */ } for(/* for each active client with fd = client_sock */) { if (FD_ISSET(client_sock, &read_fds)) { /*client socket is read ready unread data is available */ /* nbytes = recv(client_sock, */ } } } /*END main program loop */ 69
Appendix
References-other books Computer Networks, by A. S. Tanenbaum (4th edition or later). Computer Networks: A Systems Approach, by L. L. Peterson and B. S. Davie (3rd edition or later). An Engineering Approach to Computer Networking, by S. Keshav. Unix Network Programming, by W. R. Stevens, B. Fenner and A.M. Rudoff. 72
Portability Must use uniform endianity for application messages NOTE: IP & port must used network order Even in non-portable code (why?) Must use the same data types I.e, sizeof(long) can be 4 or 8! Use int16_t, uint32_t, etc For floating point number- see beej s guide. Compile with ANSI C90 Compatibility between compilers Avoid padding Include WINSOCK liberies Socket programming API in Windows is similar to LINUX 73