One Server Per City: C Using TCP for Very Large SIP Servers. Kumiko Ono Henning Schulzrinne {kumiko, hgs}@cs.columbia.edu

Transcription

1 One Server Per City: C Using TCP for Very Large SIP Servers Kumiko Ono Henning Schulzrinne {kumiko, hgs}@cs.columbia.edu

2 Goal Answer the following question: How does using TCP affect the scalability and performance of a SIP server? n Impact on the number of sustainable connections n Impact of establishing/maintaining connections on data latency n Impact on request throughput 2

3 Outline Motivation Related work Measurements on Linux Measurement results 1. Number of sustainable connections 2. Setup time and transaction time 3. Sustainable request rate Suggestions 3

4 Motivation n A scalable SIP edge server to support 300k users* n Handling connections seems costly. n Our question: How does the choice of TCP affect the scalability of a SIP server? SIP proxy servers SIP edge servers (proxy + registrar) SIP user clients * Lucent s 5E-XC TM, a high capacity 5ESS, supports 250,000 users 4

5 SIP server: Proxy and registrar Comparison with HTTP server n Signaling (vs. data) bound n No File I/O except scripts or logging n No caching; DB read and write frequency are comparable n Transactions and dialogs n Stateful waiting for human responses n Transport protocols n UDP, TCP or SCTP 5

6 Related work n A scalable HTTP server n I/O system to support 10K clients [1] n Use epoll() [2] to scale instead of select() or poll() n We built on this work. n An architecture for a highly concurrent server n Staged Event-Driven Architecture [3] n A scalable SIP server using UDP n Process-pool architecture [4] 6

7 [Ref.] Comparison of system calls to wait events n Upper limit on file descriptor (fd) set size n select(): 1,024 n poll(), epoll(): user can specify n Polling/retrieving fd set n select(), poll(): the same set both in kernel and user space n Events are set corresponding to the prepared fd set. n epoll(): n Different fd set in each by separate I/F n Optimal retrieving fd set in user space depending on APL n Events are set always from the top of the retrieving fd set. 7

8 Outline Motivation Related work Measurements on Linux Measurement results 1. Number of sustainable connections 2. Setup time and transaction time 3. Sustainable request rate Suggestions 8

9 Measurement environment Server: Pentium IV, 3GHz (dual core), 4GB memory Linux ,000 /host Clients: 8 hosts Pentium IV, 3GHz, 1GB memory Redhat Linux n System configuration n Increased the number of file descriptors per shell n 1,000,000 at server n 60,000 at clients n Increased the number of file descriptors per system n 1,000,000 at server n Expanded the ephemeral port range n [10000:65535] at clients 9

10 Measurements in two steps n Using an echo server n Number of sustainable connections. n Impact of establishing/maintaining connection on the setup and transaction response time n Using a SIP server n Sustainable request rate 10

11 Measurement tools n Number of sockets/connections n /proc/net/sockstat n Memory usage n /proc/meminfo n /proc/slabinfo n /proc/net/sockstat for TCP socket buffers n free command for the system n top command for RSS and VMZ per process n CPU usage n top command n Setup and transaction times n timestamps added at the client program n tcpdump program 11

12 Outline Motivation Related work Measurements on Linux Measurement results 1. Number of sustainable connections 2. Setup time and transaction time 3. Sustainable request rate Suggestions 12

13 Echo server measurement: Number of sustainable connections for TCP memory/connections n Upper limit n 419,000 connections with 1G/3G split n 520,000 connections with 2G/2G split n Ends by out-of-memory -> The bottleneck is kernel memory for TCP sockets, not for socket buffers. 1G/3G 2G/2G split 13

14 Echo server measurement: Slab cache usage for TCP n Static allocation: 2.3 KB slab cache per TCP connection n Dynamic allocation: only 12MB under 14,800 requests/sec. rate memory/connections 2G/2G split Slab cache usage for 520k TCP connections 14

15 Summary: Number of sustainable connections n 419,000 connections w/default VM split n 2.3 KB of kernel memory/connection n Bottleneck n Kernel memory space n More physical does not help for a 32-bit kernel. Switch to a 64-bit kernel. 15

16 Outline Motivation Related work Measurements on Linux Measurement results 1. Number of sustainable connections 2. Setup time and transaction time 3. Sustainable request rate Suggestions 16

17 Echo server measurement: Setup and transaction times n Objectives: n Impact of establishing a connection n Setup delay n Additional CPU time n Impact of maintaining a huge number of connections n Memory footprint in kernel space n Setup and transaction delay? 17

18 Echo server measurement scenarios: Setup and transaction times n Test sequences n Transaction-based n Persistent w/ TCP-open n Persistent (reuse connection) n Traffic conditions n 512 byte message n Sending request rate n 2,500 requests/second n 14,800 requests/second n Server configuration n No delay option 18

19 Echo server measurement: Impact of establishing TCP connections n CPU time: n 15% more under high loads, while no difference under mid loads n Response time n Setup delay of 0.2 ms. in our environment n Similar time for Persistent TCP to that for UDP 19

20 Echo server measurement: Impact of maintaining TCP connections n Remains constant independently of the number of connections response times/connections 20

21 Summary: Impact on setup and transaction times n Impact of establishing a connection n Setup delay n 0.2 ms in our measurement n Additional CPU time n No cost at low request rate n 15% at high request rate n Impact of maintaining a huge number of connections n Memory footprint in kernel space n Setup and transaction delay n No significant impact for TCP n Persistent TCP has a similar response time to UDP. 21

22 Outline Motivation Related work Measurements on Linux Measurement results 1. Number of sustainable connections/associations 2. Setup time and transaction time 3. Sustainable request rate Suggestions 22

23 Measurements in two steps n Echo server for simplicity n Number of sustainable connections n Impact of establishing/maintaining connection on the setup and transaction response time n SIP server n Sustainable request rate n (Impact of establishing/maintaining connection on the setup and transaction response time) 23

24 SIP server measurement: The environment n SUT n SIP server: sipd n registrar and proxy n Transaction stateful n Thread-pool model n the same host as the echo server n Clients SQL database sipd n sipstone n Registration: n TCP connection lifetime n Transaction n Persistent w/open n Persistent n 8 hosts of the echo clients REGISTER

25 SIP server measurement: Sustainable req. rate for registration n The less number of messages delivered to application, the more sustainable request rate. n Better for UDP, although persistent TCP has the same number of messages with UDP response time/request rate 25

26 What is the bottleneck of sustainable request rate? n No bottleneck in CPU time and memory usage n Graceful failure by the overload control for UDP, not for TCP Success rate, CPU time and memory usage: persistent TCP Success rate, CPU time and memory usage: UDP 26

27 Software architecture of sipd: Overload control in thread-pool model n Overload detection by the number of waiting tasks for thread allocation n Sorting and favoring specific messages n Response over requests n BYE requests Incoming Requests R1-4 n Sorting messages is easier for UDP than TCP n Message-oriented protocol enables to parse only the first line. n Byte-stream protocol requires to parse Content-Length header to find the first line. Fixed number of threads 27

28 Component test: Message processing test n Longer elapsed time for reading and parsing REGISTER message using TCP than that for UDP 28

29 Suggestions n Accelerate parsing message for sorting n By reading the first-line of buffered message without determining the exact message boundary n Not 100% accurate, but works mostly at edge server n Perform overload control at the base thread in thread-pool model n No need to wait for another thread n Use persistent connections as HTTP/1.1 29

30 Conclusions n Impact of using TCP on a SIP server n Scalable well n Memory footprint n 2.3 KB/connection in kernel memory n Setup delay n Better to use persistent connections n Parsing messages n Need to accelerate for overload control 30

31 References [1] D. Kegel. The C10K problem. [2] D. Libenzi. Improving (network) I/O performance. [3] M.Welsh, D. Culler, and E. Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. In the Eighteenth Symposium on Operating Systems Principles (SOSP-18), October [4] K. Singh and H. Schulzrinne. Failover and Load Sharing in SIP Telephony. In International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), July

32 Thank you! Any questions? mailto: 32