Survey of Web Servers

Transcription

1 Outline Survey of Web Servers Presented by Pengcheng Li & Chao Li University of Rochester What s Web Server? Web Server Architecture request-based Event Driven Models Network IO Conclusion 1 2 What s Web Server? Two Indicators Web servers need to be fast and able to service many requests concurrently Processing of HTTP request Concurrency Web Server Architectures request-based vs. event driven Throughput data volume transferred per second (bits/sec) Network IO Data not ready to be read from socket stat() and open() sys call may block on disk I/O read() and write() sys call may block too 2 5 1

2 Web Server Architectures Web Server Architectures web server architectures Multi-process and multi-thread Apache web server Single process event driven Asymmetric multi process event driven Flash web server Nginx SEDA 6 7 Multiple Process - MP Each process handles one request Disadvantages Each process has its separate address space Cannot share data: separate cache Context switch overhead Multiple Threads - MT Each thread handles one request Advantages One address space: All threads share one cache Less context switch overhead OS has to support kernel threads Apache V2.0 can use threads 5 6 2

3 Status Apache - First version is published in 1995 by Apache Group. - Most popular web server since from In 2012 it was estimated to serve 54.98% of all active websites and 58.49% of top servers. Progress model - MT: multiple threads, each thread is responsible for a request. - MP: multiple processes, each process is responsible for a request. Optimization - Cache requested static contents file in user space - Cache URL processing results Event-Driven Model Event Driven Sever Architecture Epoll Principles one thread handles many events multiplex handling of many clients and their messages use select(), poll() or epoll() to multiplex socket I/O events provide a list of sockets waiting for I/O events sleeps until an event occurs on one or more sockets can provide a timeout to limit waiting time Epoll() is far more efficient than the others create a poll instance, which returns a poll file descriptor wait for input on a set of events returns the number of file descriptors ready for I/O, or zero if the timeout occurred, or -1 on error more scalable version of poll()

4 Epoll level-triggered interrupts occur whenever the file descriptor is ready for I/O 1000 bytes of data in receive buffer you call recv() and extract 500 bytes epoll_wait() will continue to indicate the fd is ready because there are still 500 bytes in the buffer edge-triggered interrupts occur whenever the file descriptor goes from being not ready to ready 1000 bytes of data in receive buffer you call recv() and extract 500 bytes epoll_wait() will not indicate the fd is ready until the receive buffer goes down to zero and then back up to some positive number 13 Differences between select/poll and epoll(1/2) Number of file descriptors select/poll default value 1024 epoll maximum number of files can be open in the machine 1GB memory, 100,000 file descriptors compared epoll with MP Although multi-process model can be chosen to increase the number of file descriptors, process creations, synchronizations among processes and context switches cost can not omitted easily. 25 Differences between select/poll and epoll(2/2) IO efficiency select/poll - lower when the number of sockets listened by kernel becoming larger. every time kernel need to scan all sockets in the listened sockets set. But in fact only a few sockets are active during the scanning period. epoll - higher scans all the active sockets registering callback function on each socket Single Process Event Driven - SPED Single thread of execution Use non-blocking system calls to perform I/O operations Used by Zeus web server

5 Single Process Event Driven - SPED SPED should be able to process many requests at a time but Non-blocking read/write don t work on disk operations Disk I/O operations still block Asymmetric Multi Process Event Driven Combination of MP and SPED Use non-blocking calls to perform network and pipe operations Use helper process to perform blocking disk I/O operations 8 9 Asymmetric Multi Process Event Driven Helper process can be a separate thread or process Master and Helper process communicate through IPC Master and Helper mmap() the same request file Helper process reads file from disk and brings into memory Helper notifies master that file is ready Avoid data transfer between processes Flash web server Rice University, 1999 AMPED architecture Optimizations Flash Web Server aggressive caching Pathname translation caching Response header caching Caching of already mapped files Bytes position alignment

6 Status Developed by a Russian man - Igor Sysoev Open in 2004 year Over 10% of the top 1,000,000 Internet web sites 25% of the top 1,000 web sites run on Nginx Groupon, LivingSocial, Playdom, Zappos, Hulu, Dropbox, Yandex, WordPress, etc. two years ago, no more than 3%. Nginx a combination of web server cache proxy load balancing reverse HTTP proxy Progress Model master process start and stop workers control configurations listen sockets worker process handle connections events multiplexing & epoll 500 bytes each connection utilize multi-core solve blocking IO of one process Nginx Wordpress.com has found Nginx to be the only one able to handle 8000 live traffic requests per second SEDA SEDA Problems with Thread-Based Concurrency - High resource usage. - Bad performance for too many threads, throughout meltdown and response time explosion. - Static threads bound. Problem with Event-Driven Concurrency - Little OS and tool support. - No performance/failure isolation between FSMs. 22 Decompose service into stages separated by queues - Each stage performs a subset of request processing. - Stages are event-driven. - Queue is introduced execution boundary for isolation and condition. Each stage contains a thread pool to drive stage execution - Stage may block if necessary. 23 6

7 Queues for Control and Composition - Queues subject to admission control policy * Block on full queue * Drop rejected request SEDA - Queues introduce explicit execution boundary * Thread is only executed within a single stage. * Performance isolation, modularity, independence load management. - Explicit event delivery supports inspection * Trace flow of events through application * Monitor queue lengths to detect bottleneck 24 SEAD Thread Pool Controller - Determine ideal concurrency for a stage. - Observes input queue length, adds threads if over threshold. - Remove idle threads from pool. SEDA 25 SEDA Batching Controller - Batching factor: number of events consumed by each thread. - Attempt to find the smallest batching factor with stable throughput. - Reduce batching factor when throughput is high, increase if low. Network IO

8 readv/writev - Concatenate output - Save overhead of many read/write. - But hard to work with sendfile - Transfer data between file descriptors. - Transfer operation is done in kernel. - Bypass the trip through user space (mostly) entirely aio Schedule I/O operations to occur some time in the future. Calling process needs to do periodically checking. Overlap I/O operation and CPU processing. Scales poorly on default implementations. Network IO AIO Squid, Nginx sendfile Varnish, Lighttpd, Nginx writev Varnish, Nginx, Apache

9 Efficient web server: Concurrency & Throughput handle requests disk/network operations Web server architecture MP/MT Apache Event driven web server architecture Flash, Nginx, SEDA Network IO aio, sendfile, readv/writev Conclusions References [1] "Flash: An Efficient and Portable Web Server" [2] "Main Memory Caching of Web Documents" [3] "Measurement, Analysis and Performance Improvement of Apache Web Server" [4] [5] [6] [7] [8] [9] [10] [11] [12] %20Disk%20IO.pdf [13] Understand on OS enable you to write efficient/correct application code Thanks! Disclaimer Parts of the slides contain Vivek et al. s report of Flash Web Server presented at the Usenix Technical Conference in June The slides are intended for the sole purpose of survey of operating systems course at the University of Rochester. All copyrighted materials belong to their original owner(s)

10 mmap mmap 28 Pros mmap - Accessing memory mapped files is faster than directly using read/write operation. - Lazy Loading. Cons - Cost in Minor Page Fault. - Reading large size file is complex