Murray Woodside, July 2002 Techniques for Deriving Performance Models from Software Designs Murray Woodside Second Part Outline ) Conceptual framework and scenarios ) Layered systems and models C) uilding models from scenarios D) utomated model-building... from Use Case Maps, UML, traces E) Modeling insights: patterns and anti-patterns F) Some examples Murray Woodside, July 2002 2 E) Modeling Insights: Patterns and nti-patterns recurring resource-use patterns to achieve overlapping and enhance concurrency logical resources and mutual exclusion shared pool of logical resources peer-to-peer communications patterns performance pathologies associated with some patterns software bottleneck
Murray Woodside, July 2002 3 Pattern: Overlapped execution and multithreading or virtual threads Single-threaded Multi-threaded can take a second... blocks for reply request while blocked on first... server is busy while... higher capacity blocked... each thread has a long service time... long service time... in model terms, a multiserver... saturated server,... many servers, one queue lightly loaded processor Murray Woodside, July 2002 4 Overlapping: Multi-threading and virtual threads Multi-threading is supported in the kernel (Mach, Solaris, Linux, NT) or by user-level thread packages Virtual threads are programmed by the user save the context of the blocked operation in a data table accept another message and deal with it if it is a reply to a previous request, get out the state of the blocked operation and resume it called an asynchronous style of programming
Murray Woodside, July 2002 5 Threading: web server needs threads Net [2] 2 Web user [.005] n TCP ck Delay Server [.0] v Srvr Disks [.0] Srvr Processor the server blocks on the disk, and for the TCP transfer to complete CPU time is a small portion of the thread service time multi-threaded, it can serve many other requests thread-per-request, fixed thread pool, adaptive pool size Murray Woodside, July 2002 6 Overlapping through early response, or second phase service client server phase phase 2, asynchronous and parallel Idea: Give a reply as early as possible Do postponeable work after the reply, as phase 2 E. G.: Database server update operation: write to log file before returning, execute final writes later. Second-phase model may (a) place this work right after the return (approx), or (b) send an asynchronous message to a clean-up process that queues it and does it later
Murray Woodside, July 2002 7 Overlapping through asynchronous RPC a b c can execute activity b while the remote call to server is proceeding t some point must wait for the result Modeling: the entry of is defined by: Join a b c This parallel activity defines a subthread that makes a request to and blocks Murray Woodside, July 2002 8 Overlapping through forwarding ta sk ta sk ta skc Forwarding interaction typical of a service dispatcher a pipeline overlapped service: a second request can enter task as soon as it has forwarded to taskc is synchronous, is not message-handling pipeline, call setups
Murray Woodside, July 2002 9 Forwarding in COR lternative interaction patterns provided by an OR:. OR acts as RPC intermediary pp OR 2. OR forwards to method pp OR method method2 3. OR provides a handle (acts as name server) method method pp OR method2 method2 Murray Woodside, July 2002 0 Forwarding in a web-telephony system Support automated call answering by web pages speech playout is from a file encoded in a web page user enters tones; logic of decoding is also in the web page VoiceXML standard (vxmlforum.org) Call Control Web Page Interpreter Remote Web Server Web Page Cache Playout Device
Murray Woodside, July 2002 Pattern of logical resources: critical sections Tasks and must enter a critical section (call it CS) for some work... this shows the call to enter CS but it doesn t express the resource context effect of CS P CS Manager P not a suitable model So: Separate out the computation within CS into Shadow Tasks CS and CS to direct the call from to CS, make CS a pseudo-task with two pseudo-entries P CS CS CS P Murray Woodside, July 2002 2...using the critical section pattern the shadow task CS is really part of always blocked when CS runs, so separating them does not produce false concurrency it does place CS in a separate resource context... this is a general modeling pattern for separating resource contexts both and CS can make any calls to other tasks as servers SERV P CS CS CS P
Murray Woodside, July 2002 3 Logical resources: buffers a set of pplication tasks share a pool of buffers, managed by ufmgr a task needing a buffer queues until it is provided while holding the buffer, the pplications execute operations called pp uf... the same for all of them... or different pplications pplications N applications pp pp2... pplications ufmgr threads ufentry ufentry2... ufmgr pplications pp uf P pp uf pp2 uf... P2 Murray Woodside, July 2002 4 Models with buffered communications Source sends a message to a finite buffer pool (space for messages) if pool is full, Source blocks until there is space uffers replies at once to source, then sends in phase 2 to Destination if Destination needs to hold the buffer, it does its work in phase if it does not, it replies at once to uffers and does its work in phase 2. Source pplications uffers (0,) Destination May be several tasks or threads threads
Murray Woodside, July 2002 5 Pattern: Peer-to-peer interaction Peers are symmetrical they make blocking requests to each other potential exists for cyclic interactions to give layered behaviour: each peer has a Main part, and a Responder part that responds to requests from others simplest case: the Responder part does not make requests to other sites more complex, with cycles: a mechanism is needed to handle possible deadlock due to cycles. Murray Woodside, July 2002 6 Example of peer-to-peer interaction: Distributed database (Sheikh) transform a peer-to-peer relationship to reveal layered behaviour Original view Layered view User User User User pplication pplication DataManager DataManager DataManager DataManager DataStore DataStore DataStore DataStore
Murray Woodside, July 2002 7 Communications patterns overhead execution at the sender and the receiver, to make the messages and execute the protocol protocol delays for flow control network latency as a representation network bandwidth limitation Murray Woodside, July 2002 8 Communications Patterns (): Overhead tasks client send-request server send-reply send-request executes the protocol overhead at the sender, in both directions send-reply executes the overhead at the server, before the logical reply gets back to the client send-reply execution is lumped together and possibly misplaced in sequence, but the total work to provide the reply is right
Communications Patterns (2): overhead and latency client send-request Net delay INF server send-reply Murray Woodside, July 2002 9 net delay is included in the send, and again in the reply one latency each way Net delay INF Murray Woodside, July 2002 20 Communications Patterns (3):... add a bandwidth limitation to admit messages to the network client send-request Net delay INF Rate Limiter server send-reply Net delay INF Rate Limiter RL has zero first phase, second phase is T = /rate next message is admitted only after second phase is over only shown for send if the limitation is shared between the directions, the RL task may be shared too.
admission admission control control Net delay admission admission control control Murray Woodside, July 2002 2 Communications Patterns (4):... add layered flow control W threads for window tokens client INF send-request n packets W threads server send-reply m packets all packets must complete their admission and latency before logical delivery of the request to the server INF Net delay Murray Woodside, July 2002 22 nti-pattern: software bottleneck potential bottleneck pattern wherever there are multiple clients and multiple servers software bottleneck must be observed via utilizations (saturated at S and above, not saturated below) S is saturated because it is blocked on other servers cure is multiplicity (e.g. multiple threads, partitioned subsystems), reducing the blocking times multiple requesters Processor Server S multiple lower servers unsaturated (saturated) Hourglass pattern of tasks
Murray Woodside, July 2002 23 Examples of software bottlenecks in general, due to serialization at the bottleneck single threaded web server routing table single exclusive lock on an entire data base small buffer pool small flow control window Murray Woodside, July 2002 24 Multiplicity to cure a software bottleneck server level multithread a server replicated servers (this adds processing and network resources too) data level partition the data and lock each partition separately (locks on objects or pages) copies of the entire data service (distributed replicas of a routing table) subsystem level: partition a system and manage each part separately
Murray Woodside, July 2002 25 Reduced delay to cure a software bottleneck batched requests across a network of long latency avoid retrieval across a network of long latency cache data prefetch move functions out of this server (minimize the critical section) use worker threads or parallel operations or fetches Murray Woodside, July 2002 26 F) Experience in building and using models Examples: web integration with telephony router software and hardware web-based application Concerns: accuracy modeling power interpretation
Murray Woodside, July 2002 27 Example: WebTalk (web server+telephony) Support automated call answering by web pages speech playout is from a file encoded in a web page user enters tones; logic of decoding is also in the web page VoiceXML standard (vxmlforum.org) User scenario for one interaction. DTMF (tone) or begin call event 2. playout begins and continues 3. user listens and enters next DTMF event 4. next playout OR forward the call WIT 2 LISTEN/THINK 3 STOP PLYOUT Web-based telephone dialogue: Scenario over system components Murray Woodside, July 2002 28 Call Control Web Page Interpreter Remote Web Server Web Page Cache Playout Device
Murray Woodside, July 2002 29