EFFICIENT ANALYSIS OF APPLICATION SERVERS IN THE CLOUD

Transcription

1 EFFICIENT ANALYSIS OF APPLICATION SERVERS IN THE CLOUD Progress report meeting December 2012 Phuong Tran Gia Under the supervision of Prof. Michel R. Dagenais Dorsal Laboratory, École Polytechnique de Montréal

2 Outline Completed tasks Future work Challenges Questions

3 Completed tasks Build background in: Distributed and large scale systems The design and implementation of the Linux kernel Userspace and Kernel Tracing LTTng documentations Project proposal of Cloud Computing project State of the art of study, including Zipkin(Twitter) and Dapper(Google)

4 Zipkin A distributed tracing framework Why Zipkin?

5 Zipkin A distributed tracing framework Why Zipkin? Helping developers gain deeper knowledge about how certain requests perform in a distributed system

6 Zipkin A distributed tracing framework Why Zipkin? Helping developers gain deeper knowledge about how certain requests perform in a distributed system Helping developers gather timing data for the disparate services at Twitter

7 Zipkin Architecture

8 Cluster #02 Thrift Finagle MySQL Memcache Scribe daemon (client side) Zipkin Architecture Zipkin Server Zipkin-web Query daemon Cluster #01 Cassandra DB Thrift Admin Finagle (client side), Ruby Thrift Scribe daemon (client side) Scribe/ Zookeeper Zookeeper Zipkin Collector (Zipkin-collector core, Scribe-server side, Finagle server side, Zipkin collector daemon)

9 Finagle in Zipkin A core module of Zipkin. Finagle is an asynchronous network stack for the JVM that we can use to build asynchronous Remote Procedure Call (RPC) clients and servers in Java, Scala, or any JVMhosted language. github.com/twitter/finagle

10 Finagle in Zipkin [1] How Finagle-http works

11 Zipkin terminology Annotation: string data associated with a particular timestamp, service and host Time time: :17:05 value: something happened server: service: timelineservice

12 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send Time

13 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send Time T: 10ms Server Receive

14 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send Time T: 10ms Server Receive T:90ms Server Send

15 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send T: 100ms Client Receive Time T: 10ms Server Receive T:90ms Server Send

16 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send T: 100ms Client Receive Time T: 20ms Read 30 Kbytes from file T: 10ms Server Receive T:90ms Server Send

17 Zipkin terminology Span: represents one specific method call; made up of a set of annotations. Has a name and an id T: 0ms Client Send T: 100ms Client Receive Time T: 20ms Read 30 Kbytes from file T: 10ms Server Receive T:90ms Server Send Trace: A set of spans all associated with the same request.

18 Zipkin UI

19 Zipkin UI

20 Zipkin UI

21 Zipkin UI

22 Zipkin UI

23 Zipkin UI - Dependencies

24 Google s Dapper

25 Google s Dapper [2] Collect traces from production requests Low overhead Minimum of extra work for developer

26 Future work Build an analysis module for application servers by intercepting Asynchronous RPC. Start tracing MySQL/PostgreSQL, Redis/Cassandra services and tomcat/apache2 application servers Could be integrated with Streaming Data module (like Scribe) for cluster management. Large scale system performance profiling

27 Challenges Supporting many protocols Distributed tracing Performance improvements Tracing overhead management (adaptive sampling, production workloads) Flexible and easy to use. Integration with LTTng

28 Questions Typical application servers in Cloud environment of interest for industrial partners?

29 References [1] Johan Oskarsson, Zipkin- Runtime Open House, July 27, 2012 [2] Benjamin H. Sigelman, Luiz Andre Barroso, Mike Burrows, Pat Stephenson, Manoj Plakal, Donald Beaver, Saul Jaspan, Chandan Shanbhag, Dapper, a Large-Scale Distributed Systems Tracing Infrastructure, Google Technical Report dapper , April 2010