Hints for Service Oriented Architectures. Marius Twitter Inc.

Transcription

1 Hints for Service Oriented Architectures Marius Twitter Inc.

2 We went from this (circa 2010) LB web web web web queue DB cache workers

3 to this (circa 2015) ROUTING PRESENTATION LOGIC STORAGE & RETRIEVAL Tweet MySQL Monorail User T-Bird TFE API Timeline Flock Web Social Graph Memcache DMs Redis HTTP Thrift Stuff

4 Problems, circa 2010 Deploys Kiji

5 Why SOAs in the first place? Decoupling Independent scaling and distribution Isolation

6 why not? It makes everything else more complicated

7 Setting

8 Hints

9 1. The datacenter is your computer crappy machines! tie the hands of the implementor

10 2. Embrace RPC outdated

11 1 2 n 1 2 n 1 2 n replica sets

12 3. Compute concurrently def querysegment(id: Int, query: String) : Future[Result] def search(query: String): Future[Set[Result]] = { val queries: Seq[Future[Result]] = for (id < 0 until NumSegments) yield { querysegment(id, query) } Future.collect(queries) flatmap { results: Seq[Set[Result]] => Future.value(results.flatten.toSet) } }

13 queries for { tweetids < timelineids(user, id) tweets < traverse(tweetids) { id => for { (tweet, sourcetweet) < TweetyPie.getById(id) (user, sourceuser) < Stitch.join( getbyuserid(tweet.userid), traverse(sourcetweet) { t => getbyuserid(t.userid) } ) } yield (tweet, user, sourcetweet, sourceuser) } } yield (tweets)

14 4. Multiplex HTTP HTTP reverse proxy MUX auth limit api login search web

15 5. Define and use SLOs Service level objectives p50 p90 p99 API

16 6. Abstract destinations end to end problem

17 One step further: logical destinations Wily late binding /s/user

18 7. Measure liberally blind Stats.add("request_latency_ms", duration) % curl "request_latency_ms": { "average": 1, "count": , "maximum": 950, "minimum": 0, "p50": 1, "p90": 3, "p95": 5, "p99": 19, "p999": 105, "p9999": 212, "sum": },

19 aggregated avg(ts(avg, Gizmoduck, Gizmoduck/request_latency_ms.p{50,90, 99,999})) outliers distributions

20 8. Profile and trace systems in situ % open

21 % curl O % pprof heap

22 % curl H X Twitter Trace: 1 D \ HTTP/ X Trace Id: f5e0399fa51b % open

23 9. Use and evolve data schemas

24 10. Delay work new_tweet tweets queue fanout fanout fanout DB

25 11. Shunt state ROUTING PRESENTATION LOGIC STORAGE & RETRIEVAL Tweet MySQL Monorail User T-Bird TFE API Timeline Flock Web Social Graph Memcache DMs Redis HTTP Thrift Stuff

26 12. Hide caches magical scaling sprinkles in memory serving frontends

27 13. Push complexity down compose

28 14. Think end to end end to end retries

29 15. Expect failure, and fail well resilience, n designed fail well patterns

30 16. Demand less

31 17. Beware hot shards and thundering herds

32 18. Embrace Conway s law

33 19. Deploy changes gradually isolated feature flags

34 20. Keep a tab

35 21. Plan to deprecate

36 A few questions to consider prompts

37 Fault tolerance slowly degrade overload simple shed load retried

38 Scalability grow vary

39 Operability quickly predictable

40 Efficiency as little work as possible

41 Thanks. Your server as a function. Patterns for fault tolerant software.