the emperor s new consistency the case against weak consistency in data centers marcos k. aguilera microsoft research silicon valley

Size: px

Start display at page:

Download "the emperor s new consistency the case against weak consistency in data centers marcos k. aguilera microsoft research silicon valley"

Aron Atkins
8 years ago
Views:

1 the emperor s new consistency the case against weak consistency in data centers marcos k. aguilera microsoft research silicon valley

2 synopsis weak consistency widely adopted today in data center systems we will argue: this trend will change strong consistency will be the norm reasons: technological, legal, sociological

3 types of consistency replica consistency data replicas should be in sync with each other therefore: equivalent to read from any replica need to look into the system to know data consistency invariants must hold for the data set the C in ACID assumes the user behaves well

need to look into the system to know data consistency invariants

4 types of consistency semantic consistency data operations return sensible results eg: linearizability, causal consistency, sequential consistency, read-own-writes, transaction isolation properties defined by behavior of interface to storage system typically replica consistency => semantic consistency semantic consistency + correct app => data consistency

transaction isolation properties defined by behavior of interface to storage system

5 types of consistency focus of this talk: semantic consistency don t care about how things are implemented don t want to get into correctness of application

6 weak consistency best effort operations can return any response eventual consistency eventually operations return correct responses causal consistency if an operation observes an effect then it observes its causes many others

return correct responses causal consistency if an

7 consistency in data center systems yahoo pnuts: weak (timeline consistency) amazon s3: weak (eventual consistency) with option of slightly stronger consistency amazon simpledb: weak or strong microsoft azure: strong dynamo: weak (reads may not intersect writes) cops: weak (causal-plus consistency) walter: weak (parallel snapshot isolation) etc

simpledb: weak or strong microsoft azure: strong dynamo: weak (reads may not intersect

8 inconsistency hard to contain => inconsistency propagates all the way to the end user

9 origins of weak consistency in data center systems

10 why weak consistency in data center storage systems in a nutshell: benefits > drawbacks benefits better performance and availability even more in geo-distributed applications can tolerate intermittent spikes in usage storage systems easier to develop no guarantees => system correct no matter what drawbacks users see stale data but only sometimes and users are tolerant

tolerate intermittent spikes in usage storage systems easier to develop no guarantees =>

11 key argument benefits of weak consistency are shrinking technological reasons drawbacks of weak consistency are expanding technological, legal, and sociological reasons corollary: in the future, benefits < drawback next: explain factors leading to these changes

technological, legal, and sociological reasons corollary: in the

12 FACTORS: INCREASING DRAWBACKS OF WEAK CONSISTENCY

13 factor: increasing number of paid apps free apps => users tolerate strange behavior but more paid data center apps emerging social networks: facebook, linkedin, etc content provider: hulu, itunes store, netflix, etc online games: zynga, battle.net, etc online stores: amazon, most large retailers, etc office apps: office 365 business apps: salesforce, etc paying users + strange behavior = angry users

store, netflix, etc online games: zynga, battle.

14 factor: people more reliant on data center apps daily communication, shopping, banking, content sharing, appointments, medical records, etc application provider subject to liability strange behavior can lead to losses

appointments, medical records, etc application

15 factor: more app layers, more integration user of app becomes another app bank records => financial software social network data => third-app vendors s, appointments => phone client humans can identify and tolerate inconsistency programs have harder time weaker invariants => more corner cases => complexity examples: references to inexistent objects, old versions

client humans can identify and tolerate inconsistency programs have harder time weaker

16 FACTORS: DECREASING BENEFITS OF WEAK CONSISTENCY

17 factor: network partitions will disappear causes of network partitions being addressed 1. lack of bandwidth or connectivity within data center: full bisection bandwidth networks across data centers: more infrastructure around world 2. failure-prone network elements being replaced with distributed fault-tolerant systems 3. operator errors better tools to manage network centralized management via software-defined networking

centers: more infrastructure around world 2.

18 factor: wide-area latency shrinks currently, ping latencies still large 3-4 times larger than speed-of-light values due to switching delays, long paths sometimes more due to congestion speed-of-light values are small enough circumference of earth: 40,000 Km -> 133 ms Frankfurt-San Francisco: 9,000 Km -> 30 ms Frankfurt-Delhi: 7,500 Km -> 25 ms eventually get to these latencies

speed-of-light values are small enough circumference of earth: 40,000 Km -> 133 ms

19 factor: number of apps growing relative to number of storage systems storage system = infrastructure for many apps today: many storage systems eg, 25+ NOSQL storage systems active area of development much functionality overlap, few will survive meanwhile: number of apps increases argument that it is easier to build a weakly consistent storage system loses force, because it makes it harder to develop apps

much functionality overlap, few will survive meanwhile: number of apps increases argument that it is

20 combination of factors more paid data center apps people more reliant on data center apps more app layers, more integration inconsistency angers users app provider subject to liability inconsistency creates complexity more drawbacks of weak consistency no network partitions smaller wide-area latency fewer storage systems relative to apps availability and performance advantage of weak consistency shrinks better to build stronger storage system and benefit many apps fewer benefits of weak consistency

consistency no network partitions smaller wide-area latency fewer storage systems relative to apps availability and

21 factors that would undermine argument performance difference between weak and strong consistency really matters maybe, but ms not bad especially because operations generally can be batched society embraces inconsistency like software bugs, people just give up how I stopped worrying and learned to love the inconsistency

22 conclusion in the context of data center systems weak consistency creates challenges that will become more problematic weak consistency solves problems that will be solved differently

Geo-Replication in Large-Scale Cloud Computing Applications

Geo-Replication in Large-Scale Cloud Computing Applications Sérgio Garrau Almeida sergio.garrau@ist.utl.pt Instituto Superior Técnico (Advisor: Professor Luís Rodrigues) Abstract. Cloud computing applications