Cloud Datastore: A NoSQL Database at Google Scale

Transcription

1 Clud Datastre: A NSQL Database at Ggle Scale Randy Shup Engineering Directr, Ggle Clud Platfrm NSQL-Search RadShw SF June 6, 2013

2 Agenda Ggle Clud Platfrm Ggle Scale Ggle Clud Datastre Ggle Strage Infrastructure Parting Thughts

3 Ggle Clud Platfrm

4 Clud Cmputing at Ggle Ggle builds and perates ne f the largest cmputing infrastructures in the wrld Dzens f data centers lcated arund the wrld Designed frm the grund up t run massive Internet-scale services Integrated design f facility and cmputing machinery Hmgeneus hardware and system sftware Cluster-level netwrking fabric

5 Clud Cmputing at Ggle All Ggle Cmputing is Clud Cmputing Custm-built machines and netwrk Cluster is typically thusands f machines Cmmn pl f resurces with central cluster management Fungible units f cmpute, memry, strage, netwrk Sphisticated bin-packing t maximize utilizatin Hundreds t thusands f active jbs, frm ne task t thusands f tasks Mix f lw-latency, user-facing jbs and batch wrklads Massively multitenant

6 Ggle Clud Platfrm Ggle Cmpute Engine Ggle Clud Strage Ggle BigQuery Ggle App Engine Full Linux virtual machines running n Ggle's infrastructure. Stre, access, and manage applicatin data. Analyze terabytes f data in secnds. Platfrm as a Service: Pwerful, scalable applicatin develpment and executin envirnment.

7 Ggle Scale

8 Elements f Ggle Scale Layering and Cmpsitin Cmpse cmplex systems frm simple primitives As much as pssible, make it pssible t reasn independently and intuitively abut behavir f primitives All Ggle services rely n (ften many!) lwer layers f infrastructure

9 Elements f Ggle Scale At Scale, Everything Breaks Service-level utages Netwrking Pwer Ops Nde-level utages (industry average) >1% uncrrectable DRAM errrs per server per year 2-10% disk drive failure rate per year ~2 crashes per server per year >1 utility event per year => 2000 nde service sees 10 server crashes per day (!) Surce:

10 Elements f Ggle Scale Predictable Perfrmance Systems at scale highly expsed t perfrmance variability Imagine an peratin... 1ms latency median, but 1 secnd latency at 99.99%ile (1 in 10,000) Service using 1 machine -> 0.01% slw Service using 5K machines -> 50% slw Cnsistent perfrmance trumps lw average latency Lw latency + incnsistent perfrmance!= lw latency (!) Far easier t prgram fr cnsistent perfrmance Tail latencies are *much* mre imprtant than average latencies

11 Elements f Ggle Scale Opininated Platfrm Encurage scalable develpment practices Small discrete units f prcessing N single pints f failure Autmated testing Staged deplyments Make it easy t d the right thing, and hard t d the wrng thing ==> "It Just Wrks" (TM)

12 Ggle Clud Datastre

13 (Re-)Intrducing Ggle Clud Datastre Based n High Replicatin Datastre in Ggle App Engine Multiple generatins f evlutin Originally intrduced with Ggle App Engine in M+ applicatins, 300K unique develpers Petabytes f strage 4.5T+ peratins / mnth Layered n tp f MegaStre BigTable Clssus

14 Ggle Clud Datastre Accessible RESTful interface HTTP with JSON r Prtcl Buffer API Accessible frm Ggle Cmpute Engine Ggle App Engine Anywhere else Web-based interface fr cnfiguratin and management Develpment server fr lcal develpment

15 Ggle Clud Datastre Fully Managed N planned dwntime Cmpletely autmated failver Replicated acrss multiple data-centers All data replicated acrss multiple disks and multiple data centers Managed and perated as a service by Ggle 99.95% SLA

16 Ggle Clud Datastre Scalable Arbitrary hrizntal scaling Autscales as traffic increases Autshards as data increases Mre distributed as mre data is stred

17 Ggle Clud Datastre Resilient Crss-data center active-active replicatin All data replicated acrss multiple disks and multiple data centers Synchrnus replicatin via Paxs Applicatin can seamlessly migrate between data centers with n data lss Applicatins can read lcally in separate data centers with n incnsistency r replicatin lag Resilient t catastrphic failure ("meterite durability")

18 Ggle Clud Datastre Schemaless N cnfiguratin needed; just start writing data Arbitrary attributes n any entity Different entities can have different attributes Attributes can be multi-valued Arbitrary-depth parent-child relatinships "Entity grups" can assciate many related entities E.g., all s fr a user

19 Ggle Clud Datastre Cnsistency Strngly cnsistent, with atmic transactins Strng serial cnsistency within entity grup Will always Get an entity nce Put Never see partial transactins Strng cnsistency n reads and ancestr queries Multi-entity grup transactins Transactins can read / write entities within (small number f) entity grups Eventual cnsistency nly when querying acrss many entity grups

20 Ggle Clud Datastre Rich Query Features GQL is an ever-grwing subset f SQL Filters Equality (=, IN) Inequality (!=, <, <=, >=, >) AND, OR, NOT, sub-expressins Srt DISTINCT Prjectins, index-nly queries Ge radius, Date range Cursrs fr paged iteratin

21 Ggle Clud Datastre Predictable Perfrmance Fixed cst queries Query latency scales in the size f the result set, nt in the size f the verall data Cnstant latency fr queries ver 1M r 1B r 1T entities All queries are index queries "It's nt a limitatin, it's a discipline"

22 Ggle Strage Infrastructure

23 Clssus (GFSv2) Next-generatin clustered file system, successr t GFS Exabyte scale glbal strage system Autmatically sharded metadata layer Data blcks fr a given stripe replicated t multiple different fault dmains Different disks, servers, racks Blcks distributed acrss entire cluster Easy t lad-balance reads Efficient t recver "Yu knw yu have a large strage system when yu get paged at 1 AM because yu nly have a few petabytes f strage left." -- Ggle Engineer Surce:

24 BigTable Cluster-level structured strage Distributed multi-dimensinal sparse map (rw, clumn, timestamp) -> cell cntents Layered n Clssus fr file strage Autmatically splits and rebalances tablets based n size and lad Fault-tlerant within data center Asynchrnus, eventually-cnsistent replicatin "If yu lk at every NSQL slutin ut there, everyne ges back t the Amazn Dynam paper r the Ggle BigTable paper" -- Jasn Hffman, Jyent

25 MegaStre Ge-scale structured database Layered n BigTable fr structured strage Multi-rw transactins acrss machines Strng ACID cnsistency within fine-grained partitins ("entity grups") Eventual cnsistency acrss partitins Synchrnus crss-datacenter replicatin via Paxs Transparent failver

26 Parting Thughts

27 Thughts n SQL, NSQL, NearSQL "One Size Des Nt Fit All" Everything is a tradeff Data structures are fundamental t perfrmance and features f any strage system N data structure can ptimize fr every pssible use-case Plyglt persistence is expected Clumn stres fr analytics Inverted indexes fr search Simple key-value stres Scalable, pwerful NearSQL systems We use everything at Ggle (!)

28 Thughts n Scale Scale Depends On... Discipline, nt permissiveness Sharing, nt cupling Architecture, nt language r prgramming envirnment Simplicity and elegance, nt cmplexity

29 Questins? and... We are hiring!