Trainer and Consultant at IT-Visions.de

Size: px

Start display at page:

Download "Trainer and Consultant at IT-Visions.de"

Roderick Webster
8 years ago
Views:

1 By by relational databases and ACID-transactions: An introduction into BASE and NoSQL by the example Windows Azure Storage Services Manfred Steyer twitter.com/manfredsteyer Who I am Manfred Steyer Professor at CAMPUS 02, Graz (Austria) Responsible for Sowftware Engineering Trainer and Consultant at IT-Visions.de Folie 2 1

com/manfredsteyer Who I am Manfred Steyer Professor at CAMPUS 02, Graz (Austria)

2 Goal Conceptional overview of ideas behind NoSQL and BASE by the example of Azure Storage Services. Folie 3 Contents Thoughts about scaling databases Windows Azure Storage Services Folie 4 2

3 SCALING DATABASES Page 6 Big Advantage of Cloud-Solutions: Scalability and Elasticity Folie 7 3

4 RDBMS don t scale that easy! Folie 8 Scale up: More Metal Folie 9 4

5 Scale out: More Instances Folie 10 Shared Disk Folie 11 5

6 Shared Nothing Folie 12 Vertical Partitioning Products Orders Ads Folie 13 6

7 Horizontal Partitioning aka Sharding A - G H - M N - Z Folie 14 Foreign Key and Joins between Partitions are not possible Folie 15 7

8 You have to design your app for sharding Folie 16 Distribution of data is not transparent for the application Folie 17 8

9 ACID? Folie 18 Distributed Transactions: 2 Phase Commit Folie 19 9

10 But: 0,9 x 0,9 = 0,81 Folie 20 Availability is the most important criteria for huge internet-apps!!! Folie 21 10

11 Statistics Google: Delay of 0,5 sec 20 % less traffic Amazon: Every Delay of 100 ms 1 % less Sales-Amount Folie 22 Performance: Denormalize Folie 23 11

12 Recap No foreign keys No joins No ACID-Transactions Denormalized Data Folie 24 Why to use RDBMS at all?! Folie 25 12

13 NoSQL Folie 26 What is NoSQL? Simple Tables or Kay/Value-Pairs Support denormalization No Joins, No Foreignkeys, No Transactions Scale automatically Queues?! Blobs?! Folie 27 13

14 BASE Folie 28 Basically available Folie 29 14

15 soft state Folie 30 eventually consistent Folie 31 15

16 Sample Profiles Name Last Posting Date of last Posting Postings Profile Text Date Folie 32 Naive Approche Transaction-1 Insert into Postings Transaction-2 Update Profiles Folie 33 16

17 Possible Solution Transaction-1 Enqueue into Posting-Queue, My Text, , Profil XY Transaction-2 Dequeu from Posting-Queue Insert into Postings Update Profiles? Folie 34 AZURE STORAGE-SERVICES Page 35 17

18 Azure Storage Services NoSQL and BASE-Mindset Table Service Blob Service Queue Service Folie 36 APIs REST-API.NET-API JAVA-API Folie 37 18

19 Table Service Nodes PartitionKey RowKey Timestamp A 1 A 2 PartitionKey RowKey Timestamp B 1 B 2 B 3 PartitionKey RowKey Timestamp C 1 C 2 Partitions Table Services Queries via LINQ Each table get this 3 columns string PartitionKey (max. 1 KB) string RowKey (max. 1 KB) DateTime Timestamp Other columns: there is no fixed schema Index only for PartitionKey and RowKey!! Max. 1 MB per row 19

this 3 columns string PartitionKey (max. 1 KB) string RowKey (max.

20 Restrictions No Joins No Transaktionen* (* only possible for rows with the same PartitionKey) No Foreign-Keys Only Indexes for PartitionKey and RowKey Folie 40 BLOB-Storage Up to 1 TB per BLOB Folie 41 20

21 Queues Max. 8KB/ Message No Locks FIFO not guaranteed Messages can be delivered more then once No Transactions!!! Folie 42 Alternative for transactional reads Read next message + message get invisible for x seconds Step 1 Step 2... Delete Message Folie 43 21

22 Requirement: Idempotency Folie 44 Idempotency and the discussed sample Update Profile: Update is idempotent Insert Posting: Id has to be known from the beginning (GUID) Insert-or-Update -Semantic Folie 45 22

23 Take care of Poison-Messages If a message has been reat more than x times, (send it to the admin and) drop it Folie 46 Concurrency Checks Use timestamp to perform concurrency checks ( optimistic concurrency ) The first one wins (does table services for you) The last one wins Folie 47 23

24 Sample Profiles Name Last Posting Date of last Posting Postings Text Date Folie 48 Toughts There are several cases where consistency isn t necessary Beispiel: Update Profile + Update Profile-Photo Folie 49 24

25 NoSQL vs. Sharding NoSQL (+) scales transparent (+) supports denormalisation (+) simple data-structures (-) you have to do a lot by youself (-) Poor Query-Support (-) no consistency Neutral: BASE-Mindset Folie 50 NoSQL vs. Sharding Sharding (+) Foreign Keys, Transactions within a shard (+) consistency (+) good query support (+) complex data-structures (-) does not scale transparent Neutral: BASE-Mindset Folie 51 25

26 Summary Shared nothing and sharding NoSQL scales very good Queues for eventual Consistency Peek-Process-Delete instead of transactions Take care of poison-messages, concurrency and idempotency Folie 52 26

Windows Azure Storage Essential Cloud Storage Services http://www.azureusergroup.com

Windows Azure Storage Essential Cloud Storage Services http://www.azureusergroup.com David Pallmann, Neudesic Windows Azure Windows Azure is the foundation of Microsoft s Cloud Platform It is an Operating