Berlin Storage, Backup and Disaster Recovery in the Cloud AWS Customer Case Study: HERE Maps for Life

Transcription

1 Berlin 2015 Storage, Backup and Disaster Recovery in the Cloud AWS Customer Case Study: HERE Maps for Life

2 Storage, Backup and Disaster Recovery in the Cloud Robert Schmid, Storage Business Development, AWS Ali Abbas, Principal Architect, HERE Case Study: AWS Customer HERE Maps for Life: Satellite Imagery - S3 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

3 What we will cover in this session Amazon storage options Amazon Elastic File System Use cases (Backup, Archive, DR) Customer Use Case: HERE Maps for Life, Satellite Imagery-S3

4 S3 usage 102% year-over-year increase in data transfer to and from S3 (Q vs Q4 2013, not including Amazon use)

5 Amazon S3 Simple Storage Service

6 Amazon S3 Simple Storage Service % durability $0.03 per GB-month $360 per TB/year

7 Amazon Glacier Low-cost archiving service

8 Amazon Glacier Low-cost archiving service $0.01 per GB-month $120 per TB/year % durability 3 5 hours data retrieval

9 Amazon EBS Elastic Block Storage

10 EBS General Purpose (SSD) Up to 16 TB 10,000 IOPS Provisioned IOPS (SSD) Up to 16 TB 20,000 IOPS $0.10 per GB-month $0.125 per GB-month 0.065/provisioned IOPS

11 Amazon Storage Gateway

12 Storage Gateway Your on-ramp to AWS cloud storage: Back up into S3 Archive into Amazon Glacier iscsi or VTL interface

13 Summary: AWS Storage Options Object Storage (S3, Glacier) Elastic Block Storage (EBS) Storage Gateway (iscsi, VTL) Elastic File System for EC2 (EFS)

14 Introducing Amazon Elastic File System for EC2 Instances pilot availability later this summer US-WEST (Oregon)

15 What is EFS? Fully managed file system for EC2 instances Provides standard file system semantics (NFSv4) Elastically grows to petabyte scale and shrinks elastically Delivers performance for a wide variety of workloads Highly available and durable simple elastic scalable

16 Amazon Storage Use Cases: Backup, Archive, Disaster Recovery

17 Backup, Archive, Disaster Recovery Customer Data Center Block File Archive Backup Disaster Recovery Colocation Data Center Customer /CSP Assets Storage Gateways AWS SGW DirectConnect Private Storage for AWS Internet AWS Direct Connect S3 Glacier AWS Cloud S3 Glacier

18 AWS Customer Case Study Ali Abbas HERE: Maps for Life Principal Architect High Resolution Satellite Imagery Predictive Analytics/Machine Learning 18

19 19 HERE Maps HERE Drive HERE Transit HERE City Lens Explore

20 Maps for Life Web and Mobile App available on: Android/iOS/Windows Phone 20

21 Save the maps of your country or state on your phone Use your phone offline Explore anywhere without an internet connection Offline Map 21

22 Unified Route Planning Route Alternatives Turn-by-turn Navigation Pocket Nav Sat 22

23 Route Alternatives Step-by-step transit Turn-by-turn walk guidance Urban Navigation 23

24 Collections Easy location sharing Personal Maps 24

25 Train Schedule Traffic incidents 3D Maps Interactive Maps 25

26 Reality Capture Processing Satellite/Aerial Delivery Enterprise Businesses End to End User Integration 26

27 99.99% availability, % durability High throughput/good Performance for most use-cases Good price ratio Design simplifies creating integration pipelines 27

28 28 The case f Satellite Imagery

29 29 Continuous increase global coverage with a higher frequency of refresh

30 Billion of tiles Huge storage requirements due to high resolution content across zoom levels Big amount of small tile size to keep track and deliver Challenges Exponential growth rate (today some billions, tomorrow some trillions) Increased data volume refresh rate 30 Maintain low latency requirements and service level agreement

31 Behind the curtain Specialized spatial file system to deliver tile imagery with sub-ms lookup time over the network. Simple Architecture with CDN Caches and Core sites (with full dataset) Remote sites had CDN type caches with geospatial shard-ing placement algorithms. Some select cache regions suffered sometimes from inter-continental network latency due to non-optimized routing The scale of data implies massive storage infrastructure to maintain on top 31

32 Mercator based shard-ing layer Specialized Spatial Blob Store Intelligent Filter layer Specialized Adaptive Spatial Blob Store Shared Store Singleton Store 32 Core Caches

33 Given the success of S3 usage across HERE and the recent enhancement to the offering, we started to look at S3 to solve 2 main problems with 1 solution Simplify the storage handling layer with getting rid of the storage compute from our architecture and simplify Operations. Reduce the network latency from core data to our delivery instances by adding core data presence in each availability regions. 33

34 Satellite on S3 Easy life-cycle management for recurring update Big Data store requirements on-demand (ease capacity planning) Easy pipeline integration with SQS/SNS for background jobs Good performance out of the box, however did not fulfill our requirements - Too much variation in response time ~ AVG ms. 34

35 S3 Load constrain Amazon S3 maintains an index of object key names in each AWS region. Object keys are stored lexicographically across multiple partitions in the index. That is, Amazon S3 stores key names in alphabetical order. The key name dictates which partition the key is stored in. Using a sequential prefix, such as timestamp or an alphabetical sequence, increases the likelihood that Amazon S3 will target a specific partition for a large number of your keys, overwhelming the I/O capacity of the partition. 35

36 S3 Load constrain + Satellite Amazon S3 maintains an index of object key names in each AWS region. Object keys are stored lexicographically across multiple partitions in the index. That is, Amazon S3 stores key names in alphabetical order. The key name dictates which partition the key is stored in. Using a sequential prefix, such as timestamp or an alphabetical sequence, increases the likelihood that Amazon S3 will target a specific partition for a large number of your keys, overwhelming the I/O capacity of the partition. 36

37 S3 Load constrain + Satellite Stored lexicographically across S3 partitions z x y Satellite example tile ID: 15/18106/ /18089/11275 quadkey representation /72409/

38 S3 Load constrain + Satellite Stored lexicographically across S3 partitions z x y Satellite example tile ID: 15/18106/ /18089/11275 quadkey representation /72409/ Each zoom level has 4^level_detail tiles, a quadkey length is equal to the level of detail of the corresponding tile. 38

39 S3 Load constrain + Satellite Stored lexicographically across S3 partitions Alternative to quadkeys use random hash, increase base number Remaining problem At the scale of satellite, the ratio of requests in regards to the lexicographic overlap produced with a random hash was still significant and would not scale well. Performance was still unacceptable in light of our requirements. Billion of PUT requests would considerably increase recurring-updates cost. 39

40 S3 Load constrain + Satellite Stored lexicographically across S3 partitions Better solution Reduce the amount of files by creating binary blob on S3, index the tiles inside the blobs and use HTTP range-request for access. New Challenge Managing updates got more complicated, more logic requires to distribute tiles inside the blobs and more important the predicted index size was in magnitude of terabytes and growing cost and complexity overhead. 40

41 41 Back on the whiteboard

42 New Pseudo-Quad Index New compact O(1) data-structure to work around the performance constrains of S3 It minimizes the index size constrain to keep track of tiles and random hashes % size reduction in comparison to generic optimized hash tables It reduces and sets boundaries for proximity regions to cause better dispersion on the n-gram load split algorithm used by S3 Simplified Imagery updates; geometrical consistency across all S3 buckets Performance: S3: > ms S3 + PQI: <26ms 42

43 With S3 and PQI we have simplified our architecture PQI Backend Tiny ref file Simple infrastructure delivering from a few billion up to a few trillion images 43

44 Impact on Architecture Impact on day-day Operation of our services Brings us geographically closer to our customer while not compromising on design patterns to work around network latencies. Allows us to only focus on our core business and technologies while offloading compute/storage to AWS. 44

45 Thank you! please meet our Sponsors/Partners and see us in the EXPO area 2015, Amazon Web Services, Inc. or its affiliates. All rights reserved

46 Further information: