Big Data. Without a Big Database. Kate Matsudaira

Transcription

1 Big Data Without a Big Database Kate Matsudaira

2 Two kinds of data nicknames examples: created/modified by: sensitivity to staleness: user, transactional user accounts shopping cart/orders user messages users high reference, nontransactional product/offer catalogs catalogs static geolocation data dictionaries business (you) low plan for growth: hard easy access optimization: read/write mostly read

3 Two kinds of data nicknames examples: created/modified by: sensitivity to staleness: user, transactional user accounts shopping cart/orders user messages users high reference, nontransactional product/offer catalogs catalogs static geolocation data dictionaries business (you) low plan for growth: hard easy access optimization: read/write mostly read

4

5 reference data

6 reference data user data

7

8 reference data

9 reference data user data

10 Performance Reminder main memory read ms (100 ns) network round trip 0.5 ms (500,000 ns) disk seek 10 ms (10,000,000 ns) source:

11

12

13

14 The Beginning load balancer webapp webapp load balancer BIG DATABASE data loader

15 The Beginning availability problems load balancer webapp webapp load balancer BIG DATABASE data loader

16 The Beginning availability problems performance problems load balancer webapp webapp load balancer BIG DATABASE data loader

17 The Beginning availability problems performance problems load balancer webapp webapp load balancer BIG DATABASE data loader scalability problems

18 Replication load balancer webapp webapp load balancer BIG DATABASE data loader

19 Replication REPLICA load balancer webapp webapp load balancer BIG DATABASE data loader

20 Replication REPLICA load balancer webapp webapp load balancer BIG DATABASE data loader scalability problems

21 Replication REPLICA load balancer webapp webapp load balancer BIG DATABASE data loader scalability problems performance problems

22 operational overhead Replication REPLICA load balancer webapp webapp load balancer BIG DATABASE data loader scalability problems performance problems

23 Local Caching REPLICA load balancer webapp load balancer BIG DATABASE webapp data loader

24 Local Caching REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache data loader

25 Local Caching scalability problems REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache data loader

26 Local Caching scalability problems operational overhead REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache data loader

27 Local Caching scalability problems operational overhead REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache data loader performance problems

28 Local Caching scalability problems operational overhead REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache consistency problems data loader performance problems

29 Local Caching scalability problems operational overhead REPLICA cache load balancer webapp load balancer cache BIG DATABASE webapp cache long tail performance problems consistency problems data loader performance problems

30 80% of requests query 10% of entries (head) The Long Tail Problem 20% of requests query remaining 90% of entries (tail)

31 Big Cache replica load balancer webapp webapp load balancer BIG CACHE database data loader preload

32 Big Cache operational overhead replica load balancer webapp webapp load balancer BIG CACHE database data loader preload

33 Big Cache operational overhead performance problems replica load balancer webapp webapp load balancer BIG CACHE database data loader preload

34 Big Cache operational overhead performance problems replica load balancer webapp webapp load balancer BIG CACHE database data loader preload scalability problems

35 Big Cache operational overhead performance problems replica load balancer webapp webapp load balancer BIG CACHE database data loader consistency problems preload scalability problems

36 Big Cache long tail performance problems operational overhead performance problems replica load balancer webapp webapp load balancer BIG CACHE database data loader consistency problems preload scalability problems

37 Big Cache long tail performance problems operational overhead performance problems replica load balancer webapp webapp load balancer BIG CACHE database data loader consistency problems preload scalability problems

38 Big Cache Technologies

39 memcached(b) Big Cache Technologies

40 Big Cache Technologies memcached(b) Do I look like I need a cache? ElastiCache (AWS)

41 Big Cache Technologies memcached(b) Do I look like I need a cache? ElastiCache (AWS) Oracle Coherence

42

43 Targeted generic data/ use cases.

44 Dynamically assign keys to the nodes Targeted generic data/ use cases.

45 Dynamically assign keys to the nodes Targeted generic data/ use cases. Scales horizontally

46 Dynamically assign keys to the nodes Targeted generic data/ use cases. Dynamically rebalances data Scales horizontally

47 Dynamically assign keys to the nodes Targeted generic data/ use cases. Dynamically rebalances data Scales horizontally Poor performance on cold starts

48 Dynamically assign keys to the nodes Targeted generic data/ use cases. Dynamically rebalances data Scales horizontally No assumptions about loading/ updating data Poor performance on cold starts

49 Big Cache Technologies Additional hardware Additional configuration Additional monitoring Extra network hop Slow scanning Additional deserialization

50 Big Cache Technologies Additional hardware Additional configuration Additional monitoring }operational overhead Extra network hop Slow scanning Additional deserialization

51 Big Cache Technologies Additional hardware Additional configuration Additional monitoring } operational overhead Extra network hop Slow scanning Additional deserialization }performance

52 NoSQL to The Rescue? NoSQL Replica load balancer webapp webapp load balancer NoSQL Database data loader

53 NoSQL to The Rescue? NoSQL Replica load balancer webapp webapp load balancer NoSQL Database data loader some performance problems

54 NoSQL to The Rescue? NoSQL Replica load balancer webapp webapp load balancer NoSQL Database data loader some performance problems some scalability problems

55 NoSQL to some operational overhead The Rescue? NoSQL Replica load balancer webapp webapp load balancer NoSQL Database data loader some performance problems some scalability problems

56 Remote Store Retrieval Latency network remote store network client

57 Remote Store Retrieval Latency network remote store network client TCP request: 0.5 ms Lookup/write response: 0.5 ms TCP response: 0.5 ms read/parse response: 0.25 ms

58 Remote Store Retrieval Latency network remote store network client TCP request: 0.5 ms Lookup/write response: 0.5 ms TCP response: 0.5 ms read/parse response: 0.25 ms Total time to retrieve single value: 1.75 ms

59 Total time to retrieve A single value from remote store: 1.75 ms from memory: ms (10 main memory reads)

60 Total time to retrieve A single value from remote store: 1.75 ms from memory: ms (10 main memory reads) Sequential access of 1 million random keys from remote store: 30 minutes from memory: 1 second

61 The Truth About Databases

62 What I'm going to call as the hot data cliff: As the size of your hot data set (data frequently read at sustained rates above disk I/O capacity) approaches available memory, write operation bursts that exceeds disk write I/O capacity can create a trashing death spiral where hot disk pages that MongoDB desperately needs are evicted from disk cache by the OS as it consumes more buffer space to hold the writes in memory. MongoDB Source:

63 Redis is an in-memory but persistent on disk database, so it represents a different trade off where very high write and read speed is achieved with the limitation of data sets that can't be larger than memory. Redis source:

64 They are fast if everything fits into memory.

65

66 Can you keep it in memory yourself? full cache data loader load balancer webapp webapp load balancer full cache full cache data loader data loader BIG DATABASE

67 Can you keep it in memory yourself? operational relief full cache data loader load balancer webapp webapp load balancer full cache full cache data loader data loader BIG DATABASE

68 Can you keep it in memory yourself? operational relief full cache data loader load balancer webapp webapp load balancer full cache full cache data loader data loader BIG DATABASE scales infinitely

69 Can you keep it in memory yourself? operational relief full cache data loader load balancer webapp webapp load balancer full cache full cache data loader data loader BIG DATABASE performance gain scales infinitely

70 Can you keep it in memory yourself? operational relief full cache data loader load balancer webapp webapp load balancer full cache full cache data loader data loader BIG DATABASE consistency problems performance gain scales infinitely

71 Fixing Consistency deployment cell webapp full cache data loader load balancer webapp full cache data loader webapp full cache data loader

72 Fixing Consistency 1. Deployment Cells 2. Sticky user sessions deployment cell webapp full cache data loader load balancer webapp full cache data loader webapp full cache data loader

73 How do you fit all of that data into memory? credit:

74 "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered.! We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.! Donald Knuth

75 How do you fit all that data in memory?

76 The Answer

77 Domain Model Design Domain Layer (or Model Layer):! Responsible for representing concepts of the business, information about the business situation, and business rules. State that reflects the business situation is controlled and used here, even though the technical details of storing it are delegated to the infrastructure. This layer is the heart of business software.! Eric Evans, Domain-Driven Design,

78 Domain Model Design Guidelines

79 Domain Model Design Guidelines #1 Keep it immutable

80 Domain Model Design Guidelines #1 Keep it immutable #2 Use independent hierarchies

81 Domain Model Design Guidelines #1 Keep it immutable #3 Optimize Data #2 Use independent hierarchies Help! I am in the trunk!

82 intern()your immutables K1 V1 K1 V1 A A K2 V2 B C F D E E K2 V2 B C D E B C D E F

83 private final Map<Class<?>, Map<Object, WeakReference<Object>>> cache = new ConcurrentHashMap<Class<?>, Map<Object, WeakReference<Object>>>();! public <T> T intern(t o) { if (o == null) return null; Class<?> c = o.getclass(); Map<Object, WeakReference<Object>> m = cache.get(c); if (m == null) cache.put(c, m = synchronizedmap(new WeakHashMap<Object, WeakReference<Object>>())); WeakReference<Object> r = T v = (r == null)? null : (T) r.get(); if (v == null) { v = o; m.put(v, new WeakReference<Object>(v)); } return v; }

84 Use Independent Hierarchies Product id = title= Product Summary productid = Offers productid = Offers Specifications Description Specifications productid = Reviews productid = Product Info Reviews Rumors Model History Model History productid = Description productid = Rumors productid =

85 Collection Optimization

86 Leverage Primitive Keys/Values collection with 10,000 elements [0.. 9,999] java.util.arraylist<integer> java.util.hashset<integer> gnu.trove.list.array.tintarraylist size in memory 200K 546K 40K gnu.trove.set.hash.tinthashset 102K Trove ( High Performance Collections for Java )

87 Optimize small immutable collections Collections with small number of entries (up to ~20): class ImmutableMap<K, V> implements Map<K,V>, Serializable {... }! class MapN<K, V> extends ImmutableMap<K, V> { final K k1, k2,..., kn; final V v1, v2,..., public boolean containskey(object key) { if (eq(key, k1)) return true; if (eq(key, k2)) return true;... return false; }...

88 java.util.hashmap:! Space Savings 128 bytes + 32 bytes per entry!! compact immutable map:! 24 bytes + 8 bytes per entry

89 Numeric Data Optimization

90 Price History Example

91 Example: Price History Problem: Store daily prices for 1M products, 2 offers per product Average price history length per product ~2 years! Total price points: (1M + 2M) * 730 = ~2 billion

92 Price History First attempt TreeMap<Date, Double>! 88 bytes per entry * 2 billion = ~180 GB

93 Typical Shopping Price History price $ days

96 Run Length Encoding a a a a a a b b b c c c c c c 6 a 3 b 6 c

97 Price History! positive: price (adjusted to scale) negative: run length (precedes price) zero: unavailable Optimization Drop pennies Store prices in primitive short (use scale factor to represent prices greater than Short.MAX_VALUE) Memory: 15 * (array) + 24 (start date) + 4 (scale factor) = 74 bytes

98 Reduction compared to TreeMap<Date, Double>: Space Savings 155 times Estimated memory for 2 billion price points: 1.2 GB

99 Reduction compared to TreeMap<Date, Double>: Space Savings 155 times Estimated memory for 2 billion price points: 1.2 GB << 180 GB

100 public class PriceHistory {! private final Date startdate; // or use org.joda.time.localdate private final short[] encoded; private final int scalefactor;! public PriceHistory(SortedMap<Date, Double> prices) { } // encode public SortedMap<Date, Double> getpricesbydate() { } // decode! Price History Model public Date getstartdate() { return startdate; } // Below computations implemented directly against encoded data public Date getenddate() { } public Double getminprice() { } public int getnumchanges(double minchangeamt, double minchangepct, boolean abs) { } public PriceHistory trim(date startdate, Date enddate) { } public PriceHistory interpolate() { }

101 Know Your Data

102 Compress text

103 String Compression: byte arrays Use the minimum character set encoding static Charset UTF8 = Charset.forName("UTF- 8");! String s = "The quick brown fox jumps over the lazy dog ; // 42 chars, 136 bytes byte[] b a= "The quick brown fox jumps over the lazy dog.getbytes(utf8); // 64 bytes String s1 = Hello ; // 5 chars, 64 bytes byte[] b1 = Hello.getBytes(UTF8); // 24 bytes! byte[] tobytes(string s) { return s == null? null : s.getbytes(utf8); } String tostring(byte[] b) { return b == null? null : new String(b, UTF8); }

104 String Compression: shared prefix Great for URLs public class PrefixedString { private PrefixedString prefix; private byte[] public int hashcode() { public boolean equals(object o) { } }

105 String Compression: short alphanumeric case-insensitive strings public abstract class AlphaNumericString { public static AlphaNumericString make(string s) { try { return new Numeric(Long.parseLong(s, Character.MAX_RADIX)); } catch (NumberFormatException e) { return new Alpha(s.getBytes(UTF8)); } } protected abstract String public String tostring() {return value(); } private static class Numeric extends AlphaNumericString { long value; Numeric(long value) { this.value = value; protected String value() { return Long.toString(value, Character.MAX_RADIX); public int hashcode() { public boolean equals(object o) { } } private static class Alpha extends AlphaNumericString { byte[] value; Alpha(byte[] value) {this.value = value; protected String value() { return new String(value, UTF8); public int hashcode() { public boolean equals(object o) { } } }

106 String Compression: Large Strings Image source:

107 Gzip Become the master of your strings! String Compression: Large Strings Image source:

108 bzip2 Gzip Become the master of your strings! String Compression: Large Strings Image source:

109 bzip2 Just convert to byte[] first, then compress Gzip Become the master of your strings! String Compression: Large Strings Image source:

110 JVM Tuning

111 JVM Tuning Image srouce:

112 make sure to use compressed pointers (-XX:+UseCompressedOops) JVM Tuning Image srouce:

113 make sure to use compressed pointers (-XX:+UseCompressedOops) use low pause GC (Concurrent Mark Sweep, G1) JVM Tuning This s#!% is heavy! Image srouce:

114 make sure to use compressed pointers (-XX:+UseCompressedOops) use low pause GC (Concurrent Mark Sweep, G1) JVM Tuning Adjust generation sizes/ratios Overprovision heap by ~30% This s#!% is heavy! Image srouce:

115 JVM Tuning

116 JVM Tuning

117 JVM Tuning Print garbage collection

118 JVM Tuning Print garbage collection If GC pauses still prohibitive then consider partitioning

119 In summary

120 Know your Business

121 Know your data

122 Know when to optimize

123 The End! My company: My website: And much of the credit for this talk goes to Leon Stein for developing the technology. Thank you, Leon.

124 How do you load the data?

125 Cache Loading webapp full cache data loader a d b al a n c er webapp full cache data loader reliable file store (S3) webapp full cache data loader cooked datasets

126 Cache loading tips & tricks

127 Final datasets should be compressed and stored (i.e. S3) Cache loading tips & tricks

128 Final datasets should be compressed and stored (i.e. S3) Keep the format simple (CSV, JSON) Help! I am in the trunk! Cache loading tips & tricks

129 Final datasets should be compressed and stored (i.e. S3) Keep the format simple (CSV, JSON) Poll for updates Poll frequency == data inconsistency threshold Help! I am in the trunk! Cache loading tips & tricks

130 Cache Loading Time Sensitivity

131 Cache Loading: low time Sensitivity Data /tax-rates /date= tax-rates csv.gz /date= tax-rates csv.gz /date= tax-rates csv.gz

132 /prices /full Cache Loading: medium/high time Sensitivity /date= price-obs csv.gz /date= /inc /date= T csv.gz T csv.gz

133 Cache Loading Strategy Swap Image src:

134 Cache is immutable, so no locking is required Cache Loading Strategy Swap Image src:

135 Cache is immutable, so no locking is required meow Cache Loading Strategy Swap Works well for infrequently updated data sets And for datasets that need to be refreshed each update Image src:

136 Cache Loading Strategy: CRUD

137 Deletions can be tricky YARRRRRR! Cache Loading Strategy: CRUD

138 Avoid full synchronization Deletions can be tricky YARRRRRR! Cache Loading Strategy: CRUD

139 Avoid full synchronization YARRRRRR! Deletions can be tricky Consider loading cache in small batches. Use one container per partition Cache Loading Strategy: CRUD

140 Concurrent Locking with Trove Map public class LongCache<V> { private TLongObjectMap<V> map = new TLongObjectHashMap<V>(); private ReentrantReadWriteLock lock = new ReentrantReadWriteLock(); private Lock r = lock.readlock(), w = lock.writelock(); public V get(long k) { r.lock(); try { return map.get(k); } finally { r.unlock(); } } public V update(long k, V v) { w.lock(); try { return map.put(k, v); } finally { w.unlock(); } } public V remove(long k) { w.lock(); try { return map.remove(k); } finally { w.unlock(); } } }

141 Cache loading optimizations

142 I am cooking the data sets. Ha! Periodically generate serialized data/state Keep local copies Cache loading optimizations

143 I am cooking the data sets. Ha! Periodically generate serialized data/state Validate with CRC or hash Keep local copies Cache loading optimizations

144 Dependent Caches instance product summary load balancer health check status aggregator (servlet) offers dependencies matching predictions

149 Deployment Cell Status deployment cell webapp status aggregator load balancer health check cell status aggregator HTTP or JMX 1 status aggregator 2 status aggregator

150 Hierarchical Status Aggregation