Load Balancing for Distributed Stream Processing Engines Muhammad Anis Uddin Nasir EMDC 011-13
About me Ex EMDC from Batch 011 (the party batch) Currently PhD Student at KTH Royal Institute of Technology Under supervision of Šarūnas Girdzijauskas Internship Ex-intern at Yahoo Labs Barcelona Intern at Aalto University
About me 3
About me 4
About me 5
About me 6
About me 7
About me 8
Why PhD? Freedom Work timings Set of problems Flexibility Conferences Internships 9
Research Interest Stream Processing Social Network Analysis Distributed Systems Decentralized Online Social Networks
Stream Processing Engines Streaming Application Online Machine Learning Real Time Query Processing Continuous Computation Streaming Frameworks Storm, Borealis, S4, Samza, Spark Streaming 11
Stream Processing Model Streaming Applications are represented by Directed Acyclic Graphs (DAGs) Source Data Stream Source Operators Data Channels 1
Stream Grouping Key or Fields Grouping Hash-based assignment Stateful operations, e.g., page rank, degree count Shuffle Grouping Round-robin assignment Stateless operations, e.g., data logging, OLTP 13
Key Grouping Key Grouping Scalable Low Memory Load Imbalance 14
Shuffle Grouping Shuffle Grouping Load Balance Memory O(W) Aggregation O(W) 15
Problem Formulation Input is a unbounded sequence of messages from a key distribution Each message is assigned to a worker for processing Load balance properties Memory Load Balance Network Load Balance Processing Load Balance Metric: Load Imbalance 16
Partial Key Grouping (PKG) Key Splitting Split each key into two server Assign each instance using power of two choices Local Load Estimation each source estimates load on workers using the local routing history 17
Partial Key Grouping (PKG) Key Splitting Local Load Estimation 00 1 Source 1 0 Source 18
Partial Key Grouping (PKG) Key Splitting Distributed Stateless Handle Skew Local load estimation No coordination among sources No communication with workers 19
Partial Key Grouping PKG Load Balance Memory O(1) Aggregation O(1) 0
Streaming Applications Most algorithms that use Shuffle Grouping can be expressed using Partial Key Grouping to reduce: Memory footprint Aggregation overhead Algorithms that use Key Grouping can be rewritten to achieve load balance 1
Stream Grouping: Summary Grouping Pros Cons Key Grouping Scalable Memory Load Balance Load Imbalance Shuffle Grouping Partial Key Grouping Memory O(W) Aggregation O(W) Scalable Load Balance Memory O(1) Aggregation O(1)
Experiments What is the effect of key splitting on POTC? How does local estimation compare to a global oracle? How does PKG perform on a real deployment on Apache Storm? 3
Experimental Setup Metric the difference of maximum and the average load of the workers at time t Datasets Twitter, 1.G tweets (crawled July 01) Wikipedia, M access logs Twitter, 690K cashtags (crawled Nov 013) Social Networks, 69M edges Synthetic, M keys 4
Effect of Key Splitting 5
Local Load Estimation 6
Throughput (keys/s) Real deployment: Apache Storm 1600 1400 100 00 800 600 400 PKG SG 00 KG 0 0 0. 100 600s 300s 60s 10 300s 30s 60s s PKG SG KG 30s s 0.6 00 0 1 0. (a) CPU delay (ms) 600s 6 6. 4. (b) Memory (keys) 6. 7 6
Load Balancing for Distributed Stream Processing Engines Muhammad Anis Uddin Nasir EMDC 011-13
Does PKG work for highly ed data? 9
Skewness Source Source 30
Skewness Source Source 31
Skewness Source Source 3
High level ideas Splitting the high frequency keys on more than two workers Challenges How to decide which are the high frequency keys? How many workers to split the high frequency keys? 33
Solution Divide stream into two components Heavy Hitters Tail Distribution Split Heavy Hitters on all the workers Use Greedy W-Choices or Round Robin for heavy hitters Use vanilla PKG for tail 34
Skewness Source Source 35
Experiments- Threshold 5 workers I(t) (messages) 0-1 - -3-4 -5-6 -7 workers 0 W-Choice / W 1/ W 1/ W 1/4 W 1/8 W 1. W-Choice -1-1 - - - -3-3 -3-4 -4-4 -5-5 -5-6 -6-6 -7-7 -7 5 workers I(t) (messages) W-Choice -1 1. RR 1. 50 workers 0 RR RR -1-1 -1 - - - -3-3 -3-3 -4-4 -4-4 -5-5 -5-5 -6-6 -6-6 -7-7 -7 1. 1. 0-1. 1. 0 workers 0-1 W-Choice workers 0 0 workers 0 50 workers 0-7 RR 1. 36
Experiments- Load Imbalance 5 workers I(t) (messages) workers 0-1 - -3-4 -5 k unique items PKG W-Choice RR -1 - k unique items -1 - -4-5 -5-5 -6 1. 5 workers 0 1. -7 50 workers 0 0k unique items 0k unique items -1-1 -1 - - - - -3-3 -3-3 -4-4 -4-4 -5-5 -5-5 -6-6 -6-6 -7-7 -7 1. 1M unique items -7 1M unique items 1M unique items -1 - -4-4 -4-4 -5-5 -5-5 -7 1. 1. 1M unique items -3-6 -7 - -3-6 1. 0-3 0 workers 0-1 - -3-6 50 workers 0-1 - 1. 0k unique items workers 0-1 5 workers 1. 1. 0 workers 0-1 k unique items workers 0 0k unique items - -7-1 -3-7 -4-7 1. k unique items 0-4 -6-3 -6 0 workers 0-3 I(t) (messages) -6 I(t) (messages) 50 workers 0-6 -7 1. -7 1. 37
Conclusion Partial Key Grouping (PKG) reduces the load imbalance by up to seven orders of magnitude compared to Key Grouping PKG imposes constant memory and aggregation overhead, i.e., O(1), compared to Shuffle Grouping that is O(W) W-Choices improves PKG to achieve nearly perfect load balance for highly ed input 38
Load Balancing for Distributed Stream Processing Engines Muhammad Anis Uddin Nasir EMDC 011-13