CloudRank-D:A Benchmark Suite for Private Cloud Systems

Transcription

1 CloudRank-D:A Benchmark Suite for Private Cloud Systems Jing Quan Institute of Computing Technology, Chinese Academy of Sciences and University of Science and Technology of China HVC tutorial in conjunction with The 19th IEEE International Symposium on High Performance Computer Architecture () INSTITUTE OF COMPUTING TECHNOLOGY 1

2 Contents Background & Motivation Introduction of CloudRank-D Use cases

3 Contents Background & Motivation Introduction of CloudRank-D Use cases

4 What is Private Cloud? Private Cloud The cloud infrastructure is provisioned for exclusive use by a single organization comprising multiple consumers (e.g., business units). It may be owned, managed, and operated by the organization, a third party, or some combination of them, and it may exist on or off premises. "The NIST Definition of Cloud Computing" National Institute of Standards and Technology. Retrieved 24 July

5 Typical Data Processing Application Recommender systerm Social Network Search Engine Client Front End Scheduler Job Production Job Deployment Hadoop Master Node Framework MapReduce Jobs HDFS Job flow Node Node Node

6 User Concerns Xeon Xeon Xeon Xeon How to quantitatively measure systems? Which one is better (ranking systems)? How to guide optimization? Atom Atom Atom Atom

7 What is CloudRank-D? CloudRank-D Private cloud systems Ranking systems Data processing General Description CloudRank-D is a benchmark suite, used to evaluate private cloud systems that is shared for running data processing applications.

8 Why CloudRank-D? Benchmark MineBench GridMix HiBench WL suite CloudRank-D Target of Evaluation Data mining algorithms Hadoop framework Hadoop framework Hadoop framework The whole system

9 Our Focus: Evaluating the Whole System Applications (Data analysis) Applications (Data analysis) Framework (Hadoop) System platform vs Hadoop Performance Default framework (Hadoop) System platform Performanc of Software & Hardware GridMix etc. CloudRank-D

10 Comparison of Different Benchmarks Suites Mine- Bench Grid- Mix HiBench WL suite CloudSuite CloudRank-D Basic operations n y y y n y Classification y n y n y y Representative applications Clustering y n y n n y Recommendation Sequence learning n n n n n y y n n n n y Association rule mining Data warehouse operations y n n n n y n n n y n y

11 Comparison of Different Benchmarks Suites(Cont') Workloads description Submission pattern Scheduling strategies System software configuration MineB ench Grid Mix HiBench WL suite CloudSuite CloudRank-D n n n y n y n n n n n y n n n n n y Data models n n n n n y Data semantics Scalable data size Category of datacentric computation n n n n n y y y n y n y n n n y n y

12 Contents Background & Motivation Introduction of CloudRank-D Methodology Use cases

13 CloudRank-D Methodology Workloads with usage patterns System platform running Get the peak system performance Ⅰ.Measure systems Ⅱ.Find a suitable system Ⅲ.Optimize systems Performance reports feedback

14 Configurable Workloads with Tunable Usage Patterns Scalable applications and input datasets Representive applications domains User specific Scalable data size Tunable submission patterns Modeling production system logs Configurable runtime system Experiences from industry and academic

15 CloudRank-D Methodology: Workloads with Usage Patterns Scalable applications and input data sets Tunable submission patterns Configurable framework Usage patterns

16 Scalable Applications and Input Data Sets Scalable applications and input data sets Submitted jobs composed of appropriate applications Expanded data sets

17 NO. Category Application Data size Data semantics 1 sort basic 2 word count operation 3 grep 4 5 Applications and Input Data Sets classification naive bayes support vector machine scalable (scale to 10PB) automatically generated Scientist Search 6 cluster k-means scalable sougou corpus 7 recommenda tion Item based collaborative filtering scalable ratings on movies

18 Applications and Input Data Sets (Cont') NO. Category Application Data size Data semantics association rule mining sequence learning frequent pattern growth hidden morkov model grep select 11 ranking select warehouse 12 operation aggregation uservisits-ranking 13 join fixed scalable retail market basket data click-stream data, traffic accident data, collection of web html documents Scientist Search automatically generated table You can add any applications you want!

19 Applications Combinations Demonstration WebCrawling DataMining MachineLearning ImageProcessing TextIndexing LogProcessing Naive Bayes & SVM HMM & IBCF & FPG Basic Operations 35% 31% wiki.apache.org/hadoop/poweredby Reporting DataStorage Hive 34%

20 Data Set Sizes Demonstration Map Number Percentage Size < % 128MB~1.25GB 10~ % 1.25GB~62.5GB 500~ % 63.5GB~250GB > % 250GB~ Workload Characterization on a Production Hadoop Cluster: A Case Study on Taobao

21 Workloads with usage patterns Scalable applications and input data size Tunable submission patterns Configurable framework Usage patterns

22 Submission Patterns Submission patterns Submission intervals Submission orders

23 Submission Intervals Form the Facebook report, distribution of inter-arrival times was roughly exponential with a mean of 14 seconds. Ddelay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In Proceeding In Proceedings of the 5th European conference on Computer systems. Probability density function

24 Submission Orders For the workloads with different resource sizes and different catelogs Submitting jobs randomly Submitting jobs with batch model

25 Workloads with usage patterns Scalable applications and input data size Tunable submission patterns Configurable framework Usage patterns

26 Hadoop Configurations Dimensions Map/Reduce Number Scheduling Policy Main Parameters Explanation affect system utilization Hadoop chooses which job to run according to this policy mapred.tasktracker.map.tasks.maximum mapred.tasktracker.reduce.tasks.maxmum mapred.child.java.opts dfs.block.size

27 Hadoop Settings Parameter Mapred.tasktracker.tasks.r educe.maximum dis.block.size Map (adjust through the block size) Value usually, this value is equal to the core number of current node default value is 64M, you can change it to ensure there won't be too much map number for most workloads 10~100 per node, and it's would be better if the execution time was more than 1min

28 Scheduling Policy Common schedule algorithms First input first out Fair-share scheduler Capacity scheduler Fair-share scheduling can do a good job Workload Characterization on a Production Hadoop Cluster: A Case Study on Taobao

29 Focus CloudRank-D methodology: Our metrics From user perspective Easy to compare and understand Metrics Data processed per second or joule How to get it? DPS=Total data input size/total run time DPJ=Total data input size/total energy consumption

30 Contents Background & Motivation Introduction of CloudRank-D Use cases

31 How to use? CloudRank-D

32 Use Case 1: Comparing Two Hardware Platforms Xeon Xeon Cluster 1 Cluster 2 Xeon Xeon Atom Atom Atom Atom Two clusters comprise 128 nodes respectively.

33 Step 1 Prepared hardware platform Step 1 Step 2 Customize workloads Build foundation platform Procedures Step 3 Run workloads Step 4 Get results and optimize systems

34 Base Information Evaluating two private cloud systems Using all workloads we provide Deploying uniform software platform Adopting same configuration

35 Software Configuration Hadoop version software stack Hive version Mahout version 0.6 map/reduce slot Hadoop system configuration Hadoop scheduling algorithm 4 map slots and 2 reduce slots default fair schedule

36 Run your workloads Job Submission Patterns You can submit the workloads according to the exponential distribution with a specified mean submission interval seconds Submission order : Random

37 An example of result Total data processed per second (KB/S) Total data processed per joule (KB/J) Xeon Xeon Atom Atom Xeon less time, more energy Atom more time, less energy The comparion between Xeon Atom on two metrics

38 Tuning the interval Optimized (Cont') We can see that the best performance occurred when the interval value is 70.

39 Use Case 2: Scheduling Evaluation I have designed a new Hadoop scheduling algorithm, but I don t have the workloads for test. How to evaluate the scheduling? And let people trust the evaluations results. 39/

40 Using CloudRank-D Step 1 Building foundation platform with different scheduling policy Step 1 Build foundation platform Step 2 Customizing workloads with productive scenarios Step 3 Running workloads Step 4 Getting the metrics under different scheduling policy

41 Our Result Total data processed per second (KB/S) Fair scheduler FIFO scheduler Total data processed per joule (KB/J) 5 0 Fair scheduler FIFO scheduler We can see that fair scheduler works better than FIFO scheduler.

42 Contact us Websit:

43 Thanks