Benchmark Databases for Testing Big-Data Analytics In Cloud Environments

Transcription

1 North Carolina State University Graduate Program in Operations Research Benchmark Databases for Testing Big-Data Analytics In Cloud Environments Rong Huang Rada Chirkova Yahya Fathi ICA CON 2012 April 20, 2012

2 Background One major advantage of using computing clouds lies in their applicability to large-scale data warehousing and analytics. Computing clouds can host very large amounts of data and provide efficient parallelized processing of complex analytics queries on the data. Enterprise data-cloud solutions for large-scale data warehousing and analytics are highly desirable. Our goal is to provide synthetically generated benchmark databases for testing the performance and other processing aspects of database systems in a computing-cloud environment. 2

3 Relational Storage of Data Pos: itemid storeid date amount Items: itemid name category 3

4 Query Processing Q: Give me total sales Give me recent total sales for all products in the Bay by store Area ID for all appliances 4

5 Query Processing SELECT storeid, SUM(amount) FROM pos P, items I WHERE P.itemID = I.itemID AND category = 'appliances GROUP BY storeid; storeid SUM(amount) $27, $54, $41,

6 Queries and Views V: total sales by store ID and by item category Q: Give me total sales by store ID for all appliances storeid SUM(amount) $27, $54, $41, storeid category SUM(amount) appliances $27, clothing $45, electronics $50, appliances $54, clothing $60, electronics $82, appliances $41,

7 View lattice Views with grouping and aggregation on a given relation. Given a -attribute dataset, the number of views is 2. Measure the size of each view by its number of rows. {a,b,c,d} 25 {a,b,c} {a,b,d} {a,c,d} {b,c,d} {a,b} 7 {a,c} {b,c} {b,d} {c,d} {b} {c} 4 5 7

8 TPC-H Datasets The TPC-H synthetic database generator is widely recognized as a standard benchmark database generator for data analytics. We have discovered in our work, the TPC-H benchmark has potential shortcomings when used to test the quality of algorithms developed for efficient processing of complex analytics queries. The TPC-H dataset does not distinguish between view sizes. 8

9 The Potential Shortcomings of TPC-H datasets The size of a great number of views is close to that of the largest view. Total number of attributes Number of views within 0.1% size difference from the largest view Total number of views Ratio 300, , % 13 6,192 8, % 15 27,318 32, % , , % 300,000 We would always prefer to store the largest view

10 Our Contribution We define three types of synthetic datasets, which do not have the shortcomings that we have observed in the TPC-H data. We introduce algorithms for generating all the three types of datasets in any range of data sizes, which allows one to use the datasets in a variety of configurations and scales of cloud environments. Our datasets are complementary to the TPC-H datasets in testing the processing performance of complex analytics queries in the cloud environments. 10

11 The Symmetric Synthetic Datasets : total number of attributes in the dataset : number of values for each attribute Number of rows: 3, 2 A B C Example 1: 3, , Attributes: A B C Number of rows:

12 Views in the Symmetric Synthetic Datasets The size of each -attribute view over, is. The size of an ancestor is at least times the size of its descendant. Example 1 (cont d): 3, 2 {A,B,C} 8 {A,B} {A,C} 4 4 {B,C} 4 {A} 2 {B} 2 {C} 2 12

13 Symmetric Synthetic Datasets Symmetric properties of the datasets Significant size difference between each pair of ancestor-descendant views. The datasets does not distinguish between the sizes of the views with same number of attributes. 13

14 Type I Non-Symmetric Synthetic Datasets, The number of values of each attribute differs:,,, Example 2: 3; 2, 3, 4 3, 2, 3, 4 Attributes: A B C Number of rows: A B C ; 2, 3,

15 Views in the Type I Non-Symmetric Synthetic Datasets A -attribute view,,, with values,,, The size (number of rows): Example 2 (cont d): 3; 2, 3, 4 {A,B,C} 24 {A,B} {A,C} 6 8 {B,C} 12 {A} 2 {B} 3 {C} 4 15

16 Type I Non-Symmetric Synthetic Datasets Type I non-symmetric synthetic dataset distinguishes between any pair of view sizes. Relatively large difference in size between each pair of ancestordescendant views. The size of each view is at least twice of the size of its descendant. We would always prefer to store the answer of each query. 16

17 Type II Non-Symmetric Synthetic Datasets Objectives: break the symmetric properties and reduce the size difference between adjacent ancestor-descendant pair of views. Conduct an elimination procedure over the rows in a given type I non-symmetric synthetic dataset. For each attribute, we conduct a two-step sub-elimination process Step 1: Eliminate each row with probability Step 2: For each row r that is eliminated in step 1, we also eliminate the rows in the master table with the same values as r of all attributes except 17

18 Type II Non-Symmetric Synthetic Datasets Input: ;,, and Choose,,, such that Output: a type II non-symmetric synthetic dataset, such that the expected number of rows in is greater than or equal to 18

19 Type II Non-Symmetric Synthetic Datasets Example 3: Input 3; 2, 3, 4 and 10. Choose 0.9 A B C ; 2, 3, 4 A B C A B C

20 Views in the Type II Non-Symmetric Synthetic Datasets Example 3 (cont d): {A,B,C} 10 {A,B} {A,C} 5 5 {B,C} 8 {A} 2 {B} 3 {C} 4 20

21 Experimental Results A performance measure % % % % % Type I non-symmetric dataset TPC-H dataset % 0.00% β 21

22 Conclusion We define a symmetric synthetic dataset and two types of nonsymmetric synthetic datasets. We studied shortcomings of the TPC-H datasets in testing algorithms devised for improving query-processing performance for complex queries posed on large-scale data. We compare these datasets experimentally with our proposed synthetic datasets in a setting for testing in such algorithms. All the synthetic datasets that we proposed in this paper are beneficial for testing algorithms devised for improving queryprocessing performance in cloud computing 22

23 Thank You! 23