Jun Liu, Senior Software Engineer Bianny Bian, Engineering Manager SSG/STO/PAC

Transcription

1 Jun Liu, Senior Software Engineer Bianny Bian, Engineering Manager SSG/STO/PAC

2 Agenda Quick Overview of Impala Design Challenges of an Impala Deployment Case Study: Use Simulation-Based Approach to Design and Optimize an Impala Cluster What s in side: Intel Cofluent Technology for Big Data System Technologies & Optimization (STO) 2

3 Impala Overview Open-ource MPP query execution engine Built natively for Hadoop Efficiently access data stored in Hadoop using SQL Piplined execution mode enables fast data processing speed System Technologies & Optimization (STO) 3

4 Design Challenges of an Impala Cluster H/W Meet Performance Requirements Plan For the Future Not Over Provisioning 10TB 5TB 10 GB 50GB 1TB System Technologies & Optimization (STO) 4

5 Example: Cluster Sizing Requirements: a deep data analytic query over historical data should response within 10 seconds System Technologies & Optimization (STO) 5

6 Example: Storage Choice of One Use Case In general, SSD is faster than HDD, but there re exceptions ~0.0448% System Technologies & Optimization (STO) 6

7 Example: CPU Frequency No impact on the illustrated workload running on the Text formatted table Scaling well when running on the Parquet formatted table System Technologies & Optimization (STO) 7

8 Design Challenges of an Impala Cluster S/W Software Configuration Options HDFS Cache... HDFS Block Size Parquet Row Group Size System Technologies & Optimization (STO) 8

9 Example: HDFS Caching System Technologies & Optimization (STO) 9

10 Design Challenge Summary We have talked about deployment challenges, in terms of: hardware selections and settings Current Approach software configuration choices There s NO ONE SIZE FIT-ALL solution to the design challenges one would face with when deploying a system for production. Efficient Way to Predict System Performance? System Technologies & Optimization (STO) 10

11 Simulation Approach Adjust WL setting Deploy on Experimental Cluster Collect and Analyze System Log Change H/W config Change H/W knobs Simulation Plan Generate Simulation Report System Technologies & Optimization (STO)

12 Impala Simulator Overview Impala Query Execution Simulation Query Planning Flow Plan Nodes, Plan Fragments, Execution Nodes Geneation Task Scheduling and Distribution Data Processing Flow (Pull & Push) Data Distribution (Data Skew and Partitioning) Disk IO Scheduling and Scan Operations Execution nodes System Technologies & Optimization (STO) 12

13 One Banking Use Case Study Offline Customer Account Historical Data Analysis Complex and Deep Analytic Queries Low Latency Interactive Queries Reporting Queries Initially evaluated on Hive, now Impala System Technologies & Optimization (STO) 13

14 Step1: Deploy an Experimental Cluster Deploy a 4-node cluster Small scale of the data System Technologies & Optimization (STO) 14

15 Step2: Collect Simulation Input Hardware Configurations Node Count Processor Storage Network Memory Software Configurations HDFS Impala File Format Table / Column Metadata COMPUTE STATS SHOW TABLE STATS DESC FORMATTED SHOW COLUMN STATS Query Profile - PROFILE Tuple Descriptors Impala Daemon Log System Technologies & Optimization (STO) 15

16 Example: Configure Table Meta Data System Technologies & Optimization (STO) 16

17 Step 3: Baseline Validation on Experimental Cluster System Technologies & Optimization (STO) 17

18 Exchange Execution Node HashJoin Build Phase Aggregation HashJoin Probe Phase Not just query execution time. We also compare with Impala Log File to check the duration of each stage Hdfs Scan Operation disk-io-mgr.cc: disk id (1) reading for... exchange.cc: #rows... instance_id =... Disk Worker 4 Disk Worker 0 System Technologies & Optimization (STO) 18

19 Step 4: From Experimental Cluster to Production Cluster We have completed baseline verification on an experimental cluster Performance prediction for the production cluster Simulation assumptions: upper- and lower- data distribution boundaries small scale of the data System Technologies & Optimization (STO) 19

20 Step 5: Simulation Plan for Production Cluster Software Configuration Matrix Hardware Configuration Matrix File Fo rmat Compr ession Par tition Cache CPU Freq Netw or k Cluster Size Disk Type Text No Compression No Partition No Cache 2.7Gz 1GbE 2 HDD Avro Snappy Partitioned Cached 2.4Gz 10GbE 4 SDD Parquet GZIP Gz System Technologies & Optimization (STO) 20

21 Software Performance Predication System Technologies & Optimization (STO) 21

22 Software Performance Predication > 40GB data to cache System Technologies & Optimization (STO) 22

23 Cache Impact on Text Formatted Data With Cache Without Cache HdfsScanNode finishes at around 6 sec HdfsScanNode finishes at around 12 sec System Technologies & Optimization (STO) 23

24 Cache Impact on Text Formatted Data Block for a short period waiting for RowBatches Execution nodes are busy processing RowBatches System Technologies & Optimization (STO) 24

25 Cache Impact on Parquet Formatted Data With Cache Without Cache Fast Scan, CPU Bound System Technologies & Optimization (STO) 25

26 Cache Impact on Parquet Formatted Data CPU Bound,Scan Speed Does Not Have Impact on Overall Performance of Query Execution. System Technologies & Optimization (STO) 26

27 Software Configuration Recommandation Baseline Text No Compression No Partition No Cache Reporting Workload Avro Snappy Partitioned Cached 1.1% 7.37% 7.94% 4.62% Deep Analytic Workload Parquet GZIP Partitioned Cached 14.45% 2.74% -9.22% 0.49% System Technologies & Optimization (STO) 10x Files to Scan CPU Intensive 27

28 Hardware Performance Predication System Technologies & Optimization (STO)

29 Hardware Performance Predication Network Transfer Cost: MS Baseline Network Transfer Cost: MS 2 Nodes 4 Nodes 6 Nodes 8 Nodes 16 Nodes 20 Nodes System Technologies & Optimization (STO) 29

30 Hardware Performance Predication Expected Response Time System Technologies & Optimization (STO) 30

31 Overall Recommendation 1.8Gz 256 MB No Compression Text 80% 6 HDD 10GbE 4 Nodes Execution Time (Baseline): ~63.3 seconds System Technologies & Optimization (STO) Execution Time (Recommanded): ~12.4 seconds Cluster Size < 4x, 8 Nodes < 10x, 16 Nodes > 10x, 20 Nodes

32 What s Inside System Technologies & Optimization (STO) 32

33 Intel CoFluent Technology for Big Data FASTER CLUSTER DEPLOYMENT: Explore deployment options and meet performance goals OPTIMIZE CLUSTERS: Find performance bottlenecks and optimize software operation SCALE UP WITH CONFIDENCE: Simulate to determine the minimum cost to meet your future demand System Technologies & Optimization (STO)

34 Intel CoFluent Studio Based Simulation Enables fast What if? analysis with a virtual system System Technologies & Optimization (STO)

35 Layered Simulation Architecture S/W Stack Spark M/R HBase HDFS Impala System Topology Role Assignment Build a cluster OS JVM H/W Resource Monitoring and Performance Library CPU Memory Storage Ethernet Dynamic S/W & H/W Mapping Discrete Events Simulation Kernel on SystemC System Technologies & Optimization (STO)

36 Software Stack Coverage YARN System Technologies & Optimization (STO)

37 Hardware Coverage Validated: 50 Nodes SSD & HDD Pooled Compute Pooled Memory Pooled I/O 1GbE & 10GbE Rack Scale Architecture System Technologies & Optimization (STO)

38 Simulation Accuracy High Simulation Accuracy is achieved for Big Data applications running on different cluster size, hardware configurations and software stacks. System Technologies & Optimization (STO) 38

39 Fast Simulation Simulation vs. Real Time in minutes Hardware - 4 node Cluster (min) 71 Simulation Speed - Lenovo T420 (min) Abstract Modeling 36 Event Driven Simulation NUMBER OF CONCURRENT UPLOADING REQUESTS System Technologies & Optimization (STO)

40 Host machine to run simulations System Technologies & Optimization (STO)

41 Call to Actions Visit cofluent.intel.com for more information Request white papers Various customer success stories and use cases available Optimize a 50-node Hive/MR Cluster Predict the scalability of a large HBase Cluster Software Parameter tunings for Spark Applications Demo in the showcase Intel booth System Technologies & Optimization (STO) 41

42 cofluent.intel.com System Technologies & Optimization (STO)

43 Legal Notices and Disclaimers Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. Statements in this document that refer to Intel s plans and expectations for the quarter, the year, and the future, are forward-looking statements that involve a number of risks and uncertainties. A detailed discussion of the factors that could affect Intel s results and plans is included in Intel s SEC filings, including the annual report on Form 10-K. The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. Intel, CoFluent, Xeon, and the Intel logo are trademarks of Intel Corporation in the United States and other countries. *Other names and brands may be claimed as the property of others Intel Corporation. System Technologies & Optimization (STO)

44 Risk Factors The above statements and any others in this document that refer to plans and expectations for the second quarter, the year and the future are forwardlooking statements that involve a number of risks and uncertainties. Words such as "anticipates," "expects," "intends," "plans," "believes," "seeks," "estimates," "may," "will," "should" and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions also identify forward-looking statements. Many factors could affect Intel's actual results, and variances from Intel's current expectations regarding such factors could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be important factors that could cause actual results to differ materially from the company's expectations. Demand for Intel's products is highly variable and could differ from expectations due to factors including changes in business and economic conditions; consumer confidence or income levels; the introduction, availability and market acceptance of Intel's products, products used together with Intel products and competitors' products; competitive and pricing pressures, including actions taken by competitors; supply constraints and other disruptions affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Intel's gross margin percentage could vary significantly from expectations based on capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; changes in revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; excess or obsolete inventory; changes in unit costs; defects or disruptions in the supply of materials or resources; and product manufacturing quality/yields. Variations in gross margin may also be caused by the timing of Intel product introductions and related expenses, including marketing expenses, and Intel's ability to respond quickly to technological developments and to introduce new products or incorporate new features into existing products, which may result in restructuring and asset impairment charges. Intel's results could be affected by adverse economic, social, political and physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Results may also be affected by the formal or informal imposition by countries of new or revised export and/or import and doing-business regulations, which could be changed without prior notice. Intel operates in highly competitive industries and its operations have high costs that are either fixed or difficult to reduce in the short term. The amount, timing and execution of Intel's stock repurchase program could be affected by changes in Intel's priorities for the use of cash, such as operational spending, capital spending, acquisitions, and as a result of changes to Intel's cash flows or changes in tax laws. Product defects or errata (deviations from published specifications) may adversely impact our expenses, revenues and reputation. Intel's results could be affected by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust, disclosure and other issues. An unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular business practices, impacting Intel's ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. Intel's results may be affected by the timing of closing of acquisitions, divestitures and other significant transactions. A detailed discussion of these and other factors that could affect Intel's results is included in Intel's SEC filings, including the company's most recent reports on Form 10-Q, Form 10-K and earnings release. Rev. 4/14/15 System Technologies & Optimization (STO)