Performance Forecasting - Introduction -

Transcription

1 Performance Forecasting - Introduction - Matthias Mann HII3DB HVB Information Services Seite 1

2 Contents 1. Forecasting Brief Eplanation 2. How Forecasting Works Methodology and Models 3. Queueing Theory 4. Data athering and Workload Characterization 5. Methods by Eample Seite 2

3 Forecasting Brief Eplanation 1/1 Forecasting is everywhere: weather, traffic, IT Forecasting risk assessment Forecasting is not Tuning Forecasting as part of SLM: Capacity-, Continuity-, Availability-, Financial Management Performance Forecasting Focus: Does risk eist? What is at risk? When will risk occur? How to minimize the risk? Seite 3

4 Performance Forecasting: Methodology and Models 1/7 (1) Formulate the question of the study e.g.: Workload of the database server is epected to double in the net three months. Will business requirements be met? If not, which areas require attention? (2) ather workload data automated, repeatable use repository application instrumentation? measure all components (OS, DB, Middleware, Application) minimize impact on performance Seite 4

5 Performance Forecasting: Methodology and Models 2/7 (3) Characterize the data make they useful and understandable workload representation: basic unit of work (transaction)? find the baseline (4) develop / choose an appropriate model single / multiple component model input data: application / os / database related? productive / planned system time available for forecasting required precision of forecast Seite 5

6 Performance Forecasting: Methodology and Models 3/7 Common fundamental models (4a) (4b) Ballpark Figures very quick, rough forecasts: Server with 24 B Memory runs 15 DB instances (each with SA/PA). Does it support four more instances? Ratio Modelling quick, low precision relation between process categories (OLTP/Batch) and system resources (CPU) for budgeting appro., architecture validations, sizing of packaged appl. Seite 6

7 Performance Forecasting: Methodology and Models 4/7 (4c) (4d) Linear Regression Analysis typical question: How much of some business activity can occur before the system is running "out of gas"? precise, statistically validated solving of a system of equations Queueing Theory understanding of performance issues for forecasting Seite 7

8 Performance Forecasting: Methodology and Models 5/7 Model Ballpark Ratio Linear Queueing Figures Modelling Regression Theory components - single - multiple Input Data - Application - Infrastructure Usable for - living systems - planned systems Seite 8

9 Performance Forecasting: Methodology and Models 6/7 Model Ballpark Figures Ratio Linear Queueing Modelling Regression Theory Precision - low - high Project Duration - short - long Seite 9

10 Performance Forecasting: Methodology and Models 7/7 (5) Model Validation: determine precision check: numerical error statistical error (average, standard deviation, skewness) histogram analysis residual analysis (areas of good/bad forecast) go / no-go decision (6) Forecast answer the study question Seite 10

11 Queueing Theory: Basics 1/4 most environments are queueing systems: fuelling station restaurant, supermarket, call center Notation: Server (CPU, IO Device) Queue (waiting transactions) Seite 11

12 Queueing Theory: Basics 2/4 Eamples Seite 12

13 Queueing Theory: Basics 3/4 Foundation John D. Little MIT Sloan School of Management Operations Research Traffic Control Marketing Littles Law: avg. # in system (served+queued) = arrival rate times avg time spent in system (resp.time). Seite 13

14 Queueing Theory: Basics 4/4 D.. Kendall, Prof. of Math. Statistics at Cambridge: Kendalls Notation (for a single queue): A / B / m A arrival pattern (distribution) B service time distribution m # of servers in the system computing transactions follow Markovian distribution eponential, each transaction independent from others Eamples: 1 M / M / 12 5 M / M / 1 12 CPU Oracle Server 5 Device IO subsystem Seite 14

15 Queueing Theory: Terms 1/3 Term Symbol Definition Unit transaction basic unit of work tr arrival rate λ # of tr entering the system within a time period tr/ms Service Time S t how long it takes a server to service a transaction ms/tr Queue Length Q # of tr in the queue (goal: 0) tr Utilization U amount of time a resource is in use over a given time interval % Queue Time Q t time a tr waits in the queue until served ms/tr Seite 15

16 Queueing Theory: Terms 2/3 Term Symbol Definition Unit Response Time R t time of the tr in the system: R t =Q t +S t ms/tr # of servers in the system M # of servers per queue m Seite 16

17 Queueing Theory: Terms 3/3 Seite 17

18 Queueing Theory: Basic Formulas 1/3 simplified version: underestimation of response time for IO subsystem: M=1 Seite 18

19 Queueing Theory: Basic Formulas 2/3 Erlang C Mathematics A. K. Erlang: telephone engineer, worked in traffic engineering and queueing theory => Erlang C Unit Traffic of 1 Erlang refers to a single resource being in continuous use. Used in telephony as a statistical measure of telco traffic. Efficiency of call centers (calls / hour, avg. call duration, target answer time, ) Application to computing systems: CPU (single queue): o.k. IO (multiple queue): conversion to multiple single queue / single server system necessary (perfectly balanced IO ssystem across all devices) Seite 19

20 Queueing Theory: Basic Formulas 3/3 Seite 20

21 Data athering and Workload Characterization 1/12 minimize impact on system all data sources have to start / end at the same time Data Sources OS: sar, vmstat, iostat, Application: # of active OLTP sessions / Batch processes # of orders per hour, Seite 21

22 Data athering and Workload Characterization 2/12 Oracle Database: gather at system / session level use dynamic performance views v$sysstat, v$session v$sysstat joined with v$statname v$sess_io, v$mystat Seite 22

23 Data athering and Workload Characterization 3/12 gatherdata.sh # # Settings # ora_access=/ # Login in the Database seq=$(date +%H%M) # Clock Time cpu_file=${fcdir}/dat/wl.cpu.$(uname -n).${y}.${m}.${d}.dat io_file=${fcdir}/dat/wl.io.$(uname -n).${y}.${m}.${d}.dat app_file=${fcdir}/dat/wl.oracle.${oracle_sid}.${y}.${m}.${d}.dat Seite 23

24 Data athering and Workload Characterization 4/12 sqlplus $ora_access <<EOF drop table wl_stats; col value format set linesize 500 create table wl_stats as select * from v\$sysstat where name in (<any statistic#>); eit; EOF Seite 24

25 Data athering and Workload Characterization 5/12 # # Start CPU gathering # sar -u -o $sar_file $DURATION 1 >/dev/null 2>&1 & # # Start IO gathering # iostat -n $DURATION 2 >$iostat_file & # # Wait for CPU and IO gathering to complete wait Seite 25

26 Data athering and Workload Characterization 6/12 # # ather final Oracle workload values and calculate delta activity # sqlplus $ora_access <<EOF >>${log_file} select '$seq' ',' b.name ',' the_line, b.value-a.value value, ',good' from (select name,statistic#,value from v\$sysstat where name in (<any statistic#>)) b, (select name,statistic#,value from wl_stats where name in (<any statistic#>)) a where b.statistic# = a.statistic#; eit; Seite 26

27 Data athering and Workload Characterization 7/12 # # Print general Oracle workload statistics # grep good $work_file grep -v awk -F, '{print $1 "," $2 "," $3}' >>$app_file Seite 27

28 Data athering and Workload Characterization 8/12 crontab entry: 0,5,10,15,20,25,30,35,40,45,50,55 * * * * /opt/oracle/admin/bip6t8/perf/bin/gatherdata.sh -s BIP6T8 -d 300 CPU data collected: (time, %usr, %sys, %wio, %idle) 1345,59,5,1, ,59,12,1, ,57,13,1, ,66,7,1, ,57,5,1,37 Seite 28

29 Data athering and Workload Characterization 9/12 IO data collected: (time, device, selected device statistics) 1345,d40,0.0,0.1,0.2,1.2,1,3 1350,d30,0.0,0.0,6.8,1.0,0,0 1350,d40,0.0,0.0,0.2,0.6,1,2 1355,d30,0.0,0.0,9.8,1.0,0,0 1355,d40,0.0,0.0,0.2,0.4,1,2 1400,d30,0.0,0.0,7.4,0.8,0,0 1400,d40,0.0,0.1,0.1,0.6,1,8 1405,d30,0.0,0.0,6.8,1.1,0,0 Seite 29

30 Data athering and Workload Characterization 10/12 Oracle Data collected: 1345,logons cumulative, ,user calls, ,physical reads, ,physical reads cache, ,physical reads direct, ,physical read IO requests, ,physical read bytes, ,db block changes, ,physical writes, ,physical writes direct, ,physical writes from cache, ,physical write IO requests, ,physical write bytes, ,redo size, ,eecute count, ,OS User level CPU time, ,OS System call CPU time, 0 Seite 30

31 Data athering and Workload Characterization 11/12 define: what is the transaction / workload? single category model: use any suitable column from v$sysstat multiple category model: grouping of Oracle activity in v$sesstat, v$session important: Baseline construction from the gathered data for later forecasts Baseline: ref. point / interval (usually period of highest utilization during business) combine Oracle and OS data Seite 31

32 Data athering and Workload Characterization 12/12 Eample CPU subsystem Oracle: CPU used by this session user calls (Transaction) M from cpu_count => S t =(# tr/(cpu used)) OS: U (sar) => compute λ=um/s t with baseline consideration select avg ((CPU used)/(user calls)) where logoff_time Seite 32

33 Methods by eample: Ratio Modelling 1/4 Cook, Dudar, Shallahamer 1995 works only for one resource type (mostly CPU) P = U M = Σ (C i / R i ) P # of fully utilized CPUs U CPU Utilization C i # of workload occurences (eg. OLTP sess., Batch proc.) R i ratio linking a workload category to a CPU Seite 33

34 Methods by eample: Ratio Modelling 2/4 Eample 18 CPU database server U = 41 % 250 conc. OLTP User : R OLTP = conc. Batch processes: R batch = 2 What is the effect of adding 25 additional OLTP users? P = (250+25)/ /2 = 8.2 = 18 U => U = 46 % => no problem! Seite 34

35 Methods by eample: Ratio Modelling 3/4 Question: Where do the ratios come from? have to be derived taken from similar systems, get from application vendor Batch-to-CPU: shutdown OLTP, approimate OLTP-to-CPU: calculate Seite 35

36 Methods by eample: Ratio Modelling 4/4 Drawback: low precision unvalidated because of simplicity it can be easily misused Seite 36

37 Methods by eample: Queueing Theory 1/6 Situation: company acquisition customers of both companies have to use eisting database system Question: Is database server able to handle the increased workload? Consider: CPU and IO subsystem Prerequisite: data gathering is already active Workload representation: user calls from v$sysstat (tr) Seite 37

38 Methods by eample: Queueing Theory 2/6 data inspection: Baseline is tr/ms OS data: CPU: U = 33%, 32 CPUs IO: U= 45%, 60 devices Seite 38

39 Methods by eample: Queueing Theory 3/6 CPU subsystem baseline: no queuing Seite 39

40 Methods by eample: Queueing Theory 4/6 IO subsystem baseline: significant queueing Seite 40

41 Methods by eample: Queueing Theory 5/6 put formulas in Ecel spreadsheet => produce forecasts Seite 41

42 Methods by eample: Queueing Theory 6/6 Summary IO subsystem is bottleneck Message to management: workload pattern of acquired customer unknown assuming equal pattern: system may be able to support 100% workload increase risk mitigation: better understand workload pattern of acquired company reduce peak workload (Batch time shifts) continue data gathering new forecast soon after new users are using the system Seite 42

43 Seite 43