Comparing two Queuing Network Solvers: JMT vs. PDQ

Transcription

1 Comparing two Queuing Network Solvers: JMT vs. PDQ A presentation for the report of the Course CSI 5112 (W11) Adnan Faisal (CU ) Mostafa Khaghani Milani (CU ) University of Ottawa 25 March 2011

2 Introduction The company & the problem Queuing Networks Evaluation criteria & Methodology General criteria QN theory related criteria Methodology Presenting the tools JMT vs PDQ GRL evaluation Summary and Recommendation

3 The company & the problem Who are PerfTiger Inc.? A company for Performance evaluation and optimization of Computer Systems use Queuing Network techniques small size (10 people) 2 Managers: Strategic decisions 6 Junior Researchers : Data collection, analysis, report making, presentation 2 Senior Researchers: Project supervision, analysis, new ideas...

4 The company & the problem What do they need? A queuing network solver is needed QNAP2 1 : obsolete, lack of user-friendliness and functionalities Two options: 1. JMT (Java Modeling Tools) 2 2. PDQ (Pretty Damn Quick)

5 The company & the problem Problems targeted Bottleneck analysis for single and multi-class traffic Scalability analysis for web application, network etc. Capacity planning for any information system What-if analysis for change prediction of existing systems

6 Queuing Networks A Queuing Station Figure: The arrival rate and service time can take any statistical distribution (e.g., Exponential) and different queue policy (e.g, FCFS, Processor Sharing etc.)

7 Queuing Networks Network of queues Figure: An open Queuing network. We can also have closed, open-closed queuing networks. These kinds of queuing networks can be solved by both JMT and PDQ

8 General criteria Criteria Deployment: Must run on both Windows and Linux in the computers 4 the company has. Learning curve: Our researcher must be able to learn the tool by 2-4 weeks. Usability: It is good if the tool has GUI. Cost: The cost of the software must be less than 1000 dollars Accuracy: the analytic results must be accurate up to 4 decimal points. the simulated results must give Confidence Interval and Maximum Relative Error. 4 Average PerfTiger machines have quad core processor with 4 GB RAM

9 General criteria Criteria Consistency: the simulated results must fall into the same Confidence Interval every time we run a model. Compatibility: the results of the model should be compatible to be exported by MS Excel or any other spreadsheet / statistical computing program. Documentation: the tool must be well documented. Efficiency: for a model with 4 classes and 10 stations, analytic and simulated solvers must solve the model by 1 second and 3 minutes respectively. Ease of profiling: for simulation, the software should allow the logging of the jobs moving in the model.

10 QN theory related criteria Criteria Arrival rate distributions: must support Exponential, Pareto and Constant distribution. Service time distributions: must support Exponential distribution. Queue policy: FCFS must be supported. Good to have other queue policies (e.g., LCFS). Priority class: having priority class (with and without preemption) is good. Class types: Both open and closed classes must be supported.

11 QN theory related criteria Criteria BCMP models: must be able to solve BCMP models analytically. Non-BCMP models: must be able solve models with Fork, Join etc. using approximation or simulation. Load-dependent stations: must be able to solve models with load-dependent service time. What-if analysis: A built-in what-if analyzer would be highly appreciable. Bottleneck identification: Bottleneck analyzer for multi-class model will be a big advantage.

12 Methodology Methodology Identify 3 groups of stakeholders (Managers, Junior Researchers, Senior Researchers) Put weight on each of the criteria for each stakeholder based on previous experience Make 1 GRL model with 2 scenarios for the two tools Find out the tool that satisfies more stakeholders and recommend that tool.

13 JMT vs PDQ Introducing JMT Figure: JMT (developed by Politecnico di Milano) start-up screen shows 5 available tools to choose from.

14 JMT vs PDQ Introducing PDQ Figure: PDQ (developed by Performance Dynamics Company) has no GUI, is not a stand-alone application but a library of functions (written in C) for solving queuing network models.

15 JMT vs PDQ Comparison Criteria JMT PDQ Deployment Runs both in Windows Runs easily in Linux. Requires Cygwin or Virtual- and Linux since developed in Java. box to run on Windows. Learning curve Very easy and intuitive. An scripting langauge (e.g., Perl) and PDQ syntax must be learnt. Usability GUI and XML input. No GUI, script-based input. Cost Free and open-source. Free and open-source. Solver Analytic and simulation. Only analytic.

16 JMT vs PDQ Comparison Criteria JMT PDQ results are correct both Gives correct results. Accuracy for simulation and analytic solvers. Output can be exported Output can be exported Output compatibility to MS Excel via XML. to R package. User manual, book and User manual, book and Documentation online help available. online help available. Efficiency Both analytic and simulation based solvers meet time requirement. The analytic solver meets time requirement. Table: PDQ vs JMT

17 JMT vs PDQ Comparison Criteria JMT PDQ allowed during simulation N/A Profiling Exponential, Constant, Only Exponential. Supported distributions Normal and 8 more. LCFS and FCFS. LCFS and FCFS. Queue policy Supported in simulationproximation. Supported using ap- Priority traffic open, closed, mixed. open, closed, mixed. Traffic types

18 JMT vs PDQ Comparison Load-dependent stations What-if analysis iden- Bottleneck tification Criteria JMT PDQ solved by simulation analytically solved with non-bcmp models approximation. yes and very easy to do! yes Easy to do for both single class and multiclass models (using JABA) yes, but the coding is not intuitive. yes Tricky for multiclass.

19 GRL evaluation Case: Use PDQ Figure: GRL Scenario for choosing PDQ

20 GRL evaluation Case: Use JMT Figure: GRL Scenario for choosing JMT

21 Conclusion Based on our evaluation we suggest PerfTiger to choose JMT as their queuing network solver. Our personal experience was used to give weights to the different softgoals. Details of the evaluation will be given on the final report.

22 Conclusion Based on our evaluation we suggest PerfTiger to choose JMT as their queuing network solver. Our personal experience was used to give weights to the different softgoals. Details of the evaluation will be given on the final report.