Introduction to SQLIO & SQLIOSim & FIO. XLVIII Encontro da Comunidade SQLPort

Size: px

Start display at page:

Download "Introduction to SQLIO & SQLIOSim & FIO. XLVIII Encontro da Comunidade SQLPort"

Natalie Johns
8 years ago
Views:

1 Introduction to SQLIO & SQLIOSim & FIO XLVIII Encontro da Comunidade SQLPort

2 Twitter twitter.com/klunkysql Presenter: André Batista DBA - Crossjoin Solutions. PTSi PT Sistemas de Informação LinkedIn linkedin.com/in/andrebatista0 Thank you for being here today 18 st of March, 2014 Lisbon, Portugal

3 AGENDA overview 1 Backup Operation 2 SQLIOSim 3 SQLIO 4 A little bit of FIO 5 Links

4 WHAT!?!?!? BACKUP operation, is this real? ( on a production \ pre-production server )

$production \$

5 Lets start a normal backup! 2GB database.

6 Backup finished in 2 minutes. Let s look at output stats! Not bad right?

7 Let s make a new Backup of Leadtrack in pre-production environment Should be 2 minutes in average, right? Let s start backup.

8 1 minute and still no output

9 2 minutes and just 10%?!

10 After 10 minutes it s just 40% completed what s happening?

11 Do we have a monkey switching diskettes on a floppy drive?

12 NO we have some problem on storage!!!

13 SQLIOSim tool to perform reliability and integrity tests on disks subsystems

14 SQLIOSim Overview SQLIOSim simulates tests of read, write, checkpoint, backup, sort, read-ahead activities on SQL Server. SQLIOSim is not a benchmark or a tool to determine I/O capacity of storage system!!! For this we should use SQLIO ( later on session ) SQLIOSim replaces the SQLIOStress utility, formely known as SQL70IOStress Starting with SQL Server 2008, SQLIOSim is included with the SQL Server product installation. SQLIOSim have 2 executable files, SQLIOSim.com (command-line tool) and SQLIOSim.exe ( graphical application ). Both executable files provide identical simulation capabilities.

!! For this we should use SQLIO ( later on session ) SQLIOSim replaces the SQLIOStress utility, formely known as SQL70IOStress Starting with SQL

SQLIOSim command-line tool http://support.microsoft.

15 SQLIOSim command-line tool for command-line parameters. Located in BINN folder.

16 Possible to edit profiles for the SQLIOSim test, not on graphical application SQLIOSim command-line tool for command-line parameters. Located in BINN folder.

command-line tool http://support.microsoft.

SQLIOSim graphical application http://support.microsoft.

17 SQLIOSim graphical application for command-line parameters. Located in BINN folder.

18 SQLIOSim ( with SQLIOSimParser ) graphical application DEMO

19 Some warnings under stress. SQLIOSim understanding and interpreting results

20 Example of a output file: SQLIOSim understanding and interpreting results If the average I/O duration exceeds the target I/O duration, SQLIOSim utility throttles the number of outstanding I/O operations, to decrease the load and to improve I/O Target IO Duration ( ms ) = 100 Average IO Duration ( ms ) = 26 Good

operations, to decrease the load and to improve I/O Target IO Duration ( ms ) = 100 Average IO

21 Example of a output file: SQLIOSim understanding and interpreting results Running AVG. IO duration ( ms ) = 26, should be max 2ms

22 Example of a output file: SQLIOSim understanding and interpreting results In this parameter we have to take into account multiple tests on different platforms with the same duration. Low values are great because the IO is not being "discarded, its completed in less than 15s.

23 SQLIO tool to perform benchmark tests and to determine I/O capacity on the storage system

24 SQLIO Overview SQLIO should not have SQL in description, is not SQL related. SQLIO is useful to determine the hardware s I/O capacity and to identify performance-related issues before deploy. SQLIO just have a command-line tool ( there is no graphical tool available ) SQLIO is recommended to be executed against LUN s that will contain data-files and logfiles ( both user db s and tempdb ). We can use this SQLIO to test various I/O types and sizes against several disk subsystems ( DAS, NAS, SAN, ) SQLIO is not installed with SQL Server, we need to download it from Microsoft Download Center.

25 SQLIO modified folder

26 SQLIO command-line tool

27 SQLIO batch example -k[w] -t[2] -s[120] -d[f] -o[1,2,4, ] -f[random] Write I/O 2 threads Running for 120 seconds We going to use the file in parameter_file.txt Depth for outstanding I/O aka HBA queue depth Random operations, in this case random writes -b[64] I/O block size ( let s play with extents ) -B[H] Hardware buffer

28 SQLIO batch example -L[S] Gather latencies from SYSTEM -F[<param>] Specify location of parameter_file

29 SQLIO ( with Jonathan Kehayias PowerShell RegEx Parser ) command-line application DEMO

30 Example of a output file: SQLIO understanding and interpreting results Let s ETL this data on Jonathan Kehayias PowerShell parser

31 Example of a output file: SQLIO understanding and interpreting results IOPS spreadsheet

32 Example of a output file: SQLIO understanding and interpreting results MB/s spreadsheet

33 SQLIO understanding and interpreting results This output is only interested in the values of IOPS, MB \ sec and latency in storage. There are applications (Microsoft Dynamics, Microsoft CRM, Microsoft SharePoint, for example,...) that have recommended values for the infrastructure where they will be installed throughout the solution. This analysis becomes more complicated and should always be reviewed with the teams responsible (remember we are not testing SQL Server transactions, but the own storage metrics infrastructure ). Usually storage teams have reference values for each test and for each respective SAN to validate the values that we get are normal on the infrastructure. Each SAN has recommendations by the manufacturer and these values must be confronted with the results we get from the machine where it is being done the test... One of the problems we've ever come across, are with " broken-fibers " and that the problem is not the machine but the physical patch.

34 FIO I/O tool meant to be used both for benchmark and stress/hardware verification

35 FIO overview FIO is a open-source application, so we can customize for our specific environment. FIO compared to SQLIO and SQLIOSim, does not rely on zeroing the blocks of the test files. Most modern storage systems identify this pattern and compresses the IO generated. FIO uses random data to populate the datasets, so it is agnostic to zero-block detection. It has all of the capabilities of SQLIO tool. More information about FIO on links section.

36 How to use the SQLIOSim utility to simulate SQL Server activity on a disk subsystem ( Download + Configuration ) SQLIOSim links SQLIOSIMParser The SQL Server Sqliosim Utility Robert Sheldon Understanding SQLIOSIM Output Kevin Kline Interpreting Output from SQLIOSIM Kevin Kline

37 Download SQLIO Disk Subsystem Benchmark Tool SQLIO links The SQL Server Sqlio Utility Robert Sheldon SQLIO Analyzer David Klee ( currently not working ) SQLIO.rdl solution Allen McGuire Parsing SQLIO Output to Excel Charts using Regex in PowerShell Jonathan Kehayias Parse SQLIO Results to Excel Graphs with Powershell swaddc

38 FIO links FIO main page FIO project FIO Parser Khailey SQLIO vs FIO

39 Questions Please don t hesitate!

40 Twitter twitter.com/klunkysql Presenter: André Batista DBA - Crossjoin Solutions. PTSi PT Sistemas de Informação LinkedIn linkedin.com/in/andrebatista0 for your precious time and attention 18 st of March, 2014 Lisbon, Portugal

Executive Summary. Methodology

Executive Summary. Methodology Executive Summary This document describes test procedures for Diskeeper 2011 evaluation. Specifically, the tests described in this document are intended to achieve the following objectives: 1. Evaluate