Introduction to SQLIO & SQLIOSim & FIO XLVIII Encontro da Comunidade SQLPort
Email andre.batista@cross-join.com Twitter twitter.com/klunkysql Presenter: André Batista DBA - Crossjoin Solutions. PTSi PT Sistemas de Informação LinkedIn linkedin.com/in/andrebatista0 Thank you for being here today 18 st of March, 2014 Lisbon, Portugal
AGENDA overview 1 Backup Operation 2 SQLIOSim 3 SQLIO 4 A little bit of FIO 5 Links
WHAT!?!?!? BACKUP operation, is this real? ( on a production \ pre-production server )
Lets start a normal backup! 2GB database.
Backup finished in 2 minutes. Let s look at output stats! Not bad right?
Let s make a new Backup of Leadtrack in pre-production environment Should be 2 minutes in average, right? Let s start backup.
1 minute and still no output
2 minutes and just 10%?!
After 10 minutes it s just 40% completed what s happening?
Do we have a monkey switching diskettes on a floppy drive?
NO we have some problem on storage!!!
SQLIOSim tool to perform reliability and integrity tests on disks subsystems
SQLIOSim Overview SQLIOSim simulates tests of read, write, checkpoint, backup, sort, read-ahead activities on SQL Server. SQLIOSim is not a benchmark or a tool to determine I/O capacity of storage system!!! For this we should use SQLIO ( later on session ) SQLIOSim replaces the SQLIOStress utility, formely known as SQL70IOStress Starting with SQL Server 2008, SQLIOSim is included with the SQL Server product installation. SQLIOSim have 2 executable files, SQLIOSim.com (command-line tool) and SQLIOSim.exe ( graphical application ). Both executable files provide identical simulation capabilities.
SQLIOSim command-line tool http://support.microsoft.com/kb/ 231619 for command-line parameters. Located in BINN folder.
Possible to edit profiles for the SQLIOSim test, not on graphical application SQLIOSim command-line tool http://support.microsoft.com/kb/ 231619 for command-line parameters. Located in BINN folder.
SQLIOSim graphical application http://support.microsoft.com/kb/ 231619 for command-line parameters. Located in BINN folder.
SQLIOSim ( with SQLIOSimParser ) graphical application DEMO
Some warnings under stress. SQLIOSim understanding and interpreting results http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx.
Example of a output file: SQLIOSim understanding and interpreting results If the average I/O duration exceeds the target I/O duration, SQLIOSim utility throttles the number of outstanding I/O operations, to decrease the load and to improve I/O Target IO Duration ( ms ) = 100 Average IO Duration ( ms ) = 26 Good http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx.
Example of a output file: SQLIOSim understanding and interpreting results Running AVG. IO duration ( ms ) = 26, should be max 2ms http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx.
Example of a output file: SQLIOSim understanding and interpreting results In this parameter we have to take into account multiple tests on different platforms with the same duration. Low values are great because the IO is not being "discarded, its completed in less than 15s. http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx.
SQLIO tool to perform benchmark tests and to determine I/O capacity on the storage system
SQLIO Overview SQLIO should not have SQL in description, is not SQL related. SQLIO is useful to determine the hardware s I/O capacity and to identify performance-related issues before deploy. SQLIO just have a command-line tool ( there is no graphical tool available ) SQLIO is recommended to be executed against LUN s that will contain data-files and logfiles ( both user db s and tempdb ). We can use this SQLIO to test various I/O types and sizes against several disk subsystems ( DAS, NAS, SAN, ) SQLIO is not installed with SQL Server, we need to download it from Microsoft Download Center.
SQLIO modified folder http://www.microsoft.com/enus/download/details.aspx?id=20163
SQLIO command-line tool http://www.microsoft.com/enus/download/details.aspx?id=20163
SQLIO batch example -k[w] -t[2] -s[120] -d[f] -o[1,2,4, ] -f[random] Write I/O 2 threads Running for 120 seconds We going to use the file in parameter_file.txt Depth for outstanding I/O aka HBA queue depth Random operations, in this case random writes -b[64] I/O block size ( let s play with extents ) -B[H] Hardware buffer
SQLIO batch example -L[S] Gather latencies from SYSTEM -F[<param>] Specify location of parameter_file
SQLIO ( with Jonathan Kehayias PowerShell RegEx Parser ) command-line application DEMO
Example of a output file: SQLIO understanding and interpreting results Let s ETL this data on Jonathan Kehayias PowerShell parser
Example of a output file: SQLIO understanding and interpreting results IOPS spreadsheet
Example of a output file: SQLIO understanding and interpreting results MB/s spreadsheet
SQLIO understanding and interpreting results This output is only interested in the values of IOPS, MB \ sec and latency in storage. There are applications (Microsoft Dynamics, Microsoft CRM, Microsoft SharePoint, for example,...) that have recommended values for the infrastructure where they will be installed throughout the solution. This analysis becomes more complicated and should always be reviewed with the teams responsible (remember we are not testing SQL Server transactions, but the own storage metrics infrastructure ). Usually storage teams have reference values for each test and for each respective SAN to validate the values that we get are normal on the infrastructure. Each SAN has recommendations by the manufacturer and these values must be confronted with the results we get from the machine where it is being done the test... One of the problems we've ever come across, are with " broken-fibers " and that the problem is not the machine but the physical patch.
FIO I/O tool meant to be used both for benchmark and stress/hardware verification
FIO overview FIO is a open-source application, so we can customize for our specific environment. FIO compared to SQLIO and SQLIOSim, does not rely on zeroing the blocks of the test files. Most modern storage systems identify this pattern and compresses the IO generated. FIO uses random data to populate the datasets, so it is agnostic to zero-block detection. It has all of the capabilities of SQLIO tool. More information about FIO on links section.
How to use the SQLIOSim utility to simulate SQL Server activity on a disk subsystem ( Download + Configuration ) SQLIOSim links http://support.microsoft.com/kb/231619 SQLIOSIMParser http://sqliosimparser.codeplex.com/ The SQL Server Sqliosim Utility Robert Sheldon https://www.simple-talk.com/sql/database-administration/the-sql-server-sqliosim-utility/ Understanding SQLIOSIM Output Kevin Kline http://sqlblog.com/blogs/kevin_kline/archive/2007/06/28/understanding-sqliosim-output.aspx Interpreting Output from SQLIOSIM Kevin Kline http://sqlblog.com/blogs/kevin_kline/archive/2008/12/14/interpreting-output-from-sqliosim.aspx
Download SQLIO Disk Subsystem Benchmark Tool SQLIO links http://www.microsoft.com/en-us/download/details.aspx?id=20163 The SQL Server Sqlio Utility Robert Sheldon https://www.simple-talk.com/sql/database-administration/the-sql-server-sqlio-utility/ SQLIO Analyzer David Klee ( currently not working ) http://www.davidklee.net/tag/sqlio-analyzer/ SQLIO.rdl solution Allen McGuire http://allen-mcguire.blogspot.pt/search/label/sqlio Parsing SQLIO Output to Excel Charts using Regex in PowerShell Jonathan Kehayias Parse SQLIO Results to Excel Graphs with Powershell swaddc http://sqlblog.com/blogs/jonathan_kehayias/archive/2010/05/25/parsing-sqlio-output-to-excel-chartsusing-regex-in-powershell.aspx https://community.emc.com/people/swaddc/blog/2014/02/03/the-mtc-parse-sqlio-results-to-excelgraphs-with-powershell
FIO links FIO main page http://linux.die.net/man/1/fio FIO project http://freecode.com/projects/fio FIO Parser Khailey https://github.com/khailey/fio_scripts SQLIO vs FIO http://blog.delphix.com/uday/2012/09/19/sqlio_fio/
Questions Please don t hesitate!
Email andre.batista@cross-join.com Twitter twitter.com/klunkysql Presenter: André Batista DBA - Crossjoin Solutions. PTSi PT Sistemas de Informação LinkedIn linkedin.com/in/andrebatista0 for your precious time and attention 18 st of March, 2014 Lisbon, Portugal