Accelerate Oracle Performance by Using SmartCache of T Series Unified Storage This document describes why and how to use SmartCache in Oracle OLTP database to accelerate the transaction performance. SmartCache is an advanced feature of OceanStor T series unified storage, with which hot blocks on traditional hard disks could be detected and migrated to SSD cache pool, and later hit in the pool. Through this technology, the physical read response time and the OLTP throughput could be significantly improved. Using SmartCache in OLTP database, the transaction response time could be commonly reduced to 50% or lower, and the transaction throughput could be increased to 200% or higher. Jarvis WANG wangyaohui@huawei.com IT Storage Solution & Verification, Enterprise BG 2012-8-24 Version 1.0 1
What is SmartCache SmartCache is an advanced feature of OceanStor T series storage, with which hot blocks on traditional hard disks could be found and migrated to SSD cache pool, and later be hit in the pool. The following diagram shows the principle of SmartCache. Hot blocks on traditional hard disks are identified by SmartCache Engine, and then be migrated to SSD cache pool. The hot blocks in SSD cache pool have big chances to be re-read again, in other words, hit in cache pool. The read response time could be significantly improved if there s enough hit ratio. If the data of the cached hot blocks is updated, the new data will be written to hard disks and updated in the cache pool simultaneously. When the cache pool is full and there re new hot blocks detected, old blocks previous loaded into the cache pool will be evicted out using LRU-LIKE algorithm. Oracle Database T Series Storage Cold I/O Hot I/O Hot Block SmartCache Engine HDD RAID Group SSD Cache Pool 2
Why SmartCache As we know, in OLTP (online transactional processing) systems, Oracle is random I/O intensive, and hot areas exist. Besides, the increasing of capacity and performance is not in-phase for enterprise hard disk. The capacity of enterprise hard disk is too big and too slow for OLTP database nowadays. Using SmartCache in Oracle OLTP database, the hot blocks could be cached in SSDs and the physical random read response time could be improved. User could benefit from the low latency, high random IOPS and low power consumption features of SSDs. OLTP bottleneck The following table shows the top 5 wait events in one oracle OLTP database running on traditional magnet disks (10K RPM SAS disk). 96.87% of DB time is spent on waits of db file sequential read, and each wait costs 15 milliseconds. The high latency of the slow magnet disks is definitely the bottleneck of the OLTP system. OLTP Hot Areas The following chart shows read load distribution of a LUN which stores a big table and its indexes. The LBA is spitted to 100 units and read I/O is counted for each unit. We could clearly find the hot areas of the LUN. 3
% Read Load 18 16 14 12 10 8 6 4 2 0 1 11 21 31 41 51 61 71 81 91 Capacity and Performance of Hard Disk The following table shows the capacity and performance change of enterprise hard disk in the last 5 years, the capacity is increased year by year, but the performance stays the same level. When deploying Oracle OLTP database on hard disk, users will buy much more spindles than the capacity needs to suport the high random I/O requirements, while the free capacity can t be used to store other things because that will make the performance of database bad. Tiering storage or pure SSD solution are now strongly recommended in Oracle OLTP database, and T series storage with SmartCache is an advanced technology fit into the requirements. Year Capacity RPM OLTP Throughput OLAP Throughput 2008 73 GB 15 K 200 IO/s 30 MB/s 2009 146 GB 15 K 200 IO/s 30 MB/s 2010 300 GB 15 K 200 IO/s 30 MB/s 2011 600 GB 15 K 200 IO/s 30 MB/s 2012 900 GB 15 K 200 IO/s 30MB/s T Series Storage HUAWEI OceanStor T Series Unified Storage (T Series) is new-generation storage product for mid-range and high-end storage applications. It boasts integration of block-level and file-level data storage, supports for a variety of storage protocols, and GUI-based central storage management. 4
The most important performance related features of T series are DST (dynamic storage tiering) and SmartCache. When using file storage or NAS, DST could be used to dynamically detect hot files and put them to tier1 storage, while cold files which are not frequently visited are left in tier2 storage. When using block storage or SAN, SmartCache could be used to detect hot blocks and put them into SSD cache pools. Oracle database is widely deployed on SAN storage, SmartCache could significantly reduce the physical read response time for oracle database if you choose T series storage. Performance Acceleration The following chart shows the change of transaction response time (New-Order Trans.) and db file sequential read wait time before and after setting SmartCache for oracle OLTP database. The transaction response time is reduced from 247ms to 70ms, reduced to 28%. The physical read wait time is reduced from 15ms to 5ms, reduced to 33%. The end users are significantly benefited from SmartCache and SSDs. 5
Transaction Response Time (ms) Active End Users Transaction Response Time (ms) average read time (ms) db file sequential read New-Order Trans. 300.00 15 16 250.00 200.00 247.02 14 12 10 150.00 8 100.00 50.00 5 70.05 6 4 2 0.00 Base Line - Without Smart Cache With Smart Cache 0 The following chart shows the change of maximum active users and transaction response time before and after setting SmartCache for oracle OLTP database. The maximum active users (Act. Users) are increased from 2000 to 8000, increased to 400%. The transaction response time (Avg. DB RT) is reduced from 247 to 117, reduced to 47%. The IT system could host much more users in the future when setting SmartCache for oracle database. Act. Users Avg. DB RT (ms) 300.00 250.00 200.00 150.00 100.00 50.00 0.00 247.02 2000 Base Line - Without Smart Cache 8000 117.37 With Smart Cache 9000 8000 7000 6000 5000 4000 3000 2000 1000 0 The tests are performed using TPCC-LIKE workload, in which the read ratio is about 60%. Workloads with higher read ratio will benefit more from SmartCache and SSDs. 6
How to Use SmartCache It s very simple to use SmartCache, you don t have to install software on oracle host systems or apply patches for oracle database, and you don t have to migrate any database object, just plug SSDs in the empty slot of T series disk enclosure and enable SmartCache, the transaction response time will be reduced significantly and the transaction throughput will be doubled, trebled or more improved. The following steps introduce how to enable SmartCache for LUNs in ISM (Integrated Storage Management Software) of T series storage. 1. Import and active SmartCache license on ISM Connect T series storage using ISM, in Settings page, click License Management. In License Management dialog, go to Inactivated License tab, click Import button, select the license file and click open button. 7
Click Active button to active the license. 2. Set SmartCache for each controller In ISM SmartCache page, select controller A, click Disk Management button, and then add SSDs to the SmartCache pool A, then click OK. 8
In ISM SmartCache page, select controller B, click Disk Management button, and then add 3 SSDs to the SmartCache pool B, then click OK. 3. Enable SmartCache for LUNs In ISM LUNs page, select the LUNs which is used to store schema objects of OLTP database, click Advanced drop down list, select Enable SmartCache. 9
4. Check size of cached data and hit ratio You can check the SmartCache counters 4 hours after SmartCache is enabled in detailed LUN properties. In ISM LUNs page, select a LUN on which SmartCache is enabled, you will see the detailed properties of the LUN. The counter SmartCache Capacity stands for the amount of cached data in SmartCache, the counter Percentage of SmartCache stands for the percentage of the LUN s cached data in SmartCache, and the counter Hit Ratio of SmartCache stands for the LUN s block hit ratio in SmartCache. 10
5. Done 11
SmartCache Best Practices This chapter introduces best practices when using SmartCache in T series storage. SLC or emlc T series storage supports two kinds of SSDs: SLC (Single-Layer Chip) and emlc (Enterprise Multi-Layer Chip). SLC has much better random write performance and more number of block erase count, but more expensive than emlc. SLC and emlc almost have the same random read performance. SmartCache is read cache, the update data of cached blocks is both written to LUNs and SSD cache pool, so the data will NOT be lost when the SSDs fail. So choosing emlc is a better idea when using SmartCache. Size of SmartCache The maximum size of SmartCache and the maximum size of LUNs with SmartCache enabled are limited by storage controller s memory size. The following table lists the configuration limit of SmartCache. Storage Max. LUN size (TB) Max. SmartCache pool size (GB) S5500T 32 1200 S5600T 32 2400 S5800T 64 3600 S6800T 64 4800 The size of SmartCache could be changed dynamically. First you could set SmartCache pool size to 10% of user data, if the transaction response time is reduced to a good level, try to reduce the SmartCache size. If the transaction response time is also bad, double the cache size until the response time good or to the size limit. How to Place SSDs If you have more than one disk enclosures for T series storage, average distributing the SSDs onto the level 0 expanded enclosures is recommended. Through this way, you could gain maximum I/O throughput improvement. Linux I/O Scheduler Oracle is widely deployed on Linux Operating system. There re 4 kinds of I/O scheduler on block devices in Linux kernel 2.6: noop, anticipatory, deadline, and cfq. The default I/O scheduler is cfq, which is not suitable for LUNs with SmartCache enabled. 12
For Oracle database, deadline is recommended for LUNs without SmartCache enabled and noop is recommended for LUNs with SmartCache enabled. Using the following command, you could change the I/O scheduler of /dev/sdb to noop and /dev/sdc to deadline : echo noop > /sys/block/sdb/queue/scheduler echo deadline > /sys/block/sdc/queue/scheduler 13
Copyright Huawei Technologies Co., Ltd. 2012. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademark Notice HUAWEI, and are trademarks or registered trademarks of Huawei Technologies Co., Ltd. Other trademarks, product, service and company names mentioned are the property of their respective owners. General Disclaimer The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an HUAWEI TECHNOLOGIES CO., LTD. Huawei Industrial Base Bantian Longgang Shenzhen 518129, P.R. China Tel: +86-755-28780808 www.huawei.com acceptance. Huawei may change the information at any time without notice. PROVIDED BY HUAWEI SOLUTION PROVEN CENTER 14