VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Performance Report V E R I T A S W H I T E P A P E R
Table of Contents Introduction.................................................................................1 About the Benchmark..........................................................................1 Test Configuration.............................................................................2 Overview and Analysis of Results..................................................................3 Useful Tuning Tips.............................................................................6 Appendix A..................................................................................7 Appendix B..................................................................................8
Introduction This document describes the performance of VERITAS Database Edition 2.1.2 for Oracle as measured by an Online Transaction Processing (OLTP) workload. Typical OLTP systems involve processing simple to moderately complex transactions with multiple updates to the database. The benchmark used for this performance comparison was derived from the commonly known TPC-C benchmark. This document studies the impact of different memory and I/O configurations. This release of the Database Edition supports both 32-bit and 64-bit HP-UX 11i and Oracle 8.0.6 to 8.1.7. The Database Edition comprises the following components: VERITAS File System delivered through license for OnlineJFS Quick I/O Database Accelerator Storage Checkpoint and Storage Rollback VERITAS Volume Manager delivered through a license VxDBA utility VERITAS Quick I/O provides a raw device interface to regular files created under VERITAS File System. The advantages of using Quick I/O are as follows: Quick I/O bypasses the file system s buffering for reads and writes, thereby eliminating the double-buffering and double copying performance penalties. Quick I/O also results in less use of system memory, so more memory can be allocated to the database buffers. Quick I/O bypasses the file system s locking layer, eliminating the additional locking overhead associated with traditional file systems. Normally, the file system s locking mechanism maintains a single-writer-lock on files, which ensures that only one process writes to a file at a time to ensure data integrity. Because databases typically maintain their own locks at either row or page level, this is an unnecessary overhead that serializes write operations, slowing database performance. This overhead is eliminated when databases are built on VERITAS File Systems using Quick I/O. Quick I/O supports asynchronous I/O, which is inherently faster and more efficient than synchronous I/O in certain situations. Other than with Quick I/O, async I/O support is available only with raw devices. About the Benchmark The OLTP workload used for performance comparison was based on the industry-standard TPC-C benchmark from the Transaction Processing Council. The TPC-C benchmark is an established metric used to compare hardware and software platforms. This benchmark comprises a mixture of read-only and update-intensive transactions that simulate a warehouse supplier environment. (Details on this benchmark can be obtained from the Transaction Processing Council s Web page at http://www.tpc.org.) However, the goal for this report was to compare the impact of the underlying Database I/O configuration and not to achieve the best possible numbers. Hence, these numbers are shown as a reference only. The size of the database used for the test was 50 GB, with a total of 63 Oracle datafiles, including redo logs, indexes, rollback segments and temporary and user tablespaces. The size of the database was that of a fully scaled TPC-C database with a scale factor of 200 warehouses. www.veritas.com VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Page 1
Test Configuration Tests were run on a single N-Class HP9000 computer system. The N-Class was configured with four 440-MHz CPUs, 4 GB of system memory and a total of 38 disk drives. The 38 drives were composed of 30 fibre-channel drives and 8 LVD SCSI drives. Two of the LVD SCSI drives were internal, one being allocated as the system disk and the other allocated for Oracle and TPC-C executables. The six remaining LVD SCSI drives were designated to hold the TPC-C data backup. The fibre-channel drives were allocated to the TPC-C data. The 12 disks were spread across two fibre-channel controllers, six disks on each controller. The Oracle logs were striped across two fibre-channel drives on a separate controller from the data drives. Both raw and file-system tests were implemented on a 12-way striped volume. The Oracle redo logs were placed on a two-way striped volume on a separate controller. The Oracle executables and benchmark code resided on a separate LVM volume configured on the remaining internal disk. The following software releases were used in testing: Oracle 8.1.6 (32-bit) HP-UX 11i VERITAS Database Edition 2.1.2 for Oracle consisting of the following components: VERITAS Quick I/O 3.3.2 (VRTSqio) VERITAS File System 3.3.2 (OnlineJFS) VERITAS Volume Manager 3.1 for HP-UX 11i The following I/O configurations were tested: Raw I/O uses VERITAS Volume Manager raw volumes directly VERITAS File System Quick I/O uses Quick I/O feature of File System Buffered and direct I/O uses VERITAS File System HFS - HP Native File System Raw I/O traditionally delivers the best performance because no locking or buffering is performed in this mode. However, this configuration has a major shortcoming: managing databases built directly on raw volumes involves a significant load on system and database administrators. The vast array of tools that are available for monitoring and managing file systems cannot be used in this configuration, which may result in human errors from which it may be difficult to recover. Quick I/O allows regular files built on VERITAS File System to be accessed as raw devices, bypassing the double buffering and double locking associated with databases built on top of regular file systems. Although the double buffering is avoided when file systems are mounted in the direct I/O mode, this mode includes the file-system locking to ensure that only one process or thread issues writes to a file at any point in time. Because databases maintain their own locks to ensure data integrity, this extra locking imposes an unnecessary serialization of I/Os and affects performance. The double locking also imposes additional CPU overhead. All raw and Quick I/O performance runs used asynchronous I/O. File system runs were executed with asynchronous I/O disabled and the Oracle initialization parameter, dbwr_io_slaves = 20. Page 2 VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i www.veritas.com
Overview and Analysis of Results The results of the benchmark tests are shown in the following tables and graphs. The performance metric used for this comparison is the throughput rate measured by the number of transactions completed per minute (TPM). The transaction mix in this benchmark represents the processing of an order as it is entered, paid for, checked and delivered and follows the model of a complete business activity. Therefore, the TPM metric is considered a measure of business throughput. The TPM achieved for each of the I/O configurations is shown. Tests were run on five database configurations, each with a different Oracle SGA size as shown. During the test, SAR data (system activity reporter data), Oracle statistics and Quick I/O statistics (using qiostat) were gathered in addition to the throughput numbers. All performance runs were configured with an Oracle database block size of 2K. The VERITAS File System file-system block size was set to 1K with a 16-MB log. The HFS runs were completed with a default block size of 8K because HFS does not support 1K block sizes. HFS runs were performed to contrast performance of Quick I/O to that of a conventional file system. The following chart shows throughput achieved for each of the I/O configurations. 7000 Throughput Comparison 6000 5000 tpmc 4000 3000 2000 1000 0 0 500 750 1000 1250 1500 Oracle SGA (MB) Oracle SGA Size I/O Config 500 MB 750 MB 1000 MB 1250 MB 1500 MB RAW 3456 4236 4872 5527 5924 QIO 3451 4220 4854 5506 5907 DIO 2157 2573 2683 2330 2168 BIO 1822 1875 1968 1248 1214 HFS 2325 1992 1924 1235 1375 www.veritas.com VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Page 3
The preceding chart and table show transaction rates for each of the five I/O configurations tested. Raw I/O typically achieves the best performance when compared to the other I/O configurations. For all SGA sizes, Quick I/O throughput is virtually equal (less than 1 percent differential) to that of raw I/O. Analysis of the conventional file-system configurations shows VERITAS File System Direct I/O achieving the best throughput. Except for the HFS 500-MB SGA size, Direct I/O maintains a 14 percent to 50 percent throughput increase over VERITAS File System Buffered I/O and HFS. VERITAS File System Buffered I/O and HFS show similar performance characteristics overall. The following chart shows average CPU Utilization for each SGA size. CPU utilization is the sum of usr and sys time as reported by SAR (System Activity Reporter). 60 CPU Utilization 50 40 CPU % 30 20 10 0 0 500 750 1000 1250 1500 Oracle SGA (MB) Oracle SGA Size I/O Config 500 MB 750 MB 1000 MB 1250 MB 1500 MB RAW 33% 37% 44% 48% 51% QIO 31% 37% 42% 49% 52% DIO 27% 31% 32% 30% 29% BIO 27% 27% 29% 26% 20% HFS 53% 50% 52% 48% 49% The CPU utilization for Raw and Quick I/O configurations show that Quick I/O is capable of achieving raw-device performance without incurring additional overhead. As observed, the differences in CPU utilization are less than 1 percent for the Raw and Quick I/O configurations. The HFS CPU utilization is of interest; transaction throughput for HFS is practically identical to VERITAS File System Buffered I/O, but the CPU utilization is doubled. Page 4 VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i www.veritas.com
The following chart shows a measure of the overall efficiency of each configuration per SGA size. Efficiency can be determined by dividing the tpmc number achieved for each SGA size by the CPU utilization. This number is also referred to as the Figure of Merit. A higher number indicates a more efficient operation. 140 Efficiency 120 100 FOM 80 60 40 20 0 0 500 750 1000 1250 1500 Oracle SGA (MB) Oracle SGA Size I/O Config 500 MB 750 MB 1000 MB 1250 MB 1500 MB RAW 104.7 114.5 110.7 115.2 116.2 QIO 111.3 114.1 115.6 112.4 113.6 DIO 79.9 39.8 37 25.7 28.1 BIO 67.5 69.5 67.9 48 60.7 HFS 43.9 39.8 37 25.7 28.1 As seen in the previous charts and tables, Quick I/O tracks raw performance with very little variation. This pattern holds true for the efficiency data as well. As expected from the data on the following charts, HFS is the least efficient I/O configuration, using much more CPU resources than the other I/O configurations. www.veritas.com VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Page 5
Useful Tuning Tips The following are some performance-tuning guidelines that may be helpful when using VERITAS Database Edition to configure an Oracle database: Tip No. 1: Use VERITAS File System Quick I/O for Databases VERITAS File System in its default mode supports buffered I/O, as does HFS, in which all I/O incurs the double-copying and double-buffering penalties. In addition, the UNIX semantics require that only one writer have the exclusive right to update a file, which forces other read and write operations on the same file to wait. Therefore, databases on file systems often perform slower than on raw devices. Quick I/O has I/O characteristics similar to those of raw I/O (a higher degree of read/write parallelism with less CPU overhead). Using Quick I/O provides a significant advantage in OLTP scenarios, in which multiple processes access the same datafiles simultaneously. Tip No. 2: Use Striped Volumes for Performance In general, the best way to achieve the optimal throughput is by balancing the I/O across as many disks as possible. This balancing can be achieved by creating the database on a file system on a striped (RAID-0) volume using all the available disks. Traditionally, because redo logs tend to be accessed sequentially, one may see better performance by putting the redo logs on a separate file system, using a simple volume on a dedicated disk. Oracle professes a new methodology for distributing S.A.M.E. (Stripe and Mirror Everything) data, which included striping redo logs. Please see technet.oracle.com for more information on this topic. If database reliability is important, mirror (using RAID-1) at least the volume used by the redo logs. Tip No. 3: Use 2K Oracle block sizes for OLTP workloads The use of larger block sizes appears to significantly degrade database throughput. The Oracle block size must be a multiple of the file-system block size. Summary The OLTP performance measurements illustrate that the Quick I/O feature enables the VERITAS Database Edition for Oracle to have performance equal to that of raw-partition configurations. It achieves this performance while delivering the manageability attainable only by running Oracle on file systems. Using the OLTP benchmark, VERITAS Database Edition for Oracle outperforms the base VERITAS File System buffered and direct I/O configurations by a large margin. In addition, Quick I/O files can be managed as regular files and are backed by the reliability of VERITAS File System. With its log-based implementation, VERITAS File System allows fast recovery after a system crash, which is not available with HFS. VERITAS File System also offers online administration features such as online resizing and defragmentation to reduce planned downtime. Page 6 VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i www.veritas.com
Appendix A The following is the typical Oracle init.ora parameter file used in the benchmark. control_files = (/tpcc_disks/cntrl1) compatible = 8.1.5.0.0 db_name = tpcc db_files = 200 db_file_multiblock_read_count = 32 db_block_buffers = 25600 dml_locks = 500 hash_join_enabled = FALSE log_archive_start = FALSE log_checkpoint_interval = 100000000 log_checkpoints_to_alert = TRUE log_buffer = 1048576 log_archive_dest = /archlog gc_releasable_locks = 0 max_rollback_segments = 220 open_cursors = 200 processes = 200 sessions = 600 transactions = 400 distributed_transactions = 0 transactions_per_rollback_segment = 1 rollback_segments = (t1, t2, t3, t4, t5, t6, t7, t8, t9, t10, t11, t12, t13, t14, t15, t16, t17, t18, t19, t20, t21, t22, t23, t24, t25, t26, t27, t28, t29, t30, t31, t32, t33, t34, t35, t36, t37, t38, t39, t40, t41, t42, t43, t44, t45, t46, t47, t48, t49, t50) shared_pool_size = 30000000 cursor_space_for_time = TRUE www.veritas.com VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i Page 7
Appendix B The results of the database measurements are summarized in the following three spreadsheets. Most of the columns in these spreadsheets are self-explanatory, but some of the comparisons are not. The FOM is a figure of merit derived by dividing the transaction rate TPM by the CPU utilization CPU %. A larger number represents a more efficient option. The last three columns compare the two sets of results by comparing thruput (ratios greater than one represent better throughput) and CPU percent ratios (where less than one is a better result) and finally give a comparison of the figures of merit for the two compared results. Each spreadsheet represents a variation in one of the possible parameters used in this study. Parameters: File System Block Size = 1K Oracle Block Size = 2K HFS Block Size = 8K VERITAS Volume Manager Compared Type SGA # Run TPM CPU % to RAW FOM* Raw 500 MB 1 3456 33 104.7 Raw 750 MB 2 4236 37 114.5 Raw 1000 MB 3 4872 44 110.7 Raw 1250 MB 4 5528 48 115.2 Raw 1500 MB 5 5925 51 116.2 QIO 500 MB 47 3451 31 1.00 111.3 QIO 750 MB 48 4220 37 1.00 114.1 QIO 1000 MB 49 4854 42 1.00 115.6 QIO 1250 MB 50 5506 49 1.00 112.4 QIO 1500 MB 51 5907 52 1.00 113.6 BIO 500 MB 6 1822 27 0.53 67.5 BIO 750 MB 7 1875 27 0.44 69.5 BIO 1000 MB 8 1968 29 0.40 67.9 BIO 1250 MB 31 1248 26 0.23 48.0 BIO 1500 MB 32 1214 20 0.20 60.7 DIO 500 MB 9 2157 27 0.62 79.9 DIO 750 MB 10 2573 31 0.61 83.0 DIO 1000 MB 11 2684 32 0.55 83.9 DIO 1250 MB 33 2330 30 0.42 77.7 DIO 1500 MB 34 2168 29 0.37 74.8 HFS 500 MB 42 2326 53 0.67 43.9 HFS 750 MB 43 1992 50 0.47 39.8 HFS 1000 MB 44 1924 52 0.39 37.0 HFS 1250 MB 45 1235 48 0.22 25.7 HFS 1500 MB 46 1375 49 0.23 28.1 Page 8 VERITAS Database Edition 2.1.2 for Oracle on HP-UX 11i www.veritas.com
VERITAS Software Corporation Corporate Headquarters 350 Ellis Street Mountain View, CA 94043 650-527-8000 or 800-327-2232 For additional information about VERITAS Software, its products, or the location of an office near you, please call our corporate headquarters or visit our Web site at www.veritas.com V E R I T A S W H I T E P A P E R Copyright 2001 VERITAS Software Corporation. All Rights Reserved. VERITAS, VERITAS SOFTWARE, the VERITAS logo, Business Without Interruption, VERITAS The Data Availability Company, VERITAS Database Edition, VERITAS File System, VERITAS Quick I/O and VERITAS Volume Manager are trademarks or registered trademarks of VERITAS Software Corporation in the U.S. and/or other countries. Other product names mentioned herein may be trademarks or registered trademarks of their respective companies. Specifications and product offerings subject to change without notice. Printed in USA. June 2001. VER06-DBE2.1.2ORPERWPR-0002 90-01189-399