Performance Baseline of Hitachi Data Systems HUS VM All Flash Array for Oracle

Transcription

1 Performance Baseline of Hitachi Data Systems HUS VM All Flash Array for Oracle Storage and Database Performance Benchware Performance Suite Release 8.5 (Build ) November 2013

2 Contents 1 System Configuration 2 Introduction to Storage Performance Tests 3 Storage Benchmark Results Sequential I/O 4 Storage Benchmark Results Random I/O 5 Introduction to Database Performance Tests 6 Database Benchmark Results Database Load 7 Database Benchmark Results OLTP Transactions 8 Reviewing Storage and Database Benchmark Results copyright 2013 by benchware.ch slide 2

3 System Configuration Server and Cluster Configuration Server HDS UCP Large SMP CPU type Intel Xeon E Westmere #sockets 4 #cores 40 #threads 80 RAM capacity Cluster 256 GByte #server within cluster 4 #cores within cluster 160 Software HDS UCP Large SMP Operating System Oracle Linux 6.1 Oracle Database System Benchware Performance Suite 8.5 Build copyright 2013 by benchware.ch slide 3

4 System Configuration Storage System Architecture and Configuration FC attached Storage System Host connectivity DRAM Cache Flash storage #modules Capacity raw Capacity formatted Number of attached database server #hba s single db server #ports single db server HDS HUS VM All Flash Array 32 x 8 Gbps 256 GByte x 1.6 = 51.2 TByte 25.6 TByte (Raid-10) copyright 2013 by benchware.ch slide 4

5 System Configuration Volume and File Management Volume Manager Oracle ASM #LUN s for data 64 #LUN s for redo 32 Concatenated oder Striped Queue depth 128 striped Special parameters asm_au_size 4 MByte File System Oracle ASM Block size [KByte] - Direct I/O Yes Concurrent I/O Yes Asynchronous I/O Yes Special parameters - copyright 2013 by benchware.ch slide 5

6 System Configuration Database Configuration Database Oracle 11g Oracle Release Partition Option Real Application Cluster Data Guard Yes No No Basic Configuration Oracle 11g Block size [kbyte] Archiving Force Logging Flashback 8 No No No copyright 2013 by benchware.ch slide 6

7 System Configuration Database Configuration Memory Configuration Oracle 11g SGA target [GByte] PGA target [GByte] Keep pool [GByte] Recycle pool [GByte] Default pool [GByte] REDO Configuration Oracle 11g REDO log file size [GByte] #REDO log files groups #REDO log file members copyright 2013 by benchware.ch slide 7

8 System Configuration DNS Server Private Network Switch 1GbE Cluster Interconnect Public Network Switch 1GbE Hitachi X57A2 (2-way SMP) RAC Node #1 Hitachi X57A2 (2-way SMP) RAC Node #2 Hitachi X57A2 (2-way SMP) RAC Node #3 Hitachi X57A2 (2-way SMP) RAC Node #4 8-8 Gbps FC connections Gbps FC connections 8-8 Gbps FC connections Gbps FC connections Brocade port 8 Gb FC Switch Brocade port 8 Gb FC Switch Gbps FC connections 16-8 Gbps FC connections Hitachi Unified Storage VM All Flash Array 256GB cache, 32-8 Gb Fibre Channel Ports 4 - Flash Drive Enclosures with TB Flash Media Drives copyright 2013 by benchware.ch slide 8

10 Storage Performance Why measure Storage performance? Storage performance is essential not only for overall Oracle database performance, but also for system management tasks like backup, recovery and archiving Oracle uses all kinds of I/O patterns, but different o/s calls dependent upon the - operating system - system load (Oracle changes system call dependent on load) copyright 2013 by benchware.ch slide 10

11 Storage Performance Why measure Storage performance? Oracle sequential read - User processes: full table scan, full index scan - Temp segment - Backup, restore, recovery RMAN, Export, Data Pump - ARCH: reading online REDO logfile Oracle random read - User processes Oracle sequential write - Temp segment - Backup, restore RMAN, Export, Data Pump - LWGR process: small block size - ARCH processes: writing archived REDO logfile - RVWR process: flashback log file writer - CTWR process: block change tracking file Oracle random write - DBWR processes copyright 2013 by benchware.ch slide 11

12 Storage Performance What is measured? Storage performance from the Oracle point of view - Using database block size - I/O service time measured within Oracle Throughput - Data transfer in mega byte per second [MBps] - Data transfer in database blocks per second [dbps] - I/O Operations in O/S system calls per second [IOPS] Service Time - For random I/O operation in [ms] or [µs] Efficiency of - Auto-Tiering - RAID-level - Striping - Remote mirroring - Virtualization copyright 2013 by benchware.ch slide 12

13 Storage Performance Overview of Storage performance tests with Benchware test codes Oracle Storage Performance Sequential I/O Test Code for Data in-memory Test Code for Data on default Storage Test Code for Data in Flash Cache Test Code for Data in Cell Flash Cache Sequential read - STO-12 STO-13 STO-14 Sequential write - STO Sequential write (LGWR commit stress test) DBL Oracle Storage Performance Random I/O Random read 25 I/O s per SQL Test Code for Data in-memory Test Code for Data on default Storage Test Code for Data in Flash Cache Test Code for Data in Cell Flash Cache - STO-32 STO-33 STO-34 Random write (DBWR stress test) STO Mixed random read write - STO-52 STO-53 STO-54 More aggressive random read 25 I/O s per rows per SQL STO-62 STO-63 STO-64 copyright 2013 by benchware.ch slide 13

14 Storage Performance Remarks on other benchmark tools... SAP, TPC, Swingbench, Hammerora, - No specific storage performance metrics - SAP even does not document storage configuration for its benchmarks I/O load generators like IOzone, IOgen, IOmeter, vdbench - Do not reflect database reality database I/O is much more complex - Measure throughput and service times on operating system level, but not on database level dbms_resource_manager.calibrate_io - Does not recognize hybrid flash/disk architectures copyright 2013 by benchware.ch slide 14

16 Throughput in [MBps] Storage Performance Oracle storage performance: sequential read, single process 4'500 4'000 3'500 3'000 This test is to determine the optimal degree of parallelism (DOP) for a single Oracle process A single process achieves about 4 GBps sequential read! 2'500 2'000 HDS HUS VM 1'500 1' Oracle degree of parallelism (DOP) copyright 2013 by benchware.ch slide 16

17 HDS HUS VM All Flash Array Storage Performance Oracle storage performance: sequential read, single process CPU CPU Physical Physical Physical Physical Physical Physical REDO Hitrate Hitrate Elap busy sys read read read write write write write db flash exa flash time Run Tst Code #N #J #T [%] [%] [iops] [bps] [MBps] [iops] [bps] [MBps] [iops] [%] [%] [s] STO STO STO STO STO STO Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [s] elapsed time in seconds [iops] i/o operations per second [bps] blocks per second [MBps] mega byte per second This test has been run on all four nodes the throughput per process must therefore be divided by 4 copyright 2013 by benchware.ch slide 17

18 Throughput in [MBps] Storage Performance Oracle storage performance: sequential read, multiple processes 20'000 18'000 16'000 Extremely high sequential throughput of > 18 GBps Ideal platform for DWH applications 14'000 12'000 10'000 8'000 HDS HUS VM 6'000 4'000 2' Number of processes copyright 2013 by benchware.ch slide 18

19 HDS HUS VM All Flash Array Storage Performance Oracle storage performance: sequential read, multiple processes CPU CPU Physical Physical Physical Physical Physical Physical REDO Hitrate Hitrate Elap busy sys read read read write write write write db flash exa flash time Run Tst Code #N #J #T [%] [%] [iops] [bps] [MBps] [iops] [bps] [MBps] [iops] [%] [%] [s] STO STO STO STO STO STO Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [s] elapsed time in seconds [iops] i/o operations per second [bps] blocks per second [MBps] mega byte per second copyright 2013 by benchware.ch slide 19

21 HDS HUS VM All Flash Array Storage Performance Oracle storage performance: sequential write CPU CPU Physical Physical Physical Physical Physical Physical REDO Hitrate Hitrate Elap busy sys read read read write write write write db flash exa flash time Run Tst Code #N #J #T [%] [%] [iops] [bps] [MBps] [iops] [bps] [MBps] [iops] [%] [%] [s] STO STO STO STO STO STO STO STO Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [s] elapsed time in seconds [iops] i/o operations per second [bps] blocks per second [MBps] mega byte per second copyright 2013 by benchware.ch slide 21

23 Throughput in [iops] Storage Performance Oracle storage performance: random read 900' ' ' ' ' μs Excellent throughput Very low service times Ideal platform for OLTP applications HDS HUS VM 400' μs 300' ' ' μs 388 μs 418 μs 565 μs nodes Number of processes 4 nodes copyright 2013 by benchware.ch slide 23

24 HDS HUS VM All Flash Array Storage Performance Oracle storage performance: random read CPU CPU Physical Physical Physical Physical Physical Physical REDO Hitrate Hitrate Elap busy sys read read read write write write write db flash exa flash time Run Tst Code #N #J #T [%] [%] [iops] [bps] [MBps] [iops] [bps] [MBps] [iops] [%] [%] [s] STO STO STO STO STO STO STO STO STO STO STO Legend: #N number of RAC nodes [rps] rows per second #J number of jobs [tps] transactions per second #T number of threads (PX) [dbps] database blocks per second [s] elapsed time in seconds copyright 2013 by benchware.ch slide 24

25 Throughput in [dbps] Storage Performance Oracle storage performance: random write 700' '000 LGWR 10 ms DBWR 1.1ms DBWR stress test random db block write 6.4 GBps write throughput 500' '000 HDS HUS VM 300' ' ' Number of processes copyright 2013 by benchware.ch slide 25

26 HDS HUS VM All Flash Array Storage Performance Oracle storage performance: random write CPU CPU Physical Physical Physical Physical Physical Physical REDO Hitrate Hitrate Elap busy sys read read read write write write write db flash exa flash time Run Tst Code #N #J #T [%] [%] [iops] [dbps] [MBps] [iops] [dbps] [MBps] [iops] [%] [%] [s] STO STO STO STO STO STO Legend: #N number of RAC nodes [rps] rows per second #J number of jobs [tps] transactions per second #T number of threads (PX) [dbps] database blocks per second [s] elapsed time in seconds Table size rows copyright 2013 by benchware.ch slide 26

28 Database Performance Why measure Database performance? Projects need understandable key performance metrics for capacity planning - Data load - Data scan - Data aggregation - OLTP transactions - Time windows for certain operations copyright 2013 by benchware.ch slide 28

29 Database Performance What is measured? Typical Oracle database operations Speed of single thread - Rows per second [rps] - Transactions per second [tps] - SQL response time [s] Scalability - Throughput per process for n = {1, 2, 4, 8,, n} Efficiency of - All platform layers Maximum throughput of system - Rows per second [rps] - Transactions per second [tps] - SQL response time [s] - REDO rate [MBps] - REDO service time [s] copyright 2013 by benchware.ch slide 29

30 Database Performance Overview of Database load performance tests with Test Codes Database Performance Data load un-compressed Test Code for Data Load via buffer cache Test Code for Data Load direct Conventional data load (LGWR commit stress test) DBL Bulk load - DBL-21 - Database Performance Data load compressed Test Code for Data Load direct compressed Bulk load, compress BASIC DBL-31 Bulk load, compress OLTP DBL-32 Bulk load, compress HCC query low DBL-33 Bulk load, compress HCC archive low DBL-34 copyright 2013 by benchware.ch slide 30

31 Database Performance Overview of Database aggregation performance tests with Test Codes Database Performance Data Aggregation Test Code for Data on default Storage Create unique b-tree index DBA-12 Create non-unique b-tree index DBA-22 Database Performance Data Scan Test Code for Data on default Storage Test Code for Data in Flash Cache Test Code for Data in Cell Flash Cache Full table scan DBS-12 DBS-13 DBS-14 copyright 2013 by benchware.ch slide 31

32 Database Performance Overview of Database OLTP performance tests with Test Codes Database Performance Test Code for Test Code for Test Code for OLTP Transactions 1 hit per transaction Data on default Storage Data in Flash Cache Data in Cell Flash Cache Select transaction DBX-12 DBX-13 DBX-14 Update transaction DBX-22 DBX-23 DBX-24 copyright 2013 by benchware.ch slide 32

33 Database Performance Remarks on other benchmark tools... SAP, TPC, Swingbench, Hammerora, - No specific database performance metrics for data scan data load data aggregation - No explicit support for Oracle flash cache or Oracle cell flash cache - No explicit support for Oracle compression techniques copyright 2013 by benchware.ch slide 33

35 Load rate in [rps] Database Performance Database transactional load, single process, different transaction size 25'000 20'000 LGWR commit stress test: Excellent transaction rate for single process with commit per second avg SQL response time < 205 μs 15'000 10'000 HDS HUS VM 5' Transaction size in rows per transaction [rpt] copyright 2013 by benchware.ch slide 35

36 HDS HUS VM All Flash Array Database Performance Database transactional load, single process, different transaction size TX CPU Throughput Throughput SQL service Physical Physical Physical REDO REDO REDO REDO REDO Elap size busy rows/sec txn/sec time write write write size writes svt sync sync svt time Run Tst Code #N #J #T [rpt] [%] [rps] [tps] [s] [iops] [bps] [MBps] [MBps] [iops] [ms] writes [us] [s] DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [rps] rows per second [tps] transactions per second [iops] i/o operations per second [s] time in seconds [ms] time in milli seconds [μs] time in micro seconds copyright 2013 by benchware.ch slide 36

37 Load rate in [tps] Database Performance Database transactional load, 2 rows per transaction 300' ' '000 LGWR commit stress test: Excellent transaction rate for single database server with commit per second avg SQL response time < 470 μs 150'000 HDS HUS VM 100'000 50' nodes Number of processes 4 nodes copyright 2013 by benchware.ch slide 37

38 HDS HUS VM All Flash Array Database Performance Database transactional load, 2 rows per transaction TX CPU Throughput Throughput SQL service Physical Physical Physical REDO REDO REDO REDO REDO Elap size busy rows/sec txn/sec time write write write size writes svt sync sync svt time Run Tst Code #N #J #T [rpt] [%] [rps] [tps] [s] [iops] [bps] [MBps] [MBps] [iops] [ms] writes [us] [s] DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E ###### Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [rps] rows per second [tps] transactions per second [iops] i/o operations per second [s] time in seconds [ms] time in milli seconds [μs] time in micro seconds copyright 2013 by benchware.ch slide 38

39 Load rate in [rps] Database Performance Database bulk load 3'000'000 2'500'000 2'000'000 LGWR throughput stress test 287 MBps redo throughput 1 node 519 MBps redo throughput 4 nodes Very high load rate of 2.84 Mrps, each 300 byte Ideal platform for DWH applications 1'500'000 HDS HUS VM 1'000' ' nodes Number of processes 4 nodes copyright 2013 by benchware.ch slide 39

40 HDS HUS VM All Flash Array Database Performance Database bulk load TX CPU Throughput Throughput SQL service Physical Physical Physical REDO REDO REDO REDO REDO Elap size busy rows/sec txn/sec time write write write size writes svt sync sync svt time Run Tst Code #N #J #T [rpt] [%] [rps] [tps] [s] [iops] [bps] [MBps] [MBps] [iops] [ms] writes [us] [s] DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E DBL E E E Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [rps] rows per second [tps] transactions per second [iops] i/o operations per second [s] time in seconds [ms] time in milli seconds [μs] time in micro seconds copyright 2013 by benchware.ch slide 40

42 Load rate in [tps] Database Performance Oracle OLTP select performance, 1 row per transaction 700' ' ms Extraordinary transaction rate: tps 2.8 ms response time 500' ' ms HDS HUS VM 300' ' ' ms 1.8 ms node Number of processes 4 node copyright 2013 by benchware.ch slide 42

43 HDS HUS VM All Flash Array Database Performance Oracle OLTP select performance, 1 row per transaction CPU CPU Throughput Throughput SQL service Physical Physical REDO Hitrate Hitrate Physical Physical Elap busy sys rows/sec txn/sec time read write write db flash exa flash read write time Run Tst Code #N #J #T [%] [%] [rps] [tps] [s] [iops] [iops] [iops] [%] [%] [MBps] [MBps] [s] DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [rps] rows per second [tps] transactions per second [iops] i/o operations per second [s] time in seconds copyright 2013 by benchware.ch slide 43

44 Load rate in [tps] Database Performance Oracle OLTP update performance, 1 row per transaction 140' ' ' ms 14 ms 80'000 60' ms HDS HUS VM 40' ms 20' ms node Number of processes 4 node copyright 2013 by benchware.ch slide 44

45 HDS HUS VM All Flash Array Database Performance Oracle OLTP update performance, 1 row per transaction CPU CPU Throughput Throughput SQL service Physical Physical REDO Hitrate Hitrate Physical Physical Elap busy sys rows/sec txn/sec time read write write db flash exa flash read write time Run Tst Code #N #J #T [%] [%] [rps] [tps] [s] [iops] [iops] [iops] [%] [%] [MBps] [MBps] [s] DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E DBX E E E Legend: #N number of RAC nodes #J number of jobs #T number of threads (PX) [rps] rows per second [tps] transactions per second [iops] i/o operations per second [s] time in seconds copyright 2013 by benchware.ch slide 45

47 Benchmark Results Summary Storage Performance Sequential I/O Read Write Metric [MBps] [MBps] HDS HUS VM All Flash Array (6 400) Random I/O read Throughput Service time Random I/O write Throughput Throughput Service time, DBWR multi block Service time, LGWR multi block Legend: [MBps] mega byte per second [IOPS] I/O operations per second [dbps] database blocks per second [ms] milli seconds [IOPS] [ms] [IOPS] [dbps] [ms] [ms] copyright 2013 by benchware.ch slide 47

48 Benchmark Results Summary Database Performance Database Performance Data Load Un-Compressed Conventional OLTP insert [rps] Bulk load [rps] Metric [rps] [rps] HDS UCP & HDS HUS VM All Flash Array Database Performance OLTP Transactions 1 hit per transaction Select transaction Throughput Service time Update transaction Throughput Service Time Metric [tps] [ms] [tps] [ms] HDS UCP & HDS HUS VM All Flash Array copyright 2013 by benchware.ch slide 48

49 Benchmark Results Reviewing Storage and Database Performance HDS HUS VM All Flash Array combines the best of all worlds - Seamless integration in existing SAN infrastructures - Proven scalability and performance for all workloads - Extremely high I/O throughput with microseconds service time - Highly efficient DRAM storage cache in addition to flash technology - Rich and mature storage management software portfolio (cloning, snapshots, replication, dynamic provision, dynamic tiering etc.) Superior Oracle Database Platform based on HDS UCP & HUS VM - Extremely efficient transactional and bulk load - Highly scalable transaction rates - Consistent performance for ALL database objects copyright 2013 by benchware.ch slide 49

50 Benchmark Results Reviewing Storage and Database Performance Closing the I/O gap - Gap is widening between high performance servers and slow storage - Extremely high I/O performance and low response time is required to maximize utilization of server processing power - HUS VM All Flash Array reduces the I/O gap and allows Oracle platforms to fully utilize cpu capacity without waiting for I/O operations - Increasing server cpu utilization improves return on assets for customers and lowers CAPEX for server platforms copyright 2013 by benchware.ch slide 50

51 swiss precision in performance measurement