Run your own Oracle Database Benchmarks with Hammerora

Transcription

1 Run your own Oracle Database Benchmarks with Hammerora Steve Shaw Intel Corporation UK Keywords: Database, Benchmark Performance, TPC-C, TPC-H, Hammerora Introduction The pace of change in database infrastructure is rapid, both in terms of hardware and software. Innovations such as clustering or virtualization need to be assessed and measured to determine the benefits to your particular environment. However one of the overriding challenges remains of how to quantify those benefits and to compare and contrast different installations. The overriding business challenge is how to take advantage of new innovations whilst also mitigating against the risk of change. Load testing and database benchmarks provide a compass to the map of your infrastrucutre both to point you in the right direction as you begin your journey, as a constant point of reference to ensure that you are maintaining the correct course and to ensure that you are not heading in the wrong direction compared to your competitors. Despite the clear benefits of load testing and benchmarking many businesses fail to allocate both time and resources to test adequately. Justifying those resources can be a challenge especially given the cost of propritary testing solutions. Testing is rarely on the critical path to keeping the business running however not doing so can prove a false economy and put the business at a significant disadvantage compared to competitors implementing low cost solutions. Hammerora is the leading free and open source load testing tool for Oracle databases and provides an independent and cost effective solution to evaluating the performance of your database infrastructure providing you with the compass to plot your course through infrastructure change. With over 25,000 users worldwide Hammerora provides an established and proven load testing and benchmarking solution to the Oracle Database, MySQL Database with the Oracle InnoDB storage engine and HTTP environments. This paper presents a guide by the author of Hammerora to getting the most out of the software to help you learn how to compare and contrast database environments.

2 Architecture and Concepts Hammerora is released under the Gnu Public License (GPL) and was designed from the outset to leverage the advantages of Open Source technology ensuring a free, open and independent technology stack. The goal was to design and build a flexible tool aimed towards DBA s and system administrators, who potentially have limited software development experience as opposed to professional programmers. All recognised top level load testing tools require both multithreading capabilities to build efficient and scalable test systems and a easy to use scripting language with which to build tests tailored to specific environments. Designing and building a multithreaded scripting language from the ground up is time consuming yet many existing languages lacked the capabilities required either bringing either multithreading capabilities or scripting capabilities but not both combined. Coupled with the demands for an efficient scalable Oracle interface a number of languages such as Java, Python, Perl fell short of the requirements. Only one language, TCL met the requirements of the design brief with multithreading, scripting, a high performance OCI based Oracle interface called Oratcl and proven performance in complex environments such as AOL Server. Considering TCL is also open source is one of the leading projects on sourceforge and has a compatible graphical interface it provides the ideal foundation on which to build an open source load testing tool. With the foundation established Hammerora enables the building of test environments with an east to use graphical interface where lightweight threads operating independently establish sessions on the Oracle database and drive both inbuilt and user defined scenarios to test Oracle Database environments. The pluggable nature of the tool also means that tests are portable to different levels of the Oracle stack such as HTTP middleware servers with tests scripts recorded from the firefox web browser and replayed in Hammerora and the Timesten database with OCI and the MySQL Database based on the Oracle InnoDB storage engine. Installation and Configuration Hammerora can be downloaded for free from without restriction on installation and use. Pre-compiled easy to install packages are provided for the most widely deployed operating systems of both Windows and Linux however building the required packages for other operating systems is straightforward meaning Hammerora is compatible with any Oracle environment where an Oracle database or client installation is available. Hammerora can be used with any Oracle release and has been tested against all releases from Oracle 8 up to and including Oracle 11g Release 2. Installation is straightforward and for beginners the pre-installation of Oracle Express edition is recommended, for advanced users the Oracle instant client will suffice in order for Hammerora to connect across the network and test a large scale Oracle database implementation. On Linux and Windows a pre-packaged installer is provided giving a wizard driven interface to install Hammerora, after selecting the installation directory the tool extracts into the chosen location creating a standalone software environment, no software is installed into Oracle software locations.

3 Illustration Install Wizard Illustration. 1: Hammerora install wizard When installation is complete the option is provided to Launch Hammerora, if not choosing to start the software immediately it can be also be run from the installation directory. Illustration Launch Hammerora Illustration. 2: Launch Hammerora

4 Understanding Hammerora On starting, Hammerora presents a self-contained graphical user interface with which to begin building load test scenarios. Configuration for a particular environment can be defined within the configuration file config.xml. A key concept is the provision of the script editor. Within the script editor the workload is defined for the users to run against the target Oracle database although it is important to note that workloads may be defined in order that subsets can be allocated to individual users identified by name as opposed to a single workload allocated to all users. Additionally users may interact with each other during the course of a test run. When a test has been defined a number of virtual users are created, the limit on the maximum number of virtual users is only set by the hosts CPU and memory resources. Each virtual user takes the form of an operating system thread meaning Hammerora can scale efficiently to a large number of users. When creating the virtual users it is also possible to choose to view the output generated by the individual users within a grid based output to view test performance in real time. Illustration Hammerora Interface Illustration. 3:Hammerora Interface

5 Test workloads can be written bespoke, additionally Hammerora has the ability to convert Oracle Database trace files into a format that can be replayed against the target database enabling a capture and replay testing facility to build test scenarios against any database application. Of particular interest is the in-built load tests based on the specifications of the well-known TPC-C and TPC-H benchmarks for OLTP and Data Warehouse environments respectively with a TPC-C based test being provided for Oracle and MySQL and a TPC-H test for Oracle. For Database focused tests Hammerora provides a graphical transaction and query counter. Additionally Hammerora can operate in a number of modes within a Master and Slave configuration enabling Hammerora installed on a number of load testing clients to be controlled by a single interface enabling the scaling to test large scale installations with a clustered load test facility across the network. Hammerora includes an interactive console both for monitoring messages and errors from test scripts during testing runs but also to interact with the host operating system. In addition to the database tier, tests can also be defined against HTTP driven middle tiers with the automatic capture and generation of test scripts through the firefox web browser to be replayed against a target web server to complete full Oracle environment tests. Configuring and Running Load Tests Hammerora includes all of the features required to build and execute load tests based upon TPC specifications without the user requiring special scripting knowledge. Dependent on the configuration defined in config.xml or the options selected under the benchmark menu the menu options auto-configure to define your chosen benchmark. The options enable you to define a target schema within which to build your test configuration. The illustration shows a target configuration for building a schema based upon the TPC-C specification defining the target Oracle service name, the system user password, the existing target tablespace and the username and password for the test schema owner that is to be created. The number of warehouses determines the size of the dataset to be created and the number of threads enables a multithreaded schema build to fully utilize mulitcore CPU environments. The options also define parameters for the tests to be run and whether to run a standalone test or a timed tested to generate AWR reports and report the average transaction rates recorded over a test period.

6 Illustration TPC-C Options Illustration. 4: TPC-C Options With the options defined pressing the build schema button on the main menu is all that is required to both create a test schema, populate it with data and install the test PL/SQL procedures. The multithreaded build efficiently creates and installs schemas containing millions of rows in a short space of time without requiring external data or dump files for data loading. Upon completion of the schema build a compatible test driver script is defined according to the menu options given under the benchmark menu. After selecting the test script, define the required number of users and select the Run button to initiate the test. If choosing the AWR snapshot driver script the entire test is automated and will provide for a Ramp Up time for the users to begin running transactions followed by a timed test before and after which AWR snapshots are automatically taken on the target database. When the test has completed Hammerora reports the average number of transactions completed across the timed tests and provides the AWR snapshot numbers from which you may produce a report based upon the test workload. Whilst tests are running a graphical transaction counter reports the transaction rate currently measured on the system. The reported transaction rate is based upon the same

7 statistics and coincides directly with the average transaction rate reported at the end of the test and measured within the AWR snapshot. AWR reports provide a comprehensive and detailed method with which to both analyze and compare system performance and to identify performance bottlenecks from a range of system parameters. Illustration Transaction Counter Illustration. 5: Transaction Counter A similar methodology is used for testing database warehouse environments using tests based upon the TPC-H specification. Such tests use Oracle Parallel Query to define complex ad-hoc queries enabling the testing of the potential for systems to host complex business intelligence environments. Using the in-built tests it is possible to generate comprehensive performance profiles that measure the capabilities of your system and also to determine the impact of hardware and software changes upon the environment. After building familiarity with the built-in tests Hammerora also provides the facility to take a standard Oracle trace file generated by the database and automatically convert it into a format that can be replayed against the database allowing tests to be built to simulate specific applications. Testing Web Servers Many database environments also interact with web servers. Hammerora enables the generation of tests against the web-based middle-tier in addition to Database layer. Utilising the functionality provided by the Tcl Webtest Recorder released open source by Knowledgemarkets it is possible to use the firefox web browser to record interactive web browsing sessions and automatically generate a test script. This script can be replayed with multiple sessions in Hammerora to incorporate the middle tier into test scenarios.

8 Illustration Web Testing Illustration. 6: Web Testing Script Conclusion and Future Plans Based upon past success Hammerora development continues into new and exciting technology areas driven by user demand and Oracle acquisitions. Future plans include extending in-built tests to the TPC-E specification, building upon MySQL capabilities and enhancing web and HTTP testing to ensure that Hammerora remains the leading Open Source Oracle load testing alternative worldwide. Disclaimers Hammerora is Open Source released under the GNU public license under the framework laid out by the Intel Linux User s Group. Hammerora is NOT an Intel software product or developed using Intel resources or materials. Hammerora is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Contact address: Steve Shaw Intel Corporation 650 Wharfedale Road, Winnersh Triangle RG41 5TP, Wokingham, Berkshire UK Phone: +44(0) steve.shaw@intel.com Internet: