Contents. 2. cttctx Performance Test Utility Server Side Plug-In Index All Rights Reserved.

Transcription

1 c-treeace Load Test

2 c-treeace Load Test

3 Contents 1. Performance Test Description Login Info Create Tables Run Test Last Run Threads Total Results History cttctx Performance Test Utility Server Side Plug-In Index All Rights Reserved iii

4

5 1. Performance Test Description The c-treeace Load Test benchmark utility is a multithreaded client process that measures the performance of record inserts, reads, and deletes in a given set of c-treeace data files. c-treeace Load Test is written in C and uses the c-treeace ISAM API. This sample program displays a graphical interface with runtime configuration parameters that can be adjusted. These same parameters can also be specified in the command line version of the program called cttctx. To obtain the best results, this benchmark utility has been developed using FairCom's high performance ISAM C API. It is possible to import the tables created by this utility into a c-treeace SQL database and simultaneously, with the test running, perform standard relational queries on the live tables. With c-treeace, you truly have the Best of both worlds - simultaneous high performance record oriented I/O (provided by the ISAM API) and the ease of standard reporting/web integration (provided by the relational APIs)! Benchmark Overview This test program supports the following parameters: Table type Number of client threads to connect to the c-treeace Server Number of operations (called iterations) per thread All Rights Reserved 1

6 Performance Test Description Server operational model. Either: (1) traditional client/server operation (by clicking the Run button); or (2) a Server Side Plug-In model (by clicking the Run Bound Mode button). See the Run Test (page 4) chapter for more details. Benchmark Operation Specifics The test program s threads operate on a set of 23 c-tree data files, with 60% of the operations performed on 5 of the files and 40% on the remaining 18 files. Each data file has records that are 1,024 bytes long with an associated index consisting of a 90-byte key. The test program allows you to specify whether these records are fixed or variable length, though they will always be 1,024 bytes. Each thread executes the following unit of work: 1. Add a 1024-byte data record (and commit the transaction if the file is under transaction control). 2. Read the record by key value. 3. Delete the record (and commit the transaction if the file is under transaction control). The test program is designed to test the scalability of the c-treeace Server database engine and report specific metrics for performance analysis. The idea is that as long as additional CPU resources are available, dividing the work among additional threads should increase the amount of data able to be processed by the engine. 1.1 Login Info The user specifies the following required c-treeace Server parameters in this section: Login Name The user name. Any c-tree user can run this test application. The default user is ADMIN (the user is not case sensitive). User Password The user password. The default value for the ADMIN user is ADMIN (case sensitive). Server Name The server name as specified in the c-treeace Server configuration file (ctsrvr.cfg). The default value is FAIRCOMS. Server Host Machine The network IP address for the computer hosting the c-treeace Server. If the Server is running on the same machine as the client, use localhost for the best performance. All Rights Reserved 2

7 Performance Test Description 1.2 Create Tables The tables for this test program are automatically created each time you press the "Run" button. Set the file parameters as desired before hitting Run. There are 3 sets of options to configure: The level of transaction support Table definition regarding support for fixed or variable length records Option to import the tables automatically into a c-treesql database Transaction Support c-tree supports three levels of transaction control (also referred to in the industry as Journaling). These levels span the tradeoff between absolute data integrity to extreme performance. The choice of transaction control is application dependent, and is dictated by the application developer. TRNLOG Create - Full Transaction Processing -- When selecting TRNLOG, full transaction capability is enabled for the file. This provides two primary benefits: a. atomicity - means a set of operations may be grouped together and the updates to any files involved will all be committed or aborted together; and b. automatic recovery - transaction logs are written to disk at every commit, and in case of any failure in the system, such as power supply problems, the c-treeace Server will automatically recover the files to the latest synchronous state. This is the most secure and expensive in terms of performance because of the time required to write the transaction log to disk. PREIMG Create - PreImage Processing -- PREIMG files still provide atomicity, however they do not allow for automatic recovery. This option is faster than TRNLOG since the transaction logs are not maintained, thereby eliminating the overhead of writing the transactions to disk. However, this mode does not guarantee the automatic recovery of the data and index files to their latest synchronous state in case of a failure in the system. No Transaction Processing -- Finally, it is possible to create files without any level of transaction support (e.g., no atomicity or automatic recovery). The performance increases as compared to TRNLOG and PREIMG, however the data integrity of the system is not guaranteed. This may be appropriate for conditions where absolute data is not a requirement (e.g., for temporary files, real-time transient data, or situations where the data is otherwise recoverable). The best choice for these parameters in an actual application depends on the specific application design. Fixed vs Variable Length Records Create Fixed Length Records -- This test program allows you to create and test with either Fixed or Variable-length records. Your choice of record format will influence the performance results in concert with the transaction model selected above as fixed record support requires All Rights Reserved 3

8 Performance Test Description less overhead and is therefore faster than variable length support. Select the Create Fixed Length Records check box to use the fixed length option, otherwise variable length records are selected by default. c-treesql Database Support Add to "PerfTest" Database -- To support both fast ISAM operations as well as simultaneous relational support in a given file, all that is required is to import the file into a c-treeace SQL database. Note: There are items that must be taken into consideration regarding full support for an ISAM file at the c-treesql level, and these are clearly stated in our documentation. In the case where one intends to query or otherwise operate on the tables used in this c-treeace Load Test test program with a c-treeace SQL engine, click the Add to PerfTest Database checkbox. This will automatically import the table into the SQL database named PerfTest. Create the Tables Note the tables will automatically be created each time you select the "Run" option. Note: You will notice two different c-tree models are presented: Client/Server and Bound models. There is no difference in the actual tables created - the resulting tables are identical - however, the client server model passes the table creation statements over the network as multiple calls while the Server Side Plug-In creation sends a single call to the server where the create activity takes place as an entirely local process on the server. Refer to the Server Side Plug-In (page 9) section for more specific details concerning this model. When running the application, though, it is important to understand the differences between the models, as discussed in the Run Test (page 4) topic of this document. 1.3 Run Test In the Run Test pane you may adjust the number of threads and iterations (number of operations) for the test program. Number of Threads - This is the number of unique instances of this test program that will be simultaneously running against the c-treeace Server. Think of each thread as a user. Number of Iterations - This is the number of database operations that are being submitted to the c-treeace Server by each thread. For example, with the example above of 5 threads and 8,000 iterations, the c-treeace Server will receive a total of 40,000 (5 X 8,000) database operations. Also, note that each operation is actually two database requests (an add, then a read and a delete) so the number actually is 80,000 operations. For more information on what this test is doing see the write up at the end of the Performance Test Description (page 1) chapter. All Rights Reserved 4

9 Performance Test Description To execute the test, select from the following two choices: Run - This button will execute the test program in the traditional client/server model. This means the c-treeace Load Test will connect to the c-treeace Server using the TCP/IP communication protocol. Run as c-treeace Plugin -This button will execute the test program in a special Plug-In mode. In this mode of operation the c-treeace engine will be embedded within the c-treeace Load Test program using a special Server Side Plug-In module. This special mode of operation is very unique in the database industry and is a powerful c-treeace option for applications that require the fastest possible database throughput. For more details please proceed to the Server Side Plug-In chapter. Run Details Each thread will repeat the operations (add, read, delete records) the number of times specified in the Number of Iterations field. It is a useful exercise to monitor operating environment metrics such the CPU utilization and disk I/O capabilities to locate potential bottlenecks. When the test is not CPU bound, increasing the number of threads should increase performance. This multi-threaded capability demonstrates the scalability of c-treeace. The exact specification of any particular user system depends upon the requirements of the project and specific tables/files attributes desired. This benchmark utility enables the study of different feature sets impacting in the performance and security of the system. Feel free to experiment, sit back, and enjoy the results! 1.4 Last Run Threads This section of the c-treeace Load Test illustrates the per thread test results for the most recent test. Last Run Thread Columns Thread # - This is a unique number assigned by the test program for uniquely identifying each thread. You should see one thread (numbered 0 through the number of threads - 1) row for each thread specified in the Number of Threads field. Total Time (ms) - This is the total execution time it took for this specific thread to complete. Note this column is measured in milliseconds. Total Operations - This is the total number of operations the thread completed. This should be twice the setting specified in the Number of Iterations field as the add and delete operations are counted as one operation each. Maximum Time (ms) - This is the maximum time it took for the slowest operation to complete. All Rights Reserved 5

10 Performance Test Description Average Time (seconds) - This column is the average time (measured in seconds) for each thread to complete all its operations. TT Row - This row computes the sum for the Total Time and Total Operations columns (adding the values for all threads) and the highest value across all threads for the Maximum Time column. 1.5 Total Results History For each execution of c-treeace Load Test a number of statistics are saved for comparison and are displayed in this section. Total Results History Columns Total Operations - This is the total number of operations all threads completed. Maximum Time (ms) - This is the maximum time it took for the slowest operation to complete. Average Time (seconds) - This column is the average time (measured in seconds) for all threads to complete. Number of Threads - This is the Number of Threads this test was executed with. Number of Iterations - This is the number of iterations this test was executed with. Total Running Time (seconds) - This is the total elapsed time it took for this test to complete; measured from the time the first thread started until the last thread completed. Throughput (Op/s) - This is the Total Operations divided by the Total Running Time. Transaction Mode - This is the file mode specified in the Create Table window (either cttrnlog for the TRNLOG file mode, ctpreimg for the PREIMG file mode, or Non-transaction for the No Transaction support file mode). Record Mode - This column indicates if the files used for this test stored variable length or fixed length records. Target Machine - This column indicates if the test was executed on the same machine as the c-treeace Server or across the network. localhost means the same machine or an IP Address or Host Name will be displayed if across the network. This column provides the same information that is entered in the Server Host Machine box within the Login Info (page 2) area. Test Mode - This column indicates if the test was run in traditional client/server model (using the Run button) or in the Server Side Plug-In Mode (using the Run as c-treeace Plug-in) button. Client cttctx means client/server model and Bound ntrsh means the Server Side Plug-In Mode. Total Elapsed Time - This is a repeat of the Total Running Time column for the last test executed. Throughput - This is a repeat of the Throughput column for the last test executed. All Rights Reserved 6

11 Performance Test Description Note, it is also possible to save the Total History output to a text file. Right click on any row within the "Total Results History" pane to access the Save menu. All Rights Reserved 7

12 cttctx Performance Test Utility 2. cttctx Performance Test Utility cttctx is a multi-threaded c-tree client test for comprehensive testing of c-treeace Server operations. This utility was designed specifically for profiling c-treeace Server performance. Use this utility to simulate high load conditions against the c-treeace Server for verifying application performance. Usage cttctx <uid> <upw> <svn> [create [-t<trnmod>] <nthrds> [-c<concurrency>] [-d<dist>] [-e] [-h] [-j] [-o<op>] [-p] [-r<msec>] [-n<niter>] ] Arguments <uid> User name <upw> User password <svn> c-treeace Server name create Create new test data/index files -t<trnmod> Transaction mode to use when creating the files trnmod is one of the following t cttrnlog p ctpreimg n no tran <nthrds> Number of c-tree threads to spawn -c<concurrency> Change thread concurrency to <concurrency> -e Use the embedded c-treesql interface (otherwise use the ISAM interface) -h Hard exit, abrupt termination without thread cleanup -j Add to vlength files -o<op> Operation to apply <op> is one of the following: a Add r Read d Delete -p[1 2] track performance stats such as latency of operations. p1 tracks total elapsed time. p2 enables full profiling (elapsed time plus individual call times. This is the equivalent of a previous -p option). Output of these stats are written to a file named fnc.<pid>.<threadid>.log where PID is the cttctx process ID and THREADID is the executing thread ID. -r<msec> Number of milliseconds between performing each operation -n<niter> Number of loop iterations to perform All Rights Reserved 8

13 3. Server Side Plug-In In this section we discuss the support for the c-treeace Server Side Plug-In model of operation which is enabled in this test program by running the test using the "Run as c-treeace Plugin" button. The c-treeace Server Side Plug-In model tightly couples your application directly to the c-treeace database core. This model completely eliminates the communication layer required by a conventional client/server application. Instead, the data is passed between your application code and the c-treeace database core by using the process stack. This model can provide massive performance gains over traditional client/server based systems. All Rights Reserved 9

14 Copyright Notice Copyright FairCom Corporation. All rights reserved. No part of this publication may be stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise without the prior written permission of FairCom Corporation. Printed in the United States of America. Information in this document is subject to change without notice. Trademarks c-treeace, c-treertg, c-treeams, c-tree Plus, c-tree, r-tree, FairCom and FairCom s circular disc logo are trademarks of FairCom, registered in the United States and other countries. The following are third-party trademarks: AMD and AMD Opteron are trademarks of Advanced Micro Devices, Inc. Macintosh, Mac, Mac OS, and Xcode are trademarks of Apple Inc., registered in the U.S. and other countries. Embarcadero, the Embarcadero Technologies logos and all other Embarcadero Technologies product or service names are trademarks, service marks, and/or registered trademarks of Embarcadero Technologies, Inc. and are protected by the laws of the United States and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. HP and HP-UX are registered trademarks of the Hewlett-Packard Company. AIX, IBM, POWER6, POWER7, and pseries are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Intel, Intel Core, Itanium, Pentium and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Microsoft, the.net logo, the Windows logo, Access, Excel, SQL Server, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Novell and SUSE are registered trademarks of Novell, Inc. in the United States and other countries. Oracle and Java are registered trademarks of Oracle and/or its affiliates. QNX and Neutrino are registered trademarks of QNX Software Systems Ltd. in certain jurisdictions. CentOS, Red Hat, and the Shadow Man logo are registered trademarks of Red Hat, Inc. in the United States and other countries, used with permission. UNIX and UnixWare are registered trademarks of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Python and PyCon are trademarks or registered trademarks of the Python Software Foundation. OpenServer is a trademark or registered trademark of Xinuos, Inc. in the U.S.A. and other countries. Btrieve is a registered trademark of Actian Corporation. ACUCOBOL-GT, MICRO FOCUS, RM/COBOL, and Visual COBOL are trademarks or registered trademarks of Micro Focus (IP) Limited or its subsidiaries in the United Kingdom, United States and other countries. iscobol and Veryant are trademarks or registered trademarks of Veryant in the United States and other countries. All other trademarks, trade names, company names, product names, and registered trademarks are the property of their respective holders. Portions Dharma Systems, Inc. All rights reserved. This software or web site utilizes or contains material that is DUNDAS DATA VISUALIZATION, INC. and its licensors, all rights reserved. Portions Copyright Jean-loup Gailly and Mark Adler. 12/16/2015

15 4. Index C Copyright Notice x Create Tables 3 cttctx Performance Test Utility 8 L Last Run Threads 5 Login Info 2, 6 P Performance Test Description 1, 4 R Run Test 2, 4 S Server Side Plug-In 4, 9 T Total Results History 6 All Rights Reserved 11