High-Performance Oracle: Proven Methods for Achieving Optimum Performance and Availability

Transcription

1 About the Author Geoff Ingram is a UK-based ex-oracle product developer who has worked as an independent Oracle consultant since leaving Oracle Corporation in the mid-nineties. He is the creator of a free Oracle performance tuning tool DbCool downloadable from and the author of a book published by John Wiley and Sons Inc in September 2002, covering Oracle performance and availability features in Oracle8i and Oracle9i (including Release 2): High-Performance Oracle: Proven Methods for Achieving Optimum Performance and Availability ISBN: Benchmarking a Microsoft Windows Oracle Client Application Geoff Ingram, Proton Technology Ltd No part of this document may be reproduced for any purpose without the express written permission of Proton Technology Limited. This paper contains an end-to-end performance analysis of a simple application that selects 3,000 rows from an Oracle database server and populates a grid in a Visual Basic application. It's aimed at programmers that use Visual Basic and ActiveX data components in their Visual Basic programs to access Oracle database servers on a local area network (LAN) or wide-area-network (WAN). From a programmer's viewpoint the difference between a LAN and WAN is that a WAN has longer roundtrip times than a LAN, and care needs to be taken to minimize the number of network round trips caused by Oracle data-fetching in order to ensure fastest end-user application response times. The database access component in this example uses Microsoft ActiveX Data Objects (ADO) linked to an OLE DB provider based on ODBC. The ODBC driver in the example is the Oracle in ORA9i driver shipped with Oracle9i for Windows. The database in the example is located on a UNIX server accessed via TCP/IP on a WAN. Over the years, many debates have taken place on the most optimal method for fetching Oracle database into a Microsoft Windows application. Theoretically, the fastest way to access an Oracle database from a Windows client application is to use a data access interface that make direct calls to Oracle Call Interface (OCI), Oracle's low level programming API. In a Visual Basic application, most Oracle data access interfaces on Windows eventually make calls to OCI. In order to abstract the complexity of OCI programming away from the programmer, object-type interfaces layered on top of OCI are usually used in development. For example you could use Oracle's own Oracle Objects for OLE (later renamed to Oracle Data Objects). Although ADO is a more heavyweight layer on top of OCI than OO4O, it still doesn't represent a significant overhead. For example, I've never been able to isolate the extra overhead from calling ADO objects compared to performing the same task using OO4O. In case you're concerned by this overhead,

2 you can measure it yourself using the simple benchmark presented in the rest of this paper. You should also keep in mind that OLE DB providers for Oracle are now available that call OCI directly without using ODBC at all. As a programmer, all this is hidden from you if you program using ADO. Using ADO provides significant advantages compared to OO4O because its methods and properties aren't Oracle-specific whereas those used by OO4O are. As a result, you'll find a much wider choice of database-aware graphical controls available based on ADO. OO4O scores in situations where you need to access Oracle-specific data types in your application, in which case OO4O provides extra methods to make access simpler. You need to decide which is more important. From my experience the main reasons for poor performance of Windows applications that access Oracle databases on a WAN don't involve the choice of the database access method itself. To see why, you need to understand the main areas of processing where time is spent when data is fetched into a Visual Basic application grid from an Oracle database on a LAN or WAN. These are: network round trips DBMS server execution time grid rendering For a SELECT statement returning a result set of a few hundred rows, it's essential that each network round trip to the Oracle server fills each network packet full of rows. If you accidentally set your Visual Basic Oracle fetch operation to transfer rows from the server one row at a time, then you are significantly wasting network bandwidth. The effects on performance depend on the speed of your network: the slower the network, the worse the effects of wasted bandwidth. On a WAN, the waste is likely to increase end-user response times. It's essential to ensure that arrays are used to fetch results from the server in bulk rather than a single row at a time. ADO provides great features to enable underlying array interfaces to be used with minimal programming effort, as shown later. The SQL used to populate the grid in the example by fetching 3000 object definitions from the Oracle database is as follows: select * from dba_objects where rownum <=3000; It's useful to get an estimate of the time Oracle spends executing this statement within the database server, without the overhead of the network transfer, or grid rendering time. This can be performed by executing the query on the server and stepping through the result set. The following PL/SQL block can be used to run such a test because it selects all the rows and iterates through the result set entirely within the server: begin for rec in (select * from dba_objects where rownum <=3000) loop null; end loop; end;

3 This query takes less than 1 second to execute within the server and indicates that poor performance of the application isn't likely to be due to an Oracle server bottleneck. The following Visual Basic subroutine contains code that uses ADO to fetch the same result set across the network from the Oracle server into a Microsoft grid control in a client application: Sub BenchMark() Dim strsql As String strsql = "select from dba_objects where rownum <=3000" Dim rsora As New ADODB.Recordset rsora. = 500 ' set the fetch array size ' send the SQL to the server for execution ' Note: the connection moraconn is set outside this routine rsora.open strsql, moraconn, _ adopenstatic, adlockreadonly, adcmdtext rsora.movelast ' fetch all rows from the server. Time this ' populate the grid. Time this Set Me.MSHFlexGrid1.DataSource = rsora End Sub As shown, ADO provides a property for a RecordSet object to control how many records the OLE DB provider buffers in memory on the client. If the is 500, then the provider retrieves 500 records at a time into memory, fetched in batches of 500 records from the database server. In contrast, the client application code itself requests records one at a time, independent of the underlying array size. After the client has fetched the first 500 records and requests record 501, the provider fetches the next set of 500 from the database server. This is similar to the PL/SQL BULK COLLECT feature, except that the buffering and fetching is performed by the provider in the ADO case without the need for programmer to set the reflect the array sizes in the variables used. Note that for ADO, the programmer needs to do nothing more than set rsora. to change the array size. This ADO design enables much easier programming because the fetch code remains the same for an ADO application, independent of how many records are fetched in batches from the server. Also, the buffering can be controlled through a single setting without changing any other program code. Oracle's PL/SQL, Pro*C, and OCI require the programmer to handle array buffering and provide array variable types to hold fetched columns. ADO and JDBC, on the other hand, enable the buffering to be set independently of the fetch code. Using ADO or JDBC, the user can code with simple scalar types, independent of the underlying array nature of the fetch. As an aside, JDBC provides a method on a statement object to set the cache size for fetch and bulk update operations in an almost identical way to ADO. The following statement demonstrates how to set the batch size for SELECT statements using JDBC: stmt.setfetchsize(500); // set array size for SELECT to 500 For batch update operations, Oracle recommends that you don't use the addbatch() and executebatch() methods of the JDBC 2.0 PreparedStatement interface. These methods are not consistent with the functionality offered by the OraclePreparedStatement class. The following code shows how to batch up inserts into an array with 50 elements before sending the inserts to the server:

4 ((OracleConnection)conn).setDefaultExecuteBatch(50); PreparedStatement ps = conn.preparestatement ("insert into emp(empno)values(?)"); for (int i=100; i<=200; i++) { ps.setint(1,i); // bind an EMPNO value for insert ps.executeupdate(); // do the insert } ((OraclePreparedStatement)ps).sendBatch(); ps.close(); In the example, 100 EMPNO values are inserted into the EMP table in batches of 50. The executeupdate() statement doesn't send the inserts to the server upon each execution. Instead, it batches them into groups of 50 before sending them, as determined by the call to setdefaultexecutebatch() on the connection. This batch size of 50 applies to all UPDATE statements in the session unless it is overridden by the same call on an individual statement object. In the example, sendbatch() is used to send any outstanding inserts to the server once the loop completes. JDBC automatically executes the statement's sendbatch() method whenever the connection receives a commit request, the statement receives a close request, or the connection receives a close request. NOTE By default, JDBC commits all INSERT, UPDATE, and DELETE statements immediately after execution. This can harm performance and isn't usually required. You can turn this off using Connection.setAutoCommit(false) in your code and then use the COMMIT statement to commit transactions as needed. For a SELECT statement, you can confirm that the number of records fetched from the server in each batch matches the array size specified on the client by switching on session SQL trace on the ADO connection object as follows before executing the query: moraconn.execute "ALTER SESSION SET SQL_TRACE TRUE" The r= value in the raw SQL trace file for the session on the server shows the number of rows fetched in each batch based on a cache size of 500, as determined by the statement rsora. = 500 in the client application: FETCH #3:c=130000,e= ,p=0,cr=306,cu=0,mis=0,r=500,dep=0,og=4 FETCH #3:c=110000,e= ,p=0,cr=168,cu=0,mis=0,r=500,dep=0,og=4 FETCH #3:c=100000,e= ,p=0,cr=171,cu=0,mis=0,r=500,dep=0,og=4 FETCH #3:c=130000,e= ,p=0,cr=577,cu=0,mis=0,r=500,dep=0,og=4 FETCH #3:c=120000,e= ,p=0,cr=581,cu=0,mis=0,r=500,dep=0,og=4 FETCH #3:c=120000,e= ,p=0,cr=378,cu=0,mis=0,r=499,dep=0,og=4 A large cache on the client resulting from the use of batching minimizes network roundtrips, which can make a big difference to application throughput, especially in a WAN environment. You should be aware that the Oracle9i ODBC driver uses a cache size of 1 by default, so the cache size should always be increased in the first instance if performance problems are evident. Note that Ethernet physical network packets are 1,500 bytes long by default, so increasing the cache size to a huge number will not reduce the number of physical network roundtrips significantly, although it will reduce the number of Oracle network requests. The UNIX netstat -i command can be used to show the actual number of network input and output packets. The Oracle statistic "SQL*Net roundtrips to/from client" from V$SESSTAT or V$SYSSTAT is useful for determining the number of Oracle network requests. Each Oracle network roundtrip

5 corresponds to at least one physical network roundtrip. Table 1 shows the effect on elapsed query time in our example caused by varying the array batch size used to fetch rows from an Oracle database on a WAN with a latency of 100 milliseconds. Table 1 Windows Application Benchmark Results Statistic 1, SQL*Net roundtrips to/from client Elapsed Seconds for Fetch Elapsed Seconds For Grid Display In this example, significant time is saved by increasing the ADO setting from 100 to 500. A further increase from 500 to 1,000 shows less improvement. The time to display the result set in the grid is much less than the time to fetch the result set over the network. The tests were repeated using the Microsoft ODBC for Oracle driver version 2.573, and results were within 5 percent of the Oracle driver results. In general, the choice between the ODBC driver and OLE DB provider doesn't make as much difference to performance as the correct use of arrays. Some drivers save network traffic by sending numbers on the wire using Oracle's internal numeric format. For example the number requires 11 bytes of storage when represented as a string. You can use the Oracle DUMP function in SQL to show that the length of the same string as that stored in the Oracle database using the Oracle internal number format is only 2 bytes long: select dump( ) from dual; DUMP( ) Typ=2 Len=2: 198,2 Therefore, if the OLE DB provider uses the internal format to encode and decode the number before and after network transmission, 9 bytes of bandwidth can be saved in this example. For applications that process large numbers (for example many banking applications) this saving in network traffic can improve performance significantly. It's worth keeping in mind that it's possible to prevent updates to a grid from continually being redrawn when modifications are taking place during a fetch operation, and defer the changes till the grid population is complete. This can significantly reduce grid display times for grids that need to display thousands of rows. Summary When programming Windows application that access UNIX-server-based Oracle databases, the overhead introduced by various programmatic interfaces, such as ODBC, is often overstated. More often, poor performance is due either to bad choice of array size for row fetches or failure to factor in the overhead of rendering time for graphical controls used in the application to present fetched data. Using a simple benchmark, it's possible to estimate the effects of the choice of array size on application response times to eliminate the guesswork from any performance analysis.