Using TimesTen between your Application and Oracle. between your Application and Oracle. DOAG Conference 2011

Transcription

1 DOAG Conference 2011 Using TimesTen between your Application and Oracle Jan Ott, Roland Stirnimann Senior Consultants Trivadis AG BASEL 1 BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

2 Trivadis facts & figures 11 Trivadis locations with more than 550 employees Hamburg Financially independent and sustainably profitable Key figures 2010 Revenue CHF 101 / EUR 73 Mio. ~180 employees Dusseldorf Services for more than 700 clients in over 1,800 projects Frankfurt Over 170 Service Level Agreements Stuttgart Vienna Freiburg Basel Bern Lausanne 2 Zurich Munich ~20 employees ~350 employees More than 5,000 training participants Research and development budget: CHF 5.0 / EUR 3.6 Mio.

3 Why we are special Customer-specific solution competence and vendor independence Technology competence offers substantiated techniques and skills as well as self-developed approaches guarantees repeatable quality and a safe execution offers more than 17 years of expertise in Oracle and Microsoft has its own Technology Center and strives for technological excellence Solution and integration expertise has a wide and cross-sectoral customer basis and more than 1800 projects every year Combines technological expertise with an understanding of the specific business needs of the client Support for the entire IT project lifecycle 3 has a modular portfolio of services for the entire IT project lifecycle provides the appropriate combination of solutions and services for every level of maturity

4 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 4

5 Introduction Why TimesTen? Customer planned to develop the next generation of their rating application They expected an extremely high number of transactions Therefore they ended up with Oracle s TimesTen The project enables us to look deep into TimesTen TimesTen documentation/binaries are available on OTN 5

6 Introduction Terms 6

7 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 7

8 TimesTen Architecture (1) 8

9 TimesTen Architecture (2) TimesTen has 3 connection types 1) Direct driver connection TimesTen and application on the same host No IPC communication required Direct ODBC driver loads the database directly into shared memory Massive performance improvement Java/OCI applications access direct ODBC driver through JDBC/OCI 2) Driver manager connection Independent driver interface, eg. MS ODBC driver manager Application can be written independent of the database 9

10 TimesTen Architecture (3) 3) Client/Server connection TimesTen and application on a different or the same host Client communicates with a server daemon process Server daemon spawns separate processes for database connections Based on sockets and TCP/IP Network overhead, slower Client/Server offers different sub-types: TCP/IP communication UNIX domain sockets on the same box IPC on the same box 10

11 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 11

12 TimesTen In-Memory Database Setup (1) Fulfill some pre-requisities Kernel parameters (shmmax, shmmni, etc.) Enable optional HugePages support Define a port for the daemon touch/chown empty /etc/instance_info Interactive and silent installation available (on Windows with GUI) One installation directory for config-/log-/binary files (per default) Daemon is started after installation without any DataStore Creating sys.odbc.ini file needed for the DataStore the connections (direct, client/server, IPC) 12

13 TimesTen In-Memory Database Setup (2) Simple sys.odbc.ini file for DataStore and connection definitions # Located at $TT_HOME/info/sys.odbc.ini [MYDS] Driver=/opt/oracle/TimesTen/tt11.2_b/lib/libtten.so DataStore=/opt/oracle/dsn/myds LogDir=/opt/oracle/dsn PermSize=32 TempSize=32 PLSQL=1 DatabaseCharacterSet=AL32UTF8 OracleNetServiceName=DB02.world [ODBC Data Sources] MYDS=TimesTen Driver 13

14 TimesTen In-Memory Database Setup (3) Restart the daemon and load the DataStore with the first connect ttdaemonadmin restart ttisql MYDS A RAM Policy can be defined inuse, manual, always Default policy is inuse After closing the last connection the DataStore will be unloaded It s better to change to manual or always ttadmin rampolicy manual MYDS ttadmin ramload MYDS ttisql MYDS ttadmin ramunload MYDS The basic setup is now complete and we can create objects! 14

15 TimesTen In-Memory Database IMDB (1) In-Memory Database Cache Only in use together with Oracle Basically, it s a table which can be synchronized with an Oracle table Each Oracle table belongs to a CACHE GROUP in TimesTen Read only cache group AWT = Asynchronous Write Through cache group SWT = Synchronous Write Through cache group User Managed cache group All types above can be defined as dynamically loaded cache group on request by primary key pre-loaded cache group (whole table at once) One DataStore can connect only to one Oracle database 15

16 TimesTen In-Memory Database IMDB (2) 16

17 TimesTen In-Memory Database Cache Groups (1) The Oracle database must be inititally configured: SQL*Net configuration to the Oracle database (Instant Client comes along with TimesTen installation) The tables, which we are going to cache, must exist in scott Create cache admin and timesten user in Oracle via SQLNet sqlplus # Create timesten user (done by the script) cd "USERS" # Create cacheadmin user manually create user cacheadmin identified by manager default tablespace users quota unlimited on users; # Grant permissions to "cacheadmin" 17

18 TimesTen In-Memory Database Cache Groups (2) The TimesTen DataStore must be inititally configured: Create cache admin and scott (owner) user in TimesTen The TT user must match with the Oracle user, the password not! A DataStore can only connect to one Oracle database # Connect to the DataStore ttisql MYDS # Create the cacheadmin in TimesTen create user cacheadmin identified by manager; grant create session, cache_manager, create any table, drop any table to cacheadmin; # Create the table owner scott in TimesTen create user scott identified by tiger; grant create session to scott; 18

19 TimesTen In-Memory Database Cache Groups (3) Direct connect to TimesTen and set the cache admin password # Connect to TimesTen ttisql \ "DSN=MYDS;UID=cacheadmin;PWD=manager;OraclePWD=manager" call ttcacheuidpwdset('cacheadmin', 'manager'); Create a grid (required for IMDB cache) with the same connection # Create and assign the grid to the DataStore call ttgridcreate('mygrid'); call ttgridnameset('mygrid'); # Start the cache agent call ttcachestart; # Rep-agent can only be started with existing AWT CG # call ttrepstart; Cache agent is needed for the AUTOREFRESH Feature 19

20 TimesTen In-Memory Database Cache Groups (4) Create first cache group in TimesTen Some columns are excluded create readonly cache group cg_deptartments autorefresh interval 10 seconds from scott.dept ( deptno number(2) not null, dname varchar2(14), loc varchar2(13), primary key (deptno)), scott.emp ( empno number(4) not null, ename varchar2(10), job varchar2(9), deptno number(2), primary key(empno), foreign key (deptno) references scott.dept(deptno)); 20

21 TimesTen In-Memory Database Demo Scripts for the demo are attached at this presentation Load/Unload the cache group Cache group creation Load the cache group with data from Oracle Select from the cache group Insert one row into the Oracle table dept and emp Check auto-refresh feature with a select after 10 seconds Unload the cache group Select again from the cache group Show dictionary tables Partially, the dictionary names match with Oracle dba_[views] Some built-in commands Show C/S tools 21

22 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 22

23 SQL/PL-SQL in TimesTen (1) Oracle SQL / PLSQL and TimesTen SQL / PLSQL are DIFFERENT. "little" differences which make live hard. Sequence persistenz delayed sequence numbers can occure again SQL can not run in parallel (parallel query) REPLACE VIEW not supported WITH clause not supported DISTINCT/SEQUENCE not supported in same SQL SELECT s_mysequence.nextval FROM (SELECT DISTINCT 'a' AS abc FROM dual); TT3206: CURRVAL or NEXTVAL not allowed in select with distinct, aggregate function, group by or order by 23

24 SQL/PL-SQL in TimesTen (2) Selfreferencing foreign keys not supported ALTER TABLE emp ADD CONSTRAINT EMP_EMP_FK FOREIGN KEY (mgr) REFERENCES emp (empno); 3000: self-referencing foreign keys are not allowed Syntax different CREATE TABLE and PRIMARY KEY CREATE TABLE DEPT ( DEPTNO NUMBER(2) NOT NULL, DNAME VARCHAR2(14), LOC VARCHAR2(13), CONSTRAINT deptno_pk PRIMARY KEY (DEPTNO) ); TT7025: Illegal use of reserved keyword "PRIMARY", character position: file "ptsqly.y", lineno 8158, procedure "reserved_word_or_syntax_error" 24

25 SQL/PL-SQL in TimesTen (3) No implizit data converstion CREATE TABLE t_test (a VARCHAR2(30)); INSERT INTO t_test VALUES (1); 2962: Inconsistent datatypes: expected VARCHAR2 got NUMBER in expression (1) This is in ALL areas. Target table in insert INSERT INTO emp (SELECT * FROM emp); 2377: Target table of insert cannot be in the from clause CONNECT BY not supported HINTs not supported 25

26 SQL/PL-SQL in TimesTen (4) No PL/SQL function call in SQL CREATE FUNCTION my_func RETURN VARCHAR2 IS BEGIN RETURN 'a'; END; / Function created. SELECT my_func FROM dual; 2211: Referenced column MY_FUNC not found 26

27 SQL/PL-SQL in TimesTen (5) DECODE translated into CASE CREATE VIEW v_test AS SELECT DECODE(ename, NULL, 'xxx') x FROM emp; SELECT * FROM v_test; 7025: Illegal use of reserved keyword "NULL", character position: 76..._TEST (X) AS SELECT CASE WHEN (EMP.ENAME = NULL) THEN 'xxx' ELSE... 27

28 SQL/PL-SQL in TimesTen (6) Alias and views CREATE VIEW v_test AS SELECT x, x AS y FROM (SELECT 'a' AS x FROM dual); SELECT * FROM v_test; 15110:. does not exist The command failed. SELECT object_name, object_type, status FROM user_objects WHERE object_name = 'V_TEST'; < V_TEST, VIEW, VALID > 1 row found. This is not ALL 28

29 SQL/PL-SQL in TimesTen Performance Simple SQL fast really fast Complex SQL Oracle faster A commit is not a commit in Oracle terms TimesTen commits first only in the LogBuffer (memory) Later the checkpoint writes the data to the checkpoint/log files An explicit durable commit forces persistency 29

30 SQL/PL-SQL in TimesTen Explain plan explain select e.ename, d.dname, d.loc from dept d, emp e where d.deptno=e.deptno and d.deptno=10; 30 STEP: LEVEL: OPERATION: TBLNAME: IXNAME: INDEXED CONDITION: NOT INDEXED: 1 2 RowLkTtreeScan DEPT DEPT D.DEPTNO = 10 <NULL> STEP: LEVEL: OPERATION: TBLNAME: IXNAME: INDEXED CONDITION: NOT INDEXED: 2 2 RowLkTtreeScan EMP TTFOREIGN_4 E.DEPTNO = D.DEPTNO <NULL> STEP: LEVEL: OPERATION: TBLNAME: IXNAME: INDEXED CONDITION: NOT INDEXED: 3 1 NestedLoop <NULL> <NULL> <NULL> <NULL>

31 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 31

32 Analysis and Monitoring TimesTen is miles behind Oracle regarding performance tools Basically, no time elapsed for SQL, only execution count High number of execution doesn t mean the SQL is slow or bad A kind of AWR in TimesTen is really elementary (ttstats) ttstats has not been released officially Crashes with core dump increasing the stack size helps! tttracemon Dis-/enable tracing for SQL, wait, connections, etc Could generate a high number of logfiles Has different levels of detail Using the OS tools for CPU, memory and disk I/O 32

33 AGENDA 1. Introduction 2. TimesTen Architecture 3. TimesTen In-Memory Database 4. SQL/PL-SQL in TimesTen 5. Analysis and Monitoring 6. Conclusion 33

34 Conclusion - Lessons Learned The application has to be changed for TimesTen, not TimesTen for the application! Optimized queries are executed in micro seconds not milli seconds! High CPU load on a TimesTen server is often caused by a missing index No physical I/O and therefore a full table scan uses more CPU Complex SQL could cause massive performance problems Several limitations in the PL/SQL area Difficult to analyze performance problems specially if someone is familiar with the Oracle tools Differences in commit behaviour of TimesTen 34

35 THANK YOU. Trivadis AG VISIT US AT THE TRIVADIS-STAND: Floor 3, No. 304 BASEL 35 BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. Jan Ott, Roland Stirnimann Europa-Strasse 5 CH-8152 Glattbrugg Tel Fax info@trivadis.com FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN