Deep Dive into Oracle 12c Pattern Matching

Transcription

1 Deep Dive into Oracle 12c Pattern Matching Oren Nakdimon

2 Who Am I? a chronology by Oracle years When What Where Oracle 6/ Developer IAF Oracle Server Group Manager Golden Screens Oracle 8i/9i DBA Group Manager TELEknowledge Oracle 10g/11g VP R&D and Israel Site Manager Olista Oracle 11g/12c Freelance Consultant Database Expert [English] [Hebrew]

3 Agenda SQL analytics evolution The MATCH_RECOGNIZE clause 3

4 The Usual Warning Before you decide to apply any of the presented features in production, make sure: to thoroughly test them for your applications and environments that you have the required license to use them

5 SQL Analytics Evolution

6 The Most Basic SQL Row-level visibility Maximum one output row per input row WHERE clause

7 Aggregate Functions Group-level visibility Strict definition of group Each input row belongs to exactly one group Maximum one output row per group GROUP BY clause HAVING clause

8 Analytic (Window) Functions Window-level visibility Strict definition of window Each input row has its own window Window-level aggregates are added to input rows OVER PARTITION BY ORDER BY

9 Pattern Matching Enhanced analysis of row sequences Match-based output One row per match (similar to the group by concept) or All the match s input rows (similar to the window concept) Each input row may belong to 0, 1 or more matches

10 Find all the employees in department 30 The obvious solution is: SELECT * FROM EMP WHERE DEPTNO = 30 But that is too easy ;-)

11 Find all the employees in department 30 Pattern Variable EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7369 SMITH CLERK /12/ ALLEN SALESMAN /02/ SELECT * FROM EMP MATCH_RECOGNIZE ( ALL ROWS PER MATCH PATTERN (D30) DEFINE D30 AS DEPTNO=30 ) 7521 WARD SALESMAN /02/ JONES MANAGER /04/ MARTIN SALESMAN /09/ BLAKE MANAGER /05/ CLARK MANAGER /06/ SCOTT ANALYST /04/ KING PRESIDENT 17/11/ TURNER SALESMAN /09/ ADAMS CLERK /05/ JAMES CLERK /12/ FORD ANALYST /12/ MILLER CLERK /01/ D30

12 Find 2 employees hired successively to department 30 order by Pattern Variable 1 2 EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7369 SMITH CLERK /12/ ALLEN SALESMAN /02/ SELECT * FROM EMP MATCH_RECOGNIZE ( ORDER BY HIREDATE ALL ROWS PER MATCH PATTERN (D30{2}) D30) DEFINE D30 AS DEPTNO=30 ) 7521 WARD SALESMAN /02/ JONES MANAGER /04/ BLAKE MANAGER /05/ CLARK MANAGER /06/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ KING PRESIDENT 17/11/ JAMES CLERK /12/ FORD ANALYST /12/ MILLER CLERK /01/ SCOTT ANALYST /04/ ADAMS CLERK /05/ D30

13 PATTERN (Greedy) Quantifiers Pattern Number of consecutive occurrences X 1 X{n} n X+ 1 or more X{n,} n or more X{n,m} between n and m X{,m} between 0 and m X* 0 or more X? 0 or 1

14 Find one or more employees hired successively to department 30 order by Pattern Variable EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7369 SMITH CLERK /12/ ALLEN SALESMAN /02/ SELECT * FROM EMP MATCH_RECOGNIZE ( ORDER BY HIREDATE ALL ROWS PER MATCH PATTERN (D30+) DEFINE D30 AS DEPTNO=30 ) 7521 WARD SALESMAN /02/ JONES MANAGER /04/ BLAKE MANAGER /05/ CLARK MANAGER /06/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ KING PRESIDENT 17/11/ JAMES CLERK /12/ FORD ANALYST /12/ MILLER CLERK /01/ SCOTT ANALYST /04/ ADAMS CLERK /05/ D30

15 Multiple Pattern Variables order by Pattern Variables EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7369 SMITH CLERK /12/ ALLEN SALESMAN /02/ SELECT * FROM EMP MATCH_RECOGNIZE ( ORDER BY HIREDATE ALL ROWS PER MATCH PATTERN (D30 D20 LS?) DEFINE D30 AS DEPTNO=30, D20 AS DEPTNO=20, LS AS SAL<=1500 ) 7521 WARD SALESMAN /02/ JONES MANAGER /04/ BLAKE MANAGER /05/ CLARK MANAGER /06/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ KING PRESIDENT 17/11/ JAMES CLERK /12/ FORD ANALYST /12/ MILLER CLERK /01/ SCOTT ANALYST /04/ ADAMS CLERK /05/ D30 D20 LS

16 Dividing the Input Row Set order by partition by Pattern Variables EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7782 CLARK MANAGER /06/ KING PRESIDENT 17/11/ SELECT * FROM EMP MATCH_RECOGNIZE ( PARTITION BY DEPTNO ORDER BY HIREDATE ALL ROWS PER MATCH PATTERN (HS+ LS*) DEFINE LS AS SAL<=1500, HS AS SAL> 1500 ) 7934 MILLER CLERK /01/ SMITH CLERK /12/ JONES MANAGER /04/ FORD ANALYST /12/ SCOTT ANALYST /04/ ADAMS CLERK /05/ ALLEN SALESMAN /02/ WARD SALESMAN /02/ BLAKE MANAGER /05/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ JAMES CLERK /12/ LS HS

17 Adding Measures order by partition by Pattern Variables EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7782 CLARK MANAGER /06/ SELECT * FROM EMP MATCH_RECOGNIZE ( PARTITION BY DEPTNO ORDER BY HIREDATE MEASURES MATCH_NUMBER() as MATCH# ALL ROWS PER MATCH PATTERN AFTER MATCH (HS+ SKIP LS*) PAST LAST ROW DEFINE PATTERN LS (HS+ AS SAL<=1500, LS*) DEFINE LS HS AS SAL<=1500, SAL> ) HS AS SAL> 1500 ) 7839 KING PRESIDENT 17/11/ MILLER CLERK /01/ SMITH CLERK /12/ JONES MANAGER /04/ FORD ANALYST /12/ SCOTT ANALYST /04/ ADAMS CLERK /05/ ALLEN SALESMAN /02/ WARD SALESMAN /02/ BLAKE MANAGER /05/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ JAMES CLERK /12/ LS HS

18 AFTER MATCH SKIP PAST LAST ROW (the default) TO NEXT ROW TO FIRST var TO [LAST] var

19 Adding Measures order by partition by Pattern Variables EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO 7782 CLARK MANAGER /06/ SELECT * FROM EMP MATCH_RECOGNIZE ( PARTITION BY DEPTNO ORDER BY HIREDATE MEASURES MATCH_NUMBER() as MATCH# ALL ROWS PER MATCH PATTERN AFTER MATCH (HS+ SKIP LS*) TO NEXT ROW DEFINE PATTERN LS (HS+ AS SAL<=1500, LS*) DEFINE LS HS AS SAL<=1500, SAL> ) HS AS SAL> 1500 ) 7839 KING PRESIDENT 17/11/ MILLER CLERK /01/ SMITH CLERK /12/ JONES MANAGER /04/ FORD ANALYST /12/ SCOTT ANALYST /04/ ADAMS CLERK /05/ ALLEN SALESMAN /02/ WARD SALESMAN /02/ BLAKE MANAGER /05/ TURNER SALESMAN /09/ MARTIN SALESMAN /09/ JAMES CLERK /12/ LS HS

20 Uniting Adjacent Periods KNESSET_ NUMBER PRIME_MINISTER START_DATE END_DATE 0 דוד בן-גוריון 14/05/ /03/ דוד בן-גוריון 10/03/ /11/ דוד בן-גוריון 01/11/ /10/ דוד בן-גוריון 08/10/ /12/ דוד בן-גוריון 24/12/ /01/ משה שרת 26/01/ /06/ משה שרת 29/06/ /11/ דוד בן-גוריון 03/11/ /01/ דוד בן-גוריון 07/01/ /12/ דוד בן-גוריון 17/12/ /11/ דוד בן-גוריון 02/11/ /06/ לוי אשכול 26/06/ /12/ לוי אשכול 22/12/ /01/ לוי אשכול 12/01/ /02/ גולדה מאיר 17/03/ /12/ גולדה מאיר 15/12/ /03/ גולדה מאיר 10/03/ /06/ יצחק רבין 03/06/ /06/ מנחם בגין 20/06/ /08/ מנחם בגין 05/08/ /10/ יצחק שמיר 10/10/ /09/1984

21 partition by SELECT KNESSET_NUMBER, order by KNESSET_ NUMBER PRIME_MINISTER START_DATE, START_DATE END_DATE 17 END_DATE, 04/05/ /03/2009 אהוד אולמרט 15 PRIME_MINISTER, 06/07/ /03/2001 אהוד ברק 15 CONT_PERIOD#, 07/03/ /02/2003 אריאל שרון 16 CONT_PERIOD_START, 28/02/2003 אריאל שרון 16/04/ CONT_PERIOD_END 18/06/1996 בנימין נתניהו 06/07/ נתניהוFROM ISRAEL_CABINETS 31/03/2009 בנימין 18/03/2013 ( 18/03/2013 בנימין נתניהו 19 MATCH_RECOGNIZE 14/05/ PARTITION בנימין נתניהו BY 14/05/2015 PRIME_MINISTER 17/03/ /12/1969 גולדה מאיר 6 ORDER BY START_DATE 15/12/ /03/1974 גולדה מאיר 7 MEASURES MATCH_NUMBER() 10/03/ /06/1974 גולדה מאיר 8 FIRST(START_DATE) 14/05/ /03/1949 דוד בן- גוריון 0 10/03/ /11/1950 דוד בן- גוריון 1 1 ALL גוריון ROWS דוד בן- PER 01/11/1950 MATCH 08/10/ PATTERN בן- גוריון STRT )דוד 08/10/1951 NXT*) 24/12/ דוד בן- גוריון 24/12/ /01/ ) דוד בן- גוריון 03/11/ /01/ ORDER גוריון BY דוד בן- START_DATE; 07/01/ /12/ דוד בן- גוריון 17/12/ /11/ דוד בן- גוריון 02/11/ /06/ יצחק רבין 03/06/ /06/ יצחק רבין 13/07/ /11/1995 NXT CONT_PERIOD#, CONT_PERIOD_START, FINAL LAST(END_DATE) CONT_PERIOD_END DEFINE NXT AS START_DATE = PREV(END_DATE)

22 ONE ROW PER MATCH SELECT * FROM ISRAEL_CABINETS MATCH_RECOGNIZE ( PARTITION BY PRIME_MINISTER ORDER BY START_DATE MEASURES MATCH_NUMBER() CONT_PERIOD#, FIRST(START_DATE) CONT_PERIOD_START, LAST(END_DATE) CONT_PERIOD_END ONE ROW PER MATCH PATTERN (STRT NXT*) DEFINE NXT AS START_DATE = PREV(END_DATE) ) ORDER BY CONT_PERIOD_START;

23 Fibonacci Numbers X F SELECT * FROM (select rownum X from dual connect by level<=100) MATCH_RECOGNIZE ( ORDER BY X MEASURES CLASSIFIER() AS F_OR_NOT ALL ROWS PER MATCH PATTERN ((F NF)+) DEFINE F AS (X = 1 OR X = 2 OR X = LAST(F.X,1) + LAST(F.X,2)) ); Pattern Alternation

24 Only Fibonacci Numbers X F SELECT * FROM (select rownum X from dual connect by level<=100) MATCH_RECOGNIZE ( ORDER BY X MEASURES CLASSIFIER() AS F_OR_NOT ALL ROWS PER MATCH PATTERN ((F {-NF-})+) DEFINE F AS (X = 1 OR X = 2 OR X = LAST(F.X,1) + LAST(F.X,2)) ); Pattern Exclusion

25 Count Fibonacci Numbers X F SELECT * FROM (select rownum X from dual connect by level<=100000) MATCH_RECOGNIZE ( ORDER BY X MEASURES COUNT(F.X) AS F_COUNT ALL ROWS PER MATCH PATTERN ((F NF)+) DEFINE F AS (X = 1 OR X = 2 OR X = LAST(F.X,1) + LAST(F.X,2)) );

26 Thank You Oren Nakdimon