Oracle Database In- Memory & Rest of the Database Performance Features

Transcription

1 Oracle Database In- Memory & Rest of the Database Performance Features #DBIM12c

2 Safe Harbor Statement The following is intended to outline our general product direcmon. It is intended for informamon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcmonality, and should not be relied upon in making purchasing decisions. The development, release, and Mming of any features or funcmonality described for Oracle s products remains at the sole discremon of Oracle. 2

3 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 3

4 Data Loading : With In- Memory DML comes in three forms Single record transacmons Direct path / bulk data loads ParMMon exchange load In- Memory Column Store always transacmonally consistent Similar to how an index is always maintained during DML

5 Data Loading : With In- Memory DML comes in three forms Single record transacmons Inserts Updates Deletes Direct path / bulk data loads ParMMon exchange load In- Memory Column Store always transacmonally consistent Similar to how an index is always maintained during DML

6 ConvenMonal DML Oracle Database Instance System Global Area Online Redo Online Log Redo Online Log Redo Log User LGWR DBWR Data Files 6

7 ComposiMon of In- Memory Area In Memory Area IMCU IMCU IMCU IMCU IMCU IMCU IMCU IMCU Column Format Data SMU SMU SMU SMU SMU SMU SMU SMU Metadata Contains two subpools: IMCU pool: Stores In Memory Compression Units (IMCUs) SMU pool: Stores Snapshot Metadata Units (SMUs) IMCUs contain column forma^ed data SMUs contain metadata and transacmonal informamon 7

8 Single Row DML and the In- Memory Column Store In Memory Area IMCU Each IMCU contains the column entries for a subset of rows in the object It also has a transacmon journal that is used to keep the column store transacmonally consistent Column Column Format Format Data Data Metadata 8

9 Single Row DML and the In- Memory Column Store In Memory Area IMCU DML operamons processed in row store just as they are today Corresponding entry in column store marked stale Copy of changed row stored in the TransacMon Journal Column Column Format Format Data Data Metadata 9

10 Single Row DML and the In- Memory Column Store In Memory Area IMCU In- Memory Column Store is never out of date Read consistency achieved by merging contents of column and the transacmon journal Column Column Format Format Data Data Metadata 10

11 Single Row DML and the In- Memory Column Store In Memory Area IMCU When number of entries in transacmon journal hits an internal threshold CU automamcally refreshed This is an online operamon column store always available Column Column Format Format Data Data Metadata 11

12 ConvenMonal DML With In- Memory Instance Select Update/Delete System Global Area In- Memory Area IMCU JOURNAL Online Redo Online Log Redo Online Log Redo Log User LGWR IMCO Wnnn DBWR Data Files 12

13 Data Loading : With In- Memory DML comes in three forms Single record transacmons Direct path / bulk data loads Create Table As Select (CTAS) Insert /*+ APPEND */ ParMMon exchange load In- Memory Column Store always transacmonally consistent Similar to how an index is always maintained during DML

14 Direct Path loads with In- Memory Sales Table In- Memory Column Store SALES TABLE High Water Mark On Disk SQL> select segment_name, populate_status, bytes_not_populated from v$im_segments; SEGMENT_NAME POPULATE_STATUS BYTES_NOT_POPULATED SALES COMPLETED 0 In Memory 14

15 Direct Path loads with In- Memory Sales Table NEW DATA High Water Mark Direct path operamons are all or nothing Data inserted into new DB blocks above high watermark In- Memory Column Store SALES TABLE No other process in the Database sees the new data On Disk SQL> select segment_name, populate_status, bytes_not_populated from v$im_segments; SEGMENT_NAME POPULATE_STATUS BYTES_NOT_POPULATED SALES COMPLETED 0 In Memory 15

16 Direct Path loads with In- Memory Sales Table High Water Mark NEW DATA High Water Mark New DB blocks incorporated into table on commit IM column store becomes aware of new data on commit In- Memory Column Store SALES TABLE On Disk Size of new data visible in the BYTES_NOT_POPULATED column of V $IM_SEGMENTS SQL> select segment_name, populate_status, bytes_not_populated from v$im_segments; SEGMENT_NAME POPULATE_STATUS BYTES_NOT_POPULATED SALES COMPLETED STARTED In Memory 16

17 Direct Path loads with In- Memory Sales Table High Water Mark NEW DATA High Water Mark If the Sales Table has PRIORITY specified on it data is automamcally populated on commit In- Memory Column Store SALES TABLE If no PRIORITY specified then missing data is populated on next access On Disk SQL> select segment_name, populate_status, bytes_not_populated from v$im_segments; SEGMENT_NAME POPULATE_STATUS BYTES_NOT_POPULATED SALES COMPLETED 0 In Memory 17

18 Data Loading : With In- Memory DML comes in three forms Single record transacmons Direct path / bulk data loads ParMMon exchange load In- Memory Column Store always transacmonally consistent Similar to how an index is always maintained during DML

19 In- Memory s BFF - ParMMoning JANUARY 2014 SALES FEBRUARY 2014 MARCH 2014 APRIL 2014 Large Table Difficult to Manage ParMMons Divide and Conquer Easier to Manage Improve Performance Transparent to applicaaons 19

20 Oracle ParMMoning Smart distribuaon of data from disk to memory SALES DISK SSD Flash In Memory June 2014 May 2014 April 2014 April 2014 June 2014 May 2014 March 2014 March 2014 February 2014 February 2014 January 2014 January 2014 December 2013 December 2013 November 2013 October 2013 September 2013 August 2013 July 2013 June 2013 April 2013 March 2013 February 2013 October 2013 September 2013 August 2013 July 2013 June 2013 April 2013 March 2013 February 2013 November

21 Oracle ParMMoning Smart data access through paraaon eliminaaon What is the total sales figure for March 1 st 2014 compared to February 1 st 2014? SALES Jun 2014 May 2014 Apr 2014 Mar 2014 ParMMoning eliminamon DramaMcally reduces amount of data retrieved from storage Performs operamons only on relevant parmmons Transparently improves query performance and opmmizes resource umlizamon Feb 2014 Zone map pruning Jan 2014 Dec 2013 Further reduces amount of data retrieved from storage Nov 2013 For Oracle Engineered Systems Oct 2013 Transparently improves query performance and opmmizes resource umlizamon

22 PopulaMng parmmons into the In- Memory Column Store CREATE TABLE ORDERS PARTITION BY RANGE (PARTITION p1 INMEMORY MEMCOMPRESS FOR DML, PARTITION p2 INMEMORY MEMCOMPRESS FOR QUERY, PARTITION p3 INMEMORY MEMCOMPRESS FOR CAPACITY : PARTITION p200 NO INMEMORY ); Not all parmmons need to be populated in- memory Different levels of MEMCOMPRESS allow more data to be populated Easy to switch levels as part of ILM strategy Smart processing through table expansion for parmally populated parmmoned tables

23 In- Memory and ParMMon Exchange Loads Sales table currently populated in IM column store 1. Create external table for flat files 2. Use CTAS command to create non- parmmoned table 3. Set INMEMORY a^ribute 4. Populate TMP_SALES into IM column store 5. Alter table Sales exchange parmmon May_24_2014 with table TMP_SALES 23

24 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 24

25 What Are TradiMonal Summary Objects? INDEXES MATERIALIZED VIEWS OLAP CUBES Oracle ConfidenMal Internal/Restricted/Highly Restricted 25

26 TradiMonal Indexing Strategy Table 1 3 OLTP Indexes AnalyMc Indexes Most Indexes in a mixed workload environment are only used for analytic queries Inserting one row into a table requires updating analytic indexes: Slow! Indexes only speed up predictable queries & reports 26

27 In- Memory Column Store Replaces AnalyMc Indexes Table 1 3 OLTP Indexes In- Memory Column Store Fast analytics on any columns Better for unpredictable analytics Less tuning & administration Column Store not persistent so update cost is much lower OLTP & batch run faster But how do you determine which indexes to drop? 27

28 QuesMons To Determine Which Indexes To Drop Is the index needed for referenmal integrity? Primary Key Index Foreign Key Index How selecmve is the index? Number of rows returned Is the index used by transacmonal workload or analymcal queries? Hidden advantage from dropping indexes Foot print of database drops as no tablespace, redo or undo necessary for indexes 28

29 Materialized Views Summary object containing pre- calculated results Typically created to speed up Resource intensive joins and aggregamons AutomaMcally or manually refreshed Queries automamcally rewri^en to use MV where applicable Underlying table can be parmmoned Excellent candidate to be put in the In- Memory column store Together they speed up all aspects of the execuaon plan Oracle ConfidenMal Internal/Restricted/Highly Restricted 29

30 Materialized View Example The SQL SELECT d.d_year, c.c_nation, SUM(lo_revenue - lo_supplycost) profit FROM lineorder l, date_dim d, part p, supplier s, customer C WHERE l.lo_orderdate = d.d_datekey AND l.lo_partkey = p.p_partkey AND l.lo_suppkey = s.s_suppkey AND l.lo_custkey = c.c_custkey AND s.s_region = 'AMERICA' AND c.c_region = 'AMERICA' GROUP BY d.d_year, c.c_nation ORDER BY d.d_year, c.c_nation; Oracle ConfidenMal Internal/Restricted/Highly Restricted 30

31 Materialized View Example The original Plan (10 seconds) Oracle ConfidenMal Internal/Restricted/Highly Restricted 31

32 Materialized View Example Create MV CREATE MATERIALIZED VIEW maria_mv ON prebuilt TABLE WITH reduced PRECISION ENABLE query rewrite AS SELECT d.d_year, c.c_nation, SUM(lo_revenue - lo_supplycost) profit FROM lineorder l, date_dim d, part p, supplier s, customer C WHERE l.lo_orderdate = d.d_datekey AND l.lo_partkey = p.p_partkey AND l.lo_suppkey = s.s_suppkey AND l.lo_custkey = c.c_custkey AND c.c_region = 'AMERICA' GROUP BY d.d_year, c.c_nation ORDER BY d.d_year, c.c_nation; Oracle ConfidenMal Internal/Restricted/Highly Restricted 32

33 Materialized View Example The MV Plan (0.04 seconds) Oracle ConfidenMal Internal/Restricted/Highly Restricted 33

34 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 34

35 Scale- Out In- Memory Database to Any Size Scale-Out across servers to grow memory and CPUs In-Memory queries parallelized across servers to access local column data Direct-to-wire InfiniBand protocol speeds messaging on Engineered Systems 35

36 Scale- Out: Distribute by ParMMon Distribute by ParMMon (top- level parmmon for composite parmmoned tables) Ideal for Hash ParMMons Also for other parmmon types if uniformly accessed Allows in- memory parmmon- wise joins ORDERS PARTITION BY HASH ON ORDER_ID

37 Scale- Out: Distribute by Sub- ParMMon For composite parmmons, can distribute by Sub- ParMMon Ideal for Hash Sub- ParMMons Also for other sub- parmmon types if uniformly accessed Allows in- memory parmmon- wise joins ORDERS PARTITION BY RANGE ON ORDER_DATE SUBPARTITION BY HASH ON ORDER_ID Nov 13 Nov 13 Nov 13 Nov 13 Dec

38 Scale- Out: Distribute by Rowid Range Distributes IMCUs by uniform hash on first rowid For non- parmmoned tables Also for parmmoned tables with skewed access across parmmons Ensures uniform distribumon of load across instances ORDERS Rowid Ranges

39 In- Memory and RAC ALTER TABLE sales INMEMORY DISTRIBUTE BY PARTITION; ALTER TABLE COSTS INMEMORY DISTRIBUTE ROWID RANGE; DistribuMon allows in memory segments larger than individual instance memory Policy is automamc or user- specifiable Controlled by DISTRIBUTE subclause Distribute by rowid range Distribute by parmmon Distribute by subparmmon Distribute AUTO 39

40 Shared Nothing Architecture 40

41 Querying Oracle Database In- Memory in RAC environment User Serial queries will only access a fraction of the data from the IM column store on its node IMCUs not shipped across interconnect Note: Rest of the data comes from disk 41

42 Querying Oracle Database In- Memory in RAC environment Parallel execution helps as it starts multiple processes BUT we can t control where the parallel server processes are started IMCUs not shipped across interconnect Note: Rest of the data comes from disk 42

43 What is Auto DOP? OpMmizer determines execumon plan If esmmated Mme greater than threshold OpMmizer determines Auto DOP If esmmated Mme less than threshold PARALLEL_DEGREE_THRESHOLD Actual DOP = MIN(PARALLEL_DEGREE_LIMIT, Auto DOP) Statement executes serially Statement executes in parallel

44 Enable Auto DOP Set PARALLEL_DEGREE_POLICY=AUTO Enables Auto DOP, statement queuing, in- memory PX Run DBMS_RESOURCE_MANAGER.CALIBRATE_IO Default IO rate is 200MB/sec For Oracle Database In- Memory Ensures that the query coordinator is IMCU home locamon aware Ensures that at least one parallel server slave is allocated on each database instance Note: New Adaptive setting for Parallel_degree_policy not recommended for DBIM

45 Querying Oracle Database In- Memory in RAC environment Shared nothing architecture means AutoDOP must be used to access Data With AutoDOP query coordinator is aware of IMCU locations Query coordinator automatically starts parallel server processes on the correct nodes 45

46 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 46

47 Oracle Database Architecture Requires memory, processes and database files System Resources GL OE AP 47

48 New MulMtenant Architecture Memory and processes required at container level only System Resources GL OE AP GL OE AP 48

49 In- Memory and Oracle MulMtenant ERP CRM DW Shared memory and background processes In- Memory Area specified at the container Database level INMEMORY_SIZE=20G By default each Puggable Database inherits the INMEMORY_SIZE from CDB 49

50 In- Memory and Oracle MulMtenant INMEMORY_SIZE=0G INMEMORY_SIZE=4G INMEMORY_SIZE=16G ERP CRM DW Container Database INMEMORY_SIZE=20G 50

51 In- Memory and Oracle MulMtenant Over subscripmon is possible! INMEMORY_SIZE=0G INMEMORY_SIZE=10G INMEMORY_SIZE=16G ERP CRM DW Container Database INMEMORY_SIZE=20G 51

52 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 52

53 CPU Resource Manager IniMal populamon can be CPU resource intensive Resource Manager can control CPU usage during In- Memory column store populamon Default or custom resource plans can be used Default or custom consumer group can be used Oracle ConfidenMal Internal/Restricted/Highly Restricted 53

54 Resource Manager Plan With In- Memory Example Use the SET_CONSUMER_GROUP_MAPPING procedure to set the consumer group: BEGIN dbms_resource_manager.set_consumer_group_mapping( attribute => 'ORACLE_FUNCTION', value => 'INMEMORY', consumer_group => 'BATCH_GROUP'); END; CPU umlizamon will be limited to the value specified for the BATCH_GROUP 54

55 Program Agenda Data Loading and Database In- Memory TradiMonal Summary Objects and Database In- Memory Scale Out and Database In- Memory MulM- Tenant and Database In- Memory Resource Manager and Database In- Memory High Availability and Database In- Memory 55

56 Oracle Database In- Memory: Unique Fault Tolerance Similar to storage mirroring Duplicate in-memory columns on another node Enabled per table/partition Application transparent Downtime eliminated by using duplicate after failure Only Available on Engineered Systems 56

57 Oracle Database In- Memory: Unique Fault Tolerance ALTER TABLE sales INMEMORY DUPLICATE; ALTER TABLE COSTS INMEMORY DISTRIBUTE AUTO DUPLICATE ALL; Policy is user- specifiable Controlled by DUPLICATE subclause DUPLICATE DUPLICATE ALL 57

58 Example: Duplicate Strategy For a Star Schema Fact tables are distributed by parmmon Dimension tables are duplicated (DUPLICATE ALL) Co- locates joins between the distributed fact table parmmons and the dimension tables D D D D F D D F D D F D D D D 58

59 Data Guard Since no on- disk formats are changed there is no impact to Data Guard The inmemory_size parameter must be configured on the standby for failover The IM column store will be populated at failover (same rules as a normal instance startup) Supported for logical standby Primary In- Memory Column Store In- Memory Column Store SALES_1_MONTH SALES_1_MONTH 59

60 AcMve Data Guard In- memory on AcMve Data Guard Standby has its own In- Memory Column Store Must be allocated if used on the primary Currently IM Column Store not useable on standby 12.2 Enables reports run on the standby to take advantage of Database In- Memory Primary In- Memory Column Store In- Memory Column Store SALES_1_MONTH SALES_1_MONTH 60

61 Golden Gate Since no on- disk formats are changed there is no impact to Golden Gate Golden Gate replays transacmons In- Memory Column automamcally kept in sync Capture Trail Pump LAN/WAN Internet TCP/IP Trail Delivery Source Oracle & Non-Oracle Database(s) Bi-directional Target Oracle & Non-Oracle Database(s) 61

62 RMAN Since no on- disk formats are changed there is no impact to RMAN backups RMAN Tape Drive Oracle Secure Backup Amazon S3 Fast Recovery Area (FRA) Oracle Public Cloud (Future) 62

63 Oracle Database In- Memory & Security ORACLE Oracle ConfidenMal Internal/Restricted/Highly Restricted 63

64 Transparent Data EncrypMon Feature Summary Encrypted Data Disk Backups Exports ApplicaMons Managed Keys Off- Site FaciliMes! Encrypts columns or enmre applicamon tablespaces! Protects the database files on disk and on backups! Securely manages the keys, assists with key rotamon! Supports Oracle Exadata engineered systems! CompaMble with applicamons, no changes required

65 In- Memory and Column Level EncrypMon Enables encrypmon on a narrow set of columns EncrypMon is done in upper layers Data encrypted on the way in and on the way out of Database Completely transparent to the IM column store Data Stored in the IM column store is encrypted Data can be compressed in the IM column store

66 In- Memory and Tablespace Level EncrypMon All objects in the tablespace are encrypted EncrypMon is done in the data layer Data populated into the IM column store decrypted Data can be compressed in the IM column store Data from the IM column store cannot be to dumped out in clear text

67 In- Memory and VPD VPD used to enforce row and/or column level security All table column data populated into the IM column store Data can be compressed in the IM column store VPD policies applied as addimonal predicates added to queries Could actually speed up access via the column store due to extra filter

68 Takeaways Oracle Database In- Memory fully supports all data loading features Supports not just base tables but Materialized views and query rewrite RAC provides transparent In- Memory data scale out when used with Auto DOP Only fault tolerant In- Memory column store thanks to duplicate feature Fully supports the MulMtenant architecture Does not impact an of Oracle s exismng HA technologies Works seamlessly with all of Oracle s Security features 68

69 AddiMonal Resources Related Database In- Memory Free Webcasts Oracle Database In- Memory meets Data Warehousing Related White Papers Oracle Database In- Memory White Paper Oracle Database In- Memory AggregaMon Paper Join the ConversaAon h^ps://twi^er.com/db_inmemory h^ps://blogs.oracle.com/in- memory/ h^ps:// h^p:// Related Videos In- Memory Demo Database Industry Experts Discuss Oracle Database In- Memory (11:10) Sozware on Silicon Any AddiAonal QuesAons Oracle Database In- Memory Blog My 69

70 70