DirectQuery vs In-Memory in BISM

Transcription

1 DirectQuery vs In-Memory in BISM Julie Koesmarno Sydney SQL User Group MsSQLGirl.com

2 About Me 8+ years experience with SQL Server (since SQL Server 2000) MCITP 2008 Past projects: large scale 24/7 Data Integration, Data Management and Data Cleansing system Current project: Data Warehouse and Analysis Services Tabular Model project 2

3 Agenda Terminology DirectQuery vs In-Memory Hybrid Mode Out of Scope: Sharepoint Integration 3

4 Abstract The new and flashy Tabular Model for Analysis Services has been highly pitched with the in-memory (VertiPaq) capability allowing for complex queries to run very fast What are the advantages and disadvantages of using in-memory cache to store and query data? Is DirectQuery the answer to address in-memory cache mode limitations? What are the restrictions on DirectQuery tabular model? When can a hybrid mode be used? How would the selected mode can affect the Processing option during Deployment? In this session, all the above questions will be answered with live demos, comparing the two modes and the scenarios that are applicable to them. It also discusses how partition plays an important role in DirectQuery mode and choosing the correct processing mode when deploying a Tabular Model 4

5 Tabular Model Architecture and Solution Terminology v 5

6 Vertipaq DirectQuery In-Memory Cached xvelocity 6

7 Tabular Model Architecture Third Party Application Excel Reporting Services Power View Tabular Model DirectQuery In-Memory SQL Server Databases Non SQL Server Databases Files OData Cloud Services 7

8 Tabular Model Architecture Third Party Application Excel Reporting Services Power View Tabular Model In-Memory SQL Server Databases Non SQL Server Databases Files OData Cloud Services 8

9 Tabular Model Architecture Third Party Application Reporting Services Power View Tabular Model DirectQuery SQL Server Databases 9

10 Tabular Model Solutions Authoring (SSDT) Deployment Option Database Properties Connectivity demo 10

11 DirectQuery vs In-Memory Execution v 11

12 Querying DAX DirectQuery SQL DAX / MDX In-Memory VertiPaq SE Process Data Access Query Data Sources 12

13 DirectQuery Security Query DirectQuery SQL Role DirectQuery Impersonation Impersonate Current User Default (Data Source login) Data Source Impersonation Windows User Service Account 13

14 In-Memory Security Query In-Memory Role Row Level Security VertiPaq SE Process Data Source Impersonation 14

15 Other Security Consideration? Identity flow in Sharepoint aspx?sid=2027 BISM file RSDS file Simple Tabular Model Connection String 15

16 DirectQuery vs In-Memory Design v 16

17 Formula Compatibility Semantic differences xvelocity in-memory analytics engine (VertiPaq) SQL Server No Calculated Columns in DirectQuery 17

18 DAX Functions Some DAX Functions are not supported in DirectQuery: No equivalent calculations in relational engine No equivalent SQL expressions Performance of converted expressions E.g. TOTALYTD, SAMEPERIODLASTYEAR More information on BOL: 18

19 DirectQuery vs In-Memory Administration v 19

20 Processing Options (In-Memory) Mode Database Table Partition Process Default Process Full Process Data Process Clear Process Defrag Process Add Process Recalc Further reading: 20

21 Partitioning in DirectQuery One partition in Tabular Model Can take advantage of Partitioning and ColumnStore Index technology on SQL Server 2012 Partitioning / ColumnStore management done at the source database level 21

22 Partitioning in In-Memory In-Memory can have multiple mutually exclusive partitions To eliminate unnecessary processing and processor load on the AS servers Process C u r r e n t Mar 2012 Feb 2012 Jan 2012 In-Memory Partitions 22

23 DirectQuery vs In-Memory Pros and Cons v 23

24 In-Memory - Pros ALL functionality of Tabular Model Calculated Columns, ALL DAX functions Row Level security xvelocity in-memory analytics engine Many choices of client tool Excel, Power View, SSRS, MDX client* Many supported data sources 24

25 In-Memory - Cons AS requires Memory & CPU resources Caching Processing Querying Diligent Partitioning and Processing Out of date data 25

26 DirectQuery - Pros Real time access Scalable SQL Server permission SQL Server optimisation Consistency One Partition to manage No processing required 26

27 DirectQuery - Cons Restricted DAX functions Restricted client tools: PowerView, SSRS No Row Level Security Unless implemented at the Source Database level No Calculated Columns 27

28 The best of both worlds Hybrid Mode v 28

29 Tabular Model Solutions DirectQuery Mode On Off Query Mode DirectQuery DirectQuery with In-Memory In-Memory with DirectQuery In-Memory Connection String: DirectQueryMode=DirectQuery DirectQueryMode=In-Memory 29

30 Which Hybrid Mode? Query Mode DirectQuery with In-Memory In-Memory with DirectQuery Excel Connection String DirectQueryMode = InMemory No need to specify DirectQueryMode PowerView always uses the default Query Mode (i.e. the first bit of the Query Mode) can t change this! SSRS works with both Query Mode and Connection String combinations 30

31 Partitioning for Hybrid Mode One Partition for DirectQuery Processing Option: Never process this partition Multiple Partitions for In-Memory (No Overlap) C u r r e n t Aug 2012 DirectQuery Partitions Jul 2012 Jun 2012 In-Memory Partitions 31

32 Implications Inconsistency Data stale data in Cache Calculations semantic differences xvelocity vs SQL Security DirectQuery Design Flexibility at run time / client tools In-Memory (Excel) or DirectQuery (Real time)

33 Wrap Up v 33

34 Tabular Model Modes In-Memory Mode (default) DirectQuery Mode DirectQuery only DirectQuery with In-Memory In-Memory with DirectQuery 34

35 Do I DirectQuery? SQL Server database is optimised Columnstore Index Partitioning Contains all required business rules High Volume Can import non-sql data into one SQL Server database which will be the source 35

36 Further Reading Cathy Dumas blog SQLBI (Marco Russo & Alberto Ferrari) DAX resources (with links to Tabular Model) x-resource-center.aspx Julie Koesmarno s blog 36

37 Q & A signal@mssqlgirl.com 37

38 Thank you! v 38

39 MsSQLGirl.com