Best Practices for Multi-Dimensional Design using Cognos 8 Framework Manager

Size: px

Start display at page:

Download "Best Practices for Multi-Dimensional Design using Cognos 8 Framework Manager"

Derrick McCarthy
8 years ago
Views:

1 Best Practices for Multi-Dimensional Design using Cognos 8 Framework Manager Bruce Zornes The Boeing Company Business Intelligence & Competency Center

2 Best Practice: Knowledge and Experience Use Best Practice to reduce cost and produce quality Best Practice means Knowledge about the solution Theory is OK if produces results Use Patterns re-apply Knowledge Experience with the tools How to, what to do, when to do it People have Experience and Knowledge

is OK if produces results Use Patterns re-apply Knowledge Experience with

3 Best Practice: Use Design Principals Design quality in rather than Testing it in Framework Manager model is great place to implement quality design Understand the business case and drivers Understand the data schema - 3NF, star, recursive Understand the technology for optimization Teaming include DBA s and Server Admins Common model and keep it simple Understand your customer reports team too

Understand the data schema - 3NF, star, recursive Understand the technology for optimization

4 Best practices: Measures for success - Project Success criteria checklist expectations, funding, resources Complete solution phase the release(s) Our case: 11 business units, 20 cubes, one model, 4 rels Follow a process Dev Test Prod, CM, docs, SQA Design reviews and open communication Customer acceptance requirements met End-user acceptance navigation and UI, portals Cost and schedules hidden cost, unplanned work Reports performance team collaboration Support and operations team training, procedures

Design reviews and open communication Customer acceptance requirements met End-user acceptance navigation and UI, portals Cost

5 Customer Requirements: Business case How to produce quality criteria for design? Design choices: Star schema Three layer design Nice theory Too slow for large data? Narrow fact tables Familiar navigation Fast query response rqmt Minimize load time ETL Reporting mart

Design choices: Star schema Three layer design Nice theory Too

6 Best Practice: Framework Model Design Framework Model Three layer design [Physical Database Layer] SQL data source objects [Business Model Layer] Model objects and building blocks Business rules and data logic [Dimensional Presentation Layer] Multi-dimensional views and format

Model Layer] Model objects and building blocks Business rules and

7 Data Design: Parent-Child to Denormal table PARENT- CHILD tree Denormalized table for Reporting Note: Non-proprietary scrubbed test data

8 Framework model: Construct model objects Building blocks: parent-child Query Subjects (QS) and Query Sets Namespace scope Folders organize Naming convention User Naming for Reports Generation Specific QS Shortcut to physical Explicit relationship QS building block Placeholder Parents QS Query Set Union Leaf members

convention User Naming for Reports Generation Specific QS Shortcut to physical

9 Framework Model: Core Design Elements Conformed star schema (multiple fact tables) Parent-child to denormal dimension tables Unbalanced hierarchies Building block Query Subjects (QS) Parent place-holders at each generation Unions via Querysets: QSParent to QSGen Create leaf member level coalesce(case when) Link dimensions to facts at leaf level Narrow fact table - one key per dim Create dimensional views from building blocks

at each generation Unions via Querysets: QSParent to QSGen Create leaf member level coalesce(case when)

10 Framework model: with SQL Generation Building blocks: Generate Top-of-Tree SQL Cognos Connection with E123_SCENARIO21 as (select E123_SCENARIO.ESB_FKID, E123_SCENARIO.ESB_PARENT, E123_SCENARIO.ESB_CHILD, E123_SCENARIO.ALIAS1, E123_SCENARIO.DATASTORE, E123_SCENARIO.GENERATION from "FabFBI Datasource"..AFAB.E123_SCENARIO), ScenarioGen2 as (select E123_SCENARIO21.ESB_FKID as ESB_FKID, E123_SCENARIO21.ESB_PARENT as PARENT, E123_SCENARIO21.ESB_CHILD as CHILD, E123_SCENARIO21.ALIAS1 as ALIAS1, E123_SCENARIO21.DATASTORE as DATASTORE, E123_SCENARIO21.GENERATION as GENERATION from E123_SCENARIO21 where (E123_SCENARIO21.GENERATION in ('2')) ), Primary reference to PARENT- CHILD physical table Schema owner Top-of-Tree constraint

E123_SCENARIO), ScenarioGen2 as (select E123_SCENARIO21.ESB_FKID as ESB_FKID, E123_SCENARIO21.ESB_PARENT as PARENT, E123_SCENARIO21.ESB_CHILD as CHILD, E123_SCENARIO21.

11 Framework model: SQL Magic for Placeholders Building blocks: QuerySet union with placeholders SQL NULL types coerced ScenarioGen23Union517(ESB_FKID, PARENT, CHILD, ALIAS1, DATASTORE, GENERATION) as ((select E123_SCENARIO15.ESB_FKID as ESB_FKID, E123_SCENARIO15.ESB_CHILD as PARENT, cast( null as varchar (80)) as CHILD, cast(null as varchar (80)) as ALIAS1, cast(null as varchar (80)) as DATASTORE, E123_SCENARIO15.GENERATION as GENERATION from E123_SCENARIO15 where (E123_SCENARIO15.GENERATION in ('2')) ) UNION (select E123_SCENARIO16.ESB_FKID as ESB_FKID, E123_SCENARIO16.ESB_PARENT as PARENT, E123_SCENARIO16.ESB_CHILD as CHILD, E123_SCENARIO16.ALIAS1 as ALIAS1, E123_SCENARIO16.DATASTORE as DATASTORE, E123_SCENARIO16.GENERATION as GENERATION from E123_SCENARIO16 where (E123_SCENARIO16.GENERATION in ('3')) ) ), Promote CHILD as PARENT NULL CHILD Placeholder Parents at Gen 2 UNION to Members at Gen 3

ESB_CHILD as PARENT, cast( null as varchar (80)) as CHILD, cast(null as varchar (80)) as ALIAS1, cast(null as varchar (80)) as DATASTORE, E123_SCENARIO15.

12 Framework model: SQL tricks for Leaf members Building blocks: coalesce(case(tree)) as LEAF_MEMBER ScenarioLeaf as (select coalesce( case when (ScenarioGen34Union18.CHILD is NULL) then null else ScenarioGen34Union18.ESB_FKID end,case when (ScenarioGen23Union19.CHILD is NULL) then null else ScenarioGen23Union19.ESB_FKID end,case when (ScenarioGen2.CHILD is NULL) then null else ScenarioGen2.ESB_FKID end ) as LEAF_FKID, Case switch to NOT promote placeholders Walk trees in reverse coalesce( case when (ScenarioGen34Union18.CHILD is NULL) then null else ScenarioGen34Union18.PARENT end,case when (ScenarioGen23Union19.CHILD is NULL) then null else ScenarioGen23Union19.PARENT end,case when (ScenarioGen2.CHILD is NULL) then null else ScenarioGen2.PARENT end ) as LEAF_PARENT, coalesce(scenariogen34union18.child,scenariogen23union19.child,scenariogen2.child ) as LEAF_CHILD, Build leaf members

Union19.ESB_FKID end,case when (ScenarioGen2.CHILD is NULL) then null else ScenarioGen2.ESB_FKID end ) as LEAF_FKID, Case switch to NOT promote placeholders Walk trees in reverse 3 2 1.

13 Framework model: Construct leaf and case logic Building blocks: final assembly leaf model objects Placeholders promote members No cross joins allowed! Model objects need relationships Leaf links to fact tables Coalesce Case switch Coalesce is tree walker Scan right to first non-null Conditional members

14 Framework model: SQL loop joins Building blocks: SQL loop joins w/placeholder filter Join-Join causes repeating output ScenarioLeaf as. from ScenarioGen2 left outer join ScenarioGen23Union517 ScenarioGen23Union19 on (ScenarioGen2.CHILD = ScenarioGen23Union19.PARENT) left outer join ScenarioGen34Union214 ScenarioGen34Union18 on (ScenarioGen23Union19.CHILD = ScenarioGen34Union18.PARENT) where (not (coalesce( ScenarioGen34Union18.CHILD,ScenarioGen23Union19.CHILD,ScenarioGen2.CHILD) is NULL)) ) My tricky SQL? (really just ANSI sql F F041-05) Put em in & take em out Seems like extra work for unbalanced

PARENT) left outer join ScenarioGen34Union214 ScenarioGen34Union18 on (ScenarioGen23Union19.CHILD = ScenarioGen34Union18.

15 Framework model: Determinants fix multi-grain Building blocks: use determinants on leaf data groups Leaf model objects are generation specific Grouped by generation (nice to have: determinant filters) Naming suffix CHILD1 is top-of-tree Uniqueness based on composite business keys

specific Grouped by generation (nice to have: determinant filters)

16 Framework model: Dimensional construction Presentation Layer: connecting the pieces Dimensional model construction from Model objects Build presentation layer from BP model building blocks Drag drop & test Hierarchy name Member levels Unique leaf level Set businesskey and caption

objects Build presentation layer from BP model building blocks Drag

17 Framework model: Test in Query Studio Published package: validate unbalanced structure Suppress first two layers in final publish All reports connect to presentation layer First look at dimensional output Unbalanced member names promoted to leaf_level Leaf members connect to facts Note: Non-proprietary scrubbed test data

presentation layer First look at dimensional output Unbalanced member names

18 Framework model: Design challenges met Interface to legacy system OLAP data cubes Iterative design not all data structure understood ETL expensive and difficult Minimize business rules in ETL Narrow fact tables - faster than wide Link all tables via SID (business keys) High volume data (200 Million rows) partitioned Users expect 2-4 seconds click report response

business rules in ETL Narrow fact tables - faster than wide Link all tables via SID

19 Framework model: Test in Analysis Studio Verify to legacy system OLAP data cubes User familiar names Ready to build reports Is it fast In AS secs In QS 3-5 secs Do aggregate values match Mixed grains Note: Non-proprietary scrubbed test data - all aggregate values are row counts

10-30 secs In QS 3-5 secs Do aggregate values match Mixed grains Note:

20 Example 1 of Unbalanced Hierarchy Unbalanced Dimensions: promotion of members to leaf level Facts link to leaf member level Measures dimension Data Rollups Fringe Dollar Adjustments Mixedgrain data Regular Dollars Link by Root_child Note: Non-proprietary scrubbed test data

dimension Data Rollups Fringe Dollar Adjustments Mixedgrain data

21 Example 2 of Unbalanced Hierarchy - SHARED Unbalanced Dimensions: complex network members SHARED dimension members have root_child Cloned parents and child data Cloned parent Cloned child data from root_child Attached to different ancestor Note: Non-proprietary scrubbed test data

22 Design strategy: Model meets the needs Who: model end-user vs. reports end-user Three layer design - only two Developer model not end-user 3NF structure Denormalize in rdbms Adapt to Teams, Roles and Org maturity Priorities may change Key success drivers Compromise and validate Rely on principals

23 Best practices: Measures for success check! So how d we do? Success criteria / performance indicators First release due: July 2006, Delivered first reports: March 2007 Followed process: Dev Test Prod, docs Tech reviews: hold daily status with PMs Customer and End-user acceptance: improved their data quality Really appreciate common business model and Cognos rpts Cost and schedules: re-planned to meet realities Performance: designed-in, mat views added, DBA involved Support and operations: OJT, handover challenges, teaming improvements, working together, find-a-way Users: want more BI reports and faster deployments

24 Evaluation Forms Each completed evaluation qualifies you to win one of five $100 American Express gift certificates given away daily. Complete evaluations for every session you attend and qualify to win an additional $500! An overall conference survey will be available at Cognos Central on Thursday morning, and will also be ed to you. Complete the survey by May 25, 2007, and you'll qualify to win $500!

MS 20467: Designing Business Intelligence Solutions with Microsoft SQL Server 2012

MS 20467: Designing Business Intelligence Solutions with Microsoft SQL Server 2012 Description: This five-day instructor-led course teaches students how to design and implement a BI infrastructure. The