Big Data Analytics with IBM Cognos Dynamic Cubes IBM Redbooks Solution Guide

Transcription

1 Big Data Analytics with IBM Cgns Dynamic Cubes IBM Redbks Slutin Guide IBM Cgns Dynamic Cubes, which is a feature f the Cgns Business Intelligence V10.2 sftware, cmplements the existing query engine. It extends Cgns scalability t enable speed-f-thught analytics ver terabytes f enterprise data, withut being frced t rely n a new data-warehusing appliance. With this capability, which adds a level f query intelligence, yu can unleash the pwer f yur large enterprise data warehuse. Figure 1 illustrates hw the Cgns Dynamic Cubes feature is integrated int the Cgns Business Intelligence stack. Figure 1. Cgns Dynamic Cubes integrated int the Cgns Business Intelligence stack Big Data Analytics with IBM Cgns Dynamic Cubes 1

2 Did yu knw? In 2012, abut 2.5 quintillin bytes f data were created every day. Scial media is nw generating petabytes f data per day. Business value With scial data generating petabytes per day, and instrumented devices becming the nrm, data vlume grwth is accelerating at an unprecedented pace. Big data is a grwing business trend, with data frm uncnventinal surces having the ptential t be business disruptrs. Hwever, befre the pwer f these new surces can be fully used, yu must understand what is happening within yur wn business. Understanding yur wn business is added value f a data warehuse and is why taking full advantage f these data hldings is a critical first step t using these new surces f data. In additin, any rganizatin that relies n instrumented infrastructures can maximize the efficiency f its peratins. Analytics is key t accmplishing this type f ptimizatin, leading t cncrete business results. Data warehuses are the recgnized fundatin fr enterprise analytics. By using data warehuses, an rganizatin can bring tgether cleansed data frm separate surces f input, bth internal and external, such as frm partners r suppliers. Instead f garbage in, garbage ut infrmatin t supprt decisin-making, a cnsistent and cnslidated enterprise-wide view f data frm a business prvides the fundatin t imprve yur business. Building upn a trusted infrmatin platfrm fr analytics is a key cntributr t lng-term business health. Nt nly d data warehuses enable higher quality infrmatin, they enable high-perfrmance data access fr analytic-style applicatins. IBM Cgns Dynamic Cubes technlgy helps t use the cre strengths f an enterprise data warehuse and take it t the next level f perfrmance fr analytics, making the deplying and tuning easier and faster. Slutin verview The IBM Cgns Dynamic Cubes technlgy is meant t slve a specific but grwing business prblem, enabling high-perfrmance interactive analysis ver terabytes f data in an enterprise data warehuse. As data vlumes grw, analyzing that data with speed-f-thught perfrmance can be challenging. Even with mdern data warehuse technlgy, sme peratins require significant cmputatin r data mvement. This cmputatin r mvement creates delays and reduces the satisfactin f business users wh want t perfrm these analyses. Varius ways exist t accmplish perfrmance ver large vlumes f data. Frm self-cntained cubes t large in-memry appliances, different vendrs are emplying variatins f similar methdlgies t give business users timely respnse times. The Cgns Dynamic Cubes technlgy aims t give maximum flexibility in hw memry is used t accelerate interactive analysis ver terabytes f data s that yu can evlve yur deplyments ver time. Big Data Analytics with IBM Cgns Dynamic Cubes 2

3 Slutin architecture The IBM Cgns Dynamic Cubes technlgy is part f the IBM Cgns Business Intelligence query stack and is available with existing IBM Cgns entitlements. It prvides a new and pwerful tl t enable high perfrmance analytics ver large data warehuses (see Figure 2). Figure 2. A high-level representatin f the Cgns Dynamic Cubes architecture The Cgns Dynamic Cubes slutin cnsists f IBM Cgns Cube Designer (a mdeling tl ), a dynamic cube bject in the administratin envirnment (which becmes the data surce), and Aggregate Advisr (a wizard) that is started frm Dynamic Query Analyzer: IBM Cgns Cube Designer Cgns Cube Designer is a mdeling tl that brings tgether the best mdeling principles frm past successful mdeling technlgy, with a mdern and extensible architecture. The first step t deplying Cgns Dynamic Cubes is t mdel with the Cgns Cube Designer. Big Data Analytics with IBM Cgns Dynamic Cubes 3

4 IBM Cgns Dynamic Cubes Server After a dynamic cube is designed and deplyed, it becmes available in the Cgns envirnment and acts as the data surce t the interface layer fr dynamic cube applicatins. It manages all aspects f data retrieval and leverages memry t maximize respnsiveness, giving yu full flexibility t manage what is in memry and when yu want t refresh in-memry data. Yu manage dynamic cubes in the Cgns administratin envirnment. A dynamic cube cntains several in-memry elements t drive perfrmance: Metadata members Aggregates Data Results sets Expressins Aggregate Advisr (part f IBM Cgns Dynamic Query Analyzer) The Aggregate Advisr scans cube definitins and usage lgs. It als recmmends aggregates t imprve perfrmance. This apprach helps t mre easily address specific perfrmance prblems. Usage scenaris The Cgns Dynamic Cubes slutin applies t the fllwing usage scenaris. Multigrain fact scenaris A cmmn mdeling prblem is multigranularity, which ccurs when levels f dimensinal detail fr facts differ. In yur data surce, the level f infrmatin in a dimensin can be mre precise than the fact data f sme fact tables in which the dimensin takes part. Fr example, a time dimensin can have dimensin infrmatin fr the levels f year, quarter, mnth, and day. Fr a sales fact table, the facts exist at the day level. Fr a fact table with planned sales values, the fact grain r level is prbably at a higher level f detail, such as mnth. This difference in fact grain can make it difficult t plan queries crrectly if a reprt user included a level belw the fact grain in the reprt. The methd fr handling this scenari is t create rle-playing dimensins fr each instance f a dimensin that has varying levels f granularity t different fact tables. In this example, yu need t mdel a time dimensin dwn t the day level fr the sales cube and t mdel a time dimensin dwn t the mnth level fr the sales target cube. Fr the day level, the relatinship is frmed between the apprpriate keys in the fact table and the day level. Fr the mnth level, the relatinship is frmed between the mnth level and the fact table keys. If yu want t create reprts that used data frm bth cubes, yu can create a virtual cube. The virtual cube can have the tw time dimensins merged. The levels f bth surce cubes will be merged. The virtual cube might have levels that are belw the grain f measures frm ne f the surce cubes. Queries that are made in the virtual cube that use dimensin levels belw the grain f a measure will return null values, ensuring that the cnsumers f the cube d nt have duble cunting. Perhaps yu have the fllwing cubes: A sales fact cube, where the fact grain fr the time dimensin is at the day level A planned sales cube, where the fact grain fr the time dimensin is at the mnth level Big Data Analytics with IBM Cgns Dynamic Cubes 4

5 By using a virtual cube that emplys bth f these cubes as its surce, yu can make a query with day-level bjects (r a member f that level, depending n the studi that yu are using) against the sales facts. Then, yu get the results and the expected null values fr the planned sales facts. If yu used a time dimensin grain that was cmmn t bth fact tables, yu get nn-null values fr measures frm bth fact tables. Aggregate cubes A secnd scenari addresses situatins where a higher level f granularity is desirable thrugh aggregate awareness. This awareness is accmplished thrugh aggregate cubes, which are supprted by IBM Cgns Dynamic Cubes. Aggregate cubes define the measures, dimensins, and dimensin grain by which queries can be ruted t aggregate tables rather than t the detail fact table. Because aggregate tables stre fact data at a higher-than-detail level f granularity, the time necessary t aggregate values during the query can be lessened, imprving perfrmance. A query can be ruted t the aggregate table if all the measures and dimensin hierarchies f the query exist in the aggregate cube definitin. Nt all f the dimensins and measures in the aggregate cube need t be in the query. The bjective f mdeling an aggregate cube is t establish rules by which the dynamic cube can detect when it can rute a query t an aggregate table. This task is dne by specifying a mapping frm the identifiers in the dimensins and measures in the cube that have scpe t the aggregate table, t the identifiers in the aggregate table. If necessary, mapping is dne t its related tables in a rlled-up dimensin schema. This aggregate cube ruting directs a query nly t the aggregate table fr a query that uses bjects frm a dimensin grain at r abve the grain f the mapping between it and the aggregate table. Therefre, using bjects frm a grain belw the mapping grain des nt cause duble-cunting because that query cntinues t rute t the detail fact table. Integratin IBM Cgns Dynamic Cubes is tightly integrated int the Cgns Business Intelligence stack, and its data can be surfaced thrugh any f the Cgns interfaces. With this methd, existing custmers can integrate this technlgy int their applicatin envirnment withut affecting existing users. Such users are already familiar with interfaces such as Reprt Studi, Business Wrkspace, and Business Wrkspace Advanced (previusly named Business Insight and Business Insight Advanced). Different data requirements require different data slutins. One data path cannt be prficient at slving widely different data prblems. Therefre, IBM Cgns has technlgies that are built t suit specific applicatin requirements. Table 1 can help yu better understand the primary use case fr each technlgy. Hwever, carefully cnsider yur individual applicatin requirements when yu make such a decisin. Big Data Analytics with IBM Cgns Dynamic Cubes 5

6 Table 1. Use cases fr Cgns Dynamic Cubes technlgy Cube technlgy IBM Cgns TM1, in-memry cube technlgy with write-back supprt Dynamic Cubes, in-memry acceleratr fr dimensinal analysis PwerCubes, file-based cube with pre-aggregatin OLAP Over Relatinal (OOR), dimensinal view f a relatinal database Primary use cases It is ptimal fr write-back, what-if analysis, planning and budgeting, r ther specialized applicatins. It can handle medium data vlumes. The cube is run 100% in memry. Aggregatin ccurs n demand, which can affect perfrmance with high data and high user vlumes. It is ptimal fr read-nly reprting and analytics ver large data vlumes. It prvides extensive in-memry caching fr perfrmance, backed by aggregate awareness t use the pwer and scalability f a relatinal database. A star r snwflake schema is required in the underlying database (used t maximize perfrmance). It is ptimal t prvide cnsistent interactive analysis experience t many users when the data surce is an peratinal r transactinal system, and a star r snwflake data structure cannt be achieved. The pre-aggregated cube architecture requires careful management, by using cube grups t achieve scalability. Data latency is inherent with pre-aggregated cube technlgy, where data mvement int the cube is required. It is ptimal t easily create a dimensinal data explratin experience ver lw data vlumes in an peratinal r transactinal system, and where latency must be carefully managed. Caching n the Dynamic Query server helps perfrmance. Prcessing that is assciated with peratinal r transactinal system affects perfrmance. Supprted platfrms Fr infrmatin abut sftware envirnments that are supprted in IBM Cgns Business Intelligence V10.2, see "Cgns BI 10.2 Sftware Envirnments" at: Ordering infrmatin Table 2 lists the prgrams that IBM Cgns Business Intelligence cntains. Table 2. Prgram numbers and names Prgram number Prgram name 5724-W12 IBM Cgns Business Intelligence V W68 IBM Cgns Business Intelligence PwerPlay V W19 IBM Cgns Business Intelligence Reprting V W13 IBM Cgns Data Manager V W20 IBM Cgns Mbile V Big Data Analytics with IBM Cgns Dynamic Cubes 6

7 Related infrmatin Fr mre infrmatin, see the fllwing dcuments: IBM Cgns Business Intelligence V10.1 Handbk, SG IBM Cgns Business Intelligence V10.1: Intelligence Unleashed, REDP IBM Cgns Business Intelligence Infrmatin Center Dynamic Cubes Installatin and Cnfiguratin Guide Dynamic Cubes User Guide Dynamic Query Analyzer Installatin and Cnfiguratin Guide Dynamic Query Analyzer User Guide IBM annuncement letters and sales manuals On this page, enter IBM Cgns Business Intelligence, and click Search. On the next page, narrw yur search results by infrmatin type, gegraphy, language, r all three ptins. Big Data Analytics with IBM Cgns Dynamic Cubes 7

8 Ntices This infrmatin was develped fr prducts and services ffered in the U.S.A. IBM may nt ffer the prducts, services, r features discussed in this dcument in ther cuntries. Cnsult yur lcal IBM representative fr infrmatin n the prducts and services currently available in yur area. Any reference t an IBM prduct, prgram, r service is nt intended t state r imply that nly that IBM prduct, prgram, r service may be used. Any functinally equivalent prduct, prgram, r service that des nt infringe any IBM intellectual prperty right may be used instead. Hwever, it is the user's respnsibility t evaluate and verify the peratin f any nn-ibm prduct, prgram, r service. IBM may have patents r pending patent applicatins cvering subject matter described in this dcument. The furnishing f this dcument des nt give yu any license t these patents. Yu can send license inquiries, in writing, t: IBM Directr f Licensing, IBM Crpratin, Nrth Castle Drive, Armnk, NY U.S.A. The fllwing paragraph des nt apply t the United Kingdm r any ther cuntry where such prvisins are incnsistent with lcal law : INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Sme states d nt allw disclaimer f express r implied warranties in certain transactins, therefre, this statement may nt apply t yu. This infrmatin culd include technical inaccuracies r typgraphical errrs. Changes are peridically made t the infrmatin herein; these changes will be incrprated in new editins f the publicatin. IBM may make imprvements and/r changes in the prduct(s) and/r the prgram(s) described in this publicatin at any time withut ntice. Any references in this infrmatin t nn-ibm Web sites are prvided fr cnvenience nly and d nt in any manner serve as an endrsement f thse Web sites. The materials at thse Web sites are nt part f the materials fr this IBM prduct and use f thse Web sites is at yur wn risk.ibm may use r distribute any f the infrmatin yu supply in any way it believes apprpriate withut incurring any bligatin t yu. Infrmatin cncerning nn-ibm prducts was btained frm the suppliers f thse prducts, their published annuncements r ther publicly available surces. IBM has nt tested thse prducts and cannt cnfirm the accuracy f perfrmance, cmpatibility r any ther claims related t nn-ibm prducts. Questins n the capabilities f nn-ibm prducts shuld be addressed t the suppliers f thse prducts. This infrmatin cntains examples f data and reprts used in daily business peratins. T illustrate them as cmpletely as pssible, the examples include the names f individuals, cmpanies, brands, and prducts. All f these names are fictitius and any similarity t the names and addresses used by an actual business enterprise is entirely cincidental. Any perfrmance data cntained herein was determined in a cntrlled envirnment. Therefre, the results btained in ther perating envirnments may vary significantly. Sme measurements may have been made n develpment-level systems and there is n guarantee that these measurements will be the same n generally available systems. Furthermre, sme measurement may have been estimated thrugh extraplatin. Actual results may vary. Users f this dcument shuld verify the applicable data fr their specific envirnment. COPYRIGHT LICENSE: This infrmatin cntains sample applicatin prgrams in surce language, which illustrate prgramming techniques n varius perating platfrms. Yu may cpy, mdify, and distribute these sample prgrams in any frm withut payment t IBM, fr the purpses f develping, using, marketing r distributing applicatin prgrams cnfrming t the applicatin prgramming interface fr the perating platfrm fr which the sample prgrams are written. These examples have nt been thrughly tested under all cnditins. IBM, therefre, cannt guarantee r imply reliability, serviceability, r functin f these prgrams. Cpyright Internatinal Business Machines Crpratin All rights reserved. Nte t U.S. Gvernment Users Restricted Rights -- Use, duplicatin r disclsure restricted by GSA ADP Schedule Cntract with IBM Crp. Big Data Analytics with IBM Cgns Dynamic Cubes 8

9 This dcument was created r updated n December 7, Send us yur cmments in ne f the fllwing ways: Use the nline Cntact us review frm fund at: ibm.cm/redbks Send yur cmments in an t: redbk@us.ibm.cm Mail yur cmments t: IBM Crpratin, Internatinal Technical Supprt Organizatin Dept. HYTD Mail Statin P Suth Rad Pughkeepsie, NY U.S.A. This dcument is available nline at Trademarks Internatinal Business Machines Crpratin in the United States, ther cuntries, r bth. These and ther IBM trademarked terms are marked n their first ccurrence in this infrmatin with the apprpriate symbl ( r ), indicating US registered r cmmn law trademarks wned by IBM at the time this infrmatin was published. Such trademarks may als be registered r cmmn law trademarks in ther cuntries. A current list f IBM trademarks is available n the web at: The fllwing terms are trademarks f the Internatinal Business Machines Crpratin in the United States, ther cuntries, r bth: Cgns IBM PwerPlay Redbks Redbks (lg) TM1 The fllwing terms are trademarks f ther cmpanies: Other cmpany, prduct, r service names may be trademarks r service marks f thers. Big Data Analytics with IBM Cgns Dynamic Cubes 9