White Paper Data Abstractin Best Practices with Cisc Data Virtualizatin Executive Summary Enterprises are seeking ways t imprve their verall prfitability, cut csts, and reduce risk by prviding better access t infrmatin assets. Significant vlumes f cmplex, diverse data spread acrss varius technlgy and applicatin sils make it difficult fr rganizatins t meet these bjectives. T further cmplicate matters, there are a range f prblems such as separate access mechanisms, syntax, and security fr each surce; lack f prper structure fr business user r applicatin cnsumptin and reuse; incmplete r duplicate data; and a mixture f latency issues. Data abstractin vercmes these challenges by transfrming data frm its native structure and syntax int views and data services that are much easier fr applicatin develpers t use. Enterprises can apprach data abstractin three ways: manual data abstractin, creatin f data warehuse schemas, and data virtualizatin. Of the three appraches, data virtualizatin is the superir slutin fr data abstractin because it prvides the mst flexibility and agility t quickly retrieve data frm different data lcatins and surces in real time. Cisc Data Virtualizatin is cmpsed f different layers that frm a data reference architecture that supprts multiple cnsuming applicatins. The architecture aligns clsely with analyst best practices mapped ut by bth Frrester and Gartner n the tpic f data virtualizatin. This dcument explains data abstractin best practices using Cisc Data Virtualizatin that will enable yur cmpany t access the right data n demand, gain agility and efficiency, maintain end-t-end cntrl, and increase security f yur data acrss all yur data resurces. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 1 f 9
Business and IT Challenges with Data Management With large amunts f cmplex and diverse data spread acrss different applicatin sils, enterprises are finding it difficult t gain access t their data. This large spread f diverse data als makes it difficult fr enterprises t cut csts and reduce risk. A range f prblems such as: separate access mechanisms, syntax, and security fr each surce; a lack f prper structure fr business user r applicatin cnsumptin and reuse; incmplete data r duplicate data; and a mixture f latency issues demands a data management slutin that can simplify data access. (See Figure 1.) Figure 1. Data Abstractin Challenges Hw Data Abstractin Overcmes These Challenges Data abstractin vercmes surce-t-cnsumer incmpatibility by transfrming data frm its native structure and syntax int reusable views and data services that are easy fr applicatin develpers t understand and cnsume. Sme data abstractin appraches enterprises use tday include: Manual data abstractin: Sme rganizatins manually build data abstractin in Java r use business prcess management (BPM) tls. Unfrtunately, these are ften rigid and inefficient. Such appraches are nt effective fr large data sets because they lack the rbust federatin and query ptimizatin functins required t meet data cnsumers rigrus perfrmance demands. Data warehuse schemas: Data mdeling strategies fr dimensins, hierarchies, facts, and ther data rganizatin methds are well dcumented. Hwever, the data warehusing apprach brings high csts and lack f agility. Als, data warehuse-based schemas d nt include the many new grups f data (big data, clud data, external data services, and mre) that reside utside the data warehuse. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 2 f 9
Data Virtualizatin Is a Superir Slutin fr Data Abstractin Data virtualizatin is an ptimal way t implement data abstractin fr enterprises. Frm an enterprise architecture pint f view, the Cisc Data Virtualizatin slutin frms semantic abstractin, r a data services layer, in supprt f multiple cnsuming applicatins. The middle layer f reusable services decuples the underlying surce data and cnsuming slutin layers. This prvides the flexibility required t deal with each layer in the mst effective manner, as well as the agility t wrk quickly acrss layers as applicatins, schemas, r underlying data surces change. (See Figure 2.) Figure 2. Data Abstractin with Cisc Data Virtualizatin Data abstractin with data virtualizatin helps enterprises achieve a number f primary bjectives, including: The right business infrmatin at the right time: Fulfill cmplete infrmatin needs n demand by linking multiple diverse data surces tgether fr delivery in real time. Business and IT mdel alignment: Gain agility, efficiency, and reuse acrss applicatins with an enterprise infrmatin mdel r lgical business mdel. Knwn as the cannical mdel, this abstracted apprach vercmes data cmplexity, structure, and lcatin issues. Business and IT change insulatin: Insulate cnsuming applicatins frm changes in the surce and vice versa. Develpers create their applicatins based n a mre stable view f the data, allwing nging changes and relcatin f physical data surces withut affecting cnsumers. End-t-end cntrl: Use a single platfrm t design, develp, manage, and mnitr data access and delivery prcesses acrss multiple surces and cnsumers. Mre secure data: Cnsistently apply data security rules acrss all data surces and cnsumers with unified security methds and cntrls. Cisc Data Virtualizatin Data Abstractin Reference Architecture Figure 3 utlines the layers that frm the data abstractin reference architecture. Architects and analysts can use this as a guide when abstracting data using the data virtualizatin platfrm. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 3 f 9
Figure 3. Cisc Data Virtualizatin Data Abstractin Reference Architecture The varius layers included in this reference architecture are: Data cnsumers: Client applicatins need t retrieve data in different frmats and prtcls that they understand. Cisc Data Virtualizatin delivers the data t cnsumers using the mst ppular standards, including SOAP, REST, JDBC, and s n. Applicatin layer: The applicatin layer serves t map the business layer int the applicatin frmat that each cnsumer wants t see. Examples include frmatting int XML fr web services r creating views with different alias names that match the way the cnsumers are used t seeing their data. Business layer: The business layer is built n the idea that the business has a standard r cannical way t describe primary business grups such as custmers and prducts. In the financial industry, fr example, infrmatin is ften accessed accrding t financial instruments r issuers. Typically, a data mdeler wuld wrk with business experts and data prviders t define a set f lgical r cannical views that represent these business grups. These views are reusable cmpnents that can and shuld be used acrss business lines by multiple cnsumers. Physical layer: The physical layer prvides access t underlying data surces and perfrms a physical t lgical mapping by integrating physical metadata and frmatting views: Physical metadata: Data that is essentially imprted frm the physical data surces and used t nbard the metadata required by the data abstractin layer t perfrm its mapping functins. As an as-is layer, grup names and attributes are never changed in this layer. Frmatting views: These prvide a way t map the physical metadata int the data virtualizatin layer by aliasing the physical names t lgical names. The frmatting views can facilitate simple tasks such as value frmatting, data type casting, derived 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 4 f 9
clumns, and light data quality mapping. This layer is derived frm the physical surces and perfrms a ne-t-ne mapping between the physical surce attributes and their crrespnding lgical/cannical attribute name. Als, this layer serves as a buffer between the physical surce and the lgical business layer views. Therefre, caching may be intrduced at this level when it makes sense. Rebinding t different physical views during deplyment is anther rle these views take n. Naming cnventins are very imprtant and intrduced in this layer. Data surces: The data surces are the physical infrmatin assets that exist within and utside an rganizatin. These assets may be databases, packaged applicatins such as SAP, web services, Excel spreadsheets and s n. Enabling the Frrester Data Virtualizatin Visin Frrester Research prvides the fllwing guidance fr data abstractin in its Data Virtualizatin Reaches Critical Mass reprt. 1 Frrester says the mst successful implementatin f data virtualizatin uses a layered architecture that cmbines physical and virtual data stres at the apprpriate levels t fit different perfrmance requirements fr different areas within the cmpany. By funneling mappings f different surce data thrugh cannical business mdels, this creates an hurglass-shaped architecture. Besides using cannical mdels in the middle f the architecture, there are tw ther imprtant characteristics f effective data abstractin t nte. First, physical data surces tend t be lcated mre in the staging layers clse t the actual data, whereas virtual data ccurs mre as the data gets clser t the end users. Secnd, a final virtual mapping layer prvides data t cnsumers in the prper frmat. (See Figure 4.) Figure 4. Frrester Research Data Virtualizatin Reaches Critical Mass There is a striking resemblance between the Frrester and Cisc Data Virtualizatin best practice architectures shwn in Figure 5. 1 Hpkins, Brian. (2011) Data Virtualizatin Reaches Critical Mass. Frrester. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 5 f 9
Figure 5. Cmparisn f Frrester and Cisc Data Virtualizatin Best Practice Architecture Enabling the Gartner Discipline f Data Integratin Since 2005, Gartner has been researching the cncept f data services in relatin t the brader business and IT evlutin. Mre recently, Gartner discussed the discipline f data integratin at the Business Intelligence Summit, 2 as shwn in Figure 6: Figure 6. Cisc Data Virtualizatin Data Abstractin Architecture Implements Gartner Discipline f Data Integratin Gartner says data integratin includes the practices, architectural techniques, and tls used t gain cnsistent access t data, regardless f data structure type r grup, in rder t meet the requirements f applicatins and business prcesses. Data integratin capabilities are an imprtant part f an infrmatin-fcused infrastructure and will drive the alignment and delivery f data t supprt BI and perfrmance management. New challenges with data are creating a glbal surge f investment in data integratin. Business factrs such as the desire t increase speed t market r gain agility with business prcesses are causing rganizatins t manage their data differently. T accmplish these initiatives, cmpanies need better visibility f their data in rder t truly understand their perfrmance and peratins. The data virtualizatin data abstractin reference architecture can be used t implement the Gartner discipline f data integratin as fllws: 2 Ted Friedman. Advancing yur Data Integratin Cmpetency in Supprt f Analytics. Presented at Business Intelligence Summit. Gaylrd Cnventin Center, Grapevine TX. Gartner Research. Inc. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 6 f 9
Practices: Cisc Data Virtualizatin has shaped the best practices that custmers use tday t implement data virtualizatin in their rganizatins, as well as influenced the practices recmmended by leading IT analysts and system integratrs. This thught leadership and realwrld experience help users gain cnfidence when deplying data virtualizatin in their rganizatin. Architectural techniques: The Cisc Data Virtualizatin Plan and Build Service brings a wealth f knwledge and skills t help users architect their data virtualizatin slutins. Architectural techniques are included in the service, which is designed t help custmers get a prject up and running quickly and maximize their return. Custmers are intrduced t the Data Abstractin Best Practices Technical Guide, which is used as an architectural techniques blueprint. Tls: The Data Virtualizatin platfrm prvides a cmplete and prven tl t implement the Gartner Discipline f data integratin. Business cntext services: In Data Virtualizatin s reference architecture, the applicatin layer prvides the mechanisms fr mapping and publishing views r web services in the cntext f the applicatins. The applicatin layer maps int Gartner business cntext services. Applicatin cnsumers require delivery f data using different prtcls. Within the Data Virtualizatin reference mdel, data cnsumers use a variety f standard prtcls, including JDBC, ODBC, SOAP/HTTP, REST and ADO/.Net t access needed data. These standard prtcls supprt the BI, MDM, web service APIs, and enterprise bjects cnsumers included by Gartner. Semantic/lgical services: Gartner semantic/lgical services prvide fr the transfrmatin f the physical mdel int the business cntext view f the infrmatin. The terms lgical and semantic are ften referred t as cannical. It is a way f defining a cmmn data dictinary acrss the business. The terms r attributes frm this data dictinary are gruped tgether int semantically similar entities. Data Virtualizatin supprts these needs with its frmatting views. Data manipulatin services: Gartner manipulatin functins include access, strage, and delivery, which align with Data Virtualizatin s physical layer. This is where intrspectin, discvery, and surce data access tls expse the physical layer. Increasingly, Data Virtualizatin is prviding access t a wide array f data surces, including relatinal, service riented, file, packaged applicatins, and big data. Optimizatin: Bth Gartner and Cisc view ptimizatin as spanning the entire architecture frm surce t cnsumer, during bth design and runtime, perfectly matching hw Data Virtualizatin s ptimizers wrk. Recent Gartner research n the lgical data warehuse extends and enhances this guidance. Summary f Primary Benefits Data abstractin bridges the gap between the riginal frm f business needs and surce data. This best practice implementatin f Cisc Data Virtualizatin prvides the fllwing benefits: 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 7 f 9
Simplifies infrmatin access: Bridge business and IT terminlgy and technlgy s bth can succeed. Generates cmmn business view f the data: Gain agility, efficiency, and reusability acrss applicatins using an enterprise infrmatin mdel r cannical mdel. Prvides mre accurate data: Cnsistently apply data quality and validatin rules acrss all data surces. Prvides mre secure data: Cnsistently apply data security rules acrss all data surces and cnsumers using a unified security framewrk. Gains end-t-end cntrl: Use Data Virtualizatin t cnsistently manage data access and delivery acrss multiple surces and cnsumers. Insulates business and IT change: Insulate cnsuming applicatins frm changes in the surce and vice versa. Business users and applicatins develpers wrk with a mre stable view f the data. IT can make nging changes and relcatin f physical data surces withut affecting infrmatin users. Practical Next Steps Enterprises can begin achieving the primary agility and ttal cst f wnership benefits described earlier with a few simple steps. It is imprtant t get started quickly with a manageable prject that enables learning and a fundatin fr prgress: Set achievable gals: Start with prjects and a fcused team. With success, braden business and IT team invlvement t expand usage acrss departments fr ultimate full enterprise-level deplyment. Determine levels f abstractin: Are the fur recmmended layers right fr yur rganizatin? D yu need greater depth within ne r mre layers? The Cisc Data Virtualizatin Plan and Build Service can help answer these questins and get yu started n the right path. Determine mdeling and mapping apprach: Shuld yu use tp dwn, bttm up, r sme f bth? Tp dwn: Yu have a visin, and yu want t find the data t fulfill it. This is ften referred t as cntract-first design. In this apprach Data Virtualizatin allws yu t start with yur wn WSDL and map Data Virtualizatin services t yur cntract. Bttm up: Yu knw what yur data lks like, but need t determine hw yu make it usable by thers. In this apprach, Data Virtualizatin allws yu t generate r publish resurces such as SQL view and web services directly frm the Data Virtualizatin intrspected surces. Bth: Mix and match apprpriately accrding t dmains and needs. 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 8 f 9
Start nw: D nt veranalyze. Getting started nw with small steps is the best way t learn, prgress, and gain value. Fr Mre Infrmatin T learn mre abut Cisc Data Virtualizatin, speak with yur Cisc representative r visit cisc.cm/g/datavirtualizatin. Printed in USA CXX-XXXXXX-XX 10/11 2014 Cisc and/r its affiliates. All rights reserved. This dcument is Cisc Public. Page 9 f 9