Trusted, Enterprise QlikViewreporting. data Integration and data Quality (It s all about data)

Transcription

1 Trusted, Enterprise QlikViewreporting with Informatica data Integration and data Quality (It s all about data) Arjan Hijstek senior sales consultant Informatica Nederland bv ahijstek@informatica.com

2 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 2

3 Do you know Informatica? (PowerCenter, DataQuality, other) Do you use Informatica? (Data Warehouse, Migration) 3

4 Informatica The #1 Independent Leader in Data Integration Founded: Revenue: $784 million 6-year Average Growth Rate: 20% per year Employees: 2,554 Partners: 400+ Major SI, ISV, OEM and On-Demand Leaders Customers: 4,630 > 70% of the Global 500 Customers in 82 Countries Direct Presence in 26 Countries # 1 in Customer Loyalty Rankings (6 Years in a Row) $800 $700 $600 $500 $400 $300 $200 $100 $

5 Beyond Data Warehousing Empowering the Data-Centric Enterprise Business Imperatives Improve Decisions Increase Business Agility Improve Business Processes Improve Efficiency & Reduce Costs Mergers Acquisitions & Divestitures Acquire & Retain Customers Outsource Non-core Functions Governance Risk Compliance Increase Partner Network Efficiency IT Initiatives Business & Operational Intelligence Zero Latency Operations Application Portfolio Optimization Application Retirement Application Consolidation Customer, Supplier, Product Hubs BPO SaaS Risk Mitigation & Regulatory Reporting B2B Integration Data Integration Projects Data Warehouse Ultra Messaging Data Services Data Migration & Archiving Data Consolidation Master Data Management Data Synchronization Complex Event Processing B2B Data Exchange 5

6 The Tradition Approach 87% of Enterprises Use Hand-Coding for Data Integration Data Warehousing Ultra Messaging Data Services Data Migration & Archiving Data Consolidation Master Data Management Data Synchronization Complex Event Processing B2B Data Exchange 75% of enterprises reported increased maintenance costs 1 Forrester Research, The State Of Enterprise IT Budgets: 2008, March 27, Forrester Research, Addressing Data Integration Challenges with SOA,

7 The Informatica Approach Comprehensive, Unified, Open and Economical platform Data Warehousing Ultra Messaging Data Services Data Migration & Archiving Data Consolidation Master Data Management Data Synchronization Complex Event Processing B2B Data Exchange 7

8 8

9 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 9

10 Reporting: do you see unexpected results/values? (e.g. gender: M/F/1/0/missing values etc.) 10

11 Using Informatica Analyst Tools to Profile your Data Data Steward Data Analysis & Discovery Increase productivity and efficiency by enabling the business to proactively take responsibility for data quality and reduce their reliance on IT. 11

12 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 12

13 Data Quality Problem: - many systems and data-sources having information about customers/products/suppliers etc. - need to profile, standardize, cleanse, de-dup: automatically! - otherwise wrong/strange results in Data Warehouse or in new systems after migration/conversion - where to do Data Quality : at source? In Data Warehouse? - when is data quality finished? Think about Qlikview-reports/dashboards! 13

14 Reporting: do you have/see Data Quality issues? How is it solved? What about Data Ownership? 14

15 Data Quality Dimensions Completeness How do you measure data quality? Conformity Consistency What data is missing? What data is stored in nonstandard formats? What data gives conflicting information? Accuracy Duplications Integrity What data does not reflect reality or is out of date? What data or attributes are repeated? What data is not referenced or missing? 15

16 Data Quality issues, examples COMPLEETHEID STANDAARDEN CONSISTENTIE DUPLICATEN INTEGRITEIT ACCURATESSE 16

17 Frequent Requirements Data Analysis Parsing Address Matching Monitoring & & Discovery and Validation De-duplication & Standardization Reporting And do this for all domains & data types 17

18 Using Informatica Analyst Tools to Profile your Data Data Steward Data Analysis & Discovery Increase productivity and efficiency by enabling the business to proactively take responsibility for data quality and reduce their reliance on IT. 18

19 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 19

20 Data Integration / ETL - Extraction, Transformation and Load - For Data Warehousing, conversions, migrations, testdata-management, MDM, - For Informatica this means: - integration of Profiling, Data Quality and ETL - independent vendor, so connectivity to almost all databases, incl. Teradata, Netezza, Greenplum etc. - low or very high data volumes, batch and/or realtime - simple or complex environments - no coding! Infrastructure-independent (database, OS) - multi-user, multi-project 20

21 Data Integration / ETL: data mapping Source, target & transformation blocks connectors expressions 21

22 Transformation types Lego blocks Joiner Expression Sorter Lookup Aggregator Normalizer Router Sequence generator Filter Union Update strategy Transaction control Stored procedure Source qualifier Ranker External transformation Association transformation SQL transformation Java transformation Consolidation transformation Web services and more 22

23 Data Integration / ETL: workflow 23

24 Data Integration / ETL: monitoring 24

25 Data Integration / ETL: lineage 25

26 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 26

27 Qlikview connector for Informatica Powercenter 27

28 Qlikview connector for Informatica Powercenter QVX 28

29 Agenda Introduction Informatica Qlikview and reporting : data warehouse? - What data do I get? Relationships? data profile! - (bad) data quality? data quality! - Extraction, Transform and Load of any data ETL! Qlikview and Informatica Data Virtualization - faster access to new data Quick demo 29

30 Data Virtualization Problem: - Enterprise Data Warehouse takes 3-9 months (modeling, extract, reports/dashboards) - what if new data from a new source needs to be added to a report? - another 3+ months? Or just 1 day? - (complex) operational reporting? Data Virtualization 30

31 Solution with Data Services (Federation) DATA CONSUMERS Portals DWH / DM Virtual View Daily update to DWH / DM SA/ODS operational extracts from DWH and/or even directly from Data Sources, accessible via standard SQL (Reporting Tool) Daily extract (PowerCenter) Cloud Messages Mainframe Application Database Flat Files Unstructured Data Database (operational) DATA SOURCES Semi-structured Data 31

32 Questions? Quick demo! 32