Data Integration and Data Provenance. Profa. Dra. Cristina Dutra de Aguiar Ciferri

Size: px

Start display at page:

Download "Data Integration and Data Provenance. Profa. Dra. Cristina Dutra de Aguiar Ciferri cdac@icmc.usp.br"

Tyrone Cross
8 years ago
Views:

1 Data Integration and Data Provenance Profa. Dra. Cristina Dutra de Aguiar Ciferri

2 Outline n Data Integration q Schema Integration q Instance Integration q Our Work n Data Provenance q Basic Concepts q The PrInt Model 2

3 Outline n Index Structures q Biological databases q Similarity search of complex data q Spatial data warehouses q Similarity search over data warehouses of images n Mining of Medical Data n 3

4 Data Integration n Schema Level Integration q specification of mappings that describe the semantic relationships among schemas from heterogeneous data sources n Instance Level Integration q identification of which entities from heterogeneous sources refer to the same entity in the real-world q resolution of value conflicts 4

n Instance Level Integration q identification of which entities from

5 Schema Level Integration n Semantic Relativism q conflicts between two or more representations is related to the fact that different users model the same piece of the real-world in different ways according to their perceptions q Types of Conflict Identification n n n name, including homonymous and synonymous semantic structural 5

of the real-world in different ways according to their perceptions q Types of

6 Types of Schema n Global Schema (or Mediated Schema) q integration of several heterogeneous local schemas into a homogeneous global schema q development of mappings that describe the semantic relationships between the mediated schema and the schemas of the sources n Federated Schema q there is not a homogeneous global schema q there are several heterogeneous local schemas, each one related to a given source 6

relationships between the mediated schema and the schemas of the sources n Federated Schema q there is

7 Example of Structural Mappings X C C C C X C cor cor atrib Catrib atrib C X cor C X C Y X Y cor C W C Z cor C Z W Z C cor X Y a X Ca Y X a C atrib cor = corresponde a = associação atrib = atributo C,W,X,Y,Z = classe c o r Y C X aatrib atrib Y SPACCAPIETRA, S., PARENT, C. View Integration: A Step Forward in Solving Structural Conflicts. IEEE Transactions on Knowledge and Data Engineering, v.6, n.2, p ,

classe c o r Y C X aatrib atrib Y SPACCAPIETRA, S., PARENT, C.

8 Instance Level Integration n Reference Reconciliation (or Entity Resolution) q automatically detect references to the same entity of the real-world and group them in a cluster of similar entities n Value Conflict Resolution q solve the differences among values of attributes of the entities that refer to the same entity of the real-world 8

in a cluster of similar entities n Value Conflict Resolution q solve the differences

9 Reference Reconciliation Examples of entities from the class article (a) a 1 = ({ Distributed query processing in a relational database system }, { }, {p 1 ; p 2 ; p 3 }; {c 1 }) a 2 = ({ Distributed query processing in a relational database system }, { },{p 4 ; p 5 ; p 6 }; {c 2 }) Examples of entities from the class person (p) p 1 = ({ Robert S. Epstein }, null, {p 2, p 3 }, null) p 2 = ({ Michael Stonebraker }, null, {p 1, p 3 }, null) p 3 = ({ Eugene Wong }, null, {p 1, p 2 }, null) p 4 = ({ Epstein, R.S. }, null, {p 5, p 6 }, null) p 5 = ({ Stonebraker, M. }, null, {p 4, p 6 }, null) p 6 = ({ Wong, E. }, null, {p 4, p 5 }, null) p 7 = ({ Eugene Wong }, { eugene@berkeley.edu"}, null, {p 8 }) p 8 = (null, { stonebraker@csail.mit.edu"}, null, {p 7 }) p 9 = ({ mike }, { stonebraker@csail.mit.edu"}, null, null) Examples of entities from the class conference (c) c 1 = ({ ACM Conference on Management of Data }, { 1978 }, { Austin, Texas }) c 2 = ({ ACM SIGMOD }, { 1978 }, null) article: title, pages, *authors, *conference person: name, , *authors, * contact conference: name, year, local 9

Epstein }, null, {p 2, p 3 }, null) p 2 = ({ Michael Stonebraker }, null, {p 1, p 3 }, null) p 3 = ({ Eugene Wong }, null, {p 1, p 2 }, null) p 4 = ({ Epstein, R.S. }, null, {p 5, p 6 }, null) p 5 = ({ Stonebraker, M.

10 Reference Reconciliation Grouping from entities of the class article (a) grouping 1 = {a 1, a 2 } Grouping from entities of the class person (p) grouping 2 = {p 1, p 4 } grouping 3 = {p 2, p 5, p 8, p 9 } grouping 4 = {p 3, p 6, p 7 } Grouping from entities of the class conference (c) grouping 5 = {c 1, c 2 } DONG, X.; HALEVY, A.; MADHAVAN, J. Reference Reconciliation in Complex Information Spaces. In Proceedings of the ACM International Conference on Management of Data (SIGMOD), p.85-96,

entities of the class conference (c) grouping 5 = {c 1, c 2 } DONG, X.; HALEVY, A.; MADHAVAN, J.

11 Our Work n The Academic Data Reconciler Tool q semi-automates the identification of n n correspondent objects inconsistencies q helps user to n n n eliminate inconsistencies complete data exchange data n The Academic Data Reconciler Tool for Reference Reconciliation 11

user to n n n eliminate inconsistencies complete data exchange data

12 Academic Data Reconciler Tool (ADR) n Functional aspects q visualization of objects from two documents n side-by-side visualization q synchronization of objects q edition of incomplete and erroneous data q data exchange between objects from different documents 12

visualization q synchronization of objects q edition of

13 Interface 13

14 Visualization 14

15 Synchronization 15

16 Edition 16

17 Data Exchange 17

18 ADR for Reference Reconciliation! integrated entity similar entities that belong to the same grouping 18

CHAPTER-6 DATA WAREHOUSE

CHAPTER-6 DATA WAREHOUSE 1 CHAPTER-6 DATA WAREHOUSE 6.1 INTRODUCTION Data warehousing is gaining in popularity as organizations realize the benefits of being able to perform sophisticated analyses of their