Distributed Databases LBD winter 2004/2005 1
Agenda Introduction Architecture Distributed database design Query processing on distributed database Data Integration 2
Outline Introduction to DDBMS Architecture Distributed Database Design Horizontal Fragmentation Derived Horizontal Fragmentation Vertical Fragmentation Conclusion 3
Motivation integration Database Technology Computer Network distribution Distributed Database System Integration & distribution integration distribution 1994 M. Tamer Özsu & Patrick Valduriez Distributed database systems (DDBS) is a technology fostered by the development of database technology and computer network. Database systems provide data independence for user applications by offering a logical view of the data that is independent of physical implementation. On the other side, computer networks allows physically distributed machines to communicate. This is relevant to database technology because one may consider that for various reasons like privacy, efficiency, reliability, scalability users may want their data to be closer to him/her or be replicated through different nodes and still obtain an integrated global view of it. Thus, DDBSs explores the communication facilities of a computer network to offer distributed users the same level of services obtained in a centralized database system: query processing, transaction management, recovery processing, security guaranties, constraint support. 4
Introduction What s a distributed database system? a collection of data which belong logically to the same system but are spread over the sites of a computer network [Ceri and Pelagatti 1984] Logically related Data physically distributed 5
Logically related Means that applications view data as an integrated database independently of data physical placement Apps. query Logical database Logical view Computer 1 Computer 2 Computer 3 Physical placement R1,R2 R3 R4,R5 Distributed databases are still databases. Thus the data it comprises is logically related according to the database model. In the eyes of a user, there should be no logical distinction between a distributed and centralized database systems. Obviously, in the physical level the DDBMS is adapted to confront with distribution concerns: performance, distributed catalog management; distributed transactions, etc.. 6
Physically distributed The database data is placed in different computers of a network, local or wide area. Computer 1 Computer 2 Computer 3 Physical placement R1,R2 R3 R4,R5 Physical distribution reflects the fact that objects on the database logical level are distributed. This is more than just file distribution. It means that, in the Relational Model, for instance, relational views of data are placed in different nodes of a computer network. The user perceives data as globally integrated and the DDBMS is responsible for recomposing such a view from the distributed relational views. 7
Scenarios 8
Centralized DBMS in a network Site 1 Site 2 Site 5 Communication network Site 4 Site 3 A centralized DBMS available on a network node is a simplified view of a DDBMS usually known as a client-server DBMS. In this scenario, most of the DBMS functionality is done on the server side. The client side is responsible for: the user application interface, DBMS communication modules, caching of DBMS data and some transaction management functionality. 9
Shared memory architecture P 1 P n M D Examples : symmetric multiprocessors (Sequent, Encore) and some mainframes (IBM3090, Bull's DPS8), clusters SUN FIRE (6800) A shared memory architecture is a multi-processor system in which processors share a common memory space where they can exchange data without the need to send message in between processors. This is also not a DDBMS as the control is centralized in the shared memory space. Data could be distributed in the network, managed by a distributed file system, but this does not create a distributed control over the data. This architecture is also know as Symmetric multi-processor machines. Some commercial DBMSs have been implemented on top of such architecture providing for parallel execution of queries (DB2). 10
Shared Disk architecture P 1 P n D M 1 M n Examples : DEC's VAX cluster, IBM's IMS/VS Data Sharing A shared disk architecture is also a multiprocessor system in which each processor possesses a local memory space not shared with other processors. Disks are shared between processors and can be used to exchange information. A DBMS instance may run in each of the processor s nodes with shared access to data on disks. Copies of data can be maintained by each processor node requiring managing of updates in cached copies. A shared-disk system is not, essentially, a DDBMS as data is centralized even if control is decentralized. 11
Shared nothing architecture P 1 M 1 D 1 P n M n D n Exemplos : Teradata's DBC, Tandem, Intel's Paragon, NCR's 3600 and 3700 A shared-nothing is a multiprocessor system in which each node is completely independent of the other. Each processor has exclusive access to its disks and memory space. The architecture is adequate for hosting a DDBMS where each node hosts an instance of the distributed database. Teradata s DBC is a DDBMS designed on top of a shared-nothing architecture and that can support up to 1024 nodes. 12
Distributed DBMS environment Site 1 Site 2 Site 5 Communication Network Site 4 Site 3 This is a typical distributed database scenario. It is assumed that the database is distributed through some of the nodes of the network and that these nodes also hold a copy of the DBMS. Users connecting into a node served by a DBMS are offered an integrated view of the database independently of data location. 13
Transparency 14
Transparency Lower level system characteristics are hidden from users; In a DBMS environment, complex applications can concentrate on functional characteristics data management is done by the DBMS. 15
Levels of Transparency Language Transparency Network Transparency Location Transparency Fragmentation Transparency Data Independency Data Network Transparency Different transparency levels come into play regarding data distributed in a computer network. In the most basic level it comes simply the data itself. Accessing data can be achieved directly by a user program, in which case its completely dependent on the physical structure of the data, or can be abstracted by a data management service that isolates physical aspects from logical view of data. The latter corresponds to the data independency level and allows database administrators to change data structure without affecting applications. In order to offer users integrated view of distributed data, the database design process is extended with the notion of data fragments. Fragments specify a criteria for splitting a global concept into smaller fragments. Usually, in database such criteria is expressed through a language (such as the Relational Algebra). The fragmentation transparency specifies that applications should not be affected by the fragmentation process, rather they should continue to see data as defined in a global view. Fragments of a global concept may be independently associated to a node in a network, considering that a DBMS module exists to manage it. Allocation is the process of defining the location a fragment should be placed in. The location transparency property defines that applications should bother about where a fragment has been placed. An extension of the location transparency includes fragment replication policy. Thus, in addition to place a fragment into a node, allocation process may consider the replication of a fragment into nodes. So we also consider within location transparency the property of hide replication management issues for the user application. As an example, a DBMS may transparently select a copy of a fragment that is closest to the user. Considering data distribution, access depends on message exchange between network nodes. Network transparency guarantees that applications don t have to deal with network communication problems. Finally, in a heterogeneous database scenario, data can be specified according to different data models and languages. Language transparency allows users to access data using a common defined language which is mapped to the different component languages. 16
Reference Architecture ES 1 ES 2 ES 3 GCS LCS 1 LCS 2 LCS n LIS 1 LIS 2 LIS n The above reference architecture is a general model for distributed database systems. On the top level, external schemas specify the application views of data. This corresponds to the external schema level proposed by the ANSI/SPARC reference database architecture. The Global Conceptual Schema (GCS) provides a single integrated view of the data independently of distribution aspects. In the next layer, the Local Conceptual Schema corresponds to the fragmentation and allocation definitions in respect to each computer network node. Finally, the Local internal schema specifies physical characteristics for each allocated fragment. Queries are specified over the external schema or directly over the GCS. The LCS specifies the mappings between the GCS and LCS. Based on the latter, references to global elements of the GCS can be re-written in terms of local fragments. Finally, the system uses internal resources defined in the LIS to access local elements. This architecture considers a distributed database scenario where all component databases share the same model (i.e. homogeneous). Later on, we will discuss architectures where components are heterogeneous. In such scenarios, a new layer named Local Exported Schema can be introduced [Sheth and Larson 1990] 17
Architectural Models Autonomy refers to the distribution of control. It defines how each component of the DDBMS reacts in face of distributed transactions; Distribution- refers to the distribution of data through different sites; Heterogeneity refers to data representation on different components of the DDBMS 18
Architectural Models for DDBMS Distribution (A0,D2,H0) (A2,D2,H1) Autonomy Heterogeneity The three axes of the architectural models highlight orthogonal dimensions based on which we can classifying distributed database systems. They are orthogonal in the sense that a value for one dimension may be combined with any point of another orthogonal dimension giving birth to an architecture model. The Distribution dimension is graded into three levels (o: no distributed), (1: client-server), (3: distributed). Autonomy refers to the level of collaboration that a component of the system is prepared to offer. It refers to the distribution of control, not data. Autonomy (0) refers to tightly integrated systems. Autonomy (1) is a level where systems partially collaborate with the integrated view. This means that part of the DBMS may be accessed directly by non-global transactions. The complement part of the database is accessed through the global view. Autonomy level (3) corresponds to highly independent systems that do not take any action towards the global view. The integrated view, if one exists, is provided by an extra layer on top of each autonomous DBMS. Finally, heterogeneity is in respect to different database models, data models and languages. A DDBMS may be classified as homogeneous (0) or heterogeneous (1) A triple (A(X),D(Y),Z(W)) defines a possible DDBMS architecture. For instance, (A2,D2,H1) specifies a multi-dbms composed of heterogeneous DBMSS that are distributed in the network. 19
Architectures 20
Distributed Databases... View 1 View 2 View n GCS LIS 1 LIS 2... LIS n 21
Distributed Databases... View 1 View 2 View n GDBMS GCS Global Catalog LDB 1 LIS 1 LDB 2 LIS 2... LDB n LIS n 22
Federated Databases... ES 1 ES 2 ES n FDBMS GCS Global Catalog... LCS 1 LCS 2 LCS n LDBMS 1 LDBMS 2 LDBMS n... LIS 1 LIS 2 LIS n 23
Multi-DBMS GES 1 GES 2... GES n LES 11 GQP GCS Global Catalog LES nm LCS 1 LCS 2 LCS n LDBMS 1 LDBMS 2 LDBMS n LIS 1 LIS 2 LIS n 24
Mediators... ES 1 ES 2 ES n GQP GCS... LCS 1 LCS 2 LCS n... DS 1 DS 2 DS n 25
Mediators... ES 1 ES 2 ES n GQP GCS Global Catalog... LCS 1 LCS 2 LCS n Wrapper 1 Wrapper 2 Wrapper n... DS 1 DS 2 DS n 26
Peer-to-Peer ES 1 ES 2 PES 1 PES 2 PES 3 LCS 1 ICS 1 LCS ICS2 2 ICS 3 LCS 3 DS 1 DS n DS 1 DS n DS 1 DS n 27
Peer-to-Peer PQP 1 PES 1 PCS 1 ICS 1 PQP 3 PES 3 PCS 3 ICS 3 LCS 1 LCS n Local PES 2 Wrp 1 Wrp Catalog PQP 2 n LCS m LCS w + PCS 2 ICS 2 DS 1 DS n Mappings Wrp m Wrp w LCS k LCS z Local DS m DS w Wrp Wrp Catalog k z + DS k DS z Mappings Local Catalog + Mappings 28
Distributed Database Design 29
What is it? Is the process of designing a database for its deployment in a distributed environment; Two main approaches: Top-down (from centralized to distributed) Bottom-up (from existent DBMSs to an integrated view) Top-down is an approach taken in building new applications. The traditional database design approach considers the activities: Requirements analysis; conceptual model design; logical model design; physical database design. Distribution adds two new activities: fragmentation design and allocation design. The build directly from the conceptual database design enriched with view access information to produce fragments that correspond to application data access profile. Bottom-up approach is used when databases already exists and new applications aim at having an integrated view of those databases. In this scenario, a global conceptual view is defined and the local conceptual views are mapped to the global view. In the top-down approach, the GCS is a union of the LCSs that derive from it. In the reverse approach, the GCS can be a subset of the union of the LCSs as some of the existing data may not be relevant to the global conceptual view. 30
Top-Down Approach Requirements Analysis Conceptual Design System Requirements User Input View Design Global Conceptual Schema Access Information External Schema Definition Distribution Design Local Conceptual Schemas The process of top-down design begins with a requirement analysis that defines the environment of the system and elicits both the data and processing needs of all potential databases users. The requirements document is input to two process: conceptual design and view design. The former identifies the concepts and relationships that represent the domain covered by the application and the latter defines access criteria. One may see this as complementary static and dynamic information for the database design. The conceptual schema is the basis for the global conceptual schema and access information gives input to the views applications have of data. By refining access view information with frequency a fragmentation design policy is defined (see this in the next slides). 31
Why to fragment? To define a proper unit of distribution; Applications access a subset of a relation; Fragments allow for concurrent transactions to run in parallel accessing different fragments of a relation; Fragments allow queries to access in parallel different fragments of a relation; Fragmentation is not strictly necessary for distribution as one may think of distributing whole files or relations, but this would not be the most efficient approach as applications access only a subset of relations, like information of employees of a certain department or of students of a certain course. In this case, distributing whole relations would be inefficient as irrelevant data would be transferred to the application site. Thus, it is important in a distribution design process to identify an adequate distribution unit. A fragment is a distribution unit that is a subset of a relation (or object in a more general sense) that can be obtained by an expression on the data model language. Common expressions used for fragmentation of relations include: selections, projections and semi-joins. In addition to reduce network transfer cost, fragmentation also maximizes concurrency by allowing concurrent transactions that access disjoint sets of a relation to proceed independently one of the other. In addition, a single query can also benefit from fragmentation by accessing in parallel different fragments. The drawback of fragmentation occurs exactly where an optimal fragmentation point is crossed. As a result of a extreme fragmentation applications may need to access and union different fragments to build their needed view of data. As different applications access the same data and have different access view requirement, defining a best fragmentation unit is not simple and should count on the relevance of the different applications. 32
Fragmentation approach alternatives Horizontal defines a subset of a relation based on a selection predicate over relation attribute values; Vertical specifies fragments according to the subsets of the attributes accessed by applications Hybrid a combination of the two; 33
Example EMP ENO E1 E2 EName J.Doe M. Smith TITLE Elec. Eng. Syst. Anal. ALLOC ENO E1 E2 PNO P1 P1 RESP Mngr Anal. DUR 12 24 E3 A. Lee Mech. Eng. E2 P2 Anal. 6 E4 J. Miller Programmer E3 P3 Cons. 10 E5 B. Casey Syst. Anal. E3 P4 Eng. 48 E6 L. Chu Elect. Eng. E4 P2 Prog. 18 E7 R. Davis Mech. Eng. E5 P2 Mngr 24 E8 J. Jones Syst. Anal. E6 P4 Mngr 48 34
Example Proj PNO PName Budget Location P1 P2 Bioinfo ELearn. 500000 300000 Lausanne Rio P3 Plasma 1000000 Geneva P4 Aircraft 150000 Lausanne Sal Title Elec. Eng. Syst. Anal. Mech. Eng. Salary 40000 34000 27000 Programmer 24000 35
Example of Horizontal Partitioning of Proj Proj 1 PNO PName Budget Location P1 P2 Bioinfo ELearn. 500000 300000 Lausanne Rio Proj 2 PNO PName Budget Location P3 Plasma 1000000 Geneva P4 Aircraft 150000 Lausanne 36
Example of Vertical Partitioning Proj 1 PNO Budget P1 P2 500000 300000 P3 1000000 P4 150000 Proj 2 PNO PName Location P1 P2 Bioinfo ELearn. Lausanne Rio P3 Plasma Geneva P4 Aircraft Lausanne 37
Correctness criteria Completeness each data item of a relation R must appear in one of its fragments; Reconstruction Given a relation R and its fragments F= {R 1, R 2,, R n } is always possible to reconstruct R by applying operations over F. Disjointness fragments of R contain a disjoint subset of it. The completeness property guarantees that the fragmentation process produces a total partition of the global relation. This is important as users should be unaware of the fragmentation criteria and as such be free to insert data that is valid according to a centralized model, independently of any fragmentation decision. The reconstruction criteria is similar to the decomposition rule in database normalization procedure. Once data has been fragmented there should always exists an inverse operation that reconstruct the original view of the data. Finally, disjointness should simplify the recomposition process as no intersection has to be dealt with. Regaring vertical partitioning, though, fragmentation will have to consider the overlap of primary keys as a guarantee of recomposition. 38
Allocation alternatives Single copy each fragment is allocated to a single node Replicated a fragment can have multiple copies each allocated to a node in the network. 39
Horizontal Fragmentation 40
PHF Information requirements Database Information relationship S TITLE,SAL E L 1 P ENO, ENAME, TITLE PNO, PNAME, BUDGET, LOC L 2 L 3 A ENO, PNO, RESP, DUR 41
PHF Information requirements Application information (qualitative) Simple predicates: GigenR[A 1, A 2,, A n ], a simple predicate p j is p j : A i Φ value where Φ {=,<,,>,>=,<=}, value D i and D i is the domain of A i. For R define Pr = {p 1, p 2,,p m } as a set of simple predicates defined over R. Example : PNAME = BionInfo" Budget <= US$ 200000 42
PHF Information requirements Minterm predicates : Given R and Pr={p 1, p 2,,p m } defines M={m 1,m 2,,m r } as M={ m i m i = Λ pj Pr p j * }, 1<= j <=m, 1 <=i <=z where p j * = p j or p j * = (p j ). 43
PHF Information requirements Example m 1 : PNAME= BionInfo " & Budget > 200000 m 2 : NOT(PNAME = BionInfo ") & Budget > 200000 m 3 : PNAME= BionInfo "&NOT(Budget > 200000) m 4 : NOT(PNAME=" BionInfo ") & NOT(Budget > 200000) 44
PHF Information requirements Application information (quantitative) minterm selectivity : sel(m i ) Number of tuples that would be accessed by a query according to a given minterm predicate. For instance the selectivity of m 3 is zero as there is no project that corresponds to the conditions on the minterm. Access frequency: acc(q i ) Frequency with which user applications access data. Acc(qi) gives the frequency of query qi in a certain period. acc(m i ) is the access frequency of a minterm m i corresponding to acc(qi) that contains m i 45
PHF Example Two candidate relations: S e P Fragmentation of S Aplication: Verifies salary information and determines a salary raise. Arquivos de empregados mantidos em dois sites Þ aplicação executa em dois sites Simple predicates p 1 : SAL <= 30000 p 2 : SAL > 30000 Min-term predicates m 1 : (SAL < = 30000) and (SAL > 30000) m 2 : (SAL <= 30000) and NOT(SAL > 30000) m 3 : NOT(SAL <= 30000) and (SAL > 30000) m 4 : NOT(SAL <= 30000) and NOT(SAL > 30000) 46
PHF Example Fragmentation of S (cont.) Implications i 1 : (SAL <= 30000) NOT(SAL > 30000) i 2 : NOT(SAL <= 30000) (SAL > 30000) i 3 : (SAL > 30000) NOT(SAL <= 30000) i 4 : NOT(SAL > 30000) (SAL <= 30000) m 1 is contradictory to i 1, m 4 is contradictory to i 2. S 1 S 2 TITLE SAL TITLE SAL Mec. Eng. 27000 Eletr. Eng. 40000 Programmer 24000 Sist. Anal. 34000 47
PHF Example Fragmentation of relation P Aplications: 1- Find name and budget of projects given their locations Submitted in three sites 2- Accesses project s information based on their budgets One site accesses <= 200000, other accesses > 200000 Simple predicates For application (1) p 1 : LOC = Lausanne" p 2 : LOC = Geneva" p 3 : LOC = Rio" For application (2) p 4 : Budget <= 200000 p 5 : Budget > 200000 Pr = Pr' = {p 1,p 2,p 3,p 4,p 5 } 48
PHF Example Continuing Min-term fragments after elimination of contradictions m 1 : (LOC = Lausanne") and (Budget <=200000) m 2 : (LOC = Lausanne") and (Budget > 200000) m 3 : (LOC = Geneva") and (Budget <= 200000) m 4 : (LOC = " Geneva ") and (Budget > 200000) m 5 : (LOC = Rio") and (Budget <= 200000) m 6 : (LOC = Rio") and (Budget > 200000) 49
PHF Example P 1 P 2 PNO PNAME BUDGET LOC PNO PNAME BUDGET LOC P4 Aircraft 150000 Lausanne P1 Bio Info 500000 Lausanne P4 P 6 PNO PNAME BUDGET LOC P3 Plasma 1000000 Geneva PNO PNAME BUDGET LOC P2 E-learning 300000 Rio 50
Horizontal Fragmentation For each single relation in a global conceptual schema Identify the most important applications that access it; Define simple predicates based on selection criteria defined by queries over the studied relation; Build min-term predicates as a conjunction of simple predicates (include the negation of each simple predicate) Identify contradictory predicates and eliminate corresponding min-terms; Define fragments as selection operations with formula corresponding to the min-term predicates 51
Derived Horizontal Fragmentation Intuition two relations that are commonly jointly accessed define partitions of both sets that should appear together Example: Emp(ENO,ENAME,TITLE, DEPT) Alloc(ENO,DNO,Dur) If employees are managed by department M 1 = Dept=D1 M 2 = Dept=D2 M 3 = Dept=D3 The allocation information is dependent on that of associated employees, so allocation fragments are defined according to the min-terms defined over employees. A 1 = Alloc E 1 A 2 = Alloc E 2 52
Graph of dependency E1 E2 E3 A1 A2 A3 A4 Independent Fragmentation would lead To excessive joins (eventually remote) E1 E2 E3 A1 A2 A3 Defining derived fragmentation Guarantees that a single join would Be needed to access data in E and A 53
Semi-join The semi-join of relation R, defined over the set of attributes A, by relation S, defined over the set of attributes B, is the subset of tuples of R that participates in the join of R with S. It is denoted as R F S, where F is a predicate. It can be obtained as follows: R F S = π A ( R F S) 54
Example DHF Define a fragmentation criteria for the Alloc relation based on the min-terms specified for the Proj relation: P1= σ ((LOC = Lausanne") and (Budget <=200000) ) Proj P 2 =σ ((LOC = Lausanne") and (Budget > 200000)) Proj P 3 =σ ((LOC = Geneva") and (Budget <= 200000)) Proj P 4 =σ ((LOC = Geneva") and (Budget > 200000)) Proj P 5 =σ ((LOC = Rio") and (Budget <= 200000)) Proj P 6 =σ ((LOC = Rio") and (Budget > 200000)) Proj 55
Example DHF Derived Horizontal fragmentation obtained by defining semi-joins over fragments of Proj: A 1 = Alloc (Alloc.PNO=P1.PNO) P 1 A 2 = Alloc (Alloc.PNO=P2.PNO) P 2 A 3 = Alloc (Alloc.PNO=P3.PNO) P 3 A 4 = Alloc (Alloc.PNO=P4.PNO) P 4 A 5 = Alloc (Alloc.PNO=P5.PNO) P 5 A 6 = Alloc (Alloc.PNO=P6.PNO) P 6 56
DHF of Alloc over Proj A1 ENO PNO RESP DUR A2 ENO PNO RESP DUR E3 E6 P4 P4 Eng. Mngr 48 48 E1 E2 P1 P1 Mngr Anal. 12 24 A4 ENO E3 PNO RESP DUR P3 Cons. 10 A6 ENO E2 PNO P2 RESP Anal. DUR 6 E4 P2 Prog. 18 E5 P2 Mngr 24. 57