Distributed Database Design SL02

Transcription

1 Design Problem Distributed Database Systems Fall 2016 Distributed Database Design SL02 Design problem Design strategies (top-down, bottom-up) Fragmentation (horizontal, vertical) Allocation and replication of fragments, optimality, heuristics Design problem of distributed systems: Making decisions about the placement of data and programs across the sites of a computer network as well as possibly designing the network itself. In DDBMS, the distribution of applications involves Distribution of the DDBMS software Distribution of applications that run on the database Distribution of applications will not be considered in the following; instead the distribution of data is studied. DDBS16, SL02 1/60 M. Böhlen Framework of Distribution Dimension for the analysis of distributed systems Level of sharing: no sharing, data sharing, data + program sharing Behavior of access patterns: static, dynamic Level of knowledge on access pattern behavior: no information, partial information, complete information DDBS16, SL02 2/60 M. Böhlen Design Strategies/1 Top-down approach Designing systems from scratch Homogeneous systems Bottom-up approach The databases already exist at a number of sites The databases should be connected to solve common tasks Distributed database design should be considered within this general framework. DDBS16, SL02 3/60 M. Böhlen DDBS16, SL02 4/60 M. Böhlen

2 Design Strategies/2 Top-down design strategy Design Strategies/3 Distribution design is the central part of the design in DDBMSs (the other tasks are similar to traditional databases). Objective: Design the LCSs by distributing the data over the sites. Two main aspects have to be designed carefully: Fragmentation: Relation may be divided into a number of sub-relations, which are distributed Allocation and replication: Each fragment is stored at site(s) with "optimal" distribution Distribution design issues Why fragment at all? How to fragment? How much to fragment? How to test correctness? How to allocate? DDBS16, SL02 5/60 M. Böhlen Design Strategies/4 Bottom-up design strategy DDBS16, SL02 7/60 M. Böhlen DDBS16, SL02 6/60 M. Böhlen Fragmentation/1 What is a reasonable unit of distribution? Relation or fragment of relation? Relations as unit of distribution: If the relation is not replicated, we get a high volume of remote data accesses. If the relation is replicated, we get unnecessary replications, which cause problems in executing updates and waste disk space Might be an OK solution, if queries need all the data in the relation and data stays only at the sites that use the data Fragments of relation as unit of distribution: Application views are usually subsets of relations Thus, locality of accesses of applications is defined on subsets of relations Permits a number of transactions to execute concurrently, since they will access different portions of a relation Parallel execution of a single query (intra-query concurrency) However, semantic data control (especially integrity enforcement) is more difficult Fragments of relations are (usually) appropriate unit of distribution. DDBS16, SL02 8/60 M. Böhlen

3 Fragmentation/2 Fragmentation aims to improve: Reliability Performance Balanced storage capacity and costs Communication costs Security The following information is used to decide fragmentation: Quantitative information: cardinality of relations, frequency of queries, site where query is run, selectivity of the queries, etc. Qualitative information: predicates in queries, types of access of data, read/write, etc. Fragmentation/3 Types of Fragmentation Horizontal: partitions a relation along its tuples Vertical: partitions a relation along its attributes Mixed/hybrid: a combination of horizontal and vertical fragmentation (a) Horizontal Fragmentation (b) Vertical Fragmentation (c) Mixed Fragmentation DDBS16, SL02 9/60 M. Böhlen Fragmentation/4 Example of database instance DDBS16, SL02 10/60 M. Böhlen Fragmentation/5 Example (contd.): Horizontal fragmentation of PROJ relation PROJ1: projects with budgets less than PROJ2: projects with budgets greater than or equal to DDBS16, SL02 11/60 M. Böhlen DDBS16, SL02 12/60 M. Böhlen

4 Fragmentation/6 Example (contd.): Vertical fragmentation of PROJ relation PROJ1: information about project budgets PROJ2: information about project names and locations Correctness Rules of Fragmentation Completeness Decomposition of relation R into fragments R1, R 2,..., R n is complete iff each data item in R can also be found in some R i. Reconstruction If relation R is decomposed into fragments R1, R 2,..., R n, then there should exist some relational operator that reconstructs R from its fragments, i.e., R = R 1... R n Union to combine horizontal fragments Join to combine vertical fragments Disjointness If relation R is decomposed into fragments R1, R 2,..., R n and data item d i appears in fragment R j, then d i should not appear in any other fragment R k, k j (exception: primary key attribute for vertical fragmentation) For horizontal fragmentation, data item is a tuple For vertical fragmentation, data item is an attribute DDBS16, SL02 13/60 M. Böhlen Idea of Horizontal Fragmentation DDBS16, SL02 14/60 M. Böhlen Information Requirements/1 Intuition behind horizontal fragmentation Every site should hold all information that is used to query the site The information at the site should be fragmented so the queries of the site run faster Horizontal fragmentation is defined as selection operation, σ p (R) Example: σ BUDGET <200K (PROJ) σ BUDGET 200K (PROJ) Database information: Links between relations (a link models a 1:N relationship between relations that are related to each other by an equality join) Cardinality of relations: card(r) DDBS16, SL02 15/60 M. Böhlen DDBS16, SL02 16/60 M. Böhlen

5 Information Requirements/ Information Requirements/ Application information: Simple predicates used in queries Relation R[A 1, A 2,..., A n ] A i θv i a simple predicate (θ {=, <,, >,, }, v i D i, D i is the domain of A i ) For relation R we define Pr = {p1, p 2,..., p m } Example: PNAME = Maintenance, BUDGET < Minterm predicates minterm predicates are conjunctions of simple predicates Relation R, Pr = {p 1, p 2,..., p m } M = {m 1, m 2,..., m r } such that M = {m i m i = pj Pr p j }, 1 j m, 1 i z where p j = p j or p j = p j Application information (cnt d): Minterm selectivities The number of tuples of the relation that would be accessed by a user query, which is specified according to a given minterm predicate m i. Access frequencies The frequency with which a user application qi accesses data. DDBS16, SL02 17/60 M. Böhlen Horizontal Fragmentation/1 DDBS16, SL02 18/60 M. Böhlen Horizontal Fragmentation/2 We write Pr to denote the complete and minimal set of simple predicates generated from Pr: The set of predicates is complete if and only if any two tuples in the same fragment are referenced with the same probability by any application The set of predicates is minimal if and only if each simple predicate is relevant for determining the fragmentation and for each pair of fragments there is at least one query that accesses the fragments differently A horizontal fragment R i of relation R consists of all the tuples of R that satisfy a predicate F j : R j = σ Fj (R), 1 j w where F j is a selection formula, which is (preferably) a minterm predicate. Computing the horizontal fragmentation: 1. Determine Pr (80/20 rule) 2. Compute Pr 3. Determine M 4. Minimize M (eliminating impossible minterms) DDBS16, SL02 19/60 M. Böhlen DDBS16, SL02 20/60 M. Böhlen

6 Horizontal Fragmentation/3 Horizontal Fragmentation/4 Example: Fragmentation of the PROJ relation Consider the following query: Find the name and budget of projects given their location. The query is issued at all three locations Fragmentation based on LOC, using the set of predicates {LOC = Montreal, LOC = NewYork, LOC = Paris } PROJ 1 = σ LOC= Montreal (PROJ) PNO PNAME BUDGET LOC P1 Instrument Montreal PROJ 2 = σ LOC= NewYork (PROJ) PNO PNAME BUDGET LOC P2 DB Develop New York P3 CAD/CAM New York If access is only according to the location, the above set of predicates is complete i.e., in each fragment PROJ i each tuple has the same probability of being accessed If there is a second query/application that accesses only those project tuples where the budget is less than $200K, the set of predicates is not complete. P2 in PROJ 2 has higher probability to be accessed PROJ 3 = σ LOC= Paris (PROJ) PNO PNAME BUDGET LOC P4 Maintenance Paris DDBS16, SL02 21/60 M. Böhlen Horizontal Fragmentation/5 DDBS16, SL02 22/60 M. Böhlen Horizontal Fragmentation/ Example (contd.): Add BUDGET < 200K and BUDGET 200K to the set of predicates to make it complete. {LOC = Montreal, LOC = NewYork, LOC = Paris, BUDGET 200K, BUDGET < 200K} is a complete set Minterms to fragment the relation are given as follows: (LOC = Montreal ) (BUDGET 200K) (LOC = Montreal ) (BUDGET > 200K) (LOC = NewYork ) (BUDGET 200K) (LOC = NewYork ) (BUDGET > 200K) (LOC = Paris ) (BUDGET 200K) (LOC = Paris ) (BUDGET > 200K) Example (contd.): Now, PROJ i, i = 1, 2, 3 will be split in two fragments PROJ 1 = σ LOC= Montreal (PROJ) PNO PNAME BUDGET LOC P1 Instrument Montreal PROJ 3 = σ LOC= NewYork (PROJ) PNO PNAME BUDGET LOC P2 DB Develop New York Note that the following fragments are empty: σ LOC= Paris BDGT <200K (PROJ) σ LOC= Montreal BDGT 200K (PROJ) PROJ 4 = σ LOC= NewYork (PROJ) PNO PNAME BUDGET LOC P3 CAD/CAM New York PROJ 6 = σ LOC= Paris (PROJ) PNO PNAME BUDGET LOC P4 Maintenance Paris DDBS16, SL02 23/60 M. Böhlen DDBS16, SL02 24/60 M. Böhlen

7 Derived Horizontal Fragmentation/ Derived Horizontal Fragmentation/2 Defined on a member (target) relation of a link according to a selection operation specified on its owner (source). Each link is an equijoin. Fragments of member relation can be defined with a semijoin on fragments of owner relation. Given a link L where owner(l) = S and member(l) = R, the derived horizontal fragments of R are defined as Ri = R >< Si, 1 i w where w is the maximum number of fragments that will be defined on R and Si = σ Fi (S) where Fi is the formula according to which the primary horizontal fragment Si is defined. DDBS16, SL02 25/60 M. Böhlen Derived Horizontal Fragmentation/3 Given link L1 where owner(l1) = PAY and member(l1) = EMP and PAY 1 = σ SAL (PAY ) PAY 2 = σsal>30000 (PAY ) we get EMP1 = EMP >< PAY 1 EMP2 = EMP >< PAY DDBS16, SL02 26/60 M. Böhlen Vertical Fragmentation/1 Objective of vertical fragmentation is to partition a relation into a set of smaller relations so that many of the applications will run on only one fragment. Vertical fragmentation of a relation R produces fragments R 1, R 2,..., each of which contains a subset of R s attributes. Vertical fragmentation is defined using the projection operation of the relational algebra: π A1,A 2,...,A n (R) Example: PROJ 1 = π PNO,BUDGET (PROJ) PROJ 2 = π PNO,PNAME,LOC (PROJ) Vertical fragmentation has also been studied for (centralized) DBMS Smaller relations, and hence less page accesses e.g., MONET system DDBS16, SL02 27/60 M. Böhlen DDBS16, SL02 28/60 M. Böhlen

8 Vertical Fragmentation/2 Vertical Fragmentation/3 Vertical fragmentation is more complicated than horizontal fragmentation In horizontal partitioning: for n simple predicates, the number of possible minterms is 2 n ; some of them can be ruled out by existing implications/constraints. In vertical partitioning: for m non-primary key attributes, the number of possible fragments is equal to B(m) (= the mth Bell number), i.e., the number of partitions of a set with m members. For large numbers, B(m) m m (e.g., B(15) = 10 9 ) Two types of heuristics for vertical fragmentation exist: Grouping: assign each attribute to one fragment, and at each step, join some of the fragments until some criteria is satisfied. Bottom-up approach Splitting: starts with a relation and decides on beneficial partitionings based on the access behaviour of applications to the attributes. Top-down approach Results in non-overlapping fragments Optimal solution is probably closer to the full relation than to a set of small relations with only one attribute DDBS16, SL02 29/60 M. Böhlen Vertical Fragmentation/4 DDBS16, SL02 30/60 M. Böhlen Vertical Fragmentation/5 Application information: The major information required as input for vertical fragmentation is related to applications (queries) Since vertical fragmentation places in one fragment those attributes usually accessed together, there is a need for some measure that would define more precisely the notion of togetherness, i.e., how closely related the attributes are. This information is obtained from queries and collected in the Attribute Usage Matrix and Attribute Affinity Matrix. Given are the user queries/applications Q = (q 1,..., q q ) that will run on relation R(A 1,..., A n ) Attribute Usage Matrix: Denotes which query uses which attribute: { 1 iff qi uses A use(q i, A j ) = j 0 otherwise The use(qi, ) vectors for each application are easy to define if the designer knows the applications that willl run on the DB (consider also the rule) DDBS16, SL02 31/60 M. Böhlen DDBS16, SL02 32/60 M. Böhlen

9 Vertical Fragmentation/6 Example: Consider relation PROJ(PNO, PNAME, BUDGET, LOC) and queries: q 1 = SELECT BUDGET FROM PROJ WHERE PNO=Value q 2 = SELECT PNAME,BUDGET FROM PROJ q 3 = SELECT PNAME FROM PROJ WHERE LOC=Value q 4 = SELECT SUM(BUDGET) FROM PROJ WHERE LOC =Value Abbreviations: A 1 = PNO, A 2 = PNAME, A 3 = BUDGET, A 4 = LOC Attribute Usage Matrix A 1 A 2 A 3 A 4 q q q q Vertical Fragmentation/7 Attribute Affinity Matrix: Denotes the frequency of two attributes A i and A j with respect to a set of queries Q = (q 1,..., q n ): aff (A i, A j ) = ( ref l (q k ) acc l (q k )) sites l k: use(q k,a i )=1, use(q k,a j )=1 where ref l (q k ) is the cost (= number of accesses to (A i, A j )) for each execution of query q K at site l acc l (q k ) is the frequency of query q k at site l DDBS16, SL02 33/60 M. Böhlen Vertical Fragmentation/ DDBS16, SL02 34/60 M. Böhlen Vertical Fragmentation/9 Example (contd.): Let the cost of each query be ref l (q k ) = 1, and the frequency acc l (q k ) of the queries be as follows: Site1 Site2 Site3 acc 1 (q 1 ) = 15 acc 2 (q 1 ) = 20 acc 3 (q 1 ) = 10 acc 1 (q 2 ) = 5 acc 2 (q 2 ) = 0 acc 3 (q 2 ) = 0 acc 1 (q 3 ) = 25 acc 2 (q 3 ) = 25 acc 3 (q 3 ) = 25 acc 1 (q 4 ) = 3 acc 2 (q 4 ) = 0 acc 3 (q 4 ) = 0 Attribute affinity matrix aff (A i, A j ) = A 1 A 2 A 3 A 4 A A A A e.g., aff (A1, A 3 ) = 1 k=1 3 l=1 acc l(q k ) = acc 1 (q 1 ) + acc 2 (q 1 ) + acc 3 (q 1 ) = 45 (q 1 is the only query to access both A 1 and A 3 ) DDBS16, SL02 35/60 M. Böhlen Idea: Take the attribute affinity matrix (AA) and reorganize the attribute orders to form clusters where the attributes in each cluster demonstrate high affinity to one another. Bond energy algorithm (BEA) has been suggested for that purpose for several reasons: It is designed specifically to determine groups of similar items as opposed to a linear ordering of the items. The final groupings are insensitive to the order in which items are presented. The computation time is reasonable (O(n 2 ), where n is the number of attributes) DDBS16, SL02 36/60 M. Böhlen

10 Vertical Fragmentation/10 BEA: Input: AA matrix Output: Clustered AA matrix (CA) Permutation is done in such a way to maximize the following global affinity mesaure (affinity of A i and A j with their neighbors): AM = n i=1 j=1 n aff(a i, A j )[aff(a i, A j 1 ) + aff(a i, A j+1 ) + aff(a i 1, A j ) + aff(a i+1, A j )] Vertical Fragmentation/11 Example (contd.): Cluster Affinity Matrix CA after running BEA A 1 A 3 A 2 A 4 A A A A Elements with similar values are grouped together, and two clusters can be identified An additional partitioning algorithm is needed to identify the clusters in CA Usually more clusters and more than one candidate partitioning, thus additional steps are needed to select the best clustering. The resulting fragmentation after partitioning (PNO is added in PROJ 2 explicilty as key): PROJ 1 = {PNO, BUDGET } PROJ 2 = {PNO, PNAME, LOC} DDBS16, SL02 37/60 M. Böhlen BEA Algorithm/1 Input: The AA matrix Output: Clustered affinity matrix CA which is a permutation of AA Initialization: Place and fix one of the columns of AA in CA (we choose column 1 in our example) Iteration: Place the remaining n-i columns in the remaining i+1 positions in the CA matrix. For each column choose the placement that makes the largest contribution to the global affinity measure. Row order: Order the rows according to the column ordering. DDBS16, SL02 38/60 M. Böhlen BEA Algorithm/2 In order to determine the best placement we define the contribution of a placement. Contribution of a placement: cont(a i, A k, A j ) = 2 bond(a i, A k )+ 2 bond(a k, A j ) 2 bond(a i, A j ) Bond between a pair of attributes is defined as: bond(a x, A y ) = n aff (A z, A x ) aff (A z, A y ) z=1 DDBS16, SL02 39/60 M. Böhlen DDBS16, SL02 40/60 M. Böhlen

11 BEA Algorithm/3 Consider the following AA matrix and the corresponding CA matrix where A 1 and A 2 have been placed. AA = A 1 A 2 A 3 A 4 A A A A CA = A 1 A 2 A A A A bond(a 3, A 1 ) = = 4410 bond(a 3, A 2 ) = = 890 The next step is to place A 3. BEA Algorithm/4 Ordering (0-3-1) : cont(a0, A3, A1) = 2 bond(a0, A3) + 2 bond(a3, A1) 2 bond(a0, A1) = = 8820 Ordering (1-3-2) : cont(a1, A3, A2) = 2 bond(a1, A3) + 2 bond(a3, A2) 2 bond(a1, A2) = = Ordering (2-3-4) : cont(a2, A3, A4) = 2 bond(a2, A3) + 2 bond(a3, A4) 2 bond(a2, A4) = 1780 DDBS16, SL02 41/60 M. Böhlen BEA Algorithm/5 After adding A 3 the CA matrix has the form CA = A 1 A 3 A After adding A 4 and ordering rows the CA matrix has the form CA = A 1 A 3 A 2 A 4 A A A A DDBS16, SL02 43/60 M. Böhlen 02.9 DDBS16, SL02 42/60 M. Böhlen BEA Algorithm/6 The final step is to divide a set of clustered attributes {A 1,..., A n } into two sets {A 1,..., A i } and {A i+1,..., A n } such that the costs of queries that access both sets are minimized. The appropriate split point on the diagonal must be determined: A 1 A 3 A 2 A 4 A A A A This is an optimization problem: the optimal point on the diagonal must be determined. Note: Generalizations are needed to deal with partitions located in the middle of the matrix multiple split points DDBS16, SL02 44/60 M. Böhlen

12 BEA Algorithm/7 Correctness of Vertical Fragmentation AQ(q i ) = {A j use(q i, Aj) = 1} TQ = {q i AQ(q i ) TA} BQ = {q i AQ(q i ) BA} OQ = Q {TQ BQ} CQ = q i Q S j ref j (q i ) acc j (q i ) CTQ = q i TQ S j ref j (q i ) acc j (q i ) CBQ = q i BQ S j ref j (q i ) acc j (q i ) COQ = q i OQ S j ref j (q i ) acc j (q i ) CTQ CBQ COQ 2 attributes accessed by q i queries that access TA only queries that access BA only queries that access TA and BA cost of all queries cost of TQ queries cost of BQ queries cost of other queries maximize Relation R is decomposed into fragments R 1, R 2,..., R n e.g., PROJ = {PNO, BUDGET, PNAME, LOC} into PROJ 1 = {PNO, BUDGET } and PROJ 2 = {PNO, PNAME, LOC} Completeness Guaranteed by the partitioning algortihm, which assigns each attribute in A to one partition Reconstruction Join to reconstruct vertical fragments R = R1 R n = PROJ 1 PROJ 2 Disjointness Attributes have to be disjoint in VF. Two cases are distinguished: If tuple IDs are used, the fragments are really disjoint Otherwise, key attributes are replicated automatically by the system e.g., PNO in the above example DDBS16, SL02 45/60 M. Böhlen Mixed Fragmentation DDBS16, SL02 46/60 M. Böhlen Replication and Allocation In most cases simple horizontal or vertical fragmentation of a DB schema will not be sufficient to satisfy the requirements of the applications. Mixed fragmentation (hybrid fragmentation): Consists of a horizontal fragment followed by a vertical fragmentation, or a vertical fragmentation followed by a horizontal fragmentation Fragmentation is defined using the selection and projection operations of relational algebra: σ p (π A1,...,A n (R)) π A1,...,A n (σ p (R)) Replication: Which fragements shall be stored as multiple copies? Complete Replication Complete copy of the database is maintained in each site Selective Replication Selected fragments are replicated in some sites Allocation: On which sites to store the various fragments? Centralized Consists of a single DB and DBMS stored at one site with users distributed across the network Partitioned Database is partitioned into disjoint fragments, each fragment assigned to one site DDBS16, SL02 47/60 M. Böhlen DDBS16, SL02 48/60 M. Böhlen

13 Replication/1 Replication/2 Comparison of replication alternatives Replicated DB fully replicated: each fragment at each site partially replicated: each fragment at some of the sites Non-replicated DB (= partitioned DB) partitioned: each fragment resides at only one site Rule of thumb: read only queries If 1, then replication is advantageous, otherwise update queries replication may cause problems DDBS16, SL02 49/60 M. Böhlen Fragment Allocation/1 DDBS16, SL02 50/60 M. Böhlen Fragment Allocation/2 Fragment allocation problem Given are: fragments F = {F 1, F 2,..., F n } network sites S = {S 1, S 2,..., S m } and applications Q = {q 1, q 2,..., q l } Find: the optimal distribution of F to S Optimality Minimal cost Communication + storage + processing (read and update) Cost in terms of time (usually) Performance Response time and/or throughput Constraints Per site constraints (storage and processing) Required information Database Information selectivity of fragments size of a fragment Application Information RR ij : number of read accesses of a query q i to a fragment F j UR ij : number of update accesses of query q i to a fragment F j u ij : a matrix indicating which queries updates which fragments, r ij : a similar matrix for retrievals originating site of each query Site Information USC k : unit cost of storing data at a site S k LPC k : cost of processing one unit of data at a site S k Network Information communication cost/frame between two sites frame size DDBS16, SL02 51/60 M. Böhlen DDBS16, SL02 52/60 M. Böhlen

14 Fragment Allocation/3 We discuss an allocation model which attempts to minimize the total cost of processing and storage meet certain response time restrictions General Form: min(total Cost) Fragment Allocation/4 The total cost function has two components: storage and query processing. TOC = S k S F j F STC jk + QPC i q i Q subject to response time constraint storage constraint processing constraint Storage cost of fragment Fj at site S k : STC jk = USC k size(f j ) x ij where USC k is the unit storage cost at site k Decision variable x ij { 1 if fragment Fi is stored at site S x ij = j 0 otherwise Query processing cost for a query q i is composed of two components: composed of processing cost (PC) and transmission cost (TC) QPC i = PC i + TC i DDBS16, SL02 53/60 M. Böhlen Fragment Allocation/5 Processing cost is a sum of three components: access cost (AC), integrity contraint cost (IE), concurency control cost (CC) DDBS16, SL02 54/60 M. Böhlen Fragment Allocation/6 The transmission cost is composed of two components: Cost of processing updates (TCU) and cost of processing retrievals (TCR) Access cost: AC i = s k S PC i = AC i + IE i + CC i (UR ij + RR ij ) x ij LPC k F j F where LPC k is the unit process cost at site k Integrity and concurrency costs: Can be similarly computed, though depends on the specific constraints Note: AC i assumes that processing a query involves decomposing it into a set of subqueries, each of which works on a fragment,..., This is a very simplistic model Does not take into consideration different query costs depending on the operator or different algorithms that are applied TC i = TCU i + TCR i Cost of updates: Inform all the sites that have replicas + a short confirmation message back TCU i = u ij (update message cost + acknowledgment cost) S k S F j F Retrieval cost: Send retrieval request to all sites that have a copy of fragments that are needed + sending back the results from these sites to the originating site. TCR i = F j F min S k S (x jk (cost retrieval request + cost sending back result)) DDBS16, SL02 55/60 M. Böhlen DDBS16, SL02 56/60 M. Böhlen

15 Fragment Allocation/7 Modeling the constraints Response time constraint for a query q i execution time of q i max. allowable response time for q i Storage constraints for a site Sk storage requirement of F j at S k storage capacity of S k F j F Processing constraints for a site Sk processing load of q i at site S k processing capacity ofs k q i Q Fragment Allocation/8 Solution Methods The complexity of this allocation model/problem is NP-complete Correspondence between the allocation problem and similar problems in other areas Plant location problem in operations research Knapsack problem Network flow problem Hence, solutions from these areas can be re-used Use different heuristics to reduce the search space Assume that all candidate partitionings have been determined together with their associated costs and benefits in terms of query processing. The problem is then reduced to find the optimal partitioning and placement for each relation Ignore replication at the first step and find an optimal non-replicated solution Replication is then handeled in a second step on top of the previous non-replicated solution. DDBS16, SL02 57/60 M. Böhlen Conclusion/1 DDBS16, SL02 58/60 M. Böhlen Conclusion/2 Distributed design decides on the placement of (parts of the) data and programs across the sites of a computer network There are two key technical questions: fragmentation and allocation/replication of data Horizontal fragmentation is defined via the selection operation σ p (R) Rewrites the queries of each site in the conjunctive normal form and finds a minimal and complete set of conjunctions to determine fragmentation Vertical fragmentation via the projection operation π A (R) Computes the attribute affinity matrix and groups similar attributes together Mixed fragmentation is a combination of both approaches Allocation/Replication of data Type of replication: no replication, partial replication, full replication Optimal allocation/replication modelled as a cost function under a set of constraints The complexity of the problem is NP-complete Use of different heuristics to reduce the complexity DDBS16, SL02 59/60 M. Böhlen DDBS16, SL02 60/60 M. Böhlen