# A Model of Computation for MapReduce

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 number of new key; value pairs. It is crucial that the map operation is stateless that is, it operates on one pair at a time. This allows for easy parallelization as different inputs for the map can be processed by different machines. During the shuffle stage, the underlying system that implements MapReduce sends all of the values that are associated with an individual key to the same machine. This occurs automatically, and is seamless to the programmer. In the reduce stage, the reducer ρ takes all of the values associated with a single key k, and outputs a multiset of key; value pairs with the same key, k. This highlights one of the sequential aspects of MapReduce computation: all of the maps need to finish before the reduce stage can begin. Sine the reducer has access to all the values with the same key, it can perform sequential computations on these values. In the reduce step, the parallelism is exploited by observing that reducers operating on different keys can be executed simultaneously. Overall, a program in the MapReduce paradigm can consist of many rounds of different map and reduce functions, performed one after another. 1.2 MapReduce Example To better understand the power of the model, consider the following simple example of computing the k-th frequency moment of a large data (multi)-set. Let x be the input string of length n, and denote by x i the i th symbol in x. To represent the input as a sequence of key; value pairs, we look at x as a sequence of n pairs, i; x i. Let \$ be a special symbol. The program is as follows: 1. Begin by mapping every tuple to a pair with the symbol as the key, and the position as the value. Thus the first mapper, µ 1 is defined as: µ 1 ( i; x i ) = x i ; i. 2. After the aggregation by the key, the input to each reducer will be a unique string symbol, and the list of positions in which this symbol appears. We proceed to collapse that list into a single number, defining the first reducer ρ 1 as ρ 1 ( x i ; {v 1,..., v m } ) = x i ; m k. 3. At this point we just want to sum the number of remaining pairs. First, map each pair to have the same key: µ 2 ( x i ; v ) = \$; v. 4. Since all of the pairs now have the same key, they will all be mapped to the same reducer. We can simply sum them: ρ 2 ( \$; {v 1,..., v l } ) = \$; i v i. Another major attraction of MapReduce, besides its ease of parallelization, is its ease of use. As the example shows, the framework shields the programmer from the low-level details of parallel programming such as fault tolerance, data distribution, scheduling, etc. Also, all of the data shuffling and aggregation are handled by the underlying system itself. The programmer only needs to specify the map and reduce functions; the system-level issues are handled by the underlying implementation. The drawback of this model is that in order to achieve this parallelizability, programmers are restricted to using only map and reduce functions in their programs [4]. Thus, this model trades off programmer flexibility for ease of parallelization. This is a complex tradeoff, and it is not a priori clear which problems can be efficiently solved in the MapReduce paradigm. The main contribution of this work is a model for what is efficiently computable in the MapReduce paradigm. 2 The MapReduce Programming Paradigm In this section we give a more formal definition of the MapReduce programming paradigm. We begin by defining mappers and reducers. We then describe how the system executes these two functions along with the shuffle step. As mentioned above, the fundamental unit of data in map reduce computations is the key; value pair, where keys and values are always just binary strings. Definition 2.1. A mapper is a (possibly randomized) function that takes as input one ordered key; value pair of binary strings. As output the mapper produces a finite multiset of new key; value pairs. It is important that the mapper operates on one key; value pair at a time. Definition 2.2. A reducer is a (possibly randomized) function that takes as input a binary string k which is the key, and a sequence of values v 1, v 2,... which are also binary strings. As output, the reducer produces a multiset of pairs of binary strings k; v k,1, k; v k,2, k; v k,3,... The key in the output tuples is identical to the key in the input tuple. One simple consequence of these two definitions is that that mappers can manipulate keys arbitrarily, but reducers cannot change the keys at all. Next we describe how the system executes MapReduce computations. A map reduce program consists of a sequence µ 1, ρ 1, µ 2, ρ 2,..., µ R, ρ R of mappers and reducers. The input is a multiset of key; value pairs denoted by U 0. To execute the program on input U 0 : For r = 1, 2,..., R, do: 1. Execute Map: Feed each pair k; v in U r 1 to mapper µ r, and run it. The mapper will generate 939 Copyright by SIAM.

3 a sequence of tuples, k 1 ; v 1, k 2 ; v 2,.... Let U r be the multiset of key; value pairs output by µ r, that is, U r = k;v U r 1 µ r ( k; v ). 2. Shuffle: For each k, let V k,r be the multiset of values v i such that k; v i U r. The underlying MapReduce implementation constructs the multisets V k,r from U r. 3. Execute Reduce: For each k, feed k and some arbitrary permutation of V k,r to a separate instance of reducer ρ r, and run it. The reducer will generate a sequence of tuples k; v 1, k; v 2,.... Let U r be the multiset of key; value pairs output by ρ r, that is, U r = k ρ r( k; V k,r ). The computation halts after the last reducer, ρ R, halts. As stated before, the main benefit of this programming paradigm is the ease of parallelization. Since each mapper µ r only operates on one tuple at a time, the system can have many instances of µ r operating on different tuples in U r 1 in parallel. After the map step, the system partitions the set of tuples output by various instances of µ r based on their key. That is, part i of the partition has all key; value pairs that have key k i. Since reducer ρ r only operates on one part of this partition, the system can have many instances of ρ r running on different parts in parallel. 3 The MapReduce Class (MRC) In this section we formally define the MapReduce Class (MRC). There are three guiding principles that we wish to enforce in our definitions: Memory Since MapReduce allows for computation to be executed on parts of the input in parallel, the full power of the programming paradigm is realized when the input is too big to fit into memory on a single machine. Thus we require that the input to any mapper or reducer be substantially sublinear in the size of the data. Otherwise, every problem in P could be solved in the MapReduce formulation by first mapping the whole input to a single reducer, and then having the reducer solve the problem itself. Machines In order for the model to have practical relevance, we also limit the total number of available machines. For example, an algorithm requiring n 3 machines, where n is the size of the Web, will not be practical in the near future. We limit the total number of machines available to be substantially sublinear in the data size. Time Finally, there is a question of the total running time available. In a major difference from previous work [6], we do not restrict the power of the individual reducer, except that we require that both the map and the reduce functions run in time polynomial in the original input length in order to ensure efficiency. Furthermore, we will only consider programs that require a small number of Map Reduce rounds, because shuffling is a time consuming operation. We are now ready to formally define the class MRC. The input is a finite sequence of pairs k j ; v j, for j = 1, 2, 3,..., where k j and v j are binary strings. The length of the input is n = j ( k j + v j ). Definition 3.1. Fix an ɛ > 0. An algorithm in MRC i consists of a sequence µ 1, ρ 1, µ 2, ρ 2,..., µ R, ρ R of operations which outputs the correct answer with probability at least 3 /4 where: Each µ r is a randomized mapper implemented by a RAM with O(log n)-length words, that uses O(n 1 ɛ ) space and time polynomial in n. Each ρ r is a randomized reducer implemented by a RAM with O(log n)-length words, that uses O(n 1 ɛ ) space and time polynomial in n. The total space k;v U r( k + v ) used by key; value pairs output by µ r is O(n 2 2ɛ ). The number of rounds R = O(log i n). We note that while technically RAMs produce a sequence of key; value pairs as output, we interpret the sequence as a multiset of the corresponding pairs. We allow the use of randomization in MRC, and demand the final correct answer with probability at least 3/4, but there are obvious Las Vegas and deterministic variants, the latter of which we call DMRC. We emphasize that mappers process pairs one at a time, and remember nothing about the previous pairs. It also is important to remember that each reducer gets a sequence of values, in some arbitrary (not random) order. Nonetheless the output of the reducer must be correct, or must be correct with a certain probability if the algorithm is randomized, regardless of the order. Note that both mappers and reducers run in time polynomial in n, not polynomial in the length of the input they receive. At this point a careful reader may complain that the example algorithm given in Section 1 does not fit into this model. Indeed, if the string consists of n copies of the same symbol, then the input to a single reducer will be at least n, in violation of the space constraints in the model. We give an MRC algorithm for the frequency moments problem in Section Copyright by SIAM.

4 3.1 Discussion We now pause to justify some of the modeling decisions made above Machines As we argued before, it is unrealistic to assume that there are a linear number of machines available when n large. As such we assume that the total number of machines available is Θ(n 1 ɛ ). We admit that algorithms with too small an ɛ will be impractical should n be large, but it seems unnatural to to tie one s hands by limiting the number of machines to an arbitrary bound of say, O(n 1/2 ). Recall that each key gets mapped to a unique reducer instance. Since the total number of distinct keys may be as large as O(n 2 2ɛ ), the total number of reduce instances may be just as large. Therefore more than one instance of a reducer may be run on the same machine Memory Restrictions As a consequence of mappers and reducers running on physical machines, the total space available to any map or reduce computation is O(n 1 ɛ ). One important consequence of this memory restriction is that the size of every key; value pair must be O(n 1 ɛ ). Another consequence of this memory restriction is that the overall amount of memory available across all machines in the system is O(n 2 2ɛ ). Because the reducers cannot begin executing until after the last mapper has finished, the key value pairs output by the mappers have to be stored temporarily. Thus, the total space taken by all of the key; value pairs in U r must be O(n 2 2ɛ ). That the total memory available, across all machines, is O(n 2 2ɛ ) allows one to duplicate the input somewhat, but not absurdly one is not restricted to simply partitioning the input. In contrast, mappers operate on one tuple at a time, and therefore they can execute on tuples immediately as they are emitted by the reducers. As such, there is no space restriction on the total size of the output of the reducers Shuffle Step In the shuffle step the system partitions the tuples across the Θ(n 1 ɛ ) machines so that all of the pairs with the same key go to the same machine. This allows for the reducer to be executed on that machine. Observe that two pairs (k, V k,r ), (k, V k,r), with k k, may be sent to the same machine to be executed sequentially by different reduce instances. The system must ensure that the memory of no machine is exceeded. Next we prove that the space restrictions in Definition 3.1 allow the shuffle step to place all of the values associated with a key on one machine without violating the memory constraints. Lemma 3.1. Consider round r of the execution of an algorithm in MRC. Let K r be the set of keys in U r, let V r be the multiset of values in U r, and let V k,r denote the multiset of values in U r that have key k. Then K r and V r can be be partitioned across Θ(n 1 ɛ ) machines such that all machines get O(n 1 ɛ ) bits, and the pair k, V k,r gets sent to the same machine. Proof. For a set of binary strings B denote by s(b) = b B b the total space used by the strings in B. Since the algorithm is in MRC, by definition, s(v r )+s(k r ) s(u r) = O(n 2 2ɛ ). Furthermore, the space of the reducer is restricted to O(n 1 ɛ ); therefore for all k, k + s(v k,r ) is O(n 1 ɛ ). Using Graham s greedy algorithm for the minimum makespan scheduling problem [7, 13], we can conclude that the maximum number of bits mapped to any one machine is no more than the average load per machine plus the maximum size of any k, V k,r pair. Thus, s(v r ) + s(k r ) number of machines + max k K r ( k + s(v k,r )) O(n2 2ɛ ) Θ(n 1 ɛ ) + O(n1 ɛ ) O(n 1 ɛ ). We emphasize that every memory restriction in Definition 3.1 is necessary for execution of the shuffle step Time Restrictions Just as one can complain that ɛ may be too small, resulting in impractical algorithms, one can justifiably object that allowing arbitrary polynomial time per mapper and reducer is unreasonable. Our goal in defining MRC is to rigorously define the limitations imposed on the algorithm designer by the MapReduce paradigm. Just as before, we admit that algorithms with polynomial running times of too high a degree will be impractical should n be large. However, it seems unnatural to limit the running time to an arbitrary bound of, say, O(n log n). Finally, the time per MapReduce round in practice can be large, so it is important to dramatically limit the number of rounds. In fact, we strive to find algorithms in MRC 0, but will show that there are many nontrivial algorithms in MRC 1. 4 Related Work We begin by comparing the MapReduce framework with other models of parallel computation. After that we discuss other works that use MapReduce. 941 Copyright by SIAM.

6 desired L would be in P LOGSPACE, yet whether P = LOGSPACE is a long standing open question. 4.2 MapReduce: Algorithms and Models MapReduce is very well suited for naive parallelization for example, counting how many times a word appears in a data set. However, more recently algorithms have emerged for nontrivially parallelizable computations. Kang et al. [9] show how to use MapReduce to compute diameters of massive graphs, taking as an example a webgraph with 1.5 billion nodes and 5.5 billion arcs. Tsourakakis et al. [11] use MapReduce for counting the total number of triangles in a graph. Motivated by personalized news results, Das et al. [3] implement the EM clustering algorithm on MapReduce. Overall, each of these works gives practical MapReduce algorithms, but does not rigorously define the framework under which they should be analyzed. Previously, Feldman et al. [6] introduced the notion of Massively Unordered Distributed (MUD) algorithms, a model based on the MapReduce framework. While modeling the same underlying system, their approach has two crucial differences from ours. First, in the MUD framework each reducer operates on a stream of data, whereas, in our model, each reducer has random access to all of the values associated with the given key. Second, in MUD, each reducer is restricted to only using polylogarithmic space. These distinctions gives our model more power and play an important role in our algorithms. 5 Finding an MST of a Dense Graph Using MapReduce Now that we have formally defined the MapReduce model, we proceed to describe an algorithm in MRC for finding the Minimum Spanning Tree (MST) of a dense graph. As we shall exhibit, this algorithm will take advantage of the interleaving of sequential and parallel computation that MapReduce offers algorithm designers. Thus, given a graph G = (V, E) on V = N vertices and E = m N 1+c edges for some constant c > 0 (n still denoting the length of the input, not the number of vertices), our goal is to compute the minimum spanning tree of the graph. We give a new algorithm for MST and then show how it can be easily parallelized. Fix a number k, and randomly partition the set of vertices into k equally sized subsets, V = V 1 V 2 V k, with V i V j = for i j and V i = N/k for all i. For every pair {i, j}, let E i,j E be the set of edges induced by the vertex set V i V j. That is, E i,j = {(u, v) E u, v V i V j }. Denote the resulting subgraph by G i,j = (V i V j, E i,j ). Assume without loss of generality (say, by appending an index to each weight to break ties) that all of the edge weights are unique. Our algorithm proceeds as follows. First, for each of the ( k 2) subgraphs Gi,j, compute the unique minimum spanning forest M i,j. Then let H be the graph consisting of all of the edges present in some M i,j : H = (V, i,j M i,j ). Finally, compute M, the minimum spanning tree of H. The following theorem proves that this algorithm is correct. Theorem 5.1. The tree M computed by the algorithm is the minimum spanning tree of G. The algorithm works by sparsifying the input graph and then taking the MST of the resulting subgraph H. We show that no relevant edge was thrown out, that is, the minimum spanning trees of G and H are identical. Proof. Consider an edge e = {u, v} that was discarded, that is, e E(G) but e E(H); we show that e is not part of the minimum spanning tree of G. Observe that any edge e = {u, v} is present in at least one subgraph G i,j. If e M i,j then by the cycle property of minimum spanning trees, there must be some cycle C E i,j such that e is the heaviest edge on the cycle. However, since E i,j E, we have now exhibited a cycle in the original graph G on which e is the heaviest edge. Therefore e cannot be in the MST of G and can safely be discarded. The algorithm presented above is far from the optimal sequential algorithm; however, it allows for easy parallelization. Notice that the minimum spanning tree for the individual subgraphs, G i,j, can be computed in parallel. Furthermore, by setting the parameter k appropriately, we can reduce the memory used by each MST computation. As we show below, with high probability the memory used is Õ(m/k) when computing M i,j and O(Nk) when computing the final minimum spanning tree of H. These two facts imply that the algorithm is in MRC. Lemma 5.1. Let k = N c/2, then with high probability the size of every E i,j is Õ(N 1+c/2 ). Proof. We can bound the total number of edges in E i,j by bounding the total degrees of the vertices. E i,j v V i deg(v) + v V j deg(v). For the purpose of the proof only, partition the vertices into groups by their degree: let W 1 be the set of vertices of degree at most 2, W 2, the set of vertices with degree 3 or 4, and generally W i = {v V : 2 i 1 < deg(v) 2 i }. There are log N total groups. Consider the number of vertices from group W i that are mapped to part V j. If the group has a small 943 Copyright by SIAM.

7 number of elements, that is, W i < 2N c/2 log N, then v W i deg(v) 2N 1+c/2 log N = Õ(N 1+c/2 ). If the group is large, that is, W i 2N c/2 log N, a simple application of Chernoff bounds says that the number of elements of W i mapped into the partition j, W i V j is O(log N) with probability at least 1 1 N. Therefore with probability at least 1 log N N : deg(v) deg(v) v V j i v V j W i i 2N 1+c/2 log 2 N Õ(N 1+c/2 ). Lemma 5.1 tells us that with high probability each part has Õ(N 1+c/2 ) edges. Therefore the total input size to any reducer is O(n 1 ɛ ). The algorithm uses the sequential computation available to reducers to compute the minimum spanning tree of the subgraph given to that reducer. There are N c total parts, each producing a spanning tree with 2N/k 1 = O(N 1 c/2 ) edges. Thus the size of H is bounded by Õ(N 1+c/2 ) = O(n 1 ɛ ), again being small enough to fit into the memory of a single machine. 6 An Algorithmic Design Technique For MRC We begin by describing a basic building block of many algorithms in MRC called MRC-parallelizable functions. We then show how a family of such functions can used as subroutines of MRC computations. After that we show how this can be used to compute frequency moments on large inputs, and s-t connectivity on undirected graphs. Definition 6.1. Let S be a set. Call a function f on S MRC-parallelizable if there are functions g and h so that: 1. For any partition T = {T 1, T 2,..., T k } of S, where i T i = S and T i T j = for i j (of course), f can be expressed as: f(s) = h(g(t 1 ), g(t 2 ),..., g(t k )). 2. g and h can be expressed in O(log n) bits. 3. g and h can be computed in time polynomial in S and every output of g can be expressed in O(log n) bits. Intuitively, this definition says that if one wants to evaluate f on a set S, one could do so by partitioning S arbitrarily, applying g to each part of the partition, and then applying h to the results. Next we show how a family of such functions can be computed under the memory restrictions imposed by MRC. Lemma 6.1. Consider a universe U of size n and a collection S = {S 1,..., S k } of subsets of U, where k S i U, i=1 S i n 2 2ɛ, and k n 2 3ɛ. Let F = {f 1,..., f k } be a collection of MRC-parallelizable functions. Then the output f 1 (S 1 ),..., f k (S k ) can be computed using O(n 1 ɛ ) reducers each with O(n 1 ɛ ) space. This lemma says that a family of MRCparallelizable functions defined over subsets of the same universe can be computed as a subroutine of a MapReduce computation where the original input size is n. Since O(n 2 2ɛ ) is be the global amount of memory available to any MapReduce program, the lemma requires that the input has few enough subsets that they fit in memory, and that the sum of the sizes of the subsets also fits into memory. The power of the MRC-parallelizable functions lemma is that it allows an algorithm designer to focus on the structure of the problem and the input; the lemma will handle how to distribute the input across the reducers in such a way as to not overflow the memory of any one reducer. At a high level, we would like to assign a reducer for each set S i, map both the elements of S i and the function f i itself to the same reducer and compute the output. There is two technical challenge which we need to be wary of. The S i may be too large to fit on one reducer. In particular if S i > n 1 ɛ then the computation of f i (S i ) needs to be spread across several reducers. To deal with these issues we use the fact that functions f i are MRC-parallelizable to our advantage. In the first round we partition the set of reducers into t different blocks. Each set S i is then partitioned across the reducers in its assigned block, which computes the intermediate values g i (S i ). This partitioning ensures that the input to any individual reducer is not too large. In the second round, we map all of the intermediate results of block S i to the same reducer, and compute the final output using the function h i. The mapping in this step will again ensure that no reducer is inundated with an input that is larger than its memory. Input The input to the subroutine consists of pairs of the form i; u indicating that u S i, and the individual functions g i and h i for all i [k]. Initialize Let M = n 1 ɛ denote the number of reducers the subroutine will use. Partition them into blocks of size B = Θ(n ɛ ). Let t = M/B be the total number of blocks. Construct universal hash functions, hash 1 and hash 2 : [k] [t]. Map 1: For each i; u, output r; (u, i) where r is chosen uniformly at random among the reducers in block B hash1(i). Map each function g i and h i to 944 Copyright by SIAM.

### MapReduce and Distributed Data Analysis. Sergei Vassilvitskii Google Research

MapReduce and Distributed Data Analysis Google Research 1 Dealing With Massive Data 2 2 Dealing With Massive Data Polynomial Memory Sublinear RAM Sketches External Memory Property Testing 3 3 Dealing With

### MapReduce Algorithms. Sergei Vassilvitskii. Saturday, August 25, 12

MapReduce Algorithms A Sense of Scale At web scales... Mail: Billions of messages per day Search: Billions of searches per day Social: Billions of relationships 2 A Sense of Scale At web scales... Mail:

### http://www.wordle.net/

Hadoop & MapReduce http://www.wordle.net/ http://www.wordle.net/ Hadoop is an open-source software framework (or platform) for Reliable + Scalable + Distributed Storage/Computational unit Failures completely

### CSC2420 Fall 2012: Algorithm Design, Analysis and Theory

CSC2420 Fall 2012: Algorithm Design, Analysis and Theory Allan Borodin November 15, 2012; Lecture 10 1 / 27 Randomized online bipartite matching and the adwords problem. We briefly return to online algorithms

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

### A Load Balancing Technique for Some Coarse-Grained Multicomputer Algorithms

A Load Balancing Technique for Some Coarse-Grained Multicomputer Algorithms Thierry Garcia and David Semé LaRIA Université de Picardie Jules Verne, CURI, 5, rue du Moulin Neuf 80000 Amiens, France, E-mail:

### 16.1 MAPREDUCE. For personal use only, not for distribution. 333

For personal use only, not for distribution. 333 16.1 MAPREDUCE Initially designed by the Google labs and used internally by Google, the MAPREDUCE distributed programming model is now promoted by several

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### Lecture 2: Universality

CS 710: Complexity Theory 1/21/2010 Lecture 2: Universality Instructor: Dieter van Melkebeek Scribe: Tyson Williams In this lecture, we introduce the notion of a universal machine, develop efficient universal

### Discuss the size of the instance for the minimum spanning tree problem.

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

### Big Data Technology Map-Reduce Motivation: Indexing in Search Engines

Big Data Technology Map-Reduce Motivation: Indexing in Search Engines Edward Bortnikov & Ronny Lempel Yahoo Labs, Haifa Indexing in Search Engines Information Retrieval s two main stages: Indexing process

### map/reduce connected components

1, map/reduce connected components find connected components with analogous algorithm: map edges randomly to partitions (k subgraphs of n nodes) for each partition remove edges, so that only tree remains

### Analysis of Algorithms, I

Analysis of Algorithms, I CSOR W4231.002 Eleni Drinea Computer Science Department Columbia University Thursday, February 26, 2015 Outline 1 Recap 2 Representing graphs 3 Breadth-first search (BFS) 4 Applications

### Exponential time algorithms for graph coloring

Exponential time algorithms for graph coloring Uriel Feige Lecture notes, March 14, 2011 1 Introduction Let [n] denote the set {1,..., k}. A k-labeling of vertices of a graph G(V, E) is a function V [k].

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### Two General Methods to Reduce Delay and Change of Enumeration Algorithms

ISSN 1346-5597 NII Technical Report Two General Methods to Reduce Delay and Change of Enumeration Algorithms Takeaki Uno NII-2003-004E Apr.2003 Two General Methods to Reduce Delay and Change of Enumeration

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs

Discrete Mathematics & Mathematical Reasoning Chapter 10: Graphs Kousha Etessami U. of Edinburgh, UK Kousha Etessami (U. of Edinburgh, UK) Discrete Mathematics (Chapter 6) 1 / 13 Overview Graphs and Graph

### Graph Algorithms. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar

Graph Algorithms Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar To accompany the text Introduction to Parallel Computing, Addison Wesley, 3. Topic Overview Definitions and Representation Minimum

### Developing MapReduce Programs

Cloud Computing Developing MapReduce Programs Dell Zhang Birkbeck, University of London 2015/16 MapReduce Algorithm Design MapReduce: Recap Programmers must specify two functions: map (k, v) * Takes

### An Empirical Study of Two MIS Algorithms

An Empirical Study of Two MIS Algorithms Email: Tushar Bisht and Kishore Kothapalli International Institute of Information Technology, Hyderabad Hyderabad, Andhra Pradesh, India 32. tushar.bisht@research.iiit.ac.in,

### The LCA Problem Revisited

The LA Problem Revisited Michael A. Bender Martín Farach-olton SUNY Stony Brook Rutgers University May 16, 2000 Abstract We present a very simple algorithm for the Least ommon Ancestor problem. We thus

### Social Media Mining. Graph Essentials

Graph Essentials Graph Basics Measures Graph and Essentials Metrics 2 2 Nodes and Edges A network is a graph nodes, actors, or vertices (plural of vertex) Connections, edges or ties Edge Node Measures

### Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques. My T. Thai

Approximation Algorithms: LP Relaxation, Rounding, and Randomized Rounding Techniques My T. Thai 1 Overview An overview of LP relaxation and rounding method is as follows: 1. Formulate an optimization

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Lecture 2: The Complexity of Some Problems

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 2: The Complexity of Some Problems David Mix Barrington and Alexis Maciel July 18, 2000

### 8.1 Min Degree Spanning Tree

CS880: Approximations Algorithms Scribe: Siddharth Barman Lecturer: Shuchi Chawla Topic: Min Degree Spanning Tree Date: 02/15/07 In this lecture we give a local search based algorithm for the Min Degree

### Simulation-Based Security with Inexhaustible Interactive Turing Machines

Simulation-Based Security with Inexhaustible Interactive Turing Machines Ralf Küsters Institut für Informatik Christian-Albrechts-Universität zu Kiel 24098 Kiel, Germany kuesters@ti.informatik.uni-kiel.de

### 6.852: Distributed Algorithms Fall, 2009. Class 2

.8: Distributed Algorithms Fall, 009 Class Today s plan Leader election in a synchronous ring: Lower bound for comparison-based algorithms. Basic computation in general synchronous networks: Leader election

### Lecture 4 Online and streaming algorithms for clustering

CSE 291: Geometric algorithms Spring 2013 Lecture 4 Online and streaming algorithms for clustering 4.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Diversity Coloring for Distributed Data Storage in Networks 1

Diversity Coloring for Distributed Data Storage in Networks 1 Anxiao (Andrew) Jiang and Jehoshua Bruck California Institute of Technology Pasadena, CA 9115, U.S.A. {jax, bruck}@paradise.caltech.edu Abstract

### ECEN 5682 Theory and Practice of Error Control Codes

ECEN 5682 Theory and Practice of Error Control Codes Convolutional Codes University of Colorado Spring 2007 Linear (n, k) block codes take k data symbols at a time and encode them into n code symbols.

### The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,

The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints

### Lecture 6 Online and streaming algorithms for clustering

CSE 291: Unsupervised learning Spring 2008 Lecture 6 Online and streaming algorithms for clustering 6.1 On-line k-clustering To the extent that clustering takes place in the brain, it happens in an on-line

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

### MapReduce and Hadoop. Aaron Birkland Cornell Center for Advanced Computing. January 2012

MapReduce and Hadoop Aaron Birkland Cornell Center for Advanced Computing January 2012 Motivation Simple programming model for Big Data Distributed, parallel but hides this Established success at petabyte

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### The Relative Worst Order Ratio for On-Line Algorithms

The Relative Worst Order Ratio for On-Line Algorithms Joan Boyar 1 and Lene M. Favrholdt 2 1 Department of Mathematics and Computer Science, University of Southern Denmark, Odense, Denmark, joan@imada.sdu.dk

### Introduction to DISC and Hadoop

Introduction to DISC and Hadoop Alice E. Fischer April 24, 2009 Alice E. Fischer DISC... 1/20 1 2 History Hadoop provides a three-layer paradigm Alice E. Fischer DISC... 2/20 Parallel Computing Past and

Adaptive Online Gradient Descent Peter L Bartlett Division of Computer Science Department of Statistics UC Berkeley Berkeley, CA 94709 bartlett@csberkeleyedu Elad Hazan IBM Almaden Research Center 650

### Large-Scale Data Sets Clustering Based on MapReduce and Hadoop

Journal of Computational Information Systems 7: 16 (2011) 5956-5963 Available at http://www.jofcis.com Large-Scale Data Sets Clustering Based on MapReduce and Hadoop Ping ZHOU, Jingsheng LEI, Wenjun YE

### Labeling outerplanar graphs with maximum degree three

Labeling outerplanar graphs with maximum degree three Xiangwen Li 1 and Sanming Zhou 2 1 Department of Mathematics Huazhong Normal University, Wuhan 430079, China 2 Department of Mathematics and Statistics

### Finding and counting given length cycles

Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected

### 6.042/18.062J Mathematics for Computer Science October 3, 2006 Tom Leighton and Ronitt Rubinfeld. Graph Theory III

6.04/8.06J Mathematics for Computer Science October 3, 006 Tom Leighton and Ronitt Rubinfeld Lecture Notes Graph Theory III Draft: please check back in a couple of days for a modified version of these

### The Trip Scheduling Problem

The Trip Scheduling Problem Claudia Archetti Department of Quantitative Methods, University of Brescia Contrada Santa Chiara 50, 25122 Brescia, Italy Martin Savelsbergh School of Industrial and Systems

### ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN. Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015

ONLINE DEGREE-BOUNDED STEINER NETWORK DESIGN Sina Dehghani Saeed Seddighin Ali Shafahi Fall 2015 ONLINE STEINER FOREST PROBLEM An initially given graph G. s 1 s 2 A sequence of demands (s i, t i ) arriving

### JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS. Received December May 12, 2003; revised February 5, 2004

Scientiae Mathematicae Japonicae Online, Vol. 10, (2004), 431 437 431 JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS Ondřej Čepeka and Shao Chin Sung b Received December May 12, 2003; revised February

### Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:

CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm

### Offline sorting buffers on Line

Offline sorting buffers on Line Rohit Khandekar 1 and Vinayaka Pandit 2 1 University of Waterloo, ON, Canada. email: rkhandekar@gmail.com 2 IBM India Research Lab, New Delhi. email: pvinayak@in.ibm.com

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### Chapter 6: Episode discovery process

Chapter 6: Episode discovery process Algorithmic Methods of Data Mining, Fall 2005, Chapter 6: Episode discovery process 1 6. Episode discovery process The knowledge discovery process KDD process of analyzing

### On the k-path cover problem for cacti

On the k-path cover problem for cacti Zemin Jin and Xueliang Li Center for Combinatorics and LPMC Nankai University Tianjin 300071, P.R. China zeminjin@eyou.com, x.li@eyou.com Abstract In this paper we

### A Sublinear Bipartiteness Tester for Bounded Degree Graphs

A Sublinear Bipartiteness Tester for Bounded Degree Graphs Oded Goldreich Dana Ron February 5, 1998 Abstract We present a sublinear-time algorithm for testing whether a bounded degree graph is bipartite

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### arxiv:1412.2333v1 [cs.dc] 7 Dec 2014

Minimum-weight Spanning Tree Construction in O(log log log n) Rounds on the Congested Clique Sriram V. Pemmaraju Vivek B. Sardeshmukh Department of Computer Science, The University of Iowa, Iowa City,

### Distributed Computing over Communication Networks: Maximal Independent Set

Distributed Computing over Communication Networks: Maximal Independent Set What is a MIS? MIS An independent set (IS) of an undirected graph is a subset U of nodes such that no two nodes in U are adjacent.

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### Load Balancing in MapReduce Based on Scalable Cardinality Estimates

Load Balancing in MapReduce Based on Scalable Cardinality Estimates Benjamin Gufler 1, Nikolaus Augsten #, Angelika Reiser 3, Alfons Kemper 4 Technische Universität München Boltzmannstraße 3, 85748 Garching

### Page 1. CSCE 310J Data Structures & Algorithms. CSCE 310J Data Structures & Algorithms. P, NP, and NP-Complete. Polynomial-Time Algorithms

CSCE 310J Data Structures & Algorithms P, NP, and NP-Complete Dr. Steve Goddard goddard@cse.unl.edu CSCE 310J Data Structures & Algorithms Giving credit where credit is due:» Most of the lecture notes

### GRAPH THEORY and APPLICATIONS. Trees

GRAPH THEORY and APPLICATIONS Trees Properties Tree: a connected graph with no cycle (acyclic) Forest: a graph with no cycle Paths are trees. Star: A tree consisting of one vertex adjacent to all the others.

### Distance Degree Sequences for Network Analysis

Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation

### Competitive Analysis of On line Randomized Call Control in Cellular Networks

Competitive Analysis of On line Randomized Call Control in Cellular Networks Ioannis Caragiannis Christos Kaklamanis Evi Papaioannou Abstract In this paper we address an important communication issue arising

### Big Data Storage, Management and challenges. Ahmed Ali-Eldin

Big Data Storage, Management and challenges Ahmed Ali-Eldin (Ambitious) Plan What is Big Data? And Why talk about Big Data? How to store Big Data? BigTables (Google) Dynamo (Amazon) How to process Big

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### MapReduce: Algorithm Design Patterns

Designing Algorithms for MapReduce MapReduce: Algorithm Design Patterns Need to adapt to a restricted model of computation Goals Scalability: adding machines will make the algo run faster Efficiency: resources

### NP-Completeness and Cook s Theorem

NP-Completeness and Cook s Theorem Lecture notes for COM3412 Logic and Computation 15th January 2002 1 NP decision problems The decision problem D L for a formal language L Σ is the computational task:

### Parallel Programming Map-Reduce. Needless to Say, We Need Machine Learning for Big Data

Case Study 2: Document Retrieval Parallel Programming Map-Reduce Machine Learning/Statistics for Big Data CSE599C1/STAT592, University of Washington Carlos Guestrin January 31 st, 2013 Carlos Guestrin

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### GENERATING LOW-DEGREE 2-SPANNERS

SIAM J. COMPUT. c 1998 Society for Industrial and Applied Mathematics Vol. 27, No. 5, pp. 1438 1456, October 1998 013 GENERATING LOW-DEGREE 2-SPANNERS GUY KORTSARZ AND DAVID PELEG Abstract. A k-spanner

### Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design

Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design Hongsik Choi and Hyeong-Ah Choi Department of Electrical Engineering and Computer Science George Washington University Washington,

### Asking Hard Graph Questions. Paul Burkhardt. February 3, 2014

Beyond Watson: Predictive Analytics and Big Data U.S. National Security Agency Research Directorate - R6 Technical Report February 3, 2014 300 years before Watson there was Euler! The first (Jeopardy!)

### Open source software framework designed for storage and processing of large scale data on clusters of commodity hardware

Open source software framework designed for storage and processing of large scale data on clusters of commodity hardware Created by Doug Cutting and Mike Carafella in 2005. Cutting named the program after

Introduction to Hadoop 1 What is Hadoop? the big data revolution extracting value from data cloud computing 2 Understanding MapReduce the word count problem more examples MCS 572 Lecture 24 Introduction

### Introduction to Hadoop and MapReduce

Introduction to Hadoop and MapReduce THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Large-scale Computation Traditional solutions for computing large quantities of data

### NP-Completeness I. Lecture 19. 19.1 Overview. 19.2 Introduction: Reduction and Expressiveness

Lecture 19 NP-Completeness I 19.1 Overview In the past few lectures we have looked at increasingly more expressive problems that we were able to solve using efficient algorithms. In this lecture we introduce

### Lecture 7: Approximation via Randomized Rounding

Lecture 7: Approximation via Randomized Rounding Often LPs return a fractional solution where the solution x, which is supposed to be in {0, } n, is in [0, ] n instead. There is a generic way of obtaining

### Compact Representations and Approximations for Compuation in Games

Compact Representations and Approximations for Compuation in Games Kevin Swersky April 23, 2008 Abstract Compact representations have recently been developed as a way of both encoding the strategic interactions

CSE / Notes : Task Scheduling & Load Balancing Task Scheduling A task is a (sequential) activity that uses a set of inputs to produce a set of outputs. A task (precedence) graph is an acyclic, directed

### Big Data Begets Big Database Theory

Big Data Begets Big Database Theory Dan Suciu University of Washington 1 Motivation Industry analysts describe Big Data in terms of three V s: volume, velocity, variety. The data is too big to process

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### Lecture Notes on Spanning Trees

Lecture Notes on Spanning Trees 15-122: Principles of Imperative Computation Frank Pfenning Lecture 26 April 26, 2011 1 Introduction In this lecture we introduce graphs. Graphs provide a uniform model

### A Note on Maximum Independent Sets in Rectangle Intersection Graphs

A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,

### Bounded Cost Algorithms for Multivalued Consensus Using Binary Consensus Instances

Bounded Cost Algorithms for Multivalued Consensus Using Binary Consensus Instances Jialin Zhang Tsinghua University zhanggl02@mails.tsinghua.edu.cn Wei Chen Microsoft Research Asia weic@microsoft.com Abstract

### MAPREDUCE Programming Model

CS 2510 COMPUTER OPERATING SYSTEMS Cloud Computing MAPREDUCE Dr. Taieb Znati Computer Science Department University of Pittsburgh MAPREDUCE Programming Model Scaling Data Intensive Application MapReduce

### Introduction to Scheduling Theory

Introduction to Scheduling Theory Arnaud Legrand Laboratoire Informatique et Distribution IMAG CNRS, France arnaud.legrand@imag.fr November 8, 2004 1/ 26 Outline 1 Task graphs from outer space 2 Scheduling

### Generating models of a matched formula with a polynomial delay

Generating models of a matched formula with a polynomial delay Petr Savicky Institute of Computer Science, Academy of Sciences of Czech Republic, Pod Vodárenskou Věží 2, 182 07 Praha 8, Czech Republic

Hadoop and Map-reduce computing 1 Introduction This activity contains a great deal of background information and detailed instructions so that you can refer to it later for further activities and homework.

### CSE-E5430 Scalable Cloud Computing Lecture 2

CSE-E5430 Scalable Cloud Computing Lecture 2 Keijo Heljanko Department of Computer Science School of Science Aalto University keijo.heljanko@aalto.fi 14.9-2015 1/36 Google MapReduce A scalable batch processing

### COLORED GRAPHS AND THEIR PROPERTIES

COLORED GRAPHS AND THEIR PROPERTIES BEN STEVENS 1. Introduction This paper is concerned with the upper bound on the chromatic number for graphs of maximum vertex degree under three different sets of coloring

### Private Approximation of Clustering and Vertex Cover

Private Approximation of Clustering and Vertex Cover Amos Beimel, Renen Hallak, and Kobbi Nissim Department of Computer Science, Ben-Gurion University of the Negev Abstract. Private approximation of search

### 1 Basic Definitions and Concepts in Graph Theory

CME 305: Discrete Mathematics and Algorithms 1 Basic Definitions and Concepts in Graph Theory A graph G(V, E) is a set V of vertices and a set E of edges. In an undirected graph, an edge is an unordered

### P versus NP, and More

1 P versus NP, and More Great Ideas in Theoretical Computer Science Saarland University, Summer 2014 If you have tried to solve a crossword puzzle, you know that it is much harder to solve it than to verify

### Cloud Computing. Chapter 8. 8.1 Hadoop

Chapter 8 Cloud Computing In cloud computing, the idea is that a large corporation that has many computers could sell time on them, for example to make profitable use of excess capacity. The typical customer

### Keywords: Big Data, HDFS, Map Reduce, Hadoop

Volume 5, Issue 7, July 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Configuration Tuning

### The Classes P and NP

The Classes P and NP We now shift gears slightly and restrict our attention to the examination of two families of problems which are very important to computer scientists. These families constitute the