Efficient parallel string comparison
|
|
|
- Meredith Houston
- 7 years ago
- Views:
Transcription
1 Efficient parallel string comparison Peter Krusche and Alexander Tiskin Department of Computer Science ParCo 2007
2 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
3 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
4 BSP Algorithms Bulk-Synchronous Parallelism Model for parallel computation: L. G. Valiant. A bridging model for parallel computation. Communications of the ACM, 33: , Main ideas: p processors working asynchronously Can communicate using g operations to transmit one element of data Can synchronise using l sequential operations.
5 BSP Algorithms Bulk-Synchronous Parallelism Computation proceeds in supersteps Communication takes place at the end of each superstep Between supersteps, barrier-style synchronisation takes place Superstep P1 P2... Pp Barrier-Synchronization Barrier-Synchronization Computation Communication Computation Communication
6 BSP Algorithms Bulk-Synchronous Parallelism Superstep s has computation cost w s and communication h s = max(h in s, h out s ). When there are S supersteps: Computation work W = 1 s S w s Communication H = 1 s S h s Formula for running time: T = W + g H + l S.
7 Parallel Prefix in BSP Definition (Parallel Prefix) Given n values x 1, x 2,..., x n and an associative operator, compute the values x 1, x 1 x 2, x 1 x 2 x 3,..., i=1,2,...,n x i. Fact... Under some natural assumptions, we can carry out a parallel prefix operation over n elements on a BSP computer with p processors using W = O( n p ), H = O(p) and S = O(1).
8 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
9 The LCS Problem Definition (Input data) Let x = x 1 x 2... x m and y = y 1 y 2... y n be two strings on an alphabet Σ. Definition (Subsequences) A subsequence u of x: u can be obtained by deleting zero or more elements from x. Definition (Longest Common Subsequences) An LCS (x, y) is any string which is subsequence of both x and y and has maximum possible length. Length of these sequences: LLCS (x, y).
10 The Semi-local LCS Problem Definition (Substrings) A substring of any string x can be obtained by removing zero or more characters from the beginning and/or the end of x. Definition (Highest-score matrix) The element A(i, j) of the LCS highest-score matrix of two strings x and y gives the LLCS of y i... y j and x. Definition (Semi-local LCS) Solutions to the semi-local LCS problem are represented by a (possibly implicit) highest-score matrix A(i, j).
11 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
12 LCS grid dags and highest-score matrices LCS Problem can be represented as longest path problem in a Grid DAG String-Substring LCS Problem A(i, j) = length of longest path from (0, i) to (n, j) (top to bottom). Example y a b c a x a c b c
13 Extended grid dag Infinite extension of the LCS grid dag, outside the core area, everything matches: The extended highest-score matrix is now defined on indices [, + ] [, + ].
14 Additional definitions Definition (Integer ranges) We denote the set of integers {i, i + 1,..., j} as [i : j]. Definition (Odd half-integers) We denote half-integer variables using a ˆ, and denote the set of half-integers {i + 1 2, i + 3 2,..., j 1 2 } as i : j.
15 Critical point theorem Definition (Critical Point) Odd half-integer point (i 1 2, j ) is critical iff. A(i, j) + 1 = A(i 1, j) = A(i, j + 1) = A(i 1, j + 1). Theorem (Schmidt 95, Alves+ 06, Tiskin 05) 1 We can represent a the whole extended highest-score matrix by a finite set of such critical points. 2 Assuming w.l.o.g. input strings of equal length n, there are N = 2n such critical points that implicitly represent the whole score matrix. 3 There is an algorithm to obtain these points in time O(n 2 ).
16 Highest-score matrices Example (Explicit highest-score matrix)
17 Querying highest-score matrix entries Theorem (Tiskin 05) If d(i, j) is the number of critical points (î, ĵ) in the extended score matrix with i < î and ĵ < j, then A(i, j) = j i d(i, j). Definition (Density and distribution matrices) The elements d(i, j) form a distribution matrix over the entries of density (permutation) matrix D with nonzeros at all critical points (î, ĵ) in the extended highest-score matrix: d(i, j) = D(î, ĵ) (î,ĵ) i:n 0:j Can compute this efficiently using range querying data structures.
18 Querying highest-score matrix entries Theorem (Tiskin 05) If d(i, j) is the number of critical points (î, ĵ) in the extended score matrix with i < î and ĵ < j, then A(i, j) = j i d(i, j). Definition (Density and distribution matrices) The elements d(i, j) form a distribution matrix over the entries of density (permutation) matrix D with nonzeros at all critical points (î, ĵ) in the extended highest-score matrix: d(i, j) = D(î, ĵ) (î,ĵ) i:n 0:j Can compute this efficiently using range querying data structures.
19 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
20 Sequential highest-score matrix multiplication Algorithm, Tiskin 05 Given the distribution matrices d A and d B for two adjacent blocks of equal height M and width N in the grid dag, we can compute the distribution matrix d C for the union of these blocks in O(N M).
21 Sequential highest-score matrix multiplication Combine critical points by removing double crossings Trivial part (O(M)):
22 Sequential highest-score matrix multiplication Combine critical points by removing double crossings Nontrivial part (O(N 1.5 )):
23 Nontrivial part The non-trivial part can be seen as (min, +) matrix product (d A B C are now the nontrivial parts of the corresponding distribution matrices): d C (i, k) = min(d A (i, j) + d B (j, k)) j Explicit form, naive algorithm: O(n 3 ) Explicit form, algorithm that uses highest-score matrix properties: O(n 2 ) Implicit form, divide-and-conquer: O(n 1.5 )
24 Nontrivial part The non-trivial part can be seen as (min, +) matrix product (d A B C are now the nontrivial parts of the corresponding distribution matrices): d C (i, k) = min(d A (i, j) + d B (j, k)) j Explicit form, naive algorithm: O(n 3 ) Explicit form, algorithm that uses highest-score matrix properties: O(n 2 ) Implicit form, divide-and-conquer: O(n 1.5 )
25 Nontrivial part The non-trivial part can be seen as (min, +) matrix product (d A B C are now the nontrivial parts of the corresponding distribution matrices): d C (i, k) = min(d A (i, j) + d B (j, k)) j Explicit form, naive algorithm: O(n 3 ) Explicit form, algorithm that uses highest-score matrix properties: O(n 2 ) Implicit form, divide-and-conquer: O(n 1.5 )
26 Nontrivial part The non-trivial part can be seen as (min, +) matrix product (d A B C are now the nontrivial parts of the corresponding distribution matrices): d C (i, k) = min(d A (i, j) + d B (j, k)) j Explicit form, naive algorithm: O(n 3 ) Explicit form, algorithm that uses highest-score matrix properties: O(n 2 ) Implicit form, divide-and-conquer: O(n 1.5 )
27 Divide-and-conquer multiplication C-blocks and relevant nonzeros D B D A D C
28 Divide-and-conquer multiplication C-blocks and relevant nonzeros relevant nonzeros D B D A C-block sized h h D C
29 Divide-and-conquer multiplication C-blocks and relevant nonzeros relevant nonzeros D B D A C-block sized h h D C
30 Divide-and-conquer multiplication C-blocks and relevant nonzeros relevant nonzeros D B D A C-block sized h h D C
31 Divide-and-conquer multiplication C-blocks and relevant nonzeros relevant nonzeros D B D A C-block sized h h local minimum D C
32 Divide-and-conquer multiplication δ-sequences and relevant nonzeros D A D B D C j Splitting relevant nonzeros in D A and D B into two sets at a position j [0 : N], we get numbers δ A (j) of relevant nonzeros in D A up to column j 1 2 δ B (j) of relevant nonzeros in D B starting at row j + 1 2
33 Divide-and-conquer multiplication j-blocks Example (j-blocks) h δ A d = h d = 0 δ B d = h j Definition ( -sequences) δ s don t change inside a j-block A (d) = any δ B (j) j J (d) B (d) = any δ B (j) j J (d)
34 Divide-and-conquer multiplication j-blocks Example (j-blocks) h δ A d = h d = 0 δ B d = h j Definition ( -sequences) δ s don t change inside a j-block A (d) = any δ B (j) j J (d) B (d) = any δ B (j) j J (d)
35 Divide-and-conquer multiplication Local minima Definition (local minima) The sequence M (d) = min (d A(i 0, j) + d B (j, k 0 )) j J (d) contains the minimum of d A (i 0, j) + d B (j, k 0 ) in every j-block. We can use M s and s to compute the number of nonzeros in a C-block.
36 Divide-and-conquer multiplication Local minima Definition (local minima) The sequence M (d) = min (d A(i 0, j) + d B (j, k 0 )) j J (d) contains the minimum of d A (i 0, j) + d B (j, k 0 ) in every j-block. We can use M s and s to compute the number of nonzeros in a C-block.
37 Divide-and-conquer multiplication Recursive step Sequences M for every C-subblock can be computed in O(h): M (d ) = min d M (d ) = min d M (d ) = min d M (d ) = min d M (d), M (d) + B (d), M (d) + A (d), M (d) + A (d) + B (d) having (i,k, h 2 ) A (d) (i,k, h 2 ) B (d) = d with d [ h 2 : h 2 ].
38 Divide-and-conquer multiplication Recursive step Sequences A (d ) and B (d ) can also be determined in O(h) by a scan of the relevant nonzeros for each subblock. Knowing A (d ), B (d ) and M(d ) for each subblock, we can continue the recursion in every subblock. The recursion terminates when N C-blocks of size 1 are left.
39 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
40 Basic algorithm idea Start the recursion at a point where there are p C-blocks. This is at level 1 2 log p. Precompute and distribute the required sequences and M for each C-block in parallel. Every C-block has size h = N p, and hence requires sequences with O( N p ) values. After these sequences have been precomputed and redistributed, we can use the sequential algorithm to finish the computation.
41 Assumptions Assume that: p is an integer. Every processor has unique identifier q with 0 q < p. Every processor q corresponds to exactly one location (q x, q y ) [0 : p 1] [0 : p 1]. Initial distribution of nonzeros in D A and D B is assumed to be even among all processors.
42 First Step Redistribute the nonzeros to strips of width N p Send all nonzeros (î, ĵ) in D A and (ĵ, ˆk) in D B to processor (ĵ 1 2 ) p/n. Possible in one superstep using communication O( N p ). i j D A k D B D C p vertical strips N nonzeros each p }{{} N p } N p
43 Precomputing M k j i D A D B D C 1 Compute the elementary (min, +) products d A ( x, j) + d B (j, y ) along j [0 : N]. 2 Processor q holds all D A (î, ĵ) and all D B (ĵ, ˆk) for ĵ q N N p : (q + 1) p. 3 Can compute the values d A ( x, j) and d B (j, y ) by using parallel prefix/suffix. 4 After prefix and suffix computations, every processor holds N/p values d A ( x, j) + d B (j, y ) for j [q N p : (q + 1) N p ].
44 Precomputing M k j i D A D B D C 1 Compute the elementary (min, +) products d A ( x, j) + d B (j, y ) along j [0 : N]. 2 Processor q holds all D A (î, ĵ) and all D B (ĵ, ˆk) for ĵ q N N p : (q + 1) p. 3 Can compute the values d A ( x, j) and d B (j, y ) by using parallel prefix/suffix. 4 After prefix and suffix computations, every processor holds N/p values d A ( x, j) + d B (j, y ) for j [q N p : (q + 1) N p ].
45 Precomputing M k j i D A D B D C 1 Compute the elementary (min, +) products d A ( x, j) + d B (j, y ) along j [0 : N]. 2 Processor q holds all D A (î, ĵ) and all D B (ĵ, ˆk) for ĵ q N N p : (q + 1) p. 3 Can compute the values d A ( x, j) and d B (j, y ) by using parallel prefix/suffix. 4 After prefix and suffix computations, every processor holds N/p values d A ( x, j) + d B (j, y ) for j [q N p : (q + 1) N p ].
46 Precomputing M k j i D A D B D C 1 Compute the elementary (min, +) products d A ( x, j) + d B (j, y ) along j [0 : N]. 2 Processor q holds all D A (î, ĵ) and all D B (ĵ, ˆk) for ĵ q N N p : (q + 1) p. 3 Can compute the values d A ( x, j) and d B (j, y ) by using parallel prefix/suffix. 4 After prefix and suffix computations, every processor holds N/p values d A ( x, j) + d B (j, y ) for j [q N p : (q + 1) N p ].
47 Redistribution Step i j k D B D B D A D A D C p vertical strips N p nonzeros each D C p horizontal strips with nonzeros each N p
48 Analysis Computational work bounded by the sequential recursion: W = O((N/ p) 1.5 ) = O(N 1.5 /p 0.75 ) Every processor holds O(N/p) nonzeros before redistribution. Every nonzero is relevant for p C-blocks. O(N/ p) communication for redistributing the nonzeros. H = O(N/ p + p + N/ p) = O(N/ p) (if N/ p > p N > p 1.5 ) S = O(1) (parallel prefix)
49 Analysis Computational work bounded by the sequential recursion: W = O((N/ p) 1.5 ) = O(N 1.5 /p 0.75 ) Every processor holds O(N/p) nonzeros before redistribution. Every nonzero is relevant for p C-blocks. O(N/ p) communication for redistributing the nonzeros. H = O(N/ p + p + N/ p) = O(N/ p) (if N/ p > p N > p 1.5 ) S = O(1) (parallel prefix)
50 Analysis Computational work bounded by the sequential recursion: W = O((N/ p) 1.5 ) = O(N 1.5 /p 0.75 ) Every processor holds O(N/p) nonzeros before redistribution. Every nonzero is relevant for p C-blocks. O(N/ p) communication for redistributing the nonzeros. H = O(N/ p + p + N/ p) = O(N/ p) (if N/ p > p N > p 1.5 ) S = O(1) (parallel prefix)
51 Analysis Computational work bounded by the sequential recursion: W = O((N/ p) 1.5 ) = O(N 1.5 /p 0.75 ) Every processor holds O(N/p) nonzeros before redistribution. Every nonzero is relevant for p C-blocks. O(N/ p) communication for redistributing the nonzeros. H = O(N/ p + p + N/ p) = O(N/ p) (if N/ p > p N > p 1.5 ) S = O(1) (parallel prefix)
52 Outline 1 Introduction Bulk-Synchronous Parallelism String comparison 2 Sequential LCS algorithms Sequential semi-local LCS Divide-and-conquer semi-local LCS 3 The parallel algorithm Parallel score-matrix multiplication Parallel LCS computation
53 Quadtree Merging First, compute scores for a regular grid of p sub-dags of size n/ p n/ p Then merge these in a quadtree-like scheme using parallel score-matrix multiplication:
54 Analysis Quadtree has 1 2 log 2 p levels On level l, 0 l 1 2 log 2 p, we have p l = p (number of processors that work together on one 4 l merge) N l = N 2 l w l = O h l = O (block size of merge) ) ( ) = O N 1.5 p 0.75 ( ( ) 1.5 N 2 ( l ) p l ( N 2 l ( p 4 l ) 0.5 ) = O ( N p 0.5 )
55 Analysis Quadtree has 1 2 log 2 p levels Hence, we get work W = O( n2 p + N1.5 log p ) = O(n 2 /p) p 0.75 (assuming that n p 2 ), communication H = O( n log p p ), and S = O(log p) supersteps.
56 Comparison to other parallel algorithms W H S References Global LCS O( n2 p ) O(n) O(p) [McColl 95]+ [Wagner & Fischer 74] String-Substring LCS O( n2 p ) O(Cp1/C n log p) O(log p) [Alves+ 03] O( n2 p ) O(n log p) O(log p) [Tiskin 05], [Alves+:06] String-Substring, Prefix-Suffix LCS O( n2 log n p ) O( n2 log p p ) O(log p) [Alves+ 02] O( n2 p ) O(n) O(p) [McColl 95]+ [Alves+ 06], [Tiskin 05] O( n2 p ) n log p O( p ) O(log p) NEW
57 Summary and Outlook Summary We have looked at a parallel algorithm for semilocal string comparison that is communication efficient (in fact, achieving scalable communication), work-optimal,! and asymptotically better than even global LCS computation. Outlook Score matrix multiplication can also be applied to create a scalable algorithm for the longest increasing subsequence problem. Algorithm can be adapted to compute edit-distances.
58 Summary and Outlook Summary We have looked at a parallel algorithm for semilocal string comparison that is communication efficient (in fact, achieving scalable communication), work-optimal,! and asymptotically better than even global LCS computation. Outlook Score matrix multiplication can also be applied to create a scalable algorithm for the longest increasing subsequence problem. Algorithm can be adapted to compute edit-distances.
59 Thank you! Any questions?
A Load Balancing Technique for Some Coarse-Grained Multicomputer Algorithms
A Load Balancing Technique for Some Coarse-Grained Multicomputer Algorithms Thierry Garcia and David Semé LaRIA Université de Picardie Jules Verne, CURI, 5, rue du Moulin Neuf 80000 Amiens, France, E-mail:
Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new
Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of
Reminder: Complexity (1) Parallel Complexity Theory. Reminder: Complexity (2) Complexity-new GAP (2) Graph Accessibility Problem (GAP) (1)
Reminder: Complexity (1) Parallel Complexity Theory Lecture 6 Number of steps or memory units required to compute some result In terms of input size Using a single processor O(1) says that regardless of
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1
5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition
4/1/2017. PS. Sequences and Series FROM 9.2 AND 9.3 IN THE BOOK AS WELL AS FROM OTHER SOURCES. TODAY IS NATIONAL MANATEE APPRECIATION DAY
PS. Sequences and Series FROM 9.2 AND 9.3 IN THE BOOK AS WELL AS FROM OTHER SOURCES. TODAY IS NATIONAL MANATEE APPRECIATION DAY 1 Oh the things you should learn How to recognize and write arithmetic sequences
Systems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
5.3 The Cross Product in R 3
53 The Cross Product in R 3 Definition 531 Let u = [u 1, u 2, u 3 ] and v = [v 1, v 2, v 3 ] Then the vector given by [u 2 v 3 u 3 v 2, u 3 v 1 u 1 v 3, u 1 v 2 u 2 v 1 ] is called the cross product (or
Design of LDPC codes
Design of LDPC codes Codes from finite geometries Random codes: Determine the connections of the bipartite Tanner graph by using a (pseudo)random algorithm observing the degree distribution of the code
Lossless Data Compression Standard Applications and the MapReduce Web Computing Framework
Lossless Data Compression Standard Applications and the MapReduce Web Computing Framework Sergio De Agostino Computer Science Department Sapienza University of Rome Internet as a Distributed System Modern
FACTORING SPARSE POLYNOMIALS
FACTORING SPARSE POLYNOMIALS Theorem 1 (Schinzel): Let r be a positive integer, and fix non-zero integers a 0,..., a r. Let F (x 1,..., x r ) = a r x r + + a 1 x 1 + a 0. Then there exist finite sets S
Notes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
Cost Model: Work, Span and Parallelism. 1 The RAM model for sequential computation:
CSE341T 08/31/2015 Lecture 3 Cost Model: Work, Span and Parallelism In this lecture, we will look at how one analyze a parallel program written using Cilk Plus. When we analyze the cost of an algorithm
(67902) Topics in Theory and Complexity Nov 2, 2006. Lecture 7
(67902) Topics in Theory and Complexity Nov 2, 2006 Lecturer: Irit Dinur Lecture 7 Scribe: Rani Lekach 1 Lecture overview This Lecture consists of two parts In the first part we will refresh the definition
Dr. Alexander Tiskin Curriculum Vitae
Associate Professor Dr. Alexander Tiskin Curriculum Vitae Department of Computer Science, University of Warwick, Coventry CV4 7AL, United Kingdom email: tiskin@dcs.warwick.ac.uk web: http://go.warwick.ac.uk/alextiskin
Continued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
Gambling and Data Compression
Gambling and Data Compression Gambling. Horse Race Definition The wealth relative S(X) = b(x)o(x) is the factor by which the gambler s wealth grows if horse X wins the race, where b(x) is the fraction
CIS 700: algorithms for Big Data
CIS 700: algorithms for Big Data Lecture 6: Graph Sketching Slides at http://grigory.us/big-data-class.html Grigory Yaroslavtsev http://grigory.us Sketching Graphs? We know how to sketch vectors: v Mv
An example of a computable
An example of a computable absolutely normal number Verónica Becher Santiago Figueira Abstract The first example of an absolutely normal number was given by Sierpinski in 96, twenty years before the concept
The Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
Factorization Theorems
Chapter 7 Factorization Theorems This chapter highlights a few of the many factorization theorems for matrices While some factorization results are relatively direct, others are iterative While some factorization
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
The Pointless Machine and Escape of the Clones
MATH 64091 Jenya Soprunova, KSU The Pointless Machine and Escape of the Clones The Pointless Machine that operates on ordered pairs of positive integers (a, b) has three modes: In Mode 1 the machine adds
Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction
Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach
Section 8.2 Solving a System of Equations Using Matrices (Guassian Elimination)
Section 8. Solving a System of Equations Using Matrices (Guassian Elimination) x + y + z = x y + 4z = x 4y + z = System of Equations x 4 y = 4 z A System in matrix form x A x = b b 4 4 Augmented Matrix
Row Echelon Form and Reduced Row Echelon Form
These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +
ECE 842 Report Implementation of Elliptic Curve Cryptography
ECE 842 Report Implementation of Elliptic Curve Cryptography Wei-Yang Lin December 15, 2004 Abstract The aim of this report is to illustrate the issues in implementing a practical elliptic curve cryptographic
Warshall s Algorithm: Transitive Closure
CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths
IE 680 Special Topics in Production Systems: Networks, Routing and Logistics*
IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti
Distance Degree Sequences for Network Analysis
Universität Konstanz Computer & Information Science Algorithmics Group 15 Mar 2005 based on Palmer, Gibbons, and Faloutsos: ANF A Fast and Scalable Tool for Data Mining in Massive Graphs, SIGKDD 02. Motivation
Mathematics Course 111: Algebra I Part IV: Vector Spaces
Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are
Lecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
Why? A central concept in Computer Science. Algorithms are ubiquitous.
Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online
System Modeling Introduction Rugby Meta-Model Finite State Machines Petri Nets Untimed Model of Computation Synchronous Model of Computation Timed Model of Computation Integration of Computational Models
a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
Secure and Efficient Outsourcing of Sequence Comparisons
Secure and Efficient Outsourcing of Sequence Comparisons Marina Blanton 1, Mikhail J. Atallah 2, Keith B. Frikken 3, and Qutaibah Malluhi 4 1 Department of Computer Science and Engineering, University
Solutions to Homework 6
Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example
University of Lille I PC first year list of exercises n 7. Review
University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge,
The Union-Find Problem Kruskal s algorithm for finding an MST presented us with a problem in data-structure design. As we looked at each edge, cheapest first, we had to determine whether its two endpoints
SECTIONS 1.5-1.6 NOTES ON GRAPH THEORY NOTATION AND ITS USE IN THE STUDY OF SPARSE SYMMETRIC MATRICES
SECIONS.5-.6 NOES ON GRPH HEORY NOION ND IS USE IN HE SUDY OF SPRSE SYMMERIC MRICES graph G ( X, E) consists of a finite set of nodes or vertices X and edges E. EXMPLE : road map of part of British Columbia
Regular Expressions and Automata using Haskell
Regular Expressions and Automata using Haskell Simon Thompson Computing Laboratory University of Kent at Canterbury January 2000 Contents 1 Introduction 2 2 Regular Expressions 2 3 Matching regular expressions
I. GROUPS: BASIC DEFINITIONS AND EXAMPLES
I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called
9.2 Summation Notation
9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a
Yousef Saad University of Minnesota Computer Science and Engineering. CRM Montreal - April 30, 2008
A tutorial on: Iterative methods for Sparse Matrix Problems Yousef Saad University of Minnesota Computer Science and Engineering CRM Montreal - April 30, 2008 Outline Part 1 Sparse matrices and sparsity
Lecture 18: Applications of Dynamic Programming Steven Skiena. Department of Computer Science State University of New York Stony Brook, NY 11794 4400
Lecture 18: Applications of Dynamic Programming Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Problem of the Day
Elementary Matrices and The LU Factorization
lementary Matrices and The LU Factorization Definition: ny matrix obtained by performing a single elementary row operation (RO) on the identity (unit) matrix is called an elementary matrix. There are three
Analysis of an Artificial Hormone System (Extended abstract)
c 2013. This is the author s version of the work. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purpose or for creating
Lecture 22: November 10
CS271 Randomness & Computation Fall 2011 Lecture 22: November 10 Lecturer: Alistair Sinclair Based on scribe notes by Rafael Frongillo Disclaimer: These notes have not been subjected to the usual scrutiny
C H A P T E R. Logic Circuits
C H A P T E R Logic Circuits Many important functions are naturally computed with straight-line programs, programs without loops or branches. Such computations are conveniently described with circuits,
COUNTING INDEPENDENT SETS IN SOME CLASSES OF (ALMOST) REGULAR GRAPHS
COUNTING INDEPENDENT SETS IN SOME CLASSES OF (ALMOST) REGULAR GRAPHS Alexander Burstein Department of Mathematics Howard University Washington, DC 259, USA aburstein@howard.edu Sergey Kitaev Mathematics
Discrete Mathematics Problems
Discrete Mathematics Problems William F. Klostermeyer School of Computing University of North Florida Jacksonville, FL 32224 E-mail: wkloster@unf.edu Contents 0 Preface 3 1 Logic 5 1.1 Basics...............................
Direct Methods for Solving Linear Systems. Matrix Factorization
Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011
GRAPH THEORY LECTURE 4: TREES
GRAPH THEORY LECTURE 4: TREES Abstract. 3.1 presents some standard characterizations and properties of trees. 3.2 presents several different types of trees. 3.7 develops a counting method based on a bijection
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
Determinants in the Kronecker product of matrices: The incidence matrix of a complete graph
FPSAC 2009 DMTCS proc (subm), by the authors, 1 10 Determinants in the Kronecker product of matrices: The incidence matrix of a complete graph Christopher R H Hanusa 1 and Thomas Zaslavsky 2 1 Department
v w is orthogonal to both v and w. the three vectors v, w and v w form a right-handed set of vectors.
3. Cross product Definition 3.1. Let v and w be two vectors in R 3. The cross product of v and w, denoted v w, is the vector defined as follows: the length of v w is the area of the parallelogram with
Factoring & Primality
Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount
SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89. by Joseph Collison
SYSTEMS OF EQUATIONS AND MATRICES WITH THE TI-89 by Joseph Collison Copyright 2000 by Joseph Collison All rights reserved Reproduction or translation of any part of this work beyond that permitted by Sections
Arithmetic and Algebra of Matrices
Arithmetic and Algebra of Matrices Math 572: Algebra for Middle School Teachers The University of Montana 1 The Real Numbers 2 Classroom Connection: Systems of Linear Equations 3 Rational Numbers 4 Irrational
On line construction of suffix trees 1
(To appear in ALGORITHMICA) On line construction of suffix trees 1 Esko Ukkonen Department of Computer Science, University of Helsinki, P. O. Box 26 (Teollisuuskatu 23), FIN 00014 University of Helsinki,
GENERATING SETS KEITH CONRAD
GENERATING SETS KEITH CONRAD 1 Introduction In R n, every vector can be written as a unique linear combination of the standard basis e 1,, e n A notion weaker than a basis is a spanning set: a set of vectors
Performance Evaluation of Improved Web Search Algorithms
Performance Evaluation of Improved Web Search Algorithms Esteban Feuerstein 1, Veronica Gil-Costa 2, Michel Mizrahi 1, Mauricio Marin 3 1 Universidad de Buenos Aires, Buenos Aires, Argentina 2 Universidad
Reduced echelon form: Add the following conditions to conditions 1, 2, and 3 above:
Section 1.2: Row Reduction and Echelon Forms Echelon form (or row echelon form): 1. All nonzero rows are above any rows of all zeros. 2. Each leading entry (i.e. left most nonzero entry) of a row is in
AN ALGORITHM FOR DETERMINING WHETHER A GIVEN BINARY MATROID IS GRAPHIC
AN ALGORITHM FOR DETERMINING WHETHER A GIVEN BINARY MATROID IS GRAPHIC W. T. TUTTE. Introduction. In a recent series of papers [l-4] on graphs and matroids I used definitions equivalent to the following.
Parallel Algorithm for Dense Matrix Multiplication
Parallel Algorithm for Dense Matrix Multiplication CSE633 Parallel Algorithms Fall 2012 Ortega, Patricia Outline Problem definition Assumptions Implementation Test Results Future work Conclusions Problem
1.2 Solving a System of Linear Equations
1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables
DATA ANALYSIS II. Matrix Algorithms
DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where
A Note on Maximum Independent Sets in Rectangle Intersection Graphs
A Note on Maximum Independent Sets in Rectangle Intersection Graphs Timothy M. Chan School of Computer Science University of Waterloo Waterloo, Ontario N2L 3G1, Canada tmchan@uwaterloo.ca September 12,
1 Determinants and the Solvability of Linear Systems
1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped
A Turán Type Problem Concerning the Powers of the Degrees of a Graph
A Turán Type Problem Concerning the Powers of the Degrees of a Graph Yair Caro and Raphael Yuster Department of Mathematics University of Haifa-ORANIM, Tivon 36006, Israel. AMS Subject Classification:
Integer Factorization using the Quadratic Sieve
Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give
CS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 7 Binary heap, binomial heap, and Fibonacci heap 1 Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 The slides were
Zachary Monaco Georgia College Olympic Coloring: Go For The Gold
Zachary Monaco Georgia College Olympic Coloring: Go For The Gold Coloring the vertices or edges of a graph leads to a variety of interesting applications in graph theory These applications include various
CS473 - Algorithms I
CS473 - Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm View in slide-show mode 1 Reminder: Merge Sort Input array A sort this half sort this half Divide Conquer merge two sorted halves Combine
RN-coding of Numbers: New Insights and Some Applications
RN-coding of Numbers: New Insights and Some Applications Peter Kornerup Dept. of Mathematics and Computer Science SDU, Odense, Denmark & Jean-Michel Muller LIP/Arénaire (CRNS-ENS Lyon-INRIA-UCBL) Lyon,
MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
Introduction to NP-Completeness Written and copyright c by Jie Wang 1
91.502 Foundations of Comuter Science 1 Introduction to Written and coyright c by Jie Wang 1 We use time-bounded (deterministic and nondeterministic) Turing machines to study comutational comlexity of
Euler Paths and Euler Circuits
Euler Paths and Euler Circuits An Euler path is a path that uses every edge of a graph exactly once. An Euler circuit is a circuit that uses every edge of a graph exactly once. An Euler path starts and
Dot product and vector projections (Sect. 12.3) There are two main ways to introduce the dot product
Dot product and vector projections (Sect. 12.3) Two definitions for the dot product. Geometric definition of dot product. Orthogonal vectors. Dot product and orthogonal projections. Properties of the dot
How To Find An Optimal Search Protocol For An Oblivious Cell
The Conference Call Search Problem in Wireless Networks Leah Epstein 1, and Asaf Levin 2 1 Department of Mathematics, University of Haifa, 31905 Haifa, Israel. lea@math.haifa.ac.il 2 Department of Statistics,
Fast Sequential Summation Algorithms Using Augmented Data Structures
Fast Sequential Summation Algorithms Using Augmented Data Structures Vadim Stadnik vadim.stadnik@gmail.com Abstract This paper provides an introduction to the design of augmented data structures that offer
Finding and counting given length cycles
Finding and counting given length cycles Noga Alon Raphael Yuster Uri Zwick Abstract We present an assortment of methods for finding and counting simple cycles of a given length in directed and undirected
Lecture 1: Systems of Linear Equations
MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables
Universal hashing. In other words, the probability of a collision for two different keys x and y given a hash function randomly chosen from H is 1/m.
Universal hashing No matter how we choose our hash function, it is always possible to devise a set of keys that will hash to the same slot, making the hash scheme perform poorly. To circumvent this, we
An Introduction to APGL
An Introduction to APGL Charanpal Dhanjal February 2012 Abstract Another Python Graph Library (APGL) is a graph library written using pure Python, NumPy and SciPy. Users new to the library can gain an
System Interconnect Architectures. Goals and Analysis. Network Properties and Routing. Terminology - 2. Terminology - 1
System Interconnect Architectures CSCI 8150 Advanced Computer Architecture Hwang, Chapter 2 Program and Network Properties 2.4 System Interconnect Architectures Direct networks for static connections Indirect
Solving Simultaneous Equations and Matrices
Solving Simultaneous Equations and Matrices The following represents a systematic investigation for the steps used to solve two simultaneous linear equations in two unknowns. The motivation for considering
Creating Serial Numbers using Design and Print Online. Creating a Barcode in Design and Print Online. Creating a QR Code in Design and Print Online
Creating Serial Numbers using Design and Print Online Creating a Barcode in Design and Print Online Creating a QR Code in Design and Print Online 1 of 19 Creating a Serial Number 1. On the Design and Print
WRITING PROOFS. Christopher Heil Georgia Institute of Technology
WRITING PROOFS Christopher Heil Georgia Institute of Technology A theorem is just a statement of fact A proof of the theorem is a logical explanation of why the theorem is true Many theorems have this
The Ideal Class Group
Chapter 5 The Ideal Class Group We will use Minkowski theory, which belongs to the general area of geometry of numbers, to gain insight into the ideal class group of a number field. We have already mentioned
NP-Completeness and Cook s Theorem
NP-Completeness and Cook s Theorem Lecture notes for COM3412 Logic and Computation 15th January 2002 1 NP decision problems The decision problem D L for a formal language L Σ is the computational task:
The Determinant: a Means to Calculate Volume
The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are
Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.
Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that
Transportation Polytopes: a Twenty year Update
Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,
5.4 Closest Pair of Points
5.4 Closest Pair of Points Closest Pair of Points Closest pair. Given n points in the plane, find a pair with smallest Euclidean distance between them. Fundamental geometric primitive. Graphics, computer
A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called heap-order property
CmSc 250 Intro to Algorithms Chapter 6. Transform and Conquer Binary Heaps 1. Definition A binary heap is a complete binary tree, where each node has a higher priority than its children. This is called
The Assignment Problem and the Hungarian Method
The Assignment Problem and the Hungarian Method 1 Example 1: You work as a sales manager for a toy manufacturer, and you currently have three salespeople on the road meeting buyers. Your salespeople are
1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D.
1. The memory address of the first element of an array is called A. floor address B. foundation addressc. first address D. base address 2. The memory address of fifth element of an array can be calculated
CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma
CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are
Factoring Algorithms
Factoring Algorithms The p 1 Method and Quadratic Sieve November 17, 2008 () Factoring Algorithms November 17, 2008 1 / 12 Fermat s factoring method Fermat made the observation that if n has two factors