# Lecture 3: Finding integer solutions to systems of linear equations

Size: px
Start display at page:

Transcription

1 Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture is to develop machinery in order to efficiently find integer solutions to a given system of linear equations with integer coefficients. We will begin by looking at how one solves the same problem over finite fields and over the rationals, and what kind of problem one runs into while doing so naively. In particular, when using Gaussian elimination on a given matrix with rational entries, it s important to make sure that the entries don t blow up too much during the intermediate stages, if one is aiming for an efficient algorithm (polynomial time, to be more specific). After the detour, we shall get back to the original problem, introducing the Hermite Normal Form (HNF) for a matrix with integer entries, and also proving its existence by giving a constructive proof. It turns out that the algorithm implicit in the constructive proof is not efficient since the intermediate entries can blow up, and thus we will propose an efficient algorithm to find the HNF of a given integer matrix, which shall be the stepping to stone to the algorithm for finding integer solutions to a system of linear equation. This algorithm (for finding integer solutions) will be described in full detail in the next lecture, along with its analysis. 2 Solving systems of linear equations over finite fields 2.1 The setup Since we haven t yet dealt with the construction of fields whose size is a power of a prime, we shall restrict our attention to fields F p, where p is a prime. Suppose we are given n linear equations over F p in n variables, and want to find solutions to the this system. a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a n1 x 1 + a m2 x a nn x n = b n Suppose that the coefficient matrix for the above system is represented by the n n matrix A. Recall that the standard technique to solve such a system is to use Gaussian elimination to bring A into a row reduced form A (or column reduced form, but we shall stick to the row form), such that the solution space of A is exactly equal to the solution space of A. Once we have this A it s straight-forward to find the solution(s) of the system. 1

2 Remark: When employing Gaussian elimination, we will not worry about making the leading coefficient of every row into a 1. Thus, we shall only use elementary row operations of two kinds: row operations for swapping rows, and row operations for adding a multiple of one row to another. This will ensure that the determinant of the matrix remains invariant as far as the magnitude goes. 2.2 Computational complexity Every entry of the matrix A is O(log p) bits, so the size of the input to this problem is roughly n n log p. Recall that Gaussian elimination involves a sequence of elementary row operations on the matrix. It is not hard to see that the total number of such row operations is roughly O(n 2 ). Moreover, each row operation requires O(n) field operations, thus making the whole process require O(n 3 ) field operations. Another important fact is that, since we are working over F p, the result of all field operations on field elements will result in another field element, which means that at any intermediate step, every entry of the matrix is at most log p bits in size. Let s talk a bit about the complexity of the field operations now. Firstly, addition, subtraction, and multiplication in F p basically boils down to doing standard arithmetic over the integers and going mod p. In the previous lectures we saw how this can be done in time polynomial in log p. But what about division in F p? Note that performing a/b in F p boils down to finding b 1 and computing ab 1. Can we find inverses efficiently? Claim 1. For any a F p, a 1 can be computed in time polynomial in log p. Proof. First, note that over the integers, gcd(a, p) = 1. This means we can always find integers s and t such that as + pt = 1. If we go mod p on both sides, we get as 1 mod p. Thus, we are done if we can find s and t (we just need s and t mod p!) efficiently. Suppose we feed a and p into Euclid s algorithm (refer to Lecture 1), and get the following equations: p = aq 0 + r 0 a = r 0 q 1 + r 1 r 0 = r 1 q 2 + r 2 r k 3 = r k 2 q k 1 + r k 1 r k 2 = r k 1 q k + r k such that r k 1 = 1, r k = 0. Thus, the euclidean algorithm takes k + 1 steps, and gcd(p, a) = gcd(r k 2, r k 1 ) = r k 1 = 1. We know r 0 = p aq 0, r 1 = a r 0 q 1 and r 2 = r 0 r 1 q 2. Thus, using these three equations, we can write r 1 and r 2 as an integer linear combination of a and p. Now it s easy to see that by repeating this forward-substitution process roughly k times, we can express r k 1 as an integer combination of a and p, where the integers coefficients of a and p will be in terms of sums and products over {q j } k j=0. The only concern is that the intermediate or even the final coefficients (i.e. s and t) might be too large (Can this happen?). Since we are only interested in the value of s mod p, we can do all the multiplications and additions involved during the intermediate steps mod p, and only keep the value of the coefficients mod p. Thus, the complete algorithm needs roughly k steps to find gcd(a, p) (which also gives us {q j } k j=0 ) 2

3 and another k steps to do the forward substitutions, where each step requires a constant number arithmetic operations (here, by a constant number we mean a value independent of k), making the total number of arithmetic operations O(k). Now, every arithmetic operation requires O(log p) steps, and, it follows from our discussion in Lecture 1 that k O(log p), making the overall complexity O(log 2 p). We conclude this section by noting that, in the light of the above discussion, solving a system of linear equations over F p needs at most O(n 3 log 2 p) operations, which is polynomial in the input size. 3 Solving systems of linear equations over the rationals Suppose we have the same setup as in Section 2, the only difference being that this time the a ij are in Q and are presented to us as (p ij, q ij ) such that a ij = p ij /q ij and i, j, p ij, q ij are m bit integers. We want to find solution vectors (x 1,, x n ) Q n for this system. Our algorithm in this case will the same as before (i.e. Gaussian elimination), with the additional condition that the result of every arithmetic operation on two rationals will be brought to the reduced form (The reason for doing so will become clear later). 3.1 Computational complexity Note that the size of the input to this problem is O(n 2 m), and following the argument from Section 2, the total number of arithmetic operations (reducing a fraction to its lowest terms will be considered an arithmetic operation) is O(n 3 ). There is, however, a problem that comes up, which did not arise in the case of finite fields: are the intermediate and final entries of the matrix small (i.e. the number of bits is polynomial in n, m)? Assuming that they were small after every step, it is easy to see that the complexity of the algorithm would then be O(poly(n, m)), since every arithmetic operation can be performed in poly(n, m) steps (Here, by poly(n, m) we mean some polynomial in n, m). We now turn to proving that the intermediate entries are indeed small. 3.2 Bounding intermediate values Recall that the Gaussian elimination will have n 1 rounds, where, broadly speaking, the k th round involves, possibly, a row swap, followed by the subtraction of multiples of the k th row from the rows below. Suppose, after k round of Gaussian elimination, we have reduced A to the matrix A k : ( ) Bk C A k = k 0 D k where B k is an k k upper triangular matrix, the 0 denotes a 0-matrix of size (n k) k, C k is a matrix of size k (n k), and D k is then a matrix of size (n k) (n k). Claim 2. k {0, n 1}, every entry of A k can be represented using at most poly(n, m)-bits. 3

4 Proof. We can assume without loss of generality that none of the rounds involve swapping rows 1. Let us write A in a similar manner as A k : ( ) B C A = E D Let us first look at entries of D k. Consider any entry (D k ) i,j. Consider the (k + 1) (k + 1) matrix J ijk : ( ) Bk (C J ijk = k ) j 0 (D k ) i,j where (C k ) j is the j th column of C k, 0 denotes a 0-matrix of dimension 1 k. Let J ij be the matrix: ( ) B Cj J ij = 0 D i,j Since J ijk can been obtained from J ij by just doing row operations that involve subtracting a multiple of one row from another, we have By the same argument, we have det(j ijk ) = det(j ij ) det(b) = det(b k ) Moreover, det(j ijk ) = det(b k )(D k ) ij. Combining these equations, we get: (D k ) ij = det(j ij) det(b) We can assume that the matrix A has all integer entries to begin with because, if not, we can always clear the denominators while blowing up the number of bits required for each entry of the matrix only polynomially (to be precise, this blows up the number of bits from m to m 3, in the worst case). Now, the determinant of any sub-matrix of A can be represented using a polynomial (in n, m) number of bits 2, which then implies that (D k ) ij can be expressed as a rational, such that both its numerator and denominator can be expressed using a polynomial (in n, m) number of bits. Let us now turn our attention to an entry (C k ) ij, of the matrix C k. Note that the round in which this entry gets affected for the last time is the (i 1) th round. Consider the matrix A i 1 as above, representing the entries after the (i 1) th round. Clearly, (C k ) ij is an entry of (D i 1 ) in A i 1, and since we already proved that, for all l {1,, n 1}, every entry of D l can be represented using polynomially many bits, the same holds for (C k ) ij. A similar argument works for the entries of the matrix B k, which proves the complete claim. Up till now, we have only shown that the intermediate values we obtain during Gaussian elimination have a short representation. The fact that we reduce all intermediate fractions to their lowest terms ensures that we actually end up using this short representation after every round. 1 We can assume that all the required row swaps are done before hand. Let A k represent the matrix in which row swaps are done before hand, and A 0 k represent the matrix in which they are done during the rounds, then each of B k, C k and D k are equivalent to their counterparts in A 0 k upto a permutation of the rows, which really does not affect the claim we are proving. 2 The determinant is given by σ S n sign(σ) n i=1 a iσ(i), so there are n! terms, each at most 2 poly(m), and they can sum up to at most n!2 poly(m), which is representable using O(n log n + poly(m)) bits. 4

5 4 Solving systems of linear equations over the integers Suppose we are given m linear equations over Z in n variables, and want to find integer solutions to this system. We are assured that each a ij is at most N bits, making the total input size mnn. a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a m1 x 1 + a m2 x a mn x n = b m Notice that we are no longer working over a field and this basically rules out using Gaussian elimination, although we shall use something that s similar. 4.1 Enter Hermite Normal Form (HNF) Let A be the m n integer matrix representing the above system. Finding solutions to this system can be phrased as finding integer linear combinations of the columns of A that are equal to b, where b is the column vector (b 1,, b m ). In other words, is b in the integer span of the columns of A? 3 To mimic Gaussian elimination, it is important that we understand what kind of row or column operations preserve the integer span of the columns of A. The following are such operations: 1. Subtracting an integer multiple of one column from another i.e. y j y j c y i, where c Z. 2. Multiplying the entries of a column by 1 i.e. y j y j. 3. Swapping two columns, i.e. y i y j. Our goal will be to take an arbitrary integer matrix A, and perform column operations of the above time to bring it into a nice form. Definition 3 (Hermite Normal Form). An m n matrix is said to be in Hermite Normal Form (HNF) if it has the following form: ( B 0 ) such that: B is an m m lower triangular matrix. 0 denotes an m (n m) 0-matrix. i, j < i, 0 B ij < B ii. Proposition 4. Let A be an m n matrix with integer entries and column rank m. Then A can be brought into Hermite Normal Form through a sequence of column operations of the above type. Furthermore, the resulting matrix in Hermite Normal Form is unique. 3 Additionally, we can assume that the columns of A have rank m over Q, otherwise we can reduce the size of the problem and proceed. 5

6 Proof. We shall give a constructive proof to show the existence of the HNF. The procedure we give now is not efficient! This procedure is only to show the existence of an HNF for every A. We address efficiency in the next section. Consider the following algorithms: Algorithm: FindPreHNF Input: An m n integer matrix A with column rank m Output: HNF of A without the guarantee that off-diagonal entries are smaller than the diagonal ones. 1. Since we have a full rank matrix, the first row is non-zero. By multiplying columns with 1, we can ensure that all non-zero entries of the first row are positive. 2. By using column operations of the first kind (See 4.1), we compute the GCD of all the nonzero entries of the first row, such that we are left with only one non-zero entry at the end of the process, which is equal to the GCD itself. 3. We then use a column swap to ensure that this non-zero entry is at position (1, 1) (the other entries in the first row are all zero). At this stage the matrix has the following form: B B 21. B m1 4. If m = 1, the above matrix is just a row ( B ), and we return it. 5. Otherwise, let us denote the lower-left (m 1) (n 1) matrix in the above matrix by A. Notice that A is an integer matrix with column rank m 1. We can now recurse on A : C = FindPreHNF(A ). 6. We then return the following matrix: B B 21 C 11 C 1,n 1 B 21 C 21 C 2,n 1. B m1 C m 1,1 C m 1,n 1 where C i,j denotes the (i, j) entry of the matrix C obtained in the last step. Running the above algorithm on A will produce a matrix of the form ( B 0 ) where B is an m m lower-triangular matrix, though, along a row in B, the off-diagonal entries can be larger than the diagonal ones. To ensure that this does not happen, we use the following algorithm: Algorithm: ReduceOffDiagonal 6

7 Input: An m m integer lower-triangular matrix B with full rank Output: An m m integer lower-triangular matrix B with the condition that i, j < i, 0 B ij < B ii, such that to go from B to B we just need elementary column operations (See 4.1). for j = 1 to m 1 do for i = j + 1 to m do 1. If B i,j < 0, then we add a large enough of multiple of column B i to column B j so that 0 B i,j <. This is achieved by B j B j + B i,j B i. 2. Otherwise if B i,j >, we subtract a large enough multiple of column B i from column B j so that 0 B i,j <. This is achieved by B j B j B i,j B i. end for end for It can easily be verified that the above algorithm is correct, the proof idea being that after the j th iteration of the outer loop, all the off-diagonal entries present in columns 1 to j are less than their respective diagonal entries, and this inequality is not perturbed by the subsequent loop iterations. Combining FindPreHNF along with ReduceOffDiagonal proves the existence of the HNF. To prove the uniqueness, say the full-rank matrix A has two HNFs, H and H. Then look at the smallest i such that there are entries h ij and h ij such that h ij h ij. Without loss of generality, h ij > h ij. Let us look at the difference h j h j, where h j and h j are the jth columns of H and H respectively. Since both H and H have the same integer linear span, h j h j is in the integer span of H and can be written as a linear combination of the columns of H. But given our choice of i and j, h j h j has its first non-zero entry at i (where the entries with a lower index are all zero), which means that h j h j can be written as a linear combination of only columns i to m. Among these, since the i th column is the only one with a non-zero entry at row i, we have that h ij h ij = c h ii, where c Z. Now, h ij < h ij < h ii, which means that h ij h ij < h ii. Thus, c must be zero, which is a contradiction. Thus, there cannot exist an i and j such that h ij h ij. Corollary 5. Let A and A be two full-rank integer matrices with dimensions m n and m n (n > m, n > m) respectively, such that the integer span of A is equal to that of A, then: If n > n, the HNF of A is just the HNF of A appended with n n zero columns. If n < n, the HNF of A is just the HNF of A appended with n n zero columns. else if n = n, the HNF of A is the same as the HNF of A. Proof. The proof is exactly the same as the proof for the uniqueness of HNF given above: we apply the algorithm (i.e. FindPreHNF + ReduceOffDiagonal) to both A and A to get H and H, and since the integer spans of H and H are the same, the uniqueness argument from the previous proof tells us that H and H are identical in their respective lower-triangular parts. The only difference is that one of them might have more zero columns towards the end than the other if n n 0. 7

8 4.2 A polynomial time algorithm for finding the HNF The first question one might ask is, since we gave a constructive proof for the existence of the HNF, why not use the algorithm implicit in the proof? Unfortunately, unlike the previous section, there is no way of showing that the intermediate values encountered during the algorithm cannot be too large (not too large here means polynomial in n, m, N). We will now suggest a way to get around this problem. Let us choose some m independent columns from the matrix A and put them together in an m m matrix Z. Let k = det(z). Claim 6. If I m m denotes the identity matrix, then the columns of the matrix ki m m are in the integer span of the columns of Z (and thus the columns of A). Proof. ki m m can be written as (Z Z 1 ) k, which is the same as Z (Z 1 det(z)). If we can show that Z 1 det(z) has all integer entries, we would be done. Using Cramer s rule, Z 1 det(z) can be written as ((1/det(Z)) Ẑ det(z) = Ẑ, where Ẑ is the matrix of cofactors of Z. Since every entry of the cofactor matrix is an integer, the claim follows. Consider the m 2m matrix Â = ( ) Z ki m m. Using Corollary 5, we observe that the HNF for A can be obtained from the HNF of Â, since both have the same integer span, and thus it s enough to find the HNF for Â. Note that computing Â from A can be done in polynomial time based on the results of Section 3 (In particular, it involves using Gaussian elimination over Q to find m independent column vectors and to compute the determinant). The next step is to find the HNF for Â, and the algorithm for this is exactly the same as before (i.e. FindPreHNF + ReduceOffDiagonal) with a slight modification in the FindPreHNF algorithm. The modification is as follows: In step 2, whenever we use column operations of the first kind i.e. adding/subtracting a multiple of one column from another, and as a result of the computation, some entry of the matrix, say the (i, j) entry, exceeds k, we pick the i th column from the ki m m sub-matrix and subtract a large enough multiple of this column from the j th column of the matrix so that the (i, j) entry becomes less than k (This amounts to subtracting a large enough integer multiple of k from the (i, j) entry without disturbing the other entries of the j th column of the matrix). The next step of our algorithm is to make sure that the off-diagonal entries are less than the diagonal ones and this is done by simply running the ReduceOffDiagonal algorithm on the matrix. 4.3 Analysis of the modified algorithm Since we are not interested in computing the exact complexity of the algorithm, we only need a rough analysis in order to show that the running time is polynomial in n, m and N. Firstly, note that if we can show that all intermediate values during the algorithm have at most polynomially many bits, then in order to prove that the algorithm runs in polynomial time, it suffices to show that: 1. the total number of arithmetic operations is polynomial, and 8

9 2. the total number of non-arithmetic operations (swapping etc.) is also polynomial. Since both 1. and 2. can easily be verified, we will focus our attention on proving that the entries don t blow up too much during the algorithm. It can easily be verified that, since k = det(z) is at most poly(m, N) bits long, all the intermediate values encountered during this modified FindPreHNF algorithm are polynomial in n, m and N, since, after every arithmetic operation, we keep making sure that they do not become larger than k. What about the ReduceOffDiagonal algorithm? Note that after the j th iteration of the outer loop, the entries in columns j are never altered. In fact the only time the entries of the j th column are altered is in the j th iteration of the outer loop. Thus it suffices to prove the following claim: Proposition 7. Let 1 j < m (Assume that the input parameter m of the algorithm is equal to m), then, after the j th iteration of the outer loop, every entry of the j th column is at most mn bits long. The proposition follows from the following claim: Claim 8. Consider the j th iteration of the outer loop. Let j + 1 i m, then all the entries in column j are at most (i j + 1)N bits long after the (i j) th iteration of the inner loop. Proof. Let us look at the base case when i = j + 1 i.e. the 1 st iteration. Recall that, as discussed before, the j th column is unaltered up till this point and thus every entry is at most N bits. Let us assume without loss of generality that B ij > B ii. This will mean that we do the operation B j B j B i,j B i. By how much can this operation blow up the size of the entries of column j? In the worst case, an entry of B j could be zero, B i,j could be as large as N bits (for example, if is 1), and the other entries (barring the j th entry) of B i could be N bits, making an entry of column j as large as 2N. Notice that this proves the base case. Let us assume that the assertion holds up till i 1 i.e. after the (i 1 j) th iteration. So all the entries in column j are at most (i j)n bits. We will show that the assertion holds even after the (i j) th iteration, thus proving the claim. Again, in the worst case, without loss of generality, we perform the operation B j B j B i,j B i. Now, this time, B i,j can be as large as (i j)n bits (by a similar argument as the base case), and the entries of B i (except ) could be as large as N bits. In this situation, the entries of B j can become at most (i j)n + N = (i j + 1)N bits. 9

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### CS 103X: Discrete Structures Homework Assignment 3 Solutions

CS 103X: Discrete Structures Homework Assignment 3 s Exercise 1 (20 points). On well-ordering and induction: (a) Prove the induction principle from the well-ordering principle. (b) Prove the well-ordering

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### MATH 289 PROBLEM SET 4: NUMBER THEORY

MATH 289 PROBLEM SET 4: NUMBER THEORY 1. The greatest common divisor If d and n are integers, then we say that d divides n if and only if there exists an integer q such that n = qd. Notice that if d divides

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### 4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION

4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### 8 Square matrices continued: Determinants

8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### Notes on Factoring. MA 206 Kurt Bryan

The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor

### 160 CHAPTER 4. VECTOR SPACES

160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### Today s Topics. Primes & Greatest Common Divisors

Today s Topics Primes & Greatest Common Divisors Prime representations Important theorems about primality Greatest Common Divisors Least Common Multiples Euclid s algorithm Once and for all, what are prime

### Some Polynomial Theorems. John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.

Some Polynomial Theorems by John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.com This paper contains a collection of 31 theorems, lemmas,

### THE DIMENSION OF A VECTOR SPACE

THE DIMENSION OF A VECTOR SPACE KEITH CONRAD This handout is a supplementary discussion leading up to the definition of dimension and some of its basic properties. Let V be a vector space over a field

### 8 Divisibility and prime numbers

8 Divisibility and prime numbers 8.1 Divisibility In this short section we extend the concept of a multiple from the natural numbers to the integers. We also summarize several other terms that express

### Quotient Rings and Field Extensions

Chapter 5 Quotient Rings and Field Extensions In this chapter we describe a method for producing field extension of a given field. If F is a field, then a field extension is a field K that contains F.

### The Prime Numbers. Definition. A prime number is a positive integer with exactly two positive divisors.

The Prime Numbers Before starting our study of primes, we record the following important lemma. Recall that integers a, b are said to be relatively prime if gcd(a, b) = 1. Lemma (Euclid s Lemma). If gcd(a,

### 11 Ideals. 11.1 Revisiting Z

11 Ideals The presentation here is somewhat different than the text. In particular, the sections do not match up. We have seen issues with the failure of unique factorization already, e.g., Z[ 5] = O Q(

GENERATING SETS KEITH CONRAD 1 Introduction In R n, every vector can be written as a unique linear combination of the standard basis e 1,, e n A notion weaker than a basis is a spanning set: a set of vectors

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### Unit 18 Determinants

Unit 18 Determinants Every square matrix has a number associated with it, called its determinant. In this section, we determine how to calculate this number, and also look at some of the properties of

### Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### Modélisation et résolutions numérique et symbolique

Modélisation et résolutions numérique et symbolique via les logiciels Maple et Matlab Jeremy Berthomieu Mohab Safey El Din Stef Graillat Mohab.Safey@lip6.fr Outline Previous course: partial review of what

### 7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix

7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse

### The Determinant: a Means to Calculate Volume

The Determinant: a Means to Calculate Volume Bo Peng August 20, 2007 Abstract This paper gives a definition of the determinant and lists many of its well-known properties Volumes of parallelepipeds are

### Chapter 17. Orthogonal Matrices and Symmetries of Space

Chapter 17. Orthogonal Matrices and Symmetries of Space Take a random matrix, say 1 3 A = 4 5 6, 7 8 9 and compare the lengths of e 1 and Ae 1. The vector e 1 has length 1, while Ae 1 = (1, 4, 7) has length

### GREATEST COMMON DIVISOR

DEFINITION: GREATEST COMMON DIVISOR The greatest common divisor (gcd) of a and b, denoted by (a, b), is the largest common divisor of integers a and b. THEOREM: If a and b are nonzero integers, then their

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### 3. Mathematical Induction

3. MATHEMATICAL INDUCTION 83 3. Mathematical Induction 3.1. First Principle of Mathematical Induction. Let P (n) be a predicate with domain of discourse (over) the natural numbers N = {0, 1,,...}. If (1)

### 1 Determinants and the Solvability of Linear Systems

1 Determinants and the Solvability of Linear Systems In the last section we learned how to use Gaussian elimination to solve linear systems of n equations in n unknowns The section completely side-stepped

### Kevin James. MTHSC 412 Section 2.4 Prime Factors and Greatest Comm

MTHSC 412 Section 2.4 Prime Factors and Greatest Common Divisor Greatest Common Divisor Definition Suppose that a, b Z. Then we say that d Z is a greatest common divisor (gcd) of a and b if the following

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### Chapter 3. if 2 a i then location: = i. Page 40

Chapter 3 1. Describe an algorithm that takes a list of n integers a 1,a 2,,a n and finds the number of integers each greater than five in the list. Ans: procedure greaterthanfive(a 1,,a n : integers)

### LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008 This chapter is available free to all individuals, on understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### 1 Sets and Set Notation.

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

### 5.1 Radical Notation and Rational Exponents

Section 5.1 Radical Notation and Rational Exponents 1 5.1 Radical Notation and Rational Exponents We now review how exponents can be used to describe not only powers (such as 5 2 and 2 3 ), but also roots

### T ( a i x i ) = a i T (x i ).

Chapter 2 Defn 1. (p. 65) Let V and W be vector spaces (over F ). We call a function T : V W a linear transformation form V to W if, for all x, y V and c F, we have (a) T (x + y) = T (x) + T (y) and (b)

### 1. The Fly In The Ointment

Arithmetic Revisited Lesson 5: Decimal Fractions or Place Value Extended Part 5: Dividing Decimal Fractions, Part 2. The Fly In The Ointment The meaning of, say, ƒ 2 doesn't depend on whether we represent

### Math 115A HW4 Solutions University of California, Los Angeles. 5 2i 6 + 4i. (5 2i)7i (6 + 4i)( 3 + i) = 35i + 14 ( 22 6i) = 36 + 41i.

Math 5A HW4 Solutions September 5, 202 University of California, Los Angeles Problem 4..3b Calculate the determinant, 5 2i 6 + 4i 3 + i 7i Solution: The textbook s instructions give us, (5 2i)7i (6 + 4i)(

### Math 319 Problem Set #3 Solution 21 February 2002

Math 319 Problem Set #3 Solution 21 February 2002 1. ( 2.1, problem 15) Find integers a 1, a 2, a 3, a 4, a 5 such that every integer x satisfies at least one of the congruences x a 1 (mod 2), x a 2 (mod

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

### by the matrix A results in a vector which is a reflection of the given

Eigenvalues & Eigenvectors Example Suppose Then So, geometrically, multiplying a vector in by the matrix A results in a vector which is a reflection of the given vector about the y-axis We observe that

### MATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform

MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish

### Introduction to Matrix Algebra

Psychology 7291: Multivariate Statistics (Carey) 8/27/98 Matrix Algebra - 1 Introduction to Matrix Algebra Definitions: A matrix is a collection of numbers ordered by rows and columns. It is customary

### Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix multiplication).

MAT 2 (Badger, Spring 202) LU Factorization Selected Notes September 2, 202 Abstract: We describe the beautiful LU factorization of a square matrix (or how to write Gaussian elimination in terms of matrix

### CHAPTER 5 Round-off errors

CHAPTER 5 Round-off errors In the two previous chapters we have seen how numbers can be represented in the binary numeral system and how this is the basis for representing numbers in computers. Since any

### Linear Codes. Chapter 3. 3.1 Basics

Chapter 3 Linear Codes In order to define codes that we can encode and decode efficiently, we add more structure to the codespace. We shall be mainly interested in linear codes. A linear code of length

### Lecture 4: AC 0 lower bounds and pseudorandomness

Lecture 4: AC 0 lower bounds and pseudorandomness Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: Jason Perry and Brian Garnett In this lecture,

### Lecture 1: Course overview, circuits, and formulas

Lecture 1: Course overview, circuits, and formulas Topics in Complexity Theory and Pseudorandomness (Spring 2013) Rutgers University Swastik Kopparty Scribes: John Kim, Ben Lund 1 Course Information Swastik

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### Math Workshop October 2010 Fractions and Repeating Decimals

Math Workshop October 2010 Fractions and Repeating Decimals This evening we will investigate the patterns that arise when converting fractions to decimals. As an example of what we will be looking at,

### MATH10040 Chapter 2: Prime and relatively prime numbers

MATH10040 Chapter 2: Prime and relatively prime numbers Recall the basic definition: 1. Prime numbers Definition 1.1. Recall that a positive integer is said to be prime if it has precisely two positive

### Solving Systems of Linear Equations Using Matrices

Solving Systems of Linear Equations Using Matrices What is a Matrix? A matrix is a compact grid or array of numbers. It can be created from a system of equations and used to solve the system of equations.

### COMP 250 Fall 2012 lecture 2 binary representations Sept. 11, 2012

Binary numbers The reason humans represent numbers using decimal (the ten digits from 0,1,... 9) is that we have ten fingers. There is no other reason than that. There is nothing special otherwise about

### Linear Algebra Notes for Marsden and Tromba Vector Calculus

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

### SOLVING LINEAR SYSTEMS

SOLVING LINEAR SYSTEMS Linear systems Ax = b occur widely in applied mathematics They occur as direct formulations of real world problems; but more often, they occur as a part of the numerical analysis

### Homework until Test #2

MATH31: Number Theory Homework until Test # Philipp BRAUN Section 3.1 page 43, 1. It has been conjectured that there are infinitely many primes of the form n. Exhibit five such primes. Solution. Five such

### it is easy to see that α = a

21. Polynomial rings Let us now turn out attention to determining the prime elements of a polynomial ring, where the coefficient ring is a field. We already know that such a polynomial ring is a UF. Therefore

### Analysis of Binary Search algorithm and Selection Sort algorithm

Analysis of Binary Search algorithm and Selection Sort algorithm In this section we shall take up two representative problems in computer science, work out the algorithms based on the best strategy to

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### minimal polyonomial Example

Minimal Polynomials Definition Let α be an element in GF(p e ). We call the monic polynomial of smallest degree which has coefficients in GF(p) and α as a root, the minimal polyonomial of α. Example: We

### Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs

CSE599s: Extremal Combinatorics November 21, 2011 Lecture 15 An Arithmetic Circuit Lowerbound and Flows in Graphs Lecturer: Anup Rao 1 An Arithmetic Circuit Lower Bound An arithmetic circuit is just like

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### IRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL. 1. Introduction

IRREDUCIBLE OPERATOR SEMIGROUPS SUCH THAT AB AND BA ARE PROPORTIONAL R. DRNOVŠEK, T. KOŠIR Dedicated to Prof. Heydar Radjavi on the occasion of his seventieth birthday. Abstract. Let S be an irreducible

### Elementary Number Theory and Methods of Proof. CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.

Elementary Number Theory and Methods of Proof CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.edu/~cse215 1 Number theory Properties: 2 Properties of integers (whole

### The Chinese Remainder Theorem

The Chinese Remainder Theorem Evan Chen evanchen@mit.edu February 3, 2015 The Chinese Remainder Theorem is a theorem only in that it is useful and requires proof. When you ask a capable 15-year-old why

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### Elementary Matrices and The LU Factorization

lementary Matrices and The LU Factorization Definition: ny matrix obtained by performing a single elementary row operation (RO) on the identity (unit) matrix is called an elementary matrix. There are three

### Lecture 13: Factoring Integers

CS 880: Quantum Information Processing 0/4/0 Lecture 3: Factoring Integers Instructor: Dieter van Melkebeek Scribe: Mark Wellons In this lecture, we review order finding and use this to develop a method

### The Mixed Binary Euclid Algorithm

Electronic Notes in Discrete Mathematics 35 (009) 169 176 www.elsevier.com/locate/endm The Mixed Binary Euclid Algorithm Sidi Mohamed Sedjelmaci LIPN CNRS UMR 7030 Université Paris-Nord Av. J.-B. Clément,

### HOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!

Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following

### 1 Homework 1. [p 0 q i+j +... + p i 1 q j+1 ] + [p i q j ] + [p i+1 q j 1 +... + p i+j q 0 ]

1 Homework 1 (1) Prove the ideal (3,x) is a maximal ideal in Z[x]. SOLUTION: Suppose we expand this ideal by including another generator polynomial, P / (3, x). Write P = n + x Q with n an integer not

### CHAPTER 5. Number Theory. 1. Integers and Division. Discussion

CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a