# Lecture 3: Finding integer solutions to systems of linear equations

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture is to develop machinery in order to efficiently find integer solutions to a given system of linear equations with integer coefficients. We will begin by looking at how one solves the same problem over finite fields and over the rationals, and what kind of problem one runs into while doing so naively. In particular, when using Gaussian elimination on a given matrix with rational entries, it s important to make sure that the entries don t blow up too much during the intermediate stages, if one is aiming for an efficient algorithm (polynomial time, to be more specific). After the detour, we shall get back to the original problem, introducing the Hermite Normal Form (HNF) for a matrix with integer entries, and also proving its existence by giving a constructive proof. It turns out that the algorithm implicit in the constructive proof is not efficient since the intermediate entries can blow up, and thus we will propose an efficient algorithm to find the HNF of a given integer matrix, which shall be the stepping to stone to the algorithm for finding integer solutions to a system of linear equation. This algorithm (for finding integer solutions) will be described in full detail in the next lecture, along with its analysis. 2 Solving systems of linear equations over finite fields 2.1 The setup Since we haven t yet dealt with the construction of fields whose size is a power of a prime, we shall restrict our attention to fields F p, where p is a prime. Suppose we are given n linear equations over F p in n variables, and want to find solutions to the this system. a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a n1 x 1 + a m2 x a nn x n = b n Suppose that the coefficient matrix for the above system is represented by the n n matrix A. Recall that the standard technique to solve such a system is to use Gaussian elimination to bring A into a row reduced form A (or column reduced form, but we shall stick to the row form), such that the solution space of A is exactly equal to the solution space of A. Once we have this A it s straight-forward to find the solution(s) of the system. 1

2 Remark: When employing Gaussian elimination, we will not worry about making the leading coefficient of every row into a 1. Thus, we shall only use elementary row operations of two kinds: row operations for swapping rows, and row operations for adding a multiple of one row to another. This will ensure that the determinant of the matrix remains invariant as far as the magnitude goes. 2.2 Computational complexity Every entry of the matrix A is O(log p) bits, so the size of the input to this problem is roughly n n log p. Recall that Gaussian elimination involves a sequence of elementary row operations on the matrix. It is not hard to see that the total number of such row operations is roughly O(n 2 ). Moreover, each row operation requires O(n) field operations, thus making the whole process require O(n 3 ) field operations. Another important fact is that, since we are working over F p, the result of all field operations on field elements will result in another field element, which means that at any intermediate step, every entry of the matrix is at most log p bits in size. Let s talk a bit about the complexity of the field operations now. Firstly, addition, subtraction, and multiplication in F p basically boils down to doing standard arithmetic over the integers and going mod p. In the previous lectures we saw how this can be done in time polynomial in log p. But what about division in F p? Note that performing a/b in F p boils down to finding b 1 and computing ab 1. Can we find inverses efficiently? Claim 1. For any a F p, a 1 can be computed in time polynomial in log p. Proof. First, note that over the integers, gcd(a, p) = 1. This means we can always find integers s and t such that as + pt = 1. If we go mod p on both sides, we get as 1 mod p. Thus, we are done if we can find s and t (we just need s and t mod p!) efficiently. Suppose we feed a and p into Euclid s algorithm (refer to Lecture 1), and get the following equations: p = aq 0 + r 0 a = r 0 q 1 + r 1 r 0 = r 1 q 2 + r 2 r k 3 = r k 2 q k 1 + r k 1 r k 2 = r k 1 q k + r k such that r k 1 = 1, r k = 0. Thus, the euclidean algorithm takes k + 1 steps, and gcd(p, a) = gcd(r k 2, r k 1 ) = r k 1 = 1. We know r 0 = p aq 0, r 1 = a r 0 q 1 and r 2 = r 0 r 1 q 2. Thus, using these three equations, we can write r 1 and r 2 as an integer linear combination of a and p. Now it s easy to see that by repeating this forward-substitution process roughly k times, we can express r k 1 as an integer combination of a and p, where the integers coefficients of a and p will be in terms of sums and products over {q j } k j=0. The only concern is that the intermediate or even the final coefficients (i.e. s and t) might be too large (Can this happen?). Since we are only interested in the value of s mod p, we can do all the multiplications and additions involved during the intermediate steps mod p, and only keep the value of the coefficients mod p. Thus, the complete algorithm needs roughly k steps to find gcd(a, p) (which also gives us {q j } k j=0 ) 2

3 and another k steps to do the forward substitutions, where each step requires a constant number arithmetic operations (here, by a constant number we mean a value independent of k), making the total number of arithmetic operations O(k). Now, every arithmetic operation requires O(log p) steps, and, it follows from our discussion in Lecture 1 that k O(log p), making the overall complexity O(log 2 p). We conclude this section by noting that, in the light of the above discussion, solving a system of linear equations over F p needs at most O(n 3 log 2 p) operations, which is polynomial in the input size. 3 Solving systems of linear equations over the rationals Suppose we have the same setup as in Section 2, the only difference being that this time the a ij are in Q and are presented to us as (p ij, q ij ) such that a ij = p ij /q ij and i, j, p ij, q ij are m bit integers. We want to find solution vectors (x 1,, x n ) Q n for this system. Our algorithm in this case will the same as before (i.e. Gaussian elimination), with the additional condition that the result of every arithmetic operation on two rationals will be brought to the reduced form (The reason for doing so will become clear later). 3.1 Computational complexity Note that the size of the input to this problem is O(n 2 m), and following the argument from Section 2, the total number of arithmetic operations (reducing a fraction to its lowest terms will be considered an arithmetic operation) is O(n 3 ). There is, however, a problem that comes up, which did not arise in the case of finite fields: are the intermediate and final entries of the matrix small (i.e. the number of bits is polynomial in n, m)? Assuming that they were small after every step, it is easy to see that the complexity of the algorithm would then be O(poly(n, m)), since every arithmetic operation can be performed in poly(n, m) steps (Here, by poly(n, m) we mean some polynomial in n, m). We now turn to proving that the intermediate entries are indeed small. 3.2 Bounding intermediate values Recall that the Gaussian elimination will have n 1 rounds, where, broadly speaking, the k th round involves, possibly, a row swap, followed by the subtraction of multiples of the k th row from the rows below. Suppose, after k round of Gaussian elimination, we have reduced A to the matrix A k : ( ) Bk C A k = k 0 D k where B k is an k k upper triangular matrix, the 0 denotes a 0-matrix of size (n k) k, C k is a matrix of size k (n k), and D k is then a matrix of size (n k) (n k). Claim 2. k {0, n 1}, every entry of A k can be represented using at most poly(n, m)-bits. 3

4 Proof. We can assume without loss of generality that none of the rounds involve swapping rows 1. Let us write A in a similar manner as A k : ( ) B C A = E D Let us first look at entries of D k. Consider any entry (D k ) i,j. Consider the (k + 1) (k + 1) matrix J ijk : ( ) Bk (C J ijk = k ) j 0 (D k ) i,j where (C k ) j is the j th column of C k, 0 denotes a 0-matrix of dimension 1 k. Let J ij be the matrix: ( ) B Cj J ij = 0 D i,j Since J ijk can been obtained from J ij by just doing row operations that involve subtracting a multiple of one row from another, we have By the same argument, we have det(j ijk ) = det(j ij ) det(b) = det(b k ) Moreover, det(j ijk ) = det(b k )(D k ) ij. Combining these equations, we get: (D k ) ij = det(j ij) det(b) We can assume that the matrix A has all integer entries to begin with because, if not, we can always clear the denominators while blowing up the number of bits required for each entry of the matrix only polynomially (to be precise, this blows up the number of bits from m to m 3, in the worst case). Now, the determinant of any sub-matrix of A can be represented using a polynomial (in n, m) number of bits 2, which then implies that (D k ) ij can be expressed as a rational, such that both its numerator and denominator can be expressed using a polynomial (in n, m) number of bits. Let us now turn our attention to an entry (C k ) ij, of the matrix C k. Note that the round in which this entry gets affected for the last time is the (i 1) th round. Consider the matrix A i 1 as above, representing the entries after the (i 1) th round. Clearly, (C k ) ij is an entry of (D i 1 ) in A i 1, and since we already proved that, for all l {1,, n 1}, every entry of D l can be represented using polynomially many bits, the same holds for (C k ) ij. A similar argument works for the entries of the matrix B k, which proves the complete claim. Up till now, we have only shown that the intermediate values we obtain during Gaussian elimination have a short representation. The fact that we reduce all intermediate fractions to their lowest terms ensures that we actually end up using this short representation after every round. 1 We can assume that all the required row swaps are done before hand. Let A k represent the matrix in which row swaps are done before hand, and A 0 k represent the matrix in which they are done during the rounds, then each of B k, C k and D k are equivalent to their counterparts in A 0 k upto a permutation of the rows, which really does not affect the claim we are proving. 2 The determinant is given by σ S n sign(σ) n i=1 a iσ(i), so there are n! terms, each at most 2 poly(m), and they can sum up to at most n!2 poly(m), which is representable using O(n log n + poly(m)) bits. 4

5 4 Solving systems of linear equations over the integers Suppose we are given m linear equations over Z in n variables, and want to find integer solutions to this system. We are assured that each a ij is at most N bits, making the total input size mnn. a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a m1 x 1 + a m2 x a mn x n = b m Notice that we are no longer working over a field and this basically rules out using Gaussian elimination, although we shall use something that s similar. 4.1 Enter Hermite Normal Form (HNF) Let A be the m n integer matrix representing the above system. Finding solutions to this system can be phrased as finding integer linear combinations of the columns of A that are equal to b, where b is the column vector (b 1,, b m ). In other words, is b in the integer span of the columns of A? 3 To mimic Gaussian elimination, it is important that we understand what kind of row or column operations preserve the integer span of the columns of A. The following are such operations: 1. Subtracting an integer multiple of one column from another i.e. y j y j c y i, where c Z. 2. Multiplying the entries of a column by 1 i.e. y j y j. 3. Swapping two columns, i.e. y i y j. Our goal will be to take an arbitrary integer matrix A, and perform column operations of the above time to bring it into a nice form. Definition 3 (Hermite Normal Form). An m n matrix is said to be in Hermite Normal Form (HNF) if it has the following form: ( B 0 ) such that: B is an m m lower triangular matrix. 0 denotes an m (n m) 0-matrix. i, j < i, 0 B ij < B ii. Proposition 4. Let A be an m n matrix with integer entries and column rank m. Then A can be brought into Hermite Normal Form through a sequence of column operations of the above type. Furthermore, the resulting matrix in Hermite Normal Form is unique. 3 Additionally, we can assume that the columns of A have rank m over Q, otherwise we can reduce the size of the problem and proceed. 5

6 Proof. We shall give a constructive proof to show the existence of the HNF. The procedure we give now is not efficient! This procedure is only to show the existence of an HNF for every A. We address efficiency in the next section. Consider the following algorithms: Algorithm: FindPreHNF Input: An m n integer matrix A with column rank m Output: HNF of A without the guarantee that off-diagonal entries are smaller than the diagonal ones. 1. Since we have a full rank matrix, the first row is non-zero. By multiplying columns with 1, we can ensure that all non-zero entries of the first row are positive. 2. By using column operations of the first kind (See 4.1), we compute the GCD of all the nonzero entries of the first row, such that we are left with only one non-zero entry at the end of the process, which is equal to the GCD itself. 3. We then use a column swap to ensure that this non-zero entry is at position (1, 1) (the other entries in the first row are all zero). At this stage the matrix has the following form: B B 21. B m1 4. If m = 1, the above matrix is just a row ( B ), and we return it. 5. Otherwise, let us denote the lower-left (m 1) (n 1) matrix in the above matrix by A. Notice that A is an integer matrix with column rank m 1. We can now recurse on A : C = FindPreHNF(A ). 6. We then return the following matrix: B B 21 C 11 C 1,n 1 B 21 C 21 C 2,n 1. B m1 C m 1,1 C m 1,n 1 where C i,j denotes the (i, j) entry of the matrix C obtained in the last step. Running the above algorithm on A will produce a matrix of the form ( B 0 ) where B is an m m lower-triangular matrix, though, along a row in B, the off-diagonal entries can be larger than the diagonal ones. To ensure that this does not happen, we use the following algorithm: Algorithm: ReduceOffDiagonal 6

7 Input: An m m integer lower-triangular matrix B with full rank Output: An m m integer lower-triangular matrix B with the condition that i, j < i, 0 B ij < B ii, such that to go from B to B we just need elementary column operations (See 4.1). for j = 1 to m 1 do for i = j + 1 to m do 1. If B i,j < 0, then we add a large enough of multiple of column B i to column B j so that 0 B i,j <. This is achieved by B j B j + B i,j B i. 2. Otherwise if B i,j >, we subtract a large enough multiple of column B i from column B j so that 0 B i,j <. This is achieved by B j B j B i,j B i. end for end for It can easily be verified that the above algorithm is correct, the proof idea being that after the j th iteration of the outer loop, all the off-diagonal entries present in columns 1 to j are less than their respective diagonal entries, and this inequality is not perturbed by the subsequent loop iterations. Combining FindPreHNF along with ReduceOffDiagonal proves the existence of the HNF. To prove the uniqueness, say the full-rank matrix A has two HNFs, H and H. Then look at the smallest i such that there are entries h ij and h ij such that h ij h ij. Without loss of generality, h ij > h ij. Let us look at the difference h j h j, where h j and h j are the jth columns of H and H respectively. Since both H and H have the same integer linear span, h j h j is in the integer span of H and can be written as a linear combination of the columns of H. But given our choice of i and j, h j h j has its first non-zero entry at i (where the entries with a lower index are all zero), which means that h j h j can be written as a linear combination of only columns i to m. Among these, since the i th column is the only one with a non-zero entry at row i, we have that h ij h ij = c h ii, where c Z. Now, h ij < h ij < h ii, which means that h ij h ij < h ii. Thus, c must be zero, which is a contradiction. Thus, there cannot exist an i and j such that h ij h ij. Corollary 5. Let A and A be two full-rank integer matrices with dimensions m n and m n (n > m, n > m) respectively, such that the integer span of A is equal to that of A, then: If n > n, the HNF of A is just the HNF of A appended with n n zero columns. If n < n, the HNF of A is just the HNF of A appended with n n zero columns. else if n = n, the HNF of A is the same as the HNF of A. Proof. The proof is exactly the same as the proof for the uniqueness of HNF given above: we apply the algorithm (i.e. FindPreHNF + ReduceOffDiagonal) to both A and A to get H and H, and since the integer spans of H and H are the same, the uniqueness argument from the previous proof tells us that H and H are identical in their respective lower-triangular parts. The only difference is that one of them might have more zero columns towards the end than the other if n n 0. 7

8 4.2 A polynomial time algorithm for finding the HNF The first question one might ask is, since we gave a constructive proof for the existence of the HNF, why not use the algorithm implicit in the proof? Unfortunately, unlike the previous section, there is no way of showing that the intermediate values encountered during the algorithm cannot be too large (not too large here means polynomial in n, m, N). We will now suggest a way to get around this problem. Let us choose some m independent columns from the matrix A and put them together in an m m matrix Z. Let k = det(z). Claim 6. If I m m denotes the identity matrix, then the columns of the matrix ki m m are in the integer span of the columns of Z (and thus the columns of A). Proof. ki m m can be written as (Z Z 1 ) k, which is the same as Z (Z 1 det(z)). If we can show that Z 1 det(z) has all integer entries, we would be done. Using Cramer s rule, Z 1 det(z) can be written as ((1/det(Z)) Ẑ det(z) = Ẑ, where Ẑ is the matrix of cofactors of Z. Since every entry of the cofactor matrix is an integer, the claim follows. Consider the m 2m matrix Â = ( ) Z ki m m. Using Corollary 5, we observe that the HNF for A can be obtained from the HNF of Â, since both have the same integer span, and thus it s enough to find the HNF for Â. Note that computing Â from A can be done in polynomial time based on the results of Section 3 (In particular, it involves using Gaussian elimination over Q to find m independent column vectors and to compute the determinant). The next step is to find the HNF for Â, and the algorithm for this is exactly the same as before (i.e. FindPreHNF + ReduceOffDiagonal) with a slight modification in the FindPreHNF algorithm. The modification is as follows: In step 2, whenever we use column operations of the first kind i.e. adding/subtracting a multiple of one column from another, and as a result of the computation, some entry of the matrix, say the (i, j) entry, exceeds k, we pick the i th column from the ki m m sub-matrix and subtract a large enough multiple of this column from the j th column of the matrix so that the (i, j) entry becomes less than k (This amounts to subtracting a large enough integer multiple of k from the (i, j) entry without disturbing the other entries of the j th column of the matrix). The next step of our algorithm is to make sure that the off-diagonal entries are less than the diagonal ones and this is done by simply running the ReduceOffDiagonal algorithm on the matrix. 4.3 Analysis of the modified algorithm Since we are not interested in computing the exact complexity of the algorithm, we only need a rough analysis in order to show that the running time is polynomial in n, m and N. Firstly, note that if we can show that all intermediate values during the algorithm have at most polynomially many bits, then in order to prove that the algorithm runs in polynomial time, it suffices to show that: 1. the total number of arithmetic operations is polynomial, and 8

9 2. the total number of non-arithmetic operations (swapping etc.) is also polynomial. Since both 1. and 2. can easily be verified, we will focus our attention on proving that the entries don t blow up too much during the algorithm. It can easily be verified that, since k = det(z) is at most poly(m, N) bits long, all the intermediate values encountered during this modified FindPreHNF algorithm are polynomial in n, m and N, since, after every arithmetic operation, we keep making sure that they do not become larger than k. What about the ReduceOffDiagonal algorithm? Note that after the j th iteration of the outer loop, the entries in columns j are never altered. In fact the only time the entries of the j th column are altered is in the j th iteration of the outer loop. Thus it suffices to prove the following claim: Proposition 7. Let 1 j < m (Assume that the input parameter m of the algorithm is equal to m), then, after the j th iteration of the outer loop, every entry of the j th column is at most mn bits long. The proposition follows from the following claim: Claim 8. Consider the j th iteration of the outer loop. Let j + 1 i m, then all the entries in column j are at most (i j + 1)N bits long after the (i j) th iteration of the inner loop. Proof. Let us look at the base case when i = j + 1 i.e. the 1 st iteration. Recall that, as discussed before, the j th column is unaltered up till this point and thus every entry is at most N bits. Let us assume without loss of generality that B ij > B ii. This will mean that we do the operation B j B j B i,j B i. By how much can this operation blow up the size of the entries of column j? In the worst case, an entry of B j could be zero, B i,j could be as large as N bits (for example, if is 1), and the other entries (barring the j th entry) of B i could be N bits, making an entry of column j as large as 2N. Notice that this proves the base case. Let us assume that the assertion holds up till i 1 i.e. after the (i 1 j) th iteration. So all the entries in column j are at most (i j)n bits. We will show that the assertion holds even after the (i j) th iteration, thus proving the claim. Again, in the worst case, without loss of generality, we perform the operation B j B j B i,j B i. Now, this time, B i,j can be as large as (i j)n bits (by a similar argument as the base case), and the entries of B i (except ) could be as large as N bits. In this situation, the entries of B j can become at most (i j)n + N = (i j + 1)N bits. 9

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### Direct Methods for Solving Linear Systems. Matrix Factorization

Direct Methods for Solving Linear Systems Matrix Factorization Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011

### 1 Determinants. Definition 1

Determinants The determinant of a square matrix is a value in R assigned to the matrix, it characterizes matrices which are invertible (det 0) and is related to the volume of a parallelpiped described

### DETERMINANTS. b 2. x 2

DETERMINANTS 1 Systems of two equations in two unknowns A system of two equations in two unknowns has the form a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 This can be written more concisely in

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### Matrix Inverse and Determinants

DM554 Linear and Integer Programming Lecture 5 and Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1 2 3 4 and Cramer s rule 2 Outline 1 2 3 4 and

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Determinants. Dr. Doreen De Leon Math 152, Fall 2015

Determinants Dr. Doreen De Leon Math 52, Fall 205 Determinant of a Matrix Elementary Matrices We will first discuss matrices that can be used to produce an elementary row operation on a given matrix A.

### 1 Die hard, once and for all

ENGG 2440A: Discrete Mathematics for Engineers Lecture 4 The Chinese University of Hong Kong, Fall 2014 6 and 7 October 2014 Number theory is the branch of mathematics that studies properties of the integers.

### MATH 2030: SYSTEMS OF LINEAR EQUATIONS. ax + by + cz = d. )z = e. while these equations are not linear: xy z = 2, x x = 0,

MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

### 7 Gaussian Elimination and LU Factorization

7 Gaussian Elimination and LU Factorization In this final section on matrix factorization methods for solving Ax = b we want to take a closer look at Gaussian elimination (probably the best known method

### CS 103X: Discrete Structures Homework Assignment 3 Solutions

CS 103X: Discrete Structures Homework Assignment 3 s Exercise 1 (20 points). On well-ordering and induction: (a) Prove the induction principle from the well-ordering principle. (b) Prove the well-ordering

### Lecture 11. Shuanglin Shao. October 2nd and 7th, 2013

Lecture 11 Shuanglin Shao October 2nd and 7th, 2013 Matrix determinants: addition. Determinants: multiplication. Adjoint of a matrix. Cramer s rule to solve a linear system. Recall that from the previous

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### 12 Greatest Common Divisors. The Euclidean Algorithm

Arkansas Tech University MATH 4033: Elementary Modern Algebra Dr. Marcel B. Finan 12 Greatest Common Divisors. The Euclidean Algorithm As mentioned at the end of the previous section, we would like to

### Prime Numbers. Chapter Primes and Composites

Chapter 2 Prime Numbers The term factoring or factorization refers to the process of expressing an integer as the product of two or more integers in a nontrivial way, e.g., 42 = 6 7. Prime numbers are

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Topic 1: Matrices and Systems of Linear Equations.

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### Congruences. Robert Friedman

Congruences Robert Friedman Definition of congruence mod n Congruences are a very handy way to work with the information of divisibility and remainders, and their use permeates number theory. Definition

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### Mathematical Induction

Chapter 2 Mathematical Induction 2.1 First Examples Suppose we want to find a simple formula for the sum of the first n odd numbers: 1 + 3 + 5 +... + (2n 1) = n (2k 1). How might we proceed? The most natural

### Linear Systems. Singular and Nonsingular Matrices. Find x 1, x 2, x 3 such that the following three equations hold:

Linear Systems Example: Find x, x, x such that the following three equations hold: x + x + x = 4x + x + x = x + x + x = 6 We can write this using matrix-vector notation as 4 {{ A x x x {{ x = 6 {{ b General

### Lecture 5 - Triangular Factorizations & Operation Counts

LU Factorization Lecture 5 - Triangular Factorizations & Operation Counts We have seen that the process of GE essentially factors a matrix A into LU Now we want to see how this factorization allows us

### Chapter 8. Matrices II: inverses. 8.1 What is an inverse?

Chapter 8 Matrices II: inverses We have learnt how to add subtract and multiply matrices but we have not defined division. The reason is that in general it cannot always be defined. In this chapter, we

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Mathematics of Cryptography

CHAPTER 2 Mathematics of Cryptography Part I: Modular Arithmetic, Congruence, and Matrices Objectives This chapter is intended to prepare the reader for the next few chapters in cryptography. The chapter

### Solution. Area(OABC) = Area(OAB) + Area(OBC) = 1 2 det( [ 5 2 1 2. Question 2. Let A = (a) Calculate the nullspace of the matrix A.

Solutions to Math 30 Take-home prelim Question. Find the area of the quadrilateral OABC on the figure below, coordinates given in brackets. [See pp. 60 63 of the book.] y C(, 4) B(, ) A(5, ) O x Area(OABC)

### 8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

### MATH 289 PROBLEM SET 4: NUMBER THEORY

MATH 289 PROBLEM SET 4: NUMBER THEORY 1. The greatest common divisor If d and n are integers, then we say that d divides n if and only if there exists an integer q such that n = qd. Notice that if d divides

### Mathematics Course 111: Algebra I Part IV: Vector Spaces

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### Lecture Notes: Matrix Inverse. 1 Inverse Definition

Lecture Notes: Matrix Inverse Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Inverse Definition We use I to represent identity matrices,

### Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that. a = bq + r and 0 r < b.

Theorem (The division theorem) Suppose that a and b are integers with b > 0. There exist unique integers q and r so that a = bq + r and 0 r < b. We re dividing a by b: q is the quotient and r is the remainder,

### Solution to Homework 2

Solution to Homework 2 Olena Bormashenko September 23, 2011 Section 1.4: 1(a)(b)(i)(k), 4, 5, 14; Section 1.5: 1(a)(b)(c)(d)(e)(n), 2(a)(c), 13, 16, 17, 18, 27 Section 1.4 1. Compute the following, if

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### 4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION

4: EIGENVALUES, EIGENVECTORS, DIAGONALIZATION STEVEN HEILMAN Contents 1. Review 1 2. Diagonal Matrices 1 3. Eigenvectors and Eigenvalues 2 4. Characteristic Polynomial 4 5. Diagonalizability 6 6. Appendix:

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### We now explore a third method of proof: proof by contradiction.

CHAPTER 6 Proof by Contradiction We now explore a third method of proof: proof by contradiction. This method is not limited to proving just conditional statements it can be used to prove any kind of statement

### Notes on Factoring. MA 206 Kurt Bryan

The General Approach Notes on Factoring MA 26 Kurt Bryan Suppose I hand you n, a 2 digit integer and tell you that n is composite, with smallest prime factor around 5 digits. Finding a nontrivial factor

### Linear Dependence Tests

Linear Dependence Tests The book omits a few key tests for checking the linear dependence of vectors. These short notes discuss these tests, as well as the reasoning behind them. Our first test checks

### NUMERICAL METHODS C. Carl Gustav Jacob Jacobi 10.1 GAUSSIAN ELIMINATION WITH PARTIAL PIVOTING

0. Gaussian Elimination with Partial Pivoting 0.2 Iterative Methods for Solving Linear Systems 0.3 Power Method for Approximating Eigenvalues 0.4 Applications of Numerical Methods Carl Gustav Jacob Jacobi

### The last three chapters introduced three major proof techniques: direct,

CHAPTER 7 Proving Non-Conditional Statements The last three chapters introduced three major proof techniques: direct, contrapositive and contradiction. These three techniques are used to prove statements

### 2 The Euclidean algorithm

2 The Euclidean algorithm Do you understand the number 5? 6? 7? At some point our level of comfort with individual numbers goes down as the numbers get large For some it may be at 43, for others, 4 In

### 1 Gaussian Elimination

Contents 1 Gaussian Elimination 1.1 Elementary Row Operations 1.2 Some matrices whose associated system of equations are easy to solve 1.3 Gaussian Elimination 1.4 Gauss-Jordan reduction and the Reduced

### Au = = = 3u. Aw = = = 2w. so the action of A on u and w is very easy to picture: it simply amounts to a stretching by 3 and 2, respectively.

Chapter 7 Eigenvalues and Eigenvectors In this last chapter of our exploration of Linear Algebra we will revisit eigenvalues and eigenvectors of matrices, concepts that were already introduced in Geometry

### Today s Topics. Primes & Greatest Common Divisors

Today s Topics Primes & Greatest Common Divisors Prime representations Important theorems about primality Greatest Common Divisors Least Common Multiples Euclid s algorithm Once and for all, what are prime

### 2.1: Determinants by Cofactor Expansion. Math 214 Chapter 2 Notes and Homework. Evaluate a Determinant by Expanding by Cofactors

2.1: Determinants by Cofactor Expansion Math 214 Chapter 2 Notes and Homework Determinants The minor M ij of the entry a ij is the determinant of the submatrix obtained from deleting the i th row and the

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### Practice Problems for First Test

Mathematicians have tried in vain to this day to discover some order in the sequence of prime numbers, and we have reason to believe that it is a mystery into which the human mind will never penetrate.-

### Solving Linear Systems, Continued and The Inverse of a Matrix

, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing

### 1.5 Elementary Matrices and a Method for Finding the Inverse

.5 Elementary Matrices and a Method for Finding the Inverse Definition A n n matrix is called an elementary matrix if it can be obtained from I n by performing a single elementary row operation Reminder:

### Direct Methods for Solving Linear Systems. Linear Systems of Equations

Direct Methods for Solving Linear Systems Linear Systems of Equations Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University

### 1. R In this and the next section we are going to study the properties of sequences of real numbers.

+a 1. R In this and the next section we are going to study the properties of sequences of real numbers. Definition 1.1. (Sequence) A sequence is a function with domain N. Example 1.2. A sequence of real

### APPLICATIONS OF THE ORDER FUNCTION

APPLICATIONS OF THE ORDER FUNCTION LECTURE NOTES: MATH 432, CSUSM, SPRING 2009. PROF. WAYNE AITKEN In this lecture we will explore several applications of order functions including formulas for GCDs and

### Chapter 6. Number Theory. 6.1 The Division Algorithm

Chapter 6 Number Theory The material in this chapter offers a small glimpse of why a lot of facts that you ve probably nown and used for a long time are true. It also offers some exposure to generalization,

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### Continued fractions and good approximations.

Continued fractions and good approximations We will study how to find good approximations for important real life constants A good approximation must be both accurate and easy to use For instance, our

The Hadamard Product Elizabeth Million April 12, 2007 1 Introduction and Basic Results As inexperienced mathematicians we may have once thought that the natural definition for matrix multiplication would

### 1 VECTOR SPACES AND SUBSPACES

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

### Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

### 160 CHAPTER 4. VECTOR SPACES

160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

### 5.3 Determinants and Cramer s Rule

290 5.3 Determinants and Cramer s Rule Unique Solution of a 2 2 System The 2 2 system (1) ax + by = e, cx + dy = f, has a unique solution provided = ad bc is nonzero, in which case the solution is given

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Algebraic Complexity Theory and Matrix Multiplication

Algebraic Complexity Theory and Matrix Multiplication [Handout for the Tutorial] François Le Gall Department of Computer Science Graduate School of Information Science and Technology The University of

### 8 Square matrices continued: Determinants

8 Square matrices continued: Determinants 8. Introduction Determinants give us important information about square matrices, and, as we ll soon see, are essential for the computation of eigenvalues. You

### 10. Graph Matrices Incidence Matrix

10 Graph Matrices Since a graph is completely determined by specifying either its adjacency structure or its incidence structure, these specifications provide far more efficient ways of representing a

### Definition: Group A group is a set G together with a binary operation on G, satisfying the following axioms: a (b c) = (a b) c.

Algebraic Structures Abstract algebra is the study of algebraic structures. Such a structure consists of a set together with one or more binary operations, which are required to satisfy certain axioms.

### Some Polynomial Theorems. John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.

Some Polynomial Theorems by John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.com This paper contains a collection of 31 theorems, lemmas,

### a = bq + r where 0 r < b.

Lecture 5: Euclid s algorithm Introduction The fundamental arithmetic operations are addition, subtraction, multiplication and division. But there is a fifth operation which I would argue is just as fundamental

### 4. MATRICES Matrices

4. MATRICES 170 4. Matrices 4.1. Definitions. Definition 4.1.1. A matrix is a rectangular array of numbers. A matrix with m rows and n columns is said to have dimension m n and may be represented as follows:

### LINEAR ALGEBRA. September 23, 2010

LINEAR ALGEBRA September 3, 00 Contents 0. LU-decomposition.................................... 0. Inverses and Transposes................................. 0.3 Column Spaces and NullSpaces.............................

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### Methods for Finding Bases

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

### 11 Ideals. 11.1 Revisiting Z

11 Ideals The presentation here is somewhat different than the text. In particular, the sections do not match up. We have seen issues with the failure of unique factorization already, e.g., Z[ 5] = O Q(

### 7. LU factorization. factor-solve method. LU factorization. solving Ax = b with A nonsingular. the inverse of a nonsingular matrix

7. LU factorization EE103 (Fall 2011-12) factor-solve method LU factorization solving Ax = b with A nonsingular the inverse of a nonsingular matrix LU factorization algorithm effect of rounding error sparse

### GREATEST COMMON DIVISOR

DEFINITION: GREATEST COMMON DIVISOR The greatest common divisor (gcd) of a and b, denoted by (a, b), is the largest common divisor of integers a and b. THEOREM: If a and b are nonzero integers, then their

### Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

### Further linear algebra. Chapter I. Integers.

Further linear algebra. Chapter I. Integers. Andrei Yafaev Number theory is the theory of Z = {0, ±1, ±2,...}. 1 Euclid s algorithm, Bézout s identity and the greatest common divisor. We say that a Z divides

### Modélisation et résolutions numérique et symbolique

Modélisation et résolutions numérique et symbolique via les logiciels Maple et Matlab Jeremy Berthomieu Mohab Safey El Din Stef Graillat Mohab.Safey@lip6.fr Outline Previous course: partial review of what

### ORDERS OF ELEMENTS IN A GROUP

ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since

GENERATING SETS KEITH CONRAD 1 Introduction In R n, every vector can be written as a unique linear combination of the standard basis e 1,, e n A notion weaker than a basis is a spanning set: a set of vectors

### Course notes on Number Theory

Course notes on Number Theory In Number Theory, we make the decision to work entirely with whole numbers. There are many reasons for this besides just mathematical interest, not the least of which is that

### 6 Gaussian Elimination

G1BINM Introduction to Numerical Methods 6 1 6 Gaussian Elimination 61 Simultaneous linear equations Consider the system of linear equations a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 +

### Solving Sets of Equations. 150 B.C.E., 九章算術 Carl Friedrich Gauss,

Solving Sets of Equations 5 B.C.E., 九章算術 Carl Friedrich Gauss, 777-855 Gaussian-Jordan Elimination In Gauss-Jordan elimination, matrix is reduced to diagonal rather than triangular form Row combinations

### 2.5 Gaussian Elimination

page 150 150 CHAPTER 2 Matrices and Systems of Linear Equations 37 10 the linear algebra package of Maple, the three elementary 20 23 1 row operations are 12 1 swaprow(a,i,j): permute rows i and j 3 3

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### The Characteristic Polynomial

Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem

### Week 5 Integral Polyhedra

Week 5 Integral Polyhedra We have seen some examples 1 of linear programming formulation that are integral, meaning that every basic feasible solution is an integral vector. This week we develop a theory

### Quotient Rings and Field Extensions

Chapter 5 Quotient Rings and Field Extensions In this chapter we describe a method for producing field extension of a given field. If F is a field, then a field extension is a field K that contains F.