# Lecture 12: Chain Matrix Multiplication

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Lecture 12: Chain Matrix Multiplication CLRS Section 15.2 Outline of this Lecture Recalling matrix multiplication. The chain matrix multiplication problem. A dynamic programming algorithm for chain matrix multiplication. 1

2 Recalling Matrix Multiplication Matrix: An matrix is a twodimensional array! " ')((( # # \$#! " \$# %" &...! ". * which has rows and columns. Example: The - a+, following. / is 0 1 matrix: '((( 0 1.2/, 1.,, 1 31 / 43!.! * 5 2

3 Recalling Matrix Multiplication The product of a matrix and a matrix is a matrix given by for and. Example: If. / 0 1!,, 1 '( * 0. 1,, '( * then ')( * 5 3

4 5 Remarks on Matrix Multiplication If is defined, may not be defined. Quite possible that. Multiplication is recursively defined by Matrix multiplication is associative, e.g., so parenthenization does not change result. 4

5 Direct Matrix multiplication Given a matrix and a matrix, the direct way of multiplying is to compute each for and. Complexity of Direct Matrix multiplication: Note that has entries and each entry takes time to compute so the total procedure takes time. 5

6 5 Direct Matrix multiplication of Given a matrix, a matrix and a matrix, then and : can be computed in two ways The number of multiplications needed are: 5, + 1 When,, and, then. 3.. A big difference! Implication: The multiplication sequence (parenthesization) is important!! 6

7 The Chain Matrix Multiplication Problem dimensions 555 corresponding to matrix sequence, Given,555, where has dimension, determine the multiplication sequence that minimizes the number of scalar multiplications in computing. That is, determine how to parenthisize - the multiplications. Exhaustive search: +. Question: Any better approach? Yes DP 7

8 Developing a Dynamic Programming Algorithm Step 1: Determine the structure of an optimal solution (in this case, a parenthesization). Decompose the problem into subproblems: For each pair, determine the multiplication sequence for that minimizes the number of multiplications. Clearly, is a matrix. Original Problem: determine sequence of multiplication for. 8

9 5 Developing a Dynamic Programming Algorithm Step 1: Determine the structure of an optimal solution (in this case, a parenthesization). High-Level Parenthesization for For any optimal multiplication sequence, at the last step you are multiplying two matrices and for some. That is, 5 Example Here,. 9

10 Developing a Dynamic Programming Algorithm Step 1 Continued: Thus the problem of determining the optimal sequence of multiplications is broken down into 2 questions: How do we decide where to split the chain (what is )? (Search all possible values of ) How do we parenthesize the subchains and? (Problem has optimal substructure property that and must be optimal so we can apply the same procedure recursively) 10

11 Developing a Dynamic Programming Algorithm Step 1 Continued: Optimal Substructure Property: If final optimal solution of involves splitting into and at final step then parenthesization of and in final optimal solution must also be optimal for the subproblems standing alone : If parenthisization of was not optimal we could replace it by a better parenthesization and get a cheaper final solution, leading to a contradiction. Similarly, if parenthisization of was not optimal we could replace it by a better parenthesization and get a cheaper final solution, also leading to a contradiction. 11

12 Developing a Dynamic Programming Algorithm Step 2: Recursively define the value of an optimal solution. As with the 0-1 knapsack problem, we will store the solutions to the subproblems in an array. For, let " denote the minimum number of multiplications needed to compute. The optimum cost can be described by the following recursive definition. 12

13 Developing a Dynamic Programming Algorithm Step 2: Recursively define the value of an optimal solution. Proof: Any optimal sequence of multiplication for is equivalent to some choice of splitting and also are optimal. Hence for some, where the sequences of multiplications for 5 13

14 Developing a Dynamic Programming Algorithm Step 2 Continued: We know that, for some 5 We don t know what is, though But, there are only! possible values of so we can check them all and find the one which returns a smallest cost. Therefore 14

15 Developing a Dynamic Programming Algorithm Step 3: Compute the value of an optimal solution in a bottom-up fashion. Our Table:. only defined for. The important point is that when we use the equation to calculate we must have already evaluated and For both cases, the corresponding length of the matrix-chain are both less than. Hence, the algorithm should fill the table in increasing order of the length of the matrixchain. That is, we calculate in the order!! "# \$ %& ' ( ) * *+, - & ) /.0 * 1. *2# & & 15

16 Dynamic Programming Design Warning!! When designing a dynamic programming algorithm there are two parts: 1. Finding an appropriate optimal substructure property and corresponding recurrence relation on table items. Example: 2. Filling in the table properly. This requires finding an ordering of the table elements so that when a table item is calculated using the recurrence relation, all the table values needed by the recurrence relation have already been calculated. In our example this means that by the time " is calculated all of the values and were already calculated. 16

17 Example for the Bottom-Up Computation 1, Example: Given a chain of four matrices, and, 0, with, +,. Find +. S0: Initialization, and 4 1 m[i,j] 3 2 j i A1 A2 A3 A4 p0 p1 p2 p3 p4 17

18 5 Example Continued Stp 1: Computing By definition & " \$ m[i,j] 3 2 j i A1 A2 A3 A4 p0 p1 p2 p3 p4 18

19 5 Example Continued Stp 2: Computing # By definition # \$ \$ m[i,j] 3 2 j i A1 A2 A3 A4 p0 p1 p2 p3 p4 19

20 Example Continued Stp3: Computing # + By definition # m[i,j] 3 2 j i A1 A2 A3 A4 p0 p1 p2 p3 p4 20

21 5 Example Continued Stp4: Computing By definition " \$# & # m[i,j] 3 2 j 2 88 i A1 A2 A3 A4 p0 p1 p2 p3 p4 21

22 Example Continued Stp5: Computing # + By definition # + \$ + \$#& + \$# # 4 1 m[i,j] 3 2 j i A1 A2 A3 A4 p0 p1 p2 p3 p4 22

23 5 Example Continued St6: Computing + By definition + + # + # + + +,. 4 1 m[i,j] j i A1 A2 A3 A4 p0 p1 p2 p3 p4 We are done! 23

24 Developing a Dynamic Programming Algorithm Step 4: Construct an optimal solution from computed information extract the actual sequence. Idea: Maintain an array % " 55 55, where denotes for the optimal splitting in computing The array can be used recursively to recover the multiplication sequence. How to Recover the Multiplication Sequence?.. Do this recursively until the multiplication sequence is determined. 24

25 5 Developing a Dynamic Programming Algorithm Step 4: Construct an optimal solution from computed information extract the actual sequence. Example of Finding the Multiplication Sequence: Consider 1. Assume that the array has been computed. The multiplication sequence is recovered as follows , Hence the final multiplication sequence is 55 1 %

26 Hence the time complexity is. Space complexity. The Dynamic Programming Algorithm Matrix-Chain( ) for ( to ) ; for ( to ) for ( to ; ; for ( return and ) to ) if ( ) ; ; ; (Optimum in ) ; Complexity: The loops are nested three deep. Each loop index takes on values. 26

27 Constructing an Optimal Solution: Compute The actual multiplication code uses the " value to determine how to split the current sequence. Assume that the matrices are stored in an array of matrices 55, and that is global to this recursive procedure. The procedure returns a matrix. Mult( ) if ( ) ; is now, where is ; is now return ; multiply matrices and else return ; To compute, call Mult( % ). 27

28 5 Constructing an Optimal Solution: Compute Example of Constructing an Optimal Solution: Compute Consider the example 55 earlier, 55 where. Assume that the array has been computed. The multiplication sequence is recovered as follows. Mult Mult Mult Mult Mult Hence the product is computed as follows 28

### Matrix-Chain Multiplication

Matrix-Chain Multiplication Let A be an n by m matrix, let B be an m by p matrix, then C = AB is an n by p matrix. C = AB can be computed in O(nmp) time, using traditional matrix multiplication. Suppose

### Dynamic Programming. Applies when the following Principle of Optimality

Dynamic Programming Applies when the following Principle of Optimality holds: In an optimal sequence of decisions or choices, each subsequence must be optimal. Translation: There s a recursive solution.

### Dynamic Programming. Lecture 11. 11.1 Overview. 11.2 Introduction

Lecture 11 Dynamic Programming 11.1 Overview Dynamic Programming is a powerful technique that allows one to solve many different types of problems in time O(n 2 ) or O(n 3 ) for which a naive approach

### Memoization/Dynamic Programming. The String reconstruction problem. CS125 Lecture 5 Fall 2016

CS125 Lecture 5 Fall 2016 Memoization/Dynamic Programming Today s lecture discusses memoization, which is a method for speeding up algorithms based on recursion, by using additional memory to remember

### Lecture 13: The Knapsack Problem

Lecture 13: The Knapsack Problem Outline of this Lecture Introduction of the 0-1 Knapsack Problem. A dynamic programming solution to this problem. 1 0-1 Knapsack Problem Informal Description: We have computed

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

### 1.2 Solving a System of Linear Equations

1.. SOLVING A SYSTEM OF LINEAR EQUATIONS 1. Solving a System of Linear Equations 1..1 Simple Systems - Basic De nitions As noticed above, the general form of a linear system of m equations in n variables

### The Inverse of a Matrix

The Inverse of a Matrix 7.4 Introduction In number arithmetic every number a ( 0) has a reciprocal b written as a or such that a ba = ab =. Some, but not all, square matrices have inverses. If a square

### B such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix

Matrix inverses Recall... Definition A square matrix A is invertible (or nonsingular) if matrix B such that AB = and BA =. (We say B is an inverse of A.) Remark Not all square matrices are invertible.

### Lecture 6. Inverse of Matrix

Lecture 6 Inverse of Matrix Recall that any linear system can be written as a matrix equation In one dimension case, ie, A is 1 1, then can be easily solved as A x b Ax b x b A 1 A b A 1 b provided that

### Divide And Conquer Algorithms

CSE341T/CSE549T 09/10/2014 Lecture 5 Divide And Conquer Algorithms Recall in last lecture, we looked at one way of parallelizing matrix multiplication. At the end of the lecture, we saw the reduce SUM

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### The Greedy Method. Fundamental Techniques. The Greedy Method. Fractional Knapsack Problem

Fundamental Techniques There are some algorithmic tools that are quite specialised. They are good for problems they are intended to solve, but they are not very versatile. There are also more fundamental

### (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular.

Theorem.7.: (Properties of Triangular Matrices) (a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is lower triangular. (b) The product

### Similarity and Diagonalization. Similar Matrices

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

### Matrix Product. Matrix-Multiply (A, B) Cost of multiplying these matrices is p q r.

Dynamic Programming A powerful paradigm for algorithm design. Often leads to elegant and efficient algorithms when greedy or diide-and-conquer don t work. DP also breaks a problem into subproblems, but

### 1 Gaussian Elimination

Contents 1 Gaussian Elimination 1.1 Elementary Row Operations 1.2 Some matrices whose associated system of equations are easy to solve 1.3 Gaussian Elimination 1.4 Gauss-Jordan reduction and the Reduced

### ( % . This matrix consists of \$ 4 5 " 5' the coefficients of the variables as they appear in the original system. The augmented 3 " 2 2 # 2 " 3 4&

Matrices define matrix We will use matrices to help us solve systems of equations. A matrix is a rectangular array of numbers enclosed in parentheses or brackets. In linear algebra, matrices are important

### Solving Systems of Linear Equations. Substitution

Solving Systems of Linear Equations There are two basic methods we will use to solve systems of linear equations: Substitution Elimination We will describe each for a system of two equations in two unknowns,

### Lecture 2 Mathcad basics and Matrix Operations

Lecture 2 Mathcad basics and Matrix Operations Announcements No class or lab Wednesday, 8/29/01 I will be posting a lab worksheet on the web site on Tuesday for you to work through on your own. Operators

### MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors. Jordan canonical form (continued).

MATH 423 Linear Algebra II Lecture 38: Generalized eigenvectors Jordan canonical form (continued) Jordan canonical form A Jordan block is a square matrix of the form λ 1 0 0 0 0 λ 1 0 0 0 0 λ 0 0 J = 0

### Chapter 4 - Systems of Equations and Inequalities

Math 233 - Spring 2009 Chapter 4 - Systems of Equations and Inequalities 4.1 Solving Systems of equations in Two Variables Definition 1. A system of linear equations is two or more linear equations to

### Lecture 21: Dynamic Programming III

Lecture 21: Dynamic Programming III Lecture Overview Subproblems for strings Parenthesization Edit distance (& longest common subseq.) Knapsack Pseudopolynomial Time Review: * 5 easy steps to dynamic programming

### 160 CHAPTER 4. VECTOR SPACES

160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

### DERIVATIVES AS MATRICES; CHAIN RULE

DERIVATIVES AS MATRICES; CHAIN RULE 1. Derivatives of Real-valued Functions Let s first consider functions f : R 2 R. Recall that if the partial derivatives of f exist at the point (x 0, y 0 ), then we

### MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix.

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

### Merge Sort and Recurrences. COSC 581, Algorithms January 14, 2014

Merge Sort and Recurrences COSC 581, Algorithms January 14, 2014 Reading Assignments Today s class: Chapter 2, 4.0, 4.4 Reading assignment for next class: Chapter 4.2, 4.5 3 Common Algorithmic Techniques

### NOTES ON LINEAR TRANSFORMATIONS

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

### In this leaflet we explain what is meant by an inverse matrix and how it is calculated.

5.5 Introduction The inverse of a matrix In this leaflet we explain what is meant by an inverse matrix and how it is calculated. 1. The inverse of a matrix The inverse of a square n n matrix A, is another

### UNIT 2 MATRICES - I 2.0 INTRODUCTION. Structure

UNIT 2 MATRICES - I Matrices - I Structure 2.0 Introduction 2.1 Objectives 2.2 Matrices 2.3 Operation on Matrices 2.4 Invertible Matrices 2.5 Systems of Linear Equations 2.6 Answers to Check Your Progress

### December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B. KITCHENS

December 4, 2013 MATH 171 BASIC LINEAR ALGEBRA B KITCHENS The equation 1 Lines in two-dimensional space (1) 2x y = 3 describes a line in two-dimensional space The coefficients of x and y in the equation

### Systems of Linear Equations

A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Systems of Linear Equations Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

### 1.3 Induction and Other Proof Techniques

4CHAPTER 1. INTRODUCTORY MATERIAL: SETS, FUNCTIONS AND MATHEMATICAL INDU 1.3 Induction and Other Proof Techniques The purpose of this section is to study the proof technique known as mathematical induction.

### Warshall s Algorithm: Transitive Closure

CS 0 Theory of Algorithms / CS 68 Algorithms in Bioinformaticsi Dynamic Programming Part II. Warshall s Algorithm: Transitive Closure Computes the transitive closure of a relation (Alternatively: all paths

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### 1 Eigenvalues and Eigenvectors

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

### Solving Systems of Linear Equations

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

### Dynamic Programming Problem Set Partial Solution CMPSC 465

Dynamic Programming Problem Set Partial Solution CMPSC 465 I ve annotated this document with partial solutions to problems written more like a test solution. (I remind you again, though, that a formal

### Here are some examples of combining elements and the operations used:

MATRIX OPERATIONS Summary of article: What is an operation? Addition of two matrices. Multiplication of a Matrix by a scalar. Subtraction of two matrices: two ways to do it. Combinations of Addition, Subtraction,

### MATHEMATICAL INDUCTION AND DIFFERENCE EQUATIONS

1 CHAPTER 6. MATHEMATICAL INDUCTION AND DIFFERENCE EQUATIONS 1 INSTITIÚID TEICNEOLAÍOCHTA CHEATHARLACH INSTITUTE OF TECHNOLOGY CARLOW MATHEMATICAL INDUCTION AND DIFFERENCE EQUATIONS 1 Introduction Recurrence

### A Brief Primer on Matrix Algebra

A Brief Primer on Matrix Algebra A matrix is a rectangular array of numbers whose individual entries are called elements. Each horizontal array of elements is called a row, while each vertical array is

### CSE 5311 Homework 3 Solution

CSE 5311 Homework 3 Solution Problem 15.1-1 Show that equation (15.4) follows from equation (15.3) and the initial condition T (0) = 1. We can verify that T (n) = n is a solution to the given recurrence

### 9 Matrices, determinants, inverse matrix, Cramer s Rule

AAC - Business Mathematics I Lecture #9, December 15, 2007 Katarína Kálovcová 9 Matrices, determinants, inverse matrix, Cramer s Rule Basic properties of matrices: Example: Addition properties: Associative:

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

### Math 2331 Linear Algebra

2.2 The Inverse of a Matrix Math 2331 Linear Algebra 2.2 The Inverse of a Matrix Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math2331 Jiwen He, University

### Solving Systems of Linear Equations Using Matrices

Solving Systems of Linear Equations Using Matrices What is a Matrix? A matrix is a compact grid or array of numbers. It can be created from a system of equations and used to solve the system of equations.

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

### 2.5 Elementary Row Operations and the Determinant

2.5 Elementary Row Operations and the Determinant Recall: Let A be a 2 2 matrtix : A = a b. The determinant of A, denoted by det(a) c d or A, is the number ad bc. So for example if A = 2 4, det(a) = 2(5)

### Solving Systems of Linear Equations; Row Reduction

Harvey Mudd College Math Tutorial: Solving Systems of Linear Equations; Row Reduction Systems of linear equations arise in all sorts of applications in many different fields of study The method reviewed

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### Math 315: Linear Algebra Solutions to Midterm Exam I

Math 35: Linear Algebra s to Midterm Exam I # Consider the following two systems of linear equations (I) ax + by = k cx + dy = l (II) ax + by = 0 cx + dy = 0 (a) Prove: If x = x, y = y and x = x 2, y =

### MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix.

MATH 304 Linear Algebra Lecture 4: Matrix multiplication. Diagonal matrices. Inverse matrix. Matrices Definition. An m-by-n matrix is a rectangular array of numbers that has m rows and n columns: a 11

### Recurrences (CLRS )

Recurrences (CLRS 4.1-4.2) Last time we discussed divide-and-conquer algorithms Divide and Conquer To Solve P: 1. Divide P into smaller problems P 1,P 2,P 3...P k. 2. Conquer by solving the (smaller) subproblems

### MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix.

MATH 304 Linear Algebra Lecture 8: Inverse matrix (continued). Elementary matrices. Transpose of a matrix. Inverse matrix Definition. Let A be an n n matrix. The inverse of A is an n n matrix, denoted

### University of Lille I PC first year list of exercises n 7. Review

University of Lille I PC first year list of exercises n 7 Review Exercise Solve the following systems in 4 different ways (by substitution, by the Gauss method, by inverting the matrix of coefficients

### Matrices: 2.3 The Inverse of Matrices

September 4 Goals Define inverse of a matrix. Point out that not every matrix A has an inverse. Discuss uniqueness of inverse of a matrix A. Discuss methods of computing inverses, particularly by row operations.

### 2x + y = 3. Since the second equation is precisely the same as the first equation, it is enough to find x and y satisfying the system

1. Systems of linear equations We are interested in the solutions to systems of linear equations. A linear equation is of the form 3x 5y + 2z + w = 3. The key thing is that we don t multiply the variables

### CS420/520 Algorithm Analysis Spring 2010 Lecture 04

CS420/520 Algorithm Analysis Spring 2010 Lecture 04 I. Chapter 4 Recurrences A. Introduction 1. Three steps applied at each level of a recursion a) Divide into a number of subproblems that are smaller

### NON SINGULAR MATRICES. DEFINITION. (Non singular matrix) An n n A is called non singular or invertible if there exists an n n matrix B such that

NON SINGULAR MATRICES DEFINITION. (Non singular matrix) An n n A is called non singular or invertible if there exists an n n matrix B such that AB = I n = BA. Any matrix B with the above property is called

### Matrix Algebra. Some Basic Matrix Laws. Before reading the text or the following notes glance at the following list of basic matrix algebra laws.

Matrix Algebra A. Doerr Before reading the text or the following notes glance at the following list of basic matrix algebra laws. Some Basic Matrix Laws Assume the orders of the matrices are such that

### Relations Graphical View

Relations Slides by Christopher M. Bourke Instructor: Berthe Y. Choueiry Introduction Recall that a relation between elements of two sets is a subset of their Cartesian product (of ordered pairs). A binary

### Graph Theory for Articulated Bodies

Graph Theory for Articulated Bodies Alba Perez-Gracia Department of Mechanical Engineering, Idaho State University Articulated Bodies A set of rigid bodies (links) joined by joints that allow relative

### Divide and Conquer My T. UF

Introduction to Algorithms Divide and Conquer @ UF General Approach Divide the problem into a number of subproblems that are smaller instances of the same problem Conquer the subproblems by solving them

### These axioms must hold for all vectors ū, v, and w in V and all scalars c and d.

DEFINITION: A vector space is a nonempty set V of objects, called vectors, on which are defined two operations, called addition and multiplication by scalars (real numbers), subject to the following axioms

### I. GROUPS: BASIC DEFINITIONS AND EXAMPLES

I GROUPS: BASIC DEFINITIONS AND EXAMPLES Definition 1: An operation on a set G is a function : G G G Definition 2: A group is a set G which is equipped with an operation and a special element e G, called

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

### Diagonal, Symmetric and Triangular Matrices

Contents 1 Diagonal, Symmetric Triangular Matrices 2 Diagonal Matrices 2.1 Products, Powers Inverses of Diagonal Matrices 2.1.1 Theorem (Powers of Matrices) 2.2 Multiplying Matrices on the Left Right by

### Lecture Notes 2: Matrices as Systems of Linear Equations

2: Matrices as Systems of Linear Equations 33A Linear Algebra, Puck Rombach Last updated: April 13, 2016 Systems of Linear Equations Systems of linear equations can represent many things You have probably

### Basics Inversion and related concepts Random vectors Matrix calculus. Matrix algebra. Patrick Breheny. January 20

Matrix algebra January 20 Introduction Basics The mathematics of multiple regression revolves around ordering and keeping track of large arrays of numbers and solving systems of equations The mathematical

### Cross product and determinants (Sect. 12.4) Two main ways to introduce the cross product

Cross product and determinants (Sect. 12.4) Two main ways to introduce the cross product Geometrical definition Properties Expression in components. Definition in components Properties Geometrical expression.

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

### A. V. Gerbessiotis CS Spring 2014 PS 3 Mar 24, 2014 No points

A. V. Gerbessiotis CS 610-102 Spring 2014 PS 3 Mar 24, 2014 No points Problem 1. Suppose that we insert n keys into a hash table of size m using open addressing and uniform hashing. Let p(n, m) be the

### Theorem A graph T is a tree if, and only if, every two distinct vertices of T are joined by a unique path.

Chapter 3 Trees Section 3. Fundamental Properties of Trees Suppose your city is planning to construct a rapid rail system. They want to construct the most economical system possible that will meet the

### Row Echelon Form and Reduced Row Echelon Form

These notes closely follow the presentation of the material given in David C Lay s textbook Linear Algebra and its Applications (3rd edition) These notes are intended primarily for in-class presentation

### Elementary Row Operations and Matrix Multiplication

Contents 1 Elementary Row Operations and Matrix Multiplication 1.1 Theorem (Row Operations using Matrix Multiplication) 2 Inverses of Elementary Row Operation Matrices 2.1 Theorem (Inverses of Elementary

### 10.3 POWER METHOD FOR APPROXIMATING EIGENVALUES

58 CHAPTER NUMERICAL METHODS. POWER METHOD FOR APPROXIMATING EIGENVALUES In Chapter 7 you saw that the eigenvalues of an n n matrix A are obtained by solving its characteristic equation n c nn c nn...

### Solutions to Math 51 First Exam January 29, 2015

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

### Solving Recurrences. Lecture 13 CS2110 Summer 2009

Solving Recurrences Lecture 13 CS2110 Summer 2009 In the past Complexity: Big-O definition Non-recursive programs 2 Today Complexity of recursive programs Introduce recurrences Methods to solve recurrences

### Lecture No. # 02 Prologue-Part 2

Advanced Matrix Theory and Linear Algebra for Engineers Prof. R.Vittal Rao Center for Electronics Design and Technology Indian Institute of Science, Bangalore Lecture No. # 02 Prologue-Part 2 In the last

### Linear Codes. In the V[n,q] setting, the terms word and vector are interchangeable.

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

### From the Main Menu, select the MAT icon to enter the Matrix Mode and display the initial screen.

!""#"\$\$"#% From the Main Menu, select the MAT icon to enter the Matrix Mode and display the initial screen. Creating a Matrix To create a matrix, you must first define its dimensions in the MATRIX list.

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

### Mathematics of Cryptography

CHAPTER 2 Mathematics of Cryptography Part I: Modular Arithmetic, Congruence, and Matrices Objectives This chapter is intended to prepare the reader for the next few chapters in cryptography. The chapter

### data structures and algorithms exercise class 2: some answers

data structures and algorithms 2016-2017 exercise class 2: some answers 1. Show step by step how the array A = [22, 15, 36, 20, 3, 9, 29] is sorted using selection sort, bubble sort, and merge sort. First

### Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M 1 such that

0. Inverse Matrix Definition A square matrix M is invertible (or nonsingular) if there exists a matrix M such that M M = I = M M. Inverse of a 2 2 Matrix Let M and N be the matrices: a b d b M =, N = c

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 2. x n. a 11 a 12 a 1n b 1 a 21 a 22 a 2n b 2 a 31 a 32 a 3n b 3. a m1 a m2 a mn b m

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

### 4.6 Null Space, Column Space, Row Space

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

### 2.1: MATRIX OPERATIONS

.: MATRIX OPERATIONS What are diagonal entries and the main diagonal of a matrix? What is a diagonal matrix? When are matrices equal? Scalar Multiplication 45 Matrix Addition Theorem (pg 0) Let A, B, and

### Cofactor Expansion: Cramer s Rule

Cofactor Expansion: Cramer s Rule MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Introduction Today we will focus on developing: an efficient method for calculating

### is in plane V. However, it may be more convenient to introduce a plane coordinate system in V.

.4 COORDINATES EXAMPLE Let V be the plane in R with equation x +2x 2 +x 0, a two-dimensional subspace of R. We can describe a vector in this plane by its spatial (D)coordinates; for example, vector x 5

### Lecture Note on Linear Algebra 15. Dimension and Rank

Lecture Note on Linear Algebra 15. Dimension and Rank Wei-Shi Zheng, wszheng@ieee.org, 211 November 1, 211 1 What Do You Learn from This Note We still observe the unit vectors we have introduced in Chapter

### Mathematics of Cryptography Modular Arithmetic, Congruence, and Matrices. A Biswas, IT, BESU SHIBPUR

Mathematics of Cryptography Modular Arithmetic, Congruence, and Matrices A Biswas, IT, BESU SHIBPUR McGraw-Hill The McGraw-Hill Companies, Inc., 2000 Set of Integers The set of integers, denoted by Z,

### Orthogonal Diagonalization of Symmetric Matrices

MATH10212 Linear Algebra Brief lecture notes 57 Gram Schmidt Process enables us to find an orthogonal basis of a subspace. Let u 1,..., u k be a basis of a subspace V of R n. We begin the process of finding

### Lecture 1: Systems of Linear Equations

MTH Elementary Matrix Algebra Professor Chao Huang Department of Mathematics and Statistics Wright State University Lecture 1 Systems of Linear Equations ² Systems of two linear equations with two variables

### Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants

Calculus and linear algebra for biomedical engineering Week 4: Inverse matrices and determinants Hartmut Führ fuehr@matha.rwth-aachen.de Lehrstuhl A für Mathematik, RWTH Aachen October 30, 2008 Overview

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### (57) (58) (59) (60) (61)

Module 4 : Solving Linear Algebraic Equations Section 5 : Iterative Solution Techniques 5 Iterative Solution Techniques By this approach, we start with some initial guess solution, say for solution and