# General Principles in Random Variates Generation

Save this PDF as:

Size: px
Start display at page:

Download "General Principles in Random Variates Generation"

## Transcription

1 General Principles in Random Variates Generation E. Moulines Ecole Nationale Supérieure des Télécommunications 20 mai 2007

2 Why use random numbers? Random numbers lie at the heart of many scientific and financial computations. Many important analytically intractable problems can be solved using Monte Carlo techniques. Many problems that can be solved by other computational methods, can be solved much more efficiently using Monte Carlo techniques

3 Why use random numbers Random numbers are the nuts and bolts of simulation. Typically, all the randomness required by the model is simulated by a random number generator whose output is assumed to be a sequence of independent and identically distributed U(0, 1) random variables These random numbers are then transformed as needed to simulation random variables from different probability distributions, such as the normal, exponential, Poisson, etc...

4 Real source of randomness To draw a winning number for several million dollars lottery, people would generally not trust a computer! They would rather prefer a simple physical system that they understand well, such as drawing balls from a container to select successive digits. Even this requires many precautions : the balls must have identical weights and sizes, be well mixed, and changed regularly to reduce the chances that some numbers come out more frequently in the long run... Such procedure is clearly impractical for computer simulations, which often requires millions of random numbers!

5 Real source of randomness on a computer User movements (keyboard strikes, mouse). hard disk / network activity. Processor load, Cursor position, the row currently displayed. Some kernel parameters ( /dev/random )

6 Why use pseudorandom numbers rather than real RN s? As of today, the most convenient and most reliable way of generating the random numbers for stochastic simulations appears to be via deterministic algorithms with a solid mathematical basis. These algorithms produce a sequence of numbers which are in fact not random at all, but seem to behave like independent random numbers ; that is, like a realization of a sequence of IID U(0, 1) random variables Much more convenient to use pseudorandom numbers. Don t need radiation sources or lava lamps. Pseudorandom numbers can be generated on the fly and the sequences are reproducible.

7 Pseudo-Random generator A (pseudo)random number generator is a structure G = (S, s 0, T, U, G), where S is a finite set of states, s 0 S is the initial state (or seed), the mapping T : S S is the transition function, U is a finite set of output symbols, G : S U is the output function.

8 Basic Algorithm The state of the generator is initially s 0 and evolves according to the recurrence s n = T (s n 1 ), for n = 1, 2, 3,.... At step n, the generator outputs the number u n = G(s n ). The u n, n > 0 are the observations, and are also called the random numbers produced by the generator. Clearly, the sequence of states s n is eventually periodic, since the state space S is finite. Indeed, the generator must eventually revisit a state previously seen ; that is, s j = s i for some j > i > 0. From then on, one must have s j+n = s i+n and u j+n = u i+n for all n > 0.

9 Period The period length is the smallest integer ρ > 0 such that for some integer τ 0 and for all n τ, s ρ+n = s n. The smallest τ with this property is called the transient. Often, τ = 0 and the sequence is then called purely periodic. Note that the period length cannot exceed S, the cardinality of the state space. Good generators typically have their ρ very close to S (otherwise, there is a waste of computer memory).

10 Congruential Generators Linear Congruential Generator How to build a good random generator? How can one build a deterministic generator whose output looks totally random? Perhaps a first idea is to write a computer program more or less at random that can also modify its own code in an unpredictable way... However, experience shows that random number generators should not be built at random (see Knuth for more discussion on this). Building a good random number generator may look easy on the surface, but it is not. It requires a good understanding of heavy mathematics.

11 Congruential Generators Linear Congruential Generator A Linear Congruential Generator The best-known and (still) most widely used types of generators are the simple linear congruential generators (LCGs). The state at step n is an integer x n and the transition function T is defined by the recurrence x n = (ax n 1 + c) mod m, where m > 0, a > 0, and c are integers called the modulus, the multiplier, and the additive constant. One can identify s n with x n and the state space S is the set {0,..., m 1}. To produce values in the interval [0, 1], one can simply define the output function G by u n = G(x n ) = x n /m.

12 Congruential Generators Linear Congruential Generator Multiplicative Linear Congruential Generator When c = 0, this generator is called a multiplicative linear congruential generator (MLCG). It is cheaper to implement because it eliminates the addition operation. The maximal period length for the LCG is m in general. For the MLCG it cannot exceed m 1, since 0 is an absorbing state that must be avoided. Two popular values of m are n = and m = In the latter case, the modulo operation may be carried out efficiently by shift operations (or by extracting bits or words if these operations are carried out using unsigned integer arithmetics). However, these values may be too small for the requirements of today s simulations.

13 Congruential Generators Linear Congruential Generator Period of Multiplicative MLCG If the modulus m is a prime number, the maximal period ρ = m 1 results if the multiplier a is a primitive root of m, i.e. a m = 1 mod m and a q 1 mod m for 0 < q < m 1. For this restriction of m and a, each positive integer in {0,..., M} appears exactly once in any sequence of M 1 consecutive numbers produced by this generator.

14 Congruential Generators Linear Congruential Generator Period of LCG Simple conditions are available ensuring that the generator has full period - i.e. that the number of distinct values generated from any seed x 0 is m 1. If c 0, these conditions are c and m are relatively prime every prime number that divides m divides a 1, a 1 is divisible by 4 if m is. As a simple consequence, we observe that if m is a power of two, the generator has full period if c is odd a = 4n + 1 for some integer n.

15 Congruential Generators Linear Congruential Generator A running example For a concrete illustration, let m = = , c = 0, and a = These parameters were originally proposed by Lewis. Take x 0 = Then x 1 = 16807x12345 mod m = u 1 = /m = , x 2 = 16807x mod m = u 2 = /m = , x 3 = 16807x mod m = u 3 = /m =

16 Congruential Generators Multiple Recursive Generators Principle Consider the linear recurrence x n = (a l x n a k x n k ) mod m, where the order k and the modulus m are positive integers, while the coefficients a 1,..., a k are integers in the range { (m 1),..., m 1}. Define Z m as the set {0, 1,..., m 1} on which operations are performed modulo m. The state at step n of the multiple recursive generator (MRG) is the vector s n = (x n,..., x n+k 1 ) Z k m. The output function can be defined simply by u n = G(s n ) = x n /m, which gives a value in [0, 1], or by a more refined transformation if a better resolution than 1/m is required.

17 Congruential Generators Multiple Recursive Generators Properties The characteristic polynomial P of the MRG is defined by P (z) = z k a 1 z k 1 a k z. The maximal period length of the MRG is ρ = m k 1 (for all initial seeds), reached if and only if m is prime and P is a primitive polynomial over Z m, identified here as the finite field with m elements.

18 Congruential Generators Multiple Recursive Generators Primitive roots Suppose that m is prime and let r = (m k 1)/(m 1). The polynomial P is primitive over Z m if and only if it satisfies the following conditions, where everything is assumed to be modulo m 1. [( 1) k+l a k ] (m l)/q = 1 for each prime factor q of m 1 2. z r mod P (z) = ( 1) k+l a k 3. z r/q mod P (z) has degree > 0 for each prime factor q of r, 1 < q < r.

19 Congruential Generators Multiple Recursive Generators Properties If m is not prime, the period length of the MRG has an upper bound typically much lower than m k 1. For k = 1 and m = 2 e, e > 4, the maximum period length is 2 e 2, which is reached if a 1 = 3 or 5( mod 8) and x 0 is odd. Otherwise, if m = p e for p prime and e > 1, and k > 1 or p > 2, the upper bound is (p k 1)p e 1.

20 Congruential Generators Multiple Recursive Generators Choice of m Clearly, p = 2 is very convenient from the implementation point of view, because the modulo operation then amounts to chopping-off the higher-order bits. So to compute ax mod m in that case, for example with e = 32 on a 32-bit computer, just make sure that the overflow-checking option or the compiler is turned off, and compute the product ax using unsigned integers while ignoring the overflow. However, taking m = 2 e imposes a big sacrifice on the period length, especially for k > 1. For example, if k = 7 and in m = (a prime), the maximal period length is (2 31 1) But for m = 2 31 and the same value of k, the upper bound becomes ρ < (2 7 1) < 2 37, which is more than times shorter.

21 Congruential Generators Multiple Recursive Generators Implementation for prime m For k > 1 and prime m, for the characteristic polynomial P to be primitive, it is necessary that a k and at least another coefficient a j be nonzero. From the implementation point of view, it is best to have only two nonzero coefficients ; that is, a recurrence of the form x n = (a r x n r + a k x n k ) mod m with characteristic trinomial P defined by P (z) = z k a r z k r a k.

22 Congruential Generators Multiple Recursive Generators Overflow When m is not a power of 2, computing and adding the products modulo m is not necessarily straightforward, using ordinary integer arithmetic, because of the possibility of overflow. For example, if m = and a l = 16807, then x n 1 can be as large as , so the product a 1 x n 1 can easily exceed 2 31.

23 Congruential Generators Multiple Recursive Generators Floating-point arithmetic In this case, it is appropriate to represent all the numbers and perform all the arithmetic modulo m in double-precision floating point. This works provided that the multipliers a i are small enough so that the integers a i x n i and their sum are always represented exactly by the floating-point values. A sufficient condition is that the floating-point numbers are represented with at least [log 2 ((m 1)(a l + + a k ))] bits of precision in their mantissa, where [x] denotes the smallest integer larger or equal to x. On computers with good 64-bit floating-point hardware (most computers nowadays), this approach usually gives by far the fastest implementation.

24 Objectives Define the error in a meaningful. Devise a procedure for choosing which of several candidate generators with, say, the same modulus, is best according to the specified error criterion. Reduce the error achievable by a particular congruential genera- generator by modifying its structure in a way that preserves its computability, execution time, reproducibility property, and ease of implementation.

25 Structural / Statistical Methods The techniques used to evaluate the quality of random number generators can be partitioned into two main classes : The structural analysis methods (sometimes called theoretical tests) : studies the mathematical structure underlying the successive values produced by the generator, most often over its entire period length the statistical methods (also called empirical tests). It observes the output and applies a statistical test of hypothesis to catch up significant statistical defects. An unlimited number of such tests can be designed.

26 Structural methods Lattice A lattice of dimension t in the t-dimensional real space R t, is a set of the form t L def = V = z j V j, z j Z, (1) j=1 where {V 1,..., V t } is a basis of R t. The lattice L is thus the set of all integer linear combinations of the vectors V 1,..., V t ; ese vectors are called a lattice basis of L. The basis {W 1,..., W t } which satisfies V i, W j = δ i,j is the dual basis, and the lattice generated by this dual basis is called the dual lattice to L.

27 Structural methods Lattice Consider the set T t T t def = {u n = (u n,..., u n+t 1 ), s 0 = (x 0,..., x k 1 ) Z k m} of all overlapping t-uples of successive values produced by the MRG, with u n = x n /m, from all possible initial seeds. This set T t is the intersection of a lattice L t with the t-dimnensional unit hypercube I t = [0, 1] t.

28 Structural methods Lattice Property of an MRG For t k, each vector (x 0,..., x t 1 ) Z t m can be taken as s 0, so T t = Z t m/m = (Z t /m) I t ; In dimension t > k, the set T t contains only m k points, while Z t m/m contains m t points. Therefore, for large t, T t contains only a small fraction of the t-dimensional vectors whose coordinates are multiples of 1/m.

29 Structural methods (a) a = (b) a = (c) a = 51

30 Structural methods Spectral Distance When t k, the points also lie on a lattice L t composed of parallel hyperplanes. The spectral distance d t is the minimal distance between two successive hyperplanes of any family of parallel hyperplanes covering the lattice ; this distance may be shown to be the minimal norm of a non-zero vector belonging to the dual lattice. The shorter the distance d t, the better, because a large d t means thick empty slices. This spectral distance can be computed by solving a quadratic programming problem in integer random variables (which can be solved using branch and bound algorithms for example). Appropriate values of the multiplier have often been found by minimizing this spectral distance (for several values of t).

31 Structural methods Spectral distance The spectral distance is lower bounded by d t d t = 1 γ t m k/t, where γ t is a constant depending only on t. As an example, for m = t = t = 3 d t t = t = t = 6 (2) showing that, even for moderate value of t, the space between the hyperplanes can be pretty large!

32 Structural methods Discrepancy : Definition Consider the N points u n = (u n,..., u n+t 1 ), for n = 0,..., N 1, in dimension t, formed by (overlapping) vectors of t successive output values of the generator. For any hyper-rectangular box aligned with the axes, of the form R = t j=1 [α j, β j [, with 0 α j < β j < 1, let I(R) be the number of points un falling into R, V (R) = t j=1 (β j α j ) be the volume of R. Let R be the set of all such regions R. The t-dimensional (extreme) discrepancy of the set of points {u 0,..., u N 1 } is given by D (t) N = max V (R) I(R)/N. R R If we impose α j = 0 for all j, then the corresponding quantity is called the star-discrepancy

33 Structural methods Why? A low discrepancy value means that the points are very evenly distributed in the unit hypercube. To get superuniformity of the sequence over its entire period, one might want to minimize the discrepancy D (t) N or D (t) N for t = 1, 2,... A major practical difficulty with discrepancy is that it can be computed only for very special cases. For LCGs, for example, it can be computed efficiently in dimension t = 2, but for larger t, the computing cost then increases as O(N t ). In most cases, only (upper and lower) bounds on the discrepancy are available. Often, these bounds are expressed as orders of magnitude as a function of N, are defined for N = ρ, and/or are averages over a large (specific) class of generators (e.g., over all full-period MLCGs with a given prime modulus)

34 Structural methods Super-Uniformity and Discrepancy We previously argued for superuniformity over the entire period, which means seeking the lowest possible discrepancy. When a subsequence of length N is used (for N < ρ), starting, say, at a random point along the entire sequence, the discrepancy of that subsequence should behave (viewed as a random variable) as the discrepancy of a sequence of IID U(0, 1) random variables. The latter is (roughly) of order O(N 1/2 ) for both the star and extreme discrepancies.

35 Structural methods Super-Uniformity and Discrepancy Niederreiter shows that the discrepancy of full-period MLCGs over their entire period (of length ρ = m 1), on the average over multipliers a, is of order O(m 1 (log m) t log log(m + 1)). This order is much smaller (for large m) than O(m 1/2 ), meaning superuniformity. Over small fractions of the period length, the available bounds on the discrepancy are more in accordance with the law of the iterated logarithm. This is yet another important justification for never using more than a negligible fraction of the period.

36 Statistical methods Example I Suppose that one generates n random numbers from a generator whose output is supposed to imitate IID U(0, 1) random variables. Let T be the number of values that turn out to be below 1/2, among those n. For large n, T should normally be not too far from n/2... In fact, one should expect T to behave like a binomial random variable with parameters (n, 1/2). So if one repeats this experiment several times (e.g., generating N values of T), the distribution of the values of T obtained should resemble that of the binomial distribution (and the normal distribution with mean n/2 and standard deviation n/2 for large n).

37 Statistical methods Example II If N = 100 and n = 10000, the mean and standard deviation are 5000 and 50, respectively. With these parameters, if one observes, for instance, that 12 values of T are less than 4800, or that 98 values of T out of 100 are less than 5000, one world readily conclude that something is wrong with the generator. On the other hand, if the values of T behave as expected, one may conclude that the generator seems to reproduce the correct behavior for this particular statistic T (and for this particular sample size). But nothing prevents other statistics than this T to behave wrongly!

38 Statistical methods Test Statistics Define the null hypothesis H 0 as : The generator s output is a sequence of IID U(O, 1) random variables. Formally, this hypothesis is false, since the sequence is periodic and usually deterministic (except perhaps for the seed). But if this cannot be detected by reasonable statistical tests, one may assume that H 0 holds anyway. In fact, what really counts in the end is that the statistics of interest in a given simulation have (sample) distributions close enough to their theoretical ones.

39 Statistical methods Tests Statistics The most usual procedure is to devise some statistic whose distribution under the null could be obtained without too much trouble. Then, if extreme values of this statistic were observed, the null hypothesis of randomness is rejected. A statistical test for H 0 can be defined by any function T of a finite number of U(0, 1) random variables, for which the distribution under H 0 is either known or can be approximated well enough. The random variable T is called the test statistic. The statistical test tries to find empirical evidence against H 0.

40 Statistical methods Single-Level procedures When applying a statistical test to a random number generator, a single-level procedure computes the value of T, say t, then computes the p-value δ = P H0 [T > t H 0 ] (for a one-sided test) A single-sided test will reject only of δ is too close to 0, or only if it is too close to 1.

41 Statistical methods Two-levels procedures A two-level test obtains (say) N independent copies of T, denoted T 1,..., T N, and computes their empirical distribution ˆF N. This empirical distribution is then compared to the theoretical distribution of T under H 0, say F, via a standard goodness-of-fit test, such as the Kolmogorov-Smirnov (KS) or Anderson-Darling test. One version of the KS goodness-of-fit test uses the statistic D N = sup ˆF N (x) F (x), <x< for which an approximation of the distribution under H 0 is available, assuming that the distribution F is continuous. Once the value d N of the statistic D N is known, one computes the p-value of the test, defined as δ = P[D N > d N H 0 ] ; we reject H 0 if δ is too close to 0.

42 Statistical methods Which one is best? For a given test and a fixed computing budget, the question arises of what is best : to choose a small N (e.g., N = 1) and base the test statistic T on a large sample size, or the opposite? There is no universal winner. It depends on the test and on the alternative hypothesis. The rationale for two-level testing is to test the sequence not only globally, but also locally, by looking at the distribution of values of T over shorter subsequences. In most cases, when testing random number generators, N = 1 turns out to be the best choice because the same regularities or defects of the generators tend to repeat themselves over all long-enough subsequences. But it also happens for certain tests that the cost of computing T increases faster than linearly with the sample size, and this gives another argument for choosing N > 1.

43 Statistical methods Which are the best tests? Simply testing uniformity, or pair correlations, is far from enough. Good tests are designed to catch higher-order correlation properties or geometric patterns of the successive numbers. Such patterns can easily show up in certain classes of applications. If the generator is to be used to estimate the expectation of some random variable T by generating replicates of T, the best test would be the one based on T as a statistic. But this is impractical, since if one knew the distribution of T, one would not use simulation to estimate its mean.

44 Statistical methods Which are the best tests? Ideally, a good test for this kind of application should be based on a statistic T whose distribution is known and resembles that of T. But such a test is rarely easily available. Moreover, only the user can apply it. When designing a general purpose generator, one has no idea of what kind of random variable interests the user. So, the best the designer can do (after the generator has been properly designed) is to apply a wide variety of tests that tend to detect defects of different natures.

45 Statistical methods Batteries of tests The statistical tests described by Knuth have long been considered the standard tests for random number generators. A Fortran implementation of (roughly) this set of tests is given in the package TESTRAND. A newer battery of tests is DIEHARD, designed by Marsaglia. It contains more stringent tests, in the sense that more generators tend to fail some of the tests.

46 Statistical methods Die Hard test I Birthday Spacings : Choose random points on a large interval. The spacings between the points should be asymptotically Poisson distributed. The name is based on the birthday paradox. Overlapping Permutations : Analyze sequences of five consecutive random numbers. The 120 possible orderings should occur with statistically equal probability. Ranks of matrices : Select some number of bits from some number of random numbers to form a matrix over 0,1, then determine the rank of the matrix. Count the ranks. Monkey Tests : Treat sequences of some number of bits as words. Count the overlapping words in a stream. The number of words that don t appear should follow a known distribution. The name is based on the infinite monkey theorem.

47 Statistical methods Die Hard test II Count the 1 s : Count the 1 bits in each of either successive or chosen bytes. Convert the counts to letters, and count the occurrences of five-letter words. Parking Lot Test : Randomly place unit circles in a 100 x 100 square. If the circle overlaps an existing one, try again. After 12,000 tries, the number of successfully parked circles should follow a certain normal distribution. Minimum Distance Test : Randomly place 8,000 points in a 10,000 x 10,000 square, then find the minimum distance between the pairs. The square of this distance should be exponentially distributed with a certain mean. Random Spheres Test : Randomly choose 4,000 points in a cube of edge 1,000. Center a sphere on each point, whose radius is the minimum distance to another point. The smallest sphere s volume should be exponentially distributed with a certain mean.

48 Statistical methods Die Hard Test III The Squeeze Test : Multiply 2 31 by random floats on [0, 1[ until you reach 1. Repeat this 100, 000 times. The number of floats needed to reach 1 should follow a certain distribution. Overlapping Sums Test : Generate a long sequence of random floats on [0, 1[. Add sequences of 100 consecutive floats. The sums should be normally distributed with characteristic mean and sigma. Runs Test : Generate a long sequence of random floats on [0, 1]. Count ascending and descending runs. The counts should follow a certain distribution. The Craps Test : Play 200, 000 games of craps, counting the wins and the number of throws per game. Each count should follow a certain distribution.

### Portable Random Number Generators ABSTRACT

Portable Random Number Generators Gerald P. Dwyer, Jr.* a K. B. Williams b a. Research Department, Federal Reserve Bank of Atlanta, 104 Marietta St., N.W., Atlanta GA 30303 USA b. Melbourne Beach, Florida

More information

### 2WB05 Simulation Lecture 5: Random-number generators

2WB05 Simulation Lecture 5: Random-number generators Marko Boon http://www.win.tue.nl/courses/2wb05 December 6, 2012 Random-number generators It is important to be able to efficiently generate independent

More information

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.

More information

### Lecture 3: Finding integer solutions to systems of linear equations

Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture

More information

### (x + a) n = x n + a Z n [x]. Proof. If n is prime then the map

22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find

More information

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

More information

### , for x = 0, 1, 2, 3,... (4.1) (1 + 1/n) n = 2.71828... b x /x! = e b, x=0

Chapter 4 The Poisson Distribution 4.1 The Fish Distribution? The Poisson distribution is named after Simeon-Denis Poisson (1781 1840). In addition, poisson is French for fish. In this chapter we will

More information

### Elementary Number Theory We begin with a bit of elementary number theory, which is concerned

CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,

More information

### Advanced Algebra 2. I. Equations and Inequalities

Advanced Algebra 2 I. Equations and Inequalities A. Real Numbers and Number Operations 6.A.5, 6.B.5, 7.C.5 1) Graph numbers on a number line 2) Order real numbers 3) Identify properties of real numbers

More information

### N E W S A N D L E T T E R S

N E W S A N D L E T T E R S 73rd Annual William Lowell Putnam Mathematical Competition Editor s Note: Additional solutions will be printed in the Monthly later in the year. PROBLEMS A1. Let d 1, d,...,

More information

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

More information

### Collinear Points in Permutations

Collinear Points in Permutations Joshua N. Cooper Courant Institute of Mathematics New York University, New York, NY József Solymosi Department of Mathematics University of British Columbia, Vancouver,

More information

### Testing Random- Number Generators

Testing Random- Number Generators Raj Jain Washington University Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: http://www.cse.wustl.edu/~jain/cse574-08/

More information

### Random-Number Generation

Random-Number Generation Raj Jain Washington University Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: http://www.cse.wustl.edu/~jain/cse574-08/ 26-1

More information

### Notes on Determinant

ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 9-18/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without

More information

### 1 The Brownian bridge construction

The Brownian bridge construction The Brownian bridge construction is a way to build a Brownian motion path by successively adding finer scale detail. This construction leads to a relatively easy proof

More information

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

More information

### Overview of Math Standards

Algebra 2 Welcome to math curriculum design maps for Manhattan- Ogden USD 383, striving to produce learners who are: Effective Communicators who clearly express ideas and effectively communicate with diverse

More information

### Infinite Algebra 1 supports the teaching of the Common Core State Standards listed below.

Infinite Algebra 1 Kuta Software LLC Common Core Alignment Software version 2.05 Last revised July 2015 Infinite Algebra 1 supports the teaching of the Common Core State Standards listed below. High School

More information

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

More information

### > 2. Error and Computer Arithmetic

> 2. Error and Computer Arithmetic Numerical analysis is concerned with how to solve a problem numerically, i.e., how to develop a sequence of numerical calculations to get a satisfactory answer. Part

More information

### Chapter 10 Monte Carlo Methods

411 There is no result in nature without a cause; understand the cause and you will have no need for the experiment. Leonardo da Vinci (1452-1519) Chapter 10 Monte Carlo Methods In very broad terms one

More information

### Introduction to Statistics for Computer Science Projects

Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I

More information

### ORDERS OF ELEMENTS IN A GROUP

ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since

More information

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

More information

### 6 Scalar, Stochastic, Discrete Dynamic Systems

47 6 Scalar, Stochastic, Discrete Dynamic Systems Consider modeling a population of sand-hill cranes in year n by the first-order, deterministic recurrence equation y(n + 1) = Ry(n) where R = 1 + r = 1

More information

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

More information

### Linearizing Data. Lesson3. United States Population

Lesson3 Linearizing Data You may have heard that the population of the United States is increasing exponentially. The table and plot below give the population of the United States in the census years 19

More information

### Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras

Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding

More information

### Chapter II Binary Data Representation

Chapter II Binary Data Representation The atomic unit of data in computer systems is the bit, which is actually an acronym that stands for BInary digit. It can hold only 2 values or states: 0 or 1, true

More information

### Systems of Linear Equations

Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and

More information

### Current Standard: Mathematical Concepts and Applications Shape, Space, and Measurement- Primary

Shape, Space, and Measurement- Primary A student shall apply concepts of shape, space, and measurement to solve problems involving two- and three-dimensional shapes by demonstrating an understanding of:

More information

### CHAPTER 5 Round-off errors

CHAPTER 5 Round-off errors In the two previous chapters we have seen how numbers can be represented in the binary numeral system and how this is the basis for representing numbers in computers. Since any

More information

### CHAPTER 5. Number Theory. 1. Integers and Division. Discussion

CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a

More information

### Binary Number System. 16. Binary Numbers. Base 10 digits: 0 1 2 3 4 5 6 7 8 9. Base 2 digits: 0 1

Binary Number System 1 Base 10 digits: 0 1 2 3 4 5 6 7 8 9 Base 2 digits: 0 1 Recall that in base 10, the digits of a number are just coefficients of powers of the base (10): 417 = 4 * 10 2 + 1 * 10 1

More information

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

More information

### This unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.

Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course

More information

### Math 0980 Chapter Objectives. Chapter 1: Introduction to Algebra: The Integers.

Math 0980 Chapter Objectives Chapter 1: Introduction to Algebra: The Integers. 1. Identify the place value of a digit. 2. Write a number in words or digits. 3. Write positive and negative numbers used

More information

### Mathematical Induction

Chapter 2 Mathematical Induction 2.1 First Examples Suppose we want to find a simple formula for the sum of the first n odd numbers: 1 + 3 + 5 +... + (2n 1) = n (2k 1). How might we proceed? The most natural

More information

### Combinatorics: The Fine Art of Counting

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

More information

### PowerTeaching i3: Algebra I Mathematics

PowerTeaching i3: Algebra I Mathematics Alignment to the Common Core State Standards for Mathematics Standards for Mathematical Practice and Standards for Mathematical Content for Algebra I Key Ideas and

More information

### Mathematics of Cryptography

CHAPTER 2 Mathematics of Cryptography Part I: Modular Arithmetic, Congruence, and Matrices Objectives This chapter is intended to prepare the reader for the next few chapters in cryptography. The chapter

More information

### Linear Algebra I. Ronald van Luijk, 2012

Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.

More information

### FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL

FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint

More information

### MODULAR ARITHMETIC KEITH CONRAD

MODULAR ARITHMETIC KEITH CONRAD. Introduction We will define the notion of congruent integers (with respect to a modulus) and develop some basic ideas of modular arithmetic. Applications of modular arithmetic

More information

### 7 Hypothesis testing - one sample tests

7 Hypothesis testing - one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X

More information

### PYTHAGOREAN TRIPLES KEITH CONRAD

PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient

More information

### High School Algebra 1 Common Core Standards & Learning Targets

High School Algebra 1 Common Core Standards & Learning Targets Unit 1: Relationships between Quantities and Reasoning with Equations CCS Standards: Quantities N-Q.1. Use units as a way to understand problems

More information

### 8 Primes and Modular Arithmetic

8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.

More information

### Solution of Linear Systems

Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start

More information

### 1 Review of Least Squares Solutions to Overdetermined Systems

cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares

More information

### Common Core State Standards for Mathematics Accelerated 7th Grade

A Correlation of 2013 To the to the Introduction This document demonstrates how Mathematics Accelerated Grade 7, 2013, meets the. Correlation references are to the pages within the Student Edition. Meeting

More information

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

More information

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

More information

### MyMathLab ecourse for Developmental Mathematics

MyMathLab ecourse for Developmental Mathematics, North Shore Community College, University of New Orleans, Orange Coast College, Normandale Community College Table of Contents Module 1: Whole Numbers and

More information

### Senior Secondary Australian Curriculum

Senior Secondary Australian Curriculum Mathematical Methods Glossary Unit 1 Functions and graphs Asymptote A line is an asymptote to a curve if the distance between the line and the curve approaches zero

More information

### Prime Numbers. Chapter Primes and Composites

Chapter 2 Prime Numbers The term factoring or factorization refers to the process of expressing an integer as the product of two or more integers in a nontrivial way, e.g., 42 = 6 7. Prime numbers are

More information

### THE NUMBER OF REPRESENTATIONS OF n OF THE FORM n = x 2 2 y, x > 0, y 0

THE NUMBER OF REPRESENTATIONS OF n OF THE FORM n = x 2 2 y, x > 0, y 0 RICHARD J. MATHAR Abstract. We count solutions to the Ramanujan-Nagell equation 2 y +n = x 2 for fixed positive n. The computational

More information

### INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS

INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem

More information

### An Introduction to Galois Fields and Reed-Solomon Coding

An Introduction to Galois Fields and Reed-Solomon Coding James Westall James Martin School of Computing Clemson University Clemson, SC 29634-1906 October 4, 2010 1 Fields A field is a set of elements on

More information

### Algebraic and Transcendental Numbers

Pondicherry University July 2000 Algebraic and Transcendental Numbers Stéphane Fischler This text is meant to be an introduction to algebraic and transcendental numbers. For a detailed (though elementary)

More information

### Integer Factorization using the Quadratic Sieve

Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give

More information

### Notes for STA 437/1005 Methods for Multivariate Data

Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.

More information

### Six Sigma: Sample Size Determination and Simple Design of Experiments

Six Sigma: Sample Size Determination and Simple Design of Experiments Short Examples Series using Risk Simulator For more information please visit: www.realoptionsvaluation.com or contact us at: admin@realoptionsvaluation.com

More information

### Binomial lattice model for stock prices

Copyright c 2007 by Karl Sigman Binomial lattice model for stock prices Here we model the price of a stock in discrete time by a Markov chain of the recursive form S n+ S n Y n+, n 0, where the {Y i }

More information

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

More information

### Fractions and Decimals

Fractions and Decimals Tom Davis tomrdavis@earthlink.net http://www.geometer.org/mathcircles December 1, 2005 1 Introduction If you divide 1 by 81, you will find that 1/81 =.012345679012345679... The first

More information

### The Advantage Testing Foundation Solutions

The Advantage Testing Foundation 013 Problem 1 The figure below shows two equilateral triangles each with area 1. The intersection of the two triangles is a regular hexagon. What is the area of the union

More information

### Montana Common Core Standard

Algebra I Grade Level: 9, 10, 11, 12 Length: 1 Year Period(s) Per Day: 1 Credit: 1 Credit Requirement Fulfilled: A must pass course Course Description This course covers the real number system, solving

More information

### PROBLEM SET 7: PIGEON HOLE PRINCIPLE

PROBLEM SET 7: PIGEON HOLE PRINCIPLE The pigeonhole principle is the following observation: Theorem. Suppose that > kn marbles are distributed over n jars, then one jar will contain at least k + marbles.

More information

### 3.6: General Hypothesis Tests

3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.

More information

### TRANSCRIPT: In this lecture, we will talk about both theoretical and applied concepts related to hypothesis testing.

This is Dr. Chumney. The focus of this lecture is hypothesis testing both what it is, how hypothesis tests are used, and how to conduct hypothesis tests. 1 In this lecture, we will talk about both theoretical

More information

### 9.2 Summation Notation

9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a

More information

### Numerical Methods for Option Pricing

Chapter 9 Numerical Methods for Option Pricing Equation (8.26) provides a way to evaluate option prices. For some simple options, such as the European call and put options, one can integrate (8.26) directly

More information

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

More information

### Introduction Number Systems and Conversion

UNIT 1 Introduction Number Systems and Conversion Objectives 1. Introduction The first part of this unit introduces the material to be studied later. In addition to getting an overview of the material

More information

### This Unit: Floating Point Arithmetic. CIS 371 Computer Organization and Design. Readings. Floating Point (FP) Numbers

This Unit: Floating Point Arithmetic CIS 371 Computer Organization and Design Unit 7: Floating Point App App App System software Mem CPU I/O Formats Precision and range IEEE 754 standard Operations Addition

More information

### C programming: exercise sheet L2-STUE (2011-2012)

C programming: exercise sheet L2-STUE (2011-2012) Algorithms and Flowcharts Exercise 1: comparison Write the flowchart and associated algorithm that compare two numbers a and b. Exercise 2: 2 nd order

More information

### Introduction to Flocking {Stochastic Matrices}

Supelec EECI Graduate School in Control Introduction to Flocking {Stochastic Matrices} A. S. Morse Yale University Gif sur - Yvette May 21, 2012 CRAIG REYNOLDS - 1987 BOIDS The Lion King CRAIG REYNOLDS

More information

### Introduction. The Aims & Objectives of the Mathematical Portion of the IBA Entry Test

Introduction The career world is competitive. The competition and the opportunities in the career world become a serious problem for students if they do not do well in Mathematics, because then they are

More information

### A Non-Linear Schema Theorem for Genetic Algorithms

A Non-Linear Schema Theorem for Genetic Algorithms William A Greene Computer Science Department University of New Orleans New Orleans, LA 70148 bill@csunoedu 504-280-6755 Abstract We generalize Holland

More information

### OPENING THE BLACK BOX OF RANDOM NUMBERS

International Journal of Pure and Applied Mathematics Volume 81 No. 6 2012, 831-839 ISSN: 1311-8080 (printed version) url: http://www.ijpam.eu PA ijpam.eu OPENING THE BLACK BOX OF RANDOM NUMBERS Yutaka

More information

### Cryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur

Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards

More information

### SUBGROUPS OF CYCLIC GROUPS. 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by

SUBGROUPS OF CYCLIC GROUPS KEITH CONRAD 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by g = {g k : k Z}. If G = g, then G itself is cyclic, with g as a generator. Examples

More information

### Lecture 9: Random Walk Models Steven Skiena. skiena

Lecture 9: Random Walk Models Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Financial Time Series as Random Walks

More information

### Math Workshop October 2010 Fractions and Repeating Decimals

Math Workshop October 2010 Fractions and Repeating Decimals This evening we will investigate the patterns that arise when converting fractions to decimals. As an example of what we will be looking at,

More information

### Faster deterministic integer factorisation

David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers

More information

### MATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.

MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column

More information

### Lecture 13 - Basic Number Theory.

Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted

More information

### Introduction to Monte-Carlo Methods

Introduction to Monte-Carlo Methods Bernard Lapeyre Halmstad January 2007 Monte-Carlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities

More information

### MTH304: Honors Algebra II

MTH304: Honors Algebra II This course builds upon algebraic concepts covered in Algebra. Students extend their knowledge and understanding by solving open-ended problems and thinking critically. Topics

More information

### Unit 2: Number Systems, Codes and Logic Functions

Unit 2: Number Systems, Codes and Logic Functions Introduction A digital computer manipulates discrete elements of data and that these elements are represented in the binary forms. Operands used for calculations

More information

### 4. Joint Distributions

Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose

More information

### Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis

Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis 2. Polar coordinates A point P in a polar coordinate system is represented by an ordered pair of numbers (r, θ). If r >

More information

### Summary of Formulas and Concepts. Descriptive Statistics (Ch. 1-4)

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

More information

### Vocabulary Words and Definitions for Algebra

Name: Period: Vocabulary Words and s for Algebra Absolute Value Additive Inverse Algebraic Expression Ascending Order Associative Property Axis of Symmetry Base Binomial Coefficient Combine Like Terms

More information

### MATH 095, College Prep Mathematics: Unit Coverage Pre-algebra topics (arithmetic skills) offered through BSE (Basic Skills Education)

MATH 095, College Prep Mathematics: Unit Coverage Pre-algebra topics (arithmetic skills) offered through BSE (Basic Skills Education) Accurately add, subtract, multiply, and divide whole numbers, integers,

More information

### The Laws of Cryptography Cryptographers Favorite Algorithms

2 The Laws of Cryptography Cryptographers Favorite Algorithms 2.1 The Extended Euclidean Algorithm. The previous section introduced the field known as the integers mod p, denoted or. Most of the field

More information

### Operation Count; Numerical Linear Algebra

10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floating-point

More information

### South Carolina College- and Career-Ready (SCCCR) Pre-Calculus

South Carolina College- and Career-Ready (SCCCR) Pre-Calculus Key Concepts Arithmetic with Polynomials and Rational Expressions PC.AAPR.2 PC.AAPR.3 PC.AAPR.4 PC.AAPR.5 PC.AAPR.6 PC.AAPR.7 Standards Know

More information