General Principles in Random Variates Generation


 Rebecca Nash
 1 years ago
 Views:
Transcription
1 General Principles in Random Variates Generation E. Moulines Ecole Nationale Supérieure des Télécommunications 20 mai 2007
2 Why use random numbers? Random numbers lie at the heart of many scientific and financial computations. Many important analytically intractable problems can be solved using Monte Carlo techniques. Many problems that can be solved by other computational methods, can be solved much more efficiently using Monte Carlo techniques
3 Why use random numbers Random numbers are the nuts and bolts of simulation. Typically, all the randomness required by the model is simulated by a random number generator whose output is assumed to be a sequence of independent and identically distributed U(0, 1) random variables These random numbers are then transformed as needed to simulation random variables from different probability distributions, such as the normal, exponential, Poisson, etc...
4 Real source of randomness To draw a winning number for several million dollars lottery, people would generally not trust a computer! They would rather prefer a simple physical system that they understand well, such as drawing balls from a container to select successive digits. Even this requires many precautions : the balls must have identical weights and sizes, be well mixed, and changed regularly to reduce the chances that some numbers come out more frequently in the long run... Such procedure is clearly impractical for computer simulations, which often requires millions of random numbers!
5 Real source of randomness on a computer User movements (keyboard strikes, mouse). hard disk / network activity. Processor load, Cursor position, the row currently displayed. Some kernel parameters ( /dev/random )
6 Why use pseudorandom numbers rather than real RN s? As of today, the most convenient and most reliable way of generating the random numbers for stochastic simulations appears to be via deterministic algorithms with a solid mathematical basis. These algorithms produce a sequence of numbers which are in fact not random at all, but seem to behave like independent random numbers ; that is, like a realization of a sequence of IID U(0, 1) random variables Much more convenient to use pseudorandom numbers. Don t need radiation sources or lava lamps. Pseudorandom numbers can be generated on the fly and the sequences are reproducible.
7 PseudoRandom generator A (pseudo)random number generator is a structure G = (S, s 0, T, U, G), where S is a finite set of states, s 0 S is the initial state (or seed), the mapping T : S S is the transition function, U is a finite set of output symbols, G : S U is the output function.
8 Basic Algorithm The state of the generator is initially s 0 and evolves according to the recurrence s n = T (s n 1 ), for n = 1, 2, 3,.... At step n, the generator outputs the number u n = G(s n ). The u n, n > 0 are the observations, and are also called the random numbers produced by the generator. Clearly, the sequence of states s n is eventually periodic, since the state space S is finite. Indeed, the generator must eventually revisit a state previously seen ; that is, s j = s i for some j > i > 0. From then on, one must have s j+n = s i+n and u j+n = u i+n for all n > 0.
9 Period The period length is the smallest integer ρ > 0 such that for some integer τ 0 and for all n τ, s ρ+n = s n. The smallest τ with this property is called the transient. Often, τ = 0 and the sequence is then called purely periodic. Note that the period length cannot exceed S, the cardinality of the state space. Good generators typically have their ρ very close to S (otherwise, there is a waste of computer memory).
10 Congruential Generators Linear Congruential Generator How to build a good random generator? How can one build a deterministic generator whose output looks totally random? Perhaps a first idea is to write a computer program more or less at random that can also modify its own code in an unpredictable way... However, experience shows that random number generators should not be built at random (see Knuth for more discussion on this). Building a good random number generator may look easy on the surface, but it is not. It requires a good understanding of heavy mathematics.
11 Congruential Generators Linear Congruential Generator A Linear Congruential Generator The bestknown and (still) most widely used types of generators are the simple linear congruential generators (LCGs). The state at step n is an integer x n and the transition function T is defined by the recurrence x n = (ax n 1 + c) mod m, where m > 0, a > 0, and c are integers called the modulus, the multiplier, and the additive constant. One can identify s n with x n and the state space S is the set {0,..., m 1}. To produce values in the interval [0, 1], one can simply define the output function G by u n = G(x n ) = x n /m.
12 Congruential Generators Linear Congruential Generator Multiplicative Linear Congruential Generator When c = 0, this generator is called a multiplicative linear congruential generator (MLCG). It is cheaper to implement because it eliminates the addition operation. The maximal period length for the LCG is m in general. For the MLCG it cannot exceed m 1, since 0 is an absorbing state that must be avoided. Two popular values of m are n = and m = In the latter case, the modulo operation may be carried out efficiently by shift operations (or by extracting bits or words if these operations are carried out using unsigned integer arithmetics). However, these values may be too small for the requirements of today s simulations.
13 Congruential Generators Linear Congruential Generator Period of Multiplicative MLCG If the modulus m is a prime number, the maximal period ρ = m 1 results if the multiplier a is a primitive root of m, i.e. a m = 1 mod m and a q 1 mod m for 0 < q < m 1. For this restriction of m and a, each positive integer in {0,..., M} appears exactly once in any sequence of M 1 consecutive numbers produced by this generator.
14 Congruential Generators Linear Congruential Generator Period of LCG Simple conditions are available ensuring that the generator has full period  i.e. that the number of distinct values generated from any seed x 0 is m 1. If c 0, these conditions are c and m are relatively prime every prime number that divides m divides a 1, a 1 is divisible by 4 if m is. As a simple consequence, we observe that if m is a power of two, the generator has full period if c is odd a = 4n + 1 for some integer n.
15 Congruential Generators Linear Congruential Generator A running example For a concrete illustration, let m = = , c = 0, and a = These parameters were originally proposed by Lewis. Take x 0 = Then x 1 = 16807x12345 mod m = u 1 = /m = , x 2 = 16807x mod m = u 2 = /m = , x 3 = 16807x mod m = u 3 = /m =
16 Congruential Generators Multiple Recursive Generators Principle Consider the linear recurrence x n = (a l x n a k x n k ) mod m, where the order k and the modulus m are positive integers, while the coefficients a 1,..., a k are integers in the range { (m 1),..., m 1}. Define Z m as the set {0, 1,..., m 1} on which operations are performed modulo m. The state at step n of the multiple recursive generator (MRG) is the vector s n = (x n,..., x n+k 1 ) Z k m. The output function can be defined simply by u n = G(s n ) = x n /m, which gives a value in [0, 1], or by a more refined transformation if a better resolution than 1/m is required.
17 Congruential Generators Multiple Recursive Generators Properties The characteristic polynomial P of the MRG is defined by P (z) = z k a 1 z k 1 a k z. The maximal period length of the MRG is ρ = m k 1 (for all initial seeds), reached if and only if m is prime and P is a primitive polynomial over Z m, identified here as the finite field with m elements.
18 Congruential Generators Multiple Recursive Generators Primitive roots Suppose that m is prime and let r = (m k 1)/(m 1). The polynomial P is primitive over Z m if and only if it satisfies the following conditions, where everything is assumed to be modulo m 1. [( 1) k+l a k ] (m l)/q = 1 for each prime factor q of m 1 2. z r mod P (z) = ( 1) k+l a k 3. z r/q mod P (z) has degree > 0 for each prime factor q of r, 1 < q < r.
19 Congruential Generators Multiple Recursive Generators Properties If m is not prime, the period length of the MRG has an upper bound typically much lower than m k 1. For k = 1 and m = 2 e, e > 4, the maximum period length is 2 e 2, which is reached if a 1 = 3 or 5( mod 8) and x 0 is odd. Otherwise, if m = p e for p prime and e > 1, and k > 1 or p > 2, the upper bound is (p k 1)p e 1.
20 Congruential Generators Multiple Recursive Generators Choice of m Clearly, p = 2 is very convenient from the implementation point of view, because the modulo operation then amounts to choppingoff the higherorder bits. So to compute ax mod m in that case, for example with e = 32 on a 32bit computer, just make sure that the overflowchecking option or the compiler is turned off, and compute the product ax using unsigned integers while ignoring the overflow. However, taking m = 2 e imposes a big sacrifice on the period length, especially for k > 1. For example, if k = 7 and in m = (a prime), the maximal period length is (2 31 1) But for m = 2 31 and the same value of k, the upper bound becomes ρ < (2 7 1) < 2 37, which is more than times shorter.
21 Congruential Generators Multiple Recursive Generators Implementation for prime m For k > 1 and prime m, for the characteristic polynomial P to be primitive, it is necessary that a k and at least another coefficient a j be nonzero. From the implementation point of view, it is best to have only two nonzero coefficients ; that is, a recurrence of the form x n = (a r x n r + a k x n k ) mod m with characteristic trinomial P defined by P (z) = z k a r z k r a k.
22 Congruential Generators Multiple Recursive Generators Overflow When m is not a power of 2, computing and adding the products modulo m is not necessarily straightforward, using ordinary integer arithmetic, because of the possibility of overflow. For example, if m = and a l = 16807, then x n 1 can be as large as , so the product a 1 x n 1 can easily exceed 2 31.
23 Congruential Generators Multiple Recursive Generators Floatingpoint arithmetic In this case, it is appropriate to represent all the numbers and perform all the arithmetic modulo m in doubleprecision floating point. This works provided that the multipliers a i are small enough so that the integers a i x n i and their sum are always represented exactly by the floatingpoint values. A sufficient condition is that the floatingpoint numbers are represented with at least [log 2 ((m 1)(a l + + a k ))] bits of precision in their mantissa, where [x] denotes the smallest integer larger or equal to x. On computers with good 64bit floatingpoint hardware (most computers nowadays), this approach usually gives by far the fastest implementation.
24 Objectives Define the error in a meaningful. Devise a procedure for choosing which of several candidate generators with, say, the same modulus, is best according to the specified error criterion. Reduce the error achievable by a particular congruential genera generator by modifying its structure in a way that preserves its computability, execution time, reproducibility property, and ease of implementation.
25 Structural / Statistical Methods The techniques used to evaluate the quality of random number generators can be partitioned into two main classes : The structural analysis methods (sometimes called theoretical tests) : studies the mathematical structure underlying the successive values produced by the generator, most often over its entire period length the statistical methods (also called empirical tests). It observes the output and applies a statistical test of hypothesis to catch up significant statistical defects. An unlimited number of such tests can be designed.
26 Structural methods Lattice A lattice of dimension t in the tdimensional real space R t, is a set of the form t L def = V = z j V j, z j Z, (1) j=1 where {V 1,..., V t } is a basis of R t. The lattice L is thus the set of all integer linear combinations of the vectors V 1,..., V t ; ese vectors are called a lattice basis of L. The basis {W 1,..., W t } which satisfies V i, W j = δ i,j is the dual basis, and the lattice generated by this dual basis is called the dual lattice to L.
27 Structural methods Lattice Consider the set T t T t def = {u n = (u n,..., u n+t 1 ), s 0 = (x 0,..., x k 1 ) Z k m} of all overlapping tuples of successive values produced by the MRG, with u n = x n /m, from all possible initial seeds. This set T t is the intersection of a lattice L t with the tdimnensional unit hypercube I t = [0, 1] t.
28 Structural methods Lattice Property of an MRG For t k, each vector (x 0,..., x t 1 ) Z t m can be taken as s 0, so T t = Z t m/m = (Z t /m) I t ; In dimension t > k, the set T t contains only m k points, while Z t m/m contains m t points. Therefore, for large t, T t contains only a small fraction of the tdimensional vectors whose coordinates are multiples of 1/m.
29 Structural methods (a) a = (b) a = (c) a = 51
30 Structural methods Spectral Distance When t k, the points also lie on a lattice L t composed of parallel hyperplanes. The spectral distance d t is the minimal distance between two successive hyperplanes of any family of parallel hyperplanes covering the lattice ; this distance may be shown to be the minimal norm of a nonzero vector belonging to the dual lattice. The shorter the distance d t, the better, because a large d t means thick empty slices. This spectral distance can be computed by solving a quadratic programming problem in integer random variables (which can be solved using branch and bound algorithms for example). Appropriate values of the multiplier have often been found by minimizing this spectral distance (for several values of t).
31 Structural methods Spectral distance The spectral distance is lower bounded by d t d t = 1 γ t m k/t, where γ t is a constant depending only on t. As an example, for m = t = t = 3 d t t = t = t = 6 (2) showing that, even for moderate value of t, the space between the hyperplanes can be pretty large!
32 Structural methods Discrepancy : Definition Consider the N points u n = (u n,..., u n+t 1 ), for n = 0,..., N 1, in dimension t, formed by (overlapping) vectors of t successive output values of the generator. For any hyperrectangular box aligned with the axes, of the form R = t j=1 [α j, β j [, with 0 α j < β j < 1, let I(R) be the number of points un falling into R, V (R) = t j=1 (β j α j ) be the volume of R. Let R be the set of all such regions R. The tdimensional (extreme) discrepancy of the set of points {u 0,..., u N 1 } is given by D (t) N = max V (R) I(R)/N. R R If we impose α j = 0 for all j, then the corresponding quantity is called the stardiscrepancy
33 Structural methods Why? A low discrepancy value means that the points are very evenly distributed in the unit hypercube. To get superuniformity of the sequence over its entire period, one might want to minimize the discrepancy D (t) N or D (t) N for t = 1, 2,... A major practical difficulty with discrepancy is that it can be computed only for very special cases. For LCGs, for example, it can be computed efficiently in dimension t = 2, but for larger t, the computing cost then increases as O(N t ). In most cases, only (upper and lower) bounds on the discrepancy are available. Often, these bounds are expressed as orders of magnitude as a function of N, are defined for N = ρ, and/or are averages over a large (specific) class of generators (e.g., over all fullperiod MLCGs with a given prime modulus)
34 Structural methods SuperUniformity and Discrepancy We previously argued for superuniformity over the entire period, which means seeking the lowest possible discrepancy. When a subsequence of length N is used (for N < ρ), starting, say, at a random point along the entire sequence, the discrepancy of that subsequence should behave (viewed as a random variable) as the discrepancy of a sequence of IID U(0, 1) random variables. The latter is (roughly) of order O(N 1/2 ) for both the star and extreme discrepancies.
35 Structural methods SuperUniformity and Discrepancy Niederreiter shows that the discrepancy of fullperiod MLCGs over their entire period (of length ρ = m 1), on the average over multipliers a, is of order O(m 1 (log m) t log log(m + 1)). This order is much smaller (for large m) than O(m 1/2 ), meaning superuniformity. Over small fractions of the period length, the available bounds on the discrepancy are more in accordance with the law of the iterated logarithm. This is yet another important justification for never using more than a negligible fraction of the period.
36 Statistical methods Example I Suppose that one generates n random numbers from a generator whose output is supposed to imitate IID U(0, 1) random variables. Let T be the number of values that turn out to be below 1/2, among those n. For large n, T should normally be not too far from n/2... In fact, one should expect T to behave like a binomial random variable with parameters (n, 1/2). So if one repeats this experiment several times (e.g., generating N values of T), the distribution of the values of T obtained should resemble that of the binomial distribution (and the normal distribution with mean n/2 and standard deviation n/2 for large n).
37 Statistical methods Example II If N = 100 and n = 10000, the mean and standard deviation are 5000 and 50, respectively. With these parameters, if one observes, for instance, that 12 values of T are less than 4800, or that 98 values of T out of 100 are less than 5000, one world readily conclude that something is wrong with the generator. On the other hand, if the values of T behave as expected, one may conclude that the generator seems to reproduce the correct behavior for this particular statistic T (and for this particular sample size). But nothing prevents other statistics than this T to behave wrongly!
38 Statistical methods Test Statistics Define the null hypothesis H 0 as : The generator s output is a sequence of IID U(O, 1) random variables. Formally, this hypothesis is false, since the sequence is periodic and usually deterministic (except perhaps for the seed). But if this cannot be detected by reasonable statistical tests, one may assume that H 0 holds anyway. In fact, what really counts in the end is that the statistics of interest in a given simulation have (sample) distributions close enough to their theoretical ones.
39 Statistical methods Tests Statistics The most usual procedure is to devise some statistic whose distribution under the null could be obtained without too much trouble. Then, if extreme values of this statistic were observed, the null hypothesis of randomness is rejected. A statistical test for H 0 can be defined by any function T of a finite number of U(0, 1) random variables, for which the distribution under H 0 is either known or can be approximated well enough. The random variable T is called the test statistic. The statistical test tries to find empirical evidence against H 0.
40 Statistical methods SingleLevel procedures When applying a statistical test to a random number generator, a singlelevel procedure computes the value of T, say t, then computes the pvalue δ = P H0 [T > t H 0 ] (for a onesided test) A singlesided test will reject only of δ is too close to 0, or only if it is too close to 1.
41 Statistical methods Twolevels procedures A twolevel test obtains (say) N independent copies of T, denoted T 1,..., T N, and computes their empirical distribution ˆF N. This empirical distribution is then compared to the theoretical distribution of T under H 0, say F, via a standard goodnessoffit test, such as the KolmogorovSmirnov (KS) or AndersonDarling test. One version of the KS goodnessoffit test uses the statistic D N = sup ˆF N (x) F (x), <x< for which an approximation of the distribution under H 0 is available, assuming that the distribution F is continuous. Once the value d N of the statistic D N is known, one computes the pvalue of the test, defined as δ = P[D N > d N H 0 ] ; we reject H 0 if δ is too close to 0.
42 Statistical methods Which one is best? For a given test and a fixed computing budget, the question arises of what is best : to choose a small N (e.g., N = 1) and base the test statistic T on a large sample size, or the opposite? There is no universal winner. It depends on the test and on the alternative hypothesis. The rationale for twolevel testing is to test the sequence not only globally, but also locally, by looking at the distribution of values of T over shorter subsequences. In most cases, when testing random number generators, N = 1 turns out to be the best choice because the same regularities or defects of the generators tend to repeat themselves over all longenough subsequences. But it also happens for certain tests that the cost of computing T increases faster than linearly with the sample size, and this gives another argument for choosing N > 1.
43 Statistical methods Which are the best tests? Simply testing uniformity, or pair correlations, is far from enough. Good tests are designed to catch higherorder correlation properties or geometric patterns of the successive numbers. Such patterns can easily show up in certain classes of applications. If the generator is to be used to estimate the expectation of some random variable T by generating replicates of T, the best test would be the one based on T as a statistic. But this is impractical, since if one knew the distribution of T, one would not use simulation to estimate its mean.
44 Statistical methods Which are the best tests? Ideally, a good test for this kind of application should be based on a statistic T whose distribution is known and resembles that of T. But such a test is rarely easily available. Moreover, only the user can apply it. When designing a general purpose generator, one has no idea of what kind of random variable interests the user. So, the best the designer can do (after the generator has been properly designed) is to apply a wide variety of tests that tend to detect defects of different natures.
45 Statistical methods Batteries of tests The statistical tests described by Knuth have long been considered the standard tests for random number generators. A Fortran implementation of (roughly) this set of tests is given in the package TESTRAND. A newer battery of tests is DIEHARD, designed by Marsaglia. It contains more stringent tests, in the sense that more generators tend to fail some of the tests.
46 Statistical methods Die Hard test I Birthday Spacings : Choose random points on a large interval. The spacings between the points should be asymptotically Poisson distributed. The name is based on the birthday paradox. Overlapping Permutations : Analyze sequences of five consecutive random numbers. The 120 possible orderings should occur with statistically equal probability. Ranks of matrices : Select some number of bits from some number of random numbers to form a matrix over 0,1, then determine the rank of the matrix. Count the ranks. Monkey Tests : Treat sequences of some number of bits as words. Count the overlapping words in a stream. The number of words that don t appear should follow a known distribution. The name is based on the infinite monkey theorem.
47 Statistical methods Die Hard test II Count the 1 s : Count the 1 bits in each of either successive or chosen bytes. Convert the counts to letters, and count the occurrences of fiveletter words. Parking Lot Test : Randomly place unit circles in a 100 x 100 square. If the circle overlaps an existing one, try again. After 12,000 tries, the number of successfully parked circles should follow a certain normal distribution. Minimum Distance Test : Randomly place 8,000 points in a 10,000 x 10,000 square, then find the minimum distance between the pairs. The square of this distance should be exponentially distributed with a certain mean. Random Spheres Test : Randomly choose 4,000 points in a cube of edge 1,000. Center a sphere on each point, whose radius is the minimum distance to another point. The smallest sphere s volume should be exponentially distributed with a certain mean.
48 Statistical methods Die Hard Test III The Squeeze Test : Multiply 2 31 by random floats on [0, 1[ until you reach 1. Repeat this 100, 000 times. The number of floats needed to reach 1 should follow a certain distribution. Overlapping Sums Test : Generate a long sequence of random floats on [0, 1[. Add sequences of 100 consecutive floats. The sums should be normally distributed with characteristic mean and sigma. Runs Test : Generate a long sequence of random floats on [0, 1]. Count ascending and descending runs. The counts should follow a certain distribution. The Craps Test : Play 200, 000 games of craps, counting the wins and the number of throws per game. Each count should follow a certain distribution.
Portable Random Number Generators ABSTRACT
Portable Random Number Generators Gerald P. Dwyer, Jr.* a K. B. Williams b a. Research Department, Federal Reserve Bank of Atlanta, 104 Marietta St., N.W., Atlanta GA 30303 USA b. Melbourne Beach, Florida
More information2WB05 Simulation Lecture 5: Randomnumber generators
2WB05 Simulation Lecture 5: Randomnumber generators Marko Boon http://www.win.tue.nl/courses/2wb05 December 6, 2012 Randomnumber generators It is important to be able to efficiently generate independent
More informationAlgebra 2 Chapter 1 Vocabulary. identity  A statement that equates two equivalent expressions.
Chapter 1 Vocabulary identity  A statement that equates two equivalent expressions. verbal model A word equation that represents a reallife problem. algebraic expression  An expression with variables.
More informationLecture 3: Finding integer solutions to systems of linear equations
Lecture 3: Finding integer solutions to systems of linear equations Algorithmic Number Theory (Fall 2014) Rutgers University Swastik Kopparty Scribe: Abhishek Bhrushundi 1 Overview The goal of this lecture
More information(x + a) n = x n + a Z n [x]. Proof. If n is prime then the map
22. A quick primality test Prime numbers are one of the most basic objects in mathematics and one of the most basic questions is to decide which numbers are prime (a clearly related problem is to find
More informationWhat is Linear Programming?
Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to
More information, for x = 0, 1, 2, 3,... (4.1) (1 + 1/n) n = 2.71828... b x /x! = e b, x=0
Chapter 4 The Poisson Distribution 4.1 The Fish Distribution? The Poisson distribution is named after SimeonDenis Poisson (1781 1840). In addition, poisson is French for fish. In this chapter we will
More informationElementary Number Theory We begin with a bit of elementary number theory, which is concerned
CONSTRUCTION OF THE FINITE FIELDS Z p S. R. DOTY Elementary Number Theory We begin with a bit of elementary number theory, which is concerned solely with questions about the set of integers Z = {0, ±1,
More informationAdvanced Algebra 2. I. Equations and Inequalities
Advanced Algebra 2 I. Equations and Inequalities A. Real Numbers and Number Operations 6.A.5, 6.B.5, 7.C.5 1) Graph numbers on a number line 2) Order real numbers 3) Identify properties of real numbers
More informationN E W S A N D L E T T E R S
N E W S A N D L E T T E R S 73rd Annual William Lowell Putnam Mathematical Competition Editor s Note: Additional solutions will be printed in the Monthly later in the year. PROBLEMS A1. Let d 1, d,...,
More informationChapter 15 Introduction to Linear Programming
Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 WeiTa Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of
More informationCollinear Points in Permutations
Collinear Points in Permutations Joshua N. Cooper Courant Institute of Mathematics New York University, New York, NY József Solymosi Department of Mathematics University of British Columbia, Vancouver,
More informationTesting Random Number Generators
Testing Random Number Generators Raj Jain Washington University Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: http://www.cse.wustl.edu/~jain/cse57408/
More informationRandomNumber Generation
RandomNumber Generation Raj Jain Washington University Saint Louis, MO 63130 Jain@cse.wustl.edu Audio/Video recordings of this lecture are available at: http://www.cse.wustl.edu/~jain/cse57408/ 261
More informationNotes on Determinant
ENGG2012B Advanced Engineering Mathematics Notes on Determinant Lecturer: Kenneth Shum Lecture 918/02/2013 The determinant of a system of linear equations determines whether the solution is unique, without
More information1 The Brownian bridge construction
The Brownian bridge construction The Brownian bridge construction is a way to build a Brownian motion path by successively adding finer scale detail. This construction leads to a relatively easy proof
More informationContinued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
More informationOverview of Math Standards
Algebra 2 Welcome to math curriculum design maps for Manhattan Ogden USD 383, striving to produce learners who are: Effective Communicators who clearly express ideas and effectively communicate with diverse
More informationInfinite Algebra 1 supports the teaching of the Common Core State Standards listed below.
Infinite Algebra 1 Kuta Software LLC Common Core Alignment Software version 2.05 Last revised July 2015 Infinite Algebra 1 supports the teaching of the Common Core State Standards listed below. High School
More information1 Solving LPs: The Simplex Algorithm of George Dantzig
Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.
More information> 2. Error and Computer Arithmetic
> 2. Error and Computer Arithmetic Numerical analysis is concerned with how to solve a problem numerically, i.e., how to develop a sequence of numerical calculations to get a satisfactory answer. Part
More informationChapter 10 Monte Carlo Methods
411 There is no result in nature without a cause; understand the cause and you will have no need for the experiment. Leonardo da Vinci (14521519) Chapter 10 Monte Carlo Methods In very broad terms one
More informationIntroduction to Statistics for Computer Science Projects
Introduction Introduction to Statistics for Computer Science Projects Peter Coxhead Whole modules are devoted to statistics and related topics in many degree programmes, so in this short session all I
More informationORDERS OF ELEMENTS IN A GROUP
ORDERS OF ELEMENTS IN A GROUP KEITH CONRAD 1. Introduction Let G be a group and g G. We say g has finite order if g n = e for some positive integer n. For example, 1 and i have finite order in C, since
More informationVector and Matrix Norms
Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a nonempty
More information6 Scalar, Stochastic, Discrete Dynamic Systems
47 6 Scalar, Stochastic, Discrete Dynamic Systems Consider modeling a population of sandhill cranes in year n by the firstorder, deterministic recurrence equation y(n + 1) = Ry(n) where R = 1 + r = 1
More information1 Limiting distribution for a Markov chain
Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easytocheck
More informationLinearizing Data. Lesson3. United States Population
Lesson3 Linearizing Data You may have heard that the population of the United States is increasing exponentially. The table and plot below give the population of the United States in the census years 19
More informationTheory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras
Theory of Computation Prof. Kamala Krithivasan Department of Computer Science and Engineering Indian Institute of Technology, Madras Lecture No. # 31 Recursive Sets, Recursively Innumerable Sets, Encoding
More informationChapter II Binary Data Representation
Chapter II Binary Data Representation The atomic unit of data in computer systems is the bit, which is actually an acronym that stands for BInary digit. It can hold only 2 values or states: 0 or 1, true
More informationSystems of Linear Equations
Systems of Linear Equations Beifang Chen Systems of linear equations Linear systems A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where a, a,, a n and
More informationCurrent Standard: Mathematical Concepts and Applications Shape, Space, and Measurement Primary
Shape, Space, and Measurement Primary A student shall apply concepts of shape, space, and measurement to solve problems involving two and threedimensional shapes by demonstrating an understanding of:
More informationCHAPTER 5 Roundoff errors
CHAPTER 5 Roundoff errors In the two previous chapters we have seen how numbers can be represented in the binary numeral system and how this is the basis for representing numbers in computers. Since any
More informationCHAPTER 5. Number Theory. 1. Integers and Division. Discussion
CHAPTER 5 Number Theory 1. Integers and Division 1.1. Divisibility. Definition 1.1.1. Given two integers a and b we say a divides b if there is an integer c such that b = ac. If a divides b, we write a
More informationBinary Number System. 16. Binary Numbers. Base 10 digits: 0 1 2 3 4 5 6 7 8 9. Base 2 digits: 0 1
Binary Number System 1 Base 10 digits: 0 1 2 3 4 5 6 7 8 9 Base 2 digits: 0 1 Recall that in base 10, the digits of a number are just coefficients of powers of the base (10): 417 = 4 * 10 2 + 1 * 10 1
More informationa 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given
More informationThis unit will lay the groundwork for later units where the students will extend this knowledge to quadratic and exponential functions.
Algebra I Overview View unit yearlong overview here Many of the concepts presented in Algebra I are progressions of concepts that were introduced in grades 6 through 8. The content presented in this course
More informationMath 0980 Chapter Objectives. Chapter 1: Introduction to Algebra: The Integers.
Math 0980 Chapter Objectives Chapter 1: Introduction to Algebra: The Integers. 1. Identify the place value of a digit. 2. Write a number in words or digits. 3. Write positive and negative numbers used
More informationMathematical Induction
Chapter 2 Mathematical Induction 2.1 First Examples Suppose we want to find a simple formula for the sum of the first n odd numbers: 1 + 3 + 5 +... + (2n 1) = n (2k 1). How might we proceed? The most natural
More informationCombinatorics: The Fine Art of Counting
Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli
More informationPowerTeaching i3: Algebra I Mathematics
PowerTeaching i3: Algebra I Mathematics Alignment to the Common Core State Standards for Mathematics Standards for Mathematical Practice and Standards for Mathematical Content for Algebra I Key Ideas and
More informationMathematics of Cryptography
CHAPTER 2 Mathematics of Cryptography Part I: Modular Arithmetic, Congruence, and Matrices Objectives This chapter is intended to prepare the reader for the next few chapters in cryptography. The chapter
More informationLinear Algebra I. Ronald van Luijk, 2012
Linear Algebra I Ronald van Luijk, 2012 With many parts from Linear Algebra I by Michael Stoll, 2007 Contents 1. Vector spaces 3 1.1. Examples 3 1.2. Fields 4 1.3. The field of complex numbers. 6 1.4.
More informationFEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL
FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint
More informationMODULAR ARITHMETIC KEITH CONRAD
MODULAR ARITHMETIC KEITH CONRAD. Introduction We will define the notion of congruent integers (with respect to a modulus) and develop some basic ideas of modular arithmetic. Applications of modular arithmetic
More information7 Hypothesis testing  one sample tests
7 Hypothesis testing  one sample tests 7.1 Introduction Definition 7.1 A hypothesis is a statement about a population parameter. Example A hypothesis might be that the mean age of students taking MAS113X
More informationPYTHAGOREAN TRIPLES KEITH CONRAD
PYTHAGOREAN TRIPLES KEITH CONRAD 1. Introduction A Pythagorean triple is a triple of positive integers (a, b, c) where a + b = c. Examples include (3, 4, 5), (5, 1, 13), and (8, 15, 17). Below is an ancient
More informationHigh School Algebra 1 Common Core Standards & Learning Targets
High School Algebra 1 Common Core Standards & Learning Targets Unit 1: Relationships between Quantities and Reasoning with Equations CCS Standards: Quantities NQ.1. Use units as a way to understand problems
More information8 Primes and Modular Arithmetic
8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.
More informationSolution of Linear Systems
Chapter 3 Solution of Linear Systems In this chapter we study algorithms for possibly the most commonly occurring problem in scientific computing, the solution of linear systems of equations. We start
More information1 Review of Least Squares Solutions to Overdetermined Systems
cs4: introduction to numerical analysis /9/0 Lecture 7: Rectangular Systems and Numerical Integration Instructor: Professor Amos Ron Scribes: Mark Cowlishaw, Nathanael Fillmore Review of Least Squares
More informationCommon Core State Standards for Mathematics Accelerated 7th Grade
A Correlation of 2013 To the to the Introduction This document demonstrates how Mathematics Accelerated Grade 7, 2013, meets the. Correlation references are to the pages within the Student Edition. Meeting
More informationU.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra
U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationMyMathLab ecourse for Developmental Mathematics
MyMathLab ecourse for Developmental Mathematics, North Shore Community College, University of New Orleans, Orange Coast College, Normandale Community College Table of Contents Module 1: Whole Numbers and
More informationSenior Secondary Australian Curriculum
Senior Secondary Australian Curriculum Mathematical Methods Glossary Unit 1 Functions and graphs Asymptote A line is an asymptote to a curve if the distance between the line and the curve approaches zero
More informationPrime Numbers. Chapter Primes and Composites
Chapter 2 Prime Numbers The term factoring or factorization refers to the process of expressing an integer as the product of two or more integers in a nontrivial way, e.g., 42 = 6 7. Prime numbers are
More informationTHE NUMBER OF REPRESENTATIONS OF n OF THE FORM n = x 2 2 y, x > 0, y 0
THE NUMBER OF REPRESENTATIONS OF n OF THE FORM n = x 2 2 y, x > 0, y 0 RICHARD J. MATHAR Abstract. We count solutions to the RamanujanNagell equation 2 y +n = x 2 for fixed positive n. The computational
More informationINDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS
INDISTINGUISHABILITY OF ABSOLUTELY CONTINUOUS AND SINGULAR DISTRIBUTIONS STEVEN P. LALLEY AND ANDREW NOBEL Abstract. It is shown that there are no consistent decision rules for the hypothesis testing problem
More informationAn Introduction to Galois Fields and ReedSolomon Coding
An Introduction to Galois Fields and ReedSolomon Coding James Westall James Martin School of Computing Clemson University Clemson, SC 296341906 October 4, 2010 1 Fields A field is a set of elements on
More informationAlgebraic and Transcendental Numbers
Pondicherry University July 2000 Algebraic and Transcendental Numbers Stéphane Fischler This text is meant to be an introduction to algebraic and transcendental numbers. For a detailed (though elementary)
More informationInteger Factorization using the Quadratic Sieve
Integer Factorization using the Quadratic Sieve Chad Seibert* Division of Science and Mathematics University of Minnesota, Morris Morris, MN 56567 seib0060@morris.umn.edu March 16, 2011 Abstract We give
More informationNotes for STA 437/1005 Methods for Multivariate Data
Notes for STA 437/1005 Methods for Multivariate Data Radford M. Neal, 26 November 2010 Random Vectors Notation: Let X be a random vector with p elements, so that X = [X 1,..., X p ], where denotes transpose.
More informationSix Sigma: Sample Size Determination and Simple Design of Experiments
Six Sigma: Sample Size Determination and Simple Design of Experiments Short Examples Series using Risk Simulator For more information please visit: www.realoptionsvaluation.com or contact us at: admin@realoptionsvaluation.com
More informationBinomial lattice model for stock prices
Copyright c 2007 by Karl Sigman Binomial lattice model for stock prices Here we model the price of a stock in discrete time by a Markov chain of the recursive form S n+ S n Y n+, n 0, where the {Y i }
More informationFactoring & Primality
Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount
More informationFractions and Decimals
Fractions and Decimals Tom Davis tomrdavis@earthlink.net http://www.geometer.org/mathcircles December 1, 2005 1 Introduction If you divide 1 by 81, you will find that 1/81 =.012345679012345679... The first
More informationThe Advantage Testing Foundation Solutions
The Advantage Testing Foundation 013 Problem 1 The figure below shows two equilateral triangles each with area 1. The intersection of the two triangles is a regular hexagon. What is the area of the union
More informationMontana Common Core Standard
Algebra I Grade Level: 9, 10, 11, 12 Length: 1 Year Period(s) Per Day: 1 Credit: 1 Credit Requirement Fulfilled: A must pass course Course Description This course covers the real number system, solving
More informationPROBLEM SET 7: PIGEON HOLE PRINCIPLE
PROBLEM SET 7: PIGEON HOLE PRINCIPLE The pigeonhole principle is the following observation: Theorem. Suppose that > kn marbles are distributed over n jars, then one jar will contain at least k + marbles.
More information3.6: General Hypothesis Tests
3.6: General Hypothesis Tests The χ 2 goodness of fit tests which we introduced in the previous section were an example of a hypothesis test. In this section we now consider hypothesis tests more generally.
More informationTRANSCRIPT: In this lecture, we will talk about both theoretical and applied concepts related to hypothesis testing.
This is Dr. Chumney. The focus of this lecture is hypothesis testing both what it is, how hypothesis tests are used, and how to conduct hypothesis tests. 1 In this lecture, we will talk about both theoretical
More information9.2 Summation Notation
9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a
More informationNumerical Methods for Option Pricing
Chapter 9 Numerical Methods for Option Pricing Equation (8.26) provides a way to evaluate option prices. For some simple options, such as the European call and put options, one can integrate (8.26) directly
More informationCHAPTER 3 Numbers and Numeral Systems
CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,
More informationIntroduction Number Systems and Conversion
UNIT 1 Introduction Number Systems and Conversion Objectives 1. Introduction The first part of this unit introduces the material to be studied later. In addition to getting an overview of the material
More informationThis Unit: Floating Point Arithmetic. CIS 371 Computer Organization and Design. Readings. Floating Point (FP) Numbers
This Unit: Floating Point Arithmetic CIS 371 Computer Organization and Design Unit 7: Floating Point App App App System software Mem CPU I/O Formats Precision and range IEEE 754 standard Operations Addition
More informationC programming: exercise sheet L2STUE (20112012)
C programming: exercise sheet L2STUE (20112012) Algorithms and Flowcharts Exercise 1: comparison Write the flowchart and associated algorithm that compare two numbers a and b. Exercise 2: 2 nd order
More informationIntroduction to Flocking {Stochastic Matrices}
Supelec EECI Graduate School in Control Introduction to Flocking {Stochastic Matrices} A. S. Morse Yale University Gif sur  Yvette May 21, 2012 CRAIG REYNOLDS  1987 BOIDS The Lion King CRAIG REYNOLDS
More informationIntroduction. The Aims & Objectives of the Mathematical Portion of the IBA Entry Test
Introduction The career world is competitive. The competition and the opportunities in the career world become a serious problem for students if they do not do well in Mathematics, because then they are
More informationA NonLinear Schema Theorem for Genetic Algorithms
A NonLinear Schema Theorem for Genetic Algorithms William A Greene Computer Science Department University of New Orleans New Orleans, LA 70148 bill@csunoedu 5042806755 Abstract We generalize Holland
More informationOPENING THE BLACK BOX OF RANDOM NUMBERS
International Journal of Pure and Applied Mathematics Volume 81 No. 6 2012, 831839 ISSN: 13118080 (printed version) url: http://www.ijpam.eu PA ijpam.eu OPENING THE BLACK BOX OF RANDOM NUMBERS Yutaka
More informationCryptography and Network Security. Prof. D. Mukhopadhyay. Department of Computer Science and Engineering. Indian Institute of Technology, Kharagpur
Cryptography and Network Security Prof. D. Mukhopadhyay Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Module No. # 01 Lecture No. # 12 Block Cipher Standards
More informationSUBGROUPS OF CYCLIC GROUPS. 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by
SUBGROUPS OF CYCLIC GROUPS KEITH CONRAD 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by g = {g k : k Z}. If G = g, then G itself is cyclic, with g as a generator. Examples
More informationLecture 9: Random Walk Models Steven Skiena. skiena
Lecture 9: Random Walk Models Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Financial Time Series as Random Walks
More informationMath Workshop October 2010 Fractions and Repeating Decimals
Math Workshop October 2010 Fractions and Repeating Decimals This evening we will investigate the patterns that arise when converting fractions to decimals. As an example of what we will be looking at,
More informationFaster deterministic integer factorisation
David Harvey (joint work with Edgar Costa, NYU) University of New South Wales 25th October 2011 The obvious mathematical breakthrough would be the development of an easy way to factor large prime numbers
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An ndimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0534405967. Systems of Linear Equations Definition. An ndimensional vector is a row or a column
More informationLecture 13  Basic Number Theory.
Lecture 13  Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are nonnegative integers. We say that A divides B, denoted
More informationIntroduction to MonteCarlo Methods
Introduction to MonteCarlo Methods Bernard Lapeyre Halmstad January 2007 MonteCarlo methods are extensively used in financial institutions to compute European options prices to evaluate sensitivities
More informationMTH304: Honors Algebra II
MTH304: Honors Algebra II This course builds upon algebraic concepts covered in Algebra. Students extend their knowledge and understanding by solving openended problems and thinking critically. Topics
More informationUnit 2: Number Systems, Codes and Logic Functions
Unit 2: Number Systems, Codes and Logic Functions Introduction A digital computer manipulates discrete elements of data and that these elements are represented in the binary forms. Operands used for calculations
More information4. Joint Distributions
Virtual Laboratories > 2. Distributions > 1 2 3 4 5 6 7 8 4. Joint Distributions Basic Theory As usual, we start with a random experiment with probability measure P on an underlying sample space. Suppose
More informationChapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis
Chapter 5 Polar Coordinates; Vectors 5.1 Polar coordinates 1. Pole and polar axis 2. Polar coordinates A point P in a polar coordinate system is represented by an ordered pair of numbers (r, θ). If r >
More informationSummary of Formulas and Concepts. Descriptive Statistics (Ch. 14)
Summary of Formulas and Concepts Descriptive Statistics (Ch. 14) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume
More informationVocabulary Words and Definitions for Algebra
Name: Period: Vocabulary Words and s for Algebra Absolute Value Additive Inverse Algebraic Expression Ascending Order Associative Property Axis of Symmetry Base Binomial Coefficient Combine Like Terms
More informationMATH 095, College Prep Mathematics: Unit Coverage Prealgebra topics (arithmetic skills) offered through BSE (Basic Skills Education)
MATH 095, College Prep Mathematics: Unit Coverage Prealgebra topics (arithmetic skills) offered through BSE (Basic Skills Education) Accurately add, subtract, multiply, and divide whole numbers, integers,
More informationThe Laws of Cryptography Cryptographers Favorite Algorithms
2 The Laws of Cryptography Cryptographers Favorite Algorithms 2.1 The Extended Euclidean Algorithm. The previous section introduced the field known as the integers mod p, denoted or. Most of the field
More informationOperation Count; Numerical Linear Algebra
10 Operation Count; Numerical Linear Algebra 10.1 Introduction Many computations are limited simply by the sheer number of required additions, multiplications, or function evaluations. If floatingpoint
More informationSouth Carolina College and CareerReady (SCCCR) PreCalculus
South Carolina College and CareerReady (SCCCR) PreCalculus Key Concepts Arithmetic with Polynomials and Rational Expressions PC.AAPR.2 PC.AAPR.3 PC.AAPR.4 PC.AAPR.5 PC.AAPR.6 PC.AAPR.7 Standards Know
More information