Relating Turing's Formula and Zipf's Law
|
|
- Jocelyn Bradley
- 7 years ago
- Views:
Transcription
1 Relating Turing's Formula and Zipf's Law Christer Samuelsson Universit t des Saarlandes, FR 8.7, Computerlinguistik Postfach 50, D-6604 Saarbriicken, Germany chr ± st er~col i. uni- sb. de Abstract An asymptote is derived from Turing's local reestimation formula for population frequencies, and a local reestimation formula is derived from Zipf's law for the asymptotic behavior of population frequencies. The two are shown to be qualitatively different asymptotically, but nevertheless to be instances of a common class of reestimation-formula-asymptote pairs, in which they constitute the upper and lower bounds of the convergence region of the cumulative of the frequency function, as rank tends to infinity. The results demonstrate that Turing's formula is qualitatively different from the various etensions to Zipf's law, and suggest that it smooths the frequency estimates towards a geometric distribution. Introduction Turing's formula [Good 953] and Zipf's law [Zipf 935] indicate how population frequencies in general tend to behave. Turing's formula estimates locally what the frequency count of a species that occurred r times in a sample really would have been, had the sample accurately reflected the underlying population distribution. Zipf's law prescribes the asymptotic behavior of the relative frequencies of species as a function of their rank. The ranking scheme in question orders the species by frequency, with the most common species ranked first. The reason that these formulas are of interest in computational linguistics is that they can be used to improve probability estimates from relative frequencies, and to predict the frequencies of unseen phenomena, e.g., the frequency of previously unseen words encountered in running tet. Due to limitations in the amount of available training data, the so-called sparse-data problem, estimating probabilities directly from observed relative frequencies may not always be very accurate. For this reason, Turing's formula, in the incarnation of Katz's back-off scheme [Katz 987], has become a standard technique for improving parameter estimates for probabilistic language models used by speech recognizers. A more theoretical treatment of Turing's formula itseff can be found in [N das 985]. Zipf's law is commonly regarded as an empirically accurate description of a wide variety of (linguistic) phenomena, but too general to be of any direct use. For a bit of historic controversy on Zipf's law, we refer to [Simon 955], [Mandelbrot 959], and subsequent articles in Information and Control. The model presented there for the stochastic source generating the various Zipfian distributions is however linguistically highly dubious: a version of the monkeywith-typewriter scenario. The remainder if this article is organized as follows. In Section 2, we induce a recurrence equation from Turing's local reestimation formula and from this derive the asymptotic behavior of the relative frequency as a function of rank, using a continuum approimation. The resulting probability distribution is then eamined, and we rederive the recurrence equation from it. In Section 3, we start with the asymptotic behavior stipulated by Zipf's law, and derive a recurrence equation similar to that associated with Turing's formula, and from this 770
2 induce a corresponding reestimation formula. We then rederive the Zipfian asymptote from the established recurrence equation. In Section 4, similar techniques are used to establish the asymptotic behavior inherent in a general class of recurrence equations, parameterized by a real-valued parameter, and then to rederive the recurrence equations from their asymptotes. The convergence region of this parameter for the cumulative of the frequency function, as rank approaches infinity, is also investigated. In Section 5, we summarize the results, discuss how they might be used practically, and compare them with related work. 2 An Asymptote for Turing's Formula Turing's formula reestimates population frequencies locally: * = (+l). N.+~ N~ () Here N~ is the number of species with frequency count, and * is the improved estimate of. Let N be the size of the entire population and note that N = ~'N. and f~ = ~r ---- where is the count of the most populous species and f~ is the relative frequency of any species with frequency count. Let r() be the rank of the last species with frequency count. This means that quite in general r() = Z Nk k= 2. A continuum approimation i = ~ ik - ~ ik = r(~)- r( + ) (2) k= k=+l We first make a continuum approimation by etending N from the integer points =, 2,... to a continuous function N() on [, oo). This means that r() = ~Nk,~ ~ N(y)dy Differentiating this w.r.t, z, the lower bound of the integral, yields k---- dr() d f d - d N(y) dy = -N() and using the chain rule for differentiation yields dr dr d = d~'~ = -N(l.N (a) Continuum approimations are useful techniques for establishing the dependence of a sum on its bounds, to the leading term, and for determining convergence. For eample, if we wish to 7
3 n study the sum ~ in n 3 -- k 2, we note that the corresponding integral Jl 2 d = - - and conclude k--- 3 that the sum behaves like n 3. 2n 3 -F 3n 2 + n The eact formula is 6, so we in fact even got CO the leading coefficient right. Likewise, we can establish for what values of a the sum ~ k ~ converges by eplicitly calculating k=l fl a d = c~+l [~-~-ljl for o~ -~ - indicating that the integral, and thus the sum, converge for c~ < - and diverge for ~ > -. We have to be a bit careful with the transition to the continuous case. We will first let N become large and then establish what happens for small, but non-zero, values of ff = ~. So although will be small compared to N, it will be large compared to any constant C. This means that f = lim ~ = lim +C N-*oo N N--+co N for any additive constant C, and we may approimate + C with, motivating and similar approimations in the following. z+l 2.2 The asymptotic distribution For an ideal Turing population, we would have = z*. This gives us the recurrence equation Ig N=+i = + l " N= (4) implying that there are equally many inhabitants for frequency count as for frequency count +. This introduces several additional constraints, namely N= =. N and thus N= = N = 'N and thus f -.N N N (5) We are now prepared to derive the asymptotic behavior of the relative frequency f(r) of species as a function of their rank r implicit in Eq. (4). Combining Eq. (5) with Eq. (3) yields dr N N df - N(). N = ---. z N = - y This determines the rank r(f) as a function of the relative frequency f: Inverting this gives us the sought-for function f(r): r(f) = C-Nllnf (6) C--r /(r) = em- = C'e- r 72
4 - - Nlln(l+:) Utilizing the fact that the relative frequencies should be normalized to one, we find that and that thus "Turing's asymptotic law" is oo = f(r) dr = C'.Nle-~" r-- f(r) = ~-e "~ (7) Note that, reassuringly, the relative frequency of the most populous species, f, is preserved: f() = N N - f r-- Upon eamining the frequency function --N e ~, we realize that we have an eponential distribution with intensity parameter ~-, the probability of the most common species. This distribution was created by approimating our original discrete distribution with a continuous one. The discrete counterpart of an eponential distribution is a geometric distribution P(r) = p. (- p)r- r=l,2,... parameterized by p, the probability of some outcome occurring in one trial. P(r) can then be interpreted as the probability of waiting r trials for the first occurrence of the outcome. Thus, Turing's formula seems to be smoothing the frequency estimates towards a geometric distribution. 2.3 Rederiving Turing's formula To test our derivation of the asymptotic equation (7) from the recurrence equation (4), we will attempt to rederive Eq. (4) from Eq. (7). Since Eq. (7) implies Eq. (6), we start from the latter and establish that Inserting this into Eq. (2) yields r() = c - gl ln ~ z +l = NllnZ+l N~ = r(z)-r(z+l) = -N~lny+N~ln-N-- This means that We first note that < in:+! < +l - \ ] - We also note that the numerator can bewrittenasg(+l)-g()forg(y)= yln( +~), wmch in turn can be written as J g(y) dy, i.e., as ~, In
5 further note that if A < h(y) <_ B on (a, b), then A(b - a) < /b h(y) dy < B(b - a). Hence ( ) ( ~) l'+'( ( ) ) (+l)in i+.~-~-~ -ln I+ in 4~ l 'y ely 0 < = ~ - ( + )ln (+ ~) ( + )ln (+ ~) /.+(~ ) /~+' /'+' < J~ l+y dy = J y(l y) dy < dy < J V -- 2 < We have thus proved that ]N+l +l [ < ~ and since we assume that >>, this reestablishes Eq. (4) (to the second power of ~). 3 A Reestimation Formula for Zipf's Law Zipf's law concerns the asymptotic behavior of the relative frequencies f(r) of a population as a function of rank r. It states that, asymptotically, the relative frequency is inversely proportional to rank: A f(~) = B+~ (8) This implies a finite total population, since the cumulative (i.e, the sum or integral) of the relative frequency over rank does not converge as rank approaches infinity: lim F(r) ~ fi f(k) F(r) = k=~ f /(o) dp in the discrete case lim Aln(B+r) = c~ r---moo in the continuous case To localize Zipf's law, we utilize Eq. (2) and observe that r() = A' A t g~+: ~(~ + ) - ~( + 2) g ~(~) - ~(~ + ) A' A' +l This suggests "Zipf's local reestimation formula" * = (T + 2)" N+l N A m-b f() A I = ---B, ( + ). ( + 2) - +2 (9).(+l) which is deceptively similar to Turing's formula, Eq. (), the only difference being that it +2 assigns ~ more relative-frequency mass to frequency count. 3. Rederiving Zipf's law If we rederive the asymptotic behavior, we again obtain Zipf's law. Assuming the recurrence equation N~+~ = 74 ~.N~ +2 (0)
6 - C we have that C N+l = +---~'N~, = (+2)...3"N = (+2).(+l)'N ~, (+l)2 () We again use the equation for the derivative of the rank, Eq. (3), but now C I Integration yields r = 7 dr C C' = -N().N ~ -~.N = -f--~ + C" and function inversion f(r) = r - C I tt Identifying C ~ with A and C" with -B recovers Eq. (8). 4 A General Correspondence If we generalize the rederivation of Zipf's law in Eq. () to p = 2, 3,...,, we find that '..."! C N+l = +---~'N = (+p)...(l+p)'n = i.i~=l(k+p)'il,~, (+l)p C C' We integrate ~ to get r(f) - fp- + C", yielding a r-7:i" asymptote for f(r). Although a nontrivial generalization, it is in fact the case that for real-valued 0 : # 0 < z, results in the asymptote N+l = --'N~ (2) z+8 f(r) = Cr -o-i (3) The key observation here is that also for real-valued O < in general, z! C I-II_-l(k + o) ( + )o This means that we have a single reestimation equation * = ( + 0)- N+l (4) parameterized by the real-valued parameter O, with the asymptotic behavior Cr-~:r 0 # (5).f(r) = Ce 0= Although this correspondence was derived with the requirement that 0 <, we can in view of the discussion in Section 2. assume that is not only considerably larger than, but also greater than any fied value of 0. The etension to the negative real numbers is straightforward, although perhaps not very sensible. In fact, the convergence region for the cumulative of the frequency function as rank goes to infinity, oo f(r) or /(r) dr is 0 E [, 2), establishing Turing's formula and Zipf's law as the two etremes of this reestimation formula, in terms of resulting in a proper probability distribution for infinite populations; while the former does so, the latter does not. If 9 =, we have the Turing case with an eponentially declining asymptote, cf. Eq. (7). 75
7 4. Reversing the directions Finally, assuming the asymptotic behavior of Eq. (3), we rederive the recurrence equation (2). The mathematics are very similar to those used to rederive Turing's formula in Section 2.3. C t Inverting the asymptotic behavior f(r) = Cr-~=~ gives us r(f) - f0-, which in turn yields C tt r() - e- For notational convenience, let (2 denote 6 -, and assume that 0 < 0 #, i.e., - < (2 # O. ~+ +a+l ~ = I ~ ~(~+)-~(~+2)] (~+)- (~+2) +a+l 7(7-) = ;(TT T) = +(2+ ) (i ) (+l)~' --(+(2+l) (+l)a (+2)o ` _ (z+(2+ )(~ ( + )") + "b (2 +l ~ ( _._+_a (+2) a - (+l) "]- \(+l) a ~-a] (: : ) (+(2+l) ~ (+l) ~ ~ (z + )" y+(2 As before, the numerator can be written as g( + ) - g(), now for g(y) = (y + ) a ya- : I/ ~+(.- (a-- )y+ (22 -- ~ j. \ yo --T~-7-V)o-~--] dy./ ' [ (+a+)l~j-(+i). ) < I f~+l (2 dy (2- (~ (Y+l) ~ (y+l) ~+) +(2+ " " ( + )" I +l y+l (2 (2 +(2+l" [~+ a j~ y~+l dy /+l( )dy (2-., ~ (Y + ) a+i +(2+ [~+ Jz yc~+l dy (/S o+: ) (2- [ J v z~+2 dz dy + (2 + [+l J y~+l dy < < 76
8 / =+I < [a 2-[."z y,~+2dy < [a 2-[. _ < [~2_[ - + a + Jz/z+l ya+ll dy - + a This recaptures Eq. (2). Note that the derivation of Zipf's recurrence equation in Eq. (9) of Section 3 corresponds to the special case where a =, i.e., where 8 = 2. 5 Conclusions The relationship between Turing's formula and Zipf's law, which both concern population frequencies, was eplored in the present article. The asymptotic behavior of the relative frequency as a function of rank implicit in one interpretation of Turing's local reestimation formula was derived and compared with Zipf's law. While the latter relates the rank and relative frequency as asymptotically inversely proportional, the former states that the frequency declines eponentially with rank. This means that while Zipf's law implies a finite total population, Turing's formula yields a proper probability distribution also for infinite populations. In fact, it is tempting to interpret Turing's formula as smoothing the relative-frequency estimates towards a geometric distribution. This could potentially be used to improve sparsedata estimates by assuming a geometric distribution (tail), and introducing a ranking based on direct frequency counts, frequency counts when backing off to more general conditionings, order of appearance in the training data, or, to break any remaining ties, leicographical order. Conversely, a local reestimation formula in the vein of Turing's formula was derived from Zipf's law. Although the two equations are similar, Turing's formula shifts the frequency mass towards more frequent species. The two cases were generalized to a single spectrum of reestimarion formulas and corresponding asymptotes, parameterized by one real-valued parameter. Furthermore, the two cases correspond to the upper and lower bounds of this parameter for which the cumulative of the frequency function converges as rank tends to infinity. These results are in sharp contrast to common belief in the field; in [Baayen 99], for eample, we read: "Other models, such as Good (953)... have been put forward, all of which have Zipf's law as some special or limiting form." All of the Zipf-Simon-Mandelbrot distributions ehibit the same basic asymptotic behavior, C f(r) = r"'~ parameterized by the positive real-valued parameter 8- Comparing this with Eq. (5), we find that ~ - 8--Z- > 0 and thus 8 = + ~ >. In view of the established eponentially declining asymptote of the ideal Turing distribution, corresponding to 8 =, we can conclude that the latter is qualitatively different. Acknowledgements This article originated from inspiring discussions with David Milward and Slava Katz. Many thanks! Most of the work was done while the author was visiting IRCS at the University of Pennsylvania at the invitation of Aravind Joshi, and a number of New York pubs at the invitation of Jussi Karlgren, both of which was very much appreciated. I wish to thank Mark Lauer for helpful comments and suggestions to improvements, Seif Haridi for constituting the entire audience at a seminar on this work and focusing the question session on the convergence region of the parameter 0, and/~.ke Samuelsson for providing a bit of mathematical elegance. 77
9 I also gratefully acknowledge Rens Bod's encouraging comments and useful pointers to related work. Special credit is due to Mark Liberman for sharing his insights about Zipf's law, for drawing my attention to the Simon-Mandelbrot controversy, and for supplying various background material. This article, like others, has benefited greatly from comments by Khalil Sima'an. References [Baayen 99] Harald Baayen. 99. "A Stochastic Process for Word Frequency Distributions". In Proceedings of the 29th Annual Meeting of the Association for Computational Linguistics, pp , ACL. [Good 953] I. J. Good. "The Population Frequencies of Species and the Estimation of Population Parameters". In Biometrika ~0(3~4), pp , 953. [Katz 987] Slava M. Katz. "Estimation of Probabilities from Sparse Data for the Language Model Component of a Speech Recognizer". In IEEE Transactions on Acoustics, Speech, and Signal Processing 35(3), pp , 987. [N das 985] Arthur N das. "On Turing's Formula for Word Probabilities". In IEEE Transactions on Acoustics, Speech, and Signal Processing 33(6), pp , 985. [Zipf 935] G. K. Zipf. The Psychobiology off Language. Houghton Mifflin, Boston, 935. The Simon-Mandelbrot Dispute [Simon 955] Herbert A. Simon. "On a Class of Skew Distribution Functions". In Biometrika 42, pp (?), 953. [Mandelbrot 959] Benoit Mandelbrot. "A Note on a Class of Skew Distribution Functions: Analysis and Critique of a Paper by H. A. Simon". In Information and Control 2, pp , 959. [Simon 960] Herbert A. Simon. "Some Further Notes on a Class of Skew Distribution Functions". In Information and Control 3, pp , 960. [Mandelbrot 96] Benoit Mandelbrot. "Final Note on a Class of Skew Distribution Functions = Analysis and Critique of a Model due to H. A. Simon". In Information and Control 4, pp. 98-???, 96. [Simon 96] Herbert A. Simon. "Reply to 'Final Note' by Benoit Mandelbrot". In Information and Control 4, PP , 96. [Mandelbrot 96] Benoit Mandelbrot. "Post Scriptum to 'Final Note' ". In Information and Control 4, PP (?), 96. [Simon 96] Herbert A. Simon. "Reply to Dr. Mandelbrot's Post Scriptum". In Information and Control 4, PP ,
Random variables, probability distributions, binomial random variable
Week 4 lecture notes. WEEK 4 page 1 Random variables, probability distributions, binomial random variable Eample 1 : Consider the eperiment of flipping a fair coin three times. The number of tails that
More informationSection 1.3 P 1 = 1 2. = 1 4 2 8. P n = 1 P 3 = Continuing in this fashion, it should seem reasonable that, for any n = 1, 2, 3,..., = 1 2 4.
Difference Equations to Differential Equations Section. The Sum of a Sequence This section considers the problem of adding together the terms of a sequence. Of course, this is a problem only if more than
More informationOPTIMAL SELECTION BASED ON RELATIVE RANK* (the "Secretary Problem")
OPTIMAL SELECTION BASED ON RELATIVE RANK* (the "Secretary Problem") BY Y. S. CHOW, S. MORIGUTI, H. ROBBINS AND S. M. SAMUELS ABSTRACT n rankable persons appear sequentially in random order. At the ith
More informationOverview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model
Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written
More informationSection 3-7. Marginal Analysis in Business and Economics. Marginal Cost, Revenue, and Profit. 202 Chapter 3 The Derivative
202 Chapter 3 The Derivative Section 3-7 Marginal Analysis in Business and Economics Marginal Cost, Revenue, and Profit Application Marginal Average Cost, Revenue, and Profit Marginal Cost, Revenue, and
More informationExponential Functions. Exponential Functions and Their Graphs. Example 2. Example 1. Example 3. Graphs of Exponential Functions 9/17/2014
Eponential Functions Eponential Functions and Their Graphs Precalculus.1 Eample 1 Use a calculator to evaluate each function at the indicated value of. a) f ( ) 8 = Eample In the same coordinate place,
More information( ) is proportional to ( 10 + x)!2. Calculate the
PRACTICE EXAMINATION NUMBER 6. An insurance company eamines its pool of auto insurance customers and gathers the following information: i) All customers insure at least one car. ii) 64 of the customers
More informationMATH 4330/5330, Fourier Analysis Section 11, The Discrete Fourier Transform
MATH 433/533, Fourier Analysis Section 11, The Discrete Fourier Transform Now, instead of considering functions defined on a continuous domain, like the interval [, 1) or the whole real line R, we wish
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 5 9/17/2008 RANDOM VARIABLES Contents 1. Random variables and measurable functions 2. Cumulative distribution functions 3. Discrete
More informationCONTINUED FRACTIONS AND FACTORING. Niels Lauritzen
CONTINUED FRACTIONS AND FACTORING Niels Lauritzen ii NIELS LAURITZEN DEPARTMENT OF MATHEMATICAL SCIENCES UNIVERSITY OF AARHUS, DENMARK EMAIL: niels@imf.au.dk URL: http://home.imf.au.dk/niels/ Contents
More information1 if 1 x 0 1 if 0 x 1
Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or
More information9.2 Summation Notation
9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a
More information1 Error in Euler s Method
1 Error in Euler s Method Experience with Euler s 1 method raises some interesting questions about numerical approximations for the solutions of differential equations. 1. What determines the amount of
More informationLIMITS AND CONTINUITY
LIMITS AND CONTINUITY 1 The concept of it Eample 11 Let f() = 2 4 Eamine the behavior of f() as approaches 2 2 Solution Let us compute some values of f() for close to 2, as in the tables below We see from
More informationPeople have thought about, and defined, probability in different ways. important to note the consequences of the definition:
PROBABILITY AND LIKELIHOOD, A BRIEF INTRODUCTION IN SUPPORT OF A COURSE ON MOLECULAR EVOLUTION (BIOL 3046) Probability The subject of PROBABILITY is a branch of mathematics dedicated to building models
More informationReview of Fundamental Mathematics
Review of Fundamental Mathematics As explained in the Preface and in Chapter 1 of your textbook, managerial economics applies microeconomic theory to business decision making. The decision-making tools
More information7.7 Solving Rational Equations
Section 7.7 Solving Rational Equations 7 7.7 Solving Rational Equations When simplifying comple fractions in the previous section, we saw that multiplying both numerator and denominator by the appropriate
More informationA Coefficient of Variation for Skewed and Heavy-Tailed Insurance Losses. Michael R. Powers[ 1 ] Temple University and Tsinghua University
A Coefficient of Variation for Skewed and Heavy-Tailed Insurance Losses Michael R. Powers[ ] Temple University and Tsinghua University Thomas Y. Powers Yale University [June 2009] Abstract We propose a
More information10.2 Series and Convergence
10.2 Series and Convergence Write sums using sigma notation Find the partial sums of series and determine convergence or divergence of infinite series Find the N th partial sums of geometric series and
More informationPROPERTIES OF ELLIPTIC CURVES AND THEIR USE IN FACTORING LARGE NUMBERS
PROPERTIES OF ELLIPTIC CURVES AND THEIR USE IN FACTORING LARGE NUMBERS A ver important set of curves which has received considerabl attention in recent ears in connection with the factoring of large numbers
More informationChapter 4. Polynomial and Rational Functions. 4.1 Polynomial Functions and Their Graphs
Chapter 4. Polynomial and Rational Functions 4.1 Polynomial Functions and Their Graphs A polynomial function of degree n is a function of the form P = a n n + a n 1 n 1 + + a 2 2 + a 1 + a 0 Where a s
More informationMATH10212 Linear Algebra. Systems of Linear Equations. Definition. An n-dimensional vector is a row or a column of n numbers (or letters): a 1.
MATH10212 Linear Algebra Textbook: D. Poole, Linear Algebra: A Modern Introduction. Thompson, 2006. ISBN 0-534-40596-7. Systems of Linear Equations Definition. An n-dimensional vector is a row or a column
More informationProperties of sequences Since a sequence is a special kind of function it has analogous properties to functions:
Sequences and Series A sequence is a special kind of function whose domain is N - the set of natural numbers. The range of a sequence is the collection of terms that make up the sequence. Just as the word
More informationSequences and Series
Sequences and Series Consider the following sum: 2 + 4 + 8 + 6 + + 2 i + The dots at the end indicate that the sum goes on forever. Does this make sense? Can we assign a numerical value to an infinite
More informationChapter 4 One Dimensional Kinematics
Chapter 4 One Dimensional Kinematics 41 Introduction 1 4 Position, Time Interval, Displacement 41 Position 4 Time Interval 43 Displacement 43 Velocity 3 431 Average Velocity 3 433 Instantaneous Velocity
More informationBy choosing to view this document, you agree to all provisions of the copyright laws protecting it.
This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal
More information1 Gambler s Ruin Problem
Coyright c 2009 by Karl Sigman 1 Gambler s Ruin Problem Let N 2 be an integer and let 1 i N 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins
More informationMath 1B, lecture 5: area and volume
Math B, lecture 5: area and volume Nathan Pflueger 6 September 2 Introduction This lecture and the next will be concerned with the computation of areas of regions in the plane, and volumes of regions in
More informationContinued Fractions and the Euclidean Algorithm
Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction
More informationLecture 13 - Basic Number Theory.
Lecture 13 - Basic Number Theory. Boaz Barak March 22, 2010 Divisibility and primes Unless mentioned otherwise throughout this lecture all numbers are non-negative integers. We say that A divides B, denoted
More informationThe Characteristic Polynomial
Physics 116A Winter 2011 The Characteristic Polynomial 1 Coefficients of the characteristic polynomial Consider the eigenvalue problem for an n n matrix A, A v = λ v, v 0 (1) The solution to this problem
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS Systems of Equations and Matrices Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationAbout the Gamma Function
About the Gamma Function Notes for Honors Calculus II, Originally Prepared in Spring 995 Basic Facts about the Gamma Function The Gamma function is defined by the improper integral Γ) = The integral is
More informationInformation Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay
Information Theory and Coding Prof. S. N. Merchant Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 17 Shannon-Fano-Elias Coding and Introduction to Arithmetic Coding
More informationA STOCHASTIC MODEL FOR THE SPREADING OF AN IDEA IN A HUMAN COMMUNITY
6th Jagna International Workshop International Journal of Modern Physics: Conference Series Vol. 7 (22) 83 93 c World Scientific Publishing Company DOI:.42/S29452797 A STOCHASTIC MODEL FOR THE SPREADING
More information8 Primes and Modular Arithmetic
8 Primes and Modular Arithmetic 8.1 Primes and Factors Over two millennia ago already, people all over the world were considering the properties of numbers. One of the simplest concepts is prime numbers.
More informationExponential and Logarithmic Functions
Chapter 6 Eponential and Logarithmic Functions Section summaries Section 6.1 Composite Functions Some functions are constructed in several steps, where each of the individual steps is a function. For eample,
More informationThe Matrix Elements of a 3 3 Orthogonal Matrix Revisited
Physics 116A Winter 2011 The Matrix Elements of a 3 3 Orthogonal Matrix Revisited 1. Introduction In a class handout entitled, Three-Dimensional Proper and Improper Rotation Matrices, I provided a derivation
More informationCHAPTER 2 Estimating Probabilities
CHAPTER 2 Estimating Probabilities Machine Learning Copyright c 2016. Tom M. Mitchell. All rights reserved. *DRAFT OF January 24, 2016* *PLEASE DO NOT DISTRIBUTE WITHOUT AUTHOR S PERMISSION* This is a
More information6.4 Normal Distribution
Contents 6.4 Normal Distribution....................... 381 6.4.1 Characteristics of the Normal Distribution....... 381 6.4.2 The Standardized Normal Distribution......... 385 6.4.3 Meaning of Areas under
More informationLECTURE 4. Last time: Lecture outline
LECTURE 4 Last time: Types of convergence Weak Law of Large Numbers Strong Law of Large Numbers Asymptotic Equipartition Property Lecture outline Stochastic processes Markov chains Entropy rate Random
More informationSo let us begin our quest to find the holy grail of real analysis.
1 Section 5.2 The Complete Ordered Field: Purpose of Section We present an axiomatic description of the real numbers as a complete ordered field. The axioms which describe the arithmetic of the real numbers
More informationCartesian Products and Relations
Cartesian Products and Relations Definition (Cartesian product) If A and B are sets, the Cartesian product of A and B is the set A B = {(a, b) :(a A) and (b B)}. The following points are worth special
More informationCore Maths C3. Revision Notes
Core Maths C Revision Notes October 0 Core Maths C Algebraic fractions... Cancelling common factors... Multipling and dividing fractions... Adding and subtracting fractions... Equations... 4 Functions...
More informationAlgebra 2 Unit 8 (Chapter 7) CALCULATORS ARE NOT ALLOWED
Algebra Unit 8 (Chapter 7) CALCULATORS ARE NOT ALLOWED. Graph eponential functions. (Sections 7., 7.) Worksheet 6. Solve eponential growth and eponential decay problems. (Sections 7., 7.) Worksheet 8.
More information4.5 Linear Dependence and Linear Independence
4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then
More informationVisualizing Differential Equations Slope Fields. by Lin McMullin
Visualizing Differential Equations Slope Fields by Lin McMullin The topic of slope fields is new to the AP Calculus AB Course Description for the 2004 exam. Where do slope fields come from? How should
More informationSettling a Question about Pythagorean Triples
Settling a Question about Pythagorean Triples TOM VERHOEFF Department of Mathematics and Computing Science Eindhoven University of Technology P.O. Box 513, 5600 MB Eindhoven, The Netherlands E-Mail address:
More informationDevelopment of dynamically evolving and self-adaptive software. 1. Background
Development of dynamically evolving and self-adaptive software 1. Background LASER 2013 Isola d Elba, September 2013 Carlo Ghezzi Politecnico di Milano Deep-SE Group @ DEIB 1 Requirements Functional requirements
More informationLecture Notes on Elasticity of Substitution
Lecture Notes on Elasticity of Substitution Ted Bergstrom, UCSB Economics 210A March 3, 2011 Today s featured guest is the elasticity of substitution. Elasticity of a function of a single variable Before
More informationSection 3-3 Approximating Real Zeros of Polynomials
- Approimating Real Zeros of Polynomials 9 Section - Approimating Real Zeros of Polynomials Locating Real Zeros The Bisection Method Approimating Multiple Zeros Application The methods for finding zeros
More informationMathematics Review for MS Finance Students
Mathematics Review for MS Finance Students Anthony M. Marino Department of Finance and Business Economics Marshall School of Business Lecture 1: Introductory Material Sets The Real Number System Functions,
More informationMoving Least Squares Approximation
Chapter 7 Moving Least Squares Approimation An alternative to radial basis function interpolation and approimation is the so-called moving least squares method. As we will see below, in this method the
More informationRandom variables P(X = 3) = P(X = 3) = 1 8, P(X = 1) = P(X = 1) = 3 8.
Random variables Remark on Notations 1. When X is a number chosen uniformly from a data set, What I call P(X = k) is called Freq[k, X] in the courseware. 2. When X is a random variable, what I call F ()
More informationReview of Basic Options Concepts and Terminology
Review of Basic Options Concepts and Terminology March 24, 2005 1 Introduction The purchase of an options contract gives the buyer the right to buy call options contract or sell put options contract some
More informationAn Extension Model of Financially-balanced Bonus-Malus System
An Extension Model of Financially-balanced Bonus-Malus System Other : Ratemaking, Experience Rating XIAO, Yugu Center for Applied Statistics, Renmin University of China, Beijing, 00872, P.R. China Phone:
More information1 Determine whether an. 2 Solve systems of linear. 3 Solve systems of linear. 4 Solve systems of linear. 5 Select the most efficient
Section 3.1 Systems of Linear Equations in Two Variables 163 SECTION 3.1 SYSTEMS OF LINEAR EQUATIONS IN TWO VARIABLES Objectives 1 Determine whether an ordered pair is a solution of a system of linear
More informationDIFFERENTIABILITY OF COMPLEX FUNCTIONS. Contents
DIFFERENTIABILITY OF COMPLEX FUNCTIONS Contents 1. Limit definition of a derivative 1 2. Holomorphic functions, the Cauchy-Riemann equations 3 3. Differentiability of real functions 5 4. A sufficient condition
More informationVersion. General Certificate of Education (A-level) January 2013. Mathematics MPC3. (Specification 6360) Pure Core 3. Final.
Version General Certificate of Education (A-level) January Mathematics MPC (Specification 66) Pure Core Final Mark Scheme Mark schemes are prepared by the Principal Eaminer and considered, together with
More informationsin(x) < x sin(x) x < tan(x) sin(x) x cos(x) 1 < sin(x) sin(x) 1 < 1 cos(x) 1 cos(x) = 1 cos2 (x) 1 + cos(x) = sin2 (x) 1 < x 2
. Problem Show that using an ɛ δ proof. sin() lim = 0 Solution: One can see that the following inequalities are true for values close to zero, both positive and negative. This in turn implies that On the
More informationGEARING UP EXAMPLES. 4 to 3 4:3
GEARING UP EXAMPLES B 2 Teeth A 8 Teeth DEFINITION - RATIO As gear A revolves times, it will cause gear B to revolve times. Hence, we say that gear ratio of A to B is to. In mathematics, a ratio is a comparison
More informationUnderstanding Poles and Zeros
MASSACHUSETTS INSTITUTE OF TECHNOLOGY DEPARTMENT OF MECHANICAL ENGINEERING 2.14 Analysis and Design of Feedback Control Systems Understanding Poles and Zeros 1 System Poles and Zeros The transfer function
More informationRESULTANT AND DISCRIMINANT OF POLYNOMIALS
RESULTANT AND DISCRIMINANT OF POLYNOMIALS SVANTE JANSON Abstract. This is a collection of classical results about resultants and discriminants for polynomials, compiled mainly for my own use. All results
More informationStudent Outcomes. Lesson Notes. Classwork. Discussion (10 minutes)
NYS COMMON CORE MATHEMATICS CURRICULUM Lesson 5 8 Student Outcomes Students know the definition of a number raised to a negative exponent. Students simplify and write equivalent expressions that contain
More informationSECTION P.5 Factoring Polynomials
BLITMCPB.QXP.0599_48-74 /0/0 0:4 AM Page 48 48 Chapter P Prerequisites: Fundamental Concepts of Algebra Technology Eercises Critical Thinking Eercises 98. The common cold is caused by a rhinovirus. The
More informationLies My Calculator and Computer Told Me
Lies My Calculator and Computer Told Me 2 LIES MY CALCULATOR AND COMPUTER TOLD ME Lies My Calculator and Computer Told Me See Section.4 for a discussion of graphing calculators and computers with graphing
More informationSection 1-4 Functions: Graphs and Properties
44 1 FUNCTIONS AND GRAPHS I(r). 2.7r where r represents R & D ependitures. (A) Complete the following table. Round values of I(r) to one decimal place. r (R & D) Net income I(r).66 1.2.7 1..8 1.8.99 2.1
More information3 Contour integrals and Cauchy s Theorem
3 ontour integrals and auchy s Theorem 3. Line integrals of complex functions Our goal here will be to discuss integration of complex functions = u + iv, with particular regard to analytic functions. Of
More informationNonlinear Systems of Ordinary Differential Equations
Differential Equations Massoud Malek Nonlinear Systems of Ordinary Differential Equations Dynamical System. A dynamical system has a state determined by a collection of real numbers, or more generally
More informationS-Parameters and Related Quantities Sam Wetterlin 10/20/09
S-Parameters and Related Quantities Sam Wetterlin 10/20/09 Basic Concept of S-Parameters S-Parameters are a type of network parameter, based on the concept of scattering. The more familiar network parameters
More informationZero and Negative Exponents and Scientific Notation. a a n a m n. Now, suppose that we allow m to equal n. We then have. a am m a 0 (1) a m
0. E a m p l e 666SECTION 0. OBJECTIVES. Define the zero eponent. Simplif epressions with negative eponents. Write a number in scientific notation. Solve an application of scientific notation We must have
More informationVersion 1.0. General Certificate of Education (A-level) January 2012. Mathematics MPC4. (Specification 6360) Pure Core 4. Final.
Version.0 General Certificate of Education (A-level) January 0 Mathematics MPC (Specification 660) Pure Core Final Mark Scheme Mark schemes are prepared by the Principal Eaminer and considered, together
More informationProbability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur
Probability and Statistics Prof. Dr. Somesh Kumar Department of Mathematics Indian Institute of Technology, Kharagpur Module No. #01 Lecture No. #15 Special Distributions-VI Today, I am going to introduce
More information1 Teaching notes on GMM 1.
Bent E. Sørensen January 23, 2007 1 Teaching notes on GMM 1. Generalized Method of Moment (GMM) estimation is one of two developments in econometrics in the 80ies that revolutionized empirical work in
More informationApproximation of Aggregate Losses Using Simulation
Journal of Mathematics and Statistics 6 (3): 233-239, 2010 ISSN 1549-3644 2010 Science Publications Approimation of Aggregate Losses Using Simulation Mohamed Amraja Mohamed, Ahmad Mahir Razali and Noriszura
More informationHOMEWORK 5 SOLUTIONS. n!f n (1) lim. ln x n! + xn x. 1 = G n 1 (x). (2) k + 1 n. (n 1)!
Math 7 Fall 205 HOMEWORK 5 SOLUTIONS Problem. 2008 B2 Let F 0 x = ln x. For n 0 and x > 0, let F n+ x = 0 F ntdt. Evaluate n!f n lim n ln n. By directly computing F n x for small n s, we obtain the following
More informationLecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization
Lecture 2. Marginal Functions, Average Functions, Elasticity, the Marginal Principle, and Constrained Optimization 2.1. Introduction Suppose that an economic relationship can be described by a real-valued
More informationI remember that when I
8. Airthmetic and Geometric Sequences 45 8. ARITHMETIC AND GEOMETRIC SEQUENCES Whenever you tell me that mathematics is just a human invention like the game of chess I would like to believe you. But I
More informationThe sample space for a pair of die rolls is the set. The sample space for a random number between 0 and 1 is the interval [0, 1].
Probability Theory Probability Spaces and Events Consider a random experiment with several possible outcomes. For example, we might roll a pair of dice, flip a coin three times, or choose a random real
More informationThe continuous and discrete Fourier transforms
FYSA21 Mathematical Tools in Science The continuous and discrete Fourier transforms Lennart Lindegren Lund Observatory (Department of Astronomy, Lund University) 1 The continuous Fourier transform 1.1
More informationIdentities for Present Values of Life Insurance Benefits
Scand. Actuarial J. 1993; 2: 100-106 ORIGINAL ARTICLE Identities for Present Values of Life Insurance Benefits : RAGNAR NORBERG Norberg R. Identities for present values of life insurance benefits. Scand.
More informationC3: Functions. Learning objectives
CHAPTER C3: Functions Learning objectives After studing this chapter ou should: be familiar with the terms one-one and man-one mappings understand the terms domain and range for a mapping understand the
More informationLeast-Squares Intersection of Lines
Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a
More informationSolutions to Homework 10
Solutions to Homework 1 Section 7., exercise # 1 (b,d): (b) Compute the value of R f dv, where f(x, y) = y/x and R = [1, 3] [, 4]. Solution: Since f is continuous over R, f is integrable over R. Let x
More informationSolving Linear Systems, Continued and The Inverse of a Matrix
, Continued and The of a Matrix Calculus III Summer 2013, Session II Monday, July 15, 2013 Agenda 1. The rank of a matrix 2. The inverse of a square matrix Gaussian Gaussian solves a linear system by reducing
More informationStandard Deviation Estimator
CSS.com Chapter 905 Standard Deviation Estimator Introduction Even though it is not of primary interest, an estimate of the standard deviation (SD) is needed when calculating the power or sample size of
More informationCHAPTER 5 Round-off errors
CHAPTER 5 Round-off errors In the two previous chapters we have seen how numbers can be represented in the binary numeral system and how this is the basis for representing numbers in computers. Since any
More informationMath 115 Spring 2011 Written Homework 5 Solutions
. Evaluate each series. a) 4 7 0... 55 Math 5 Spring 0 Written Homework 5 Solutions Solution: We note that the associated sequence, 4, 7, 0,..., 55 appears to be an arithmetic sequence. If the sequence
More information1. First-order Ordinary Differential Equations
Advanced Engineering Mathematics 1. First-order ODEs 1 1. First-order Ordinary Differential Equations 1.1 Basic concept and ideas 1.2 Geometrical meaning of direction fields 1.3 Separable differential
More informationMPE Review Section III: Logarithmic & Exponential Functions
MPE Review Section III: Logarithmic & Eponential Functions FUNCTIONS AND GRAPHS To specify a function y f (, one must give a collection of numbers D, called the domain of the function, and a procedure
More informationChem 115 POGIL Worksheet - Week 4 Moles & Stoichiometry
Chem 115 POGIL Worksheet - Week 4 Moles & Stoichiometry Why? Chemists are concerned with mass relationships in chemical reactions, usually run on a macroscopic scale (grams, kilograms, etc.). To deal with
More informationDifferentiation of vectors
Chapter 4 Differentiation of vectors 4.1 Vector-valued functions In the previous chapters we have considered real functions of several (usually two) variables f : D R, where D is a subset of R n, where
More informationMathematics 31 Pre-calculus and Limits
Mathematics 31 Pre-calculus and Limits Overview After completing this section, students will be epected to have acquired reliability and fluency in the algebraic skills of factoring, operations with radicals
More informationChapter 31 out of 37 from Discrete Mathematics for Neophytes: Number Theory, Probability, Algorithms, and Other Stuff by J. M.
31 Geometric Series Motivation (I hope) Geometric series are a basic artifact of algebra that everyone should know. 1 I am teaching them here because they come up remarkably often with Markov chains. The
More information1 Interest rates, and risk-free investments
Interest rates, and risk-free investments Copyright c 2005 by Karl Sigman. Interest and compounded interest Suppose that you place x 0 ($) in an account that offers a fixed (never to change over time)
More informationInfinitely Repeated Games with Discounting Ù
Infinitely Repeated Games with Discounting Page 1 Infinitely Repeated Games with Discounting Ù Introduction 1 Discounting the future 2 Interpreting the discount factor 3 The average discounted payoff 4
More information1 Short Introduction to Time Series
ECONOMICS 7344, Spring 202 Bent E. Sørensen January 24, 202 Short Introduction to Time Series A time series is a collection of stochastic variables x,.., x t,.., x T indexed by an integer value t. The
More informationSubstitute 4 for x in the function, Simplify.
Page 1 of 19 Review of Eponential and Logarithmic Functions An eponential function is a function in the form of f ( ) = for a fied ase, where > 0 and 1. is called the ase of the eponential function. The
More informationChapter 4: Exponential and Logarithmic Functions
Chapter 4: Eponential and Logarithmic Functions Section 4.1 Eponential Functions... 15 Section 4. Graphs of Eponential Functions... 3 Section 4.3 Logarithmic Functions... 4 Section 4.4 Logarithmic Properties...
More informationPolynomial Degree and Finite Differences
CONDENSED LESSON 7.1 Polynomial Degree and Finite Differences In this lesson you will learn the terminology associated with polynomials use the finite differences method to determine the degree of a polynomial
More information