# IEEE Information Theory Society Newsletter

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 IEEE Information Theory Society Newsletter Vol. 53, No.4, December 2003 Editor: Lance C. Pérez ISSN The Shannon Lecture Hidden Markov Models and the Baum-Welch Algorithm Lloyd R. Welch Content of This Talk The lectures of previous Shannon Lecturers fall into several categories such as introducing new areas of research, resuscitating areas of research, surveying areas identified with the lecturer, or reminiscing on the career of the lecturer. In this talk I decided to restrict the subject to the Baum-Welch algorithm and some of the ideas that led to its development. I am sure that most of you are familiar with Markov chains and Markov processes. They are natural models for various communication channels in which channel conditions change with time. In many cases it is not the state sequence of the model which is observed but the effects of the process on a signal. That is, the states are not observable but some functions, possibly random, of the states are observed. In some cases it is easy to assign the values of the parameters to model a channel. All that remains is to determine what probabilities are desired and derive the necessary algorithms to compute them. In other cases, the choice of parameter values is only an estimate and it is desired to find the best values. The usual criterion is maximum likelihood. That is: find the values of parameters which maximizes the probability of the observed data. This is the problem that the Baum-Welch computation addresses. Preliminaries Let N be the set of non-negative integers. Let s introduce some useful notation to replace the usual n-tuple notations: [a k ] j k=i (a i, a i+,...,a j ) [a(k)] j k=i (a(i), a(i + ),...,a( j)) The k = will be dropped from the subscript when it is clear what the running variable is. Of particular use will be the concept of conditional probability and recursive factorization. The recursive factorization idea says that the joint probability of a collection of events can be expressed as a product of conditional probabilities, where each is the probability of an event conditioned on all previous events. For example, let A, B, and C be three events. Then Pr(A B C) = Pr(A)Pr(B A)Pr(C A B) Using the bracket notation, we can display the recursive factorization of the joint probability distribution of a sequence of discrete random variables: Pr ( [X(k)] 0 N =[x ) k] 0 N = Pr(X(0) = x0 ) N n=0 Pr ( X(n)=x n [X(k)] n 0 = [x k ] n ) 0 Markov Chains and Hidden Markov Chains We will treat only Markov Chains which have finite state spaces. The theory is more general, but to cover the more general case will only obscure the basic ideas. Let S be a finite set, the set of states. Let the number of elements in S be M. It will be convenient to identify the elements of S with the integers from to M. Let {S(t) : t N } be a sequence of random variables with Pr(S(t) S) = for all t N. That is, the values of S(t) are confined to S. Applying the above factorization to the joint distribution of the first N + random variables gives: continued on page 0

4 4 manifold: A geometric approach to the noncoherent multipleantenna channel, which appeared in the IEEE Transactions on Information Theory, Vol. 48, pp , February The last 2003 Board of Governors (BoG) meeting took place in the beautiful setting of the Allerton House (see picture), the conference center of the University of Illinois, in conjunction with the 4st Annual Allerton Conference on Communication, Control, and Computing. At the meeting we decided to have Seattle as the location for the 2006 ISIT with general co-chairs Joseph A. O Sullivan (Washington University) and John B. Anderson (Lund University). Alexander Barg (University of Maryland) and Raymond Yeung (The Chinese University of Hong Kong) will Call for Nominations for the 2004 IEEE Information Theory Society Paper Award Nominations are invited for the 2004 IEEE Information Theory Society Paper Award. Outstanding publications in the field of interest to the IT Society appearing anywhere during 2002 and 2003 are eligible. The purpose of this award is to recognize exceptional publications in the field and to stimulate interest in and encourage contributions to the fields of interest of the IT Society Paper Award Winners Announced The Information Theory Society is pleased to announce the winners of the 2003 Joint Information Theory/Communication Society Paper Award and the 2003 IEEE Information Theory Society Paper Award Joint IT/ComSoc Paper Award The winners of the 2003 IEEE Communications Society and Information Theory Society Joint Paper Award are Shlomo Shamai (Shitz) and Igal Sason for their article, Variations on the Gallager bounds, connections and applications, which appeared in IEEE Transactions on Information Theory, Vol. 48, No. 2, pp , December, Shlomo Shamai (Shitz) (S'80-M'82-SM'89-F'94) received the B.Sc., M.Sc., and Ph.D. degrees in Electrical Engineering from the Technion- Israel Institute of Technology, in 975, 98 and 986, respectively. From 975 to 985 he was with the Communications Research Labs in the capacity of a Senior Research Engineer. Since 986 he is with the Department of Electrical Engineering, Technion-Israel Institute of Technology, where he is now the William Fondiller Professor of Telecommunications. His research interests include topics in information theory and statistical communications. lead the program committee. Tutorials connected to the ISIT are highly appreciated by our members. If the tutorials are successful at the 2004 ISIT in Chicago, we will certainly add these to the program in Seattle. The elections for the 2004 presidents resulted in Hideki Imai for President, Steven McLaughlin for st Vice President and David Neuhoff for 2nd Vice President. I very much enjoyed being the President for the year The most important part of the job is to improve the communication with IEEE, the Board of Governors and you, the IT members. I thank all the members that supported the Board of Governors and me during this year. I hope to be able to contribute as past president to the improvement of our infrastructure. The Award consists of an appropriately worded certificate and an honorarium of US\$000 for a single author, or US\$2000 equally split among multiple authors. NOMINATION PROCEDURE: Please a brief rationale (limited to 300 words) for each nominated paper explaining its contributions to the field by Friday, March 5, 2004 to the Transactions Editor-in-Chief at with a cc to Katherine Perry at Call for Nominations for the 2004 Joint Information Theory/Communications Society Paper Award The Joint Information Theory/Communications Society Paper Award recognizes one or two outstanding papers that address both communications and information theory. Any paper appearing in a ComSoc or IT Society publication during the year 2003 is eligible for the 2004 award. Please send nominations to Steve McLaughlin by February, A Joint Award committee will make the selection by April 0, 2004 with practical constraints, multi-user information theory and spread spectrum systems, multiple-input-multiple-output communications systems, information theoretic models for wireless networks and systems, information theoretic aspects of magnetic recording, channel coding, combined modulation and coding, turbo codes and LDPC, in channel, source, and combined source/channel applications, iterative detection and decoding algorithms, coherent and noncoherent detection and information theoretic aspects of digital communication in optical channels. He is especially interested in theoretical limits in communication 2003 Joint IT/ComSoc Paper Award winners Igal Sason and Shlomo Shamai (Shitz). IEEE Information Theory Society Newsletter December 2003

8 8 GOLOMB S PUZZLE COLUMN Irreducible Divisors of Trinomials Solutions Solomon W. Golomb. A primitive polynomial f (x) of degree n 2 divides infinitely many trinomials over GF(2). Proof. Let α be a root of f (x). By primitivity, all the values,α,α 2,α 3 2n 2,...,α are distinct, and are all the non-zero elements of GF(2 n ). Therefore, for each j, 0 < j < 2 n, + α j = α k with 0 < k < 2 n and j k. Hence, f (x) divides the trinomial + x j + x k for these values of j and k. In addition, f (x) divides + x J + x K for every J with J j (mod 2 n ) and K k (mod 2 n ), since + α J + α K = + α j + α k = 0, in view of α 2n =. 2. If f (x) is irreducible with primitivity t and f (x) divides no trinomials of degree < t, then f (x) divides no trinomials. Proof (by contradiction). Suppose f (x) divides the trinomial x N + x A + of degree N > t, and let α be a root of f (x). Since f (x) has primitivity t, α t =. Since f (α) = 0, where α is a root of f (x), any polynomial g(x) divisible by f (x) also has α as a root, since g(x) = f (x) q(x) gives g(α) = f (α) q(α) = 0 q(α) = 0. Thus, α N + α A + = 0, from which α n + α a + = 0 where n N (mod t) and a A (mod t), where we choose both n and a to be less than t, from which f (x) divides the trinomial x n + x a +, of degree <t. 3. If p 5 is a prime for which 2 is primitive modulo p, then f (x) = (x p )/(x ) = + x + x 2 + +x p is an irreducible polynomial which divides no trinomials. Proof. For each prime p, p (x) = (x p )/(x ) is the cyclotomic polynomial over the rational field Q, whose roots are the φ(p) = p primitive p th roots of unity. While all cyclotomic polynomials are irreducible over Q, p (x) remains irreducible over GF(2) if and only if 2 is primitive modulo p. In this case, f (x) = p (x) has primitivity t = p, and any root α of this f (x) has α p =. Note that for p 5, the minimum polynomial for such a root of unity has p > 3 non-zero terms. By Result 2, above, if this f (x) divides any trinomial, it must divide a trinomial of degree <t = p, say x n + x a + with n < p. But then the root α of f (x) is a root of this trinomial of degree p, whereas the unique polynomial of degree p with α as a root is the minimal polynomial of α, f (x) = p (x), of degree p,which has more than three terms. Note. There are also many other irreducible polynomials which divide no trinomials. These three problems are the easy results. Gerard J. Foschini Named Bell Labs Fellow Alex Dumas Bell Labs President Bill O'Shea and Lucent's R&D leadership have chosen seven employees as 2002 Bell Labs Fellows. The annual award, Bell Labs' highest honor, recognizes sustained research and development contributions to the company. The 2002 awards mark the 20th year of the program. Since it began, 98 scientists and engineers have joined this elite group. Each new Fellow will receive a sculpture, a personal Fellows plaque and a cash award. A plaque of each will be added to the wall of honor in the Murray Hill, N.J., lobby. A formal luncheon to honor the recipients is scheduled for September. The 2002 winners are: *Alvin Barshefsky, Lucent Worldwide Services, for his sustained performance in the area of software and services development. Gerard J. Foschini *Gerard J. Foschini, Bell Labs Research, for his breakthrough invention of the BLAST concept that has the potential to revolutionize wireless technology. *Theodore M. Lach, Integrated Network Solutions, for his innovation and technical leadership in switching system component and product reliability, silicon fabrication techniques and contributions to industry standards. *Rajeev R. Rastogi, Bell Labs Research, for contributions in the areas of network management and database systems, and the successful application of these innovations to Lucent products. *William D. Reents, Supply Chain Networks, for pioneering work in analytical science and development of leadingedge characterization tools and methodologies. *Young-Kai Chen, Bell Labs Research, for his pioneering working in developing high-speed devices and circuits. *Joseph A. Tarallo, Mobility Solutions, for sustained contributions to second- and third-generation wireless technology, and base station architecture and design. continued on page 9 IEEE Information Theory Society Newsletter December 2003

10 0 Hidden Markov Models and the Baum-Welch Algorithm (continued from page ) Pr ( [S(k)] 0 N=[s ) k] 0 N = Pr(S(0) = s0 ) N Pr(S(n)=s n [S(k)] n 0 = [s k ] n ) () 0 n= For the sequence of random variables to be a Markov Chain the conditional probabilities must only be a function of the last random variable in the condition so that equation () reduces to Pr ( [S(k)] 0 N=[s ) k] 0 N = Pr(S(0) = s0 ) N (2) Pr(S(n)=s n S(n ) = s n ) n= In my work, the transition probabilities were stationary, that is they are constant functions of time: Pr(S(n) = j S(n ) = i) = Pr(S() = j S(0) = i) def = p ij In addition to the Markov Chain, let {Y(t) : t N } be a sequence of random variables (called random observations). It will be convenient to assume that the values are confined to a discrete set and an experiment consists of observing values of T consecutive random variables. Again, treating a more general case will only obscure the basic ideas. Pr ( [S(t)] T 0 = [s t] T 0 and [Y(t)]T = [y t] T ) = p(s, y) = p s0 T p st s t f(y t s t ) This formula gives the probability of the atoms of the model, that is, those events that can not be subdivided into smaller events. The probability of any event describable in the model is the sum of the probability of the atoms. Of course, the probabilities are functions of the parameters of the model, which we will denote by: λ def = t= { } ps : s M, p sσ : s,σ M, f(y s) : y Y, s M and use λ as a function argument where appropriate. Questions What questions are of interest? One question is what is the probability that an T-tuple of Y(t) will be observed. This, of course, is a function of the parameters. The probability of an T-tuple of observations is just the sum over all state sequences of the probabilities of the corresponding atoms: (5) Applying recursive factorization to the joint distribution of the first T + random states and first T random observations: p(y; λ) def = Pr ( [Y(t)] T = [y t] T ; λ) = s p(s, y; λ) (6) Pr ( [S(t)] 0 T = [s t] 0 T and [Y(t)]T = [y t] ) T = T Pr(S(0) = s 0 ) Pr ( S(t) = s t [S(k)] t 0 = [s k ] t 0 ) t= T Pr ( Y(t) = y t [S(k)] 0 T = [s k] 0 T t= ) and [Y(k)]t = [y k ] t The next simplifying assumption is that the conditional probability distribution of Y(t) given all states and all previous random observations is only a function of S(t) (and not of time). I considered also the case when the distribution of Y(t) depends on S(t) and S(t ). However, though it added little to the computational complexity, it added significantly to the number of parameters to be estimated. Making use of the above conditions, Pr ( [S(t)] 0 T = [s t] 0 T and [Y(t)]T = [y t] ) T = T Pr(S(0) = s 0 ) Pr(S(t) = s t S(t ) = s t ) t= (3) T Pr(Y(t) = y t S(t) = s t ) (4) t= To simplify notation, define f(y s) def = Pr(Y(t) = y S(t) = s), s def = [s t ] 0 T, y def = [y t ] T, and p(s, y) def = Pr([S(t)] 0 T = s and [Y(t)]T = y). With this notation we have Then p(y; λ) is the probability of the observations, y. It is also the likelihood function for λ given the observations, y. A standard problem is to choose λ to maximize the likelihood function. It is frequently the case that the random observables, {Y(t) : t N }, are observed for some period of time and it is desired to find some information about the state sequence from those observations. For example, given the event, {[Y(t)] T = [y t ] T }, we may wish to find the probability distribution of the state at a specified time, τ. That is we wish to find, for a given sequence of observations, the probability distribution of s τ given those observations: Pr ( S(τ) = s τ [Y(t)] T = [y ) t] T Since, Pr ( S(τ) = s τ [Y(t)] T = [y t] ) T Pr ( S(τ) = s = τ and [Y(t)] T = [yt] ) T Pr ( [Y(t)] T = [y ) t] T the computation of the a posteriori probability is equivalent to computing the joint probability. Referring to equation (5), we see that this reduces to computing Ɣ τ (s τ ) def = Pr ( S(τ) = s τ and [Y(t)] T = [y ) t] T = N p st s t f(y t s t ) (7) [s k] τ 0 p s0 [s k] T t= τ+ for each choice of s τ. With the exception of s τ, each indexed state is a summation variable and, with the exception of s 0, occurs in exactly two factors. It is easily deduced that the equation can be expressed in terms of the product of matrices. IEEE Information Theory Society Newsletter December 2003

11 However, there is a better (at least to me) approach to computing this probability. We begin with the joint probability of the T-tuple of observations and the state at time τ and apply recursive factorization where the first event is the set of observations up through time, τ, and the state at time τ. Pr ( [Y(t)] T = [y t] T and S(τ) = s ) τ = Pr ( [Y(t)] τ = [y t] τ and S(τ) = s τ ) Pr ( [Y(t)] τ+ T = [y t] τ+ T [Y(t)]τ = [y t] τ and S(τ) = s ) τ Now Markovity of the state sequence implies that the probablility of [S t ] τ+ T and therefore the probability of [Y t] τ+ T are independent of history prior to time τ. So the condition on the Y in the second term drop out and the factorization reduces to Pr ( [Y(t)] T = [y t] T and S(τ) = s ) τ = Pr ( [Y(t)] τ = [y t] τ and S(τ) = s ) τ Pr ( [Y(t)] τ T + = [y t] τ T + S(τ) = s ) τ We next address the problem of computing these factors, and I remark that December 2003 α τ (s) def = Pr ( [Y(t)] τ = [y t] τ and S(τ) = s), β τ (s) def = Pr ( [Y(t)] τ+ T = [y t] τ+ T S(τ) = s) Ɣ τ (s) = α τ (s) β τ (s) α T (s) = Pr ( [Y(t)] T = [y t] ) T = p(y; λ). s Now recursive factoring of α τ (s) where the first factor is the observations up through time τ and the state at time τ gives α τ (s) Pr ( [Y(t)] τ = [y t] τ and S(τ) = s) = Pr ( [Y(t)] τ = [y t ] τ and S(τ ) = σ ) σ Pr ( Y(τ) = y τ and S(τ) = s [Y(t)] τ = [y t ] τ and S(τ ) = σ ) Again, Markovity implies that the condition, [Y(t)] τ = [y t ] τ can be dropped from the second factor: α τ (s) = Pr ( [Y(t)] τ = [y t ] τ and S(τ = σ ) σ Pr ( Y(τ) = y τ and S(τ) = s S(τ ) = σ ) The first factor is just α τ (σ ) and the second factor is p σ s f(y τ s) and above equation becomes the recursion: α τ (s) = α τ (σ )p σ s f(y τ s) (8) σ Similarly, a reverse time recursion exists for β τ (s): Finally we have and β τ (s) = σ p sσ f(y τ+ σ)β τ+ (σ ) (9) Pr ( S(τ) = s and [Y(t)] T = [y t] T ) = Ɣτ (s) = α τ (s)β τ (s) Pr(S(τ) = s y) = α τ (s) β τ (s) σ α T(σ ). Once we have routines for computing α and β we can compute not only Pr(S(t) = s y) but also the a posteriori probability of other local events, such as the event, {S(t) = s and S(t + ) = σ }. In this case the expression is Pr(S(t ) = s, S(t) = σ and [Y(τ)] T = y) = α t (s)p sσ f (y t+ σ)β t+ (σ ) Ɣ t (s,σ)(0) Improving on Estimates of Parameters At this point Leonard Baum and I found that we had both been working independently on Hidden Markov Chains and had both come up with essentially the same calculation for a posteriori probabilities of local events. At that point we joined forces. Now, the above calculations were based upon specified parameter values. What if those parameter values did not adequately represent the phenomena under investigation? My thoughts proceeded as follows. While the parameters may not be accurate, the a posteriori probabilities may translate to better parameters. For example, from the frequency interpretation of probability, if we could observe the state sequence over a long period of time and count the number of times the state, s, occurs, the frequency of occurrence should be approximately Pr(S(t) = s) If the assumed parameters are correct, we will get p s, where for a large enough period of time p will be the stationary distribution (the eigenvector with eigenvalue ) of the transition matrix. Furthermore, if the observation sequence, [y t ] T, is a typical sequence in the Information Theoretic sense, that is, it has high probability using the parameters of the model, then the expected frequencies of states given the observations should also be approximately p s, where p s is the stationary distribution, not the initial distribution. Expressed in equation form: T t= Pr( S(t) = s [Y(τ)] T = [y τ ] T ) p s. T Similarly, from the frequency interpretation of probability, if we could observe the state sequence over a long period of time and count the number of times the state, s, occurs followed by σ, the frequency of occurrence should be approximately Pr(S(t ) = s and S(t) = σ). If the assumed parameters are correct, we will get p s p sσ. Again if the observation sequence, [y t ] T, is a typical sequence in IEEE Information Theory Society Newsletter

12 2 the Information Theoretic sense, then the expected frequencies of state transitions given the observations should also be approximately p s p sσ. Expressed in equation form: T t= Pr( S(t ) = s, S(t) = σ [Y(τ)] T = [y τ ] T ) T p s p sσ. Finally, if we could observe the state sequence and the observation sequence and count the number of times S(t) = s, Y(t) = y, and the frequency should approximate p s f(y s). Again, if the observed sequence is a typical sequence for the given parameters, the a posteriori expected frequencies should approximate p s f(y s), i.e., ) p s f(y s) T Pr( S(t) = s,y(t) = y [Y(τ )] T t= = [yτ ]T T Pr( S(t) = s [Y(τ )] T t {t:y=y t } = [yτ ]T T My next thought was that if y was generated by a model with different parameter values and therefore not a typical sequence for the assumed values, the a posteriori frequencies, influenced by behavior of [y τ ] T, may be a better indication of the true parameters that the initial guess. So I replaced the parameter values by the expected frequencies and recomputed p(y; λ ) where λ def = p s (λ ) = T p sσ (λ ) = f(y s; λ ) T Pr( S(t) = s [Y(τ)] T t= = [yτ ]T T Pr( S(t ) = s,s(t) = σ [Y(τ)] T t= = [yτ ]T T T Pr( S(t) = s,y(t) = y [Y(τ)] T t= = [yτ ]T Tp s(λ ) ) ) ) () ) I was please to find that p(y; λ )>p(y; λ). In other words, this substitution increased the likelihood function! I tried this transformation on several Hidden Markov Models and the likelihood function always increased. Leonard Baum tried it on a number of examples and again the likelihood function always increased. That is my contribution to the Baum-Welch `algorithm, the easy part. I tried to provide a mathematical proof that the likelihood always increases but I failed. Baum, in cooperation with J. Eagon did the hard part by proving that this transformation either increases the likelihood function or leaves it constant. In the latter case, λ is a fixed point of the transformation. Their proof involved rather complex computations and applications of Hölder s inequality and the fact that the geometric mean is less than or equal to the arithmetic mean. Later Baum, together with T. Petrie, G. Soules and N. Weiss, all at CRD/IDA at the time found a more elegant proof with the flavor of Information Theory which I will now discuss. The Q Function They began with the Kullback-Leibler divergence of two distributions: D(p, p 2 ) ( ) p (ω) p (ω) log ω p 2 (ω) where p and p 2 are two probability distributions on a discrete space and ω is summed over that space. The interpretation of this number is that for an experiment consisting of multiple selections from the distribution p, D(p, p 2 ) is the expected log factor of the probability in favor of p against p 2. It is an information theoretic measure and is known to be non-negative, equaling zero only when the p 2 (ω) = p (ω) for all ω for which p (ω) > 0. How does this apply to Hidden Markov Models? We let p(s, y; λ) p (s) = and p 2 (s) = p(s, y; λ ) p(y; λ) p(y; λ ). Then p and p 2 are distributions and 0 D(λ, λ ) = ( ) p(s, y; λ) p(s, y; λ)p(y; λ s p(y; λ) log ) p(s, y; λ )p(y; λ) ( ) p(y; λ ) = log + ( ) p(s, y; λ) p(s, y; λ) p(y; λ) s p(y; λ) log p(s, y; λ. ) We simplify this by defining Then Q(λ, λ ) s p(s, y; λ) log(p(s, y; λ )). ( ) p(y; λ ) 0 D(λ, λ ) = log + Q(λ, λ) Q(λ, λ ) p(y; λ) p(y; λ) and rearranging the inequality we have Q(λ, λ ) Q(λ, λ) p(y; λ) log ( ) p(y; λ ) p(y; λ) and this implies that if Q(λ, λ )>Q(λ, λ) then p(λ )>p(λ) Hill Climbing (2) We obtain a hill climbing algorithm by finding that λ which maximizes Q(λ, λ ) as a function of its second argument. If Q(λ, λ )>Q(λ, λ) then p(λ )>p(λ) and we have succeeded in increasing p(λ) which is the probability of the observations. To maximize Q(λ, λ ) we begin by finding the critical points of Q as a function of λ and subject to the stochastic constraints on the components of λ. (A sample constraint is j p ij = ). Before proceeding, let s manipulate the expression for Q. In equation (5) the expression for p(s, y; λ) is a product, so its logarithm is a sum of log factors. Replacing λ by λ in equation(5) and taking logarithms we have: log(p(s, y; λ )) = log(p s(0) (λ )) T + log(p s(t )s(t) (λ ) f(y(t); λ s(t )s(t))) t= Substituting into the definition for Q gives: IEEE Information Theory Society Newsletter December 2003

13 3 Q(λ, λ ) = s + s p(s, y,λ)log(p s(0) (λ )) (3) p(s, y,λ) T log(p s(t )s(t) (λ ) f (y(t); λ s(t )s(t))). t= From this equation it can be seen that it is easy to differentiate with respect to the components of λ, add the appropriate Lagrange factors and solve. The result has already been displayed in equation (). Applications There are too many papers published on applications to list here. In the area of speech recognition, here is a small sample: F. Jelinek, L. Bahl, and R. Mercer, Design of a linguistic statistical decoder for the recognition of continuous speech, IEEE Trans. Inform. Theory, vol. 2, May 975. L.R. Rabiner, A tutorial on Hidden Markov models and selected applications in speech recognition, Proceedings of the IEEE, vol. 77, no. 2, Feb A. Poritz, Linear predictive hidden Markov models and the speech signal, in Proceedings of ICASSP 82, May 982. EM Theory In 977, Dempster, Laird and Rubin collected a variety of maximum likelihood problems and methods of solving these problems that occurred in the literature. They found that all of these methods had some ideas in common and they named it the EM Algorithm, (standing for Expectation, Maximization.) The common problem is to maximize Prob(y; ), as a function of where y is observed. (The probability of y is used in the case of discrete random variables and a density is maximized in the case of continuous random variables.) The observation, y, is viewed as incomplete data in the sense that there is a larger model containing complete data and y inherits its distribution by way of a mapping from the larger model to the observation model. Mathematically: There is a probability space, X with a family of probability measures, p(x; ), and a mapping function, F with F(x) = y. The distribution, q, of y is q(y; ) = {x:f(x)=y} p(x; ). The conditional distribution of x given y is provided F(x) = y. p(x y; ) = p(x; ) q(y; ), Given a second value of, say they define Q(, ) = E{log(p(x; )) y; }, where E is the notation for the expected value function. This is the Expectation step. It gives a formula in. The Maximization step is to vary to maximize Q. In many problems the maximization step is easy and in many others it is as difficult as the original maximum likelihood problem. This leads to a Generalized Estimation Maximization which simply finds any which increases Q. The Baum-Welch algorithm fits right into the EM scheme. x is (s, y) and the observation is y. The Q function is exactly the Q function that Baum et al. introduced to prove that the transformation increases the likelihood. Recommended Reading There are many papers published on these subjects. A few are: L.E. Baum and J. Eagon An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology, Bulletin of the American Mathematical Soc., vol. 70, pp L.E. Baum, T. Petrie, G. Soules and N. Weiss, A maximization technique occurring in the statictical analysis of probabilistic functions of Markov chains, Ann. Math. Stat., vol. 4, 970. A. Dempster, N. Laird and D. Rubin, Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, B, vol. 39, 977. A paper which extends theory to observations with continuous distributions. L. Liporace, Maximum likelihood estimates for multivariate observations Markov sources, IEEE Trans. Inform. Theory, vol. 28, Sept December 2003 IEEE Information Theory Society Newsletter

15 Information Theory Society Board of Governors Meeting Pacifico Conference Center, Yokohama, Japan, June 29, 2003 Mehul Motani 5 Attendees: John Anderson, Daniel Costello, Thomas Cover, Michelle Effros, Tony Ephremides, Ivan Fair, Tom Fuja, Marc Fossorier, Joachim Hagenauer, Chris Heegard, Hideki Imai, Torleiv Klove, Ralf Koetter, Ioannis Kontoyiannis, Ryuji Kohno, Steven McLaughlin, Mehul Motani, Paul Siegel, David Tse, Alexendar Vardy, Han Vinck, Sriram Viswanath. The meeting was called to order at 0:00 AM by Society President Han Vinck. The members of the Board were welcomed and introduced themselves.. The agenda was approved and distributed. 2. The minutes of the previous meeting in Paris, France on March 3, 2003 were approved as distributed. 3. The President began by reporting on the IEEE TAB meeting which was held earlier in the year. He reported that IEEE was very concerned about its financial situation. In 2002, IEEE suffered a \$6M deficit. It was reported that due to redistribution of the IEL and ASPP income, the Society would be getting \$00K more based on content (the society has the largest journal offering) and downloading statistics. The Society will also be repaid 20% of the investment it has made in the digital library. The President also reported that the Society was successful in participating in IEEE awards and suggested that more Society members be nominated for such awards. 4. The Awards Committee report was presented by Hideki Imai. The Awards Committee has nominated Bob Gallager for the 2003 Cristofore Colombo International Communications Award. It was reported that the IT Society Members on the Joint IT/ComSoc Paper Award Committee and the chair of the IT Society Awards Committee selected one paper out of three nominations from members of the Awards Committee and submitted it to the Joint IT/ComSoc Paper Award Committee on April 6. There was no nomination from ComSoc. On June 8, the Joint IT/ComSoc Paper Award Committee decided to award the single nominated paper the Joint IT/ComSoc Best Paper Award. The winning paper is: S. Shamai (Shitz) and I. Sason, Variations on the Gallager bounds, connections and applications, IEEE Transactions on Information Theory, Vol. 48, No. 2, pp , December Action Item: It was agreed by the Board that the Joint IT/ComSoc Paper Award procedures be added to the Bylaws. This needs to be in conjunction with ComSoc. It was reported that the total number of nominations for the IT December 2003 Society Paper Award for the period was 0. Three papers shown survived the last round of voting of the Award Committee: - Lizhong Zheng and David N. C. Tse, Communication on the Grassmann manifold: A geometric approach to the noncoherent multiple-antenna channel, IEEE Transactions Information Theory, vol. IT-48, No. 2, pp , February Vladimir I. Levenshtein, Efficient reconstruction of sequences, IEEE Transactions on Information Theory, vol. IT-47, no., pp. 2-22, Jan S. Verdu, Spectral efficiency in the wideband regime, IEEE Transactions on Information Theory, special issue in memory of A. Wyner, on Shannon theory: perspective, trends and applications, vol. 48, no. 6, pp , June According to the bylaws, the Board shall vote for the nominees by ballot, conducted by the Society President or designee, at the first Board Meeting following June st of the award year. The President informed the Board that several member of the Board could not attend the meeting due to visa problems. The Board voted unanimously to delay the vote and conduct an ballot by August. Action Item: The Board voted unanimously to revisit the Bylaws with respect to voting for the IT Paper Award at the BOG meeting. 5. The membership report was presented by Steven McLaughlin. He reported that the Society membership has dropped by 0% since last year (at the same time of the year). Since IEEE has seen a similar drop in its membership, it was suggested that this is the reason for the Society s drop. Steven also reported that the Society was participating in an IEEE wide questionnaire dealing with some membership issues. Steven also noted that the Society chapter luncheon would be held on Thursday and invited any interested Board members to attend. Action Item: The Board requested that Steven report more details on the 0% drop in Society membership at the next Board meeting. Steven reported that there were 00 ISIT attendees who were not IT Society members and suggested contacting them regarding membership. Action Item: The Board requested that Steven, in cooperation with Michelle Effros, present a proposal at the next meeting addressing membership issues including ideas and suggestion for increasing Society membership. 6. The treasurer s report prepared by Marc Fossorier was distrib- IEEE Information Theory Society Newsletter

18 8 IEEE Information Theory Society Newsletter December 2003

19 9 December 2003 IEEE Information Theory Society Newsletter

20 20 IEEE Information Theory Society Newsletter December 2003

### U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009. Notes on Algebra

U.C. Berkeley CS276: Cryptography Handout 0.1 Luca Trevisan January, 2009 Notes on Algebra These notes contain as little theory as possible, and most results are stated without proof. Any introductory

### Integer roots of quadratic and cubic polynomials with integer coefficients

Integer roots of quadratic and cubic polynomials with integer coefficients Konstantine Zelator Mathematics, Computer Science and Statistics 212 Ben Franklin Hall Bloomsburg University 400 East Second Street

### ALGEBRA 1/ALGEBRA 1 HONORS

ALGEBRA 1/ALGEBRA 1 HONORS CREDIT HOURS: 1.0 COURSE LENGTH: 2 Semesters COURSE DESCRIPTION The purpose of this course is to allow the student to gain mastery in working with and evaluating mathematical

### WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT?

WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT? introduction Many students seem to have trouble with the notion of a mathematical proof. People that come to a course like Math 216, who certainly

### MOP 2007 Black Group Integer Polynomials Yufei Zhao. Integer Polynomials. June 29, 2007 Yufei Zhao yufeiz@mit.edu

Integer Polynomials June 9, 007 Yufei Zhao yufeiz@mit.edu We will use Z[x] to denote the ring of polynomials with integer coefficients. We begin by summarizing some of the common approaches used in dealing

### Math Review. for the Quantitative Reasoning Measure of the GRE revised General Test

Math Review for the Quantitative Reasoning Measure of the GRE revised General Test www.ets.org Overview This Math Review will familiarize you with the mathematical skills and concepts that are important

### ECEN 5682 Theory and Practice of Error Control Codes

ECEN 5682 Theory and Practice of Error Control Codes Convolutional Codes University of Colorado Spring 2007 Linear (n, k) block codes take k data symbols at a time and encode them into n code symbols.

### FORDHAM UNIVERSITY GABELLI SCHOOL OF BUSINESS

FORDHAM UNIVERSITY GABELLI SCHOOL OF BUSINESS TOGETHER WITH THE GABELLI CENTER FOR GLOBAL INVESTMENT ANALYSIS Value Investing 20 Years Later: A Celebration of the Roger Murray Lecture Series 1993 _ 2013

### Graduate Program Policies and Procedures

The University of British Columbia Department of Physics & Astronomy Graduate Program Policies and Procedures Sept 2011 Table of Contents 1. Introduction!... 4 2. Programs!... 4 3. Admissions!... 4 3.1.

### Graduate Program Policies and Procedures

The University of British Columbia Department of Physics & Astronomy Graduate Program Policies and Procedures May 2007 Table of Contents 1 Introduction 4 2 Programs 4 3 Admissions 4 3.1 Eligibility 4 3.2

### minimal polyonomial Example

Minimal Polynomials Definition Let α be an element in GF(p e ). We call the monic polynomial of smallest degree which has coefficients in GF(p) and α as a root, the minimal polyonomial of α. Example: We

### Prime Numbers and Irreducible Polynomials

Prime Numbers and Irreducible Polynomials M. Ram Murty The similarity between prime numbers and irreducible polynomials has been a dominant theme in the development of number theory and algebraic geometry.

### 2.1 Complexity Classes

15-859(M): Randomized Algorithms Lecturer: Shuchi Chawla Topic: Complexity classes, Identity checking Date: September 15, 2004 Scribe: Andrew Gilpin 2.1 Complexity Classes In this lecture we will look

### The program also provides supplemental modules on topics in geometry and probability and statistics.

Algebra 1 Course Overview Students develop algebraic fluency by learning the skills needed to solve equations and perform important manipulations with numbers, variables, equations, and inequalities. Students

### Breaking Generalized Diffie-Hellman Modulo a Composite is no Easier than Factoring

Breaking Generalized Diffie-Hellman Modulo a Composite is no Easier than Factoring Eli Biham Dan Boneh Omer Reingold Abstract The Diffie-Hellman key-exchange protocol may naturally be extended to k > 2

### Ronald Graham: Laying the Foundations of Online Optimization

Documenta Math. 239 Ronald Graham: Laying the Foundations of Online Optimization Susanne Albers Abstract. This chapter highlights fundamental contributions made by Ron Graham in the area of online optimization.

### NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS

NEW YORK STATE TEACHER CERTIFICATION EXAMINATIONS TEST DESIGN AND FRAMEWORK September 2014 Authorized for Distribution by the New York State Education Department This test design and framework document

### CONTRIBUTIONS TO ZERO SUM PROBLEMS

CONTRIBUTIONS TO ZERO SUM PROBLEMS S. D. ADHIKARI, Y. G. CHEN, J. B. FRIEDLANDER, S. V. KONYAGIN AND F. PAPPALARDI Abstract. A prototype of zero sum theorems, the well known theorem of Erdős, Ginzburg

### LOOKING FOR A GOOD TIME TO BET

LOOKING FOR A GOOD TIME TO BET LAURENT SERLET Abstract. Suppose that the cards of a well shuffled deck of cards are turned up one after another. At any time-but once only- you may bet that the next card

### Evaluation of Intellectual Property Course Offerings for undergraduate engineering students in the Mid Atlantic Region

Evaluation of Intellectual Property Course Offerings for undergraduate engineering students in the Mid Atlantic Region Abstract S. Jimmy Gandhi, M.S, Stevens Institute of Technology Donald Merino, PhD,

### Students in their first advanced mathematics classes are often surprised

CHAPTER 8 Proofs Involving Sets Students in their first advanced mathematics classes are often surprised by the extensive role that sets play and by the fact that most of the proofs they encounter are

### SECRET sharing schemes were introduced by Blakley [5]

206 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 1, JANUARY 2006 Secret Sharing Schemes From Three Classes of Linear Codes Jin Yuan Cunsheng Ding, Senior Member, IEEE Abstract Secret sharing has

### FINAL ASSESSMENT REPORT Graduate Review of Electrical and Computer Engineering (MEng, MSc, PhD) March 2013

Review Process FINAL ASSESSMENT REPORT Graduate Review of Electrical and Computer Engineering (MEng, MSc, PhD) March 2013 The self-study was produced with the assistance of a committee which included the

### GOING TO GRADUATE SCHOOL IN THE MATHEMATICAL SCIENCES

GOING TO GRADUATE SCHOOL IN THE MATHEMATICAL SCIENCES By Ricardo Cortez and Colette Patt. Produced by the Physical Sciences Student Diversity Program, Dean's Office of the College of Letters and Science,

### 9.2 Summation Notation

9. Summation Notation 66 9. Summation Notation In the previous section, we introduced sequences and now we shall present notation and theorems concerning the sum of terms of a sequence. We begin with a

### Queen s Master of Entrepreneurship & Innovation

Queen s Master of Entrepreneurship & Innovation A 12-month program starting in September at Queen s School of Business in Kingston, ON. Faculty of Engineering and Applied Science Offered in Partnership

### Factoring & Primality

Factoring & Primality Lecturer: Dimitris Papadopoulos In this lecture we will discuss the problem of integer factorization and primality testing, two problems that have been the focus of a great amount

### Notes 11: List Decoding Folded Reed-Solomon Codes

Introduction to Coding Theory CMU: Spring 2010 Notes 11: List Decoding Folded Reed-Solomon Codes April 2010 Lecturer: Venkatesan Guruswami Scribe: Venkatesan Guruswami At the end of the previous notes,

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### Polynomials and Factoring. Unit Lesson Plan

Polynomials and Factoring Unit Lesson Plan By: David Harris University of North Carolina Chapel Hill Math 410 Dr. Thomas, M D. 2 Abstract This paper will discuss, and give, lesson plans for all the topics

### Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics

Data Science: An Action Plan for Expanding the Technical Areas of the Field of Statistics William S. Cleveland Statistics Research, Bell Labs wsc@bell-labs.com Abstract An action plan to enlarge the technical

### The Basics of Graphical Models

The Basics of Graphical Models David M. Blei Columbia University October 3, 2015 Introduction These notes follow Chapter 2 of An Introduction to Probabilistic Graphical Models by Michael Jordan. Many figures

### Principles to Guide the Design and Implementation of Doctoral Programs in Mathematics Education

Principles to Guide the Design and Implementation of Doctoral Programs in Mathematics Education A Task Force Report for the Association of Mathematics Teacher Educators Forward This report, in some ways,

### A simple criterion on degree sequences of graphs

Discrete Applied Mathematics 156 (2008) 3513 3517 Contents lists available at ScienceDirect Discrete Applied Mathematics journal homepage: www.elsevier.com/locate/dam Note A simple criterion on degree

### Students of Agronomy, Soils, & Environmental Sciences

SASES Students of Agronomy, Soils, & Environmental Sciences agronomy.org/students crops.org/students soils.org/students Students of Agronomy, Soils, and Environmental Sciences (SASES) is the undergraduate

### Industrial Engineering Definition of Tuning

Industrial Engineering Definition of Tuning Tuning is a faculty-led pilot project designed to define what students must know, understand, and be able to demonstrate after completing a degree in a specific

### THE CLASS TOP OF. The Swanson School of Engineering at the University of ANSYS ACADEMIC ADVANTAGE ANSYS.COM/81CLASS THOUGHT LEADER

TOP OF THE CLASS The University of Pittsburgh s Swanson School of Engineering was an early adopter of engineering simulation, using ANSYS software for decades to train students in industrystandard practices

### SCHOOL OF URBAN AFFAIRS & PUBLIC POLICY CRITERIA AND PROCEDURES FOR PROMOTION AND TENURE

SCHOOL OF URBAN AFFAIRS & PUBLIC POLICY CRITERIA AND PROCEDURES FOR PROMOTION AND TENURE The School of Urban Affairs and Public Policy is an interdisciplinary graduate and professional school, designated

### 350 Serra Mall, Stanford, CA 94305-9515

Meisam Razaviyayn Contact Information Room 260, Packard Building 350 Serra Mall, Stanford, CA 94305-9515 E-mail: meisamr@stanford.edu Research Interests Education Appointments Large scale data driven optimization

### PRACTICE BOOK COMPUTER SCIENCE TEST. Graduate Record Examinations. This practice book contains. Become familiar with. Visit GRE Online at www.gre.

This book is provided FREE with test registration by the Graduate Record Examinations Board. Graduate Record Examinations This practice book contains one actual full-length GRE Computer Science Test test-taking

### Vilnius University. Faculty of Mathematics and Informatics. Gintautas Bareikis

Vilnius University Faculty of Mathematics and Informatics Gintautas Bareikis CONTENT Chapter 1. SIMPLE AND COMPOUND INTEREST 1.1 Simple interest......................................................................

### 4. KNOW PARLIAMENTARY PROCEDURE

Duties of the President 1. PLAN * Plan the business meeting with the leader(s) and other officers before the meeting. This can be done over the phone if necessary, and it helps maintain order in the meeting.

### The Applied and Computational Mathematics (ACM) Program at The Johns Hopkins University (JHU) is

The Applied and Computational Mathematics Program at The Johns Hopkins University James C. Spall The Applied and Computational Mathematics Program emphasizes mathematical and computational techniques of

### PRACTICE BOOK MATHEMATICS TEST (RESCALED) Graduate Record Examinations. This practice book contains. Become familiar with

This book is provided FREE with test registration by the Graduate Record Examinations Board. Graduate Record Examinations This practice book contains one actual full-length GRE Mathematics Test (Rescaled)

### SECTION 0.6: POLYNOMIAL, RATIONAL, AND ALGEBRAIC EXPRESSIONS

(Section 0.6: Polynomial, Rational, and Algebraic Expressions) 0.6.1 SECTION 0.6: POLYNOMIAL, RATIONAL, AND ALGEBRAIC EXPRESSIONS LEARNING OBJECTIVES Be able to identify polynomial, rational, and algebraic

### Row Ideals and Fibers of Morphisms

Michigan Math. J. 57 (2008) Row Ideals and Fibers of Morphisms David Eisenbud & Bernd Ulrich Affectionately dedicated to Mel Hochster, who has been an inspiration to us for many years, on the occasion

### 8.7 Mathematical Induction

8.7. MATHEMATICAL INDUCTION 8-135 8.7 Mathematical Induction Objective Prove a statement by mathematical induction Many mathematical facts are established by first observing a pattern, then making a conjecture

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### Introduction to Finite Fields (cont.)

Chapter 6 Introduction to Finite Fields (cont.) 6.1 Recall Theorem. Z m is a field m is a prime number. Theorem (Subfield Isomorphic to Z p ). Every finite field has the order of a power of a prime number

### An Extension Model of Financially-balanced Bonus-Malus System

An Extension Model of Financially-balanced Bonus-Malus System Other : Ratemaking, Experience Rating XIAO, Yugu Center for Applied Statistics, Renmin University of China, Beijing, 00872, P.R. China Phone:

### The van Hoeij Algorithm for Factoring Polynomials

The van Hoeij Algorithm for Factoring Polynomials Jürgen Klüners Abstract In this survey we report about a new algorithm for factoring polynomials due to Mark van Hoeij. The main idea is that the combinatorial

### Communication on the Grassmann Manifold: A Geometric Approach to the Noncoherent Multiple-Antenna Channel

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 48, NO. 2, FEBRUARY 2002 359 Communication on the Grassmann Manifold: A Geometric Approach to the Noncoherent Multiple-Antenna Channel Lizhong Zheng, Student

### Cyclotomic Extensions

Chapter 7 Cyclotomic Extensions A cyclotomic extension Q(ζ n ) of the rationals is formed by adjoining a primitive n th root of unity ζ n. In this chapter, we will find an integral basis and calculate

### CSCMP Roundtable Marketing Guidebook

CSCMP Roundtable Marketing Guidebook 2 Table of Contents Section 1 Introduction 4 CSCMP Mission CSCMP Vision CSCMP Goals Section 2 Roundtable Marketing Objectives 4 Support the CSCMP Mission, Vision and

### Celebrate National Medical Assistants Recognition Week October 21 25, 2013

Celebrate National Medical Assistants Recognition Week October 21 25, 2013 Registered Medical Assistants (RMA) and the American Medical Technologists (AMT) will be celebrating National Medical Assistants

MA 134 Lecture Notes August 20, 2012 Introduction The purpose of this lecture is to... Introduction The purpose of this lecture is to... Learn about different types of equations Introduction The purpose

### Juniper Networks Global Bandwidth Index

Juniper Networks Global Bandwidth Index Published December 2014 Summary In 1994, only 25 million people had access to the Internet. This year that number will reach three billion, including 2.3 billion

### Diablo Valley College Catalog 2014-2015

Mathematics MATH Michael Norris, Interim Dean Math and Computer Science Division Math Building, Room 267 Possible career opportunities Mathematicians work in a variety of fields, among them statistics,

### DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN

DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN How to Be a Distinguished Club Rev. 4/2015 WHERE LEADERS ARE MADE DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN How to be a Distinguished Club TOASTMASTERS

### CHAPTER SIX IRREDUCIBILITY AND FACTORIZATION 1. BASIC DIVISIBILITY THEORY

January 10, 2010 CHAPTER SIX IRREDUCIBILITY AND FACTORIZATION 1. BASIC DIVISIBILITY THEORY The set of polynomials over a field F is a ring, whose structure shares with the ring of integers many characteristics.

### DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN

DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN How to Be a Distinguished Club Rev. 4/2014 WHERE LEADERS ARE MADE DISTINGUISHED CLUB PROGRAM AND CLUB SUCCESS PLAN How to be a Distinguished Club TOASTMASTERS

### College of Medicine Promotion and Tenure Procedure FLORIDA ATLANTIC UNIVERSITY

College of Medicine Promotion and Tenure Procedure FLORIDA ATLANTIC UNIVERSITY Approved by majority vote of College Faculty March 25, 2014 Table of Contents Introduction... 1 Governance... 1 a. Membership...

### 2015 Media Planning Handbook

2015 Media Planning Handbook Promoting the U.S. Professors of the Year Award www.usprofessorsoftheyear.org 2015 U.S. PROFESSORS OF THE YEAR MEDIA PLANNING HANDBOOK 1 TABLE OF CONTENTS Capitalizing on Award

### 5. Factoring by the QF method

5. Factoring by the QF method 5.0 Preliminaries 5.1 The QF view of factorability 5.2 Illustration of the QF view of factorability 5.3 The QF approach to factorization 5.4 Alternative factorization by the

### 4. Factor polynomials over complex numbers, describe geometrically, and apply to real-world situations. 5. Determine and apply relationships among syn

I The Real and Complex Number Systems 1. Identify subsets of complex numbers, and compare their structural characteristics. 2. Compare and contrast the properties of real numbers with the properties of

### Homework 5 Solutions

Homework 5 Solutions 4.2: 2: a. 321 = 256 + 64 + 1 = (01000001) 2 b. 1023 = 512 + 256 + 128 + 64 + 32 + 16 + 8 + 4 + 2 + 1 = (1111111111) 2. Note that this is 1 less than the next power of 2, 1024, which

### 1 Limiting distribution for a Markov chain

Copyright c 2009 by Karl Sigman Limiting distribution for a Markov chain In these Lecture Notes, we shall study the limiting behavior of Markov chains as time n In particular, under suitable easy-to-check

### On the greatest and least prime factors of n! + 1, II

Publ. Math. Debrecen Manuscript May 7, 004 On the greatest and least prime factors of n! + 1, II By C.L. Stewart In memory of Béla Brindza Abstract. Let ε be a positive real number. We prove that 145 1

### Analysis of an Artificial Hormone System (Extended abstract)

c 2013. This is the author s version of the work. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purpose or for creating

### Continued Fractions and the Euclidean Algorithm

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction

The Graduate School: A Model to Enhance Quantity and Quality in PhD Production Maresi Nerad South African PhD Panel Report October 2009 South Africa s universities are confronted with two major challenges.

### Some Polynomial Theorems. John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.

Some Polynomial Theorems by John Kennedy Mathematics Department Santa Monica College 1900 Pico Blvd. Santa Monica, CA 90405 rkennedy@ix.netcom.com This paper contains a collection of 31 theorems, lemmas,

### The Expectation Maximization Algorithm A short tutorial

The Expectation Maximiation Algorithm A short tutorial Sean Borman Comments and corrections to: em-tut at seanborman dot com July 8 2004 Last updated January 09, 2009 Revision history 2009-0-09 Corrected

### Panel on Emerging Cyber Security Technologies. Robert F. Brammer, Ph.D., VP and CTO. Northrop Grumman Information Systems.

Panel on Emerging Cyber Security Technologies Robert F. Brammer, Ph.D., VP and CTO Northrop Grumman Information Systems Panel Moderator 27 May 2010 Panel on Emerging Cyber Security Technologies Robert

### Network Consulting Engineer

Brochure Network Consulting Engineer February, 2012 2012 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public Information. Page 1 of 7 The Cisco Support Center in Krakow To understand

### IEEE INSTRUMENTATION AND MEASUREMENT SOCIETY CHAPTER CHAIR REFERENCE GUIDE

IEEE INSTRUMENTATION AND MEASUREMENT SOCIETY CHAPTER CHAIR REFERENCE GUIDE 1. INTRODUCTION This reference guide is intended to assist you in your role as a Chapter Officer. It relies on and derives from

### Capacity Limits of MIMO Channels

Tutorial and 4G Systems Capacity Limits of MIMO Channels Markku Juntti Contents 1. Introduction. Review of information theory 3. Fixed MIMO channels 4. Fading MIMO channels 5. Summary and Conclusions References

### A Survey on Bilingual Teaching in Higher Education Institute in the Northeast of China

ISSN 1798-4769 Journal of Language Teaching and Research, Vol. 1, No. 4, pp. 353-357, July 2 2 ACADEMY PUBLISHER Manufactured in Finland. doi:.44/jltr.1.4.353-357 A Survey on Bilingual Teaching in Higher

### Lecture Notes: Basic Concepts in Option Pricing - The Black and Scholes Model

Brunel University Msc., EC5504, Financial Engineering Prof Menelaos Karanasos Lecture Notes: Basic Concepts in Option Pricing - The Black and Scholes Model Recall that the price of an option is equal to

### By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

This material is posted here with permission of the IEEE Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology's products or services Internal

### An On-Line Algorithm for Checkpoint Placement

An On-Line Algorithm for Checkpoint Placement Avi Ziv IBM Israel, Science and Technology Center MATAM - Advanced Technology Center Haifa 3905, Israel avi@haifa.vnat.ibm.com Jehoshua Bruck California Institute

### 2015 Russian Nanotechnology Investment Enabling Technology Leadership Award

2015 Russian Nanotechnology Investment Enabling Technology Leadership Award 2015 Contents Background and Company Performance... 3 Technology Leverage and Customer Impact of OJSC RUSNANO... 3-5 Conclusion...

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### Statement of. William D. Green. Executive Chairman of the Board, Accenture. and

Statement of William D. Green Executive Chairman of the Board, Accenture and Member, Committee on Research Universities Board on Higher Education and Workforce Policy and Global Affairs National Research

### A Direct Numerical Method for Observability Analysis

IEEE TRANSACTIONS ON POWER SYSTEMS, VOL 15, NO 2, MAY 2000 625 A Direct Numerical Method for Observability Analysis Bei Gou and Ali Abur, Senior Member, IEEE Abstract This paper presents an algebraic method

### University of Delaware College of Health Sciences Department of Behavioral Health and Nutrition

University of Delaware College of Health Sciences Department of Behavioral Health and Nutrition GUIDELINES FOR PROMOTION, TENURE AND REVIEW I. INTRODUCTION The mission of the Department of Behavioral Health

### Brian Alan Snyder, Ph. D.

Brian Alan Snyder, Ph. D. Current work address Department of Mathematics and Computer Science Work phone: (906) 635-2658 Lake Superior State Home phone: (906) 253-9822 650 West Easterday Ave. Department

### CRC Website: http://www.acosoc.org/regchapters/

GUIDELINES FOR THE OPERATION OF A REGIONAL CHAPTER OF THE ACOUSTICAL SOCIETY OF AMERICA (rev2010) The objective of these guidelines is to make operating a Chapter as effortless as possible. The suggestions

### IADR Global Oral Health Inequalities Research Network Appointment of Regional Councilors

IADR Global Oral Health Inequalities Research Network Appointment of Regional Councilors IADR - GOHIRN The IADR Global Oral Health Inequalities Research Network (GOHIRN) was approved by the IADR Council

### Master s Theory Exam Spring 2006

Spring 2006 This exam contains 7 questions. You should attempt them all. Each question is divided into parts to help lead you through the material. You should attempt to complete as much of each problem

### Forensic & Investigative Accounting (FIA) Section American Accounting Association Mission, Objectives and 2013-2015 Strategy.

Forensic & Investigative Accounting (FIA) Section American Accounting Association Mission, Objectives and 2013-2015 Strategy Mission The mission of the Forensic & Investigative (FIA) Section of the American

### CHAPTER EIGHT. BEYOND THE GRADUATE PROGRAM: APPLYING FOR POSTDOCTORAL FELLOWSHIPS AND OTHER RESEARCH POSITIONS IN THE FIELD OF THE PhD

CHAPTER EIGHT BEYOND THE GRADUATE PROGRAM: APPLYING FOR POSTDOCTORAL FELLOWSHIPS AND OTHER RESEARCH POSITIONS IN THE FIELD OF THE PhD APPLYING FOR A POSTDOCTORAL POSITION IN THE SCIENCES The postdoctoral

### Induction. Mathematical Induction. Induction. Induction. Induction. Induction. 29 Sept 2015

Mathematical If we have a propositional function P(n), and we want to prove that P(n) is true for any natural number n, we do the following: Show that P(0) is true. (basis step) We could also start at

### APPLICATION FOR PART-TIME EMPLOYMENT AS A TUTOR TUTOR IN THE DOLCIANI MATHEMATICS LEARNING CENTER

APPLICATION FOR PART-TIME EMPLOYMENT AS A TUTOR TUTOR IN THE DOLCIANI MATHEMATICS LEARNING CENTER Dear Applicant, As you consider applying for a position in the Dolciani Mathematics Learning Center, there

### Determinants can be used to solve a linear system of equations using Cramer s Rule.

2.6.2 Cramer s Rule Determinants can be used to solve a linear system of equations using Cramer s Rule. Cramer s Rule for Two Equations in Two Variables Given the system This system has the unique solution

### Lecture 1: Elementary Number Theory

Lecture 1: Elementary Number Theory The integers are the simplest and most fundamental objects in discrete mathematics. All calculations by computers are based on the arithmetical operations with integers

### MATHEMATICS. Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Degree Requirements

MATHEMATICS Administered by the Department of Mathematical and Computing Sciences within the College of Arts and Sciences. Paul Feit, PhD Dr. Paul Feit is Professor of Mathematics and Coordinator for Mathematics.

### Fellowship Applications / Nominations

Ratified May 1989 Last reviewed Feb 2015 Review due Feb 2016 Fellowship Applications / Nominations Policy Fellowship of Speech Pathology Australia is one of the highest public professional honours the