Robert Piché Tampere University of Technology

Size: px

Start display at page:

Download "Robert Piché Tampere University of Technology"

Bethany Hopkins
7 years ago
Views:

1 model fit: mu Statistical modelling with WinBUGS Robert Piché Tampere University of Technology diff sample: y/n !1 0 1 x Literature WinBUGS free download Read the first 15 pages of For a full intro to Bayesian statistics, take my course math.tut.fi/~piche/bayes My course is based on Antti Penttinen s course users.jyu.fi/~penttine/bayes09/

5-0.25 0.0 0.25 0!1 0 1 x Literature WinBUGS free download www.mrc-bsu.cam.ac.uk/bugs/ Read the first 15 pages of www.stat.uiowa.

2 in this lesson we look at 6 basic statistical models inferring a proportion comparing proportions diff sample: inferring a mean comparing means linear regression model fit: mu slide 3 of 20 logistic regression y/n 0!1 0 1 x

5-0.25 0.0 0.25 comparing means linear regression 35.0 30.0 25.

3 inferring a proportion What proportion of the population of Ireland supports the Lisbon agreement? (let s call this:!) When we ask n Irish adults, s of them say that they are in favour. What can we conclude about!? 1. construct an observation model p( s! ) 2. construct a prior model p(!) 3. compute posterior p(! s ) with WinBUGS slide 4 of 20

) When we ask n Irish adults, s of them say that they are in favour.

4 inferring a proportion (2) Observation model Let yi = 1 if person i is in favour ( success ), otherwise yi = 0 assume: given!, the probability that case i is a success is!, i.e. p(y i θ)= assume: given!, the observations are statistically independent, so p(y 1:n θ)= θ (yi = 1) 1 θ (y i = 0) n! θ y i (1 θ) 1 y i = θ s (1 θ) n s, i=1 = θ y i (1 θ) 1 y i (y i {0,1}) where s = " n i=1 y i Thus, given!, s is a random variable with Binomial distribution s θ Binomial(θ,n) slide 5 of 20

, the observations are statistically independent, so p(y 1:n θ)= θ (yi = 1) 1 θ (y i = 0) n!

5 inferring a proportion (3) The observation model ( likelihood ) tells us how we could generate random s, given! : s θ Binomial(θ,n) Inference is the inverse problem: given s, what is!? Bayesian inference The unknown proportion! is treated like a random variable Its probability density function (pdf) lives in [0, 1] The prior pdf ( before observation ) and posterior pdf are related by Bayes law: p(θ s)= p(s θ)p(θ) p(s θ)p(θ)dθ 2 p(!) prior posterior 20 p(! y) slide 6 of !

is treated like a random variable Its probability density function (pdf) lives in [0, 1] The prior pdf ( before

6 inferring a proportion (4) The prior, p(!) It is our state of belief about! before we make observation We can use any pdf that lives in [0, 1], e.g. Unif(0, 1) It s convenient to use Beta distribution, with 2 parameters ">0, #>0 x p(x) x E(x) mode(x) V(x) { } (µ σ ) "(α+β) Beta(α,β) (x µ)2 R µ µ σ "(α)"(β) xα 1 (1 x) β 1 α α 1 αβ [0,1] α+β α+β 2 (α+β) 2 (α+β+1) (λ) λ λ { } λ λ λ Beta(1, 1) = Unif small " and # give vague prior slide 7 of 20 plot the Beta pdf with Matlab s disttool

Unif(0, 1) It s convenient to use Beta distribution, with 2 parameters ">0, #>0 x p(x) x E(x) mode(x) V(x) { } (µ σ )

7 inferring a proportion (5) WinBUGS model model { s dbin(theta,n) theta dbeta(1,0.667) ypred } dbin(theta,1) data list(s=650,n=1000) initialisation list(theta=0.1) p(!) ! try dbeta(1,1) try s=65, n=100 try s=6500, n=10000 results node mean sd 2.5% median 97.5% theta slide 8 of 20 95% credibility interval is [0.6185,0.6802]

) 2 0 0 1! try dbeta(1,1) try s=65, n=100 try s=6500, n=10000 results node mean sd 2.

8 comparing proportions In a study of larynx cancer patients, s1 of the n1 patients who were treated with radiation therapy were cured, compared to s2 of the n2 patients who were treated with surgery. What can we say about!1 (success rate of radiation) vs!2 (surgery)? 1. construct an observation model p( s! ) 2. construct a prior model p(!) 3. compute posterior p(! s ) with WinBUGS 4. compute posterior probability that (!1 $!2 ) slide 9 of 20

What can we say about!1 (success rate of radiation) vs!2 (surgery)? 1. construct an observation model p( s!

9 comparing proportions (2) Observation model assume: given!=(!1,!2), the probability that a radiation therapy patient is cured is!1 and the probability that a surgery therapy patient is cured is!2 assume: given!, the observations are independent i.e. p(s 1,s 2 θ 1,θ 2 )=p(s 1 θ 1 )p(s 2 θ 2 ) s 1 θ 1 Binomial(θ 1,n 1 ) s 2 θ 2 Binomial(θ 2,n 2 ) The prior, p(!1,!2) independent vague slide 10 of 20 p(θ 1,θ 2 )=p(θ 1 )p(θ 2 ) θ 1 Beta(0.5,0.5) θ 2 Beta(0.5,0.5) plot these with dissttool

1 and the probability that a surgery therapy patient is cured is!2 assume: given!, the observations are independent i.e. p(s 1,s 2 θ 1,θ 2 )=p(s 1 θ 1 )p(s 2 θ 2 ) s 1 θ 1 Binomial(θ 1,n 1 ) s 2 θ 2 Binomial(θ 2,n 2 ) The prior, p(!

10 comparing proportions (3) WinBUGS model { s_1 ~ dbin(theta_1,n_1) s_2 ~ dbin(theta_2,n_2) theta_1 ~ dbeta(0.5,0.5) theta_2 ~ dbeta(0.5,0.5) diff <- theta_1-theta_2 P <- step(diff) } # data list(s_1=15,n_1=18,s_2=21,n_2=23) # initialisation list(theta_1=0.8,theta_2=0.8) variables that are deterministic functions of stochastic variables are specified with <- step(diff) = 1 if diff! 0, = 0 otherwise comments are indicated with # slide 11 of 20 results node mean sd 2.5% median 97.5% theta theta diff P diff sample:

8) variables that are deterministic functions of stochastic variables are specified with <- step(diff) = 1 if diff!

11 inferring a mean In 1798, Henry Cavendish performed experiments to measure the specific density of the Earth (!). He repeated the experiment n times, obtaining results y1, y2,..., yn. What can we conclude about!? Observation model Assume that observation noise is zero-mean gaussian with precision ", and that noises are independent given " and! y i = µ + e i slide 12 of 20 e i µ,τ Normal(0,τ) p(y 1,...,y n µ,τ)= n i=1 p(y i µ,τ) precision = 1/variance

org/pss/106988 Observation model Assume that observation noise is zero-mean gaussian with precision ", and that noises

12 inferring a mean (2) The prior, p(",#) µ~gamma(10,2) Assume independence: p(µ, τ)=p(µ)p(τ) We can use any pdfs that live in [0,!) It s convenient to use Gamma distributions small " and # give vague prior x p(x) { x } E(x) β Gamma(α,β) µ σ α "(α) xα 1 e βx ( µ) α (0,#) R µ β (λ) λ λx 1 ( #) p(µ) 0 10 granite 2.5 µ! ~ Gamma(2.5,0.1) lead ore 7.5 p(!) slide 13 of 20 note: Matlab s Gamma parameters are A= ", B= 1/# "=25 #= !

) It s convenient to use Gamma distributions small " and # give vague prior x p(x) { x } E(x) β Gamma(α,β)

13 WinBUGS inferring a mean (3) model { mu ~ dgamma(a_mu,b_mu) tau ~ dgamma(a_tau,b_tau) for (i in 1:n) { y[i] ~ dnorm(mu,tau) } } # data try y1 = 15.36, i.e. outlier try robust distribution y[i] ~ dt(mu,tau,4) list(y=c(5.36,5.29,5.58,5.65,5.57,5.53,5.62,5.29,5.44,5.34, 5.79,5.10,5.27,5.39,5.42,5.47,5.63,5.34,5.46,5.30, 5.78,5.68,5.85),n=23,a_mu=10,b_mu=2,a_tau=2.5,b_tau=0.1) # initialisation list(mu=5,tau=25) slide 14 of 20 results node mean sd 2.5% median 97.5% mu mu sample:

79,5.10,5.27,5.39,5.42,5.47,5.63,5.34,5.46,5.30, 5.78,5.68,5.85),n=23,a_mu=10,b_mu=2,a_tau=2.5,b_tau=0.

14 comparing means Cuckoo eggs found in m dunnock nests have diameters x1, x2,..., xn (mm). Cuckoo eggs found in n sedge warbler nests have diameters y1, y2,..., yn (mm). Do cuckoos lay bigger eggs in the nests of dunnocks than in the nests of sedge warblers? Observation model p(x 1,...,x n,y 1,...,y n µ x,τ x, µ y,τ y )= n i=1 p(x i µ x,τ x )p(y i µ y,τ y ) x i µ x,τ x Normal(µ x,τ x ), y i µ y,τ y Normal(µ y,τ y ) Prior p(µ x,τ x, µ y,τ y )=p(µ x )p(τ x )p(µ y )p(τ y ) µ x Gamma(0.22,.01), τ x Gamma(0.1,0.1) µ y Gamma(0.22,.01), τ y Gamma(0.1,0.1) plot these with dissttool slide 15 of 20

Do cuckoos lay bigger eggs in the nests of dunnocks than in the nests of sedge warblers? Observation model p(x 1,...,x n,y 1,.

15 WinBUGS results comparing means (2) model { for(i in 1:m){ x[i] ~ dnorm(mu_x,tau_x) } for(i in 1:n){ y[i] ~ dnorm(mu_y,tau_y) } mu_x ~ dgamma(0.22,0.01) mu_y ~ dgamma(0.22,0.01) tau_x ~ dgamma(0.1,0.1) tau_y ~ dgamma(0.1,0.1) diff <- mu_x - mu_y P <- step(diff) } # data list(x=c(22,23.9,20.9,23.8,25,24,21.7,23.8,22.8,23.1),m=10, y=c(23.2,22,22.2,21.2,21.6,21.9,22,22.9,22.8),n=9) # init list(mu_x=22,mu_y=22,tau_x=1,tau_y=1) do the sizes of cuckoo eggs in dunnock nests have greater variance than those in sedge warbler nests? diff sample: 4500 slide 16 of 20 node mean sd 2.5% median 97.5% diff P

8,22.8,23.1),m=10, y=c(23.2,22,22.2,21.2,21.6,21.9,22,

16 linear regression In1875, Scottish physicist James D. Forbes published a study relating data on the boiling temperature of water x1, x2,..., xn (deg F) and the atmospheric pressure y1, y2,..., yn (inches of Hg). If water boils at 190 deg F, what is the atmospheric pressure? Observation model p(y 1,...,y n µ 1,...,µ n,τ)= i y i µ i,τ Normal(µ i,τ) ln(µ i )=α + β(x i x) p(y i µ i,τ) physics predicts a straight line fit to ln(y) as a function of x slide 17 of 20 Prior p(α,β,τ)=p(α)p(β)p(τ) α Normal(0,10 6 ), β Normal(0,10 6 ) τ Gamma(0.001, 0.001)

17 slide 18 of 20 WinBUGS results linear regression (2) model { x_bar <- mean(x[ ]) for ( i in 1 : n ) { log(mu[i]) <- alpha+beta*(x[i]-x_bar) y[i] ~ dnorm(mu[i],tau) } alpha ~ dnorm( 0.0,1.E-6) beta ~ dnorm( 0.0,1.E-6) tau ~ dgamma(0.001,0.001) y190 <- exp(alpha+beta*(190-x_bar)) } # data model fit: mu list( x=c(210.8, 210.2, 208.4, 202.5, 200.6, 200.1, 199.5, 197, 196.4, 196.3, 195.6, 193.4, 193.6, 191.4, 191.1, 190.6, 189.5, 188.8, 188.5, 185.7, 186, 185.6, 184.1, 184.6, 184.1, 183.2, 182.4, 181.9, 181.9, , 180.6), P=c(29.211, , , , , , 23.03, , , , , 20.48, , , 19.49, , , , , , , , , , , , , , , , ), n=31) # init list(alpha=0,beta=0,tau=1) try y[i] ~ dt(mu[i],tau,4) node mean sd 2.5% median 97.5% y Inference > Compare node = mu other = y axis = x

5, 197, 196.4, 196.3, 195.6, 193.4, 193.6, 191.4, 191.1, 190.6, 189.5, 188.8, 188.5, 185.7, 186, 185.6, 184.1, 184.6, 184.1, 183.2, 182.4, 181.9, 181.9, 181.15, 180.6), P=c(29.211, 28.559, 27.972, 24.

18 logistic regression n1 lab mice are injected with a substance at log concentration x1, and y1 of them die. The experiment is repeated 4 times with different concentrations, yielding further data (n2, x2, y2), (n3, x3, y3), (n4, x4, y4). What dosage corresponds to a 50% chance of mortality? Observation model y i θ i Binomial(θ i,n i ) 1 1/2! slide 19 of 20 Prior logit(θ i ) log θ i 1 θ i α Normal(0,.001), = α + βx i 0 p(α,β)=p(α)p(β) β Normal(0,.001)!"/# plot prior with dissttool (note: Matlab s parametrization of Normal differs from WinBUGS usage) x

What dosage corresponds to a 50% chance of mortality? Observation model y i θ i Binomial(θ i,n i ) 1 1/2!

19 WinBUGS logistic regression (2) model { for (i in 1:nx) { logit(theta[i]) <- alpha + beta*x[i] y[i] ~ dbin(theta[i],n[i]) } alpha ~ dnorm(0.0,0.001) beta ~ dnorm(0.0,0.001) LD50 <- (logit(0.50)-alpha)/beta } # data list(y=c(0,1,3,5), n=c(5,5,5,5), x=c(-0.863,-0.296,-0.053,0.727), nx=4) # init list(alpha=0,beta=1) what log-concentration corresponds to a mortality probability of 1%? x i n i y i slide 20 of 20 results node mean sd 2.5% median 97.5% alpha beta LD y/n 0!1 0 1 x

727), nx=4) # init list(alpha=0,beta=1) what log-concentration corresponds to a mortality probability of 1%? x i n i y i -0.863 5 0-0.296 5 1-0.053 5 3 0.

1 Prior Probability and Posterior Probability

Math 541: Statistical Theory II Bayesian Approach to Parameter Estimation Lecturer: Songfeng Zheng 1 Prior Probability and Posterior Probability Consider now a problem of statistical inference in which