Bayesian Hidden Markov Models for Alcoholism Treatment Tria

Transcription

1 Bayesian Hidden Markov Models for Alcoholism Treatment Trial Data May 12, 2008

2 Co-Authors Dylan Small, Statistics Department, UPenn Kevin Lynch, Treatment Research Center, Upenn Steve Maisto, Psychology Department, Syracuse University Dave Oslin, Treatment Research Center, UPenn

3

4 The Problem N subjects measured on T days: daily drink counts. Want to estimate the average treatment effect on outcome. Day Subject

5 Sample Time Series Subject 61 Subject Drinks 2 Drinks Day Day Subject 142 Subject Drinks 2 Drinks Day Day

6 The Goal of Treatment The main goal: Reduce Alcohol Consumption 1. Does the treatment reduce the frequency of all drinking events - or only certain types of drinking events? Is moderate drinking an acceptable outcome? How does the treatment affect different complex drinking patterns and behaviors? 2. Does the treatment reduce the frequency and/or duration of relapses? What is a relapse? Everybody agrees on the notion of a relapse, but there is no concensus for an operational definition of relapse.

7 What is the Outcome? It s complicated. The subjects are recovering alcoholics, whose drinking behaviors are complex processes that evolve and change through time. Simple models lack the structure to adequately describe these processes (Wang, et. al., 2002).

8 Simple Models Time until first drink/relapse (Ignores all behavior after first drink) Percentage of days drinking (Ignores amount of alcohol that is consumed) Multiple failure time models (Requires definition of a relapse) Drinks per Day 3 Y it day

9 HMM Motivation A well-known theory of relapse, the cognitive-behavioral model of relapse (McKay, et. al. 2006, Marlatt and Gordon, 1985), suggests that the cause of a relapse is two-fold: 1. First, the subject must be in a mental and/or physical condition in which he or she is vulnerable to drinking. That is, if presented with an opportunity to drink, the subject would not be able to mount a coping response. 2. Second, the subject must actually encounter such a high-risk drinking situation.

10 HMM structure Y it is the observation for subject i at time t. Y i1 Y i2 Y i, t 1 Y it Y i, t+1 Y i, T 1 Y it H i1 H i2 H i, t 1 H it H i, t+1 H i, T 1 H it H it is the hidden state for subject i at time t.

11 A Simple HMM with no covariates The complete-data likelihood for an HMM factors into three parts: p(y, H θ) = N p(h i1 θ) (1) i=1 N i=1 t=2 N i=1 t=1 T p(h it H (i,t 1),θ) (2) T p(y it H it,θ), (3) where Y and H denote observations and hidden states, and parts (1), (2), and (3) refer to the initial state distribution, the hidden state transitions, and the observations, respectively.

12 Simple HMM Fit: S=5 Fit multinomial distributions for hidden state transitions and observations conditional on hidden states. Data is pooled across individuals: ˆπ = (.79,.11,.01,.07,.01) ˆQ = ˆP = where ˆQ and ˆP denote the hidden state transition matrix, and the observation distributions, respectively.

13 Interpretation of hidden states for S=5 1. Large probabilities on the diagonal of ˆQ hidden states are persistent. 2. Observation Distributions are clinically interpretable: Y it = 1 Y it = 2 Y it = A IM ˆP = SM IH C A SH Abstinence Intermittent Moderate Drinking Steady Moderate Drinking Intermittent Heavy Drinking Steady Heavy Drinking Fitting additional latent states (S = 6, 7) yielded no additional interpretable drinking behaviors.

14 Choosing the number of Hidden States 10-fold CV to make out-of-sample predictions; measure deviance N T D = 2 log ˆP(Y it = y it ). i=1 t=11 HMM Markov MTD Deviance Number of Hidden States Order Number of Lags

15 Question 1: Is Moderate Drinking OK? Question: If the hidden states are persistent, can a subject drink moderately, and not resort to heavy drinking soon after? Define states 4 and 5 as Relapse States. Probability of Avoiding Relapse as a Function of Time Initial State = 1 (A) Initial State = 2 (IM) Initial State = 3 (SM) Probability Day

16 Question 2: What is a Relapse? Currently, there is no universally agreed upon operational definition of relapse. Furthermore, different definitions can have an impact on the estimates of treatments (Maisto, et. al, 2003). Any drink of alcohol A day of heavy drinking Four consecutive drinking days (any amount of alcohol) Any drink of alcohol that follows at least 4 days of abstinence The HMM offers a new data-based definition: Any time point at which a subject has a high probability of being in hidden state 4 or 5 ( Intermittent Heavy Drinking or Steady Heavy Drinking ). Estimate the most likely hidden state sequence for each subject using the Viterbi algorithm.

17 Most Likely Sequence 1 Subject (SH) 4(IH) Y it 2 3(SM) Latent State 2(IM) 1 1(A) day

18 Most Likely Sequence (2) Subject (SH) 4(IH) Y it 2 3(SM) Latent State 2(IM) 1 1(A) day

19 A More Complex HMM Incorporate Covariates, possibly time-varying Random Effects Missing data, assuming MAR

20 The Model For hidden state transition probabilities, use a multinomial logit model, where P(H it = s H (i,t 1) = r, X it, β) = exp(xq it β rs ) k exp(xq it β rk ). β rs0 N(µ rs, σ rs ). For observation probabilities, use an ordinal probit model, where P(y sj = 1) P(y sj = 2) P(y sj = 3) x P sj β s 2 4 γ s1 γ s2

21 The Hidden State Transition Matrix The hidden state transition matrix parameters are organized as follows (for S = 3 hidden states): {0} (0,0,...,) {β 120i } (β 121,β 122,...) {β 130i } (β 131,β 132,...) 2 {0} (0,0,...,) {β 220i } (β 221,β 222,...) {β 230i } (β 231,β 232,...) 3 {0} (0,0,...,) {β 320i } (β 321,β 322,...) {β 330i } (β 331,β 332,...) where braces {β} denote a set of random effects, and the rest are fixed effects.

22 The Data The outcome (N = 240 subjects and T = 168 days) is distributed as follows: Y % Missing 17 Total 100

23 Covariates This clinical trial, conducted at UPenn s Treatment Research Center, had 6 arms: treatment/control for Naltrexone, and two therapies vs. control. In the hidden state transition matrix, we include: 1. Treatment (Naltrexone) 2. Therapy 1 3. Therapy 2 4. Female 5. Time In the observation model, we include: 1. Weekend indicator 2. Past Drinking Behavior

24 The Gibbs Sampler 1. Initialize the parameters θ = (β, η, γ, π, µ, σ). 2. H Y obs, θ from its full conditional distribution by evaluating the likelihood using the forward recursion, and then using a stochastic backward recursion for all subjects i = 1, 2,..., N (Scott, 2002). 3. β H from its posterior using Scott s DAFE algorithm (2007), which involves augmented variables and a Metropolis-Hastings step, or using a random-walk Metropolis step. 4. µ β, σ from their full conditional distributions, assuming flat or weakly informative priors (Gelman, forthcoming). 5. σ β, µ from their full conditional distributions. 6. Y mis H, η, γ assuming it is missing at random (MAR) using the current batch of parameters. 7. γ H, Y obs, Y mis, η using Cowles (1996) random-walk Metropolis-Hastings step. 8. η H, Y obs, Y mis, γ in the standard data augmentation way (Albert and Chib 1993). 9. π H from its full conditional Dirichlet distribution. 10. Repeat steps 2-9 for g = 2,..., G.

25 Characterizing the Fit: S = 3 ˆπ = (.94,.04,.02) ˆQ = ˆP =

26 The Treatment Effect (Treat = Red, Control = Black) Q(1,1) Q(1,2) Q(1,3) Density Density Density Q(2,1) Q(2,2) Q(2,3) Density Density Density Q(3,1) Q(3,2) Q(3,3) Density Density Density

27 Missing Data Subject Drinks Day

28 Hidden States Posterior Distribution Subject Drinks Day

29 Missing Data Posterior Distribution Subject Drinks Day

30 Missing Data Subject 61 3 Drinks Day

31 Hidden States Posterior Distribution Subject Drinks Day

32 Missing Data Posterior Distribution Subject 61 3 Drinks Day

33 An HMM is a model with a rich structure that can capture complex drinking behaviors as they evolve through time. It corresponds to a well-known theoretical model for relapse, the cognitive-behavioral model of relapse. We can (1) assess the danger of moderate drinking, and (2) define relapse in a data-based way. We can measure treatment effects. We can fit the model to subjects with incomplete data, and we can incorporate random effects.

34 Thanks! shirley