Choosing Outcome Measures in Pediatric IBD. Anne M Griffiths, MD Hospital for Sick Children University of Toronto Toronto, Canada

Transcription

1 Choosing Outcome Measures in Pediatric IBD Anne M Griffiths, MD Hospital for Sick Children University of Toronto Toronto, Canada

2 Disclosures The content of this presentation reflect my personal opinions based on data review and may differ from the opinions of others, including the FDA I was the Principal investigator for the development of the PUCAI and collaborator in the development of the PCDAI (J Hyams PI) I have participated in multiple studies evaluating the PUCAI and PCDAI subsequent to their development PUCAI is copyrighted to SickKids Hospital, Toronto; modest user fees are payable to SickKids Research Institute for industrysponsored clinical trials

3 Objectives Understand principles of multi-item measure development and assessment Discuss specific outcome measures for pediatric IBD Disease activity in UC Disease activity in CD 3

4 Measuring disease activity Disease activity is not adequately represented by any single laboratory test Therefore multi-item measures have been developed to assess activity

5 Derivation of a multi-item measure Development Assessment Item generation Sensibility Item reduction Formatting, grading Weighting of items Reliability Validity Responsiveness Definitions of response and remission

6 Considerations in development process Item generation and reduction: want to identify all items or variables (item generation) and then reduce the number via a process which selects out those contributing most to accurate measurement of the concept

7 Considerations in assessment SENSIBILITY (includes Content Validity) Does the index seem appropriate in content given its purpose; is it user friendly? RELIABILITY/REPRODUCIBILITY Are similar scores obtained on subsequent assessments if no change in disease status has occurred? VALIDITY How well does the index measure what it was intended to measure? RESPONSIVENESS Is the score sensitive to a change in disease status?

8 How to choose an outcome measure? 1. What does it measure (i.e. validity)? 2. Is it accurate (i.e. reliability)? 3. Can is discriminate between categories? 4. Can it detect change (responsiveness)? 5. Is it feasible?

9 Measures of disease activity in ulcerative colitis Pediatric Ulcerative Colitis Activity Index (PUCAI) Several UC activity indices used in adult studies Mayo score Seo index: seldom used, but the most rigorously developed of the adult indices others

10 ITEM 1. Stool frequency Normal number for patient 1-2 more than normal 3-4 more than normal >/= 5 more than normal 2. Rectal bleeding No blood seen Streaks of blood with stool less than half the time Obvious blood with stool most of the time Blood alone passed 3. Sigmoidoscopic appearance Normal Mild disease (erythema, decreased vascular pattern, mild friability) Moderate (marked erythema, absent vascular pattern, friability, erosions) Severe (spontaneous bleeding, ulceration) 4. Physician global assessment Normal Mild disease Moderate disease Severe disease POINTS The Mayo score Endoscopic subscore widely used to define endoscopic remission Schroeder KW et al, NEJM 1987; 317: SUM 0-12

11 ITEM 1. Abdominal pain No pain Pain can be ignored Pain cannot be ignored 2. Rectal bleeding None Small amount only in < 50% of stools Small amount with most stools Large amount (>50% of the stool content) 3. Stool consistency of most stools Formed Partially formed Completely unformed 4. Number of stools per 24 hours >8 POINTS Nocturnal bowel movement (any diarrhea episode causing wakening) No Yes 6. Activity level No limitation of activity Occasional limitation of activity Severe restricted activity SUM OF PUCAI (0-85) The Paediatric UC Activity Index (PUCAI) Remission: <10 Mild: Moderate: Severe: Turner D et al; Gastroenterology 2007;133: Validation in adults: Turner D, Seow C et al: Clin Gastro Hepatol 2009; 7:

12 The history of the PUCAI Development began as a result of CCFA-sponsored Pediatric Clinical Trials workshop (December 2004) No existing measure of UC activity (adult or pediatric) had been rigorously formulated and consistently used Colonoscopy was the gold standard for assessing activity of UC in adult clinical trials, but mandatory repeated colonoscopic assessments would deter children and families from participating in trials

13 Concept, Purpose, Population (what?; why?; where?) We wanted an instrument for use in clinical trials, which would evaluate activity (inflammation) in children with ulcerative colitis Could serve to defining inclusion criteria and as endpoint i.e. would tell us does the colon look like or or

14 Symptoms versus inflammatory Activity Activity implies symptoms due to inflammation There are other causes of symptoms in IBD irritable bowel symptoms in UC strictured disease in CD pyschophysiologic symptoms or anxiety-related magnification of symptoms non-ibd causes of symptoms adverse drug effects

15 Considerations in development process Item generation and reduction: want to identify all items or variables (item generation) and then reduce the number via a process which selects out those contributing most to accurate measurement of disease activity (the concept) So..we asked experts what they relied on to make the judgment how active is the child s colitis? ( how inflamed is the colon?) and therefore how should I treat it?.. how do you, the physician, judge whether the colon looks like. :? or? or?

16 PUCAI development by a Delphi group of pediatric IBD specialists Literature search 20 items added by 48 IBD experts Generation of 21 items 41 items Item Generation IBD Panel Literature search Rank order by the IBD experts 11 items in final index Item Reduction IBD Panel Very careful wording of response options Input from the IBD experts Draft PUCAI for exploring by the patients Item Gradation

17 PUCAI-Phases of development and Item Generation Item Reduction Item Gradation assessment 157 UC children (2-18yr), enrolled prospectively at 5 pediatric IBD centers. 2 physicians independently completed a PGA of disease activity and PUCAI items. Item Weighting Validity Reliability (inter-observer, test-retest) Responsiveness (sensitivity to change) Define cutoff points PGA X X X 11, betas are the weighting (rounded to nearest 5)

18 ITEM 1. Abdominal pain No pain Pain can be ignored Pain cannot be ignored 2. Rectal bleeding None Small amount only in < 50% of stools Small amount with most stools Large amount (>50% of the stool content) 3. Stool consistency of most stools Formed Partially formed Completely unformed 4. Number of stools per 24 hours >8 POINTS Nocturnal bowel movement (any diarrhea episode causing wakening) No Yes 6. Activity level No limitation of activity Occasional limitation of activity Severe restricted activity SUM OF PUCAI (0-85) The Pediatric UC Activity Index (PUCAI) Remission: <10 Mild: Moderate: Severe: Turner D et al; Gastroenterology 2007;133: Validation in adults: Turner D, Seow C et al: Clin Gastro Hepatol 2009; 7:

19 Top concerns of children and adolescents with chronic ulcerative colitis 1 Medications 2 Unfair 3 Bleeding 4 Worry about flare-up 5 Worry about future health 6 IBD is lifelong 7 Problems Travelling 8 Needing a bathroom fast 9 Angry 10 Diarrhea 11 Cramps/Pain Based on patient interviews and item reduction during development of IMPACT (Griffiths et al. JPGN 1999; 28:S46)

20 PUCAI Rigorously developed over 2 years: Extensively evaluated: as part of original assessment and subsequently Performs well

21 PUCAI is Feasible: Completion rate in the North American Pediatric IBD registry 97.6% P< % completion 47.6% nflam Bowel Dis :

22 PUCAI is reproducible/reliable xcellent inter-observer reliability as measured by intra-class correlation (ICC) for individual items and for total score Total PUCAI ICC= 0.95, 95%CI ). xcellent Test-retest reliability among patients whose disease activity was unchanged at a follow-up visit ICC=0.94, 95%CI astroenterology 2007;133:

23 PUCAI is Valid Construct validity testing (n=48 children) Correlates well with physician global assessment of disease activity, with endoscopic appearance, with Mayo score PGA Endoscopic appearance The Mayo score PUCAI 0.91* 0.76* 0.95* PUCAI+labs 0.90* 0.77* 0.92* PUCAI+ Sigmoidoscopy 0.90* 0.83* 0.97* Lichtiger index 0.83* 0.71* 0.89* Seo index 0.77* 0.69* 0.84* Turner D et al, Gastroenterology 2007;133: *p<0.001

24 PUCAI correlates well with colonoscopic findings Colonoscopic Score (0 15) r=0.78; n=76 PUCAI score astroenterology 2007;133: and unpublished data (Dan Turner)

25 Discriminative validity and cutoffs r=0.91 P<0.001 n=25 Definition AUC of ROC Sens/Spec Severe: > ( ) 96%/ 91% PUCAI score n=81 n=42 n=57 Mod: Mild: None: < ( ) 0.98 ( ) 0.99 (0.99 1) 96%/ 91% 89%/ 94% 95%/ 99% None Based on 205 Mildpatients Moderate severe urner D et al; Gastroenterology 2007;133:

26 Discriminant validity and cutoffs The IBD registry validation n= r=0.87, P<0.001 PUCAI score N=45 N=13 65 points points 15 0 N=100 N=57 Remission Mild Moderate Severe 10 points nflam Bowel Dis :

27 Replication in adults (n=86) 100 r=0.87, P<0.001 PUCAI score N=20 N= points 35 points AUC of ROC (95%CI) 0.97 ( ) 0.96 ( ) N= N=34 10 points 0.92 ( ) Remission Mild Moderate Severe Clin Gastroenterol Hepatol 2009

28 Discriminative validation Differentiating remission from active disease Area under the ROC curve Sensitivity 90% Specificity 86% Turner D et al. CGH 2009;7:

29 Comparison of PUCAI with endoscopy PedUC Pediatric infliximab study: remission rates at week 8 % Hyams et al. Clin Gastro Hepatol 2012

30 Comparison of PUCAI with endoscopy Beclomethasone 17,21 dipropionate (BDP) RCT in mild mod pediatric UC BDP for 8wks Asacol (n=15) Asacol 80mg/kg (n=15) Remission rate by 12 weeks P<0.05 P<0.05 omano et al. JPGN 2010; 50: 385-9

31 A PUCAI<10 points reflects complete endoscopic remission Complete endoscopic remission is the desirable outcome as it is associated with better outcome than partial mucosal healing (i.e. Mayo subscore of 1) 31

32 Clinical remission at wk 8 in the combined ACT trials, predicts wk 30 clinical outcomes % at week 30 * * * * Clinical remission Steroid free remission olombel JF et al. Gastroenterology 2011

33 PUCAI is Responsive to change r=0.84, p< Delta PGA Delta PUCAI astroenterology 2007;133:

34 80 Responsiveness of the PUCAI Delta PUCAI r= 0.85, p<0.001 The minimal clinically important difference (MCID): AUC of ROC: 0.97 (0.93 1) Sensitivity: 85% Specificity: 88% Significant improvement Moderate improvement Mild improvement No change Physician global assessment of changegastroenterology 2007;133:

35 How to choose an outcome measure? 1. What does it measure (i.e. validity)? 2. Is it accurate (i.e. reliability)? 3. Can is discriminate between categories? 4. Can it detect change (responsiveness)? 5. Is it feasible?

36 Choosing an outcome measure of disease activity in pediatric UC The rigorously developed PUCAI is a non-invasive, feasible reliable, valid and very responsive index, by which to assess disease activity in pediatric UC Limitations: ceiling effect relevant in acute severe UC

37 Measures of disease activity in Crohn s disease CDAI: long history of consistent use in adult Crohn s disease PCDAI: developed ~ 1990: disease activity has different manifestations in children vs adults; growth impairment an important manifestation of inflammation in CD

38 he heterogeneity of Crohn s disease omplicates activity assessment Prevalence Symptom at presentation Abdominal Pain 72% 86% Diarrhea 56% 78% Weight loss 58% 80% Perianal fistula/abscess 7% 8% Fever NR 38% Blood in stool 22% 49% Sick Kids, Toronto IBD database (n=386) Sawczenko et al, UK surveillance study, Arch Dis Child 2003 ( ; n=379)

39 PCDAI developed for use among children, but item selection and weighting by consensus only Problems: some variables not relevant to disease activity in all patients e.g. perianal disease, growth in post-pubertal patients some variables, especially linear growth rate, will not change in short-term (affects Responsiveness)

40 History Abdominal pain Stools Well-being Physical examination Abdominal examination Perianal lesions Extraintestinal manifestations Weight gain/loss Linear growth PCDAI Laboratory values hematocrit ESR serum albumin Total score: 0-100

41 CDAI Rigorously developed but not with children in mind Problems: 7-day diary impairs feasibility poor reliability when tested among adults omits linear growth children may underreport symptoms

42 Domains of PCDAI vs adult CDAI A comparison of weightings Laboratory Ht/Wt adult CDAI Examination PCDAI History 0% 10% 20% 30% 40% 50% 60%

43 CDAI versus endoscopic appearance Modigliani, Gastroenterology 1990; 98:

44 Comparison of PCDAI vs CDAI Construct Validity adult CDAI score * n=6 n=19 n=18 n=36 Quiescent Mild Moderate Severe PCDAI score ** 25 * 0 n=6 n=18 n=18 n=36 Quiescent Mild Moderate Severe Physician Global Assessment R=0.77 Physician Global Assessment R=0.86 Otley AR et al, Gastroenterology 1999; 116:

45 Comparison of responsiveness in clinical trial setting CDAI scher, ur J Gastro epatol 2004 Crohn s disease activity index treatment week Budesonide * Prednisolone 8 wks: budesonide 149 vs prednisolone 97 *p=0.048

46 Comparison of responsiveness in clinical trial setting PCDAI Escher Eur J Gastro Hepatol 2004 Pediatric Crohn s disease activity index NS treatment week Budesonide Prednisolone 8 wks: budesonide 24 vs. prednisolone 18 p=ns

47 PCDAI versions Original PCDAI (Hyams et al; JPGN 1991) Abbreviated PCDAI (Loonen et al JPGN 2003) Short PCDAI (Kappelman et al, Inflamm Bowel Dis 2009) Modified PCDAI (Leach et al JPGN 2010) Weighted PCDAI (Turner et al, Inflamm Bowel Dis 2012)

48 Comparison of different PCDAI versions urner D et al, Inflamm Bowel Dis 2012:18: 55-62

49 Comparison of responsiveness of different PCDAI versions urner D et al, Inflamm Bowel Dis 2012:18: 55-62

50 Goals of treatment in pediatric IBD TRADITIONAL TREATMENT GOALS NEW TREATMENT GOALS Achieve and maintain mucosal healing Induce and maintain clinical remission (alleviate symptoms) Facilitate growth (a marker of success of therapy) Optimize well-being, quality of life Deep remission (minimize pan-intestinal damage)

51 Measures of intestinal inflammation/healing

52 CD Endoscopic Index of Severity (CDEIS) Deep ulcerations 12 points Superficial ulcerations 6 points Surface of ulcerations (0-10 cm) Surface of lesions (0-10 cm) Ileum 0 or 12 0 or Right colon 0 or 12 0 or Transverse 0 or 12 0 or Left colon 0 or 12 0 or Rectum 0 or 12 0 or TOTAL (sum of all cases) N TOTAL/number of explored segments N/ if ulcerated stenosis if nonulcerated stenosis 0-3 CDEIS : 0 to 44

53 EVOLUTION OF CROHN S DISEASE OVER TIME Protocol for development of damage score Lemann M et al, Inflamm Bowel Dis 2010

54 The ImageKids Study Aim: To develop and validate MRbased indices of inflammatory activity and structural change (damage) in pediatric Crohn s

55 Item generation Item reduction Exact wording of items & response options Weighting and further formatting 4 Validity testing 5 Reliability testing Responsiveness testing Systematic review of the literature 1 Delphi group of ~25 pediatric radiologists and gastroenterologists Second stage of the Delphi process Retrospective reading of 50 MRE exams performed on Pediatric Crohn s patients during the last year 2 Third stage of the Delphi process Derivation Retrospective reading of 50 MRE exams performed on Pediatric Crohn s patients during the last year 2 Prospective enrolment of 120 children at the time of endoscopic assessment and MRE/pelvic MRI 3 Prospective enrolment of another 120 Children at the time of endoscopic Assessment and MRE/pelvic MRI 3 Interobserver reliability: using the two independent readings of the centralized radiologists Children will be followed through 18 months (4 visits) when MRE/pelvic MRI will be repeated; stool & serum collected Test retest reliability: by repeated readings over time in stable pts Using the repeated readings after 18 months 6 Define cutoff scores on both cohorts 7 and end of study Evaluation

56 Summary: Viewpoint of an IBD Clinician/investigator In assessing drugs in a clinical trial setting and for registration, we need to know if the drug (safely) treats intestinal inflammation, when present Therefore we need multi-item measures of disease activity in CD and in UC that can be used to define inclusion criteria, response, and remission Such choices should be made based on principles of measure evaluation as discussed

57 Summary: Viewpoint of an IBD Clinician/investigator PUCAI fills the need in pediatric ulcerative colitis Endoscopic assessment of mucosal healing in UC is complementary and generally feasible at baseline and at time of primary outcome assessment in clinical trials (but not more frequently) Despite identified weaknesses (sensibility, responsiveness) PCDAI has long history of successful use in prospective pediatric CD clinical trials; further exploration of possible superiority of weighted PCDAI is indicated