Practical examples [& pitfalls] of addressing bias in child treatment trials

Transcription

1 Practical examples [& pitfalls] of addressing bias in child treatment trials Dr Matt Woolgar Consultant Clinical Psychologist & Senior Researcher National Academy for Parenting Research

2 Quality of Measures Highest quality measures That address the relevant issues Harder in a new area of higher risk, e.g. RCT for a novel Tx of Conduct Disorder +/- the new CU modifier CD well established, but for CU & its confounds? That can be completed in time More information of higher validity good. More=good? But the greater the Assessment effort, more likely to lose participants, especially those of higher risk Efficiency of measures [vs.] reliability/validity Both relate to translatability back to clinics

3 Blinding Keep Therapist and Assessor roles distinct as far as possible Easiest for the non-clinical measures Hardest for smaller studies Unblinding, e.g., talk about the nice therapist

4 Blinding in Small Treatment Trial 1 RA and 1.5 Therapists for clinical interviews Treating therapists does pre-assessments and other therapist does post-assessment [blind to condition]. Good quality assessment at both time points, blind to starting state But perhaps more likely than RA to be able to detect who has been in which condition. Therapist does pre-assessment, and then RA does post-assessment Good quality initial data, post-assessment blind to starting state and less likely to be biased by detecting who was in which treatment arm But possibly poorer post data [subject to specific training, supervision and monitoring] or at least of a potentially different kind [better?]. RA does both assessments. Similar assessment quality [quality dependent on training etc], less likely to be biased by detecting who was in which treatment arm But not blind to starting state unless using a second RA [e.g. 2x0.5 RA] Video record all assessment sessions and watch these back for quality and consistency

5 Randomization Assigning participants to conditions done outside of the therapy / assessing roles Sometimes outside of the team altogether Sometimes balance randomisation by features such as age or gender Sometimes Randomization could undermine the logic of a study

6 Example of a problem randomizing A study of the psychological & psychiatric outcomes for 2 surgical procedures for the same child disorder, with equivalent surgical outcomes, but performed at different ages One as neonate, the other as an older child Highly specialist surgical procedures carried out in highly specialised regional units specialising in only one or the other procedure Surgeons rationales completely psychological

7 Conditions cannot be randomized Multi site study good but treatment conditions cannot be randomly assigned Require sites to perform treatments they are not experts in what would that measure? Assessors will know what treatment based on address, plus other geographical confounds Assessors blind to site can still tell the treatment based on age of child and how they look Non-randomized design but the bias confounded with the research question??

8 Categorical vs. Dimensional Measures Categories Potential for clear classifications & answers If many cases fall around category cut-offs, consistent calibration can be difficult especially if algorithm complex [CD, attachment etc] and composed of different domains Might a category mask different trajectories/ sub-groups?

9 Categorical vs. Dimensional Dimensional Overall severity of functioning Just above to just below cut off vs. far above to just above [the latter a bigger Tx effect] Type of presentation e.g. CD heterogeneous disorder, with different domains Adjustment across domains of disorder Treatment may be more pertinent to some aspects than others, hence more valid to focus on specific outcomes within the diagnosis

10 Example of an RCT for CD +/- CU Outcomes CD diagnosis & CU Traits [categories] CD domains also measured dimensionally Cruelty more salient than disobedience Positive aspects of relationships Observed in multiple contexts Home & school reports

11 Multiple Informant Reports from Child, Parent, Teacher, Peers etc, More=better... [except when different!] Validity may vary by informant Child > Adults re. internalising problems Adults > Child re. bad behaviour [+ve presentation bias] Child > Adults re. covert delinquency [knowledge] Behaviour may change as a result of context Home vs. school [e.g., F91.0 different kind ] Adults vs. peers [e.g., relational aggression]

12 Multiple Methods Observational methods Data unbiased by recall/insight/presentational biases etc of parent or child Bias may occur in rating Trained to Gold Standard with manualised program & pass reliability test Reliability monitoring & group supervision Double code whole sample or subset & test reliability First two most important to minimise bias cf. quantify it at the end More bias if observational context not ecologically valid? Require different raters to code different measures on same tape to avoid contamination across measures?

13 Multiple Methods Semi-structured Interviews vs. Questionnaires Rater biases from choosing prompts & follow ups non-systematically Desire to avoid difficult questions/answers Abuse uncomfortable (& produces extra admin) Habit of skipping over rarer items Rarity can be sample specific, define new subsamples, or of unknown prevalence in higher risk / novel groups Stick to algorithms, perhaps using laptops

14 Clips from SHO training