Measurement Properties of the Whiplash Disability Questionnaire in Acute Whiplash-associated Disorders

Transcription

1 Measurement Properties of the Whiplash Disability Questionnaire in Acute Whiplash-associated Disorders by Maja Stupar A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Institute of Health Policy Management and Evaluation University of Toronto Copyright by Maja Stupar, 2013

2 Measurement Properties of the Whiplash Disability Questionnaire in acute Whiplash-associated Disorders Abstract Maja Stupar Doctor of Philosophy (Clinical Epidemiology) Institute of Health Policy, Management and Evaluation University of Toronto 2013 Whiplash-associated disorders (WAD) include physical and psychological symptoms that may lead to disability. However, measuring disability following whiplash injuries is challenging because we lack valid and reliable measurement tools. The assessment of WAD-related disability relies on self-reported instruments that are specific to neck pain and do not comprehensively target the constructs associated with WAD-related disability. Designing new tools and evaluating their measurement properties is challenging because of the apparent inconsistencies in the theoretical frameworks (psychometrics and clinimetrics) used in instrument development and in reliability, validity and responsiveness evaluation. A scoping review design was used to develop a conceptual theory on the difference between clinimetrics and psychometrics in order to provide recommendations for future application. The scoping review of psychometric and clinimetric methods suggested that the two frameworks are not as divergent as reflected in the current protracted debates. Content analysis revealed that differences only exist in the scope of what is measured and in instrument development methods with no operational differences in the testing phases. Based on content analysis, I developed a ii

3 new framework that bridges the two measurement schools with an overlapping informed zone between them. I designed a cohort study of 130 participants with acute WAD to assess the measurement properties of the Whiplash Disability Questionnaire (WDQ). The WDQ is a recently developed instrument designed to capture the broad construct of WAD-related disability. The WDQ measurement properties were determined in adults with WAD recruited within 21 days of their collision. My study indicates that the WDQ and its subscales are reliable and valid for clinical and research use. The WDQ can demonstrate change over time as a single scale or as the daily activities subscale. However, WDQ users should be aware of its measurement error when demonstrating change over time. Furthermore, the emotional subscale should not be used alone to demonstrate change over six weeks because it was not responsive. My thesis proposes a unified framework for studying the measurement properties of assessment tools used in clinical practice. I also demonstrated that the WDQ possesses the necessary properties to be used in patients with acute WAD. iii

4 Acknowledgments The journey of life is not a journey taken alone. I would like to thank everyone who has contributed to my journey through this doctoral program. First, I would like to thank my supervisor, Dr. Pierre Côté, for his mentorship, support and availability throughout my doctoral journey. I thank him for challenging me and helping me mold into the young investigator that I have become. His infectious enthusiasm for research and excellence in scientific rigor continue to be inspiring. I was also privileged to work with a dedicated and supportive advisory committee, Dr. Dorcas Beaton, Dr. Eleanor Boyle and Dr. J. David Cassidy. I am thankful for their guidance and tireless feedback. Several individuals and teams contributed support directly and indirectly in the completion my thesis. I would like to thank everyone who contributed to the recruitment, data collection, processing and completion of the UHN Whiplash Intervention Trial that, in turn, helped the completion of this thesis project. Without funding support, the completion of my doctoral program would not be possible. I would like to thank the Canadian Institute of Health Research (CIHR) for providing three years of financial support toward my doctoral studies through the Vanier Canada Graduate Scholarship. My doctoral education experience was also enriched with the opportunity to study abroad at Karolinska Institute in Stockholm, Sweden through the support of the CIHR Michael Smith Foreign Study Supplement. I am thankful to the Department of Clinical Epidemiology and Health Care Research within the Institute of Health Policy, Management and Evaluation at the University of Toronto for providing additional support. Finally, without AVIVA Canada s vision of investing in research to improve business practices, the UHN Whiplash Intervention Trial would not be possible and, in turn, my thesis projects would not have been completed. I thank all these institutions for making it possible for me to dedicate time to my doctoral education. iv

5 I thank my personal friends for their smiles and laughter that made my journey that much more enjoyable and for their support during those more challenging times. I am thankful to my family for their unwavering support; to my wonderful parents Milica and Ilija Stupar, for guiding me through the rollercoaster of life and teaching me the value of hard work; and to my dear sister Biljana for always standing by me with an attentive ear and for all her insightful advice. v

6 Table of Contents Abstract... ii Acknowledgments... iv Table of Contents... vi List of Tables... xi List of Figures... xiii List of Appendices... xiv List of abbreviations... xv Preface... 1 Chapter 1 : Introduction Measuring disability in health research Epidemiology of Whiplash-associated Disorders Definition The burden of whiplash-associated disorders in the population Prognosis of Whiplash-associated Disorders Treatment of Whiplash-associated Disorders Outcome measures currently used in WAD research The measurement divide Objectives General Objectives Specific Objectives vi

7 1.5 Structure of the Thesis Chapter 2 : Measurement Properties: A new framework to contribute to the debate between the field of clinimetrics and psychometrics Introduction Methods Research question Search for relevant studies Study selection Data charting Collation, summarizing and reporting results including synthesis Results Literature search Study selection Data charting Collation, summarizing and reporting of results Synthesis Discussion Conclusion Chapter 3 : Can Recovery from Whiplash-associated Disorders be Measured Reliably in Patients with Acute Whiplash-Associated Disorders? A Test-retest Reliability Study of the Whiplash Disability Questionnaire Introduction Methods Participants vii

8 3.2.2 Procedure Data Sample Size Analysis Test-Retest Reliability Minimal detectable change Sensitivity Analyses Results Descriptive statistics Completeness of WDQ Test-retest reliability Individual item test-retest reliability Minimal detectable change Discussion Conclusion Acknowledgement Chapter 4 : Exploratory Factor Analysis, Validity and Responsiveness of the Whiplash Disability Questionnaire in Adults with Acute Whiplash-associated Disorders Introduction Methods Participants and Procedures Data Collection Whiplash Disability Questionnaire Numerical Pain Rating Scale viii

9 Neck Disability Index Neck Bournemouth Questionnaire CES-D SF-36 Health Survey Self-report Recovery Analysis Descriptive statistics Factor Structure Validity Responsiveness Sample Size Results Sample characteristics Data completion Factor structure Validity Responsiveness Discussion Conclusion Acknowledgement Chapter 5 : Discussion Context and summary of the thesis Contribution of the research to the whiplash literature ix

10 5.3 Implications of the research Future research Content validity using qualitative methods Minimizing measurement error Longitudinal and structural construct validity Predictive validity Direct comparison with other relevant instruments Applicability of the conceptual framework References Appendices x

11 List of Tables Table 2.1: Position statement of our framework and the evidence that is in support of the framework Table 2.2a: Studies using empirical methods to test differences between clinimetric and psychometric methods Table 2.2b: Studies using empirical methods to test differences between clinimetric and psychometric methods Table 3.1: Baseline demographic characteristics of patients with acute whiplash associated disorders Table 3.2: Intra-class Correlation Coefficient for the Total Summary Score categorized by the report of no recovery on the change in neck pain question and memory effects Table 3.3: Sensitivity Analysis for the Intra-class Correlation Coefficient for the Total Summary Score Table 3.4: Intra-class Correlation Coefficient for individual items of the WDQ Table 4.1: Baseline demographic characteristics of patients with acute whiplash associated disorders Table 4.2: Baseline means, medians and normality values of WDQ total score and individual items Table 4.3: Model fit statistics for the models with different number of factors in the WDQ Table 4.4: Factor analysis of the WDQ: The 2-factor solution Table 4.5: Results of construct validation (n=130). A priori expected Pearson correlations between the WDQ, its subdomains and constructs shown (E) followed by observed/achieved results (A) xi

12 List of Tables (continued) Table 4.6: Effect size, Guyatt s responsiveness statistic (RS) and standardize response mean (SRM) for participants reporting recovery on the global recovery question (N=62) Table 4.7: Spearman s rank correlations and AUCs for responsiveness based on the a priori hypotheses xii

13 List of Figures Figure 1. 1: Data collection and data use in analysis addressing objectives two to six Figure 2.1: Literature search for the measurement divide scoping review Figure 2.2: Latent construct relationship with causal and indicator variables Figure 2.3: Conceptual framework bridging clinimetrics and psychometrics Figure 4.1: Total WDQ baseline distribution Figure 4.2: Factor analysis scree plot xiii

14 List of Appendices Appendix 1: Questionnaires A-1.1: Baseline Questionnaire A-1.2: Three-to-Five Day Follow-up Questionnaire A-1.3: Six-week Follow-up Questionnaire A-1.4: Addition to WIT Baseline Questionnaire A-1.5: Addition to WIT Six-week Follow-up Questionnaire Appendix 2: Ethics Certificates A-2.1: University Health Network Ethics Approval A-2.2: University of Toronto Ethics Approval Appendix 3: Baseline WDQ Distributions Appendix 4: COSMIN Checklist completed with criteria relevant to this thesis xiv

15 List of abbreviations AIC ANOVA AUC CES-D CINAHL COSMIN DASH EFA ES GTA ICC ICF IRT KMO MCID MDC MeSH NDI NPTF NRS PCA QTF RMSR ROC SBC SEM SF-36 SRM TLRC UHN VAS WAD WDQ WHO WIT WOMAC Akaike information criteria Analysis of Variance Area under the curve Center for Epidemiologic Studies Depression Scale Cumulative Index to Nursing and Allied Health COnsensus-based Standards for the selection of health Measurement INstruments Disabilities of the Arm, Shoulder and Hands Exploratory Factor Analysis Effect Size Greater Toronto Area Intra-class Correlation Coefficient International Classification of Functioning, Disability and Health framework Item Response Theory Kaiser-Meyer-Olkin Minimal Clinically Important Difference Minimal Detectable Change Medical Subject Heading Neck Disability Index Neck Pain Task Force Numerical Rating Scale Principal Component Analysis Quebec Task Force Root Mean Square Residual Receiver operating characteristic Schwarz Bayesian criteria Standard Error of Measurement Short-Form Health Survey containing 36 items from the Medical Outcomes Study Standardized Response Mean Tucker and Lewis Reliability coefficient University Health Network Visual Analog Scale Whiplash-associated Disorders Whiplash Disability Questionnaire World Health Organization Whiplash Intervention Trial Western Ontario and McMaster Universities Osteoarthritis Index xv

16 1 Preface Background The general purpose of my thesis was to determine the measurement properties of the Whiplash Disability Questionnaire (WDQ) in patients with acute Whiplash-Associated Disorders (WAD) and to develop a conceptual theory on the difference between clinimetrics and psychometrics. In order to meet these goals, I performed a scoping literature review and I designed a cohort study that involved primary data collection. Potential participants for this cohort study were recruited alongside the University Health Network (UHN) Whiplash Intervention Trial (WIT) but participation for this study was offered regardless of their eligibility for the trial.[26] The UHN WIT investigated the effectiveness of programs of care in improving recovery of patients with recent WAD. The recruited population for the UHN WIT included adults who made an insurance claim for traffic injuries to a large Ontario insurer (Aviva Canada) between February 2008 and June 2012 with WAD diagnoses Grades I-II[113] of less than 3 weeks duration. Participants were given the opportunity to participate in both studies but the cohort study also included WAD Grade III and had a shorter recruitment period from February 2008 to August The UHN WIT was led by Dr. Pierre Côté. I was a clinical research coordinator of the UHN WIT and the cohort study as well as one of the co-authors. The objectives of the research conducted for my doctoral dissertation are separate from the randomized controlled trial lead by Dr. Pierre Côté. The UHN WIT provided the infrastructure for recruiting participants within 21 days of their collision. Without this infrastructure, recruiting participants with acute WAD would not be possible in Ontario for a small cohort study. Within this infrastructure, claims adjusters identified potential study subjects when policy holders contacted AVIVA s claim center to report an injury. A short screening tool was designed to assist adjusters in identifying eligible participants. The tool prompted the adjusters to inquire about their location of residence (GTA, Barrie, Brantford, Burlington, Cambridge, Guelph, Hamilton, Kitchener-Waterloo, New Market, Oshawa, and surrounding towns); their age (18 years or older); whether they were making an injury claim and whether their collision was within 21 days of reporting the injury. If they satisfied these conditions, the adjusters invited them to enter a study at UHN, and asked permission to release their name and phone number to the UHN

17 2 research team. If they agreed, the claimant was referred immediately to one of the clinical research coordinators and booked for eligibility assessment. The offer to participate in both studies was given if potential participants were determined to be eligible after the history, physical exam and, if needed, a radiological exam performed by the clinical research coordinators. Informed consent was obtained separately for each study. Some baseline and sixweek follow-up data was the same for both studies and that data was collected only once for participants in both studies with only a few additional questions asked for the cohort study. These data collection procedures reduced the burden on study participants and provided the cohort with a rich dataset appropriate for analysis of measurement properties of the WDQ in acute WAD. Roles and Responsibilities As a clarification of the roles and responsibilities, my specific tasks in the conduct of this research over the past six and a half years are outlined below: i. Designed the study and defended the protocol in May 2008; ii. iii. iv. Wrote the ethics applications to the University Health Network and University of Toronto; Coordinated participant recruitment and data collection including baseline, 3-5 day reliability study follow-up and the 6-week responsiveness study follow-up; Developed, cleaned, validated and managed databases used for the cohort study; v. Conducted the analysis for the test-retest reliability, factor analysis, construct validity and responsiveness; vi. vii. viii. Designed, led and contributed to the scoping review of literature as one of two reviewers; Conceptualized the scoping review framework based on content analysis with one other author; I was the primary author and lead writer of all the papers presented in this thesis.

18 3 Chapter 1 : Introduction 1.1 Measuring disability in health research In the era of evidence-based medicine and health care accountability, measuring outcomes with validated outcome measures is the essential building block for developing evidence and implementing it into practice.[52,104] How health outcomes are assessed directly impacts on the development of effective therapies and on the evaluation of their cost-effectiveness. Outcome measures need to be validated for use in research and in clinical practice.[33] Validation of an instrument means that measurement properties have been tested to ensure that the instrument measures what it purports to measure and that it can accurately demonstrate change over a clinically relevant period of time. Without adequate measurement properties, results of clinical trials would be biased and change demonstrated in clinical settings would be inaccurate. Results from clinical studies are only as good as the instruments used to measure outcomes in those studies. Unlike weight or blood pressure, many health outcomes cannot be directly measured. Therefore, many outcomes are defined and measured as latent constructs. The development and testing of instruments used to measure latent constructs is complex. Developers of instruments need to consider the definition of the construct that is being measured, the time component of what is measured (e.g. change over time, current state), items that should be included in self-report outcome measures to capture the scope of the construct and how to score the measure.[33] Once developed, outcome measures must be tested to establish reliability, validity, and responsiveness. Evaluating an instrument s ability to accurately measure latent constructs can be complicated by the lack of consistency in the terminology and methods used in the field of measurement. I used the definitions provided by the consensus-based standards for the selection of health measurement instruments (COSMIN) group because they used Delphi methods to reach consensus on taxonomy, terminology and definitions related to measurement.[89] They defined reliability as the extent to which scores for patients who have not changed are the same for repeated measurement under several conditions. These conditions can be categorized into

19 4 different types of reliability including: 1. internal consistency (e.g. using a different set of items from the same health related-patient reported outcomes); 2. test-retest (e.g. testing change over time); 3. inter-rater (e.g. testing the condition by different persons on the same occasion); and 4. intra-rater reliability (e.g. testing the condition by the same raters on different occasions).[89] Validity was defined as the degree to which an instrument measures the construct(s) it purports to measure. Validity can also be assessed using different criteria and was categorized into: 1. content validity (e.g. if the instrument adequately reflects the construct); 2. construct validity (e.g. if the instrument is consistent with hypotheses relating to internal and external instrument relationships and relevant group differences demonstrating measurement of the construct); and 3. criterion validity (e.g. if the instrument is an adequate reflection of a gold standard). Responsiveness was defined as the ability of an instrument to detect change over time in the construct to be measured. Finally, the outcome measure must be interpretable meaning that a qualitative meaning can be assigned to the quantitative or change scores to some degree.[89] Adequate measurement properties including reliability, validity, responsiveness and interpretability are necessary when using an instrument in clinical and research settings to ensure accurate measurement of outcomes. However, measurement properties are specific to the condition, setting and population in which the instrument is assessed.[119] Therefore, researchers and clinicians must consider the conditions, settings and populations in which instruments are to be used or to which the instrument needs to be applicable when determining their properties. 1.2 Epidemiology of Whiplash-associated Disorders Definition The Quebec Task Force (QTF) on Whiplash-Associated Disorders defined whiplash as an acceleration-deceleration mechanism of energy transfer to the neck which may result in bony or soft tissue injuries that commonly occurs in motor vehicle collisions.[113] The resulting whiplash associated disorders (WAD) are defined as a clinical manifestation of, or the disability caused by, whiplash injury and may include biologic, psychological, and social symptoms of the

20 5 potential tissue damage.[99,113] Common WAD symptoms include neck pain, back pain, headache, dizziness, arm pain, concentration problems and depression.[13,19,113] The burden of whiplash-associated disorders in the population Whiplash injuries are common following motor vehicle collisions. In the United States, whiplash-related injuries were reported as the most common emergency department-treated motor vehicle injury in 2000 with an incidence of 328 visits per 100,000 inhabitants.[101] In 2008, a systematic review of literature on the burden of neck pain and associated disorders such as WAD was published by The Bone and Joint Decade Task Force on Neck Pain and Its Associated Disorders.[61] This systematic review estimated the annual incidence to be at least 300 per 100,000 inhabitants in North America and western Europe.[61] It also reported that the incidence of WAD differed substantially between countries. Similarly, a 2008 study by the European Insurance Committee found that the incidence of minor cervical trauma (defined as a percentage of overall claims) varied widely across ten European countries with the lowest incidence found in France (3% of all bodily injuries) and the highest in Great Britain (76% of all bodily injuries).[20] This study also found that the cost of minor cervical trauma varied greatly between countries with Switzerland having higher costs (average cost of euro per claim) compared to other European countries (average cost of 9000 euro per claim). However, these cost differences did not reflect the difference in incidence across countries. The incidence of WAD also varies across Canadian provinces. In Saskatchewan, the six-month incidence of WAD was approximately 300 cases per 100,000 inhabitants in 1995.[19] WAD were reported by 83% of all eligible participants in this cohort.[19,25] In contrast, the 12-month incidence was 70 cases per 100,000 inhabitants in Quebec.[113] Different compensation systems have been shown to influence the incidence and prognosis of WAD and may provide part of the explanation for the varied reporting of injuries across provinces and countries.[19] Multiple studies have reported the incidence of WAD is higher in women and more common in younger ages.[25,27,61,113] It can also be influenced by several risk factors including personal, societal, and environmental.[61]

21 Prognosis of Whiplash-associated Disorders Whiplash injuries are an important cause of persistent disability. Although the QTF originally reported that WAD is a self-limiting condition with a favourable prognosis, subsequent studies found that the course of WAD varies greatly between jurisdictions and insurance systems.[28,113] In Saskatchewan, the median time to recovery decreased from 433 days in 1994 for claimants under the tort system to 200 days in 1995 for those insured under the no-fault system.[19] In contrast, the original study from the QTF on WAD in 1987, reported the median time on compensation to be 30 days with 4.1% of individuals were receiving compensation oneyear after the collision.[28,113] A review of literature published by the NPTF on the course and prognosis of WAD reported that approximately 50% of those with WAD will report neck pain symptoms one-year after their injuries.[15] Prognostic factors for Recovery From WAD The prognosis of WAD is complex and influenced by physical and psychological factors. Studies have found that greater initial pain intensity, more symptoms and greater initial disability predict slower recovery from WAD.[15,135] Pre- and post-injury psychosocial factors such as passive coping, depressed mood and fear of movement are also predictive of slower recovery.[17,97,114] Other studies reported that sociodemographic factors (e.g. female gender, lower education), general health before the injury and insurance/compensation systems under which benefits can be claimed were associated with WAD recovery.[15,19,28,135] In addition, an individual s expectation of recovery is an important prognostic factor for delayed recovery with those reporting poor expectations showing much slower rates of recovery than those who expect to get better soon after their injury.[62,94,95] Treatment of Whiplash-associated Disorders Identification of effective therapies through research studies is important in providing evidencebased care that can influence the prognosis of WAD. The NPTF systematic review on the treatment of neck pain reported that there is evidence that educational videos, mobilization and

22 7 exercises appear more beneficial than usual care or physical modalities in promoting the recovery of patients with WAD.[67] However, the role of education in the management of WAD is being debated as evidenced by two recent systematic reviews that reached different conclusions.[53,130] Moreover, evidence from observational studies suggests that early intensive management of WAD may delay recovery.[29,30] Similarly, a population-based cohort study from Saskatchewan has shown that individuals receiving fitness training and outpatient rehabilitation had a 19-50% slower recovery from WAD.[18] The effectiveness of rehabilitation, training programs and other health care services commonly provided to patients with WAD needs to be determined in randomized controlled trials.[71] For clinical trials to accurately demonstrate therapy effectiveness, appropriate outcome measures must be used to evaluate the clinical evolution of a condition. Currently used measures in WAD clinical trials have focused on the assessment of disability related to the neck. Considering that WAD commonly present with a constellation of symptoms, currently used measures may be missing the full spectrum of disability and recovery from WAD.[59] Furthermore, clinical outcome measures must demonstrate good reliability, validity and responsiveness in order to be useful clinically and for research purposes.[119] Outcome measures currently used in WAD research The construct of disability is difficult to define and measure. It is a concept that is not physically tangible and can be highly contextualized; therefore, it may differ from person to person and from situation to situation.[5] While previous definitions focused on activity limitations, the most current International Classification of Functioning (ICF) framework proposes that disability includes impairments, activity limitations, and participation restrictions.[142] The new ICF model attempts to capture aspects of the condition covered not only by impairment and activity limitation but also its effect on the individual s participation in life events. Because it encompasses the effect of the disability on all aspects of the individual, the ICF is a useful model to base the measurement of WAD disability on. To be valid, self-report outcome measures need to capture all components of a construct. Most measures currently used to measure WADrelated disability do not have a body of evidence that supports their construct definition, comprehensiveness, validity or reproducibility.

23 8 A commonly used outcome measure in whiplash research is the Neck Disability Index (NDI). The NDI was developed to capture neck-specific disability and consists of 10 items, each with 6 response options rated from 0 (no disability) to 5 (maximal disability).[56,116,131,139] The items include questions on pain intensity and related to the effect of neck pain on function relevant to personal care, lifting, reading, headaches, concentration, work, driving, sleeping and recreation.[131] The NDI has been reported to have good construct validity, reliability and responsiveness in different populations.[56,116,139] However, it was not designed for WAD and therefore it does not capture all aspects of WAD disability. A review of the published literature demonstrated that the NDI omits important components of WAD disability because it centers on neck pain.[63] Specifically, only three of nine disability items (i.e. work, driving, and sleep) identified by WAD patients as being important are included in the NDI.[63] Other items important to WAD patients that are not included in the NDI include fatigue, participation in sports, depression, socializing with friends, frustration and anger.[63] Furthermore, neck pain patients have been found to have lower general health scores based on the SF-36 outcome measure specifically in the energy/fatigue, mental health and role-emotional domains compared to those without neck pain.[70] Therefore, a comprehensive instrument that includes a range of items that are important to patients would measure the construct of WAD disability more accurately and perform better as an outcome measure. An instrument recently developed to measure WAD disability is the Whiplash Disability Questionnaire (WDQ). The WDQ was developed based on the ICF framework of disability.[99] However, the developers have shown that in chronic WAD patients, the WDQ only includes one domain/factor suggesting that it is does not fully represent the ICF framework. The psychometric properties of the WDQ were studied in Australian patients with chronic WAD.[49,99,140] In this population, the WDQ demonstrated good validity, reliability and responsiveness. Recently, a German translation of the WDQ was also shown to have adequate measurement properties for patients with chronic whiplash injuries.[87,110] However, the WDQ s reliability, validity and responsiveness in patients with acute WAD remain unknown. To be useful clinically and in research, an outcome measure must have strong measurement properties throughout the course of the condition. Moreover, because validation of an outcome measure is specific to the population and setting studied, it is therefore necessary to establish its measurement properties in a population of patients with acute WAD.[119]

24 9 1.3 The measurement divide Different schools of measurement can have different approaches to instrument development and evaluation. Two schools relevant to health care, clinimetrics and psychometrics, have been the source of some debate.[46,92] The international COSMIN research group recently developed a set of measurement standards using Delphi methods to reach consensus on taxonomy, terminology and definitions related to measurement properties for health-related patient-reported outcomes.[89] However, this group consisted largely of researchers adhering to clinimetric measurement methods. Clinimetrics is a measurement school developed by Feinstein and focused on measurement relevant to clinical outcomes.[46] Psychometrics is an older school of measurement developed in psychology with a focus on personal and interpersonal behaviour and educational testing or examination.[92] Many of the psychometric measurement methods are clinically relevant and have, therefore, been applied across health care. Inconsistency in methods used to develop and evaluate instruments is complicated by the existence of these different schools of measurement mainly because they use different theoretical and empirical methods.[31,118,143] The debate between the two schools, clinimetrics and psychometrics, has lead to confusion on the appropriateness of various instruments used to measure health outcomes. I compared and contrasted the clinimetric and psychometric methods to advance this debate and provide suggestions on the use of outcome measures in the future. As demonstrated in the next chapter, this comparison led to the development of a conceptual framework that integrated the theories and methods of both schools. I, therefore, use the term measurement properties in this thesis instead of psychometric or clinimetric properties when discussing the evaluation of the WDQ. I propose that using measurement properties will minimize confusion and focus the discussion on properties of the instrument rather than the school of measurement. 1.4 Objectives General Objectives My first objective is to determine the measurement properties of the Whiplash Disability Questionnaire (WDQ) in a cohort of patients with acute WAD. My second objective is to

25 10 analyze the divide between clinimetrics and psychometrics and develop a conceptual framework for the evaluation of measurement properties Specific Objectives In a clinical cohort of patients with recent WAD (less than 21 days duration), we aim to: To clarify conceptual differences between psychometric and clinimetric methods; Determine the short-term test-retest reliability of the WDQ; Determine the factor structure of the WDQ; Determine the internal consistency of the WDQ; Determine the construct validity of the WDQ using the Neck Disability Index and Short Form General Health Status Survey (SF-36); Determine the short-term responsiveness of the WDQ using the global perceived improvement question as an indicator of improvement 1.5 Structure of the Thesis This thesis is presented as a multiple-paper dissertation with five chapters: an overall introduction, three papers that address specific objectives of the thesis and an overall discussion. The sequence of papers was ordered to address conceptual issues first and follow the traditional order determining measurement properties of an outcome measure. Specifically, the papers were ordered to present reliability first, followed by validity, factor structure, internal consistency and responsiveness. Each of the three manuscripts was written in a publishable format and includes an introduction, a methods section, a results section and a discussion. The thesis consists of the following three papers. Chapter Two: Measurement Properties: A new framework to contribute to the debate between the field of clinimetrics and psychometrics addresses the conceptual issues relevant to two fields within measurement leading to a new conceptual framework which was the first objective of this thesis. Chapter Three: Can Recovery from Whiplash-associated Disorders be Measured Reliably in Patients with Acute Whiplash-Associated Disorders? A Test-retest Reliability Study of the Whiplash Disability

26 11 Questionnaire presents information relevant to the second objective of the thesis; specifically, the 3-5 day test-retest reliability of the WDQ in adults with acute WAD (Figure 1.1). Chapter Four: Exploratory Factor Analysis, Validity and Responsiveness of the Whiplash Disability Questionnaire in Adults with Acute Whiplash-associated Disorders includes information relevant to specific objectives three through six (Figure 1.1). Specifically, the factor structure, internal consistency and construct validity of the WDQ (i.e. objective three through five) were determined using baseline data from 130 participants with acute WAD. Objective six (i.e. shortterm responsiveness over six weeks) was established using baseline and six-week follow-up data (Figure 1.1). All three papers will be submitted for publication before or soon after the doctoral examination. Figure 1.1: Data collection and data use in analysis addressing objectives two to six This dissertation also includes several appendices. The first appendix includes intervieweradministered baseline and follow-up questionnaires used to addresses the different specific objectives of the thesis. Appendix 2 includes ethics approval certificates for this thesis from the University Health Network and the University of Toronto.

27 12 Chapter 2 : Measurement Properties: A new framework to contribute to the debate between the field of clinimetrics and psychometrics 2.1 Introduction Measurement is a core science located at the heart of many intersecting health disciplines. Consequently, different measurement paradigms have been developed to support the metrics used in the various disciplines. Most common to health research are two fields: psychometrics and clinimetrics. Psychometrics is the measurement of phenomena that are best measured by multiple items or attributes reflecting a specific construct (i.e., anxiety or depression).[57,92,112,145] While psychometrics is popular in health care, the second most prevalent measurement school is clinimetrics which was introduced by Feinstein in the early 1980 s.[46] Clinimetrics focuses on prognostic and diagnostic indices, which may combine different constructs (e.g., blood pressure, symptoms or previous risk factors) to create a composite weighted score of risk of a distinct construct. The APGAR is an example of a robust, well-used clinimetric index using distinct constructs (e.g. grimace, appearance, pulse) to diagnose/classify the health status of newborns and identify newborns in need of medical attention.[2] Various disciplines have developed their own measurement theories and methodologies (e.g., sociometrics, biometrics, anthropometrics) which address measurement concepts central to their discipline or area of research.[4,83,123] As previously mentioned,, the two most prevalent schools in health research are psychometrics or clinimetrics. The theories and methodologies promoted by these two schools are mainly used to design indicators or outcome measures for research and clinical purposes. They will, therefore, be the focus of this paper. These two schools are often described as discordant with proponents of each school promoting the strengths of their approach and highlighting the limitations of the other framework.[31,118,143] We will deconstruct these similarities and differences in order to move beyond the current debates. This debate is more acute in the current period of patient-centered care where reimbursement is often contingent on patient outcomes. Furthermore, regulators are increasingly aware that

28 13 measurement standards must be met before the benefits of an intervention are established.[128] The Era of Health Care Accountability described by Relman is dependent on good measurement reinforcing Nunally s call that accurate measurement of key variables sets the pace of scientific progress.[92,104,141] However, clinicians and researchers who need to select patient-based outcomes are confronted with the tensions that exist between the two paradigms of measurement: clinimetrics and psychometrics. This polarized debate often leads to ambiguity by making some tools appear inadequate in their development or manifest properties when approached from one perspective compared to the other. In this paper, we argue that psychometrics and clinimetrics have more similarities than differences. For example, both fields emphasize that outcome measures should be standardized, reproducible and accurate.[47,92] They also share a common interest in the measurement of various latent constructs such as pain, disability, self-efficacy, appraisal, perceptions and depression. Despite their common interests, the co-existence of clinimetrics and psychometrics has given rise to a hearty debate leading some psychometricians to question the need for clinimetrics.[39,118] These proponents of psychometric theory suggest that the existence of clinimetrics is redundant because it is not substantially different from the older field of psychometrics.[39,118] They suggest that, like any classification, distinctions should continue to exist only if they facilitate accurate communication about clinical and other outcome measures.[39,76] Alternatively, proponents of clinimetrics suggest that their school is necessary because it offers clinically-based methods to construct measures even if the measures are of latent constructs such as pain, anxiety or functioning.[31,143] We propose that the current division between clinimetrics and psychometrics creates an unneeded schism in a area where much work is needed: measurement of key health constructs and variables. The debate has led to an unnecessary confusion in clinical research and has created a barrier for the appropriate choice and use of measures. Moreover, it has kept the fields separate and limited the advancement of measurement methodology. Reconciliation between the two schools might lie in revisiting the roots of each school rather than in continuing to debate the differences. The purpose of our paper was twofold. First, we performed a scoping literature review to describe the attributes of the clinimetric-psychometric divide. The aim of a scoping study is to map rapidly the key concepts underpinning a research area... especially where an area has not been reviewed comprehensively before.[3] Second, we synthesized the findings and developed

29 14 a revised framework that highlights the similarities and differences of each, respecting the nature of the measurement theory. 2.2 Methods We conducted a scoping review of the literature. Our search included five stages: a) development of a research question; b) search for relevant studies; c) study selection; d) data charting; and e) collation, summarizing and reporting the results.[3] Research question What are the methodological similarities and differences between clinimetrics and psychometrics in the development and evaluation of a clinical measure? Search for relevant studies We performed a literature search in Medline between 1950 and March 2012 using a combination of the MeSH terms psychometrics (exploded) and health status (exploded) and text terms clinimetric* and psychometric*. The terms were combined in the search using and as the combination link (i.e., MeSH psychometrics and clinimetric* or psychometric* and clinimetric* ). The search was limited to publications in English. We performed a similar search in PsychINFO, CINAHL and Embase databases using the same subject headings as search terms. Finally, we performed a textbook (title) search in the University of Toronto catalogue using terms clinimetric*, psychometric*, measurement and health. Article bibliographic reference lists were also searched for relevant literature Study selection The lead author (MS) reviewed all titles and abstracts and selected articles relevant to the research question. An article was considered relevant if the major theme of the article was on the comparison of clinimetrics and psychometrics Data charting Through a series of iterative meetings (MS, DEB), we performed a content analysis of relevant articles to identify emerging themes and stances taken by different authors. Our data charting

30 15 was guided by previous frameworks used to assess health indices and the measurement of disease-specific quality of life.[55,78] These frameworks identified several categories that were useful in categorizing our data when investigating the instrument development stages: item selection, reduction, scaling and questionnaire formating.[55,78] Moreover, these frameworks included stages for the process of instrument testing and evaluation including reliability/reproducibility, validity and responsiveness. We used these categories to chart similarities and difference between clinimetric and psychometric methods. For example, our content analysis of articles included identification of similarities and differences in methods used by clinimetrics and psychometrics in the stages of item selection and reduction Collation, summarizing and reporting results including synthesis We synthesized the themes and findings from the relevant literature through iterative consensus meetings between two of the authors (MS, DEB). This synthesis led to the development of a position statement for our framework. We verified the results by revisiting each article to extract features supporting or contradicting our position statement and presented it to the larger author group (PC, JDC, EB) for debate and critique. Results from articles demonstrating empirical testing were also summarized in a separate table to provide more relevant elements in reporting of empirically based studies. 2.3 Results Literature search The Medline search using a combination of the MeSH term psychometrics and the key word clinimetric* or key words psychometric* and clinimetric* yielded 90 results (Figure 2.1). A Medline search using the MeSH term health status and the keyword clinimetric* yielded 41 results. CINAHL, PhychINFO and EMBASE searches did not identify any new articles and therefore are not represented in Figure 2.1.

31 16 Article Search in Medline Textbook Search MeSH: Psychometrics Text word: Clinimetric* Text word: Psychometric* Text word: Clinimetric* MeSH: Health Status Text word: Measurement Text word: Health AND AND OR AND AND 711 Duplicates or not relevant Text word: Clinimetric Text word: Psychometric Selection based on relevance to the debate: main topic of the article is the differences between clinimetrics and psychometrics 15 Reasons for exclusion: 55 articles assessed a specific instrument 20 articles not on main topic for other reasons 894 textbooks not on main topic 7 articles added from bibliography search 22 0 Figure 2.1: Literature search for the measurement divide scoping review Study selection Our search yielded 15 relevant articles (Table 2.1). All articles were published in the early 1990s following Feinstein s description of clinimetrics in the 1980s. Additional articles were obtained from searching article bibliographies (five articles and two replies to included articles). Five of the relevant articles used empirical methods to test the proposed methodological differences between clinimetrics and psychometrics (Table 2.2). The textbook search yielded three citations (one textbook and two theses) using the term clinimetric, 180 citations using the term

32 17 psychometric and 711 citations for the combination of terms measurement and health. Textbook citations were focused on the methods of each field and not on the differences between psychometrics and clinimetrics. Therefore, we selected relevant textbooks of each field to inform our article review.[33,47,92] Most articles (61%) were excluded because they focused on evaluating the measurement properties of specific measures without comparing clinimetric and psychometric methods Data charting Based on our content analysis, we found several emerging themes and positions by different authors. Specifically, several proponents of clinimetrics reported that the construction of clinimetric indexes involves a deliberate combination of multiple attributes that are not expected to produce a homogenous measure (Table 2.1).[31,32,41,42,91,143,146] They also suggested that clinimetric measures usually contain fewer items than psychometric measures and that the items are chosen to combine multiple clinical constructs in a single index. This contrasts with the psychometric approach where different facets of the same construct are preferable. Furthermore, proponents of clinimetrics propose that dissected intuition (defined as stakeholder or expert input including clinician or patient input) is the fundamental distinction between the two metric fields in the construction of instruments.[31,143] In contrast, proponents of psychometrics suggest that there is an equal amount of dissected intuition involved in constructing psychometric instruments, and that the need for the homogeneity amongst items (i.e. internal consistency) in the outcome measure is dependent on the purpose of the measure.[118] However, a more purist psychometric instrument development would perhaps use less opinion or appraisal from stakeholders and focus on indicators of similarity in response patterns (correlations, factor analysis) to determine items to keep. Another important distinction between clinimetrics and psychometrics is the nature of the variables included in an instrument. Clinimetricians suggest that psychometric approaches include indicator (variables that result from the measured construct) rather than causal variables (variables that may induce change in the measured construct rather than be the result of it).[32,44,45] In contrast, clinimetric approaches include mainly causal variables.[32,44,45] As pointed out by Fayers et al, this is significant because changes in the latent (measured) construct should be directly and proportionally reflected by the indicator variables, but not necessarily by