External Field Review Results for the Canadian Multiple Sclerosis Monitoring System July 2012
Acknowledgements The Canadian Institute for Health Information (CIHI) wishes to acknowledge and thank the individuals and organizations that volunteered their time and effort to contribute to the external field review for the Canadian Multiple Sclerosis Monitoring System (CMSMS) minimum data set. The external field review provided CIHI with an opportunity to gather a range of views on the information that the CMSMS will collect. The feedback provided by reviewers significantly contributes to CIHI s efforts to build a system to record, measure and monitor the evolution and treatment of multiple sclerosis in Canada. The results of the external field review, in combination with the guidance and insight provided by the CMSMS Project Advisory Committee and Technical Advisory Committee, have helped to ensure the development of a data set that is valid and useful for its stated purpose. Overview This report presents an overview of the results of an external field review conducted for the Canadian Institute for Health Information s Canadian Multiple Sclerosis Monitoring System (CMSMS). The Canadian Institute for Health Information (CIHI) is collaborating with an extensive network of experts including people living with multiple sclerosis (MS) and their caregivers, the MS Society of Canada, the Canadian Network of Multiple Sclerosis Clinics, clinicians, researchers, international experts and various governments to develop the CMSMS. The CMSMS will allow for the recording, measuring and monitoring of the evolution and treatment of MS in Canada. The information from the CMSMS will help patients, clinicians, researchers and the public better understand disease patterns across Canada, variation in use of treatments and long-term patient outcomes. In April 2012, CIHI launched an external field review with a targeted group of experts to gather a range of views on the validity and utility of the minimum data set for the CMSMS. Specifically, the objectives were to assess the following: 1. The degree to which the minimum data set has sound methodological properties: Relevance: The data element captures an important aspect of the evolution and treatment of MS in Canada. Feasibility: The data is available and can be collected consistently over time. Clarity: Definitions are clear and easy to understand and the data element is conceptually sound. 1
Breadth: The coding options listed for the data element are inclusive, appropriate and accurately capture the intended details. Usefulness: Data will give participating data providers access to relevant, meaningful information to improve the management and delivery of patient care. 2. The degree to which the minimum data set is acceptable for use. In consultation with key stakeholders, CIHI was able to identify reviewers with a wide range of expertise, including people with MS and their caregivers, clinicians, researchers, government representatives and organizations from both within and outside of Canada. Participants in the external field review were selected based on one or more of the following criteria: Knowledge of the evolution and treatment of MS; Knowledge of priority health information needs related to MS; Working knowledge of health information standards; and Experience living with MS and/or being a personal caregiver for someone living with MS. Reviewers accessed the minimum data set using a survey questionnaire that facilitated evaluation of six areas (demographics, leisure time exercise, clinical, treatment, diseasemodifying therapy [DMT] and outcomes) against four criteria (relevance, feasibility, clarity and usefulness). In addition, three text boxes in each section provided the option for more detailed comments and suggestions. The target groups were well represented in the responses received, and the input was reflective of their diverse range of expertise. Feedback from within and outside of Canada was detailed and thorough, with clear direction provided. In those cases when sufficient information was not available on the questionnaire, additional follow-up consultation was completed with the reviewer. Analysis of survey responses included quantitative and qualitative methods. Levels of reviewer agreement for survey questions were quantified, and a qualitative review of written comments was conducted to identify themes and key findings for specific data elements and data domains in general. 2
Summary of Results The external field review for the CMSMS minimum data set met all of its objectives, with a positive response from reviewers. Analysis of the survey results demonstrated that the targeted group of reviewers assessed the minimum data set to be valid and useful for its stated purpose. Overall, there was a high level of agreement that data elements within the areas of demographics, leisure time exercise, clinical, treatment, DMT and outcomes met the criteria of relevance, feasibility, clarity and usefulness. Areas or data elements with the weakest support among reviewers were identified and evaluated, and all changes were subsequently validated by the CMSMS Project Advisory Committee and Technical Advisory Committee. Conclusion The CMSMS will provide a minimum data set and data standards, adding quality and value to current data collection and analysis capabilities. It will fill an important information gap by providing longitudinal data that is pan-canadian and comparable across jurisdictions and that enables sound policy and effective health system management. The results of the external field review represent one component of CIHI s evaluation of the minimum data set for the CMSMS. If you have any questions or would like more information on the CMSMS, the members of the two advisory committees or the development process, please send an email to ms@cihi.ca or visit our website at www.cihi.ca/multiplesclerosis. 3