ADDIS: towards on-demand support for evidence based decision making based on structured data sources Gert van Valkenhoef 2014-11-21 @ NLM / ClinicalTrials.gov
Section 1 Background
About me MSc Artificial Intelligence (2009) Researcher and lead developer, ADDIS project (2009-now) PhD Medical Sciences (2009-2012) Funded through 2016 Based in the Netherlands Visiting scholar @ Brown, Oct-Dec 2014 https://gertvv.nl
Acknowledgements PhD supervisors: Prof. Hans Hillege, Prof. Bert de Brock Key collaborators: Dr. Tommi Tervonen, Dr. Douwe Postmus, Prof. A.E. Ades, Dr. Sofia Dias, Dr. Nicky Welton, Guobing Lu, Dr. Byron Wallace, Dr. Tom Trikalinos Programmers and students: Joël Kuiper, Dr. Daan Reid, Connor Stroomberg, Bob Goeree, and previously many others
ADDIS 1.x: Project Escher Escher (2007-2013) was a national research project of the Dutch Top Institute Pharma aiming to improve drug regulation through science 16 PhD students and 4 PostDocs working in 5 universities (RUG/UMCG, UU/UMCU, Erasmus MC) in collaboration with industry (MSD, GSK, Amgen, WINap) Now (2014) rebooted as research platform
ADDIS 1.x: Escher WP 3.2 Goals (2009-2013) Develop a drug information system: Effective knowledge access and management Answer drug efficacy and safety questions in an efficient, transparent and accountable way within and across compounds for a broad audience (including regulators) Improve consistency in regulatory decision making Based on systematic review and meta-analysis
ADDIS 1.x: Escher WP 3.2 Results ADDIS decision support system for health care policy: Database of clinical trials Evidence synthesis (network meta-analysis) Decision aiding (multi-criteria benefit-risk analysis) Research output: 7 journal articles + PhD thesis + additional journal and conference papers Primary limitations: Gathering data is time consuming Collaboration and sharing of data is difficult
ADDIS 2.x: IMI GetReal (2014-2016) GetReal is a European project of the Innovative Medicines Initiative (IMI) that aims to integrate randomized and observational data to best inform relative effectiveness 5 work packages, 13 academic partners, 15 industry partners, ties with regulatory and reimbursement networks EUR 16mln funding, 130 person-years of effort over 3 years, primarily senior scientists
ADDIS 2.x plans Web-based multi-user system Collaborative database building Flexible (ad hoc) data integration / harmonization Predictive modeling / relative effectiveness
Current status ADDIS 1.x no longer developed ADDIS 2.x progressing Most key components in place But functionality is rough / incomplete Work on regulatory BR continues (IMI PROTECT)
Section 2 Introducing ADDIS
ADDIS: Aggregate Data Drug Information System ADDIS is a decision support system For health care policy decision making Bridging the gap between aggregated clinical data and evidence-based drug regulation using state of the art methods for benefit risk decision making Software should (eventually) also apply to HTA, hospital, pharmacy, etc. decision making
Evidence-based health care policy Basing policy on evidence is challenging: Data acquisition Evidence synthesis Decision aiding / making Each step needs to be driven by the end goal
ADDIS: Aggregate Data Drug Information System How could evidence-based decision making be supported or improved if clinical trials data were available in a structured format?
Case: EMA EPAR Edarbi (Azilsartan Medoxomil)
Case: EMA EPAR Edarbi (Azilsartan Medoxomil) Dossier investigates three doses: 20, 40, 80 mg/day With various populations, comparators Is there a benefit of 80 mg/day over 40 mg/day? If so, does that benefit outweigh additional harms?
Case: EMA EPAR Edarbi (Azilsartan Medoxomil)
Case: EMA EPAR Edarbi (Azilsartan Medoxomil)
Case: EMA EPAR Edarbi (Azilsartan Medoxomil)
ADDIS 1.x demo
ADDIS 2 Can the availability of structured clinical trials data be improved through an on-line collaborative platform for sharing and improving data extractions?
ADDIS 2
ADDIS 2
ADDIS 2 status Most components in place Closing in on ADDIS 1.x feature parity Data entry is next big thing Not yet ready for general use
Section 3 Experiences with ClinicalTrials.gov
ClinicalTrials.gov import in ADDIS 1.x ClinicalTrials.gov import is a key feature Helped show feasibility of ADDIS concept Import was remarkably easy to achieve Data models similar, despite independent development Some stumbling blocks Caveat: experiences 1-2 years out of date
Key advantages Huge time saver: well-reported CT record saves many hours Loads of data available, also helps when papers are unclear Most key dimensions represented: easily maps User input is required mainly for harmonization Typically good trade-offs between text and structured data Links to literature and other IDs
Lack of referential integrity There are no key/keyref constraints in the XML schema: Duplicate IDs References to undefined IDs Especially arm/group references become confusing IDs can be (re-)defined in each section Use of XML enum could clarify range of some attributes e.g. Number, Mean,... Other data types (e.g. dates) also help
Categorical as a catch-all A category can mean: A true categorical variable Stratified reporting Reporting at multiple time points This is complex to disentangle. Proper support for time points is #1 on my wish list! Missing dimension work for every data point
Reporting thresholds Political issue Investigator-set reporting thresholds for AEs seriously reduce the value of datasets We ve modelled regulatory dossiers where the key events discussed were not reported on ClinicalTrials.gov
Further wish-list items XML elements instead of structured text for study design, eligibility criteria, etc. Distinguish uses of Number: count versus percentage Add MedDRA IDs when MedDRA is used
Section 4 Discussion
Discussion Your feedback to the ADDIS concept & system! Are changes to the schema planned / expected / possible? Is it / will it be possible to flag problems in records? Are there further plans for cross-linking other services? Best way to poll for new and recently changed records?
Thank you! Thank you! https://drugis.org/