FULL DESCRIPTION OF QUARTERWATCH METHODOLOGY

Transcription

1 FULL DESCRIPTION OF QUARTERWATCH METHODOLOGY The FDA s Adverse Event Reporting System (AERS) data present numerous challenges to systematic analysis as presented in QuarterWatch. We summarize the fundamentals of our methods in each report. The detailed criteria we use are outlined below. This guide is intended to assist other researchers who might want to conduct analysis using the QuarterWatch criteria. Contents Report Selection Criteria Report Versions, Counts, and Revisions Coding and Classification of Drug Names Characterization of Adverse Events Proportional Reporting ration Other Details. Report Selection Criteria REPORTS INCLUDED We report on adverse event cases received by the FDA for the first time in the calendar quarter under study and originating in the United States or its territories. The reports we analyze are classified and specifically coded as serious under FDA regulation, meaning that the health outcome indicated one of the following: death, disability, congenital anomaly, hospitalization (initial or prolonged), required intervention to prevent harm, life threatening or other medically serious outcome. A report from a drug manufacturer that is classified as Expedited is always considered as serious under the QuarterWatch definition. REPORTS EXCLUDED We exclude foreign reports because QuarterWatch focuses on potential risks to the health of patients in the United States. Also, because foreign and domestic reporting rules differ, and countries may have different requirements, reports from outside the United States

2 also introduce new sources of variability. However, foreign reports contain valid information and may be useful in other special analyses. Also excluded are reports from clinical studies. In clinical studies, all serious events are reported whether or not an association of the drug was suspected. Also excluded are reports with occupation = lawyer in the occupation field. While case reports associated with health claims in the legal system are as valid as any other kind of report, these cases are excluded because of a quirk in reporting requirements. We have discovered that the legal departments of pharmaceutical companies routinely submit adverse event reports for each lawsuit filed against the company identifying a company drug product. Typically these reports provide minimal detail about an event that may have occurred years earlier, and may duplicate adverse event reports already filed by the patient making the legal claim. In addition we exclude reports about drugs withdrawn for safety reasons after the date of withdrawal because these cases do not indicate a current hazard to patient health. In historical data prior to 2006 we exclude direct reports to the FDA with a health outcome of other and similar cases in manufacturer reports unless they are marked as Expedited. Report Versions, Counts And Revisions Quarterly report totals for individual drugs and categories are subject to variability in the underlying data. The AERS data may include more than one version of a specific case report. This occurs because manufacturers revise the reports as the companies obtain additional information. We use the most recent revision (called the last best case) but retain the original reporting date. The date used in QuarterWatch is the date the report was initially received by the FDA, regardless of when the event occurred. Variability in the report totals comes from multiple sources. Each quarterly distribution contains hundreds of reports that were received (and belong in) the previous quarter but were not processed in the FDA system in time to be included in the quarterly totals. Thousands of reports from earlier quarters are revised by the manufacturer. Also, manufacturers may submit reports for events that occurred in previous quarters or even previous years thus inflating one quarter s totals. These reports from earlier periods usually occur for two reasons. In some cases a company had a reporting problem and submitted reports for an extended period. Also, FDA regulations allow companies to submit Periodic Reports on an annual basis rather than a quarterly basis three years after approval. Finally, QuarterWatch criteria are changed from time to time as more is learned about the overall system performance. The consequence of this variability is that any previously reported drug, quarter or event total may change over time. As a result, QuarterWatch recalculates the historical and drug trend data every quarter. However the more current historical series may differ slightly from those previously reported. Coding And Classification Of Drug Names We define for analysis a drug as an active, unique chemical moiety that is FDA approved for the diagnosis or treatment of any disorder or disease. Most drug names are standardized to

3 an ingredient name based on the National Library of Medicine drug name terminology called RxNorm. This means that salts, esters, extended release formations, and different routes of administration still appear under the same drug/ingredient name in QuarterWatch analysis. In most cases this makes little or no difference (e.g. amoxicillin sodium or amoxicillin trihydrate) but could have an effect on the analysis in special cases. For example, the immunosuppressant tacrolimus is available as an oral capsule to prevent the rejection of organ transplants as well as a topical cream for atopic dermatitis. The risks and benefits of the two formulations may differ substantially. Where dosage form or route of administration is relevant to our reporting on a specific drug, we indicate this in that section of the report. Special Cases of Drug Names A few drug names and categories present special problems that require a more or less arbitrary solution. A more general category is needed because of a large number of chemically related drugs combined with weak product identification on the original reports. The cases with the greatest effect on results are as follows: All estrogenic drugs are currently combined, including those for contraception and hormone replacement. From time to time we publish detailed breakout showing the brand names that are included in a current quarter results. We also group all interferon products in to the following groups: interferon alfa, interferon beta and interferon gamma. For similar reasons we group together all forms of insulin products. Special Reporting Drugs We classify in the special reporting category certain drugs in which the report volume is influenced by special circumstances that appear unrelated to the safety profile of the product. Certain drugs are approved only for restricted distribution and have special programs for reporting all adverse events whether or not the drug was suspected of causing the event. Examples include thalidomide, lenalidomide, ambrisentan and deferasirox. Since we know of no central listing of restricted distribution products, drugs are added to this category as we learn of their special status. The category includes other drugs where manufacturer direct contact with patients leads to large numbers of reports where the drug was not necessarily suspected of causing the adverse event. Examples include the peritoneal dialysis solution called Dianeal, the injectable bisphosphonate drug ibandronate (Boniva). Drugs We Regularly Monitor In QuarterWatch the overall trends and event totals include Special Reporting Drugs and those such as insulin which are grouped into more general categories. However, most other rankings, including overall serious report rankings, rankings of patient deaths, and rankings for SMQs exclude Special Reporting Drugs. When Special Reporting drugs are excluded we describe the drugs that are the main focus as drugs we regularly monitor.

4 Characterization of Adverse Events In the FDA adverse event reports released for research, the narrative that described the event that occurred is replaced by one or more standardized medical terms drawn from the global standard Medical Dictionary for Regulatory Affairs (MedDRA). The MedDRA terminology was developed by the pharmaceutical industry to standardize international reporting of data from clinical trials and postmarketing adverse events. The dictionary is updated twice a year and QuarterWatch updates its version annually. The most granular (or specific) terms are called Preferred Terms (PT) and number approximately 29,000. An adverse event might be described by a single PT term (e.g. nausea, myocardial infarction) or several terms (i.e. both of the preceding). QuarterWatch analysis also uses two other features of the MedDRA terminology that permits grouping similar or related terms together to provide a broader event description. The terminology provides High Level Terms (HLTs) that collect together a group of closely related terms. For example, 30 different forms of specific intestinal infections are grouped together under the HLT called intestinal infections. The MedDRA terminology provides another tool for grouping together terms that are clinically related (even though not near synonyms) into an umbrella term called a Standardized MedDRA Query (SMQ). The SMQs allow grouping terms that are clinical related but functionally different. For example, a liver function laboratory test result could be grouped with other kinds of terms that might indication a liver disorder. SMQ definitions can be either broad scope or narrow scope; QuarterWatch utilizes broad scope SMQs. Because each case report might contain more than one MedDRA term (on the average more than 3 terms), the problem of possible duplication arises. In QuarterWatch each case can appear in a specific SMQ only once. However, each case might possibly appear in more than one SMQ. (for example, the term nausea could cause a case to be classified in at least three SMQs.) When reporting on specific Preferred Terms we describe the totals as mentions because it means that, for example, 10 reports mentioned the term nausea. However, in any list of Preferred Terms it is possible that one case report might contain terms on that list. PROPORTIONAL REPORTING RATIO We use a statistical technique called the proportional reporting ratio to measure the strength of the association between reports of a side effect and the suspect drug. By comparing the observed number of reports for a side effect with the expected number, given the total report volume, it becomes possible to detect a valid signal from a relatively small number of reports, and to discount some cases where reports were more numerous, but more likely a chance effect. Several factors, unrelated to the safety of a drug, might influence the number of reports received. This includes greater or lesser patient exposure, differences in reporting rate, and the play of random chance.

5 The proportional reporting ratio for a side effect calculates the proportion of reports for the side effect under study compared with the proportion of reports for the side effect for all other drugs in the preceding four calendar quarters. For example, suppose the side effect under study was MedDRA term Depression. We would compare the proportion of reports (suspect drug depression cases/suspect drug all cases) with the proportion for all drugs in the preceding four quarters (all depression cases/all serious reports). For example, if a suspect drug had 10 cases of depression among 100 reports overall, we conclude that the observed proportion was For all drugs in 2011 we identified 3,450 reported mentions of depression among 179,855 cases overall, or an expected value of Therefore the proportional reporting ratio would be 0.1/0.019 or PRR = 5.2. This shows that the suspect drug had 5.2 times the expected value given the reports for all other drugs. The next step is to determine whether this five-fold difference could have occurred by chance. Using the chi-square statistical test (with Yates correction) we calculated that the chi-square value was 40.4 and the probability this association occurred by chance was p < Other Details We use the word signal to characterize the evidence we see of a safety issue. The term signal as used in QuarterWatch means evidence of sufficient weight to justify an alert to the public and scientific community, and to warrant additional investigation to assess a causal relationship to the drug and determine its incidence. The QuarterWatch master database of all adverse event reports submitted to the FDA is maintained on a MySQL open source database ( and analyzed with the R Package for Statistical Computing (