CONCERN ABOUT THE LOW SENsitivity

ORIGINAL CONTRIBUTION Evaluation of Human Papillomavirus Testing in Primary Screening for Cervical Abnormalities Comparison of Sensitivity, Specificity, and Frequency of Referral Shalini L. Kulasingam, PhD James P. Hughes, PhD Nancy B. Kiviat, MD Constance Mao, MD Noel S. Weiss, MD, DrPH Jane M. Kuypers, PhD Laura A. Koutsky, PhD CONCERN ABOUT THE LOW SENsitivity of the conventional Papanicolaou (Pap) smear has prompted a search for newer methods to either supplement or replace it. 1-3 A recent review concluded that 47% of women who developed invasive cervical cancer have had adequate screening histories within 5 years of detection. 4 Some women failed to follow up on an abnormal smear result, while many had a history of negative smear results. The Bethesda system for reading and interpreting cervical cytology specimens was introduced to improve the accuracy of Pap smear diagnoses. 3 However, the diagnostic categories introduced by the Bethesda System appear to be no more reproducible than diagnostic categories of other systems. 5-7 Another concern is the high proportion of smears (over 10% in some populations) read as atypical cells of undetermined significance (ASCUS) and the fact that 5% to 10% of women with this diagnosis have an underlying highgrade lesion. 8 Human papillomavirus (HPV) is now established as a cause of cervical can- Context Human papillomavirus (HPV) DNA testing of women having Papanicolaou (Pap) smears showing atypical squamous cells of undetermined significance (ASCUS) has clinical usefulness. Whether HPV DNA testing alone is useful in primary screening remains to be determined. Objective To determine the accuracy of HPV DNA testing for detecting cervical intraepithelial neoplasia (CIN) grade 3 or cancer (the criterion standard). Design, Setting, and Participants Between December 1997 and October 2000, 4075 women who attended Planned Parenthood clinics in Washington State were screened simultaneously using thin-layer Pap and HPV DNA testing by a polymerase chain reaction (PCR) based method and by a liquid-based RNA-DNA hybridization capture with signal amplification assay (signal amplification). Women who were positive for high-risk HPV types, or had Pap results of ASCUS or higher, were considered to have positive screening test results and were referred for colposcopy and biopsy. Additionally, a random sample of women with negative screening test results was referred for colposcopy. Based on individual and combined thin-layer Pap, HPV PCR, and HPV signal amplification test results from the screening and the colposcopy visits, 7 colposcopy triage strategies were defined and evaluated. Main Outcome Measure Sensitivity and specificity for detecting cervical lesions graded CIN 3 or higher for each of the 7 triage strategies. Results The estimated prevalence of CIN 3 or higher was 3.2%. The sensitivity (95% confidence interval) of thin-layer Pap (with a result of ASCUS) for identifying women with CIN 3 or higher was only 61.3% (48.5%-70.9%) compared with 88.2% (78.9%- 93.8%) for HPV testing by PCR and 90.8% (83.1%-95.8%) by signal amplification. Differences in specificities were also observed: 82.4% (81.8%-83.1%) for thin-layer Pap (with a result of ASCUS), 78.8% (77.9%-79.7%) for PCR, and 72.6% (69.4%- 75.0%) for signal amplification. Compared with referral for colposcopy of all women with ASCUS or higher, signal amplification testing of women with ASCUS and referral of those with a positive result was about as sensitive (61.3% vs 60.3%, respectively) and significantly more specific (82.4% vs 88.9%, respectively). The strategy requiring repeat positive PCR tests on 2 visits had a sensitivity of 84.2% (75.3%-91.0%) and a specificity of 86.2% (85.1%-87.3%). All tests were more specific and less sensitive in older ( 30 years) vs younger women. Conclusions Testing for HPV has higher sensitivity but lower specificity than thinlayer Pap screening. In some settings, particularly where screening intervals are long or haphazard, screening for HPV DNA may be a reasonable alternative to cytologybased screening of reproductive-age women. JAMA. 2002;288:1749-1757 www.jama.com Author Affiliations: Departments of Epidemiology (Drs Kulasingam, Weiss, and Koutsky), Biostatistics (Dr Hughes), Pathology (Drs Kiviat and Kuypers), and Obstetrics and Gynecology (Dr Mao), University of Washington, Seattle. Dr Kulasingam is now with the Department of Obstetrics and Gynecology, Duke University, Durham, NC. Corresponding Author and Reprints: Laura A. Koutsky, PhD, HPV Research Group, 1914 N 34th St, Suite 300, Seattle, WA 98103 (e-mail: kouts@u.washington.edu). 2002 American Medical Association. All rights reserved. (Reprinted) JAMA, October 9, 2002 Vol 288, No. 14 1749

cer. 9,10 In particular, persistent detection of high-risk HPV types is a strong predictor of development of highgrade cervical precancer and invasive cervical cancer. 11-13 A handful of studies, to date, have examined the role of testing for HPV DNA in primary screening. Most of these studies did not refer women with completely negative test results to colposcopy, so the resulting measures of sensitivity may be overly optimistic. In addition, no measures of specificity can be calculated. 14,15 One study of 2098 Canadian women, reported by Ratnam et al, 16 did include referral of women with completely negative test results to colposcopy. The estimated sensitivity of HPV DNA testing was higher than that of conventional Pap smear (68.1% vs 40.2%). The estimated specificity of Pap testing was 91.6% compared with 90.6% for HPV DNA testing. The thin-layer Pap technique was introduced in the early 1990s. It may have a higher sensitivity than the conventional Pap smear and allows for HPV testing of residual liquid that remains after a cytology slide has been made. 17 Despite this improved sensitivity, triage studies of women with Pap smears graded as ASCUS indicate that, compared with a repeat thin-layer Pap smear, a test for high-risk HPV types has a better sensitivity for detecting high-grade lesions or cancer. 8,18 A comparison of the accuracy of thin-layer Pap vs HPV DNA testing in primary screening has not yet been reported. Testing for HPV DNA may be more specific in older vs younger populations. While HPV DNA is commonly detected in cervical specimens of sexually active women younger than 30 years and may indicate transient infection, it is detected less often in women older than 30 years, where it is more likely to be associated with highgrade lesions or cancer. 14 The present study was undertaken to evaluate the potential of HPV testing for improving screening accuracy for detection of high-grade lesions or cancer among a population of women of reproductive age and to see whether the level of accuracy varies by age. Two different assays for HPV DNA testing were compared: a polymerase chain reaction (PCR) based assay and a liquidbased DNA-RNA hybridization capture with signal amplification assay (signal amplification). METHODS Study Population From December 1997 to October 2000, 4358 consecutive women presenting for annual examinations at 1 of 3 Planned Parenthood clinics in Washington State were invited to participate in the study. Women who were 18 to 50 years of age were eligible if they had no history of hysterectomy, chronic immune suppression, or treatment for cervical neoplasia. Women who provided written informed consent were enrolled. The University of Washington Institutional Review Board approved all protocols. Clinical Examinations After consent was obtained, a standardized questionnaire was used to collect demographic, reproductive, and sexual history information. A gynecologic examination was then performed, and screening test samples were obtained according to the following protocol. For cervical cytology, an Ayres spatula was used to collect cells from the transformation zone and a cytobrush was used to collect endocervical cells. Both the brush and the spatula were repeatedly tapped and rinsed in PreservCyt (Cytyc Corp, Boxborough, Mass) for processing the final thin-layer cytology slide (ThinPrep, Cytyc Corp). Next, a specimen for HPV DNA testing was obtained by rotating a dacron-tipped swab in the endocervical os and swabbing it on the ectocervical epithelium. This swab was placed in standard transport medium (STM) (Digene, Gaithersburg, Md). All STM specimens from the screening visit were tested for HPV DNA by PCR. Prior to January 2000, STM specimens from the screening visit of the 712 women who attended colposcopy were tested for HPV DNA by signal amplification (Hybrid Capture 2, Digene). Starting in January of 2000, HPV DNA signal amplification testing of residual thin-layer Pap liquid from all screening visits was introduced (n=1150). No change was made to the specimen collection protocol. Before April 1999, a urine sample was collected (n=1533). Poor sensitivity and a high proportion of inadequate samples (data not presented) led to a decision to stop collecting urine for HPV testing. Women were referred for colposcopy and biopsy if they had ASCUS, atypical glandular cells of undetermined significance (AGUS), low-grade squamous intraepithelial lesion (LSIL), or high-grade squamous intraepithelial lesion (HSIL) or higher on thin-layer Pap; or a positive PCR test result for high-risk HPV types. A 45% random sample of the first 1000 women with negative Pap and HPV DNA test results was invited to have colposcopy performed. This high percentage was initially selected to determine rates of colposcopy participation. Oversampling for this group continued, however, due to low participation. Of 2631 women with negative cytology and HPV results, 1079 (41%) were randomly selected, and 202 (7.7%) had colposcopy performed. In January 2000, with the initiation of screening by signal amplification, women who were positive for highrisk HPV types by signal amplification were also referred for colposcopy. All women eligible for colposcopy were telephoned a minimum of 3 times and sent a letter when telephone contact failed. The average time between the screening visit and the colposcopy visit was 3 months. At the colposcopy visit, a detailed standardized questionnaire was completed and cervical samples for thinlayer Pap and HPV DNA testing were obtained as outlined above for the screening visit. One of 2 experienced colposcopists performed colposcopy using the DenVu computerized colposcopy system (DenVu, Tucson, Ariz). Ectocervical biopsies of visible lesions (or the 12 o clock location if no lesion was visible) were obtained. Endocervical cu- 1750 JAMA, October 9, 2002 Vol 288, No. 14 (Reprinted) 2002 American Medical Association. All rights reserved.

rettage was performed when the lesion extended into the endocervical canal, the Pap test showed HSIL but the lesion was not visible on colposcopy, or the Pap test showed AGUS. Women with histologically confirmed cervical intraepithelial neoplasia (CIN) grade 2 or 3 or invasive cervical cancer were referred for treatment. Women with HSIL on thin-layer Pap and less than CIN 1 on histology underwent repeat biopsy. Women with CIN 1 returned to Planned Parenthood for follow-up care. Cytology and Histology Thin-layer slide preparation and processing was performed according to the manufacturer s specifications (Cytyc Corp). The thin-layer slide was then stained with the Papanicolaou stain, screened by a cytotechnologist (certified by Cytyc), and reviewed by pathologists at Harborview Medical Center in Seattle, Wash, having no knowledge of other laboratory or clinical data. Smears were classified as unsatisfactory based on finding more than 60% of the slide target area without epithelial cells. Cellular changes were classified according to the Bethesda classification system as negative, ASCUS, AGUS, LSIL, HSIL, or suggestive of cancer. 19 A random 10% sample of all slides read as normal was rescreened manually as mandated by federal law. 20 Hemotoxylin-eosin stained slides of the biopsy tissue were prepared and reviewed by the pathologist without knowledge of other clinical or laboratory data. Diagnoses were assigned using both the CIN and Bethesda classification systems. HPV DNA Testing HPV DNA PCR amplification reactions were performed using 5 biotinylated MY09, MY11, and HMB01 primers (Invitogen Co, Carlsbad, Calif) and Amplitaq Gold polymerase (Applied Biosystems, Foster City, Calif). To prevent PCR product carryover, dttp was replaced by dutp and uracil-nglycosylase was added. The human -globin gene was coamplified in the HPV reaction mix using 5 biotinylated primers PC04 and GH20 to monitor specimen adequacy. Two microliters of each specimen was added to 100 µl of reaction mix. Amplification by PCR was carried out in a TC 9600 thermal cycler (Perkin Elmer, Wellesley, Mass) with the following profile: 95 C for 9 minutes to activate the polymerase; 40 cycles at 95 C for 1 minute, 55 C for 1 minute, and 72 C for 1 minute, and a 5-minute terminal extension at 72 C. HPV DNA typing analysis was performed according to the manufacturer s specifications 21 using the reverseline strip test (Roche, Emeryville, Calif) to detect high-risk HPV types 16, 18, 26, 31, 33, 35, 39, 45, 51, 52, 55, 56, 58, 59, 68, 73, 82, or 84. The signal amplification test was configured to detect in a single assay one or more of the following high-risk HPV types: HPV 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, or 68. Prior to January 2000, the assay was performed according to the manufacturer s specifications on residual screening STM samples (after the aliquot for PCR was removed) from 712 women who returned for colposcopy. Starting in January 2000, testing of residual PreservCyt liquid was performed on all screening samples (n=1150). A positive result was recorded for specimens with a relative light unit of 1 or greater, corresponding to 5000 or more HPV DNA copies. As configured and performed in this study, both the PCR and the signal amplification assays had analytic sensitivities of 10 to 100 copies of HPV DNA per sample. Data Analysis Estimates of sensitivity and specificity for detection of CIN 3 or higher and of the percentage of women referred for colposcopy were obtained for 7 screening strategies. These strategies included referral for colposcopy of women with the following positive test results: (1) a screening thin-layer Pap test showing LSIL or higher or a screening thin-layer Pap test showing ASCUS with a follow-up thin-layer Pap test showing ASCUS or higher; (2) a screening thin-layer Pap test showing ASCUS or higher; (3) a screening thin-layer Pap test showing LSIL or higher or a screening thin-layer Pap test showing ASCUS with a screening PCR test result that was positive for high-risk HPV types; (4) a screening thin-layer Pap test showing LSIL or higher or a screening thinlayer Pap test showing ASCUS with a screening signal amplification test result that was positive for high-risk HPV types; (5) a screening PCR test result that was positive for high-risk HPV types; (6) a screening and follow-up PCR test result that was positive for high-risk HPV types; or (7) a screening signal amplification test result that was positive for high-risk HPV types. A strategy of repeat testing for highrisk HPV DNA by signal amplification could not be evaluated because only 163 women had these results for both the screening and follow-up visits. To calculate sensitivity and specificity, 2 2 tables were generated using results from the screening and colposcopy visits. Analyses were first performed using the total population and then separately for women younger than 30 years and for women aged 30 years or older. The definition of a positive screening test result was based on the strategy-specific criteria for referral for colposcopy. Depending on the strategy, women with an abnormal Pap result ( ASCUS) or high-risk HPV DNA were categorized as test positive. Women with negative results (including women who were positive for lowrisk HPV types) were categorized as test negative. Since women with inadequate Pap results are usually asked to return for a repeat test, Pap and HPV DNA test results that were inadequate, missing, or insufficient were initially coded as positive. A sensitivity analysis was performed to determine how coding the inadequate, insufficient, and missing results as negative affected the estimates of sensitivity and specificity. Additional sensitivity analyses were based on the use of CIN 2 or higher as the criterion standard, and on separate analyses for signal amplifica- 2002 American Medical Association. All rights reserved. (Reprinted) JAMA, October 9, 2002 Vol 288, No. 14 1751

Table 1. Demographic Characteristics of Screened Women (N = 4075)* Characteristic No. (%) Age, y 18-19 736 (18) 20-24 1686 (41) 25-29 888 (22) 30-34 438 (11) 35-39 214 (5) 40-44 88 (2) 45-50 20 ( 1) Race/ethnicity African American 425 (10) American Indian 52 (1) Asian 132 (3) Hispanic 146 (4) White 2924 (72) Other 391 (10) Lifetime No. of sex partners 0 4 ( 1) 1 484 (12) 2 365 (9) 3-5 1127 (27) 6-9 753 (18) 10 1337 (33) Parity 0 2468 (60) 1 859 (22) 2 469 (11) 3 274 (7) Using birth control Yes 1436 (35) No 2634 (65) Education High school 1648 (40) High school 2422 (60) Monthly income, $ 400 1028 (25) 401-800 1083 (27) 801-1300 989 (24) 1300 970 (24) *Five women had missing demographic information. tion prior to and after January 2000 to account for differences in specimen collection and testing. Using the screening results, we defined screening strata based on the 6 possible combinations of thinlayer Pap results (ASCUS/LSIL, HSIL, normal) and PCR result (positive/ negative for high-risk HPV DNA). For women with normal Pap and negative PCR results, we deliberately invited only a random subsample to return for colposcopy. Analyses indicated that the likelihood of referral for colposcopy depended on the perceived severity of the screening results. Thus, estimates of sensitivity, specificity, and prevalence that ignore these sampling issues will be biased. It is possible to correct for this bias when estimating sensitivities, specificities, and prevalence. Further details in each case are provided below. To determine the expected number of women testing positive by signal amplification overall and prior to 2000, the number of positive screening signal amplification samples was divided by the sampling fraction for each screening stratum. Summation across all strata provided an estimate of the total number of positive results that would have been obtained if all women screened had been tested by signal amplification. Uncorrected sensitivity, corrected sensitivity, and corrected specificity were calculated as follows. Uncorrected sensitivity was calculated with the usual formula for sensitivity: k 1 x i (1) Uncorrected Sensitivity = i =1 k 1 n i i =1 where n i is the number of true positives who returned for colposcopy in the ith stratum and x i is the number of those true positives that screen positive. The sums are over 5 of the 6 screening strata. Specifically, we omit the stratum of women who screened negative by both Pap and PCR. Thus, uncorrected sensitivity refers to calculations based only on women who returned for colposcopy and who had a positive screening result for at least 1 of the tests. Such estimates can only be used to determine whether more cases were associated with a positive result on one test than on another test. The lack of outcome information for women with negative results for both tests means that there is incomplete information for the total number of false-negative results and true-negative results for each test (ie, there is verification bias). Corrected sensitivity was calculated from the following formula: (2) Corrected Sensitivity = i =1 k k i =1 xi ù i ni ù i where f i is the proportion of those in screening stratum i who return for colposcopy. Corrected specificity was defined analogously. The corrected sensitivity and specificity estimate the sensitivity and specificity that would have been obtained if all women who were screened had returned for colposcopy. Within each screening stratum, the distribution of lesion grades among those who did and did not return for colposcopy was assumed to be the same. This is a plausible assumption, since HPV DNA and thin-layer Pap results were the only factors found to be independently associated with both return for colposcopy and detection of CIN 3 or higher. Other potentially confounding factors, including age, race/ethnicity, education, income, number of sex partners, hormonal contraceptive use, time between screening and colposcopy visit, and parity, were not independently associated with both colposcopy attendance and detection of CIN 3 or higher (data not shown). Means, medians, and SDs were calculated for continuous variables. Differences in proportions were tested using the 2 test. Univariate and multivariate logistic regression analyses were performed to estimate odds ratios (ORs) and 95% confidence intervals (CIs) for colposcopy attendance and high-grade histology results. Variances and 95% CIs for corrected sensitivities and specificities were calculated using bootstrap resampling in which the sampling fractions were considered fixed. 22 All analyses were performed using STATA 7.0 (STATA Corp, College Station, Tex). RESULTS Population Characteristics Of 4358 eligible women, 4075 (93.5%) were enrolled. These women were a mean age of 25 years (SD, 5.7), were predominantly white, and reported a lifetime median of 6 sex partners (range, 0-148) (TABLE 1). Compared with women younger than 30 years (n=3310), women aged 30 years or older (n=760) reported a higher lifetime number of sex partners (median of 5 vs 9, respectively), and higher incomes (median of $800 vs $1100 per month, respectively). Older and younger women were simi- 1752 JAMA, October 9, 2002 Vol 288, No. 14 (Reprinted) 2002 American Medical Association. All rights reserved.

lar in terms of race/ethnicity, parity, use of oral contraceptives, and education. The screening thin-layer Pap results of 678 (16.6%) of 4075 women were abnormal (TABLE 2). High-risk HPV DNA was detected by PCR in screening samples of 747 (18.3%) women, of whom 231 (31%) women were infected with multiple types (including high- and low-risk types). Among women who were positive for 1 or more high-risk types, HPV type 16 was the most common type detected (28.7%, n=214), followed by HPV type 52 (13.2%, n=99) and HPV type 51 (11.4%, n=85). HPV type 18 was detected in 8.4% (n=63), HPV type 45 in 4.9% (n=37), and HPV type 31 in 7.1% (n=53) of women screening positive for HPV DNA. Extrapolation of the screening signal amplification results of 712 women who returned for colposcopy to all women screened prior to 2000 provided a high-risk HPV DNA prevalence estimate of 26%. The prevalence estimate for the 1150 women who were screened post-2000, when signal amplification screening results were available for all women, was 28.5%. Since these 2 prevalence estimates were similar, the weighted estimate (27.4%) that was based on a weighted summation of the percentage positive within each screening cytology and PCR stratum was used in calculations of sensitivity and specificity for all strategies that included a signal amplification test. The percentages of women with different cytologic findings who were positive by PCR for high-risk HPV types were as follows: 82.0% (91/111) with HSIL, 64.4% (104/166) with LSIL, 35.7% (143/401) with ASCUS, 12.0% (398/3318) with normal cytology results, and 11.3% (9/79) with inadequate cytology specimens.the estimated percentages (in the overall population) of women who had positive results by signal amplification were as follows: 85.6% (95/111) for HSIL, 81.9% (136/166) for LSIL, 47.4% (190/ 401) for ASCUS, and 20.5% (680/ 3318) for women with normal smear results. The prevalence of HPV (by PCR) increased in women aged 18 to 24 years and then declined sharply (FIGURE 1). The prevalence of LSIL declined steadily with increasing age. The prevalence of ASCUS averaged approximately 10% across all age categories. The esti- Table 2. Screening Test Results* Result No. (%) Thin-layer Papanicolaou test HSIL 111 (2.7) LSIL 166 (4.1) ASCUS/AGUS 401 (9.8) Normal 3318 (81.4) Unsatisfactory 79 (1.9) Test for HPV DNA by PCR Positive 747 (18.3) Positive for low-risk 158 (3.9) types only Negative 3004 (73.7) Insufficient 159 (3.9) Missing 7 ( 1) Test for HPV DNA by signal amplification Positive for high-risk 327 (28.4) types Negative 799 (69.5) Insufficient 24 (2.1) *HSIL indicates high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion; ASCUS, atypical squamous cells of undetermined significance; AGUS, atypical glandular cells of undetermined significance; HPV, human papillomavirus; and PCR, polymerase chain reaction. Four women were diagnosed with AGUS by a screening Papanicolaou test. Low-risk HPV types 6, 11, 40, 42, 53, 54, 57, 66, and 84 detected by the reverse-line strip test. Based on a subsample of 1150 women screened by signal amplification beginning in January 2000. Figure 1. Age-Specific Prevalence of High-Risk HPV DNA (PCR), ASCUS, LSIL, HSIL (Histology), and CIN 3 (Cytology) 25 HPV by PCR ASCUS ASCUS 20 15 10 5 Positive or Abnormal, % 0 25 20 LSIL HSIL CIN 3 15 10 5 0 18-19 20-24 25-29 30-34 35-50 Age, y 18-19 20-24 25-29 30-34 35-50 Age, y 18-19 20-24 25-29 30-34 35-50 Age, y See Table 1 for number of women in each age group. HPV indicates human papillomavirus; PCR, polymerase chain reaction; ASCUS, atypical squamous cells of undetermined significance; LSIL, low-grade squamous intraepithelial lesion; HSIL, high-grade squamous intraepithelial lesion; and CIN, cervical intraepithelial neoplasia. 2002 American Medical Association. All rights reserved. (Reprinted) JAMA, October 9, 2002 Vol 288, No. 14 1753

mated prevalence of histologically confirmed CIN 3 or higher increased slightly with increasing age. Of the 1015 women who had biopsies performed, 87 had results of CIN 3 or higher (FIGURE 2), including 1 woman with endocervical adenocarcinoma in situ. Sixty-four (73.6%) of these lesions were detected in women younger than 30 years. The results for Figure 2. Enrolled Women Cross-Classified by Screening Test Results, Colposcopy Attendance, and Detection of CIN 3 380 Had ASCUS HPV-Positive (by PCR) 298 Returned for Colposcopy 58 Had CIN 3 377 Had ASCUS HPV-Negative (by PCR) 219 Returned for Colposcopy 7 Had CIN 3 4358 Patients Eligible 283 Refused 4075 Enrolled 533 Had Normal Cytology HPV-Positive (by PCR) 238 Returned for Colposcopy 19 Had CIN 3 2631 Had Normal Cytology HPV-Negative (by PCR and Signal Amplification) 202 Returned for Colposcopy 0 Had CIN 3 154 Had Normal Cytology HPV-Negative (by PCR) and Were HPV-Positive (by Signal Amplification or Urine Test ) 62 Returned for Colposcopy 3 Had CIN 3 CIN indicates cervical intraepithelial lesion; ASCUS, atypical squamous cells of undetermined significance; HPV, human papillomavirus; and PCR, polymerase chain reaction. *Includes 24 women with inadequate Papanicolaou (Pap) samples, 29 with insufficient HPV DNA samples, and 1 with missing HPV DNA test results. Includes 55 women with inadequate Pap samples. Includes 134 women with insufficient HPV DNA samples and 2 with missing results. Test dropped early in the study and omitted from the Results section. Includes 29 women with insufficient HPV DNA samples and 8 with missing signal amplification results. Includes 4 women who did not undergo biopsy. women with lower than CIN 3 were as follows: 50 had CIN 2, 185 had CIN 1, 137 had atypia, and 556 had negative histologic findings. Three women (ages 30, 32, and 34 years) with CIN 3 or higher diagnosed by biopsy were found to have microinvasive cancer in the specimen from the loop electrosurgical excision procedure and subsequently underwent hysterectomy. All 3 were positive for HPV DNA by signal amplification and had HSIL Pap results at the time of screening. Two of the 3 women were PCR-positive at screening; all 3 were PCR-positive at their colposcopy visit. Sensitivity and Specificity The corrected (for colposcopy attendance and verification bias) estimate of CIN 3 or higher prevalence in this population was 3.2%. The 3 screening strategies that were based on testing for highrisk HPV DNA by PCR or by signal amplification were significantly more sensitive than any of the 4 strategies that included thin-layer Pap (84.2%-90.8% vs 57.2%-61.3%) (TABLE 3). However, the specificity estimates for the HPV DNA detection strategies tended to be lower than for those of the thin-layer Pap based strategies (72.6%-86.2% vs Table 3. Detection of CIN 3: Sensitivity, Specificity, and Percentage of Women Referred for Colposcopy for Each Screening Strategy* Screening Strategy Result Uncorrected Sensitivity, % Corrected (95% CI) Corrected Specificity (95% CI), % Referred for Colposcopy, % Negative Predictive Value ASCUS and repeat ASCUS 72.4 57.2 (46.2-66.6) 89.9 (89.0-90.7) 11.6 98.4 ASCUS 74.7 61.3 (48.5-70.9) 82.4 (81.8-83.1) 19.0 98.5 with PCR test with signal amplification test ASCUS and positive for high-risk HPV DNA ASCUS and positive for high-risk HPV DNA 73.6 59.8 (47.1-68.9) 89.8 (89.2-90.5) 11.8 98.5 73.6 60.3 (47.4-69.6) 88.9 (88.1-89.6) 12.7 98.5 PCR test for HPV DNA Positive 88.5 88.2 (78.9-93.8) 78.8 (77.9-79.7) 23.4 99.5 PCR test for HPV DNA Repeat positive for 85.1 84.2 (75.3-91.0) 86.2 (85.1-87.3) 16.1 99.3 high-risk types Signal amplification test Positive at 1 RLU 90.8 90.8 (83.1-95.8) 72.6 (69.4-75.0) 29.4 99.6 *CIN indicates cervical intraepithelial neoplasia; CI, confidence interval; Pap, Papanicolaou; ASCUS, atypical squamous cells of undetermined significance; LSIL, low-grade squamous intraepithelial lesion; PCR, polymerase chain reaction; HPV, human papillomavirus; and RLU, relative light unit. Estimates based only on data from women who were screened, had a positive screening test result, and returned for colposcopy. Estimates corrected for verification bias and bias due to loss to follow-up. 1754 JAMA, October 9, 2002 Vol 288, No. 14 (Reprinted) 2002 American Medical Association. All rights reserved.

82.4%-89.9%). Compared with referral for colposcopy of all women with ASCUS or higher, signal amplification testing of women with ASCUS and referral of those with a positive result was about as sensitive (61.3% vs 60.3%, respectively) and significantly more specific (82.4% vs 88.9%, respectively). A single signal amplification test for highrisk HPV DNA referred the highest proportion of women for colposcopy. Compared with strategies that required repeat Pap or HPV DNA testing of women with ASCUS, the referral strategy based on repeated PCR positivity was significantly more sensitive (84.2% vs 57.2%-60.3%) but significantly less specific (86.2% vs 88.9%-89.9%). Although the differences were not significant, all screening strategies were more sensitive for CIN 3 or higher among women younger than 30 years than for women aged 30 years or older (TABLE 4). Estimates of specificity, however, were significantly higher for women aged 30 years or older than for those who were younger than 30 years. Similar results were obtained when CIN 2 or higher was used as the definition of high-grade disease, and again all sensitivity estimates for those aged 30 years or older were lower than for those younger than 30 years. Estimates of sensitivity and specificity were not appreciably affected by coding inadequate or missing screening test results as negative instead of positive. Lastly, estimates of sensitivity and specificity for strategies that included signal amplification prior to and after 2000 were not significantly different (data not shown). COMMENT In this first US-based study comparing screening by thin-layer Pap smear to screening by HPV DNA tests for detection of CIN 3 or higher, HPV DNA testing was significantly more sensitive, but significantly less specific, than thinlayer Pap testing. Compared with referring all women with abnormal smear results for colposcopy, strategies that incorporated HPV DNA testing of Table 4. Corrected Sensitivity and Specificity Estimates for Detecting CIN 3 or CIN 2-3, by Age* Screening Strategy Result Corrected Sensitivity, % Corrected Specificity, % Age 30 y Age 30 y Age 30 y Age 30 y CIN 3 (95% CI) ASCUS and repeat ASCUS 61.2 (47.9-73.6) 45.7 (29.0-67.5) 88.6 (87.6-89.5) 95.4 (93.6-96.9) ASCUS 65.4 (51.9-79.1) 49.7 (32.9-71.5) 81.5 (80.7-82.3) 86.4 (84.8-88.1) with PCR test with signal amplification test PCR test for HPV DNA PCR test for HPV DNA Signal amplification test ASCUS and positive for high-risk HPV DNA ASCUS and positive for high-risk HPV DNA Positive for high-risk types Repeat positive 63.4 (50.4-77.1) 49.7 (32.9-71.5) 88.5 (87.7-89.3) 95.4 (93.9-96.7) 64.0 (51.1-77.6) 49.7 (32.9-71.5) 87.6 (86.7-88.4) 94.7 (92.8-96.1) 91.1 (81.0-97.2) 80.0 (58.8-92.2) 76.8 (75.7-77.8) 87.4 (85.7-89.6) 87.2 (77.7-95.0) 76.0 (55.8-89.4) 84.6 (83.3-85.9) 93.6 (91.2-95.7) Positive at 1 RLU 92.5 (83.5-97.3) 86.0 (59.7-96.9) 70.1 (66.5-73.1) 83.0 (76.8-87.1) CIN 2-3 (95% CI) ASCUS and repeat ASCUS 45.9 (32.7-58.1) 35.6 (19.4-62.6) 89.4 (88.3-90.4) 95.6 (93.8-97.2) ASCUS 50.1 (35.2-62.2) 38.3 (19.3-63.3) 82.1 (81.3-83.0) 86.4 (84.7-88.3) with PCR test with signal amplification test PCR test for HPV DNA PCR test for HPV DNA Signal amplification test ASCUS and positive for high-risk HPV DNA ASCUS and positive for high-risk HPV DNA Positive for high-risk types Repeat positive *See Table 3 footnote for expansions of terms. Estimates corrected for verification bias and bias due to loss to follow-up. 47.6 (33.4-59.5) 38.3 (19.3-63.3) 89.3 (88.4-90.2) 95.7 (94.1-97.0) 47.9 (34.1-60.0) 38.3 (19.3-63.3) 88.3 (87.4-89.2) 95.0 (93.0-96.4) 69.9 (49.4-85.2) 56.5 (30.3-85.5) 77.8 (76.7-78.9) 87.3 (85.5-89.5) 63.1 (45.1-77.3) 53.9 (28.1-83.1) 85.5 (84.2-86.9) 93.6 (91.4-95.8) Positive at 1 RLU 73.5 (53.3-87.7) 62.7 (34.1-93.2) 71.1 (67.3-74.0) 83.0 (76.6-87.2) 2002 American Medical Association. All rights reserved. (Reprinted) JAMA, October 9, 2002 Vol 288, No. 14 1755

ASCUS resulted in significant improvements in specificity, with only slight decreases in sensitivity. The strategy of requiring 2 positive PCR tests for HPV DNA was both more sensitive and specific than referral of women with abnormal smear results for colposcopy. All results were robust to differences in assumptions regarding the definition of high-grade disease, age of the screening population, and coding of missing or unsatisfactory screening test results. Our uncorrected sensitivity estimates (for detection of CIN 2 or higher) were generally similar to estimates presented in other studies. 15 A comparison of our results with those from a study of 2098 Canadian women (which included referral of women with negative screening results) 16 showed similar corrected sensitivities for Pap tests (47.5% vs 40.2%) and for signal amplification HPV DNA tests (71.4% vs 68.1%), but lower corrected specificity estimates for Pap testing (82.9% vs 91.6%), and for signal amplification testing (73.4% vs 90.6%). These differences may be due in part to the fact that, in the Canadian study, biopsies were obtained at the discretion of the colposcopist, women older than 50 years were included, conventional Pap smears were used, and a lesssensitive signal amplification test was used for most of the study. Overall, more women were positive for high-risk HPV DNA by signal amplification (27.4%) than by PCR (18.3%). Although a similar proportion of women with HSIL were positive by signal amplification and by PCR, more women with LSIL, ASCUS, or normal cytology results were positive by signal amplification than by PCR. One explanation for this difference is that the signal amplification test for highrisk HPV types will detect the low- and indeterminate-risk types that are frequently found in samples from women with normal, ASCUS, or LSIL Pap results, but not in samples from women with HSIL. 23-25 Limiting HPV testing to women aged 30 to 50 years resulted in improved specificity. However, thinlayer Pap testing was also more specific, and both HPV testing and thinlayer Pap testing were less sensitive in the older population. Due to the agerelated shift of the squamocolumnar junction from the ectocervix to the endocervix, sample collection for both Pap and HPV DNA tests may be more difficult among older vs younger women. The characteristics of our population were similar to those of the population enrolled in the ASCUS/LSIL Triage Study (ALTS), 8 a multicenter study of triage strategies for women with ASCUS or LSIL sponsored by the National Institutes of Health. In particular, the CIN 3 or higher prevalence among 401 women with ASCUS in our study was comparable to the CIN 3 or higher prevalence reported for 1149 women with ASCUS enrolled in ALTS (5.2% vs 5.1%, respectively), where histologic outcomes were determined by an expert pathology panel. Despite differences in age and ethnicity between our Planned Parenthood population and the population of women from health maintenance organizations studied by Manos et al, 18 both studies showed that testing of women with ASCUS for high-risk HPV DNA by signal amplification was more sensitive for detection of CIN 2 or higher than was a repeat Pap test showing ASCUS or higher. Our study extends these findings by showing that, compared with referring all women with ASCUS to colposcopy, referring only those with positive signal amplification test results achieved a significant improvement in specificity, with only a slight loss in sensitivity. Whether estimates of sensitivity and specificity for all of the 7 screening strategies can be reproduced in older and less sexually active populations remains to be determined. Our most serious potential limitation was low colposcopy attendance. However, prior studies of screening in the United States have observed nonattendance of 23% to more than 60%. 26-27 In our study, women who returned for colposcopy tended to have more severe Pap smear findings and positive HPV test results. To address the potential bias arising from differences in colposcopy attendance for women with different test results, our estimates of sensitivity and specificity were corrected. Importantly, while there were changes in the value of estimates after correction, the overall conclusions did not change. Additionally, none of the women with completely negative screening results who returned for colposcopy had CIN 3 or higher, suggesting that referral based on high-risk HPV DNA testing in conjunction with thinlayer Pap testing identified most, if not all, women with CIN 3 or higher, although our sample of women with completely negative results was relatively small. In our study, a higher percentage of women with abnormal Pap results (72.7%) than women with positive HPV DNA results (67.9%) complied with recommended follow-up. Schneider et al 28 reported reversed percentages, with 80.1% of women with positive HPV DNA tests and 61.9% of women with abnormal cytology findings complying with recommended follow-up. When a significant portion of a referred population fails to comply with follow-up, it suggests that patients and/or their clinicians are uncertain as to the significance of an abnormal or positive screening test result. 29 Better tests, clearer guidelines, and more education will help. Cervical cancer screening practices are quite variable. In settings where programs invite women to participate and virtually all women are screened at regular intervals, the costs (psychological and monetary) of screening with a less-specific test may outweigh the risk of delayed treatment associated with use of a less-sensitive test. In settings where screening is more haphazard than routine, or performed infrequently, screening with a highly sensitive test provides greater reassurance that disease has not been missed in women who screen negative. In summary, a single HPV DNA test was more sensitive but less specific for detection of CIN 3 or higher than any of the cytology-based strategies. All screening strategies were somewhat less sensitive and significantly more spe- 1756 JAMA, October 9, 2002 Vol 288, No. 14 (Reprinted) 2002 American Medical Association. All rights reserved.

cific among women aged 30 years and older vs women younger than 30 years. In some settings, screening for highrisk HPV DNA may be a reasonable alternative to cytology-based screening. Author Contributions: Study concept and design: Kulasingam, Kiviat, Koutsky. Acquisition of data: Kulasingam, Mao, Kuypers. Analysis and interpretation of data: Kulasingam, Hughes, Weiss, Koutsky. Drafting of the manuscript: Kulasingam, Koutsky. Critical revision of the manuscript for important intellectual content: Kulasingam, Hughes, Kiviat, Mao, Weiss, Kuypers, Koutsky. Statistical expertise: Kulasingam, Hughes. Obtained funding: Kiviat, Koutsky. Administrative, technical, or material support: Kiviat, Kuypers, Koutsky. Study supervision: Weiss, Kuypers, Koutsky. Clinical supervision: Mao. Funding/Support: This study was funded by grant CA 34493 from the National Cancer Institute. Acknowledgment: We thank the staff of Tacoma and Lakewood Planned Parenthood Clinics for their generous participation, Kimberle Tomlinson, BA, for her clinical coordinating expertise, and Connie Nelson, BA, for her programmatic oversight. REFERENCES 1. Koss LG. The Papanicolaou test for cervical cancer detection: a triumph and a tragedy. JAMA. 1989;261: 737-743. 2. Gay JD, Donaldson LD, Goellner JR. False negative results in cervical cytologic studies. Acta Cytol. 1985;29:1043-1046. 3. National Cancer Institute. The 1988 Bethesda system for reporting cervical/vaginal cytological diagnoses. JAMA. 1989;262:931-934. 4. Sasieni PD, Cuzick J, Lynch-Framery E. Estimating the efficacy of screening by auditing smear histories of women with and without cervical cancer. Br J Cancer. 1996;73:1001-1005. 5. Confortini M, Biggeri A, Cariaggi MP, et al. Intralaboratory reproducibility in cervical cytology: results of the application of a 100-slide set. Acta Cytol. 1993; 37:49-54. 6. Klinkhamer PJ, Vooijs GP, de Haan AF. Intraobserver and interobserver variability in the diagnosis of epithelial abnormalities in cervical smears. Acta Cytol. 1988;32:794-800. 7. Stoler MH, Schiffman M. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA. 2001;285:1500-1505. 8. Solomon D, Schiffman M, Tarone B, for the ALTS Group. Comparison of three management strategies for patients with atypical squamous cells of undetermined significance (ASCUS): baseline results from a randomized trial. J Natl Cancer Inst. 2001;93:293-299. 9. International Agency for Research on Cancer (IARC) Working Group on the Evaluation of Carcinogenic Risks to Humans. Human Papillomaviruses. Lyon, France: World Health Organization; 1995. IARC Monographs on the Evaluation of Carcinogenic Risks to Humans. Vol 64. 10. Bosch FX, Manos MM, Munoz N, et al. Prevalence of human papillomavirus in cervical cancer: a worldwide perspective. J Natl Cancer Inst. 1995;87: 796-802. 11. Koutsky LA, Holmes KK, Critchlow CW, et al. A cohort study of the risk of cervical intraepithelial neoplasia grade 2 or 3 in relation to papillomavirus infection. N Engl J Med. 1992;327:1272-1278. 12. Ho GY, Burk RD, Klein S, et al. Persistent genital human papillomavirus infection as a risk factor for persistent cervical dysplasia. J Natl Cancer Inst. 1995;87: 1365-1371. 13. Wallin KL, Wiklund F, Angstrom T, et al. Typespecific persistence of human papillomavirus DNA before the development of invasive cervical cancer. N Engl J Med. 1999;341:1633-1638. 14. Cuzick H, Beverley E, Ho L, et al. HPV testing in primary screening of older women. Br J Cancer. 1999; 81:554-558. 15. Cuzick J, Szarewski A, Terry G, et al. Human papillomavirus testing in primary cervical screening. Lancet. 1995;345:1533-1536. 16. Ratnam S, Franco E, Ferenczy A. Human papillomavirus testing for primary screening of cervical cancer precursors. Cancer Epidemiol Biomarkers Prev. 2000;9:945-951. 17. Bernstein SJ, Sanchez-Ramos L, Ndubisi B. Liquidbased cervical cytologic smear study and conventional Papanicolaou smears: a meta-analysis of prospective studies comparing cytologic diagnosis and sample adequacy. Am J Obstet Gynecol. 2001;185: 308-317. 18. Manos MM, Kinney WK, Hurley LB, et al. Identifying women with cervical neoplasia using human papillomavirus DNA testing for equivocal Papanicolaou results. JAMA. 1999;281:1605-1610. 19. Luff RD. The Bethesda system for reporting cervical/ vaginal cytologic diagnoses: report of the 1991 Bethesda Workshop. Hum Pathol. 1992;23:719-721. 20. Center for Devices and Radiological Health, US Food and Drug Administration. Clinical Laboratory Improvement Methods. 2002. Available at: http:// www.fda.gov/cdrh/clia. Accessibility verified September 13, 2002. 21. Gravitt PE, Peyton CL, Apple RJ, Wheeler CM. Genotyping of 27 human papillomavirus types using L1 consensus PCR products by a single hybridization reverse line-blot detection method. J Clin Microbiol. 1998;36:3020-3027. 22. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York, NY: Chapman & Hall; 1993: 436. 23. Terry G, Ho L, Londesborough P, Cuzick J, Mielzynska-Lohnas I, Lorincz A. Detection of high risk HPV types by the signal amplification 2 test. J Med Virol. 2001;65:155-162. 24. Peyton CL, Schiffman M, Lörincz AT, et al. Comparison of PCR- and signal amplification-based human papillomavirus detection systems using multiple cervical specimen collection strategies. J Clin Microbiol. 1998;36:3248 3254. 25. ALTS Group. Human papillomavirus testing as a triage strategy for women with cytologic evidence of low grade squamous intraepithelial lesions: baseline data from a randomized trial. J Natl Cancer Inst. 2000; 92:397-402. 26. Eger RR, Peipert JF. Risk factors for noncompliance in a colposcopy clinic. J Reprod Med. 1996; 41:671-674. 27. Paskett ED, Phillips KC, Miller ME. Improving compliance among women with abnormal Papanicolaou smears. Obstet Gynecol. 1995;86:353-359. 28. Schneider A, Hoyer H, Lotz B, et al. Screening for high-grade cervical intra-epithelial neoplasia and cancer by testing for high-risk HPV, routine cytology or colposcopy. Int J Cancer. 2000;89:529-534. 29. Khanna N, Phillips MD. Adherence to care plan in women with abnormal Papanicolaou smears: a review of barriers and interventions. J Am Board Fam Pract. 2001;14:123-130. 2002 American Medical Association. All rights reserved. (Reprinted) JAMA, October 9, 2002 Vol 288, No. 14 1757