Radiol Oncol 2021; 55(4): 426-432. doi: 10.2478/raon-2021-0043 426 research article Diagnostic performance of p16/Ki-67 dual immunostaining at different number of positive cells in cervical smears in women referred for colposcopy Ursula Salobir Gajsek1, Andraz Dovnik2, Iztok Takac2,3, Urska Ivanus4,5, Tine Jerman4, Simona Sramek Zatler6, Alenka Repse Fokter6 1 Department of Obstetrics and Gynecology, General Hospital Celje, Celje, Slovenia 2 University Department of Gynecology and Perinatology, University Medical Center Maribor, Maribor, Slovenia 3 Department of Gynecology and Obstetrics, Faculty of Medicine, University of Maribor, Maribor, Slovenia 4 National Cervical Cancer Screening Programme and Registry ZORA, Cancer Screening Department, Institute of Oncology Ljubljana, Ljubljana, Slovenia 5 Faculty of Medicine, University of Ljubljana, Ljubljana, Slovenia 6 Department of Pathology and Cytology, General Hospital Celje, Celje, Slovenia Radiol Oncol 2021; 55(4): 426-432. Received 7 July 2020 Accepted 28 September 2021 Correspondence to: Uršula Salobir Gajšek, M.D., Department of Obstetrics and Gynecology, General Hospital Celje, Oblakova 5, 3000 Celje, Slovenia. E-mail: urska.salobir.gajsek@gmail.com Disclosure: No potential conflicts of interest were disclosed. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Background. The aim of the study was to evaluate the diagnostic accuracy of p16/Ki-67 dual immunostaining (p16/ Ki-67 DS) in cervical cytology and the number of positive p16/Ki-67 cells to diagnose high grade cervical intraepithelial neoplasia (CIN2+) in colposcopy population. Subjects and methods. We performed an analysis on a subset cohort of 174 women enrolled within a large-scale randomised controlled human papillomavirus (HPV) self-sampling project organised as part of the population-based Cervical Cancer Screening Programme ZORA in Slovenia. This subset cohort of patients was invited to the colposcopy clinic, underwent p16/Ki-67 DS cervical cytology and had the number of p16/Ki-67 positive cells determined. Results. Among analysed women, 42/174 (24.1%) had histologically confirmed CIN2+. The risk for CIN2+ was increas- ing with the number of positive cells (p < 0.001). The sensitivity of p16/Ki-67 DS for detection of CIN2+ was 88.1%, speci- ficity was 65.2%, positive predictive value was 44.6% and negative predictive value was 94.5%. Conclusions. Dual p16/Ki-67 immunostaining for the detection of CIN2+ has shown high sensitivity and high negative predictive value in our study, which is comparable to available published data. The number of p16/Ki-67 positive cells was significantly associated with the probability of CIN2+ detection. We observed a statistically significant and clini- cally relevant increase in specificity if the cut-off for a positive test was shifted from one cell to three cells. Key words: cervical cytology; high-grade dysplasia; p16/Ki-67 immunostaining Introduction For many decades, cervical cancer prevention has been based on screening with cervical cytology.1 This method has two major drawbacks: high varia- bility in interpretation among cytopathologists and relatively low sensitivity, which requires shorter screening intervals.2 The interpretation of cervical cytology requires experience and long-term train- ing.3 Inevitable factor in development of cervical can- cer is infection with high-risk human papillomavi- Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears 427 rus (HPV)4, but it is not sufficient. However, other cofactors, such as smoking, have been identified to increase the risk of cervical cancer in HPV positive women as well.5,6 Some European countries have already implemented primary HPV screening in women aged 30–35 years and older due to the higher sensitivity of validated HPV tests compared to cytology, taking into account the lower specific- ity of HPV tests due to high HPV prevalence in younger women.1,7 Due to the challenges of cytology and HPV cer- vical screening, novel biomarkers have been stud- ied. Dual p16/Ki-67 immunostaining (p16/Ki-67 DS) has shown promising sensitivity and specific- ity for the detection of high-grade cervical intraepi- thelial neoplasia (CIN2+).8-12 Tjama et al. reported in a systematic literature review that in the Belgian screening population (age 25–65), p16/Ki-67 DS cy- tology was significantly more sensitive and slightly less specific than cytology, but in the population with low-grade changes (atypical squamous cells of undetermined significance [ASC-US], low-grade intraepithelial lesion [LSIL]) and the population referred to colposcopy dual-stain with p16/Ki-67 specificity was statistically significantly higher (+ 25–30%) and sensitivity statistically significantly lower (– 5–6%) than HPV testing.13 p16/Ki-67 DS is based on simultaneous detection of p16 and Ki67 proteins in cervical smears. p16 protein is an im- portant cycline-dependent kinase (CDK) inhibitor which directly controls the progression of the cell cycle from the G1 phase to the S phase and induces cell cycle arrest under physiological conditions. It is expressed in cells, which are infected by HPV, a sign of HPV E7 action on tumour suppressor gene Rb.14-18 Ki-67 is a cell proliferation marker, strongly associated with tumour cell proliferation and growth and is widely used as a proliferation marker. It is a nuclear non-histone protein and is expressed in all phases of the cell cycle, except dur- ing the G0 phase.2,19,20 Normally, over-expression of p16 and expression of Ki-67 should not occur in the same cell under physiological conditions. Simultaneous detection of tumour-suppressor protein p16 and a proliferation marker Ki-67 co- expression within the same cell should indicate deregulation of the cell cycle as the consequence of oncogenic transformation after long term infection induced by high-risk HPV.2,10 The presence of 1 or more cervical epithelial cell(s) showing p16/Ki-67 double immunoreactiv- ity is defined as a positive test result for p16/Ki- 67 DS cytology, independent from morphology interpretation.10 This study has been designed to evaluate the diagnostic accuracy of p16/Ki-67 DS for detection of high-grade cervical intraepithelial neoplasia (CIN2+) and the possible diagnostic role of the number of p16/Ki-67 positive cells. The goal was to determine whether taking a different num- ber of positive cells as the cut-off in the p16/Ki-67 DS test has a statistically significantly different re- sult in detection of CIN2+. Subjects and methods We performed the analysis on a subset cohort of women enrolled within a large-scale HPV self- sampling project within the organised, population- based Cervical Cancer Screening Programme ZORA in Slovenia that was conducted in 2013–2016 in two Slovenian regions.21 The project was approved by the National ethics committee (Approval Nos. 154/03/13, 136/04/14 and 102/11/15). All enrolled women with permanent residence in the Celje re- gion, who had p16/Ki-67 DS of the cervical smear and colposcopy in the Celje General Hospital re- gion were included in the analysis. Women were invited to colposcopy to Celje General Hospital either due to high-grade cytology or HPV-positive triage test after low-grade cytol- ogy or during follow-up after treatment of CIN2+ according to national cervical cancer screening guidelines or due to a positive HPV-self sampling result from an open label, multi-arm trial with a randomised design. A cervical smear was taken prior to the colposcopy. Conventional cytology with split sample technique was used. The first smear was stained with the standard Papanicolaou method and assessed according to national guide- lines (Bethesda classification). The second smear was stained with p16/Ki-67 DS (CINtec PLUS, Cytology CE; Ventana Medical Systems, Inc 2015, Tucson, Arizona USA) according to the manufac- turer’s instructions.22,23 All women underwent col- poscopy. In the case of an abnormal colposcopy result, a biopsy was taken, and the result was in- cluded in the analysis. If the patient had a nega- tive colposcopy, no biopsy was taken, and she was regarded as negative for CIN2+. All patients were managed according to the national guidelines.24 p16/Ki-67 DS was performed in the cytopa- thology laboratory of the Institute of Oncology Ljubljana and sent to the cytopathology laboratory of Celje General Hospital for assessment. All slides were blinded at the Institute of Oncology Ljubljana and independently assessed by a cytotechnolo- gist and cytopathologist in Celje General Hospital. Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears428 The cytopathologist’s result was included in the analysis. A positive reaction was defined as a p16 signal (brown) and a Ki-67 signal (red) present in the same cell with red stained nucleus and brown stained cytoplasm (Figure 1). One dual-stained cell was an indicator of a positive result.10 All evalu- ators recorded the number of positive or suspi- cious cells (one to five). A suspicious category was introduced to identify cases that were difficult to interpret. For the purpose of these analyses, sus- picious DS results were considered positive, and inadequate as negative.22,23 Number of p16/Ki-67 DS positive cells and CIN2+ according to Pap test results were calculat- ed. The diagnostic accuracy of p16/Ki-67 DS for the detection of CIN2+ was assessed with sensitivity (true positive rate), specificity (true negative rate), positive predictive value (PPV) and negative pre- dictive value (NPV). The association between the number of p16/Ki-67 positive cells and the detec- tion of CIN2+ was evaluated with Mann–Whitney U test. Statistical analysis was performed with R version 4.0.5. A p value of less than 0.05 was con- sidered statistically significant. Results Of 212 enrolled women from the Celje region, 38 were excluded due to the lack of p16/Ki-67 DS, leaving 174 women who had both p16/Ki-67 DS and colposcopy performed to be included in the analysis. The average age of women was 45.1 years. 73 women (42.0%) had a pathologic smear, and 101 women (58.0%) had a normal smear. The types of pathologic smears were high-grade intraepithelial lesion (HSIL) in 29 women (16.7%), ASC-US in 24 women (13.8%), LSIL in 14 women (8.0%), atypi- cal squamous cells-cannot exclude high-grade squamous intraepithelial lesion (ASC-H) in 4 women (2.3%), invasive squamous cell carcinoma in 1 woman (0.6%) and atypical glandular cells, not otherwise specified (AGC-N) in 1 woman (0.6%). The smear was interpreted as p16/Ki-67 DS posi- tive in 83 women (11 of which were originally eval- uated as suspicious) and negative in 91 (1 of which was initially inadequate). The analysis of p16/Ki-67 DS positivity among different smear results is pre- sented in Table 1. FIGURE 1. Positive reaction was defined as p16 brown signal and Ki-67 red signal (red arrow) present in the same cell with red stained nucleus and brown stained cytoplasm. Note: negative p16/Ki-67 dual immunostaining (p16/Ki-67 DS) reaction (black arrow) (p16/Ki-76 DS, magnification 400x). TABLE 1. p16/Ki-67 dual immunostaining (p16/Ki-67 DS) positivity and number of positive cells among different smear results Cervical cytology Number of p16/Ki-67 positive cells (n, [%]) Total0 (Negative) 1 2 3 4 ≥ 5 1+ (Total Positive) Normal 70 (69.3) 13 (12.9) 9 (8.9) 1 (1.0) 1 (1.0) 7 (6.9) 31 (30.7) 101 ASC-US 14 (58.3) 0 (0.0) 1 (4.2) 2 (8.3) 1 (4.2) 6 (25.0) 10 (41.7) 24 LSIL 7 (50.0) 3 (21.4) 0 (0.0) 0 (0.0) 1 (7.1) 3 (21.4) 7 (50.0) 14 AGC-N 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (100.0) 1 (100.0) 1 HSIL 0 (0.0) 1 (3.4) 0 (0.0) 0 (0.0) 0 (0.0) 28 (96.6) 29 (100.0) 29 ASC-H 0 (0.0) 0 (0.0) 0 (0.0) 1(25.0) 0 (0.0) 3 (75.0) 4 (100.0) 4 Inv. cancer 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 1(100.0) 1 (100.0) 1 Total 91 (52.3) 17 (9.8) 10 (5.7) 4 (2.3) 3 (1.7) 49 (28.2) 83 (47.7) 174 ASC-H = high-grade squamous intraepithelial lesion; AGC-N = atypical glandular cells, not otherwise specified; ASC-US = atypical squamous cells of undetermined significance; HSIL = high-grade intraepithelial lesion; LSIL = low-grade intraepithelial lesion Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears 429 Among the 83 women with a positive p16/Ki-67 DS result, 17 women (20.5%) had one positive cell, 10 women (12.0%) had two positive cells, 4 women (4.8%) had three positive cells, 3 women (3.6%) had four positive cells, and 49 women (59.0%) had at least five positive cells (Table 1). Among analysed women, 42/174 (24.1%) had histologically confirmed CIN2+, 92 women (52.9%) had CIN1 or normal histology and 40 (23.0%) wom- en had only colposcopy performed. Among the CIN2+ women, 37 (88.1%) had a p16/Ki-67 DS posi- tive smear, and among the women without CIN2+, 46 (34.8%) had a p16/Ki-67 DS positive smear. The analysis of the number of p16/Ki-67 DS pos- itive cells according to CIN2+ outcome is present- ed in Table 2 and Figure 2. Among the 91 women with negative p16/Ki-67 DS, 5 women (5.5%) had CIN2+. Among p16/Ki-67 DS positive women, the risk for CIN2+ was higher in those with more posi- tive cells (p < 0.001: one cell: 2/17 [11.8%], two cells: 0/10 [0.0%]; three cells: 1/4 [25.0%]; four cells: 1/3 [33.3%], five or more cells: 33/49 [67.3%]). The diagnostic accuracy of p16/Ki-67 DS for the detection of CIN2+ is presented in Table 3. For the total population, sensitivity was 88.1% (50% for women with ASC-US or LSIL), specificity was 65.2% (61.1% for women with ASC-US and 50% for LSIL), PPV was 44.6% and NPV was 94.5%. Discussion We evaluated the diagnostic accuracy of p16/Ki-67 DS to detect CIN2+ at different cut-offs defined by the number of positive cells. Our analysis showed 88.1% sensitivity of p16/ Ki-67 DS for the detection of CIN2+, which is comparable to several other studies that reported sensitivity between 86.4 and 98.2%3,10,12,25-28 and somewhat higher than some other reported data, including previous data from our group where were analysed postmenopausal women with low- grade cytology.9,29,30 Our group also reported that additional training contributes to higher sensitiv- ity of p16/Ki-67 DS for detecting CIN 2+ without a decrease in specificity.22,23 Additional analyses showed only 50% sensitivity in women with LSIL, which might reflect the low number of enrolled pa- tients (95% CI: 1.3–98.7%). Other authors reported p16/Ki-67 DS as an effective triage of patients with LSIL.2,10,26 Peeters et al. reported in a meta-analysis that sensitivity of p16/Ki-67 DS for detection of CIN 2+ in triaging women with ASC-US and LSIL was similar - 84% (95% CI: 77–89%) and 86% (95% CI: 82–89%) - than that of the HPV test - 93% in ASC-US (95% CI: 91–95%) and 95% (95% CI: 94– 96%) in LSIL. Specificity of p16/Ki-67 DS for detec- tion of CIN 2+ in ASC-US and LSIL were 77% (95% CI: 70–77%) and 66% (95% CI: 59–72%). In contrast, the HPV test was less specific: in ASC-US 45% (95% CI: 38–53%) and LSIL 27% (95% CI: 23–33%), re- spectively.28 In cases of ASC-US and LSIL, the rec- ommended subsequent follow-up strategy is HPV triage. However, this strategy has its limitations because of the high HPV positivity in women with low-grade cytology.28,31 According to Frega et al. the sensitivity and specificity in the ASC-US group were high for CIN 2 (90.09% CI: 89.4–92.4%; 81.8% CI: 74.2–89.4) and CIN 3 (99.9% CI: 92.2–99.9%; FIGURE 2. The association between the number of p16/Ki-67 dual immunostaining (p16/Ki-67 DS) positive cells and the risk for cervical intraepithelial neoplasia (CIN2+). Observed values are marked as points. Smoothed line (Method spline) is added for better trend representation. TABLE 2. Cervical intraepithelial neoplasia (CIN2+) according to the number of p16/Ki-67 dual immunostaining (p16/Ki-67 DS) positive cells p16/Ki-67 Histology Positive cells n < CIN2 n (%) CIN2+ n (%) 0 91 86 (94.5) 5 (5.5) 1 17 15 (88.2) 2 (11.8) 2 10 10 (100.0) 0 (0.0) 3 4 3 (75.0) 1 (25.0) 4 3 2 (66.7) 1 (33.3) ≥ 5 49 16 (32.7) 33 (67.3) Total 174 132 (75.9) 42 (24.1) Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears430 73.7% CI: 65.0–82.4%). In LSIL group the sensitiv- ity was 95.2% for CIN 2 (CI: 88.7–99.9%) and 94.1% for CIN 3 (CI: 82.9–99.9%), however specificity was only 61.8% for CIN 2 (CI: 54.4–69.2%) and 49% for FIGURE 3. Diagnostic performance of p16/Ki-67 dual immunostaining (p16/Ki-67 DS) at different cut-offs (number of positive cells). TABLE 3. Diagnostic performance of p16/Ki-67 dual immunostaining (p16/Ki-67 DS) according to cytology results and according to different cut-offs (number of positive cells) in detecting cervical intraepithelial neoplasia (CIN2+) CIN2+(n) p16/Ki-67 Sensitivity (%, 95% CI) Specificity (%, 95% CI) PPV† (%, 95% CI) NPV‡ (%, 95% CI) Cytology result Negative (n = 101) 3 66.7 (9.4–99.2) 70.4 (60.3–79.2) 6.5 (0.8–21.4) 98.6 (92.3–100.0) ASC-US (n = 24) 6 50.0 (11.8–88.2) 61.1 (35.7–82.7) 30.0 (6.7–65.2) 78.6 (49.2–95.3) LSIL (n = 14) 2 50.0 (1.3–98.7) 50.0 (21.1–78.9) 14.3 (0.4–57.9) 85.7 (42.1–99.6) HSIL (n = 29) 26 100.0 (86.8–100.0) 0.0 (0.0–70.8) 89.7 (72.6–97.8) / Number of positive p16/Ki-67 cells cut-off 1+ (n = 174) 42 88.1 (74.4–96.0) 65.2 (56.4–73.2) 44.6 (33.7–55.9) 94.5 (87.6–98.2) 2+ (n = 174) 42 83.3 (68.6–93.0) 76.5 (68.4–83.5) 53.0 (40.3–65.4) 93.5 (87.1–97.4) 3+ (n = 174) 42 83.3 (68.6–93.0) 84.1 (76.7–89.9) 62.5 (48.5–75.1) 94.1 (88.2–97.6) 4+ (n = 174) 42 81.0 (65.9–91.4) 86.4 (79.3–91.7) 65.4 (50.9–78.0) 93.4 (87.5–97.1) 5+ (n = 174) 42 78.6 (63.2–89.7) 87.9 (81.1–92.9) 67.3 (52.5–80.1) 92.8 (86.8–96.7) †positive predictive value; ‡negative predictive value; ASC-US = atypical squamous cells of undetermined significance; HSIL = high-grade intraepithelial lesion; LSIL = low-grade intraepithelial lesion; NPV = negative predictive value; PPV = positive predictive value CIN 3 (CI: 41.4–56.6%), respectively. In contrast, the HPV test was more sensitive in all groups but far less specific (17.5% [CI: 2.2–32.8%] – 29.7% [CI: 22.7-36.7%]) in their study group of young women aged 21–24 years.32 It has been reported that by combining high sensitivity and specificity, p16/ Ki-67 DS could decrease referrals to colposcopy by 50% in women with ASC-US and LSIL.8,10,33-35 Previous studies in women older than 30 years have shown statistically significantly higher sensi- tivity of p16/Ki-67 DS compared to Pap cytology. However, HPV was statistically significantly more sensitive than dual-stained cytology (93.3% vs. 84.7%; P = 0.03), but statistically significantly less specific (93.0% vs. 96.2%; P < 0.001).12 In our study, the specificity of p16/Ki-67 DS was 65.2%, while the specificity in ASC-US was 61.1% and the specificity in LSIL was 50.0%. Triage stud- ies reported similar results.2,25,26,33,35,36 Schmidt et al. reported specificity of 80.6% for the detection of CIN2+ in the ASC-US group and 68.0% in the LSIL group, respectively.10 Danish researchers reported 51.3% specificity of p16/Ki56 DS for the detection of CIN2+ and 48.2% for the detection of CIN 3+.25 In other studies, the reported specificity for the de- tection of CIN2+ were 59.5% (Wentzensen), 60,0% (Luttmer), 61.9% (Killeen), 82.5% (Zhu), and 95,2% (Ikenberg).2,3,12,26,37 Studies involved different popu- lations, which is the reason for the range of spe- cificities reported for the p16/Ki-67 DS test results. Wentzensen and Killeen have similar studies of women referred to colposcopy, Luttmer enrolled HPV-positive women referred to colposcopy, Zhu enrolled only women with ASC-US cytological di- agnosis, and Ikenberg involved women 18 years or older undergoing routine cytology-based cervical cancer screening. PPV and NPV for the detection of CIN2+ in our study were 44.6% and 94.5%, respectively. Killeen et al. reported in a group of women with abnormal Pap smear PPV and NPV of 30.6% and 98.4%, re- spectively.2 Waldstrom et al. reported 29.3% PPV and 95.2% NPV for p16/Ki-67 DS LSIL smear for the detection of CIN2+.25 Zhu Y. et al. reported 55.2% PPV and 99.25% NPV for p16/Ki-67 DS ASC- US smear for detection of CIN2+.3 The major limitation of our study is the small number of participants. Only one positive cell is required for a positive result of the p16/Ki-67 DS.10 Ziemke reported in his study that using a score of 10 p16/Ki-76 DS positive cells as a positive result instead of one led to signifi- cantly higher specificity (89.0 vs. 70.2%, p < 0.001) and that this threshold offers better risk assessment Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears 431 in LISL.38 In our study, we report the association of the number of p16/Ki-67 DS positive cells with the detection of CIN2+ that could be used to im- prove real-time diagnostic performance without long-term data. We have investigated the threshold of the number of positive cells where we achieve a statistically significant better specificity but do not lose the sensitivity of the test. We have shown that women with a positive p16/Ki-67 DS result have a significantly higher risk for CIN2+ when the num- ber of p16/Ki-67 DS positive cells is increasing. The probability of detecting a CIN2+ result in a patient with five or more p16/Ki-67 DS positive cells was 67.3% compared to only 11.8% in a patient with on- ly one positive cell. A few longitudinal studies ex- ist that are not directly comparable with ours since they are concerned with long-term cumulative risk rather than current diagnostic implications. They investigated the long-term predictive value of p16/ Ki-67 DS cytology and explored additional assess- ments using different numbers of dual stained positive cells as a cut-off for a positive test result. The cumulative risk of CIN2+ increased with the increasing number of positive dual-stained cells.39 A similar result was observed by Uijterwaal et al. in the study of triaging HPV-positive women with normal cytology by p16/Ki-67 DS cytology testing.40 We have observed a statistically significant in- crease in p16/Ki-67 DS specificity at the cut-off for p16/Ki-67 DS positivity at 3 cells compared to 1 cell, with statistically insignificant decrease in sensitiv- ity (Figure 3). This finding opens a new research question, whether changing the cut-off in p16/Ki- 67 DS test could improve performance of p16/Ki-67 DS triage in terms of a further increase in specific- ity, which would lower the colposcopy referrals even further without a significant loss in longitudi- nal sensitivity and NPV, which would still enable a safe prolongation of follow-up intervals. References 1. Maver PJ, Poljak M. Primary HPV-based cervical cancer screening in Europe: implementation status, challenges, and future plans. Clin Microbiol Infect 2020; 26: 579-5. 83. doi: 10.1016/j.cmi.2019.09.006 2. Killeen JL, Dye T, Grace C, Hiraoka M. Improved abnormal Pap smear triage using cervical cancer biomarkers. J Low Genit Tract Dis 2014; 18: 1-7. doi: 10.1097/LGT.0b013e31828aeb39 3. Zhu Y, Ren C, Yang L, Zhang X, Liu L, Wang Z. Performance of p16/Ki-67 immunostaining, HPV E6/E7 mRNA testing, and HPV DNA assay to detect high-grade cervical dysplasia in women with ASCUS. BMC Cancer 2019; 19: 271. doi: 10.1186/s12885-019-5492-9 4. Sawaya GF, Smith-McCune K, Kuppermann M. Cervical cancer screen- ing: more choices in 2019. JAMA 2019; 321: 2018-9. doi: 10.1001/ jama.2019.4595 5. White CM, Bakhiet S, Bates M, Ruttle C, Pilkington LJ, Keegan H, et al; CERVIVA consortium. Exposure to tobacco smoke measured by urinary nico- tine metabolites increases risk of p16/Ki-67 co-expression and high-grade cervical neoplasia in HPV positive women: A two year prospective study. Cancer Epidemiol 2020; 68: 101793. doi: 10.1016/j.canep.2020.101793 6. Gavrić Lovrec V, Erlah T, Dobnik S, Takač I. The influence of smoking on the frequency and characteristics of complications following large loop excision of the transformation zone (LLETZ). Acta medico-biotechnica 2019; 1: 40-6. 7. Arbyn M, Ronco G, Anttila A, Meijer CJ, Poljak M, Ogilvie G, et al. Evidence regarding human papillomavirus testing in secondary preven- tion of cervical cancer. Vaccine 2012; 30(Suppl): F88-99. doi: 10.1016/j. vaccine.2012.06.095. Erratum in: Vaccine 2013; 31: 6266. doi: 10.1016/j. vaccine.2012.06.095 8. Petry KU, Schmidt D, Scherbring S, Luyten A, Reinecke-Lüthge A, Bergeron C, et al. Triaging Pap cytology negative, HPV positive cervical cancer screening results with p16/Ki-67 dual-stained cytology. Gynecol Oncol 2011; 121: 505- 9. doi: 10.1016/j.ygyno.2011.02.033 9. Wright TC Jr, Behrens CM, Ranger-Moore J, Rehm S, Sharma A, Stoler MH, et al. Triaging HPV-positive women with p16/Ki-67 dual-stained cytology: results from a sub-study nested into the ATHENA trial. Gynecol Oncol 2017; 144: 51-6. doi: 10.1016/j.ygyno 10. Schmidt D, Bergeron C, Denton KJ, Ridder R; European CINtec Cytology Study Group. p16/Ki-67 dual-stain cytology in the triage of ASCUS and LSIL papanicolaou cytology: results from the European equivocal or mildly ab- normal Papanicolaou cytology study. Cancer Cytopathol 2011; 119: 158-66. doi: 10.1002/cncy.20140 11. Bergeron C, Ikenberg H, Sideri M, Denton K, Bogers J, Schmidt D, et al; PALMS Study Group. Prospective evaluation of p16/Ki-67 dual-stained cytology for managing women with abnormal Papanicolaou cytology: PALMS study results. Cancer Cytopathol 2015; 123: 373-81. doi: 10.1002/ cncy.21542 12. Ikenberg H, Bergeron C, Schmidt D, Griesser H, Alameda F, Angeloni C, et al.; PALMS Study Group. Screening for cervical cancer precursors with p16/ Ki-67 dual-stained cytology: results of the PALMS study. J Natl Cancer Inst 2013; 105: 1550-7. doi: 10.1093/jnci/djt235 13. Tjalma WAA. Diagnostic performance of dual-staining cytology for cervi- cal cancer screening: a systematic literature review. Eur J Obstet Gynecol Reprod Biol 2017; 210: 275-80. doi: 10.1016/j.ejogrb 14. Khleif SN, DeGregori J, Yee CL, Otterson GA, Kaye FJ, Nevins JR, et al. Inhibition of cyclin D-CDK4/CDK6 activity is associated with an E2F- mediated induction of cyclin kinase inhibitor activity. Proc Natl Acad Sci U S A 1996; 93: 4350-4. doi: 10.1073/pnas.93.9.4350 15. Wentzensen N, von Knebel Doeberitz M. Biomarkers in cervical cancer screening. Dis Markers 2007; 23: 315-30. doi: 10.1155/2007/678793. doi: 10.1155/2007/678793 16. Sano T, Oyama T, Kashiwabara K, Fukuda T, Nakajima T. Expression status of p16 protein is associated with human papillomavirus oncogenic potential in cervical and genital lesions. Am J Pathol 1998; 153: 1741-8. doi: 10.1016/ S0002-9440(10)65689-1 17. Klaes R, Friedrich T, Spitkovsky D, Ridder R, Rudy W, Petry U, et al. Overexpression of p16(INK4A) as a specific marker for dysplastic and neo- plastic epithelial cells of the cervix uteri. Int J Cancer 2001; 92: 276-84. doi: 10.1002/ijc.1174 18. Wang HR, Li YC, Guo HQ, Yu LL, Wu Z, Yin J, et al. A cocktail of p16INK4a and Ki-67, p16INK4a and minichromosome maintenance protein 2 as triage tests for human papillomavirus primary cervical cancer screening. Oncotarget 2017; 8: 83890-9. doi: 10.18632/oncotarget.19870 19. Schlüter C, Duchrow M, Wohlenberg C, Becker MH, Key G, Flad HD, et al. The cell proliferation-associated antigen of antibody Ki-67: a very large, ubiquitous nuclear protein with numerous repeated elements, representing a new kind of cell cycle-maintaining proteins. J Cell Biol 1993; 123: 513-22. doi: 10.1083/jcb.123.3.513 20. Yu L, Fei L, Liu X, Pi X, Wang L, Chen S. Application of p16/Ki-67 dual-staining cytology in cervical cancers. J Cancer 2019; 10: 2654-60. doi: 10.7150/ jca.32743 21. Ivanus U, Jerman T, Fokter Repse A, Takac I, Prevodnik VK, Marcec M, et al. Randomised trial of HPV self-sampling among non-attenders in the Slovenian cervical screening programme ZORA: comparing three differ- ent screening approaches. Radiol Oncol 2018; 52: 399-412. doi: 10.2478/ raon-2018-0036 Radiol Oncol 2021; 55(4): 426-432. Salobir Gajsek U et al. / p16/Ki-67 DS in cervical smears432 22. Kloboves Prevodnik V, Jerman T, Nolde N, Repše Fokter A, Jezeršek S, Pohar Marinšek Ž, et al. Interobserver variability and accuracy of p16/Ki-67 dual immunocytochemical staining on conventional cervical smears. Diagn Pathol 2019; 1: 48. doi: 10.1186/s13000-019-0821-5 23. Prevodnik VK, Marinsek ZP, Zalar J, Rozina H, Kotnik N, Jerman T, et al. Evaluation of the training program for p16/Ki-67 dual immunocytochemical staining interpretation for laboratory staff without experience in cervical cytology and immunocytochemistry. Radiol Oncol 2020; 2: 201-8. doi: 10.2478/raon-2020-0018 24. Guidelines for management of women with cervical precancerous lesions. Ursic-Vrscaj M, Rakar S, Možina A, Kobal B, Takač I, Deisinger I, et al, edi- tors. [Slovenian]. Ljubljana: Institute of Oncology Ljubljana; 2011. [Accessed 2021 Feb 5]. Available at: https://zora.onko-i.si/fileadmin/user_upload/ dokumenti/strokovna_priporocila/2011_Smernice_web.pdf 25. Waldstrøm M, Christensen RK, Ørnskov D. Evaluation of p16(INK4a)/Ki-67 dual stain in comparison with an mRNA human papillomavirus test on liquid-based cytology samples with low-grade squamous intraepithelial le- sion. Cancer Cytopathol 2013; 121: 136-45. doi: 10.1002/cncy.21233 26. Wentzensen N, Schwartz L, Zuna RE, Smith K, Mathews C, Gold M A, et al. Performance of p16/Ki-67 immunostaining to detect cervical cancer precur- sors in a colposcopy referral population. Clin Cancer Res 2012; 18: 4154-62. doi: 10.1158/1078-0432.CCR-12-0270 27. Šekoranja D, Repše Fokter A. Triaging atypical squamous cells-cannot ex- clude high-grade squamous intraepithelial lesion with p16/Ki-67 dual stain. J Low Genit Tract Dis 2017; 21: 108-11. doi: 10.1097/LGT.0000000000000297 28. Peeters E, Wentzensen N, Bergeron C, Arbyn M. Meta-analysis of the ac- curacy of p16 or p16/Ki-67 immunocytochemistry versus HPV testing for the detection of CIN2+/CIN3+ in triage of women with minor abnormal cytology. Cancer Cytopathol 2019; 127: 169-80. doi: 10.1002/cncy.22103 29. Edgerton N, Cohen C, Siddiqui MT. Evaluation of CINtec PLUS® testing as an adjunctive test in ASC-US diagnosed SurePath® preparations. Diagn Cytopathol 2013; 41: 35-40. doi: 10.1002/dc.21757 30. Dovnik A, Repše Fokter A. p16/Ki-67 Immunostaining in the triage of post- menopausal women with low-grade cytology results. J Low Genit Tract Dis 2020; 24: 235-7. doi: 10.1097/LGT.0000000000000539 31. Dovnik A, Repše-Fokter A. p16/Ki-67 immunostaining in the triage of young women with LSIL, ASC-US, and ASC-H cytology. Diagn Cytopathol 2020; 48: 96-7. doi: 10.1002/dc.24339 32. Frega A, Pavone M, Sesti F, Leone C, Bianchi P, Cozza G, et al. Sensitivity and specificity values of high-risk HPV DNA, p16/Ki-67 and HPV mRNA in young women with atypical squamous cells of undetermined significance (ASCUS) or low-grade squamous intraepithelial lesion (LSIL). Eur Rev Med Pharmacol Sci 2019; 24: 10672-7. doi: 10.26355/eurrev-201912-19765 33. Wentzensen N, Clarke MA, Bremer R, Poitras N, Tokugawa D, Goldhoff PE, et al. Clinical evaluation of human papillomavirus screening with p16/Ki-67 du- al stain tri-age in a large organized cervical cancer screening program. JAMA Intern Med 2019; 179: 881-8. doi: 10.1001/jamainternmed.2019.0306 34. Stoler MH, Baker E, Boyle S, Aslam S, Ridder R, Huh WK, et al. Approaches to triage optimization in HPV primary screening: Extended genotyping and p16/Ki-67 dual-stained cytology-Retrospective insights from ATHENA. Jr. Int J Cancer 2020; 146: 599-607. doi: 10.1002/ijc.32669 35. Giorgi Rossi P, Carozzi F, Ronco G, Allia E, Bisanzi S, Gillio-Tos A, et al; the New Technology for Cervical Cancer 2 Working Group. p16/Ki-67 and E6/E7 mRNA accuracy and prognostic value in triaging HPV DNA-positive women. J Natl Cancer Inst. 2021; 3: 292-300. doi: 10.1093/jnci/djaa105 36. Han Q, Guo H, Geng L, Wang Y. p16/Ki-67 dual-stained cytology used for triage in cervical cancer opportunistic screening. Chin J Cancer Res 2020; 2: 208-17. doi: 10.21147/j.issn.1000-9604.2020.02.08 37. Luttmer R, Dijkstra MG, Snijders PJF, Berkhof J, van Kemenade FJ, Rozendaal L, et al. p16/Ki-67 dual-stained cytology for detecting cervical (pre)cancer in a HPV-positive gynecologic outpatient population. Mod Pathol 2016; 29: 870-8. doi: 10.1038/modpathol.2016.80 38. Ziemke P. p16/Ki-67 Immunocytochemistry in gynecological cytology: limita- tions in practice. Acta Cytologica 2017; 61: 230-6. doi: 10.1159/000475979 39. Clarke MA, Cheung LC, Castle PE, Schiffman M, Tokugawa D, Poitras N, et al. Five-year risk of cervical precancer following p16/Ki-67 dual-stain triage of HPV-positive women. JAMA Oncol 2019; 5: 181-6. doi:10.1001/ jamaoncol.2018.4270 40. Uijterwaal MH, Polman NJ, Witte BI, van Kemenade FJ, Rijkaart D, Berkhof J, et al. Triaging HPV-positive women with normal cytology by p16/Ki-67 dual- stained cytology testing: baseline and longitudinal data. J Cancer 2015; 136: 2361-8. doi: 10.1002/ijc.29290