Reports of Major Impact
Gynecology
Predicting the risk of malignancy in adnexal masses based on the Simple Rules from the International Ovarian Tumor Analysis group

https://doi.org/10.1016/j.ajog.2016.01.007Get rights and content

Background

Accurate methods to preoperatively characterize adnexal tumors are pivotal for optimal patient management. A recent metaanalysis concluded that the International Ovarian Tumor Analysis algorithms such as the Simple Rules are the best approaches to preoperatively classify adnexal masses as benign or malignant.

Objective

We sought to develop and validate a model to predict the risk of malignancy in adnexal masses using the ultrasound features in the Simple Rules.

Study Design

This was an international cross-sectional cohort study involving 22 oncology centers, referral centers for ultrasonography, and general hospitals. We included consecutive patients with an adnexal tumor who underwent a standardized transvaginal ultrasound examination and were selected for surgery. Data on 5020 patients were recorded in 3 phases from 2002 through 2012. The 5 Simple Rules features indicative of a benign tumor (B-features) and the 5 features indicative of malignancy (M-features) are based on the presence of ascites, tumor morphology, and degree of vascularity at ultrasonography. Gold standard was the histopathologic diagnosis of the adnexal mass (pathologist blinded to ultrasound findings). Logistic regression analysis was used to estimate the risk of malignancy based on the 10 ultrasound features and type of center. The diagnostic performance was evaluated by area under the receiver operating characteristic curve, sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR–), positive predictive value (PPV), negative predictive value (NPV), and calibration curves.

Results

Data on 4848 patients were analyzed. The malignancy rate was 43% (1402/3263) in oncology centers and 17% (263/1585) in other centers. The area under the receiver operating characteristic curve on validation data was very similar in oncology centers (0.917; 95% confidence interval, 0.901–0.931) and other centers (0.916; 95% confidence interval, 0.873–0.945). Risk estimates showed good calibration. In all, 23% of patients in the validation data set had a very low estimated risk (<1%) and 48% had a high estimated risk (≥30%). For the 1% risk cutoff, sensitivity was 99.7%, specificity 33.7%, LR+ 1.5, LR– 0.010, PPV 44.8%, and NPV 98.9%. For the 30% risk cutoff, sensitivity was 89.0%, specificity 84.7%, LR+ 5.8, LR– 0.13, PPV 75.4%, and NPV 93.9%.

Conclusion

Quantification of the risk of malignancy based on the Simple Rules has good diagnostic performance both in oncology centers and other centers. A simple classification based on these risk estimates may form the basis of a clinical management system. Patients with a high risk may benefit from surgery by a gynecological oncologist, while patients with a lower risk may be managed locally.

Introduction

Ovarian cancer is a common and lethal disease for which early detection and treatment in high-volume centers and by specialized clinicians is known to improve survival.1, 2, 3, 4 Hence, accurate methods to preoperatively characterize the nature of an ovarian tumor are pivotal. In 2008 the International Ovarian Tumor Analysis (IOTA) group described the Simple Rules.5 These are based on a set of 5 ultrasound features indicative of a benign tumor (B-features) and 5 ultrasound features indicative of a malignant tumor (M-features). When using the Simple Rules, tumors are classified as benign if only B-features are observed and as malignant if only M-features are observed. If no features are observed or if conflicting features are present, the Simple Rules cannot classify the tumor as benign or malignant (inconclusive results). Masses in which the Simple Rules yield an inconclusive result can be classified using subjective assessment by an experienced ultrasound operator or, given the high prevalence of malignancy in this group, they can all be classified as malignant to increase the sensitivity for ovarian cancer.6 On prospective validation both by the IOTA group (2 studies including 1938 and 2403 patients, respectively)7, 8 and by other research teams (9 studies including a total of 2101 tumors),9, 10, 11, 12, 13, 14, 15, 16, 17 the Simple Rules were applicable in 77-94% of tumors (range between studies). The malignancy rate ranged from 1-9% in cases classified as benign, from 69-94% in cases classified as malignant, and from 13-53% in inconclusive cases. In a metaanalysis comparing the ability of 19 methods to discriminate between benign and malignant adnexal masses before surgery, the Simple Rules had a sensitivity of 93% and a specificity of 81% when classifying inconclusive tumors as malignant.18 In the metaanalysis the Simple Rules and the IOTA logistic regression model 219 were superior to all other methods. This suggests that evidence-based approaches to the preoperative characterization of adnexal masses should incorporate the use of Simple Rules or the logistic regression model 2. Logistic regression model 2 is a mathematical risk prediction model based on age and 5 ultrasound variables (presence of blood flow in a papillary structure, irregular cyst walls, ascites, acoustic shadows, and maximum diameter of the largest solid component).

The Simple Rules have been well received by clinicians, and the Royal College of Obstetricians and Gynecologists in the United Kingdom has included the Simple Rules in their Green Top guideline on the assessment and management of ovarian masses in premenopausal women.20

Despite a combination of simplicity and excellent performance, important limitations of the Simple Rules are the inconclusive results in a proportion of cases and the absence of an estimated risk of malignancy. The ability to provide accurate risk estimates is highly relevant for risk stratification and individualized patient management. The objective of this study was to develop and validate a model to calculate the risk of malignancy in adnexal masses based on the 10 ultrasound features in the Simple Rules.

Section snippets

Study design and setting

This international multicenter cross-sectional cohort study involves patients from 22 centers (oncology centers and other hospitals) (Table 1) with at least 1 adnexal (ovarian, paraovarian, or tubal) tumor selected for surgery by the managing clinician. Exclusion criteria were: (1) pregnancy at the time of examination, (2) refusal of transvaginal ultrasonography, (3) declining participation, and (4) surgical intervention >120 days after the ultrasound examination. Data collection was carried

Results

During IOTA phases 1b, 2, and 3, data on 5020 patients were recorded at 22 centers (2 centers from IOTA phase 1 did not take part in later phases). Data on 172 patients were excluded because the patients fulfilled an exclusion criterion (n = 124; 43 women were pregnant and 81 women were operated on >120 days after the ultrasound examination), data errors or uncertain/missing final histology (n = 47), or protocol violation (n = 1). This leaves data on 4848 patients (Tables 1, 2, and 3). The

Principal findings of the study

In this study we have developed a method to estimate the individual risk of malignancy in an adnexal mass using the ultrasound features in the IOTA Simple Rules. On prospective validation the risk estimates showed good ability to discriminate between benign and malignant tumors (AUC 0.917) and good agreement between the calculated risks of malignancy and the true prevalence of malignancy.

Implications of the work

The Simple Rules are intuitively attractive because of their ease of use.9, 10, 11, 12, 13, 14, 15, 16, 17,

References (40)

  • D. Timmerman et al.

    Simple ultrasound-based rules for the diagnosis of ovarian cancer

    Ultrasound Obstet Gynecol

    (2008)
  • J. Kaijser et al.

    Improving strategies for diagnosing ovarian cancer: a summary of the International Ovarian Tumor Analysis (IOTA) studies

    Ultrasound Obstet Gynecol

    (2013)
  • D. Timmerman et al.

    Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group

    BMJ

    (2010)
  • A.C. Testa et al.

    Strategies to diagnose ovarian cancer: new evidence from phase 3 of the multicenter international IOTA study

    Br J Cancer

    (2014)
  • C.A. Hartman et al.

    Ultrasound criteria and CA 125 as predictive variables of ovarian cancer in women with adnexal tumors

    Ultrasound Obstet Gynecol

    (2012)
  • J.L. Alcázar et al.

    IOTA simple rules for discriminating between benign and malignant adnexal masses: prospective external validation

    Ultrasound Obstet Gynecol

    (2013)
  • A. Sayasneh et al.

    Multicenter external validation of IOTA prediction models and RMI by operators with varied training

    Br J Cancer

    (2013)
  • C. Tantipalakorn et al.

    IOTA simple rules in differentiating between benign and malignant ovarian tumors

    Asian Pac J Cancer Prev

    (2014)
  • N. Nunes et al.

    Use of IOTA simple rules for diagnosis of ovarian cancer: meta-analysis

    Ultrasound Obstet Gynecol

    (2014)
  • D. Tinnangwattana et al.

    IOTA simple rules in differentiating between benign and malignant adnexal masses by non-expert examiners

    Asian Pac J Cancer Prev

    (2015)
  • Cited by (0)

    This study was supported by the Flemish government [Research Foundation–Flanders (FWO) project G049312N, Flanders’ Agency for Innovation by Science and Technology (IWT) project IWT-Translational Biomedical Research 070706-International Ovarian Tumor Analysis phase 3, and iMinds 2015] and Internal Funds KU Leuven (project C24/15/037). LW is a doctoral fellow of IWT. DT is a senior clinical investigator of FWO. TB is supported by the National Institute for Health Research (NIHR) Biomedical Research Center based at Imperial College Healthcare National Health Service (NHS) Trust and Imperial College London. The views expressed are those of the authors and not necessarily those of the NHS, NIHR, or Department of Health. LV is supported by the Swedish Medical Research Council (grants K2001-72X-11605-06A, K2002-72X-11605-07B, K2004-73X-11605-09A, and K2006-73X-11605-11-3), funds administered by Malmö University Hospital and Skåne University Hospital, Allmänna Sjukhusets i Malmö Stiftelse för bekämpande av cancer (the Malmö General Hospital Foundation for fighting against cancer), and 2 Swedish governmental grants (ALF-medel and Landstingsfinansierad Regional Forskning). The sponsors had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the work for publication. The researchers performed this work independently of the funding sources.

    All authors declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

    1

    Drs Timmerman and Van Calster are joint first authors.

    View full text