Review
Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis

https://doi.org/10.1016/j.ejca.2016.01.007Get rights and content

Highlights

  • •

    This is the first meta-analysis on subjective assessment to classify adnexal masses.

  • •

    Subjective assessment performed best of all methods under investigation.

  • •

    All methods yielded superior test accuracy when compared to the currently used risk of malignancy index.

  • •

    Simple rules added with subjective assessment matched test performance of experts.

  • •

    The logistic regression model 2 model can be used as an alternative if an expert is not available.

Abstract

Introduction

Many national guidelines concerning the management of ovarian cancer currently advocate the risk of malignancy index (RMI) to characterise ovarian pathology. However, other methods, such as subjective assessment, International Ovarian Tumour Analysis (IOTA) simple ultrasound-based rules (simple rules) and IOTA logistic regression model 2 (LR2) seem to be superior to the RMI.

Our objective was to compare the diagnostic accuracy of subjective assessment, simple rules, LR2 and RMI for differentiating benign from malignant adnexal masses prior to surgery.

Materials and methods

MEDLINE, EMBASE and CENTRAL were searched (January 1990–August 2015). Eligibility criteria were prospective diagnostic studies designed to preoperatively predict ovarian cancer in women with an adnexal mass.

Results

We analysed 47 articles, enrolling 19,674 adnexal tumours; 13,953 (70.9%) benign and 5721 (29.1%) malignant. Subjective assessment by experts performed best with a pooled sensitivity of 0.93 (95% confidence interval [CI] 0.92–0.95) and specificity of 0.89 (95% CI 0.86–0.92). Simple rules (classifying inconclusives as malignant) (sensitivity 0.93 [95% CI 0.91–0.95] and specificity 0.80 [95% CI 0.77–0.82]) and LR2 (sensitivity 0.93 [95% CI 0.89–0.95] and specificity 0.84 [95% CI 0.78–0.89]) outperformed RMI (sensitivity 0.75 [95% CI 0.72–0.79], specificity 0.92 [95% CI 0.88–0.94]). A two-step strategy using simple rules, when inconclusive added by subjective assessment, matched test performance of subjective assessment by expert examiners (sensitivity 0.91 [95% CI 0.89–0.93] and specificity 0.91 [95% CI 0.87–0.94]).

Conclusions

A two-step strategy of simple rules with subjective assessment for inconclusive tumours yielded best results and matched test performance of expert ultrasound examiners. The LR2 model can be used as an alternative if an expert is not available.

Introduction

In order to ensure that ovarian cancer patients receive appropriate treatment, an accurate characterisation of any adnexal mass that needs surgery is pivotal to improve the outcome of this disease. Subjective assessment by experienced examiners, also called ‘pattern recognition’, is generally accepted to be the best way to classify adnexal masses prior to surgery. Several individual reports have demonstrated that subjective assessment is superior to the use of scoring systems and mathematical models, such as International Ovarian Tumour Analysis (IOTA) simple ultrasound-based rules (simple rules), IOTA logistic regression model 2 (LR2) or the risk of malignancy index (RMI) [1], [2], [3], [4]. However, both LR2 and simple rules closely approximate the performance of subjective assessment by expert examiners [5], [6]. An advantage of these models over subjective assessment is their objectivity and simplicity which facilitates their use by ultrasonographers with different backgrounds and various levels of experience [7], [8], [9], [10]. Despite accumulating and compelling evidence in favour of both subjective assessment and the ultrasound-based models such as simple rules and LR2, many national guidelines concerning the management of ovarian masses still advocate the use of RMI in the classification of adnexal masses. Consequently, the RMI is still the most commonly used model in clinical practice.

Several reviews have critically appraised the evidence relating to this subject [5], [6], [11], [12], [13], [14], [15], [16]. However, none of these has provided a meta-analysis on the test performance of subjective assessment of adnexal tumours, while in general this method is considered the most accurate way to distinguish benign from malignant adnexal tumours. The aim of this meta-analysis was to compare the diagnostic accuracy of subjective assessment, simple rules, LR2 and RMI for the pre-operative differentiation of benign and malignant adnexal masses.

Section snippets

Protocol and registration

All methods described in this manuscript were determined in advance and recorded in a study protocol (Prospero CRD42013004334, http://www.crd.york.ac.uk/PROSPERO). The conduct of this systematic review and meta-analysis was done in accordance with prevailing guidelines (http://www.prisma-statement.org and http://srdta.cochrane.org/handbook-dta-reviews).-

Eligibility criteria

Eligible studies had to evaluate diagnostic accuracy of subjective assessment, simple rules, LR2 and/or RMI for the characterisation of adnexal

Study selection and characteristics

In total 60 studies were included in the qualitative data synthesis (Fig. 1) [1], [6], [7], [8], [18], [19], [20], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75]. Of these, 47 were valid for quantitative data synthesis (meta-analysis),

Main results

This review and meta-analysis summarises the evidence currently available on the diagnostic accuracy of different pre-operative ultrasound methods for differentiating benign from malignant adnexal masses. According to this systematic review and meta-analysis, we believe an evidence-based approach should incorporate either simple rules with referral for subjective assessment of ultrasound findings by expert examiners if the rules are not applicable, or alternatively the LR2 model if such

Implications for research

Side-by-side comparison studies in which all methods are validated in the same population should be performed in order to prove which method demonstrates the best diagnostic performance. Furthermore, studies should employ strict blinding of the ultrasonographer to the outcome (histology) and vice versa.

More research should be performed on new methods, such as the ADNEX-model, or combinations of current methods.

Implications for practice

Evidence of this meta-analysis shows that, although RMI is used most commonly, both

Conflicts of interest statement

None declared.

Role of the funding source

EM, RK and TVG received grants from CZ Fund and the Academic Fund of Maastricht University Medical Centre+. This study is supported by the Flemish Government: FWO project G049312N, Flanders’. DT is Senior Clinical Investigator of the Research Foundation - Flanders (Belgium) (FWO).

The sponsors had no role in design of the review and meta-analysis; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the work for publication. The

Acknowledgements

The authors acknowledge the authors of included articles who provided us with additional data, whenever possible; Dr. S. Granberg [25], Dr. B. Hagen [50], Dr. F Strigini [34], Dr. L. Roman [32], Dr. O. Lucidarme [27], Dr. A. Rossi and Dr. L. Forzano [51], Dr. S. Derchain [9], Dr. N. Nunes and Dr. D. Jurkovic [6], [7], [77], Dr. E Vaes and Dr. R. Manchanda [56], Dr. J.L. Alcazar [10], [70], [73], Dr. R. Moszynski [30], Dr. V. Arun-Muthuvel [46], Dr. T. Tongsong [58] and the members of the IOTA

References (85)

  • A. Rossi et al.

    A proposal for a new scoring system to evaluate pelvic masses: pelvic masses score (PMS)

    Eur J Obstet Gynecol Reprod Biol

    (2011)
  • M. Terzic et al.

    Risk of malignancy index validity assessment in premenopausal and postmenopausal women with adnexal tumors

    Taiwan J Obstet Gynecol

    (2013)
  • K. Fathallah et al.

    External validation of simple ultrasound rules of Timmerman on 122 ovarian tumors

    Gynecologie Obstetrique Fertilite

    (2011)
  • M.J. Engelen et al.

    Distinguishing benign and malignant pelvic masses: the value of different diagnostic methods in everyday clinical practice

    Eur J Obstet Gynecol Reprod Biol

    (2008)
  • J. Kaijser et al.

    A comparison between an ultrasound based prediction model (LR2) and the risk of ovarian malignancy algorithm (ROMA) to assess the risk of malignancy in women with an adnexal mass

    Gynecol Oncol

    (2013)
  • P.A. van den Akker et al.

    External validation of the adapted risk of malignancy index incorporating tumor size in the preoperative evaluation of adnexal masses

    Eur J Obstet Gynecol Reprod Biol

    (2011)
  • B. Ruiz de Gauna et al.

    Diagnostic performance of IOTA simple rules for adnexal masses classification: a comparison between two centers with different ovarian cancer prevalence

    Eur J Obstet Gynecol Reprod Biol

    (2015)
  • A. du Bois et al.

    Borderline tumours of the ovary: a cohort study of the Arbeitsgmeinschaft Gynakologische Onkologie (AGO) Study Group

    Eur J Cancer

    (2013)
  • L. Valentin et al.

    Adding a single CA 125 measurement to ultrasound imaging performed by an experienced examiner does not improve preoperative discrimination between benign and malignant adnexal masses

    Ultrasound Obstet Gynecol

    (2009)
  • D. Timmerman et al.

    Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group

    BMJ

    (2010)
  • J. Kaijser et al.

    Presurgical diagnosis of adnexal tumours using mathematical models and scoring systems: a systematic review and meta-analysis

    Hum Reprod Update

    (2013)
  • N. Nunes et al.

    Use of the IOTA simple rules for the diagnosis of ovarian cancer: a meta-analysis

    Ultrasound Obstet Gynecol

    (2014)
  • N. Nunes et al.

    A prospective validation of the IOTA logistic regression models (LR1 and LR2) in comparison to subjective pattern recognition for the diagnosis of ovarian cancer

    Int J Gynecol Cancer

    (2013)
  • A. Sayasneh et al.

    Multicentre external validation of IOTA prediction models and RMI by operators with varied training

    Br J Cancer

    (2013)
  • C.A. Hartman et al.

    Ultrasound criteria and CA 125 as predictive variables of ovarian cancer in women with adnexal tumors

    Ultrasound Obstet Gynecol

    (2012)
  • J.L. Alcazar et al.

    IOTA simple rules for discriminating between benign and malignant adnexal masses: a prospective external validation

    Ultrasound Obstet Gynecol

    (2013)
  • P. Geomini et al.

    The accuracy of risk scores in predicting ovarian malignancy: a systematic review

    Obstet Gynecol

    (2009)
  • K. Kinkel et al.

    Indeterminate ovarian mass at US: incremental value of second imaging test for characterization–meta-analysis and Bayesian analysis

    Radiology

    (2005)
  • M. Stukan et al.

    Usefulness of diagnostic indices comprising clinical, sonographic, and biomarker data for discriminating benign from malignant ovarian masses

    J Ultrasound Med

    (2015)
  • D. Timmerman et al.

    Simple ultrasound-based rules for the diagnosis of ovarian cancer

    Ultrasound Obstet Gynecol

    (2008)
  • I. Jacobs et al.

    A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer

    Br J Obstet Gynaecol

    (1990)
  • S. Tingulstad et al.

    Evaluation of a risk of malignancy index based on serum CA125, ultrasound findings and menopausal status in the pre-operative diagnosis of pelvic masses

    Br J Obstet Gynaecol

    (1996)
  • S. Tingulstad et al.

    The risk-of-malignancy index to evaluate potential ovarian cancers in local hospitals

    Obstet Gynecol

    (1999)

  • Minimum training recommendations for the practice of medical ultrasound

    Ultraschall Med

    (2006)
  • P.F. Whiting et al.

    QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies

    Ann Intern Med

    (2011)
  • D. Timmerman et al.

    Logistic regression model to distinguish between the benign and malignant adnexal mass before surgery: a multicenter study by the International Ovarian Tumor Analysis Group

    J Clin Oncol

    (2005)
  • A. Daemen et al.

    Improving the preoperative classification of adnexal masses as benign or malignant by second-stage tests

    Ultrasound Obstet Gynecol

    (2011)
  • S. Guerriero et al.

    Transvaginal color Doppler imaging in the detection of ovarian cancer in a large study population

    Int J Gynecol Cancer

    (2010)
  • O. Lucidarme et al.

    A new computer-aided diagnostic tool for non-invasive characterisation of malignant ovarian masses: results of a multicentre validation study

    Eur Radiol

    (2010)
  • A. Mancuso et al.

    The role of transvaginal ultrasonography and serum CA 125 assay combined with age and hormonal state in the differential diagnosis of pelvic masses

    Eur J Gynaecol Oncol

    (2004)
  • O. Meray et al.

    Risk of malignancy index is not sensitive in detecting non-epithelial ovarian cancer and borderline ovarian tumor

    J Turk Ger Gynecol Assoc

    (2010)
  • R. Moszynski et al.

    Analysis of false negative results of subjective ultrasonography assessment of adnexal masses

    Ginekol Pol

    (2013)

    • Cited by (185)

    • Current uptake and barriers to wider use of the International Ovarian Tumor Analysis (IOTA) models in Dutch gynaecological practice

      2023, European Journal of Obstetrics and Gynecology and Reproductive Biology

    • O-RADS scoring system for adnexal lesions: Diagnostic performance on TVUS performed by an expert sonographer and MRI

      2023, European Journal of Radiology

    • Simple rules, O-RADS, ADNEX and SRR model: Single oncologic center validation of diagnostic predictive models alone and combined (two-step strategy) to estimate the risk of malignancy in adnexal masses and ovarian tumors

      2023, Gynecologic Oncology

    • Radiomics and Radiogenomics of Ovarian Cancer: Implications for Treatment Monitoring and Clinical Management

      2023, Radiologic Clinics of North America

    • Development and validation of an ultrasound-based deep learning radiomics nomogram for predicting the malignant risk of ovarian tumours

      2024, BioMedical Engineering Online

    • Machine learning and radiomics for segmentation and classification of adnexal masses on ultrasound

      2024, npj Precision Oncology
    View all citing articles on Scopus
    1

    Present address: Montessoriweg 1, 3083 AN, Rotterdam, The Netherlands.

    View full text
    Copyright © 2016 Elsevier Ltd. All rights reserved.