Article Text

Download PDFPDF
467 Ask RoBERTa: operative note-driven outcome forecasting in advanced ovarian cancer cytoreductive surgery
  1. Alexandros Laios1,
  2. Marios Evangelos Mamalis2,
  3. Evangelos Kalampokis2,
  4. Amudha Thangavelu1,
  5. Timothy Broadhead1,
  6. David Nugent1 and
  7. Diederick Dejong1
  1. 1Department of Gynaecologic Oncology, ESGO Centre of Excellence for ovarian cancer surgery, St James’s University Hospital, Leeds, UK
  2. 2Information Systems Lab, Department of Business Administration, University of Macedonia, Thessaloniki, Greece


Introduction/Background Contemporary efforts to forecast surgical outcomes traditionally rely on evaluating discrete surgical risk factors. This study sought to investigate whether employing Natural Language Processing (NLP) techniques to scrutinise unstructured operative notes could enhance the prediction of complete cytoreduction (CC0) in patients with advanced epithelial ovarian cancer (EOC) who underwent cytoreductive surgery.

Methodology Electronic Health Records were accessed to identify patients with advanced EOC and extract their operative notes. The Term Frequency – Inverse Document Frequency (TF-IDF) score was used to assess the discriminative capacity of word sequences (n-grams) in predicting the presence of residual disease. A state-of-the-art RoBERTa-based classifier was employed via transfer learning to extract structured information from unstructured operative notes. Discrimination performance was measured using the area under curve (AUC). An XGBoost Machine Learning model was trained in the same dataset to evaluate the additional value of the probabilities generated by the RoBERTa classifier when combined with discrete features.

Results The study cohort comprised 555 cases of EOC cytoreduction between January 2014 and December 2019. Distinctive word clouds, weighted by n-gram TF-IDF, that effectively discriminated between non-CC0 and CC0 predictions featured words such as ‘adherent’ and ‘miliary disease’ (figure 1). Performance metrics included an Area under the Receiver Operating Characteristic (AUROC) of 0.86, an Area under Precision-Recall Curve (AUPRC) of 0.87, and precision, recall, and F1 score values of 0.81. When the probabilities derived from the RoBERTa classifier were integrated with conventional discrete clinical and engineered predictors in the XGBoost model, a modest improvement in overall model performance was observed, resulting in an AUROC and AUPRC of 0.91.

Conclusion We employed a sui-generis approach to refine information retrieved from the extensive textual surgical data, showcasing the effective modelling of extraction tasks for classification prediction. The application of state-of-the-art NLP methods to biomedical texts holds promise for advancing modern EOC patient care.

Disclosures There are no conflicts of interest.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.