Article Text
PDF
Abstract
Introduction/Background Contemporary efforts to forecast surgical outcomes traditionally rely on evaluating discrete surgical risk factors. This study sought to investigate whether employing Natural Language Processing (NLP) techniques to scrutinise unstructured operative notes could enhance the prediction of complete cytoreduction (CC0) in patients with advanced epithelial ovarian cancer (EOC) who underwent cytoreductive surgery.
Methodology Electronic Health Records were accessed to identify patients with advanced EOC and extract their operative notes. The Term Frequency – Inverse Document Frequency (TF-IDF) score was used to assess the discriminative capacity of word sequences (n-grams) in predicting the presence of residual disease. A state-of-the-art RoBERTa-based classifier was employed via transfer learning to extract structured information from unstructured operative notes. Discrimination performance was measured using the area under curve (AUC). An XGBoost Machine Learning model was trained in the same dataset to evaluate the additional value of the probabilities generated by the RoBERTa classifier when combined with discrete features.
Results The study cohort comprised 555 cases of EOC cytoreduction between January 2014 and December 2019. Distinctive word clouds, weighted by n-gram TF-IDF, that effectively discriminated between non-CC0 and CC0 predictions featured words such as ‘adherent’ and ‘miliary disease’ (figure 1). Performance metrics included an Area under the Receiver Operating Characteristic (AUROC) of 0.86, an Area under Precision-Recall Curve (AUPRC) of 0.87, and precision, recall, and F1 score values of 0.81. When the probabilities derived from the RoBERTa classifier were integrated with conventional discrete clinical and engineered predictors in the XGBoost model, a modest improvement in overall model performance was observed, resulting in an AUROC and AUPRC of 0.91.
Conclusion We employed a sui-generis approach to refine information retrieved from the extensive textual surgical data, showcasing the effective modelling of extraction tasks for classification prediction. The application of state-of-the-art NLP methods to biomedical texts holds promise for advancing modern EOC patient care.
Disclosures There are no conflicts of interest.
