Article Text
PDF
Abstract
Introduction Sentinel lymph node dissection is widely used in the staging of endometrial cancer. Variation in surgical techniques potentially impacts diagnostic accuracy and oncologic outcomes, and poses barriers to the comparison of outcomes across institutions or clinical trial sites. Standardization of surgical technique and surgical quality assessment tools are critical to the conduct of clinical trials. By identifying mandatory and prohibited steps of sentinel lymph node (SLN) dissection in endometrial cancer, the purpose of this study was to develop and validate a competency assessment tool for use in surgical quality assurance.
Methods A Delphi methodology was applied, included 35 expert gynecological oncology surgeons from 16 countries. Interviews identified key steps and tasks which were rated mandatory, optional, or prohibited using questionnaires. Using the surgical steps for which consensus was achieved, a competency assessment tool was developed and subjected to assessments of validity and reliability.
Results Seventy percent consensus agreement standardized the specific mandatory, optional, and prohibited steps of SLN dissection for endometrial cancer and informed the development of a competency assessment tool. Consensus agreement identified 21 mandatory and three prohibited steps to complete a SLN dissection. The competency assessment tool was used to rate surgical quality in three preselected videos, demonstrating clear separation in the rating of the skill level displayed with mean skills summary scores differing significantly between the three videos (F score=89.4; P<0.001). Internal consistency of the items was high (Cronbach α=0.88).
Conclusion Specific mandatory and prohibited steps of SLN dissection in endometrial cancer have been identified and validated based on consensus among a large number of international experts. A competency assessment tool is now available and can be used for surgeon selection in clinical trials and for ongoing, prospective quality assurance in routine clinical care.
- sentinel lymph node
- surgical oncology
- surgical procedures
- operative
- uterine cancer
- endometrial neoplasms
Data availability statement
Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information. The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.
Statistics from Altmetric.com
HIGHLIGHTS
There is an urgent need to standardize surgical technique in sentinel lymph node dissection for endometrial cancer, and to develop tools for assessment of surgical quality.
A delphi study achieved consensus on the mandatory, optional and prohibited steps of a sentinel lymph node dissection in endometrial cancer.
A validated competency assessment tool is now available for use in sentinel lymph node dissection surgical quality assurance.
A consensus on the mandatory, optional and prohibited steps of a sentinel lymph node dissection in endometrial cancer has been developed and validated.
A competency assessment tool is now available for use in surgical quality assurance.
Introduction
Surgical trials pose methodological challenges1 because surgeon training, experience, and skills influence the delivery of surgical interventions, leading to a variability in health practices and outcomes.2 Surgical quality assurance can aid adherence to pre-defined standards and outcome measures, and enable reliable comparison across multiple clinical trial sites.3–7
Management guidelines for apparent uterine-confined disease prescribe total hysterectomy, bilateral salpingo-oophorectomy for removal of the primary tumor, and assessment of locoregional lymph nodes to establish the stage of disease (‘staging’).8 This information is prognostic and may guide postoperative treatment decisions.8 9 Historically, surgical staging entailed full or limited pelvic/paraaortic lymph node dissection. This practice was informed by the results of observational, clinicopathologic studies10 11 then adopted by FIGO in 1988.12 Subsequent prospective studies failed to demonstrate differences in survival outcomes.13 14 Contemporary surgical staging involves sentinel lymph node (SLN) dissection.15 According to the SLN concept, tumor cells metastasize to one or two lymph nodes first, before involving further lymph nodes.16 Presumed benefits of SLN dissection include increased surgical staging precision, while sparing removal of other regional lymph nodes.17 Sentinel nodes are examined histopathologically using immunohistochemical ultrastaging.18 SLN dissection obtains accurate information about lymph node status18 such that many clinicians now elect it in place of a full lymphadenectomy.19
With rapid and global adoption of SLN dissection19 comes variability of surgical technique. Local institutional guidelines have been developed to minimize variation in outcomes.20 However, these algorithms are insufficient to facilitate harmonization of the detailed surgical technique across a group of surgeons. There remains a need to define the precise surgical steps required to accomplish satisfactory bilateral SLN dissection; assess a surgeon’s proficiency before enrolment of patients into clinical trials; and assist with ongoing surgical quality assurance.21
The purpose of this study was to establish a consensus on the specific mandatory and prohibited steps of SLN dissection in endometrial cancer, as well as develop a competency assessment tool. This facilitates assessment of surgical quality in clinical trials aiding in both the selection of surgeons and prospective quality assurance.
Methods
Study Participants
The study was approved by the institution's Human Research Ethics Committee and informed written and/or eConsent was obtained from all participants.
Participants were expert gynecological oncology surgeons from five continents currently performing SLN dissection, henceforth referred to as ‘the group’. Experts were recruited using snowball sampling, that is, first contacting surgeons known to perform SLN dissection per scientific reports or presentations in peer-reviewed forums, and then asking these surgeons to nominate other experts. Participant characteristics were summarized using descriptive statistics.
Standardization of Sentinel Lymph Node Dissection
A four-round Delphi methodology was applied in order to achieve standardization of SLN dissection steps and tasks. Several rounds of questionnaires were sent out to experts with the responses then aggregated, de-identified, and shared with the group after each round. Experts adjusted their answers in subsequent rounds, based on their interpretation of the group response provided to them. Over multiple rounds of questionnaires, the Delphi method seeks to reach best response through consensus.22 Study data was collected and managed on a secure, web-based REDCap electronic database hosted at The University of Queensland.23 24
Delphi Consensus Process and Hierarchical Task Analysis
Round One (Semi-Structured Interviews)
After providing written informed consent, interviewees described their opinion about the mandatory, optional, and unwarranted steps taken in performing SLN dissection for endometrial cancer.
The interviews were conducted individually and were audio recorded. Recordings were transcribed and thematically analyzed by two reviewers (KM, AO). Each reviewer independently identified important and recurring codes (eg, uterine manipulation, identifying anatomy, troubleshooting). Codes were then compared to confirm important themes. The reviewers jointly examined codes and themes and interpreted the data. Where discordance in coding was identified, themes were refined through discussions between the two reviewers. Interviews were conducted until saturation in variations of technique. Key steps and tasks of SLN dissection were identified by a process of hierarchical task analysis.
Rounds Two–Four (Consensus Process)
An initial questionnaire was devised including all of the variations identified in the interviews. Members of the group were invited to indicate their agreement or disagreement with variations. In accordance with other published work,25 consensus agreement level was set at 70%.4 22 Variations where consensus was reached were iteratively moved into an operation guide: those with <70% agreement remained for a subsequent survey round.
In accordance with the journal’s guidelines, we will provide our data for the reproducibility of this study in other centers if such is requested.
Operation Guide
A SLN dissection operation guide was created including the mandatory, optional, and prohibited/unwarranted steps that reached 70% agreement level.
Competency Assessment Tool
Development
The final competency assessment tool was limited to mandatory and prohibited steps in the intraoperative phase of SLN dissection. A score of one to four was allocated to each step – ‘skillful’, ‘adequate’, ‘inconsistent’, ‘lacking/deficient’: for troubleshooting steps ‘not applicable’ was also offered.
Content Validity
Three surgical videos were selected having been agreed by KM and AO to represent poor, inconsistent, or optimal technique of SLN dissection according to the ratings conferred by application of the competency assessment tool. The videos featured the 11 surgical steps of SLN dissection assessed by the competency assessment tool, but did not include tracer preparation and injection, surgical troubleshooting, or pathological assessment of tissues. Content validity was assessed by KM and AO who discussed each step of the competency assessment tool in detail, before watching those individual steps performed with various skill levels across the three surgical videos, confirming that competency assessment tool items adequately reflected the skill required.
Contrast Validity and Internal Reliability
Contrast validity26 was assessed via invitation of the group members to use the competency assessment tool in rating the three pre-selected videos, each representing distinct performance levels. Due to the occurrence of some cells with a cell size <5, Fisher’s exact tests were performed to assess if the proportion of experts who rated each of the three videos as ‘skillful’, ‘adequate’, ‘inconsistent’, or ‘lacking/deficient’ differed according to the quality of the video. An average competency assessment tool score (possible range 11–44) was computed for each video. One-way ANOVA modeling determined if the overall competency assessment tool score assigned by the SLN dissection experts to each video differed significantly. The summary score was used to assess the internal consistency (Cronbach alpha) of the competency assessment tool.
Results
Thirty-five international gynecological oncology surgeons and experts in SLN dissection from 16 countries participated. Some demographic data was not available for five participants, but 28 surgeons were above 40 years of age (80%) and 27 were male (77%) (Table 1).
Participating surgeons' demographic characteristics
Twenty-four surgeons had practiced gynecological oncology for more than 10 years (69%), and 21 had performed SLN dissection for more than 5 years (60%). Nineteen surgeons (54%) reported that their institution had an endometrial cancer SLN dissection standard protocol. Twenty-one surgeons (60%) performed more than 50 SLN dissections annually, excluding those performed for cancer of the vulva. Participating surgeons reported using between one and eight methods to learn SLN dissection, for example, being self-taught (46%), learning from research papers (43%) or being trained by a senior colleague (31%).
Standardization of Sentinel Node Dissection
Delphi Round One (Hierarchical Task Analysis)
Saturation in the variation of the SLN dissection technique was reached after 25 interviews. Analysis of transcripts allocated themes into four phases: preoperative (dye selection and preparation, injection); intraoperative (pelvic dissection, identification of key anatomical structures, definition and dissection of sentinel node, extraction of tissue); troubleshooting; and a postoperative (pathology) phase. Task variations were defined as management of specific surgical steps in different ways. In total, 107 task variations were identified across the interviews (Table 2).
Hierarchical task analysis including task variations
Delphi Rounds Two–Three (Consensus Process)
The first survey (Delphi round two) featured 107 task variations and was completed by all 35 participants (Online supplemental table 1). The second survey (Delphi round three) was informed by the results of the first survey and 33 of 35 participants responded (Online supplemental table 2). Over rounds two and three, >70% consensus was achieved in 33 of the 107 (30.8%) task variations4 22 on mandatory, optional, and prohibited steps of SLN dissection. Of the variations that reached consensus, 21 were classified as mandatory, nine optional, and three prohibited. For example, in round two, 79% of participants agreed that ‘a transperitoneal approach of injecting dye into the uterus’ should be prohibited, while 75% of participants agreed that ‘the internal iliac artery must be identified for sentinel node mapping’ was mandatory. An operation guide consisting of the final list of steps for which consensus was obtained is provided in Table 3.
Supplemental material
Supplemental material
Operation guide
There was consensus that the tracer of choice must be indocyanine green (ICG) but adding other tracers is optional. There was consensus that ICG should be administered by superficial injection (1–2 mm) into the cervix. Superficial injection was defined by the group as sub-mucosal injection into the epithelium of the ectocervix (akin to intra-dermal injection techniques used in SLN mapping for vulvar cancer). Deep injection (1 cm) was considered optional only when combined with mandatory superficial injection. Deep injection alone was considered prohibited by consensus. There was no consensus about the dilution of ICG (between 0.5 mg/L and 1.5 mg/mL), the total volume injected, or timing of injection (before or after establishing a pneumoperitoneum). The use of a uterine manipulator was considered optional, but if used, it should be inserted after tracer injection. There was consensus that dividing the round ligament and the infundibulopelvic ligament can be performed either before or after SLN dissection. The pelvic structures and spaces that should be demonstrated for SLN dissection include external and internal iliac vessels, ureter, obliterated umbilical ligament, and the paravesical space. The direction of the SLN dissection was considered optional (starting close to the cervix or dissecting toward the cervix).
The group agreed that the sentinel node should be defined as the most proximal node irrespective of the nodal station in which the node is found. Eighteen participants felt that mapping of presacral nodes should be optional (56.3%). There was lack of consensus on a side-specific lymphadenectomy if no nodes are mapped on one side. Participants agreed that the sentinel node should be a single mapped node with or without its next station (second echelon node(s)). A majority of participants (59.4%) but less than required for consensus, agreed that not all second, third, and fourth echelon nodes should be removed. Greater than 70 percent of participants agreed that specimen extraction should be within a containment device. There was consensus that ex-vivo fluorescence should be used to prove the sentinel node; that labeling of the sentinel node should be according to laterality and nodal station; and enhanced pathology techniques for ultrastaging of sentinel nodes should be used.
Contrast Validity and Internal Reliability
Twenty-seven (77.1%) Delphi participants were involved in rating the quality of surgery of the three preselected videos using the competency assessment tool (Figure 1). For each of the 10 initial surgical steps, there was clear separation in the rating of the skill level displayed between the three videos (Table 4).
SLND competency assessment tool.
Assessment of contrast validity
For example, while 78% of experts rated the ‘optimal technique video’ as skillfully performing the dissection of the iliac vessels, only 19% and 0% of experts rated the ‘inconsistent technique video’ and ‘poor technique video’ as skillful (Fishers' exact test=56.0; P<0.001). For the last step (‘completion of SLND in one hemipelvis before proceeding to the contralateral side’), 25 of the 27 group members rated this step as not applicable. Overall, the mean skills summary score differed significantly between the three videos from 35.6 (SD=4.7) for the ‘optimal technique video’, to 25.3 (SD=5.9) for the ‘inconsistent technique video’ and 17.7 (SD=4.1) for the ‘poor technique video’ (one-way ANOVA F score=89.4; P<0.001). Internal consistency of the items was high (Cronbach α=0.88).
Discussion
Summary of Main Results
We report the creation of a competency assessment tool, derived by consensus among a large number of international experts, outlining the mandatory, optional,and prohibited steps of a SLN dissection procedure for endometrial cancer. The competency assessment tool is validated by gynecological oncology surgeons and can be used by trial governance committees as a decision aide for surgeon selection and for ongoing quality assurance in surgical clinical trials.
Results in the Context of Published Literature
While local health service protocols20 suggest specific steps for a SLN dissection, the present publication summarizes an operating consensus based on the opinion of a considerable number of international experts in SLN dissection. Consensus was achieved about definition of the sentinel node (the node closest to the uterus) regardless of whether it is located at the lateral pelvic wall, the aortic/caval, or the presacral area. There was also agreement that the number of sentinel nodes removed should be kept to a minimum. There was no consensus on the mandatory need for completion lymphadenectomy on the ipsilateral side of a pelvis that fails to map. This most likely reflects the possibility of patient and uterine factors indicating against full lymph node dissection. Greater than 70 percent consensus was reached on the need to extract nodes through a containment system, the need for ex-vivo green fluorescence to prove the sentinel node, on specimen labeling, and on pathologic ultrastaging.
The competence assessment tool development undertaken in this study follows similar efforts in other surgical specialties. In general surgery, a recent systematic review reporting on quality assurance in randomized controlled trials of laparoscopic colorectal surgery identified three distinct categories of surgical quality assurance measures: trial entry criteria for surgeons and centers; standardization of surgical techniques; and continuous monitoring of surgeons and/or units.25 A competence assessment tool was developed, validated, and implemented to assess technical surgical performance in the context of a summative assessment process for the National Training Program in Laparoscopic Colorectal Surgery (Lapco).27 Subsequently, COLOR III4 investigators have described the standardization of surgical interventions followed by the development and assessment of objective surgical quality assurance tools for use in colorectal trials.
Strengths and Weaknesses
The strengths of our study include the large number of international experts who identified the mandatory, optional, and prohibited steps of SLN dissection based on consensus. In addition, the competence assessment tool was able to demonstrate contrast validity and internal reliability. Our expert participants reported a range of experience in SLN dissection despite recruitment using snowball sampling. We did not use more objective measurements of expertise such as a requirement for study participants to submit their own outcomes data, or videos of their individual technique. Another weakness of our study relates to the impact of our findings on meaningful clinical outcomes such as sensitivity for detection of metastases, or false negative rates. These parameters were not the focus of our study, but we aknowledge the absolute necessity for individual surgeons to monitor their performance including prscopective audit of sensitivity, false negatives, recurrence patterns, and rates. This predictive clinical validity of the SLN dissection technique can only be determined with accumulation of clinical outcomes after using the competence assessment tool in SLN dissection clinical trials and educational programs, as has been demonstrated for a colorectal competence assessment tool in the Lapco program.28
Implications for Practice and Future Research
Despite the benefits of SLN dissection, including shorter operating times compared with a lymphadenectomy, it remains unknown in which ways SLN dissection impacts clinically relevant outcomes, such as the need for postoperative radiation treatment or chemotherapy, recovery from surgery and quality of life, the incidence of adverse events, disease recurrence, and survival.21 Additionally, while new surgical procedures may appear promising, there remains a need to evaluate novel surgical procedures for safety and effectiveness.29 Such surgical trials rely on the standardized delivery of the intervention (with minimal variation) to allow a meaningful and reliable comparison between intervention and control groups across multiple surgeons or trial sites. In the context of SLN dissection, variability in technique, and failure to identify sentinel nodes could translate into the need for frozen section assessment of the uterus, acceptance of unknown nodal status, or may increase the risk of an “empty package”,30 all of which may confound the results of SLN dissection efficacy trials. Depending on local protocols, some patients may even require re-staging, a full ipsilateral lymphadenectomy,20 or might warrant adjuvant chemotherapy or radiation treatment based on uterine risk factors. These scenarios may have significant impact on short- and long-term patient and trial outcomes. Using this competency assessment tool, institutions or clinical trialists can define quality standards of sentinel lymph node dissection and measure individual surgical performance.
Significant efforts are made by chief investigators and trial management committees to minimize the variability in surgical techniques and outcomes, including limiting the trial to sites with a high surgical volume. Recently, principal investigators completed a site visit and all surgeons were observed in-person18 31 or unedited videos were reviewed to confirm standardization of the technique.32 In other trials, participating surgeons were required to have completed a minimum number of procedures, before the initiation of enrollment.33 While these measures were valuable within institutions, volume, minimal number, or observation of one surgery, they may be inaccurate without application of a standardized assessment tool.
Conclusion
The output from this work includes a list of mandatory and prohibited steps of a SLN dissection that independent assessors can use to check for both surgical proficiency as well as whether SLN dissection has been performed in accordance to an agreed standard. The work provides specific steps of SLN dissection, and the quality assurance criteria developed as part of this process will help with selection of prospective surgeons into surgical trials evaluating SLN dissection. The goal is to shorten the learning curve34 but also to control for heterogeneity in surgical performance that could override the true efficacy.4
Supplemental material
Data availability statement
Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information. The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.
Acknowledgments
We thank all participants for their expertise in developing this tool. We also thank Vanessa Behan, Queensland Centre for Gynaecological Cancer Research, University of Queensland, Centre for Clinical Research, and QLD Australia for administrative support with the project.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
GH and AO are joint senior authors.
Twitter @frumovitz, @leitaomd, @drjuliolau, @DrFMartinelli, @ThomasInd
Contributors KM, AO, MJ, and GH contributed to the design of the trial. KM, AO, and MJ contributed to manuscript writing. All authors contributed to data acquisition, interpretation of data, revising the draft for intellectual content, final approval of the version to be published, and agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy and integrity of any part of the work were appropriately investigated and resolved.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial, or not-for-profit sectors.
Competing interests AO reports grants and personal fees from Surgical Performance PTY LTD, grants from Medtronic, outside the submitted work; NRA-R reports grants from Stryker/Novadaq, outside the submitted work; MF reports grants from Astra Zeneca, grants from Tesaro/GSK, grants and personal fees from Stryker, grants from Biom’Up, outside the submitted work; ML reports Ad hoc consulting from Intuitive Surgical, serves on advisory board for Ethicon, partial grant support from NIH/NCI Memorial Sloan Kettering Cancer Center Support, outside the submitted work; TEI reports personal fees from Medtronic, personal fees from Intuitive Surgical, outside the submitted work; RK reports personal fees from Intuitive Surgical Inc., personal fees from Medtronic, personal fees from Medicaroid, outside the submitted work, and President of SERGS and Council Member of IGCS; HF reports personal fees from Intuitive Surgical Inc, outside the submitted work; JP reports personal fees from Intuitive Surgical Inc., outside the submitted work; AA reports grants and a site PI, speaker fees and serves on advisory board for Astrazeneca, serves on the advisory board for GSK, grants and grats and site Co-PI and speaker fees from Merck, speaker fees from Sanofi, grants from Pfizer, grants from Clovis, grants from CancerCare Manitoba Foundation, grants from Canadian Clinical Trials group, outside the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.