Original investigationAssessment of Radiologist Performance in the Detection of Lung Nodules: Dependence on the Definition of “Truth”1
Section snippets
Patient Image Data
A total of 25 thoracic helical CT scans were collected from a single LIDC site in accordance with the previously published inclusion criteria (7, 14). Appropriate local institutional review board approval was obtained for the research use of scans that had been acquired in accordance with established clinical or ongoing research imaging protocols. Each CT scan had been acquired from a different patient (10 females, 15 males; age 40–75 years, median 59) on Aquilion (Toshiba Medical Systems,
Number of Nodules
A total of 91 lesions were identified as “nodule ≥3 mm” by at least one of the four radiologists. The number of nodules identified by each of the four radiologists is shown in Table 1. Radiologist C defined the fewest lesions as nodules (n = 20), and Radiologist A defined the most lesions as nodules (n = 63). For the nodules that were identified by each radiologist, Figure 1 presents the numbers of those nodules that were identified by that radiologist alone, by the radiologist and one other
Discussion
Several limitations are inherent in this study. First, the task of identifying nodules in the context of establishing “truth” for research studies differs from the identification task in the clinical setting, and the radiologists were asked to identify lesions without the benefit of accompanying clinical data. Second, pathologic information was not available for any of the lesions. Third, to define the study targets, radiologists were forced to make binary decisions as to the presence of
References (20)
- et al.
The Lung Image Database Consortium (LIDC): an evaluation of radiologist variability in the identification of lung nodules on CT scans
Acad Radiol
(2007) - et al.
Assessment methodologies and statistical issues for computer-aided diagnosis of lung nodules in computed tomography: contemporary research topics relevant to the Lung Image Database Consortium
Acad Radiol
(2004) - et al.
The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation
Acad Radiol
(2007) - et al.
National Cancer Institute initiative: lung image database resource for imaging research
Acad Radiol
(2001) - et al.
Evaluation of lung MDCT nodule annotation across radiologists and methods
Acad Radiol
(2006) - et al.
The Lung Image Database Consortium (LIDC): a comparison of different size metrics for pulmonary nodule measurements
Acad Radiol
(2007) - et al.
An analysis of early studies released by the Lung Imaging Database Consortium (LIDC)
Acad Radiol
(2007) - et al.
Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT
Eur Radiol
(2005) - et al.
Pulmonary nodule detection with low-dose CT of the lung: agreement among radiologists
Am J Roentgenol
(2005) - et al.
Inter-observer variations on interpretation of multi-slice CT lung cancer screening studies, and the implications for computer-aided diagnosis
SPIE Proc
(2002)
Cited by (0)
- 1
Supported in part by USPHS Grants U01CA091085, U01CA091090, U01CA091099, U01CA091100, and U01CA091103.