Purpose: To assess and validate the performance of commercially available deep learning (DL) algorithm for automatic classification of subsolid nodule comparison with expert thoracic radiologists and radiology residents
Materials and methods: Between January 2020 and July 2021, a thoracic radiologist collected subsolid nodules using the electronic radiology database of our institution by searching for the following terms: subsolid/sub-solid, nonsolid/non-solid, part-solid, ground glass opacity/ nodule. Total 189 subsolid nodules (182 patients) were included. Two experienced chest radiologists re-evaluated chest CT data to classify these nodules in consensus and used as reference standard. The deep learning algorithm implemented in this study was a commercially available DL algorithm (VUNO Med-Lung CT AI, version 1.0.0; VUNO). Four Resident (1-4 years of trainee) categorized each subsolid nodule into part-solid or non-solid and measured mean axial diameter. The rate of agreement of nodule classification was performed with Cohen κ statistics. Intraclass correlation coefficients (ICCs) were used to assess agreement of diameter between DL algorithm and readers. . Chi-square test was used to compare the disagreement in part solid nodule classification.
Results: Among 189 nodules, nodule classification by thoracic radiologists as reference standard was identified as 94 non-solid nodules and 95 part-solid nodule. Agreement of subsolid nodule classification between reference standard and DL algorithm was weak (κ value=0.41). And this value was lower than agreement of between most residents and thoracic radiologists (κ range, 0.67 – 0.69). Disagreement of subsolid nodule classification by DL algorithm was significantly frequent when DL algorithm classified part-solid nodule. (Chi-square = 12.45, p < 0.001). Agreement of classification by DL algorithm was significantly higher in group of larger than 20 mm (28/39, 71.8%, Chi-square = 6.87, p = 0.009) in cases of part-solid nodules
Conclusion: The performance of commercially available deep learning algorithm in subsolid nodule classification in chest CT was not enough compared to experienced chest radiologists. Especially, DL have to be improved for the classification of larger than 20 mm size of part-solid nodule.