2.1

Data collection and preprocessing

A total of 295 lesions were detected in 289 patients at the Shanghai Public Health Clinical Center (hospital 1). Additionally, in a recent examination, 127 patients with 138 lesions were re-selected at the same hospital (hospital 1.1). The basic patient information is detailed in Table 1. The selection criteria included: (1) patients who had undergone thin-slice chest CT imaging, (2) patients who had undergone an unenhanced chest CT exam, and (3) patients diagnosed with lung adenocarcinoma by doctors. Among these patients, one pGGN was detected in 187 patients and two in 6 patients. All lesions were pathologically diagnosed as 174 MIA, 54 AIS, and 67 IAC.

Table 1.

Demographic characteristics

Ages and size are shown as mean ± standard deviation;

M male, F female

*

p value < 0.05

For the study, 236 lesions were allocated to the training set, while 59 were included in the testing set. Two to eight images with clear features were selected from the CT scans of each lesion, resulting in 853 CT images in the training set and 213 in the testing set. Additionally, 138 lesions diagnosed as 75 MIA, 36 AIS, and 27 IAC from two other hospitals were included. One to three images with clear features were selected from the CT scans of each lesion in this new dataset, yielding 267 CT images in the validation set. The patient selection flowchart is depicted in Figure 1.

Figure 1.

Flow chart of case sample selection

2.2

CT scanning

An unenhanced chest CT exam was performed during the entire lung scan of each patient with a United-Imaging 760 CT device (42-126 mA 120 kV, slice thickness of 1 mm), a Siemens Emotion 16 CT device (34-123 mA, 130 kV, 1 mm) and a Siemens Perspective 77427 CT device (155mA, 110kV, 1mm), under a 512×512 resolution.

2.3

Feature extraction

A semi-automated, unsupervised method was employed to segment the region of interest (ROI) for each CT image[26]. This segmentation was subsequently confirmed by a radiologist with a decade of experience in chest CT interpretation.

Radiographic characteristics were independently evaluated by two experienced thoracic radiologists, with 6 and 13 years of experience in chest CT interpretation, respectively, who were blinded to the pathological results. Discrepancies between the observers were resolved through consensus. The radiographic characteristics analyzed for each lesion included: (1) margin (clear, blurred), (2) lobulation (absent, present), (3) spiculation (absent, present), (4) pleural attachment, including pleural tag and indentation (absent, present), (5) air bronchogram (absent, present), (6) vessel change (absent, present), and (7) bubble lucency (absent, present) (Table 2).

Table 2.

The Parameters of lesions in datasets

A total of 1,068 features were extracted from the ROI using commercial software (PyRadiomics 3.0.1)[27]. These features included tumor size, shape, first-order statistical descriptors (histogram features), and high-order texture features (gray level co-occurrence matrix and gray level run length). The original images were normalized prior to feature extraction.

2.4

Feature selection

Given that the initial number of features extracted was close to the number of patients, a systematic feature selection process was implemented to avoid overfitting. The feature set was divided into radiographic characteristics and radiomic features. All radiographic characteristic features were preserved due to their clinical relevance. For the radiomic features, an initial screening was conducted where features with low variance were removed as they provide minimal information and can introduce noise. Furthermore, highly correlated features (correlation coefficient > 0.9) were identified, and one feature from each pair was removed to reduce multicollinearity. Feature importance was then assessed using the ’RandomForest’ [28] package to compute the ’MeanDecreaseGini’ metric, ranking features in decreasing order of their importance scores. To determine the optimal subset of radiomic features, cross-validation was employed, calculating the root mean squared error (RMSE) for various feature subsets and identifying the subset that minimized RMSE. This iterative process ensured the retention of the most predictive features. Additionally, the correlation of each feature in the optimal radiomic feature set was analyzed to ensure that the selected features were not redundant and provided unique information. The final selected features included original_shape2D_Perimeter, squareroot_firstorder_90Percentile, original_shape_Maximum2DDiameterRow, wavelet.LL_glcm_ClusterShade, wavelet.HL_glszm_SmallAreaLowGrayLevelEmphasis, original_shape2D_Sphericity, diagnostics_Image.original_Mean, wavelet.LL_firstorder_Minimum, and logarithm_firstorder_90Percentile.

The optimal radiomic feature subset was then combined with the radiographic characteristics feature set to form a comprehensive training feature set, which was used for model training. This thorough feature selection process ensured that the final model included only the most relevant and non-redundant features, thereby enhancing its predictive performance and robustness.

Figure 2.

The optimal radiomic feature subset and its performance are obtained through cross validation. (a) represents the performance of cross validation. (b) represents the correlation diagram of 9 features. (c) represents the important value of 9 features.

2.5

Model training and validation

Based on the preliminary training of all features, the optimal values of parameters ’mtry’ and ’ntree’ required by the random forest model were calculated (mtry=20, ntree=900, Figure 3). Then, on the premise of the same parameters, the random forest model was trained with those three feature sets respectively (training feature set as model 1, radiographic characteristics feature set as model 2 and the optimal radiomic feature set as model 3). At the same time, SVM, Logistic Regression and C5.0 algorithm were used to verify the results.

Figure 3.

Optimal parameters of random forest model. (a) represents the performance of the ‘mtry’. (b) represents the performance of the ‘ntree’.

2.6

Statistical analysis

Quantitative data were presented as mean ± SD or median (25th–75th percentile), while qualitative data were described as counts (n). Comparisons between groups for qualitative variables were performed using Fisher’s exact test, and comparisons for quantitative variables were performed using either a t-test or the Wilcoxon test. A p-value of less than 0.05 was considered statistically significant.

The Area Under the Curve (AUC) represents the degree of separability and serves as a summary measure of the accuracy of a quantitative diagnostic test. The Receiver Operating Characteristic (ROC) curve, a commonly used graph to summarize the performance of a classifier across all possible thresholds, is generated by plotting the true positive rate (y-axis) against the false positive rate (x-axis). The ROC curve and AUC were used to evaluate the diagnostic performance, precision, and discrimination accuracy of the models.

All statistical analyses were performed using R statistical software (http://www.Rproject.org, version 3.6.0) with the following packages: ’randomForest’, ’ggplot2’, ’caret’, ’pROC’, ’e1071’, and ’reshape2’.

2.7

Results

The performance of the radiomics model constructed using random forest was evaluated by precisions and accuracies. The ROC curves of random forest models based on three feature set are shown in Figure 4. Because there are three types of lesions, the subsequent evaluation is to combine two of them into one and compare them with the remaining one (MIA represents the comparison between MIA and IAC & AIS, AIS represents the comparison between AIS and IAC & MIA, IAC represents the comparison between IAC and MIA & AIS).

Figure 4.

The performance of random forest model based on different feature sets on testing dataset and verification dataset. The yellow line represents the performance of the model in MIA compared with AIS&IAC. The blue line represents the performance of the model in AIS compared with MIA&IAC. The red line represents the performance of the model in IAC compared with MIA&AIS.

For the testing dataset, the AUC scores of model 1, model 2, and model 3 were 0.974, 0.483, and 0.835, respectively. The sensitivity values of model 1, model 2, and model 3 at the optimum critical point were 0.873(MIA:0.967, AIS:0.825, IAC:0.826), 0.814(MIA:0.866, AIS:0.750, IAC:0.826), and 0.641(MIA:0.902, AIS:0.378, IAC:0.642). The specificity values of model 1, model 2, and model 3 at the optimum critical point were 0.936(MIA:0.837, AIS:0.994, IAC:0.976), 0.897(MIA:0.791, AIS:0.965, IAC:0.934), and 0.829(MIA:0.589, AIS:0.972, IAC:0.925).

For validation dataset, the AUC scores of model 1, model 2, and model 3 were 0.964, 0.915, and 0.841, respectively. The sensitivity values of model 1, model 2, and model 3 at the optimum critical point were 0.867(MIA:0.940, AIS:0.837, IAC:0.824), 0.825(MIA:0.880, AIS:0.756, IAC:0.838), and 0.648(MIA:0.927, AIS:0.245, IAC:0.772). The specificity values of model 1, model 2, and model 3 at the optimum critical point were 0.932(MIA:0.855, AIS:0.986, IAC:0.955), 0.906(MIA:0.803, AIS:0.949, IAC:0.965), and 0.798(MIA:0.521, AIS:0.986, IAC:0.886). The performance of the random forest models based on the three feature sets are shown in the Table 3.

Table 3.

The performance of random forest models in testing and validation datasets

At the same time, SVM, Logistic Regression and C5.0 algorithm used the same training dataset and training feature set to train the model. Evaluate the performance of the model on the testing dataset. For SVM model, the AUC score was 0.734. The sensitivity values were MIA:0.921, AIS:0.750, IAC:0.348. The specificity values were MIA:0.547, AIS:0.959, IAC:0.976. For Logistic Regression model, the AUC score was 0.780. The sensitivity values were MIA:0.850, AIS:0.750, IAC:0.587. The specificity values were MIA:0.709, AIS:0.971, IAC:0.892. For C5.0 model, the AUC score was 0.700. The sensitivity values were MIA:0.843, AIS:0.475, IAC:0.522. The specificity values were MIA:0.558, AIS:0.965, IAC:0.886. The details are shown in the table 4.

Table 4.

The performance of validation models in testing datasets