Main points

• Early monitoring and intervention for patients with coronavirus disease-2019 (COVID-19) will benefit both patients and the medical system.

• Chest computed tomography (CT) radiomics provide more information for the prognosis of COVID-19 pneumonia.

• The area under the curve of each model demonstrated good predictive accuracy [death: 0.846; stage: 0.918; complication: 0.919; acute respiratory distress syndrome (ARDS): 0.852]. After finding the optimal cut-off for each outcome, the respective accuracy, sensitivity, and specificity were 0.854, 0.700, and 0.864 for the prediction of death of COVID-19 patients; 0.814, 0.949, and 0.732 for the prediction of higher-stage COVID-19; 0.846, 0.920, and 0.832 for the prediction of complications of COVID-19; and 0.814, 0.818, and 0.814 for ARDS in COVID-19 patients.

The novel coronavirus disease-2019 (COVID-19) has caused a global pandemic, which presents a threat to human health. The COVID-19 infection causes a fever, cough, and diarrhea, among other symptoms. It can affect several tissues, lead to rapid organ failure, and has a poor prognosis and high mortality rate. Once patients progress to a severe stage of pneumonia, over 60% of them die.¹ To date, there is no effective treatment for COVID-19. However, early diagnosis, immediate patient isolation, and extensive vaccination could effectively prevent the transmission of the SARS-CoV-2 virus.² Accurate predictive models are needed to identify the risk of patients experiencing a poor clinical outcome and plan early intervention to improve outcomes.^3,4,5

A chest computed tomography (CT) scan combined with a positive molecular polymerase chain reaction (PCR) test is the most important diagnostic method for COVID-19. Compared with the test conducted in standard laboratories, the CT scan procedure has a faster turnaround time and can provide more detailed information about the prognostic significance of the severity of lung damage. Several studies on quantitative CT radiomics or deep-learning techniques have shown the efficiency of a rapid diagnosis of COVID-19.^6,7 It is unknown whether quantitative CT radiomics could provide more information for patients. The quantitative image provides data on clinical decisions and prediction prognoses in many fields,^8,9 and radiomics provide more detailed information on the severity of the lung damage and prognosis of patients with COVID-19.

In this paper, we have developed a radiomics prediction model, a novel tool that extracts hundreds of quantitative features based on the shape, intensity, size, or volume of the target lesions, to predict the outcomes of COVID-19.

Methods

Patients

We retrospectively analyzed 157 patients with confirmed positive results of COVID-19 from a viral nucleic acid reverse transcription-PCR test of respiratory secretions via a nasopharyngeal or oropharyngeal swab in Wuhan Leishenshan Hospital. The Ethics Committee of Shanghai Sixth’s People’s Hospital approved this retrospective study, and written informed consent was waived (approval no.: 2020-KY-013).

All patients’ first CT scans after hospitalization were included (Incisive CT, Philips Healthcare and Revolution Maxima, GE Healthcare). The scanning range was from the apex to the lung base. The main scanning parameters were as follows: tube voltage = 120 kVp, tube current = 360 mAs/287 mAs, matrix = 512 × 512, slice thickness = 5 mm, spacing between slices = 5 mm, field of view = 350 mm × 350 mm, window level = 600 Hounsfield units (HU), and window width = 1.200 HU.

Clinical variables and the primary outcome

Clinical data were collected, including the clinical signs and symptoms (fever, headache, cough, expectoration, fatigue, dyspnea, nausea and vomiting, diarrhea, arthralgia, and myalgia), imaging results, demographic variables (age, sex, smoking status, and time between onset of symptoms to admission), and medical history (comorbidities, respiratory diseases, diabetes, hypertension, coronary artery disease, cerebrovascular disease, cancer, and chronic renal disease).

The primary endpoint in the study was efficacy in the predictions of death, clinical stage, and complications. Complications, including stroke, acute kidney injury, acute respiratory distress syndrome (ARDS), and heart failure, which appeared secondary to pneumonia, were defined as positive if the patient had one or more of these complications.

Image segmentation and blinding

All non-contrasted CT images were performed using ITK-SNAP software (version 2.2.0; www.itksnap.org) for manual segmentation of the regions of interest (ROIs). Since the presence of lesions interfered with the automatic identification of the chest, we manually delineated along the edge of the pulmonary parenchyma, slice by slice, for each patient. A three-dimensional ROI of the whole lung was then automatically generated by the software. The hilus pulmonis and the trachea were also included in the ROI (Figure 1). All the images were evaluated by two experienced radiologists who were blinded to the patients’ clinical information (Ran-ying Zhang, Reader 1, with seven years of radiologist experience; Ting Yao, Reader 2, with four years of experience).

Radiomic signature building

Figure 1 demonstrates our workflow. The radiomic features were extracted from each ROI using PyRadiomics on Python (version 3.7).¹⁰ Before extraction, all the chest CT images were subjected to image normalization (the intensity of the image was scaled to 0–500). During the normalization process, the binwidth was set to 25, and the intensity of the image of from 1 to 25 bin, 26 to 50, 51 to 75 and so on was regarded as the same intensity in avoid of diversity due to the different parameter setting of CT ma­chine and personal difference. Then, the normalized image was resampled to the same resolution (1 mm × 1 mm × 1 mm) using the interpolation method of sitkBSpline to avoid any possible data heterogeneity. This procedure was followed by a filtering process to implement image smoothing. After filtering, the radiomic features were extracted from the ROI of the original image and its corresponding filtered results, which included features of first-order statistics, shape, grey-level co-occurrence matrix, grey-level run-length matrix, grey-level size-zone matrix, gray-level dependence matrix, and wavelet features.

The radiomic features of all patients were standardized using the z-score method. Intra-/inter-class correlation coefficients (ICCs) were calculated for each extracted radiomic feature, and those with ICCs of >0.8 were selected. In addition, we calculated the P value of the paired t-test for radiomic features with ICCs of >0.8. We chose the least absolute shrinkage and selection operator (LASSO) algorithm to complete the radiomic signature building and form radiomic models with features of non-zero regression coefficients. Each endpoint (stage, death, complication, respiratory failure) had a corresponding model. In total, four radiomic models were constructed to predict the occurrence of the endpoints.

To build a predictive radiomics model for each outcome, we followed several steps. First, the method of normalization to z distribution [(value – mean value)/standard deviation] was applied for each extracted feature. Second, the ICCs were calculated for each extracted radiomic feature, and those with ICCs of >0.8 were selected. Third, the LASSO algorithm was applied for further feature reduction. The most significant features with the smallest deviance were then selected using the LASSO algorithm for the final features. The LASSO algorithm is a penalized regression method that has been successfully applied to oncologic research. The LASSO algorithm can estimate the regression coefficients by maximizing the log-likelihood function (or the sum of squared residuals) with the constraint, reduce the coefficients of indistinctive covariates to zero, and enable the non-zero features to be combined into a radiomics model.^11,12 With this model, the risk score for each patient was calculated using the following formula weighted by regression coefficients for each outcome: risk score = constant + coefficients × features.

Statistical analysis

The predictive accuracy of the radiomic signature was evaluated using a receiver operating characteristic curve analysis. To determine the optimal cut-off value to predict each outcome, the Youden index was calculated for all possible cut-off values (c) [(Youden index = maxc (sensitivity + specificity – 1)],¹³ and the value of c that achieves the maximized index was considered optimal. For each model, the accuracy, sensitivity, and specificity were also measured using the defined optimal cut-off values. For internal validation, the corrected area under the curve (AUC) was calculated using bootstrapping validation (1,000 bootstrap resamples).¹⁴ In addition, a decision curve analysis (DCA) was performed to evaluate the clinical usefulness of the radiomic signature by quantifying the net benefit at different threshold probabilities.¹⁵

To explore the clinical utility of the addition of a radiomics signature for each outcome to the models with only clinical data included, we first constructed the clinical model using stepwise backward regression. We initially included the demographics of patients, their symptoms, and their past medical history by calculating the AUC for each outcome. Then, the AUC was calculated for the mixed models by including the clinical models and radiomics signature. Meanwhile, the net reclassification index (NRI), an alternative to AUC to assess the improvement in risk prediction and measure the usefulness of a new model,¹⁶ was calculated to evaluate the clinical benefits and utility of the mixed models compared with the clinical models. A statistical analysis was performed using R software (version 3.5.0, packages: irr, caret, glmnet, caTools, OptimalCutpoints, rms, rmda), and P < 0.05 was considered statistically significant.¹⁷

Results

Patient characteristics

We collected data from 157 patients in Wuhan Leishenshan Hospital between February 19, 2020, and April 10, 2020. The mean (standard deviation) age of these patients was 63.13 (14.14), and 86 of them were women (55.13%). At hospital admission, 59 patients were severe, and 25 patients had severe complications. The overall mortality was 6.3% (Table 1).

Feature selection and radiomic signature building

For each ROI, a total of 833 quantitative features were extracted. Using an ICC of 0.80 as a cut-off for determining good reproducibility, a total of 257 radiomic features were selected for the next assessment. As shown in Supplementary Table 1, almost all the P values of the paired t-test for radiomic features for all 257 radiomic features were larger than 0.05. After applying the LASSO logistic algorithm, 60 radiomic features were used to develop all the radiomic models.

As shown in Table 2, the AUC of each model demonstrated good predictive accuracy (death model, 0.846; stage model, 0.918; complications model, 0.919; ARDS model, 0.852). After finding the optimal cut-off for each outcome, the respective accuracy, sensitivity, and specificity were 0.854, 0.700, and 0.864 for the prediction of death of COVID-19 patients; 0.814, 0.949, and 0.732 for the prediction of higher-stage COVID-19; 0.846, 0.920, and 0.832 for the prediction of complications of COVID-19 patients; and 0.814, 0.818, and 0.814 for ARDS of COVID-19 patients. The AUCs after bootstrapping were 0.846 for the death prediction model, 0.919 for the stage prediction model, 0.919 for the complications prediction model, and 0.853 for the ARDS prediction model in the internal validation, which indicates that the models were stable. The DCA for the four radiomic models with different endpoints is presented in Figure 2 and shows good performance in terms of clinical application.

We next explored the clinical utility of the addition of the radiomics signature for each outcome to the models with only clinical data included. As shown in Table 3, the AUCs of the clinical models were 0.728, 0.952, 0.726, and 0.861 for the higher stage, death, complications, and ARDS prediction models, respectively. After combining the radiomics signatures and clinical parameters, the AUCs of the mixed models were 0.925, 0.990, 0.929, and 0.903 for the higher stage, death, complications, and ARDS prediction models, respectively. The AUCs of the mixed models were higher than the clinical models. In addition, a significantly increased NRI (stage: P < 0.001; death: P = 0.013; complications: P < 0.001; ARDS: P < 0.001) was found for the mixed models compared with the clinical models.

Discussion

In this study, we described a prediction model for COVID-19 based on radiomic signatures. Based on the first CT scan after hospitalization, we can predict the prognosis of these patients early with high accuracy and intervene where necessary.

COVID-19 can influence several tissues and lead to organ failure rapidly. It has a poor prognosis and a high mortality rate. A chest CT combined with a positive molecular PCR test is the most important diagnostic method for COVID-19. Compared with tests conducted in standard laboratories, the CT scan procedure has a faster turnaround time and can provide more detailed information regarding lung damage severity and acute respiratory failure.^18,19 Features of CT images can present with ground-glass opacities, linear opacities, consolidation, bronchial wall thickening, lymph node enlargement, pericardial effusion, or pleural effusion. However, the CT characteristics in some stages are somewhat similar, such as in severe and critical cases. Therefore, a single qualitative radiological diagnosis cannot fully meet our needs to predict the prognosis of the disease. Radiomics features can quantitatively reflect the invisible details of the lesions. First-order features (e.g., entropy, skewness, and kurtosis) describe the distribution of the values of individual voxels without concern for spatial relationships. Second-order (texture) features describe the statistical interrelationships between voxels with similar (or dissimilar) contrast values. Higher-order statistical methods impose filter grids on the image to extract repetitive or non-repetitive patterns. For instance, among the final selected features, firstorder_10Percentile indicated the 10^th percentile of intensity in the ROI, which may reflect the relationship between the density of lesions and the disease grade.

Several studies on CT radiomics and the deep-learning technique have shown the efficiency of a rapid diagnosis of COVID-19. In a large cohort of 3,777 patients, the artificial intelligence diagnostic model can differentiate NCP from other common pneumonia with 92.49% accuracy, 94.93% sensitivity, 91.13% specificity, and an area under the ROC curve of 0.9797.⁶ Another deep-learning artificial intelligence-enabled rapid diagnosis system also showed a clinical benefit. However, studies focusing on prognosis prediction using quantitative image features are rare. Our research was the first study to investigate the role of CT radiomics in predicting the prognosis of patients with COVID-19. The AUCs of each model demonstrated good predictive accuracy (0.85–0.92). The DCA also indicated a good performance in terms of clinical application.

Several retrospective cohort studies have described the multi-organ damage caused by COVID-19, including respiratory, cardiovascular, digestive, urinary, endocrine, and nervous system damage.^20,21 Accurate predictive models are needed to identify the risk of patients experiencing a poor clinical outcome and plan early intervention to improve outcomes. Previous studies have found several variables that are risk factors for a severe prognosis related to COVID-19 and have built effective prediction models for patient management.^22,23 The following factors contain comprehensive clinical data: chest radiography abnormality, age, interleukin-6, dyspnea, number of comorbidities, cancer history, lower lymphocyte count, higher lactate dehydrogenase neutrophil-to-lymphocyte ratio, lactate dehydrogenase, creatinine, and direct bilirubin. However, these data rely on large data collection samples and patient follow ups for the entire study, which might lead to economic issues. In our preliminary study, the first CT image on arrival at the medical center could bring us more information than chest lesions. An important advancement in the use of imaging is assisting clinical management in identifying high-risk groups and intervening early to reduce mortality. However, the lack of widely used CT scanning equipment and experienced radiologists might affect the clinical application of these prediction models. Similar to previous research, the data models used in the present research relied on accurate labeling by professional radiologists. Moreover, the clinical characteristics and outcomes were estimated by the expert radiologists for the description of the state of the patient but did not consider the real severity.⁵ One limitation of this study is the small sample size for validation and the use of patients in the same country, which could cause bias. This retrospective study could also contain missing data. Additional prospective global multi-center validation studies of COVID-19 are recommended.

In conclusion, the radiomic signature provided vital information for predicting the prognosis of COVID-19. We built a model consisting of a radiomic signature that had maximum accuracy in the prediction of the prognosis. Our study provided vital insight into important preoperative clinical decisions and is expected to be applied in multiple medical centers to optimize future diagnoses and treatments.

Conflict of interest disclosure

The authors declared no conflicts of interest.

Funding

This study was supported by Shanghai Science and Technology Commission Clinical Research Project (grant number: 19411951500); Shanghai Sailing Program (grant no. 20YF1436300).