Development And Validation of a Risk Scoring System to Predict Pneumothorax in CT-Guided Percutaneous Transthoracic Needle Biopsy


 Objective: To develop and validate a risk scoring system using variables easily obtained for the prediction of pneumothorax in CT-guided percutaneous transthoracic needle biopsy (PTNB).Methods: The derivation cohort was comprised of 1001 patients who underwent CT-guided PTNB. Multivariate logistic regression was used to identify risk factors for pneumothorax, which were treated as the foundation to develop the risk scoring system. To validate the system, a validation cohort group of 230 patients was enrolled.Results: Age, puncture times, puncture depth, smoking index, number of specimens, bleeding from the needle path, and lobular lesion were identified as risk factors in the derivation cohort. A risk scoring system (Hosmer-Lemeshow goodness-of-fit test p =0.33) was developed. The area under the receiver operating characteristic curve (AUROC) was 0.601 by using the risk score system. This risk score system demonstrated a better diagnostic effect with increasing age. In the group of patients older than 80 years, the AUROC was 0.76, showing good predictive power. This risk scoring system was confirmed in the validation cohort with an AUROC of 0.736.Conclusion: This scoring system has a good predictive effect in both derivation and validation cohort.


Introduction
Percutaneous transthoracic needle biopsy (PTNB) has been widely used to obtain lung lesion specimens because of its high diagnostic accuracy rate (> 90%) and low complication rate (1,2) . Pneumothorax and pulmonary hemorrhage are the two most common complications caused by PTNB (3) . Pneumothorax is a common but potentially dangerous complication. In previous studies, the average rate of pneumothorax episodes was reported to range from 1.1 to 29.8% (2,4) , while the development of severe pneumothorax post-biopsy requiring intervention occurred in 1%-14.2% of patients undergoing CT-guided PTNB (2) . The risk factors for pneumothorax have been less studied (1)(2)(3)(4) . An evidence-based and easily available risk scoring system for predicting pneumothorax could be valuable for interventionists who perform PTNB.

Methods
This study was designed as a retrospective single-center study and was conducted in accordance with the Health Insurance Portability and Accountability Act. This retrospective study was approved by the Institutional Review Board Committee of the hospital. Patient informed consent requirements were waived.

Patients and definitions
Patients were enrolled if they underwent CT-guided PTNB between September 11, 2019, and August 30, 2021. Repeated PTNBs were recognized as new procedures in this study.

Biopsy procedures
The procedure was conducted under CT guidance. All PTNBs were performed using a co-axial needle system with 18-gauge automated biopsy needles (Huaxing Medical Devices, China). Basically, there were two steps during the procedure. First, put the outer sheath into the lesion. Second, punctured the lesion. During procedure breath-holding was not required.

Assessment of pneumothorax
Pneumothorax was assessed by an on-site CT scan. Cases with delayed pneumothorax were also included, which was de ned as pneumothorax occurring within 24 hours after the procedure. Severe pneumothorax was de ned as 2 cm depth from apex to cupola distance or interpleural distance at level of the hilum or even not meet the standard but with the obvious breathless symptom which need for thoracic closed drainage.

Covariates
A total of 47 variables were extracted, including patient-related variables, procedure-related variables, and lesion-related variables (radiomic feature extraction).
The patient-related variables included patient demographics, comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, history of tumor), smoking history (packyear), and body mass index (BMI).
The lesion-related variables were the following: location (divided into right/left upper, middle, and lower zones); central or peripheral; size (largest diameter); lesion characteristics (nodules or masses were de ned based on the largest diameter that was larger than 3 cm; patches were de ned as not quali ed to be a mass or nodule); pure ground glass, solid, subsolid, or cavitary lesions; subsolid lesions were de ned as containing a mixture of discrete solid and ground-glass areas; pleural adhesions, pleural effusion, lobular lesions; bone metastasis; pericardial effusion; lesion enhancement (mild/moderate/strong), feasibility (de ned as an increase of at least 10 Houns eld units after contrast administration for each degree); and the histopathological biopsy results.
Procedure-related variables included the patient's position (supine or prone), number of lesion puncture, number of specimens, maximum puncture depth and number of CT scanning during the procedure. After the biopsy, the CT images were taken to detect procedure-related complications, including lung tissue bleeding and pneumothorax.

Statistical analysis
Data were analyzed using statistical software (R 4.2; IBM Corp., Armonk, NY). Data are reported as the mean and standard deviation (SD) or as the median and interquartile range, as appropriate. The Gower distance, partitioning around medoids (PAM), and silhouette width were evaluated in order to cluster the data of mixed types including all kinds of variables (continuous, binary or qualitative) (5)(6)(7) .
The risk model was developed with the following steps. First, univariate analysis was performed to identify variables associated with pneumothorax caused by PTNB. Second, variables with a p value of <0.2 in the rst step were included in a stepwise multivariate logistic regression analysis model to identify independent risk factors associated with pneumothorax (5) . The probability used for stepwise removal was 0.05 or less for entry and 0.05 or less for removal. Then, we obtained a risk model and a forest tree plot. The area under the receiver operating characteristic curve (AUROC) was applied to assess the ability to predict pneumothorax (8) . The Hosmer-Lemeshow test was carried out to evaluate the goodness of t for the predicting model (p > 0.05). Third, we followed the method proposed by Sullivan et al. and Jun Duan et al. to create the risk scoring system (9,10) . The variables in the prediction model were classi ed into clinically meaningful categories, and the midpoint was calculated. In each category, we set the lowest risk for pneumothorax as the within-group reference, which was assigned zero points; then, the weight in each category was calculated. Finally, one point was assigned to the category with the lowest weight, which was set as the betweengroup reference. The value of that weight in the other category divided by the between-group reference was then calculated, which was rounded off to the nearest integer as the assigned points. The risk scale for pneumothorax was the sum of the points. Then, the total points associated with each category of each risk factor were calculated. The risk scoring system was then applied to the training and validation data. AUROC was applied for internal and external validation. The cutoff value was determined based on the AUROC.
The sample size was calculated by Buderer's formula (11) . Based on clinical experience and a literature review, we estimated that the risk scoring system for pneumothorax achieved a 70% sensitivity and a 90% speci city. The average prevalence of pneumothorax was approximately 15% in previous studies (2,4,12) . We chose α = 0.05 and maximum marginal error of estimate = 5%. Thus, a minimal sample size of 807 cases was required in derivation cohorts. The validation cohort was a prospective aspect on this retrospective study. Figure 1 shows ow chart of this study.

Results
General clinical data A total of 1001 patients were enrolled in the test cohort from September 2019 to February 2021, while another 230 patients were enrolled in the validation cohort from March 2021 to August 2021. The baseline characteristics of patients who underwent PTNB are shown in Table 1. Among the derivation cohort, 611 were men (61%), and the mean age of all patients was 63.81±12, ranging from 14 to 90. In the validation cohort, 81 patients were men (70%), and the mean age of all patients was 66.56±10.86, ranging from 41 to 87.
According to the pathology reports, 660 were malignancies, 85 were confirmed as tuberculosis (n=83) and mycosis (n=2), 224 were nonspecific inflammations and 32 were judged to be nondiagnostic in the derivation cohort. One hundred forty were malignancies, 18 were confirmed as tuberculosis (n=10) and mycosis (n=8), 60 were nonspecific inflammations and 16 were judged to be nondiagnostic in the validation cohort.
Of all the 169 pneumothorax cases among the patients, (16.9%) occurred after PTNB, while 12 cases (7%) were de ned as severe pneumothorax that needed closed thoracic drainage in the derivation cohort. The corresponding rate in the validation cohort was 36 pneumothorax cases (15.7%), and 9 case (3.9%) needed intervention. The overall rate of hemoptysis was 3.4% (n=34), with most cases having volumes of hemoptysis < 20 mL. One patient died because of massive hemoptysis in the derivation cohort. The corresponding hemoptysis rate in the validation cohort was 3%(n=7) and no one died.

Clustering for pneumothorax
After calculating the silhouette width for clusters ranging from 2 to 10 for the PAM algorithm, we noticed that 2 clusters yielded the highest value ( gure 2A). After running the algorithm and selecting two clusters ( gure 2B), we interpreted the clusters by running a summary on each cluster ( gure 2 B). Based on these results, it seemed that Cluster 1 was mainly younger/more number of lesion puncture/higher puncture depth with a heavy smoking index. Cluster 2, on the contrary, was mainly older/fewer number of lesion puncture/lower puncture depth with a light smoking index.
Derivation of a risk prediction model for pneumothorax and development of a risk scoring system.
The nal risk prediction model with an AUROC of 0.673 ( gure 3A) contained seven variables, including age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion (table 1), that were all statistically signi cantly associated with pneumothorax (p < 0.05) in the regression analyses in the derivation cohort, which could be visualized by a forest tree plot ( gure 2C). We used these seven variables to develop a risk scale to predict pneumothorax. The categories and assignment of points in each variable are summarized in Tables 2 and 3. The scale ranged from -13 to 26 points. Risks associated with point totals are demonstrated in Table 4.   Validation of the risk scoring system

Internal validation
In the derivation cohorts, the risk scoring system could only achieve an AUROC of 0.601 in the evaluation of predicting pneumothorax ( gure 3B). However, the subgroup analysis demonstrated that with increasing age, the predictive effect of the risk scoring system was high. In the group with patients older than 80 years, the AUROC was as high as 0.766 ( gure 3C/D). To obtain a high predictive effect, we excluded protective predictors, such as the number of specimens, needle path bleeding, and lobularity of the lesion, one by one to test whether the prediction effect could be higher, but combining all predictors had the highest AUROC ( gure 3E). The predictions made from the risk model were in alignment with the observed outcomes suggested by Hosmer-Lemeshow tests (P=0.33). The cutoff score suggested by Youden's index was 15 points in the derivation cohorts.

External validation
In the validation cohort, the risk scoring system obtained an AUROC of 0.736 in the evaluation of predicting pneumothorax ( gure 3F). The cutoff score suggested by Youden's index was 11 points in the validation cohort.

Discussion
This is a retrospective single-center study attempting to identify risk factors and then set up a risk score system for CT-guided PTNB-associated pneumothorax complications in a tertiary hospital. In our study, the incidence of pneumothorax and severe pneumothorax were 16.9 and 1.2% in the derivation cohorts, respectively. These pneumothorax complication rates in our study were in line with the results of previous studies, implying that the results of our study can be extrapolated to other studies.
Clustering allows us to better understand how a sample might be comprised of distinct subgroups given a set of variables. After calculating the silhouette width for clusters for the PAM algorithm, we observed 2 clusters of pneumothorax complications related to PTNB. To the best of our knowledge, this is the rst study to use this method to cluster and visualize these clusters. This result provides us with a new angle to better understand the complications of pneumothorax.
The identi ed variables further served as the foundation for the development of a risk scoring system connecting CT-guided PTNB. This risk scoring system takes into account age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion, which are easily obtained. To the best of our knowledge, this is the rst study in the development and validation of a risk score system for predicting pneumothorax. Among those factors, the number of specimens, needle path bleeding, and lobular lesion were protective factors for pneumothorax, which could not be excluded from the model for obtaining the highest AUROC. Three things interest us among those factors. First, the number of lesion puncture was a risk factor, but the number of specimens was a protective factor. It is understood that each puncture may not obtain one specimen because off-target puncture sometimes occurs, which results in damage to the normal lung and is not helpful for diagnosis. We hypothesize more numbers of lesion puncture mean more destruction, especially puncture into normal lung. Second, more specimens in the lesion do not necessarily mean more destruction of the normal lung, and more specimens result in more tissue bleeding, which may stop the development of pneumothorax, which is why it is a protective factor for pneumothorax. Third, it was noticed that pneumothorax usually happened at two stages during the procedure. First stage was the moving forward of the outer sheath to reach the lesion, while second was the lesion puncture. We tried to use parameters including lesion location, puncture depth and number of CT scanning to mimic the rst procedure stage, but these parameters were hard to describe the complexity such as the angle adjusting during the outer sheath moving forward.
We used seven variables to develop this scoring system to predict pneumothorax in 1001 patients and validated it in another 230 patients. All the variables were readily available; among these, two were general information (age, smoking index), four were procedure-related variables (number of lesion puncture, puncture depth, number of specimens, needle path bleeding), and one was a radiologic feature (lobularity of the lesion). We found that the diagnostic accuracy of pneumothorax was acceptable in both the test and validation cohorts, while in the older age group, the diagnostic power was better. We therefore conclude that this scoring system is a good tool to help clinical practitioners predict PTNB-related pneumothorax. We developed the scoring system using this presentation format because of its straightforward use in clinical practice. During the CT-guided PTNB procedure, patients were scored 11 or higher, and we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to prevent subsequent pneumothorax. Our study has a larger sample size than the calculated sample size that was needed but had a relatively low incidence in the outcome. Thus, the possibility of type II errors in this study may not be low, and the results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggested an appropriate t of the scoring system.
There were some limitations that need to be mentioned. First, due to its retrospective design, information bias might have occurred during the outcome ascertainment. Second, the sample size was small in the higher age group, such as the group of patients aged older than 70 or 80 years, and the e cacy of the scoring system in these patients may be skewed. Third, other centre may use smaller gauge needle and this may make a difference to pneumothorax risk. Fourth, external validation by a future prospective study would be warranted, as it would make this scoring system more relevant for clinical scenarios.

Figure 1
Flow chart of this study.

Figure 2
A: Two clusters yield the highest value after calculating silhouette width for clusters ranging from 2 to 10 for the PAM algorithm.
B: The plot shows the two well-separated clusters of pneumothorax that PAM was able to detect.
C: Logistic regression plot of odds ratios and 95% confidence intervals. Number of specimens, needle path bleeding, and lobular of the lesion was protective factors of pneumothorax.