Development And Validation of a Risk Scoring System to Predict Pneumothorax in CT-Guided Percutaneous Transthoracic Needle Biopsy

Abstract

Objective: To develop and validate a risk scoring system using variables easily obtained for the prediction of pneumothorax in CT-guided percutaneous transthoracic needle biopsy (PTNB).

Methods: The derivation cohort was comprised of 1001 patients who underwent CT-guided PTNB. Multivariate logistic regression was used to identify risk factors for pneumothorax, which were treated as the foundation to develop the risk scoring system. To validate the system, a validation cohort group of 230 patients was enrolled.

Results: Age, puncture times, puncture depth, smoking index, number of specimens, bleeding from the needle path, and lobular lesion were identified as risk factors in the derivation cohort. A risk scoring system (Hosmer-Lemeshow goodness-of-fit test p =0.33) was developed. The area under the receiver operating characteristic curve (AUROC) was 0.601 by using the risk score system. This risk score system demonstrated a better diagnostic effect with increasing age. In the group of patients older than 80 years, the AUROC was 0.76, showing good predictive power. This risk scoring system was confirmed in the validation cohort with an AUROC of 0.736.

Conclusion: This scoring system has a good predictive effect in both derivation and validation cohort.

Introduction

Percutaneous transthoracic needle biopsy (PTNB) has been widely used to obtain lung lesion specimens because of its high diagnostic accuracy rate (> 90%) and low complication rate^{(1, 2)}. Pneumothorax and pulmonary hemorrhage are the two most common complications caused by PTNB ⁽³⁾. Pneumothorax is a common but potentially dangerous complication. In previous studies, the average rate of pneumothorax episodes was reported to range from 1.1 to 29.8% ^{(2, 4)}, while the development of severe pneumothorax post-biopsy requiring intervention occurred in 1%-14.2% of patients undergoing CT-guided PTNB ⁽²⁾. The risk factors for pneumothorax have been less studied ^(1-4). An evidence-based and easily available risk scoring system for predicting pneumothorax could be valuable for interventionists who perform PTNB.

Methods

This study was designed as a retrospective single-center study and was conducted in accordance with the Health Insurance Portability and Accountability Act. This retrospective study was approved by the Institutional Review Board Committee of the hospital. Patient informed consent requirements were waived.

Patients and definitions

Patients were enrolled if they underwent CT-guided PTNB between September 11, 2019, and August 30, 2021. Repeated PTNBs were recognized as new procedures in this study.

Biopsy procedures

The procedure was conducted under CT guidance. All PTNBs were performed using a co-axial needle system with 18-gauge automated biopsy needles (Huaxing Medical Devices, China). Basically, there were two steps during the procedure. First, put the outer sheath into the lesion. Second, punctured the lesion. During procedure breath-holding was not required.

Assessment of pneumothorax

Pneumothorax was assessed by an on-site CT scan. Cases with delayed pneumothorax were also included, which was defined as pneumothorax occurring within 24 hours after the procedure. Severe pneumothorax was defined as 2 cm depth from apex to cupola distance or interpleural distance at level of the hilum or even not meet the standard but with the obvious breathless symptom which need for thoracic closed drainage.

Covariates

A total of 47 variables were extracted, including patient-related variables, procedure-related variables, and lesion-related variables (radiomic feature extraction).

The patient-related variables included patient demographics, comorbidities (diabetes, hypertension, chronic obstructive pulmonary disease, history of tumor), smoking history (packyear), and body mass index (BMI).

The lesion-related variables were the following: location (divided into right/left upper, middle, and lower zones); central or peripheral; size (largest diameter); lesion characteristics (nodules or masses were defined based on the largest diameter that was larger than 3 cm; patches were defined as not qualified to be a mass or nodule); pure ground glass, solid, subsolid, or cavitary lesions; subsolid lesions were defined as containing a mixture of discrete solid and ground-glass areas; pleural adhesions, pleural effusion, lobular lesions; bone metastasis; pericardial effusion; lesion enhancement (mild/moderate/strong), feasibility (defined as an increase of at least 10 Hounsfield units after contrast administration for each degree); and the histopathological biopsy results.

Procedure-related variables included the patient’s position (supine or prone), number of lesion puncture, number of specimens, maximum puncture depth and number of CT scanning during the procedure. After the biopsy, the CT images were taken to detect procedure-related complications, including lung tissue bleeding and pneumothorax.

Statistical analysis

Data were analyzed using statistical software (R 4.2; IBM Corp., Armonk, NY). Data are reported as the mean and standard deviation (SD) or as the median and interquartile range, as appropriate. The Gower distance, partitioning around medoids (PAM), and silhouette width were evaluated in order to cluster the data of mixed types including all kinds of variables (continuous, binary or qualitative)^(5-7).

The risk model was developed with the following steps. First, univariate analysis was performed to identify variables associated with pneumothorax caused by PTNB. Second, variables with a p value of <0.2 in the first step were included in a stepwise multivariate logistic regression analysis model to identify independent risk factors associated with pneumothorax⁽⁵⁾. The probability used for stepwise removal was 0.05 or less for entry and 0.05 or less for removal. Then, we obtained a risk model and a forest tree plot. The area under the receiver operating characteristic curve (AUROC) was applied to assess the ability to predict pneumothorax⁽⁸⁾. The Hosmer–Lemeshow test was carried out to evaluate the goodness of fit for the predicting model (p > 0.05). Third, we followed the method proposed by Sullivan et al. and Jun Duan et al. to create the risk scoring system ^{(9, 10)}. The variables in the prediction model were classified into clinically meaningful categories, and the midpoint was calculated. In each category, we set the lowest risk for pneumothorax as the within-group reference, which was assigned zero points; then, the weight in each category was calculated. Finally, one point was assigned to the category with the lowest weight, which was set as the between-group reference. The value of that weight in the other category divided by the between-group reference was then calculated, which was rounded off to the nearest integer as the assigned points. The risk scale for pneumothorax was the sum of the points. Then, the total points associated with each category of each risk factor were calculated. The risk scoring system was then applied to the training and validation data. AUROC was applied for internal and external validation. The cutoff value was determined based on the AUROC.

The sample size was calculated by Buderer’s formula ⁽¹¹⁾. Based on clinical experience and a literature review, we estimated that the risk scoring system for pneumothorax achieved a 70% sensitivity and a 90% specificity. The average prevalence of pneumothorax was approximately 15% in previous studies ^{(2, 4, 12)}. We chose α = 0.05 and maximum marginal error of estimate = 5%. Thus, a minimal sample size of 807 cases was required in derivation cohorts. The validation cohort was a prospective aspect on this retrospective study. Figure 1 shows flow chart of this study.

Results

General clinical data

A total of 1001 patients were enrolled in the test cohort from September 2019 to February 2021, while another 230 patients were enrolled in the validation cohort from March 2021 to August 2021. The baseline characteristics of patients who underwent PTNB are shown in Table 1. Among the derivation cohort, 611 were men (61%), and the mean age of all patients was 63.81±12, ranging from 14 to 90. In the validation cohort, 81 patients were men (70%), and the mean age of all patients was 66.56±10.86, ranging from 41 to 87.

According to the pathology reports, 660 were malignancies, 85 were confirmed as tuberculosis (n=83) and mycosis (n=2), 224 were nonspecific inflammations and 32 were judged to be nondiagnostic in the derivation cohort. One hundred forty were malignancies, 18 were confirmed as tuberculosis (n=10) and mycosis (n=8), 60 were nonspecific inflammations and 16 were judged to be nondiagnostic in the validation cohort.

Of all the 169 pneumothorax cases among the patients, (16.9%) occurred after PTNB, while 12 cases (7%) were defined as severe pneumothorax that needed closed thoracic drainage in the derivation cohort. The corresponding rate in the validation cohort was 36 pneumothorax cases (15.7%), and 9 case (3.9%) needed intervention. The overall rate of hemoptysis was 3.4% (n=34), with most cases having volumes of hemoptysis < 20 mL. One patient died because of massive hemoptysis in the derivation cohort. The corresponding hemoptysis rate in the validation cohort was 3%(n=7) and no one died.

Clustering for pneumothorax

After calculating the silhouette width for clusters ranging from 2 to 10 for the PAM algorithm, we noticed that 2 clusters yielded the highest value (figure 2A). After running the algorithm and selecting two clusters (figure 2B), we interpreted the clusters by running a summary on each cluster (figure 2 B). Based on these results, it seemed that Cluster 1 was mainly younger/more number of lesion puncture/higher puncture depth with a heavy smoking index. Cluster 2, on the contrary, was mainly older/fewer number of lesion puncture/lower puncture depth with a light smoking index.

Derivation of a risk prediction model for pneumothorax and development of a risk scoring system.

The final risk prediction model with an AUROC of 0.673 (figure 3A) contained seven variables, including age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion (table 1), that were all statistically significantly associated with pneumothorax (p < 0.05) in the regression analyses in the derivation cohort, which could be visualized by a forest tree plot (figure 2C). We used these seven variables to develop a risk scale to predict pneumothorax. The categories and assignment of points in each variable are summarized in Tables 2 and 3. The scale ranged from -13 to 26 points. Risks associated with point totals are demonstrated in Table 4.

Table 1 Demographics of patients with or without pneumothorax undergone CT-guided PTNB in derivation and validation cohort. Data are presented as the mean ± SD or as a number with/without the percentage in parenthesis, as appropriate. Number of lesion puncture, Puncture number. Number of specimens, Specimen. Needle path bleeding, tissue bleeding. Lobularity of the lesion, lobularity. NA, not available.

Table 2 Scores associated with each of the categories of the risk factors. βi, regression units; B, constant. Number of lesion puncture, Puncture number. Number of specimens, Specimen. Needle path bleeding, tissue bleeding. lobularity of the lesion, lobularity

Table 3 Risk scoring system for predicting pneumothorax in PTNB.

Table 4 Scores associated with estimate of risk.

Validation of the risk scoring system

Internal validation

In the derivation cohorts, the risk scoring system could only achieve an AUROC of 0.601 in the evaluation of predicting pneumothorax (figure 3B). However, the subgroup analysis demonstrated that with increasing age, the predictive effect of the risk scoring system was high. In the group with patients older than 80 years, the AUROC was as high as 0.766 (figure 3C/D). To obtain a high predictive effect, we excluded protective predictors, such as the number of specimens, needle path bleeding, and lobularity of the lesion, one by one to test whether the prediction effect could be higher, but combining all predictors had the highest AUROC (figure 3E). The predictions made from the risk model were in alignment with the observed outcomes suggested by Hosmer-Lemeshow tests (P=0.33). The cutoff score suggested by Youden’s index was 15 points in the derivation cohorts.

External validation

In the validation cohort, the risk scoring system obtained an AUROC of 0.736 in the evaluation of predicting pneumothorax (figure 3F). The cutoff score suggested by Youden’s index was 11 points in the validation cohort.

Discussion

This is a retrospective single-center study attempting to identify risk factors and then set up a risk score system for CT-guided PTNB-associated pneumothorax complications in a tertiary hospital. In our study, the incidence of pneumothorax and severe pneumothorax were 16.9 and 1.2% in the derivation cohorts, respectively. These pneumothorax complication rates in our study were in line with the results of previous studies, implying that the results of our study can be extrapolated to other studies.

Clustering allows us to better understand how a sample might be comprised of distinct subgroups given a set of variables. After calculating the silhouette width for clusters for the PAM algorithm, we observed 2 clusters of pneumothorax complications related to PTNB. To the best of our knowledge, this is the first study to use this method to cluster and visualize these clusters. This result provides us with a new angle to better understand the complications of pneumothorax.

The identified variables further served as the foundation for the development of a risk scoring system connecting CT-guided PTNB. This risk scoring system takes into account age, number of lesion puncture, puncture depth, smoking index, number of specimens, needle path bleeding, and lobular lesion, which are easily obtained. To the best of our knowledge, this is the first study in the development and validation of a risk score system for predicting pneumothorax. Among those factors, the number of specimens, needle path bleeding, and lobular lesion were protective factors for pneumothorax, which could not be excluded from the model for obtaining the highest AUROC. Three things interest us among those factors. First, the number of lesion puncture was a risk factor, but the number of specimens was a protective factor. It is understood that each puncture may not obtain one specimen because off-target puncture sometimes occurs, which results in damage to the normal lung and is not helpful for diagnosis. We hypothesize more numbers of lesion puncture mean more destruction, especially puncture into normal lung. Second, more specimens in the lesion do not necessarily mean more destruction of the normal lung, and more specimens result in more tissue bleeding, which may stop the development of pneumothorax, which is why it is a protective factor for pneumothorax. Third, it was noticed that pneumothorax usually happened at two stages during the procedure. First stage was the moving forward of the outer sheath to reach the lesion, while second was the lesion puncture. We tried to use parameters including lesion location, puncture depth and number of CT scanning to mimic the first procedure stage, but these parameters were hard to describe the complexity such as the angle adjusting during the outer sheath moving forward.

We used seven variables to develop this scoring system to predict pneumothorax in 1001 patients and validated it in another 230 patients. All the variables were readily available; among these, two were general information (age, smoking index), four were procedure-related variables (number of lesion puncture, puncture depth, number of specimens, needle path bleeding), and one was a radiologic feature (lobularity of the lesion). We found that the diagnostic accuracy of pneumothorax was acceptable in both the test and validation cohorts, while in the older age group, the diagnostic power was better. We therefore conclude that this scoring system is a good tool to help clinical practitioners predict PTNB-related pneumothorax. We developed the scoring system using this presentation format because of its straightforward use in clinical practice. During the CT-guided PTNB procedure, patients were scored 11 or higher, and we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to prevent subsequent pneumothorax. Our study has a larger sample size than the calculated sample size that was needed but had a relatively low incidence in the outcome. Thus, the possibility of type II errors in this study may not be low, and the results of the statistical evaluation can be sensitive to the differences between the observed and predicted values. After all, the results in both cohorts suggested an appropriate fit of the scoring system.

There were some limitations that need to be mentioned. First, due to its retrospective design, information bias might have occurred during the outcome ascertainment. Second, the sample size was small in the higher age group, such as the group of patients aged older than 70 or 80 years, and the efficacy of the scoring system in these patients may be skewed. Third, other centre may use smaller gauge needle and this may make a difference to pneumothorax risk. Fourth, external validation by a future prospective study would be warranted, as it would make this scoring system more relevant for clinical scenarios.

Conclusion

We established and validated a risk scoring system to predict pneumothorax in patients who underwent CT-guided PTNB. For patients who have high scores during the PTNB procedure, we recommend limiting the number of lesion puncture and choosing the shortest needle path to the lesion to lower the risk of pneumothorax.

Declarations

Approval was obtained from our Institutional Review Board for this manuscript.

All the authors are qualified for authorship and agree to submit this paper.

All data and materials are available.

All authors declare no conflict of interest.

There is no funding support for this research.

Doc Qiuhong Yang performed the puncture, collected the data and analyzed the data. Doc Lincheng Luo performed the puncture, collected and analyzed the data. Xinyi Peng was the radiologist, controlled the CT machine and collected the procedure-related data. Doc Hailong Wei performed the puncturing procedure. Doc Qun Yi proposed the idea, revised and translated the manuscript. Doc Wei Luo performed the puncturing procedure, analyzed the data, conducted the literature review and wrote the manuscript.

Acknowledgments: Not applicable.

References

1. Tian P, Wang Y, Li L, Zhou Y, Luo W, Li W. CT-guided transthoracic core needle biopsy for small pulmonary lesions: diagnostic performance and adequacy for molecular testing. J Thorac Dis. 2017;9:333–43.

2. Yang BR, Kim M-S, Park CM, Yoon SH, Chae KJ, Lee J (2020) Patterns of percutaneous transthoracic needle biopsy (PTNB) of the lung and risk of PTNB-related severe pneumothorax: A nationwide population-based study. PLoS ONE15(7): e0235599. .

3. Heerink WJ, de Bock GH, de Jonge GJ, Groen HJM, Vliegenthart R, Oudkerk M. Complication rates of CT-guided transthoracic lung biopsy: meta-analysis. Eur Radiol. 2017;27:138–48.

4. Wang Y, Jiang F, Tan X, Tian P. CT-guided percutaneous transthoracic needle biopsy for paramediastinal and nonparamediastinal lung lesions: Diagnostic yield and complications in 1484 patients. Medicine (Baltimore). 2016;95(31):e4460.

5. Sjolander A. Regression standardization with the R package stdReg. Eur J Epidemiol. 2016;31(6):563-74.

6. Irigoien I, Sierra B, Arenas C. ICGE: an R package for detecting relevant clusters and atypical units in gene expression. BMC Bioinformatics. 2012;13:30.

7. Rousseeuw PJ: Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis. Journal of Computational and Applied Mathematics 1987, 20: 53–65.

8. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.

9. Sullivan LM, Massaro JM, D'Agostino RB, Sr. Presentation of multivariate data for clinical use: The Framingham Study risk score functions. Stat Med. 2304;23(10):1631-60.

10. Duan J, Han X, Bai L, Zhou L, Huang S. Assessment of heart rate, acidosis, consciousness, oxygenation, and respiratory rate to predict noninvasive ventilation failure in hypoxemic patients. Intensive Care Med. 2017;43(2):192-9.

11. Buderer NM (1996) Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Acad Emerg Med 3:895–900.

12. Chen CH, Huang WM, Liang SH, Jhou ZY, Chen CW, Chien YC, et al. Does biopsy needle traversing through central portion of lesion increase the risk of hemoptysis during percutaneous transthoracic needle biopsy? Jpn J Radiol. 2018;36(3):231-7.