Reducing image artifacts in sparse projection CT using conditional generative adversarial networks

doi:10.21203/rs.3.rs-3304857/v1

Download PDF

Article

Reducing image artifacts in sparse projection CT using conditional generative adversarial networks

https://doi.org/10.21203/rs.3.rs-3304857/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 16 Feb, 2024

Read the published version in Scientific Reports →

You are reading this latest preprint version

Background: Reducing the amount of projection data in computed tomography (CT), called sparse-view CT, can reduce the exposure dose; however, image artifacts can occur. We quantitatively evaluated the effects of conditional generative adversarial networks (CGAN) on image quality restoration for sparse-view CT using simulated sparse projection images and compared them with autoencoder (AE) and U-Net models.

Methods: To simulate sparse-view CT, we acquired fan-beam projections at rotation angles of 1°, 2°, 5°, and 10° for the chest images (4250 slices). Four types of sinograms with different decimating projections were generated to simulate sparse-view CTs. The AE, U-Net, and CGAN models were trained using pairs of artifacts and original images, with 90% of all data used for training and the remaining for evaluation. Restoration of CT value was evaluated using mean error (ME) and mean absolute error (MAE). The image quality was evaluated using structural image similarity (SSIM) and peak signal-to-noise ratio (PSNR).

Results: Organ structures were restored up to a sparse projection of 2°; however, slight deformation in tumor and spine regions was observed, with a dispersed projection of over 5°. Image resolution decreased, and blurring occurred in AE and U-Net; therefore, large deviations in ME and MAE were observed in lung and air regions, and the results of SSIM and PSNR were degraded.

Conclusions: The CGAN demonstrated higher image reproducibility than AE and U-Net, particularly for accurate CT value restoration. However, over a decimation angle of 5°, the accuracy of the reconstruction of exact organ structures is limited.

Health sciences/Health care/Medical imaging

Health sciences/Health care/Medical imaging/Three dimensional imaging

Health sciences/Health care/Medical imaging/Tomography

Sparse projection

CGAN

Deep learning

Artifact reduction

Computed tomography (CT) has been widely applied for the clinical diagnosis of many organs, and its clinical benefits are indispensable in modern medical strategies. However, excessive X-ray doses increase the risk of tissue diseases and cancers, and the overuse of CT scans in medical environments raises concerns [1, 2]. Methods for achieving the lowest radiation dose are imperative for many CT tasks [3]. One easy way to handle this problem is to reduce the number of X-ray photons; however, this leads to an increase in image noise. Iterative reconstruction methods combine the statistical properties of the data in the image domain and the projection space to optimize the objective function and can remove noise and artifacts from low-dose CT images [3, 4]. This technique depends on the CT manufacturer’s specifications, which limit the clinical applications of conventional image-based denoising methods, such as total variation minimization [5]. Another effective strategy is to decrease the number of projections for a given trajectory, which is called the sparse-view CT technique [6–8]. Sparse projections cannot be implemented in conventional multidetector CTs because of the continuous helical acquisition of projection data. In contrast, sparse-view CT may can reduce the irradiated dose in cone-beam CT (CBCT)s equipped with dental CT, mammography tomosynthesis, and linear accelerator, and clinical benefits will be sufficient [9, 10]. Indeed, by reducing the number of projections in 4-demensional CBCT, it is possible to achieve a shorter imaging time. This potential reduction in imaging time can be realized by decreasing the amount of data acquisition and processing required, thereby improving the efficiency and practicality of 4D CBCT imaging. However, insufficient projection views lead to inaccurate image reconstruction and can result in prominent streak artifacts [11–13]. Therefore, various artifact reduction methods have been used to restore image quality with sparse projection data. Compressed-sensing-based regularization techniques have shown good performance for severely undersampled data [14, 15]. However, this method requires prior knowledge of the image to be restored, and it successfully removes local noise but does not remove linear artifacts that appear as large structures on limited-angle tomography [16].

Recently, with the development of deep learning in the medical imaging field, several approaches have been proposed to remove the noise associated with low-dose CT [17–19]. Various convolutional neural network (CNN)-based methods have been proposed ; however, they often suffer from disappearing edges and the introduction of image blurring [20, 21]. Moreover, the streaking artifacts caused by projection reduction are difficult to remove using conventional CNNs [22]. A generative adversarial network (GAN) enforces an inverse transformation and achieves highly accurate consistency when the underlying structures are similar even when mapped to nonlinear domains [23, 24]. In particular, the conditional generative adversarial network (CGAN) algorithm has shown a superior ability to synthesize images from uniform label maps and is used as a solution to image-to-image translation [25]. Therefore, the CGAN may be advantageous for reducing image artifacts in sparse-view CT using a huge paired supervised image with artifacts.

In this study, we investigated the effect of image artifact correction in sparse-view CT using the CGAN model on lung CT images based on numerous computational artifact simulation images. The accuracy of the streak artifact correction was evaluated by comparing and verifying its dependence on the number of sparse projection angles. The effect of image quality correction by the CGAN-based model was compared with that of the Autoencoder (AE) and U-Net models, which are wildly applied medical image correction, to clarify the image quality and improvement effect. The contribution of this study leases in demonstrating the feasibility of utilizing sparse projection CT for practical applications.

Image data acquisition

Lung CT images were used to train a deep learning model, and image artifacts in sparse-view CT images were simulated. The lung CT images were obtained from a publicly available dataset, the Cancer Imaging Archive, an open-access information source created by the US National Cancer Institute [26]. In these datasets, CT images were acquired using multidetector CT with helical scanning. The tube voltage was 120 kV, the thickness of each CT image 3 mm, the matrix size 512 × 512 pixels, and the field of view 45 × 45 cm. These images were corrected in 40 cases, and CT images were acquired without image artifacts using sparse projections.

Generation of simulated sparse projection image

Simulated artifact images of the sparse projection were generated by decimating the projection data from the original image. The original CT image was decimated and forward-projected to create sparse sinogram data by accumulating the CT values in each projection path. These sparse projection data were acquired at intervals of 1°, 2°, 5°, and 10°, and a sparse sinogram was obtained using these projection data. Then, simulated artifact images were created from these sparse sinograms using a fast Fourier transform with a Shepp and Logan filter. An overview of the generation of the simulated sparse projection image is shown in Fig. 1. In this study, we used a supervised image paired with the original and simulated images to train and evaluate the deep learning model.

Artifacts correction in sparse-view CT by deep learning

In each deep learning model, the training dataset comprised 4250 images with lung CT images, and various amounts of artifacts occurred at each dispersed projection angle. Therefore, the number of simulated artifact images was 4250 for each dispersed projection angle of 1°, 2°, 5°, and 10°. In each projection, 90% of all images were used for training, and the remaining 10% were used for image quality evaluation, which included randomly selected patient case data. In the input data, each pixel value is normalized to (0, 1) between the maximum and minimum intensities of the pixel value. An artifact-corrected image was generated based on the CGAN model. Moreover, the AE and U-Net models were used to generate artifact-corrected images for comparison. The details of these deep-learning models are presented below.

CGAN

The CGAN has two CNNs, one generator, and one discriminator, which work in opposite functions. During the training process, the generator attempts to produce an artifact-reduction image, whereas the discriminator networks attempt to distinguish the artificially created data as being either the original CT image or being artificially generated. In this study, the conditional label was an image pair of the original and simulated artifact images. Figure 2 presents a flowchart of the CGAN model and the architectures of the generator and discriminator. The modules of these models were in the form of a convolution-BatchNorm-ReLu. Patch-GAN was utilized in the discriminator, which classified either a true or generated image in each given patch. In addition, four convolutional layers were added to the discriminator architecture. The total loss function in CGAN training is as follows [25]:

$${L}_{cGAN}\left(G, D\right)={\mathbb{E}}_{x, y}\left[\text{log}D\left(x, y\right)\right]+{\mathbb{E}}_{x, z}\left[\text{log}\left(1-D\left(x, G\left(x, z\right)\right)\right)\right]$$

where the generator ($G)$ attempts to minimize the loss function (${L}_{cGAN}\left(G, D\right)$), whereas the discriminator ($D$) strives to maximize it to distinguish between $G\left(x, z\right)$ and real samples $y$. Furthermore, estimation error loss to the discriminator feedback is added for effective training of the generator. Therefore, the final objectives were as follows:

$${G}^{*}=arg\underset{G}{\text{min}}\underset{D}{\text{max}}{L}_{cGAN}\left(G, D\right)+\lambda {L}_{{L}_{1}}\left(G\right)$$

where ${L}_{{L}_{1}}\left(G\right)$ is an additional L1-norm-based loss function in the generator to get closer to the ground truth output, and $\lambda$ is a tunable parameter; in this study, we set this value to 100.

We conducted the experiments on a personal computer equipped with two GPUs (Quadro RTX 5000, NVIDIA Corporation) and a CPU (Intel Xeon Silver 4210R) with 96 GB of memory. Our algorithm was implemented using MATLAB 2022b (MathWorks Inc., Natick, MA, USA). The network was optimized using the Adam optimizer with a learning rate of 0.0002 for both the generator and discriminator networks. We used a batch size of one, and the training was stopped at 200 epochs. The loss curves decreased as the number of epochs increased and then remained almost unchanged as they approached 200 epochs.

AE

The AE model consists of an encoder-decoder process with four layers [27]. Each layer has modules of convolution-ReLu-maxPooling and reconstruction and creates the same size of output as the input image. The filter size was 3×3, and the initial number of filters was 32. In this model, the input image is a simulated artifact image with sparse projection, and the output is the original image without artifacts. This network learns to match the input and output and extracts only the important information necessary for restoration from the training data. The network was optimized using the Adam optimizer with a learning rate of 0.001 and a batch size of 4. The training was stopped after 200 epochs. The loss curves decrease continuously and remain almost unchanged after 200 epochs.

U-Net

The U-Net model has four encoder and decoder depths, and each layer has convolution-ReLu-maxPooling modules. Skip connections are utilized in the channels at each layer to restore the overall location information while preserving local features [28]. The filter size was 3×3, and the number of first filters was 64. This network comprises a context aggregation pathway that represents the input and a localization pathway that recombines these representations with shallower features. The input image was a simulated sparse artifact image, and the output image was the original one paired with the input. The learning rate and batch size are similar to those of the AE model. The training was stopped at 200 epochs, and the loss curves increased as the number of epochs increased and then remained almost unchanged as they approached 200 epochs.

Image quality evaluation

ME and MAE

To compare the accuracy of artifact correction in sparse-view CT using each deep learning model, synthesized images, which were not included in the training data, were generated using the trained CGAN, AE, and U-Net models. The quality of the corrected images was quantitatively evaluated by comparison with the original CT images. To evaluate the differences in the CT number with respect to the original images, we set regions of interest (ROIs) in the lung, soft tissue, bone, and air regions and measured the mean error (ME) and mean absolute error (MAE) as follows:

$$ME\left(X,Y\right)=\frac{1}{M\times N}{\sum }_{i=1}^{M}\sum _{j=1}^{N}\left(X\right(i,j)-Y(i, j))$$

$$MAE\left(X,Y\right)=\frac{1}{M\times N}{\sum }_{i=1}^{M}\sum _{j=1}^{N}\left|X(i, j)-Y(i, j)\right|$$

where $M$ and $N$ indicate the width and height of the pixels within an ROI, respectively, $X(i, j)$ is the CT number of the i-th and j-th pixels in the spared projection image or artifact reduction image, and $Y(i, j)$ is the CT number of the i-th and j-th pixels in the original CT image. The sizes of the ROIs were 40 × 40, 40 × 40, 15 × 15, and 50 × 50 pixels in the lung, soft tissue, bone, and air regions, respectively. An example of these ROI positions is shown in Fig. 3.

SSIM and PSNR

To evaluate the differences in overall image quality, the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) in the artifact-reduced image were calculated based on the original CT image [29, 30]. The SSIM of images $X$ and $Y$ is defined as follows:

$$SSIM\left(X, Y\right) =\frac{\left(2{\mu }_{X}{\mu }_{Y}+{C}_{1}\right)\left(2{\sigma }_{X}{\sigma }_{Y}+{C}_{2}\right)}{\left({{\mu }^{2}}_{X}^{}+{{\mu }^{2}}_{Y}^{}+{C}_{1}\right)\left({{\sigma }^{2}}_{X}^{}+{{\sigma }^{2}}_{Y}^{}+{C}_{2}\right)}$$

where ${\mu }_{X}$ and ${\mu }_{Y}$ are the average pixel values of the image pair ($X, Y$), ${\sigma }_{X}$ and ${\sigma }_{Y}$ are the variances, and the $C$ terms are regularization constants, where ${C}_{1}$ equals ${\left(0.01 \times 2000\right)}^{2}$, ${C}_{2}$ equals ${\left(0.03 \times 2000\right)}^{2}$, and 2000 is the dynamic range of the images. The PSNR is defined as follows:

$$PSNR=10{\text{log}}_{10}\frac{max{\left|X\left(i,j\right)\right|}^{2}}{MSE}$$

$$MSE=\frac{1}{M\times N}\sum _{i=1}^{M}\sum _{j=1}^{N}{\left(X\left(i,j\right)-Y(i,j)\right)}^{2}$$

The PSNR is defined as the maximum value in an input image $X\left(i,j\right)$ divided by the mean squared error between images $X$( with artifacts or the corrected image) and image $Y$ (the original CT image). In addition, $M$ and $N$ represent the widths and heights of the images, respectively. Differences in SSIM and PSNR were evaluated as statistically significant using a two-tailed t-test.

Figure 4 shows the results of the artifact correction image in the sparse-view CT using each deep learning method. (a) shows representative axial slice images with a dispersed projection angle of 1°. The lower images show subtracted images (the artifact correction image minus the original image), as shown by the absolute difference in CT values. Moreover, (b), (c), and (d) show the results for each corrected image with dispersed projection angles of 2°, 5°, and 10°, respectively. The simulated sparse-view CT image degraded the image quality, large deviations in the CT values are shown in the subtraction image, especially for dispersed projection angles greater than 5°. In contrast, the simulated sparse projection artifacts were small at dispersed projection angles of 1° and 2°. Through artifact reduction using each deep learning model, differences in CT values in the lung and air regions were observed in the subtraction image of the AE and U-Net results. Figure 5 shows the enlarged images of each artifact-reduced image. Image artifacts caused by sparse projections were reduced even when the dispersed projection angle was greater than 5°. However, the image resolution was significantly decreased, and image blurring occurred in the results of the AE and U-Net models, resulting in missing fine structures within the lung region. In contrast, we observed that the CGAN model reduced the artifacts while maintaining the image resolution. However, the detailed shape of the tumor and spine changed, and it was not possible to completely restore the image at dispersed projection angles of 5° and 10°.

Tables 1 and 2 present the ME and MAE results compared with the original CT image. For a large decimation projection angle, the CT values differed from those of the original images. In the results of AE and U-Net, large deviations were observed in each region; in particular, the lung and air regions had a difference of over 45 hounsfield unit (HU). In the results of the CGAN, the HU values were similar to those of the original image for each sparse projection angle; however, a slight deviation was observed in the bone region, and the differences were generally less than 10 HU.

Table 1

Mean error in the CT values of each site: lung, soft tissue, bone, and air regions. These values were calculated with respect to the CT numbers in the original images and are shown in terms of mean ± standard deviation for evaluation images.
Lung	Interval of projection data acquisition angle
Lung	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	-0.08	± 0.06	-0.07	± 0.13	-0.07	± 0.67	3.80	± 3.51
AE	-52.90	± 0.84	-48.70	± 0.71	-47.49	± 1.73	-75.42	± 6.98
U-Net	-54.01	± 0.31	-54.96	± 0.45	-54.89	± 0.78	-59.74	± 2.24
CGAN	1.47	± 0.20	-0.85	± 0.25	0.97	± 1.70	-5.38	± 2.84
Soft tissue	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	-0.08	± 0.09	-0.09	± 0.12	-0.59	± 0.55	-2.38	± 1.59
AE	-17.28	± 0.34	-9.89	± 0.61	-5.81	± 1.44	-3.63	± 2.12
U-Net	-19.68	± 0.28	-25.55	± 0.20	-14.60	± 1.08	-21.09	± 1.78
CGAN	2.98	± 0.11	0.82	± 0.14	0.32	± 1.74	-0.16	± 1.93
Bone	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	0.14	± 0.20	-0.33	± 0.50	-0.40	± 3.07	-6.38	± 12.80
AE	-10.57	± 2.65	-3.20	± 3.10	-8.75	± 4.25	7.13	± 10.25
U-Net	-14.35	± 1.27	-20.62	± 1.39	-8.83	± 2.55	-0.85	± 10.77
CGAN	3.26	± 0.45	0.14	± 0.74	4.15	± 4.25	-2.02	± 11.55
Air	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	-1.30	± 0.08	-0.66	± 0.24	12.20	± 2.49	35.95	± 1.55
AE	-62.27	± 0.24	-62.47	± 0.14	-61.19	± 0.46	-58.75	± 0.96
U-Net	-61.95	± 0.19	-60.97	± 0.21	-60.48	± 0.46	-61.95	± 0.48
CGAN	-0.45	± 0.09	-1.13	± 0.17	0.86	± 0.50	1.23	± 0.45

Table 2

Mean absolute error in the CT numbers of each site: lung, soft tissue, bone, and air regions. These values were calculated with respect to the CT numbers in the original images and are shown in terms of mean ± standard deviation for evaluation images.
Lung	Interval of projection data acquisition angle
Lung	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	0.08	± 0.05	0.12	± 0.09	0.49	± 0.44	3.88	± 3.44
AE	53.11	± 0.84	48.89	± 0.72	47.67	± 1.74	75.72	± 7.01
U-Net	54.22	± 0.31	55.17	± 0.45	55.11	± 0.79	59.97	± 2.24
CGAN	1.48	± 0.20	0.85	± 0.26	1.61	± 1.05	5.40	± 2.86
Soft tissue	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	0.10	± 0.07	0.12	± 0.08	0.66	± 0.46	2.51	± 1.38
AE	17.35	± 0.34	9.93	± 0.62	5.83	± 1.44	3.64	± 2.13
U-Net	19.75	± 0.28	25.65	± 0.20	14.66	± 1.08	21.18	± 1.79
CGAN	2.99	± 0.11	0.83	± 0.14	1.38	± 1.03	1.28	± 1.39
Bone	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	0.21	± 0.10	0.48	± 0.34	2.64	± 1.39	11.89	± 7.33
AE	10.61	± 2.66	3.96	± 1.94	8.78	± 4.27	9.56	± 7.83
U-Net	14.41	± 1.27	20.71	± 1.40	8.86	± 2.56	9.35	± 4.52
CGAN	3.27	± 0.46	0.59	± 0.43	4.17	± 4.27	9.42	± 6.36
Air	1 deg.		2 deg.		5 deg.		10 deg.
with artifacts	1.30	± 0.08	0.66	± 0.25	12.25	± 2.50	36.09	± 1.56
AE	62.51	± 0.24	62.71	± 0.14	61.43	± 0.46	58.98	± 0.97
U-Net	62.20	± 0.19	61.21	± 0.21	60.71	± 0.46	62.19	± 0.48
CGAN	0.45	± 0.09	1.13	± 0.18	0.86	± 0.50	1.23	± 0.45

Table 3 presents the results of the SSIM and PSNR for each dispersed projection angle with artifact reduction in each model. Through artifact reduction using the CGAN model, the results of the SSIM and PSNR were significantly improved for all sparse projection angles. In contrast, the results of SSIM in AE and U-Net were shown under 0.5, and the PSNR was degraded compared with the original image at sparse projection angles of 1° and 2°.

Table 3

SSIM and PSNR values in each dispersed projection angle with artifacts reduction in deep learning models. These values were calculated based on the original CT image. *p-values of < 0.005 were deemed significant for differences from the results of corresponding results of artifacts image.
	Interval of projection data acquisition angle
SSIM	1deg				2deg				5deg				10deg
with artifacts	0.87		± 0.02			0.58	± 0.02		0.29		± 0.02		0.17			± 0.02
AE	0.40		± 0.03*			0.38	± 0.02*		0.37		± 0.02*		0.31			± 0.03*
U-Net	0.41		± 0.03*			0.41	± 0.03*		0.40		± 0.03*		0.31			± 0.02*
CGAN	0.94		± 0.02*			0.90	± 0.03*		0.87		± 0.02*		0.79			± 0.02*
PSNR
PSNR		1deg				2deg				5deg				10deg
with artifacts		33.6		± 0.3		26.8		± 0.3		19.7		± 0.4		16.0	± 0.5
AE		24.8		± 0.1*		24.5		± 0.1*		23.3		± 0.2*		20.4	± 0.4*
U-Net		24.9		± 0.1*		24.8		± 0.1*		24.0		± 0.1*		21.1	± 0.3*
CGAN		36.7		± 0.7*		33.3		± 0.4*		29.3		± 0.4*		23.7	± 0.6*
* p < 0.005, showing a significant difference from the artifact conditions.

Our study attempts to reduce image artifacts in sparse-view CT images using deep learning. Sparse-view CT reduces the patient exposure dose, and artifact reduction is essential for applying this technique to clinical CT images. Therefore, deep learning with the CGAN model, which is expected to achieve accurate synthetic image generation, was evaluated for artifact correction using simulated sparse-view CT images.

As shown in Figs. 4 and 5, image quality degradation with sparse projection occurred in the simulated sparse-view CT, especially at decimation angles of 5° and 10°, as shown in the subtraction image. In the results of the AE model, restoration of the decoding process was insufficient because of the suppression of the artifact region with a relatively high-contrast resolution component; This was noticeable at the decimation angles of 5° and 10°. The U-Net model reduced artifacts while maintaining the image resolution at a decimation angle of up to 5°. However, partial over-smoothing was observed at the boundary between the adipose and muscle regions. This result shows the same tendency as that reported in previous studies [30, 31]. Moreover, the lung and air regions were different from the original image in the artifact-reduced images of AE and U-Net. The low-density CT value in the lung region tends to be smoothed with an excessively low CT value; therefore, the microvessel structure in the lung vanishes; This can be seen in the reconstructed image in Figs. 4 and 5, even for decimation angles of 1° and 2°. As shown in Tables 1 and 2, the ME and MAE had large differences in the lung and air regions from the original image. The AE and U-Net set the loss function of MSE compared with the training data, and learning with total variation regularization; therefore, these over-smoothing corrections and filling with a uniform value were possibly shown in the artifact reduction images of the AE and U-Net. This result could affect the accuracy of computational analyses using CT values in the image, such as computer-assisted detection/diagnosis and radiation treatment planning. In contrast, the artifacts of sparse projection were properly corrected, and over-smoothing of the pixel value did not occur in the results of the CGAN method. Therefore, the results of ME and MAE were also low values by almost under 5 HU in all regions, and the artifact correction image by the CGAN was suitable for applying the computational analysis image.

Table 3 shows the results of the SSIM and PSNR in sparse projection and correction images by each model. The highest values of SSIM and PSNR were shown in the results of CGAN. The SSIM over 0.8 and the PSNR over 20 dB were accomplished even for the decimation angles of 10 degrees. In the previous study for image correction of sparse-view CT, results of SSIM and PSNR reached about 0.8 and 30 dB by using dual CNN-based methods [32]. Our results of CGAN were comparable with previous studies; therefore, the CGAN model can synthesize accurate images of the sparse-view CT. On the other hand, the results of SSIM and PSNR were significantly degraded by the AE and U-Net models because ME and MAE had large values in these models.

The CGAN model significantly improved the image quality index in terms of SSIM and PSNR compared to the U-Net and AE models. With the addition of conditions and L1 norm regularization, the CGAN significantly improved artifact correction for sparse-view CT and restored the synthetic image close to the original image. In this study, up to a decimation angle of 2°, accurate restoration of image quality, including organ structure and CT values, was achieved using the CGAN model. Over a decimation angle of 5°, the details of the organ structures appear to be transformed, and accurate restoration is limited in the CGAN model. On the other hand, the image similarity index in the SSIM and PSNR were significantly improved using the CGAN correction compared with sparse projection image in all decimated angle cases. Therefore, the accurate restoration of pixel values in the lung region, soft tissues, and bones can enhance the accuracy of image registration using pixel value information and improve the accuracy of calculating the distribution of radiation dose in the radiation therapy. Many groups have explored sinogram synthesis methods based on CNNs in the projection domain and proposed filling in missing view data in sinograms [33, 34]. Our study applied the reconstructed images as training and evaluation data; therefore, artifact correction was performed on the reconstructed CT images. Therefore, because there is no sinogram-based correction, our study has the advantage of not being affected by filter characteristics such as high-frequency enhancement by the FBP. And, since the image reconstruction process can be speeded up with directly corrected in the reconstruction image, it is considered to be more practical in clinical practice. Moreover, implementing artifact correction directly on the reconstructed images is considered more practical and versatile, because the users cannot acquire the sinograms directly from clinical CT scanners. On the other hand, focusing on the details of the tumor contour, the details of the tumor structure were distorted, limiting the complete reconstruction of the structure using the CGAN model.

It is difficult to collect a large number of pixel-by-pixel paired CT images with sparse projections in a clinical CT unit because conventional CT equipment involves continuous rotation data acquisition. Therefore, in this study, many virtual sparse-view CT images were created from sufficiently projected CT images using computational simulation, and deep learning was accomplished using these images. The CGAN needs to add a conditional label using paired images, and the effectiveness of image quality improvement is expected. In this study, the evaluated image of the deep-learning model was an artifact image generated by the computational simulation of sparse projection, and the correction effect for the artifacts caused by the actual sparse projection was not verified. However, because sparse projection is not possible with current clinical CT units, modification of the CT data acquisition system is needed to apply artifact correction methods with deep learning. We believe that our research findings can contribute to reducing radiation exposure and shortening imaging time (by reducing the projection data per phase due to 4D reconstruction) in cone-beam CT images that can be acquired through sparse projection. Our study clarified the effect of image quality improvement for sparse-view CT using three deep learning models and revealed that the CGAN model can synthesize the most similar image, including consistency of CT values. For the clinical application of artifact correction of sparse-view CT images, it is necessary to evaluate the practicality of artifact correction by CGAN by verifying the accuracy of this learning model for actual sparse projection images in future studies.

To suppress the image artifacts of sparse-view CT, a deep learning model, CGAN, was constructed using artifact images created by computational simulation as training data, and its correction effect was compared and evaluated with other deep learning models. The CGAN model demonstrated high image reproducibility compared to AE and U-Net, as well as particularly accurate CT value restoration. However, over a decimation angle of 5°, the accuracy of reconstructing the exact organ structures was limited.

Computed tomography

CBCT

Cone-beam computed tomography

CNN

Convolutional neural network

GAN

Generative adversarial network

CGAN

Conditional generative adversarial network

Autoencoder

ROI

Regions of interest

Mean error

MAE

Mean absolute error

SSIM

Structural similarity index

PSNR

Peak signal-to-noise ratio

Hounsfield unit.

Acknowledgments

The authors would like to thank Juntendo University Hospital for their continuous support of this project. We would like to thank Editage (www.editage.com) for English language editing.

Consent for publication

Not applicable

Funding

This work was supported by JSPS KAKENHI Grant Numbers 22K07778.

Competing Interests

The authors have no relevant financial or non-financial interests to disclose.

Author Contributions

KU designed the study. Material preparation, data collection and analysis were performed by KU and SK. AA and KO contributed to analysis and interpretation of data, assisted in the preparation of the manuscript. HS contributed to statistical analysis of the results. All authors contributed to the study conception and design. The first draft of the manuscript was written by KU. YS, SK and HD revised the draft for important intellectual content. All authors read and approved the final manuscript.

Ethical approval

All procedures performed in this study were in accordance with the ethical standards of the institution or the practice at which the study was conducted.

Data availability

The datasets used and analyzed during the current study available from the corresponding author on reasonable request.

Ethics approval and consent to participate.

Not applicable. CT data were obtained from The Cancer Imaging Archive (http://www.cancerimagingarchive.net/).

Trial registration

None.

Krishnamoorthi, R., Ramarajan, N., Wang, N. E., Newman, B., Rubesova, E., Mueller, C. M., & Barth, R. A. Effectiveness of a staged US and CT protocol for the diagnosis of pediatric appendicitis: reducing radiation exposure in the age of ALARA. Radiology, 2011;259:231–239.
McCollough, C. H., Primak, A. N., Braun, N., Kofler, J., Yu, L., & Christner, J. Strategies for reducing radiation dose in CT. Radiologic Clinics, 2009;47:27–40.
Pickhardt, P. J., Lubner, M. G., Kim, D. H., Tang, J., Ruma, J. A., del Rio, A. M., & Chen, G. H. Abdominal CT with model-based iterative reconstruction (MBIR): initial results of a prospective trial comparing ultralow-dose with standard-dose imaging. AJR. American journal of roentgenology, 2012;199:1266.
Ghetti, C., Ortenzia, O., & Serreli, G. CT iterative reconstruction in image space: a phantom study. Physica medica, 2012;28:161–165.
Beck, A., & Teboulle, M. Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE transactions on image processing, 2009;18:2419–2434.
Kudo, H., Suzuki, T., & Rashed, E. A. Image reconstruction for sparse-view CT and interior CT—introduction to compressed sensing and differentiated backprojection. Quantitative imaging in medicine and surgery, 2013;3:147.
Sidky, E. Y., Kao, C. M., & Pan, X. Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT. Journal of X-ray Science and Technology, 2006;14:119–139.
Hansis, E., Schafer, D., Dossel, O., & Grass, M. Evaluation of iterative sparse object reconstruction from few projections for 3-D rotational coronary angiography. IEEE Transactions on Medical Imaging, 2008;27:1548–1555.
Bian, J., Siewerdsen, J. H., Han, X., Sidky, E. Y., Prince, J. L., Pelizzari, C. A., & Pan, X. Evaluation of sparse-view reconstruction from flat-panel-detector cone-beam CT. Physics in Medicine & Biology, 2010;55:6575.
Shieh, C. C., Gonzalez, Y., Li, B., Jia, X., Rit, S., Mory, C., … Keall, P. SPARE: Sparse-view reconstruction challenge for 4D cone‐beam CT from a 1‐min scan. Medical physics, 2019;46:3799–3811.
Katsevich, A. Theoretically exact filtered backprojection-type inversion algorithm for spiral CT. SIAM Journal on Applied Mathematics, 2002;62:2012–2026.
Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Yu, G., … Ma, J. Sparse-view x-ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 2014;59:2997.
Brooks, R. A., Glover, G. H., Talbert, A. J., Eisner, R. L., & DiBianca, F. A. Aliasing: a source of streaks in computed tomograms. Journal of computer assisted tomography, 1979;3:511–518.
Candès, E. J., Romberg, J., & Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 2006;52:489–509.
Sidky, E. Y., & Pan, X. Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Physics in Medicine & Biology, 2008;53:4777.
Donoho, D. L. Compressed sensing. IEEE Transactions on information theory, 2006;52:1289–1306.
Chen, H., Zhang, Y., Kalra, M. K., Lin, F., Chen, Y., Liao, P., … Wang, G. Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE transactions on medical imaging, 2017;36:2524–2535.
Kang, E., Min, J., & Ye, J. C. A deep convolutional neural network using directional wavelets for low-dose X‐ray CT reconstruction. Medical physics, 2017;44:e360-e375.
Madesta, F., Sentker, T., Gauer, T., & Werner, R. Self-contained deep learning‐based boosting of 4D cone‐beam CT reconstruction. Medical Physics, 2020;47:5619–5631.
Zhao, T., McNitt-Gray, M., & Ruan, D. A convolutional neural network for ultra‐low‐dose CT denoising and emphysema screening. Medical Physics, 2019;46:3941–3950.
Kim, B., Han, M., Shim, H., & Baek, J. A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low‐dose CT images. Medical physics, 2019;46:3906–3923.
Mao, X., Shen, C., & Yang, Y. B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Advances in neural information processing systems, 2016;29.
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. Generative adversarial networks. arXiv preprint arXiv:2014;1406.2661, 1406.
Harms, J., Lei, Y., Wang, T., Zhang, R., Zhou, J., Tang, X., … Yang, X. Paired cycle-GAN‐based image correction for quantitative cone‐beam computed tomography. Medical physics, 2019;46:3998–4009.
Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition 2017;(pp. 1125–1134).
Clark, K., Vendt, B., Smith, K., Freymann, J., Kirby, J., Koppel, P., … Prior, F. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. Journal of digital imaging, 2013;26:1045–1057.
Zhao, T., McNitt-Gray, M., & Ruan, D. A convolutional neural network for ultra‐low‐dose CT denoising and emphysema screening. Medical Physics, 2019;46:3941–3950.
Chen, S., Qin, A., Zhou, D., & Yan, D. U-net‐generated synthetic CT images for magnetic resonance imaging‐only prostate intensity‐modulated radiation therapy treatment planning. Medical physics, 2018;45:5659–5665.
Renieblas, G. P., Nogués, A. T., González, A. M., Gómez-Leon, N., & Del Castillo, E. G. Structural similarity index family for image quality assessment in radiological images. Journal of medical imaging, 2017;4:035501–035501.
Kim, B., Han, M., Shim, H., & Baek, J. A performance comparison of convolutional neural network-based image denoising methods: The effect of loss functions on low‐dose CT images. Medical physics, 2019;46:3906–3923.
Li, M., Du, Q., Duan, L., Yang, X., Zheng, J., Jiang, H., & Li, M. Incorporation of residual attention modules into two neural networks for low-dose CT denoising. Medical Physics, 2021;48:2973–2990.
Zhang, P., & Li, K. A dual-domain neural network based on sinogram synthesis for sparse-view CT reconstruction. Computer Methods and Programs in Biomedicine, 2022;226:107168.
Lee, H., Lee, J., Kim, H., Cho, B., & Cho, S. Deep-neural-network-based sinogram synthesis for sparse-view CT image reconstruction. IEEE Transactions on Radiation and Plasma Medical Sciences, 2018;3:109–119.
Fu, J., Dong, J., & Zhao, F. A deep learning reconstruction framework for differential phase-contrast computed tomography with incomplete data. IEEE Transactions on Image Processing, 2019;29:2190–2202.

No competing interests reported.

Download PDF

Journal Publication

published 16 Feb, 2024

Read the published version in Scientific Reports →

Editorial decision: Revision requested
05 Dec, 2023
Reviews received at journal
01 Dec, 2023
Reviewers agreed at journal
30 Oct, 2023
Reviews received at journal
11 Oct, 2023
Reviewers agreed at journal
08 Oct, 2023
Reviewers invited by journal
18 Sep, 2023
Editor assigned by journal
04 Sep, 2023
Editor invited by journal
29 Aug, 2023
Submission checks completed at journal
29 Aug, 2023
First submitted to journal
28 Aug, 2023

You are reading this latest preprint version

Reducing image artifacts in sparse projection CT using conditional generative adversarial networks

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Image data acquisition

Generation of simulated sparse projection image

Artifacts correction in sparse-view CT by deep learning

CGAN

AE

U-Net

Image quality evaluation

ME and MAE

SSIM and PSNR

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Journal Publication

Version 1