Participants
A total of 1387 subjects from two different cohorts were included in this study, namely the Alzheimer's Disease Neuroimaging Initiative (ADNI) and Huashan Hospital. The ADNI was launched in 2003 as a public–private partnership and its objective is to develop a list of disease biomarkers and to advance understanding relevant to the pathophysiology of AD 34. All visits and procedures from Huashan Hospital occurred at the Movement Disorders Clinic, Department of Neurology, Huashan Hospital, Fudan University (Shanghai, China). In both cohorts, individuals who underwent a brain tau-PET imaging and corresponding sMRI scan within two months from were included in our study.
The ADNI cohort consists of 985 participants, including 483 cognitive normal (CN) subjects, 398 mild cognitive impairment (MCI), patients and 104 AD patients. Detailed definition criteria for CN status, MCI, and AD can be found at http://adni.loni.usc.edu/. The participants were randomly split into three subsets according to a ratio of 6:2:2 for model training (n = 589), validation (n = 196) and testing (test1, n = 200). Demographic information (gender, age and education) and neuropsychological examinations scores (Clinical Dementia Rating–Sum of Boxes (CDR-SB), Mini-Mental State Examination (MMSE)) were also collected for all participants. An additional independent test cohort was also compiled based on the following criteria to fully explore the relationship between the synthetic tau-PET images and MRI atrophy patterns: (1) available sMRI scan images and basic demographic information (age, sex and APOE); (2) available Standardized Uptake Value Ratio(SUVR) of global amyloid beta provided by ADNI. This resulted in a subset of 239 individuals(191NC, 29MCI and 19 AD) and 661 sMRI (test2). Participant from ADNI cohort was approved by institutional review boards of ADNI and written informed consent was obtained from all participants or authorized representatives
Huashan cohort consists of 163 participants, including 74 CN subjects, 33 MCI patients and 56 AD patients. Clinically probable AD was diagnosed with the 2011 NIA-AA guidelines 35, and the diagnosis of MCI required to meet Petersen’s criteria 36. The participants were similarly split into three subsets for model training (n = 96), validation (n = 31) and testing (test3, n = 36). Demographic information (gender and age) and neuropsychological examinations scores (CDR-SB and MMSE) were collected for all participants. All participants provided written informed consent in accordance with the Helsinki declaration, and approval was granted by the Institutional Review Board of the Huashan Hospital (identifiers: KY2019-284, KY2019-433, and KY2020-1160).
Image data acquisition and preprocessing
ADNI cohort
Tau-PET imaging in the ADNI cohort was performed using 18F-flortaucipir (FTP; 18F-AV-1451). The acquisition protocols of sMRI and FTP PET imaging are detailed at http://www.loni.ucla.edu/ADNI/Research/Cores/. The interval between the acquisition of sMRI and FTP PET images did not exceed two months for any subject. The raw imaging data were downloaded from the ADNI dataset.
Huashan cohort
Tau-PET imaging in Huashan cohort was performed using 18F-florzolotau (a second-generation tau tracer also termed 18F-PM-PBB3; 18F-APN-1607). Because of its light sensitivity, both manufacturing and injection of 18F-Florzolotau were carried out under a green light-emitting diode (510 nm). 18F-Florzolotau tau PET imaging was acquired using a 20-min protocol (90−110 min) on a Biograph mCT Flow PET/CT system (Siemens Healthcare GmbH, Erlangen, Germany) and reconstructed using a 3D-ordered subset expectation maximization algorithm. T1-weighted MR imaging was acquired using a 3.0-T horizontal magnet (Discovery MR750; GE Medical Systems, Milwaukee, WI, USA). The details for imaging acquisition could be found in the previous publication 37.
Image preprocessing
Image processing was performed using Statistical Parametric Mapping 12 (SPM12; Wellcome Trust Center for Neuroimaging, London, UK, http://www.fil.ion.ucl.ac.uk/spm) implemented in MATLAB 2017a (MathWorks Inc.). The original PET images were first converted to analysis format and registered with the corresponding sMRI images. Subsequently, both tau-PET and sMRI images were standardised to the Montreal Neurological Institute (MNI) space and smoothed with an 8 mm full width at half maximum (FWHM) Gaussian kernel. The preprocessed images were sliced by the sagittal, coronal, and axial planes respectively. The data were then resampled to 128×128 using linear interpolation and normalised to 38 using min-max normalisation. FreeSurfer v.5.3 was used to segment and parcellate the MRI image, defining regions of interest and extracting measures of cortical thickness.
Deep neural network set up
We developed a dual perception-enhanced conditional generative adversarial network for translating sMRI to tau-PET (PCGAN4Tau), containing three modules: ResNet generator G, patch-level discriminator D, and dual VGG-Net (VGG-16 and VGG-19), as shown in Fig. 1. For thorough details, please see the supplementary materials. First, a sequence of consecutive slices, containing the sMRI slices and the target tau-PET (FTP PET or PM-PBB3 PET) slices, were fed into the ResNet generator, which created the synthetic tau-PET slices after learning the distribution features of the data. The synthetic samples and the real tau-PET data were then sent to a patch-level discriminator, which was tasked with discrimination between the real and synthetic data at the patch scale. During the training process, the results of the discriminator were fed back to the layers of the network to update the parameters. The network eventually learned to fit the features of the real tau-PET as closely as feasible during this continuous iterative optimisation process.
We compared our model with four conventional GAN approaches (i) UNIT 39, (ii) pix2pix40, (iii) cycleGAN41 and (iv) BicycleGAN42. The comparison models are implemented based on previous medical image synthesis studies that achieved the best performance. For each model, two tracer-specific models, denoted as FTP-model (ADNI cohort) and PM-PBB3-model (Huashan cohort), were trained and tested respectively. All preprocessed images were sliced by the sagittal, coronal, and axial views respectively. For each view, individual sMRI 2D slices (128×128) were used as the input, and the FTP PET (for FTP-model) or PM-PBB3 PET (for PM-PBB3-model) 2D slices with respect to the same patients were taken as output to train and validate PCGAN4Tau model. We obtained the corresponding synthetic tau-PET images from three independently trained perspectives and integrated them into the 3D volume 12. After then, the 3D volume was normalised to the cerebellar gray matter 43 to generated the synthetic 3D SUVR image for further analysis.
Quantitative evaluations on the synthetic tau-PET images
Overall evaluations between the synthetic and real tau-PET images
Clinical assessment
The synthetic images using different models were selected for visual comparison, together with the residual map between the synthetic and corresponding real slices. Here, slices are obtained from the synthetic FTP/PM-PBB3 PET SUVR images. All synthetic PET images, including the synthetic FTP PET and PM-PBB3 PET images, were evaluated by two experienced physicians (i.e., one with 11 years of experience, and another with 5 years of experience for MRI and PET diagnosis). They provided the opinion ratings to assess the clinical feasibility, e.g., image clarity, structural features and contrast (especially in lesion areas) in synthetic images. The images in test sets are label-removed and presented to the two physicians for independent reading in a randomized order. According to the widely recognized reading standard in many recent studies 44,45 , the physicians assigned an image quality score for each PET image on a five-point scale: 1 = uninterpretable, 2 = poor, 3 = adequate, 4 = good, 5 = excellent.
Quantitative evaluation
For quantitative evaluation, mean absolute error (MAE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and learned perceptual image patch similarity (LPIPS) were calculated to evaluate the image quality of the synthetic images on all test datasets 46 (the Supplementary materials contain the calculation formulas of the index).
Regional-specific evaluations between the synthetic and real tau-PET images
For the regional-specific evaluation, the intraclass correlation coefficient (ICC) 47, was calculated. Here, we use the level of absolute agreement across measurements to compare the reconstructed data to ground truth 48. After the regional-specific SUVR of a single region of interest (ROI) was computed in a volume-weighted manner, we computed the index in voxel-wise to generate ICC maps. Secondly, we computed ICC values by atlas. The Automatic Anatomic Labeling (AAL, https://www.gin.cnrs.fr/en/tools/aal)-based ROI were defined using FreeSurfer and applied to all ICC maps (122 parcels covering cerebral cortex, subcortical nuclei and 32 cerebellum regions). In addition, the associations between the SUVR of individuals' real and synthetic tau-PET images in several AD-susceptible regions were further explored, including posterior cingulum, hippocampus, parahippocampal, amygdala, fusiform and inferior temporal lobe.
Cross-phenotypic associations between the synthetic tau-PET images and sMRI atrophy maps
The correlation between the two modalities (i.e. tau-PET and sMRI) was also calculated to explain the association between the synthetic tau-PET images and sMRI. Our primary analyses were performed within the FTP-model (test2: n = 239). We evaluated the association between the synthetic FTP-PET and sMRI brain atrophy maps across participants with different clinical information: age (above and below the median 77.5), sex (male and female), and presence of Apolipoprotein E (ApoE) status (positive and negative). We also approximately evaluated the changes in the different pathological stages of AD. Participants were classified as either positive (+) or negative (-) for each biomarker, resulting in four A/T groups (i.e., A+T+, 21.33%; A+T-, 34.64%; A-T+, 12.71%; A-T-, 31.32%). To be more precise, global normalized β-amyloid SUVR were used to distinguish between β-amyloid abnormal and β-amyloid normal status (threshold: 1.11) 49. SUVR in a temporal metaROI (entorhinal, amygdala, parahippocampal, fusiform, inferior temporal, and middle temporal) 50 were calculated to represent tau deposition (threshold: 1.23), which has been commonly used to detect AD-related tau deposition in the human brain 50.
Application evaluation – disease classification
The synthetic tau-PET images were utilized as input for multiple binary disease classification to evaluate the diagnostic performance along the AD continuum. A classical convolutional neural network, ResNet18, was configured to perform binary classification. The evaluation of the classification performance involved quantitative metrics such as accuracy (ACC), specificity (SPE), sensitivity (SEN), and area under the receiver operating characteristic curve (AUC). To visualize the relationship between real and synthetic images, the distribution of image features was displayed using t-distributed stochastic neighbour embedding (t-SNE) 51. Features were extracted from the real and synthetic images for each class label using the ResNet18 model, standardised by subtracting the mean and scaling the variance to 1, and then condensed to three-dimension using t-SNE (perplexity = 40.0).
Statistical analysis
Differences between the demographic information and neuropsychological scores of subjects in each subset were assessed using the chi-squared test, the two-sample t-test, and ANOVA (as appropriate). We adopt t-tests to describe the agreements between the two physicians in opinion scores. To assess the individual relationship between the SUVR of the region of interest in the synthetic images and that of real images, linear regression model was used. Then, based on our previous assumptions, using bivariate correlations to assess the link between the synthetic FTP PET images and sMRI atrophy maps, with brain cortical thickness as the dependent variable. All statistical analyses were performed using IBM SPSS Statistics Version 25, and two-sided p-values less than 0.05 were considered statistically significant.