DW implementation and benchmarking
DW is a publically available, open-source software that can be run on a laptop computer and features an intuitive interface through which multiple z-stacks or whole-slide images can be processed after specifying a few intuitive parameters (Fig. 1a). DW builds on the Richardson-Lucy (RL) method4,5 with three crucial improvements: (i) a highly efficient implementation of the Biggs acceleration method25 to reduce the number of required RL iterations; (ii) a high precision point spread function (PSF) calculator; and (iii) automatic lateral and axial boundary handling with minimal artifacts (Methods). The entire DW package is freely available under a GPL3 license at https://github.com/elgw/deconwolf/.
We first tested whether the Biggs acceleration module in DW improves deconvolution as compared to Huygens (HG) and DeconvolutionLab2 (DL2), which are considered Reference deconvolution tools. To this end, we used a synthetic image of microtubules (http://bigwww.epfl.ch/deconvolution/data/microtubules/) that was previously used to benchmark DL2 against HG. Using the RL algorithm implemented in DW (with Biggs acceleration turned off, see Methods) and comparing the deconvolved image to the ground truth image, the smallest mean squared error (MSE) was 1.4\(\times\)105 at 1,500 iterations, whereas adding Biggs acceleration yielded the same MSE using only 115 iterations (Fig. 1b-d). We then used the same image to benchmark DW against DL2 and HG. The sharpness and contrast of the image deconvolved by DW were clearly superior compared to the same image processed by DL2 using the same number (115) of iterations or by HG, as judged by eye as well as based on the MSE (1.4\(\times\)105, 2.3\(\times\)105, and 2.1\(\times\)105 for DW, DL2, and HG, respectively) (Fig. 1e, f and Supplementary Fig. 1a). Notably, performing the same number (115) of iterations with DW was ~16 times faster than using DL2 (on an 8-Core AMD Ryzen 7 3700X machine). We predict that reaching the same MSE (1.4\(\times\)106) achieved by DW using DL2 on the same machine would require 1,500 iterations and ~4 hours, whereas this only took 115 iterations and 52 sec with DW—a ~200-fold decrease in computing time. This dramatic difference in deconvolution speed depends mostly on the use of Biggs acceleration in DW, but also on the FFTW3 Fast Fourier Transform (FFT) library26 used in DW as opposed to AcademicFFT27, which is used by default in DL2 (Methods). Similarly, DW outperformed DL2 both in quality and speed when applied to a C. elegans whole-embryo image or to an image of a synthetic hollow bar available from the same source28 as the microtubule image described above (Supplementary Fig. 1b-f and Methods). These results demonstrate that the combination of Biggs acceleration and FFTW3 in DW drastically improves both the quality and the speed of deconvolution compared to the reference open-source deconvolution software DL2, rendering routine deconvolution of fluorescence microscopy images a practical option.
The second key feature of DW is the use of a PSF calculator (PC) based on the Born-Wolf (BW) model29, which integrates over the sensor pixels (collecting all photons) instead of only sampling the pixels at their center as in the gold-standard tool for PSF calculation—the PSF Generator (PG)30—thus theoretically resulting in higher localization precision (Methods). To benchmark PC against PG, we used the PSFs generated by both tools (using the same optical settings and PSF size) as an input to DW, to deconvolve microscopy images previously generated by OligoFISSEQ13. These images contain multiple (near-)diffraction limited fluorescence dots corresponding to the DNA loci targeted by OligoFISSEQ (Supplementary Fig. 2a and Methods). In the images deconvolved using the PC tool implemented in DW, the fluorescent dots were more distinguishable in the (x, y) plane compared to the dots in the images deconvolved using PG (Fig. 1g, h and Supplementary Fig. 2b, c). Accordingly, the size of the fluorescence dots was significantly smaller in the former compared to the latter images (full width at half maximum, FWHM: 439.9 ± 79.7 nm vs. 477.4 ± 87.1 nm, mean ± s.d. P value: 1.3\(\times\)10−29, Wilcoxon test, two-tailed) (Fig. 1i), which in turn resulted in higher resolvability of OligoFISSEQ dots when using the PC tool. Altogether, these results demonstrate that the implementation of the BW model in the PC tool used by DW outperforms the PSF Generator.
The third key feature of DW is the implementation of a method for handling image boundary effects originally developed in astrophysics31, which considers the outside of an image as missing data in contrast to standard boundary handling (BH) methods that either use an explicit guess of what is outside of the imaged region (padding) or that treat the image boundary circularly (with or without apodization) (Methods). To benchmark the BH method implemented in DW (DW-BH), we first assessed how it performs in comparison to any of the five BH methods implemented in DL2 (DL2-BH), using the same C. elegans whole-embryo image described above and the same number of iterations (50). The images deconvolved using DW-BH were much sharper and resulted in considerably fewer lateral boundary artifacts compared to the images processed using the default DL2-BH or any of the other BH methods available in DL2 (Fig. 2a, b and Supplementary Fig. 3a-f). We then assessed the ability of DW-BH to handle boundary effects that may arise during the deconvolution of z-stacks, particularly when an object is only partially imaged along the z-direction. To this end, we deconvolved a z-stack image of adherent human HAP1 cell nuclei stained with the DNA dye Hoechst 33342, either using DW-BH or DL2-BH. The latter—but not DW-BH—introduced clearly visible image distortions along the z-axis, even when using 8 times more iterations (Fig. 2c-f and Supplementary Fig. 4a-c). Notably, these artifacts were only partially prevented by using the padding option in DL2 (Fig. 2f) and became much more pronounced when we cropped the bottom focal planes to mimic a scenario frequently encountered in fluorescence microscopy experiments—where an object is not imaged entirely along the z-axis (Fig. 2g-j). Accordingly, while this had a modest effect on the fluorescence intensity profile along the z-axis of the images deconvolved with DW, the same procedure drastically changed the z-profile of the images deconvolved by DL2 (Fig. 2k). To further highlight the ability of DW to avoid boundary effects, we compared a z-stack image of HAP1 cells immunostained for histone H3 tri-methylated on lysine 27 (H3K27me3) before and after cropping the bottom focal planes. The maximum z-projections of the full and cropped stack processed by DW nearly perfectly overlapped, whereas the z-projections of the stacks deconvolved with DL2 showed largely different patterns (Fig. 2l-n). Altogether, these results demonstrate that DW outperforms DL2 both in terms of quality and fidelity of the deconvolved images generated, as well as in terms of the time required to generate them.
DW generates images comparable in quality to those obtained with confocal microscopy
To further assess the reliability of DW, we next sought to compare it with confocal microscopy, considering images generated with the latter as the ground truth. To this end, we imaged the same region of a human brain tissue section stained with an antibody against the glial fibrillary acidic protein (GFAP)—which marks multiple branched structures localized in the cytoplasm of astrocytes, glial, and ependymal cells in the brain—using both a widefield and confocal microscope (Methods). As expected, the confocal images were sharper and had more structural details compared to the non-deconvolved widefield images acquired in the same tissue region using similar optical magnification (60x for widefield and 63x for confocal) (Fig. 3a). However, when we deconvolved the same widefield images using DW, the GFAP pattern became considerably sharper and more resolved, matching the pattern visible in the corresponding confocal images (Fig. 3a and Supplementary Fig. 5a). In fact, the widefield images processed by DW displayed more fine structural details compared to the corresponding confocal images, especially when they were acquired with a higher magnification objective (100x), which is frequently the case in widefield imaging as opposed to confocal (Fig. 3b and Supplementary Fig. 5b). Notably, neither DL2 nor HG were able to closely match the quality and level of structural details obtained by applying DW (Supplementary Fig. 5c). Importantly, widefield imaging was 210 times faster compared to using a confocal microscope (~10 vs. 2,100 sec per field of view, respectively). Altogether, these results highlight the power of DW and suggest that widefield microscopy combined with image deconvolution by DW can outperform confocal imaging in terms of costs and speed, while providing results of comparable, if not superior, quality.
DW enables in silico separation of individual transcripts in crowded smFISH images
Motivated by these results, we then explored whether DW could be used to increase the spatial resolution—and therefore the throughput—in high-resolution smFISH and DNA FISH experiments, which often generate images with spatially crowded (near-)diffraction limited fluorescence dots. To this end, we first generated in silico z-stack images containing different densities of diffraction limited fluorescence dots, including a noise component to simulate real smFISH images (Methods). Again, DW clearly outperformed DL2 even when the number of iterations was twice as high in DL2 compared to DW, allowing to identify and count diffraction limited dots even in highly crowded images (~4 dots per mm3, which corresponds to ~17,000 dots in a 20 mm diameter spherical cell) (Fig. 4a-e and Supplementary Fig. 6a-e). We then assessed whether DW would also help resolve crowded transcripts in real smFISH images acquired with a widefield microscope. To this end, we performed smFISH with a probe targeting GAPDH gene transcripts in human SKBR3 breast adenocarcinoma cells (Supplementary Table 1 and Supplementary Methods). As expected, the GAPDH gene was expressed at very high levels in most of the cells, and the corresponding smFISH dots were often too crowded to be resolved by eye in non-deconvolved (raw) images (Fig. 4f). However, when we deconvolved the images with DW, the resolution drastically improved, making individual dots become visible even in very crowded regions (Fig. 4g and Supplementary Fig. 7a). This, however, was not the case when we used DL2 or HG for deconvolution (Supplementary Fig. 7b, c).
To quantitatively assess the performance of DW on these crowded smFISH images (Supplementary Fig. 7a, d), we applied our in-house software DOTTER, which is specifically tailored for detecting diffraction limited dots in smFISH and high-resolution DNA FISH images (Methods). In images processed by DW, DOTTER managed to detect individual transcripts simply based on their fluorescence intensity, whereas this was not possible in the original (raw) images, where instead a significantly higher (P = 3.16\(\times\)10−7, t-test, two-sided) number of dots was detected, especially in very bright regions (Fig. 4h, i and Supplementary Fig. 8a-c). In these regions, the high local concentration of dots produces high levels of blurred or out-of-focus light, elevating nearby dim signals and making it hard to differentiate true from false positive dots (Fig. 4h, i and Supplementary Fig. 8a, b). Indeed, the dots detected by DOTTER in the images deconvolved with DW showed a narrow size distribution characteristic of smFISH signals23, whereas the dots identified in the corresponding raw images had a broader size distribution, suggesting that many of them represented false positive signals (Supplementary Fig. 8d, e).
To confirm the ability of DW to resolve smFISH signals in crowded images, we used a different dot detection procedure (difference of Gaussians or DoG), which is also implemented in DOTTER (Methods). In this case, the number of dots detected by DOTTER was significantly lower (P = 3.66\(\times\)10−5, t-test, two-sided) in the raw images compared to those deconvolved with DW (Supplementary Fig. 8c). Importantly, both the intensity-based and the DoG-based dot identification approaches yielded very similar dot counts in the case of deconvolved images (Supplementary Fig. 8c). Furthermore, the size distribution of the dots detected using the DoG approach was still considerably broader for the raw images compared to the same images after deconvolution, suggesting that many of the dots detected in the former represent false positive signals (Supplementary Fig. 8f, g). Accordingly, the corresponding distribution of counts per field of view (FOV) was significantly different in the case of raw images analyzed using the DoG approach compared to the images deconvolved with DW and analyzed by any of the two dot detection methods, further showcasing the robustness of DW (Supplementary Fig. 8h). These results demonstrate that DW can dramatically improve the sensitivity and specificity of smFISH, particularly when highly expressed genes are targeted.
DW resolves densely packed individual DNA loci in crowded DNA iFISH images
Next, we assessed the performance of DW on crowded images of DNA loci visualized using high-resolution DNA FISH. To this end, we used our previously developed iFISH method24 to simultaneously visualize 63 DNA loci in different A/B chromatin sub-compartments32 along chromosome (chr) 16, using widefield microscope (Supplementary Fig. 9a, Supplementary Table 2, and Methods). In the raw images or in the images deconvolved by DL2 or HG, the iFISH signals in the cell nuclei appeared as clouds of poorly distinguishable fluorescence dots, corresponding to individual chromosomal territories (Fig. 4j and Supplementary Fig. 9b). In contrast, after deconvolution of the raw images by DW, the dots representing individual DNA loci became clearly separated and, as a result, the dot counts per nucleus were significantly higher, closer to the expected number of dots per cell (Fig. 4k, l and Supplementary Fig. 9a, c-e). Notably, the deconvolution process resulted in the loss of large and poorly contrasted dots and, at the same time, in the gain of dots with a FWHM close to the mean—representing bona fide iFISH signals—which were not detected in the non-deconvolved raw images (Fig. 4m, n and Supplementary Fig. 9f-k). These results demonstrate that DW can drastically improve the specificity and spatial resolution of iFISH, allowing the simultaneous visualization and localization of many individual DNA loci within the same chromosomal territory.
DW enables high throughput smFISH in tissue sections
Having demonstrated the ability of DW to resolve crowded (near-)diffraction limited dots in smFISH and iFISH images, we then wondered whether DW would also enable the detection of individual transcripts in smFISH images acquired at low magnification (20x air objectives). To test this, we targeted the mRNA product of the MKI67 gene (also known as Ki-67)—a cell proliferation marker often used in pathology—in a tumor microarray and imaged an entire tissue core from a breast carcinoma sample using a widefield microscope (Supplementary Table 1 and Supplementary Methods). As expected, individual transcripts were poorly distinguishable in non-deconvolved (raw) images acquired at 20x magnification (Fig. 5a, b). However, when we deconvolved the same images by DW, the contrast drastically improved, making individual transcripts clearly visible throughout the entire core (Fig. 5a, b). Importantly, the same spatial patterns of Ki-67 transcripts observed in the 20x deconvolved images were recapitulated in both raw and deconvolved images from the same FOVs but acquired at higher magnification (60x oil objective) (Fig. 5c and Supplementary Fig. 10a).
We then assessed whether robust automatic detection and counting of individual transcripts would be feasible in images acquired using a 20x air objective. We first applied DOTTER to the same Ki-67 smFISH images but did not manage to automatically identify a proper threshold for distinguishing real signals from noise, most likely because of the lower signal-to-noise ratio (SNR) when smFISH is performed on tissues. We therefore devised a different approach by plotting the SNR against the intensity of hundreds of thousands of fluorescence dots identified by DOTTER in the images (Methods). Except for non-deconvolved (raw) 20x images, this approach revealed two clearly distinct point clouds: one corresponding to high-quality (HQ) smFISH dots with high SNR and intensity—most likely representing true signals—and the other corresponding to low-quality (LQ) smFISH dots with lower intensity and low-to-intermediate SNR—most likely representing noise (Fig. 5d-g and Supplementary Fig. 10b-e). In all the images, except for 20x raw ones, the boundary between the two clouds corresponded to a local minimum clearly visible in the density plots of the fluorescence intensity of the DoG-filtered dots (Fig. 5d-g and Supplementary Fig. 10b-q). We therefore used this local minimum to set a threshold to automatically identify HQ dots in the 60x as well as in the 20x deconvolved images. For 20x raw images, we selected the lower-density tail of the point cloud as containing HQ dots (Fig. 5g and Supplementary Fig. 10n-q). In five FOVs analyzed, 91.6% of the HQ dots identified in the raw images at 60x magnification matched the HQ dots found in the corresponding deconvolved images (Supplementary Fig. 11a-e). Conversely, 94.1% of all the HQ dots identified in the deconvolved images overlapped with the HQ dots in the corresponding raw images, suggesting that these represent true positive signals (Supplementary Fig. 11a-e). We then used the HQ dots shared between raw and deconvolved 60x images as reference and found that 58.3% (n = 8,105) of the HQ dots in the 20x raw images did not match the reference dots, suggesting that they represent false positive signals (Supplementary Fig. 11f-j). In contrast, 81.3% (n = 6,313) of the HQ dots identified in the 20x deconvolved images matched the reference dots (Supplementary Fig. 11f-j). Of note, the HQ dots identified in the deconvolved images displayed the narrowest size distribution, further suggesting that they represent true positive signals (Supplementary Fig. 11k, l). Altogether, these results demonstrate that widefield imaging using low magnification (20x air) objectives, followed by deconvolution with DW, can be used to visualize and reliably count thousands of transcripts across large tissue sections, including sections of clinically relevant samples.
DW improves the sensitivity of in situ spatial transcriptomics
Having demonstrated that DW drastically improves the sensitivity and specificity of dot detection in both crowded and low magnification smFISH images, we wondered whether it could also improve signal detection in images generated by in situ spatial transcriptomics (ISST)17. To this end, we applied DW to an image dataset previously generated by ISST to simultaneously detect 120 genes in a large tissue section of the human middle temporal gyrus (MTG) cortex, using five consecutive cycles of in situ ligation and imaging with four fluorescence channels17,33 (Supplementary Table 3). Decoding of each individual transcript in this dataset requires that five signals ((near-)diffraction limited fluorescent dots) originating from consecutive imaging rounds are spatially colocalized (Methods). We first examined how the number of detected dots varied across a broad range of fluorescence intensity thresholds, in both non-deconvolved (raw) images as well as after deconvolution with DW. For thresholds lower than 0.1%, the number of assigned (A) and unassigned (U) dots—i.e., dots with a correctly or incorrectly decoded barcode, respectively—was comparable between raw and deconvolved images (Fig. 6a, b). The dots detected in the latter on average also had a higher quality score (Fig. 6b). For thresholds comprised between 0.1% and 2%, the number of U dots decreased sharply, while the quality score of A dots sharply increased in both raw and deconvolved images, plateauing at a threshold of 2% in the latter (Fig. 6a, b). Above this threshold, the ISST automatic dot detection pipeline started discarding true signals instead of noise, causing a sharp decline in the number of dots detected for thresholds above 10%, especially in the case of raw images (Fig. 6a). Hence, we used a threshold of 2% for all subsequent analyses.
Next, we quantitatively assessed the effect of DW on the number of transcripts correctly decoded and on cell type calling, which is based on which transcripts are expressed in each cell. Across the MTG cortical section profiled by ISST, we observed a ~3.4-fold increase in the number of transcripts identified in images deconvolved with DW in comparison to raw ones (328,437 and 96,934, respectively) (Fig. 6c). The decoded transcripts were distributed along a gradient decreasing from the supragranular to the infragranular extremity of the cortical section, and the transcript counts remained strongly correlated between deconvolved and raw images throughout the length of the section (Supplementary Fig. 12a). Only one target gene, SMYD1, showed a 50% reduction in transcript counts upon deconvolution of the images, perhaps related to the fact that this gene was very lowly expressed. We then annotated different cell types based on the relative expression of each of the 120 profiled genes in individually segmented cells (Methods). The number of cells being successfully annotated increased from 55–75% after applying DW, thanks to the substantially higher number of cell-type specific genes identified in deconvolved images (Fig. 6d and Supplementary Fig. 12b-d). Together, these results demonstrate that DW can considerably improve the sensitivity of target detection and the efficiency of cell type calling in ISST experiments. Therefore, we propose that DW is incorporated into existing ISST analysis pipelines to facilitate multiple applications of this powerful technology.
DW dramatically improves the detection sensitivity in OligoFISSEQ
Lastly, we sought to determine whether DW could also improve the detection sensitivity of OligoFISSEQ13, a microscopy-based multiplexed method that enables the reconstruction of DNA trajectories by labeling multiple DNA loci with Oligopaint FISH probes34, followed by multiple cycles of in situ sequencing (ISS) to decode the barcodes embedded in the oligos. Like ISST, OligoFISSEQ depends on the colocalization of (near-)diffraction limited fluorescent dots generated from the same target locus during multiple rounds of in situ sequencing. Importantly, even though OligoFISSEQ barcodes include redundancies to maximize their detection, the method remains sensitive to the colocalization procedure used to detect the barcodes. We therefore tested the potential of DW to overcome this issue, by using a previously generated OligoFISSEQ image dataset consisting of 46 DNA loci along chrX that had been visualized together in the same cells using five ISS cycles13 (Methods). Visual inspection of the images in the original dataset revealed the presence of densely packed clouds of fluorescence dots in different colors inside each nucleus. Notably, these dots had been only partially resolved by applying a commercial deconvolution software (Nikon NIS Elements AR), which is incorporated in the OligoFISSEQ image processing pipeline (Fig. 7a). To test whether DW would generate more resolved images and improve barcode decoding in OligoFISSEQ, we applied it to the same image dataset, which rendered individual fluorescence dots inside each cloud clearly visible (Fig. 7a). As a result, the efficiency of OligoFISSEQ barcode detection (i.e., the fraction of the 46 barcodes identified in each cell) dramatically increased from 74.1 ± 1.1% (mean ± s.d.) to 97.2 ± 0.5% (mean ± s.d.) after applying DW (Fig. 7b), demonstrating the ability of DW to augment the efficiency with which OligoFISSEQ can detect and count genomic targets. Importantly, when using the original OligoFISSEQ image processing pipeline13, a considerable fraction of the barcodes was consistently detected at a lower frequency (Fig. 7c). In contrast, all the barcodes were detected with similarly high efficiency using DW instead of the NIS software to deconvolve the OligoFISSEQ images (Fig. 7c).
We then applied the same chromosome tracing pipeline that was previously developed to reconstruct chromosome trajectories from OligoFISSEQ data13, using the coordinates of the fluorescence dots identified in the images deconvolved by DW as input. The increase in barcode detection efficiency enabled by DW yielded, for the first time, multiple complete chrX traces for which no interpolation of missing targets was required (Fig. 7d). Such fully decoded traces featured more nodes compared to the single-cell traces reconstructed from OligoFISSEQ images deconvolved with the NIS software (Fig. 7e-h). To quantitatively compare the chromosome traces reconstructed after applying the NIS software or DW, we compared the contact frequency between the 46 DNA loci visualized by OligoFISSEQ (calculated at different threshold distances for calling two loci as being in contact) with the contact frequency between the same loci assessed by Hi-C35. The 3D chromosome traces reconstructed from OligoFISSEQ images deconvolved with DW displayed a consistently higher correlation with Hi-C data compared to the traces reconstructed from images deconvolved with the NIS software, for every distance threshold used to call a pair of DNA loci as being in contact (Fig. 7i). Furthermore, the contact frequency map obtained using OligoFISSEQ images deconvolved with the NIS software was nosier and showed higher contact frequency values near the diagonal, compared to the contact frequency maps generated from OligoFISSEQ images deconvolved with DW or compared to Hi-C maps (Fig. 7j, k). Altogether, these results demonstrate that DW can greatly improve the barcode detection efficiency in OligoFISSEQ experiments, and consequently the fidelity of chromosome topology reconstructions, further highlighting the performance and broad applicability of our software.