Computational immune synapse analysis reveals T-cell interactions in distinct tumor microenvironments

The tumor microenvironment (TME) and the cellular interactions within it can be critical to tumor progression and treatment response. Although technologies to generate multiplex images of the TME are advancing, the many ways in which TME imaging data can be mined to elucidate cellular interactions are only beginning to be realized. Here, we present a novel approach for multipronged computational immune synapse analysis (CISA) that reveals T-cell synaptic interactions from multiplex images. CISA enables automated discovery and quantification of immune synapse interactions based on the localization of proteins on cell membranes. We first demonstrate the ability of CISA to detect T-cell:APC (antigen presenting cell) synaptic interactions in two independent human melanoma imaging mass cytometry (IMC) tissue microarray datasets. We then generate melanoma histocytometry whole slide images and verify that CISA can detect similar interactions across data modalities. Interestingly, CISA histoctyometry analysis also reveals that T-cell:macrophage synapse formation is associated with T-cell proliferation. We next show the generality of CISA by extending it to breast cancer IMC images, finding that CISA quantifications of T-cell:B-cell synapses are predictive of improved patient survival. Our work demonstrates the biological and clinical significance of spatially resolving cell-cell synaptic interactions in the TME and provides a robust method to do so across imaging modalities and cancer types.


Introduction
The immune system is a major component of the tumor microenvironment (TME) and plays a central role in the modern approach to oncology. The TME has been the target of recent therapeutics that increase the activity of the immune system 1 , improving survival for patients with tumors including metastatic melanoma 2 . Pre-existing anti-tumor immunity has long been a prognostic factor in patient survival 3,4 and has been associated with response to immune checkpoint blockade 5,6 . Conversely, absent or dysfunctional anti-tumor immunity generally leads to poor survival and response to treatment 7,8 . Better parsing of the TME factors and mechanisms mediating anti-tumor immunity is therefore critical to the improvement of immunotherapy.
Cellular interactions within the TME modulate the immune response, often suppressing anti-tumor immunity and worsening patient survival 9 . T-cells are of particular interest owing to their ability to eliminate cancer cells 10 and as key mediators in the immune checkpoint blockade pathway [11][12][13][14] .
Immunosuppressive interactions targeting T-cells can prevent their in ltration into the TME, dampen antitumor functions, and lead to worse outcomes 7 , and tumor-resident cells such as dendritic cells may also play a role in T-cell-mediated anti-tumor immunity 15 . Further study to understand T-cell-TME interactions and their clinical implications is an important need.
A central aspect of T-cell interaction and function Is the immune synapse (IS). The IS forms following recognition of antigens on antigen-presenting cells (APCs), triggering a molecular cascade where the Tcell receptor (TCR) and other signaling molecules aggregate at the point of contact with the APC 16 . This process is initially required for T-cell activation by professional APCs where T-cells are primed against non-self antigen 17,18 . Once primed, IS formation is necessary for and precedes acquired immunity functions such as delivering a cytotoxic payload to target (e.g. cancer) cells or triggering the release of cytokines 19,20

. Thus, IS formation is indicative of T-cell antigen recognition and induction of its functional capabilities. Recent studies have described important new T-cell behaviors by curating
examples from image data, e.g. by using lattice light sheet microscopy to show macrophages with exhausted CD8 T-cells 21 or by using 3D CyCIF to show CD8 T-cells with melanoma cells 22 . However, such approaches are limited by manual selection, and automated quanti cation would be bene cial.
Advances in tumor imaging provide new opportunities to understand immune synapses, but new analysis methods are needed. Proteomic imaging platforms can simultaneously measure dozens of epitopes at subcellular resolution within tumor samples 23 , and a number of groups have harnessed multiplex imaging methods to associate spatial distribution patterns of immune cells with patient survival or therapy response [24][25][26][27][28][29][30][31][32][33][34][35][36] , typically realized through analysis of cell-cell colocalization or cellular feature intensities. However, approaches based on subcellular information, such as the formation of immune synapses, remain little explored. New methods to automatically evaluate IS formation thus could be valuable to improve T-cell mechanistic quanti cation and discovery of TME-related biomarkers.
In this work we introduce a computational framework to investigate immune synapses in the TME, which we refer to as Computational Immune Synapse Analysis (CISA). We rst assess its performance in capturing relevant T-cell IS biology and then demonstrate its effectiveness in identifying functionally relevant cell interactions from high-resolution multiplex TME imaging data across multiple imaging modalities and datasets. Our ndings demonstrate the capability and value of interrogating computationally-de ned immune synapse formation from imaging data to evaluate biologically and translationally relevant interactions in the in situ TME.

Image-Based Computational Immune Synapse Analysis
We hypothesized that immune synapses between cells can be detected based on the preferential localization of proteins at interfaces where cells contact one another. While this concept is intuitive, it has been unknown whether current spatial proteomic pro ling technologies, such as imaging mass cytometry, have su cient resolution and precision to reveal such synapses. It also remains poorly understood what fraction of cell-cell contacts will have active synapses in a given tissue sample, as this quantity will depend on the cell types of interest, cell densities, the strength and persistence of synapses, and the tissue type. In addition, synapse quanti cation is affected by the accuracy of cell segmentation algorithms in identifying cell-cell interfaces in 2D spatial proteomic images. Given these uncertainties, a statistical approach that not only evaluates individual cell-cell contacts, but also integrates across an image, would be valuable to quantifying synaptic activity.
To interrogate cell-cell interactions in the in situ immunological context, we de ned an image-based T-cell immune synapse metric. T-cells are known to interact with antigen presenting cells (APCs), which motivated us to focus on the behavior of TCR proteins at interfaces between T-cells and APCs; however, the approach we describe here is generalizable to any cell types and synapse markers. A T-cell's synapse strength σ is de ned as the logarithm of the mean CD3 signal intensity (a proxy for TCR signal) at the membrane region in contact with an antigen presenting cell (APC) divided by the mean signal in the noncontact region (Fig. 1A). A positive synapse strength indicates T-cell synapse formation and potential functional interaction with another cell in the TME, modeling how the TCR aggregates at the synapse with an APC upon recognition of the presented antigen 16 . The synapse strength σ can also be considered for each tissue sample, by averaging the values of σ for all contacts between T-cells and APCs (or speci ed cell type) in the sample. We also implemented a null synapse model based on random sets of contiguous T-cell membrane pixels to account for baseline CD3 aggregation (see Methods). This computational immune synapse analysis (CISA) provides a way to investigate the functional importance of cell-cell contacts from images.
To test whether CISA is capable of quantifying immune synapses in high-resolution multiplexed images, we rst applied it to a published melanoma IMC dataset from Moldoveanu et al. 35 This dataset contains 30 pre-treatment patient tissue microarray samples from melanoma patients that received immune checkpoint inhibition therapy. We segmented cells in each image and mapped to the cell annotations from Moldoveanu et al 35 . (see Methods), shown for example in Fig. 1B. We then used CISA to compute σ CD3 (T-CD8+, APC) for each T-cell:APC contact, i.e. the relative localization of CD3 to the contact region between a CD8 + T-cell and an adjacent APC. We also computed σ CD8 (T-CD8+, APC), the localization of CD8 toward the adjacent APC. We observed strong correlation (Fig. 1C) in the directional enrichment of CD3 and CD8 toward adjacent APCs, with r = 0.69 between σ CD3 (T-CD8+, APC) and σ CD8 (T-CD8+, APC), as would be expected from active T-cell:APC interactions. CD8 + T-cells can have a minor CD4 signal not expected to co-localize with CD3 in CD8 + cells, providing a control. Consistent with these expectations, the correlation of σ CD3 and σ CD4 (r = 0.17) in CD8 + cells is less than that between σ CD3 and σ CD8 (Fig. 1C).
While our results show that colocalization of TCR proteins toward APCs can be detected by CISA, some protein colocalization might occur regardless of an APC contact. To evaluate the contact-independent effect, we calculated pixel-wise correlations between CD3 and CD4 (or CD8) intensity within the complete T-cell areas in each spot. CISA scores exhibited stronger correlations than the contact-naïve pixel-wise methods (Fig. 1D). This was observed for correlations of CD3 and CD8 in CD8 + T cells (t-test, p = 5.8e-9), as well as for correlations of CD3 and CD4 in CD4 + T cells (t-test, p = 3.9e-7). These results indicate that CISA captures information on contact-dependent synapse activity.
To verify the robustness of these results, we analyzed a second IMC dataset from Hoch et al. 34 It consists of 167 tissue microarray (TMA) spot images from melanoma patients. This dataset showed CISA behaviors consistent with the Moldoveanu et al dataset 35 . CD3 and CD8 CISA correlations in CD8 + T cells were strong (r = 0.57, Fig. 1E) and higher than CD3-CD4 CISA correlations in the same cells (r = 0.09).
CD3 and CD4 CISA correlations in CD4 + T cells were also strong (r = 0.43, Fig. 1E), and higher than CD3-CD8 CISA correlations in those cells (r = 0.14). As in the Moldoveanu et al dataset, CISA correlations were stronger than pixel-wise correlations (Fig. 1F), both for CD3:CD8 in Tc cells (p = 1.8e-12) and for CD3:CD4 in Th cells (p = 3.8e-12). We also analyzed the correlations of T-cell:APC CISA scores for all pairs of proteins in the IMC data (Fig. S1). These showed a rich clustering structure re ecting behaviors such as TCR co-occurrence, T-cell speci city, and APC-speci city.
To clarify the observed behaviors, example images of T-cell:APC contacts with positive or negative CISA scores are shown in Fig. 2A and B. These images show the variability in signal along membrane boundaries, supporting the importance of statistical approaches that integrate information along over entire cells and tissue images to provide robustness. For example, in addition to CD3, CD4 and CD8a, we were also able to evaluate correlations in CISA scores between CD3 and other T cell surface proteins, including ICOS (CD278) and CD7 (Fig. 2C, D). It is worth noting that ICOS has a signi cantly (p = 3e-9) lower expression level than CD4 in CD8 + T cells (Fig. 2E), but the correlation between σ CD3 (T-CD8+, APC) and σ CD278 (T-CD8+, APC) ( Fig. 2D) is stronger than between CD3 and CD4 in those cells. This suggests CISA can detect polarization of cell membrane proteins even at a low expression level. Together, these results support CISA as a robust method to quantify immune synapses between T cells and neighboring APCs.
Whole-slide images enable regional analysis of tumor microenvironmental interactions To further verify the applicability of CISA to other types of image data, we generated and analyzed histocytometry 37 whole slide images of metastatic melanoma 38 . These WSIs are much larger than TMA spots and are based on uorescence, providing substantial differences from IMC. We analyzed a cohort of 21 human metastatic melanoma samples from 20 patients (see Methods). A 6-marker panel was employed to characterize cell phenotypes across whole-slide sections, providing extensive data for identi cation of TME features and interactions. The images in our cohort averaged 67 mm 2 of tissue imaged at a resolution of 663 nm per pixel.
We rst analyzed regional cell type prevalence and spatial co-occurrence in these datasets, as cellular composition 39 and APC expression 38 could vary between the tumor stroma and tumor nests. The large size of WSI images makes them better than TMA spots for investigating potential region-speci c T-cell interactions. After segmenting images into intratumor and stromal regions (see Methods, Fig. 3A, B), we observed that T-cells and macrophages are both abundant in the stroma, but macrophages make up the bulk of the intratumor immune in ltrate. Within the tumor, the cohort median density of T-cells is 86.3 cells per mm 2 , while the density of macrophages is 252.6 cells per mm 2 (macrophage > T-cell: p = 2.4x10 − and stromal regions, where loading is de ned by the presence of melanoma antigen in the cytoplasm (Fig. 3D, see Methods). In the intratumoral region, the majority of macrophages are loaded with unprocessed melanoma antigen (Fig. 3E, cohort median = 72.3%). In the stroma, a signi cantly lower fraction of macrophages is loaded (cohort median = 7.7%, p = 9.5x10 − 7 ).
T-cells colocalize with macrophages in a region-dependent manner in melanoma Because of the differential antigen-loading of macrophages between the intratumoral and stromal regions, we hypothesized that cells of the TME interact in a region-speci c manner. We investigated this using a radial distribution function (RDF) analysis to identify spatial relationships between cell types in the TME (Fig. S2A,B). We applied RDF analysis to quantify the distance-dependent density of TME cells relative to T-cells (see Methods). We additionally devised a metric to quantify T-cell/TME cell colocalization relative to null expectations, ΔCDF, de ned from the RDF curve as the excess of colocalization relative to a label-permuting null model -an approach that controls for variations in local cell density (Fig. S2C).
The use of RDF analysis to reveal cell-cell spatial associations is illustrated in Fig. 4 for sample Mel-512, with T-cells as the reference cell. In the intratumoral region, a peak in the RDF for loaded macrophages at approximately 12 µm indicates that T-cells and loaded macrophages are often in close proximity ( In the stromal regions, cell co-localization relationships differ from those within the tumor. For example, in the stroma of sample Mel-512, unloaded macrophages have a small RDF peak with respect to T-cells at a distance of approximately 10 µm (Fig. 4C, blue). This colocalization exceeds the null expectation by a statistically signi cant but small margin ( =2.642, 1-sample t-test p = 1.5x10 − 6 , Fig. 4D). On the other hand, loaded macrophages have lower colocalization with stromal T-cells than expected (Fig. 4C).
Negative colocalization between these cell types is observed across the cohort ( =−3.567, 1sample t-test p = 1.0, Fig. 4D). The unloaded macrophage ΔCDF is signi cantly greater than for loaded

CISA reveals region-speci c T-cell/macrophage synapses in whole slide images
Given the distinct cell co-localization behaviors in the intratumor and stromal regions, we hypothesized that CISA analysis would detect regional differences in immune synapse formation within whole slide histocytometry images. Indeed, CISA analysis of intratumoral regions showed that intratumor T-cells tend to form synapses to loaded macrophages ( =0.098, 1-sample t-test p = 3.9x10 −3 , Fig. 5A), consistent with the IMC results. However, this relationship was not observed in the stroma, and stromal T-cells instead tend to form synapses to unloaded macrophages ( =0.059, 1-sample t-test p = 5.0x10 −3 ). Both synapse strengths are signi cantly greater than the null model (p = 2.4x10 −5 and 2.9x10 −6 respectively). T-cell synapses to loaded macrophages are signi cantly stronger within the tumor than in the stroma, while Tcell synapses to unloaded macrophages are signi cantly stronger in the stroma than in the tumor We employed synapse-focused super-resolution imaging to further verify T-cell synapses in the TME. Figure 5C shows two T-cell-macrophage contacts with different CD3 aggregation behavior via stimulated emission depletion microscopy (Fig. 5C). The left T-cell exhibits relatively homogenous CD3 along its membrane without aggregation towards its neighboring macrophage. Volumetric rendering suggests a CD3-containing T-cell protrusion towards the macrophage which may signify early synapse formation or antigen sampling (Fig. S3A), though there is not a reciprocal concentration of ICAM-1 in the macrophage that would demonstrate mature synapse formation (Fig. S3B). The right T-cell shows CD3 concentration towards the dendrite protrusion on the target macrophage (Fig. 5C, yellow box). Volumetric rendering shows not only contact between the T-cell CD3 and the macrophage dendrite but also reciprocal macrophage ICAM-1 in the contact area, as expected from mature synapse formation and functional interaction (Fig. 5D, Fig. S3B).

T-cell-macrophage interactions are associated with T-cell proliferation in metastatic melanoma
T-cells recognizing their cognate antigen in conjunction with pro-in ammatory signals undergo clonal expansion 40 typically within the lymph node, but in vivo mouse data suggest this may occur in the tumor as well 15 . We therefore hypothesized that proliferating T-cells in the TME might have stronger synapse strengths than other T-cells. We extended CISA to address this by classifying T-cells as proliferating (Ki-67+) or non-proliferating (Ki-67−) from KI-67 imaging data. We found that intratumor Ki-67 + T-cells have signi cantly stronger synapses to loaded macrophages than Ki-67 − T-cells ( On the other hand, T-cell synapse formation with unloaded macrophages was not associated with a proliferative response. We observed no signi cant difference in average synapse strength between Ki-67 + and Ki-67 − T-cells in contact with unloaded macrophages, either within the tumor or in the stroma (Fig. 6A, B). Additionally, intratumoral Ki-67 + T-cells have signi cantly higher average synapse strengths with loaded macrophages than with unloaded macrophages (Fig. 6A, p = 6.0x10 − 5 ). In the stroma, T-cell Ki-67 status is not associated with synapse strength to an adjacent macrophage, regardless of whether the macrophage has an antigen load.

Synapse analysis of breast cancer imaging mass cytometry data reveal T-cell/B-cell interactions
To test the applicability of the RDF and CISA approaches to other types of tumors, we applied these methods to a cohort of breast cancer IMC images of primary tumors from 281 patients 25 , of which 275 had associated survival data and clinical subtyping. Because these IMC images were derived from tissue microarrays, individual IMC spot images had far fewer cells than histocytometry images, resulting in noisy and uninformative sample-level T-cell RDF curves. Consequently, we aggregated all breast cancer IMC images to generate a single cohort RDF for each class of cell-cell co-localizations. We did not distinguish antigen-loaded from -unloaded macrophages in this analysis because loaded macrophages make up only 0.3% of macrophages in the stroma and only 11% of the total macrophage content. We then applied CISA to the breast cancer IMC images to investigate T-cell synapses with macrophages and B-cells. In the intratumoral region (Fig. 7A),

Discussion
Image analysis of in situ human tumor samples can deepen our understanding of the functional and compositional heterogeneity within the TME. Here we have presented CISA, a novel algorithm to quantify T-cell immune synapses by analyzing the distribution of proteins at cell-cell interfaces in multiplexed images of tumor tissues. We have veri ed the effectiveness of CISA across multiple imaging mass cytometry datasets in melanoma and breast cancer and further validated it with whole slide histocytometry images and super resolution emission depletion microscopy images generated by our team. Using CISA, we were able to elucidate many aspects of T-cell synaptic interactions, including the quanti cation of T-cell:macrophage interactions in melanomas, the increased proliferation of T-cells forming synapses with macrophages, and the differential behavior of T-cell:macrophage interactions between tumor and stromal regions. To our knowledge, CISA is the rst computational method able to mine high resolution imaging data to quantify immune synapses. While we have focused on TCR proteins, CISA can handle any synapse-associated molecules. For example, CISA could be used to study other clinically important synapse-mediated mechanisms such as reciprocal PD-1/PD-L1 engagement at immune synapses 42,43 and their downstream effects on TCR engagement strength 44 .
Our analysis of a large breast cancer cohort highlights the exibility and potential clinical utility of CISA. We discovered that CISA quanti cation of T-cell: B-cell synapses is predictive of improved survival.
Remarkably, this predicted bene t is beyond what would be expected simply from lymphocyte in ltration 3 , particularly in HR + HER2 − and triple-negative breast cancer tumors. These ndings are notable given the small sizes of TMA spots, the rarity of B-cells, and the small number of immune markers in the IMC data, suggesting that the scale of data considered here will be su cient for further synapse biology discoveries.
The T-cell synaptic associations we have observed are relevant to a number of prior studies. For example, enhancement of macrophage tumor cell phagocytosis with anti-CD47 antibodies has been shown to yield tumor regression [45][46][47][48] in a T-cell-dependent manner 49  We expect that CISA analyses will improve as multiplexed tumor imaging with broader marker panels increase, as these will enable ner interrogation of synaptic protein co-localization patterns. Multiplexed imaging approaches are limited by the number of concurrent assayable markers 37 or capture duration 58 , and current panel sizes necessitate tradeoffs among tumor and immune proteins. This make it di cult to resolve drivers of spatial effects such as the proliferation observed in the melanoma histocytometry data.
Larger marker panels would also re ne understanding of how speci c T-cell populations such as Tregs 59 contribute to prognosis. 3-dimensional imaging 60 is also likely to improve the quanti cation of immune synapses and their impact on tumor cell killing in time and space 61 .
A caveat of CISA is that it depends on the quality of cell segmentation to accurately identify membrane pixels. We showed that melanoma datasets imaged and segmented by distinct approaches yielded similar synaptic behaviors when analyzed with CISA, but there remain uncertainties due to variations in data characteristics and associated segmentation approaches. For example, the cell segmentations from our histocytometry cohort were based on Imaris' proprietary algorithms. In the breast cancer cohort, segmentations were generated by the combination of Ilastik 62 and CellPro ler 63 , a common protocol but potentially dependent on manual tuning. Machine-learning segmentation techniques such as StarDist 64 , Cellpose 65 , and MESMER 66 offer high-throughput options that should improve as more publiclyavailable datasets emerge. A related consideration is that cell classi cation methods also vary, e.g. cell classi cation schemes designed speci cally for single-cell mass cytometry applications such as X-shift 67 may be important to optimal analysis of IMC data. Nevertheless, the reason that CISA is able to yield robust results in spite of such uncertainties is likely because it integrates over the many cell-cell interactions within a TMA spot or whole slide image, clarifying the signal.
In conclusion, CISA is a powerful approach capable of capturing region-speci c T-cell synaptic interactions in the TME associated with biological and clinical outcomes. Moreover, it can in principle be applied to any highly-multiplexed epitope imaging modalities. We expect that CISA analysis will be valuable for understanding the functional and clinical correlates of speci c T-cell interactions to guide future therapies.

Whole section scan
Whole tissue scans were acquired on the Leica SP8 confocal microscope (Leica Microsystems) equipped with an automated motorized stage. Sequential acquisition was performed with a 20X objective. For each tile, focal plane was de ned by autofocus function based on nuclear staining. Tiles were max projected and stitched using Leica LAS X software.

Super resolution microscopy (3D STED)
Tissue sections were stained following a modi ed immuno uorescence staining protocol with 2-fold increase in antibody concentration and one additional wash at each washing step. Vector Shield mounting media was left to cure for 72 hours. Super resolution acquisition was performed on an inverted Leica SP8 confocal microscope equipped with STED modules, with 3 depletions lasers and an HC PL APO 100x GLYC objective (Leica Microsystems). Z-stacks were acquired with the 3D STED function using the 775nm depletion laser. Images were analyzed and surface rendering was performed using the Imaris software (Bitplane).

Histocytometry
In situ quantitative analysis of melanoma tissue was based on published methodology 37,68,69 . Brie y, whole tissue scans were acquired using a Leica SP8 confocal microscope (Leica Microsystems, Germany). Each scan was then analyzed using image analysis software Imaris 8.4 (Bitplane). Using the "spot" function in Imaris, the images were segmented into individual cells, de ned based on having a nucleus diameter equal or larger than 5um, used as a seeding point for each cell's spot. The accuracy of the segmentation was visually assessed and adjusted if needed for each sample. Finally, for each generated spot, x;y coordinates and the mean intensity values for all channels were exported into an fcs le to be visualized and quanti ed using Flowjo software (version 10, Flowjo LLC).

Histocytometry image processing and analysis strategy
Single-cell masks from cell segmentation were exported from Imaris, and processed using the Python package scikit-image 70 . Small holes in the masks were lled and per marker single-cell average uorescence intensities recalculated from the modi ed masks. The one pixel-wide internal mask boundary was considered as the membrane and the remaining pixels as the cytoplasm for additional signal quanti cation.
We used a K-means-based adaptive thresholding scheme for both tumor/stroma segmentation and cell label assignment in the histocytometry cohort to accommodate for inter-sample signal variation.
Tumor/stroma segmentation for each image was conditioned on the signal intensity of the melanoma and CD45 channels respectively. The two image channels were passed through a gaussian lter (σ = 1) and log 2 (x + 1) transformed, where x represents the pixel value, to prevent negative values. We used Multi-Otsu thresholding, which is equivalent to the K-means clustering of the intensity histograms 71 , to determine thresholding levels. We considered four clusters which approximate to non-tissue background, tissue background, low intensity foreground, and high intensity foreground. The nal signal threshold value was chosen as the mean signal in the low intensity foreground cluster to help limit undersegmentation of the tumor. Intensities greater than the threshold were considered part of the individual mask. Individual masks were furthered processed with a morphological closing (10-pixel square) followed by removing small holes (fewer than 2500 pixels) to eliminate small gaps. Small objects (fewer than 2500 pixels) were also removed to ignore isolated cell debris. We considered the tumor segmentation as the processed melanoma channel mask; the stroma segmentation was the processed CD45 channel mask excluding intersection with the melanoma channel mask. This allows for immune cells which reside in tumor nests to be differentiated from those in the stromal regions of the tumor.
Our K-means clustering approach for cell label assignment considered two clusters, approximating nonmembership and membership. We clustered the log 2 -transformed single cell average uorescence intensities (again biased by + 1) for each channel within a sample separately. To reduce the potential for false positives, we chose the mean of the membership cluster as our threshold to label cells. We modi ed the quanti cation of single cell signal intensities to call antigen-loaded macrophages in our histocytometry cohort. Instead of using the entire macrophage area to determine melanoma signal to call antigen-loaded or unloaded, we took the upper-quartile average of the melanoma signal in the cytoplasm mask and determined loaded macrophage membership using the melanoma channel threshold. Our gating strategy allows for membership in multiple classes, such as cells labeled as both CD3+ (T-cell) and CD14+ (macrophage) which should not be expressed in a single immune cell together. However, we observed that these cases were due to undersegmentation of cells such that T-cells in contact with macrophages were pressed up closely enough such that nuclei were indistinguishable for segmentation.
This did not affect our spatial analyses signi cantly but is an artifact to consider that may cause our immune synapse analysis to underestimate the strength of T-cell synapses with macrophages.
External Imaging Mass Cytometry data analysis Data from the breast cancer IMC cohort 25 were obtained from https://doi.org/10.5281/zenodo.3518284.
Data were minimally processed, as the necessary elements for our RDF and synapse analysis were provided from the publication. This includes spillover-corrected images, tumor/stroma masks, single-cell masks, and cell labels. We used the provided cell labels in our analysis, combining the metaclusters considered epithelial cells into a single tumor cell label for our general analysis. In addition, we combined the two T-cell only metaclusters into a single cluster and the two macrophage metaclusters into a single cluster. We excluded metacluster 2 for B-and T-cell labeling due to the ambiguity. Reclustering only immune cells did not produce meaningful separation in the metacluster to discern T-and B-cells (data not shown).
To mitigate the bias of random single strong pixels in IMC data, all raw images were log-transformed before analyzed by CISA. To avoid inducing additional bias, low intensity pixels were not removed as noise. Such noise is addressed implicitly as CISA compares the contacting region with the non-contacting region.

CISA correlation analyses
For the correlations in CISA scores ( Fig. 1B and Fig. 1D Cell densities and the radial distribution function Immune cell densities in histocytometry were calculated using the tumor/stroma masks and cell labels. For sample-level densities (Fig. 3C), we took the label count divided by the area of the respective tumor and stroma mask. B-cells were excluded from analysis due to antibody cross-reactivity with tumors cells, limiting our ability to reliably detect B-cells. The radial distribution function (Fig. 4) was calculated using the nearest neighbors algorithm from the Python package scikit-learn 72 . RDFs describe the density of target particles for a given distance from a reference particle 73 (Fig. S2A). RDFs have been used previously to calculate cell-cell separation distances in cell cultures 74 , but have not been applied to quantify whole-slide TME spatial relationships to our knowledge. We adapted this concept using T-cells as the reference particle, with the other TME cell types assayed as target particles. RDFs quantify the average relationships between cell types, integrating the full information from selected regions. An advantage of RDFs over nearest neighbor statistics is that the RDF can be interpreted as a function of cell-cell distance.
After calculating all neighbors within 100 µm for each T-cell, we collected the frequency of the cell types of interest at distances binned into 1 µm increments. 100 µm is the range at which most RDF curves reached an asymptotic density; further distances did not yield additional information. We calculated histograms for both stromal and intratumor T-cells, then normalized to the number of T-cells in each region. Each bin of the histograms was further normalized by the area of the annulus representing each binned distance (Fig. S2A, yellow). This formulation of the RDF differs from the textbook de nition in the nal normalization step, where a traditional RDF de nition further normalizes to the overall particle density 73 . We chose to omit this step to re ect the true density near T-cells as an additional value to compare spatial behaviors. We modi ed the nal normalization step for the IMC analysis, as the smaller imaging area led to edge effects. For each T-cell, we adjusted the counts in each distance bin by the area of the annulus residing in the image divided by the original area of the annulus. Expected RDF curves were generated using label permutation. As we iterated through each T-cell to aggregate distance histograms for the observed RDF curve, we randomly permuted cell labels within 100 µm 100 times while keeping distances constant. We repeated the original RDF procedure for each permutation and averaged all the permutations together to generate the expected RDF curves for each cell type of interest.
The T-cell RDF can be thought of as a probability distribution function (PDF) of nding a target cell a given distance away from the reference cell. We used this concept in our ΔCDF calculations to determine T-cell spatial associations. RDFs were divided by the sum of RDF distance bins out to 100 µm to generate a pseudo-PDF which were then converted to discrete cumulative distribution functions (CDF). ΔCDF was calculated by summing the differences between the observed and expected CDFs at each of the 100 1µm bins.

Immune synapse model
To study immune synapse formation in imaging, we formulated the T-cell synapse strength as a means to quantify TCR localization. A T-cell's synapse strength with an APC is calculated as the average signal intensity of CD3 at the membrane in contact with an APC divided by the average signal in the noncontact area followed by a log 2 transformation (Fig. 1A). For a given T-cell, we identify the two pixel-wide internal boundary of the cell mask as the cell membrane. A one pixel-wide internal boundary was used for IMC synapse analysis due to the lower resolution. We use a 2-pixel diamond element for morphological dilation on each single cell mask of cells near T-cells to determine which are in contact. The dilated pixels of the neighboring cell intersecting with the cell of interest's membrane pixels are considered the contact interface (Fig. 1A). The remaining membrane pixels are part of the non-contact interface. In cases where a T-cell is in contact with multiple cells of the same cell type, the non-contact interface is modi ed to contain only pixels not considered in contact with that cell type. This model is a simple quantitative improvement to a previous in situ approach which requires multiple molecule colocalization to identify qualitative synapses 75 .
We made adjustments to the synapse strength calculation for IMC images due to high signal-to-noise ratio compared to histocytometry for which our synapse analysis was originally designed. The mean CD3 signal in either the contact and/or non-contact interface was 0 for a non-trivial number of T-cells. Rather than discarding these T-cells from the analysis, we added a noise term to provide a ll-in value in these situations. We estimated this noise term for each sample as the background cell CD3 level, calculated as the average of the CD3 signal intensity in cells not labeled as a T-cell. In cases where the mean signal at the contact interface was 0 but the mean signal in the non-contact interface was less than the estimated noise term, we set the synapse strength to 0.
We devised a null synapse model for our histocytometry cohort to account for potential cell segmentation biases and serve as a control comparison in our synapse analysis. The null synapse model was derived by generating randomly-selected contiguous pixel regions on T-cell membranes. We used a seed-andgrow approach to create contiguous regions from membrane pixels to act as the null contact pixels in our synapse model, with the remaining membrane pixels acting as the non-contact pixels for synapse strength calculations. In the seed-and-grow approach, a pixel was randomly chosen from the membrane as a seed. Neighboring pixels were iteratively added until the region consisted of at least 15 pixels. A cell's null synapse value was calculated from the average of ve randomly seeded null synapses. This process was repeated for each T-cell in contact with another cell, with the null model synapse value for an iteration calculated from the sample average of these cell's null synapse values. The mean of 100 iterations of sample averages were calculated to generate the nal null model synapse value. In our initial histocytometry synapse analysis (Fig. 5), T-cells were split into intratumor and stroma to generate separate null models for each region. We further split these populations by Ki67 positivity for the proliferation analysis (Fig. 6).
Breast cancer IMC survival analysis