COVID-19 causes neurological symptoms that can be potentially life-threatening in up to 67 % of the patients. The underlying pathophysiological mechanisms of COVID-19 associated encephalopathy, the involved immune cells, their spatial distribution and their cellular interactions during disease remain largely unclear. In this study, we performed a 38-biomarker imaging mass cytometry analysis of the brain stem from 25 patients and additional controls to understand the local immune response during SARS-CoV-2 infection at a spatially resolved, high-dimensional single-cell level. Importantly, utilizing an unbiased image segmentation and cell classification pipeline, we observed a significant immune activation in the central nervous system (CNS) and identified novel context-specific CD8 T cell and microglial clusters. Spatially resolved single-cell analysis identified distinct phenotypes of T cells and microglial clusters, their presence in specific anatomical regions and their cellular interactions. Our analysis further highlights microglial nodules and perivascular immune cell clusters as key sites of the local immune response. It also demonstrates that disease-associated neuroinflammation is associated with severe axonal damage as a structural basis for the neurologic deficits. Finally, we identified compartment- and cluster-specific immune checkpoints that can be used for future therapeutic interventions.