The outbreak and rapid spread of COVID-19 not only caused an adverse impact on physical health but also brought about mental health problems among the public. To assess the causal impact of COVID-19 on psychological changes in China, we constructed a city-level panel data set based on the expressed sentiment in the contents of 13 million geotagged tweets on Sina Weibo, the Chinese largest microblog platform. Applying a difference-in-differences approach, we found a significant deterioration in mental health status after the occurrence of COVID-19. We also observed that this psychological effect faded out over time during our study period and was more pronounced among women, teenagers and older adults. The mental health impact was more likely to be observed in cities with low levels of initial mental health status, economic development, medical resources, and social security. Our findings may contribute to the understanding and control of COVID-19’s mental health impact.