It is of great theoretical and practical significance to explore the impact of population concentration on the efficiency of urban carbon emissions in order to build a long-term mechanism of green development in China. Based on the panel data of 284 prefecture-level cities in China from 2003 to 2019, this paper uses the super-efficient SBM model to measure the urban carbon emission efficiency, and depicts the upgrading of urban industrial structure with the upgrading and rationalization of industrial structure, and then makes quantitative analysis on this basis. The results show that population concentration can significantly improve the efficiency of urban carbon emissions. The conclusion is still valid after dealing with endogenous problems and robustness testing. Population concentration can improve carbon emission efficiency by stimulating regional technological innovation level; The upgrading and rationalization of industrial structure have a threshold effect on the urban carbon emission efficiency. The impact of the upgrading of industrial structure on the carbon emission efficiency before and after the threshold value presents a trend of "first restraining and then promoting", while the rationalization of industrial structure presents a more obvious promoting effect after crossing the threshold; Spatial measurement shows that urban carbon emission efficiency has a significant positive spillover effect, and also has a significant promotion effect on the carbon emission efficiency of neighboring cities, and the intensity is higher than the local level. Finally, based on the empirical conclusion, the corresponding policy recommendations are put forward from the perspective of the government.