How to effectively release the carbon reduction effect of artificial intelligence(AI) is the urgent task to promote China to achieve carbon peak and carbon neutrality. In this study, we systematically investigated the impacts and mechanisms of action of AI on CO2 emissions by constructing econometric models using dynamic panel data from 30 provinces in mainland China from 2006 to 2019. We found that the use of AI significantly reduced CO2 emissions under various robustness tests, though such reductions were regionally heterogeneous, with the strongest effects observed in central followed by eastern China; no effect was observed in western China. Further analysis of mediating effects revealed that AI reduced CO2 emissions by promoting advanced industrial structures and the rationalization of industrial structures, as well as through more ecological industrial structures and by mediating effects among regions. In particular, high-quality advanced industrial structures exhibited reduced CO2 emissions with the use of AI. This study provides theoretical evidence for the reduction of CO2 emissions via AI and provides references for the orderly development of AI technologies and the promotion of carbon peaking and neutrality in China.