Ambitious climate policy requires acceptance by millions of people whose daily lives would be affected in costly ways. How to get the mass public on board to prevent a political backlash against costly climate policies? Many scholars regard ‘framing’ as an effective communication strategy for changing climate beliefs, attitudes, and behaviors. In contrast, skeptics argue that people hold relatively stable opinions and doubt that framing can alter public opinion on salient issues like climate change. We contribute to this debate by conducting a first systematic review of 121 experimental studies on climate and environmental policy framing, published in 46 peer-reviewed journals. We find that the vast majority of these experiments report significant framing effects. However, the robustness of these results cannot easily be checked because only few studies make their data publicly available. A survey of framing researchers suggests that when scholars successfully publish non-significant effects, these were typically bundled together with other, significant effects. Re-analysis of studies focusing on framing differences by partisanship (a key driver of climate change attitudes) also shows that these effects are often not robust when accounting for omitted interaction bias. To improve confidence in climate communication research, we propose some best-practice standards, including preregistration of study designs, publication of replication materials, and use of advanced post-design solutions.