The COVID-19 pandemic by non-stop infections of SARS-CoV-2 has continued to ravage many countries worldwide. Here we report the discovery of suramin, a 100-year-old drug, as a potent inhibitor of the SARS-CoV-2 RNA dependent RNA polymerase (RdRp) through blocking the binding of RNA to the enzyme. In biochemical assays, suramin and its derivatives are at least 20-fold more potent than remdesivir, the currently approved nucleotide drug for COVID-19. The 2.6 Å cryo-EM structure of the viral RdRp bound to suramin reveals two binding sites of suramin, with one site directly blocking the binding of the RNA template strand and the other site clash with the RNA primer strand near the RdRp catalytic active site. Furthermore, suramin potently inhibits SARS-CoV-2 duplication in Vero E6 cells. These results provide a structural mechanism for the first non-nucleotide inhibitor of the SARS-CoV-2 RdRp and a rationale for repurposing suramin for treating COVID-19.