The long-awaited roll-out of vaccination programmes against COVID-19 from across the globe has fuelled hope for a reduction in the incidence of cases and deaths, as well as the resumption of economic and social activities. Despite being the most effective measure to mitigate the pandemic, especially in regions where non-pharmaceutical interventions had been ineffective, many people suffered from the lack of efforts by government officials to conduct vaccination. In Brazil, vaccination has always been cutting across party political and ideological lines, which have delayed the start of vaccination and brought the whole process into disrepute. Such disputes put the immunisation of the population in the background and create additional hurdles beyond the pandemic, mistrust and scepticism over vaccines. We conduct a mathematical modelling study to analyse the impacts of late vaccination and with slowly increasing coverage, as well as how harmful it would be if part of the population refused to get vaccinated or missed the second dose in the state of Rio de Janeiro, Brazil. The general framework we propose can be extended to analyse the epidemic situation in any region. Our results indicate that if the start of vaccination had been 30 days earlier, combined with efforts to drive vaccination rates up, about 18,000 deaths could have been averted. Furthermore, the slow pace of vaccination and the low demand for the second dose could cause a resurgence of cases as early as 2022.