The study of the contamination patterns and characteristics of Sb and As in the soils of the legacy contaminated sites of antimony smelters is important for the redevelopment and utilization of industrial sites. In this study, 13 heavy metals were determined in the soil and plants of an antimony smelter in Gansu Province to study the biogeochemical behavior, health risk, and pollution source. The results showed that Nemerow index of Sb (728) exceeded the value of As (43.6) by 17.6 times, and the average Igeo values of Sb and As were 10.1 and 1.97, respectively, categorized as extremely and moderately contaminated classes. Compared to As, Sb had a larger proportion of oxidizable fraction and a smaller proportion of reducible fraction. Even though the bioaccumulation factors (BAFs) for As were about 10 times higher than those for Sb, the accumulation of Sb in plants was not negligible. For the USEPA model, the mean HQ values of As by oral ingestion, dermal absorption, and inhalation accounted for 99.0%, 0.97%, and 0.0002% of the total, and 54.2%, 45.3%, and 0.51% for MEEPRC model. There were significant positive correlations between Sb, As, Zn, Se, Cd, and Ba (P < 0.05). The results of the source identification analysis revealed that antimony smelting, solid waste pollution and natural origin were identified as the main pollution sources. The PCA and PMF methods differed by more than 20% in the analysis of the contribution of antimony smelting activities and solid waste pollution sources.