The granite and limestone powders are commonly exploited as a replacement for cement; however, the effects of different mixing dosages of them on the mechanical properties and durability of concrete have not been scrutinized carefully. Under different environmental conditions, the compressive strength of the specimens is measured using cube compressive, splitting tensile, freeze-thaw cycles, and sulfate immersion tests. The phase composition of hydration products and microstructure is evaluated by SEM scanning analysis. The results indicate that the composite mixture of granite and limestone powders shows a complementary synergistic effect and improves the mechanical properties, freeze-thaw resistance, and sulfate erosion resistance of the concrete. The best values for the mechanical properties and freeze-thaw resistance are obtained when the dosages of granite and limestone powders in order are 10% and 5%. For the case of granite and limestone powders equal to 10% and 15%, respectively, the best sulfate erosion resistance is reported.