Silver vanadates are promising visible-light-responded photocatalysts with suitable bandgap for solar absorption. However, the easy recombination of photogenerated carriers limits their performance. To overcome this obstacle, a novel 2D graphene oxide (GO) modified α-AgVO3 nanorods (GO/α-AgVO3) photocatalyst was designed herein to improve the separation of photocarriers. The GO/α-AgVO3 was fabricated through a facile in-suit coprecipitation method at room temperature. It was found that the as-prepared 0.5 wt.% GO/α-AgVO3 exhibited the most excellent performance for Rhodamine B (RhB) decomposition, with an apparent reaction rate constant 18 times higher than that of pure α-AgVO3 under visible-light irradiation. In light of the first-principles calculations and the heterojunction analysis, the mechanism underpinned the enhanced photocatalytic performance was proposed. The enhanced photocatalytic performance was ascribed to the appropriate bandgap of α-AgVO3 nanorods for visible light response and efficient separation of photocarriers through GO nanosheets. This work demonstrates the feasibility of overcoming the easy recombination of photogenerated carriers and provides a valuable GO/α-AgVO3 photocatalyst for pollutant degradation.