As is well known, the surface texture of asphalt pavement is adversely affected by the long-term rainwater erosion, leading to a rapid decrease in skid resistance. Through a small accelerated loading device, high-precision 3D scanner, and digital image correlation technology (DIC), this study investigates the changes of surface texture and wear mechanism of basalt asphalt pavement and steel slag asphalt pavement under the influence of water erosion and driving load, respectively. It is found that the 3D-texture characteristic parameters, namely root mean square height (Sq), skewness (Ssk), peak support volume (Vmp), core support volume (Vmc), central void volume (Vvc), and valley void volume (Vvv), decrease significantly, the asphalt film peels off, leading to surface layer deformation, when subjected to loads ranging from 200,000 to 500,000. Moreover, the asphalt film peels off obviously when subjected to 500000 loads. It is noteworthy that the wear resistance of steel slag asphalt pavement is significantly better than that of the basalt asphalt pavement, providing a practical basis for the asphalt mixtures design with high wear resistance.