The characteristic frequency of vibration was obtained by extracting the vibration characteristics of fluorophlogopite in quick-point grinding, and the characteristic frequency deviation rate was proposed to study the effect of vibration characteristics on surface roughness. Meanwhile, the influence of five process parameters, such as the grinding speed, table feed speed, grinding depth, deflection angle, and inclining angle, on the characteristic frequency deviation rate and surface roughness was also investigated. It reflected the relationship between deviation rate and surface roughness. The experimental results indicated that the process parameters influenced the deviation rate and surface roughness, and the changes of both were synchronized. Moreover, the modified model of the quick-point grinding surface roughness was proposed in the paper, which was based on the prediction model of several scholars. The model included vibration characteristic parameters and five process parameters, which not only kept the accuracy of the prediction model but also realized the introduction of vibration characteristics. It was proved that the prediction effect of the modified model was accurate and reliable. Based on the prediction model of surface roughness including vibration features, the idea of monitoring and controlling surface roughness by real-time vibration signal was put forward.