The emergence of ultrasonic vibration-assisted polishing technology has effectively improved the machining accuracy and efficiency of hard and brittle materials in modern optical industry, however, the material removal mechanism of ultrasonic vibration-assisted polishing (UVAP) still needs to be further revealed. This paper focuses on the material removal mechanism of ultrasonic vibration-assisted polishing of optical glass (BK7), the application of ultrasonic vibration to axial vibration and the atomization of polishing slurry, the material removal model was established. Based on the analysis of the relationship between the nominal distance d of the polishing pad and the actual contact area distribution, the prediction of the material removal profile is realized. In addition, the effects of different parameters on the material removal rate (MRR) were analyzed, including polishing force, spindle speed, abrasive particle size, ultrasonic amplitude, feed rate, and flow-rate of polishing slurry. Based on the motion equation of abrasive particles, the trajectory of abrasive particles in the polishing slurry was simulated, and the simulation results show that the introduction of the ultrasonic vibration field changes the motion state and trajectory of embedded and free abrasive particles. The new model can not only qualitatively analyze the influence of different process parameters on MRR, but also predict the material removal depth and MRR, providing a possibility for deterministic material removal and a theoretical basis for subsequent polishing of complex curved surfaces of optical glass.