Contact stiffness and backlash model of harmonic reducer is related to robot’s positioning accuracy and vibration characteristics. Harmonic reducer tooth pair height is typically less than 1 mm. Thus, backlash and contact stiffness measurement and modeling are relatively complex. In this paper, contact stiffness and backlash model is proposed by establishing a relationship between fractal parameters and tooth contact load. Non-contact optical profiler and RMS method are combined to obtain fractal roughness parameters of real machined tooth surface. Finally, the effect of rough tooth surface and contact force fractal parameters on contact stiffness and gear backlash is studied. The results indicate that surface topography parameters and contact force have significant effects on contact stiffness and backlash. By increasing the fractal dimension, a decrease of gear backlash and contact stiffness is observed. However, the opposite is true for the fractal roughness parameter. Lastly, an increase in contact force improves the contact stiffness.