To improve the frequency sensitivity of a single tuned mass damper (STMD) and solve the accuracy problem of multiple tuned mass dampers (MTMD) in determining suitable installation location and effective number of tuned mass dampers (TMDs), this paper combines the construction characteristics of the rocking structures and the damping principle of TMDs to form a new type of MTMD, i.e., rocking wall tuned mass dampers (RW-TMDs). To verify the effectiveness of RW-TMDs, Finite element analyses were conducted to systematically compare the damping performance of the structure with STMD and RW-TMDs under different earthquake excitations. The results show that the RW-TMDs with different dynamic characteristics not only significantly suppress structural dynamic response but also improve the frequency sensitivity of TMD, reducing structural damage. Besides, the RW-TMDs can also improves structural inter-story deformation pattern under white noise excitation, preventing the occurrence of the layer collapse mechanism. Overall, the RW-TMDs exhibit the damping effect of the MTMD system and the advantage of rocking structures to control inter-story deformation. Therefore, the RW-TMDs possess a high potential for practical applications for new and existing buildings.