In the manufacturing of dies and molds, vibrations may represent serious problems, since the finishing tool used is usually slender (high Length / Diameter ratio) in order to machine deep cavities with complex geometry, typical of these products. Vibration is an undesirable phenomenon in any machining operation as it can lead to poor surface finish, lower material removal rate and increased tool wear. The use of impact dampers in the tool has proven to be an effective method for reducing vibration in machining processes. Damping occurs through energy dissipation and linear momentum exchange during intermittent collisions between the main structure (in this case the milling tool) and a free mass (spheres or cylinders placed within a tool cavity). Although efficient, these types of dampers are highly nonlinear. Thus, the aim of this work is to analyze the effect of different materials and geometries (steel spheres, tungsten spheres and steel cylinders) acting as impact dampers inside a ball nose end milling tool. To do so, milling of a convex D6 steel surface was performed, comparing commercial tool holders with dampened ones. The results showed that the tools with impact dampers generated lower values of roughness in the workpiece (around 30% of the value observed in the conventional steel tool holder for the case of steel cylinders and around 40% for both spheres) and presented lower levels of vibration when compared to the same tool without the impact damper, mainly in the machining of workpiece regions where radial and tangential forces are predominant. The tool which used tungsten spheres generated roughness surfaces similar to those obtained with steel spheres, while the tool that used steel cylinders only generated lower roughness in the regions where the axial force component is not predominant, which shows that their performance is highly dependent on the resulting force direction.