Many tasks such as physical rehabilitation, vehicle co-piloting or surgical training, rely on the physical assistance from a partner. While this assistance may be provided by a robotic interface, how to implement the necessary haptic support to help improve performance without impeding learning is unclear. In this paper, we study the influence of haptic interaction on the performance and learning of a shared tracking task. We compare the interaction with a human partner, the trajectory guidance traditionally used in training robots, and a human-like robotic partner in a tracking task. While trajectory guidance resulted in the best performance during training, it dramatically reduced error variability and hindered learning. In contrast, the reactive human and robot partners did not impede the adaptation and allowed the subjects to learn without modifying their movement patterns. Moreover, interaction with a human partner was the only condition that demonstrated an improvement in retention and transfer learning compared to a subject training alone. These results suggest that for movement assistance and learning, algorithms that react to the user's motion and change their behaviour accordingly are better suited.