Does interactivity on Twitter initiated by political elites yield dividends in terms of indirect outcomes such as online political mobilization? Twitter has traditionally been utilized by political elites mainly as broadcasting media, characterized by its limited interactions with the wider public. In doing so, political elites may be overlooking Twitter's utility as an interactive media for mobilizing their supporters, which we seek evidence for in this paper. We adopt a naturalistic quasi-experimental design utilizing a unique dataset comprising tweets of users who were mentioned (36.62 million tweets) and not mentioned (29.73 million tweets) by Prime Minister Narendra Modi on 16th March, 2019 during his general election campaign. A controlled intervention analysis using Autoregressive Integrated Moving Average (ARIMA) models was carried out, which suggests that, on average, the total tweets count increased more for users mentioned by Modi (treatment group) as compared to those not mentioned by Modi (control group). However, no substantive lasting effect could be detected, indicating therefore that a mention by political elites helps generate immediate engagement which is not necessarily self-sustaining. To determine if this activation behavior is more prominent for less or more active users, we categorized the users based on their Twitter activity and assessed the significance of any movement between categories; no evidence could be found in support of this claim. Our findings thus provide important insights to understand how Twitter can be used as an effective platform for political campaigns by exploiting some of its interactive features such as replies and mentions to mobilize supporters.