Modeling the interactions between users and items to accurately predict ratings is very crucial for improving the performance of recommendation. Although existing graph-based methods have achieved great progress in predicting ratings for recommendation, they usually need additional side information which is difficultly obtained, and ignore the temporal associations between items (users) when constructing graphs. In this paper, we propose a time series association based dynamic graph evolution model for recommendation, which can capture not only the information propagation on multiple graphs but also the temporal association between items (users) by constructing a time series item association graph and a user similarity graph. Specifically, the proposed model consists of two main components: recurrent graph construction component and message propagation component. The former recurrently constructs the time series item association graph and the user similarity graph only from the interactions between items (users) to capture the temporal associations between items (users), which further helps the process of constructing two auxiliary graphs. The latter refines user and item representations by aggregating the influence information propagated from multiple high-order neighbors. Finally, the refined representations of users and items are used to predict ratings on all items a user has not interacted with. The experimental results illustrate that our method outperforms the state-of-the-art methods on five real-world datasets.