Vehicle routing problem with time windows (VRPTW), which is a typical NP-hard combinatorial optimization problem, plays an important role in modern logistics and transportation systems. Although the particle swarm optimization (PSO) algorithm exhibits very promising performance on continuous problems, how to adapt PSO to efficiently deal with VRPTW is still challenging work. In this paper, we propose a neighborhood comprehensive learning particle swarm optimization (N-CLPSO) to solve VRPTW. To improve the exploitation capability of N-CLPSO, we introduce a new remove-reinsert neighborhood search mechanism. We calculate an information matrix (IM) recording the probability of adjacency between two clients based on the information about the clients themselves and the local information about the elite individuals to guide the removal operation of the neighborhood search. At the same time, we combine the cost matrix (CM) that records the cost of customer removal with IM to create a guided reinsertion operator based on local information to guide the routing. Moreover, to enhance the exploration of N-CLPSO, a semi-random disturbance strategy is proposed. To prevent degradation of the population, the longest common sequences of elites are saved when performing the disturbance. To illustrate the effectiveness of N-CLPSO, this paper conducts extensive experiments