Deep convolutional neural networks have achieved great success in the field of medical image segmentation and assisted diagnosis due to their excellent autonomous feature extraction capability and good feature representation ability. Traditional convolutional neural networks usually use standard convolutional layers to extract features. However, in medical image segmentation, a pixel may correspond to multiple different scale structures, such as lesion edges, organ boundaries, etc. Therefore, traditional convolutional layers may not be able to capture this complex feature. In this paper, we propose a multi-scale convolution (MC) which realizes more comprehensive feature extraction by introducing multiple differently sized convolution kernels. Additionally, a novel multi-scale attention pooling network (MAPNet) is proposed, including a multi-scale pooling convolution block that combines pooling with multi-scale attention gates to better focus and utilize feature information at different scales and achieves effective skip connections. The proposed model is evaluated on two different medical image segmentation datasets, and the results show that our model achieves good performance in terms of speed and accuracy.