Seed purity detection is essential for crop growth and development. Real-time detection can identify and screen impurities or other varieties in time, thereby improving seed quality, increasing crop yields, and ensuring food security. However, traditional seed purity detection methods are often have certain limitations and may struggle to meet the requirements of real-time detection. In order to make the real-time seed purity detection used in seed factories, markets or other scenarios, we propose the deep model SeedNet, which is formed by adjusting the VGG16 network structure. Building upon this foundation, we embedd the Ghost module with the aim of reducing the model’s parameter count and computational complexity, resulting in SeedNet-Ghost. Pruning techniques further lighten the network by removing redundant parameters from the model’s convolutional layers, meeting the requirements for real-time detection. Finally, SeedNet-Ghost achieves optimal performance under the pruning rate of 5%, with classification accuracy 99.17%, memory usage 3.54MB, and inference time 4.27ms. In the consistency verification system designed by us, the model also shows good performance. The average speed of detecting the image purity of multiple seeds reaches 254ms. The research results indicate that the approach in this paper can offer high precision, fast speed, and low memory usage for real-time detection, safeguarding crop growth and development.