The positioning accuracy of coal and gangue is related to the discharge accuracy of gangue, which will affect the utilization rate of coal. But the detectability of the small coal and gangue is poor due to the fewer number of pixels and texture information in coal and gangue dual-energy X-ray images. So, the Otsu with crotch structure based on Adaptive Partical Swarm Optimization (APSO) for small targets detection is proposed, called after APSO-C_Otsu. Firstly, the Otsu with crotch structure is used to perform multi-threshold segmentation of coal and gangue dual-energy X-ray images to increase the contrast between small target and background. Meanwhile, the APSO algorithm was used to optimize the Otsu algorithm with crotch structure in order to improve its convergence speed and reduce its calculation amount. Finally, the processed image is binarized, and the location of the target was labeled based on the bwlabel algorithm. The experimental results revealed that the APSO-C_Otsu algorithm could effectively detect the small pixel size (less than 8 × 8 pixels) of coal and gangue with a particle size of 6 ~ 30 mm, and was also applicable to the coal and gangue with the particle size larger than 30 mm, which was of great significance for accurate separation of coal and gangue and the improvement of coal utilization.