During the manufacturing process, hot-rolled steel strip surface defects occur frequently. These defects cause economic losses and risks in the use of these products. Therefore, it is crucial to develop automatic inspection systems to identify these defects. In the last few years, computer vision has emerged as an effective tool to identify these defects. Deep learning-based computer vision techniques, especially Convolutional Neural Networks (CNN), achieved state-of-the-art results for most computer vision tasks, including image classification. These results are obtained using a large amount of data. However, collecting such large datasets in the manufacturing field remains a challenging task. To overcome such a problem, a transfer learning-based framework with multiple CNN variants is proposed in this study. Therefore, different state-of-theart and widely used pre-trained CNN architectures, including VGG-16, VGG-19, ResNet50, ResNet50V2, InceptionV3, InceptionResNet-V2, MobileNet-V1, MobileNet-V2, MobileNet-V3 Small, and NASNetMobile, combined with transfer learning were investigated to evaluated their performances in classifying hot-rolled steel strips surface defects. The experimental results showed that MobileNet-V2 and InceptionResNetV2- based methods achieve better performance than all other models in terms of accuracy, loss, training and inference times, and model size.