The Gannet Optimization Algorithm (GOA) has shown good competence in solving constrained optimization problems, whereas the algorithm is unable to solve binary optimization problems. Hence, this paper proposes a new S-shaped time-varying transfer function and implements a binary version of the Gannet Optimization Algorithm (BGOA). The performance of the proposed time-varying transfer function was verified by comparing it with five other popular S-shaped transfer functions on 23 benchmark test functions. Eventually, BGOA is used in the feature selection domain to find feature subsets, which minimizes the number of selected features and maximizes the classification accuracy. An assessment metric to evaluate the performance of BGOA over 15 different datasets from the UCI repository and compare it with Binary Particle Swarm Optimization Algorithm (BPSO), and Binary Grey Wolf Optimization Algorithm (BGOA). The results show that the proposed BGOA is capable of searching for the optimal subset of features in the binary search space.