DNA-binding proteins (DBPs) play a crucial role in numbers of biological processes and have received wide attention in recent years. Meanwhile, the rapid development of sequencing technologies lead to the explosive growth of new protein sequences, it is highly desired to develop a fast and accurate method for DNA-binding proteins prediction. Experimental methods such as chromatin immunoprecipitation on microarray (ChIP-chip) and X-ray crystallography are highly accurate but expensive and time-consuming. To address this issue, many computational methods have been proposed, they usually exploit multiple information about protein sequence, e.g., sequence composition information, physicochemical properties, evolutionary information, structural information, etc. Despite the effectiveness of these approaches, they heavily depend on prior biological knowledge and undergo a very complex process of feature extraction. In view of these shortcomings, here, we present a novel method, named DBP2Vec, to predict DNA-binding proteins directly from pre-trained protein language model (e.g., ESM-1b) which effectively encode biological properties without any prior knowledge by virtue of deep representation learning (e.g., BERT) on large protein sequences databases (e.g., UniParc). Tests on two benchmark datasets (e.g., PDB296, UniSwiss-Tst) and one newly constructed DBPs dataset demonstrate that our proposed method significantly outperforms existing state-of-the-art methods.