With the burden of deaths and disability caused by neurological disorders like Alzheimer's disease, Parkinson’s disease, and epilepsy on the rise, speeding up the novel neurotherapeutic drug discovery process becomes crucial. Determining whether a drug is blood-brain barrier permeable or not is a prerequisite for discovering all central nervous system (CNS) drugs. Accurate and high throughput screening of clinical CNS drugs candidates based on their blood–brain barrier (BBB) permeability therefore becomes important, given the high failure rate of drug candidates due to their inability to penetrate the BBB. Molecules being richly annotated molecular graphs with atom, bond, and substructure features make graph representation learning for blood–brain barrier permeability worth studying. In this work, we holistically benchmark the performance and generalizability of 4 graph neural network models - Message Passing Neural Network (MPNN), Graph Convolutional Network (GCN), Graph Attention Network (GAT), and Graph Isomorphism Network (GIN) to quantitatively and qualitatively predict the BBB permeability of molecules.