This paper presents a real-time obstacle perception method (ROPM) for unmanned aerial vehicles (UAVs) with an RGB-D camera, which aims to address the difficulty of perceiving obstacles in real-time for UAVs in low-light environments. First of all, a novel obstacle detector is designed based on the ensemble detection strategy to rapidly detect dynamic and static obstacles in low-light environments. Moreover, a new tracker is constructed based on Kernel Correlation Filter (KCF), which uses the detection results to obtain obstacle features for regression matching. At the same time, a constant-acceleration Kalman filter is used to estimate the state of the dynamic obstacles in order to achieve the objective of constant and stable dynamic tracking. Furthermore, an obstacle reposition method in the region of obstacle tracking loss is designed, in order to address the problem of occlusion during obstacle perception in unknown low-light environments. Finally, an experiment conducted on a UAV platform in a low-light environment demonstrates the efficiency and accuracy of the aforementioned method.