In order to improve the convenience of parking and increase parking density, automated valet parking systems based on parking AGVs (P-AGVs) are gradually becoming popular in cities. Parking AGVs can move toward any position and attain any orientation simultaneously with its Mecanum wheels, which makes it very convenient to transport vehicles in the parking lot. This paper proposes a nonlinear disturbance observer-based sliding mode controller for the trajectory tracking problem of a Parking AGV. First, a kinematics and dynamics models of a Parking AGV tracking trajectory are analyzed sequentially, and the influences of disturbing forces are considered. Afterwards, a nonlinear disturbance observer (NDO) is designed to estimate the disturbing forces and torques generated by castor wheels. Based on the designed NDO, a robust nonsingular terminal sliding mode (NTSM) controller is utilized to track reference trajectories. In addition, the stability of the NDO and NDO-NTSM control system is theoretically verified by using Lyapunov function. Finally, simulations and experiments are carried out to certify the effectiveness of the proposed control scheme. Experiment results show that the proposed NDO-NTSM controller can track the reference trajectories well and improve the trajectory tracking stability by 42–68% in comparison with a traditional NTSM controller.