Random Matrix (RM) model is an effective method for modeling extended objects, and has been widely used in extended object tracking. However, the existing RM based tracking methods usually assume that the measurement models obey Gaussian distribution, which will lead to the decrease of accuracy when applied to Lidar system. This paper proposed a new observation model and used it to modified the RM smoother by considering the characteristics of Lidar data. Simulation results show that the proposed approach achieved better performance compared with the original RM tracker in Lidar system.