In this study, we present a real-time free-fps algorithm that utilizes a multi-dimensional vector matrix Walsh transform to address some of the current challenges in real-time video ghost imaging of moving objects. The algorithm allows for adjustable ghost imaging quality and frame rate, which solves the issues of less reconstruction sampling times, non-smooth imaging, and fixed imaging frame rate. To achieve this, a four-dimensional vector Walsh transform kernel matrix is used to transform and reconstruct the high-quality images of the target object, and then reasonable frame rate parameters are set based on the corresponding relationship between the bucket detection values and speckle in the front and back frames, resulting in smooth live video ghost imaging. After simulation and experimental verification of moving objects, our results demonstrate that our proposed algorithm outperforms existing video ghost imaging methods. Specifically, our algorithm solves the DMD refresh rate limitation, resulting in smoother and more stable real-time imaging. Moreover, objective evaluation indexes such as PSNR, the subjective non-reference evaluation standard NRSS structure retention, and Brisque subjective evaluation all show significant improvements of 8%, 7%, and 7%, respectively.