A sub-pixel spectral reduction algorithm for the echelle spectrometer is proposed here to ensure a high spectral resolution for it. The initial model is established by separating cross-dispersion and calculating the imaging offset from the center. The geometric optics formula and mathematical function are used to calculate the offset in two vertical directions respectively. The deviation of the initial model is less than two pixels which is caused by coupling effect of the two vertical directions,and it is too difficult and complicated to calculate true value of the deviation. Therefore, the method of mathematical fitting is used to update the model that the deviation can be reduced. The coordinates were contrast between algorithm and both ray tracing and experimental results to determine the deviation. The results show that the maximum deviation of the whole image plane is not greater than one pixels (sub-pixel level). The proposed spectral reduction algorithm thus significantly improves the spectral accuracy of the echelle spectrometer, and the model is simple to establish and modify, which renders it suitable for applications.