This paper presents a new local calibration method for a weak stiffness on-machine measuring (OMM) device. First, the original volumetric error models’ union is established, and an equivalent dimensionality reduction algorithm is proposed to reduce the model’s dimensionality, thus, the novel equivalent volumetric error models’ union (EVEMU) is derived, which is more suitable for limited calibration data. For better flexibility and efficiency of error measurements, a new 3-D ball array calibrator is proposed, and its local calibration procedures can be executed directly in the 3-D space. Then, the smoothly clipped absolute deviation (SCAD) regularization algorithm with oracle property is introduced to identify equivalent coefficients as well as select the optimal volumetric error sub-model of EVEMU, which can closely represent the real error state of the weak stiffness OMM device. Experiments demonstrated that methods proposed in this paper are effective, and provide new insights for the error compensation of similar devices.