Accurate measurement of the distance between any two points under endoscopic examination of the gastrointestinal tract is difficult due to the fisheye lens currently used on endoscopes. To overcome this issue, we developed a three-dimensional visual inspection system that incorporates a phase-shift method for direct measurement of a three-dimensional shape. Projection of a striped pattern is used to enable three-dimensional renderings of a surface using an industrial camera. We evaluated the accuracy of measurement relative to that of reference measurements obtained using a measurement microscope, for three-dimensional shapes on a plane and curved surface. For 60 regions of interests, with various configurations of the reference locations relative to the camera, the measurement error for our system was <0.1 mm on the plane grid and <0.5 mm on the curved surface. The accuracy of our novel three-dimensional visual inspection system could improve determination of the size of lesions with different shapes, flat or depressed, regardless of the curved surface of the gastrointestinal tract.