Three-dimensional surface imaging has proven itself as a valid and reliable imaging modality for evaluation of several surgical treatment results of the face. New handheld scanning systems are being released which advertise even higher accuracy than previously widely used static systems. Reliability studies of these new scanners are needed before they can be safely used in clinical practice and research.
Intra-system and inter-system reliability in this study was established by scanning both healthy volunteers and a mannequin’s head. We found that the Artec Space Spider had a better intra-system reliability for both volunteers and the phantom head. The Artec Space Spider captures more details, since little wrinkles and small protuberances of the face can be visualized very well. Comparisons between the Artec Space Spider and Artec Eva showed small differences in reliability. However, compared to these handheld scanners the 3dMD system introduced more inaccuracy as illustrated by the distance maps especially under the nose, under the chin and at the side of the face. These differences could possibly be explained by the fixed orientation of the 3dMD system. The Artec handheld systems can scan in several angles around the face to capture iterative meshes that are stitched together to produce a complete 3D model. Therefore, geometry underneath the nose or chin can be imaged more completely using handheld systems. Because of the narrow field of view of the Artec Space Spider, a lot of iterative meshes must be captured and stitched together to create a 3D model. This process can hypothetically introduce a processing error when scanning large areas. However, since we found an overall high reliability and validity of the Artec Space Spider when scanning an area of the size of the face, we regard the relevance of this error low.
Moreover, since errors were all below 0.5mm, the clinical relevance of these differences are very limited. Other properties such as ease-of-use or time needed to generate a 3D model could dictate the choice for a specific system. However, when measuring volume by comparing larger areas of surface scans of the face, these small differences add up to more inaccurate volume measurements. Therefore, it is recommended for pre- and post-surgery volume measurements, to use a system with the smallest errors such as the Artec Space Spider.
In previous studies, 3dMD was used as reference to compare other scanners with 6. Since reliability of other scanners is better than 3dMD in this study, we believe that 3dMD should not act as a gold standard anymore. We 3D printed a mannequin phantom head with cylinders. Since 3D printing introduces errors, we took linear measurements of the cylinders, in multiple directions on the mannequin’s head, to operationalize this as gold standard. These measurements showed that Artec Space Spider is more valid in capturing the geometry of the face. Based on the linear measurements error, it seems that the Artec Eva was overestimating the volume of the face and the 3dMD was underestimating the volume of the face.
Most studies analyzed reliability and validity of the 3dMD stereophotogrammetry system 2,3,9. A recent study scanning dental casts showed that the Artec Space Spider had high reliability, but low validity (0.4mm) 10. However, the authors attributed the error to the automatic reconstruction function of the software, a function we did not use. Artec Space Spider showed one of the best validity measures when compared to Primescan, Trios, Pritiface and iPhone when used to scan nasal, orbital and auricular models that were manufactured using stereolithography11. Artec Space Spider was found to be most valid compared to Artec Eva, Vectra H1, Bellus and SNAP when used to scan plaster statues with balls attached of which linear measurements were used for comparison12. In the peri-orbital region an excellent reliability of 0.1–0.2 mm of the Artec Space Spider was found 13.
In this study we showed that all scanners produce clinically acceptable results, with errors below 0.3mm for the intra-system reliability. Variation in facial expression is known to introduce an error when comparing multiple facial 3D surface scans, which explains the differences between the human volunteers and the phantom head14. The substantial difference between the phantom head and the face RMS distances shows that the variability of the expression of the face introduces more error than the scanning systems themselves. This finding highlights that standardization of facial expression during scanning could be even more important than scanning system accuracy. For clinical use, certain properties can differentiate their usefulness for specific applications. The Artec Space Spider is especially useful for acquiring scans of objects with complex geometry, since it showed the best validity and captured most detail. Artec Eva and Artec Space Spider are mobile scanners, a property which makes it especially useful when the scanner needs to be used in multiple sites, such as the operating room or multiple clinical centers. In addition to this, the storage space of these mobile scanners is substantially less than the static scanning system 3dMD, which needs a designated room for its setup. However, training and experience is required for the mobile scanners to produce an acceptable quality of the scans since slow and homogenous movements are required during the recording. The 3dMD obtains surface models with a single flash which reduces the risk of movement artefacts during scanning and provides fast acquisition times.
One of the strengths of our study is that we included both intra- and inter-system reliability of a mannequin’s head and human subjects. This facilitates proper assessment of the reliability of both a rigid object with the geometry of the face and the face of a human subject. We also included validity analysis of a 3D printed face and linear measurements of cylinders on this 3D print. However, the main limitation of this study is that for validity measures the true geometry of the human face is not known. It is therefore impossible to define a gold standard which facilitates assessment of the validity of capturing the true complex geometry of the face. We therefore included assessment of a 3D printed face and compared this with the original printed mesh. 3D printing however also introduces a printing error. We therefore included assessment of linear measurements with calipers and compared these with the linear measurements of the 3D scan which showed again that the Artec Space Spider was most valid. Another limitation is the small sample size in this study. Despite these limitations multiple analyses showed consistent results: the Artec Space Spider scanning system was most reliable and valid. Future research on validity of new 3D surface imaging systems should therefore focus on establishing a proper gold standard for facial scanning.
We provided evidence that three scanning systems in our comparison were reliable and valid and concluded that the Artec Space Spider was most reliable and valid. Other properties such as ease-of-use or mobility of the 3D surface imaging systems can dictate which system to choose in practice. Nonetheless, 3D surface imaging is a promising and radiation-free imaging modality that can be safely implemented to assess facial surgical outcomes during follow-up.