The institutional review board for the protection of human subjects reviewed and approved the research protocol (IRB-201626016).
A total of 28 pairs of pre- (T1) and post-treatment (T2) cephalograms were selected. They were taken by the same X-ray machine. The subjects consisted of 21 females and 7 males. The age for T1 ranged from 12 to 27 years, with a mean age of 15.32 years; the T2 ages ranged from 14 to 29 years, with a mean age of 18.03 years. The experimental design is shown in Figure 1. Calibration rulers were used to control distortions and resolution errors during the printing and scanning process.
One operator identified landmarks on the T1 and T2 digital lateral films using customised software produced by the State Key Laboratory of Virtual Reality Technology and Systems. The landmarks included the upper reference point (URP), lower reference point (LRP), sella (S), nasion (N), pterygoid point (Pt), posterior nasal spine (PNS), anterior nasal spine (ANS), subspinale (A), supramental (B), pogonion (Pg), menton (Me), gonion (Go), condylion (Co), upper incisor edge (UIE), upper incisor root apex (UIA), upper first molar mesial buccal cusp (UM), upper first molar mesial root apex (UMA), lower incisor edge (LIE), lower incisor apex (LIA), lower first molar mesial buccal cusp (LM), and lower first molar mesial apex (LMA).
The structural superimposition method developed by Johnston  for the anterior cranial base, maxilla and mandible was independently conducted by each operator.
Hand tracing superimposition
Three senior orthodontic residences, who finished the superimposition course in our department and attended the hands-on lecture given by Johnston in person, were selected as operators in this study. They independently performed hand tracings of T1 and T2 side by side on acetate paper. Information from the hand superimposition was recorded using a similar method as that developed by the University of California, San Francisco . A series of ten pinholes were drilled into T1 films in the non-information-bearing area surrounding the anatomic region of interest. Four corner pinholes on the T1 films, called positioning reference pinholes, were used to register the scanned hand tracings onto corresponding digital films. The other six pinholes on the T1 films, called superimposing reference pinholes, were used in pairs to register the between-film relationships onto T2 tracings for the three types of hand superimposition methods and to convert the between-film relationship of the hand superimpositions into a digital record by scanning. For the T2 films, only the four corner pinholes were drilled. The lateral films with pinholes and the corresponding hand tracings were scanned (HP Color LaserJet 2840, Hewlett-Packard Company, Palo Alto, CA, USA) in original size and at 600 dpi. The operators identified the scanned pinholes that carried the information on superimposition and image size from the T1 and T2 tracings.
Figure 2a shows the rectangular region of interest enclosed by landmarks on the anterior cranial base, maxilla and mandible used to detect the keypoints. On the anterior cranial base, this area was defined by the URP, S, Pt and N. On the maxilla, this area was enclosed by the Pt, PNS, ANS and A. On the mandible, this area was enclosed by the LM, Pg, Me and Go.
The ORB’s oriented FAST algorithm [23,26] was used to detect keypoints on each confined area. We then used the ORB’s steer BRIEF  algorithm for keypoint description, a modified algorithm used to solve the problem that BRIEF is not rotationally invariant. Finally, a brute-force Hamming distance  was used to match keypoints on two cephalograms.
Considering that the ORB results might not be completely accurate, the Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence (GMS)  algorithm was applied to filter the matching results. Then, we calculated the relative offset distances and rotation angles of each pair of successfully matched keypoints and transferred the T2 cephalogram onto the T1 image accordingly. The automated superimposition results were exported as Photoshop files.
Figure 2b shows the flowchart of the automated superimposition process on maxilla.
Calculation of T2 landmark distances
Photoshop CC 2017 (Adobe, San Jose, CA, USA) was used to register hand tracings onto the corresponding digital cephalograms. The T1 cephalograms in each automated superimposition file were used as templates to measure the T2 landmark distances (T2LDs). The T1 tracings were registered onto the templates by 4 positioning reference pinholes. To avoid landmark identification errors, one operator used the automatically superimposed T2 cephalograms to identify landmarks by the brush tool with a 3-pixel diameter and noted down the landmarks’ coordinators as automated superimposition results. Then, the T2 cephalograms were duplicated, and the corresponding T2 tracings were registered on duplicated films at four positioning reference pinholes. Subsequently, they were superimposed onto the templates registered by the bilateral superimposing reference pinholes of the same superimposition type as the automated one. Finally, the landmarks’ coordinators on the duplicated T2 cephalograms as the operator’s hand superimposition results were noted down. Before importing the next operator’s tracings, we deleted the tracings of the previous one to prevent mutual influence of the superimposition results among operators.
To calculate the operative differences of hand superimposition, T2LDs between operators’ corresponding coordinators were measured (Figure 3).
Before evaluating hand superimposition accuracy, the average coordinators among three operators’ coordinators for each landmark were set as the true values. The T2LDs between each operator’s coordinators and the corresponding true values were measured.
To evaluate the accuracy of automated superimposition, the T2LDs were calculated between the coordinators of automated superimpositions and the corresponding true values.
Statistical analyses were carried out with SPSS 25.0 (SPSS Inc., Chicago, IL, USA). The mean T2LDs of operative differences in hand superimposition, hand superimposition accuracy and automated superimposition accuracy for multiple cephalometric landmarks under each superimposition method were calculated. A paired t-test was applied to examine the statistical accuracy of automated superimposition with hand superimposition. P-values less than 0.05 were considered significant.