Four different types of targets were selected for this study: C-shaped target, oropharyngeal cancer, metastatic spine cancer, and prostate cancer (Fig. 1). The C-shaped target was delineated on the water phantom, according to the TG-119 report , and the remaining three types of targets were outlined by senior residents in our research institution. The structures of the targets were exported from treatment planning system (TPS) Raystation (Raysearch, Stockholm, Sweden) in the form of a DICOM file, and the position information of the contours were read by an in-house developed Python software (version 3.7.3), and then used to perform the geometric transformations. Finally, the transformed structures were imported back to Raystation system in the form of a DICOM file. The targets (original contours) before the transformation were regarded as the delineation results by junior residents (test contours), and the transformed targets were regarded as reference contours after systematic and random errors correction by senior doctors.
In this study, the contour errors were introduced in the form of geometric transformations. The translation transformations were divided into the following three cases: right, anterior, and posterior direction. Based on the location of the original contours, at intervals of 1 mm, the data were moved 10 times to each of the right, anterior, and posterior directions to obtain the reference contours (see Additional file 1: Supplementary Figures S1–S3 for the contours after the right, anterior, and posterior directions translation, respectively). Scaling transformation represented an equidistant expansion or reduction transformation in reference to the position of the original contour. Considering the fast speed of scaling transformation changes, in the patient modeling module of the Raystation planning system, 10 equidistant transformations were performed at 0.5 mm intervals, excluding the anterior and posterior directions (see Additional file 1: Supplementary Figures S1–S2 for the contours after the expansion and reduction transformation, respectively). The rotation transformation involved taking the origin of the CT image coordinates as the rotation center point, using 1° as the interval, and rotating clockwise 10 times (see Additional file 1: Supplementary Figures S6 for the contours after the rotation transformation). For the sine function transformation, we extracted the coordinate values (x0, y0) of the original contour first, and then, we used the function y = sinωy0 (ω = 3, 4, 5, 6, …, 12) to carry out periodic transformations 10 times with a fixed amplitude (see Additional file 1: Supplementary Figures S7 for the contours after the sine function transformation).
Through the above seven geometric transformations, systematic errors were introduced into the C-shaped target. In order to introduce random errors, the 5 layers of CT images were randomly selected to keep the position information of the structure unchanged, the remaining CT image layers were divided into three parts, and the above geometric transformations were carried out randomly, and 20 delineation results were obtained for each target.
In this study, we chose five widely used geometric indices for the evaluations, including three distance-type indices HD (maximum, mean, 95%) and two volumetric indices (DSC and Jaccard). These five geometric indices were calculated by 3DSlicer version 4.10.2 , which is open source software. The calculation of HD was performed on the RT-DICOM structures. The HD indices calculated by the 3DSlicer represent bi-directional distances, and the bi-directional distance is symmetrical; this type of distance is more stable than the unidirectional distance calculated by other methods.
The original clinical plans for these four targets all used IMRT technology. The C-shaped target met the requirements for a simple version in the TG-119 report, the dose of 5000 cGy received by 90% of the target volume was taken as the prescription, and the dose prescriptions for oropharyngeal cancer, metastatic spine cancer and prostate cancer were 95% of the target volume receiving 5400, 3000 and 5600 cGy, respectively, and the dose grid was 2 mm. After geometric transformation, the RTstructures were imported into the radiotherapy plan of the original clinical plans, and then, on the dose distribution, D98%, Dmean, and D2% of the PTV were obtained. According to the ICRU-83 report , these dosimetric indices represent the minimum dose, mean dose, and maximum dose received by the target, respectively. In this study, the dose differences (ΔD) of three dosimetric indices D98%, Dmean and D2% were calculated and normalized according to their respective clinical goals . Here, , where x represents the type of dosimetric index.
An SPSS version 21.0 software (SPSS, Chicago, IL, USA) was used for linear regression analysis. The correlation coefficient R² was used to quantify the correlations between the geometric indices HD (maximum, mean, 95%), DSC, and Jaccard, and the dosimetric indices D98%, Dmean, and D2%. Two-sided p-values were obtained, and p-values <0.05 were considered significant. In addition, the geometric indices obtained from the equidistant scaling transformations and the right, anterior, and posterior directions of the C-shaped PTV translation transformations were compared, and the difference between them was tested for statistical significance using the Wilcoxon signed ranks test in SPSS, and from scatterplots of geometric indices versus dose difference, the feasibility of assessing the accuracy of contours with geometric indices was analyzed.
valuations. The subjective evaluation is only based on the experiences and personal preferences of the evaluators. Evaluators were guided to turn off the original contour display and grade all research contours using 3 levels: useful as test contours (= 1), useful with minor edits (= 2), and not useful (= 3). The definition of minor edits was that the test contours would be acceptable after minor modifications . This evaluation method is deeply affected by the individual differences among the evaluators and required considerable time. Most contour accuracy studies are performed directly by using quantitative evaluation, which involves the employment of geometric indices to characterize the similarity between the test contour and the reference contour . Geometric indices widely used in contour evaluations include distance-type geometric indices (e.g., the maximum (HD), mean (HDmean), and 95% Hausdorff distances (HD95)) and volumetric geometric indices (e.g., Dice-similarity coefficient (DSC) and Jaccard) . Although these indices are easy to calculate, they do not consider the clinical effect and may lack clinical relevance [10, 11]. Under the assumption of a reference contour, the method for clinically assessing the accuracy of radiotherapy (RT) contours is to determine and predict the deviation of its dosimetric indices based on the dose distribution of the radiation treatment plan [10, 12-14].However, the relationships between geometric indices and dosimetric indices are yet to be further studied. In addition, different geometric indices have different properties, but some automatic segmentation studies randomly select geometric indices to evaluate the contour results [15-17].
In the present study, we explored the evaluation accuracy of the geometric indices for evaluating the contours under the systematic and random errors. This study artificially introduced contour errors through the following four geometric transformations: translation, scaling, rotation, and sine function transformation. Then, based on these transformations, we investigated the correlations between the distance-type (HD, HDmean, HD95) and volumetric geometric indices (DSC, Jaccard) and the dosimetric indices (D98%, Dmean, D2%); explored the ability of geometric indices to distinguish the contours with the same transformation type but different directions.