Study on the necessity of establishing customized limits for the daily QA program performed with Daily QA3

Background: Due to no corresponding action limits for some new daily QA items , limits of the monthly or annual QA procedures have to be used for daily QA program at present. The general purpose of this paper is to discuss whether it is feasible to do so. Methods: 3 groups daily quality control process during 2017-2019 performed with Daily QA3 were analyzed using SPC. In Calculation of capability indices for output, daily QA tolerance ±3% in the TG142 was used as specification limit, while for flatness and symmetry, annual QA tolerance ±1% was used. An appropriate number of data points were analyzed , capability indices (C p , C pk , C pm and C pmk ) were calculated, and customized tolerance and action limits were established for each process. Meanwhile, the locations of the UCL 、 CL 、 LCL in I-MR charts of daily QA performed by 5 therapists in turns were compared with that of daily QA performed by only one physicist during the same period . Results: Process capability indices of output were all ≥1, some were even up to 3-4. While, for symmetry, most of them failed to meet the requirements. For the same daily QA item, the calculated customized limit of different quality control processes were quite different. For the daily QA performed with only one physicist, the range of UCL and LCL was smaller or the central line was closer to the target value 0 in I-MR charts. Conclusions: It is necessary to establish customized limits for daily QA program performed with Daily QA3, because customized limits and improved operation constancy both can improve the daily quality control level.


Introduction
If the performance of linear accelerators deviates from its normal performance, the effectiveness of radiotherapy may be affected [1] .In AAPM TG-142 , it is recommended that the geometry or dose parameters that may influence the radiotherapy dose should be checked as daily QA procedures [2], and gives the tolerance limit of corresponding procedures.
However, the tolerance limits in AAPM TG-142 is the minimum requirement for quality control, which is mainly based on clinical efficacy without considering the differences during actual quality control process or just assuming that all quality control processes have the same deviation [3]. In addition, with the rapid development of equipment such as morning check equipment and smart radiotherapy platforms [4,5,6], some monthly QA procedures such as flatness and symmetry have gradually become check items for daily quality control. But, there is no corresponding tolerance limits for these new daily QA items currently and limits for the monthly or annual QA procedures have to be used in most cases recently.
Tolerance limits based on the process are needed, and the method of statistical process control (SPC) can be used establishing such customized limits. Firstly, SPC was applied to industrial manufacturing [7,8,9], and gradually used to the aviation and health care [10,11]. In 2005, Pawlicki et al [12] analyzed the quality control of radiotherapy through SPC.
Subsequently, this method was also applied to the QA of linear accelerator [13,14], tomotherapy [15], proton accelerator [16] and intensity modulated radiotherapy plans [17,18,19]. In 2013, Sanghangthum et al [20] proposed to use SPC to establish customized tolerance and action limits. In AAPM TG-142 , it is also recommended that there are three types of actions, with an action priority ranking from lowest to highest, as follows: Level 1 Inspection Action, Level 2 Scheduled Action, and Level 3 Immediate Action. The level 1 threshold is not a critical requirement but it can lead to significant improvements in the QA program. However, there is no literature reporting on the impact of only taking Level 3 action 4 limit.
The purpose of this study is to calculate capability indices of 3 groups daily quality control processes that only taken Level 3 action limit, then to discuss the necessity of setting customized tolerance and action limits.

Background
Statistical process control (SPC) is a process control tool that uses mathematical statistics to find signs of systemic factors in time and take measures to eliminate their effects. It is mainly applied in two aspects: one is to use the control chart to analyze the stability of the process, distinguish the random error and system error in the process by the threshold of the upper and lower control lines, and provide early warning for abnormal factors in the process; the other one is to evaluate the process quality using process capability indices and provide numerical measures of whether or not a QA process is capable of meeting a predetermined level of machine tolerance.
Shewhart chart is one of the the most commonly used control charts for continuous data. It includes upper control line (UCL), central line (CL) , lower control line (LCL), and some data points arranged by sequence of time. If all the data points fall between UCL and LCL, it indicates that the process is stable, that is, the process is only affected by random errors.
Usually, the upper and lower control lines are equal to the center line ± 3σ, which means that when the data follow a normal distribution, 99.7% of the data points will fall between the upper and lower control lines, i.e., when the process is controllable, the data point only has a 0.3% chance of falling outside the control lines. If the subgroup size of the quality control item is 1, the individuals and moving range (I-MR) control chart is used. The calculation formula for UCL, CL and LCL in I-MR control chart is respectively: LSL USL C p   Where x is the mean value;  is the standard deviation; USL and LSL are the upper and lower specification limits, and T is the target value preset by the product designer.

Methods of setting tolerance limits and action limits
We use SPC to analyze equipment, quality control personnel, and measurement as a whole process. The tolerance limit was established using the larger one of the upper control limit (UCL) and the lower control limit(LCL) in the I-MR control. The action limit is calculated by the formula of Sanghangthum [20] as: Where the A is used to adjust the width of the limit to balance the occurrence of type I errors (errors rejecting true null hypothesis) and type II errors (errors accepting the false null hypothesis) in the statistical process. The calculation is performed with A being 6, 5, 4, 3 respectively, and the calculated value of which covers the tolerance limit with the minimum A value is used as the action limit.

2.3Materials and Methods
All measurements were carried out using Elekta Synergy linear accelerator at Shanxi Before the calculation, it is determined whether the data meets the normal distribution. If not, the calculation is performed after Johnson transformation. Before calculating capability indices , the process should be in control. In Calculation of capability indices for output, daily QA tolerance ±3% in the TG142 was used as specification limit, while for flatness and symmetry, annual QA tolerance ±1% was used. The requirements are met when the values of Cp, Cpk, Cpm and Cpmk are ≥1. 8 The tolerance and action limits of four groups quality control processes were calculated respectively. Taking the data in 2019 for an example, process-based tolerance and action limits were used to monitor and analyze the rest 60 data points, and for verification, the measurement results of ionization chamber and Mapcheck2 can used to compare.

The effect of individual operation constancy
The process control charts were calculated using 30 sets daily check data points collected by 5 therapists in turns and only one physicist respectively. Then the effect of individual operation constancy for the location of the UCL、CL、LCL was analyzed .

Discussions
The study of Sanghangthum et al [13] found , the results become more consistent when the number of data points to calculate limits was increased. If the control line is too wide, some errors will be omitted , while too narrow, the error may be a false positive . So they recommended at least 8-12 (2-3 months) weekly QA data points should be used for calculating limits, and they speculated 2-3 weeks daily QA data points maybe needed. The same method was used in our study, and we found at least 6-8 weeks (30-40) in control daily check data points should be used for calculating the I-MR control chart to ensure the stability of control lines. Due to the increase of uncontrollable risks during long-term operation of the quality control process, the more data points may not be the better [13]. Therefore, when using SPC to study the quality control process, we should firstly calculate the minimum number of calculation points required for the control line to be stable by analyzing the rule of changes of 10 x /σ . When calculating the process capability index, the size of sampled data shall be greater than 25 to be representative. In our study, 40 data points were used based on the study result of control line stability .In our analysis of the process capability indices, we found that the indices part of the output was up to 3-4, which was much higher than A ++ (Cpk ≥ 2.0) level.
But the result does not mean that the quality control process has reached an excellent performance and cost reduction can be considered. That was due to the specification limit was too loose as which tolerance ±3% in the TG142 was used. Because the tolerance ± 3% is the minimum requirement which meets the clinical requirements. In order to improve the quality control level, tolerance specifications shall be more strict. In addition, the fact that the capability indices of transverse symmetry failed to meet the requirements does not mean 4 groups of quality control processes are all poor. It was due to the specification limits were too strict as which annual tolerance ±1% was used. Since a three-dimensional water tank or a large parallel plate detector are used for the annual check of symmetry, the performance of these equipments is better than daily check equipment and the tolerance requirement shall be more stricter [2]. Binny D et al. [4] ever pointed out transverse and axial symmetry measurement variation from baselines measured with water tank were within ±1.5% and flatness variations from baselines were within ±0.5% . That explains why most process capability indices of symmetry failed to meet the requirements while the flatness can for the same quality control process. Therefore, only establishing more suitable specification limit , the capability index can be used to better evaluate the quality control process.
The tolerance limits were used in this study for Level 1 and 2 actions in TG-142 report.
Exceeding the tolerance limits indicate that the process may change, but observation can be continued or actions can be taken as appropriate. The action limits were used for Level 3 actions in report TG-142, and immediate action shall be taken if exceeding the action limits. 11 There are two types of action limits. One kind is those that are defined by professional societies, guidance documents, or best practice documents, such as the recommended limits in report No. TG-142. These are taken as universal or absolute recommendations that are not varied from institution to institution and exceeding these limits are likely to affect clinical treatment effect. The other kind is unspecified action limits that are determined empirically by an analysis of available data or sometimes simply by using clinical experience and exceeding these limits does not necessarily affect the clinical treatment effect, but staying within these limits can improve the treatment process [20]. In this study, the action limits belong to the second type and are process-based action limits. What's more, the calculation of the action limits are set considering the Taguchi-type process capability index Cpm = 1.33. It can be seen that when the quality control process is not managed by the SPC method, even if the tested accelerator, quality control equipment and testing personnel remain basically unchanged, the quality control process may be significantly different(see Table 2). The maximum value of A for the action limit shall not be greater than 6, which means that the process meets the 6 management requirements; the smaller the value of A, the smaller the change in data and the better the process capability [20] In the work of Sanghangthum et al [20] for the weekly check process of linear accelerator output (the measuring equipment is the morning check equipment RBA-3) and the monthly check process (the measuring equipment is the finger-type ionization chamber), the chosen value of A for the action limits were all 3. The same method was used in this study, the chosen value of A was 4-5. It indicates that our daily quality control capability of the accelerator needs to be improved . the lower control line is significantly higher than the that of the upper control line, and the tolerance and action limit are determined by the control line with a larger value. So it is necessary to manage and adjust the quality control process using the SPC method before setting customized tolerance and action limits.
When using tolerance and action limits to monitor the quality control process, the system deviations in the process should be detected in time to ensure effective monitoring of changes in accelerator dose parameters. Seen from figure 2, it appeared a system error in the process from the 45th data point. However, during the same period, the measurement results of Mapcheck2 did not show systematic deviations. So the possibility for the dose parameters of linear accelerator to change is small. In addition, the daily check process was getting worse progressively, so the system error may be caused by the worsening constancy of detector of Daily QA3. After recalibrating the detector constancy, the system error was eliminated. So as the morning check equipment has been used for many years and the detector is severely aged, it is necessary to increase the frequency of calibration or replace it with new test equipment. It was also found that the range of tolerance limits of axial symmetry was greater than that in transverse direction in 2019, which was consistent with the measurements of Mapcheck2. It can be concluded that this phenomenon is indeed the true state of accelerator, and can be effectively detected using customized tolerance and action limits.
Measurement accuracy refers to the degree of consistency between the measured value and its true value; and precision refers to the degree of consistency between the measured values.
The smaller the deviation between the average value and the target value in the Shewhart chart, the higher the accuracy of the process; the smaller the range between the upper and lower control lines, the higher the precision of the process. The result of Figure 3 indicates that when the operation constancy is good, the daily quality control level is higher.
Limitations of this study: none of the 4 groups of quality control process were managed by 13 SPC method. If having a comparison, it could better explain the positive effect of SPC method in the improvement of quality control process. The authors intend to compare the improved quality control data with existing data in the next work and further study the positive effect of SPC method.

Conclusions
It is necessary to establish customized limits for daily QA program performed with Daily QA3, because customized limits and improved operation constancy both can improve the daily quality control level. Figure 1 Signal to noise ratio ( x /σ) normalized by the number of in-control data points Figure 2 Monitoring charts using the process-based tolerance and action limit in 2019 Note: "△" represents the first 40 data points used for calculating the tolerance and action limit; "□" represents the 60 data points requiring monitoring analysis. The dashed lines represent the upper and lower control lines and central lines of the control chart calculated from the first 40 data points; the solid lines represent the range of action limits calculated from the first 40 data points.