Variable importance and selection
To analyze the severity of the intervention based on the MGS image scores, a total of 4944 images were randomly selected for evaluation using a picture selection tool similar to our previous studies 15. Of these images, 749 could not be included because of poor quality or non-recognizability (are marked as -1= rejected in the raw data) of the evaluation criteria (e.g., whisker change). Data were integrated for mean values in terms of repeated measurements from different video sources. Further, in addition to the five MGS criteria, the time resolution of the measurements was noted in two variables “week” (0, 1, 2, 3, 4) and “day” (day 1, 2, and 3) as well as the variables treatment (Oil, CCl4), intervention (baseline, pre and post), and animal ID. The final data set had the dimensions of 498 rows with n=24 unique animal identifiers.
Initially, the priority of the different MGS evaluation criteria was determined with the MoPBs algorithm. As a result, the expressiveness of certain parameters was ranked and quantified relative to the most meaningful value (defined as 100%). Figure 1 shows the result of these analyses and identifies orbital tightening as the first-ranked parameter and whisker change as the last-ranked parameter. Further, the algorithm explored criteria combinations like OT and NB as second best, etc.
In addition to the expressiveness, time- and intervention-independent correlations of the grimace scale criteria in each treatment group were analyzed (Table 1). The overall correlations in the CCl4 group were higher than in the Oil group. In both treatment groups, the NB~CB combination shows the highest correlation of all criteria (Oil, rNB~CB= 0.817; CCl4, rNB~CB= 0.901).
Table 1. Time- and intervention-independent correlations of the grimace scale criteria.
|
|
Orbital Tightening
|
Nose Bulge
|
Cheek Bulge
|
Ear Position
|
Whisker Change
|
Oil
|
Orbital Tightening
|
1
|
|
|
|
|
Nose Bulge
|
0,718
|
1
|
|
|
|
Cheek Bulge
|
0,728
|
0,817
|
1
|
|
|
Ear Position
|
0,572
|
0,707
|
0,777
|
1
|
|
Whisker Change
|
0,615
|
0,78
|
0,812
|
0,81
|
1
|
CCl4
|
Orbital Tightening
|
1
|
|
|
|
|
Nose Bulge
|
0,876
|
1
|
|
|
|
Cheek Bulge
|
0,898
|
0,901
|
1
|
|
|
Ear Position
|
0,836
|
0,835
|
0,848
|
1
|
|
Whisker Change
|
0,801
|
0,843
|
0,855
|
0,861
|
1
|
In general, however, the results show that all parameters are highly correlated and will, therefore, show strong collinearity in regular regression analysis. To compensate for this, we used a penalized maximum likelihood regression that was capable of both, variable selection and regularization of the model. We used 10-fold cross-validation to minimize the mean squared error on the λ estimator (λ1SE,Oil=0.001, λ1SE, CCL4=0.306). Figure 2 shows the result of the coefficient ranking from the LASSO regression. A time-independent analysis showed that the orbital tightening parameter in both treatment groups and interventions had the largest values βCCL4,OT,post=0.295, βCCL4,OT,pre= 0.293, compared to βOil,OT,post=0.215, βOil,OT,pre= 0.214. Interestingly, the second strongest parameter in both treatment groups was found to be the EP parameter (βCCL4,EP,pre= 0.289, βCCL4,EP,post= 0.288, compared to βOil,OT,pre= 0.182 and βOil,OT,post= 0.182). Although, not a combination of parameters, this is similar to the findings of the MoBPs algorithm, where the second-best full parameter is also ear position (Fig. 1, full green bar). However, in terms of the weakest contributing variable, the two different methods showed different results. The MoBPs algorithm finds whisker change as the worst-performing variable, while the LASSO regression finds nose bulge, again in both treatment methods. In the regression model, whisker change is performing better than cheek bulge in the CCl4 group. In the control group, this was reversed.
Due to the overall agreement of the high applicability of the orbital tightening in our results and the simultaneous easy recognizability also for future automated examination procedures, we have selected the orbital tightening as a "target parameter" for subsequent examinations.
The regression model of the OT analysis
In the second part of the analysis, multiple linear mixed regression models with orbital tightening as the dependent variable were built to analyze different treatments and interventions over time affecting the orbital tightening variable (Table 2). The main target factor is the investigation of the effects of the parameter OT on the treatment, the intervention and the time.
Table 2. Overview of the regression models with Fixed Effects (FE) and Random Effects (RE) parameters.
Model
|
FE
|
RE
|
type
|
I
|
treatment : day : intervention
|
animal ID & week/day
|
between-treatments
|
II
|
week: intervention
|
animal ID
|
within-treatment CCl4
|
III
|
week: intervention
|
animal ID
|
within-treatment Oil
|
Model I - Orbital tightening between-treatments analysis
In model I (Supplemental Material S2-3), the highest available time resolution “day” was included in an interaction with the “intervention” variable and the “treatment” groups (Oil and CCl4). The between-treatments model (I) with animal ID as RE was extended by a random intercept term in which “day” was nested within the “week” variable (βIntercept=2.59, CI95%[2.04; 3.14], p<0.001). From the total variance, the animal ID was able to explain 21.56% (τID= 0.32), the interaction day:week 5.33% (τday:week= 0.08) and week 0.77% (τweek= 0.01) of the variance in the data. The remaining unexplained variance remained high with 72.33% (σ2= 1.09). With the between-treatments model (I), no significant difference between treatment groups was found. However, there was evidence for a potential difference (βCCl4= 0.601, CI95%[-0.05; 1.26], p=0.069). Compared to the given default levels in the oil group, CCl4 showed higher values in orbital tightening (βIntercept=2.59 + βCCl4= 0.601 = 3.191). Despite this large estimate, the effect was not significant at the α=0.05 level and the given variance. The model found a significant general difference for the “intervention” predictor between treatments (βCCl4= 0.52, CI95%[0.07; 0.96], p=0.022). In terms of the time- and treatment-independent intervention effect, post-intervention was significantly higher than pre-intervention. This difference was most prominent in the CCl4:intervention interaction, when compared to the default levels of the treatment-model (βCCl4:intervention= 1.03, CI95%[0.36; 1.7], p=0.003). While the between-treatments predictor was not significant, the interaction with intervention shows that CCl4-post-intervention was higher than Oil-pre-intervention. In model I, “day” or its interactions with “treatment” or “intervention” did not show significant differences (Fig., 3A).
Model II - Orbital Tightening within-CCl4 analysis
The analysis in model II focused on CCl4 data (Supplemental Material S2, S4). Here, the within-treatment development of severity over time was modeled. Therefore, baseline data (at week 0) with missing interventions were excluded. As a result, the default level of “week” was 1 in this model. Baseline level comparisons are shown in model I. Orbital tightening was modeled as a function of the interaction terms “week” and “intervention” (βIntercept=3.30, CI95%[2.77; 3.83], p<0.001) with animal ID as random effects. The animal ID was able to explain 24.51% (τweek= 0.341) of the model variance. The residual variance remained high at 75.49% (σ2= 1.392). Compared to the default levels, only the time-independent “intervention” predictor was significant (βintervention= 1.73, CI95%[1.12; 2.34], p<0.001). Thus, an intervention increased the orbital tightening value from 3.3 to 5.03 units. No other within-treatment coefficient or interaction with “week” was significant (Fig. 3 B). Nevertheless, the week:intervention estimates in the model showed a continuous decrease over time, indicating a return of the orbital tightening values towards the default levels (week 1, pre-intervention) (βweek2:intervention=-0.57, CI95%[-1.47; 0.33], p=0.21; βweek3:intervention= -0.40, CI95%[-1.31; 0.51], p=0.391; βweek4:intervention= -0.75, CI95%[-1.73; 0.22], p=0.22).
Model III - Orbital tightening within-Oil analysis In the third model (III), baseline data were excluded in the same way as in model II (Supplemental Material S2, S5). The orbital tightening was modeled as a function of the interaction terms “week” and “intervention” (βIntercept= 2.70, CI95%[2.24; 3.15], p<0.001) with animal ID as random effects. The animal ID was able to explain 29.80% (τweek= 0.283) of the model variance. The residual variance remained high at 70.20% (σ2= 1.392). No significant coefficients were found (Fig. 3 C). The week:intervention coefficients remained inconclusive of a trend and were small (βweek2:intervention= 0.38, CI95%[-0.31; 1.07], p=0.282; βweek3:intervention= 0.01, CI95%[-0.64; 0.66], p=0.978; βweek4:intervention= 0.11, CI95%[-0.56; 0.77], p=0.754), indicating no intervention effect in general or over time.
Severity classification and Pain assessment
Figure 4 A shows the time-dependent group contrasts in the treatment groups, colorized by within-subjects differences of interventions. Notably, the variance was high in all contrasts. The regression models I-III have shown that there are large amounts of variance in the groups that cannot be explained with any of the experimental variables. The resulting intra-class correlation coefficients were, therefore, small (ICCI= 0.28, ICCII= 0.20, ICCIII= 0.23).
In the control group, the median development of the post-interventional severity was not as high as it was in the CCl4 group (see “intervention (post)” in models II and III, Fig. 3 B and C). Both treatment groups started at different baseline values (bootstrapped estimates: Oilweek0=2.76, CI95%[2.37; 3.16], and CCl4,week0=3.25, CI95%[2.74; 3.76]). This difference was significant (W=105, p=0.029). Further, the distribution of data into the three discretized severity classes was also different in the group comparisons. CCl4 showed more directionality towards higher severity in the post-intervention group (red points in the red area) than the control group. Figure 4 B explores the cumulative and time-independent development of severity in the data. For this, data in the discrete classes were counted (Table 3) and expressed as percentages (for absolute numbers, see Supplemental S6). There was a clear trend towards higher severity in the post-intervention procedure in the CCl4 group (also see the “intervention (post)” coefficient in model II). Here, the severity in the post-intervention was always higher than before an intervention (Χ2CCl437.15, df=4, p≤0.001, with padj,mild/moderate=≤0.001, padj,mild/severe≤0.001, padj,moderate/severe≤0.006). In the control group (Oil) this was only found in the mild severity class (Χ2Oil=10.579, df=4, p=0.03, with padj,mild/moderate=≤0.044, padj,mild/severe≤0.285, padj,moderate/severe≤0.627).
Table 3. Severity class distribution in the treatment and intervention groups.
|
|
counts per severity class
|
|
treatment
|
intervention
|
mild
|
moderate
|
severe
|
total
|
Oil
|
bsl
|
11
|
7
|
0
|
18
|
Oil
|
pre
|
93
|
41
|
2
|
136
|
Oil
|
post
|
63
|
59
|
5
|
127
|
CCL4
|
bsl
|
8
|
12
|
0
|
20
|
CCL4
|
pre
|
49
|
54
|
3
|
106
|
CCL4
|
post
|
9
|
69
|
13
|
91
|
Sum
|
|
233
|
242
|
23
|
498
|
Orbital Tightening data were summarized and grouped by “treatment” and “intervention”. Since the orbital tightening variable showed mixed distributions over time (Supplemental Material S7) and the time-independent distribution was also not normally distributed (Shapiro Wilk’s test, p<0.0001), value development was characterized as medians using a 10000-fold bootstrapping from which also the 95% confidence intervals were obtained. The treatment-based medians were depicted and grouped by the intervention (“pre” (steel blue) / “post” (red)), and the corresponding confidence bands (Fig. 5). Week 0 had no injected animals and served as baseline measurement in both treatments. The control group showed no significant difference between the animals at the baseline and after the intervention (week 0 vs 1, W = 291.5, p-value = 0.7875, rcontrol= 0.04). However, in the CCl4 group, a significant difference after treatment between weeks 0 and 1 was found, resulting in a medium-sized effect (W = 62.5, p-value <0.0001, rCCL4= 0.64) and was considered highly significant.