Ordinary Least Products Regression: A Robust Statistical Tool for Assessing Agreement Between Measures Attended by High Variability


 Background: Agreement (and disagreement) assessments are essential steps in the evaluation of new and existing methods. We aimed to provide a statistical approach to assess systematic disagreement between two measures/methods when both are attended by random error and high variability. Methods: We applied ordinary least products (OLP) regression and the Bland-Altman method in six simulated pairs of samples. In OLP regression, fixed bias defined if 95% confidence intervals (CIs) of the intercept did not include 0. Proportional bias was defined if 95% CIs of the slope did not include 1. As a comparator, we assessed fixed and proportional bias by the Bland-Altman method. Results: We found divergence between studied statistical method outcomes only for measures with low variability (coefficient of variation, CV < 25,0%). Conclusion: OLP regression is a simple and powerful tool for detecting systematic disagreement when the measures are attended by high variability, as well as behavioral variables.


Introduction
Agreement (and disagreement) assessments are essential steps in the evaluation of new and existing methods [1][2][3] . In agreement studies, measures from one or more methods compared with the measures obtained with one reference method on the same subjects 3

. However, biostatisticians
insist that what should be sought is not an agreement between methods or measurers but disagreement or bias 4 . Disagreement estimates are essential for two main reasons: (i) to calibrate one method or measure against another and (ii) to detect bias, fixed or proportional 1,4 . Fixed bias means that one method gives values that are higher (or lower) than those from the other by a constant amount 5 . Proportional bias means that one method gives values that are higher (or lower) than those from others by an amount that is proportional to the level of the measured variable 5 .
In this sense, although there are still inappropriate applications of statistical methods in some studies, the Bland-Altman approach is the most popular method used to assess fixed and proportional bias in agreement research 6 . Conversely, Ludbrook argues that in favor of using ordinary least products (OLP) regression for comparing methods of measurement 1,4,5,7 . The author argument least products regression analysis is suitable for calibrating one approach against another; as well as, and it is also a sensitive technique for detecting and distinguishing fixed and proportional bias between methods 4,5 . However, this theory was built or at least supported based on a dataset (or simulated dataset) with low variability 4,5,7 . In these studies, the author suggests that OLP regression is a simple statistical tool to assess fixed and proportional using simulated measures of blood pressure (in mmHg). However, all tests were conducted in datasets with coefficient of variation (a variability estimator) lower than 25.0% and sometimes these tests showed divergent outcome (diagnosis) of systematic disagreement, fixed and proportional bias, in comparison with Bland-Altman method.
On the other hand, the agreement between measures from emotion (e.g., Well-Being) or behaviors (e.g., physical activity, sedentary behavior, sleep time), commonly assessed by subjective methods, present more considerable variability [8][9][10][11] than blood pressure measures 12 . A key point in behavioral sciences is an understanding of the sources of variability in behavior: i) systematic variability explains differences in behavior that result from manipulation of the independent variable and ii) unrelated variability represents differences in behavior that result from a systematic error (confounds) and random error 13 . Thus, the researcher attempts to maximize systematic variability and to eliminate systematic error and to minimize random error 13 . In this sense, it will be essential that the instrument has reduced fixed and proportional bias even in situations of the high variability of measurements.
In this article, we present the ordinary least products (OLP) regression intercept and slope confidence intervals (CIs) rationale and their implementation, and interpretation for assessing potential sources of systematic disagreement between two measures from different behaviors. We hypothesized that the OLP regression could also be a simple and powerful tool for detecting systematic disagreement, even in cases in which the measures present high variability.

Materials and methods
To assess properties of OLP regression for assessing systematic disagreement, fixed and proportional bias, we have used a simulated scenario with hypothetical measures based on the literature and created with Stata 14 software (Stata Corp., College Station, TX, USA). The Bland-Altman method was used as a comparator to assess systematic disagreement. The hand-held calculator conducted both statistical analyses.

Simulated dataset scenario
To perform examples of agreement assessments, we simulated six pairs of samples (N=100) using random normal distribution (SEED 010101). To generate the measures, the mean and standard deviation were retrieved from the literature 11,14,15 . The measures were (moderate-to-vigorous) physical activity in minutes/day, sedentary behavior in hours/day and sleep time (duration) in hours/day. To recreate typical agreement research 15-17 in each pair, one measure was a simulation of the variable assessed by questionnaire in youth, and the second one was the same simulation, however, assessed by accelerometer.
In this simulated scenario (described in Table 1), we simulated three pairs of measures with high correlation (r > 0.9) 18 , similar to Ludbrook studies 4,5 ; as well as, we simulated three pairs of measures with low correlation (r < 0.3) 18 , similar to behavior studies 10,11,14 . In this case, the correlation was an outcome of agreement 4,5,7 . Also, among these variables, we also simulated two pairs of measures with low variability [coefficient of variation (CV) < 25.0%] as control agreement, similar to blood pressure measures where OLP regression was validated 4,5 . The simulated dataset presented in Supplementary Table 1.

Ordinary least products regression
The OLP regression, also known as standard major axis regression 5 , is a technique designed to assess systematic disagreement obtained from two subjects or methods attended by random error 4 , especially when it is impossible to decide which should be regarded as a dependent 5 . Using the OLP regression intercept (constant) and slope confidence intervals (CIs). We can examine potential sources of systematic disagreement between two measures: fixed and proportional bias.
The OLP regression assumptions are 1 :  The relationship between x (tested method) and y (reference method) is linear;  Both x and y may be measured with an error or attended by random error;  Errors are independent and normally distributed;

Fixed and proportional bias assessment
In OLP regression, the sum of the products of the vertical and horizontal distances of the x (tested method) and y (reference method) values from the line minimized 1,5 . A detailed discussion about OLP regression models is available elsewhere 1,4,5 . The general formula for the OLP regression line is the predicted value of y as a function of x, in this sense, the OLP regression coefficients and their CIs can calculate several ways 5 . Here, we used one way to assess OLP coefficients and two ways to evaluate their 95% CIs that seem to be less sophisticated approaches 1,4,19 . Initially, it is necessary to find values of b (slope) in the ordinary least square (OLS) models where they are estimated by OLS regression of y on x and x on y, respectively.
where r is the Pearson (product-moment) correlation coefficient 20 Secondly, the coefficient of the slope, b', of the OLP regression is given by Finally, because the OLP regression line passes through the point mean x and mean y, the intercept, a', can be obtained from the formula 5 : To calculate the OLP regression intercept and slope (95%) CIs, it is necessary to assess the standard error based on standard major axis (SMA) regression (also called reduced major axis or RMA regression) technique 21 . CIs can calculated by the Student's t-distribution and z-distribution method with n-2 degrees of freedom 22,23 . However, there are other complex methods to calculate OLP regression intercept and slope CIs. For a detailed discussion about this topic, please see Ludbrook 5 .
In the SMA technique, the OLP intercept and slope standard errors, SE, can be assessed by 21 and Where r is the Pearson (product-moment) correlation coefficient between x and y. 95% ( ′) = ′ ± 2 × ( ′ ) (11) and Another way used to assess OLP coefficients and their CIs can be using values of the z-distribution method (α = 0.05) based on normal sample distribution and the central limit theorem 24 and Finally, to interpret the sources of disagreement, fixed bias was defined if 95% CIs of the intercept did not include 0. Proportional bias was defined if 95% CIs of the slope did not include 1 4 .

Bland-Altman method
Agreement between two methods or measures can be quantified using the differences between observations made using the two methods on the same subjects. Bland and Altman established a method to quantify the agreement between two quantitative measurements by constructing limits of agreement 25 . In the Bland-Altman method, the calculation of the 95% CIs for the mean difference provide a fixed bias estimate; whereas, the presence or absence of the correlation between averages and differences indicate proportional bias 26 . Another import inference assessed by Bland-Altman is the 95% limits of agreement (LOA, not addressed in this study) estimated by mean difference ±1.96 standard deviation of the differences 27 . These limits indicate the range within which a single new observation taken from the same population would be expected to lie 26 . In this sense, LOA is not exactly a parameter for fixed and proportional bias. A comprehensive description of LOA characteristics has published elsewhere 26,27 . Moreover, the Bland-Altman method based on the assumption that the differences are normally distributed 27 . Thus, investigators also should check the normality of disagreement between the tested and reference measure or method 27 .

Fixed and proportional bias assessment
In the Bland-Altman method, the Bland-Altman (average difference or LOA) plot and analysis are used to compare two measurements of the same variable 27 . That is, it is a method comparison technique.
Firstly, it is necessary to estimate the average (a) and difference (d) between the tested (x) and reference (y) method or measure using the equations 25,27 ( ̅) = ∑( and Thus, to assess fixed bias, mean differences, and their 95% CIs should be calculated 27 . The formula for 95% CIs for the mean difference is as follows 27 where t is the value of Student's t-distribution corresponding to two-sided α = 0.05 at n-1 and n is the sample size, and SE is the standard error of the mean for difference 27 assessed by ( ̅ ) = √ (18) and the SD is the sample standard deviation for differences 26 Besides, to examine proportional bias, the correlation between average and difference should be checked 27 . The (Pearson product-moment) correlation coefficient can be obtained via 20 The level of significance (p-value, p) of the correlation coefficient assessed by the formula where t is the value of Student's t-distribution corresponding to two-sided with df = n-2 20 .
In disagreement assessment, an obvious starting point is a difference between measurements (fixed bias) of the two methods on the same subject 27 . Additionally, if the difference increases or decreases (proportional bias) progressively as the average value increases is another essential aspect of the disagreement diagnosis 26 . If the difference limits include 0, then it is assumed that no fixed bias is present 25,27 . On the other hand, if there is no correlation (p > 0.05) between average and difference, the differences are uniform (homoscedasticity), and there is no proportional bias; however, if the differences increase or decrease progressively (heteroscedasticity, p ≤ 0.05), there is proportional bias 25,27 . Table 2 shows the summary of OLP regression disagreement analysis. The six hypothetical pairs of measures were then re-analyzed by the Bland-Altman method (Table 3), and the disagreement outcomes from the OLP and Bland-Altman methods shown in Table 4. In this sense, we found partial divergence between studied statistical method outcomes only for measures with low variability (CV < 25,0%), also called control agreement. In addition, we found complete divergence when the measures presented low variability (CV < 25,0%) and low correlation (r ≤ 0.1). An example of the statistical implementation is presented in the Appendix.

Discussion
In this study, we provide a statistical procedure to assess systematic disagreement between two measures or methods when both attended by random error and high variability. Thus, the step-by-

Fixed and proportional bias assessments
In our simulated scenario, we assessed the disagreement between six pairs of measures: physical activity, sedentary behavior, and sleep time. As we hypothesized, we found the same outcomes (diagnoses) for fixed and proportional bias for all agreements with high variability. On the other hand, for measures with low variability, we found divergent outcomes of fixed and/or proportional bias between OLP regression and Bland-Altman method (Table 4). Our results are in line with Ludbrook studies 4,5 , where outcomes for fixed and proportional bias cannot be equal between OLP regression and Bland-Altman method for blood pressure measures with high correlation and low variability (in a simulated scenario).
The divergence between OLP regression and Bland-Altman method can also be seen in a real sample (with high correlation and low variability) 19,28 . A potential explanation for this finding is attributable to the presence of a large amount proportional bias 26 . Briefly, if there is proportional bias, then the mean difference will almost inevitably deviate from 0 4,26 . Thus, there is a risk that fixed bias will be overdiagnosed 4 . The exception to this is if proportional bias is in one direction (e.g., proportional bias > 0) and fixed bias in the opposite direction. Then the mean difference may be close to 0, and fixed bias will be underdiagnosed 4 .
Additionally, we hypothesized another potential explanation based on the evaluation of gene expression measurements 29,30 , where the coefficient of variation (variability) used as a degree of reproducibility. In this case, the comparison of measure agreement on the same set of subjects can lead to a comparison of dependent measures. And, the low variability can reinforce this dependence and confound systematic disagreement diagnoses. As suggested by Tan et al. 29 , a dependency between the datasets would confound any inferences we could make about the differences.
OLP regression is a simple way to evaluate the disagreement between two methods or measures, where the sum of the products of the vertical and horizontal deviations of x, y values from the line minimized 5 . Likewise, the Bland-Altman method is a simple way to evaluate bias between the mean differences and to estimate an agreement interval 27 . Thus, note that OLP regression (as well as the Bland-Altman method) defines if there is fixed and/or proportional bias significantly different from 0. These statistical procedures do not say whether those differences are acceptable or not. Acceptable disagreements must be defined a priori and based on clinical necessity, biological considerations, or other goals.

Limitations and strengths
Some restrictions should be noted in this study. Firstly, the OLP regression confidence intervals are assessed by the and-held calculator. Although least products analysis is more difficult to execute by hand-held calculator than the Bland-Altman method, the bias diagnosis is easier to interpret.
Performing OLP regression in statistical software can eliminate this difficulty. Besides, we achieved different values of confidence intervals coming from the t-distribution (used in this study) and z-distribution methods; however, the diagnosis/outcomes were the same. Some methodologists favor the use of more precise CIs for effect sizes; for example, CIs estimation based on bootstrap or bias-corrected bootstrap methods 5,22 . However, in previous experimental agreement assessments, we achieved more approximated (but not the same) by t-distribution confidence intervals compared with estimations made by statistical software 4 .
Moreover, in both cases, t-distribution and z-distribution, the diagnosis agreed with those found by Ludbrook 4 . Besides, our results and discussion were grounded in a simulated scenario, which can be considered a potential limitation. Studies in a real sample should be conducted to confirm our findings.
Despite these limitations, our study has several strengths. The present study provides a simple statistical method made by hand-held calculator to assess sources of disagreement, fixed and proportional bias, as well as an analytical method to evaluate disagreement when the reference method is sometimes known as the 'gold standard', but this does not, or should not, imply that it measured without error. Our study also provides an alternative to quantify proportional bias and to deal with cases in which there is a full presence of proportional bias affecting fixed bias estimation for the Bland-Altman method, which has been extensively used worldwide 6 . And, finally, this is the first time that OLP regression compared with the Bland-Altman method for assessing systematic disagreement when the measures attended by high variability.
In conclusion, ordinary least products regression is a simple and powerful tool for detecting systematic disagreement when the measures attended by high variability, as well as behavioral variables. The convergence between least products regression and Bland-Altman method seems to improve when the variables accompanied by high variability. Our findings, in a simulated scenario, demonstrate that ordinary least products regression using intercept-slope confidence intervals interpretations could achieve an acceptable convergence for systematic disagreement outcome in comparison with the most popular statistical method used in agreement studies, providing scientific evidence that ordinary least products regression could be implemented as an analytical technique as We assessed the significance level by Based on Student's t-distribution, the significance level (using d.f. = n-2 = 98 and t =-1.719, twosided t-value) was 0.0887.