Vitamin D recommendations in nutritional guidelines: A systematic quality evaluation using AGREE-2 and analysis of potential predictors

Vitamin D has been widely promoted for bone health through supplementation and fortification of the general population. However, there is growing evidence that does not support these strategies. Our aim is to review the quality and recommendations on vitamin D nutritional and clinical practice guidelines and explore predictive factors for their direction and strength. We searched Medline, TRIP, G-I-N and guidelines.gov databases for guidelines from January 2010 to January 2018. We performed a descriptive analysis, a quality appraisal using AGREE II scores (Appraisal of Guidelines Research and Evaluation) and a bivariate analysis evaluating the association between direction and strength of recommendations, three AGREE II domains’ scores (Rigor of development, Applicability and Editorial Independence) and pre-specified characteristics. We included 24 guidelines in our review. Nine recommended supplementations, eight suggested its use, and seven did not recommended it. AGREE II mean scores were 40.1% for “Rigor of development”, 34.5% for “Applicability, and 39.6% for “Editorial independence”. Guidelines with higher scores in these domains were less likely to recommend or suggest vitamin D supplementation (64% recommending against vs 29% suggesting for, and 27% recommending for; p=0.002, 52% vs 26% and 25%; p=0.003, 61% vs 35% and 24%; p=0.001, respectively). Guidelines with potential conflicts of interest were more likely to recommend or suggest vitamin D supplementation (likelihood ratio: p=0.018).


Abstract
Background Vitamin D has been widely promoted for bone health through supplementation and fortification of the general population. However, there is growing evidence that does not support these strategies. Our aim is to review the quality and recommendations on vitamin D nutritional and clinical practice guidelines and explore predictive factors for their direction and strength.

Methods
We searched Medline, TRIP, G-I-N and guidelines.gov databases for guidelines from January 2010 to January 2018. We performed a descriptive analysis, a quality appraisal using AGREE II scores (Appraisal of Guidelines Research and Evaluation) and a bivariate analysis evaluating the association between direction and strength of recommendations, three AGREE II domains' scores (Rigor of development, Applicability and Editorial Independence) and pre-specified characteristics.

Results
We included 24 guidelines in our review. Nine recommended supplementations, eight suggested its use, and seven did not recommended it. AGREE II mean scores were 40.1% for "Rigor of development", 34.5% for "Applicability, and 39.6% for "Editorial independence". Guidelines with higher scores in these domains were less likely to recommend or suggest vitamin D supplementation (64% recommending against vs 29% suggesting for, and 27% recommending for; p=0.002, 52% vs 26% and 25%; p=0.003, 61% vs 35% and 24%; p=0.001, respectively). Guidelines with potential conflicts of interest were more likely to recommend or suggest vitamin D supplementation (likelihood ratio: p=0.018).

Conclusions
Policy makers, clinicians and patients should be aware that lower quality guidelines, and those reporting conflicts of interest are more likely to promote vitamin D supplementation.
Guideline organizations should improve the quality of their recommendations' development, and the management of conflicts of interest. Users and editors should be aware of these findings when using and appraising guidelines.

Background
Vitamin D plays a vital role in several physiological processes (1)(2)(3)(4). The best understood of these functions is calcium regulation, counter regulating parathyroid hormone secretion to maintain calcium serum levels(5) and fixing calcium absorption. It is predominantly synthesized via direct sunlight (ultraviolet B radiation) exposure(6), as it is limited to only a few natural sources(7) in human diet. Severe deficiency leads to rickets in children and osteomalacia in adults; its deficit has been associated with low bone mineral density (8), and increased risk of fractures (9).
Public health concerns about suboptimal vitamin D intake have led to the development of dietary and supplementation recommendations(10) in nutritional guidelines. However, the link between vitamin D intake and fracture risk is controversial (11). These recommendations were initially addressed to populations at risk of low sun exposure, particularly in northern latitudes (12). Early systematic reviews (13)(14)(15) showed a decrease in fractures among older institutionalized women, while more recent ones have demonstrated the absence of beneficial effect (16)(17)(18)(19)(20)(21)(22). Of note is the absence of randomized clinical trials in the general population to evaluate outcomes such as fractures, with only surrogate outcomes (e.g. bone mineral density and 25hydroxycholecalciferol (25-OH-D)(23) being available. Vitamin D supplementation has not shown any effect in preventing cardiovascular disease or cancer (24)(25)(26), and high doses paradoxically, have been associated with an increase in falls in the elderly (27).
Another subject of debate is what constitutes vitamin D deficiency. While some studies declared epidemic proportions of deficiency(28), others have not confirmed these claims (10). Typically assessed with 25-OH-D serum levels, the threshold to maintain adequate bone health is also disputed(10).
Our aim is to review and analyze Vitamin D recommendations present in nutritional and clinical practice guidelines (CPGs) as well as potential related factors for these recommendations.

Methods
We searched Medline via PubMed, Guidelines.gov, the Guidelines International Network (G-I-N), and Trip Database. We conducted our search for CPGs published between January 2010 and January 2018. The full search strategy can be found in the Supplementary File 1.

Eligibility criteria
We included guideline publications, following the definition of the Institute of Medicine(29). We included guidelines that formulated recommendations for vitamin D intake, and/or screening for the general, healthy population. We excluded guidelines addressed to pediatric population (to prevent rickets), guidelines on specific conditions (e.g. chronic corticoid users), guidelines addressed to osteoporotic patients exclusively, or for secondary prevention of osteoporotic fractures.

Data extraction
We collected: year of publication, institution, region, target population, proposed vitamin D levels, calcium advice (if given), sun exposure advice, screening advice, fortification advice, suggested methods for vitamin D obtention, and supplementation recommendations. Three reviewers (DF, EN and JK) conducted screening, data extraction, and quality assessment. We resolved disagreements through discussion or with the help of another reviewer (PA). We classified the recommendations following the distinction that the Grading of Recommendations Assessment, Development and Evaluation (GRADE) provides (30) in three categories: "Recommends in favor", for unequivocal, strong recommendations; "Suggests in favor", for weak or conditional recommendations; and "Does not recommend", for guidelines which did not provided advice in favor or recommended against supplementation.
We also collected information concerning authors' potential conflicts of interest (COIs) and how these were managed. We focused on evaluating individual COIs reporting and the potential Institutional COIs in accordance with methodology implemented in previous evaluations (31,32). We recorded if organizations addressed affiliation, financial and intellectual COIs in their report. We also collected the type of organization that formulated the recommendations and, if available, the institution's potential financial or advocatory COIs (32). Two reviewers (DF and CC) collected this data independently. In case of disagreement, we solved it by discussion and the help of a third reviewer (PA).

Quality appraisal
We used the AGREE II (Appraisal of Guidelines Research and Evaluation version 2) (33) instrument. This tool evaluates the methodological rigor and transparency with which a guideline is developed. We restricted our assessment to the Rigor of Development, Applicability, and Editorial Independence domains (Supplementary Box 1). AGREE II does not provide a "general" score of the guideline quality, hence, given the scope of our review, we focused on three specific subdomains for our quality appraisal. "Rigor of development" provides a rationale of the methodological strength of the guideline. "Applicability" addresses issues regarding the advice given on its recommendations' implementation. Finally, besides our specific analysis of conflicts of interest, "Editorial independence" provided us additional information on this matter. After calibration, two researchers evaluated publications independently. We calculated the percentage of the maximum possible score for each domain, and its standardized range (from 0% to 100%).

Data analysis
We performed a descriptive analysis for each included guideline. We assessed the reproducibility of quantitative measurements made by different observers by calculating the intra-class correlation coefficients (ICC), and the associated 95% confidence intervals (CI) for each AGREE II domain. We conducted a bivariate analysis to evaluate the potential association between recommendations and AGREE II scores with the following factors: region, type of organization, target population, suggested method to obtain Vitamin D, advice on sun exposure, advice on fortifying strategies, and conflicts of interest. We included in our logistic regression model the results that remained significant in the bivariate analysis.

Results
The initial search yielded 491 references. Forty-five guidelines were selected for full text review and 24 guidelines were finally included in our analysis (Figure 1). Nine (38%) guidelines were from the USA or Canada, seven (29%) from Europe, four (16%) from Asia, two (8%) from Australia and New Zealand, and two (8%) from international organizations.
By type of society: fourteen (56%) guidelines were developed by scientific societies, two (8%) by international governmental organizations, and nine (36%) by national governmental organizations.
Nine (38%) guidelines recommended in favor of some form of supplementation of vitamin D at some stage in life, seven (29%) suggested it, and eight (33%) did not recommend or recommended against supplementation. By population, eleven (46%) guidelines recommended or suggested supplementation for the general population, eight (33%) recommended supplementation for pediatric population, six (25%) recommended or suggested supplementation in pregnant women, and twenty-three (96%) recommended it for the elderly and institutionalized patients. The daily intake recommended varied from 400 International Units (IU) to 4000 IU (SD 621.33), the mode being 800 IU. (Table 1 and Table 2 and Supplementary Table 1).
With respect to vitamin D obtention, six guidelines (25%) highlighted the importance of sun exposure to achieve enough vitamin D levels. Twenty guidelines (83%) recommended supplementation as the favored method to obtain adequate vitamin D levels, two (8%) recommended sun exposure, and two guidelines (8%) did not provide a specific method. In the subdomains of AGREE II evaluated (Figure 2 and  reported an explicit process to manage them, in two (8%) the process was unclear and four (17%) did not report any related information. Twenty three out of 24 (96%) reported affiliations, whereas 2 (8%) did not. In 19 out of 24 (79%) reported financial ties, two (8%) did not reported them clearly and four (17%) did not report them at all. Intellectual COI were reported in seven out of 24 guidelines (29%), with 18 (75%) not providing any information. Regarding institutional financial ties five guidelines (21%) reported them, thirteen (54%) did not provide any information, and seven (29%) were unclear. When examining if the institution potentially played an advocatory role favoring intervention, we detected this in five cases (21%), eight (33%) being unclear. Overall, we considered 15 out of 24 guidelines (63%) to have potential COIs ( Table 3).
The guidelines that do not recommend vitamin D supplementation (compared with those that suggested or recommended it) scored significantly higher in the Rigor of Development domain (64% recommending against vs 29% suggesting, and 27% recommending for; p=0.002), Applicability (52% vs 26% and 25%; p=0.003) and Editorial Independence (61% vs 35% and 24%; p=0.001) respectively. A post-hoc analysis showed no significant differences between the "suggest" and "recommend" categories.
Guidelines that provided recommendations in favor of vitamin D supplementation were associated with conflicts of interest, either detected or declared (likelihood ratio p=0.018). In the logistic regression model, Rigor of Development and Applicability were independent predictors of recommending vitamin D supplementation; the higher the score, the less likely supplementation was recommended Odds Ratio (OR) 1.11; CI95% 1.02-1.19 and OR 1.15; CI95% 1.03-1.28 respectively. With respect to vitamin D screening, we found a significant association between recommending screening and lower AGREE II scores for Rigor of Development, Applicability and Editorial Independence (50% vs 24% p=0.002; 41% vs 24% p=0.007; 46% vs 31% p=0.05).
Finally, guidelines that declared no conflicts of interest, and guidelines in which no conflicts of interest were detected scored significantly higher in "Rigor of Development", "Applicability" and "Editorial independence" domains, compared to those guidelines that did not, or for which conflicts of interest were detected (54% vs 32% p=0.033; 48% vs 27% p=0.022, and; 51% vs 33% p=0.024).We did not find any other associations with respect to the rest of predefined variables.

Discussion
Our systematic assessment of vitamin D recommendations shows that despite the lack of conclusive evidence, most organizations recommend vitamin D supplementation for the general population. With a few exceptions, the guidelines included were generally of low quality. More importantly, those with lower scores or with conflicts of interest were more likely to recommend wider supplementation.
In general, our results resonate with previous AGREE II evaluations where low scores were reported for the majority of guideline topics (34)(35)(36)(37)(38)(39). This finding could suggest that guideline development is not directly related to the quality of the underlying evidence.
While the guidelines identified recommended population supplementation, the Institute of Medicine (IOM) landmark systematic report, recognized the low quality of the available evidence about vitamin D supplementation, favoring supplementation only in specific situations with minimum sun exposure (10). Paradoxically, this report is cited in most of the guidelines evaluated, to support strong recommendations towards supplementation.
Furthermore, the IOM report acknowledged that the thresholds proposed by some organizations amplify the sense of an "epidemic of deficiency that does not correlate with epidemiological data" (10). By contrast, renowned organizations evaluated in our study, such as the National Osteoporosis Foundation, do not rate the quality of the evidence, nor the strength of the recommendations for populations with very different baseline risk (institutionalized vs community), and extrapolate effectiveness evidence from women to men. GRADE contemplates five exceptional situations in which a strong recommendation may be warranted in the face of low-quality evidence. For example, when it suggests benefit in a life-threatening situation (40). None of these apply for the situation we described.
Our results are also in agreement with previously reported ties between industry, advocacy and academia in this field (41,42). We provide much needed empirical observational evidence illustrating that conflicts of interest are associated with more interventionist recommendations. According to one systematic review about conflicts of interest in practice guidelines, the authors were not able to identify studies informing about the effect of COI on recommendations (43). Although our assessment is an indirect measure, the observed signal is significant, and is consistent with the expected direction (41,42).
Although we did not focus on pediatric guidelines, we detected population-level vitamin D supplementation recommendations for children, even when the incidence of rickets continues to be rare in the developed world, despite some reports of an increase (44,45).
In developing countries, given that most of them are situated relatively close to the equator, which grants enough UVB sun exposure all year-round, putting emphasis on vitamin D rather than on correct nutrition seems a futile approach to child health care.
Similarly, the World Health Organization, in their publication on vitamin D supplementation in pregnant women, acknowledges that adequate nutrition in pregnant women is best achieved through a healthy balanced diet(46). Supplementation or fortification may be justified in northern latitudes, especially in higher risk subpopulations such as dark-skinned individuals, people who do not spend enough time outdoors, and women who fully cover themselves for religious or cultural reasons (10).
Some additional concerns in the recommendations we evaluated were mostly based on the use of surrogate markers, such as bone mineral density or serum levels of 25-OH-D for defining vitamin D deficiency. This is problematic given that the correlation between intake, desirable serum levels and preventive effects is unclear (10). Also, despite the fact that there might be a potential beneficial effect of supplementation with calcium and vitamin D in specific subgroups, such as institutionalized elderly women (13), extrapolating supplementation to the general population is not necessarily straight forward. Especially when the certainty of evidence is low. Finally, there is also the potential risk for medicalization and the economic implications of testing and clinical appointments (47).
We encourage end users of recommendations to remain cautious when using recommendations in this field, especially those developed with unclear methods, those with potential conflicts of interest, and those targeting the general population.
Recommendation developers should adhere to rigorous methods and use actionable, standardized language. We urge scientific societies and governmental organizations to adopt a cautionary approach when recommending preventive or screening interventions addressed to the general population. Lastly, organizations and journals should address conflicts of interest more transparently.

Limitations
We restricted our literature search to guideline repositories and Medline. However, most of the relevant guidelines are likely to have been captured, given how representative of published guidelines MEDLINE and the searched databases are. We did not perform a full AGREE II evaluation, but the direction and scoring of the domains analyzed were However, this study had several strengths. To our knowledge, is the first systematic assessment of published guidelines that inform about the effect of COI on recommendations. We performed our analysis using rigorous and transparent methods. The evaluation of the relationship of several pre-specified factors with more interventionist recommendations provides relevant information for developers, and users, including policy makers.

Conclusions
Vitamin D has been one of the most widely studied topics in medicine. Nonetheless, high quality, conclusive evidence about a net beneficial effect with its supplementation is lacking. The scientific community should probably start recognizing that vitamin D is not the "magic bullet" it appeared to be, and that its implication in physiological processes is likely to be tangential, rather than causal. Future research should probably be directed to improving lifestyles(48) rather than medicalizing healthy populations.

Consent for publication
Not applicable.

Availability and data material
All data generated or analyzed during this study are included in this published article (and its supplementary information files).

Competing interests:
The authors declare that they have no competing interests.

Funding:
The project was not funded. Funding for English language editing and publication fees was obtained via an award from the foundation for Biomedical Research and innovation in Primary Care (Fundación para la Investigación e Innovación Biomédica en Atención Primaria). The project had no other sources of funding.

Authors' contributions
DF contributed to design, data extraction, data analysis and wrote this manuscript. AL collaborated in design, helped with data analysis and collaborated in manuscript redaction. EN and EP collaborated in data extraction and manuscript redaction. CC and JK collaborated in data extraction. IG collaborated in data analysis and statistical advice. PA collaborated in design, data analysis and manuscript redaction.