In the present study, we examined the validity and reproducibility of a 178-item multiple-choice SQ-FFQ which was assessed in a long-term clinical trial. The present results demonstrated a reasonable relative validity in relation to WDRs for energy and all nutrients, except for vitamin A, (median 0.46). The agreement between these two methods was reasonably acceptable (median 76.2%) and the median correlation between FFQs was 0.56 for all nutrient intakes. The present study tried to include a reasonable number of participants. Furthermore, to reduce the random error due to within-individual variation, both energy-adjusted and de-attenuated correlation coefficients were calculated. In addition, two blood samples were collected with 3-months intervals to reduce the influence of measurement errors.
It is proposed that measuring the dietary intakes using multiple DRs that are not dependent on memory and has a great specificity in describing foods is a suitable choice to be used as a reference method in validation studies [4, 23]. Biochemical markers are also used in epidemiological studies to measure the participants' status regarding specific nutrients or dietary compounds [24, 25]. Previous studies indicate high correlations between dietary intake and some biochemical markers [26, 27]. It should be noted that disease and homeostatic regulations might affect biomarkers’ status; furthermore, biomarkers should be assessed several times to show the long-term dietary intakes. These problems might reduce the applicability of biochemical markers to be used as the sole indicator of the dietary intakes [27]. It is suggested that validation studies would provide a better insight if they compare FFQs with both DRs and biomarkers [19]. Therefore, we used 27-days WDRs which have the least correlated error [4], as a reference to compare the energy and nutrients intakes from the questionnaire and biochemical markers.
We observed a general overestimation of nutrient intake using FFQs in comparison with WDRs. It is probably due to the seasonal availability of food items like fruits and vegetables, the misconception of portion size, and a long list of food items. In line with our results, other validation studies also reported that the FFQs, as compared with food record or 24-h recall, overestimate the nutrient and energy intake [28–30]. Likewise, Considering that breads and rice are stapled foods, the overestimation of carbohydrates intake was found in another validation study in Iran [13]. It is proposed that compared with reference methods, FFQs estimate higher intakes for most of the nutrients particularly when FFQ exceeds 100 food items [31, 32]. We also observed the mean daily intake of nutrients is higher in men compared with women (Supplementary Tables 1 and 2). Sex differences in reporting energy intake exist and women were more likely to under-report energy intake [33]. Furthermore, according to sex differences in the food portion size, the sex-specific typical portion weights are recommended to be used instead of standard portion size [34, 35].
The range of reproducibility of our questionnaire was 0.43 for thiamin to 0.73 for vitamin D for adjusted data which is comparable to other validation studies [7, 12, 13, 28, 29]. According to the reports of a comprehensive review, the time interval in the validation studies varied from 2 hours to 15 years [36]. We chose 3-month intervals between the FFQs and tried to administer them at the same time of blood sample collection to diminished the difficulties for participants. The participants were asked not to change their diet during the study period.
Although the mean daily nutrient intake estimates between FFQs were not significantly different for the majority of the nutrients and energy intakes, the third FFQ showed a better correlation with WDRs, perhaps because of the learning bias, that can result from participnts learned how to answer the questions in the same way as previous questionnaires or WDRs, or change in participant's diet [37]. Moreover, FFQ3, administered at the end of the study, could comprice all WDRs in the period of the study which might explain the better correlation. In addition, we observed the higher median ICC in men between nutrients assessed by FFQs (0.59 for men and 0.58 for female) (Supplementary Table 2) or between FFQs and WDRs (0.27 for men and 0.24 for women) (Supplementary Table 3), which is in line with other reports [38, 39]. As men tend to be unconcerned about their daily diets, it might have been easier for men to complete the FFQ, which requires simplified dietary habits [39].
We expected that the random error correction for within-individual variation increases the correlation values. However, similar to the finding from other studies the de-attenuation correlations were not substantially different from non-corrected estimates [13, 28]. A large number of dietary records (27 days) or the low within-individual variation compared to between-individual variation might explain this similarity [28].
Energy adjustment appears to improve correlation coefficients and diminish the measurement errors in the FFQ instrument [4]. However, along with the finding of other studies [13, 40, 41], using energy adjustment in our study, the median correlation coefficient of nutrients tends to lower the correlation values. It seems that the low between-individual variation in nutrients’ intakes measured by WDRs has led to lower correlation coefficients after adjustment [42].
The FFQs are mainly used to rank individuals based on their dietary intake and this is important in obtaining correct risk estimates of diseases [4, 5, 43]. The present study demonstrated that about 33% of participants were classified in the same quartiles using FFQs and WDRs. Furthermore, above 70% of participants were classified to the same or adjacent quartiles which are in agreement with other validation studies. Furthermore, the present study demonstrated that the proportion of complete disagreement was in a range of 3–13.4% (median 6%). These results were in line with other studies that used quartiles to classify their participants and were conducted in Asian adults [29, 44].
As biomarkers represent the quantitative measurements and not rely on subjects’ memory which is the main source of bias in dietary assessment methods [19], we also used the serum biomarkers in our validation study. Using the method of triads, the correlation between estimated nutrients intakes using third FFQ and WDRs and measured biomarkers was calculated. Although the validity coefficients of a nutrient are not common to compare between studies because of differences in sample size, duration of studies, number of food items, food consumption which is culture-specific, and intrinsic variability of biomarkers (bioavailability and metabolism of nutrients) [4, 45], the FFQ validity coefficients for all biomarkers except for vitamin C (0.13 for vitamin C, 0.62 for calcium, 0.89 for magnesium, and 0.66 for zinc) were considered as moderate and high which is similar to findings from Mc Naughten et al. (0.50, 0.63, 0.45, 0.62) [20], and Andersen et al. (0.58, 0.51) [46]. In addition, Mirmiran et al. [13] found that the range of validity coefficient (ρ QI) was 0.21 to 0.95 (TLGS) which is in line with our results (0.02 to 0.89).
The realistic correlation coefficients of validation studies tend to be in a range of 0.5 to 0.7 [4]. In the TLGS the mean Pearson correlation coefficient and the mean intraclass correlation coefficient between twelve 24-h dietary recalls and FFQ for men were 0.53 and 0.59, and for women were 0.39 and 0.60 in energy-adjusted values [13]. In the Golestan cohort study, the correlations coefficient between twelve 24-h dietary recalls and mean of four FFQs ranged from 0.49 to 0.82 and the intraclass correlations were between four FFQs vary from 0.66 to 0.89 [12]. The validity correlation coefficients reported being lower in the present study compared to the previous Iranian studies. We observed that the median Pearson correlation coefficient and median Intraclass correlation coefficient between 27-d WDRs and SQ-FFQ were 0.35 and 0.46. It should be noted that the previous investigations had used 24-h dietary recalls for examining the validity which both rely on memory and this might increase the correlation coefficients by error [4]. This is while our study used 27-d WDRs for validity assessment in the Iranian population for the first time which is not the same in the sources of bias [4]. The range of correlation coefficients in our study was similar to studies previously conducted in Asia [29, 47–51] which compared FFQs with dietary records. They found that the rage of correlation coefficients of nutrient intakes between FFQs and WDRS were 0.06 to 0.81 and the median ranged between 0.3 to 0.5. It should be noted that serving sizes and foods consumed in Asian regions are different; furthermore, meals are served as family-style and the family members share their foods. Thus, it might lead to a low perception of portion size when reporting their dietary intake using FFQs [29].
The present study has some limitations that should be considered. First, the same portion size was used for both sexes which may result in substantial errors in the estimation of nutrient intakes. Second, as no complete Iranian FCT exists, we used the USDA FCT to calculate the energy and nutrient intakes for the majority of foods. This point might not affect the correlation coefficients and the assessment of misclassifications, however, might lead to biased absolute intakes. As the present study did not aim to assess the absolute intakes, the present results using USDA FCT might not have tangible effects on the present results. Furthermore, the same FCT was used to calculate the dietary intakes reported using FFQs and WDRs. This might lead to higher correlation coefficients. As the present study was conducted in the context of a clinical trial aimed to examine the effect of different plant oils on cardio-metabolic outcomes, the reproducibility and validity of FFQs might be prone to bias for dietary fatty acids. Therefore, we removed the validity and reproducibility statistics for different dietary fatty acids.
In summary, the present study found that the present 178 item SQ-FFQ has overall acceptable levels of validity and reproducibility for assessing the dietary nutrients intake. Thus, the SQ-FFQ used in this study seems to be a useful instrument to measure the dietary nutrients in epidemiological studies conducted in Yazd province, central Iran. Furthermore, compared to the previous version, the SQ-FFQ has been quickly administered (takes about 20 to 30 minutes).