Validity and reproducibility of a semi-quantitative multiple-choice food frequency questionnaire in adults living in central Iran

Background: To the best of our knowledge, no study has tried to develop and validate a multiple-choice food-based FFQ in Iran using weighed dietary records.This study aimed to investigate the validity and reproducibility of a multiple-choice semi-quantitative food frequency questionnaire (SQ-FFQ) in adults living in central Iran. Methods: Participants attended a large long-term clinical trial were asked to complete three SQ-FFQs by interview, and nine 3-day weighed dietary records (WDRs), over nine months. They provided two blood samples to assess serum calcium, magnesium, zinc, and vitamin C levels. The Pearson and intraclass correlation coefficients (ICC) were calculated to assess reproducibility and validity. The degree of misclassification was explored by using a contingency table of quartiles which is compare the information between third FFQ and WDRs. The method of triads was incorporated to assess validity coefficients between estimated intakes using third FFQ, WDRs, and biochemical markers and assumed true intakes. Results: A total of 180 adults aged 48.9±8.4 years completed the study. Compared to WDRs, FFQs overestimated all nutrients intakes except for iron. The median Pearson correlation coefficient was 0.31, 0.44, and 0.38 for FFQ1-FFQ2, FFQ1-FFQ3 and FFQ2-FFQ3, respectively and ICC ranged from 0.43 (thiamin) to 0.73 (vitamin D, median: 0.56). The de-attenuated, age, sex, and education adjusted correlation coefficients ranged from 0.01 for vitamin A to 0.40 for vitamin B12 and -0.05 for vitamin A to 0.41 for manganese (median: 0.17 and 0.26) for FFQ1-WDR and FFQ3-WDR, respectively. The median exact agreement and complete disagreement between FFQ3 and WDRs were 33% and 6%, respectively. The FFQ3 validity coefficients for vitamin C, calcium, magnesium, and zinc were 0.13, 0.62, 0.89, and 0.66, respectively, using the triads method. Conclusions: The SQ-FFQ seems to be an acceptable tool to assess the long-term dietary intake for future large-scale studies in this population. sex revealed that frequency questionnaires provided higher estimations for the of compared to WDRs in both sexes for iron.


Introduction
Diet plays an important role in the development and control of chronic diseases [1][2][3]. Valid and reliable dietary assessment methods developed for each population are needed to find out the dietary determinants of health [2]. Several methods such as 24-hour recall, dietary record (DR), and food frequency questionnaire (FFQ) have been used to assess the dietary intake in different studies [4]. All of these methods have some individual limitations in estimating the dietary food intake; therefore, they are selected by researchers to be used based on the aim and the design of each study [5,6].
The semi-quantitative food frequency questionnaire (SQ-FFQ) is a widely used method to assess longterm dietary intake in population-based studies [7]; because it is inexpensive and easy to complete in large populations [4]. The questionnaires are mostly composed of a food list and a frequency response section. Some SQ-FFQs also ask about the usual portion size of each consumed food item.
FFQs are prone to bias and misclassification because they rely on memory, and miss some food items that are high in the targeted nutrients, frequently consumed in the population, and are differently used by individuals living in the target population [4]. Therefore, FFQs should be validated using the other dietary assessment methods which are not prone to the same sources of bias [4]. Biological markers are likely be able to improve the estimations of dietary intake assessment, because of random errors are independent of other dietary assessment tools [4,8]. Nevertheless, the majority of biomarkers are expensive and it is not desirable to replace the other methods of dieary assessment as part of a large epidemiological study [8,9]. Furthermore, the reproducibility of the FFQs should be assessed because they are designed to provide data on long-term intakes [4]. Any changes in the design of FFQs might lead to a change in the performances of the questionnaire [10]. In addition, these questionnaires are culture-specific which means that the dietary culture and foods consumed are highly variable between populations even in the same country. Therefore, the validity and reproducibility of a questionnaire are needed to be measured for any population [4,6].
Several FFQs are developed in Iran [11][12][13][14]. For instance, Tehran Lipid and Glucose Study (TLGS) [13] and Golestan cohort study [12], have used open-ended FFQs which were designed to be used in Tehran and Golestan provinces, respectively. Furthermore, the FFQ used in the TLGS asked about the frequency and portion size of the intake of 168 food items [13]. The participants' food consumption frequency over the previous year was asked on daily (e.g. bread), weekly (e.g. rice, meat) or monthly (e.g. fish) basis. Likewise, a 150-item FFQ was designed for the Golestan cohort study and the frequency of consumption was recorded as times per day, week, month, year, and never. For 51 food items, pictures of different portion sizes were used to increase the precisions of estimations [12]. Both FFQs were validated using 24-hour recalls as a reference method to validate the dietary intakes and showed good validity and reproducibility [12,13].
To the best of our knowledge, no study has tried to develop and validate a multiple-choice food-based FFQ in Iran using weighed dietary records. People residing in Yazd province, central Iran are living in arid and semi-arid areas that have different dietary habits and food items. Therefore, we designed a semi-quantitative food-based FFQ with 178 food items in order to assess the habitual food and nutrient intake of adults living in Yazd to be used in large-scale epidemiologic studies. The current FFQ used a multiple-choice approach; therefore it is easy to perform for participants and interviewers to fill the questionnaire [4]. The present study aimed to assess the validity and reproducibility of the semi-quantitative multiple-choice food frequency questionnaire using weighed dietary records biochemical markers in a sample of adults living in Yazd city, who were attended in a clinical trial.

Materials And Methods Study design
The present study was conducted in the context of a clinical trial. The study protocol has been described in detail, elsewhere [15]. In brief, a triple-blind randomized three-way cross-over clinical trial which aimed to compare the effect of the replacement of the regular consumed oils with three edible oils (canola, sesame, and sesame-canola oils) on cardiovascular risk factors in adults with diabetes and their spouse, who were recruited from the diabetes research center, Yazd, Iran. A total of 102 adults with diabetes (50 males, 52 females) and 101 spoused (50 males, 51 females), aged between 18 and 60 years old entered the original clinical trial. Informed consent were taken from all study participants. The methodology of the current study was ethically approved by the research ethics committee of Shahid Sadoughi University of Medical Sciences (approval code: IR.SSU.SPH.REC.1396.155).
Each participant entered three 9-week intervention periods and randomly received all intervention oils. The intervention periods were separated by 4 weeks of washout periods. The participants were visited at baseline, in the middle and at the end of each intervention period. Therefore, each study attendant was visited 9 times for 35 weeks (about 9 months) in the clinical trial. The participants were asked to provide three weighed dietary food records (two weeks and one weekend days) for each visit; therefore, each participant would provide data on dietary intake for 27 days during the study period. For the current investigation, three FFQs separated by three months were recruited to address the reproducibility. The first FFQ was filled at the 6th month of the clinical trial (visit 6), the second one was administered at the end of the study (9 months from baseline, visit 9) and the participants were invited to fill the third FFQ three months after the end of the clinical trial. The study flow diagram is provided in Fig. 1.

Food frequency questionnaire (FFQ)
The semi-quantitative FFQ used for the present study was a modified version of an open-ended 168item FFQ which was validated and used in Tehran Lipid and Glucose Study (TLGS) [13]. In the current version, we added 10 food items that were typically consumed in Yazd. Besides, compared to the TLGS FFQ, the current questionnaire was designed to be a multiple-choice questionnaire. Altogether, 178 food items were included in the questionnaire: breads and grains (n = 23), beans (n = 7), meats, fish and shellfish (n = 19), milks and dairy products (n = 17), vegetables (n = 26), fruits (n = 40), fats and nuts (n = 13), beverages (n = 5), Snacks and sweets (n = 28). The study participants had to answer two questions by the interview regarding each food item: 1) the frequency of consumption, 2) the portion size. The frequency responses for each food item were as follows: never or less than one month, 1-3 times per month, 1 time per week, 2-4 times per week, 5-6 times per week, 1 time per day, 2-4 times per day, 5-7 times per day, 7-9 time per day and 10 times and more per day.
The portion sizes were estimated by natural units (for example one banana) or standard quantities (for example one spoon of olive oil). The portion size of all the items which were listed in the questionnaire was asked in separate questions. Furthermore, a separate section was considered to estimate the supplements' consumption: fish oil (or omega-3), calcium, vitamin D, folic acid, iron, and multivitamin-mineral supplements. The reported frequencies of each item were converted to the number of intakes per day and multiplied by the indicated portion size to convert the reported intakes to gram/day.

Dietary food records
Participants were asked to provide 3-day (Two weekdays and one weekend day) weighed dietary records (WDRs) for each visit (data on 27 days were recruited for each participant). A digital kitchen scale (model Electronic kitchen scale, SF-400) was provided for the study attendees to help the participants record their dietary intake with maximum accuracy. They were trained how to fill dietary food records by a nutritionist. The participants were asked to write down the consumed foods, and beverages with their weight; in addition, they were asked to describe all supplements and medications which were consumed each day. Prior to starting the study, a protocol for coding WDRs was prepared by a research supervisor (ASA). Based on the protocol, trained nutritionist collected the WDRs and asked the participants to clarify unclear descriptions, errors, and doubtful entries. Trained nutritionists checked all completed WDRs for accuracy. All food items were converted to grams and the energy and nutrients' intakes were calculated using Nutritionist IV software (version 3.5.2, Axxya Systems, Redmond, Washington, USA) which was modified for Iranian foods. In general, we used the United States Department of Agriculture's (USDA) food composition table (FCT) [16]

Biochemical markers
Blood samples from participants were taken after an overnight fast (10-12 hours) and stored at -80 ºC in DNase and RNase free micro tubes until analysis. The average of blood samples recruited at visits 6 and 9 before the time that FFQ 1 and FFQ 2, respectively, were used for the current analysis. Serum calcium, magnesium, and zinc levels were measured by using an auto-analyzer (Alpha-classic, model: AT ++ ) using Pars-azmoon (for serum calcium and magnesium levels) and Biorex Fars (for serum zinc levels) standard kits. The inter-and intra-assay coefficients of variability (CV) were 2.4% and 1.2% for serum calcium, 1.4% and 1.1% for serum phosphorus, 1.3% and 0.8% for serum magnesium, and 1.7% and 1.2% for serum zinc assessment, respectively. Moreover, serum vitamin C levels were determined by Enzyme-linked immunosorbent assay (ELISA) kit (Zellbio standard kit). The inter-and intra-assay for serum vitamin C measurements were 4.7% and 3.5%, respectively.

Statistical analysis
All nutrients and energy intake values were log-transformed (Log 10 ) prior to analysis to optimize the normality of distribution. Energy adjustment was performed by computing residual method using the linear regression model in which the nutrient intakes were defined as dependent variables and the energy intakes as an independent variable [4,18]. To compare the absolute nutrients intakes from is the within-individual variance, is the between-individual variance, and n is the number of replicate measurement (here n=27) [4].
The misclassification of questionnaires was assessed by classifying the participant's nutrients intakes measured by FFQs and 27-day WDRs into quartiles and evaluating the degree of agreement between the third FFQ and WDRs using contingency tables.
The method of triads was used for calcium, magnesium, zinc and vitamin C to evaluate the validity coefficient between assumed true intake and estimated intakes from the third FFQ, WDR and biochemical markers (Fig. 2) [19]. We considered the validity coefficient of FFQ (ρ QI) as the upper limit and the correlation coefficient of FFQ and biomarker (r QB) as the lower limit of the validity coefficient between FFQ and the true intake [20]. We considered the validity coefficients as weak (ρ < 0.2), moderate (0.2 ≤ ρ ≤ 0.6), and high (ρ > 0.6) [19]. If we observed the validity coefficient of greater than one for any of the assessment methods, which is known as Heywood case [21], we truncated it to one and the validity coefficient of the method was considered as the upper limit and the correlation coefficient of the method and biomarker as the lower limit of the validity coefficient [20,22]. All analyses were performed using statistical package for social sciences (SPSS), version 21 (SPSS Inc., Chicago, III, USA). P-values < 0.05 were considered statistically significant.

Results
A total of 180 subjects (89%) aged 48.9 ± 8.4 years after excluding 22 participants who did not administer three FFQs were included in the current analysis (50.2% female and 49.8% male). The general characteristics of the study participants are provided in Table 1. The mean daily nutrient intake based on nine 3-day WDRs and FFQs are shown in Table 2. Compared to WDRs, the mean daily intakes calculated from FFQs tended to overestimate the energy and nutrients intake except for the iron intake. The FFQs were not significantly different in estimating the dietary nutrients intake except for the dietary protein, fat, cholesterol, niacin, folate, beta-carotene, vitamin E, zinc, copper, selenium, and manganese. The sex-stratified mean daily nutrient intakes estimated by using the three FFQs and WDRs are also described in Supplementary Tables 1 and 2.
The analysis based on sex revealed that Food frequency questionnaires provided higher estimations for the intakes of dietary nutrients compared to WDRs in both sexes except for iron.   respectively. The sex-stratified analyses are also provided in Supplementary Tables 5 and 6. Table 4 Correlation coefficients for the association between energy and nutrient intake measured by the nine 3-  and WDRs (ρ BI: 0.32). The validity coefficient for FFQ and the correlation between questionnaire and biomarker were considered as the upper and the lower limit of the validity coefficient between FFQ and the true intakes, respectively. Therefore, the lower and upper limits ranged from 0.02 to 0.13, 0.13 to 0.62, 0.20 to 0.89, and 0.21 to 0.66 for vitamin C, calcium, magnesium, and zinc, respectively. .0 were truncated as this is the highest possible value, ρ QI: validity coefficient for the food frequency questionnaire, ρ BI: validity coefficient for biomarkers, and ρ RI: validity coefficient for WDRs ‡ The lower limit for the FFQ and the biomarker is the correlation between the FFQ and biomarker, and the lower limit for the WDR is the correlation between the biomarker and the WDR, and the lower limit for the biomarker is the correlation between the FFQ and the WDR, and the upper limit is calculated by the method of triads.

Discussion
In the present study, we examined the validity and reproducibility of a 178-item multiple-choice SQ-FFQ which was assessed in a long-term clinical trial. The present results demonstrated a reasonable relative validity in relation to WDRs for energy and all nutrients, except for vitamin A, (median 0.46).
The agreement between these two methods was reasonably acceptable (median 76.2%) and the median correlation between FFQs was 0.56 for all nutrient intakes. The present study tried to include a reasonable number of participants. Furthermore, to reduce the random error due to withinindividual variation, both energy-adjusted and de-attenuated correlation coefficients were calculated.
In addition, two blood samples were collected with 3-months intervals to reduce the influence of measurement errors.
It is proposed that measuring the dietary intakes using multiple DRs that are not dependent on memory and has a great specificity in describing foods is a suitable choice to be used as a reference method in validation studies [4,23]. Biochemical markers are also used in epidemiological studies to measure the participants' status regarding specific nutrients or dietary compounds [24,25]. Previous studies indicate high correlations between dietary intake and some biochemical markers [26,27]. It should be noted that disease and homeostatic regulations might affect biomarkers' status; furthermore, biomarkers should be assessed several times to show the long-term dietary intakes.
These problems might reduce the applicability of biochemical markers to be used as the sole indicator of the dietary intakes [27]. It is suggested that validation studies would provide a better insight if they compare FFQs with both DRs and biomarkers [19]. Therefore, we used 27-days WDRs which have the least correlated error [4], as a reference to compare the energy and nutrients intakes from the questionnaire and biochemical markers.
We observed a general overestimation of nutrient intake using FFQs in comparison with WDRs. It is probably due to the seasonal availability of food items like fruits and vegetables, the misconception of portion size, and a long list of food items. In line with our results, other validation studies also reported that the FFQs, as compared with food record or 24-h recall, overestimate the nutrient and energy intake [28][29][30]. Likewise, Considering that breads and rice are stapled foods, the overestimation of carbohydrates intake was found in another validation study in Iran [13]. It is proposed that compared with reference methods, FFQs estimate higher intakes for most of the nutrients particularly when FFQ exceeds 100 food items [31,32]. We also observed the mean daily intake of nutrients is higher in men compared with women ( Supplementary Tables 1 and 2). Sex differences in reporting energy intake exist and women were more likely to under-report energy intake [33]. Furthermore, according to sex differences in the food portion size, the sex-specific typical portion weights are recommended to be used instead of standard portion size [34,35].
The range of reproducibility of our questionnaire was 0.43 for thiamin to 0.73 for vitamin D for adjusted data which is comparable to other validation studies [7,12,13,28,29]. According to the reports of a comprehensive review, the time interval in the validation studies varied from 2 hours to same time of blood sample collection to diminished the difficulties for participants. The participants were asked not to change their diet during the study period.
Although the mean daily nutrient intake estimates between FFQs were not significantly different for the majority of the nutrients and energy intakes, the third FFQ showed a better correlation with WDRs, perhaps because of the learning bias, that can result from participnts learned how to answer the questions in the same way as previous questionnaires or WDRs, or change in participant's diet [37]. Moreover, FFQ3, administered at the end of the study, could comprice all WDRs in the period of the study which might explain the better correlation. In addition, we observed the higher median ICC in men between nutrients assessed by FFQs (0.59 for men and 0.58 for female) (Supplementary Table 2) or between FFQs and WDRs (0.27 for men and 0.24 for women) (Supplementary Table 3), which is in line with other reports [38,39]. As men tend to be unconcerned about their daily diets, it might have been easier for men to complete the FFQ, which requires simplified dietary habits [39].
We expected that the random error correction for within-individual variation increases the correlation values. However, similar to the finding from other studies the de-attenuation correlations were not substantially different from non-corrected estimates [13,28]. A large number of dietary records (27 days) or the low within-individual variation compared to between-individual variation might explain this similarity [28].
Energy adjustment appears to improve correlation coefficients and diminish the measurement errors in the FFQ instrument [4]. However, along with the finding of other studies [13,40,41], using energy adjustment in our study, the median correlation coefficient of nutrients tends to lower the correlation values. It seems that the low between-individual variation in nutrients' intakes measured by WDRs has led to lower correlation coefficients after adjustment [42].
The FFQs are mainly used to rank individuals based on their dietary intake and this is important in obtaining correct risk estimates of diseases [4,5,43]. The present study demonstrated that about 33% of participants were classified in the same quartiles using FFQs and WDRs. Furthermore, above 70% of participants were classified to the same or adjacent quartiles which are in agreement with other validation studies. Furthermore, the present study demonstrated that the proportion of complete disagreement was in a range of 3-13.4% (median 6%). These results were in line with other studies that used quartiles to classify their participants and were conducted in Asian adults [29,44].
As biomarkers represent the quantitative measurements and not rely on subjects' memory which is the main source of bias in dietary assessment methods [19], we also used the serum biomarkers in our validation study. Using the method of triads, the correlation between estimated nutrients intakes using third FFQ and WDRs and measured biomarkers was calculated. Although the validity coefficients of a nutrient are not common to compare between studies because of differences in sample size, duration of studies, number of food items, food consumption which is culture-specific, and intrinsic variability of biomarkers (bioavailability and metabolism of nutrients) [4,45] [13] found that the range of validity coefficient (ρ QI) was 0.21 to 0.95 (TLGS) which is in line with our results (0.02 to 0.89).
The realistic correlation coefficients of validation studies tend to be in a range of 0.5 to 0.7 [4]. In the TLGS the mean Pearson correlation coefficient and the mean intraclass correlation coefficient between twelve 24-h dietary recalls and FFQ for men were 0.53 and 0.59, and for women were 0.39 and 0.60 in energy-adjusted values [13]. In the Golestan cohort study, the correlations coefficient between twelve 24-h dietary recalls and mean of four FFQs ranged from 0.49 to 0.82 and the intraclass correlations were between four FFQs vary from 0.66 to 0.89 [12]. The validity correlation coefficients reported being lower in the present study compared to the previous Iranian studies. We observed that the median Pearson correlation coefficient and median Intraclass correlation coefficient between 27-d WDRs and SQ-FFQ were 0.35 and 0.46. It should be noted that the previous investigations had used 24-h dietary recalls for examining the validity which both rely on memory and this might increase the correlation coefficients by error [4]. This is while our study used 27-d WDRs for validity assessment in the Iranian population for the first time which is not the same in the sources of bias [4]. The range of correlation coefficients in our study was similar to studies previously conducted in Asia [29,[47][48][49][50][51]] which compared FFQs with dietary records. They found that the rage of correlation coefficients of nutrient intakes between FFQs and WDRS were 0.06 to 0.81 and the median ranged between 0.3 to 0.5. It should be noted that serving sizes and foods consumed in Asian regions are different; furthermore, meals are served as family-style and the family members share their foods. Thus, it might lead to a low perception of portion size when reporting their dietary intake using FFQs [29].
The present study has some limitations that should be considered. First, the same portion size was used for both sexes which may result in substantial errors in the estimation of nutrient intakes.
Second, as no complete Iranian FCT exists, we used the USDA FCT to calculate the energy and nutrient intakes for the majority of foods. This point might not affect the correlation coefficients and the assessment of misclassifications, however, might lead to biased absolute intakes. As the present study did not aim to assess the absolute intakes, the present results using USDA FCT might not have tangible effects on the present results. Furthermore, the same FCT was used to calculate the dietary intakes reported using FFQs and WDRs. This might lead to higher correlation coefficients. As the present study was conducted in the context of a clinical trial aimed to examine the effect of different plant oils on cardio-metabolic outcomes, the reproducibility and validity of FFQs might be prone to bias for dietary fatty acids. Therefore, we removed the validity and reproducibility statistics for different dietary fatty acids.
In summary, the present study found that the present 178 item SQ-FFQ has overall acceptable levels of validity and reproducibility for assessing the dietary nutrients intake. Thus, the SQ-FFQ used in this study seems to be a useful instrument to measure the dietary nutrients in epidemiological studies conducted in Yazd province, central Iran. Furthermore, compared to the previous version, the SQ-FFQ has been quickly administered (takes about 20 to 30 minutes).

Ethics approval and consent to participate
Parents of the study participants were given verbal and written information about the study and signed informed consent before participation. The present study was conducted in accordance with the declaration of Helsinki and the methodology of the current study was ethically approved by the research ethics committee of Shahid Sadoughi University of Medical Sciences (approval code: IR.SSU.SPH.REC.1396.155).

Consent for publication
No individual detail is presented in this manuscript.

Availability of data and material
The data of the present study will be available for the corresponding author. The data used for the current study are already published in individual papers. The data can be obtained from the corresponding author.  Figure 1 The study flow diagram. BV: baseline visit, BS: blood sample Triangular comparison between three dietary exposure measurements (triads method). R: reference method (WDR), Q: food frequency questionnaire, B: biomarker, I: true intake, r QR: correlation between food frequency questionnaire and reference method, r BR:

Figures
correlation between biomarker and reference method, r QB: correlation between food frequency questionnaire and biomarker, ρ QI: validity coefficient for food frequency questionnaire, ρ BI: validity coefficient for biomarker, and ρ RI: validity coefficient for reference method.

Supplementary Files
This is a list of supplementary files associated with this preprint. Click to download.