Development and Validation of Food Frequency Questionnaire for Food and Nutrient Intake of Adults in Butajira, Southern Ethiopia

DOI: https://doi.org/10.21203/rs.3.rs-26794/v2

Abstract

Background: To date, there is no culture-specific and validated Food Frequency Questionnaire (FFQ) available in Ethiopia. We developed a FFQ and evaluated its validity as compared to estimates of a food group and nutrient intakes derived from two 24-Hour Dietary Recalls (24-HRs).

Method: The FFQ has a total of 89 food items. A total of 105 adults aged 20 to 65 years participated in this study. Wilcoxon- signed rank test, Spearman’s correlation, Cross-classification, kappa, and Bland-Altman analysis were used to validate food group intakes and nutrients derived from the FFQ against two 24-HRs.

Results: Mean energy and macronutrient intakes obtained from the FFQ were significantly higher than those obtained from the mean of two 24-HRs. For energy and macronutrients, the correlation between two instruments ranged from 0.05 (total fat) to 0.32 (carbohydrate). Whereas, for micronutrients, it ranged from 0.1 (calcium) to 0.49 (vitamin B1). Visual inspection of the Bland-Altman plots for both energy and macronutrients shows no consistent trend across the intake values. For the majority of the food groups, no significant difference was observed in median intake of foods and nutrients between 24-HRs and the FFQ. The correlation coefficients for food groups ranged from 0.12 (egg) to 0.78 (legumes). The FFQ showed a fair classification agreement with the 24-HRs for cereals, legumes, and roots and tubers intake. A systematic trend of overestimation for roots and tubers and under estimation of beverage intakes at higher values was observed when we used FFQ.

Conclusion: The FFQ is valid to assess and rank individuals in terms of intakes of most food groups according to high and low intake categories. However, estimates of roots and tuber and beverages should be interpreted with caution.

Introduction

Food frequency questionnaire (FFQ) asks respondents about their usual frequency of consumption of each food during a specified time (1). Compared to other dietary assessment methods, such as short-term recall and diet record, FFQ is easier to administer, place less burden on respondents, relatively inexpensive, and provide a rapid estimate. This makes FFQ more feasible and best suited for measuring long term dietary intake for most epidemiological studies and large cohort studies (2).

Interpretation of results from studies of diet-disease that use FFQ is often difficult unless it has been adapted and validated in a population reasonably similar to that being investigated (3). Incorrect information may give rise to false associations between dietary factors and diseases or disease-related markers. On the other hand, the null association could also be attributed to a lack of variation in the dietary exposure in the study population or the inability of the tool to find out existing differences in the diet. Therefore, it is important to assess the degree to which the questionnaire measures the aspect of a diet for which it has been designed (3, 4).

Because of the lack of gold standard in dietary assessment, validation studies cannot compare a test method with absolute truth rather they compare one method with another method that is judged to be superior (5). Among the available and feasible comparison methods to validate FFQ, diet records represent an optimal comparison method because of having the least correlated error with FFQ. However, when co-operation or literacy of study subjects is limited, 24-hour recall (24-HR) may be more appropriate (3, 6).

To our knowledge, there is no validated standard FFQ in Ethiopia that can help to assess dietary habits among adults. Therefore, this study aimed to develop context-specific FFQ and evaluate the relative validity against two 24-HR methods.

Methods

Study design and Participants

We validated the FFQ against the average of two 24-HRs. The FFQ was obtained after the second 24-HR. There was a 15 days interval between the first and the second 24-HR. We have used an interactive, multiple passes 24-HR question adapted and validated for use in developing countries (7). We conducted the study  among randomly selected 120 Ethiopian adults aged 20 to 65 in Butajira Health and Demographic Surveillance Site (HDSS), from March to April 2019. We employed a simple random sampling procedure to identify study participants. Households with adults age 20-65 were filtered out from the Butajira HDSS data registry to form a sampling frame. From this frame, we randomly selected 120 households with adults 20-65 of age. We visited all houses with support from the health extension workers, local guides and study supervisors.

Development of the FFQ

We followed five steps to develop the FFQ: choosing appropriate foods, prioritization, and categorization of food items, assembling a list of selected foods, frequency and portion size, and expert review and pre-testing. First, we obtained information on dietary intakes was from an unpublished cross-sectional dietary survey of women living in Butajira from 2018-2019. Information on dietary intake was collected using a single 24-HR. We undertook Market and mini-market visits on non-consecutive days to identify common brand names and foods that could be relevant and were added accordingly. Besides, we conducted a focus group discussion in Butajira district two weeks before the interview. It was organized by the principal investigator and field supervisors. We interviewed a group of 6 women about foods consumed in the area. We undertook the discussion to identify food items that are typically consumed, including ingredients used, and methods of preparation. Second, we combined similar foods and beverages into a single group of food items. Third, we clustered the related food items together. For closely related foods, we placed more specific items before general items. Forth, we evaluated the frequency of intake based on the usual intake over 1 month before data collection. We included seven frequency categories were included ranging from daily to never/one less than per month. Three women were involved in the cooking process and portion size estimation. We assigned a portion size for each food item. We employed a pre-specified portion size estimation method for the estimation of portion size in FFQ using local house-hold units such as bowl, plate, spoons of different sizes (tablespoon, teaspoon), coffee-cups, tea-cups, water glasses, as well as using photographs. Fifth, experts reviewed the newly developed FFQ (nutritionist from Addis Ababa University) to confirm its content validity. We have discussed the food list extensively to ensure all relevant food items were included; this was followed by pre-testing of the developed questionnaire in a group of 10 adult women who are comparable to the study participants from nearby sites. Some minor changes were made based on the finding of the pre-test.

Dietary assessment

24-Hour Dietary Recall

We have used an interactive, multiple passes 24-HR question adapted and validated for use in developing countries (7). We conducted the two 24-HR on non-consecutive days. We interviewed on weekdays and weekends to capture variance in the intakes across various days of the week. Before data collection, we gave a rigorous training and conducted a pre-test. We recruited three (3) interviewers who had a previous experience in dietary data collection and fluent in local language. Each interview involved a stepwise series of questions.

First, we asked the participants to report everything that they consumed the previous day, including the night. The opening question was; “After you got up this morning/yesterday morning, when was the first time that you had something to eat or drink?” followed by the questions “What did you eat or drink at that time?” and “Did you eat or drink anything else at that time?” The same three questions were repeatedly been asked until the participants recall all the food and drink items consumed over 24-hr period. The first pass ended with the questions “Can you remember any other times you had something to eat or drink to?” In the second pass, participants were asked to provide additional detailed information about each item of food and drinks consumed. This includes the name of the food item, where they ate it, brand names, cooking methods, amounts served, and the amount consumed. For homemade dishes, participants were asked for the recipes and ingredients.

On the third pass we used common household utensils such as bowl, plate, spoons of different sizes (tablespoon, teaspoon), coffee cups, teacups, water glasses to improve the memory of the respondents and to assist in completing the recall. We used a digital food scale (Electronic kitchen scale) to measure the weight of the consumed as well as the ingredients used in food preparation to the nearest 1g. The final pass reviewed all previously recalled information to confirm the accuracy of the record. During the final pass, the data collectors asked the participants to prompt for information about foods and drinks not mentioned that were considered to be easy to forget, such as snacks, fruits, water, and juices (8).

Food Frequency Questionnaire

We evaluated the frequency of intake based on the usual intake over the previous month. We included nine (9) frequency categories ranging from daily to never/one less than per month each food item was assigned a portion size. We employed a pre-specified portion size estimation method based on the average consumption for the estimation of portion size using local house-hold units such as bowl, plate, spoons of different sizes (tablespoon, teaspoon), coffee cups, teacups, water glasses, as well as using photographs. We visited local markets and shops to purchase and arrange equipment for data collection. To determine the weight of the food items used we made commonly consumed dishes in the Ethiopian public Health Institute laboratory by the principal investigator. We did the measurement with an Electronic Seca scale and the average of the 3 measurements was taken. We gave codes for different prepared portions. To help participants understanding, we prepared photographs for each measurement done and the interviewer showed them to the participants.

Calculation of Daily Food and nutrient Intake

We used the Ethiopian food composition table was to derive nutrient and energy estimates from the dietary data. The names of foods and drinks, their description, cooking methods, and amounts from both 24-HR and FFQ, were coded and entered into NutriSurvey2007. The FFQ consisted of 89 food and drink items. We organized the food lists into 14 food groups on the basis of prior information. We calculated food estimates from FFQ using the product sum method. We converted the average frequency of food intake per week and month of the FFQ to a daily intake value (e.g., frequency of 2–3 times per month = 2.5/30.5 times per day. Once the frequency of consumption per day was calculated, we computed the daily food intake using the product sum method. Daily food intake =∑ (reported consumption frequency of the food item, converted to times per day) *(portion size consumed of that food).

Statistical test of validity

We checked both the FFQ and 24-HR data for completeness and potential errors. We then entered the data on socio-demographic characteristics using Epi-Data version 3.1 and exported to STATA version 15 for further processing and analysis.   

We checked the normality of the average intake of nutrient and food groups using the Shapiro-Wilk normality test and visualized using Q-Q plots. We used parametric tests for normally distributed variables, while non-parametric tests were used for most of the variables as the distributions significantly deviated from normality. Those which fulfilled the assumption of normality were described using mean with standard deviation (SD) and those which were not normally distributed, we used median with inter-quartile range (IQR).

We evaluated the performance of the FFQ against two 24-HRs using several statistical tests. First, to compare median daily food intakes obtained from the averages of the two 24-HRs and the FFQ, we used the Wilcoxon signed-rank test. To compare mean daily food intakes obtained from the averages of the two 24-HRs and the FFQ, we used paired t-test. Second, to measure the strength and direction of the correlation between the two methods, we computed the Pearson correlation for normally distributed variables, whereas Spearman’s rank correlation for those not normally distributed. The cut-off points used for correlation coefficient were as follows; <0.20 as low correlation (poor outcome), 0.20 - 0.49 as moderate correlation (acceptable outcome), and ≥0.50 as high correlation (good outcome) (5).

Third, for both the test and reference methods, subjects were divided into categories relating to the distribution of dietary intake; quartiles of intake. A comparison of the subjects’ categories showed whether subjects were classified in the same or different categories by the two methods. The result permitted an assessment of the proportion of subjects who were classified correctly. We used a weighted kappa statistic to account for both the correctly classified percentage and the expected participant proportion classified by chance. The cut-off points used for weighted kappa statistics were as follows; <0.20 as low kappa (poor outcome), 0.20 - 0.60 as moderate kappa (acceptable outcome), and ≥0.50 as high kappa (good outcome) (5). At last, we used a Bland and Altman plot for assessing limits of agreement between the two methods. The Bland-Altman method is preferable to compare two measurements each of which produced some error in their measures (9).

Results

Study Participant Characteristics

Table 1 shows the socio-demographic characteristics of the study participants. Of the 120 participants, 115 (95.8%) completed both the 2 day 24-hour dietary recall and the FFQ. A total of 105 study participants were included in the final analysis, of which 43 (41%) were male and 62 (59%) female. The mean age of participants was 31.9 years (SD: 9.2), 33.3% of them had primary education, and 43 (41%) were housewives.

 Table 1. Socio-demographic characteristics of study participants (n=105)

Characteristics of participants

Frequency

Percent

Sex

Male

43

41

Female

62

59

Age category

20-29

49

46.7

30-39

35

33.3

40-49

13

12.4

50-65

8

7.6

Education

Primary

35

33.3

Secondary

21

20

College/university

10

9.5

No formal education

39

37.2

Occupation

Farmer and housewife

5

4.8

Housewife

43

41

Employee/private

3

2.9

Merchant

25

23.8

Daily laborer

8

7.6

Unemployed

8

7.6


Development of Food Frequency Questionnaire

The developed FFQ consisted of 89 food and drink items. The food groups include cereals, bread and potatoes, Legumes and pulses, Roots and tubers, vegetables, fruits, egg, milk and dairy, fish and fish-products, meat and poultry, fat and oil, sweets, drinks, and fast foods and pastry. Information obtained from the focal grouped discussion included foods during the different seasons, traditional and ceremonial dishes consumed and foods that are infrequently consumed. The aim is to use the tool particularly for chronic disease dietary risk assessment; for this we compared the mean nutrient intakes previously hypothesized to be associated with chronic diseases. Which includes: macronutrients (carbohydrate, energy, protein and, fat), micronutrients (calcium, iron, vitamin A, vitaminB1, B2) and food groups.

Relative validity analysis

Table 2 shows the mean (SD), median, and 25th, 75th percentiles daily nutrient intakes estimated by the average of two 24-HRs and the FFQ. The mean energy and macronutrient intakes obtained from the FFQ were significantly higher than the average of two 24-HRs. The highest mean difference was for energy 367.6 (CI: 259.0, 476.1), while the lowest was for total fat intake 4.1 (CI: 2.5, 5.7). Similarly, a significant median difference was found in micronutrient intakes between the two measures. The median difference ranged from 0.09mg/day for vitamin B2 to 391.8ugRAE for vitamin A intake.

Table 2. Mean (SD), median, and 25th, 75th percentiles of daily Energy and nutrients intakes estimated by the average of two 24-Hour dietary recalls and FFQ

 

Energy and nutrients

     Average of 24-Hour 

        Dietary Recalls

                  FFQ

 

 

Independent sample

t-test

Mean

SD

Mean

SD

Energy (Kcal)

1449.50

421.60

1817.10

482.30

367.60 *

Protein (g)

39.40

12.20

49.90

12.50

10.50*

Total fat (g)

17.20

5.70

21.30

6.10

4.10*

Carbohydrate (g)

297.60

92.30

375.20

109.30

77.60*

 

Median

     IQR

(25%, 75%)

Median

     IQR

(25%, 75%)

Wilcoxon signed Rank test

Calcium (mg)

463.10

337.60, 587.90

684.40

518.00, 796.80

0.000**

Iron (mg)

56.00

41.90, 83.70

67.40

54.30, 86.80

0.009**

Vitamin A (ugRAE)

259.30

51.50, 581.40

651.10

295.90, 976.60

0.000**

Vitamin B1 (mg)

0.70

0.49, 0.98

0.81

0.54, 1.17

0.002**

Vitamin B2 (mg)

0.70

0.54, 0.84

0.79

0.64, 0.94

0.002**

*p-value ≤ 0.05        **p- value < 0.01


Table 3 presents the results of correlations between nutrient intakes obtained from the average of two 24-HRs and the FFQ. The Pearson correlation coefficient varied from 0.05 (total fat) to 0.32 (carbohydrate). Except for total fat the correlations were statistically significant. Spearman correlation (rho) obtained for micronutrients ranged from 0.1 (calcium) to 0.49 (vitamin B1). A statistically significant correlation was obtained for vitamin A (p < 0.05) and vitamin B1 (p < 0.05).

Table 3. Correlations of daily Energy and nutrient intakes when comparing the FFQ to the average of two 24-Hour dietary recalls

 

Macro-nutrients

Pearson correlation (95% CI)

                            Bland-Altman statistics

Mean Difference (95% CI)

95% Limit of Agreement

Energy (Kcal)

0.24* (0.05, 0.41)

367.6 (259.0, 476.1) *

-731.9, 1467.1

Protein (g)

0.22* (0.03, 0.36)

10.5 (7.6, 13.5) *

-19.7, 40.8

Total fat (g)

0.05 (-0.14, 0.24)

4.4 (2.5, 5.7) *

-13.7, 22.5

Carbohydrate (g)

0.32* (0.14, 0.48)

77.6 (54.5, 100.6) *

-155.9, 311

Micro-nutrients

Spearman correlation (95% CI)

Bland and Altman statistics for Log transformed data

Mean Difference (95% CI)

95% Limit of Agreement

Calcium (mg)

0.10 (-0.09, 0.29)

0.15 (0.11, 0.18) *

-0.26, 0.55

Iron (mg)

0.12 (-0.07, 0.30)

0.06 (0.007, 0.11) *

-0.45, 0.61

Vitamin A (ugRAE)

0.45* (0.28, 0.59)

0.5 (0.34, 0.66) *

-1.1, 2.1

Vitamin B1 (mg)

0.49* (0.33, 0.62)

0.1 (0.02, 0.11) *

-0.4, 0.5

Vitamin B2 (mg)

0.17 (-0.02, 0.35)

0.05 (0.01, 0.09) *

-0.41, 0.51

*p-value ≤ 0.05        **p- value < 0.01

CI: Confidence Interval;


Table 4 shows cross-classification and weighted Kappa statistics of daily intakes of energy, nutrients and food group in quartiles assessed with average of two 24-HRs and the FFQ. The proportion of individuals classified by the FFQ and the average of two 24 hour dietary recalls into the same quartile ranged from 13.4% for total fat to 38.1% for vitamin A. However, the proportion classified into opposite quartiles varied from 3.8% (vitamin B1) to 23.8% (total fat). Weighted kappa values ranged from -0.04 (total fat) to 0.18 (vitamin A).

Table 4.  Cross-classification and Weighted Kappa statistics of daily Energy and nutrient intakes of in quartiles as assessed with the average of two 24-Hour dietary recall and the FFQ.

 

Energy and nutrients

           Cross-classification

Kappa statistics

% in same quartile individuals

% in opposite quartile of individuals

Kappa value

Energy (Kcal)

34.3

8.6

0.13

Protein (g)

33.4

8.6

0.11

Total fat (g)

13.4

23.8

-0.04

Carbohydrate (g)

34.3

8.6

0.12

Calcium (mg)

25.7

7.6

0.09

Iron (mg)

28.6

9.5

0.05

VitaminA (ugRAE)

38.1

5.7

0.18

Vitamin B1 (mg)

33.3

3.8

0.11

Vitamin B2 (mg)

33.3

8.6

0.11


Table 5 present’s the median, and 25th, 75th percentiles of daily food group intakes estimated by the average of two 24-HRs and FFQ. Both methods provide similar median intake estimates for fruits, eggs, meat/poultry/fish, and daily products. For roots and tubers, the two 24-HR show a higher estimate of median intake. FFQ provides a higher estimates of median vegetable intake. A statistically significant median difference was observed for roots and tubers, eggs and vegetable intake.

Table 5. Mean (SD), median, and 25th, 75th percentiles daily food group intakes estimated by the average of two 24-Hour dietary recalls and FFQ

 

Food group

Average of 24-Hour 

Dietary Recalls

              FFQ

 

 

 

Median

(25%, 75%)

Median

(25%, 75%)

P-value 

Cereals

710

(548.5, 817)

648

(520.9, 852)

0.997

Legumes

94.5

(0, 145.5)

93

(21, 134.9)

0.347

Roots and Tubers

24.5

(0, 45)

11.2

(0, 31.3)

0.013*

Vegetables

79.5

(23.5, 156)

109

(45.3, 159)

0.048*

Fruits

0.0

0.0

0.0

(0, 15.4)

0.367

Eggs

0.0

0.0

0.0

0.0

0.000**

Dairy products

0.0

0.0

0.0

(0, 4.9)

0.087

Meat/Poultry/Fish

0.0

0.0

0.0

0.0

0.068

Beverages

243

(152, 334)

230.4

(183, 320.6)

0.971

Wilcoxon signed Rank test     *p-value ≤ 0.05        **p- value < 0.01


Table 6 shows the correlations between food group intakes obtained from the average of two 24-HRs and the FFQ. Spearman correlation coefficients ranged from 0.12 for egg to 0.78 for legumes. Greater than 0.5 correlation coefficient was observed for legumes (r=0.78). Correlation (0.2-0.49) were observed for cereals (r=0.33), Meat/poultry/fish (r=0.47), fruits (r=0.46), dairy products (r=0.45), roots and tubers (r=0.34), vegetables (r=0.3) and beverages (r=0.2). Correlation was low (<0.2) for egg (r=0.12).

Table 6: Correlations of food group intakes when comparing the FFQ to the average of two 24-Hour dietary recalls.

 

Food groups

Spearman correlation (95% CI)

                         Bland-Altman statistics

Mean Difference (95% CI)

95% Limit of Agreement

Cereals

0.33* (0.15, 0.49)

9.9 (-41.9, 61.8)

535.3, -515.5

Legumes

0.79* (0.71, 0.85)

2.6 (-8.9, 14.1)

-113.8, 118.9

Vegetables

0.33* (0.15, 0.49)

7.1 (-15.7, 29.8)

-223.7, 237.8

Beverages

0.20* (0.01,0.38)

2.9 (-30.9, 36.6)

-339, 344.7

 

 

Spearman correlation (95% CI)

 

     Bland and Altman statistics

Bland and Altman statistics for Log transformed data

Mean Difference (95% CI)

95% Limit of Agreement

Mean Difference (95% CI)

95% Limit of Agreement

Roots and Tubers

0.34* (0.16, 0.45)

16.2 (6.2,26.3) *

-87.5, 120.1

-0.04(-0.23,0.15)

-1.96, 1.88

Fruits

0.46* (0.23, 0.56)

7.9 (1.4, 14.6) *

-60.2, 76.2

0.15 (0.01,0.29) *

-1.25, 1.55

Eggs

0.12 (-0.07, 0.30)

-2.1 (-3.4,0.8) *

-15.9, 11.7

0.19 (0.11,0.27) *

-0.64, 1.03

Dairy products

0.45* (0.29, 0.59)

6.2 (1.9, 10.4) *

-37.8, 50.1

0.04 (-0.07, 0.14)

-1.03, 1.1

Meat/Poultry/Fish

0.47* (0.31, 0.61)

3.8 (-0.8, 8.4)

-43.2, 50.8

0.06 (-0.01, 0.13)

-0.68, 0.8

*p-value ≤ 0.05         **p- value < 0.01


Table 7 shows cross-classification and weighted Kappa statistics of daily intakes of food groups in quartiles as assessed with average of two 24-HRs and the FFQ. The highest correct classification into the same quartile was observed for cereals and legumes-i.e., 50.5% and 51.4%, respectively. For the other food groups, the classification into same quartile ranged from 30.5% (beverages) to 40% (roots and tubers). Oppositely classified individuals ranged from 1% (cereals) to 11.4% (beverages). No gross misclassification was observed for intake of legumes. The weighted kappa values ranged from 0.07 (beverages) to 0.35 (legumes).

Table 7. Cross-classification and Weighted Kappa statistics of daily intakes of food group in quartiles as assessed with the average of two 24-Hour dietary recall and the FFQ

 

 

Food groups

                          

                   Cross-classification

 

Kappa statistics

% in same quartile individuals

% in opposite quartile of individuals

Kappa value

Cereals

50.5

1

0.32

Legumes

51.4

0

0.35

Roots and Tubers

40

7.6

0.18

Vegetables

38.1

6.7

0.17

Beverages

30.5

11.4

0.07


Figure 1 presents the Bland-Altman plots for energy, protein, carbohydrate, total fat, vitamin B1, vitamin A, vitamin B2, calcium and iron. The Bland-Altman plot was used to evaluate the agreement between the FFQ and 24-HR by plotting for each nutrient the difference between the two methods versus the average of the two methods and calculating the limits of agreement and their corresponding 95% CI. Visual inspection of the Bland-Altman plots for both energy and macronutrients shows no consistent trend across the intake values. The FFQ overestimated energy and macronutrient intakes. Except for total fat intake, increased variability of data points was observed for all nutrients both at low, average and high values (wider limits of agreement). Overall, majority of the data points lied between the Limits of Agreements (LOAs). Some outliers were observed for energy and macro-nutrients. Since differences in nutrient intakes were associated with the mean measurement, data related to the micro-nutrient intake were log transformed for Bland and Altman statistics. The result indicates a trend as the FFQ consistently over estimated vitamin A and iron intake at lower value.

Figure 2 shows the Bland-Altman plots for legumes, cereals, vegetables, beverages, roots and tubers, fruits, egg, dairy product and meat/poultry/fish. Data related to roots and tubers was log transformed for Bland and Altman statistics. A systematic trend of overestimation for roots and tubers and FFQ underestimate beverage intakes at higher values. Majority of the data points are in the 95% of LOA for almost all food groups. A wide LOA was observed for roots and tubers. A wide limit of agreement was observed for roots and tubers.

Discussion

In this study, we developed and validated a food frequency questionnaire to assess food and nutrient intakes of adults in Ethiopia. We have observed a higher intake of energy and nutrients when the FFQ was used as compared to the average of the two 24-HRs.

Bland-Altman plots show an overestimation of energy and macro-nutrients (carbohydrate, protein and fat) for various data points. We found a low to moderate level of agreement (correlation coefficients) for energy and nutrient intakes between the two methods. The FFQ did not adequately classify subjects with respect to energy, macro-nutrients and most of the micro-nutrients. For the majority of the food groups, median differences in the intake of foods and nutrients between 24-HRs and FFQ were, overall, small and statistically insignificant. For food groups, a moderate correlation was found between the average of the two 24-HRs and the FFQ. The FFQ showed a fair agreement for cereals, legumes, and roots and tubers.

We found that the FFQ overestimated energy and nutrient intakes relative to the average of the two 24-HRs recalls. Overestimation is a common issue reported in various validation studies (10-15). Overestimation in the present study was moderate for intakes of energy (367.9 kcal), protein (10.5 g) and carbohydrate (77.6 g) and slight for intakes total fat (4.4 g) compared to other validation studies conducted using 24-HRs as their reference method (8, 10-13). Overestimation can be attributed to the subject’s tendency to overestimate their actual intake when they are asked to recall the frequency of a large number of foods consumed in a FFQ. Besides, difficulty in conceptualizing the assigned portion sizes and difficulties in reporting the frequencies of usual intake could be an attributing factor(13). It may have also occurred as a result of purposeful over-reporting of food consumption by subjects (7). The use of shorter questionnaires and advances in portion size estimation techniques are suggested to improve overestimation of intakes by FFQs.

According to the cutoff by Lombard MJ and his colleagues (5), this study found moderate correlations (0.2-0.49) between the average of the two 24-HRs and FFQ for energy (r=0.24), protein (r=0.22), and carbohydrate (r=0.32) and low correlation (<0.2) for fat (r=0.05). The low to moderate correlation coefficients found between the two methods for energy and macro-nutrient intakes are comparable with other previous validation studies (8, 10, 12, 16). However, our finding was lower than those reported by other studies using 24-HRs as their reference method (8, 10, 12). The observed decrease in correlation coefficients could be interpreted as being the result of using only a two day 24-HRs as a reference method. It was found that nutrient correlations were lower when the reference method of the dietary questionnaire was conducted fewer than eight times (17). We believe that the observed correlation estimates could be improved with additional days of recall as well as with multiple studies over different seasons.

The lower correlation coefficients for iron and vitamin intakes observed in our study is not uncommon in FFQ validation studies (8, 10, 11, 13, 18). A meta-analysis of FFQ validation studies showed that pooled correlation coefficients of nutrient intakes ( total fat, protein, carbohydrate, alcohol, calcium, iron and vitamins) were lower for FFQ validated against 24-HRs rather than food record (17). The possible reason for a low correlation for the vitamin is that vitamin intake tends to vary greatly from day to day (as many vitamins are found in only a small selection of foods)(8).

We observed a moderate to good correlation for almost all food groups according to  the previous cutoff point (5). This is in good agreement with previous validation studies assessing food group intakes (8, 10, 19, 20). The good correlation found for vegetable intakes in our study is higher than those reported by other validation studies (8, 10, 12). This may have occurred because of ease of quantifying vegetable intakes as they are often consumed independently in Butajira. The lower correlation of egg intake in our study, in contrast to other studies (10, 11) may have occurred because of not consuming egg on the days where 24-HR was conducted.

The Bland-Altman plot showed a moderate agreement between the two methods for energy and macro-nutrients. Trend was not observed across the intake level in energy and macronutrient intakes. Similarly, another study also showed a moderate level of agreement with no persistent trend across intake levels using a Bland-Altman plot (12, 21). However, ranges for limits of agreement were relatively wide as opposed to another study (13). The observed wide limits of agreements between FFQ and the reference method are common, hence highlighting the limitation of the FFQ in assessing absolute nutrient intake due to wide variability in how the FFQ measured energy and macronutrient intake relative to the average of the two 24-HRs (4).

A tendency towards a poorer agreement in Vitamin A and iron intake between methods was observed with lower levels of intake as shown by the Bland-Altman plot. This poor agreement  in iron intake is also reported in another validation study (22). As indicated by a Bland-Altman plots, a systematic mean difference was not observed across the intake levels of Cereals, legumes, vegetables and beverages. Most of the data points are found between the 95% limits of agreement. However, the plot indicated wide limits of agreement which occurs as a result of increased variability.

The present study showed that the FFQ did not adequately classify subjects with respect to energy, macro-nutrients and most of the micro-nutrients as indicted by cross-classification and weighted kappa results (percentage of individuals in same quartile < 50 , K values < 0.2) (5). However, the FFQ showed a fair quartile classification agreement for cereals, legumes, and roots and tubers (K values 0.2-0.6). This finding is consistent with previous studies which reported cross-classification and kappa by categorizing intakes into quartiles (10, 11, 22). We found lower values for energy and nutrients with respect to those reported by other studies using similar intake categories (8, 12, 13, 23). The misclassification and low kappa reported in our study may have occurred due to the insensitivity of FFQ to classify individuals into intake categories. The use of a Food diary as a reference method may have also increased classification agreement in the previous studies. FFQ showed a fair quartile classification agreement for cereals, legumes, and roots and tubers (K values 0.2-0.6). Similarly, other studies reported a fair classification agreement for this particular food group(13, 19). For beverage intake, our study indicated a misclassification (30.5%) into opposite quartile supported by low kappa value (k value <0.2) showing the poor outcome. Other study reported a similar finding for beverage intake (19).

The present study has some limitations that must be acknowledged. First, given that we used a 24-HR dietary assessment method as our reference method, the sources of error between 24-HR and FFQ may tend to be correlated due to conceptualization of portion sizes. However, to lessen this effect we have used a salted replica of actual foods, pictures and calibrated equipment to estimate portion size. Second, we conducted two 24-HR per participant. This might have influenced the result obtained, particularly for estimating usual intakes of foods not consumed on a daily or regular bases such as meat/poultry/fish and the intake of other specific food groups. Third, participants may have purposefully over reported their intake due to social desirability. However, we gave a detailed explanation for the interviewers on how to explain the purpose of the FFQ to the participants using role-playing, small group exercises, and discussions. Fourth, we did not administer the FFQ at the onset of the study; therefore, we cannot assess the reproducibility of the instrument. Fifth, the assessment was not carried out at several seasons that might lead to variation in estimation of food intake. Sixth, inter-rater reliability between raters for this study was not calculated.

The strength of the present study are the development of the FFQ based on the latest local dietary survey, focal group discussions, pre-test, and expert reviews, the use of comprehensive statistical analysis, to assess the validity of the FFQ and the use of an interactive, multiple-pass 24-HR adapted and validated for use in developing countries as our reference method..

Conclusions

The study showed that FFQ had good validity to capture intakes of cereals, legumes, vegetables, and beverages both at individual and group levels. However, intakes of root and tuber and beverages are associated with potential systematic bias. Therefore, caution must be exercised when using FFQ for this particular food groups. The FFQ can be used to rank individuals based on cereals, legumes, roots and tubers and vegetable intakes. The supporting individual level validity was acceptable for energy and macro-nutrients as indicated by correlation coefficients and Bland-Altman plots. However, estimates of minerals and vitamins should be interpreted with caution. In summary, FFQ is capable of classifying an individual’s food group intake into quartiles, which is useful in examining the relationships between diet and chronic disease. Food group intake agreement analysis also indicated the usefulness of the tool to assess dietary intakes in large epidemiological studies. However, the use of the tool to assess absolute nutrient intakes at the food group level should be exercised with caution.

Declarations

Ethics approval and consent to participate

Ethical clearance was obtained from the institutional review board of School of Public Health, College of Health Sciences, Addis Ababa University. Informed written consent was obtained from the study participants. The study is in compliance with the principles of the declaration of Helsinki.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to the confidentiality regulations of our University, but are available from the corresponding author upon request.

Competing interests

The authors declare that they have no competing interests.

Funding

The study did not receive funding. All participants voluntarily agreed to take part.

Authors’ Contributions

Conceptualization, I.F., B.S.E., S.H.G., and E.H.; methodology, I.F., B.S.E., S.H.G., and E.H.; software, I.F., B.S.E., S.H.G., and E.H.; validation, I.F., B.S.E., S.H.G., and E.H.; formal analysis, I.F., B.S.E., S.H.G., E.H., and H.Y.H.; investigation, I.F.; writing original draft preparation, I.F., B.S.E., S.H.G., E.H., and H.Y.H.; writing, review and editing, I.F., B.S.E., S.H.G., E.H., and H.Y.H. All authors approved the final version to be submitted.

Acknowledgments

We are grateful to supervisors, data collectors and study participants

Authors’ information  

Ilili Feyesa1, Bilal S Endris1, Esete Habtemariam1, Hamid Y Hassen2, Seifu H Gebreyesus1

1Department of Nutrition and Dietetics, School of Public Health, College of Health Sciences Addis Ababa University, Ethiopia; [email protected] (IF); [email protected] (BSE); [email protected] (EH); [email protected] (SHG)

2Department of Primary and interdisciplinary care, Faculty of Medicine and health sciences, University of Antwerp, 2610, Antwerp, Belgium. [email protected] (HYH)

*Correspondence: [email protected]; Tel.: +251919324253

References

  1. Thompson FE, Subar AF. Dietary assessment methodology. Nutrition in the Prevention and Treatment of Disease (Third Edition): Elsevier; 2013. p. 5-46.
  2. Gibson RS. Principles of nutritional assessment. 2005.
  3. Willet W. Nutritional Epidemiology. Third ed. New york: Oxford university press; 2013.
  4. Cade JE, Burley V, Warm D, Thompson R, Margetts B. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutrition research reviews. 2004;17(1):5-22.
  5. Lombard MJ, Steyn NP, Charlton KE, Senekal M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. Nutrition journal. 2015;14(1):40.
  6. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires–a review. Public health nutrition. 2002;5(4):567-87.
  7. Gibson RS, Ferguson EL. An interactive 24-hour recall for assessing the adequacy of iron and zinc intakes in developing countries. Washington, DC: IFPRI and CIAT. 2008.
  8. Jackson MD, Motswagole BS, Kwape LD, Kobue-Lekalake RI, Rakgantswana TB, Mongwaketse T, et al. Validation and reproducibility of an FFQ for use among adults in Botswana. Public health nutrition. 2013;16(11):1995-2004.
  9. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-10.
  10. Rodríguez MM, Méndez H, Torún B, Schroeder D, Stein AD. Validation of a semi-quantitative food-frequency questionnaire for use among adults in Guatemala. Public health nutrition. 2002;5(5):691-8.
  11. Zack RM, Irema K, Kazonda P, Leyna GH, Liu E, Gilbert S, et al. Validity of an FFQ to measure nutrient and food intakes in Tanzania. Public health nutrition. 2018:1-10.
  12. Liu L, Wang PP, Roebothan B, Ryan A, Tucker CS, Colbourne J, et al. Assessing the validity of a self-administered food-frequency questionnaire (FFQ) in the adult population of Newfoundland and Labrador, Canada. Nutrition journal. 2013;12(1):49.
  13. Dehghan M, Martinez S, Zhang X, Seron P, Lanas F, Islam S, et al. Relative validity of an FFQ to estimate daily food and nutrient intakes for Chilean adults. Public health nutrition. 2013;16(10):1782-8.
  14. Denova-Gutiérrez E, Ramírez-Silva I, Rodríguez-Ramírez S, Jiménez-Aguilar A, Shamah-Levy T, Rivera-Dommarco JA. Validity of a food frequency questionnaire to assess food intake in Mexican adolescent and adult population. salud pública de méxico. 2016;58:617-28.
  15. El Kinany K, Garcia-Larsen V, Khalis M, Deoula MMS, Benslimane A, Ibrahim A, et al. Adaptation and validation of a food frequency questionnaire (FFQ) to assess dietary intake in Moroccan adults. Nutrition journal. 2018;17(1):61.
  16. Boucher B, Cotterchio M, Kreiger N, Nadalin V, Block T, Block G. Validity and reliability of the Block98 food-frequency questionnaire in a sample of Canadian women. Public health nutrition. 2006;9(1):84-93.
  17. Molag ML, de Vries JH, Ocké MC, Dagnelie PC, van den Brandt PA, Jansen MC, et al. Design characteristics of food frequency questionnaires in relation to their validity. American journal of epidemiology. 2007;166(12):1468-78.
  18. Zang J, Luo B, Chang S, Jin S, Shan C, Ma L, et al. Validity and reliability of a food frequency questionnaire for assessing dietary intake among Shanghai residents. Nutrition journal. 2019;18(1):30.
  19. Lin P-I, Bromage S, Mostofa MG, Allen J, Oken E, Kile ML, et al. Validation of a dish-based semiquantitative food questionnaire in rural Bangladesh. Nutrients. 2017;9(1):49.
  20. Steinemann N, Grize L, Ziesemer K, Kauf P, Probst-Hensch N, Brombach C. Relative validation of a food frequency questionnaire to estimate food intake in an adult population. Food & nutrition research. 2017;61(1):1305193.
  21. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative Food Frequency Questionnaire for Argentinean adults. PLoS One. 2012;7(5):e37958.
  22. Tollosa DN, Van Camp J, Huybrechts I, Huybregts L, Van Loco J, De Smet S, et al. Validity and reproducibility of a food frequency questionnaire for dietary factors related to colorectal cancer. Nutrients. 2017;9(11):1257.
  23. Goulet J, Nadeau G, Lapointe A, Lamarche B, Lemieux S. Validity and reproducibility of an interviewer-administered food frequency questionnaire for healthy French-Canadian men and women. Nutrition journal. 2004;3(1):13.