Development and validation of a quantitative food frequency questionnaire to assess free sugar intake among Sri Lankan preschool children

Abstract

Background: Monitoring the free sugar intake of preschool children is important for population health surveillance and evaluation of community-wide efforts at non-communicable diseases and dental caries prevention. Appropriate dietary assessments tools are also needed to underpin clinical prevention for patients. Thus, anappropriate dietary assessment tool applicable for large-scale, community-based studies without an excessive participant burden is needed, such as a quantitativefood frequency questionnaire that assesses dietary data retrospectively. Suchtools also need to be tailored and validated for the cultural context andpopulation concerned. Sri Lanka is currently lacking such a tool for use amongits pre-school population.

Methods: In the development phase, a sample of 518 pre-school children and their caregivers were selected from pre-schools in the District of Colombo, Sri Lanka. Three 24-hour dietary recalls were collected from each child and based on that, a 67-item food frequency questionnaire was developed, including commonly consumed free sugar-containing food items. In the validation phase, 108 pairs of preschool children aged 04-05-years, and their caregivers participated. The relative validity of the food frequency questionnaire was assessed by comparing it with three, 24-hour dietary recalls. The test-retest reliability was assessed by repeated application of the food frequency questionnaire to the same population in six weeks intervals.

Results: The free sugar intake values calculated from the food frequency questionnaire when compared with three, 24-hour dietary recalls indicated a good correlation, good agreement in cross classifying participants to tertiles based on their sugar intake levels and a good agreement in bland and Altman graphs at different food group levels and total free lsugar intake. The food frequency questionnaire measured sugar intake levels slightly higher than three 24-hour dietary recalls. Similarly, the free sugar intake values calculated by repeated application of the food frequency questionnaire also showed a good agreement at the group and individual levels as well as a good correlation.

Conclusion: The newly developed quantitative food frequency questionnaire was found as a valid and reliable instrument to assess the free sugar intake among preschool children quantitatively as a total or by different food and beverage group levels.

Background

The World Health Organization strongly recommends that consumption of free sugars should be limited to less than 10% of energy intake [1]. Free sugars are therefore defined as "all monosaccharides and disaccharides added to food by the manufacturer, cook or the consumer, plus the sugar naturally found in honey, syrups and fruit juices" [1]. These do not include the sugars present in milk or in fruit and vegetables (intrinsic sugars), since the latter are bound by a cell wall and tend to be digested more slowly and take longer to enter the blood stream than free sugars. Excess free sugar intake is associated with numerous adverse health outcomes: predominantly non-communicable diseases and specifically, obesity, diabetes, cardiovascular diseases, several forms of cancers and dental diseases [1] [2]. Thus health authorities have looked for mechanisms to facilitate curtailing the intake of sugars in recent years [3].

In the Sri Lankan context, early childhood dental caries (ECC) is a significant public health issue affecting about 63% of 5-year-olds and persisting for many decades [4]. In Sri Lanka, obesity is also an emerging health issue in children, with obesity rates among teenagers and adults already a significant problem in the country [5]. Dietary assessment tools provide a means of evaluating progress towards nutritional goals at both a population on an individual (clinical basis). For example, the computer-assisted 24-HDRmSACINA tool has allowed comparison of free sugar consumption between eight European countries [6], showing that less than 20% of children are within the less than 10% of energy from free sugars recommendation.

Implementation of these tools in a global health setting such as Sri Lanka however, is challenging; firstly because of limited finances dedicated to health monitoring and secondly because of the need to tailor it to the traditional food context. Also, with ECC a key concern in Sri Lanka, a dietary tool which especially reflects aetiological factors of dental caries is needed, both for population surveillance and policy making (9, 10), and also as a tool for clinical prevention. Several dietary assessment tools are available in Sri Lanka, although there are no specifically designed tools to assess the free sugar intake particularly among pre-school children.

Despite weighted dietary records being the gold standard in nutritional epidemiology, they have inherited limitations like being costly to use in large-scale community-based studies, requiring great commitment from participants, and having a high burden on them [7]. Oppositely, a food frequency questionnaire (FFQ) is inexpensive, simple to administer particularly for a large cluster imposing less burden on the respondent. Moreover, it captures the habitual day-to-day dietary intake of an individual, over an extended period of time retrospectively, avoiding short term disparities, like seasonal variations and dietary alterations during the periods with illnesses [8] [9] [10] which make the FFQ the tool of choice in the present study. FFQ should be specifically designed for the target population as dietary habits are culture specific [11] [12], and further, should assess the validity and reliability prior to use [9][13]. The objective of the current study was therefore to develop a quantitative FFQ to assess the free sugar intake among pre-school children and to assess the relative validity and reliability of it.

Methods

Study design

A cross sectional descriptive study was conducted to identify the commonly consumed free sugar containing food and beverage items by the pre-school children in Colombo. Prior to conducting the study, ethics approval was obtained from the ethics review committee, Faculty of Medicine, University of Colombo, Sri Lanka (EC-17-001). Written informed consent was obtained from all the primary caregivers of the selected children who participated in the study. 

Development Phase

Study population

For the initial phase of the study 518, apparently healthy 04-05-year-old preschool children were enrolled randomly from 26 preschools in the district of Colombo, Sri Lanka. All included children were those who had resided in the district from birth, were not on a special diet and had a primary caregiver was available for data collection.  These 26 preschools were selected as two random preschools from all 13 district secretariat divisions in the Colombo district.

Development of the food frequency questionnaire:

During this phase, all the food and beverage consumption data of included children were collected by face-to-face interviews with the primary caregivers based on 24-hour dietary recalls (24hDR), covering one weekend day and two weekdays. From these, all food and beverage items that contained sugar were extracted, and then the foods and beverages that contributed 95% of the sugar intake were identified by stepwise regression. The final dietary inventory list included 60 sugar-containing food and beverage items, as well as seven food and beverage items to which caregivers commonly added sugar at the time of consumption.

Finally, the identified sugar-containing food and beverage items were classified into seven categories by two nutrition specialists based on similarity of consistency and preparation methods [14] [15] [16] as: bakery products, biscuits, sugar confectionery, chocolate confectionery, sugar-sweetened beverages, desserts. Remaining items were classified as miscellaneous sweets. In the FFQ these were asked in order of descending frequency of intake. Frequency options were included as ‘never’ or ‘times per day/per week or per month’ for the respondent to write the frequency in numbers in the relevant column. In measuring the intake of a particular nutrient, precise quantification of the food and beverage intake is vital. Thus, commonly used measures for quantification of each food and beverage item were identified: for example, different sizes of glasses, cups and spoons. Along with those, some actual food items were identified in available smallest portions in the market, such as toffees, pieces of chocolate, biscuits some more. A power-point presentation was developed, including these, and measuring instruments were identified to demonstrate at the time of data collection. The FFQ was originally developed in Sinhala and translated to Tamil and English using backwards and forward translation methodology. It was pretested among 20 caregivers of pre-school children for the clarity of the instructions given and the food names, according to the findings, a few alterations were done.

Compilation of the food composition database

Since the existing food composition databases for Sri Lankan foods do not provide accurate free sugar content of these sugary foods, the researchers compiled a comprehensive database on the free sugar content of the identified 60 foods and beverage items employing a number of methods used in the compilation of  food composition databases [17] [18]: the recipes from reputed Sri Lankan culinary books, information gleaned from food labels, recipes and analytical reports from local manufacturers were among them. 

Validation Phase

Due to non-availability of a gold standard method for assessment of diet, criterion validity of the food frequency questionnaire cannot be evaluated. Thus, to assess relative validity, free sugar intake measures derived from FFQ were compared with free sugar intake measures derived from three 24-hour dietary recalls (24hDR) for three days as the reference method [19] [20] [21] [22] [23]. The test-retest reliability of the FFQ was determined by administering the same FFQ twice to the same population over a six-week period. 

Study population

During the validation phase of the study, we recruited an additional 113 preschool children aged 04-05 from 10 preschools, as a minimum sample size was required to achieve a 5% significance level and 80% power to demonstrate a minimum correlation of 0.3 (24) and 10% of non-response. However, only 108 participants completed both stages of data collection. As the FFQ was developed to assess the sugar intake of preschool children in Colombo District in the next phase, to avoid contamination validation study was conduct in two adjacent district secretariat outside Colombo District. Simple random sampling was used to select five preschools from each area. Healthy children who did not follow a special diet and whose primary caregivers were available for data collection were included in the study after receiving informed consent.

Dietary assessment

Data was obtained by meeting with the child’s primary care giver twice in two days. The same FFQ was completed as a self-administered questionnaire at the first visit (FFQ1) and at the second visit (FFQ2) six weeks later. Participants gathered in groups of 10 in a hall, and the researcher gave clear instructions to complete the FFQ, followed by exercises with some hypothetical examples. The FFQ was then completed based on the child's usual diet for the past three months. When participants completed the FFQ, use a power-point presentation to illustrate serving sizes for each food and drink. 

Again, all necessary directions to complete the 24hDR were provided both orally and in print, and caregivers were trained to complete these at home over two weekdays and one weekend. They were encouraged to choose days that reflected the child’s usual diet. They received printed manual on how to choose the serving size.

Data Analysis     

Calculation of sugar intake;

The total daily free sugar intake of each child was calculated, separately through FFQ1, FFQ2 and the three 24hDR.

• Using the FFQ, free sugar intake from each food and beverage item per day was calculated.

Sugar intake from particular = Amount of intake x Frequency of x Free sugar concentration

       Food/ beverage item                                                 intake                 of the item                  

Daily intake of sugar was calculated by dividing weekly intake by 7 and monthly intake by 30.4. The total daily intake was calculated adding all these daily sugar intake values

• Using 24HDR sugar intake was calculated by,

Sugar intake from particular = Amount of intake x Free sugar concentration

     Food/ beverage item                                                         of the item

Adding sugar intake from each item the daily sugar intake was calculated, mean sugar intake for the three days was taken for final comparison. 

Statistical analysis 

Validity was assessed through comparison of sugar intake calculated from FFQ1 and FFQ2 separately with the mean sugar intake calculated from three 24hDRs, while reliability was assessed through comparison of sugar intake values calculated from FFQ1 and FFQ2.

Data was analyzed using SPSS version 21. The following statistical methods were used for the comparison; comparison of the means by calculating the mean percentage difference, Wilcoxon sign rank test, a relative agreement between two methods was assessed by ranking cross-classification and weighted kappa coefficient, Spearman correlation was used to measure the degree to which the two administrations are related along with intraclass correlation (reliability only) and Bland–Altman method was used to measure the agreement between the daily sugar intake values obtained from FFQ and 3hDR throughout the range of intake [14][24][25]. Several statistical approaches were utilized simultaneously as they analyze various aspects of validity and reliability at group level, individual-level and throughout the range of free sugar intake levels.

Results

For comparison, we looked at the sugar intake from different categories of foods and beverages separately, as well as the total amount of free sugar taken in. The results were interpreted according to the criteria established by Lombard et al. [25].  

Table 1

 Interpretation criteria for statistical tests assessing the validity and the reliability of the FFQ [25] 

Mean (Standard Deviation) free sugar intake from repeated application of the food frequency questionnaire FFQ1, FFQ2 and three, 24hDR was 83.03 (66.6) grams/day, 82.0 (84.81) grams/day and 81.18 (69.0) grams/day, respectively. Comparing the first (FFQ1) and second (FFQ2) application of food frequency questionnaire with three 24hDRs, the mean percentage difference was 3.4% and 2.6%, respectively, which was less than 10%, indicating satisfactory agreement between the two methods. Similarly, when it comes to reliability, there were 9.3% differences between FFQ1 and FFQ2, which is an acceptable agreement. 

Sugar intake values from FFQ1, FFQ2, and three 24hDRs were skewed, deviating from a normal distribution with median values (Inter Quartile Range) of 64.5(39.9-111.5) grams/day, 56.9(29.4-102.8) grams/day, and 61.8 (41.3-97.8) grams/day, respectively. The Wilcoxon sign rank test demonstrated no significant difference between the FFQ1 and three 24 hDR (p=0.13), FFQ2 and three 24 hDR (p=0.63), and FFQ1 and FFQ2 (p=0.45), which was also present at the levels of food and beverage groups (Table 2).

Table 2

Comparison of median sugar intake from different food groups according to FFQ1, FFQ2, and three 24hDR; significance of differences (N= 108) 

 * Intake was measured in g/day (median and IQR)       

**IQR= Interquartile range 

According to the results of the FFQ1, FFQ2, and 24hDR, the degree of potential cross-classification was assessed through the classification of participants into tertiles based on their free sugar intake levels, which indicates the capacity of the dietary assessment method to rank participants (Table 2). In the comparison of the two methods, the percentages correctly classified into the same tertiles for different sugary food groups were over 50% and the percentages misclassified into the opposite tertile were less than 5%, when comparing FFQ1 and  FFQ2 independently with three, 24hDR and weighted kappa coefficient values were above 0.61 for allmost all the food groups (Table 3). 

The percentage correctly classified into the same tertile according to the free sugar intake data measured by two applications of FFQ was higher than 50%, and the percentages misclassified were below 10%, while the weighted kappa coefficient was also above satisfactory level (Table 3). 

Table 3

Cross classification of total sugar intake assessed by FFQ1, FFQ2 and three 24hDR (N= 108)

* As the same values were repeated over time, chocolate confectionery and table sugar could not be classified into tertiles

 The Spearman correlation coefficient for sugar intake obtained from FFQ1 and FFQ2 separately with that of three 24-hDRs was above 0.5 for all sugary food groups. The Spearman correlation coefficient for repeated application of the FFQ was also above 0.5 for all sugary food groups. Additionally, intra-class correlation coefficients (ICC) were calculated and all these values showed a relatively good correlation (Table 4).  

Table 4

Spearman correlation coefficient between sugar intake assessed by FFQ1 and 24hDR (N= 108)

* Data on the intake of chocolate confectionery and table sugar from FFQ can't be classified into tertiles since the same values were repeated, therefore, ICC is not calculated.

The agreement between the two methods was evaluated graphically by plotting the Bland-Altman plots. Comparisons of FFQ1 and FFQ2 with three 24hDRs showed a good agreement (Figure 1). Visual inspection of the graphs showed no difference throughout the range of intake and  less than 5% of participants were found outside the limits of agreement. 

The Bland and Altman plot (Figure 2) indicates good agreement between total sugar intake from repeated application of FFQ (FFQ1 and FFQ2) and only 5% of participants were found outside the limits of agreement.  

All these assessments reveal a high level of agreement when comparing the two applications of FFQ separately with the three 24hDR and also between the repeated applications of the FFQ (FFQ1 and FFQ2).

Discussion

The FFQ was developed targeting the 04–05-year age group, based on the food intake data from a representative sample of children. This FFQ consists of 67 food items and according to Cade et al.[26], less than 100 items per FFQ would be optimal, thus this number can consider as sufficient to obtain accurate data. Previous studies which develop FFQ to assess sugar intake has identified an almost close number of items as 64 [22] and 77 [27].

The literature on the development of FFQ for the assessment of sugar intake is sparse, and most were designed for various age groups, and the types of sugar they were referring to were different, which makes it difficult to compare it to the current FFQ. Sugar intake values calculated using the FFQ were slightly higher than those calculated using the 24hDR, which may be due to the relatively long list of foods included in the FFQ, as in most earlier studies [22] and [28]. 

The mean percentage difference and the Wilcoxon sign rank test between FFQ1 and three 24hDR, between FFQ2 and three 24hDR and between the two applications of FFQ demonstrated no significant difference. It can therefore be established that the FFQ was valid and reliable for estimating free sugar intake at the group level.

The Spearman correlation is frequently used, to measure the relationship between two applications of FFQ and three 24hDR, all these values were above 0.5, which can be considered as a reasonably good agreement on individual level as it was above 0.3 [13]. These values were almost compatible with the other FFQ validation studies [29] and [22]. Current findings were higher than those of the FFQ development study that assessed sugar intake among Australian toddlers [30]. This may be due to the inconsistent dietary intake patterns of toddlers  compared to the present study population, which targets children between the ages of 04 and 05. 

All sugary food categories and total sugar have Spearman correlation coefficients above 0.5. Intra-class correlation coefficients (ICC) were calculated because they account for both within- and between-subject variability [31] and are the most appropriate test for assessing the agreement between the repeated FFQs in ranking individuals by their intake of free sugar. Interestingly, all of these values have shown a good correlation with a minimum of 0.5 and these findings were compatible with the ICC values obtained in another study for Asia pacific region [29]. 

By classifying participants by both test and reference methods into tertiles, we can calculate the percentage of participants correctly classified into the same tertile and the percentage misclassified in the opposite tertile. This indicates the ability of the dietary intake assessment method to rank the participants correctly, reflecting the agreement on the individual level. These findings were almost identical to those found in the previous study of Pacific Islanders in South Auckland [29]. 

 Bland and Altman's plots were used to visually compare two methods and determine to what extent they agree across a wide range of intakes. It can identify systematic differences (bias) between two comparison methods throughout the range of values and calculate the limits of agreement. Sugar intake values were in good agreement between the two methods with no observable difference within the range of intake, which was consistent with previous studies done on Australian toddlers [30] and Malaysian adults[22].  

Limitations

As both FFQ and 24hDR depend on memory, overestimation and underestimation are possible. FFQ validation was performed using 24hDR, but these two methodologies are completely opposed, since FFQ is a retrospective method for a long period of time, whereas 24hDR is a prospective method for a shorter period.

Conclusion

FFQ contains a reasonable number of food and beverage items that contribute to 95% of the variation in sugar intake among pre-school children. This can also be adapted to other parts of the country by slightly modifying the food list according to the local context. The new quantitative food frequency questionnaire (FFQ) was found to be a valid and reliable method of assessing the amount of free sugar consumed and ranking participants accordingly. Additionally, it is well suited to assessing the amount of free sugar consumed by different sugar-containing food groups. Consequently, this tool can be used to assess free sugar intake at the population level or individual sugar intake in clinical settings.

Abbreviations

95% CI- 95% Confidence Interval

ECC – Early Childhood Caries

FFQ – Food Frequency Questionnaire

24hDR – 24 Hour Dietary Recall

ICC- Intra Class Correlation

IQR - Inter Quartile Range

P – probability value

ROC curve – Receiver Operating Characteristic curve

SCC- Spearman Class Correlation

SD- Standard Deviation

WHO – World Health Organization

Z- Z value

Declarations

• Ethics approval and consent to participate 

Ethcal approval to conduct the study was gained from the Ethics Review committee, Faculty of Medicine, University of Colombo prior to data collection (EC-17-001). According to our confirmation, all methods followed relevant guidelines and regulations.

Written informed consents were obtained from all caregivers of the participating children, and those who agreed were included in the study. 

• Consent for publication 

Not Applicable 

• Availability of data and materials 

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. 

• Competing interests 

The authors declare that they have no competing interests. 

• Funding 

Authours did not receive any funding for the above study 

• Author’s contributions 

SM- Designing the study, data collection, data analysis, interpretation of data, article writing

TT and AS - Designing the study, data interpretation, reviewed the article, Supervision and mentorship

RH- Technical supervision, reviewed the article and editing 

• Acknowledgements 

Not Applicable

References

1. World Health Organization. Guideline: sugars intake for adults and children. Available from: https://www.who.int/publications/i/item/9789241549028
2. Steyn K, Damasceno A. Lifestyle and Related Risk Factors for Chronic Diseases. Disease and Mortality in Sub-Saharan Africa. The International Bank for Reconstruction and Development / The World Bank; 2006. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21290651
3. Popkin BM, Hawkes C. Sweetening of the global diet, particularly beverages: Patterns, trends, and policy responses. Vol. 4, The Lancet Diabetes and Endocrinology. Lancet Publishing Group; 2016. p. 174–86. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4733620/
4. Ministry of Health Sri Lanka. National Oral Health Survey Sri Lanka 2015- 2016. 2018. Available from: http://www.health.gov.lk/moh_final/english/public/elfinder/files/publications/2019/NOHS2015-2016.pdf
5. Department of Census and Statistics [Internet]. Demographic and Health Survey, Sri Lanka. Available from: http://www.statistics.gov.lk/Health/StaticalInformation
6. Graffe MIM, Pala V, De Henauw S, Eiben G, Hadjigeorgiou C, Iacoviello L, et al. Dietary sources of free sugars in the diet of European children: the IDEFICS Study. 2020;59(3):979–89. Available from: https://doi.org/10.1007/s00394-019-01957-y
7. Ortega RM, Perez-Rodrigo C, Lopez-Sobaler AM. Dietary Assessment Methods: Dietary Records. 2015;31(April):38–45.
8. Shim J-S, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. 2014;36:1–8.
9. Sauvageot N, Alkerwi A, Albert A, Guillaume M. Use of food frequency questionnaire to assess relationships between dietary habits and cardiovascular risk factors in NESCAV study : validation with biomarkers. 2013;1–11.
10. Carpenter CL. Dietary Assessment. Nutritional Oncology. 2006. 367–375 p.
11. Shim J-S, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. 2014 Jul 22;36:e2014009. Available from: /pmc/articles/PMC4154347/?report=abstract
12. Welch AA. Dietary intake measurement: Methodology. 2013 Jan 1;2–4:65–73.
13. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. 2002 Aug;5(4):567–87. Available from: https://pubmed.ncbi.nlm.nih.gov/12186666/
14. Nik Shanita S, Norimah N, Abu Hanifah S. Development and validation of a food frequency questionnaire (FFQ) for assessing sugar consumption among adults in Klang Valley, Malaysia. 2012;18(3):283–93.
15. Sheehy C, McNeill G, Masson L, Craig L. Survey of sugar intake among children in Scotland. 2008; Available from: http://www.food.gov.uk/sites/default/files/multimedia/pdfs/publication/surveyofsugarscotland0308.pdf
16. O’Connor L, Imamura F, Brage S, Griffin SJ, Wareham NJ, Forouhi NG. Intakes and sources of dietary sugars and their association with metabolic and inflammatory markers. 2018 Aug 1;37(4):1313–22.
17. Rand WM, Pennington J a T, Murphy P, Klensin JC. Compiling Data for Food Composition Data Bases. 1991;1–65.
18. Ramírez-Luzuriaga MJ, Silva-Jaramillo KM, Belmont P, Freire WB. Tabla de composición de alimentos para Ecuador: Compilación del Equipo técnico de la ENSANUT - ECU 2012. 2014;8:19. Available from: http://nutritionj.biomedcentral.com/articles/10.1186/1475-2891-5-2
19. Dehghan M, del Cerro S, Zhang X, Cuneo JM, Linetzky B, Diaz R, et al. Validation of a semi-quantitative food frequency questionnaire for argentinean adults. 2012;7(5).
20. Haftenberger M, Heuer T, Heidemann C, Kube F, Krems C, Mensink GBM. Relative validation of a food frequency questionnaire for national health and nutrition monitoring. 2010;9(1):1–9.
21. Kroke A, Klipstein-Grobusch K, Voss S, Möseneder J, Thielecke F, Noack R, et al. Validation of a self-administered food-frequency questionnaire administered in the European Prospective Investigation into Cancer and Nutrition (EPIC) Study: Comparison of energy, protein, and macronutrient intakes estimated with the doubly labeled water,. 1999;70(4):439–47.
22. Shanita NS, Norimah AK, Hanifah AS. Food Frequency Questionnaire (FFQ) for sugar. 2012;18(3):283–93. Available from: http://nutriweb.org.my/mjn/publication/18-3/a.pdf
23. Patterson RE, Kristal AR, Tinker LF, Carter RA, Bolton MP, Agurs-Collins T. Measurement characteristics of the Women’s Health Initiative food frequency questionnaire. Vol. 9, Annals of Epidemiology. 1999. p. 178–87.
24. Hafizah YN, Choo Ang L, Yap F, Najwa WN, Cheah WL, Talib Ruzita A, et al. Validity and Reliability of a Food Frequency Questionnaire (FFQ) to Assess Dietary Intake of Preschool Children. 2019;16:4722. Available from: www.mdpi.com/journal/ijerph
25. Lombard MJ, Steyn NP, Charlton KE, Senekal M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. 2015;14(1). Available from: http://dx.doi.org/10.1186/s12937-015-0027-y
26. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires – a review. 2002 Aug;5(4):567–87.
27. Decarli A, Franceschi S, Ferraroni M, Gnagnarella P, Parpinel MT, La Vecchia C, et al. Validation of a food-frequency questionnaire to assess dietary intakes in cancer studies in Italy results for specific nutrients. 1996 Mar 1;6(2):110–8.
28. Marques-Vidal P, Ross A, Wynn E, Rezzi S, Paccaud F, Decarli B. Reproducibility and relative validity of a food-frequency questionnaire for French-speaking Swiss adults. 2011;55.
29. Boniface OT. Validation of A Short Food Frequency Questionnaire Which Ranks Individuals by Sugar Intakes in Pacific Islanders in South Auckland, New Zealand. 2013;(December).
30. Devenish G, Mukhtar A, Begley A, Do L, Scott J. Development and relative validity of a food frequency questionnaire to assess intakes of total and free sugars in Australian toddlers. 2017;14(11):5–7.
31. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. 2016 Jun 1;15(2):155. Available from: /pmc/articles/PMC4913118/