This study complies with the International Ethical Guidelines for Biomedical Research Involving Human Subjects (24) and received ethical approval from the WHO Ethics Board (Ref 004583), followed by ethical approval from institutional ERCs of individual study sites. From Pakistan, approval was sought from the National Bioethics Committee NBC (Ref 4–87/NBC-/422/19/1170) and Aga Khan University AKU (Ref. 1567). For the Bangladesh site, approval was obtained from the Institutional Review Board (IRB) of the Projahnmo Research Foundation (PR-190002) and Johns Hopkins Bloomberg School of Public Health (IRB No.: 00009615). In the United Republic of Tanzania-Pemba, the study was approved by the Zanzibar Health Research Ethics Committee (Ref: ZAHREC/03/PR/Sept/2019/02).
Study Settings and Participants
The feasibility study was conducted from January 2020 to March 2020 in Bangladesh, Pakistan, and the United Republic of Tanzania. In all three countries, most children were enrolled from existing cohorts of the Alliance for Maternal and Newborn Health Improvement (AMANHI) study group (25). In sites where children had outgrown the needed age groups, newborn and younger children were recruited from the Antenatal CorTicosteroids for Improving Outcomes in preterm Newborns (ACTION) trial in Bangladesh (JHSPH IRB # 00007684) (26) and the Demographic Surveillance System in Pakistan (27).
In Bangladesh, the GSED study was implemented in Sylhet district, particularly the two subdistricts of Zakiganj and Kanaighat, where the AMANHI study group maintains a health and demographic surveillance of 500,000 people with an annual birth cohort of approximately 12,500, and the catchment areas include three tertiary care hospitals in Sylhet city. In Pakistan, the study site was a fishing village (Ibrahim Hyderi) located on the outskirts of the metropolitan city of Karachi. In 2022, the number of children under the age of 5 was approximately 15,393, and the annual birth cohort was 3500 (unpublished data). The Department of Pediatrics and Child Health at Aga Khan University maintains a Primary Health Centre (PHC) at the site staffed by medical doctors, paramedical staff, and community health workers. In the United Republic of Tanzania, the study was undertaken on Pemba Island in Wete and Chake Chake districts, covering a population of ~ 450,000 with an annual birth rate of ~ 12,000 (data from the ongoing surveillance system of AMANHI-Pemba). The AMANHI-Pemba study group has digitized the whole island with each household numbered and geo-referenced, and therefore census of the whole island has been undertaken.
Recruitment and Consent
Children and caregivers were approached at home during a first visit by GSED-trained community health workers. Eligibility criteria included the presence of a respondent who was the biological mother, legal guardian if the mother was deceased, or the primary caregiver who spent the most time with the child. In addition, the caregiver respondent was eligible if they were over 18 years, understood the local language used in the GSED forms (i.e., Bangla, Swahili, and Urdu), and spoke to the child in the same language as translated for the forms. Last, children who were acutely ill in the previous five days were rescheduled for a later date. Standard formal consenting procedures were followed.
Sample Size and Sampling Scheme
A minimum sample of 32 caregiver-child dyads from each country site was deemed sufficient based on the joint judgment of statistical and subject matter experts regarding the amount of data needed to be collected to achieve the feasibility objectives (22) (28). A quota sampling scheme was drawn up to ensure comprehensive coverage of the target age range, stratified into eight age groups (0–2, 3–5, 6–8, 9–11, 12–17, 18–23, 24–29, 30–41 months) and balanced by sex (see Additional File 1). Although our study focused on children aged 0–3 years, we sampled children up to 41 months because older children were needed for the psychometric evaluation of the items in the main study.
Data Collection
Study Measures
The complete set of GSED measures and other contextual measures, listed in Table 1, were administered to all participants. The kit with props used in the GSED LF administration is shown in Additional File 2.
Table 1
Summary of GSED and other contextual measures used in the feasibility study
Construct
|
What the Measure Captures
|
Measure
|
Administration Mode
|
Average Administration time (minutes)
|
Child neurological development
|
Cognitive, motor, language, and social-emotional development
|
GSED SF(18)
|
Caregiver Report
|
15–25
|
GSED LF (18)
|
Child Assessment
|
30–75
|
Child Behavioural and regulatory function
|
Indication of early precursors of nonnormative behaviours and regulatory issues
|
GSED PF
|
Caregiver Report
|
20
|
Child health and household socioeconomic status (SES)
|
• Eligibility (exclusion -criteria)
• Demographic information
• Information about acute child health
• Delivery and Perinatal conditions
• Child’s health history
• Maternal health/chronic illness
|
Eligibility and Household Form (Specifically developed for the study)
|
Caregiver Report
|
35
|
Child anthropometry
|
• Weight
• Infant Length/Child Height
• Child’s Mid-upper arm circumference
• Child’s head circumference
|
Anthropometry Form (according to WHO Child Growth standards.) (29)
|
Child Assessment
|
15
|
Family environment
|
• Home Environment (HOME only)
• Play/Stimulation/Interactions between the child and other family members in the home (HOME)
|
Home Observation for Measurement of the Environment Inventory (HOME) (30)
|
Caregiver report & Observation
|
45
|
• Child neglect/abuse
• Exposure to violence or conflict
|
Childhood Psychosocial Adversity Scale (CPAS)(31) †
|
Caregiver Report
|
15
|
• Family resilience
|
Brief Resilience Scale (BRS) (32) †
|
Caregiver Report
|
1
|
• Family social support
|
Family Support Scale (FSS) (33) †
|
Caregiver Report
|
5
|
Caregiver health and wellbeing
|
• Caregiver Depressive Symptoms
|
The Patient Health Questionnaire- 9 (PHQ-9) (34)
|
Caregiver Report
|
5
|
† These measures have been minimally adapted for the study
|
GSED App
The data were collected via a newly created tablet-based GSED Application (App) developed by the Center for Public Health Kinetics Global (United Republic of Tanzania) in collaboration with the social enterprise company Universal Doctor (www.universaldoctor.com). The GSED App is built on a core Open Data Kit (ODK) platform (available at: http://Getodk.org/), a free and open-source software platform for off-grid electronic data collection and management in resource-constrained environments. The data collection version v.1.25 of the ODK Collect App was adapted and customized for the GSED project. In addition to the overall appearance and appearance, the App incorporated a grid-based interface for the GSED LF to aid administration. Additionally, the GSED App provided other utility tools, such as a timer and information button, which facilitated the long-form administration by displaying administrative guidelines and images for each item in the grid-based user interface. ODK aggregate with MySQL 5.7 community edition was used as the aggregator at the back end. The data were collected on Android-based tablets with a 10-inch screen for better visibility and user interface. A screenshot of the App's home page is given in Figs. 1a, and the GSED LF grid is shown in Figs. 1b.
Feasibility Outcomes
The methods for addressing each feasibility objective are detailed below. The feasibility of the implementation processes is addressed in section 1, and the acceptability of the processes and measures is explained in more detail in section 2. It should be noted that only one FGD was held with each country team at the end of the study to collect feedback on the feasibility and acceptability of the processes described.
1. Assessing the feasibility of the implementation process:
a) Fidelity of translation and adaptation processes of GSED and other measures
Translation was needed for all the GSED measures (LF, SF, and PF) and other contextual measures described in Table 1. The forms were translated from English to Bangla, Urdu, and Swahili for Bangladesh, Pakistan, and the United Republic of Tanzania, respectively. A standardized translation and back translation process was carried out in each country. First, the forms were translated from English to the local language by two independent local professional translators recruited by the study managers at each site (35). Second, each translation was reviewed by the local study teams to reach a consensus on the wording. Third, the agreed-upon local language versions were back-translated by two separate independent translators into English, and back translations were then compared with the original English version. Finally, the back translations underwent an iterative review and revision process by the WHO team and SMEs, identifying and revising items where the meaning had altered from the original before being finalized and approved for data collection (36). For the PHQ9 and HOME, local translations were already available, so they were only back-translated once and then reviewed and approved. Eligibility forms also went through a single round of translation and back translation, as they were brief questions with direct and easy meaning.
Further feedback from assessors regarding clarity and perceived comprehensibility for caregivers was obtained via the structured FGD at the end of the feasibility study.
b) Refining the training processes
The feasibility study was used to test and refine the training processes and packages that had been developed for the validation study. An in-person Training of Trainers (ToT) event for supervisors of all three country teams was conducted for one week in the United Republic of Tanzania, led by a team from the WHO and SMEs from various international universities and institutions with sizable experience in developmental psychology, pediatrics, early childhood development, and psychometrics and measure creation. The training involved i) theoretical sessions about child development principles and measurement, ii) a detailed review of study procedures, and iii) an item-by-item review of the GSED measures and other measures used in the study. This was followed by live demonstrations of best-practice GSED implementation by SMEs and practice sessions that gave further explanations for the "difficult-to-administer" items. Training participants also played a role under supervision to ensure that they understood the administration of items correctly. Draft standard operating procedures (SOPs) for study implementation were developed during the ToT event. The SOPs outlined processes for approaching eligible households, seeking informed consent, administering the measures, and data collection and management, along with item guides and manuals for the GSED measures.
The site supervisors who were participants in this training then served as local "master trainers" who trained their respective country team assessors. To train the assessors at each study site, the site supervisors designed a two-week training program in consultation with the WHO team. The training and certification process included the following:
-
Pre- and post training quizzes helped keep participants focused on the set objectives. In addition, post training quizzes were part of the certification process.
-
Each assessor was needed to perform three administrations of the GSED SF, LF, PF, CPAS, HOME, PHQ9, BRS, and FSS on children aged 1) less than six months, 2) 7–18 months, and 3) 19–36 months. The supervisors simultaneously scored assessments. To be approved to collect data for the GSED study, field assessors were needed to complete a certification process that involved achieving an agreement of 90% on the forms’ scoring between the assessor and the local supervisor.
-
For certification of anthropometric measurements of head circumference, mid-upper arm circumference, length, height, and weight, assessors were trained on standardized procedures (37). Each country site already had master trainers trained by anthropometry specialists. They served as “gold standard” assessors during training. For inter-rater and intra-rater agreement, assessors and trainees were needed to take anthropometric measurements on ten children in two rounds. Their measurements were checked for intra-rater agreement (precision), and against the measurements, the gold standard assessor took for inter-rater agreement (accuracy). Differences in measurements falling within the defined margins of error (MOE) were considered acceptable. The MOE for length, height and head circumference was ± 0.5 cm, and the mid-upper arm circumference was ± 0.2 cm. Additional rounds of standardization were implemented for those who did not pass the initial round.
The FGDs held with assessors and supervisors at the end of the feasibility study elicited their feedback on the training sessions. They were asked i) if they thought the training objectives were met, ii) whether any modifications were needed, and iii) what challenges they faced during data collection.
c) Trialing visit scheduling and administration processes
One of the essential objectives of the feasibility study was to trial and devise the most practical way of scheduling visits to administer all the study measures. Due to the large number of measures to be administered, the schedule was divided into two visits to minimize the burden on the families. In all three sites, the first visit was performed at home. In the United Republic of Tanzania and Pakistan, the second visit was performed in a mobile clinic or clinic setting. In Bangladesh, it was performed at home due to the absence of clinic or center facilities. The visit schedule is shown in Table 2. Within each visit, half of the children/caregivers (Group 1) received the GSED PF cognitive testing (see section 2a for details) and GSED PF exit interview, and half (Group 2) received the GSED LF exit interview and comprehensive exit interview. In addition, at the Bangladesh site, the feasibility sample was divided into two subgroups to assess the feasibility of having one or two study visits to see if conducting all the assessments in one day was feasible. The risk of conducting the assessments over two days was that caregivers might not return to the clinic the next day with their child. However, the risk of conducting the assessments in one day was that the caregivers and children would feel overburdened and become too restless or tired.
We conducted exit interviews to gather feedback from caregivers about their experience. We asked them about the length of the visits, whether they found it to be a major disruption to their routines, how well the study teams maintained confidentiality and privacy, and the order in which questionnaires were asked. Feedback from assessors was collected regarding the overall challenges they faced during the scheduling of visits and administration of the measures during the FGD administration.
Table 2
Summary of Visit Schedules
Group 1
|
Group 2
|
1st Visit at home
|
Same for both groups
|
• Eligibility and consent form
|
Same for both groups
|
• Eligibility and consent form
|
Household information
|
• Household information
|
• GSED Short form (SF) (audio-recorded)
|
• GSED Short form (SF) (audio-recorded)
|
• GSED Psychosocial Form (PF) (audio-recorded)
|
• GSED Psychosocial Form (PF) (audio-recorded)
|
• HOME Inventory tool
|
• HOME Inventory tool
|
• Anthropometric assessment
|
• Anthropometric assessment
|
Qualitative data collection only for group 1
|
• GSED Psychosocial form (PF) Cognitive Testing (audio-recorded and notes on paper)
|
No qualitative data collection during visit 1 for mothers in
group 2
|
• GSED Psychosocial Form (PF) Exit Interview (audio-recorded and notes on paper)
|
2nd Visit at center/clinic (within 48 hours of visit 1)
|
Same for both groups
|
• GSED Long form(LF) (video recorded)
|
Same for both groups
|
• GSED Long form (LF) (video recorded)
|
• CPAS
|
• CPAS
|
• PHQ9
|
• PHQ9
|
• Family support & resilience
|
• Family support & resilience
|
No qualitative data collection during visit 2 for mothers in group 1
|
Qualitative data collection for group 2
|
• GSED Long Form Exit (LF) Interview [Immediately After GSED LF] (audio-recorded and notes on paper)
|
• Comprehensive Visit Exit Interview [At the end of all testing] (audio- recorded and notes on paper)
|
d) Assessing the robustness of the data management systems
Data were checked for completeness, accuracy, and quality by manually monitoring the data collection process at the end of each day. Data were collected on tablets and extracted to CSV format for each data collection form. These CSV files were then merged using pre-written software and shared with the WHO in a password-locked folder by each country's data manager for analysis purposes.
e) Comparing "in-person" inter-rater reliability assessment with "video-based"
A further objective of the feasibility study was to evaluate two methods to assess the inter-rater reliability for the GSED measures to be implemented in the main validation study. The first method consisted of an assessor administering the measure while recording a video (for the GSED LF using a camera fixed on a tripod) or audio (for the GSED SF and PF). The videos and audio were then independently assessed and scored by other assessors. The second method consisted of an independent supervisor (acting as master rater) in-person scoring live assessments simultaneously with the primary assessor.
2. To evaluate the acceptability of:
a) The GSED measures and the supplementary battery of contextual measures to be administered
A further objective of the GSED feasibility study was to establish the overall acceptability of the GSED measures in terms of item appropriateness to context and comprehensibility. Feedback was sought from i) caregivers (n = 16 per country) whose feedback regarding cultural acceptability and comprehensibility of GSED measures was critical via exit interviews, ii) field-site supervisors and assessors from the three countries involved in operationalizing each step of the study process via FGDs conducted at the end of the feasibility study, and iii) a subsample of caregivers reviewing 9 problematic items in the newly created GSED PF via cognitive interviews. Table 3 summarizes the data collected.
Table 3
Summary of qualitative data collection
Qualitative Measures
|
Tool assessed
|
Administered to
|
Intent
|
Exit Interviews
|
GSED PF, GSED LF, and overall for all other study measures
|
Caregivers
|
To understand acceptability, the ease of administration, workflows and visit schedules, and respondent comprehension for the battery of measures used in the GSED validation
|
Focus group discussions (FGDs)
|
All
|
Site supervisors and assessors
|
Feedback on the experience of various aspects of the study:
• consenting process
• ease of administration of the forms
• feedback on visit schedules
• use of GSED App
• training needs
• comprehension of the items
• familiarity of objects in the GSED LF kit
|
Cognitive testing of GSED PF
|
GSED PF
|
Caregivers
|
Evaluation of how the caregiver understood the items to construct his or her answers
|
The FGDs helped understand the viewpoints of both caregivers and assessors within each country, which were fed back by the supervisors and assessors. Table 4 lists the prompts given in the FGDs.
Table 4
Topics and examples of prompts during the FGD sessions held with assessors
Domains
|
Few examples
|
Consenting Process
|
• Can you tell us about the consenting process?
|
• Did parents have follow-up questions?
|
Overall experience of administration
|
• What is your overall experience with administering the forms to the parents?
|
• Were there specific forms/questions they struggled/hesitated with?
|
• For you, what was the most difficult form to administer?
|
• How do you feel the flow of the form administration went- that is how the forms are sequenced
|
• Did you feel the length of the interviews was a challenge for the respondents or the child?
|
GSED LF
|
• How did parents respond to the administration of the GSED LF?
|
• Were there any activities that the parents did not understand?
|
• Were there any activities that seemed to make the parents feel uncomfortable?
|
• Were there any test-related equipment materials or pictures that were difficult to use?
|
Training
|
• Please share your training experience with us.
|
• Did you feel you had enough practice with the children/respondents prior to administration.
|
• Did your training include videos of administrations – if yes- was this helpful?
|
• Did the training include any reliability assessment?
|
• What training activity did you enjoy the most?
|
App
|
• Please share your experience with the App use
|
• Please share the challenges that you faced during the App use.
|
• Did you have any concerns with the App distracting the rapport with respondents and children?
|
• Which forms were easiest to use with the App?
|
• Which forms were challenging to use and what were the challenges?
|
• What changes do you suggest to improve the App?
|
Video recording
|
• What was your experience with the video – recording?
|
• Did you feel that it was disruptive to the process of form administration?
|
• Please share your challenges and concerns with the video recording
|
The caregiver exit interviews comprised semi-structured questions about i) the GSED LF, ii) the GSED PF, and iii) the overall administration experience at the end of the second visit. As the GSED LF was directly administered to a child, it was important to know how easy or difficult this interaction was for the families. Hence, a question asked during the GSED LF exit interview was, "Was there anything during the administration of the tests with your child that you did not feel comfortable with?”. Another question asked during the comprehensive caregiver exit interview was, "Did you feel uncomfortable with any of the questions or how any of the questions were asked?”. The GSED SF was not included specifically in this part of the work as it was very similar both in content and methodology to the Infant and Young Child Development (IYCD) (38) and Caregiver Reported Early Developmental Instrument (CREDI) (39) where these exercises with caregivers have already been carried out and thus it was deemed as conveying unnecessary burden on caregivers. An example of an exit interview is given in Additional File 3.
The GSED PF was a newly created measure comprising 62 items. In preliminary field work, 9 items (see Table 5) had been identified with unusual response patterns, and we took the opportunity to refine and retest these items in this study. Caregiver feedback was gathered while administering the form through cognitive testing. ‘Think-aloud’ techniques were used to improve the instrument's reliability by ensuring that the meanings of the items were clear to respondents and matched the conceptual framework of the instrument developers (40). The method consisted of administering open-ended questions about the items on the measure to the caregiver and asking them to 1) rephrase or explain the items and 2) explain what the items would look like in their child. eliciting their interpretation and understanding of them (41). The question asked for each item was “Can you tell me in your own words what you think this question is asking OR describe what you picture when you think of this behavior?”. These two questions aimed at eliciting an explanation of what the caregiver interpreted and whether any rephrasing, restructuring, or cultural adaptations were needed
Table 5 Subset of 9 items from GSED PF used in cognitive testing
b. Using a tablet-based GSED App for administration
Following the development of the GSED App, web-based training sessions were held to train country supervisors and assessors on its usage which led to the setup of a system of data transfer to the server and cloud storage for each site. Challenges in developing the GSED App, web-based training, and setting up the data management system will be discussed in detail in a separate paper.
Data Analysis
Information about cultural acceptability and comprehensibility of GSED measures was gathered from the exit interviews and cognitive interviews in parallel as the administration of the GSED measures progressed. The country specific FGDs were conducted after data collection had been completed. The qualitative data were compiled and synthesized with Dedoose, an online tool for examining qualitative data (42). It allowed researchers to identify themes and extract excerpts from the FGDs as well as compile quantitative data about how participants responded (e.g., number of comments made that included a certain response or theme, such as feeling that some materials were unfamiliar or suited for older children). The Yes and No responses received from exit interviews are summarized using counts and percentages.
After the data analysis had been completed, the feedback and lessons learned were shared at a virtual technical meeting between the WHO coordinators, SMEs, and country teams to discuss whether further revision of the measures and the overall administration processes was needed before the main validation study began.