Twenty-three participants (18 male) with autism participated in this study. The average age was 79.17 months with a standard deviation of 16.93 months. Recruitment occurred through flyers and consent forms distributed to parents within approved school districts, via special education teachers and Speech Language Pathologists. An Autism Diagnostic Scale (ADOS) was completed by a research-reliable provider to confirm diagnosis and document severity of autism symptoms. The Comparison Score on the ADOS provides a number from 1-10 indicating the level of autism spectrum-related symptoms present; a score of 1-2 indicates minimal to no evidence, 3-4 indicates low evidence, 5-7 indicates moderate evidence, and 8-10 indicates high evidence. The scores for participants ranged from 6-10, M = 7.4. At the time of the first data collection, all participants said fewer than 40 different words according to both parent and teacher report and none of the participants were putting words together into meaningful phrases. The Vineland Adaptive Behavior Scales-3 (VABS-3) was used to gather information on participants’ communication, social, daily living and motor skills. It was completed online by participants’ classroom teachers. The mean adaptive behavior composite score was 54.73, SD= 9.37.
Students were participating in a randomized clinical trial of a multimodal word learning intervention. The clinical trial was paused due to COVID 19. Data for the current study were all obtained from the intervention phase of the study.
Phonemic Feature Scoring
To better capture small gains or changes in speech production for minimally verbal participants, a phonemic feature scoring system was used. Each phoneme attempted by the participant during the probe was evaluated for accuracy as compared to the target consonant-vowel-consonant (CVC) word. For consonants, three features were measured-Place, Manner, Voice. Place indicates where along the speech system the air is constricted to make the specific sound, examples include bilabial, velar, glottal, etc. Manner indicates which articulators are used to shape the air stream, examples include stop, fricative and glide. Voice indicates if the vocal folds are used for the sound or not to produce voiced or voiceless consonants. For vowels, four features were measured-Height, Advancement, Rounding, Tenseness. Height indicates where the tongue is during articulation and can be categorized by placement of low, mid or high in the mouth. Advancement also refers to tongue placement, but from front to back of the mouth. Advancement can be categorized by tongue placement of front, back or central. Rounding refers to rounding of the lips, while Tenseness indicates whether the vowel uttered was tense, with greater vocal tract constriction, or lax, less vocal tract constriction.
Each child production was evaluated according to these features. Each correct feature was given 1 point, for a total raw score of up to 10 points per word (3 for C1, 4 for V, 3 for C2). This means that the higher the total points, the closer the approximation is to the target word.
Examples of Phonemic Scoring. In Example 1, the target CVC word is “bat.” In Trial 1, the participant produces /baek/. For C1 they would receive 1 point each for voice, place and manner for the correct production of /b/. For the vowel, they would receive 1 point each for height, advancement, rounding and tenseness for the correct production of /ae/. For C2, the participant produces /k/ instead of /t/. As a result, they are given 1 point for voice, as both /k/ and /t/ are unvoiced. They are also given 1 point for manner because both the target consonant /t/ and the uttered consonant /k/ are stops. However, the participant is given 0 points for placement because the target /t/ is an alveolar while the uttered /k/ is a velar sound. In this case, the participant would receive a total of 9 points for trial 1.
In example 2, the target word is “cake”. In trial 1 the participant produces /dae/, leaving off C2 completely. For C1 the participant receives only 1 point for manner, as both /k/ and /d/ are stops. The participant is given 0 points for voice because /d/ is voiced and /k/ is not. They also receive 0 points for place because /k/ is a velar while /d/ is an alveolar sound. The vowel /ae/ is given 1 point for advancement and 1 point for roundness, as the target /e/ and the approximation/ae/ both share these phonemic features. For C2, the participant receives no points since that sound was not attempted at all. Thus, the total score for example 2 is 3.
We applied the phonemic scoring system to child vocalizations during weekly probe sessions. For this, participants were asked to name pictures representing each of 5 target words, comprising a word set. Each picture was presented 3 times each in a random order for a total of 15 trials. Weekly probes continued for each set until the participant passed 3 of the 5 target words in 2 of 3 opportunities or 4 weeks elapsed, whichever came first. For our purposes, a score of at least 6 with the initial consonant correctly produced was considered a “passing” score for a word attempt.
Weekly probes were video-recorded during the session and later transcribed in our lab. Two trained transcriptionists, blind to observation details including the week of the probe, and whether or not the participant was receiving intervention transcribed each session. To optimize accuracy, each recording was first independently coded and then disagreements were resolved through consensus.
Scoring Reliability. Two research assistants were trained to use the phonemic scoring system until they achieved a criterion of 85% agreement with videos previously coded by the fourth author, an experienced speech language pathologist. Subsequently, the research assistants each independently applied the phonemic scoring system to the transcribed data. Twenty percent, or 1851 total probes, were coded by both research assistants. The intraclass correlation coefficient (ICC) for these scores is .92. If trials with scores of zero are removed, the ICC is .88. These coefficients indicate excellent reliability (Koo & Li, 2016).
Vineland-3: Vineland Adaptive Behavior Scale-3 (Sparrow, et al., 2016). We used the composite adaptive behavior score as a control variable for developmental differences in our analyses. The Vineland Adaptive Behavior Scales-3 is administered individually by either a professional in an interview form, a parent/caregiver, or a teacher for ages birth to 91 years. For our purposes, we had teachers fill out the form, which has a narrower focus of age 3-18 years. The form was completed online which provided information on the participant in a structured setting and allowed for more precise scoring using the online platform. The VABS-3 Adaptive Composite Score consists of skills in communication, daily living skills, socialization domains. The ABC standard scores are scaled so the mean is 100 within each age group and the standard deviation is 15. Averages of reliability for the teacher form fall in the excellent range.
To address our research questions, we examined the associations between the scores from probes obtained from the end of the first 4 weeks of intervention to later scores. Our rationale for selecting week 4 was that this amount of time allowed participants to become familiar with the probe procedures. Hence, we were looking at early learning progress as an indicator of subsequent progress. Specifically, we obtained the correlations between the following derived scores:
- Week 4 phonemic score. The total phonemic score from the probe obtained at week 4 divided by the total number of trials (i.e.,15).
- Week 4 words passed per trial. - The number of words passed during the Week 4 session, divided by the number of trials in the probe.
- Phonemic Score for Set 2. The total phonemic score for the best probe session during each word set that followed week 4 (Set 2). For example, if a participant’s best phonemic score during probes for set 2 was a 17, this score was used in our analyses.
- Words passed per trial Set 2. The number of words passed during the best probe session of set 2, divided by the number of trials in the probe. For example, if a participant met our pass criterion for 5 words on their best probe for set 2, and there were 15 trials, their score for this variable would be .33. A score of 1.0 is the maximum for this variable.
- Average words passed per week in all subsequent weeks. The total number of words meeting our pass criteria for all sets after Week 4 combined, divided by the total number of intervention weeks, after week 4.
For our research questions, we considered variables 1 and 2 above to be predictor variables and variables 3- 5 as outcomes.