2.1 METHODS
2.1.1 Design
This study aims to investigate whether 1) participating in a live learning session improves learning online compared to recorded videos of either a previous interaction or of a teacher alone and ii) whether these conditions impact learning differently in adults with ASC compared to neurotypicals. To answer these questions, this study adopted a 2 (group) x3 (learning condition) x2 (time) repeated-measures design, with between- and within-subjects factors. The between-subjects factor is group (Autistic Spectrum Condition (ASC) vs neurotypical (NT)), the within-subjects factors are i) learning condition (live vs recorded of another social learning episode vs recorded of the teacher alone), and ii) time of recall (immediate vs delay quiz). Specifically, facts about 15 items were presented with two minutes per item. Five items were assigned to each condition: 1) live condition: participant participated in a live video-call when they learned in interaction with the teacher; 2) recorded-observant condition: participant was shown a pre-recorded video of the teacher presenting the learning material to a student (confederate); 3) recorded alone condition: participant was shown a pre-recorded video of the teacher alone presenting the learning material (Figure 1). Learning score (outcome measure) for each participant was obtained from a multiple-choice quiz (see Materials). Items assigned to each condition and trial order within each condition remained fixed for the whole duration of this experiment. Order of conditions were randomised across participant.
2.1.2 Materials
A selection of 15 items were selected from De Felice et al., (2021), three from the exotic food category (Rambutan, Kiwano, Cherimoya), four from the antiques category (Strigil, Porte-joupe, Scotch Hands, Chatelaine), four from the animals category (Tarsier, Axolotl, Glaucus, Anhinga) and four from the musical instruments category (Kalimba, Caxixi, Agogo, Hulusi). Each condition presented a mixture of objects from these categories, which remained fixed for all participants for this experiment, as follow: 1) live condition [Tarsier, Kalimba, Strigil, Axolotl, Rambutan]; 2) recorded-observant condition [Porte-joupe, Kiwano, Caxixi, Scotch Hands, Glaucus]; 3) recorded-alone condition [Agogo, Cherimoya, Anhinga, Chatelaine, Hulusi]. Learning was tested via the same multiple-choice quiz used in De Felice et al., (2021). Full details of item information and multiple-choice quiz are reported in Appendix Table 1.
2.1.3 Procedure
Participants recruitment
This study was approved by the UCL ethic committee. Participants were recruited via the online platform Prolific (www.prolific.co). The platform retains demographic details as well as information on any disabilities/diagnosis of users, as reported by the users at the time of account registration. Such anonymous information can be used by researchers to create adverts targeting a specific pool. Two separate adverts were published: one targeting neurotypical participants and one targeting people with Autistic Spectrum Disorder (ASC). As a further check, users who responded to our adverts were asked to confirm their diagnosis via a questionnaire on Gorilla Experiment Builder (www.gorilla.sc). To ensure that the experimenter was blind to participants’ diagnosis, recruitment was done by a researcher who was not involved in data collection.
To be eligible, all participants had to i) be fluent in English (speaking English regularly for >5 years); ii) be aged 18-65; iii) give consent to having their camera and microphone on; and iv) give consent to being recorded for the whole duration of the experiment. Participants were paid at the hourly rate of £7.50 for a total of £15 over two hours. An additional £3 were offered for those who completed a 10 min quiz a week later.
Participants who responded to our advert were asked to complete four main parts: 1) background battery (independently online, on Gorilla Experiment Builder), 2) learning session (over a video-call), 3) online learning multiple-choice quiz immediately after the learning session (independently online on Gorilla Experiment Builder), and 4) repeat the quiz a week later.
Background Battery
Users who responded to the Prolific adverts were redirected to Gorilla Experiment Builder (www.gorilla.sc) where they received instructions on the study and gave consent for participation. They then completed the Background Battery tasks. This comprises of i) Spot-the-word test, a measure of verbal fluency (Baddeley et al., 1993), ii) matrix reasoning item bank (MaRs-IB), a measure of non-verbal reasoning (Chierchia et al., 2019) and iii) Animated Triangle test, a measure of mentalising (Abell et al., 2000; White et al., 2011).
Upon completion of the Background battery task, an independent researcher sent the participant ID to the experimenter (teacher), who arranged a video-call with the participant (via Prolific chat), while remaining blind to their diagnosis.
Video-call
The experimenter greeted the participant and checked that audio and video worked properly. Participant was asked to open the zoom window in full-screen mode and chose the gallery view (i.e. everyone in the call is shown equal size, this ensured that view during live session was comparable to view during pre-recorded video watching). Participants were told that the aim of the study was to investigate how people learned online and whether this differed in people with ASC, and were asked not to disclose any personal information to the teacher, who was blind to their diagnosis. The experimenter then explained that the participant will learn some facts about exotic food, animals, antiques and rare musical instruments over three formats: in live interaction with the teacher (live condition), and through watching of pre-recorded videos showing either the teacher with a previous participant (recorded-observant condition) or the teacher alone (recorded-alone condition). They were instructed to memorize as much information as possible, as at the end of the video-call they will complete a multiple-choice quiz to test their learning. During the live condition, participants were told that they were free to ask questions and interact with the teacher. Before starting the learning sessions, participant’s pre-knowledge was tested. If any item was known, it was excluded from the analysis (but not from the learning session). Learning sessions started with either the live, recorded-alone or recorded-observant condition, in a counterbalanced and semi-randomised order. The whole call lasted approximately 40 minutes (i.e. 10 minutes per condition, with 2 min per item and five items in each condition, plus 10 minutes for instructions).
Learning quiz
Immediately after the learning session, participants were redirected to Prolific, where their IDs were included into a ‘white list’, so that a new advert was visible to them only. By replying to that advert, participants were redirected to Gorilla Experiment Builder (www.gorilla.sc), where they reported on the quality of the video call (audio and video), before completing the learning quiz. After the learning quiz, they also completed an ‘enjoyment questionnaire’ and inclusion of other in self questionnaire (Aron et al., 1992). This part lasted approximately 20 minutes and was completed by the participant independently (note that the ‘immediateness’ of the quiz was ensured by the experimenter who terminated the video-call only a few moments later participant initiated the quiz part on Gorilla Experimenter Builder).
Exactly one week after the learning sessions, participants invited through Prolific to the final stage of the experiment, and directed to Gorilla Experimenter Builder to complete the same learning quiz. Additionally, participants filled in a history questionnaire, to check for potential revision of any of the items (e.g. search on Google). This part lasted approximately 10 minutes.
2.2 RESULTS
2.2.1 Sample
53 participants took part in the study (Table 2). Participants were excluded when reporting 3 or less on a 1 (poor) to 5 (excellent) video-call quality scale (N=3), and being visibly distracted during the video-call (N=1). Of the remaining 49 participants, 46 (NASC=20, male=11, female=4, non-binary=4; NNT=26, male=11, female=15) completed the full experiment, including the one-week delay quiz (see 2.1 Design and 2.3 Procedure). We lost demographic data from one participant (ASC group) due to a technical fault.
Participant with ASC either received a diagnosis by a clinician (N=18) or were self-diagnosed (N=2). ASC and NT group did not differ on age (meanASC (sd) = 27.79 (9.22), meanNT (sd) = 29.85 (9.90), t(43)=.71, p=.48), verbal fluency (Spot the word test, meanASC (sd) = 47.63 (6.71), meanNT (sd) = 44.73 (6.23), t(43)=-1.48, p=.15) non-verbal reasoning (MaRs-IB, meanASC (sd) = 64.48 (18.57), meanNT (sd) = 61.64 (16.98), t(43)=-.52, p=.60) and mentalising test (Animated Triangle, meanASC (sd) = 9 (2.54), meanNT (sd) = 9.33 (1.92), t(43)=.38, p=.71). ASC scored significantly higher on AQ than NT (meanASC (sd) = 33.37 (6.73), meanNT (sd) = 19.19 (7.29); t(43) = -6.73, p<.0001).
2.2.2 Data pre-processing
Single trials were excluded when: i) participants reported that they knew the item; ii) internet connection dropped during the single trial but was good for the rest of the experiment; iii) the experimenter reported incorrect information about the item; iv) participants reported revising information about a given item before the delay quiz (excluded from delay performance only). Performance was calculated for each learning condition separately, as an average over the included trials (score= points collected on all trials / total points available on all included trials).
2.2.3 Analysis of Variance
An analysis of variance (ANOVA) was run to test the difference in learning performance between 2 (groups: ASC and NT) x 3 (learning conditions: live, recorded-alone and recorded-observant) x 2 (time of learning quiz: immediate and delay). Means and SD for all conditions are reported in Table 3. Results for main and interaction effects are reported in Table 4 and Figure 2.
Main effects. Findings show a main effect of time: unsurprisingly, people remembered more things straight after the learning session (mean (sd) = 4.2(.58) than a week later (Mean=3.8, sd=.73; F(1,44)=56.16, p<.001, η2=.56, large effect size; Bakeman, 2005). More interestingly, we found a main effect of learning condition (F(2,43)=3.86, p=.03, η2=.15, medium effect size; Bakeman, 2005). The pairwise comparison revealed a significant learning advantage associated with Live compared to Recorded-alone condition (MLive (sd) = 4.1 (.09); MRecorded-alone (sd) = 3.9 (.1); t(44) = .19 p = .008). No other significant difference between learning conditions were found. No main effect was found for group: in other words, people with ASC showed a NT-equivalent performance (meanASC (sd) = 4.01 (.09); meanNT (sd) = 3.99 (.1); t(44)=-.01 p=.93).
Interaction effects. No significant interaction effects where found between the main factors of interest (group, learning conditions and time). However, visualization of the data (Fig. 1.C) revealed a trend specific to ASC group: while immediate recall showed a similar pattern across conditions between groups, delayed recall dropped specifically for items learned during the Live condition for ASC group. A 2(group) x2(time) was therefore run to test the hypothesis that delay performance was significantly more affected for ASC compared to NT specifically for things learned during Live condition. Results revealed a group*time interaction effect: F(1,44)=4.88, p=.03, η2=.1): for things learned during Live condition, a week later people with ASC recalled significantly less things compared to NT (meanASC (sd) = 3.75 (.18); meanNT (sd) = 3.97 (.16)).
2.3 Conclusions from Experiment 1 & Hypothesis for Experiment 2
Experiment 1 found that for both NT and people with ASC, learning during Live session was associated with better recall both immediately after the session and one week later. In addition, it was found that the ASC group exhibits a decline in recall for items learned over Live interaction specifically, to a significantly greater extent than what was observed in the NT group.
Based on these results, a follow-up experiment was pre-registered (https://archive.org/details/osf-registrations-5pga3-v1) to confirm two main hypotheses:
- Participants from both groups will learn more from live calls (Live condition) compared to pre-recorded video calls (Recorded-alone and Recorded-observant condition).
- There will be an interaction between learning condition, group and time: while neurotypical adults will show a consistent advantage for interactive learning (Live condition) over time, the ASC group will show better immediate learning for material learnt in Live condition, and better long-term learning for materials learned from pre-recorded videos (Recorded-alone and Recorded-observant condition).
Experiment 2 consisted of two equal sub-experiments, which followed the same methods and procedure as experiment 1. Each sub-experiment was run by a different teacher.