All groups completed the first task, recognizing that each interaction with Misty’s buttons and sensors elicited a response. Some responses were verbal, and some were physical. The task was meant to function as a quick and easy introduction to interacting with Misty. Students spent between 4 and 8 minutes exploring different interactions and laughter and repeated trials indicate that they found the task motivating and purposeful as a precursor to the following tasks. The second task had larger variation. Some groups spent around 5 minutes exploring the code and tried to make Misty use different languages and some spent upwards 20 minutes. This variation was reflected in different levels of engagement in the task. Several groups wanted to spend a lot of time exploring Misty’s language affordances, some groups had problems understanding the task and some groups moved quickly though the task on their way to the next. The students were explicitly encouraged to change the code, but there was no explicit initial instruction given for what to do other than to explore what they could do. In most of the groups the test leader, however, pointed out the possibility to try to change the verbal message and the language choice. Most groups needed some initial assistance to do so, but soon learnt how to change the text. By 8 minutes of exploring the different language settings afforded, all but one of the thirteen groups also managed to discover how the change of language used in the text box messages and the outcome were connected to the change of language settings, resulting in phrases in for example Italian, Japanese and Arabic pronounced in Italian, Japanese and Arabic, respectively. Most of the groups initially tested a phrase written in Swedish, and the selection of a different language to pronounce it in. One of the groups only tested one phrase in Swedish (“hej det är Ike and Michael som programmerar dig”, names are pseudonyms, eng. Hi it’s Ike and Michael who are programming you) with several different language settings, resulting in Swedish pronounced with Arabic, Danish, Italian, Indonesian and French accents. The following transcript is from this student pair. They struggled to interpret the code and Misty’s corresponding actions. They managed to make an alteration for the head-sensor (Fig. 3) but struggled with eliciting the reaction in the altered code. The researcher leading the test tried to support the students in figuring out the problem.
Researcher: Where did you alter the code?
Michael: Ehm, here.
Researcher: What button is that programmed on?
Ike: One!
Researcher: What does it say here?
Michael: Number of activations by one.
Researcher: Yes, what else?
Ike: Hed back (spelled as it was pronounced)
Michael: Pitch one!
Researcher: Head back, what does that mean?
Michael: “Go back”?
Researcher: “Head”… what does that mean?
Michael: “Go back”?
Ike: No.. head
Michael: Lean your head back!
Michael and Ike have correctly identified where to alter the code, but it seems that they have not understood the condition to activate the clause or the meaning of the code. Prior to this interaction they tried this alteration for about 2.5 minutes with repeated prompts from the researcher to consider which button they were programming. In the transcript above, the test leader again asks which button they are currently programming but the students give random answers of what they see on the screen. “Number of activations…” can be seen in Fig. 3 and “Pitch one” is further down on the screen in code connected to another button. When the researcher points at “Head back”, which is the name of the button on Misty, Michael interprets it as “Go back” which is a reasonable translation of the whole phrase but leads to a misinterpretation in this particular context where the individual words in the code rather refer to the location of the button. However, not being able to identify “Head back” as the location, he also fails to connect it with Misty’s verbal response. The researcher proceeds by showing them how to activate the phrase by stroking the back of Misty’s head.
Michael and Ike’s failure to see the conditional aspects of the code might, thus, be attributed to difficulties in grasping the nature of the code, eventually in combination with the fact that the code is in English, in this case allowing for a phrasal interpretation. Hence, Michael and Ike seem to have difficulties with understanding the different ways of interacting with Misty, as users or as programmers. The meaning of Hi it’s Ike and Michael who are programming you seems to function as an initiative to communicate with Misty rather than writing its code. This in turn might indicate that they see no difference between interacting with Misty through the VPE or through physical touch by activating her buttons. The interactive features of the social robot might in that case encourage a misconception of the role of instructions in programming.
Michael and Ike never reach the third task during the lesson, nor do they identify the way Misty’s event or speak blocks work during their exploration. Misty’s affordances kept Michael and Ike engaged and on task for the full 25 minutes. Judging by number of trials and repetitions, 6 different languages and 25 trial runs, they found interacting and programming Misty meaningful. However, it seems as if the combination of a VPE and physical interaction was not helpful to create opportunities for learning conditionals in their case.
The fact that Misty’s VPE was mostly in English posed some challenges. The following transcript is from another group, who with the help of visual cues (Fig. 4) in the VPE figured out connections between the code, their physical interaction and Misty’s response.
Researcher: Try pressing the same button again, can you see that it lights up?
Jhurmie: Aaahhh
Researcher: And if we try another one
Jhurmie: It is that one!
Researcher: And the next one
Jhurmie: These ones are for these and those ones are for those (points at screen and on Misty)
With some encouragement from the test leader, Jhurmie and his companions connect the code with actions. As displayed in Fig. 4, when a particular section of code is activated it lights up a box in the VPE. This connection becomes gradually clearer for Jhurmie as he first responds with an Aaahhh to Misty’s action, then tries to predict the outcome of activating the next button and finally connects specific sections of the code to specific buttons. This understanding becomes apparent in their patterns of trial and error, where they thoroughly activate each section to test their alterations. Figure 5 illustrates how the group has altered the chin response to “kan du lägga din hand igen” which translates to can you put your hand there again. Compared to the previous group, Michael and Ike, the wording here indicates that they have a better grasp on the difference between altering Misty’s code in the VPE and interacting physically with Misty to elicit a response. The wording also indicates that they are well aware of the conditional that activates this response since they instruct Misty to answer correctly to the physical act of putting a hand on its chin.
Just as Michael’s group, Jhurmie’s group does not advance from task 2 but continues with making alterations for the full 25 minutes. There is, though, a difference between the groups. Whereas Michael’s group continues with a random pattern of trial and error making alterations without clear intent, Jhurmie’s group has a clear intent with each alteration and tweaks them until they work as intended. Creating opportunities for discovering the connection between code and condition seems to be key and utilizing Misty’s visual cues in the VPE might be a feasible addition to the design principle.
Several groups successfully not only tried out but also improved the verbal output of Misty in different languages by changing the spelling, thus showing that they had grasped the causative connection of their actions and Misty’s performance:
(Jhurmie’s group)
Swedish kan du lega din hand igen (6 repetitions)
Swedish kan du läga din hand igen (3 repetitions)
Swedish kan du lägga din hand igen (4 repetitions) (seemingly satisfied
(Eliza’s group)
Arabic bdi nam (2 repetitions) (”bidi 'anam“ I want to sleep), lots of
Arabic biddi anam (2 repetitions) (seemingly satisfied with the
The exercise turned into an effortless play with language to achieve a passable pronunciation of phrases in different languages. The two examples above show how the groups adhered to a trial-and-error pattern until they were satisfied. The following transcript is from Eliza’s group trying to get Misty to respond in Arabic for the first time.
Saida: How?
Eliza: So we write in Swedish and it says it in Arabic?
Researcher: No you have to write in Arabic.
Saida: How do we write Arabic letters?
Researcher: You write it in regular, Swedish letters. Try something simple like Salam
Eliza: Wait, “jag” (writing I, first person singular pronoun in Swedish), now she’s supposed to say “Anaa” (I in Arabic)
Eliza and Saida struggle with the input format of the VPE. Even though they have a clear idea of what they want Misty to do, it is not clear to them how to put it in the VPE. They wonder if Misty can translate for them, since there is no obvious way of inputting Arabic into Misty’s English oriented VPE. We tried copy pasting Arabic lettering from google translate with another group and it worked fine. However, Saida and Elizas reflections suggest that the functionality should be among Misty’s affordances. Figure 6 show their final version before moving on to other alterations. It shows a similar understanding as the one displayed by Jhurmie’s group. They formulate a correct and appropriate phrase, test it with its condition and use Misty’s perspective in writing the phrase.
Eliza and Saida move on to task 3 after trying seven different Arabic phrases and altering Misty’s facial expressions. Task 3 was to create a new conditional clause by mimicking a previous one. Figure 7 show their work in progress, with their code on the left and the code they are trying to mimic on the right.
As displayed in Fig. 7, most of the code is identical to the code in task 2. However, there are crucial differences that indicate that task 1 and 2 has created learning opportunities for understanding several of the blocks used. The action that becomes the condition for triggering the code is pressing the front right bumper rather than the front left as in the example. And when the code is complete, they proceed with altering the display image to elicit different facial expressions.
In general, the different groups seemed to enjoy repeated trials. The trials generated a lot of laughter both when the result of their programming was successful and when it was not so successful. Only one student at a time was able to do the actual programming, but there was a lot of engagement of the others, and even some competition of who was to do the writing or touching of the sensory buttons. Also, the test leader needed to interfere, sometimes repeatedly, and stop the groups solving the task, as the next group was waiting for their turn. In average, the groups spent around half an hour with Misty, and between four minutes and 45 seconds and nineteen minutes and 50 seconds on the language task. They tested from one to seven different languages with one to fourteen different phrases. There was, thus, some variation both to the way the groups solved the task and for how long time they stayed in the task. Hence, the task was motivating and purposeful and functioned as a precursor to the following tasks.
The results indicate that the affordance to play around with both language settings and the text messages offered by Misty were both appreciated and engaging. There were, however, also some challenges. Misty supports 41 language settings, but students did not always find the languages they wanted to use among these. This occasionally led to disappointment, a rather quick dismissal of the language task and a decision to move on to the next task. Also, even if a language, for example Arabic, was offered as a choice, not all students speaking Arabic write Arabic. Further, there is a discrepancy between the language of verbal and written instructions and affordances. The former are in Swedish, the latter only in English. The first author has translated the written texts to Swedish – unfortunately, though, the design team behind the robot has not (yet) been able to implement the translations. We noticed that this had negative impact on the student performance on the task, as they received little, or no support of the English written in the blocks. For example, the text Head Back, displayed in Fig. 1, was by one of the groups initially interpreted not as the location of the sensor on Misty, but rather as a command to ‘return’, which slowed down the progress.