The Sentiment of a Virtual Rock Concert

doi:10.21203/rs.3.rs-1361876/v1

Download PDF

Research Article

The Sentiment of a Virtual Rock Concert

https://doi.org/10.21203/rs.3.rs-1361876/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

We created a virtual reality version of a 1983 performance by Dire Straits. To understand the responses of participants we carried out two studies which used sentiment analysis of texts written by the participants. Study 1 (n = 25) (Beacco et al., 2021, Disturbance and Plausibility in a Virtual Rock Concert, IEEE Virtual Reality) had the unexpected finding that negative sentiment was caused by the virtual audience, where e.g., some were fearful of being harassed by them. In Study 2 (n = 26) notwithstanding some changes, the audience again led to negative sentiment – e.g., a feeling of being stared at. For Study 2 we compared sentiment with questionnaire scores, finding that the illusion of being at the concert was associated with positive sentiment for males but negative for females. Overall, we found sentiment was dominated by responses to the audience rather than the band. We relate our findings to the concept of Plausibility, the illusion that events and situations in the VR are real. We conclude with the need for co-design of VR scenarios, and the use of sentiment analysis in this process, rather than sole reliance on concepts proposed by researchers which may not reflect the experiences of participants.

Virtual reality

sentiment analysis

evaluation

concert

performance

plausibility

presence

The traditional paradigm for the evaluation of a new method, application or scenario in virtual reality (VR) is to carry out an experimental study, with response variables as answers on Likert scale questionnaires and possibly some behavioural or physiological measures. The point is to understand how the responses vary between different levels of the factors of the experiment. For example, one of the earliest ever VR studies was concerned with transfer of training in a physical manipulation task from VR to real world performance (Kozak, Hancock, Arthur, & Chrysler, 1993) and found that VR training offered no advantage compared to a waiting group (in the particular system used at that time). More generally, a long-standing theme in the evaluation of VR experiences has been the concept of presence (the feeling of ‘being there’) in the place depicted by the VR (Sheridan, 1992). Since this is a unique affordance of VR the achievement of high presence has been thought to be a fundamental goal of VR experiences. A stream of studies started in the early 1990s that analysed different factors that may contribute to presence, for example, (Slater, Usoh, & Steed, 1995) examined how walking-in-place compared to point-and-click methods of moving through an environment influenced presence, and (Barfield & Hendrix, 1995) examined the impact of display update rate.

Presence is usually evaluated by questionnaire (Lessiter, Freeman, Keogh, & Davidoff, 2001; Usoh, Catena, Arman, & Slater, 2000; Witmer & Singer, 1998), physiological responses (Meehan, Insko, Whitton, & Brooks Jr, 2002), breaks in presence (Slater & Steed, 2000) or psychophysical approaches where factors can be varied in real time in order to find their optimal balance (Llobera et al., 2021; Slater, Spanlang, & Corominas, 2010).

Research on presence is reviewed in (Sanchez-Vives & Slater, 2005; Skarbez, Brooks Jr, & Whitton, 2018) with a meta-analysis concerned with factors found to influence presence in (Cummings & Bailenson, 2016). However, the concept of presence has evolved and has been deconstructed into two orthogonal components (Slater, 2009). Place Illusion (PI) refers to the illusion that participants have of being in the place depicted by the VR displays, even though they know that this is not true. The root of this is that perception be based on the extent to which natural sensorimotor contingencies (O'Regan & Noë, 2001a, 2001b) are afforded by the VR system. This refers to using the whole body for perception (e.g., head turns, looking around and underneath objects, turning the whole body, eye movements) resulting in the same changes in sensory input as in reality. For example, a stereo wide field-of-view head mounted display with head 6 degrees of freedom head tracking meets many of the requirements for natural sensorimotor contingencies for vision, and in the case of spatialised sound, for audition too. The second component of presence is referred to as Plausibility (Psi). This is the illusion that the events that are perceived to be happening in the VR are really happening, even though this is known not to be true. Psi depends on (i) events in the VR responding to the actions of the participant (for example, a virtual character looks back when looked at), (ii) events that spontaneously refer to the participant (e.g., a virtual character contingently looks at the participant and smiles), (iii) that where the VR depicts events or a situation that participants are quite familiar with in reality, that their expectations are met. Requirement (iii) is often difficult to satisfy since it requires detailed domain knowledge by the application designers, and is complex in itself. For example, participants might well accept a VR with strange creatures or where normal physical laws are not obeyed - for example, in the case of 3D chess in VR where chess pieces fly through the air of their own accord (Slater, Linakis, Usoh, & Kooper, 1996) – but not accept a situation where some detail fails to meet expectations – for example, in our work on violence between soccer fans in a bar, our first rendition of the bar was rejected by participants on the grounds that a bar decorated in that way would never be visited by soccer fans (Rovira, Swapp, Spanlang, & Slater, 2009). Medical doctors experienced less Psi in an interview with virtual patients because they were unable to look up patient details on a virtual computer display that was on their virtual desk (Pan et al., 2016). Plausibility is probably the more difficult (and interesting) illusion to generate and has been increasingly studied, for example recently (Hofer, Hartmann, Eden, Ratan, & Hahn, 2020) examined the relationship between PI and Psi with results suggesting their independence, (Galvan Debarba, Chague, & Charbonnier, 2020) studied the impact of different levels of body tracking on Psi using the psychophysics methodology of (Slater et al., 2010), the impact of virtual human character behaviour and other factors on Psi were considered in (Bergström, Azevedo, Papiotis, Saldanha, & Slater, 2017; Skarbez, Neyret, Brooks, Slater, & Whitton, 2017).

The standard experimental paradigm and methods of measurement are appropriate when there are specific hypotheses in mind, or when we know what relationships we are interested in investigating. For example, whether spatialised sound is likely to result in greater scores on a presence questionnaire (Poeschl, Wall, & Doering, 2013), or to examine how display latency influences presence (Meehan, Razzaque, Whitton, & Brooks Jr, 2003). However, in the case of a novel application where there is no or little prior knowledge about how participants may respond, or what factors may be important, this paradigm may be uninformative or even misleading. A questionnaire score can mask critical information.

We embarked on a new research field concerned with the recreation of historical rock concerts in VR. On the technical side the idea was to employ computer vision techniques (Beacco, Gallego, & Slater, 2020) to extract the appearance and movements of the band players in 3D, to use agent based models and crowd rendering to reconstruct a virtual audience, and to place the players and audience in a model of the theatre in which the concert took place. The sound from the original video at the basis of the reconstruction was used for the audio. Our scientific interest has been to explore how people would respond to the virtual concert – would they reject it because of the inevitable lack of realism? Would they join in dancing along with the audience? How much would they feel as if they were at an actual concert? and so on. Which factors might contribute to or detract from these? The particular performance on which we have focussed is from the 1983 Alchemy concert by Dire Straits playing ‘Sultans of Swing’ at the Hammersmith Odeon in London. In our pilot study (Beacco, Oliva, Cabreira, Gallego, & Slater, 2021) 25 participants were recruited online and from an overseas University class. The scenario involved a recreation of the Hammersmith Odeon, the players on stage modelled partially from a video of the live performance, and a virtual audience that surrounded the participant. The audience had a number of realistic male avatars standing in the immediate vicinity of the participant, which were created from photographs of men using our computer vision techniques so that they looked like actual people, and further away from the participant the audience members were standard graphics-based avatars, and further away still were impostors. The audience moved with the music though the dancing animations that were taken from online repositories of animations.

Since our questions were very open, exploring this quite new area of application, instead of questionnaires and behavioural measures, we used sentiment analysis. Sentiment analysis (Bakshi, Kaur, Kaur, & Kaur, 2016; Liu, 2012) relies on prior classifications of millions of words in dictionaries which have been assigned positive or negative valence, we discuss this further below. Pieces of text then obtain a score, for example, as the average score over all the relevant words in the text. The major response variable was derived from a sentiment analysis of short essays that participants were asked to write immediately after their experience. This analysis led to the quite unexpected result that the virtual audience was more impactful than the actual performance by the band. In particular, some participants felt vulnerable, and alone amongst the audience, had a feeling of being stared at by audience members (even though this was not programmed to occur), and especially women felt that they would be on the receiving end of unwelcome approaches from the surrounding men. Such feelings were classified as ‘disturbing’. However, this also signified a high degree of Plausibility of the experience, since a prerequisite of feeling disturbed is that the events in question must be experienced as really happening, an automatic response, not a belief. High disturbance was associated with low sentiment scores. A second contributor to lower sentiment scores was a failure of expectations – examples being the band not interacting with the audience, or the drummer not visually beating in time to the sound of the drums. On the positive side, higher sentiment scores were associated with a feeling of immersion in the concert, people joining in with the dancing of those around them, spatial audio from the band, and the movement of the crowd around.

We refer to this as Study 1. An important overall conclusion from the results of Study 1 was that a standard VR experimental design, with a questionnaire asked after the experience with Likert scale questions, would never have picked up on these deeper findings about the responses of participants. For example, Plausibility may have scored highly, but the underlying disturbance associated with this, might never have been discovered.

Here we report the results of a second experiment (Study 2) of exposure to the VR concert with a number of changes:

To further examine the impact of the audience the virtual audience members were all depicted as female. This deliberately went to the other extreme compared to the first study. Would this lessen the chance of disturbance, especially as earlier reported amongst women participants?

All audience members visible to the participant were generated with the software Character Creator 3, being more pleasant and realistic than the earlier ones.

The movements of the audience members, depicted as dancing along with the music, were based on motion capture of a few individuals actually dancing in rhythm to the same music.

There were various small improvements to the portrayal and movement of the band members.

Methods for the two studies are described in the next section, including a description of the sentiment analysis used. We then present results for Study 2, and then combine the data from both Studies and analyse those together. Conclusions about sentiment analysis, the concert scenario and the way forward are presented in the Discussion section.

2.1 The Scenario

The VR scenario was a reproduction of the complete live performance of ‘Sultans of Swing’ by Dire Straits which lasts just over 10 minutes. The participant was placed amongst a standing audience about 6 rows back from the stage. There was audience chatter to start and eventually an announcement welcoming Dire Straits. The band members ran onto the stage and took their positions, and then started to play. The crowd cheered and clapped, and did so again at various times during the performance, for example, clapping their hands above their heads, and cheering. Various aspects of the scenario are shown in Fig. 1. The video on https://youtu.be/2qdvNGjavEg shows the opening scenes.

Fig. 1 Images of the scenario (a)-(b) the band playing. (c) the audience is dancing along with the rhythm and one of the characters looks towards the participant. (d) the character to the right of the participant after looking at the participant

2.2 Procedures and Ethics

Both studies were carried out during the COVID-19 pandemic so that face-to-face laboratory studies were not possible. Instead the Qualtrics software[1] was used for both Studies 1 and 2. This is survey tool accessed through a web interface. Participants were sent a link which opened to the information sheet, ethics consent form, instructions including installation of the software, demographic questions such as age, prior experience with VR, instructions on how to run the experience in the head-mounted display, and follow-up questions. Hence, participants could participate in their own time and place.

When a participant had completed the experiment, the experimenters could access their responses from the Qualtrics pages. All responses were anonymous and IP addresses were not transmitted.

The studies were approved by the Bioethics Committee of the University of Barcelona, IRB00003099. All participants gave written and informed consent.

2.3 Recruitment

Study 1 is described in detail in (Beacco et al., 2021). There were two groups of participants, 15 were recruited through an advertisement on social media, and 10 were students from a class in a United States University, which was not a technical computer science class. Of the 25 participants 17 identified as female and the remainder as male.

For Study 2, 17 were recruited from amongst students of various University classes (in Australia, New Zealand, and the United States), and a further 9 from advertisements on social media. 19 identified as male and 7 as female amongst those who finally participated. 65 individuals started the Qualtrics survey and 26 completed it. All of those who did not complete it stopped at the point that they were asked to download the application and upload it to their head-mounted display.

Full demographic information for Studies 1 and 2 is shown in Supplementary Table S1.

Participants were not paid for their participation.

2.4 Equipment

All 25 of the participants in Study 1 used the Oculus Quest 1 head-mounted display supplied by their University. For Study 2 participants used either an Oculus Quest 1 (5), an Oculus Quest 2 (16), or the Pico NEO (5). The breakdown is given in Supplementary Table S1.

2.5 Experimental Design

For Study 1 all participants experienced the same environment, as described earlier. After the VR experience participants returned to the Qualtrics page, and answered the question labelled as ‘Essay’ in Table 1.

For Study 2 there was one binary factor that we refer to as Gaze with levels (NoLookAt = 0, n=15) and (LookAt=1, n=11). Participants were allocated to these randomly by Qualtrics (across all 65 initial respondents). For those in the LookAt condition if they looked towards a nearby audience member the corresponding avatar had a high probability of returning their gaze with a smiling face. The return look lasted 1-3 seconds at random. For those in the NoLookAt condition the gaze behaviour of the participant had no effect on the surrounding virtual audience members.

After the VR experience participants continued with the Qualtrics, and the questions are shown in Table 1.

Table 1 – Post experience questions

Variable Name	Question
Essay	Please now write your answer below paying attention to the following points: The feeling to be at a concert. Your movements along with the crowd (e.g., dancing, clapping, cheering). Aspects that drew you into the experience. Aspects that drew you out of the experience. Anything else you want to comment on. (It would be helpful if your answer could be at least 150 words excluding copying the phrases above).
	After completing the essay question the Qualtrics moved to a separate block with 3 questions each on a 1-7 Likert scale, 1 = ‘Not at all’, 4 = ‘Sometimes’, 7 = ‘Almost all the time’.
copresence	How much, if at all, did you feel to be amongst a crowd of people?
dancing	How much, if at all, were you swaying or dancing along with the audience?
concert	Thinking back on your experience how much was this like attending a concert?

2.6 Implementation

Details of the major aspects of the implementation can be found in (Beacco et al., 2021). The following changes were made for Study 2:

Improved colouring of the crowd – i.e., of the clothing
Crowd animations were replaced by new mocap recordings that were properly synchronized with the music.
Animations of the band members improved with a broader set of motions. We avoided penetration of guitars with skin meshes by using physics.
Close crowd behaviour: when staring at some agents for a certain short amount of seconds, they would look back at the participant for between 1-3 s.
Added emulated dynamic lighting. By having two different baked lightmaps with different configurations (lights turned on and lights turned off), we could swap between them to simulate changes in the environment lightning.
And there is another point that we should add, but I am not 100% sure they were in the executed version of the experiment.
Random facial micro gestures on close crowd characters to give a more natural and organic look.

2.7 Sentiment Analysis

We used 4 different sentiment analysis methods available through R. Different systems use different dictionaries and various criteria. For example, the R package sentimentr (Rinker, 2021) uses 9 dictionaries and aims particularly at ‘valence shifters’ i.e., modifiers where ‘I do not like it’ is correctly recognized as negative and ‘I really like it’ is an enhanced positive valence. Rinker (2021)[2] includes a comparative evaluation of several sentiment analysis packages.

The VADER system (Hutto & Gilbert, 2014) was designed for the analysis of social media text but also is used more generally. Here we use the R implementation[3] by Katherine Roehrick.

The syuzhet package (Jockers, 2017)[4] includes 4 sentiment lexicons and was originally designed for analysis of the latent structure in narrative, although it has been used widely for other applications.

SentimentAnalysis[5] is the fourth R package that we use (Feuerriegel & Proellochs, 2019; Feuerriegel, Proellochs, & Feuerriegel, 2018) which exploits 3 different dictionaries which has applications in the analysis of financial text, but again has been used more widely.

There are many other sentiment analysis packages available, and comparisons are discussed in (Naldi, 2019; Yoon et al., 2017). Rather than choose one particular package or complete an analysis using all of them and compare results, here we obtain results from all 4 packages mentioned above and treat them together. We obtain an matrix, where is the number of texts and the 4 columns are the sentiment scores for the 4 packages. Then cluster analysis is used to find subsets of the texts that have similar scores. Keywords and commonalities within each subset can be obtained, in order to identify the major themes that emerge in response to the concert experience.

We first consider only the texts produced in Study 2, and consider them per individual participant ( n = 26. In this way we can also compare the sentiment analysis scores with the questionnaire responses. We then move on to analyse the combined results of Study 1 and Study 2 at the sentence level (n = 611 sentences).

2.8 Data Availability

All data and programs for analysis are available on https://www.kaggle.com/melslater/analysis-of-concert-data.

[1] www.qualtrics.com

[2] https://github.com/trinker/sentimentr#comparing-sentimentr-syuzhet-meanr-and-stanford

[3] https://CRAN.R-project.org/package=vader

[4] https://github.com/mjockers/syuzhet

[5] https://CRAN.R-project.org/package=SentimentAnalysis

3.1 Observations from Study 2

The overall impression from reading the essays written by participants in Study 2 is that once again their evaluations of the scenario were largely based on their responses to the audience. Those in the Gaze-LookAt condition were bothered by the virtual audience members staring back whenever they looked towards them, for example:

… whenever I turned around the people closest to me stopped watching the concert and turned their attention toward me. I may have been imagining it, but it was none the less, a little creepy.

When I turned in the direction of someone they’d stare at me until I turned back, which was a little unsettling.

“Every time I looked at the woman to my right and the woman to my left a little behind me, they turn to look at me and stare for the amount of time I would expect a friend to stare. Because they are strangers to me, I would not expect them to acknowledge my glances or to only look over with their eyes for a second. Turning their bodies and looking at me for such a long time made me feel uncomfortable.”

A second issue was that the audience consisted only of women:

It was a bit weird that I was the only guy among an all women audience. Made it feel a bit like some sort of fantasy experience.

I personally felt a bit paralyzed watching the other concertgoers, most if not all who were beautiful women.

Furthermore, I felt very out of place as the only man between exclusively women.

Furthermore, it was very strange to be in an audience with all women. At the concerts I find myself going to, I am usually one of the very few women that makes it this close to the front of the stage.

A third major issue was failure of expectation, which of course was also associated with the fact of there being a mainly female audience. Other examples include:

The clapping did not sound right to me. It sounded as if it were also coming out of the guitar amps in the venue like the music. I would have expected it to sound much louder and closer to me.

The crowd stood farther apart than expected.

At no time did I really have the feeling of being at a real concert. Starting with the band, I missed the interaction with the audience as well as within the band itself.

The crowd seemed too far apart - like it was a socially distanced gig, which drew me out.

The fourth major issue was simply technical issues or glitches:

The models [of the band] also seemed to slide in a strange way when they walked sideways.

Overall it was a very entertaining experience, although it was clear that it is not a real concert, due to repetitive models, scaling issues, clashes of players on stage.

The restricted movement and graphics of the musicians and audience drew me out of the experience.

… the band was not actually playing in time to the music in a realistic way. The drummer was not playing on beat. The guitarist’s fingers were moving more like a bass player than like a guitarist.

They did not seem to move much and their movements seemed clunky.

The lead singer’s arm was clipping through his jacket sleeve, which seriously disrupted the immersiveness of the experience.

On the positive side the spatialized sound was frequently mentioned as an important aspect of the experience.

The spatial sound helped make it feel more real as well, along with crowd movements and noises.

The sound was quite good. As I moved my head around it felt like it was localised to the stage so that helped me feel more like I was there.” “Directionally, it felt like the sound was coming from in front of me and I was in the audience, which gave me an experiential concert feel and got me clapping along with the virtual audience.

Also it is noteworthy that exactly the same scenario was experienced quite differently by different participants. For example,

While the genre of music hits my taste, I found the experience unconvincing, lacking creativity, emotionless and unengaging.

I felt like I was at an unfulfilling virtual replica of a concert.

On the other hand:

This was a really cool experience! I immediately started dancing as if I were at the performance.

Overall, it was a very nice and interesting experience. I really felt that I am there in some way flexing with the musicians and the crowd. I was really about to clap at the end! And it was great to be at least at some kind of concert after several years of covid.

3.2 Sentiment Analysis for Study 2

Sentiment analyses were carried out for the complete texts of each the 26 participants. For the 4 packages positive sentiment scores indicate positive sentiment, negative scores negative sentiment, and scores around 0 indicate no sentiment expressed.

Table 2

– Statistics for the Sentiment Scores for Study 2 by Sentiment Method (n = 26)
Package	VarName	Min	Max	Mean	SD
sentimentr	$sr$	-0.059	0.232	0.088	0.074
vader	$sv$	0.064	0.993	0.764	0.260
syuzhet	$sz$	-1.950	10.250	4.025	3.050
SentimentAnalysis	$sa$	-0.097	0.219	0.052	0.075

Table 2 shows the range, means and standard deviations of the 4 sentiment scores, and Fig. 2 the distributions. The vader method stands out as having a different distribution from the other 3. Table 3 shows the correlations between the scores of the different methods. While the correlation between sentimentr ($sr$) and vader ($sv$) is low, the others are positively correlated. Low correlation is useful since it means that the different variables are not responding to the same textual properties in the same way.

Figure 2 Histograms of sentiment scores for the 4 methods of Table 1 for Study 2. (a) sentimenr. (b) Vader. (b) syuzhet. (d) SentimentAnalysis

Table 3

– Pearson correlations between the sentiment scores (with significance levels), n = 26
	$sv$	$sz$	$sa$
$sr$	0.27 (0.187)	0.51 (0.008)	0.43 (0.027)
$sv$	1	0.41 (0.036)	0.46 (0.018)
$sz$		1	0.53 (0.005)

Kmeans clustering (using kmeans in R) was used to cluster the resulting scores. 4 clusters resulted in the greatest separation between the clusters, more than 4 resulted in considerable overlap. An elegant way to show the clusters is to find the principal components (PCs) of the 4 sentiment scores, and plot the scores per individual of the first two PCs. The R package factoextra⁶ (Kassambara, 2017; Kassambara & Mundt, 2017) includes the function fviz_cluster that achieves this. Figure 3 shows the clusters plotted on the first two PCs. The first PC accounts for 58% of the total variance and the second 19%.

Figure 3 The first two principal components (Dim1 and Dim2) of the $26\times 4$ matrix of sentiment scores with the clusters shown by the convex hulls of their corresponding points. The clusters contain 7, 5, 7 and 7 in the order cluster 1 to cluster 4 respectively

Table 4 shows the loadings for the PCs. The first PC is approximately proportional to the sum of the 4 sentiment scores (all loadings are 0.5 to one decimal place). ${PC}_{2}$ is proportional to the difference between the sentimentr score and the vader score. ${PC}_{1}$ and ${PC}_{2}$ account for almost 79% of the total variance, and we do not consider the remaining PCs further.

Table 4

– Loadings for the principal components. The $j=\text{1,2},\text{3,4}$ PC score for individual $i=\text{1,2},\dots ,n\left(=26\right)$ is the row vector $({sr}_{i},{sv}_{i},{sz}_{i},{sa}_{i})$ multiplied by the column vector ${PC}_{j}$. The brackets show the proportion of variance explained by the corresponding PC.
	${PC}_{1}$ (58%)	${PC}_{2}$ (19%)	${PC}_{3}$ (12%)	${PC}_{4}$ (11%)
$sr$	0.47	0.64	-0.46	0.39
$sv$	0.45	-0.74	-0.50	0.08
$sz$	0.54	0.17	0.10	-0.82
$sa$	0.53	-0.12	0.73	0.41

Table 5 shows the correlations between the PCs and the sentiment variables. It can be seen that ${PC}_{1}$ is strongly positively correlated with all 4 variables. ${PC}_{2}$ is strongly positively correlated with $sr$, negatively with $sv$ and not with the remaining two PCs. Figure 4 shows the means and standard errors of the PC scores by the clusters.

Figure 4 Mean and standard errors of the PC scores by the clusters (a) ${PC}_{1}$ (b)${PC}_{2}$

Table 5

– Correlations between the first two principal components and the sentiment scores (significance levels)
	$sr$	$sv$	$sz$	$sa$
${PC}_{1}$	0.72 (0.00003)	0.69 (0.0001)	0.82 (0.0000003)	0.81 (0.0000006)
${PC}_{2}$	0.55 (0.003)	-0.64 (0.0005)	0.15 (0.5)	-0.11 (0.6)

Using this information, we can interpret Fig. 3. The first dimension ${PC}_{1}$ is positively associated with all sentiment scores. Clusters 2, 3 and 4 are in the order of increasing levels of sentiment on the first dimension. Cluster 1, however, is low on the first dimension and higher on the second dimension. This indicates ambivalence in the responses: ‘one the one hand …[something positive]’, and ‘on the other hand … [something negative]’.

We can summarise the text associated with the 4 clusters using a keyword extraction technique and sentence summaries. We use the udpipe package (Straka, Hajic, & Straková, 2016; Straka & Straková, 2017)⁷and in particular dependency parsing, which finds nominal subjects and the adjectives that describe them. See also⁸. We use the lexRank (Erkan & Radev, 2004)⁹ method to summarise the text in the clusters. The results are shown in Table 5.

Each cluster contains a mix of positive and negative sentiment, following Fig. 3. For the clusters 2, 3, 4 even though they show increasing sentiment for ${PC}_{1}$ they are generally on the low side of ${PC}_{2}$. However, from 2 to 4 the proportion of positive statements increases. Cluster 1 is low on ${PC}_{1}$ indicating that it is low across all 4 sentiment scores, whereas it is high on ${PC}_{2}$ indicating a low score on vader. Correspondingly most of the comments show negative sentiment.

Table 5

– Keyword pairs, the context of those keyworks, and summary
Keyword pairs	Context	Summary
Cluster 1
effective one	As far as virtual experiences go, this one was pretty effective at reproducing the concert experience.	[1] "Part of the magic of a concert experience is the collective effervesance of the crowd that the music draws and I felt that this was missing from the experience." [2] "Knowing that this was a Dire Straits concert, I would assume that the crowd would be more diverse." [3] "As a female one of my favorite things about going to a concert is looking around at the crowd during the show and taking fashion inspiration from the women around me." [4] "The music and crowd motion drew me into the experience." [5] "At the concerts I find myself going to, I am usually one of the very few women that makes it this close to the front of the stage."
weird drummer	The drummer and pianist running across the stage while hunched over was a bit weird as well, but did not impact my overall experience to a large degree.
evident concert	The environment was well designed and the feeling of being at a real concert was evident during certain moments.
long song	I felt that I was really at the concert. However I am not a huge fan of Dire Straits so the song was very long.
diverse crowd	Knowing that this was a Dire Straits concert, I would assume that the crowd would be more diverse.
female audience	I was drawn out of the presence by the fact that the audience was all female and not very varied in race and ethnicity.
generic clothing	I found that the avatars clothing and style was quite generic and modest in style in comparision to what I would typically see at a show.
Cluster 2
female audience	The audience was all female, and they seemed to be looking at me a lot of the time, also disconcerting. Interestingly, it took me until about halfway through the song to realize that the entire audience was female.	[1] "Sometimes I decided to move a bit along with the crowd, dancing, waving arms..." [2] "Avatars would start clapping just a little bit after the audio of the crowd started cheering, which as pretty eerie, and when the crowd in the audio was clapping to the beat, avatars were clapping off of the beat, which destroyed that cool synchronous collective feeling you get being in a crowd at a concert." [3] "The crowd was mostly realistic, but the things that broke immersion were the timing on clapping and the limited number of avatars." [4] "At times it felt a bit like being part of a crowd." [5] "I did sometimes have the feeling of being at a concert, especially when the crowd started cheering, but I am doing this in the afternoon, while I am a little tired and not exactly in a concert-going mood."
female crowd	The crowd stood farther apart than expected. It was unusual that the crowd was only female.
high motivation	The motivation to dance and clap along was rather high
disconcerting performer	It was hard to feel immersed in the experience, the music felt real, but the avatar performers and audience were more disconcerting than immersive.
realistic crowd	The crowd was mostly realistic, but the things that broke immersion were the timing on clapping and the limited number of avatars.
repetitive movement	Movements were repetitive and animations imprecise.
Cluster 3
great sound	It definitely felt like I was at a concert, the sounds were great at drawing me into the experience. The sound traveling from one ear to the other when I changed direction was great.	[1] "I wouldn’t say that it felt like I was alone but I would much prefer to be completely alone without any of the audience in that concert." [2] "It definitely felt like I was at a concert, the sounds were great at drawing me into the experience." [3] "The audiences near me on the other hand are very disturbing." [4] "The timing of screaming and clapping sound of the audiences did not match with their animation either." [5] "I noticed myself swaying along with the crowd at some points during the concert."
nice music	The live music was nice.
effective audience	The audiences cheering was also effective.
good effect	No spatial audio effect is not good for the VR experience I think.
mandatory audio	Spatial audio should be mandatory for this kind of application since people can move their bodies and head around.
immersive sound	Upon beginning the experience, I noticed two things: the sounds being very immersive, but the human models taking away from some of this feeling of immersion.
believable movement	On the other hand, the movements of the crowd were mostly believable.
great direction	The sound traveling from one ear to the other when I changed direction was great.
slight sense	The sense of being at concert was slight.
impactful audio	In particular, the high fidelity audio of a live recording was impactful in that way [the sense of being at a concert].
distract use	The guys on stage didn’t look realistic and that pulled me out. Their use of the instruments was distracting.
weird thumb	Didn’t look like they were playing guitar and the thumbs especially were weird.
ok clapping	The clapping in the audience was ok if you didn’t look too close.
fun seconds	First few seconds was fun. Hearing the band starting the song and adressing the audience, hearing and seeing the audience response and move for the first time are really exciting.
gone immersion	[continuation of previous entry] But as soon as I looked around, my immersion was instantly gone because the audiences around me just stopped reacting to the music and looked at me instead
strange animation	Some audiences animations were really strange.
decent venue	The venue and lighting were decent in my opinion.
disturbing audience	The audiences near me on the other hand are very disturbing.
Cluster 4
irresistible movement	When the audience put their hands in the air I also felt compelled to do the same. The movements were quite irresistible.	[1] "So it didn’t feel close to real in that sense but it still felt like I experience something." [2] "Not having hands made me feel a bit less in the experience." [3] "Made it feel a bit like some sort of fantasy experience." [4] "Having the crowd was also fun as I felt that I somehow have shared the experience with others." [5] "The crowd moved a lot more naturally than a lot of other NPC based experiences so that helped me get in to the groove more."
good sound	The sound was quite good. As I moved my head around it felt like it was localised to the stage so that helped me feel more like I was there.
excellent quality	One thing I will say is that the recorded sound quality of the performance was excellent and I felt in the Quest 2 headset that it did simulate a live sound experience.
stationary hand	They should have been moving their arms as if they had a pick in their hand. Both the guitarists and bassist’s left hand were mostly stationary, while they should have been sliding up and down the neck of their instruments according what notes they were playing at a given time.
quantifiable programmer	Again, I am not a computer programmer at all, but a lot of musical data would be easily digitized (stuff like tempo, meter, pitch, etc is all very quantifiable) and if you could get an application like Logic/Protools/Garageband to play the song in midi, perhaps it would be possible to send that midi-data into the animation software to get the Avatars to move in time with the song?
enough stage	The stage was not large enough and could have been better dressed with front monitors and lighting, amps, steps, etc.
female crowd	The crowd were exclusively female, which would be highly unlikely for a Dire Straits concert – and none of them were drinking (also, if this was in the 1980s, people would have been smoking).
friendly face	However, there were a girl on the right of me that was literally staring at me when I looked at her. And her face was not friendly at all. At some point she even started to move closer and dancing in a very uncomfortable proximity from me. But having an eye-contact with a third girl who had a really friendly face was super nice!
nice move	And the clapping thing of the audience as well as some synchronized moves was nice and inviting.
distracting creeps	The creeps from the audience and the standing-too-close girl were very distracting.
important audience	But the audience was sometimes more important that the band itself in this experience."
unnerving fact	Also, the fact that they turn to look at me every time I glance at them was unnerving.

An important point throughout is that overwhelmingly negative comments relate to the audience not to the band itself:

Lack of diversity
Incorrect clothing
Lack of synchrony between sounds and observed movements of the audience
Audience members looking at the participant inappropriately.

On the other hand the audience was also effective in pulling people into the experience, so that they would find themselves dancing along with the audience.

Sound was important in generating positive sentiment, in particular the spatialised sound (though one participant did not perceive this). With respect to the band, incorrect movement while playing the guitar generated negative sentiment.

3.3 Analysis of the questionnaire scores of Study 2

Here we consider the scores for the questions shown in Table 1. In particular we were interested in whether the Gaze condition had any influence on the variables concert, copresence, or participants dancing along with the audience (dancing).

Figure 5 shows the box plots. Overall the levels of response were low in the case of concert, and approximately symmetrically distributed around the median in the other two cases. In terms of the effect of the Gaze condition there appears to be no or little difference between the NoLookAt and LookAt conditions (formal analysis confirms this).

Figure 5 Box plots of the questionnaire responses (a) Overall. (b) By the Gaze condition. The thick horizontal lines are the medians, the boxes are the interquartile ranges (IQR), the whiskers extend from max(min value, lower quartile – 1.5*IQR) to min(max value, upper quartile + 1.5*IQR)

Next we consider whether there is any relationship between the sentiment scores and the questionnaire variables. The three questionnaire response variables are highly correlated. Spearman’s rho between concert and copresence is $\rho =0.66$ (P = 0.002), between concert and dancing $\rho =0.57$ (P = 0.0025) and between copresence and dancing $\rho =0.50$ (P = 0.009). Therefore we only consider concert since these are not 3 independent variables.

The questionnaire score concert is highly correlated with one of the sentiment scores $sr$ ($\rho =0.53$, P = 0.006) but not with any of the others. The variable concert is also positively correlated with ${PC}_{2}$ ($\rho =0.47$ (P = 0.016), but not with ${PC}_{1}$. There is also an influence of gender.

Figure 6 shows the scatter plots of concert on the PCs by gender. Figure 6a suggests that for ${PC}_{1}$ for males there is a positive correlation between concert and sentiment whereas for females negative. The same is shown for ${PC}_{2}$ in Fig. 6b. (The same results can be found if $sr$ is used instead of the PCs). The interpretation is that for males there is a positive association between being at the concert and positive sentiment. However, for females, a high sense of being at the concert is associated with lower sentiment. As noted in the analysis above the feeling of being stared at, for example, causing negative sentiment, only makes sense if there is an illusion of actually being at the concert, i.e., the events are interpreted as if really occurring. (There are very similar graphs for copresence and to a lesser extent for dancing). The result indicates a difference in response between males and females.

Figure 6 Scatter plot of concert on PCs by gender (a) ${PC}_{1}$ (b)${PC}_{2}$

A Bayesian logistic regression was carried out for each of the three questionnaire response variables concert, copresence, and dancing, in one overall model that includes all three. For each the linear predictor is of the form:

Gender + PC1 + PC1⋅Gender + PC2 + PC2⋅Gender

where Gender (Male = 0, Female = 1) is the main effect of gender, PC1 is the main effect of ${PC}_{1}$, and PC1×Gender is the interaction term, and similarly for PC2. Formally the linear predictor of the model for the $i$th individual is:

$${\beta }_{0}+ {\beta }_{1}{gender}_{i}+{\beta }_{2}{PC}_{1,i}+{\beta }_{3}\left({gender}_{i}\bullet {PC}_{1i}\right)+{\beta }_{4}{PC}_{2i}+{\beta }_{5}\left({gender}_{i}\bullet {PC}_{2i}\right)$$

where the ${\beta }_{j}$ are replaced by the coefficients indicated in the first column of Table 6 for the three different response variables. The prior distribution for each coefficient was taken as normal(mean = 0, SD = 10), so that the prior 95% credible intervals are − 20 to 20. The logistic model also requires cut points, and the prior distributions were similarly assigned normal(mean = 0, SD = 10). These are weakly informative prior distributions (Gelman, Jakulin, Pittau, & Su, 2008; Lemoine, 2019), meaning that they are proper probability distributions, but with high variance representing high uncertainty.

Table 6

– Summaries of the posterior distributions showing for each parameter the mean of the distribution, the standard deviation, the 95% credible interval. Prob > 0 is the posterior probability that the parameter > 0.
Parameter	Coefficient of:	Mean	SD	2.5%	97.5%	Prob > 0
concert
${\beta }_{con,0}$		-2.44	3.85	-10.02	5.08	0.263
${\beta }_{con,1}$	Gender	0.60	0.95	-1.24	2.48	0.738
${\beta }_{con,2}$	PC1	1.08	0.38	0.37	1.88	0.999
${\beta }_{con,3}$	PC1⋅Gender	-1.55	0.70	-2.99	-0.27	0.009
${\beta }_{con,4}$	PC2	1.91	0.61	0.78	3.16	1.000
${\beta }_{con,5}$	PC2⋅Gender	-4.12	1.61	-7.42	-1.08	0.005
copresence
${\beta }_{cop,0}$		-0.09	2.76	-5.36	5.36	0.478
${\beta }_{cop,1}$	Gender	0.87	0.92	-0.91	2.73	0.829
${\beta }_{cop,2}$	PC1	0.60	0.33	-0.04	1.26	0.969
${\beta }_{cop,3}$	PC1⋅Gender	-0.86	0.60	-2.01	0.30	0.075
${\beta }_{cop,4}$	PC2	1.01	0.49	0.08	2.00	0.982
${\beta }_{cop,5}$	PC2⋅Gender	0.02	1.43	-2.74	2.89	0.504
dancing
${\beta }_{dan,0}$		-0.30	9.91	-19.91	19.65	0.488
${\beta }_{dan,1}$	Gender	1.10	0.87	-0.57	2.82	0.896
${\beta }_{dan,2}$	PC1	0.40	0.33	-0.25	1.07	0.891
${\beta }_{dan,3}$	PC1⋅Gender	-0.65	0.60	-1.80	0.53	0.135
${\beta }_{dan,4}$	PC2	0.40	0.48	-0.53	1.35	0.794
${\beta }_{dan,5}$	PC2⋅Gender	-0.64	1.39	-3.43	2.02	0.330

The results are summarised in Table 6. The entries with high posterior probabilities of being either positive or negative (i.e., do not include or only marginally include 0 in their 95% credible intervals) are highlighted.

Regarding the concert response variable, the evidence is very strong that higher values of PC1 or PC2 are associated with lower values in the case of females, and higher values in the case of males (i.e., when Gender = 0).

In the case of copresence, PC1 has a negative association in the case of females, and positive for males. There is no interaction of gender with PC2, but PC2 is positively associated with copresence independently of gender.

There are weaker associations in the case of dancing. The interaction terms do not contribute to the model fit, but there is some evidence that females are associated with greater dancing scores, but there is no effect for males. Also PC1 and PC2 are positively associated with dancing, irrespective of gender.

From analysis of the questionnaire data there is no evidence that the Gaze condition influenced these scores, but there is quite strong evidence of an association with the sentiment scores, moderated by gender.

These results should be treated with caution because there are only 7 out of 26 females. However, the Bayesian method clearly updated the high variance prior distributions of the parameters, indicated by the narrow posterior 95% credible intervals (compared with the prior intervals of -20 to 20). Thus these data were sufficient to move from a probability model with very wide variances to one with low variances.

3.4 Analysis at the Sentence Level (Studies 1 and 2 combined)

Above we only analysed the results of Study 2, treating the entire essay written by each person as the text for the sentiment score. As we have seen, each essay may contain both positive and negative sentiment, and the single overall score will reflect that with positive and negative sentiment cancelling each other. Here we treat all the sentences of Study 1 and Study 2 together, and create sentiment scores at the sentence level. There are n = 611 sentences altogether. We follow a similar strategy of obtaining scores on the same variables $sr$, $sv$, $sz$ and $sa$, and then find clusters amongst these with the help of the principal components.

Table 7

– Statistics for the Sentiment Scores for all sentences by Sentiment Method (n = 611)
Package	VarName	Min	Max	Mean	SD
sentimentr	$sr$	-1.736	1.292	0.083	0.293
vader	$sv$	-0.818	0.947	0.157	0.357
syuzhet	$sz$	-2.350	4.350	0.425	0.813
SentimentAnalysis	$sa$	-1.000	1.000	0.073	0.218

Table 7 shows the statistics for the 4 methods, and Fig. 7 the corresponding histograms. The vader method ($sv$) stands out as a different distribution. Table 8 shows that all scores are highly positively correlated with each other.

Figure 7 Histograms of sentiment scores for the 4 methods of Table 7 for all sentences combined. (a) sentimenr. (b) Vader. (c) syuzhet. (d) SentimentAnalysis

Table 8

– Pearson correlations between the sentiment scores over all sentences, n = 611. All significance levels are 0.
	$sv$	$sz$	$sa$
$sr$	0.509	0.579	0.492
$sv$	1	0.608	0.479
$sz$		1	0.505

Next we find the principal components with the loadings shown in Table 9. Note that ${PC}_{1}$ is reversed compared to the sentiment scores (i.e., higher values correspond to lower sentiment), and is approximately proportional to the (negative) sum of the 4 sentiment scores. ${PC}_{2}$ is approximately $sv+sz-sa$. Since the first two PCs account for 79% of the variance we do not consider the remaining two.

Table 9

– Loadings for the principal components. The $j=\text{1,2},\text{3,4}$ PC score for sentences $i=\text{1,2},\dots ,n\left(=611\right)$ is the row vector $({sr}_{i},{sv}_{i},{sz}_{i},{sa}_{i})$ multiplied by the column vector ${PC}_{j}$. The brackets show the proportion of variance explained by the corresponding PC.
	${PC}_{1}$ (65%)	${PC}_{2}$ (14%)	${PC}_{3}$ (12%)	${PC}_{4}$ (9%)
$sr$	-0.50	0.06	-0.80	-0.32
$sv$	-0.50	0.42	0.55	-0.52
$sz$	-0.52	0.31	0.03	0.79
$sa$	-0.47	-0.85	0.22	0.01

Table 10 shows the correlations between the PCs and the sentiment variables. ${PC}_{1}$ is highly negatively correlated with all scores, and ${PC}_{2}$ is positively correlated with $sv$ and $sz$ and negatively with $sa$.

Table 10

– Correlations between the first two principal components and the sentiment scores over all sentences (n = 611). All significance levels are effectively 0 except between ${PC}_{2}$ and $sr$ which is 0.25.
	$sr$	$sv$	$sz$	$sa$
${PC}_{1}$	-0.80	-0.81	-0.84	-0.76
${PC}_{2}$	0.05	0.31	0.23	-0.63

Figure 8 The first two principal components (Dim1 and Dim2) of the $611\times 4$ matrix of sentiment scores over all sentences with the clusters shown by the convex hulls of their corresponding points. The clusters contain 183, 334, and 94 in the order cluster 1 to cluster 3 respectively

The number of clusters keeping clear separation between them is 3 (more than this and the clusters overlap considerably). The clusters are shown in Fig. 8. Recall that the ordering inverted with 1 representing the highest sentiment and 3 the lowest. Table 11 shows the means and standard errors of the PCs. For ${PC}_{1}$ as expected there are clear differences between the means ranging from low to high in the order from 1 to 3. However, for ${PC}_{2}$ there is not much difference between the clusters, which is also clear from Fig. 8 since the clusters are not well distinguished on the second dimension. ${PC}_{1}$ being approximately the sum of the 4 sentiment scores (though with sign reversed) is the more useful dimension.

Table 11

– Means and Standard Errors of the PCs by cluster
	Mean${PC}_{1}$	SE${PC}_{1}$	Mean${PC}_{2}$	SE${PC}_{2}$
Cluster 1	-1.901	0.065	0.105	0.069
Cluster 2	0.352	0.031	-0.062	0.031
Cluster 3	2.452	0.086	0.015	0.085

Figure 9 Boxplots of sentiment by cluster for ${PC}_{1}$ and${PC}_{2}$

Figure 9 shows the clear separation in sentiment scores represented by ${PC}_{1}$ and that ${PC}_{2}$ has little variation across the clusters, though with more low value outliers in cluster 1, and more high value outliers in cluster 3.

Figure 10 Word clouds of keyword pairs showing nominal subjects and the adjectives that describe them, by cluster. All frequencies are 1 except for those with the lighter colour

Figure 10 shows the keyword pairs, i.e., nominal subjects and their corresponding adjectives. The purpose of this is just to get a quick overview of the sentence types in the different clusters. It is clear that cluster 1 has the highest sentiments and 3 the lowest, with 2 in-between.

Table 12 shows the summary sentences for each cluster. Cluster 1 (with the highest sentiment) concentrates almost wholly on the positive impact of the sound and the audience behaviour. Where the band is mentioned (sentence [14]) it is the only negative aspect.

Considering cluster 3 (with the lowest sentiment) three aspects emerge: the audience members staring at the participant and ‘creepy’ eye contact, inappropriate movements of the crowd, the fact that the crowd was women only. Again almost all sentences refer to the crowd, with one referring to the band ([7]).

Table 12

– Summaries of the text in the clusters using lexRank, showing the top 15 sentences in each case. Sentences are classified as either relevant to PI, or Psi factors (i), (ii), (iii). A – sign indicates a contribution to negative sentiment.
Cluster 1 (n = 183)
[1] "The sound was good and felt like real live music at times." (Psi (iii)) [2] "It definitely felt like I was at a concert, the sounds were great at drawing me into the experience." [3] "I felt included in the concert crowd, watching them dancing would encourage me to move and dance as well but after a while, the movements were repetitive so I felt a bit more self-aware." [4] "I didn’t clap my hand because I just didn’t feel like in concert even though the sound quality and song sounded like in the actual concert." [5] "Having the crowd was also fun as I felt that I somehow have shared the experience with others." [6] "So it didn’t feel close to real in that sense but it still felt like I experience something." [7] "The spatial sound actually makes it feel like at an actual concert, the audiences really helps this experience to be very interactive." (PI) [8] "I enjoyed the movement around me, being surrounded by people dancing was good, although not realistic/appropriate for the performance or particularly in time with the music." (Psi -(iii)) [9] "The spatial sound helped make it feel more real as well, along with crowd movements and noises." (PI) [10] "It felt nice to be in a crowd of people dancing and I felt compelled to join in." [11] "The set up is so familiar - a crowd, darkness, a well lit stage, the music, the fact that if you turn your head, the sound is directional, and the identifiable members of the band - all drew me, making it feel more like a real experience." (PI, Psi (iii)) [12] "When the people around me started dancing I really felt like dancing as well, especially during the parts when they were all cheering and clapping." [13] "Lastly, towards the end of the performance, I clapped and made sounds (woohoo, etc.) but because I could not hear anyone else making sounds it felt weird: it would have been awesome if there was more noise from the crowd like a real concert." (Psi -(iii)) [14] "The band performance did not feel real so I wasn't so engaged with the music as I normally would do in a concert, but would be keen to see and actual live concert through VR..." [15] "I enjoyed being able to move around the crowd."
Cluster 2 (n = 334)
[1] "I was feeling the need to move closer to the stage during the whole experience (as I would do in a real concert)." (Psi –(iii)) [2] "I did not really feel like moving at all mainly because the band wasn't moving, the music was not loud enough, the people around me didn't move too much and also didn't move like the crowd in the back." (Psi -(iii)) [3] "Movements: I moved as if I was at a real concert." [4] "The crowd did not move the way people usually move in a concert and usually they sing along, there is always some crowd noise." (Psi -(iii)) [5] "The sound music, the lighting and the crowd really drew me into the experience." [6] "Feeling the crowd dancing around me drew me in." [7] "The band themselves drew me out a bit- they didn't move much while they played." (Psi -(iii)) [8] "The audience really drew me into the experience." [9] "Not having hands made me feel a bit less in the experience." (Psi -(iii)) [10] "The music and crowd motion drew me into the experience." [11] "Otherwise, I thought it was a very immersive experience and I felt like I was in a crowded concert!" [12] "At no time did I really have the feeling of being at a real concert." [13] "One aspect that drew me out was there were times in the audio that it sounded like the audience was clapping but when I looked around, the audience was not visually clapping." (Psi -(iii)) [14] "The whole illusion felt like a real concert venue- the lighting, the crowd (but with more room to move than in real life)." [15] "It was nice to be a part of concert but the movement of the crowd was a bit odd, some of the people only moved when I looked in their direction." (Psi -(iii))
Cluster 3 (n = 94)
[1] "It was hard to feel immersed in the experience, the music felt real, but the avatar performers and audience were more disconcerting than immersive the audience was all female, and they seemed to be looking at me a lot of the time, also disconcerting." (Psi –(ii) –(iii)) [2] "Two very creepy looking people on my right kept staring at me the whole time and that made me uncomfortable." (Psi –(ii)) [3] "At one point the avatar to my left made a strange arm movement that broke the illusion of presence." (Psi –(iii)) [4] "As negative, I have to say that everything seemed too static: starting from my inability to move, but also the fact that the crowd didn't move and that there were no groups amongst the crowd, just people dancing alone." (Psi –(iii)) [5] "The two people dancing on the left and right of me definitely drew me into the experience, but also made me quite uncomfortable." [6] "Turning their bodies and looking at me for such a long time made me feel uncomfortable." (Psi –(ii)) [7] "Furthermore, the quite static and unnatural movements of the band and the audience were always a point that continuously reminded me that this is not a real concert." (Psi –(iii)) [8] "Some audiences animations were really strange." (Psi –(iii)) [9] "In many moments when I turned my head or some other movement, the image of my virtual hands appeared that were staying in strange ways, that aspect disconnected me a bit from the experience and I had to move my arms so that the image of the hands disappeared." (Psi –(iii)) [10] "I noticed that all of the crowd seemed to be woman, all of 5–6 different avatar types which was a little strange." (Psi –(iii)) [11] "The crowd was mostly realistic, but the things that broke immersion were the timing on clapping and the limited number of avatars." (Psi –(iii)) [12] "The solid eye contact of the man on my right was a little creepy, and as a woman, made me feel uncomfortable." (Psi –(ii)) [13] "When I looked around the woman to my right seemed to hold my gaze, this felt weird, I guess it drew me in, or it made me feel uncomfortable, I then looked further around and another woman looked at me, just for too long to be normal." (PI, Psi –(i)) [14] "Furthermore, it was very strange to be in an audience with all women." (Psi –(iii)) [15] "The faces were also occupying a bit of the Uncanny Valley, which made me feel a little bit uncomfortable and took me out of the experience a bit."

Cluster 2 shows a mixed set of responses, mentioning both some positive and negative aspects – the inability to move towards the stage, the movements and noises from the crowd not being appropriate, the lack of movement of the band, but the audience drawing them into the experience, and the venue, lighting and crowd drawing them into the experience.

The notations following some of the sentences approximately classify the meaning as corresponding to PI or the three factors contributing to Psi, with positive or negative sentiment. For example cluster 1 [11] references a sensorimotor contingency “… the fact that if you turn your head, the sound is directional …” which contributes to PI, and also “… the identifiable members of the band …” which conforms to expectation appropriate to a Dire Straits performance, Psi condition (iii). Cluster 3 [7] refers to “ … quite static and unnatural movements of the band and the audience …” referring also to a failure of expectations, resulting in negative sentiment.

[6] https://cran.r-project.org/web/packages/factoextra/index.html

[7] https://CRAN.R-project.org/package=udpipe

[8] http://www.bnosac.be/index.php/blog/77-an-overview-of-keyword-extraction-techniques

[9] https://www.rdocumentation.org/packages/lexRankr/versions/0.5.2/topics/lexRank

The most useful experiments are often those where the results are unexpected since this is when we can learn something new. In Study 1 we were surprised to find how significant the appearance and behaviour of the audience were in shaping the responses of some participants. In Study 2 the same happened, even though we had aimed at removing the aspects of audience appearance that had caused distress for some participants (i.e., that the audience members appeared as male) and simulated some interaction with the audience through their returning participant glances. Our focus has been mainly on aspects that resulted in low sentiment, because this was the most unexpected and striking, and also is critical for the production of later improved versions of the VR scenario. As one participant wrote positively: “The sound was good and felt like real live music at times.” This is nice to know, but it is not too informative. On the other hand “... the quite static and unnatural movements of the band and the audience were always a point that continuously reminded me that this is not a real concert.” This points to an actual improvement that later versions of the scenario need to address.

Almost all of the negative sentiment issues in the Table 12 summaries for cluster 3 are related to Plausibility. Recall the three components of Psi (i) responsiveness, (ii) contingent actions towards the participant and (iii) expectations. For example, Cluster 3 [1] refers to “the audience was all female” which is a failure of expectation, and “they seemed to be looking at me a lot of the time, also disconcerting”. This is the second aspect contributing to Psi, where from the standpoint of the participant contingent events occur that refer directly to the participant. From this statement is it not clear whether the “looking at” was in response to the participant looking at the character, in which case it would correspond to condition (i), or whether the participant thought that the character spontaneously looked. It is important to note that Psi does not necessarily result in positive sentiment. In this case a character looks at the participant (for too long) and this is experienced as uncomfortable. The very operation of Psi is itself what has resulted in negative sentiment. In fact the negative sentiment occurs because there is Psi, the illusion that the events are really happening – that someone was looking at the participant. If participants would not have the illusion of actually being stared at then there would be nothing to be concerned about.

Previous work has shown that virtual characters looking towards a participant contributes to Psi. In (Bergström et al., 2017) participants were located close to a virtual string quartet. Psi was greater when the players occasionally looked toward the participant. In (Steed, Pan, Watson, & Slater, 2018) participants were located on a beach with virtual characters representing refugees waiting for a boat to pick them up. It was found that Psi was greater in a condition when the characters would return glances of the participant towards them compared to a control condition. In (Llobera et al., 2021) participants were amongst a virtual crowd walking towards a theatre. Participants were more likely to choose a condition where the surrounding characters occasionally looked towards themselves rather than one where there was no such feedback. Kyriakou, Pan, and Chrysanthou (2017) found that the Psi was increased with respect to a surrounding crowd when a number of realistic crowd behaviours were introduced, including gaze. Gaze behaviour of virtual crowds is important not just when crowd members look towards the participant. Jorjafki, Sagarin, and Butail (2018) found that even if a small proportion of members of a virtual crowd exhibit gaze behaviour such as looking upwards, then participants will tend to follow their gaze. Ruhland et al. (2015) provide a comprehensive review of eye gaze in virtual characters.

Overall, evidence suggests that gaze of virtual characters towards the participant, whether imagined (Study 1) or programmed (Study 2) is likely to increase the illusion of realness of the events. However, this does not imply that there will be corresponding positive sentiment. In both Study 1 and Study 2 being ‘stared at’ was associated with negative sentiment (even though in Study 1 the virtual characters were not even programmed to look at the participants). Factors that enhance Psi do not necessarily lead to positive sentiment, and there are applications where negative sentiment is the appropriate response, for example in the various studies of bystander response to violent incidents – a recent one reported in (Rovira et al., 2021).

Generating strong Psi is difficult, far more than PI, which depends on sensorimotor contingencies associated with the display device and body tracking. In the case of Psi for a virtual event that is a simulation of real events (such as a rock music performance) meeting expectations is essential. Some of these are obvious, such as the technical capability to match up movements of virtual characters (such as clapping) with the corresponding sounds, making the movements of the drummer or guitarist match with the music, or stopping limbs intersecting bodies. Others depend on deliberate choices of the designers, such as making the audience all appear to be female, or choosing the number and spatial layout of the crowd. Other failures of expectation are more domain specific – such as the clothing and appearance of the virtual crowd not matching the 1980s, or there being no smoking amongst the audience (not fully banned in England in indoor spaces until 2007) and no drinking. To produce a scenario that is Psi-effective it is essential to incorporate as much domain knowledge as possible. This should ideally involve interviews with potential participants, to try to find out what is important to them. Often they will not consciously know this until presented with an actual example and then realise that a particular feature is wrong, or another one is missing. As mentioned earlier, in our bystander studies on soccer violence, it never occurred to us that the decoration of the bar in which the violence took place would be an important factor for Plausibility. This leads to the conclusion that co-design, where potential participants in a VR scenario should be involved from the outset in its design and evaluation, should be employed in the creation of novel scenarios. For example, (García et al., 2021) involved schizophrenia patients in the design and testing of a system for the embodiment of auditory hallucinations in VR. Dietrich et al. (2021) describe two case studies on VR applications for alcohol abuse prevention, and compare different co-design methodologies. Brassel, Power, Campbell, Brunner, and Togher (2021) provide a review of design principles for VR applications in the field of brain injury rehabilitation, and conclude that co-design is an important component.

While there has been significant use of sentiment analysis in a wide variety of domains – see (Alamoodi et al., 2021; Birjali, Kasri, & Beni-Hssane, 2021) – it has seen little use in the design and evaluation of VR applications. Fagernäs et al. (2021) point out that although there have been experimental studies with participants in the field of VR for the promotion of relaxation techniques, sentiment analysis can be used to find out what users actually think of these techniques, and point the way to improvements in applications. As mentioned earlier, when we carry out experimental studies essentially we are testing the models of the researchers involved, not necessarily what is important to participants. For decades, VR researchers (including ourselves) have concentrated heavily on presence (as ‘being there’) but it is possible that this concept is imposed on participants in experimental studies through the questions that they are required to answer (Slater, 2004). However, something more basic, and perhaps obvious than ‘being there’ is simply to follow the preferences of participants. We have started to employ this methodology in recent work – in (Murcia-López et al., 2020) participants experienced a talk from a virtual character and were able to select different options to change the characteristics of the character and aspects of the setting in real-time throughout the session to match their preferences. In (Llobera et al., 2021) the same technique was used, except that the possible changes available to participants were proposed by a reinforcement learning agent. However, in work that has used this methodology, for example recently (Fribourg, Argelaguet, Lécuyer, & Hoyet, 2020), the possible changes that can be made to the environment are fixed and chosen in advance by the researchers. We suggest that the use of sentiment analysis as a methodology to understand how participants respond to VR applications would be a useful way forward in the co-design process of building applications that are more likely to be preferred. This does not necessarily mean designing applications that result in positive sentiment, since negative sentiment, for example in psychological therapy applications, might be part of the goal. Rather the idea is to uncover features of the virtual environment that unexpectedly result in negative or inappropriate sentiment, and in the next phase of application development, overcome those problems.

The methodology adopted in this paper has illustrated the power of this type of qualitative to quantitative analysis to give deep insight into the responses of people to the virtual reality scenario. If we would have just followed traditional approaches we would have had answers to some fixed questions and Likert scale scores. We would not have known the features that enhanced or detracted from the illusion of being at a concert, only a set of scores that scratch the surface of the responses of people. We have discovered that the overwhelming reaction to the concert is not so much the performance of the band but the quality of the audience, and the relationship between the audience and the participant.

The resulting analysis has pointed out several features that we must pay attention to in subsequent versions of our applications. The participant is alone – which can be overcome by supporting several friends to simultaneously attend the concert. The characters around stare at the participant for too long – using data about glance times could help to overcome this problem. Audience members do not interact with the participant except for gaze – research into how members of a concert audience actually interact is essential. The lack of diversity of the crowd is a major issue – this was a deliberate choice for experimental reasons but is being addressed in current work. The clothing and behaviour of the virtual audience does not match the time at which the real concert took place (1983) – research on this issue would be required to properly match these expectations. The band does not interact with the audience nor the audience with each other – this is mainly due to the fact that the entire video of the ‘Sultans of Swing’ recording was not transformed into animated 3D, which would have displayed the actual interactions between members of the band. There are multiple technical problems such as the lack of sync between movements of the crowd members and the corresponding sounds, and also the musicians and the song – this requires improvements in the implementation. The audio aspect was well considered but the sounds of the crowd around need to originate from close distance rather than blended in with the sound that appears to be coming from the stage – the sound recording needs to be separated into different streams, the band and the audience sound, but also sounds from the immediate surrounding audience members need to be incorporated into the scenario. These issues all have mainly technical solutions that can be addressed in our future work.

The Likert scale was introduced in the 1930s, nearly a century ago. It is time that the incredible developments in machine learning and analysis of natural language be brought into the domain of experimental studies. The ‘meta message’ of this paper is to bring VR experimental studies into the 21st Century.

Conflict of interest. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research was funded by the European Research Council Advanced Grant MoTIVE: Moments in Time in Immersive Virtual Environments (#742989). Gizem Senel is supported by ‘la Caixa’ Foundation (ID 100010434) with Fellowship code LCF/BQ/DR19/11740007. The authors would like to thank Mr Ken Zolot, Prof. Mark Billinghurst, and Prof. Joseph K. Kearney, for help with the recruitment of participants.

Author Contributions

Design of the study, data collection and analysis were performed by Mel Slater. Design and implementation of the scenario was carried out by Alejandro Beacco, Jaime Gallego, Carlos Cabriera and Ramon Oliva. Contributions to animations were by Domna Banakou and Gizem Senel. The first draft of the manuscript was written by Mel Slater and all authors contributed to and commented on subsequent versions. All authors read and approved the final manuscript.

Alamoodi, A., Zaidan, B. B., Zaidan, A. A., Albahri, O. S., Mohammed, K., Malik, R. Q., et al. (2021). Sentiment analysis and its applications in fighting COVID-19 and infectious diseases: A systematic review. Expert Systems with Applications, 167, 114155. doi:doi.org/10.1016/j.eswa.2020.114155
Bakshi, R. K., Kaur, N., Kaur, R., & Kaur, G. (2016). Opinion mining and sentiment analysis. Paper presented at the 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).
Barfield, W., & Hendrix, C. (1995). The Effect of Update Rate on the Sense of Presence within Virtual Environments. Virtual Reality: The Journal of the Virtual Reality Society, 1, 3-16.
Beacco, A., Gallego, J., & Slater, M. (2020). Automatic 3D Character Reconstruction from Frontal and Lateral Monocular 2D RGB Views. Paper presented at the 2020 IEEE International Conference on Image Processing (ICIP).
Beacco, A., Oliva, R., Cabreira, C., Gallego, J., & Slater, M. (2021). Disturbance and Plausibility in a Virtual Rock Concert: A Pilot Study. 2021 IEEE Virtual Reality and 3D User Interfaces (VR), 538-545. doi:10.1109/VR50410.2021.00078
Bergström, I., Azevedo, S., Papiotis, P., Saldanha, N., & Slater, M. (2017). The plausibility of a string quartet performance in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 23(4), 1352-1359. doi:10.1109/TVCG.2017.2657138
Birjali, M., Kasri, M., & Beni-Hssane, A. (2021). A comprehensive survey on sentiment analysis: Approaches, challenges and trends. Knowledge-Based Systems, 226, 107134. doi:doi.org/10.1016/j.knosys.2021.107134
Brassel, S., Power, E., Campbell, A., Brunner, M., & Togher, L. (2021). Recommendations for the Design and Implementation of Virtual Reality for Acquired Brain Injury Rehabilitation: Systematic Review. Journal of medical Internet research, 23(7), e26344.
Cummings, J. J., & Bailenson, J. N. (2016). How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology, 19(2), 272-309.
Dietrich, T., Guldager, J. D., Lyk, P., Vallentin-Holbech, L., Rundle-Thiele, S., Majgaard, G., et al. (2021). Co-creating virtual reality interventions for alcohol prevention: living lab vs. co-design. Frontiers in public health, 9, 185.
Erkan, G., & Radev, D. R. (2004). Lexrank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 22, 457-479.
Fagernäs, S., Hamilton, W., Espinoza, N., Miloff, A., Carlbring, P., & Lindner, P. (2021). What do users think about Virtual Reality relaxation applications? A mixed methods study of online user reviews using natural language processing. Internet Interventions, 24, 100370.
Feuerriegel, S., & Proellochs, N. (2019). SentimentAnalysis Vignette. Massachusetts Institute of Technology. Retrieved from https://mran.microsoft.com/snapshot/2017-08-06/web/packages/SentimentAnalysis/vignettes/SentimentAnalysis.html
Feuerriegel, S., Proellochs, N., & Feuerriegel, M. S. (2018). Package ‘SentimentAnalysis’. CRAN: London, UK.
Fribourg, R., Argelaguet, F., Lécuyer, A., & Hoyet, L. (2020). Avatar and Sense of Embodiment: Studying the Relative Preference Between Appearance, Control and Point of View. IEEE Transactions on Visualization and Computer Graphics, 26(5), 2062-2072.
Galvan Debarba, H., Chague, S., & Charbonnier, C. (2020). On the Plausibility of Virtual Body Animation Features in Virtual Reality. IEEE Transactions on Visualization and Computer Graphics. doi:10.1109/TVCG.2020.3025175
García, A. S., Fernández-Sotos, P., Vicente-Querol, M. A., Sánchez-Reolid, R., Rodriguez-Jimenez, R., & Fernández-Caballero, A. (2021). Co-design of avatars to embody auditory hallucinations of patients with schizophrenia. Virtual Reality, 1-16. doi:doi.org/10.1007/s10055-021-00558-7
Gelman, A., Jakulin, A., Pittau, M. G., & Su, Y.-S. (2008). A weakly informative default prior distribution for logistic and other regression models. The Annals of Applied Statistics, 2(4), 1360-1383.
Hofer, M., Hartmann, T., Eden, A., Ratan, R., & Hahn, L. (2020). The role of plausibility in the experience of spatial presence in virtual environments. Frontiers in VIrtual Reality, 1, 2.
Hutto, C., & Gilbert, E. (2014). Vader: A parsimonious rule-based model for sentiment analysis of social media text. Paper presented at the Proceedings of the International AAAI Conference on Web and Social Media.
Jockers, M. (2017). Package ‘syuzhet’. URL: https://cran. r-project. org/web/packages/syuzhet.
Jorjafki, E. M., Sagarin, B. J., & Butail, S. (2018). Drawing power of virtual crowds. Journal of The Royal Society Interface, 15(145), 20180335.
Kassambara, A. (2017). Practical guide to principal component methods in R: PCA, M (CA), FAMD, MFA, HCPC, factoextra (Vol. 2): Sthda.
Kassambara, A., & Mundt, F. (2017). Package ‘factoextra’. Extract and visualize the results of multivariate data analyses, 76.
Kozak, J., Hancock, P. A., Arthur, E., & Chrysler, S. T. (1993). Transfer of training from virtual reality. Ergonomics, 36(7), 777-784. doi:doi.org/10.1080/00140139308967941
Kyriakou, M., Pan, X., & Chrysanthou, Y. (2017). Interaction with virtual crowd in Immersive and semi‐Immersive Virtual Reality systems. Computer Animation and Virtual Worlds, 28(5), e1729.
Lemoine, N. P. (2019). Moving beyond noninformative priors: why and how to choose weakly informative priors in Bayesian analyses. Oikos, 128(7), 912-928.
Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). A cross-media presence questionnaire: The ITC-Sense of Presence Inventory. PRESENCE: Teleoperators and Virtual Environments, 10, 282-298.
Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1), 1-167.
Llobera, J., Beacco, A., Oliva, R., Şenel, G., Banakou, D., & Slater, M. (2021). Evaluating participant responses to a virtual reality experience using reinforcement learning. R. Soc. Open Sci., 8, 210537. doi:https://doi.org/10.1098/rsos.210537
Meehan, M., Insko, B., Whitton, M., & Brooks Jr, F. P. (2002). Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics (TOG), 21(3), 645-652.
Meehan, M., Razzaque, S., Whitton, M. C., & Brooks Jr, F. P. (2003). Effect of latency on presence in stressful virtual environments. Virtual Reality, 2003. Proceedings. IEEE, 141-148.
Murcia-López, M., Collingwoode-Williams, T., Steptoe, W., Schwartz, R., Loving, T. J., & Slater, M. (2020). Evaluating Virtual Reality Experiences Through Participant Choices. Paper presented at the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).
Naldi, M. (2019). A review of sentiment computation methods with R packages. arXiv preprint arXiv:1901.08319.
O'Regan, J. K., & Noë, A. (2001a). A sensorimotor account of vision and visual consciousness. Behav Brain Sci, 24, 939-1031.
O'Regan, J. K., & Noë, A. (2001b). What it is like to see: A sensorimotor theory of perceptual experience. Synthese, 129, 79-103. doi:10.1023/A:101269922
Pan, X., Slater, M., Beacco, A., Navarro, X., Swapp, D., Hale, J., et al. (2016). The Responses of Medical General Practitioners to Unreasonable Patient Demand for Antibiotics - A study of medical ethics using immersive virtual reality PLOS ONE, 11(2), e0146837. doi:10.1371/journal.pone.0146837
Poeschl, S., Wall, K., & Doering, N. (2013). Integration of spatial sound in immersive virtual environments an experimental study on effects of spatial sound on presence. Paper presented at the IEEE Virtual Reality.
Rinker, T. W. (2021). sentimentr: Calculate Text Polarity Sentimentversion 2.9.0. Retrieved from https://github.com/trinker/sentimentr
Rovira, A., Southern, R., Swapp, D., Campbell, C., Zhang, J. J., Levine, M., et al. (2021). Bystander Affiliation Influences Intervention Behavior: A Virtual Reality Study. SAGE Open, 11(3), 21582440211040076.
Rovira, A., Swapp, D., Spanlang, B., & Slater, M. (2009). The use of virtual reality in the study of people’s responses to violent incidents. Frontiers in Behavioral Neuroscience, 3, 1-10. doi:10.3389/neuro.08.059.2009
Ruhland, K., Peters, C. E., Andrist, S., Badler, J. B., Badler, N. I., Gleicher, M., et al. (2015). A review of eye gaze in virtual agents, social robotics and hci: Behaviour generation, user interaction and perception. Paper presented at the Computer Graphics Forum.
Sanchez-Vives, M. V., & Slater, M. (2005). From Presence to Consciousness Through Virtual Reality. Nature Reviews Neuroscience, 6, 332-339.
Sheridan, T. B. (1992). Musings on Telepresence and Virtual Presence. PRESENCE: Teleoperators and Virtual Environments, 1, 120-126.
Skarbez, R., Brooks Jr, F. P., & Whitton, M. C. (2018). A survey of presence and related concepts. ACM Computing Surveys (CSUR), 50(6), 96.
Skarbez, R., Neyret, S., Brooks, F. P., Slater, M., & Whitton, M. C. (2017). A psychophysical experiment regarding components of the plausibility illusion. IEEE Transactions on Visualization and Computer Graphics, 23(4), 1369-1378.
Slater, M. (2004). How Colorful Was Your Day? Why Questionnaires Cannot Assess Presence in Virtual Environments. PRESENCE: Teleoperators and Virtual Environments, 13, 484-493. doi:10.1162/1054746041944849
Slater, M. (2009). Place Illusion and Plausibility can lead to realistic behaviour in immersive virtual environments. Philos Trans R Soc Lond, 364, 3549-3557. doi:10.3389/frobt.2016.00074
Slater, M., Linakis, V., Usoh, M., & Kooper, R. (1996). Immersion, presence, and performance in virtual environments: An experiment with tri-dimensional chess. ACM virtual reality software and technology (VRST). 163-172.
Slater, M., Spanlang, B., & Corominas, D. (2010). Simulating virtual environments within virtual environments as the basis for a psychophysics of presence. ACM Transactions on Graphics, 29, Paper: 92. doi:10.1145/1833349.1778829
Slater, M., & Steed, A. (2000). A Virtual Presence Counter. PRESENCE: Teleoperators and Virtual Environments, 9(5), 413-434.
Slater, M., Usoh, M., & Steed, A. (1995). Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans. Comput.-Hum. Interact., 2, 201-219.
Steed, A., Pan, Y., Watson, Z., & Slater, M. (2018). “We Wait”—The Impact of Character Responsiveness and Self Embodiment on Presence and Interest in an Immersive News Experience. Frontiers in Robotics and AI, 5(112). doi:10.3389/frobt.2018.00112
Straka, M., Hajic, J., & Straková, J. (2016). UDPipe: trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, pos tagging and parsing. Paper presented at the Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).
Straka, M., & Straková, J. (2017). Tokenizing, pos tagging, lemmatizing and parsing ud 2.0 with udpipe. Paper presented at the Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies.
Usoh, M., Catena, E., Arman, S., & Slater, M. (2000). Using presence questionnaires in reality. PRESENCE: Teleoperators and Virtual Environments, 9, 497-503.
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: a presence questionnaire. PRESENCE: Teleoperators and Virtual Environments, 7, 225-240.
Yoon, S., Parsons, F., Sundquist, K., Julian, J., Schwartz, J. E., Burg, M. M., et al. (2017). Comparison of different algorithms for sentiment analysis: Psychological stress notes. Studies in health technology and informatics, 245, 1292.

Supplementary.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

The Sentiment of a Virtual Rock Concert

Status:

Version 1

Abstract

Figures

1. Introduction

2. Methods

2.1 The Scenario

2.2 Procedures and Ethics

2.3 Recruitment

2.4 Equipment

2.5 Experimental Design

2.6 Implementation

2.7 Sentiment Analysis

2.8 Data Availability

3. Results

3.1 Observations from Study 2

3.2 Sentiment Analysis for Study 2

3.3 Analysis of the questionnaire scores of Study 2

3.4 Analysis at the Sentence Level (Studies 1 and 2 combined)

4. Discussion

Declarations

References

Supplementary Files

Status:

Version 1

Package	VarName	Min	Max	Mean	SD
sentimentr	\(sr\)	-0.059	0.232	0.088	0.074
vader	\(sv\)	0.064	0.993	0.764	0.260
syuzhet	\(sz\)	-1.950	10.250	4.025	3.050
SentimentAnalysis	\(sa\)	-0.097	0.219	0.052	0.075

	\(sr\)	\(sv\)	\(sz\)	\(sa\)
\({PC}_{1}\)	0.72 (0.00003)	0.69 (0.0001)	0.82 (0.0000003)	0.81 (0.0000006)
\({PC}_{2}\)	0.55 (0.003)	-0.64 (0.0005)	0.15 (0.5)	-0.11 (0.6)

Parameter	Coefficient of:	Mean	SD	2.5%	97.5%	Prob > 0
concert
\({\beta }_{con,0}\)		-2.44	3.85	-10.02	5.08	0.263
\({\beta }_{con,1}\)	Gender	0.60	0.95	-1.24	2.48	0.738
\({\beta }_{con,2}\)	PC1	1.08	0.38	0.37	1.88	0.999
\({\beta }_{con,3}\)	PC1⋅Gender	-1.55	0.70	-2.99	-0.27	0.009
\({\beta }_{con,4}\)	PC2	1.91	0.61	0.78	3.16	1.000
\({\beta }_{con,5}\)	PC2⋅Gender	-4.12	1.61	-7.42	-1.08	0.005
copresence
\({\beta }_{cop,0}\)		-0.09	2.76	-5.36	5.36	0.478
\({\beta }_{cop,1}\)	Gender	0.87	0.92	-0.91	2.73	0.829
\({\beta }_{cop,2}\)	PC1	0.60	0.33	-0.04	1.26	0.969
\({\beta }_{cop,3}\)	PC1⋅Gender	-0.86	0.60	-2.01	0.30	0.075
\({\beta }_{cop,4}\)	PC2	1.01	0.49	0.08	2.00	0.982
\({\beta }_{cop,5}\)	PC2⋅Gender	0.02	1.43	-2.74	2.89	0.504
dancing
\({\beta }_{dan,0}\)		-0.30	9.91	-19.91	19.65	0.488
\({\beta }_{dan,1}\)	Gender	1.10	0.87	-0.57	2.82	0.896
\({\beta }_{dan,2}\)	PC1	0.40	0.33	-0.25	1.07	0.891
\({\beta }_{dan,3}\)	PC1⋅Gender	-0.65	0.60	-1.80	0.53	0.135
\({\beta }_{dan,4}\)	PC2	0.40	0.48	-0.53	1.35	0.794
\({\beta }_{dan,5}\)	PC2⋅Gender	-0.64	1.39	-3.43	2.02	0.330