Facilitating Situated Crowdsourcing of 3D Cultural Heritage via Asynchronous Virtual Collaboration

doi:10.21203/rs.3.rs-2245108/v1

Download PDF

Research Article

Facilitating Situated Crowdsourcing of 3D Cultural Heritage via Asynchronous Virtual Collaboration

https://doi.org/10.21203/rs.3.rs-2245108/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Crowdsourcing via mass photogrammetry is a viable approach for acquiring collections of 3D cultural heritage objects. The essence is to outsource onsite image acquisitions to the crowd whose contributions can be aggregated and processed into 3D models. Prior research has proven that synchronous offline collaboration can leverage positive collective behaviours. However, forming an ad-hoc team requires external operations and can hinder potential participation. Therefore, this study investigates how technology-supported collaboration can be integrated into situated crowdsourcing to reduce organisational costs, lower participation barriers, and achieve high-quality crowdsourcing performance. The goal is to develop a systematic approach to facilitate virtual collaboration where volunteers can self-select, engage, and contribute asynchronously. We proposed and implemented an operational model comprising multiple information and communication technologies. The platform was deployed and tested through field experiments, and its performance was compared with two other crowdsourcing means with different levels of organisational control. The cross-evaluations confirmed the effectiveness and efficiency of our mechanism. It stimulates increased participation and better-quality contributions than conventional open calls. Its self-selecting nature provides the necessary freedom and flexibility to promote gathering volunteers with shared interests at far less coordination than synchronous offline collaboration. Surveys of user experiences also reveal that individuals find the process more enjoyable with less effort on the users’ side. This empirical study derives concrete design implications for situated collaborative crowdsourcing, offering practical insights into volunteer-based technology use and user-generated content contribution in the digital heritage field.

Awareness

Collaborative Crowdsourcing

Computer-Supported Cooperative Work

Digital Heritage

Mass Photogrammetry

Situated Crowdsourcing

As digitisation becomes a major process in cultural heritage fields, a large-scale 3D digital heritage database can certainly bridge historical gaps and open up cultural heritage locally and globally (Ch’ng 2013; Ch’ng et al. 2019; CyArk 2021; Europeana n.d.; Sketchfab n.d.; Katz 2017; Nishanbaev 2020; The Palace Museum n.d.; Tsirliganis et al. 2004). Driven by close-range photogrammetry techniques, cultural institutions have been exploring the potential of the crowd for 3D reconstruction (Alsadik 2016; Bonacchi et al. 2019; Hartmann, Havlena, and Schindler 2016; Microsoft Live Labs n.d.; Oomen and Aroyo 2011; Photo Tourism n.d.; Ridge 2020; Ross 2013; Stathopoulou et al. 2015). General photogrammetric procedures include onsite image acquisitions and offsite multi-image processing (Gruyter et al. 2014). Whilst offsite steps can be completed quickly due to rapidly developed algorithms and computational power, the onsite capture remains laborious and time-consuming. Crowdsourcing can offload this effort-intensive digital capture work from researchers to volunteers through mass photogrammetry (Ch’ng et al. 2019; Cheng and Ch’ng 2022). Quality multi-image configuration requires holistic, multi-image, multi-angle capture at different distances and with adequate overlaps between image sequences (Ch’ng et al. 2019; Gruyter et al. 2014; Mudg et al. 2010). Since conventional crowdsourcing methods focus on encouraging isolated contributions from individuals with diverse backgrounds (Zhao and Zhu 2014), the imageries crowdsourced via conventional open calls often present problems in data quality for photogrammetry-based reconstructions.

Extensive research supports the viability of open collaboration for complex tasks, inviting contributors to team up and jointly work towards the same goal (Blohm et al. 2018; Dissanayake, Zhang, and Gu 2015; Fuger et al. 2017; Lykourentzou et al. 2016; Rahman et al. 2019; Riedl and Woolley 2017; Wang 2020). Our previous studies (Cheng and Ch’ng 2022) empirically confirmed that high-quality mass photogrammetry results could be obtained by integrating synchronous offline collaboration into crowdsourcing. However, recruiting and assigning volunteers for ad-hoc face-to-face teams is not trivial. These tedious processes can hinder potential and long-term participation and contribution. As the evolution of mobile systems and social space has changed the way of communication (Gilson et al. 2015; Ludwig, Reuter, and Pipek 2016), virtual collaboration can be a possible and beneficial solution to reduce external organisational efforts while promoting collective behaviour to ensure quality mass photogrammetry.

With the advent of various forms of computer-supported collaboration, awareness has been repeatedly emphasised as the key to efficient and effective virtual collaboration (Gross 2013; Gross and Prinz 2004; Heath et al. 2002). Awareness refers to the information collaborating individuals perceive about cooperative activities, peer actions, and shared artefacts, etc (Berlage and Sohlenkamp 1999; Dourish and Bellotti 1992; T. Erickson et al. 1999; Gutwin and Greenberg 1998). In situations where participants are physically within sight and earshot, awareness can be easily perceived and processed at the periphery of their attention (Kozlowski and Bell 2008; Reiter-Palmon, Wigert, and Vreede 2012). However, when participants are present at different places or times, awareness information is not naturally available and may be lost during the mitigation process facilitated by social computing technologies (Gross and Prinz 2004; Haines and Vehring 2012; Heath et al. 2002; Schuetz 1945). The challenge therefore lies in how we can use ICTs to support effective virtual collaboration and awareness perceptions for voluntary crowdsourcing of 3D cultural heritage.

Extensive CSCW research has investigated the conceptual frameworks of shared contexts and coordination (Berlage and Sohlenkamp 1999; Cabitza and Simone 2013; Gross 2013; Gross and Prinz 2004; Heath et al. 2002; Simone and Bandini 2002). As there is a social-technical gap existing between the contextualised social needs and technical mechanisms of CSCW (Ackerman 2000), many studies have examined the effectiveness of ICT designs in diverse disciplines and methods (Dailey and Starbird 2014; Gadiraju et al. 2019; Handler and Ferrer Conill 2016; Hansson and Ludwig 2019; Harandi 2019; Kulkarni, Can, and Hartmann 2012; Ludwig, Reuter, and Pipek 2016; Qiu, Gadiraju, and Bozzon 2020). However, few field experiments have been conducted in the interdisciplinary research of digital heritage and the concrete design directions for integrating ICTs into crowdsourcing 3D cultural heritage models remain relatively unexplored. Therefore, we aim to develop a systematic open collaboration approach to stimulate the voluntary undertaking of mass photogrammetry tasks. Explicitly, we seek to create a social space where self-selected volunteers can asynchronously participate, collaborate, and contribute to the onsite image acquisition.

Due to its heterogeneity, the achievement of any computer-supported cooperative work should be assessed taking its specific contextualised nature into account (Schmidt and Bannon 2013). The performance of our proposed approach can be evaluated by comparing it with different means of crowdsourcing photogrammetry-based 3D models. Based on our previous research (Cheng and Ch’ng 2022), we can assume that high-quality photogrammetric 3D models can be achieved by synchronous offline collaboration with volunteers assigned to face-to-face groups. However, this requires additional operations at the expense of user experience. On the other hand, conventional open calls that encourage isolated contributions from disorganised participants can offer flexible time options with little coordination. Nevertheless, this method may not yield quality contributions. Our proposed asynchronous virtual collaboration possesses a degree of freedom and flexibility lying between these two means. Therefore, we can assess our crowdsourcing approach through cross-evaluations regarding performance and user experience. The following RQs and hypotheses guide our research investigation:

RQ₁: Can asynchronous virtual collaboration facilitate effective crowdsourcing in reconstructing photogrammetry-based 3D models?

H₁_a: Asynchronous virtual collaboration generates better photogrammetric 3D models than conventional open calls.

H_1b: Asynchronous virtual collaboration yields photogrammetric 3D models of the same quality as synchronous offline collaboration.

RQ₂: To what extent can asynchronous virtual collaboration stimulate participation and contribution?

H_2a: Compared to conventional open calls, more participants self-select in asynchronous virtual collaboration within the same time period.

H_2b: Individual participants contribute more data in asynchronous virtual collaboration than in conventional open calls.

RQ₃: To what extent can asynchronous virtual collaboration enhance the user experience?

H_3a: Synchronous offline collaboration requires less user effort when compared to asynchronous virtual collaboration.

H_3b: Synchronous offline collaboration provides better performance when compared to asynchronous virtual collaboration.

This article is organised as follows. We contextualise our crowdsourcing scenario concerning related research and propose an operational model. On this basis, a web app is developed to facilitate mass photogrammetry. Our empirical experiments deploy and test three crowdsourcing approaches. Their crowdsourcing performance is cross evaluated through quantitative and qualitative analysis. Finally, we discuss the strengths and limitations of our mechanism and draw practical implications for further research.

This section analyses crowdsourcing scenarios and examines useful ICTs to facilitate task-oriented onsite collaboration. We learn from the general theoretical frameworks in CSCW literature and propose a feasible crowdsourcing model for our asynchronous virtual collaboration.

2.1 Targeting at right crowd and suitable cultural heritage objects

The success of crowdsourcing depends critically on whether or to what extent initiators can match crowdsourcing needs to the key characteristics of the crowd (L. B. Erickson, Petrick, and Trauth 2012; Li, Zhao, and Fuxman 2014; Naderi 2018). To acquire successive images of cultural heritage objects, museums and heritage sites are the most suitable cultural venues. Visitors, consequently, become our candidate volunteers who may respond to the open calls with expected contributions. For the initial research, we choose museums as our starting point for the artefacts are all conveniently housed in the same location. Therefore, our goal can be narrowed down to encouraging museum visitors to contribute photos of museum exhibits of sufficient quantity and necessary quality.

The quality of the photogrammetric 3D model is determined by the image acquisition process, which can be affected by the object’s characteristics and the environment (Ch’ng et al. 2019; Gruyter et al. 2014). Mass photogrammetry can achieve the optimal (not perfect) photogrammetric accuracy if the three conditions are guaranteed: 1) sufficient space for 360-degree image acquisition; 2) high contrast and properly coloured materials in relation to the background; 3) appropriate lighting conditions (Ch’ng et al. 2019; Cheng and Ch’ng 2022). In traditional museum settings, scattered exhibits are organised and displayed in the windows. Some are placed side-by-side or that the displays have poor lighting conditions. This creates natural barriers that may prevent some angles from being captured. For quality mass photogrammetry, rather than letting amateurs decide which artefacts to photograph, it would be more efficient to give facilitators the discretion to choose and assign artefacts. Imitators can choose exhibits placed in individual display cases with adequate lighting conditions to increase the success rate of 3D reconstructions.

2.2 Situated crowdsourcing for onsite image acquisition

Our crowdsourcing outsources onsite image acquisition tasks to museum visitors. These location-based tasks, such as geo-tagging images (Jonathan and Mokbel 2018), can be challenging to solve for existing online crowdsourcing platforms. Because online crowdsourcing activities often fail to match tasks to demographic characteristics (L. B. Erickson, Petrick, and Trauth 2012; Li, Zhao, and Fuxman 2014) and limit the ability to attract volunteers with desired backgrounds or skills (Rubya, Numainville, and Yarosh 2021), especially when contextualised knowledge is required (Huang 2015). Situated crowdsourcing fills the gap, as it can embed interfaces in physical spaces and target at geographically or temporally situated crowds (Jonathan and Mokbel 2018; Rubya, Numainville, and Yarosh 2021). It has many benefits over generic open calls on online platforms. It can reach an untapped source of potential workers, leverage their serendipitous availability, and use their local knowledge or cognitive states (Goncalves et al. 2013, 2014; Hosio et al. 2014). Based on these motivating factors, we consider situated crowdsourcing suitable for our crowdsourcing campaign. The challenge remains on how we can create a contextually controlled crowdsourcing scenario in the museum.

Obtaining accurate indoor positioning information is essential for our situated crowdsourcing. GPS, the most commonly used positioning technique for mobile devices, does not work well inside buildings since the satellite signal can be blocked (Nikander et al. 2013). Bluetooth and WLAN-based technologies are less accurate and may require additional setups and user-granted permissions (Resch 2013). QR codes can be beneficial indoor positioning solutions due to the requirements of having a line-of-sight on the code (Nikander et al. 2013). Moreover, many museums have adopted QR codes in diverse ways (Fuks et al. 2012; Giaccardi 2012; Haworth and Williams 2012; Pérez-Sanagustín et al. 2016; Psomadaki et al. 2019). They are often positioned on the artefact’s label, allocating extended information to specific cultural heritage objects.

Therefore, QR codes can serve as interfaces, connecting our crowdsourcing platform with the museum’s established infrastructures. This cost-effective and easy-to-deploy method can map corresponding virtual collaboration to specific cultural heritage objects. Such practices also align with the conventions of museum visits, for participants can take and upload photos while observing artefacts on the spot. It can leverage the serendipitous availability of museum visitors and reduce extra operations on the users’ side, such as photo sorting. The accurate location information obtained from QR codes can also help initiators to avoid complex data pre-processing and classification.

2.3 Technology-supported Collaboration and Awareness Perception

Effective and efficient virtual collaboration requires technological support for the cooperation process and awareness perception (Gross and Prinz 2004; Heath et al. 2002). Awareness is the information that collaborators perceive about cooperative activities (Dourish and Bellotti 1992; T. Erickson et al. 1999; Gutwin and Greenberg 1998). Various prototypes and implementations have been presented to provide sufficient awareness information for virtual cooperative settings in CSCW, HCI and Ubiquitous Computing literature. There are sensors for capturing situational context (Gellersen, Schmidt, and Beigl 2002), models for data processing (Gross and Prinz 2004), and toolkits or widgets for visualisation and presentation (Sohlenkamp, Prinz, and Fuchs 2000). It has been concluded that the general awareness model requirements include: 1) precise mapping of real situations; 2) close correspondence to the cooperative settings; 3) adaptation to changing situations; 4) little effort required on the users’ side (Gross and Prinz 2004; Heath et al. 2002).

Prior findings in team studies also suggested that an emergent cognition that enables team members to predict and anticipate one another’s actions is essential for team function (DeChurch and Mesmer-Magnus 2010; Maynard and Gilson 2014; Müller and Antoni 2021). Establishing a shared understanding of teamwork among team members is crucial (Cannon-Bowers, Salas, and COnverse 1993; Mathieu et al. 2000). Because shared mental models can constantly emphasise cooperative activities, stimulate synergetic effects (Hertel 2011), and produce positive outcomes, including team performance and coordination efficiency (DeChurch and Mesmer-Magnus 2010). The shared knowledge structures include three main aspects: members’ expertise and skills; deadlines or task duration; and collective goals, strategies, and tasks (Cannon-Bowers, Salas, and COnverse 1993; Mathieu et al. 2000; Mohammed et al. 2015; Müller and Antoni 2021).

Our platform should facilitate the awareness perception and emergence of shared mental models. We first need to demonstrate the mass photogrammetry principles and break down the steps so that our outsourced tasks can align with participants’ daily practices of photo taking. In addition to emphasising collective goals, cooperative individuals should be able to sense the collaborative dynamics, for it can affect their adoption of teamwork strategies. Due to our crowdsourcing campaign's voluntary and self-selecting nature, our platform is restricted in collecting user data. Thus, group chat programs can be a potential solution to capture cooperative activities. Participants can join whenever they prefer, share their actions or feelings, and view the chat logs to track peer activities. Since volunteers may not know each other and may have concerns about their personal privacy, we can incorporate anonymity and privacy protection features. Administrative roles may also be required to configure and maintain the group chats. As participation and contributions increase, our platform should be able to adapt to the changes to facilitate a more complex cooperation environment. We can employ visualisation toolkits and accumulation algorithms to automatically calculate the total number of participants and contributions at the current stage. Moreover, we can present a partially reconstructed 3D model processed using the collected data so far. To integrate all the functionalities into one stable platform, a web app is preferred for the following reasons. Firstly, a web app is lightweight, platform-independent, and functions well on multiple mobile devices (Android and IOS). As our crowdsourcing settings are moving beyond the desktop, it can lower accessibility barriers for participants to make instant responses. The platform can link multiple HTML-based pages, which can be converted to QR codes separately for corresponding virtual collaboration. It can also access mobile devices’ sensors, such as cameras and photo storage, which is sufficient to serve our needs.

Our platform (digitalheritage.site) features a responsive web design and can be accessed via multiple devices, from desktops to tablets and mobile phones. It is multilingual, with English and Chinese language options. All the functionalities are embedded in four main pages.

The Home page lists all necessary information, including project description, introduction, and knowledge about mass photogrammetry.
The Archive page presents our collection of reconstructed 3D cultural heritage.
The Open Collaboration page facilitates virtual collaboration using various ICTs, such as visualisations, chatrooms, and 3D model displays.
The Contribute Now page allows users to submit data via the front-end uploader.

We highlight two key features in the section below.

3.1 Basic Functionalities

The platform highlights the meaning, significance, and impact of digitalising cultural heritage (Fig. 1) to communicate collective goals and encourage voluntary participation. As most volunteers are non-professional, it is critical to ensure that they understand the task execution procedures for mass photogrammetry. For immediate guidance, a short video is used to visualise the essential principles of mass photogrammetry, followed by step-by-step image acquisition methods and quality requirements.

Gaining the trust of participants can inspire engagement and contribution. We, therefore, present transparent and verifiable information about our research and institutions to make our campaign more reliable and trustworthy. Users can provide instant feedback via the accessible Contact Us buttons (Fig. 2), allowing us to contact participants for future improvement.

Multiple buttons redirect users to the Contribution Now page (Fig. 3), where they can submit content via the embedded front-end uploader. Users can click the File-Upload buttons to select multiple images from their album or take photos directly. They can get instant visual feedback from the upload progress bar and cancel any uploads as needed. The default setting is to upload a full-size image with its EXIF file that contains behavioural data such as geo-locations.

3.2 Open Collaboration

We also implemented the Open Collaboration features to facilitate asynchronous virtual collaboration. In this case, each target artefact is associated with a virtual collaboration block containing its caption, essential historical info, the number of participants and photos, a process bar for 3D modelling, and a partially reconstructed 3D model generated using photos collected so far (Fig. 4). All numbers and process bar animate from 0 to the current state, providing an impression of accumulation. The interactable 3D models allow users to zoom in/out and rotate to discover any missing or imperfect parts of the current reconstruction.

Each block contains a Check/Leave Traces button (Fig. 4), through which participants can join the corresponding group chat (Fig. 5). The maximum capacity of simultaneous users for the chatroom is 40. The system identifies different users based on their devices and automatically assigns each user a random nickname at first, which can be changed manually. Users can view chats and send messages, including text, images and emojis, to share their actions, express feelings, and leave thoughts or comments. Participants can join, check, share, and re-join anytime they want. They can also check the user list to see who is online and interact with others. Every chatroom has an administrator responsible for moderating and coordinating online communication. Administrators can remotely manage group chats by banning illegal IPs, deleting inappropriate messages, and answering questions.

4.1 Pilot Tests and Ethical Considerations

In August 2020, 21 volunteers were recruited to freely explore all features of our web app on various devices, including laptops, tablets, and iOS/Android mobile phones (iPhone, Huawei, Xiaomi, Samsung, iQoo, OnePlus, and Vivo). Pilot tests indicated our platform could technologically support asynchronous collaboration and confirmed its usability and stability.

Regarding ethical issues, we obtained ethical approval from our institution prior to the experiments. The platform displays detailed information explaining our crowdsourcing activities, purposes, and research labs. There are also consent checks before participants take further actions. For instance, there are privacy notices before joining the group chat and checkboxes before submitting user-generated content.

4.2 Three Crowdsourcing Approaches

We examined our crowdsourcing approaches through controlled field experiments in a way that can reduce the constraints of environmental challenges. First, three crowdsourcing scenarios (Table 1) were formalised with decreasing flexibilities and increasing organisational costs. Participants in both Scenario I (Disorganised) and II (Virtual) are spontaneous and can self-select into crowdsourcing activities via scanning QR codes. The difference is that participants in Disorganised groups are unaware of collaboration and only have access to our platform’s basic functionalities (Section 3.1). The Open Collaboration features (Section 3.2) are only enabled to facilitate asynchronous virtual collaboration for participants in Virtual scenarios, who can perceive the presence of other participants and view the collaborative processes. Scenario III (F2F) is the most controlled one that requires external operations to recruit volunteers in advance and randomly assign them to face-to-face groups. Each group is then coordinated to perform synchronous collaboration using the basic functionalities of our platform (Section 3.1).

Table 1

Three Crowdsourcing Settings. Scenario I follows disorganised crowdsourcing and encourages isolated contributions. Scenario II utilises ICT-based virtual collaboration to support awareness perception and facilitate asynchronous participation and contribution. Scenario III employs offline face-to-face team structures to leverage collective behaviours.
	Scenario I (Disorganised)	Scenario II (Virtual)	Scenario III (F2F)
Approach	Conventional open calls	Asynchronous virtual collaboration	Synchronous offline collaboration
Features	- Disorganised - Asynchronous - Unaware of collaboration	- Self-organised - Asynchronous - Aware of collaboration	- Self-organised - Synchronous - Aware of collaboration
Partici-pation	- Spontaneous - Self-select - Work independently - Access to the basic features	- Spontaneous - Self-select - Work as a virtual group - Access to full features (basic & open collaboration)	- Recruited - Randomly assigned - Work in a F2F group - Access to the basic features

We chose five artefacts (Fig. 6) from the Shanghai Museum as target objects because they have separate display cases, proper lighting conditions and sufficient space for sequential multi-angle capture at different distances. These objects vary in form, shape, and size, leading to varying complexities in 3D reconstruction. This also makes them suitable experimental subjects to eliminate the potential effect imposed by objects’ characteristics and attractiveness on the comparison of crowdsourcing performance.

4.3 Data Collection

The series of experiments were conducted from 3rd to 10th December 2021. We set up both Disorganised and Virtual scenarios for three regular museum open days with similar visitor flows, during which we placed QR codes of our platform near the targets’ display cases so that visitors could decide whether or how they wanted to participate and contribute. For F2F scenarios, we recruited 15 volunteers and coordinated them into five groups to perform onsite image acquisition tasks for five target objects.

Our platform links the front-end submissions with our Cloud drive which automatically stores every submission as a data entry created in a designated format, including entry ID (IDs automatically assigned to each submission), date, IP, images, textual feedback, and optional info (such as email, caption, and descriptions). Chat logs of each chatroom could be exported as CSV files with time, text, and image links. We also published follow-up questionaries (Fig. 2) to learn about users’ crowdsourcing experience. Since Scenario III (F2F) requires external coordination, it allowed us to keep track of every volunteer’s contribution and feedback. However, the situations became complicated for Scenario I (Disorganised) and II (Virtual) due to their self-selective nature. Not all participants who contributed photos completed the survey. Some issued multiple submissions for multiple artefacts. Some may use the same device to participate and contribute. To avoid missing and duplicated data, we counted every submission as one participation so that we could obtain a file recording each submission with its associated time, images, number of images per submission, and textual information.

This section details crowdsourcing performance obtained from field experiments. We compared crowdsourcing outcomes regarding the quality of the 3D model generated in three scenarios and investigated crowd behaviours in terms of participation, contribution, user experience, and awareness perception.

5.1 Quality of 3D Reconstruction

We pre-processed collected photos and reconstructed 3D models for each group respectively using the RealityCapture software. As observed (Fig. 7), Disorganised groups failed to produce complete 3D models regardless of the objects' complexities. F2F and Virtual groups successfully generated complete 3D models for four artefacts (Fig. 8). As for Object E, both Virtual and F2F groups produced 3D models with a missing bottom (Fig. 9).

This can be attributed to the fact that Object E has the largest size among all five objects. It has a height of 93.1cm, a width of 75.6cm, and was placed in a display case, making the object even taller and more difficult for participants to get a top view or bottom of it. Considering its complexity and reconstruction difficulty, we can conclude that Virtual and F2F groups obtained similar crowdsourcing performance in model completion.

Thus, we the alternate hypotheses are true. H_1a: Asynchronous virtual collaboration generates better photogrammetric 3D models than conventional open calls and H_1b: Asynchronous virtual collaboration yields photogrammetric 3D models of the same quality as synchronous offline collaboration. This confirmed the effectiveness of our asynchronous virtual collaboration in reconstructing photogrammetry-based 3D models.

Table 2

Results of Statistical Tests. Basic statistics are calculated for three groups in terms of number of images contributed and time spent. Kruskal-Wallis H Tests are computed, and the results confirm the statistically significant differences among three groups in the number of images contributed (H(2) = 50.52, p < .01) and time spent (H(2) = 23.05, p < .01). The post-hoc Mann-Whitney U Tests are adopted for further comparison between two groups, and the results verify statistical significance in all pairs, expect for the time spent in Virtual and Disorganised scenarios. It means that Virtual (x̄=10.0min) and Disorganised groups (x̄=9.5min) spent similar amount of time on crowdsourcing tasks.
	Basic Statistics					H-Test df = 2, N = 216	Post-hoc Comparison (U-Test)
		n	Σ	x̄	x̃	H-Test df = 2, N = 216		U-value	z-score	p-value	Sig.
Num. of Images	D	42	278	6.6	6	H = 50.52 p < 0.001 Reject H₀	V vs D	2226.5	3.332	0.0009	Y
	V	159	1439	9.1	8		F vs V	13.5	− 6.320	< 0.001	Y
	F	15	228	15.2	26		F vs D	0	− 5.799	< 0.001	Y
Time Spent (mins)	D	42	399	9.5	8	H = 23.05 p < 0.001 Reject H₀	V vs D	2964	1.117	0.263	N
	V	159	1519	10.0	10		V vs F	310.5	− 4.727	< 0.001	Y
	F	15	258	17.2	16		F vs D	100.5	− 5.700	< 0.001	Y

Note: D = Disorganised, V = Virtual, F = F2F, n = sample size, Σ = total number, x̄=mean, x̃=median, df = degree of freedom, Sig.=Significance.

5.2 Participation and Contribution

Participation and contributions can be reflected using the number of participants and images contributed during the crowdsourcing activities at both the group and individual levels.

Group-Level Participation and Contribution

Three crowdsourcing approaches differed significantly (Table 2) in the total number of participants (n_Disorg.=42, n_Virtual=159, n_F2F=15), image collected (Σ_img.Disorg. = 278, Σ_img.Virtual = 1439, Σ_img.F2F = 228). The statistics reveal that Virtual scenarios successfully motivated more participation over three days, which confirmed our H_2a: Compared to conventional open calls, more participants self-select in asynchronous virtual collaboration within the same time period.

As observed, even if Disorganised groups (Σ_img.Disorg = 278) contributed more images than F2F groups in total, they failed to produce complete 3D models. Despite the high-quality 3D reconstructions, the F2F scenarios recruited the fewest volunteers (n_F2F=15) and used the least total number of images (Σ_img.F2F = 228) compared to the other two settings. It implies that the simple accumulation of images could not guarantee desired outcomes for mass photogrammetry. High-quality contributions from individual participants are required to achieve effective crowdsourcing.

Individual-Level Image Contribution

When it comes to the individual level, Fig. 10 illustrates that individual participants in three scenarios behaved and contributed differently. Since distribution shapes among the three groups differ (Fig. 11), we adopted the Kruskal-Wallis and post-hoc Mann-Whitney tests to verify the statistical significance (Table 2). It indicates that individual contributions of three groups were statistically different, which can be ranked from most to least as: F2F (x̄=15.2, x̃=26), Virtual (x̄=9.1, x̃=8) and Disorganised (x̄=6.6, x̃=6). It verifies H_2b: Individual participants contribute more data in asynchronous virtual collaboration than in conventional open calls. We can conclude that our asynchronous virtual collaboration successfully motivated more participation and better contributions for mass photogrammetry than conventional open calls.

Other Data Contributions

We further investigated the communication that occurred in the crowd-based self-organised collaboration in F2F and Virtual scenarios. As coordinated (Table 1), F2F groups that gathered offline usually had a quick brief prior to task executions. During their image acquisition, explicit communication among group members was rarely observed. This can be ascribed to three main reasons. 1) the task is relatively easy with low interdependence. Group members can adjust behaviours through implicit observation. 2) the ad-hoc teams are formed randomly and temporally, and participants barely know each other. 3) their image acquisition actions do not align with conventions of practices in museum visits during which visitors are supposed to be quiet.

We processed the chat logs obtained from the anonymous online chatrooms in Virtual scenarios. The content was encoded with multiple tags: Action.Des: Describe actions (e.g., "I took pictures of..."). Angles: Describe shooting angles (e.g., "top", "bottom", "inside"). Artefacts: Specific reference to the cultural heritage objects. Barriers: Mention barriers during the task execution (e.g., "Reflective glass", "Bar frame"). Concerns: Raise concerns/issues about crowdsourcing (e.g., "afraid", "concerned"). Emotions: Express emotional reactions (e.g., emojis; "wow"; "ha-ha"). History.Info: Share historical information. History.Q: Ask questions about history/artefacts. Obj.Features: Mention specific features of an artefact (e.g., "paw", "eyes", "camel's hump"). Photograph: Mention the image acquisition/photo-taking action. Suggestion: Give suggestions for platform/activities (e.g., "should", "it's better to"). Tech.Info: Share information about 3D technology or mass photogrammetry. Tech.Q: Ask questions about technology or techniques. 3D.Model: Specific reference to photogrammetry-based 3D model.

As depicted (Fig. 12), participants in Virtual groups were not intuitively triggered to report their actions. Instead, their share desires were more likely to be stimulated by emotional reactions towards the artefacts. Although the information exchange seemed less effective in terms of task execution, it covered a more comprehensive range of topics than that within F2F groups.

5.3 User Experience

User experience consists of two main aspects: user effort and crowdsourcing experience. User workload can be examined by combining objectively recorded task execution times with subjective feedback. Crowdsourced experiences are primarily related to self-reported app interactions and overall enjoyment. Our questionnaires collected participants’ responses using 7-point Likert scale items ranging from 'strongly disagree' to 'strongly agree'. The Likert scale distributions of three groups were plotted to compare and reflect user experience.

User Effort

First, we examined the time users spent performing tasks, following the method of analysing image contribution (Section 5.2). It can be observed (Fig. 10 and Fig. 11) that the three groups appeared to have varying task execution times. Nevertheless, our statistical tests (Table 2) indicate that there was no significant difference in time spent between the Disorganised (x̄=9.5min, x̃=8min) and Virtual (x̄=10min, x̃=10min) groups. Both groups took much less time to perform tasks than the F2F groups (x̄=17.2min, x̃=16min). Less execution time implies that Virtual groups required less effort on the users’ side. Participants also rated effort regarding time spent and image contribution for the crowdsourcing. A lower score indicates less workload for participants. As illustrated (Fig. 13), the shapes of Likert scale distributions generated by the Virtual groups are skewed towards lower scores for both questions, especially compared to F2F groups. Combining the above observations, we can deduce that our H_3a is true: Compared to synchronous offline collaboration, asynchronous virtual collaboration requires less user effort.

Crowdsourcing Experience

We learnt about participants’ experiences from their feelings about the overall campaign and interactions with our platform. A higher score implies a better experience. Figure 13 shows that all three groups responded positively as most scores were higher than 4 (i.e., neutral response). Fluctuations in the distributions generated by F2F groups may be due to the varying complexities of target objects handled by different participants. Objects of larger sizes require users to contribute more images and spend more time interacting with the app. Scores obtained from Virtual groups for both questions produced two stronger and steadier upward trends, implying the overall experience was generally good regardless of object complexities. These observations can validate our H_3b: Compared to synchronous offline collaboration, asynchronous virtual collaboration provides a better crowdsourcing experience.

User Experience of Virtual Groups

Figure 13 also reveals that participants in Virtual groups tended to give better reviews to our platform (app interaction) than those in Disorganised and F2F groups. Unlike the other two experimental settings (Table 1), Virtual scenarios allowed participants to access the full features of our platform, consequently differentiating their user experience, especially when it comes to app evaluation. Hence, we assessed the user experience of Virtual groups in details.

Specifically, the Likert plots (Fig. 14) demonstrate that most participants reported that they had a positive crowdsourcing experience in platform interactions (78%) and overall enjoyment (67%). The distribution of user effort ratings no longer consistently favours high scores (a higher score means a higher workload). 65% of participants felt they did not put much effort into image contributions. Relatively speaking, the evaluation of time spent was evenly distributed, with roughly the same number of participants considering they spent a lot (47%) or little time (39%) on task execution. This may be due to the varying complexity of target objects experienced by different participants. We can thus validate the usability of the Open Collaboration functionalities and infer that asynchronous virtual collaboration requires little user effort and promotes a better crowdsourcing experience.

5.4 Awareness Perception

Performance differences can be closely related to awareness perception during the collaboration. Hence, we further explored whether and to what extent our virtual collaboration can support awareness perception, and how it can affect user behaviour. We measured participants’ perceived information using self-reported 7-point Likert scores and adopted Spearman’s rho to determine the relations between ranked variables.

Perceived Meaningfulness of the Crowdsourcing Campaign

Our platform attempted to emphasise the impact of digitalising cultural heritage and the significance of our crowdsourcing campaign (Section 3.1). Here we reflect on our design and investigate the correlations between user experience and their perception of the meaningfulness of the activities.

We focused on the user experience in Virtual scenarios. The plots in Fig. 14 demonstrate that 86% of participants recognised the importance of our crowdsourcing activities. Participants’ awareness of meaningfulness was strongly correlated with their evaluation of our platform (r_s=.74, p < .01, n = 51) and moderately related to their enjoyment of the overall experience (r_s=.46, p < .01, n = 51). However, there were no statistically significant relationships between perceived importance and self-reported user effort (p-values > .01 for contribution and time spent). The results indicate that our platform had successfully conveyed the meaning of our crowdsourcing campaign, which can, to some extent, improve users’ favourability of our platform.

Perceived Knowledge of Principles and Steps of Mass Photogrammetry

Knowledge of photogrammetric principles and procedures was critical to the success of mass photogrammetry. Since such information was communicated to participants differently in three experimental settings (Table 1), we assessed participants’ understanding of principles and steps within each of the three groups respectively.

Different results were computed (Fig. 15), although the p-values (< .001) for all three groups statistically confirmed a positive correlation between these two factors. F2F groups had the most significant proportion of participants who understood the principles (81%) and steps (81%), with a strong correlation between them (r_s=.89, p < .01, n = 15). This can be explained by the way they cooperated. Participants in F2F groups discussed strategies prior to fieldwork, which guided subsequent image acquisition steps. Disorganised groups (r_s=.85, p < .01, n = 28) also obtained a high coefficient even though they had a minor proportion of participants who responded positively to both questions (principles: 54%, steps: 39%). It might suggest that participants who could understand the principles were funnelled to be those who knew task execution steps. These phenomena highlight the significance of illustrating technical principles, for it can help contributors develop a taskwork mental model

Virtual groups were generally more aware of mass photogrammetry knowledge (principles: 65%, steps: 69%) than Disorganised groups, but they were less knowledgeable than F2F groups. Interestingly, among three experimental settings, the Virtual scenario was the only one in which more participants knew the steps than those who understood the principles. And these two variables were moderately correlated (r_s=.69, p < .01, n = 51). We consider possible explanations as follows. Since Virtual groups could generate 3D reconstructions of a similar quality to F2F groups, they should have been able to obtain relevant information during the activities. Many participants in Virtual groups may feel lost prior to task execution, like those in Disorganised groups, as there was no one to assist them directly on the spots. The virtual chatrooms can provide a channel where they can ask and learn instant feedback from others. Moreover, rather than spending efforts figuring out complex principles, most participants tended to pick up executable actions they observed. The results suggest that asynchronous virtual collaboration can facilitate the creation and development of the shared mental model within groups.

Our empirical research has investigated and compared three crowdsourcing approaches for 3D cultural heritage objects, namely, conventional open calls (Disorganised), asynchronous virtual collaboration (Virtual), and synchronous offline collaboration (F2F). The Disorganised and F2F scenarios can be considered representatives of two different crowdsourcing means with relatively extreme levels of flexibility and organisational control (Table 1). The results obtained from these two scenarios can establish baselines for examining the performance of the proposed asynchronous virtual collaboration.

The photogrammetric outcomes (Figs. 7, 8 and 9) illustrate that the 3D models generated by the Virtual groups are superior to the Disorganised groups and similar to the F2F groups. These demonstrations answer our RQ₁ that asynchronous virtual collaboration can facilitate effective crowdsourcing in reconstructing photogrammetry-based 3D models. Comparing two experimental settings that allow spontaneous and self-selected participation, the Virtual scenarios evoke higher response rates and better quality of data contribution (Table 1, Figs. 10 and 11) than the Disorganised scenarios. Participants in the Virtual collaboration also communicate more than members in the F2F collaboration, exchanging a broader and more diverse range of information in the online chatrooms (Fig. 12). These findings have answered RQ₂, that asynchronous virtual collaboration can stimulate more participation and contribution than conventional open calls, and more explicit information exchange than synchronous offline collaboration. Furthermore, compared to F2F groups, individual participants in Virtual groups can spend significantly less time (Table 1, Figs. 10 and 11) and less user effort (Fig. 13) to produce complete 3D models. During the activities, the Virtual groups can obtain satisfying overall experience and platform interactions (Figs. 13 and 14). Our RQ₃ can thus be answered that asynchronous tasks can enhance user experience by reducing user effort and providing a positive crowdsourcing experience. Our further exploration of awareness perception reveals that our platform effectively communicates collective goals, and that participants’ perceived meaningfulness strongly influences their crowdsourcing experiences. It also confirms the usability and effectiveness of our Open Collaboration functionalities in facilitating the process of taskwork knowledge acquisition, which can lead to better task execution.

The findings of field experiments provide insights and practical implications. The chat logs (Fig. 12) showed that the exchange of information around specific artefacts can extend beyond traditional social communication in museum settings. There is great potential to embed the interfaces in physical infrastructures. Virtual spaces linked to interfaces can asynchronously connect situated visitors with diverse backgrounds, creating a social context where participants can mark or view unique experiences associated with specific objects or locations. User-generated content can be wide-ranging, including expressing feelings, sharing thoughts, recording stories, tagging anecdotes, etc. The accumulation of such extensive information can certainly benefit digitalising campaigns in the long run. Second, our platform can be improved as a tour guide for visitors exploring the museum. We can cooperate with curators to select and connect artefacts into contextualised stories to align with designated scenic routes. Moreover, online communities and game mechanisms can be further explored to sustain participation and stimulate better user engagement. Even though extra functionalities may trade off the simplicity of use, all potential implementations can be iteratively tested in future research.

There are also some limitations in our field study. First, as we focused on five target objects (Fig. 6), our platform did not allow participants to issue a personalised thread for the object they were interested in. Although we opened multiple channels for participants to share feedback or send additional photos, no one else can view the threads or collaborate on them. It may discourage volunteers from contributing more pictures, particularly for those artefacts not listed. Moreover, we observed that participants who participated at the early stage might feel alone and isolated since only a few messages were displayed. They needed to wait longer to view the 3D modelling process, for it took time to aggregate enough photos to reconstruct 3D models. Some participants lose interest even if they can recheck the progress later at any time.

Furthermore, this current work mainly examined crowdsourcing performance in task accomplishment; the collaboration dynamics remain understudied. The temporal dynamics that are emergent during crowdsourcing are the drivers for the success of this particular crowdsourcing activity. Exploring the crowd behaviours during the temporal collaborative dynamics can indeed shed light on how best to involve the crowds in cultural heritage activities. All these issues will be further investigated prior to any large-scale, cross-border mass photogrammetry activity.

This research presents a systematic crowdsourcing approach that uses virtual collaboration to facilitate asynchronous participation and contributions for mass photogrammetry of 3D cultural heritage objects. Combining existing literature and our crowdsourcing purposes, we implemented an operational model and empirically tested it on five artefacts displayed in a museum. Its performance was evaluated in comparison with conventional open call and synchronous offline collaboration regarding quality of photogrammetric 3D model, participation, contribution, and user experience. The results obtained confirmed the comparative advantages of our asynchronous virtual collaboration. Our mechanisms can be well adapted to the conventions of practices in museum visits and its self-selection nature can provide necessary degrees of freedom and flexibilities to promote the gathering of volunteers with common interests. It significantly reduces external operational costs and leverages the serendipitous availability of participants to encourage engagement. Moreover, our research has effectively facilitated the interplay and awareness of volunteers on collective goals and tasks, resulting in better performance and contribution. Furthermore, surveys of user experience reveal that participants spend less effort and obtain better experience. These findings have demonstrated that our situated asynchronous collaborative crowdsourcing can successfully harness collective behaviour to achieve high-quality models with low operations and coordination costs. It demonstrated the potentials of integrating ICTs into crowdsourcing activities and enlighten the benefits of creating a virtual crowdsourcing space that can connect participants with diverse backgrounds, and that gathers crowd-generated content and contextual information pertaining to the physical environment inhabited by the grassroots.

Acknowledgement

This work was carried out at the NVIDIA Joint-Lab on Mixed Reality, NVIDIA Technology Centre at the University of Nottingham Ningbo China. The authors wish to express their gratitude to CapturingReality, and Epic Games for their support with RealityCapture, the photogrammetry software that made this research possible. Our sincere thanks to all the volunteers for participating in this exciting and worthwhile project. Finally, our special appreciation to the NVIDIA AI Technology Center, Singapore, for their continued support and knowledge exchange in technologies we use for our global cultural heritage research. This research is partly funded by the AHRC ’Shaping the Connected Museum II’ (Grant number: AH/T011246/1).

Ethical Approval

Prior to experiments, we obtained ethical clearance and approval for this research project from our institution – the University of Nottingham Ningbo China. The detailed documentations can be accessed via our institution if required.

Consent to participate

Both a participant consent form and participant information sheet were prepared for potential participants during the experiments. The platform we used for the field experiments displays detailed information, explaining our research purposes and institution. There are checkboxes before participants take further actions.

Consent to publish

Both authors give our consent to “Computer Supported Cooperative Work (CSCW): The Journal of Collaborative Computing and Work Practices” for the publications of this article, including the figures, tables, and data used.

Competing Interests

This work was carried out at the NVIDIA Joint-Lab on Mixed Reality, NVIDIA Technology Centre at the University of Nottingham Ningbo China. Eugene Ch’ng is the director of this lab, in cooperation with the NVIDIA AI Technology Center, Singapore. However, there is no financial interest in the materials discussed in this manuscript.

We used the RealityCapture, the photogrammetry software, in reconstructing our 3D cultural heritage objects. However, both authors certify that they have no affiliations with or involvement in the CapturingReality, the Epic Games, or the RealityCapture. And there is no financial interest or non-financial interest to disclose in the subject matter.

This research is partially funded by the Arts and Humanities Research Council (AHRC): Shaping the Connected Museum II (Grant number: AH/T011246/1). Details can be found via the link: https://app.dimensions.ai/details/grant/grant.8856193.

Authors’ Contributions

The authors acknowledge their contributions to the paper as follows:

Danzhao Cheng: study conception and literature review, field experiments and data collection, data analysis and result interpretation, and manuscript drafting and preparation.

Eugene Ch'ng: ideation, research and experimental guidance, and manuscript reviews.

Both authors reviewed the findings and approved the final version of the manuscript.

Funding

Availability of data and materials

Data used to support the results reported in this article can be found. The datasets collected and processed during the current study are available for the corresponding author (Danzhao Cheng) on reasonable request.

Ackerman, Mark S. 2000. “The Intellectual Challenge of CSCW: The Gap Between Social Requirements and Technical Feasibility.” Human–Computer Interaction 15(2–3): 179–203. https://www.tandfonline.com/doi/full/10.1207/S15327051HCI1523_5.
Alsadik, Bashar. 2016. “Crowdsource and Web-Published Videos for 3D Documentation of Cultural Heritage Objects.” Journal of Cultural Heritage 21: 899–903. http://dx.doi.org/10.1016/j.culher.2016.03.010.
Berlage, Thmoas, and Markus Sohlenkamp. 1999. “Visualizing Common Artefacts to Support Awareness in Computer-Mediated Cooperation.” Computer Supported Cooperative Work (CSCW) 8(3): 207–38. http://link.springer.com/10.1023/A:1008608425504.
Blohm, Ivo, Shkodran Zogaj, Ulrich Bretschneider, and Jan Marco Leimeister. 2018. “How to Manage Crowdsourcing Platforms Effectively?” California Management Review 60(2): 122–49. http://journals.sagepub.com/doi/10.1177/0008125617738255.
Bonacchi, Chiara et al. 2019. “Participation in Heritage Crowdsourcing.” Museum Management and Curatorship 34(2): 166–82. https://www.tandfonline.com/doi/full/10.1080/09647775.2018.1559080.
Cabitza, Federico, and Carla Simone. 2013. “Computational Coordination Mechanisms: A Tale of a Struggle for Flexibility.” Computer Supported Cooperative Work: CSCW: An International Journal 22(4–6): 475–529.
Cannon-Bowers, Janis A., Eduardo Salas, and Sharolyn COnverse. 1993. “Shared Mental Models in Expert Team Decision Making.” In Individual and Group Decision Making: Current Issues, ed. Lawrence Erlbaum. , 221–46. https://books.google.com.hk/books?hl=en&lr=&id=HklF9o_x9JEC&oi=fnd&pg=PA221&dq=Shared+mental+models+in+expert+team+decision+making&ots=wI4Ti2BlSC&sig=3aIoGis6ZU8cH6CxlhdPqGVwz2c&redir_esc=y#v=onepage&q=Shared mental models in expert team decision making&f (January 9, 2022).
Ch’ng, Eugene. 2013. “The Mirror Between Two Worlds: 3D Surface Computing Interaction for Digital Objects and Environments.” Digital Media and Technologies for Virtual Artistic Spaces 21(3): 166–85. https://www.academia.edu/646062/The_Mirror_between_Two_Worlds_3D_Surface_Computing_for_Objects_and_Environments (October 9, 2022).
Ch’ng, Eugene, Shengdan Cai, Tong Evelyn Zhang, and Fui Theng Leow. 2019. “Crowdsourcing 3D Cultural Heritage: Best Practice for Mass Photogrammetry.” Journal of Cultural Heritage Management and Sustainable Development 9(1): 24–42.
Cheng, Danzhao, and Eugene Ch’ng. 2022. “Harnessing Collective Differences in Crowdsourcing Behaviour for Mass Photogrammetry of 3D Cultural Heritage.” Journal on Computing and Cultural Heritage 1(1): 1–24. https://dl.acm.org/doi/10.1145/3569090.
“CyArk.” 2021. https://www.cyark.org/ (November 27, 2021).
Dailey, Dharma, and Kate Starbird. 2014. “Journalists as Crowdsourcerers: Responding to Crisis by Reporting with a Crowd.” Computer Supported Cooperative Work: CSCW: An International Journal 23(4–6): 445–81.
DeChurch, Leslie A., and Jessica R. Mesmer-Magnus. 2010. “The Cognitive Underpinnings of Effective Teamwork: A Meta-Analysis.” Journal of Applied Psychology 95(1): 32–53.
Dissanayake, Indika, Jie Zhang, and Bin Gu. 2015. “Virtual Team Performance in Crowdsourcing Contest: A Social Network Perspective.” In 2015 48th Hawaii International Conference on System Sciences, IEEE, 4894–97. http://www.tandfonline.com/doi/full/10.1080/07421222.2015.1068604.
Dourish, Paul, and Victoria Bellotti. 1992. “Awareness and Coordination in Shared Workspaces.” In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work - CSCW ’92, New York, New York, USA: ACM Press, 107–14. http://dourish.com/publications/1992/cscw92-awareness.pdf.
Erickson, Lee B., Irene Petrick, and Eileen M. Trauth. 2012. “Hanging with the Right Crowd: Matching Crowdsourcing Need to Crowd Characteristics.” In 18th Americas Conference on Information Systems 2012, AMCIS 2012, , 77–85. https://aisel.aisnet.org/amcis2012/proceedings/ VirtualCommunities/3.
Erickson, Thomas et al. 1999. “Socially Translucent Systemss: Social Proxies, Persistent Conversation, and the Design of Babble.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems the CHI Is the Limit - CHI ’99, New York, New York, USA: ACM Press, 72–79. http://portal.acm.org/citation.cfm?doid=302979.302997.
“Europeana.” https://www.europeana.eu/en (November 27, 2021).
Fuger, Simon, Robert Schimpf, Johann Füller, and Katja Hutter. 2017. “User Roles and Team Structures in a Crowdsourcing Community for International Development – a Social Network Perspective.” Information Technology for Development 23(3): 438–62. https://www.tandfonline.com/doi/full/10.1080/02681102.2017.1353947.
Fuks, Hugo et al. 2012. “Collaborative Museums.” : 681.
Gadiraju, Ujwal, Gianluca Demartini, Ricardo Kawase, and Stefan Dietze. 2019. “Crowd Anatomy Beyond the Good and Bad: Behavioral Traces for Crowd Worker Modeling and Pre-Selection.” Computer Supported Cooperative Work (CSCW) 28(5): 815–41. http://link.springer.com/10.1007/s10606-018-9336-y.
Gellersen, Hans W., Albrecht Schmidt, and Michael Beigl. 2002. “Multi-Sensor Context-Awareness in Mobile Devices and Smart Artifacts.” Mobile Networks and Applications 7(5): 341–51.
Giaccardi, Elisa. 2012. Heritage and Social Media. https://ebookcentral.proquest.com/lib/nottingham/detail.action?docID=988002 (August 19, 2021).
Gilson, Lucy L. et al. 2015. “Virtual Teams Research: 10 Years, 10 Themes, and 10 Opportunities.” Journal of Management 41(5): 1313–37.
Goncalves, Jorge et al. 2013. “Crowdsourcing on the Spot: Altruistic Use of Public Displays, Feasibility, Performance, and Behaviours.” In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, New York, NY, USA: ACM, 753–62. https://dl.acm.org/doi/10.1145/2493432.2493481.
Goncalves, Jorge, Simo Hosio, Denzil Ferreira, and Vassilis Kostakos. 2014. “Game of Words: Tagging Places Through Crowdsourcing on Public Displays.” In Proceedings of the 2014 Conference on Designing Interactive Systems, New York, NY, USA: ACM, 705–14. https://dl.acm.org/doi/10.1145/2598510.2598514.
Gross, Tom. 2013. “Supporting Effortless Coordination: 25 Years of Awareness Research.” Computer Supported Cooperative Work (CSCW) 22(4–6): 425–74. http://link.springer.com/10.1007/s10606-013-9190-x.
Gross, Tom, and Wolfgang Prinz. 2004. “Modelling Shared Contexts in Cooperative Environments: Concept, Implementation, and Evaluation.” Computer Supported Cooperative Work: CSCW: An International Journal 13(3–4): 283–303.
Gruyter, De Walter et al. 2014. Close-Range Photogrammetry and 3D Imaging. 2nd ed. eds. Thomas (Thomas) Luhmann, Stuart (Engineering professor) Robson, Stephen (Engineering professor) Kyle, and Jan (Engineering professor) Boehm. Berlin, Germany: De Gruyter. https://www.degruyter.com/document/doi/10.1515/9783110607253/html.
Gutwin, Carl, and Saul Greenberg. 1998. “Design for Individuals, Design for Groups: Tradeoffs Between Power and Workspace Awareness.” In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work - CSCW ’98, New York, New York, USA: ACM Press, 207–16. http://portal.acm.org/citation.cfm?doid=289444.289495.
Haines, Russell, and Nadine Vehring. 2012. “Increasing Team Coordination and Social Motivation Through Awareness Practices: A Case Study.” In European Conference on Information Systems (ECIS) at AIS Electronic Library (AISeL), , 5–15. http://aisel.aisnet.org/ecis2012.
Handler, Reinhard A., and Raul Ferrer Conill. 2016. “Open Data, Crowdsourcing and Game Mechanics. A Case Study on Civic Participation in the Digital Age.” Computer Supported Cooperative Work (CSCW) 25(2–3): 153–66. http://dx.doi.org/10.1007/s10606-016-9250-0.
Hansson, Karin, and Thomas Ludwig. 2019. “Crowd Dynamics: Conflicts, Contradictions, and Community in Crowdsourcing.” Computer Supported Cooperative Work (CSCW) 28(5): 791–94. http://link.springer.com/10.1007/s10606-018-9343-z.
Harandi, Mahboobeh. 2019. “Supporting Occasional Groups in Crowdsourcing Platforms.” In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing, New York, NY, USA: ACM, 52–55. https://dl.acm.org/doi/10.1145/3311957.3361856.
Hartmann, W., M. Havlena, and K. Schindler. 2016. “Towards Complete, Geo-Referenced 3D Modles From Crowd-Sourced Amatur Images.” ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences III–3(July): 51–58. http://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/III-3/51/2016/isprs-annals-III-3-51-2016.pdf.
Haworth, Annette, and Peter Williams. 2012. “Using QR Codes to Aid Accessibility in a Museum.” Journal of Assistive Technologies 6(4): 285–91. https://www.emerald.com/insight/content/doi/10.1108/17549451211285771/full/html.
Heath, Christian et al. 2002. “Configuring Awareness.” Computer Supported Cooperative Work (CSCW) 11(3–4): 317–47. http://link.springer.com/10.1023/A:1021247413718.
Hertel, Guido. 2011. “Synergetic Effects in Working Teams” ed. Guido Hertel. Journal of Managerial Psychology 26(3): 176–84. https://www.emerald.com/insight/content/doi/10.1108/02683941111112622/full/html.
Hosio, Simo et al. 2014. “Situated Crowdsourcing Using a Market Model.” In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA: ACM, 55–64. https://dl.acm.org/doi/10.1145/2642918.2647362.
Huang, Yi Ching. 2015. “Designing a Micro-Volunteering Platform for Situated Crowdsourcing.” In Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing, New York, NY, USA: ACM, 73–76. https://dl.acm.org/doi/10.1145/2685553.2699324.
Jonathan, Christopher, and Mohamed F. Mokbel. 2018. “Stella: Geotagging Images via Crowdsourcing Christopher.” In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, New York, NY, USA: ACM, 169–78. https://dl.acm.org/doi/10.1145/3274895.3274902.
Katz, Jared. 2017. “Digitized Maya Music: The Creation of a 3D Database of Maya Musical Artifacts.” Digital Applications in Archaeology and Cultural Heritage 6(August): 29–37. http://dx.doi.org/10.1016/j.daach.2017.08.004.
Kozlowski, Steve W J, and Bradford S Bell. 2008. “Team Learning, Development, and Adaptation.” In V. I. Sessa & M. London (Eds.), Work Group Learning: Understanding, Improving and Assessing How Groups Learn in Organizations, Taylor & Francis Group/Lawrence Erlbaum Associates, 15–44. https://www.taylorfrancis.com/chapters/edit/10.4324/9780203809747-7/team-learning-development-adaptation-steve-kozlowski-bradford-bell (October 17, 2021).
Kulkarni, Anand, Matthew Can, and Björn Hartmann. 2012. “Collaboratively Crowdsourcing Workflows with Turkomatic.” In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work and Companion CSCW ’12 : Seattle, Washington, USA, February 11-15, Association for Computing Machinery.
Li, Hongwei, Bo Zhao, and Ariel Fuxman. 2014. “The Wisdom of Minority: Discovering and Targeting the Right Group of Workers for Crowdsourcing.” In Proceedings of the 23rd International Conference on World Wide Web - WWW ’14, New York, New York, USA: ACM Press, 165–76. http://dl.acm.org/citation.cfm?doid=2566486.2568033.
Ludwig, Thomas, Christian Reuter, and Volkmar Pipek. 2016. “From Publics to Communities: Researching the Path of Shared Issues Through ICT.” Computer Supported Cooperative Work (CSCW) 25(2–3): 193–225. http://dx.doi.org/10.1007/s10606-016-9252-y.
Lykourentzou, Ioanna, Shannon Wang, Robert E. Kraut, and Steven P. Dow. 2016. “Team Dating: A Self-Organized Team Formation Strategy for Collaborative Crowdsourcing.” In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, New York, NY, USA: ACM, 1243–49. https://dl.acm.org/doi/10.1145/2851581.2892421.
Mathieu, John E et al. 2000. “The Influence of Shared Mental Models on Team Process and Performance.” Journal of Applied Psychology 85(2): 273–83.
Maynard, M. Travis, and Lucy L. Gilson. 2014. “The Role of Shared Mental Model Development in Understanding Virtual Team Effectiveness.” Group and Organization Management 39(1): 3–32.
Microsoft Live Labs. “Photosynth.” https://web.archive.org/web/20070205184512/http://labs.live.com/photosynth/) (May 30, 2022).
Mohammed, Susan et al. 2015. “Time for Temporal Team Mental Models: Expanding beyond ‘What’ and ‘How’ to Incorporate ‘When.’” European Journal of Work and Organizational Psychology 24(5): 693–709. http://www.tandfonline.com/doi/full/10.1080/1359432X.2015.1024664.
Mudg, M et al. 2010. “Principles and Practices of Robust, Photography-Based Digital Imaging Techniques for Museums.” In The 11th International Symposium on Virtual Reality, Archaeology and Cultural Heritage VAST, http://eprints.soton.ac.uk/271658/.
Müller, Rebecca, and Conny Herbert Antoni. 2021. “Effects of ICT Shared Mental Models on Team Processes and Outcomes.” Small Group Research.
Naderi, Babak. 2018. “Who Are the Crowdworkers?” In T-Labs Series in Telecommunication Services, , 17–27. http://www.mturk.com.
Nikander, Jussi, Juha Järvi, Muhammad Usman, and Kirsi Virrantaus. 2013. “Indoor and Outdoor Mobile Navigation by Using a Combination of Floor Plans and Street Maps.” In Lecture Notes in Geoinformation and Cartography, , 233–49. http://link.springer.com/10.1007/978-3-642-34203-5_13.
Nishanbaev, Ikrom. 2020. “A Web Repository for Geo-Located 3D Digital Cultural Heritage Models.” Digital Applications in Archaeology and Cultural Heritage 16(February): e00139. https://doi.org/10.1016/j.daach.2020.e00139.
Oomen, Johan, and Lora Aroyo. 2011. “Crowdsourcing in the Cultural Heritage Domain: Opportunities and Challenges.” C and T 2011 - 5th International Conference on Communities and Technologies, Conference Proceedings (July): 138–49.
Pérez-Sanagustín, Mar et al. 2016. “Using QR Codes to Increase User Engagement in Museum-like Spaces.” Computers in Human Behavior 60: 73–85. https://linkinghub.elsevier.com/retrieve/pii/S0747563216300644.
“Photo Tourism.” http://phototour.cs.washington.edu/ (May 30, 2022).
Psomadaki, Ofilia I., Charalampos A. Dimoulas, George M. Kalliris, and Gregory Paschalidis. 2019. “Digital Storytelling and Audience Engagement in Cultural Heritage Management: A Collaborative Model Based on the Digital City of Thessaloniki.” Journal of Cultural Heritage 36: 12–22. https://linkinghub.elsevier.com/retrieve/pii/S1296207417302583.
Qiu, Sihang, Ujwal Gadiraju, and Alessandro Bozzon. 2020. “TickTalkTurk: Conversational Crowdsourcing Made Easy.” In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing, New York, NY, USA: ACM, 53–57. https://dl.acm.org/doi/10.1145/3406865.3418572.
Rahman, Habibur et al. 2019. “Optimized Group Formation for Solving Collaborative Tasks.” The VLDB Journal 28(1): 1–23. http://link.springer.com/10.1007/s00778-018-0516-7.
Reiter-Palmon, Roni, Ben Wigert, and Triparna de Vreede. 2012. “Team Creativity and Innovation.” In Handbook of Organizational Creativity, Elsevier, 295–326. http://dx.doi.org/10.1016/B978-0-12-374714-3.00013-6.
Resch, Bernd. 2013. K. J. M (Ed.) Progress in Location-Based Services Progress in Location-Based Services. ed. Jukka M. Krisp. Berlin, Heidelberg: Springer Berlin Heidelberg. http://link.springer.com/10.1007/978-3-642-34203-5.
Ridge, Mia. 2020. “Crowdsourcing in Cultural Heritage.” In Routledge International Handbook of Research Methods in Digital Humanities, Routledge, 363–83. https://www.taylorfrancis.com/books/9780429671746/chapters/10.4324/9780429777028-26.
Riedl, Christoph, and Anita Williams Woolley. 2017. “Teams vs. Crowds: A Field Test of the Relative Contribution of Incentives, Member Ability, and Emergent Collaboration to Crowd-Based Problem Solving Performance.” Academy of Management Discoveries 3(4): 382–403. http://journals.aom.org/doi/10.5465/amd.2015.0097.
Ross, Parry. 2013. Museums in a Digital Age Museums in a Digital Age. ed. Ross Parry. Routledge. https://www.taylorfrancis.com/books/9781135666248.
Rubya, Sabirat, Joseph Numainville, and Svetlana Yarosh. 2021. “Comparing Generic and Community-Situated Crowdsourcing for Data Validation in the Context of Recovery from Substance Use Disorders.” In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, New York, NY, USA: ACM, 1–17. https://dl.acm.org/doi/10.1145/3411764.3445399.
Schmidt, Kjeld, and Liam Bannon. 2013. “Constructing CSCW: The First Quarter Century.” Computer Supported Cooperative Work (CSCW) 22(4–6): 345–72. http://link.springer.com/10.1007/s10606-013-9193-7.
Schuetz, Alfred. 1945. “On Multiple Realities.” Philosophy and Phenomenological Research 5(4): 533. https://www.jstor.org/stable/2102818?origin=crossref.
Simone, Carla, and Stefania Bandini. 2002. “Integrating Awareness in Cooperative Applications through the Reaction-Diffusion Metaphor.” : 495–530.
“Sketchfab.” https://sketchfab.com/.
Sohlenkamp, Markus, Wolfgang Prinz, and Ludwin Fuchs. 2000. “PoliawaC: Design and Evaluation of an Awareness-Enhanced Groupware Client.” AI & Society 14(1): 31–47. http://link.springer.com/10.1007/BF01206127.
Stathopoulou, E. K., A. Georgopoulos, G. Panagiotopoulos, and D. Kaliampakos. 2015. “Crowdsourcing Lost Cultural Heritage.” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 2(5W3): 295–300.
The Palace Museum. “Digital Heritage Database.” The Palace Museum. https://digicol.dpm.org.cn/.
Tsirliganis, Nestor et al. 2004. “Archiving Cultural Objects in the 21st Century.” Journal of Cultural Heritage 5(4): 379–84. https://linkinghub.elsevier.com/retrieve/pii/S1296207404000780.
Wang, Rong. 2020. “Marginality and Team Building in Collaborative Crowdsourcing.” Online Information Review 44(4): 827–46. https://www.emerald.com/insight/content/doi/10.1108/OIR-09-2018-0269/full/html.
Zhao, Yuxiang Chris, and Qinghua Zhu. 2014. “Effects of Extrinsic and Intrinsic Motivation on Participation in Crowdsourcing Contest: A Perspective of Self-Determination Theory.” Online Information Review 38(7): 896–917.

Competing interest reported. This research is partially funded by the Arts and Humanities Research Council (AHRC): Shaping the Connected Museum II (Grant number: AH/T011246/1).

Download PDF

Version 1

posted

You are reading this latest preprint version

Facilitating Situated Crowdsourcing of 3D Cultural Heritage via Asynchronous Virtual Collaboration

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work And System Design Rationale

2.1 Targeting at right crowd and suitable cultural heritage objects

2.2 Situated crowdsourcing for onsite image acquisition

2.3 Technology-supported Collaboration and Awareness Perception

3. Implementation And System Overview

3.1 Basic Functionalities

3.2 Open Collaboration

4. Process

4.1 Pilot Tests and Ethical Considerations

4.2 Three Crowdsourcing Approaches

4.3 Data Collection

5. Findings

5.1 Quality of 3D Reconstruction

5.2 Participation and Contribution

5.3 User Experience

5.4 Awareness Perception

6. Discussion

7. Conclusion

Declarations

References

Additional Declarations

Status:

Version 1