This research aims to innovate a method called NewRelics for interactive installation development, making traditional relics present new forms. This enhances the learning of museum visitors with a multisensory interactive experience. The method realizes the installation's appearance by reshaping the features of the relics with innovative cultural symbols. Meanwhile, the approach enables kinetic sculpture and the simulation of artifact sound through the design of mechanical structures. Thereby a multisensory integration and interaction between tactile, visual, and auditory senses are achieved. The installations developed based on this method have a variety of interaction modes that facilitate user communication and experience. In this paper, NewRelics contributed to the museum's education by creating interactive installations that can be manipulated, which let visitors make a profound impression on the cultural relics and its history, thus improving museum their overall experiences and education effect.
NewRelics method framework
Based on the NewRelics method, each cultural relic can be transformed into an interactive installation representing its characteristics. Users can operate the installation to learn about culture and enhance communication. The development of the NewRelics approach followed three steps: characteristic shape, structural composition, and interaction mode. Figure 1 illustrates an overview of the NewRelics.
Step 1: Reshape the cultural characteristics of the installation. In this step, the relics facing the installation are determined, and its relics' characteristics will be shown. The style of the main pattern is determined by the extraction of the outer contours, the arrangement of the secondary patterns, and the comparison of different geometric evolutionary methods. Moreover, further deriving the external pattern and then selecting the color and material scheme to derive the overall representation of the appearance.
Step 2: Design the mechanical structure of the installation. In this step, we built a mechanism for rotating sculpture patterns and generating sounds based on traditional heritage elements. The kinetic sculpture is based on a rotating structure with gears turning to achieve dynamic changes. Furthermore, set the sound structure based on the relic's material and the traditional scale to complete the simulation of different artifact tones and scales.
Step 3: Build the interaction modes of the installation. The installation is designed to be interacted with in one of two ways: individually and with a group. For individuals, the focus is on the entertainment and educational value of the experience. For a group, the focus is on fostering cooperation and communication. The interaction of installations is determined based on these two focuses.
NewRelics demonstration for Jing Chu culture
In this section, we illustrated the development process of the NewRelics method by taking a representative artifact from the Jing Chu period as an example. Thereby a prototype interactive installation of the Jing Chu culture was finalized.
Styling features
Artifact selection. China's Yellow River and Yangtze River civilizations represent one of the four ancient civilizations along with ancient Babylon, Egypt, and India (Caiwu and Rong, 2013). Jing Chu culture was a crucial part of the Yangtze River civilization, whose brilliant achievements include bronze casting, silk embroidery, lacquering, prose, and music (Ruihong and Lisheng, 2013), with more than 260 000 cultural relics, Hubei Provincial Museum is a window onto Jing Chu culture. It is not only a center for archaeological exploration, excavation, protection, and research but also the most significant exhibition hall of ancient musical instruments in China.
In order to maximize the knowledge about cultural relics through the installation, it is necessary to show the artifacts' characteristics on the installation's appearance. Original objects are used as a database for exploring the installation's visual elements, and the characteristic features of the relics are extracted. As shown in Figure 2, photos that depict the visual elements of Jing Chu culture artifacts were used to create a dataset (Hubei Provincial Museum, 2013).
Three collections were selected to ensure representation of the far-reaching history and wide influence of Jing Chu culture in Hubei Province: the Bianzhong of Marquis Yi of Zeng, Tiger bird frame drum, and Four Blue-and-White Love-Figure Plum Bottles from the Yuan Dynasty (hereafter, these artifacts will be referred to as the Bianzhong, Tiger-Bird Drum, and Yuan Plum Bottles, respectively).
Feature reshaping.
(1) Contour extraction. We adjusted the threshold of the contour photos of the relics to obtain a rough extract as a line draft. While ensuring the integrity of the image's lines and improving the color block problem for the same material as the lines, we reduced the impact of the color blocks on the extracted lines of the cultural relic as much as possible. The contour of the artifacts was printed above the base of the installation to highlight characteristics of the cultural relics.
(2) Vice pattern layout. After clarifying the line extraction method and obtaining the base pattern, we need to extract further patterns and apply them to the formation of vice patterns. The extractable elements of each cultural relic were illustrated in Fig. 3.
The extracted edge patterns are integrated and summarized to determine the outer edge of the base and vice pattern.
(3) Main pattern research. The representative patterns underwent local amplification, and lines with unclear connections or broken contact were connected to ensure the fluency of the lines and obtain clear images of the main patterns. To maintain a connection with the original patterns after they were transformed, we performed change processing on the extracted patterns based on basic methods of geometric transformation: translation, rotation, and reflection (Wang et al., 2005), three styles of graphics were obtained (Fig 4).
The translation style was rendered by copying the basic pattern to form the main pattern shape. The upper part of the main pattern was moved vertically, and it presented the effect that the main pattern slides smoothly up and down. The main pattern of the rotation style was formed by copying the basic pattern around the center. The main pattern would be rotated around the center point and have the dynamic characteristic of divergence or contraction. The reflection style was presented by replicating the basic pattern symmetrically with a central axis, resulting in an overall master pattern. The main pattern would gather and expand symmetrically in the middle, creating an undulating dynamic in space.
As a symbolic reproduction of the heritage elements, the innovative pattern needs to achieve the unity of design expression and design connotation in design semiotics (Yuxiao et al., 2005). To further filter out the main pattern that best represents the relic elements, we used the semantic difference method questionnaire for experimental evaluation of the design expression (Heise, 1969). Respondents to the questionnaire received different degrees of external stimulation. They were able to state their feelings by responding to a questionnaire. The questionnaire transforms the perceptual cognition of the participants who observed the dynamic patterns into quantitative data to provide a reference for pattern selection. We used three elements from common visual design elements in the description of semiotics (shape, form, and pattern) (Qixiang and Changhong, 2020). The Likert 5-point scale (1 = strongly disagree to 5 = strongly agree) was used to rate the three styles. A total of 144 questionnaires were collected. We analyzed the reliability and validity of the participants' data according to the average score of each element when watching different styles. The calculated Cronbach's alpha was 0.884, the Kaiser-Meyer-Olkin (KMO) measure was 0.736, Bartlett's spherical test was 19.010, and the corresponding two-tailed significance test was less than 0.05. Figure 5 shows the average scores for each term.
According to the perceptual questionnaire score, all three styles show almost the same degree of the pattern. In the other two evaluation indexes, the mean score of the rotating style is significantly higher than the other styles. The combined scores show that the semiotic expression of the rotation type is more representative of the relic elements for the subjects. That identifies the main pattern style as the rotational style.
The placement of vice and main patterns followed the design connotation, which is the historical meaning of the artifact. For example, the Tiger-Bird Drum is an important musical instrument of the State of Chu during the Warring States period, consisting of two crouching tigers as the base, with a phoenix bird on each tiger's back (Xueyu, 2013). Some scholars believe that the tiger element (vice pattern) depicts the enemy state of Ba; the phoenix pattern (main pattern) represents the Chu, with the phoenix bird above the tiger pattern at the upper end of the drum frame, indicating the suppression of the enemy state (Can, 2019). The cultural relics are arranged by design connotation for the main patterns and vice patterns, which helps us to realize the reshaping of artifact features (Fig. 6).
Material and colour matching. The container for the installation was made of red oak, Brazilian red pear, and other wood combined with acrylic materials. Wood absorbs ultraviolet rays, softening the light and reflecting infrared rays, giving a warm and smooth feeling (Duan and Lang, 2010). In addition, wood is highly resonant, supporting the acoustics needed for the sound element of the installation (Evans et al., 2020). After the materials were determined, the container was made from 30° inclined trapezoids. Moreover, the side and back were engraved with characteristic patterns to stimulate from all angles visually.
The external surface of the installation utilizes patterns and colors that are characteristic of Jing Chu culture. Color conveys profound cultural messages. Wood tones were selected because wood is an elastic-plastic material with neat and primitive visual effects, a mild touch, and UV absorption (Yijie, 2021). Additionally, the mythical origin of the Jing Chu royal family was that they were the offspring of Zhu Rong, the God of fire (Wei, 2008). The fire is red, so the Chu people loved red (Wuxiang, 2012). Such as objects in Marquis Yi Zeng's tomb (Jinggang, 2013), like clothes and hat boxes, were made of red lacquer wood. This color has been widely used in the lacquerware of representative cultural relics' of the Jing Chu culture. Therefore, we added red to highlight the nature of Jing Chu culture.
Structural composition
Evaluation of rotation rate and direction. After reshaping the appearance of the cultural relics preferred and recognized by the users, we needed to further the effects of their rotation. This section describes the experiments conducted on rotation direction and speed and how a scheme was decided upon. Ensure that the installation pattern has an ideal external presentation so that visitors feel comfortable and attracted when viewing the dynamic pattern, thus enhancing visitors' knowledge of the artifact. We conducted an experiment to evaluate the impact of rotation direction and speed, selecting ten subjects to participate who had visited the museum and were interested in interactive design.
Patterns could rotate in the same direction (identical rotation) or different directions (opposed rotation). These two different dynamic patterns will give a different visual focus. Figure 7(a) shows the animation sequence in two rotated forms. And the eye-tracking recordings revealed that the user's visual focus of gravity is in the pattern's center during opposed rotation. In contrast, the visual focus is at the edge during identical rotation.
To eliminate the influence of different independent variables, we tested the rotational direction of the patterns. In Experiment 1, the subjects watched two videos containing identical and opposed rotations. Subjects watched the videos in different orders, reducing the order error of the experiment. Since the perceptual indices of the two rotation modes are similar, we chose an objective measurement method to evaluate the psychological impact of different rotation modes on the subjects by evaluating their heart rates. The higher a subject's average heart rate, the stronger the physiological response to the visual stimulation (Pollatos et al., 2020). We recorded the subject's heart rate every 12 s through a heart rate sensor over 2 min; each subject generated 10 records during this process to judge the attractiveness of the rotation direction. The directions of rotation differed, and the pattern impressions were inconsistent. The positive degree of the subjects' first impression of the video is positively correlated with their heart rate. Since the style of rotation was selected by the perceptual questionnaire, users expressed approval for both rotation styles. Figure 7(b) shows that the change in heart rate was higher for the opposed rotation direction. We interpreted this as greater enthusiasm for the opposed rotation on the part of the subjects. Therefore, we selected opposed direction pattern for further evaluation.
In Experiment 2, subjects were required to watch five videos. The rate of 10 revolutions per minute was shown at speeds of 0.5×, 0.75×, 1×, 1.25×, and 1.5× the original. Different subjects watched the videos in different orders to reduce sequence errors. The heart rates of the subjects were recorded as in Experiment 1 (Fig. 8). It can be seen that the user's heart rate range is 90-100 b/min at 1.25× and 1.5×. A normal heart rate range should be 50-90 b/min (Yanhong, 2018). If the rotation speed is too fast, visitors may feel vertigo and nystagmus due to vestibular stimulation, which will hinder the interaction with the artifact (Mian et al., 2013). Within the otherwise normal quiet heart rate range, knowledge of heart rate shows that the most potent visual stimulus can be given to the user at 1x. That affects the generation of mental imagery and thus stimulates the user's imagination (Brogaard and Gatzia, 2017). Therefore, we chose a rate of 10 revolutions per minute, represented by red line above, as the ideal rate for the installation.
Rotating and sounding mechanism. After determining how the pattern rotates, the corresponding mechanical structure was designed as shown in Fig. 9. The rotating structure controlled the rotation of the kinetic sculpture. Because the kinetic sculpture adds to rotate in opposite directions on the same shaft, it is necessary to ensure that patterns moved in a uniform circular motion while visitors were operating the installation. The automatic rotation occurs if no one is operating it. The rotating structure is divided into left and right gears, one at the bottom and three that are meshed to each other; the solid and hollow shafts have the same axis; the left gear is connected to the solid shaft while the right is connected to the hollow shaft. When the motor drives the left gear to rotate forward, the right g rotates in the opposite direction according to the basic principle of gear transmission. Therefore, the solid and hollow shafts have both positive and negative rotations (Xiaocen, 2014).
To enable installation to simulate the sound of the material of the artifact itself, we designed the sound generating mechanism. That sounding mechanism composes primarily of reduction motors, shift rods, springs, paddles, and bells. The reduction motors drive the paddle to rotate. During rotation, the paddle moves the spring to swing up and down, and the spring strikes the plectrum and bell above to make a sound, thus, a scale is played. The different materials of the paddles produce different tones, enabling the simulation of the sound of striking different artifacts. At the same time, the thickness of the paddles can be controlled to produce different scales, which are based on the traditional Chinese pentatonic scale. These notes correspond to the contemporary scales of C, D, E, G, and A on the piano.
Interaction mode
In order to preserve cultural relics, visitors are not permitted to touch them during a museum visit (Pistofidis et al., 2021). Nevertheless, we further decreased the distance between visitors and the cultural relics by associating the music with the visual element, helping them better understand the culture. As shown in Fig. 10, the interactive installations of different cultural relics correspond to different sound compositions: the bells of the Bianzhong, the percussion of the Tiger-Bird Drum, and porcelain percussion, similar to a triangle of the Yuan Plum Bottles. In ancient times, the pitched bells of the Bianzhong were arranged according to size on a wooden rack. A small mallet would be used to strike the bells in a rhythmic pattern. Innovating upon traditional use, visitors can rotate the installation to produce the tones of the scale while also rotating the vice and dynamic patterns. The rate of rotation of the percussion module controls the rate of sound production. Though a single interactive installation plays only a specific series of notes, a fuller sound can be created when multiple installations are used at once, switching the installation from single player to multiplayer. The intent is to recreate a scene from the Jing Chu period and thus deepen visitors' interaction with Jing Chu culture.