A bionic soft robotic glove mimicking nger actions based on sEMG recognition

Compared with rigid robots, soft robotics is more suitable to develop anthropomorphic digits that mimics the biological structures and dexterous motions of human finger. This study proposed a surface electromyogram (sEMG) sensors-based soft robotic glove system which was able to recognize the finger activities and execute the same operation via the bionic glove. Finger activities can be recognized by using electrodes sensors to monitor the electric potential variations on specific surface of the forearm muscle regions. A hybrid robotic digit was designed that utilizes pneumatic bellow actuators to satisfy the anatomical range of the finger motion in order to mimic finger action according to sEMG information. The moving trajectory of digit tip and the range motion of each joint of the robotic digit were measured in experiments under the pressure from 0kPa to 70kPa. The bionic soft robotic glove successfully demonstrated the finger action recognition and robotic digits controlling for a variety of manipulation tasks. The feasible results provided a novel technique for controlling the soft robotic glove through sEMG signals holistically and practically, and also give inspiration and guidance for multiple fingers remote operational applications.


Introduction
Soft robotic glove system has been widely explored from many aspects [1] [2] [3], including the linear or nonlinear soft materials, controlling module, and applications (e.g. gripper [4], and rehabilitation training [5]) [6]. Polygerinos et al. [7] used to develop a soft robotic glove using molded elastomeric bladders with anisotropic fiber reinforcements to overcome personal grasp problems. Trivoramai et al. [8] incorporated a rotary and bending actuator in a polyester glove that was tested among subjects for daily life assistance. Chua et al. [9] proposed a therapeutic glove device with two internal inflatable actuators to provide lateral compression on the fingers arthritis treatment. A variety of soft actuators have been adopted by these gloves according to different application requirements. Soft actuators with chambers can offer smooth and flexible bending motion, which makes them ideal to be used in soft robotic gloves for hand rehabilitation [10] [11]. A bending bellow actuator is a typical long beam-like element in which one side is longer than the other along the longitudinal direction, resulting in actuator bending [12]. To achieve different flexion motions, the bellows are partitioned into several compartments, thereby different bending angles of each chamber could be achieved by inflating with different pneumatic pressures [13]. Rotary flexible actuators can rotate its particular geometry to provide movement, which is typically used in two robotic links [14]. Single-structure actuators for fingers are easy to manufacture, but the freedom of hand motion is limited by the single direction of the movement [15] [16]. Therefore, it is necessary to develop flexible soft actuator with high degrees of freedom to satisfy versatile finger motions controlled by the robotic glove system.
The finger range of motion (ROM) is the commonly measured parameters in finger motion relevant studies [17] [18] [19]. The bending angle of the robotic digit are usually measured through experiments or analyzed via mathematical and finite element models [20] [21] [22]. For actuators with complex structures and multiple materials, the bending angle is given by experimental testing and data curve fitting [23]. The mathematical model is difficult to construct because of the highly nonlinear characteristics of the applied material and the complex coupling between the human fingers and actuators [24] [25]. Additionally, finger exoskeleton need to be studied due to the interactive relation between fingers and wearable robots [11] [26]. With the development of soft robotic technology, the traditional rigid mechanical and single-structure designs have gradually transitioned to soft-actuated and bionic finger musculoskeletal structure designs [27] [28] [29]. Wearable soft robotic gloves with basic function of hand rehabilitation have been developed to improve the daily life quality of people with hand trauma or neurological impairment [30] [31] [32]. However, the bionic design of an anthropomorphic digit and its simultaneous mimicking ability for dexterous motions of human finger are still long-term challenges for researchers.
Surface electromyogram (sEMG) is a typical medicine technique for collecting muscular electrical signals of skeletal muscle activities via skin surface electrode sensor, which has been applied in the wearable devices for medical and healthcare applications [33] [34] [35]. Myoelectric pattern-recognition techniques have been developed for finger motion intentions identification based on EMG information which would be applied in controlling a prosthetic robot to mimic the recognized intentions [36] [37]. For instance, Lu, et al. [38] developed a wearable gesture sensing device worn on the forearm to manipulate a mobile phone using 19 predefined gestures or even personalized ones, and Feng et al. [39] realized the online sEMG control of the soft robotic hand to reproduce the gestures behavior of human. Although numerous EMG-driven robots and exoskeletons have been designed and used for neurorehabilitation [40] [41], the combinational work mechanism of finger curvature recognition using sEMG sensors for controlling soft robotic glove to mimic finger action should be explored further and optimized.
This study aims to design and develop an sEMG-sensor-based soft robotic glove bionic system for finger curvature recognition and action mimicking. Surface EMG sensors were stuck on the forearm muscles in five different locations in corresponding to five fingers, and the intensity of sEMG signals for each active finger was measured for recognizing the fingers activities (e.g. flexion, extension). The proposed soft robotic glove system consists of five hybrid robotic digits, which were constructed by pneumatic bellows with flexible chambers; rigid blocks to fix bellows; and a glove base to provide support. The moving trajectory of the digit tip and the range motion of each robotic joint were measured in experiments under various constant pressures. To mimic finger actions, a pneumatic feedback controlling unit that integrated into the sEMG monitoring module was developed for the precise control of the dynamic pressure dosages. Performances of the developed soft robotic glove with sEMG sensor monitoring system were examined on subjects through a series of finger active tasks.

Methods and fabrication
The bionic soft robotic glove The digit of bionic soft robotic glove was engineered with a certain number of pneumatic bellows that configured in alternating order, rigid blocks located on both sides of pneumatic bellows, and glove base connected two rigid blocks under the flexible bellows chambers. When air pressure is inflated in the bellows chambers, the upper part of the bellows is free to expand and the lower part is fixed by the glove base that limits the expansion of the bottom part of the bellows, thus the bending motion occurs as shown in Figs.1(a-b). The pneumatic bellows and rigid blocks were installed individually outside the glove base according to the natural skeletal structure of human fingers as shown in Fig.1 (c). A soft robotic glove which composed of five hybrid robotic digits was fabricated and presented in Fig. 1(d). Fig. 1(e) shows the detail structure of hybrid robotic digit using three sections of pneumatic bellows for the index, middle and ring finger. Robotic digits correspond to the small finger and the thumb consisted of two pneumatic bellows. Thereby, two types of hybrid robotic digits were engineered in the glove that allow all fingers bend together on the same plane. The finger sensing and robotic controlling system Hand exercises require more than 20 muscles to work together [42], and each finger involves specific muscle active region [33] [43]. The traditional sEMG sensors with three disposable electrodes were adopted in this study to collect myoelectric signals, and the corresponding data acquisition (DAQ) board was developed. Two active electrodes were stuck on active muscle region and a reference electrode was attached closely to the elbow where observed minimum muscle electric potential during finger activities [42] [44]. Five different muscle regions on the forearm were identified for sticking electrodes as illustrated in Fig.2(a) to monitor particular finger activities. The selected regions (1) to (5) were considered have the maximum intensity of sEMG signals for the activities of the thumb, index, middle, ring and small fingers, respectively.
The hardware platform of the sEMG detection module mainly consist of three parts: a STM32 chip (STMicroelectronic, Geneva, Switzerland) for controlling and data analysis, five precision instrumentation amplifiers (AD8221, Analog Devices Co., Ltd.) for signal processing, and a BLE module for data communication with remote device [45]. Since the dimension and depth of the muscles contracting underneath the electrodes differ, the produced amplitude of the original sEMG signals may vary from uV to mV [46] [47]. The differential input of the precision n instrumentation amplifiers was used to amplify the sEMG signals for STM32 chip acquisition. The full-wave precision rectifiers were applied to filter the noise with the amplified signals, and operational amplifiers were applied to amplify them again [48] [49]. The hardware processed sEMG signals were delivered to the STM32 chip for converting digital value, and the obtained data were sent to a remote receiver via the BLE module for further analysis and finger activity recognition. Fig.2 (b and c) presents the whole soft robotic glove bionic system, which consists of an sEMG monitoring module, a bionic soft robotic glove with hybrid digits as an execution module, and a controlling module with pneumatic tubes connecting to controller. The controller as shown in Fig. 2(c) was composed of a pneumatic pump (model no. KPV04, max pressure 75 kPa), a data acquisition control board (STM32F103 chip) with BLE module, differential pressure sensors (model no. MPX5100DP, Freescale Semiconductor Co.), and valves (SMC V124A, Sintered Metal Corporation, China agent). Fig.2 (d) shows the preset control scheme of robotic digits. The sEMG signals of each finger were collected and analyzed to give controlling commands. Simultaneously, the microcontroller of the bionic glove regulates the pressurization level of the hybrid robotic digit to mimic finger action. In the pressure controlling progress, to obtain the precise pressure for finger movements, the pulse width modulation (PWM) wave [50] generated by the robotic glove controller was applied to open and close the valves via the proportional-integral-derivative method [51]. Additionally, differential pressure sensors were adapted on each hybrid robotic digit to collect pressure feedback.

Analysis on finger range of motion
The pneumatic bellows of hybrid robotic digit should follow the ROM of the human finger to achieve finger flexion and extension comfortably and generate adequate force to bend the finger properly. A simple interphalangeal configuration was adopted for the motion analysis with the kinematic model presented in Fig. 3 (a). References were set up based on the finger frames and assigned to this model to depict the bending actions of joint points and the tip point C. The position variation of the digit tip point C in bending direction can be described using its components as follows: , , where θ1, θ2, and θ3 are the first, second, and third finger joint bending angles, respectively, which are related to the supplied pressure, and L1, L2, and L3 are the lengths of DIP, PIP, and MCP, respectively. The index robotic digit with three pneumatic bellows was taken as an example to show the working progress from the initial stage to the bending stage as well as its moving trajectory (Fig.3b). Six markers were attached to the hybrid robotic digit base and formed the levers of MCP, and PIP, and DIP joints. The moving trajectory and the ROM of each joint were measured by digit bending experiments, in which different points were tested under the pressure from 0kPa to 70kPa, repeated for three times.  Additionally, the relative error variations of the flexion angles were calculated to explore the consistency of bellows under repeated pressure test. The relative error can be determined as at a certain supplied pressure, where ∆θ indicates the maximum deviation of the flexion angle and θ0 represents the average flexion angle. The relative errors towards the MCP, PIP, and DIP joints are 7%, 9%, and 5% respectively at the supplied pressures from 10 kPa to 70 kPa, which are less than 10% presenting an acceptable result. However, when the pressure varies from 0 kPa to 10 kPa, the relative errors are larger than 10 % because of the need for start pressure in the bellows which could be called the "pressure dead zone" that has been observed in similar studies [52] [53]. Similarly, the working progress of a specific sized digit with two bellows from the initial stage to the bending stage was explored by the same experiments and presented in Fig. 4(a). Figs. 4(b-c) show the flexion angles of MCP and DIP joints were 57° and 75° respectively at the approximate max air pressure of 70 kPa. The moving trajectory of the digit tip (locations of the reference (0, 0) and tip (0, 70) points) was illustrated in Fig.4 (d). When the supplied pressures varied from 10 kPa to 70 kPa, the flexion angle is observed to be larger than 90°, the curves are approximately linear, and the relative errors for the DIP and MCP joints are less than 10%. The results indicated that the two-bellow hybrid robotic digit also provide good motion accuracy that can be controlled precisely for finger action simulation.

Finger sEMG and robotic digit controlling
Although the hardware circuit smoothened the sEMG signal, the raw data outputted by the data acquisition board are not suitable for analysis due to the real-time noise and offset [54] [55]. Refer our previous study [56], a normalization process for the root mean square (RMS) of sEMG signal was engineered to reduce individual differences as follows: where RMSN is normalized, and RMSM means the maximum root mean square of sEMG. The maximum RMS value was determined before the experiment via repeated tests under the maximum state. A group of finger motions were designed and tested with the analysis of RMSN towards each finger as well as all fingers in a continuous action (Figs. 5). The results demonstrated that for the flexion and extension actions of each finger, the activity intensity of each muscle was significantly different. For real-time gesture monitoring and analysis, classical linear classifiers have been widely applied due to the low computational complexity and good real-time performance [57] [39]. Therefore, in this study, a linear discriminant analysis classifier is adopted to recognize finger versatile action. The average recognition rate (AVRR) of five fingers from extension to flexion in small, ring, middle, index, thumb are 97%, 95%, 95%, 96% and 98% respectively. An electro-pneumatic unit was assembled into a controlling box to provide isolated control for each digit of the glove while differential air pressure sensors were used to monitor the pressure of each digit. Fig. 6(a) illustrates the closed-loop working diagram of a robotic digit, which contains the microcontroller that provides the control orders, the solenoid valves that switch the inlet and outlet flows, and the differential pressure sensors that monitor the air pressure inside the robotic digit actuator. When there is an obvious difference between the desired and measured pressure signals (from 0 kPa to 50 kPa), the microcontroller outputs the PWM wave to the solenoid valve and regulates the hybrid robotic digit pressure (p) to track Pref as presented in Fig. 6(b). The solenoid valves are opened at a high-level voltage and closed at a low-level voltage with a peak PWM frequency of 20 Hz because of the nominated response time of 50 ms. Therefore, a frequency of 20 Hz could be output by the control unit, which is higher than the mechanical frequency (0.5 Hz) of the required actuator.
To evaluate the dexterity of the single and multiple robotic digits and the performance of the controlling unit, a series of the robotic digit actions were set up as shown in Figs. 6(c-j). For examples, to achieve the action of tripod pinch grasp (Fig.  6j), the valves of the thumb, index, and middle digit and the pump would be open for inflation while other digits remained unpressurized.

Soft robotic glove mimicking finger action
The implemented control scheme of soft glove system for each hybrid robotic digit was shown in Fig. 2(d). The working logic hierarchy of the developed system allows the sEMG sensor to continuously monitor and controls the glove in advance by comparing the collected signals, e.g. set the predefined conditions based on the initial experiment to determine the threshold parameters of the user's muscle intensity. Three predefined conditions of a) 'flex', b) 'extend', and c) 'hold' would enable the glove to achieve flexing, extending, or holding in advance and reduce the glove operation delay time. Six combinational finger gestures were designed for the soft robotic glove mimicking based on the sEMG signals (Fig.7). A pneumatic plastic glove filled with 1kPa air pressure was adopted inside a soft textile glove for supporting the shape and providing a small resistance during mimicking actions. Table 1 shows the average recognition rates (AVRR) are above 90% for six types of combinational finger gestures.  The mimicking results of the bionic soft robotic glove successfully demonstrated the finger action recognition and hybrid robotic digits mimicking controlling. The hybrid architecture under pneumatic pressure control provides specific mechanical characteristics and the glove base serves as the connection and also facilitates force transfer [23] [14] [58]. Specifically, single finger motion recognition rate (all above 95%), which based on the sEMG signals collected from the aforementioned selected regions as presented in Fig. 2(a), seems to be higher than that of combinational finger gestures that may due to the interference of multiple muscles in simultaneous activities [59] [60]. This bionic robotic glove system worth further investigation which could be applied in stroke hand rehabilitation training for assisting in variety of manipulation tasks.

Conclusions
This study proposed a bionic soft robotic glove with the finger motion recognition and robotic digit controlling functions based on sEMG signals to achieve finger action mimicking. Given the complexity of the coupling interaction between the finger and hybrid robotic digit, the natural skeletal structure of human finger was considered within the glove design in order to achieve perfect compliance. The bionic soft robotic glove composed of five hybrid robotic digits which was connected by a certain number of pneumatic bellow actuators to satisfy the anatomical range of finger motion for each joint. Therefore, controlling the air pressure of each digit can realize the desired finger bending angles, and the combinational control of different hybrid robotic digits is feasible to mimic human finger actions in daily life, such as grasp and fist. Surface EMG sensors were attached to the specified muscle regions in the forearm where have the maximum sEMG signals intensity for each finger. Thus, depending on the sEMG characteristics of the selected muscles, hybrid robotic digit mimicking action can be controlled following specific muscle intensity commands. A series of hand activity tasks were design to evaluate the performance of the robotic system with the results indicating that the developed system could satisfy the desired motion characteristics. Many operation applications with multiple fingers could be built based on this wearable sEMG monitoring and glove mimicking system avoiding external finger model simulation calculation and measurement issues. In further study, experiments would be performed among a wider range of participants to improve the AVRR of finger activities and to investigate the robustness or other influences of the robotic glove controlling algorithms.        The sEMG analysis information of ngers: (a-e) Designed hand motions; (f-j) the RMSN of sEMG towards each nger; (k) the continues RMSN signal of all ngers in a bending action.  Six combinational nger gestures for mimicking: (a) Fist; (b) "Two", (c) "Six", (d) tripod pinch grasp, (e) "OK", (f) orchid nger.