A Novel 6-DOF Force-Sensed Human-Robot Interface for an Intuitive Bilateral Teleoperation


 The teleoperation of a 6 degrees-of-freedom (DOF) manipulator is one of the basic methods to extend people’s capabilities in the wide variety of applications. The master interface based on the force/torque (FT) sensor could provide the full-dimension intuitive bilateral teleoperation of a 6-DOF robot since it has the ability to trigger 6-DOF command input. However, due to the force coupling, noise disturbance and unlimited input values of the FT sensor, this forces-sensed interface could not be widely used in practice. In this paper, we present an intuitive teleoperation method based on the force-sensed sensor from mechanical design and input force signal process to overcome these challenges. In this method, the input signals from the force-sensed joystick were filtered and then processed to the force commands by force limit algorithm, with the merits of anti-interference, output limitation, and online velocity adjustment. Furthermore, based on the admittance control and position controller, the manipulator could be teleoperated by the force commands. Three experiments were conducted on our self-designed robotic system. The result of the first experiment shows that the interfered force from the force coupling could be effectively suppressed with the limitation of the input force through force limit algorithm. Then, a parameter was introduced in the other two experiments to adjust the velocity online practically with force limit algorithm. The proposed method could give a practical solution to the bilateral teleoperation based on the FT sensor.


Introduction
The teleoperation of robots is an effective method to extend human capabilities in various kinds of applications where the site is inaccessible or remote, like underwater and space exploration, mining toxic materials, surgery, and rehabilitation [1]. Though the different kinds of teleoperation systems have been applied in different fields [2], bilateral teleoperation is still the common method. In terms of whether the device is touched or not when using the device, there are three types of teleoperation. The first form is the one with the tactile interface that the robot is tele-controlled by the device that the user contacts, like joysticks, gamepads, the haptic devices. The second is contactless control, where the robot can be tele-operated through visual and audio signals. The final one is the fusion form, which combines the contact and non-contact devices.
There are quite a lot of studies focusing on the telerobotic with touchable interfaces in different application fields. The joystick is a typical one since it could bring the intuitive experience on the teleoperation of 2-DOF or 3-DOF slaves to people. A haptic joystick with force-feedback was employed as a master for a 3-DOF telerobotic by Chciuk et al. in 2020 [3]. The self-designed 3-DOF joystick was validated to help the operator to better control the robot with the same dimensions during drilling. However, the low-dimension tele-control could not satisfy the requirements with high-dimensional manipulators in the other applications. A 7-DOF manipulator mounted on a wheelchair was tele-operated by a joystick through switching mode [4]. However, the low-dimension joystick could not provide ·3· full dimension control of the robot, resulting in the increase of difficult teleoperation as well as the limitation of the robot capabilities. To tele-control the high-dimension slave, the low-dimension master has usually empowered the intelligence by the algorithms. In 2016, Herlant et al. proposed an automatic time-optimal mode switching method of the joystick, which significantly improved the user experience of teleoperating a 6-DOF manipulator [5]. Besides, by embedding the robot's high-dimensional actions into low-dimensional and human-controllable latent actions, Losey et al. designed a personalized alignment model to improve the efficiency of the utility of low-dimension inputs [6]. In 2020, Wu et al. designed a smartphone-based interface to tele-control robot by using the 2D screen and 3D IMU in the phone [7]. Since only 5 user input values from the phone can be used, another dimension was activated by using a specific gesture for full-dimension teleoperation. The intelligence was embedded into the low-dimensional master to compensate for the insufficient control dimensions. In addition, several studies focused on the design and use of Multi DOF interfaces. The haptic device with characteristics of sufficient teleoperation DOF, force feedback, and the one-hand operation had been widely studied in minimally invasive robotic-assisted surgery [8][9][10]. The tactile sensing feedback has a significant effect on the performance and accuracy of expert surgeons, which would create telepresence and achieve high transparency in robot-assisted surgery [11]. Furthermore, the intuitive pose control of robot with one-hand operation is a requirement for the surgeons which makes them feel like using their own hands during surgical instruments teleoperation, leading to the full demonstration of robot motion performance and the improvement of operation efficiency [9].
Besides, in the recent decade, many other teleoperation methods without manual control, like body motion and biosignals, have also been discussed in the studies. By using the high-dimension mobilities on of human body, the body-machine interfaces were designed to provide full-dimension control for telemanipulation [12]. In most cases, these methods aim to help the operators with limb mobility impairments. Moreover, the non-invasive Brain-Computer Interface (BCI), by using the human biosignals, were designed for teleoperation of the mobile robots [13][14][15], since the biosignal is also hard to provide high dimension signals. Moreover, the manipulator could also be teleoperated by a specific command for a specific motion from BCIs [15][16], where the motion contained the high-dimension operation.
Different from tactile interfaces, the contactless teleoperation needs to process the complex information from the sensors with the comprehensive analysis. With the process of vocal or optical signals, the manipulators could be tele-controlled not only for the simple motion in the Cartesian space [17][18], but also for the specific motion [19][20]. The complicated processing of the voice and pictures empowered the manipulator with much more intelligence and helped the robot interact with people naturally. Sensors fusion teleoperation is based on tactile and non-tactile sensors [21][22]. The tactile sensors in the fusion teleoperation were usually small but were unable to provide high-dimension control directly. Therefore, the other touchless sensor was introduced to compensate for the dimensions.
Generally, the teleoperation should be intuitive, simple-operated, efficient and accurate. The accuracy of the telemanipulation is the basic requirement for the master to tele-control the robot under the user's expectation. The efficiency of the teleoperation is related to the real-time operation, so the robot would respond quickly after getting commands without any latency. Moreover, the telemanipulation should be convenient and simple for the users so that it would decrease their learning time of device utilization as well as the cognitive load during learning. Besides, if the dimensionalities of the master and the slave could match, it would provide the direct control without dimension compensation, which leads to the efficiency and simple operation. Furthermore, the control should be intuitive so that every dimension of the robot motion could set up a mapping to the master with the same dimension respectively, which would reduce false triggering of teleoperation signals from the misunderstanding of the dimension mapping. The teleoperation with the haptic wand [8][9][10] could provide an intuitive control with sufficient DOF of slave for teleoperation in minimally invasive surgery, leading to making the operators feel like using their own hands during teleoperation. To obtain haptic feedback, the motors are usually installed in the device which results in heavy device with a lot of wires. However, for most common applications, the importance of the interface size usually outweighs the tactile sensation. Therefore, how to make a small interface with intuitive, simple-operated, efficient and accurate telemanipulation for the high-dimension slave remains a challenge in the field.
In 2020, Black et al. integrated an FT sensor into a joystick for the intuitive, simple-operated and real-time teleoperation in da Vinci Standard Surgical system [24].

·4·
The joystick was used to monitor the surgeon's interaction forces which improved the haptic experience with the use of impedance control with high-fidelity and low latency. In their method, only one axis was controlled at one time by the corresponding direction in the FT sensor, as a result of the force coupling on the other axes. The mechanical design was detailed in their study since this device was embedded into the da Vinci Research Kit as a master for surgical application. However, due to the effect of the environment humidity and temperature, the noise from the FT sensor is unpredictable on each axis, leading to an unsmooth motion of the robot. Besides, the force from people could increase the uncertainty of noise generation as well, since the people are not able to perform a certain and stable force on a certain dimension of the joystick. Therefore, the original input force signals need to be filtered to eliminate the noises. Moreover, the FT sensor would decouple the force or torque into other axes to generate the component forces as a disturbance when the input force is not just applied in a certain direction. Besides, although the FT sensor has a limited measurement range with maximum input, the limitation is usually beyond the force that the user can provide so that the input force from the operator is without restriction. Considering the admittance control framework, the velocity of the manipulator is proportional to the input force. The input force without limitation has a great possibility to generate a larger velocity of the robot, which would lead to an increase in the unsafety of the telemanipulation. As a result, the limitation of the input force is also necessary during the teleoperation by using force -sensed joystick.
Although the master interface based on the FT sensor could provide the intuitive, simple-operated and efficient telemanipulation, the noise should be filtered and the force signal process should not be ignored as well to avoid the interference of coupling and the unsafety factor of unlimited input force. Moreover, from the perspective of practical use, online velocity adjustment is also essential for teleoperation when the robot intends to complete the delicate task, while it is impossible to be achieved by adjusting the input force since people could not apply a steady force continuously. In this paper, to solve the above problems by using a force-sensed joystick, we present an intuitive teleoperation method for a 6-DOF manipulator with the force limit algorithm based on admittance control framework with anti-interference, amplitude limitation and velocity adjustment. The original force signals with noises would be filtered at first. After that, the force limit algorithm which is proposed in this paper would process the force signal with the merits of anti-interference, velocity adjustment, and output limit. Then, the output force commands from the force limit algorithm would convert to the joint velocity commands based on the admittance control framework and Jacobian matrix. After obtaining the velocity of each joint, the desired position can be calculated by the current position and the product of velocity and specific time. In the following section, the method will be described in detail in the next section. In the third section, the experiments and results will be illustrated and a practical method for velocity adjustment would be validated through the water experiment, while the final section is about the discussion and conclusion.

System Design
The system (as shown in Figure 1), including the mechanical design, the signal process and the control framework, shows the process of the entire bilateral telemanipulation. The force intends to be more likely to be applied on a certain axis through the mechanical design, which would decrease the force coupling from the way of the force triggering. From the perspective of the signal process, the original force signals ori F obtained from the designed joystick is filtered as fil F by using a moving average filter and then, processed as the force commands cmd F by the force limit algorithm avoid interference, restrict the output and adjust the velocity, which intends to solve the force coupling after obtaining the forces. After that, based on the admittance control, the robot could be controlled by cmd F . The signal process based on force limit algorithm is the main contribution to solve the force coupling in the FT sensor. The detail of each module is illustrated in the subsections.

Mechanical Design
The FT sensor can decouple the input force to the other axes, leading to the interference. To specifically, when an input force is applied to the expected dimension, the force might break down on the other axes as the components, which results in the generation of the disturbance. The input force from the user is not steady and pure, so that it ·5· is very difficult to apply a certain force to a certain dimension completely. To avoid the force coupling and obtain the pure force, the mechanic design is necessary for helping the user apply force to the FT sensor on one certain dimension directly.
When the force is applied on one certain axis, the component wrenches would be easily-triggered on the orthogonal axis if the force is acting on the sensor with a distance. Besides, a pure wrench command could be triggered perfectly with 2 opposite torques on two sides along an axis with the same values and the same applied time. If the wrench is applied on one side along an axis, it would trigger a wrench command under the expectation, as well as be decoupled into a component forces on the orthogonal axis. Therefore, the joystick for triggering force commands on the three axes should be designed close to the FT sensor and the knobs for triggering the wrenches should be placed along three axes. Figure 2 shows the mechanical design of the joystick based on the FT sensor and the relationship between the joystick frame (JF) and the end-effector frame (EF). In this figure, part 1 on the joystick is a quadrangular prism for triggering forces signals in three axes, while the wrench signals on z-x-y axes could be triggered by the rotary knobs in parts 2, 3, and 4 correspondingly. Furthermore, from the relationship, three forces ( x F , y F R & ) of the end-effector referred to WF accordingly. Therefore, the user could apply the force to the joystick to tele-control the desired axes of the end effector through this mapping with intuitive teleoperation. However, the force coupling can not just be solved by the mechanical design, which is a way to reduce the interference from the force applied by the user. Furthermore, to filter the noises and diminish the interference, the original input force signals should be processed after acquirement.

Moving Average Filter
As mentioned previously, due to the influences of environment and human factors, the noise would be accompanied by the generation of the force signals from the FT sensor. To eliminate the noise in the original force signals ori F , the moving average filter is introduced to filter the original signals as follows: where M is the number of the data, and i is the current signal order. From the equation, there is a certain lag in the outputs compared to the inputs, which is dependent on M . However, the larger M is, the smoother the outputs are, but the bigger the lag is. The setting of M should keep balanced. Figure 3 shows the input forces in the x-axis with the green lines while the blue, cyan and red lines show the filter results when 30 M  , 70 M  and 100 M  .

Figure 3
The result of moving average filter with the original signal in the x-axis.

Force Limit Algorithm
This algorithm is presented to solve the force coupling described previously. Besides, the unsafety of motion should be taken into consideration by restricting the input force signals. Therefore, the force limit algorithm, inspired by the sigmoid function, is introduced can be presented as follows: Zihao Li et al.    Figure 4 illustrates part 1, while the cyan dotted line relates to part 2. In this algorithm, different settings of parameters would make a different impact on the output, which would be detailed in the next paragraph only for the case when 0 fil F  since the function is symmetry. Figure 5 shows the effects of parameters variation on the output cmd F . As for k , the variation would mainly lead to the size adjustment of dead zone. The dead zone would increase with the increase of k in Figure 5 ⅰ). The variation of b can be seen in Figure 5 ⅱ), which determines the maximum output (equals to b ). Moreover, increasing b would also magnify the dead zone. From in the dead zone, decreasing k c would make no difference when two parts overlapped. Besides, the rising rate of part 1 is generally greater than that of part 2 out of the dead zone and saturation zone, which means that a small change of fil F would lead to a large response of cmd F in part 1, while it would have a relatively slight change of cmd F in part 2. Therefore, when the force is applied to the joystick at the beginning, the robot would respond fast with a small velocity before the force reaches the TP. Then, when the filter force is bigger than the force at TP, the variation of velocity would slow down, in case of the dangerous motion with big velocity. Moreover, d is designed to affect the rising rate from the view of safety, especially after the turning point, shown in Figure 5 iv). The smaller d is, the smoother the output is.

Control Framework
The control framework is based on the admittance control while there are quite a lot of studies on this compliance control where the detail can be found in Ref. [25]. Admittance control can be implemented with the inner loop of the position-controlled system and the outer loop of the torque-controlled system, displayed in Figure  1, which can be written as follows:  x . In this mass-spring-damping system, if "mass" ( t de Mx && ) and "spring" ( t de Kx) parts are ignored, then the desired velocity of the robot end effector in Cartesian space could be obtained and the equation (3) can be given as: Moreover, the relationship between joint velocities and end-effector velocities can be given as follows: t ee x Jq  && (6) where J is the Jacobian matrix and e q & is the error between desired joint velocities and current joint velocities. Moreover, the desired joint positions d q can be acquired from the joints current position c q and the specific time t of the real-time system, which can be given as: where d q is the next command that the controller sent to the joints after the specific time t . Then, the teleoperation is complete after the manipulator responds to the command.

Setup
In this study, a robotic system with a controller and a redundant manipulator, named as THCobot, developed in our lab has been used. The controller is built and designed on ROS with a real-time operating system. Moreover, the position controller is used in every joint in THCobot where the desired joint position commands were input in real-time. The joystick based on the FT sensor is considered as a master for the user to tele-control the end-effector intuitively with full dimension control by using one hand. The data can be obtained from the master 100 times in a second. Figure 6 shows the detail of this experiment setup with WF on its base link. The robot state would display on the screen simultaneously, in case of safety accidents during the telemanipulation. Three experiments were conducted to validate the proposed method. The first experiment intended to validate the feasibility of the method to avoid the interfered force due to coupling, and restrict the output force commands. The second experiment aimed to introduce a variable to adjust the velocity online, which had also been used in the third experiment to complete a watering experiment.

Experiments
In the first experiment, the operator manipulated the robot to move or orientate in Cartesian space by applying the force to the force-sensed joystick in real-time with the ·8· Figure 7 The results of the first experiment with full dimension teleoperation.  Figure 7 shows the results of this experiment. The "Origin", "Filter" and "Limit" lines, with the y-label on the left, represent the original force, filter force, and the limited force commands from the force limit algorithm respectively. The red lines are the pose variation of the end-effector with the y-label on the right. Moreover, the filter data from the three torques had expanded 5 times before using the force limit algorithm, since the input wrench was generally small and hard to be triggered as a large value by the user. But in this figure, the orientation limited force commands had reduced 5 times for display. From this figure ("origin" and "filter" lines), it was obvious that when the user attempted to apply the force on a certain axis, the force would be decoupled and generate the components as interference on the other axes without expectation. However, also from this picture ("limit" lines), the unexpected interference was eliminated after force limit algorithm. Therefore, the user could apply the force on all dimensions of the joystick at the same time and not worry about the interference force on the other axes during the telemanipulation. Furthermore, the output force commands from this algorithm would also be limited so that the robot velocity would be restricted to guarantee the safety during the teleoperation.
The maximum output of the force limit algorithm would directly affect the robot velocity, and it is mainly dependent on parameter b . The decrease of b would decrease the size of the dead zone, which leads to reducing anti-interference. However, anti-interference is much more necessary for some delicate motions with the small velocity, like water pouring or door unlock. From the previous analysis of this algorithm, the increase of k would increase the size of the dead zone. Therefore, a variable  had been introduced for the online velocity adjustment through the following equations: From this figure, when  is large, the output is large, and the dead zone is relatively small. While  is small, the output is relatively small and the dead zone becomes larger, which improves the ability of anti-interference. Moreover, the maximum output can be obtained rapidly with a smaller input when  is set smaller (the orange dotted line in Figure 8). Figure 9 shows the results of the second experiment with the same parameters as the first experiment. The input simulated force in axis z was set as 30 N, while the input interfered force in axis x was set as 12sin(0.2 ) t N, where t is a time variable. In this experiment,  was set from 1 to 0.6 and to 0.2 online.
Then the limit output in axis z varied from 5 N to 3 N and 1 N, shown as the green line in the right graph with the movement of the robot in the red line. Moreover, from the left picture, the influence of the interfered force in axis x was reduced with the change of  online. When 0.2

 
, the interference made no effect on robot. Adjustment of  online could adjust the velocity of the end effector with the same input force, which was used in the next experiment for the delicate motion.
In the third experiment, the manipulator was teleoperated A Novel 6-DOF Force-Sensed Human-Robot Interface for an Intuitive Bilateral Teleoperation ·9· Figure 8 The effects of changing  on the output cmd F .
to grasp a jug and pour water into a glass with online velocity adjustment. Figure 10 displays the detail of the third experiment with 30 and the diagonal of the damping matrix is   100,100,100, 20, 20, 20 . The limit force commands and positions of x-y-z are shown on the left y label in the top and bottom pictures, while the limit wrench commands and rotations of x-y-z are displayed on the right one. In the beginning, THCobot was teleoperated to move close to the jug with water when 1   . Before the robot was close to the jug,  was re-set to 0.4 for decreasing the robot 2 2 Figure 9 The results of the second experiment velocity.
After the jug was grasped and moved close to the glass, the robot was tele-controlled to pour water into the glass with wrench commands on the y-axis. Then, the velocity increased by re-setting  to 1.0 after the jug was placed on the table. Finally, it was manipulated to leave away from the jug. From Figure 10 Figure 10 The results of the third experiment that THCobot was teleoperated to grasp a jug and pour water into a glass.

·10·
experiment validated that the velocity of the robot could be adjusted online by changing  with the use of the force limit algorithm.

Discussions & Conclusions
The setting of parameters is very essential to telemanipulation since it has a strong influence on the force limit algorithm and finally determines its performance. The setting is quite individual since it is dependent on the force that the user can apply to the joystick. Moreover, velocity adjustment is commonly used during telemanipulation, especially for some delicate and elaborate operations, and it could be achieved online with a force limit algorithm by changing the variable  in practice. Besides, although the velocity could also be adjusted by the damping matrix in admittance control, the experiments validate feasibility of the online velocity adjustment through a force limit algorithm with the ability of anti-interference.
However, the human-robot interaction is one of the main considerations in teleoperation. It is relative to the human sensation which has an individual difference with difficulty in quantification. In this method, the problem to influence the human-robot interaction is that the joystick is triggered by force, which would reduce the tactile sensation during operation, compared to the position-triggered interface. However, this problem might be solved by adding springs, which has a transformation from position to force, on the joystick to improve the teleoperation experience in the future. Besides, the force-sensed joystick should be fixed for the force applied on it during the teleoperation, which is inconvenient compared to the mobile joystick. However, this joystick could be well-used as a master in some situations, such as fixed on a surgical system for surgery or on a wheelchair to help the disabled or elder.
This paper presented an intuitive teleoperation method with a force limit algorithm for a 6-DOF manipulator based on the force-sensed joystick. From three experiments, the translation and the rotation of the end-effector were telemanipulated by this force-sensed joystick to validate the anti-interference, online velocity adjustment, and amplitude limitation of the force limit algorithm. In the future, the human-robot interaction would be considered mainly and it would be improved the experience of the haptic sense. Moreover, the electrical part is also necessary to be added to improve the functions of the joystick.