Modeling and analysis of upper limb kinematics in sign language recognition

doi:10.21203/rs.3.rs-2473127/v1

Download PDF

Research Article

Modeling and analysis of upper limb kinematics in sign language recognition

https://doi.org/10.21203/rs.3.rs-2473127/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Analysis of sign language characteristics is of great significance to the establishment of sign language recognition systems. However, a unified standard is required to evaluate whether the collected and analyzed characteristics are complete and can effectively characterize sign language gestures. In this study, a complete upper limb kinematics model is established for sign language based on the Denavit–Hartenberg (DH) robotics kinematics model; the expression of key nodes and the determination method of model parameters based on upper limb sign language are analyzed; the inverse kinematics solving equation is established according to the actual sign language expression, and the correctness of the platform inverse kinematics equation is verified. This study establishes a theoretical basis for the application research of the platform so as to provide model support for judging the completeness of the characteristic analysis.

sign language recognition

kinematic analysis

Sign language is the main language in the daily life of deaf-mute individuals and is also one of their primary methods of communicating with others. Since the 1990s, many scholars have been engaged in the study of sign language recognition, aiming to convert sign language gestures into text or voice information that is easy for non-disabled people to understand using computer programming. There are two crucial issues in the field of sign language recognition [1]. One is how to design characteristics of sign language gestures and obtain corresponding representation data; the second is how to build a recognition model for characteristics of sign language gestures. For the second question, the modeling and recognition of sign language is mainly based on speech recognition, combined with some algorithm models in machine learning theory.

Most studies on the design of characteristics of sign language gestures are based on data obtained by a sensor to further analyze the characteristics reflected by the original data; that is, using a reverse process to infer the characteristics through measured data. However, due to the lack of directionality in the collection process, there is no effective basis for judging whether the collected data contains all or part of the key features required for effective identification [6]. Hence, it is necessary to determine in-depth requirements for the expression of sign language in sensor networks and to establish a complete sign language movement equation in order to clarify all or key necessary information contained in its sign language expression. This will provide a basis for subsequent judgments on whether the information collected by sensor data is complete. Relevant scholars have carried out a partial analysis of the state of sign language movement through a combined method with sensor data or using sign language shape representation. For instance, from the perspective of expression of sign language, the hand movements of sign language can be classified in detail. The characteristics of sign language have been extracted and classified using appearance information such as chirality [2][3] through visual information. By non-visual sensor collecting action characteristics, the information containing characteristics of sign language have been obtained by focusing on the analysis of single or multiple collection points [4][5]. However, the above-mentioned studies are still reverse studies, inferring the characteristics of sign language through the collected data, and there is no clear method for whether the information is complete or how to judge whether it is complete.

Based on the motion analysis method and starting from the overall situation of the upper limb, we establish a complete upper limb kinematics model and analyze the overall situation of the sign language movement to obtain information containing complete characteristics of sign language, thus providing a basis for judgment in the feature extraction of the sign language recognition system.

The composition of sign language gestures is generally divided into hand shape (DEZ), location (TAB), and movement (SIG) [7]. Herein, hand shape is represented by the position and movement of the palm or finger. Hence, if the position and movement of the target can be clarified, the sign language gesture can be completely expressed in theory. Therefore, further modeling and analysis of the moving target are needed.

The kinematics model must first be selected. The motion links involved in sign language gestures are all rigid links; that is, the length of each rod will not change. Except for the shoulder joint and the wrist joint, the other joints will only move in one direction [8]. Based on the above two characteristics, a model for sign language gestures can be established using the Denavit–Hartenberg (DH) model applied to robotics motion analysis.

Compared with other motion models, the DH model has more mature theory, the modeling process is relatively simple, and for a single joint point, only one parameter will change, which reduces the amount of calculation in the simulation process and is more conducive to the need for simple models of sign language gesture analysis which is only an intermediate process of sign language recognition.

3.1 Homogeneous transformation

In the process of kinematics modeling, a four-dimensional homogeneous matrix is often used to represent the coordinate transformation between two adjacent coordinate systems in a three-dimensional space.

As shown in Fig. 1, θ is the rotation angle around the Z-axis, d is the distance between two adjacent common perpendiculars in the Z-axis direction, a is the length of each male perpendicular (in most cases, it is the length of the connecting rod), and α is the angle between the adjacent Z-axis (usually 0 or 90°). Next, the change relationship from joint n + 1 to adjacent joint n + 2 is calculated, and the solution is divided into four steps.

(1) Rotate θn + 1 around the Zn axis to make Xn and Xn + 1 parallel, and the transformation matrix is Rot (Z, θn + 1);

(2) Translate dn + 1 along the Zn axis so that Xn and Xn + 1 are collinear, and the transformation matrix is Trans (0,0, dn + 1);

(3) Translate an + 1 along the Xn axis, so that the origins of Xn and Xn + 1 coincide, and the transformation matrix is Trans (0,0, dn + 1);

(4) Rotate αn + 1 around the Xn + 1 axis so that the Zn and Zn + 1 axis are collinear, and the transformation matrix is Rot(X, αn + 1).

Herein, the coordinate system n and n + 1 coincide, and the four transformation matrices are multiplied on the right to get the total transformation matrix T (Sθ = sinθ, Cθ = cosθ).

$$=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{n+1}& -S{\theta }_{n+1}C{\alpha }_{n+1}\\ S{\theta }_{n+1}& C{\theta }_{n+1}C{\alpha }_{n+1}\end{array}& \begin{array}{cc}S{\theta }_{n+1}S{\alpha }_{n+1}& {a}_{n+1}C{\theta }_{n+1}\\ -C{\theta }_{n+1}S{\alpha }_{n+1}& {a}_{n+1}S{\theta }_{n+1}\end{array}\\ \begin{array}{cc}0& S{\alpha }_{n+1}\\ 0& 0\end{array}& \begin{array}{cc}C{\alpha }_{n+1}& {d}_{n+1}\\ 0& 1\end{array}\end{array}\right]$$

$$=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{n+1}& -S{\theta }_{n+1}\\ S{\theta }_{n+1}& C{\theta }_{n+1}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& 0\\ 0& 1\end{array}\end{array}\right]\times \left[\begin{array}{cc}\begin{array}{cc}1& 0\\ 0& 1\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& {d}_{n+1}\\ 0& 1\end{array}\end{array}\right]\times \left[\begin{array}{cc}\begin{array}{cc}1& 0\\ 0& 1\end{array}& \begin{array}{cc}0& {a}_{n+1}\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& 0\\ 0& 1\end{array}\end{array}\right]\times \left[\begin{array}{cc}\begin{array}{cc}1& 0\\ 0& C{\alpha }_{n+1}\end{array}& \begin{array}{cc}0& 0\\ -S{\alpha }_{n+1}& 0\end{array}\\ \begin{array}{cc}0& S{\alpha }_{n+1}\\ 0& 0\end{array}& \begin{array}{cc}C{\alpha }_{n+1}& 0\\ 0& 1\end{array}\end{array}\right]$$

Through homogeneous transformation, the coordinate system of all related nodes is expressed through the initial coordinates so that the coordinates of all points are unified into one coordinate system. Thus, the posture change of each point is transformed into a pure angle change, which is a solvable motion equation.

3.2 DH modeling

DH modeling is a classic standard method for the modeling of robotics motion. Its basic idea is as follows: each joint will consolidate a coordinate system according to a certain rule from the first joint, then the transformation matrix from the previous coordinate system to the next coordinate system is determined, and these adjacent coordinate transformation matrices are multiplied on the right in turn to obtain the total transformation matrix. Using the total transformation matrix, we can obtain the expression of a posture in the known space relative to the end effector coordinate system in the base coordinate system, which is the solution of the forward kinematics problem. On the basis that the forward kinematics problem is solved, the inverse problem of the above forward kinematics problem can be solved by the geometric method based on the spatial correspondence between the coordinate systems or the solving method of directly solving the equation set, which is the inverse kinematics problem.

3.3 Modeling of upper limb kinematics

First, the number of joint points in the model is determined. According to the human body structure, there are six connecting rods with constant length in the upper limb part, including one upper arm, one forearm, one palm, and three fingers. There are also six joints, including one shoulder joint, and one elbow joint, one wrist joint, and three finger joints (two thumb joints), of which the shoulder and wrist joints can move in two directions. Hence, two coordinate transformation matrices are required; that is, the number of joint points is two. With the first coordinate system as the initial coordinate system, the complete single-sided upper limb kinematics model includes eight joint points. From the first point to the last joint point, a total of seven coordinate transformations and seven conversion matrices are required.

The coordinate system with the direction of the shoulder joint perpendicular to the front of the body as the z-axis is the initial coordinate system. The translation distance of the second coordinate system relative to the initial coordinate system was 0 in the x and z directions, and it rotated 90° around the x-axis, i.e., α1 = 90°. Relative to the second coordinate system, the third coordinate system moved the upper arm length d3 along the x-axis direction, and other changes were all 0. Relative to the third coordinate system, the fourth coordinate system rotated 90° around the z-axis; that is, θ = θ4 + 90°, and moved the arm length d4 along the x-axis, and other changes were 0. Relative to the fourth coordinate system, the fifth coordinate system rotated 90°around the z-axis and x-axis, that is, θ = θ5 + 90°, α5 = 90°, and other changes were all 0. Relative to the fifth coordinate system, the length of each finger movement in the sixth coordinate system was different. Each finger moved in the x-axis and z-axis directions with the movement length of d6 and a6; the sixth coordinate system rotated 90° around the x-axis and z-axis; that is, θ = θ6 + 90°, α6 = 90°. The seventh coordinate system moved d7 along the x-axis direction relative to the sixth coordinate system, and other changes were all 0. The eight-coordinate system moved d8 along the x-axis direction relative to the seventh coordinate system, while other changes were 0.

According to the above analysis, the parameters of the transformation matrix among various coordinate systems are shown in Table 1.

Table 1

Parameters of transformation matrix.
Serial number	Transformation relationship	θ	d	a	α
1	1–2	θ2	0	0	90°
2	2–3	θ3	d3	0	0
3	3–4	θ4 + 90°	d4	0	0
4	4–5	θ5 + 90°	0	0	90°
5	5–6	θ6 + 90°	d6	a6	90°
6	6–7	θ7	d7	0	0
7	7–8	θ8	d8	0	0

The number in the transformation relationship refers to the serial number of the joint point, and in order to keep the initial sign language model state in the L-shaped state in Fig. 3, additional values are set in some of the solution parameters so that the maximum and minimum values of each parameter are the same, at − 180° and 180°, and there will be no situation of − 270°~90°. In the process of data acquisition and the application of the model and actual sensor, this range of parameters can be used to monitor whether the sensor is working properly and reduce the content involved in the judgment step. Obviously, the actual action cannot reach such a large range, and as a result, the actual parameter range needs to be determined according to the specific sign language gesture.

According to Table 1, each transformation matrix can be obtained as:

$${}_{2}{}^{1}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{2}& 0\\ S{\theta }_{2}& 0\end{array}& \begin{array}{cc}S{\theta }_{2}& 0\\ -C{\theta }_{2}& 0\end{array}\\ \begin{array}{cc}0& 1\\ 0& 0\end{array}& \begin{array}{cc}0& 0\\ 0& 1\end{array}\end{array}\right]$$

$${}_{3}{}^{2}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{3}& -S{\theta }_{3}\\ S{\theta }_{3}& C{\theta }_{3}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& {d}_{3}\\ 0& 1\end{array}\end{array}\right]$$

$${}_{4}{}^{3}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{4}& -S{\theta }_{4}\\ S{\theta }_{4}& C{\theta }_{4}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& {d}_{4}\\ 0& 1\end{array}\end{array}\right]$$

$${}_{5}{}^{4}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{5}& 0\\ S{\theta }_{5}& 0\end{array}& \begin{array}{cc}S{\theta }_{5}& 0\\ -C{\theta }_{5}& 0\end{array}\\ \begin{array}{cc}0& 1\\ 0& 0\end{array}& \begin{array}{cc}0& 0\\ 0& 1\end{array}\end{array}\right]$$

$${}_{6}{}^{5}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{6}& 0\\ S{\theta }_{6}& 0\end{array}& \begin{array}{cc}S{\theta }_{6}& {a}_{6}C{\theta }_{6}\\ -C{\theta }_{6}& {a}_{6}S{\theta }_{6}\end{array}\\ \begin{array}{cc}0& 1\\ 0& 0\end{array}& \begin{array}{cc}0& {d}_{6}\\ 0& 1\end{array}\end{array}\right]$$

$${}_{7}{}^{6}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{7}& -S{\theta }_{7}\\ S{\theta }_{7}& C{\theta }_{7}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& {d}_{7}\\ 0& 1\end{array}\end{array}\right]$$

$${}_{8}{}^{7}T=\left[\begin{array}{cc}\begin{array}{cc}C{\theta }_{8}& -S{\theta }_{8}\\ S{\theta }_{8}& C{\theta }_{8}\end{array}& \begin{array}{cc}0& 0\\ 0& 0\end{array}\\ \begin{array}{cc}0& 0\\ 0& 0\end{array}& \begin{array}{cc}1& {d}_{8}\\ 0& 1\end{array}\end{array}\right]$$

The final transformation matrix is:

$${}_{8}{}^{1}T={}_{2}{}^{1}T{}_{3}{}^{2}T{}_{4}{}^{3}T{}_{5}{}^{4}T{}_{6}{}^{5}T{}_{7}{}^{6}T{}_{8}{}^{7}T$$

According to the definition of the posture matrix, compared to the initial matrix, the transformation matrix of the end fingertip is:

$$pose R=\left[\begin{array}{ccc}{n}_{x}& {o}_{x}& {a}_{x}\\ {n}_{y}& {o}_{y}& {a}_{y}\\ {n}_{z}& {o}_{z}& {a}_{z}\end{array}\right]$$

$$coordinate P=\left[\begin{array}{c}{p}_{x}\\ {p}_{y}\\ {p}_{z}\end{array}\right]$$

where

$${n}_{x}=\text{C}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)- \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)\right)\right)- \text{S}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)- \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)- \text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)\right)\right)$$

$${n}_{y}=\text{S}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)+ \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)\right)\right)- \text{C}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)+ \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)- \text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)\right)\right)$$

$${n}_{z}=- \text{C}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)- \text{C}\left({\theta }6\right)\text{*}\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)\right)- \text{S}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)+ \text{C}\left({\theta }6\right)\text{*}\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)\right)$$

$${o}_{x}=- \text{C}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)- \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)- \text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)\right)\right)- \text{S}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)- \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)\right)\right)$$

$${o}_{y}=\text{C}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)+ \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)\right)\right)+ \text{S}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)+ \text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)- \text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)\right)\right)$$

$${o}_{z}=\text{S}\left({\theta }8\right)\text{*}\left(\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)- \text{C}\left({\theta }6\right)\text{*}\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)\right)- \text{C}\left({\theta }8\right)\text{*}\left(\text{C}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)+ \text{C}\left({\theta }6\right)\text{*}\text{S}\left({\theta }7\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)\right)$$

$${a}_{x}=- \text{C}\left({\theta }6\right)\text{*}\text{S}\left({\theta }2\right)- \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)$$

$${a}_{y}=\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }6\right)- \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)$$

$${a}_{z}=\text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)$$

$${p}_{x}=\text{d}6\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)\right)- \text{d}8\text{*}\left(\text{C}\left({\theta }6\right)\text{*}\text{S}\left({\theta }2\right)+ \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)- \text{d}7\text{*}\left(\text{C}\left({\theta }6\right)\text{*}\text{S}\left({\theta }2\right)+ \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{d}3\text{*}\text{S}\left({\theta }2\right)+ \text{d}4\text{*}\text{S}\left({\theta }2\right)- \text{a}6\text{*}\text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)+ \text{a}6\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)$$

$${p}_{y}=\text{d}7\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }6\right)- \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{d}8\text{*}\left(\text{C}\left({\theta }2\right)\text{*}\text{C}\left({\theta }6\right)- \text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)\right)+ \text{d}6\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)\right)- \text{d}3\text{*}\text{C}\left({\theta }2\right)- \text{d}4\text{*}\text{C}\left({\theta }2\right)- \text{a}6\text{*}\text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)- \text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }2\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)- \text{a}6\text{*}\text{C}\left({\theta }2\right)\text{*}\text{S}\left({\theta }6\right)$$

$${p}_{z}=\text{a}6\text{*}\text{C}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)- \text{d}6\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)- \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)\right)+ \text{d}7\text{*}\text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)+ \text{d}8\text{*}\text{S}\left({\theta }6\right)\text{*}\left(\text{C}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)+ \text{C}\left({\theta }4\right)\text{*}\text{S}\left({\theta }3\right)\right)+ \text{S}\left({\theta }5\right)\text{*}\left(\text{C}\left({\theta }3\right)\text{*}\text{C}\left({\theta }4\right)- \text{S}\left({\theta }3\right)\text{*}\text{S}\left({\theta }4\right)\right)\right)$$

According to Table 1, only the z-axis rotation angle θ in the kinematics model is an unknown parameter, which is called the joint angle. The arm length represented by d and the finger length represented by a, are known parameters.

4.1 Model analysis

The upper limb kinematics modeling is based on the modeling method of the industrial manipulator, and the industrial manipulator has two inverse solving methods, namely the geometric method and the algebraic method [9]. The geometric method can handle the unknown parameters through the plane geometric relationship between the joint points and the manipulator; hence, a clear geometric relationship of each part is required as the basis for the solution. The algebraic method directly iteratively solves the equation system composed of the terminal transformation matrix. The solution is straightforward but requires a lot of solution resources and is generally suitable for equation systems with fewer unknown parameters.

According to the modeling analysis in Section 2, there are seven unknown parameters θ2 ~ θ8 in the upper limb motion virtual model, and there are a total of 12 parameters in the terminal fingertip transformation matrix T, including nine parameters in the posture matrix R and three parameters in the coordinate matrix P. Therefore, 12 equations can be listed. However, the posture matrix is orthogonal, and each row and column has a unit length, which introduces six constraints. In fact, the nine parameters of the posture matrix only have three independent variables. Only six independent equations can be listed in the solution process, and the uniqueness of the solution result cannot be guaranteed. Therefore, it is necessary to add other qualifications to reduce the number of unknowns and ensure the uniqueness of the result.

First, the analysis of the inverse solution of sign language movement involving the arm is considered. The representation space of sign language is generally in front of the chest [11], and the upper arm and forearm form an L-shaped representation. It can be assumed that the arm state at this time is the initial state of sign language. At this time, the meaning of sign language is only related to the shape of the hand; that is, the position and state of the palm and fingers, and the arm motion state can be ignored.

According to the initial model, at this time, θ2 = θ3 = θ4 = 0°, the original seven unknowns can be reduced to four unknowns. Therefore, the solution of the kinematics equation only involves the palm to the finger part, and the kinematics equation is:

Sign language in the non-sign language initial state is generally related to body parts, such as "sleeping", "ear", and "eating".

It can be seen that, at this time, the sign language gesture is generally an action of five fingers together straightening or single or multiple fingers straightening. Generally, it does not include complex changes such as angled bending or multi-finger coordination. It is not necessary to completely calculate the palm movements of each finger [10], and the fingertip movement can be briefly solved with the solution result of the arm part. If only the arm part is considered, the kinematics equation is:

In the actual model application process, the coordinate value range of the joint point at the wrist can be used to determine whether the sign language being represented is in the general position. If it is in the general position, will be solve$d$; if it is in the non-general position, is solved. Therefore, the number of unknowns of the equations actually solved does not exceed four, and its uniqueness can be guaranteed under the condition of six independent equations.

4.2 Posture transformation

According to the analysis of the model, the value of the joint angle to be solved can be obtained through the position matrix and the posture matrix of the end during the solution process. However, in the actual data collection process, due to the existence of the constraint relationship, the data of the posture matrix is not independent, cannot be obtained directly, and needs to be further transformed [12]. In practical applications, the commonly used methods for attitude description are Euler angle, axis angle, and quaternion. In this study, the European angle is used to perform posture matrix transformation.

Euler angles can uniquely determine the three independent angular variables of a rigid body; that is, it uses three angles to describe the attitude of the rigid body in the coordinate system so as to meet the requirements of independence. Now suppose that the rotating angles around the z-axis, x-axis, and z-axis are α, β, and γ, respectively. The posture matrix R can be obtained as follows:

$$R={Rot}_{z}{\left(\alpha \right)Rot}_{x}{\left(\beta \right)Rot}_{z}\left(\gamma \right)=\left[\begin{array}{ccc}{C}_{\alpha }{C}_{\gamma }-{C}_{\beta }{S}_{\alpha }{S}_{\gamma }& -{C}_{\alpha }{S}_{\gamma }-{C}_{\beta }{C}_{\gamma }{S}_{\alpha }& {S}_{\alpha }{S}_{\beta }\\ {C}_{\gamma }{S}_{\alpha }+{C}_{\alpha }{C}_{\beta }{S}_{\gamma }& {C}_{\alpha }{C}_{\beta }{C}_{\gamma }-{S}_{\alpha }{S}_{\gamma }& -{C}_{\alpha }{S}_{\beta }\\ {S}_{\beta }{S}_{\gamma }& {C}_{\gamma }{S}_{\beta }& {C}_{\beta }\end{array}\right]$$

where C and S refer to cos and sin, respectively. In this study, Euler angles are obtained around the x and z coordinate axes, which is called the ZXZ rigid body coordinate system. Similarly, Euler angles can also be obtained around other coordinate axes. Here, the ZXZ coordinate system is used because only the x and z coordinate axes are also used in the model building process. It is not necessary to decompose the y axis during the posture transformation process, which reduces the calculation steps.

The upper limb model in this paper is based on the DH model, which can be verified by robotics modeling and simulation tools. Using the Robotics Toolbox in MATLAB, the upper limb kinematics model was constructed according to the parameter table in Table 2 for kinematics simulation verification. The Robotics Toolbox is a powerful tool for robotics calculation and simulation based on the MATLAB platform developed by the Peter Croke team. It is suitable for kinematics simulation and rigid rod simulation. By establishing a motion system for the forward and inverse direction of rotation angles of the joint points, the required solution parameters are linked with the posture data of the key parts in the motion process.

According to part of the sign language gesture expression, the position and posture of the end of the robotic arm that meet the requirements of the connecting rod restriction are randomly given, and the designed simulation tool is used to perform the inverse kinematics solution so as to obtain the value of each joint angle. These angles are then substituted into the forward kinematics solution to obtain the end posture, which is compared with the given posture to verify the correctness of the motion model. The two models and are verified separately, and the verification results are shown in Tables 2 and 3.

Table 2

Inverse kinematics verification.
Number	Position (cm)			Posture (radians)			Joint angle (radians)
	x	y	z	α	β	γ	θ5	θ6	θ7	θ8
1	−7	−37	3	−0.372	−0.639	−1.860	1.963	2.558	2.134	0.399
2	−3	−46	-4	0.107	−1.995	−0.999	2.531	0.648	1.369	1.824
3	4	−37	6	−1.944	−1.416	2.093	0.118	2.649	2.943	1.576
4	−2	−43	7	−0.308	−1.496	3.156	1.032	1.599	1.431	0.867
5	−5	−44	5	−0.125	−1.726	−2.679	1.521	1.364	1.489	0.561

Table 3

Inverse kinematics verification.
Number	Position (cm)			Posture (radians)			Joint angle (radians)
	x	y	z	α	β	γ	θ2	θ3	θ4
1	39	−10	0	1.330	1.571	−3.042	1.232	1.330	1.460
2	13	−38	0	0.334	1.571	−2.857	2.111	0.335	1.645
3	−3	40	0	−3.068	1.571	1.904	1.465	3.215	0.123
4	29	28	0	2.338	1.571	−2.845	2.338	1.657	0.214
5	33	−23	0	0.961	1.571	−2.029	0.961	0.285	2.398

In the verification process, the posture data is collected using a three-axis acceleration sensor, converted inside the sensor, and output based on the ZXZ coordinate system. In the actual action expression process, the action is expressed according to the range of the sign language movement. It may not have actual meaning, but it meets the conditions that may occur in actual actions.

The inverse solution process of the kinematics model is equivalent to solving the combination of multiple equations, and the solution of the equation system will involve its solvability. If the model is not valid, the real number solution cannot be obtained when it is verified by substituting actual data. Hence, obtaining the real number solution in the verification process indicates that the model is valid under actual conditions [13].

Based on the needs of feature analysis of the actual sign language recognition system, a standard kinematics simulation model is established, and the segmented idea is used to limit the model with multiple solutions. An equation system that could be actually solved is obtained, which is decomposed into 4-link and 3-link models using the seven degrees of freedom connecting rod model, and the solvability and accuracy of the kinematics solution is verified based on actual data. Combined with the analysis of the collection points in the data collection process, the importance of the collection point to the overall recognition system can be individually analyzed in the actual sign language recognition process, and the movement state of the sign language representation process can be further analyzed according to a reasonable basis for judgment.

Author Contributions: Conceptualization, C.H.; methodology, C.H.; validation, C.H.; resources, C.H.; writing-original draft preparation, C.H.; writing-review and editing C.H.; visualization, C.H.; supervision, D.C.; project administration, D.H.; f All authors have read and agreed to the published version of the manuscript.

Funding: No funding.

Institutional Review Board Statement: Not applicable.

Informed Consent Statement: Not applicable.

Conflicts of Interest: The authors declare no conflict of interest.

M. P. Paulraj, S. Yaacob, H. Desa, C. R. Hema, W. M. Ridzuan and W. A. Majid, "Extraction of head and hand sign language features for recognition of sign language," 2008 International Conference on Electronic Design, Penang, Malaysia, 2008, pp. 1-6, doi: 10.1109/ICED.2008.4786633.
Goldin-Meadow S, Brentari D. Sign language, sign and language: The coming of age of sign language and sign language studies. Behavioral & Brain Sciences, 2015: 1-82.
Duta N. A survey of biometric technology based on hand shape. Pattern Recognition, 2009, 42 (11): 2797-2806.
Khokhar Z O, Xiao Z G, Menon C. Surface EMG pattern recognition for real-time control of a wrist exoskeleton. Biomedical Engineering Online, 2010, 9 (1): 41-41.
Mario, Munoz-Organero, Ahmad, et al. Human Movement Recognition Based on the Stochastic Characterisation of Acceleration Data. Sensors, 2016.
Tavari, N. V., Deorankar, A. V., & Chatur, P. N. (2013). A Review of Literature on Hand Sign language Recognition for Indian Sign Language. International Journal of Advance Research in Computer Science and Management Studies, 1 (7), 13–20.
Liu H, Wang L. Sign language recognition for human-robot collaboration: A review. International Journal of Industrial Ergonomics, 2017: S0169814117300690.
Gkouvatzi A N, Mantis K, Kambas A. Comparative Study of Motor Performance of Deaf and Hard of Hearing Students in Reaction Time, Visual-Motor Control and Upper Limb Speed and Dexterity Abilities. International Journal of Special Education, 2010, 25 (2): 15-25.
Singh G K, Claassens J. An analytical solution for the inverse kinematics of a redundant 7DoF Manipulator with link offsets[C]// IEEE/RSJ International Conference on Intelligent Robots & Systems. IEEE Xplore, 2010: 2976-2982.
] Oh Y J, Jang H Y, Jung J W, et al. Soft Sign Language Expression Method of 3D Avatar. Kips Transactions Partb, 2007, 14B (2): 107-118.
Alaghband, Marie & Yousefi, Niloofar & Garibay, Ivan. (2020). Facial Expression Phoenix (FePh): An Annotated Sequenced Dataset for Facial and Emotion-Specified Expressions in Sign Language.
Wang C, Jonckheere E. Simulated versus reduced noise quantum annealing in maximum independent set solution to wireless network scheduling. Quantum Information Processing, 2018, 18 (1).
Chaoping, Zang, D, et al. Model validation for structural dynamics in the aero-engine design process. Frontiers of Energy & Power Engineering in China, 2009.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Modeling and analysis of upper limb kinematics in sign language recognition

Status:

Version 1

Abstract

Figures

1. Introduction

2. Modeling Of Sign Language

3. Modeling Of Upper Limb

3.1 Homogeneous transformation

3.2 DH modeling

3.3 Modeling of upper limb kinematics

4. Kinematics Inverse Analysis

4.1 Model analysis

4.2 Posture transformation

5. Model Validation

6. Conclusions

Declarations

References

Additional Declarations

Status:

Version 1