Grasping is an essential skill that humans possess and replicating or imitating its functionality has been a significant focus in robotics research. Robotic hands by imitating human grasping behavior, can perform versatile grasping tasks and enhance human-robot interactions. Replication of such ability in robots is a challenging problem. To tackle this challenge, deep learning, and computer vision methods are proposed. Through object recognition and transfer learning, these techniques have made robotic grasping more accurate and robot hands can become more autonomous. The main objective of this paper is to implement and compare deep learning and reinforcement learning (RL) methods for achieving a semi-automatic grasp of different objects. This paper proposes a humanoid 5-DoF robot hand designed specifically for grasping tasks. The robotic hand is fabricated using a 3D printer and its fingers are driven by 5 servo motors. In this direction, a pre-trained Convolutional Neural Network (CNN) structure was used to train the robot hand. Additionally, a 5-finger robot hand is simulated in the MuJoCo environment. The RL agent plans and executes appropriate actions in the simulated hand robot and provides positive or negative rewards based on the Q-learning algorithm. Finally, the performance of methods is evaluated on objects. The results demonstrate the RL method achieved a higher grasp accuracy of 95% compared to the CNN method, which achieved a grasp accuracy of 85%. This indicates that the RL method outperformed the CNN method in terms of grasp accuracy for the robot hand and improved results.
Research Article
Dexterous learning of the robot hand using machine learning algorithms for object grasping
https://doi.org/10.21203/rs.3.rs-3772624/v1
This work is licensed under a CC BY 4.0 License
Version 1
posted
You are reading this latest preprint version
Deep learning
Grasping
Convolutional neural network
Robot hand
Reinforcement learning
The study of robotic hands is a diverse and interdisciplinary field that encompasses various disciplines, including mechanics, mathematics, control theory, and machine learning. Researchers aim to replicate the dexterity and movement capabilities of the human hand in mechanical structures known as humanoid or robotic hands. An example of such dexterity is the grasping and manipulation of objects. Grasping an object with a robot hand is a challenging task, and its success depends on several factors such as the shape and position of the object. Accordingly, skilled hands have been investigated in different grasping applications. Hardware-based methods require reprogramming when dealing with new objects or slight variations in the grasping process. The advancement of artificial intelligence (AI) has indeed had a significant impact on robotic hands, surpassing traditional hardware-based methods. Deep learning and RL are subfields of AI that have revolutionized the way robots perceive and interact with objects, particularly in the context of grasping. Furthermore, the applications of robotic hands extend to sections such as surgery, prosthetics, the manufacturing industry, and rehabilitation.
The anatomy of the human hand is complex and attractive, enabling a wide range of dexterous movements. The hand comprises several bones that work together to provide a wide range of motion and dexterity. Finger joints can be classified into Metacarpophalangeal (MCP) joints, Proximal Interphalangeal (PIP) joints, and Distal Interphalangeal (DIP) joints [1]. Figure 1 shows the joints of the fingers.
The human hand possesses a total of 27 degrees of freedom. These degrees of freedom enable the hand's considerable ability to perform intricate movements and handle a wide range of tasks. These intricate movements allow humans to grasp objects with precision, adaptability, and finesse. Grasping an object involves securely holding it using your hand or fingers. There are various taxonomies in the field of grasp analysis that are widely used. These classifications, categorize hand movements and grasping patterns based on functional aspects such as the shape and orientation of the objects being grasped, the use of fingers, and the type of grasping applied. For example, one commonly used classification is the Cutkosky classification [3]. Cutkosky's classification is given in Fig. 2.
This classification includes power grasps and precision grasps. The power grasp involves achieving full contact between the hand and the object being grasped. This type of grasp is employed when stability and grasping strength are crucial. Precision grasps are characterized greater by the utilization of the index finger and thumb to achieve dexterity and control. These grasps are employed when tasks require more delicate or precise manipulation. Both power grasp and precision grasp are necessary for human dexterity and the ability to interact with objects in our environment effectively. The Cutkosky classification is a widely recognized reference and framework used for designing robotic hands. The Shadow Hand [4] is a highly dexterous robotic hand with 24 degrees of freedom and developed by Shadow Robot Company. This robot hand mimics the human hand's structure and capabilities, allowing for complex grasping and manipulation tasks. The DLR Hand II [5] is an anthropomorphic robotic hand with five fingers and 20 degrees of freedom and was developed by the German Aerospace Center. DLR Hand II features tactile sensing capabilities and is designed for complex manipulation tasks and human-robot interaction. The use of 3D printing and rapid prototyping techniques has indeed become increasingly popular in the development of robotic hands. These technologies offer several advantages, including cost-effectiveness, customization, and faster iteration cycles. OpenBionics [6] is a company that focuses on creating affordable and customizable robotic hands using 3D printing and is designed for amputees. InMoov [7] is an open-source project that provides a framework for building a functional and humanoid robotic hand. InMoov robot hand is primarily designed to be fabricated using 3D printing technology and is driven by servo motors.
The integration of AI, particularly deep learning, into robot hand grasping research has brought significant advancements. It has enabled researchers to overcome the limitations of hardware-specific approaches, achieve generalization across different systems, and develop adaptive and intelligent grasping capabilities. Deep learning algorithms such as CNNs [8], can be used for analyzing visual data, image classification, and object detection. CNNs apply sequential operations on the input image, typically consisting of input, convolution, Pooling, and fully connected layers. Deep CNNs excel at solving complex problems with high accuracy. These networks are computationally intensive and time-consuming. However, several solutions have been proposed to these problems. Transfer learning is a technique where a pre-trained model, typically trained on a large-scale dataset (e.g., ImageNet), is utilized as a starting point for a new task or a different dataset [9]. This approach reduces the need for training a network from scratch, saving both time and resources. Fine-tuning is a technique used in transfer learning and involves adjusting the parameters of the pre-trained model to adapt it to the specific characteristics and requirements of the new task. Visual Geometry Group (VGG) [10] is one of the common pre-trained CNN architectures. VGG-16 consists of 16 layers and was proposed by the Visual Geometry Group at the University of Oxford. Devaraja et al. [11] trained the robotic hand on multiple objects and evaluated its performance in recognizing object shapes. They employed a support vector machine (SVM) classifier to recognize the shape of objects and reported an average object shape recognition accuracy of 94.4%. Levin et al. [12] focus on a learning-based approach for robotic grasping using monocular images. They used deep learning techniques to develop a grasp success prediction. The experimental results obtained illustrate the effectiveness of their method in grasping a diverse range of objects.
In recent years, to minimize human participation in data capture and improve accuracy in the context of grasping hand robots, the RL method has provided suitable solutions. RL is one other subfield of machine learning that specifically focuses on sequential decisions. In RL, an agent learns to take actions in an environment to maximize a reward signal. The agent receives feedback in the form of rewards or penalties based on its actions, enabling it to learn optimal strategies through trial and error. An MDP is a mathematical model used to formalize the RL problem and consists of five components (a set of states, actions, transition probabilities, rewards, and a discount factor). The key assumption in an MDP is the Markov property, which states that the current state contains all the necessary information to predict the next state. RL algorithms aim to find an optimal policy by estimating the value function, which measures the expected cumulative reward from a specific state or state-action pair [13]. The value function allows the agent to prioritize actions that lead to higher rewards. RL algorithms can be broadly classified into two categories model-based and model-free methods. These classifications are based on how the RL agent learns and utilizes knowledge about the environment. Model-free RL is more flexible and can handle environments with complex dynamics since it does not explicitly rely on a learned model. However, it might require more exploration to discover the optimal policy and can take longer to converge compared to model-based approaches. Model-free RL algorithms often work with state-action value functions. Q-learning is a known model-free algorithm used to solve RL problems [14]. To balance exploration and exploitation, Q-learning often employs an ε-greedy policy. According to this policy, with a high probability of (1 - ε), the agent selects the action that has the highest Q value for a given state. This exploitation step aims to choose actions that are believed to have higher expected rewards based on the learned Q values. On the other hand, with a low probability of ε, the agent selects a random action.
Training RL agents on physical robots requires collecting a significant amount of training data. This process can be time-consuming and impractical, as it often involves repetitive interactions with the robot over extended periods. Also, RL agents typically explore and learn through trial and error, which can be unsafe when applied directly to physical robots [15]. To address these challenges, robotic simulators are employed. Simulators provide virtual environments that mimic the behavior and physics of real-world robotic systems. RL agent by using simulators, can learn and explore different tasks in a faster, more scalable, and cost-effective manner [16]. MuJoCo [17] is a physics engine developed by Emmanuel Todorov. MuJoCo is used in robotics research, particularly in studies related to robotic hands and manipulation tasks. Training simulated robotic arms using RL algorithms to perform tasks like reaching a goal has shown promising results in robotics research [18]. Researchers by applying RL techniques to simulated environments, can train robotic arms to acquire the necessary skills to complete specific tasks effectively. Grasping complicated objects can be a challenging task due to their shape and potential slippage. It demonstrates the effectiveness of the RL algorithm in training the robotic arm to learn a grasp policy that enables it to securely and reliably grasp cylindrical objects. Overall, the successful application of RL to grasp complicated objects demonstrates the potential of this approach in training robotic arms to handle challenging and diverse object manipulation tasks [19].
This study proposed a 5-finger robot hand for grasping a wide range of objects. This robot hand is manufactured using 3D printing technology. The CNN method is used to learn stable grasping and control the automatic opening and closing of the robot's hand fingers. This method focuses on fine-tuning the last layer of a pre-trained neural network structure and eliminates the need for human control and decision during the grasping process. To improve and increase accuracy, the RL method is applied to the simulated robot hand in the MuJoCo environment. The use of simulation reduces the cost and risks associated with training on real robotic hardware. The Comparison of CNN and RL methods to determine which one leads to the best performance in the grasping of a hand robot is indeed a significant contribution to the present paper.
In this section, the construction of the robot hand and the electronic system used are briefly explained. In addition, the details regarding the model of the simulated robot hand, its physical properties, and the object model are also presented.
2.1 Structure of 3d-printed robot hand
For this study, a 5-finger robotic hand was fabricated by a 3D printer with the aim of rapid and low-cost prototyping. The robot hand has 5 degrees of freedom. The 5 degrees of freedom in the robotic hand refer to the ability of each finger to move independently in flexion/extension. All fingers, except the thumb, have the same design and dimensions. As seen in Fig. 3, the suggested 4-finger design consists of three metacarpophalangeal, proximal interphalangeal, and distal interphalangeal joints, and the thumb consists of two metacarpophalangeal and distal interphalangeal joints.
Accordingly, a tendon-based transmission system was chosen. The tendon-based transmission system involves routing the fishing line through the fingers' joints and connecting them to servo motors. When the servo motors pull or release the tendons, the fingers can flex or extend accordingly. Adding the foam sheet on the palm of the robot hand can serve as a cushioning material and provide added grasping and dexterity. A 5-megapixel camera into the robot's hand to collect visual information is integrated. The camera's resolution of 720 x 1280 can provide a reasonable level of detail for capturing visual data. This camera is connected to the robot's hand using a piece of wood at a distance of 40 cm. Wood acts as a mount or support structure to securely position the camera at the desired distance and angle. An image of the connection of the camera to the robot's hand is shown in Fig. 4.
2.2 Hardware implementation
The 5 servo motors that move the fingers are controlled through the Pulse Width Modulation (PWM) pins of the Arduino. PWM is a technique where a digital signal with a fixed frequency is used, and the width of the pulses within that signal is varied to control the servo motor's position. The position values of the servo motors are read through the analog pins of the Arduino. The required components of the robot hand are listed in Table 1.
Table 1 Electrical components of the robot hand
|
Number |
Component image |
Component name |
|
5 |
|
Servo motor(model HS-81 Hitec) |
|
1 |
|
Camera module(model ELP-USB100W05MT) |
|
1 |
|
Arduino mega 2560 |
The circuit connections are Placed on a shield connected to the Arduino Mega board. Shields are expansion boards that provide additional functionalities and allow for easier connectivity. The Arduino Mega board receives processed information through the serial port. The Arduino Mega board and the camera are connected to the computer via USB. It's important to note that while the USB connection can power the Arduino Mega board, requires a separate power supply for the servo motors since they draw more current than the Arduino board can provide. Figure 5 shows the hardware connections and communication between the servo motor, Arduino, camera, and computer.
2.3 Simulation of robot hand model and object
Among the robotic simulators considered, the MuJoCo simulator was chosen for this work because it performs better than many other simulators, especially in the fields of robotics and grasping. To simulate the robot hand, the 3D CAD model of the real robot hand was used. The CAD model provides detailed geometric information about the hand's structure, including the shape, size, and dimensions of its components. This model consists of a forearm, wrist, palm, and five fingers and has 14 joints. The simulated hand must be combined with suitable motion programming and controllers. There are servos to move the fingers, which are used to move and control the joints. Also, the inertial and dynamic properties of the joints should be met. Figure 6 shows the simulated robot hand in the MuJoCo environment.
For a grasping test in the MuJoCo environment, a 3D object resembling a real glue stick was simulated. The simulated object was designed with a diameter of 2 cm and a length of 8 cm. The chosen model is a cylinder that is ideal for grasping. The image of the simulated object in the MuJoCo environment is given in Fig. 7.
To conduct this research, two techniques of CNN and RL were used to train the robot hand. To collect data, images are labeled and trained with convolutional neural network architecture. Also, to achieve high accuracy a transfer learning technique is adopted. In addition, the components of RL such as agent, action, reward function, and the algorithm presented are explained.
3.1 Training with the CNN method
3.1.1 Data collection
Two datasets ImageNet, and data collected with a camera, were used to train the robot's hand. ImageNet is a large-scale dataset that contains millions of labeled images across various categories. It is used for pre-training CNNs due to its size and diversity. In addition to the ImageNet data, a dataset of 669 images collected with a camera was used. These images were captured in the same light and location conditions for a glue stick with a cylindrical shape. Dataset by capturing images of the glue stick from various angles and positions, can help the CNN understand the unique characteristics of the object and its spatial relationships with the hand and estimate its position. Finally, Position each finger is used as a label for training the network. Figure 8 shows a frame of this dataset.
3.1.2 Network architecture
The VGG-16 architecture is used for automatic grasping selection. The choice of VGG-16 is due to its simple architecture compared to other architectures. The input tensor size for the VGG-16 architecture is (3, 224, 224), which contains images with a fixed size of 224 x 224 pixels and three RGB channels (red, green, blue). The VGG-16 architecture consists of a series of convolutional computation layers followed by integration layers, typically maxpooling layers. The convolution and maxpooling layers are repeated several times to gradually extract features from the input image. Towards the end of the VGG-16 architecture, after the convolutional and pooling layers, fully connected layers are employed to process the extracted features and generate the final output. The output of the fully connected layers represents the predicted class probabilities or scores for the given input image. The neural network diagram used is summarized in Fig. 9.
3.1.3 Network training
A VGG-16 is adopted to predict the use of each finger for grasp objects. The pre-trained VGG-16 model, with its initial layers' weights kept constant, is used as the starting point for the network. The network by leveraging the pre-trained weights, can benefit from the learned general image features. Only the fully connected layers of the VGG-16 model are retrained and adjusted using the collected data. This allows the network to adapt and learn the specific task of predicting the use of each finger for grasping objects. The network is trained for 30 epochs, which involves iterating over the collected dataset multiple times. A learning rate of 0.00005 is used to control the magnitude of weight updates during training. After training each finger using five networks, the networks are combined into a single network. This combination aims to improve performance and capture the relationships between the fingers, enabling more effective grasping decisions. The TensorFlow framework, a popular deep learning platform, is used to facilitate the training process. The code for the application is written in Python, which allows for seamless integration of pre-trained networks, cameras, and Arduino. Finally, the output of the network is transformed into a "used" and "unused" prediction for each finger. For example, the output of 11000 means that the thumb and index finger are used and the other three fingers are not used. The hand performs the grasping using the activated fingers and the closing command. By automating the grasping decisions, the system can perform tasks with minimal human input. Figure 10 shows the steps of grasping objects using the robot's hand.
3.2 Training with the RL method
3.2.1 System configuration
The robotic hand grasping task is formulated as a MDP and an episodic process. The objective of RL is to train agent to successfully do various grasping tasks. The agent's role is to observe the environment and take actions such as opening and closing the fingers of the robotic hand. Following each action, the agent receives feedback in the form of the new state of the environment, which comprises a set of object position values. The agent by using an ε-greedy policy, balances between exploiting its current knowledge and exploring new possibilities. Initially, during the training process, the robotic hand performs different movements randomly. The agent observes these random movements and monitors the behavior of the robot. Through RL, the agent gradually refines its policy by exploring the environment and exploiting the knowledge gained. As training progresses, the agent learns to choose the next action based on the current state, leveraging its acquired policy. In this article, the state space for the grasping task consists of the position vector (x, y, z) of the beginning and end of the object in three-dimensional space. Furthermore, the action space for each finger comprises two actions open and closed. These actions can be encoded as binary values, where 1 represents a closed finger position and 0 represents an open finger position. The agent by considering the state and action, can learn a policy that maps states to appropriate actions, enabling the robotic hand to perform the grasping task effectively in a three-dimensional space with multiple fingers. Selection of an appropriate reward function is indeed a crucial aspect of RL for the robotic hand grasping task. The reward function serves as a feedback mechanism that guides the agent's learning process by providing information on the desirability of its actions. Here it is desirable that the robotic hand can hold objects for at least 2 seconds. If the hand successfully holds the object, the agent receives a reward of 1. On the other hand, if the object falls to the ground, the agent receives a reward of 0 to indicate that the action was not desirable. By doing so, the agent is encouraged to perform actions that result in successful grasping and holding of objects. The agent learns to associate actions that lead to the successful holding of objects with positive rewards, thereby gradually improving its grasping capabilities. Figure 11 shows the RL interaction loop for the robot hand.
3.2.2 Robot hand training using Q learning algorithm
The RL algorithm employed is Q-learning, which enables the robotic hand agent to predict the opening and closing movements of its fingers and achieve successful grasping of objects. According to Fig. 12, the Q-learning algorithm updates the values of state-action pairs by considering the reward obtained upon transitioning to the next state and the maximum value achievable in the subsequent state.
For this purpose, a Q network is defined for each finger, which represents the learned action-value function. The teaching process of the robotic hand can be outlined as follows:
a. The agent begins by requesting the current state of the simulator, which comprises the start and end positions of the object.b. Agent using its policy selects the next action. Initially, the agent may choose random actions with low probability. However, as training progresses, the agent gradually shifts towards selecting actions with higher expected values.
c. The agent sends the chosen action to the simulator, which executes the action and returns a new state. Additionally, the reward for the action is calculated, and the necessary updates are done.
d. The values in the action-value function are updated frequently based on the newly obtained rewards, allowing the robotic hand to optimize its actions towards reaching the goal state. The optimize-function optimizes the network for a single step, ensuring convergence towards an optimal policy.
During each step, the agent may successfully grasp the object for 2 seconds, or it may fail the object. At each simulation environment reset, the location and angle of the object are changed using a random-pose function and enable the agent to adapt to various scenarios. The random-pose function by placing the object in different locations and orientations, ensures that the agent encounters a diverse range of scenarios during training. This randomness helps the agent learn to adapt its grasping strategy to various object placements and orientations, leading to a more robust and generalized policy. Now the robotic hand agent is programmed in a Q-learning approach to take the best action to reach the goal state in the next action by determining the rewards for the current actions taken and using these rewards.
In this section, the test results of both methods are presented and analyzed. Next the results of validation and comparison of algorithms are presented.
4.1 Training results of CNN
4.1.1 Network analysis
The article describes the use of CNN to predict grasping based on images of a glue stick for a real 5-finger robot hand. The network's performance was evaluated using Huber's error function as a measure of network error. Figures 13 and 14 show the graphs of network error and accuracy for each finger, respectively. According to Fig. 13, the error function decreased rapidly during the training process. This rapid decrease indicates that the network effectively fits the training data, allowing it to make accurate predictions. Furthermore, each finger achieved similar accuracy rates, with a maximum accuracy of 100% at the end of training. This indicates that the trained network accurately predicted actions by the robotic hand. The experiments demonstrated that the accuracy and error rates for each finger were considered acceptable after 30 epochs. These results are obtained at a low computational cost and in little time.
4.1.2 Objects grasping trials in the real environment
In this research, the focus is on the grasping task where the robot's fingers are used to grasp objects without the need to lift them. The classification of grasping distinguishes between power grasp, which involves the use of the robot's palm and applies more pressure on the object, and precision grasp, which solely relies on the fingers. Several test was conducted to assess the grasping system under similar conditions. The tested objects included a highlighter, a roll of paper tape, a tube of cream, a plastic egg, and the object used in training. This set is shown in Fig. 15.
The success of a grasp was defined by the robot hand stable holding the object for at least 2 seconds. Otherwise, grasp is considered unsuccessful. Success and failure percentages were calculated for each object based on more than 100 grasping attempts and 30 training epochs. Table 2 shows the success and failure rates for each object.
| Object | Success percent | Failure percent |
|---|---|---|
| Glue stick | 85 | 15 |
| Highlighter | 83 | 17 |
| Tube of cream | 80 | 20 |
| Roll of paper tape | 62 | 38 |
| Plastic egg | 51 | 49 |
The glue stick had the highest accuracy with an 85% success rate, while the plastic egg had the lowest accuracy with a 51% success rate compared to other objects. Overall, the success rates for grasping objects other than the glue stick demonstrate the generalization, efficiency, and flexibility of the algorithm. For previously unseen objects, the robot hand achieved a 66% success rate on average, while for the object used in training, the success rate was 85%. Comparisons with previously reported studies in Table 3 indicate that the results obtained in this study are acceptable. Figure 16 shows some examples of successful attempts to grasp a glue stick and other objects.
| Hand | Success grasp percent | Similar works |
|---|---|---|
| Multi-finger gripper | 30–60 | [20] |
| Boris robot hand | 77.8 | [21] |
| Proposed robot hand | 85 | Present study |
4.2 Training and Simulation Results of RL
4.2.1 Parameters tuning
The process of finding the appropriate parameters and simulator settings for RL in the context of grasping testing was a challenging task. It requires careful adjustment of various parameters, with some being more crucial than others. The selection of these parameters was based on experimental testing and aimed to achieve high accuracy in the grasping test. Table 4 presents the adjusted parameters specifically for the grasping test. These parameters include learning rate, episode, and time to hold.
| Parameter | The set value |
|---|---|
| learning rate | 0.06 |
| episode | 90 |
| time to hold | 2 Second |
4.2.2 Algorithm learning performance
In this section, the results of RL on the simulated robot hand are presented. Experiments were carried out using the Q learning algorithm for the problem of learning grasping in a simulation environment. The results of the algorithm training in the simulation environment are depicted in Fig. 17. In each episode of the training process, the agent's performance is evaluated, and it receives a reward of 1 or 0 based on its performance. As can be seen in Fig. 17, there is more continuity in the graph as we approach the final episodes. It seems that the Q learning algorithm learns relatively fast and achieves good results. Also, the increased continuity in the graph towards the final episodes of training suggests that the Q-learning algorithm is effectively learning and adapting its policies to improve the grasping task.
4.2.3 Results of object grasping in the simulation environment
In this section, the performance of grasping the object with the simulated robot hand is examined. At the beginning of each simulation, the environment is reset and the object is randomly placed in a new position in front of the robot's hand. The trained robot hand then attempts to grasp the object by opening and closing its fingers. A grasp is considered successful if the robot hand can hold the object for 2 seconds, and unsuccessful if it cannot. To assess the performance, the simulation was repeated 100 times. The results showed that the simulated robot hand can successfully grasping with 95% accuracy. This indicates that the trained robot hand was able to grasp and hold the objects effectively in the majority of the simulations. The simulation results show that the proposed approach can effectively reduce the total training time and the total number of grasping attempts. This suggests that the RL framework applied to the simulated robot hand enables efficient learning of grasping strategies, leading to successful grasping. Figure 18 shows samples of successful robot hand grasping where the trained hand can grasp and hold the objects securely. Figure 19 illustrates a scenario where the grasp attempt is unsuccessful.
It can be concluded that the RL method outperforms the CNN method in terms of success rate for grasping objects and is more independent in grasping the robot hand. It also, provides a more holistic approach to grasping learning, taking into account the interaction between the robot hand and the environment. However, it does not mean that CNN methods are lower for all grasping scenarios. The simulation experiments conducted for RL, provide reliable and accurate estimates of the grasping performance. Through training the model in a simulated environment, can optimize the grasp strategy without the need for costly and potentially risky experiments in real-world training scenarios.
This paper introduces a 5-finger robot hand equipped with a camera and servo motor, which is trained using a CNN. The focus of the research is to develop a deep learning algorithm that can predict the optimal use of each finger for grasping objects. Using the proposed pre-trained network led to a reduction in computation and time. The trained robotic hand was tested on various objects with different shapes and sizes. The results indicate that the highest success rate achieved in capturing a cylindrical object is 85%. Also, the network demonstrates acceptable performance in grasping unknown objects, highlighting the generalization capability of the trained model. Moreover, the research explores the implementation of a RL approach using the MuJoCo simulator environment. The Q-learning algorithm is employed to train the robot hand to grasp objects. The results obtained through RL demonstrate superior performance in grasping and achieve a higher success rate of 95% in grasping objects. The CNN-based approach achieves good results in grasping objects, while the RL approach further enhances the performance and success rate of the robotic hand.
Author Contribution
All authors contributed equally: Hamidreza Heidari (H.H), Tahereh Ghahri Saremi (T.G.S), and Tayebeh Ghahri Saremi (T.G.S).
- Palastanga, N., & Soames, R. (2011). : Anatomy and human movement, structure and function with PAGEBURST access, 6: anatomy and human movement. Elsevier Health Sciences.
- Wheatland, N., Wang, Y., Song, H., Neff, M., Zordan, V., & Jörg, S. (2015). : State of the art in hand and finger modeling and animation. in Computer Graphics Forum. 34(2), 735–760 https://doi.org/10.1111/cgf.12595.
- Cutkosky, M. R. (1989). : On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on robotics and automation, 5(3), 269–279.
- https://www.shadowrobot.com/dexterous-hand-series/.
- Butterfaß, J., Grebenstein, M., Liu, H., & Hirzinger, G. (2001). : DLR-Hand II: Next generation of a dextrous robot hand. n Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164). 1, 109–114 https://doi.org/10.1109/ROBOT.2001.932538.
- Liarokapis, M., Zisimatos, A., Kontoudis, G., Mavrogiannis, C., & Kyriakopoulos, K. : OpenBionics prosthetic hands: Open source, affordable, lightweight, anthropomorphic.
- Montagut, G. E. (2016). : Inmoov robot: building of the first open source 3D printed life-size robot. (Doctoral dissertation, Universitat de Lleida).
- LeCun, Y., & Bengio, Y. (1995). : Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks. 3361(10).
- Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). : Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25.
- Simonyan, K., & Zisserman, A. (2014). : Very deep convolutional networks for large-scale image recognition. https://doi.org/10.48550/arXiv.1409.1556.
- Devaraja, R. R., Maskeliūnas, R., & Damaševičius, R. (2020). : Design and evaluation of anthropomorphic robotic hand for object grasping and shape recognition Computers 10(1), https://doi.org/10.3390/computers10010001.
- Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018). : Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection The International journal of robotics research 37(4–5), 421–436 https://doi.org/10.1177/02783649177103.
- Sutton, R. S., & Barto, A. G. (1999). : Reinforcement learning: an introduction. second edition. 3(9).
- Watkins, C. J., & Dayan, P. (1992). : Q-learning. Machine learning. 8, 279–292.
- Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., & Hassabis, D. (2015). : Human-level control through deep reinforcement learning. nature. 518(7540), 529–533.
- Levine, S., Finn, C., Darrell, T., & Abbeel, P. (2016). : End-to-end training of deep visuomotor policies The Journal of Machine Learning Research. 17(1), 1334–1373 https://doi.org/10.48550/arXiv.1504.00702.
- Todorov, E., Erez, T., & Tassa, Y. (2012). : Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems. 5026–5033 https://doi.org/10.1109/IROS.2012.6386109.
- Franceschetti, A., Tosello, E., Castaman, N., & Ghidoni, S. (2021). : Robotic arm control and task training through deep reinforcement learning. In International Conference on Intelligent Autonomous Systems. 532–550.
- Stulp, F., Theodorou, E., Buchli, J., & Schaal, S. (2011). : Learning to grasp under uncertainty. In IEEE International Conference on Robotics and Automation. 5703–5708 (2011). https://doi.org/10.1109/ICRA.2011.5979644.
- Kootstra, G., Popović, M., Jørgensen, J. A., Kuklinski, K., Miatliuk, K., Kragic, D., & Krüger, N. (2012). : Enabling grasping of unknown objects through a synergistic use of edge and surface information The international journal of robotics research 31(10), 1190–1213 https://doi.org/10.1177/0278364912452621.
- Kopicki, M., Detry, R., Adjigble, M., Stolkin, R., Leonardis, A., & Wyatt, J. L. : One-shot learning and generation of dexterous.
- grasps for novel objects. The International Journal of Robotics Research. 35(8), 959–976 (2016).
- https://doi.org/10.1177/0278364915594244.
No competing interests reported.