An on-line path planning of assembly robots based on FOVS


 To improve the accuracy and efficiency of path planning for the mechanical assembly process of products, an on-line path planning method for mechanical assembly process robots based on visual field space is proposed in this paper. Firstly, to predict and describe the assembly process, the concept of field-of-view space (FOVS) is proposed. Secondly, image processing is carried out by knowledge base to judge the assembly type and current assembly state, and the initial assembly path is given. Then, the assembly process is integrated and solved, and the location estimation of obstacles are given according to the FOVS. Finally, the ant colony algorithm is improved to get the final assembly optimization path. Comparing the algorithm with the ACS algorithm in the aspect of path planning. The length of path planning is reduced by 2%, and the algorithm time is reduced by 0.5s, the accuracy and efficiency have been effectively improved. the result shows that the algorithm is effective.


Introduction
Recently, with the proposal of the continuous development of industrial robotics technology, unmanned operation is becoming the inevitable trend of the future development of the manufacturing industry [1][2][3]. With the improvement of the manufacturing level, robots must adapt to the new conditions in the unstructured environment, and they must also use their wisdom adaptively and effectively. In the modern manufacturing industry, the workload of the assembly process accounts for 20% to 70% of the whole manufacturing process, and the cost of assembly process accounts for 40% or more of the production cost. Flexible automatic assembly has always been a major difficulty in manufacturing automation [4].
At present, the assembly process automation of small variety and large batch is widely used, but the automation of small batch and multi variety assembly process is rarely used. There is environmental uncertainty in the assembly process of small batch and multi variety. This uncertainty includes two points. On the one hand, to meet the requirements or avoid obstacles, assembly path planning is an important part of Xiaoqiao Wang wxq20061087@163.com assembly operations, and the solution of assembly path planning can also be used to verify the rationality of product design and assembly sequence planning. On the other hand, as the working object will change its position because of deformation or offset, it is very difficult for the executing agency to complete the processing of the target object only through teaching. This requires that the executing agencies can perceive and adjust the execution path when the environment changes.
In view of the above two difficulties, this paper obtains the environmental information and work piece information in real time by constructing the field of view space, and plans the initial assembly path, and then optimizes the path by optimizing ant colony algorithm. Aiming at the two problems mentioned in the introduction, the literature review will be carried out according to the two aspects of visual image processing and assembly path optimization.
At present, many machine vision and robot control systems are widely used [5][6].
At the same time, human-computer interaction [7][8], tactile [9], and predicting ergonomic risk [10] have made some progress in robot control. Paper [11][12][13] are the mature and in-depth application in machine vision. Yao Guo [14] sensed the patient's gait in the home through machine vision, accurately judged the abnormal gait of the person, and let the assistant robot play a very important role in improving the quality of life of the patient at home. Noriaki Hirose [15] proposed a depth vision model predictive control strategy learning method, which can avoid collision with invisible objects in the navigation path while performing visual navigation. He  and Piotr Dollar [21][22][23] have done a lot of research on visual perception and image processing.
Given the bottleneck of region recommendation computation, he proposed a region recommendation network (RPN). It also predicts the object boundary and score of each location. RPN produces high-quality region construction through end-to-end training. In the ILSVRC and COCO 2015 competition, faster R-CNN and RPN are the basis for winning the first place in many competitions. Wei Liu [24] proposed a new method of haze removal before image restoration based on sky segmentation and dark channel to solve the prior defects of the dark channel. This method can effectively restore the Inland Waterway image. Evan Shelhamer [25] defined the full convolution network for semantic segmentation, transformed the existing classification network (Alexnet, VGG network, and google net) into the full convolution network, and transferred the learned representation to the segmentation task through fine-tuning. Then, a skip architecture is defined, which combines the semantic information from deep and rough layers with the appearance information from shallow and fine layers to produce accurate and detailed segmentation. Multi-manipulator [26], mobile robot [27] and UAV [28] related visual perception and image processing are also gradually in-depth; in addition to camera recognition, there are also ultraviolet or near-infrared spectroscopy system detection used in visual perception [29]. To sum up, the application mode of machine vision in the field of application is relatively single, lacking of deconstruction of space. The fusion of vision and surrounding multiple information is worth studying In the field of human-computer interaction and robot assembly, Wang [30] proposes the assembly tasks related to space robots, Shi [31] uses flexible grippers for micro-assembly tasks, Xu [32] has a good effect on dual-arm robots, human-computer assistance, and isolation. Anouar Benamor [33] proposed a new multi-objective design method for optimal control of robots. The robot system was described as a linear time-varying model. The weight was optimized by a genetic algorithm, which not only reduced the tracking error but also improved the tracking response under the condition of reducing the oscillation. Yue Wang [34] programmed assembly tasks for industrial robots by decomposing human demonstrations into a series of assembly skills and compiling them into robot executable files. Wafa Boukadida [35] mainly studies robust optimal sliding mode control law for uncertain discrete-time robotic systems with high nonlinearity, unmodeled dynamics, and uncertainty. Seyed Mohammad Ahmadi [36] proposed a robust task space control method. The adaptive Taylor series uncertainty estimator was used to estimate the motor manipulator. At the same time, the upper bound of approximation error was estimated to form a robust term. The asymptotic convergence of tracking error and its time derivative was proved based on stability analysis. To improve the sorting accuracy and efficiency of the sorting system of large inertia robots, a new trajectory planning method based on S-shaped acceleration and deceleration algorithm is proposed. It can be seen that vision based multi-objective path planning is still lack of research, and assembly path optimization in dynamic environment is needed.
The rest of the paper is organized as follows. In section 2, we give the overview of methodology and gives specific methods and elaborate on the concept of FOVS and optimization of path planning. Section 3 validates by an example of assembling the lower body of a solenoid valve. The conclusion of the paper is discussed in section 4.

Overview of methodology
Visual-based path planning is an effective path planning method so far. It has been widely used in the industrial field because of its many advantages [37]. The main advantages are low cost, easy installation, capturing real scene maps and collision detection before actual contact, etc. Therefore, vision-based machine monitoring and guidance is one of the important ways to be applied in all walks of life [38].
In Fig. 1, the environment space includes information of robots, obstacles, machine vision and so on. The target path needs to be planned in a limited space. In order to describe the path planning in the limited space, the Field of View Space (FOVS) is proposed. The FOVS consists of three aspects: the reachable space of robots, the position space of products and parts, and the field of view. Firstly, the product model knowledge base is established by product modeling, and the complete product parameters are given.
Secondly, the initial image information is obtained by taking pictures and comparing images, after visual processing algorithm library, the product shape and position direction are identified by edge searching and model library comparison. Then, according to the machine, the product shape and position direction are identified. The motion characteristics and posture of the robot are used to get the actuator status information. After visual information processing, product-related parameters processing, and robot status information processing, multi-source information fusion is carried out, and the real-time path planning model is given, and the initial path is obtained. Then the initial path is validated and optimized according to the relevant algorithm to complete the assembly task in an uncertain environment.
One of the most important directions in the research of robot cooperative environment using vision-guided active collision avoidance system is to improve flexibility and productivity. Therefore, this paper proposes a path planning method for assembly process robot based on FOVS. The space state is judged in real-time by-product process parameters, accurate field-of-view image information, and robot state parameters. The robot is guided to assemble by image information, which reduces the requirement of fixture positioning accuracy and improves assembly efficiency.

FOVS
Given how to collect and process different assembly processes in different environments, depending on a single visual field information can not reflect the real situation of the field of view, and considering the assembly process parameters and robot status information, a complete position status information can be obtained.
According to the relationship between this information, this paper proposes the concept of field of view space.
As shown in Figure 2, the robot path planning is a dynamic process. This process consists of the Field-of-view expression, Path planning, Application path, Actuator action, System perception module. It is a closed-loop control system. The relationship among product process parameters, robot state information and visual field information constitutes the Field-of-view expression. 1, Process parameters of products gives Initial state information and initial path planning. 2, Process parameters of actuator gives Actuator status information and Fixture, Positioning state. 3, Initial visual information and System perception module determine the Update visual information and Obstacle examination and location.
The FOVS gives the spatial model, the type of work piece, the number of target points, the location of obstacles and other information. Firstly, the FOVS can solve the problem of spatial expression of field of view for obstacles or environmental changes, and secondly, it can solve the corresponding relationship among the field of view, actuator and assembly space. The unconstrained optimization form of the total variation model is: Where X is the vector space of finite dimension,  represents gradient operator,  represents regularization parameter, u represents denoised image, g represents observed image. In the formula, the first term is the regular term, representing the prior information of the original image, and the second term is the fidelity term, which is used to ensure the similarity between the denoised image and the observed image. The regularization parameter is used to balance the regularization term and the fidelity term in the model. When the optimal value is taken, the effect of image restoration is the best.
Image denoising is an inverse problem of recovering clear image from noisy image.
In denoising method, variational method models the inverse problem as an optimization problem of energy functional.

The gradient descent equation of energy functional is
The initial value is set to ug  , and the image is iteratively calculated by gradient descent method until an optimal solution is obtained. The process of image information processing: firstly, the image is acquired by the system perception module, then the edge is searched, and the shape recognition, relative position and direction recognition are carried out according to the edge information.
Then, the actual position and direction are obtained by the pixel transformation.
Target recognition is essentially a process of finding the data model matching the target from the established 3D data model library. For ease of description, the model base is represented as a set. An element in a set represents a 3D data model, which in turn is represented as a set: ( , ),( ) Formula: n P -estimated pose.

Ant Colony Algorithm and its optimization
Assuming that there are i N target points in the model base, the probability of ant Ｋ transferring from city i to j is expressed by j P at t (city represents each assembly location point in the assembly space): is a heuristic factor, indicating the degree of expectation of ants from city i to city j .  and  represent the relative importance of pheromones and heuristics, respectively. k allowed is the next step to allow a collection of cities, expressed as k k allowed S tabu   (6) S represents the initial point and the set of all target points; k tabu represents the tabu table of ant Ｋ.
To balance the residual pheromones and heuristic information, the following rules are adjusted: In the formula, represents the pheromone residue factor, while is the pheromone volatilization factor, . denotes the pheromone increment on path in a cycle, and at the initial time ; denotes that the ant Ｋretains the pheromone on path in this cycle. Local pheromone updating rules: When ( is initial pheromone concentration values on each path), the algorithm can get a better solution in the call time, r is the local pheromone Volatilization Coefficient. The detailed calculation steps are as follows: Table 1 The detailed calculation steps of the algorithm Steps Content Step 1 Initializes pheromones and sets other parameters Step 2 When d is less than c N , repeat step 3-step 5.

dd 
Step 4 1 kk  Step 5 Chooses the next target point depending on k ij P and adds the target Point to S and S until ant Ｋ reaches the end point.
Step 6 Updates the pheromone if a kn  .
Step 7 Outputs the current optimal solution. At that time (as a constant), the algorithm can be better solved in the call time. The detailed calculation steps are as Table1 ( c N is Maximum number of iterations, d is Current iterations, a n is Ants number).
When the algorithm falls into the local optimum, the penalty function is introduced to make the pheromone of the current optimal path drop rapidly and reduce the impact of the positive feedback of the next ant search. When the algorithm falls into local optimum, penalty function is introduced .
1) In the process of ant search, when encountering complex terrain, it is easy to form path deadlock. In order to make the pheromone on the path around the trap drop rapidly without affecting the next ant search, a penalty function is introduced.
2) The length of the optimal path has not changed for 20 consecutive generations, and the penalty function is introduced. An improved pheromone updating formula: (13)

General Process of Path Planning
The path planning of assembly process execution mechanism refers to the process that fixtures, work pieces, and tools are located and generated by visual methods in the assembly process, and the generated path is optimized.
The main steps are as Table 2: Table 2 The main steps of path planning Step 1 Obtains and processes the field of view spatial information (unit information e , spatial type S , topological structure T ); Step 2 Generates assembly path 0 P according to the result of FOV spatial information processing Step 3 Judges whether the generated path collides or not, and if there is collision (the estimated pose and path 0 P intersect), the collision analysis is carried out by the system perception module, and then returns to 1; if there is no collision, go to 4; Step 4 Determines whether or not the target points Moves the actuator according to the non-collision path.
Step 6 Saves all points on the path according to the motion information of the actuator.
Step 7 Judges whether the assembly is complete or not, and returns to 1 if the assembly is not completed; if the assembly is completed, the process is completed.

HCU-010
Install the locating pin HCU-020 Install the lower body bushing and valve element HCU-030 Install spring HCU-040 Test spring force HCU-050 Install O-ring and blank HCU-060 Install the upper body solenoid HCU-070 Install the right solenoid ball and spring HCU-080 Install the upper body bushing and valve element HCU-090 Test spring force HCU-100 Install the upper body solenoid HCU-110 Install the left solenoid ball and spring

HCU-120
Install the accumulator spring, piston and upper and lower plates HCU-130 Install 21 connecting bolts of clutch pressure sensor and valve body HCU-140 Install 9 bolts connecting the valve body and pre-tighten them

HCU-170
Insert the wire harness of solenoid valve into the plug of solenoid valve and clutch sensor, recheck the state of engagement After assembly, as shown in Figure 3: Co. Ltd(JAC). This paper will take the assembly of the valve plate on the solenoid valve of DCT dual-clutch transmission of JAC as an example to verify the assembly path planning.
It is verified in this paper that 9 bolts are pre-tightened for huc-140 valve body.
In this example, applying the concept of field space, the work piece type is the valve body of DCT gearbox solenoid valve, the number of different valve body target points may be 9, 12 or 7, the position of obstacles may be blocked by the position of harness and bracket, and the position of cage may be blocked.
In the case of map in this case, when other parameters remain unchanged, several groups of and are selected for comparison. It is found that when ， , the distance average value and standard deviation of the optimal path are the smallest, and the average time is small.
Parameter settings are shown in Table 4.

Examples to verify ideas
According to the detailed description of the assembly environment in the construction diagram, this paper uses the system model shown in Figure 4 to study the path planning of a single manipulator in the assembly environment. Line body, manipulator, fixture and valve body constitute assembly space. Guppy camera monitors obstacles and environmental changes. FANUC robot and fixture are used to assemble valve plate bolts on solenoid valves in static and dynamic environments respectively.   Table 5 shows that the ant colony algorithm can quickly find the initial assembly path of parts in complex environment. The whole calculation process is 0.2800-0.7000s.
The ant colony algorithm has the advantages of swarm intelligence and so on. It has high solving efficiency in path planning.  It can be seen from figure 7 that the iterations of ACS algorithm and this algorithm are 117 and 63 respectively, so the convergence speed of this algorithm is faster than that of ACS algorithm, control group 2 and control group 3, and the final shortest path length of convergence is better than ACS algorithm, control group 2 and control group 3.
As shown in Table 6. Compared with Ant Colony Algorithm, Genetic Algorithm has faster convergence speed but longer optimal path. The optimal path of Simulated Annealing Algorithm is almost the same, but its convergence speed is slow. As shown in Table 7. The comparison of simulation data under the three conditions of 7, 9 and 12 target points is given. It can be seen that the number of convergence iterations, iteration time and shortest path length of this algorithm are less than those of ACS algorithm, control group 2 and control group 3.  Table 2, 3 represent control group 3 in Table 2, 4 represent method of this paper.

Conclusion
The accuracy and efficiency of mechanical manufacturing process is the focus of manufacturing process. Monitoring and adjusting the manufacturing process through machine vision technology can improve the accuracy and efficiency. The adjustment of machine vision technology is a necessary way to promote the optimization of manufacturing process In this paper, the single manipulator operation in assembly process is taken as the research background. Considering the equipment resources, the assembly sequence and assembly path planning of products are studied. The product information model based on FOVS is established, and the assembly sequence generation algorithm considering equipment resource information is given. The simulation of assembly planning structure shows that the result of assembly planning is better than ACS algorithm. The product information model in view space is established, the data structure of the product model is analyzed, and the part information model including assembly features is established, which realizes the unified expression of geometric data information, expresses the dynamic constraints of parts and their assembly resources in the assembly process, and reduces the complexity of the path planning of single manipulator in the assembly process.
The online path planning method of assembly robots based on FOVS can further agglomerate and strengthen the core competitiveness and improve the improve the accuracy and efficiency of manufacturing process. It can be applied to any situation in the manufacturing process, not just assembly robots. Its disadvantage is that it does not consider the dynamic change, or the external factors of industrial production system, which is the next research direction.

Declarations Ethical approval
Ethical approval was not required for this study.

Consent to participate
Written informed consent was obtained from individual or guardian participants.

Consent to publish
Manuscript was approved by all authors for publication.

Funding
Funding information is not applicable.

Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Availability of data and materials
The datasets used or analyzed during the current study are available from the corresponding author on reasonable request.