Multi-agent Based Scheduling Method for Tandem Automated Guided Vehicle Systems

Contributions: This study presented a novel scheduling approach to effectively deal with global and local request by minimizing the total traveling time within the individual zone and the waiting time at the transfer points between zones based on the cooperating control of multi-agent. The proposed approach is verified at the AGV system for book printing process as the real-world experiment and it is showed that the scheduling for multi-agent based tandem AGV system is effective Ethics declarations Abstract The way a AGV in each zone in tandem AGV (Automated Guided Vehicles) systems handles the local request for transport while schedules effectively the global request for transport involving the pickup/delivery operation at the transfer point by cooperating a AGV in other zone, affect the traveling time of the AGV (and thus energy consumption), as well as the waiting time of each node in the system. So far, almost all studies on tandem AGV systems focus on the optimal number of partitioned zones (and thus the number of AGVs), the optimal location of nodes into zones, the optimal location of transfer points between each two zones, the design of AGV’s direction to minimize the total handling flow or the traveling distance of AGV in each zone. This study presented a novel scheduling approach to effectively deal with global and local request by minimizing the total traveling time within the individual zone and the waiting time at the transfer points between zones based on the cooperating control of multi-agent. The proposed approach is verified at the AGV system for book printing process as the real-world experiment and it is showed that the scheduling for multi-agent based tandem AGV system is effective


Introduction
Transporting efficiently materials in the manufacturing system is one of the most important issues for improving the system performances.Especially, material handing operations cover nearly 20% -50% of the overall operational cost [1] Using AGV in material handling have advanced highly the traditional material handling systems, since AGV has the advantages of routing flexibility, space utilization, safety and reduction of overall operating cost [2] One of the most important problems in designing AGV systems is the guide-path design.The AGV guide-path configurations discussed in previous research include Conventional/Traditional [3], Tandem [4], Single loop [5], bi-directional shortest path [6] and segmented flow [16].
A tandem guide-path is configured by partitioning all the nodes into single-vehicle, non-overlapping zones.Additional pickup/delivery points are positioned between adjacent zones to serve as transfer points (Fig. 1).Fig. 1 A tandem AGV system [20] In Fig. 1, the circles represent the nodes, and the solid line represent the guide-path of an AGV.The boxes next to each node show the input/output to/from each node.This configuration offers some advantages such as the deletion of blocking and congestion, simplicity of control, and flexibility due to system modularity.It also has some disadvantages including the handling a load by two or more vehicles and thus longer movement times, extra space and cost requirements, resulting from the use of additional pickup/delivery points and conveyors.Amir Salehipour is breaking the problem of designing a tandem AGV system into following five problems [7]:  Decision on the number of required zones, and hence, the number of AGVs (problem 1)  Dividing up nodes into zones, i.e., node partitioning (problem 2)  Determining node layout in each zone, i.e., the optimal location of nodes (problem 3)  Optimal location of transition points between each two zones (problem 4)  Design of AGVs routes'direction, i.e., clockwise or counterclockwise (problem

5)
Studies on these problems is rich enough; problem 1 is the most studied one [8], For problem 2, Bozer and Srinivasan typically studied partitioning of nodes into different zones [9].For problem 3, Amir Salehipour typically studied the optimal location of nodes in tandem AGV systems.[7] For problem 4, [10] and [11] studied the design of material flow handling systems with respect to the pickup/delivery points.The most relevant and important study on problem 5 are [12].In these studies, different objective functions than minimizing total lateness of all nodes(in a zone) have been considered.
In this study, we focus on implementing the cooperation of AGVs together at the transfer points while minimizing the traveling time of AGV within the zone and the waiting time at the transfer nodes.This issue is not included in the above problems.
For the sake of this, the tandem AGV system typically have to be considered as the transportation networks and operated using a multi-agent, or distributed, control approach [13].In such an approach the overall network consists of multiple smaller subnetworks.Each of the subnetworks is controlled by an agent.
In the literature, an agent has been introduced as an entity that is capable of autonomous performance in its environment in order to meet its design objectives [15] The reminder of this paper is organized as follows: in Section 2, we give the dynamics of the system, the notations used in the study and an exact definition of the problem in the individual zones and the overall system.In Section 3, we describe the solution process.Section 4 is devoted to show the effectiveness of problem formulation and the solution algorithm through the real-world experiment of book printing process.The paper ends with Conclusion.

Problem formulation
In this section, we describe the dynamics of the system, notations for modeling the Tandem AGV system and the model for the scheduling within each zone and the overall system.To achieve this, it is assumed that each agent controls the AGV traveling within the zone.
The dynamics of the Tandem AGV system are modeled using Discrete Event Systems(DES) and represent the following behavior: Once a scheduling of tandem AGV system is determined, the AGVs travel until the global request or local request to be of the "exogenous" occur.With this occurrence of event, a rescheduling is done based on the new available information and according to this, the traveling continues It determines the delivery times of all intermediate products and products to any node according to the scheduling, which depends on global and local requests In tandem AGV systems the agents are geographically distributed and it is difficult to develop the uniformly optimization model, since each agent only performs the optimization based on its zone information.To deal with this issue, the global scheduling problem is decomposed, in a hierarchical approach, into sub-problems.
Each of these sub-problem is solved separately and based on information sending by every agent, global optimization is done.The aim of each agent's optimization is to minimize the traveling time of AGV within its zone as well as the transporting order for intermediate products and products and the waiting time of AGVs at transfer points In order to restrict the exchanged information among agents, they only share the predicted times for delivery and due times for pickup as the necessary information to reach an agreement.
2.1 The system dynamics DES are defined as those in which the events cause a transition from one state, k y , to another state, 1 k y  , of the system.In addition, the states can assume non-numerical or logic values [17].Let the set of all admissible events, namely the space of events, hereafter indicate as SE, whereas the set of all possible system states, namely the space of states, hereafter indicate as Y .The dynamics of tandem AGV system is given by the following state equation: Which provides the ( 1) k  th state, In tandem AGV systems, the event set SE can be defined as the union of the set of "internal events" In addition, the set exogenous SE gathers all the events requiring the calculation of a new scheduling such as the occurrence of new requests, the occurrence of delays, the changes in the AGV system, etc.The state variables x represent the positions(in which node) and the logic states(loaded/unloading/traveling) of the AGV in tandem AGV system.
In more detail, the occurrence of the internal events are related to the scheduling and at the occurrence of any internal event, k e , the clock value is updated to its occurrence time, k  , and the new state is determined.The exogenous events, l e , may stochastically occur at  With addition of node 0 and node 21 n  , as the AGV's initial start point and end point, within the zone i , the node set will became as, The numbers of nodes corresponding to local request and global request are defined : ,1 ,1 The pickup and delivery points in the zone i are, respectively, including into two sets as x   ; AGV moves from node m to node m , else 0 Initially, 0 0 i Y  and 0 And then the objective function and constraints for the scheduling of AGV at individual zone can be written as; Subject to: i mm 0, , , 1, ,(2 1) ; i mm ; , ; ; ; , ; , i mm i mm i mm i mm , i mm x   , i mm , , 1 1, Constraints ( 2),(3) define the multi-load AGV loading condition .The load on multi-load AGV will be increased or decreased by 1 after visiting any pickup or delivery point.Initially, the multi-load AGV starts travelling from node 0 and it follows a pickup point while moving on a guide path.The AGV is allowed to move to any delivery point or pickup point after picking upload from its first pickup point.
before delivering the job to the last node, the multi-load AGV will deliver the job to the second last node.The arriving time of each node estimated by considering the arriving at current node, the pickup or delivery time at current node and the traveling time between the current node and the next node.
Constraints (6) ensure that a multi-load AGV will visit every pickup point once, and constraints (7,8) represent that if an AGV gets an entry into a node, then the AGV will exit that node and if a multi-load AGV visits any pickup node, then it will go to the associated delivery node also.Constraints (4,5,9) define that each multiload AG will make the first visit to the pickup node and last visit to the delivery node.
Constraint (10) present that the load carried by the AGV will not exceed its capacity.

Formulation for the scheduling problem in the overall area of AGVS
We define the scheduling problem in the overall area of tandem AGV system as the problem formed by the aggregation of the local control problems including the definition of the consensus input and output (11～25): , 1 min ( ) and constraints ( 2)-( 10) 3.Solution process

Augmented Lagrange formulation
The scheduling problem in the overall area ( 27) is not separable into sub-problems of only each agent i due to consensus constraints(28).In order to remove the consensus one from constraint set, we represent the set of two zones associated with and add to the objective function in the form of additional linear cost terms, based on Lagrange multipliers and additional quadratic terms.[18] The augmented Lagrange function is defined as ( , , ,  , , ) Where c is a positive constant and , in ji  is the Lagrange multiplier associated with the consensus constant ,, By duality theory [18], the resulting optimization problem follows as maximization over the multipliers while minimizing over the other variable i X ( , , , max min , , ) Subject to, for 1, , i n  K ,constraints( 2)-( 10) of zone i .
Under convexity assumptions on the objective functions and affinity of the scheduling formulation constraints, it can be proved that a minimum of the scheduling problem in the overall area (27) can be found iteratively through repeatedly solving the minimization part of (30) for fixed Lagrange multiplier, followed by updating of the Lagrange multipliers using the solution of the minimization, until the Lagrange multipliers do not change any more from one iteration to the next [19].

Distributing the solution approach
The iterations to compute the solution of the scheduling problem in the overall area based on the augmented Lagrange formulation (29) include quadratic terms and can therefore not directly be distributed over the agents.To handle this, the non-separable issue(29) can be approximated by solving n separated issues of the form: .
The variables ()   , s out ij  are received by agent i through communication with agent j , which computed these variables for its variables for its consensus constraints with respect to agent i .The general multi-agent solution algorithm that results from this comprises at some event k the following: (ⅰ) for 1, i n  K , the agents cooperatively solve their scheduling problems in the following iterative way: (a) Set the iteration counter s to 1 and initialize the Lagrange multipliers ()   , tt   computed at the current iteration s for each agent i j  N that has solved its problem before agent i in the current iteration s , and given the previous information () , rr  of the last iteration 1 s  for the other agents, agent i solves problem (31) using the following additional objective function , , Agent i sends to each agent i j  N the computed values ,( , (c) Update the Lagrange multipliers.
  (d) Move on to the next iteration 1 s  and repeat steps (a)-(c).The iterations stop when the following stopping condition is satisfied: Where  is a small positive scalar and   denotes the infinity norm.The one important thing is that satisfaction of this stopping condition can be determined in a distributed way, because each individual component of the infinity norm depends only on variables of one particular agent [14] (ⅲ) The agent implement the traveling of AGV until the next event is occurred.
(ⅳ) The next scheduling is started.

The real-world Verification
We provide the real-world experiment at the tandem AGV system of book-printing process and evaluate our proposed approach.
The physical layout of the first floor at the tandem AGV system is showed in Fig. 2 Fig. 2 The physical layout of the first floor at the tandem AGV system for book printing process 3 railed-guided vehicles(RGVs) are employed to transport a tremendous amount of pallets and communicate with the control panel with wireless devices.A RGV is operated on a linear track to transport pallets between workstations.In each zone, 20 workstations(nodes) are located along both side of the track.pallets are queued at workstations, and will be picked up from and delivered to their destined workstations on either sides of the track.This is achieved by the RGV via the two-sided loading/unloading operations.The RGV and workstations are equipped with roller decks to support the two-sided loading/unloading operations.When the RGV is docked to a workstation, the roller decks rotate accordingly forward or backward to load or unload pallets sequentially, The RGV can carry 2 pallets at a time.
We perform simulation experiments by cooperating at transfer points between two adjacent zones for minimizing the waiting time to verify the effectiveness of the proposed method and the node numbers are one to twenty at zone 1, thirty-one to fifty at zone 2 and the transfer point numbers for pickup are twenty-one to twentyfive at zone 1, and for deliver are twenty-six to thirty, and for pickup are fifty-one to fifty-five at zone 2, and for deliver are fifty-six to sixty.
The scheduling in each zone is solved by using particle swarm optimization(PSO) algorithm and the program developed and run in C++.In order to verify the results of the multi-agent based scheduling method, they were compared with those of the First-Come-First-Served(FCFS) strategy.
The velocity of vehicle is 0.9m/s, all the distance between nodes is 6m.And Since this tandem AGV system is modeled using DES, when the maximum number of global transport requests and local requests initially is 8, the scheduling for vehicles is performed and once the exogenous events will occur, the system is rescheduled.In table4, 5, 6, the waiting time at the transfer point according to FCFS strategy is greatly varied according to the order of global requests to be occurring.Here, it is considering that global requests are occurred at the end of 8 requests.In table 4, 5, 6, We can see that the total traveling time and the waiting time at transfer points using the proposed method are smaller than FCFS strategy.

Conclusions
We have proposed a new multi-agent based scheduling strategy for minimizing the traveling time of a AGV within each zone and the waiting time of transfer points by the cooperating control of AGVs in the tandem AGV system.The proposed approach is verified at the AGV system for book printing process as the real-world experiment.
Results show that our strategy is capable of producing better solutions, compared with FCFS strategy.
In the future, we are finding better cooperating strategy based on multi-agent in tandem AGV system.

.
The first gathers the events characterizing the normal dynamics of the tandem AGV system as determined by the scheduling; the event arrival e represents the arrival by AGV at a node, the event loading e represents the beginning of the loading(resp., unloading) operation of a pallet, and the event departure e represents the departure of a pallet by AGV from a node.

iPiTTQXYt 2 . 3
Nodes set for pickup and delivery points for global requests at i i LP Nodes set for pickup and delivery points for local requests in the zone i h Index for global requests i m Pickup or delivery time of node m in the zone i 0 Time at which the AGV leaves the initial node in the zone i i m Time at which the AGV arrive at node m in the zone i , i mm T  Time at which the AGV in the zone travels from the physical location of node m , m L , to the physical location of node m , m L  (for each pair of m and m in the i N ) Capacity of AGV , i mm x  Movement of AGV from node m to node m in the zone i i Set of movements of AGV in the zone i i m Load at AGV when it leaves node m in the zone i Traveling time of pallet from zone i to zone j , h ii Predicted delivery time of global request h announced by i to j , h ij r local deadline of global request h required by i from j node m and node nm  Pickup and delivery location of m th request in the zone Lagrangian multiplier weighting the difference between , The scheduling formulation at individual zone

Table 1
represents the case that the global request is one and local requests are seven, table 2 the case that the global requests are two and local requests are six, table 3 the case that the global requests are three and local requests are five.Table 1 the case that the global request is one and local requests are seven

Table 2
the case that the global requests are two and local requests are six

Table 3
the case that the global requests are three and local requests are five For each case, 10 experiments are iterated, and the mean values of total traveling time and the waiting time are compared with FCFS values.Table4represents the results in the case that the global request is one and local requests are seven, table5the results in the case that the global requests are two and local requests are six, table6the results in the case that the global requests are three and local requests are five.Table 4 the results in the case that the global request is one and local requests are

Table 5
the results in the case that the global requests are two and local requests are Table6the results in the case that the global requests are three and local requests