Determining natural pathway for side-by-side robotic wheelchair in passing pedestrian ﬂows using dynamic density

—Side-by-side is the preferred mode of walking for pairs compared to Leader-Follower mode in which one follows the other. Yet, in passing, this walking mode tends to occupy more space on the pathway and reduces the travel space of pedestrians in the opposite direction. Therefore, humans often intuitively consider solutions to optimize the balance between side-by-side walking mode and moving space for others in passing. This is also a problem that designers of side-by-side robotic wheelchairs often have to solve. This paper proposes a novel model to determine natural navigation pathways for side-by-side robotic wheelchairs to pass multiple pedestrians walking in the opposite directions, mimicking human passing behaviors by taking into account the habit of moving with the ﬂow and density in moving (called dynamic density). Based on an experimental observation and data collection, the model was developed and then validated by comparing the pathways generated by the model and the natural moving plans of the pairs in the same situations in a corridor. The simulation results show that they are highly similar.


I. INTRODUCTION
Among the recent research directions on the development of robotic devices to support elderly and the disabled, developing robotic wheelchairs is an area that has attracted the attention of robotic researchers. As a result, a variety of assistant wheelchairs have been developed, from the development of brainwave control systems to autonomous navigation systems [1], [2].
Following this research trend, developing wheelchairs being able to autonomously move alongside a caregiver, mimicking the walking habits of pairs is a relatively new topic. Due to causes such as impaired mobility in the elderly, or the temporary or permanent loss of control in people with disabilities, etc.; wheelchair users may have difficulties in controlling them. In those situations, the wheelchair users need support from a caregiver to control the wheelchair whenever they want to move from one place to another place. This leads to an extra burden for caregivers. Moreover, the total reliance on a caregiver's control can also lead to inconveniences, such as caregivers being distracted by simultaneously moving and controlling wheelchairs, which can lead to sub-optimal decisions in crowded environments, for example in choosing traveling route or moving velocity, or not easily maintaining the side-by-side walking habits. Therefore it may reduce the comfort of everyone, including wheelchair users, caregivers, Vinh The Nguyen and Thang Duc Tran are with Institute of Information Technology, Vietnam Academy of Science and Technology. I-Han Kuo is with Unitec Institute of Technology, New Zealand.
vinh.nt@ioit.ac.vn and pedestrians involved in various scenarios. One of the approaches to tackling these issues is to develop a robot that can mimic the human walking behaviors, in which the robot should be able to help the users move alongside their caregivers in the same way that they themselves walk like normal pairs. This type of robotic wheelchair is called sideby-side robotic wheelchairs, as side-by-side walking mode is the preferred walking habit of humans [3], [4].
The main idea about developing side-by-side robotic wheelchairs is simple, but the technical solution is not [5], [6]. To succeed in that, multiple factors need to be considered simultenesouly during their movements. The wheelchairs need to able to maintain a stable distance to the caregiver, move at some comfortable velocities, minimize velocity variations, and effectively avoid obstacles, etc. [7], [8].
Moreover, to be truly accepted by humans when using the wheelchairs in crowded environments, side-by-side robotic wheelchairs need to respect the comfort of people involved in the scene [9]. This not only includes maintaining the preferred side-by-side formation between caregivers and wheelchair users [3], [8], but it also should not make excessive use of space, or block the pathway, or hinder the movement of others more than is normally accepted.
In order to find answers to these requirements, we need to understand the way that people respond in typical cases. In this paper, we focus on tackling a common problem that sideby-side robotic wheelchairs have to deal with, namely passing multiple pedestrians walking in the opposite direction in a corridor. Assuming that a side-by-side robotic wheelchair is moving with a caretaker in a passageway, Figure 1 illustrates four main modes in which a walking pair passes multiple pedestrians walking in the opposite direction (a to d).
In modes (a) and (b), the pair maintains their connection for the entire passing time. The pair moves to only one side of the corridor, left or right respectively, and no pedestrian goes into the gap between the pair. Moreover, the pair also tries to maintain side-by-side formation whenever possible. It means that the pair only switch to the leader-follower mode where one goes after the other when there is not enough space left to sustain their side-by-side formation. In mode (c), each member of the pair finds his/her own way to pass the pedestrians. One or more pedestrians may interfere with the pair by moving through the gap between them. In mode (d), the pair maintains their connection and no pedestrian goes into the gap between the pair, however the pair wriggles through the gaps in between the crowd in passing. As can be seen from our study, which is described in detail in Section III, all the situations observed happened in passing modes (a) or (b). This phenomenon has also been observed through previous studies [?], [10], [11]. Here, two main things can be seen from the peoples' natural walking habits. First, pairs normally avoid modes (c) and (d) in most situations, and the pedestrians walking in the opposite direction also do likewise. Secondly, pairs not only usually select modes (a) or (b) to follow, but they also try to maintain the side-by-side formation between two members of the pair at the optimal state possible; i.e. although pairs normally respect the movements of pedestrians walking in the opposite direction, they balance it with the need to maintain the side-by-side formation.
While many studies have been conducted for developing the navigation function for robots in crowded environments, most of that research has focused only on robots moving independently, or leading a group, or following a group [12], not on robots designed to travel with a companion. Thus, if these models are applied, the robot may randomly select one of the modes (a), (b), (c), or (d). Moreover, the robots do not have a mechanism to help the caregivers maintain a friendly connection with the wheelchair users as human pairs. Among the few studies which were developed for a robot moving together with a human as a pair, we found three main approaches [8], [13], [14].
Sato et al. [14], [15] suggested a technical solution for passing pedestrians for wheelchair robots moving alongside a person. By default, the robot moves alongside the caregiver in a side-by-side mode. Whenever it detects one or more pedestrians walking in the opposite direction, the robot changes its moving mode to leader-follower mode in which the robot follows the caregiver. The advantage of this method is that it allows the robot to maintain the connection with the caregiver, i.e. no pedestrian will interfere with the pair. However, the side-by-side mode cannot be maintained during the passing, and the robot passively follows the pathway chosen by the caregiver like a tail. This is not an natural pattern, especially in cases where the corridor is not too narrow, or not too many pedestrians walk in the corridor, or follow each other in a line; i.e. the side-by-side walking formation is canceled while it is still maintainable.
By employing social-force model and proxemics concepts, Ferrer et al. [13], [16], in another approach, developed a navigation model for a companion mobile robot. Their model mainly focused on maintaining a comfortable distance between the robot and its companion, whereas the robot tries to keep away from other pedestrians. Physically, this model is able to help the robot pass multiple pedestrians. However, one of the main disadvantages of this model is that the robot has no mechanism to select one of the four modes (a), (b), (c), and (d); i.e. with the model, all these modes are equal. It accepts mode (c) and (d) in which some pedestrians can move into the gap between the pair, or the pair wriggles through the gaps in between the pedestrians, and does not understand that humans prefer modes (a) or (b) in passing.
The last approach, developed by Nguyen et al., based on Morales et al.'s model [2], [8], incorporated ten factors that influence pairs in passing, allowing the robot to understand that it should maintain the connection with the caregiver and also maintain the side-by-side walking formation whenever possible. The limitation of this model is that it was developed for passing only one pedestrian in the corridor. It means that if we employ this model in the scenarios of passing multiple pedestrians, the model may randomly choose one of the modes (a), (b), or (d). In other words, it still lacks a mechanism to propose a natural pathway in passing pedestrian flows.
Another thing that can be seen is, to maintain the sideby-side formation, pairs need to occupy a pathway in the corridor with the width w p wider than the pathway needed for a single person, i.e. more than the width needed if they switch to leader-follower mode. However, pairs also understand that they need to respect the moving space of the other pedestrians walking in the opposite direction, as it may be too small if the side-by-side formation is always in an ideal state. Therefore, here everyone tries to reach a state of equilibrium by trying to choose the optimum value of w p for each situation. This width w p illustrated in Figure 2 varies depending on the state of the people in the corridor, i.e the shape of the pathway that the pairs process depends on each real situation. The question here is, how the robot can mimic the pathway that each particular pair processes; i.e. how to determine the shape of the pathway P w each pair follows.
To the best of our knowledge, the navigation model developed in [8] is one of the state-of-the-art models for a sideby-side robotic wheelchair. In this paper, based on this model, we present a novel model to help the robot understand that, in a relatively narrow corridor, pairs normally choose mode (a) or (b) to pass pedestrian flows. Once mode (a) or (b) has been determined, the model helps to determine the shape of the path in the same way that pairs follow. In this situation, the model, at each time t, will predict a suitable w p for the time t + 1 that matches the real w p that a pair of two normal people processes. In our study, by observing the pathway of walking pairs in a lobby, the data of the positions of people in the lobby was collected. Then the model has been developed, calibrated, and validated. The results show that, at each time t, the model can predict w p values for the time t + 1 highly similar to the real values.

A. Human walking habits
The question of what has influenced people's behavior while walking has recently attracted the research attention. Helbing & Molnar [17] discovered some main hidden forces that affect pedestrians when they walk from one point to another. They include attractive forces, such as destinations that make people want to get closer; or repulsive forces, such as obstacles that make people try to stay away.
With pairs, Costa et al. [3] commented that when pairs move, both members of the pair usually prefer to walk side by side (side-by-side mode) rather than one going first and the other following (leader-follower mode). The side-by-side mode was later recorded in many other studies, in which two members of the pair not only want to walk in side-by-side mode but they automatically try to synchronize with each other to maintain this mode without exchanging words or actions [11], [18]. Hall [19], on the other hand, coined the concept of proxemics in which the distances between people are not randomly determined but followed some hidden rules, called hidden dimensions. Depending on the relationship between people participating in a group, social interaction, the status of each person, etc. the distances between the people in the group are automatically selected in certain ranges. These rules are not only being applied in static environments but also in interactive and dynamic environments [4], [20], [21]. In dynamic environments, Hoogendoorn and Bovy [22], Ferrer and Sanfeliu [9] proposed a study on pedestrian routing options. Henderson [23] presented a study of crowd movement behaviors in which people move like liquids. Alternatively, moving with the flow is also a preferred choice for pedestrians [24], [25]. In general, there are hidden rules that control human habits in standing and walking. These hidden rules are prevalent but not confined to some small groups of people. Depending on the circumstances, the optimal values for each habit may differ, but often people try to reach to the optimal state of each value in order to make people feel more comfortable.

B. Navigation model for companion robots
Recently, there have been many efforts to develop a robot that can move along with another person as a friend. Moreover, there have been studies focusing on developing a wheelchair robot that helps caregivers maintain a friendly relationship with wheelchair users, while reducing the workload for carers and improving the comfort of everyone involved. Basically, these studies are aimed not just at solving purely technical problems, but also improving human comfort by applying generally accepted hidden rules of movement. Starting with some research, e.g. Kirby [26], a robot that use the leader-follower mode in which the follower is the robot was developed. Then in later studies, Kobayashi et al. [14], [15] employed side-by-side walking mode in their robotic wheelchair. Zanlungo et al [27], Robicquet et al [28] also pointed out that the companion robots working in the human environment need to be able to understand and replicate pedestrian walking behaviour, otherwise they cannot be accepted by humans. Some prediction models were developed to anticipate the future actions and trajectories of the caregivers and pedestrians. Some preliminary effort began with Prassler et al. [29], they developed a prediction model for their side-by-side robotic wheelchair, in which the robot can predict the future positions of its companions. The situation is more complicated in dynamic environments where the robot has to deal with a lot of people moving around. There are some major approaches. Some researchers have developed several Social Reward related cost functions, e.g. Dalmasso et al. [30] or Ferrer et al. [10], [13], [31], others try to determine the future trajectories of pedestrians [28], [32], [33], whereas some others have tried to apply their understanding of the pedestrian flow mechanism [34]. Each approach has its pros and cons. Using Social Reward related cost functions, there are many factors affecting the movement that have not been discovered and evaluated, and what affects the flow of pedestrians is not fully understood. Based on statistics, there was not enough sample data to train motion prediction models in many situations. Therefore, those models still need to be further developed.
As mentioned above, we believe that the model developed in [8] is one of the best models currently applicable to side-byside robotic wheelchairs. It can be considered as a combination of the Social-Force model, along with the proxemics concept and other personal and social factors like face-to-face talking habits. Hitherto, the model was developed for passing one pedestrian walking in the opposite direction. In this study, by applying our understanding of pedestrian flow and the dynamic density factor, the model is developed to determine the natural pathway for a side-by-side robotic wheelchair passing pedestrians in a corridor by mimicking the pathway of a real human walking with his/her partner.

III. USER STUDY
The aim of the user study is to understand human walking habits when they walk as pairs in a corridor facing multiple pedestrians walking in an opposite direction. From that, if we replace one person in the pair with a side-by-side robotic wheelchair, then the wheelchair should enable a wheelchair user to move in a pathway in a same way that user would walk by himself. Following the other studies mentioned above, that brings comfort to the wheelchair user and all the surrounding people, including the caregiver and other pedestrians, feeling that they can walk and act as they normally do in normal situations but not with atypical situations, i.e. with an appearance of an object that might act in an abnormal way that people cannot predict.
Aside from collecting the pathway P w that pairs take to pass pedestrian flows by recording their positions in the corridor, we also wanted to identify the factors which affect their decisions in choosing the pathway.

A. Data collection
The user study was carried out in a simulated corridor with a width of 2.0m and a length of 11.0m. Learning from previous experiments [8], [35], we considered that this width is a common corridor size, and the length of the corridor provides enough space to simulate a passing scenario between a pair and multiple pedestrians in it. Two laser scan finders (LRs) were put in the middle of the corridor, and the distance between two LRs is 5.9m. The height of each sensor above the ground is 30cm. We also designed a parallel passage so that participants can circle around to keep the flow of people moving in the  Figure 3.
Here, two LRs with a maximum radius of 5 meters were set up at 10Hz scanning speed. The starting point and the destination of the pairs were A and B respectively, in which A and B were located at both ends of the corridor, in the middle of the width of the corridor. A flow of n pedestrians walking inversely to a pair i, j, from B to A. All the people knew in advance that they needed to walk from A to B or B to A, depending on their role as a member of the pairs or as a pedestrian. The tracking algorithm of Leigh et al. [36] was used to record the positions of all the people in the corridor. To increase accuracy, we prepared and asked all participants to wear white pants. In Fig. 3, LR scanning area is shown in light yellow color. Aside from that, we used a video camera to record the walking sessions. Because the sensor and the tracking algorithm have some limitations, in case of doubt about the accuracy of the position of the people in the corridor, we used images from video to correct the values manually.
The walking rules for the pairs and pedestrians were as follows: • Each pair (i, j) was asked to walk from A to B. All the pairs were asked to act as companions pretending that they were strolling around a building or around a park. They were instructed to maintain a cordial connection appropriately as close friends, but not so intimate as lovers.
Each pair was permitted to have their own conversation during the move. • In the opposite direction, some people acted as pedestrians walking in a continuous stream from B to A. In addition, they were also asked to walk like strolling on a street or in a park, meaning not in a hurry. There was no relationship between pedestrians except that they happened to walk in the same corridor and in the same direction. • Everyone was asked to actively walk in a natural way to their destination, and respect the movements of all the others in the environments as humans normally do.
By this arrangement, we simulated real-life situations in which a caregiver escorts a patient / disabled person wandering or moving from one location to another.
Seventeen volunteers participated in the study. They were all undergraduate students from three universities, including eight women and nine men. We did not ask their ages, but No   Passing mode  Quantity  1  a  12  2  b  38  3  c  0  4 d 0  Figure 1 estimated that they are all from nineteen to twenty-two years old. Although a few a few knew each other in advance, basically many of them did not know the others before they joined our study. Those students had no research relationship to our project, and we did not tell them about our previous studies and the final aim of our research. The volunteers only knew that they came to help us to develop a computational model about human walking habits. Everyone was informed about the rules of participation in advance.
We divided the user study into three stages. In the first stage, each pedestrian was asked to move alone and separated from others. In the second stage, in order to simulate a crowded corridor, some participants were asked to start to walk almost simultaneously from B, forming a crowd of two or three people. In the last stage, we maximized the crowds by asking up to four participants to start simultaneously. Therefore, for the second and the last stage we had crowds with numbers of pedestrians ranging from one to four. Because of the sensors and human detecting algorithm limitations, we ask participants not to form crowds of more than four. In addition, pairs were asked to wait at A if another pair was still walking in the corridor and had not yet reached B. Similarly, when each pair approached the end of B, pedestrians temporarily stopped and waited at B, aiming to initiate a completely new walking session for each pair. Thus, only one pair at a time were in the corridor. For each move from A to B of a pair, we counted as one session.
One thing we want to clarify here is that although the participants were asked to start simultaneously and formed pedestrians' groups, they are not a combination of people who had some close relationships with each other but they were still independent pedestrians. The pedestrians' groups were just formed randomly, simulating a situation where some people happen to have the same need to move from B to A, and on the way they share the corridor's space with each other.
We randomly selected participants to form the pairs in which each session had two to three pairs. The rest acted as pedestrians.
Some sessions were eliminated from the data where one or more people did not obey the walking rule. For example, some pairs' members forgot to maintain their relationship, or some pedestrians suddenly stopped in the middle of the corridor and obstructed the pedestrian flows. The data was also deleted in cases where the sensor and/or tracking algorithm lost too many participants' positions and could not be recovered manually via the video. Finally, a total of 50 sessions were recorded in the data set. The data is summarized in Table I.

B. Trajectory standardization
To make it easier to analyze the data, we applied a standardization process for collected trajectory sets as follows: • We defined the time when the last pedestrian passed the last member of the pair as the final point of time t e for each set of trajectories T . Each set of trajectories T included all the trajectories of everyone in the corridor in a particular session. • All the recorded trajectories in T after t e were discarded.
Because here we only focused on the passing process, and after t e , the passing process was totally completed and no one needed to handle the passing process anymore, the pairs could freely occupy any space in the corridor as they wanted, therefore we did not need to determine the shape of the pathway after t e . • From t e , we moved backward for tn seconds, where tn is a natural number, until we reached t b where t b is the moment that the pair departed from A. Yet, the difference between t b and t e may not be an integer. In that case the fraction less than 1 second from the time t b was cut off, in order to simplify the calculation. This means we only kept all the segments in each T from t e − n T to t e ; all the segments outside this time range were removed from the dataset. • After all the trajectories T were removed all the unused portions, we set t b of each T as the starting point of the T , i.e. pairs and pedestrians started walking from t b .
C. Observation 1) Passing modes: It can be seen from the user study that, although all the pedestrians walked independently, to pass a pair in a session, pedestrians chose to follow each other spontaneously without having to talk orally in advance or during the move. Everyone in a group chose the same side, left or right, to pass the pair. The next groups followed the previous ones and avoided the pairs on the same side, and all chose the same side as the first group. In other words, once the first pedestrian chose one side, left or right, to pass the pairs, all the others behind followed him/her to pass the pairs on the same side as the first pedestrian. On the other hand, once one pair chose to pass the first pedestrian on the left, then they would keep passing all the other following pedestrians on the left. It happened similarly if at the beginning one pair decided to pass the first pedestrian group on the right side. That means, without any discussion in advance, everyone automatically chose passing modes (a) or (b), and no one chose mode (c) or (d).
The passing process is illustrated in Figure 4. It can be seen that the pedestrians chose to follow the others moving in the same direction without any discussion beforehand.
As explained in the previous section, in the study, basically the participants did not know the others in advance, and they had no idea about the previous relevant studies or our target of this study either. However, all the participants handled the passing situation in a similar manner, whether they played the role of independent pedestrians or members of pairs. We believe that this passing strategy was chosen for the benefits Fig. 4: The process of passing pedestrian flows of pairs in the study it offers. For example it helps the passing process to be done easily, quickly, and safely; i.e. to minimize hassles for everyone. Therefore, we believe that these passing processes are common, not limited to a specific group of people.
2) Pairs pathway: It can be seen that, when a pair achieves the optimal state of side-by-side mode, they will occupy the width of the corridor to the maximum extent w pmax . That means the remaining width, i.e the passing space for people walking in the opposite direction, will be at a minimum. Conversely, when the pair is in the leader-follower mode, the width of the corridor they occupy will be minimal w pmin and the width of the corridor for those traveling in the opposite direction will be at the maximum. The minimum width w pmin of a pair is not smaller than the minimum physical width of a member of the pair. In this user study, another thing that can be observed is the space occupied by the pairs and the pedestrians during the move, or the shape of the pathway P w that pairs occupy in the move from A to B. When the distance between a pair and the first person in the opposite flow is large, pairs normally occupy w pmax . As the distance between them is smaller, both sides automatically adjust the amount of space they occupy in the corridor, i.e the width w p that the pair occupy is reduced. If the corridor is not very crowded, for example the pedestrian group contains only one pedestrian, then w p is larger than in the situations where the corridor are crowded with a lot of people moving simultaneously together as they take more space. That means w p is adjusted depending on the numbers of the pedestrians simultaneously walking in the corridor, i.e. the shape of P w is changed following each real situation in the corridor. We find that this adjustment is completely automatic without verbal or gesture communication between pairs and opposite pedestrians. On the other hand, we can also see that if pairs try to occupy a large value of w p in a crowded corridor, then the moving space for opposite pedestrians is smaller than normal. This will reduce their walking velocity and make them feel uncomfortable as their walking space is narrower than normal [8]. However, if there are not so many people in the corridor but w p is smaller than normal, pairs will also feel uncomfortable. This shows that w p needs to be balanced to optimize everyone's comfort. w p values vary depending on the number of pedestrians in the corridor, i.e the density of the pedestrians in the corridor, and the distance from them to the pairs. Therefore, in order to determine the shape of P w , the model needs to find two answers for two steps. First, the robot needs to determine the passing mode that everyone will process, (a) or (b), and eliminate (c) and (d). Secondly, to mimic the pathway P w that pairs will take in each situation, at each time t, the model needs to find the most natural value w p at time t + 1 for its next movement.

A. Setting assumptions
In this study, we assume the conditions for the model as follows: • Two people are walking together as a pair in a corridor, from one end of the corridor to the other (A to B).
• The two of them have an equal role, keeping intimate relationship while moving to the destination. • While on the move, they encounter a number of pedestrians walking in the opposite direction from B to A. The number of pedestrians is unknown in advance and they are also free to move at their own will. • We assume that all people in the scene not only consider the best route to their destinations but also respect the movements of the others.
After the model is developed, one of the pair will be replaced by a caregiver, and the other will be replaced by a side-by-side robotic wheelchair with a person sitting on it. The mission of the model installed on the robot is to identify the P w that the robot will traverse as if the person on the wheelchair walks according to his own will.

B. Passing process hypothesis
Based on the above assumption, we assume that a person s (s = i, j) will process the following steps: • As soon as a pedestrian m 1 is detected in the opposite direction, the person s needs to determine which passing mode it needs to process, mode (a) or mode (b). After mode (a) or (b) is determined, Step (1) begins. The person s can assume that all the other pedestrians behind m 1 , if any, will follow him. plans of each individual pedestrian m k (k ∈ (2, n) as many possible cases may happen; the number of cases to be calculated will increase exponentially with the number of pedestrians in the corridor. Instead, he should apply a strategy to estimate the parameters that correspond to the entire group of pedestrians in front of him. He will then estimate the status for the entire group in the next step based on the positions in the next step of himself, his companions, and m 1 . We also believe that this is a common human way of thinking. We will describe the solution and the model for this step in section IV-F. • At Step (4), person s compares all the feasible moving plans P. The plan that he believes achieves the optimal solution for all the people in the scene is called the optimum plan P * . After P * is identified, the next positions of the pair are identified and therefore, w p for t + 1 is identified.
• At Step (5), the person s processes his move to the new position in his feasible position set which forms P * . This step is finished at time t + 1. • To continue the next movement for the time t + 2, the person s repeats from Step (1) until he finishes the passing process with all the pedestrians in front of him.

C. The standard navigation model
The navigation model for the side-by-side robotics wheelchairs developed by Nguyen et al. in [8], [37] is able to find a pathway for the situation that a pair walking together facing a pedestrian walking in the opposite direction. Because this study employed this model, we briefly describe it as follows.
In the model, ten factors that affect the pair in walking are identified and incorporated. They include the distance to other pedestrians and obstacles O, moving toward subgoals S, relative distance between two members of the pair R d , relative angle R α , relative velocity R V , relative vision R β , moving acceleration M a , moving velocity M V , angular velocity M w , and friendly link F L [8]. When two people walk together as an intimate pair from one place to another, basically they want to choose the next positions that allow all the above factors to achieve optimal values. Similarly, after removing the factors related to the interaction between the two in a pair, solitary walkers also want to achieve an optimal state with the rest of the factors.
Assume that i is one member of the pair positioned atp i , the member i has a utility of U i toward his partner positioned atp j and a moving pedestrian positioned atp m as follows: The function f O was modeled as follows: f Mw functions were modeled as follows: In Eq. 2 and 3, x is a variable, it is one of the eight factors O, S, R d , R α , R V , M a , M V , and M w ; each individual utility f has a unique set of a, b, c. a, b, c are constant co-efficients, except c Mv in f Mv is calculated as follows: here, t − 1, t and t + 1 denote the past time, the current time, and the predicted time respectively.
The friendly link utility f F L was modeled as follows: • µ t and µ t+1 are the midpoints of the straight line between p i t andp j t and betweenp i t+1 andp j t+1 respectively. • z i and z j are the parts of the Euclidean plane covered by two angles α i and α j respectively. z 1 is the rest of the Euclidean plane after subtracting z i and z j .
Here, A ang is a part of the Euclidean-plane area, determined by the inside area covered by an angle variable α. A tri is the inside area of the triangle determined by three vertices (p s , µ,p m where s = i, j). A EP is the entire Euclidean-plane area.
The above variables and notations is illustrated in Figure 5.
The utility for the pedestrian is as follows: Assume that, at time t, the positions of i, j, and m in the environment are represented by the set P t = {p i t ,p j t ,p m t }. P t+1 = {p i t+1 ,p j t+1 ,p m t+1 } is one positions set among all feasible positions sets that they can move to. Then, at time t, the overall utility of three people for calculating their positions for the next step is as follows: The new positions set P * t+1 is selected if: In this study, we have multiple pedestrians, not only one as in this model, therefore this model needs to be improved to reflect the impact of other pedestrians to the moving decisions of everyone. To employ this model, we choose the nearest pedestrian in front of the pair to be the person m in the above utility. From now on, it is denoted m 1 .

D. Identify pedestrian groups
There are two points that can be seen from the Passing process. Firstly, after passing the pair, the pedestrians can move freely at their will without being affected by the pair. Therefore the model only needs to consider the pedestrians in front of the pair, at the time their passing process is not finished. Everyone who completes the passing process is removed from the model. Thus, m 1 may change from one particular person to another. Secondly, pedestrians do not always spread evenly across the corridor, but can form crowds, called local groups, in some situations. For example if they simultaneously happen to depart from the same location in the corridor, or some people start late but catch up with the people in front of them. It means some pedestrians can be compressed into a small space, while the average density of all the pedestrians in the corridor can be low. Obviously, when more people are gathered in a small area, they will be more affected by each other than when they travel one after another over long distances. Therefore, we need a mechanism to identify these local groups. Here, we consider that "the ability to move in all directions without causing a collision" is the operative factor in identifying local groups. If two pedestrians m k and m k+1 can freely choose any positions that they want for the next step, they are in different groups. We note that a pedestrian can stop right after completing a step. However, if a step is being taken, it is difficult for him to stop halfway. Therefore, if two pedestrians are too close, and if the pedestrian in the front m k suddenly stops for some reason, a collision with the pedestrian right behind him m k+1 may occur because m k+1 needs a certain distance to stop. In this case, m k+1 has to consider the next movement of the former before taking his own next step, therefore he is in the same group with the former. On the contrary, if m k+1 is far enough away from the preceding person, m k+1 may be considered to be able to freely move in all directions in the sense that even if m k stops suddenly, m k+1 has sufficient distance to complete his current step and then stop without causing collision. Therefore, if we call M v k+1 is the current velocity of m k+1 , we define that m k+1 is in the same group with m k if the current distance d k+1|k between m k and m k+1 following the moving direction of the group is smaller than one step and a half d k+1 lim = 1.5 * M v k+1 .

E. Determining passing mode
As soon as m 1 is detected, everyone needs to determine which passing mode they want to process. Based on the observed data, we now know that basically people prefer modes (a) or mode (b). This means that pairs need to decide whether to pass the pedestrians to the left or to the right. To avoid collisions, the pair and pedestrians have to choose different sides. To accomplish this, we employ the work presented by Nguyen et al. in [8] to determine the passing mode by using the subgoal concept and the method to determine the sub-goals of the pairs and m 1 . We only need to determine the sub-goal of m 1 as we assume that basically all the other pedestrians behind m 1 should have the same sub-goal with m 1 . We briefly describe the sub-goal detection method in [8] as follows: where g = (lef t, right), S visible = {S lef t , S right }. leader is the caregiver. Here, S g lef t and S g right are the possible subgoals of the pedestrian m 1 , they are also the possible sub-goals In this model, we also prioritize decision making for the caregiver, because the caregiver is the one who takes the responsibility to lead the robot for a walk. Thus, if agent i is the caregiver, then it is also the leader, and vice versa. Using this method, if the pairs' sub-goal is S p lef t and pedestrian group's sub-goal is S g right , mode (a) is determined, otherwise mode (b) is selected. In the base model that we employ, the factor F L in Eq. 16 was developed to make sure the robot understands that mode (c) will not occur, as everyone in the scene should respect the relationship between two members of the pairs. Similarly, from the observation, based on the tacit cooperating of the caregiver and all the other pedestrians in the environment, it can be seen that mode (d) will not occur if no one change their sub-goals. Therefore, the next mission is to determine the width w p of the pathway that the pairs should take.

F. Dynamic density factor
As mentioned in Section IV-B above, in a crowded environment, it would be unreasonable if before taking a new step, a pedestrian has to calculate all possible movements of all others in the environment in detail. Instead, we believe the better option is to estimate the parameters of the entire group in front of him. In this regard, we make the following comments: would have the greatest impact on the moving decision of them. • Secondly, everyone can see that the movement of one pedestrian not only affects to the other pedestrians walking in the opposite direction but also the other people in their walking group. For example, if one person of the team decides to move to the left, the people behind him may need to take appropriate action to avoid collisions because they are too close to each other. • Thirdly, although theoretically the pedestrian may want to calculate the movements of all the pedestrian groups behind the nearest group, we believe that the influence of the groups behind the nearest group is not very significant in comparison with the nearest group. In addition, in a moving scenario of a pretty narrow corridor where all the groups share a lot in common, for example, they may choose the same passing mode with the groups in the front of them (the same sub-goal), and the same destination. Thus, to simplify the model but still achieve acceptable results, we remove those groups from the calculation. This means that in this model, pedestrians will only calculate their next move based on consideration of the next movement of the nearest group in front of them. • Fourthly, as mentioned above, all persons who complete the passing process are considered to be free to move, which means that they have the full right to use the corridor space without affecting the people who walk in the opposite direction but they are already behind them. Therefore these people will also be removed from the calculation.
With a single walker, we already know some factors that influence his choice for the next step, including the distance to other pedestrians and obstacles O, moving toward subgoals S, moving acceleration M a , moving velocity M V , angular velocity M w , and friendly link F L. From the viewpoint of the pairs, supposing the group has q members that contain m 1 , then m 1 can be considered as the leader of the group. Therefore his factors, including distance to the pairs, subgoal, moving acceleration, moving velocity, moving angular velocity, and friendly link can also be considered as the values for the corresponding factors of the whole group. Thus, the main thing that should be considered is the shape of the group. Indeed, a group extends not only in length but also in the width of the corridor. However, when encountering obstacles, they need to shrink by width to pass the obstacles and avoid collisions. The question of the group is, how much does the width of the group need to shrink? It can be seen that when narrowing horizontally, the distances between the members of the group become smaller, leading to a reduction in comfort related to the distance. Some members going behind will have to reduce their moving velocity to avoid collisions and to avoid the distance between members becoming too uncomfortable. Similarly, with the pairs -the group's obstacle -can also change their shape. Side-by-side mode is the most preferred walking formation of pairs but it also occupies the maximum width of the corridor. By contrast, the leader-follower mode is the less preferred form, but it only occupies a minimum width. Thus, the pairs' question here is how much their width is needed to reduce to maintain side-by-side mode as much as possible, while still allowing them to pass the pedestrians' groups and avoid collisions. Because we assume that everyone tries to respect each other's comfort, everyone needs to find a balance in the answers to these questions. We illustrate these behaviors in Figure 7. In this study, we believe that the balance lies in the density of the groups in moving, i.e. the dynamic density factor. If the pair think that the group occupies too much corridor space, they may want the group to reduce their width. On the other hand, if the pair feels they take up too much space, they might think they should reduce their width more to give way to the opposite side. In order to model the above approach, we give some definitions to describe the density factor illustrated in Figure 8 as follows: We define an Euclidean plane coinciding with the floor of the corridor where everyone walks on this plane. In this plane, the width of the corridor is parallel to the horizontal axis (x), while the length of the corridor is placed parallel to the vertical axis (y) on the plane. Pairs move following y-axis in an ascending direction, which means that the starting point for all pairs is set to p by = 0, and at the final point we have p ey > 0.
• min py and max py are the current minimal and maximal positions of the pair along y-axis respectively. Similarly, min gy and max gy are the current minimal and maximal positions of the group nearest to the pair along y-axis respectively. • min px and max px are the current minimal and maximal positions of the pair along x-axis respectively. If subgoal of the pair is on the right S p right , then min px is the leftmost edge of the pair, and max px is the right side of the corridor; otherwise if sub-goal of the pair is on the left S p lef t then min px is the the left side of of the corridor, and max px is the rightmost edge of the pair. Similarly, we have min gy and max gy are the leftmost edge of the group or left side of of the corridor, rightmost edge of the group or right side of of the corridor respectively, this depends on the sub-goal of the group is S g lef t or S g right . • If there is a overlap of the two segments L p |min px ,max px | and L g |min gx ,max gx | on the x-axis, then the mid gx and mid px mark the starting point and the the min py = argmin(s y ) s ∈ (i, j) A g nc = area(min gx , mid gx , min gy , max gy ) A g c = area(mid gx , max gx , min gy , mid gy ) A p c = area(min px , mid px , min py , max py ) A p nc = area(mid px , max px , min py , max py ) If mode (a) is detected, S p ≡ S p lef t and S g ≡ S g right , that means the pair and the pedestrian groups simply swap their sub-goals from right to left and vice versa respectively, then the values of the variables in the equations from Eq. 22 to Eq. 36 are calculated in the similar way.
We also define: From the above discussion and definition, it can be seen that if the pair and the group keep moving straight forward without changing their current shape, a collision may occur when two conflict areas A g c and A p c meet. Therefore, they should have a desire to avoid that collision by reducing A g c and A p c to zero. In order to do that, in mode (b), the pair needs to increase min px , whereas the group needs to decrease max gx . Finally, the collision will not occur if: From our study, the Eq. 39 may not immediately happen just after one or more pedestrians walking in the opposite direction are detected, but it may happen gradually depends on the value of d c . If d c is large, i.e. the two sides are still far away, it is not necessary for the two sides to reach the status of Eq. 39 immediately. On the other hand, if d c is small, i.e. when the two sides are very close to each other, the status of Eq. 39 needs to be quickly reached.
However, as we assume that everyone tries to respect the comfort of others, even if collisions are guaranteed to not occur, both sides need to ensure that the space occupied by each side reaches an optimal balance.
If we define: are the densities of the pair and the pedestrian group respectively, and the average values of them areρ p andρ g respectively, then we propose our solution as follows: • if A p c + A g c = 0: if ρ p <=ρ p and ρ g <=ρ g : both parties can continue to move forward without having to change anything regarding their density. otherwise ρ p and ρ g should both change and the adjustments towards the optimal ratio ρ p ρ g =ρ p ρ g . If d nc is small, the speed of change needs to take place faster.
• if A p c + A g c > 0: ρ p and ρ g should both reduce and the adjustments towards the optimal ratio ρ p ρ g =ρ p ρ g and A p c + A g c = 0. If d c is small, the speed of change needs to take place faster. We define a new utility f D to reflect the impact of the pedestrian group to the combined utility of three people i, j, m 1 (Eq. 16); f D is calculated as follows: if A g ct + A p ct = 0: here, t and t + 1 denote the values of the variables at the present time and the predicted time. If f D = 0, this utility has no effect to the moving decision of everyone. The two zero values of f D in Eq. 42 are explained as follows: • if the pedestrian group's density is lower than average, they are occupying more space than the acceptable value, then the model does not need to interfere to the overall utility, i.e. f D = 0. Instead, the pair can freely use the other factors (side-by-side formation, relative distance, etc.) to find their optimal solution for the next step without considering about reserving more moving space for the pedestrians. • if the pair's density is higher than average, the pair is occupying the smaller space than the acceptable value. The pair is allowed to expand their space with the pedestrian group's implicit unanimous consent. Thus the pair can increase their moving space to the acceptable value in the next step without considering about the pedestrian group's moving space, i.e. f D = 0. In Eq. 43, if the pair's density in the next step is lower than in the current step whereas a collision may soon occur, f D should receive a large negative value c to inform the model that this may not to be a good choice.
Based on the above reasoning, we model f D in Eq. 42 as follows: and f D in Eq. 43 as follows: here, a, b, and ∆ d are the coefficients, they are constant; ρ pg is calculated as follows: A new utility for the pedestrian group is defined as follows: here, k g is a weight constant. Eq. 16 is amended as follows:

A. Parameter calibration
In order to have a working model, the coefficients in Eq. 43, Eq. 44, and Eq. 45 need to be calibrated. Some other coefficients might need to be re-calibrated as well.
1) We used the tracking data for calibrating the new model. 2) A grid was applied to the navigation plane with cell dimensions of 20x20cm. We developed a simulator for calibrating. 3) One member of the pair was replaced with our simulated agent, which represents our robotic wheelchair. All the next positions of this agent were calculated by the model. At each time t, the positions of its partner and all the other pedestrians were picked up from the real data. By doing this, in each session the robot had to use continuously its positions calculated from the model and the real data of the others to calculate the value of w p in the next step. 4) As can be seen from the scenario, the member of the pair which moves in between their partner and the pedestrians will determine the value of w p . For example, in Figure 9, i will determine the value of w p but not j, hence, it makes no sense to apply the model to j. Thus, we only replace this member by the robot. 5) Because the model needs at least two steps before it can collect full data for its calculation, hence in the initial step, the robot started with the real data. In the second step, it uses c Mv t+1 = c Mv . 6) Initially, all the coefficient values found in [8] were kept.
We estimate some values that allows f D shows its effect at d nc = 5m, therefore we set initial values a = 0.4, b = 1.5, c = −100, ∆ d = 5, k = 1. 7) Similar to the simulator in [8], in Figure 9, SC s t+1 is a point on the line stretching between two pointsp s t−1 andp s t with s = (i, j, m) toward the destinations. The distance between SC s t+1 andp s t was a length determined by M v in Eq. 4 multiplied by ∆t.   At time t, to find w p at t + 1, the simulator scans a region of 5x5 cells around SC s t+1 , from that U in Eqs. 16 is calculated. 9) The simulation step is set with ∆t = 1 second. At time t, by applying Eq. 16, the simulator scans a region of 5x5 cells around SC s t+1 , since then U in Eq. 16 is calculated, i.e. the position of the robot at t + 1 is found, from that w p of the pair at t + 1 is found. 10) All calculations for the step t + 1 are finished here. 11) The model continues calculating the position of the simulated follower at time t + 2 and so on. We chose the dimension of one person is 0.5*0.5m, thus the minimum width w pmin = 0.5. The average densities of pairs and groups from the data are listed in Table II. The coefficients are adjusted so that the w p values generated by the model are closest to the real value.
The adjusted values of coefficients are listed in Table III. Most values from previous studies were kept, except k O and c f R d were reduced. There were too many people in the corridor therefore people accepted a smaller distance between them and obstacles, thus we reduced k O . Also, the average distance between the pair c f R d was smaller than in the previous studies. In addition, we set ∆ d = 5.

B. Performance verification
We used Wilcoxon Signed-Rank Test for Equivalence using TOST (Two One-Sided Tests) in R to evaluate the performance   It can be seen that there are two main steps that affect the performance of the model: the accuracy of the sub-goal determination step (passing mode detection), and the width determination w p after the sub-goals have been determined.
By evaluating the entire model, we obtained 343 pairs of data (the real w p and the simulated w p ). The value p obtained is p = 0.977306, where the average widthw p between the actual data and the data extracted from the model is 1.18m and 1.17m respectively; showing that w p rendered from a model is very close to reality; i.e. the model can generate routes similar to the actual pathways that the pairs have taken.
In the above results, there are some cases where the algorithm to find sub-goal works incorrectly, such as mode (a) is recognized but then people actually choose mode (b). Therefore we took one more step to test the performance of the model by eliminating the cases of passing mode misidentification. In this case, 335 pair values were remained. The statistics are presented in Tables IV and V, and illustrated in Figure 10. The value p = 0.937084, showing that the real data and the predicted data are very similar as well, i.e. the new model is able to predict the pathway that humans intend to take. This paper presents a novel model for determining natural pathways for side-by-side robotic wheelchairs; the model helps the robots mimic the natural route that human pairs normally take when passing pedestrians in crowded environments. From our observation and the data collected from a user study, we improved the previous model by incorporating one new human walking habit -moving in line -and the dynamic density factor into the navigation function. The model tackled the limitations of navigation functions in previous studies. Although the scenarios in this study were mainly focused on passing pedestrians in a relatively narrow corridor, we believe that this is a common situation that side-by-side robotic wheelchairs have to deal with in daily scenarios.
In real-world scenarios, we can envisage more complex environments. For example if people stroll in a shopping mall, their walking trajectories may be affected by their attentions to some featured goods; or if a lot of people move on a wide sidewalk or an intersection, other situations may happen. However, we believe that this model can be developed further to deal with those environments.