The areas of drone application have been growing rapidly in recent years. Autonomous drones that perform tasks in automatic mode are increasingly being used. The industry of unmanned drones for the transportation of both goods and people is developing. The drones themselves are being improved. Software tools are being improved for controlling the activity of drones both in the human-operator control mode and in the autonomous mode.
Today, the world has accumulated a lot of experience in using drones in various situations. However, this experience is usually limited to either the control of the drone/drones by a human operator, or the use of drones in automatic mode. Therefore, there is increasing attention to the use of swarms of drones that can work cooperatively to achieve a goal. This requires moving to a new level of drone control design.
This next stage is the transition to social systems that will consist of people and drones. At the same time, they will perform a joint task. However, the formation of such systems requires new approaches, especially to the organization of management of joint activities. Thus, in the review article [1], attention is focused on ideas and concepts that may be promising for the development of swarm robotics, especially with specific tasks. As the most promising areas for the development of swarm robotics, the authors identify "an increasing need for a swarm engineering, that is, a need for methods for (1) requirement modeling and specification, (2) design and implementation, (3) verification and validation, and (4) operation and maintenance". The authors identify three areas as important. The first is the requirements for both models and drones/robots. The second is the creation of universal approaches to the design and implementation of collective behavior, based on the behavior of the drone itself and ending with the behavior of the swarm as a whole. Thirdly, the problem of communication and cooperation between people and drones.
One of the main challenges is that it is necessary to develop new approaches to modeling the distribution of control for drone swarms. First, many individual elements of drone behavior must be performed individually. Secondly, drones must perform a large array of behaviors together, as part of a single swarm. Thirdly, it is important to manage the communication between humans and drones for such a social system aimed at the joint achievement of goals [2].
The problem of controlling a swarm of drones is considered in a large number of articles today. Thus, in [3,4] an algorithm for organizing a collective movement based on the movement of a swarm “behind the leader”, which is identified quite randomly, is considered.
In several works, for example [5, 6, 7], the attention of researchers is focused on the development of collision avoidance models in a moving swarm.
A quantity of work is devoted to the transition from the local movement of the drone to the organized movement of the swarm as a whole. In [8], drones with limited sensory capabilities are considered, which "feel" only a few nearest neighbors and "know" only one constant direction of movement. The control model proposed by the authors is very promising, but before application, it needs to be improved. However, this indicates one direction for further development. In [9], a control model is presented for the distribution of a swarm in space to achieve a given density of the number of drones. It also relies on information about the local position of the drones in the swarm. In [10], a decentralized problem is also considered. Attention in this work is focused on studying the effect of internal noise or interference on the collective motion. To overcome interference, it is proposed to use the availability of communication channels between drones. In [11], optimized flocking of drones in a confined environment with multiple conflicting objectives is proposed. The considered objectives are collision avoidance (with each other and the wall), speed, correlation, and communication (connected and disconnected agents).
In [12], the influence of drone characteristics on the tasks that can be assigned to the swarm is considered. An example is a swarm of aquatic robots designed to remove harmful algae from the water. This is a rather promising approach, moving from drone parameters to swarm parameters. At the same time, there is an inverse approach from the required characteristics of the swarm behavior to the characteristics of the drone that are necessary to implement such behavior. In [13], it is proposed to use the patterns of coordination for interacting agents for observations of the evolution of the swarm density over time to describe a swarm. As a result, the swarm is described by macroscopic parameters. This approach is motivated by the analysis of large biological swarms. In [14] it is proposed to calculate the trajectories of each of the drones in the swarm, taking into account the presence of conflicts between drones. This allows for taking into account the ability of individual robots to make quick turns. Within the framework of the trajectory tube models proposed in our article below the topological solution for them completely solves this problem already at the level of "ready-made fragments", which are then connected like Lego.
In [15], the requirements for the interface of an operator controlling drones in a swarm are considered. However, the operator must still pay attention to each of the drones in the swarm. In [16], the authors proposed “a two-step scheme which consists of task partitioning and autonomous task allocation to address these issues. In the first step, the original task is partitioned into simpler subtasks to reduce the complexity of designing fitness functions. In the second step, evolutionary approaches are adopted to synthesize a composite artificial neural network-based controller to generate autonomous task allocation for the robotic swarm.” To do this, the task is laid out at the hardware level of the drone itself.
In [17], the situation is considered when the swarm is divided into groups with the same number of drones. Drones have both firmware (fixed, built-in programs) and regular programs for their traffic. A group of drones is controlled from the command center. In the models proposed in our article below, individual clusters in a swarm can consist of a different number of drones. In addition, a drone-coordinator, rather than a human operator, can be used as a cluster control center. In [18], a swarm model with a central control agent is described. In the model, several drones are located above the swarm and serve as repeaters to control the swarm. This model allows for solving a narrower class of problems than is proposed below in our article. In [19], a comparison of centralized and distributed control for a swarm of drones is studied. It was revealed that centralized control is more profitable than distributed one. However, centralized management has significant scalability limitations. The results obtained in our article below combine centralized and distributed control to achieve a high level of scalability.
We think that further development of research in the area of swarm robotics can be carried out in the direction of developing models in which not a person, but a special drone controls a group of drones (or a separate swarm). And, of course, the particular interest is the development of models for the implementation of both joint and coordinated behavior of several swarms containing a different number of drones, and swarms with the ability to exchange drones with each other.
The purpose of the article is to describe the perspective approaches to modeling the organization of distributed human control over the behavior of drone swarms, each of which has a hierarchical structure.
Concept
There are two main approaches to modeling drone swarm behavior.
The first approach is based on the fact that existing drones need to be combined into a single structure. Today each drone, as a rule, are either controlled by a person or has uniquely programmed behavior (and therefore highly specialized). As a result, the behavior of a drone swarm, in the final case, will be defined as the control of one or several drones by a person. The other drones in the swarm mimic (in one sense or another) the behavior of the human-controlled drones. As a result, the majority of the swarm should consist of drones that are arranged in the same way, and that have practically unambiguous behavior. All this significantly reduces the stability of the swarm’s adaptation to the changing environment.
The second approach is based on the fact that drones should form a certain “social system”. Here, each of the drones can show a kind of “personality”, that is, show unique behavior and have unique capabilities. In such a “social swarm of drones," there can be a “leader” (ringleader, superintendent, coordinator, etc.), who directs the behavior of the entire swarm. But even when fulfilling a joint goal, individual drones in a swarm show a certain level of individual behavior and carry out their adaptation to the changing environment. At the same time, the stability and adaptive properties of the swarm will be sufficiently high. The predictability of the behavior of an individual drone in a flock under these conditions sharply decreases.
Below will be considered one of the possible options for implementing the second approach. At the same time, the swarm will have a drone-coordinator (thus the swarm will have a hierarchical structure). Its task is to manage the methods and technologies of achieving a given goal, set before the swarm as a single entity. The behavior of an individual drone will have individual features that will distinguish one drone from another. This makes it possible to also create drone swarms, which differ from each other in methods, technologies, and tools for the manifestation of their activity. Such a swarm can perform tasks in autonomous (offline) mode. The human-operator will influence only the drone-coordinator, and during the operation only in exceptional cases.
Note. This organization of control of swarm behavior corresponds to the management of human teams. They consist of workers who perform simple work. As a rule, a human worker is taught to perform a certain set of simple actions for a long enough time. It is the simple and “programmed” actions of people today that are increasingly becoming automated. But each team has a person who performs the functions of managing workers and works. This person is focused on management functions, that is, his activity is to make decisions and adjust, and change the program of activities for individual people. This person coordinates the activities/behavior of the workers in such a way that the goal of the joint activity is achieved as a result. Examples include an officer/sergeant in the military or a lower-level manager in an industry or enterprise. This is distributed and hierarchically organized management.
The article will consider the behavior of the swarm directly during the operation. Getting the swarm to the location of the operation is a fairly obvious task (however, some aspects of this will be discussed below).
The behavior of an individual drone in a swarm
An individual drone during operation can be characterized by such a set of parameters.
1. The trajectory of the drone during the operation is given as follows. The trajectory D(r,t) and speed V(r,t) of the general drift of the drone in the swarm are set in space. Then a curvilinear (open or closed) tube of trajectories T(r,t) (for example, a torus) is set, beyond which the drone’s trajectory must not go. The diameter d of such a tube of trajectories can change along the axis and change with time (within the given limits). The drone moves chaotically within such a curved cylinder (one can set the distance d(r,t) and the speed v(d) from the axis of such a cylinder). Also, the speed of the drone changes randomly (within the given limits). Therefore, as a result, the trajectory and speed of the drone are divided into the drift and stochastic components.
The trajectory tube is required to take into account statistical changes of 1) random changes in the environment (gust of wind, stone under the manipulator, etc.) and 2) random deviations of the drone from the axis of the tube due to internal changes (for example, during the operation: shots, missile launches, etc.).
In some cases, it is possible to allow the boundary of the tube to be set also statistically.
Note that, in the general case, the trajectory of the drone in the swarm will be in the given “drone trajectories tube (DTT)” (which may have loop-intersections, depending on the drift speed).
2. Avoiding collisions with other drones can be done as follows. First, the drone may have an automatic that warns of a collision (see, e.g., [5-7]). At the same time, the trajectory and speed of the drone may have a stochastic additive. Second, to avoid a collision, the drone can choose to switch to another curvilinear tube (or transit to a new curvilinear tube).
3. The drone has a set of options W(r,t) of behavior (manifests of activity) during the operation.
Thus, the behavior B of an individual drone in a swarm is given by the following tuple.
4. The drone informs the drone-coordinator about changes in its state and behavior characteristics that occur due to internal and external causes.
Note that the presence of a stochastic component in the drone’s motion can be important for some tasks. For example, it can be a countermeasure against the identification of a drone (radar or laser) in a swarm. As a result of the presence of a stochastic component, the drone will be able to go beyond the identification area. As a result, the identification devices will either be forced to pause to find the drone again, or another drone in the swarm will come into their “field of view”. In the latter case, the swarm will also receive a certain amount of time to perform the planned actions.
An interesting mathematical problem arises regarding the topology of drone trajectories in a swarm during the operation (or given activity). This topology will take into account the specifics of the task that was given before the swarm. At the same time, several different topologies for “tubes of trajectories” can be developed for each of the drones in the swarm to accomplish the task. This allows one to switch from one topology to another during an operation.
Behavior and tasks of the drone-coordinator in the swarm
The drone-coordinator has the following set of parameters and characteristics.
Note. We remind that the task of such a drone-coordinator is to make decisions and give commands to change/correct the activity of individual drones in the swarm. This imposes specific requirements for the construction of a drone-coordinator, that is, its design and programming. For example, it must have specific sensor systems and software (which are different from other drones in the swarm). But it is desirable to keep the appearance as similar as possible to all other drones of the swarm. This is necessary for the stability of the swarm concerning external factors (this is especially clear for military conditions: the officer’s clothes should not unmask him).
1. In the swarm, itself and its behavior should not differ from other drones. Therefore, it is also described by the same characteristics (1) and (2). However, the “tubes of trajectories" of the drone-coordinator are located in those places of the swarm that depends on the purpose of the swarm’s activity as a whole. It can also migrate to different areas in the swarm as environmental conditions change.
2. The drone-coordinator controls the entire operation plan. Its activities are as follows.
1) It gives the entire trajectory of the swarm’s movement and topology for “tubes of trajectories" for every drone in the swarm from the beginning of the operation to its end.
2) It selects and sets the entire set of behaviors bij of all drones in the swarm during an operation. The numbering for this is double: the first number corresponds to the i-th drone, and the second index means the j-th behavior of the drone.
3) It chooses a specific strategy siÎS for the swarm behavior as a whole during the operation from a set of possible strategies S. Each strategy si is assembled from the temporally and spatially ordered strategies of the behaviors of individual drones B={bij} (like Lego).
4) It chooses the strategy “to interrupt the operation” (“rapid exit”) in the event that all the planned strategies from set S are ineffective. For example, in the case of the loss of a certain number (or some) of drones (this can be a kind of “reconnaissance by combat”). It also ensures the delivery of information about the realized situation to the control center (“headquarters”) either by itself or by transferring it to one of the drones.
5) It also communicates with the human-operator (control center, “headquarters”) as it is needed. At the same time, only short pulses are used that correspond to the strategy chosen by it. It can also inform the human-operator (“center”) about the execution (partial execution) of the current (given) stage of the operation. To do this, it is important to create a database of options for “partial execution” of the operation stages and/or the operation as a whole.
6) It does all the current control of the drones by using the indices i and j which specify the behavior of the particular drone bij. This allows the use of low-intensity signals within the swarm (radio waves, light signaling, signaling by elements of the drone body, etc.). In this way, the protection of information channels is ensured.
Tasks of a human-operator
The tasks of the human-operator are not to implement the permanent control of the swarm through the drone-coordinator. The task of the human-operator is to prepare the operation and carry out the training for both the drones and the drone-coordinator, dividing the operation into stages, elaborating the goal tree for the drones in the swarm, taking into account possible changes in the environment, and operational area, develop decision-making algorithms for the coordinator drone, etc.
Among other tasks, the following can be distinguished (the list is not complete and exhaustive).
1) He chooses the behavior B={bij} for each drone in the swarm.
2) He specifies a set of strategies S for the drone-coordinator using the previous reconnaissance.
3) He selects the drift trajectories D(r,t) and velocity V(r,t) of the swarm for a) movement towards the location of the operation, b) stages of the operation, and c) movement away after the operation.
4) He selects the shapes and topologies for “drone trajectory tubes” for various stages of the operation.
5) He tests the strategies and behaviors of the drones in a swarm over computer simulations with other operators and “headquarters” workers, as well as with the experts. In some cases, it is convenient to do this in the form of a computer game. For example, online gaming platforms to optimize various stages of control of the swarm can be used.
Algorithm for modeling the behavior of a drone swarm
The algorithm for choosing the optimal behavior of a drone swarm can be specified in the following stages.
1. The geographical relief of the place of operation is specified and a set of targets for the manifestation of drone activity is determined.
2. The method of forming a drone swarm is specified. It is formed from drones that are capable of performing a given activity for a given set of targets (which have the necessary set of tools for this).
3. The path (trajectory) is set for 1) the formation of the swarm (because not all the necessary drones can be in one place) and 2) the swarm movement to the operation area. During the swarm movement to the operation area, the requirements for the ‘trajectory tubes” for each of the drones can be weakened (for example, practically straight lines).
4. A set of operation execution scenarios is specified. It takes into account geographical features and the specificity of the behavior of targets and possible obstacles. The possibility of damage and loss of a certain number of drones is also taken into account.
5. According to the set of scenarios, corresponding sets of “trajectory tubes” are formed for each of the drones at each stage of the scenario execution.
6. According to points 3-6, a coding system is formed for possible variants of the behavior of each of the drones.
7. Taking into account 3-6, a set of operation execution scenarios is structured. Including, a tree of possible solutions is formed (in the sweep of the operation in time) for changing the scenarios, using fragments of different scenarios.
8. Considering 3-6, codes are created for the formed set of scenarios and a tree of possible decisions (behavior) for the drone-coordinator, which will make decisions regarding the control of the behavior of each of the drones in the swarm.
9. A set of possible behavior of the targets and a set of obstacles is specified, taking into account their geographical reference and deployment in time.
10. Computer simulation of the operation is carried out. The parameters by which the results are optimized are as follows (the list is not exhaustive):
1) swarm drift trajectories and speeds;
2) characteristics of “trajectory tubes” for each of the drones (including different probabilities for stochastic manifestations);
3) characteristics of targets and obstacles;
4) mission performance characteristics (for example, number of successful drone activity results, drone damage, number of lost drones, changes in target characteristics as a result of swarm activity, etc.).
11. The optimal option (variant) is selected and the drones in the swarm and the drone-coordinator are programmed.
12. A series of natural (field) experiments is carried out.
13. Client massively multiplayer online games in real-time (for example, World of Tanks, Massively multiplayer online role-playing games, Massively multiplayer online first-person shooters, etc.) can be used as field (natural) experiments too.
14. Taking into account the results of field experiments, points 1-13 may be repeated. In such computer simulations (especially with the participation of real people-players), a database is created regarding possible choices of geographic references and target reactions, operational obstacles, etc.).
15. The database of the results of computer simulations (supplemented by the description of real field experiments) can be used to optimize both the behavior of individual drones during the operation (as well as to form requests for their modernization) and to train for drone-coordinator. Tournaments can also be used as drone-coordinator training, where one/several drone-coordinator is pitted against another drone-coordinator (one or several). This allows us to determine the best programs for a set of drone-coordinator.
A detailed description of the algorithms for the computer simulation of the behavior of the drone swarm during the operation will be determined by the purpose of the operation.