Paper—A Review: On Using ACO Based Hybrid Algorithms for Path Planning of Multi-Mobile Robotics A Review: On using ACO Based Hybrid Algorithms for Path Planning of Multi-Mobile Robotics

The path planning for Multi Mobile Robotic (MMR) system is a recent combinatorial optimisation problem. In the last decade, many researches have been published to solve this problem. Most of these researches focused on metaheuristic algorithms. This paper reviews articles on Ant Colony Optimisation (ACO) and its hybrid versions to solve the problem. The original Dorigo’s ACO algorithm uses exploration and exploitation phases to find the shortest route in a combinatorial optimisation problem in general without touching mapping, localisation and perception. Due to the properties of MMR, adaptations have been made to ACO algorithms. In this review paper, a literature survey of the recent studies on upgrading, modifications and applications of the ACO algorithms has been discussed to evaluate the application of the different versions of ACO in the MMR domain. The evaluation considered the quality, speed of convergence, robustness, scalability, flexibility of MMR and obstacle avoidance, static and dynamic environment characteristics of the tasks. Keywords—Multi-Robotics, path planning (PP), ACO algorithm, uncertainties


Introduction
When multiple mobile robots cooperate to complete a specified task, the group of the robots are referred to as Multi-Mobile Robotics [1]. It is a specific case of Swarm Robotics (SR) where thousands of decentralised mobile robots work together to perform a specific job [2]. These types of robots normally have special properties e.g. member identicality, asynchronism, decentralised control, robustness, scalability, and flexibility. Deterministic methods are too complex for large workspaces and intangible applications. Beni [3] introduced the term SR and emphasised that there are many similar behaviours between SR and insect colonies in biology. This comparison led the robotic community to propose and to use algorithms based on the species societies behaviours to solve problems concurred with SR and MMR. These algorithms referred to as nature-inspired algorithms; they are heuristic or metaheuristics. The ACO algorithm [4,5] the ABC algorithm [6,7], the Particle Swarm Optimisation (PSO) algorithm [8,9], the Cuckoo Search (CS) algorithm [10,11], the Grey Wolf Optimisation Algorithm (GWO) [12], the Whale Optimisation Algorithm (WOA) [13,14] are some of the mathematically mature optimisation algorithms based on the social behaviour of animal colonies. After three decades and developing tens of heuristic and metaheuristic algorithms, these algorithms did not find their solid way into practical applications [15]. All survey studies of the applications of nature-inspired algorithms addressed ACO as one of the premier optimisation algorithms for Path Planning (PP) of mobile robots [16][17][18]. This paper reviews the pros and cons of the ACO algorithm and its variants to solve the MMR optimisation problem.

2
The MMR Path Planning Problem The main goal of PP is to find an optimal, collision-free path from an initial point to an endpoint in the environment with uncertainties to a group of autonomous robots. The general problem of PP for the MMR may be formulated as follows (see figure 1). "Given a space X contains obstacles O, find the paths pi ⊂X\O such that pi connects any two points sn,fm∊X\O and pi∊X\pj i≠j, minimizing the measure J".
Here s and f are the start and final targets, respectively, X is the free searching space, O is the constrained space due to obstacles and J is an objective measure to be minimised, P is the connected path between s and f, n is the initial and m is the final pose.
As the MMR applications are rapidly progressing, different applications have different requirements for path planning. To the knowledge of the authors, there is no universal algorithm for all types of applications. Most algorithms support only a few task requirements. The hybridisation of algorithms is a good alternative to embed non-supported functions to the existing algorithms. Let us call the original algorithm the base and the added methods the embedded algorithms. Although ACO has been the base algorithm for many hybrid algorithms [4,19,20], however only a few of them have been implemented in practical models.

Ant Colony Optimisation (ACO)
According to the natural ant characteristics, initially individual ants in a colony come out of the nest and depart in different directions searching for food randomly. Once they find a food source, they take some of the food back home. In their way back, they secret a chemical hormone called Pheromone. The amount of Pheromone may depend on the quality and quantity of the food. As a designated path e.g. shorter than other paths is used more and more by the ants, more Pheromone is deposited on that path. In this way, other ants are stimulated to follow the path/trail with more Pheromone and would probably prefer it to the other paths. Hence, the ants collectively use the shortest path as explained in figure 2 [21]. This process is called Stigmergy which is an indirect communication between the ants via Pheromone. It supports efficient collaboration between simple agents who lack any memory or even individual awareness of each other [22].
The ACO algorithm was originally introduced by Marco Dorigo in his PhD in 1992 [23] . He converted the ant colony behaviour on how they select a specific path during seeking and collecting food into an artificial optimisation algorithm to solve combinatorial problems [24].
The ACO algorithm is a metaheuristic algorithm which is used to determine an approximate solution to complex combinatorial optimisation problems in a reasonable amount of time [25]. The algorithm works as follows. Initially, a grid search space is defined and n nodes are selected from the search space. Then each ant is positioned at a starting node randomly. At each node, each ant applies a state transition rule to go to an adjacent node and build a complete path. The next node is selected according to the probability function given in Error! Reference source not found.) [26]: Here τi, j is the Pheromone trail of the path between node i to node j, , = 1 , Li,j is the Euclidean distance between the nodes i to j, α and β are the stimulating of Pheromone concentration and relative importance of visibility, respectively. Roulette wheel method is used to select the best probability [27,28].
After each complete path, the Pheromone concentration of the path is updated by Pheromone amount Δτij which is given by Error! Reference source not found. according to [29]: The ACO algorithm selects the best solution with the maximum Pheromone laid on the path. If the evaporation is considered, the pheromone amount is modified by inserting the evaporation factor ρ, (0<ρ<1) as given by Error! Reference source not found..
The pseudocode for the ACO algorithm is given in figure 3.

Embedded Functions in ACO Base Algorithm in MMR Applications
According to industrial magazines, MMR is rapidly developing in a way that it will change the way human lives [30]. Every day new applications of MMRs are emerging [31][32][33]2].
As MMR is being deployed increasingly in real life, MMR needs to work in different scenarios. It is well known that a basic requirement for any MMR is that the robots must be able to move safely through the workspace; hence, proper PP must be designed for each application [34,35].
The PP algorithm for MMR is part of a wider interacted engineering problem which contains Simultaneous Localisation Mapping and Perception (SLMP). Various *ACO Algorithm* Phase1: * initialization phase* For each (r, s), τ(r,s)=τo, End-For *Pheromone concen-tration for each path (i,j)* For each (r,s), d(r,s)= distance between r and s *search space construction* End For Set algorithm parameters: Desirability of each path (η), evaporation rate(ρ), Stimulating factor of pheromone concentration (α), relative important between pheromone and distance(β), number of ants(m), number of nodes (n) Phase2: *Main Loop* Set ant k to a node randomly For ant k {1, …. , m} *Path construction* For S= {1, ….., n} *searching space scanning* Select J=next node according to the maximum prob bility from equation (1) End For Calculate the best path using the shortest length fitness function End For For all i, j Update Pheromone according to Equation (3) End For Repeat Phase 2 until a stopping criteria is met (number of iteration, best path achievement) Output the best path found variables like distance, speed, time, task allocation reliability, energy consumption, adaptation to environment, etc. have been considered in most optimisation strategies.
The original ACO algorithm has been the base algorithm for many new and modified nature-inspired algorithms e.g. [36,37,5], etc. that have been developed in recent years. However, the fundamental basic phases i.e. exploitation and exploration of ACO have been repeated.
A modified version of the ACO called the Ant Colony System (ACS) added three updates to the original ACO algorithm [38]. These modified versions were: the first addition was a local Pheromone update which offered a solution to overcome the local optima problem while were ants were building a solution. The second one introduced a state transition rule which provided a direct way to balance between new edges exploration and exploitation of a priori accumulated knowledge to raise the speed during obstacle avoidance process. The third one applied a global updating rule only to edges that belong to the best ant path. The new ACS produced better results than the original ACO when applied to the Travelling Salesman Problem (TSP) [39].
In recent years, the ACO has been revisited by many researchers as a base algorithm to build new algorithms. For example, the ACO has been used for feature selection in the Support Vector Machine (SVM) classifier and they described the problem of redundant features and falling into local optimum [40]. They modified the fitness function and pheromone updating to achieve better results [41]. The limitations of the stagnation of ACO has been studies in [41]. This paper introduced a hybrid ACO-ABC algorithm to reduce these limitations by using employee bee in the exploration of the ACO algorithm and participated in the exploitation into two parts in the ABC algorithm. The study shows that these two limitations are reduced but not eliminated. The trapping into a local optimum solution of the ACO has been addressed in [42]. The researchers modified the ACO by using the mutation and crossover characteristics of the Genetic Algorithm (GA). They showed successful results for this to the no-wait flow shop scheduling combinatorial problem. The stagnation appears due to Pheromone accumulation on the explored paths which leads to failure in finding new best paths [29].
In the MMR domain, the problem of PP is to find paths for a team of autonomous mobile robots sharing the same workspace to achieve a common task while avoiding interference with each other [43]. There are different classifications for MMR Systems; the two major ones are based on the obstacle behaviour and search strategy. Based on the obstacles and environment, they are classified into static i.e. motionless obstacles and dynamic i.e. moving obstacles. The other classification i.e. search strategy deals with local or offline MMR that searches for obstacles while it is making progress towards the objective and global or online when the environment is forecasted before the robots start their missions [44].
There are specific technical requirements for MMR PP to be addressed, the main ten requirements to be considered are listed below: Mao and Yu have studied Multi robotic PP from its early appearance [45]. They added a new function to update the trial intensity of the pheromone. They presented a solution to the conflict between moving ants. They applied their algorithm to a static environment with dead corners.
A hybrid algorithm from ACO and the Cellular Automata (CA) has been developed in [46]. The study dealt with the collision existence of the MMR and tried to determine collision-free trajectories. They simulated ten e-puck agents in the webots platform. This algorithm is very well formulated for the dynamic environment.
Another hybrid algorithm has been presented by embedding GA into ACO [47]. The study covered the shortest path requirement in a static environment. A global optimum solution is considered with real-time constraints in this study.
In [48], the authors introduced a hybrid algorithm by embedding Fuzzy Logic into ACO to avoid premature convergence when it appears. The study used adaptive control to tune ACO parameters to maintain the diversity in the colony.
A technique which was called marginalisation to select an initial location for the ants has been introduced in [49]. This technique locates the ants on the edge of the nodes to prevent touching in the first phase. The research adjusted α and β in the ACO model to prevent local convergence and determined the optimum values for α= [1,3.5] and β= [1.5,5.5].
Paper [28] embedded A* algorithm into ACO to accelerate the convergence speed of the base algorithm. The study dealt with the deadlock through introducing a subroutine for local diffusion pheromones. It simulated three map types with the new algorithm. This algorithm is suitable for tunnels.
An online PP algorithm for a typical MMR system with homogeneous, independent and limited communication skills agents has been presented in [50]. This algorithm is efficient to lead the robots in real time from the initial position to their final goals with known initial and target positions only. The algorithm used the Pheromone concept of ACO in combination with Fuzzy logic. The V-Rep simulation of the algorithm shows satisfactory results with limitations in special case waypoints.
Another hybrid algorithm has been introduced by embedding Improved Potential Field into the ACO algorithm [19]. This paper presented a new method for Pheromone updating. The researchers implemented the algorithm to the Traveller II robot.
An improved algorithm based on ACO that uses the roulette wheel algorithm of GA is developed in [20]. In this algorithm, the probability function of ACO has been adjusted to solve the deadlock problem in the MMR environments. The algorithm is verified via simulation analysis.
Based on the discussions presented above, table 1 shows a summary of the embedded algorithms in the ACO algorithm. This review shows that ACO is currently the dominant base algorithm in the field of PP for MMR. This is because of its flexibility for adjustment and adaptation through the Pheromone updating strategy and probability function tuning. These two adjustable parameters help in finding the shortest collision-free path, with a guaranteed global optimum solution. To achieve these two goals, different metaheuristic algorithms have been embedded in the ACO to build new hybrid algorithms. Most of these algorithms have been verified via simulation but rarely have been tested in a real environment with real robots. The results of this review study show that there is no general algorithm exists yet to solve the PP problem of the MMR that meets all nine requirements.

Conclusion
A thorough search of the relevant literature has found that ACO is the most used algorithm for PP of MMR systems in recent years. The ACO algorithm has been used as a base algorithm and it has been utilised to solve the PP problem. In this paper, the recent algorithms with major modifications to ACO have been reviewed and their strengths and weaknesses to MMR task applications have been presented. The ACO consists of two phases; the exploration and exploitation and can tune Pheromone updating and probability function selection. These two attributes have been the key parameters in all the reviewed hybrid algorithms. The original ACO performs well in complex combinatorial optimisation problems but it has convergence speed and local optima problems. These two problems have been solved by embedding Genetic, A*, Cellular automata, Improved Potential Field and Fuzzy algorithms.
Based on the survey and results of the reviewed algorithms, industrial applications of MMR systems are increasing, and new applications are arising every day. Therefore, this is still an open and active research field. These new applications need their own algorithms to achieve their objectives. There are needs for studies to introduce new hybrid algorithms in the presence of bottlenecks, deadlocks, dynamic obstacles, environment uncertainties, robustness and hardware applicability.