Flexible Route Planning for Multiple Mobile Robots by Combining Q-Learning and Graph Search Algorithm

The use of multiple mobile robots has grown significantly over the past few years in logistics, manufacturing and public services. Conflict-free route planning is one of the major research challenges for such mobile robots. Optimization methods such as graph search algorithms are used extensively to solve route planning problems. Those methods can assure the quality of solutions, however, they are not flexible to deal with unexpected situations. In this article, we propose a flexible route planning method that combines the reinforcement learning algorithm and a graph search algorithm for conflict-free route planning problems for multiple robots. In the proposed method, Q-learning, a reinforcement algorithm, is applied to avoid collisions using off-line learning with a limited state space to reduce the total learning time. Each vehicle independently finds the shortest route using the A* algorithm, and Q-learning is used to avoid collisions. The effectiveness of the proposed method is examined by comparing it with conventional methods in terms of computation time and the quality of solutions. Computational results show that for dynamic transportation problems, the proposed method can generate the solutions with approximately 10% of the computation time compared to the conventional Q-learning approach. We found that the required computation time is linearly increased with respect to the number of vehicles and nodes in the problems.