Framework of Active Obstacle Avoidance for Autonomous Vehicle Based on Hybrid Soft Actor-Critic Algorithm: Fid-Bau Portal

Framework of Active Obstacle Avoidance for Autonomous Vehicle Based on Hybrid Soft Actor-Critic Algorithm

Chen, Yuanhang / Wu, Shaofang

In this paper, a framework of active obstacle avoidance for autonomous vehicles based on the hybrid soft actor-critic (SAC) algorithm is proposed. In the stage of local path planning, a comprehensive cost function considering collision risks, deviation from the global route, road lines crossing, and driving comfortability is developed to provide a local optimal path avoiding both static and dynamic obstacles considering multiple predicting timesteps. Then, a path tracking controller on the foundation of a hybrid SAC algorithm is designed to mitigate the problem of high sample complexity caused by random initialization of parameters in conventional reinforcement learning approaches. Model predictive control (MPC) plays a guiding role by applying its control action to combine with the action of SAC online to obtain a more effective state and reward information for training. The mechanism of the combination of MPC with SAC to balance the exploration and reliability is explained in detail. In order to improve the convergence rate and learning efficiency, a dual actor network structure for two different control actions is adopted. With considerations of various relevant factors influencing the control effect, the reward for the hybrid SAC algorithm is designed carefully. Finally, the results of simulation experiments illustrate that the proposed approach performs effectively with the assurance of safety and driving comfortability. In summary, the hybrid SAC algorithm with dual actor networks performs better than other algorithms for comparison in all test scenarios in this paper.

An autonomous vehicle is an attractive topic with the development of artificial intelligence. Obstacle avoidance is a significant challenge to autonomous vehicles, which is also the focus of this paper. To promote the reliability of autonomous vehicles in the real driving environment, this paper proposes a framework of active obstacle avoidance for autonomous vehicles based on the hybrid soft actor-critic algorithm divided into two steps of local path planning and path tracking. In the first step, a comprehensive cost function considering various factors is developed to provide a local optimal path avoiding both static and dynamic obstacles, which can be embedded into the planning module of autonomous vehicles and achieve real-time collision-free path planning by an onboard computer. In the second step, the proposed hybrid soft actor-critic algorithm for path tracking control can solve the problem of low sample efficiency of conventional deep reinforcement learning. In addition, it is trained in the simulation environment without the risk of collisions in a real driving environment and could be migrated to the real vehicle platform conveniently as it is designed based on the vehicle dynamics model. The proposed approach is both academic and practical and can promote the development of autonomous driving in the real world.