Hierarchical behavior decision-making framework based on reinforcement learning for driverless cars

Qinghe Liu; Yankun Zhang; Mingze Xu; Ke Li

doi:10.1117/12.2645685

23 November 2022 Hierarchical behavior decision-making framework based on reinforcement learning for driverless cars

Qinghe Liu, Yankun Zhang, Mingze Xu, Ke Li

Proceedings Volume 12302, Seventh International Conference on Electromechanical Control Technology and Transportation (ICECTT 2022); 123021P (2022) https://doi.org/10.1117/12.2645685
Event: Seventh International Conference on Electromechanical Control Technology and Transportation (ICECTT 2022), 2022, Guangzhou, China

Abstract

As the brain of driverless cars, behavioral decision-making module plays an important role. In recent years, reinforcement learning has been gradually introduced into this field. However, the algorithms proposed in many articles are only used to solve a certain scenario or a certain driving task, and cannot completely solve the unmanned driving in urban scenarios. Some articles try to use one framework to solve the whole urban scenario, which does not take into account the differences between individual scenarios in the city and has poor expansibility. In this paper we propose a hierarchical behavioral decision-making framework. In the top layer, we consider four typical scenarios in the city and adopt rule-based finite state machine (FSM) to realize reasonable switch between them, which makes the system have good expansibility. At the bottom layer, we train an agent for each scenario using reinforcement learning to realize deep traversal of the specified scenario to improve decision accuracy. First, FSM determines which scenario the car is in and then an according agent is selected to make decisions. Using SMARTS as the simulation environment, we test each agent in according scenario and the agent fused with our framework in urban scenario under free, stable and congested three traffic conditions. On the whole, the completion rate remains above 90% and the collision rate remains below 5% for every agent showing that our method is safe and effective.

Citation Download Citation

Qinghe Liu, Yankun Zhang, Mingze Xu, and Ke Li "Hierarchical behavior decision-making framework based on reinforcement learning for driverless cars", Proc. SPIE 12302, Seventh International Conference on Electromechanical Control Technology and Transportation (ICECTT 2022), 123021P (23 November 2022); https://doi.org/10.1117/12.2645685

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Roads

Decision making

Rule based systems

Design and modelling

Lithium

Switches

Safety

Show All Keywords

Keywords/Phrases

Search In:

Publication Years