Deep Merging: Vehicle Merging Controller Based on Deep Reinforcement Learning with Embedding Network

Ippei Nishitani1, Hao Yang2, Rui Guo3, Shalini Keshavamurthy4, Kentaro Oguchi5

  • 1TOYOTA MOTOR CORPORATION
  • 2McMaster University
  • 3Toyota InfoTechnology Center USA
  • 4Toyota North America
  • 5Toyota InfoTechnology Center, USA

Details

09:15 - 09:30 | Mon 1 Jun | Room T6 | MoA06.1

Session: Autonomous Driving I

Abstract

Vehicles at highway merging sections must make lane changes to join the highway. This lane change can generate congestion. To reduce congestion, vehicles should merge so as not to affect traffic flow as much as possible. In our study, we propose a vehicle controller called Deep Merging that uses deep reinforcement learning to improve the merging efficiency of vehicles while considering the impact on traffic flow. The system uses the images of a merging section as input to output the target vehicle speed. Moreover, an embedding network for estimating the controlled vehicle speed is introduced to the deep reinforcement learning network architecture to improve the learning efficiency. In order to show the effectiveness of the proposed method, the merging behavior and traffic conditions in several situations are verified by experiments using a traffic simulator. Through these experiments, it is confirmed that the proposed method enables controlled vehicles to effectively merge without adversely affecting to the traffic flow.