Faster R-CNN-Based Deep Q-Network for Smart Traffic Light Control

Abstract

Abstract—Conventional fixed-time traffic light control systems cannot accommodate real-time fluctuations in vehicle volume, making adaptive approaches based on reinforcement learning an increasingly popular alternative. This study implements a Deep Q-Network (DQN) architecture combined with a Faster R-CNN ResNet-50 FPN v2 object detection sensor for smart traffic light control. Traffic data were collected from CCTV recordings of a T-intersection in the city of Surabaya via the SITS application and processed into a Discrete Traffic State Encoding (DTSE) representation sized 4×20×3 as the model input. Evaluation was carried out under three time conditions (morning, afternoon, and night) using the average queue length, average reward, and total departed metrics. The Faster R-CNN detector achieved a recall of 96.21%, outperforming comparable single-stage detection methods and providing more reliable traffic observations for the reinforcement learning agent. Among the evaluated checkpoints, the model trained for 500 episodes delivered the best overall performance, with an average queue length of 127.34, an average reward of 161.75, and 1,046.97 departed vehicles. These findings suggest that accurate vehicle detection plays an important role in improving reinforcement learning performance for adaptive traffic signal control.

Keywords—Deep Q-Network, Faster R-CNN, Adaptive Traffic Light, DTSE, Reinforcement Learning.