Reinforcement Learning With Function Approximation for Traffic Signal Control

LA, Prashanth ; Bhatnagar, Shalabh (2011) Reinforcement Learning With Function Approximation for Traffic Signal Control IEEE Transactions on Intelligent Transportation Systems, 12 (2). pp. 412-421. ISSN 1524-9050

Full text not available from this repository.

Official URL: http://doi.org/10.1109/TITS.2010.2091408

Related URL: http://dx.doi.org/10.1109/TITS.2010.2091408

Abstract

We propose, for the first time, a reinforcement learning (RL) algorithm with function approximation for traffic signal control. Our algorithm incorporates state-action features and is easily implementable in high-dimensional settings. Prior work, e.g., the work of Abdulhai , on the application of RL to traffic signal control requires full-state representations and cannot be implemented, even in moderate-sized road networks, because the computational complexity exponentially grows in the numbers of lanes and junctions. We tackle this problem of the curse of dimensionality by effectively using feature-based state representations that use a broad characterization of the level of congestion as low, medium, or high. One advantage of our algorithm is that, unlike prior work based on RL, it does not require precise information on queue lengths and elapsed times at each lane but instead works with the aforementioned described features. The number of features that our algorithm requires is linear to the number of signaled lanes, thereby leading to several orders of magnitude reduction in the computational complexity. We perform implementations of our algorithm on various settings and show performance comparisons with other algorithms in the literature, including the works of Abdulhai and Cools , as well as the fixed-timing and the longest queue algorithms. For comparison, we also develop an RL algorithm that uses full-state representation and incorporates prioritization of traffic, unlike the work of Abdulhai We observe that our algorithm outperforms all the other algorithms on all the road network settings that we consider.

Item Type:Article
Source:Copyright of this article belongs to Institute of Electrical and Electronics Engineers.
Keywords:Q-Learning With Full-State Representation (QTLC-FS); Q-Learning With Function Approximation (QTLC-FA); Reinforcement Learning (RL); Traffic Signal Control.
ID Code:116537
Deposited On:12 Apr 2021 06:45
Last Modified:12 Apr 2021 06:45

Repository Staff Only: item control page