Decentralized Coordinated Optimal Ramp Metering using Multi-agent Reinforcement Learning.
详细信息
文摘
Freeways are the major arteries of the transportation networks. In most major cities in North America,including Toronto,infrastructure expansion has fallen behind transportation demand,causing escalating congestion problems. It has been realized that infrastructure expansion cannot provide a complete solution to congestion problems owed to economic limitations,induced demand,and,in metropolitan areas,simply lack of space. Furthermore,the drop in freeway throughput due to congestion exacerbates the problem even more during rush hours at the time the capacity is needed the most. Dynamic traffic control measures provide a set of cost effective congestion mitigation solutions,among which ramp metering (RM) is the most effective approach. This thesis proposes a novel optimal ramp control (metering) system that coordinates the actions of multiple on-ramps in a decentralized structure. The proposed control system is based on reinforcement learning (RL)； therefore,the control agent learn the optimal action from interaction with the environment and without reliance on any a priori mathematical model. The agents are designed to function optimally in both independent and coordinated modes. Therefore,the whole system is robust to communication or individual agent's failure. The RL agents employ function approximation to directly represent states and action with continuous variables instead of relying on discrete state-action tables. Use of function approximation significantly speeds up the learning and reduces the complexity of the RL agents design process. The proposed RM control system is applied to a meticulously calibrated microsimulation model of the Gardiner Expressway westbound in Toronto,Canada. The Gardiner expressway is the main freeway running through Downtown Toronto and suffers from extended periods of congestion every day. It was chosen as the testbed for highlighting the effectiveness of the coordinated RM. The proposed coordinated RM algorithm when applied to the Gardiner model resulted in 50% reduction in total travel time compared with the base case scenario and significantly outperformed approaches based on the well-known ALINEA RM algorithm. This improvement was achieved while the permissible on-ramp queue limit was satisfied.