Title:Graph Attention Reinforcement Learning for Multicast Routing and Age-Optimal Scheduling

Abstract:Age of Information (AoI) has emerged as a prominent metric for evaluating the timeliness of information in time-critical applications, such as video streaming, virtual reality, and metaverse platforms, which often rely on multicast communication. Optimizing AoI in multicast networks is challenging due to the coupling of multicast routing and scheduling decisions, the complexity of the multicast, and the graph representation. This paper focuses on dynamic multicast networks and aims to minimize the expected average AoI by integrating multicast routing and scheduling. To address the inherent complexity of the problem, we first decompose the original problem into two subtasks amenable to hierarchical reinforcement learning (RL) methods. We propose the first RL framework to address the multicast routing problem, also known as the Steiner Tree problem, by incorporating graph embedding and the successive addition of nodes and links. For graph embedding, we propose the Normalized Graph Attention mechanism (NGAT) framework with a proven contraction mapping property, enabling effective graph information capture and superior generalization within the hierarchical RL framework. We validate our framework through experiments on four datasets, including the real-world AS-733 dataset. The results demonstrate that our proposed scheme can be $11.56\times$ more computationally efficient than traditional multicast routing algorithms while achieving approximation ratios of 1.1-1.3, comparable to state-of-the-art (SOTA) methods. Additionally, our age-optimal TGMS algorithm reduces the average weighted Age of Information (AoI) by 25.6% and the weighted peak age by 29.2% in low-energy scenarios.