Hybrid Large Language Models and Reinforcement Learning for Energy-Efficient Multisatellite Scheduling: Boosting the Performance From Scratch

초록

Low Earth orbit (LEO) satellite constellations are crucial for global connectivity by providing extensive coverage and reduced delays. However, scheduling data transmission in these dynamic networks is challenging due to rapidly changing satellite positions. This research introduces boosted reinforcement learning (BoostRL), a novel framework integrating large language models (LLMs) with reinforcement learning (RL) for efficient scheduling in LEO satellite constellations. BoostRL leverages LLM-generated initial policies to accelerate convergence, thereby guiding the early stages of policy learning while adapting swiftly to dynamic network conditions. It uses a hybrid policy approach which transitions smoothly from LLM recommendations to autonomous RL policies. A tailored initialization of Q -value parameters and an enhanced loss function further optimize learning efficiency, by aligning the initial learning phase with LLM-generated insights. Simulations using two-line element (TLE) orbital data demonstrate that BoostRL achieves rapid convergence and improved efficiency, thereby validating its potential as a scalable, adaptive solution for managing satellite communication networks.

키워드