RECCE: Deep Reinforcement Learning for Joint Routing and Scheduling in Time-Constrained Wireless Networks

Time Division Multiple Access-based Medium Access Control protocols tend to be the choice for wireless networks that require deterministic delay guarantees, as is the case in many Industrial Internet of Things (IIoT) applications. As the optimal joint scheduling and routing problem for multi-hop wireless networks is NP-hard, heuristics are generally used for building schedules. However, heuristics normally result in sub-optimal schedules, which may result in packets missing their deadlines. In this paper, we present RECCE, a deep REinforcement learning method for joint routing and sCheduling in time-ConstrainEd networks with centralised control. During training, RECCE considers multiple routes and criteria for scheduling in any given time slot and channel in a multi-channel, multi-hop wireless network. This allows RECCE to explore and learn routes and schedules to deliver more packets within the deadline. Simulation results show that RECCE can reduce the number of packets missing the deadline by as much as 55% and increase schedulability by up to 30%, both relative to the best baseline heuristic. RECCE can deal well with dynamic network conditions, performing better than the best baseline heuristic in up to 74% of the scenarios in the training set and in up to 64% of scenarios not in the training set.