Solving Mixed Influence Diagrams by Reinforcement Learning

While efficient optimisation methods exist for problems with special properties (linear, continuous, differentiable, unconstrained), real-world problems often involve inconvenient complications (constrained, discrete, multi-stage, multi-level, multi-objective). Each of these complications has spawned research areas in Artificial Intelligence and Operations Research, but few methods are available for hybrid problems. We describe a reinforcement learning-based solver for a broad class of discrete problems that we call Mixed Influence Diagrams, which may have multiple stages, multiple agents, multiple non-linear objectives, correlated chance variables, exogenous and endogenous uncertainty, constraints (hard, soft and chance) and partially observed variables. We apply the solver to problems taken from stochastic programming, chance-constrained programming, limited-memory influence diagrams, multi-level and multi-objective optimisation. We expect the approach to be useful on new hybrid problems for which no specialised solution methods exist.