Controlling Large, Graph-Based MDPs with Global Control Capacity Constraints: An Approximate LP Solution

Abstract

We consider controlling graph-based MDPs (GMDPs) with two special properties: (i) Anonymous Influence and (ii) Symmetry. Large-scale spatial processes such as wildfires, disease epidemics, opinion dynamics, and robot swarms are well modeled by GMDPs with these properties. We derive two efficient and scalable algorithms for computing approximately optimal control policies for large GMDPs with Anonymous Influence and Symmetry and derive sub-optimality bounds for these policies. Unlike prior work, our algorithms explicitly enforce a global control capacity constraint. Our methods scale linearly in the number of equivalence classes in the GMDP rather than the total number of MDPs in the graph. We demonstrate our methods in simulations of controlling a wildfire with a global fire retardant constraint and controlling an Ebola outbreak with a global medicine constraint. Our Ebola model is derived from data from the 2014 West Africa outbreak.