SMDP homomorphisms: an algebraic approach to abstraction in semi-Markov decision processes

摘要: To operate effectively in complex environments learning agents require the ability to selectively ignore irrelevant details and form useful abstractions. In this article we consider question of what constitutes a abstraction stochastic sequential decision problem modeled as semi-Markov Decision Process (SMDPs). We introduce notion SMDP homomorphism argue that it provides tool for rigorous study SMDPs. present an minimization framework factored MDPs based on homomorphisms. also model different classes abstractions arise hierarchical systems. Although use options purposes illustration, ideas are more generally applicable. show conditions employ generalization earlier work by Dietterich applied framework.