Hierarchical POMDP controller optimization by likelihood maximization

摘要: Planning can often be simplified by decomposing the task into smaller tasks arranged hierarchically. Charlin et al. [4] recently showed that hierarchy discovery problem framed as a non-convex optimization problem. However, inherent computational difficulty of solving such an makes it hard to scale real-world problems. In another line research, Toussaint [18] developed method solve planning problems maximum-likelihood estimation. this paper, we show how in partially observable domains tackled using similar maximum likelihood approach. Our technique first transforms dynamic Bayesian network through which hierarchical structure naturally discovered while optimizing policy. Experimental results demonstrate approach scales better than previous techniques based on optimization.