On environment difficulty and discriminating power

摘要: This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on very general setting for tasks: interactive (possibly multi-agent) environments where an agent acts upon observations rewards. Instead analysing complexity environment, state space or actions that are performed by agent, we analyse performance population policies against task, leading distribution is examined in terms policy complexity. then sliced algorithmic analysed through several diagrams indicators. The notion environment response curve also introduced, inverting results into ability scale. apply all these concepts, indicators two illustrative problems: class agent-populated elementary cellular automata, showing how may vary environments, multi-agent system, agents can become predators preys, need coordinate. Finally, discuss tools be applied characterise (interactive) tasks (multi-agent) environments. These characterisations used get more insight about facilitate development adaptive tests evaluation abilities.