Improving Spatiotemporal Self-supervision by Deep Reinforcement Learning

摘要: Self-supervised learning of convolutional neural networks can harness large amounts cheap unlabeled data to train powerful feature representations. As surrogate task, we jointly address ordering visual in the spatial and temporal domain. The permutations training samples, which are at core self-supervision by ordering, have so far been sampled randomly from a fixed preselected set. Based on deep reinforcement propose sampling policy that adapts state network, is being trained. Therefore, new according their expected utility for updating representation. Experimental evaluation unsupervised transfer tasks demonstrates competitive performance standard benchmarks image video classification nearest neighbor retrieval.