Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering
作者： Risto Miikkulainen , Elliot Meyerson
DOI:
关键词:
摘要: The technology disclosed identifies parallel ordering of shared layers as a common assumption underlying existing deep multitask learning (MTL) approaches. This assumption restricts the kinds of shared structure that can be learned between tasks. The technology disclosed demonstrates how direct approaches to removing this assumption can ease the integration of information across plentiful and diverse tasks. The technology disclosed introduces soft ordering as a method for learning how to apply layers in different ways at …
arxiv.org
storage.googleapis.com 参考文章(43)
Ziwei Liu, Xiaoou Tang, Xiaogang Wang, Ping Luo, Deep Learning Face Attributes in the Wild arXiv: Computer Vision and Pattern Recognition. ,(2014)
Tom Schaul, Volodymyr Mnih, Koray Kavukcuoglu, Joel Z Leibo, Max Jaderberg, Wojciech Marian Czarnecki, David Silver, Reinforcement Learning with Unsupervised Auxiliary Tasks arXiv: Learning. ,(2016)
Yoshimasa Tsuruoka, Richard Socher, Caiming Xiong, Kazuma Hashimoto, A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks arXiv: Computation and Language. ,(2016)
Brendan Jou, Shih-Fu Chang, Deep Cross Residual Learning for Multitask Visual Recognition acm multimedia. pp. 998- 1007 ,(2016) , 10.1145/2964284.2964309
Ilya Sutskever, Minh-Thang Luong, Oriol Vinyals, Quoc V. Le, Lukasz Kaiser, Multi-task Sequence to Sequence Learning international conference on learning representations. ,(2016)
Andrea Vedaldi, Hakan Bilen, Integrated perception with recurrent multi-task neural networks neural information processing systems. ,vol. 29, pp. 235- 243 ,(2016)
Amir R Zamir, Te-Lin Wu, Lin Sun, William B Shen, Bertram E Shi, Jitendra Malik, Silvio Savarese, Feedback Networks arXiv: Computer Vision and Pattern Recognition. ,(2016)
Kevin Bache, Moshe Lichman, UCI Machine Learning Repository University of California, School of Information and Computer Science. ,(2007)
Richard Socher, Andrew Y. Ng, Cliff C. Lin, Chris Manning, Parsing Natural Scenes and Natural Language with Recursive Neural Networks international conference on machine learning. pp. 129- 136 ,(2011)
Zhizheng Wu, Cassia Valentini-Botinhao, Oliver Watts, Simon King, Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis international conference on acoustics, speech, and signal processing. pp. 4460- 4464 ,(2015) , 10.1109/ICASSP.2015.7178814