FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On

作者: Haoye Dong , Xiaodan Liang , Xiaohui Shen , Bowen Wu , Bing-Cheng Chen

DOI: 10.1109/ICCV.2019.00125

关键词:

摘要: Beyond current image-based virtual try-on systems that have attracted increasing attention, we move a step forward to developing video system precisely transfers clothes onto the person and generates visually realistic videos conditioned on arbitrary poses. Besides challenges in (e.g., fidelity, image synthesis), further requires spatiotemporal consistency. Directly adopting existing approaches often fails generate coherent with natural textures. In this work, propose Flow-navigated Warping Generative Adversarial Network (FW-GAN), novel framework learns synthesize of based image, desired series target FW-GAN aims while manipulating pose clothes. It consists of: (i) flow-guided fusion module warps past frames assist synthesis, which is also adopted discriminator help enhance coherence quality synthesized video; (ii) warping net designed warp for refinement textures; (iii) parsing constraint loss alleviates problem caused by misalignment segmentation maps from images different poses various Experiments our newly collected dataset show can high-quality significantly outperforms other methods both qualitatively quantitatively.

参考文章(38)
Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, Thomas Brox, None, FlowNet: Learning Optical Flow with Convolutional Networks 2015 IEEE International Conference on Computer Vision (ICCV). pp. 2758- 2766 ,(2015) , 10.1109/ICCV.2015.316
Thomas Brox, Andrés Bruhn, Nils Papenberg, Joachim Weickert, None, High Accuracy Optical Flow Estimation Based on a Theory for Warping european conference on computer vision. pp. 25- 36 ,(2004) , 10.1007/978-3-540-24673-2_3
Xue Bai, Jue Wang, David Simons, Guillermo Sapiro, Video SnapCut: robust video object cutout using localized classifiers international conference on computer graphics and interactive techniques. ,vol. 28, pp. 70- ,(2009) , 10.1145/1531326.1531376
Peng Guan, Loretta Reiss, David A. Hirshberg, Alexander Weiss, Michael J. Black, DRAPE: DRessing Any PErson international conference on computer graphics and interactive techniques. ,vol. 31, pp. 35- ,(2012) , 10.1145/2185520.2185531
Tao Chen, Jun-Yan Zhu, A. Shamir, Shi-Min Hu, Motion-Aware Gradient Domain Video Composition IEEE Transactions on Image Processing. ,vol. 22, pp. 2532- 2544 ,(2013) , 10.1109/TIP.2013.2251642
Masahiro Sekine, Kaoru Sugita, Frank Perbet, Bjorn Stenger, Masashi Nishiyama, Virtual Fitting by Single-Shot Body Shape Estimation 5th International Conference on 3D Body Scanning Technologies, Lugano, Switzerland, 21-22 October 2014. pp. 406- 413 ,(2014) , 10.15221/14.406
, Generative Adversarial Nets neural information processing systems. ,vol. 27, pp. 2672- 2680 ,(2014) , 10.3156/JSOFT.29.5_177_2
Simon Osindero, Mehdi Mirza, Conditional Generative Adversarial Nets arXiv: Learning. ,(2014)
E. Shechtman, Y. Caspi, M. Irani, Space-time super-resolution IEEE Transactions on Pattern Analysis and Machine Intelligence. ,vol. 27, pp. 531- 545 ,(2005) , 10.1109/TPAMI.2005.85
Anna Hilsmann, Peter Eisert, Tracking and Retexturing Cloth for Real-Time Virtual Clothing Applications international conference on computer vision. pp. 94- 105 ,(2009) , 10.1007/978-3-642-01811-4_9