Don't Unroll Adjoint: Differentiating SSA-Form Programs.
作者： Michael Innes
DOI:
关键词:
摘要: This paper presents reverse-mode algorithmic differentiation (AD) based on source code transformation, in particular of the Static Single Assignment (SSA) form used by modern compilers. The approach can support control flow, nesting, mutation, recursion, data structures, higher-order functions, and other language constructs, output is given to an existing compiler produce highly efficient differentiated code. Our implementation a new AD tool for Julia language, called Zygote, which high-level dynamic semantics while transparently compiling adjoint under hood. We discuss benefits this both usability performance tools.
arxiv.org
harvard.edu
arxiv-vanity.com参考文章(22)
Christophe Foket, Bjorn De Sutter, Tim Besard, Effective Extensible Programming: Unleashing Julia on GPUs IEEE Transactions on Parallel and Distributed Systems. ,vol. 30, pp. 827- 841 ,(2019) , 10.1109/TPDS.2018.2872064
Scott Cyphers, Arjun K Bansal, Anahita Bhiwandiwalla, Jayaram Bobba, Matthew Brookhart, Avijit Chakraborty, Will Constable, Christian Convey, Leona Cook, Omar Kanawi, Robert Kimball, Jason Knight, Nikolay Korovaiko, Varun Kumar, Yixing Lao, Christopher R Lishka, Jaikrishnan Menon, Jennifer Myers, Sandeep Aswath Narayana, Adam Procter, Tristan J Webb, None, Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning arXiv: Distributed, Parallel, and Cluster Computing. ,(2018)
Mike Innes, Flux: Elegant machine learning with Julia Journal of Open Source Software. ,vol. 3, pp. 602- ,(2018) , 10.21105/JOSS.00602
Bart van Merriënboer, Olivier Breuleux, Arnaud Bergeron, Pascal Lamblin, Automatic differentiation in ML: Where we are and where we should be going neural information processing systems. ,vol. 31, pp. 8757- 8767 ,(2018)
Elliot Saba, Keno Fischer, Automatic Full Compilation of Julia Programs and ML Models to Cloud TPUs arXiv: Programming Languages. ,(2018)
Luca Antiga, Sam Gross, Soumith Chintala, Alban Desmaison, Adam Paszke, Zeming Lin, Adam Lerer, Edward Yang, Gregory Chanan, Zachary DeVito, Automatic differentiation in PyTorch ,(2017)
D Yuret, M Innes, J Bradbury, A Edelman, S Karpinski, T Besard, D Barber, J Revels, Pleps Saito Stenetorp, S Danisch, J Malmaud, Churavy, Shah, On Machine Learning and Programming Languages usenix workshop on tackling computer systems problems with machine learning techniques. ,(2018)
Jeffrey Mark Siskind, Atılım Günes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Automatic differentiation in machine learning: a survey Journal of Machine Learning Research. ,vol. 18, pp. 5595- 5637 ,(2017)
Tianqi Chen, Ziheng Jiang, Mu Li, Efficient Deep Learning Inference on Edge Devices ,(2018)
B. Speelpenning, Compiling fast partial derivatives of functions given by algorithms University of Illinois at Urbana-Champaign. ,(1980) , 10.2172/5254402