Distributed deep learning on edge-devices: Feasibility via adaptive compression

摘要: A large portion of data mining and analytic services use modern machine learning techniques, such as deep learning. The state-of-the-art results by come at the price an intensive computing resources. leading frameworks (e.g., TensorFlow) are executed on GPUs or high-end servers in datacenters. On other end, there is a proliferation personal devices with possibly free CPU cycles; this can enable to run users' homes, embedding operations. In paper, we ask following question: Is distributed computation WAN connected feasible, spite traffic caused tasks? We show that setup rises some important challenges, most notably ingress hosting up-to-date model have sustain. order reduce stress, propose AdaComp, novel algorithm for compressing worker updates server. Applicable stochastic gradient descent based approaches, it combines efficient selection rate modulation. then experiment measure impact compression, device heterogeneity reliability accuracy learned models, emulator platform embeds TensorFlow into Linux containers. report reduction total amount sent workers server two magnitude 191-fold convolutional network MNIST dataset), when compared standard asynchronous descent, while preserving accuracy.