Large Scale Estimation in Cyberphysical Systems using Streaming Data: a Case Study with Smartphone Traces

摘要: Controlling and analyzing cyberphysical robotics systems is increasingly becoming a Big Data challenge. Pushing this data to, processing in the cloud more efficient than on-board processing. However, current cloud-based solutions are not suitable for latency requirements of these applications. We present new concept, Discretized Streams or D-Streams, that enables massively scalable computations on streaming with latencies as short second. experiment an implementation D-Streams top Spark computing framework. demonstrate usefulness concept novel algorithm to estimate vehicular traffic urban networks. Our online EM can very large city network (the San Francisco Bay Area) by tens thousands observations per second, few seconds. Note Practitioners This work was driven need at scale, setting, using commodity hardware. Machine Learning algorithms combined new, but it still requires deep expertise both Computer Systems achieve scale tractable manner. The Streaming project aims providing interface abstracts out all technical details computation platform (cloud, HPC, workstation, etc.). As shown work, imple- menting calibrating non-trivial cluster, provides intuitive yet powerful programming interface. readers invited refer source code referred article examples. presents sample compute densities Gamma random variables restricted hyperplane (i.e. distributions form Tij P j jTj = d Tj independant distributions). It common case use Gaussian because closed solve. If one considers positive valued heavy tails, our formulas gamma may be suitable.