StatStream: statistical monitoring of thousands of data streams in real time

摘要: Consider the problem of monitoring tens thousands time series data streams in an online fashion and making decisions based on them. In addition to single stream statistics such as average standard deviation, we also want find high correlations among all pairs streams. A stock market trader might use a tool spot arbitrage opportunities. This paper proposes efficient methods for solving this Discrete Fourier Transforms three level interval hierarchy. Extensive experiments synthetic real world financial trading show that our algorithm beats direct computation approach by several orders magnitude. It improves previous Transform approaches allowing time-delayed correlation over any size sliding window delay. Correlation lends itself grid-based structure. The result is first know compute time. incremental, has fixed response time, can monitor pairwise 10,000 PC. embarrassingly parallelizable.