Scalable Distributed Stream Processing

摘要: Stream processing fits a large class of new applications for which conventional DBMSs fall short. Because many stream-oriented systems are inherently geographically distributed and because distribution offers scalable load management higher availability, future stream will operate in fashion. They run across the Internet on computers typically owned by multiple cooperating administrative domains. This paper describes architectural challenges facing design large-scale systems, discusses novel approaches addressing management, high federated operation issues. We describe two Aurora* Medusa, being designed to explore complementary solutions these challenges. issues systems. begin Section 2 with brief description our centralized system, Aurora [4]. then discuss efforts extend environment: Medusa. assumes an environment all nodes under single domain. Medusa provides infrastructure support boundaries. After describing architectures 3, we consider three common both: infrastructures protocols supporting communication amongst (Section 4), sharing response variable network conditions 5), availability presence failures 6). also high-level policy specifications employed 7. For issues, believe that push-based nature stream-based not only raises but possibility domain-specific solutions.