Processing Exact Results for Windowed Stream Joins in a Memory-Limited System: A Disk-Based, Adaptive Approach

摘要: We consider the problem of processing exact results for sliding window joins over data streams with limited memory. Existing approaches either, (1) deal memory limitations by shedding loads, and therefore cannot provide or even highly accurate showing time-varying rate arrivals, (2) suffer from large I/O overhead due to random disk flushes disk-to-disk stages a stream join, making inefficient handle joins. an Adaptive, Hash-partitioned Exact Window Join (AH-EWJ) algorithm incorporating storage as archive. Our spills onto on periodic basis, refines output result properly retrieving disk-resident data, maximizes employing techniques manage blocks, continuously adjusting allocated within windows. The managing blocks in memory—similar nature caching issue—captures both temporal frequency related properties arrivals. present baseline called Rate-based Progressive Joins (RPWJ), which extends existing tune performance reducing while experimental demonstrating effectiveness proposed algorithm.