The Fast and The Frugal: Tail Latency Aware Provisioning for Coping with Load Variations

摘要: Small and medium sized enterprises use the cloud for running online, user-facing, tail latency sensitive applications with well-defined fixed monthly budgets. For these applications, adequate system capacity must be provisioned to extract maximal performance despite challenges of uncertainties in load request-sizes. In this paper, we address problem provisioning under budget constraints goal minimizing latency. To tackle problem, propose building systems using a heterogeneous mix low expensive resources cheap that provide high throughput per dollar. As changes through day, more faster reduce during periods cheaper handle periods. achieve benefits, introduce novel heterogeneity-aware scheduling autoscaling algorithms are designed Using software prototypes by experiments on public cloud, show our approach can outperform existing reducing as much 45% fixed-budget settings.