Popularity-based covering sets for energy proportionality in shared-nothing clusters

Energy management for large-scale clusters has been the subject of significant research attention in recent years. The principle of energy proportionality states that we can save energy by activating only a subset of cluster nodes, in proportion to the current load. However, achieving the energy proportionality in shared-nothing clusters is challenging, because the arbitrary deactivation of nodes would make some data become unavailable. In this paper, we propose a new algorithm, named popularity-based covering sets (PCS), to achieve the energy proportionality in large-scale shared-nothing clusters. PCS determines the set of active nodes dynamically, in order to achieve the design goals of (a) guaranteeing the minimum level of availability for every data so that any job can execute promptly, and (b) providing more replicas for popular data to mitigate contention on the data. This differs from previous studies, where some data may become unavailable, or they provide the same number of replicas for every data. Furthermore, PCS is rack-aware and thus it can reduce the energy consumption of power-hungry rack components. Experiment results indicate that PCS improves the overall energy savings by up to 62% compared to previous algorithms without significant performance loss.