Adaptive component composition and load balancing for distributed stream processing applications

Providing real-time and QoS support to stream processing applications running on top of large-scale overlays is challenging due to the inherent heterogeneity and resource limitations of the nodes and the multiple QoS demands of the applications that must concurrently be met. In this paper we propose an integrated adaptive component composition and load balancing mechanism that (1) allows the composition of distributed stream processing applications on the fly across a large-scale system, while satisfying their QoS demands and distributing the load fairly on the resources, and (2) adapts dynamically to changes in the resource utilization or the QoS requirements of the applications. Our extensive experimental results using both simulations as well as a prototype deployment illustrate the efficiency, performance and scalability of our approach.

Vana Kalogeraki (Corresponding author)Email:

---

Thomas Repantis   is a PhD candidate at the Computer Science and Engineering Department of the University of California, Riverside. His research interests lie in the area of distributed systems, distributed stream processing systems, middleware, peer-to-peer systems, pervasive and cluster computing. He holds an MSc from the University of California, Riverside and a Diploma from the University of Patras, Greece, and has interned with IBM Research, Intel Research and Hewlett-Packard. Yannis Drougas   is currently a Ph.D. student in the Department of Computer Science and Engineering at University of California, Riverside. He received the Diploma in Electrical and Computer Engineering from Technical University of Crete, Greece in 2003. His research interests include peer-to-peer systems, real-time systems, stream processing systems, resource management and sensor networks. Vana Kalogeraki   is currently an Associate Professor in the Department of Computer Science and Engineering at the University of California, Riverside. She received the Ph.D. in Electrical and Computer Engineering from the University of California, Santa Barbara, in 2000. Previously she was an Assistant Professor in the Department of Computer Science and Engineering at the University of California, Riverside (2002–2008) and held a Research Scientist Position at Hewlett Packard Labs in Palo Alto, CA (2001–2002). Her research interests include distributed systems, peer-to-peer systems, real-time systems, resource management and sensor networks.