Adaptive component composition and load balancing for distributed stream processing applications

Providing real-time and QoS support to stream processing applications running on top of large-scale overlays is challenging due to the inherent heterogeneity and resource limitations of the nodes and the multiple QoS demands of the applications that must concurrently be met. In this paper we propose an integrated adaptive component composition and load balancing mechanism that (1) allows the composition of distributed stream processing applications on the fly across a large-scale system, while satisfying their QoS demands and distributing the load fairly on the resources, and (2) adapts dynamically to changes in the resource utilization or the QoS requirements of the applications. Our extensive experimental results using both simulations as well as a prototype deployment illustrate the efficiency, performance and scalability of our approach.

PatchPeer: A scalable video-on-demand streaming system in hybrid wireless mobile peer-to-peer networks

Video-on-demand service in wireless networks is one important step to achieving the goal of providing video services anywhere anytime. Typically, carrier mobile networks are used to deliver videos wirelessly. Since every video stream comes from the base station, regardless of what bandwidth sharing techniques are being utilized, the media stream system is still limited by the network capacity of the base station. The key to overcome the scalability issue is to exploit resources available at mobile clients in a peer-to-peer setting. We observe that it is common to have a carrier mobile network and a mobile peer-to-peer network co-exist in a wireless environment. A feature of such hybrid environment is that the former offers high availability assurance, while the latter presents an opportunistic use of resources available at mobile clients. Our proposed video-on-demand technique, PatchPeer, leverages this network characteristic to allow the video-on-demand system scale beyond the bandwidth capacity of the server. Mobile clients in PatchPeer are no longer passive receivers, but also active senders of video streams to other mobile clients. Our extensive performance study shows that PatchPeer can accept more clients than the current state-of-the-art technique, while maintaining the same Quality-of-Service to clients.

An inexpensive unstructured platform for wireless mobile peer-to-peer networks

In this paper, we propose an unstructured platform, namely I nexpensive P eer-to- P eer S ubsystem (IPPS), for wireless mobile peer-to-peer networks. The platform addresses the constraints of expensive bandwidth of wireless medium, and limited memory and computing power of mobile devices. It uses a computationally-, memory requirement- and communication- wise inexpensive gossip protocol as the main maintenance operation, and exploits location information of the wireless nodes to minimize the number of link-level messages for communication between peers. As a result, the platform is not only lightweight by itself, but also provides a low cost framework for different peer-to-peer applications. In addition, further enhancements are introduced to enrich the platform with robustness and tolerance to failures without incurring any additional computational and memory complexity, and communication between peers. In specific, we propose schemes for a peer (1) to chose a partner for a gossip iteration, (2) to maintain the neighbors, and (3) to leave the peer-to-peer network. Simulation results are given to demonstrate the performance of the platform.

Revisiting free riding and the Tit-for-Tat in BitTorrent: A measurement study

In this paper we report new results of our continuous effort on analyzing the impact of incentive mechanisms on user behavior in BitTorrent. In this second measurement and analysis study we find that free riders’ population has significantly increased comparing to our previous measurement study. We relate this increase to the advance in end-users’ connection speeds and to users’ increased knowledge in BitTorrent. We also categorize free riders based on the behavior they exhibit in multiple-torrent system into three types: cheaters, strategic and lucky peers. Furthermore, refuting the findings of other studies, we show that peers who exploit the system in BitTorrent are both high bandwidth capacity peers and low bandwidth capacity peers. Moreover, we argue that the Tit-for-Tat mechanism does not discriminate peers based on their bandwidth capacities and that it reacts successfully against inter-class bandwidth capacity strategic peers. Finally, we propose a memory-backoff approach to the optimistic unchoke policy that reduces the volume of free riding in BitTorrent.

An analysis of peer-to-peer networks with altruistic peers

We develop a new model of the interaction of rational peers in a Peer-to-Peer (P2P) network that has at its heart altruism, an intrinsic parameter reflecting peers’ inherent willingness to contribute. Two different approaches for modelling altruistic behavior and its attendant benefit are introduced. With either approach, we use Game Theoretic analysis to calculate Nash equilibria and predict peer behavior in terms of individual contribution. We consider the cases of P2P networks of peers that (i) have homogeneous altruism levels or (ii) have heterogeneous altruism levels, but with known probability distributions. We find that, under the effects of altruism, a substantial fraction of peers will contribute when altruism levels are within certain intervals, even though no incentive mechanism is used. Our results corroborate empirical evidence of large P2P networks surviving or even flourishing without or with barely functioning incentive mechanisms. We also enhance the model with a simple but powerful incentive scheme to limit free-riding and increase contribution to the network, and show that the particular incentive scheme on networks with altruistic peers achieves its goal.

Peer sampling with improved accuracy

Node sampling services provide peers in a peer-to-peer system with a source of randomly chosen addresses of other nodes. Ideally, samples should be independent and uniform. The restrictions of a distributed environment, however, introduce various dependancies between samples. We review gossip-based sampling protocols proposed in previous work, and identify sources of inaccuracy. These include replicating the items from which samples are drawn, and imprecise management of the process of refreshing items. Based on this analysis, we propose a new protocol, Eddy, which aims to minimize temporal and spatial dependancies between samples. We demonstrate, through extensive simulation experiments, that these changes lead to an improved sampling service. Eddy maintains a balanced distribution of items representing active system nodes, even in the face of realistic levels of message loss and node churn. As a result, it behaves more like a centralized random number generator than previous protocols. We demonstrate this by showing that using Eddy improves the accuracy of a simple algorithm that uses random samples to estimate the size of a peer-to-peer network.

Context-aware trust negotiation in peer-to-peer service collaborations

Service-oriented architecture (SOA) and Software as a Service (SaaS) are the latest hot topics to software manufacturing and delivering, and attempt to provide a dynamic cross-organisational business integration solution. In a dynamic cross-organisational collaboration environment, services involved in a business process are generally provided by different organisations, and lack supports of common security mechanisms and centralized management middleware. On such occasions, services may have to achieve middleware functionalities and achieve business objectives in a pure peer-to-peer fashion. As the participating services involved in a business process may be selected and combined at run time, a participating service may have to collaborate with multiple participating services which it has no pre-existing knowledge in prior. This introduces some new challenges to traditional trust management mechanisms. Automated Trust Negotiation (ATN) is a practical approach which helps to generate mutual trust relationship for collaborating principals which may have no pre-existing knowledge about each other without in a peer-to-peer way. Because credentials often contain sensitive attributes, ATN defines an iterative and bilateral negotiation process for credentials exchange and specifies security policies that regulate the disclosure of sensitive credentials. Credentials disclosure in the iterative process may follow different orders and combinations, each of which forms a credential chain. It is practically desirable to identify the optimal credential chain that satisfies certain objectives such as minimum release of sensitive information and minimum performance penalty. In this paper we present a heuristic and context-aware algorithm for identifying the optimal chain that uses context-related knowledge to minimize 1) the release of sensitive information including both credentials and policies and 2) the cost of credentials retrieving. Moreover, our solution offers a hierarchical method for protecting sensitive policies and provides a risk-based strategy for handling credential circular dependency. We have implemented the ATN mechanisms based on our algorithm and incorporated them into the CROWN Grid middleware. Experimental results demonstrate their performance-related advantages over other existing solutions.

Exploiting unlabeled data to improve peer-to-peer traffic classification using incremental tri-training method

Unlabeled training examples are readily available in many applications, but labeled examples are fairly expensive to obtain. For instance, in our previous works on classification of peer-to-peer (P2P) Internet traffics, we observed that only about 25% of examples can be labeled as “P2P”or “NonP2P” using a port-based heuristic rule. We also expect that even fewer examples can be labeled in the future as more and more P2P applications use dynamic ports. This fact motivates us to investigate the techniques which enhance the accuracy of P2P traffic classification by exploiting the unlabeled examples. In addition, the Internet data flows dynamically in large volumes (streaming data). In P2P applications, new communities of peers often join and old communities of peers often leave, requiring the classifiers to be capable of updating the model incrementally, and dealing with concept drift. Based on these requirements, this paper proposes an incremental Tri-Training (iTT) algorithm. We tested our approach on a real data stream with 7.2 Mega labeled examples and 20.4 Mega unlabeled examples. The results show that iTT algorithm can enhance accuracy of P2P traffic classification by exploiting unlabeled examples. In addition, it can effectively deal with dynamic nature of streaming data to detect the changes in communities of peers. We extracted attributes only from the IP layer, eliminating the privacy concern associated with the techniques that use deep packet inspection.

A unifying framework of rating users and data items in peer-to-peer and social networks

We propose a unifying family of quadratic cost functions to be used in Peer-to-Peer ratings. We show that our approach is general since it captures many of the existing algorithms in the fields of visual layout, collaborative filtering and Peer-to-Peer rating, among them Koren spectral layout algorithm, Katz method, Spatial ranking, Personalized PageRank and Information Centrality. Besides of the theoretical interest in finding common basis of algorithms that where not linked before, we allow a single efficient implementation for computing those various rating methods. We introduce a distributed solver based on the Gaussian Belief Propagation algorithm which is able to efficiently and distributively compute a solution to any single cost function drawn from our family of quadratic cost functions. By implementing our algorithm once, and choosing the computed cost function dynamically on the run we allow a high flexibility in the selection of the rating method deployed in the Peer-to-Peer network. Using simulations over real social network topologies obtained from various sources, including the MSN Messenger social network, we demonstrate the applicability of our approach. We report simulation results using networks of millions of nodes.

Automated music video generation using multi-level feature-based segmentation

We show how to create a music video automatically, using computable characteristics of the video and music to promote coherent matching. We analyze the flow of both music and video, and then segment them into sequences of near-uniform flow. We extract features from the both video and music segments, and then find matching pairs. The granularity of the matching process can be adapted by extending the segmentation process to several levels. Our approach drastically reduces the skill required to make simple music videos.

Receiver-side semantic reasoning for digital TV personalization in the absence of return channels

Experience has proved that interactive applications delivered through Digital TV must provide personalized information to the viewers in order to be perceived as a valuable service. Due to the limited computational power of DTV receivers (either domestic set-top boxes or mobile devices), most of the existing systems have opted to place the personalization engines in dedicated servers, assuming that a return channel is always available for bidirectional communication. However, in a domain where most of the information is transmitted through broadcast, there are still many cases of intermittent, sporadic or null access to a return channel. In such situations, it is impossible for the servers to learn who is watching TV at the moment, and so the personalization features become unavailable. To solve this problem without sacrificing much personalization quality, this paper introduces solutions to run a downsized semantic reasoning process in the DTV receivers, supported by a pre-selection of material driven by audience stereotypes in the head-end. Evaluation results are presented to prove the feasibility of this approach, and also to assess the quality it achieves in comparison with previous ones.

Self-stabilization in preference-based systems

Participants of a decentralized system often use some local ranking informations, for selection of effective collaborations. We say that such systems are preference-based. For most practical types of preferences, such systems converge towards a unique stable configuration. In this paper, we investigate the speed and quality of the convergence process with respect to the model parameters. Our results provide an interesting insight into the design of system parameters, such as the number of connections or the algorithm for choosing new partners.

Presenting software engineering results using structured abstracts: a randomised experiment

When conducting a systematic literature review, researchers usually determine the relevance of primary studies on the basis of the title and abstract. However, experience indicates that the abstracts for many software engineering papers are of too poor a quality to be used for this purpose. A solution adopted in other domains is to employ structured abstracts to improve the quality of information provided. This study consists of a formal experiment to investigate whether structured abstracts are more complete and easier to understand than non-structured abstracts for papers that describe software engineering experiments. We constructed structured versions of the abstracts for a random selection of 25 papers describing software engineering experiments. The 64 participants were each presented with one abstract in its original unstructured form and one in a structured form, and for each one were asked to assess its clarity (measured on a scale of 1 to 10) and completeness (measured with a questionnaire that used 18 items). Based on a regression analysis that adjusted for participant, abstract, type of abstract seen first, knowledge of structured abstracts, software engineering role, and preference for conventional or structured abstracts, the use of structured abstracts increased the completeness score by 6.65 (SE 0.37, p < 0.001) and the clarity score by 2.98 (SE 0.23, p < 0.001). 57 participants reported their preferences regarding structured abstracts: 13 (23%) had no preference; 40 (70%) preferred structured abstracts; four preferred conventional abstracts. Many conventional software engineering abstracts omit important information. Our study is consistent with studies from other disciplines and confirms that structured abstracts can improve both information content and readability. Although care must be taken to develop appropriate structures for different types of article, we recommend that Software Engineering journals and conferences adopt structured abstracts.

Analyzing the efficacy of using digital ink devices in a learning environment

There has been increased interest on the impact of mobile devices such as PDAs and Tablet PCs in introducing new pedagogical approaches and active learning experiences. We propose an intelligent system that efficiently addresses the inherent subjectivity in student perception of note taking and information retrieval. We employ the idea of cross indexing the digital ink notes with matching electronic documents in the repository. Latent Semantic Indexing is used to perform document and page level indexing. Thus for each retrieved document, the user can go over to the relevant pages that match the query. Techniques to handle problems such as polysemy (multiple meanings of a word) in large databases, document folding and no match for query are discussed. We tested our system for its performance, usability and effectiveness in the learning process. The results from the exploratory studies reveal that the proposed system provides a highly enhanced student learning experience, thereby facilitating high test scores.

Using visual and text features for direct marketing on multimedia messaging services domain

Traditionally, direct marketing companies have relied on pre-testing to select the best offers to send to their audience. Companies systematically dispatch the offers under consideration to a limited sample of potential buyers, rank them with respect to their performance and, based on this ranking, decide which offers to send to the wider population. Though this pre-testing process is simple and widely used, recently the industry has been under increased pressure to further optimize learning, in particular when facing severe time and learning space constraints. The main contribution of the present work is to demonstrate that direct marketing firms can exploit the information on visual content to optimize the learning phase. This paper proposes a two-phase learning strategy based on a cascade of regression methods that takes advantage of the visual and text features to improve and accelerate the learning process. Experiments in the domain of a commercial Multimedia Messaging Service (MMS) show the effectiveness of the proposed methods and a significant improvement over traditional learning techniques. The proposed approach can be used in any multimedia direct marketing domain in which offers comprise both a visual and text component.

T-MAESTRO and its authoring tool: using adaptation to integrate entertainment into personalized t-learning

Interactive Digital TV opens new learning possibilities where new forms of education are needed. On the one hand, the combination of education and entertainment is essential to boost the participation of viewers in TV learning (t-learning), overcoming their typical passiveness. On the other hand, researchers broadly agree that in order to prevent the learner from abandoning the learning experience, it is necessary to take into account his/her particular needs and preferences by means of a personalized experience. Bearing this in mind, this paper introduces a new approach to the conception of personalized t-learning: edutainment and entercation experiences. These experiences combine TV programs and learning contents in a personalized way, with the aim of using the playful nature of TV to make learning more attractive and to engage TV viewers in learning. This paper brings together our work in constructing edutainment/entercation experiences by relating TV and learning contents. Taking personalization one step further, we propose the adaptation of learning contents by defining A-SCORM (Adaptive-SCORM), an extension of the ADL SCORM standard. Over and above the adaptive add-ons, this paper focuses on two fundamental entities for the proposal: (1) an Intelligent Tutoring System, called T-MAESTRO, which constructs the t-learning experiences by applying semantic knowledge about the t-learners; and (2) the authoring tool which allow teachers to create adaptive courses with a minimal technical background.

Web data retrieval: solving spatial range queries using k-nearest neighbor searches

As Geographic Information Systems (GIS) technologies have evolved, more and more GIS applications and geospatial data are available on the web. Spatial objects in a given query range can be retrieved using spatial range query − one of the most widely used query types in GIS and spatial databases. However, it can be challenging to retrieve these data from various web applications where access to the data is only possible through restrictive web interfaces that support certain types of queries. A typical scenario is the existence of numerous business web sites that provide their branch locations through a limited “nearest location” web interface. For example, a chain restaurant’s web site such as McDonalds can be queried to find some of the closest locations of its branches to the user’s home address. However, even though the site has the location data of all restaurants in, for example, the state of California, it is difficult to retrieve the entire data set efficiently due to its restrictive web interface. Considering that k-Nearest Neighbor (k-NN) search is one of the most popular web interfaces in accessing spatial data on the web, this paper investigates the problem of retrieving geospatial data from the web for a given spatial range query using only k-NN searches. Based on the classification of k-NN interfaces on the web, we propose a set of range query algorithms to completely cover the rectangular shape of the query range (completeness) while minimizing the number of k-NN searches as possible (efficiency). We evaluated the efficiency of the proposed algorithms through statistical analysis and empirical experiments using both synthetic and real data sets.

DAg-stream: Distributed video adaptation for overlay streaming to heterogeneous devices

Combining the advantages of Peer-to-Peer (P2P) content distribution concept and metadata driven adaptation of videos in compressed domain, in this paper, we propose a simple but scalable design of distributed adaptation and overlay streaming using MPEG-21 gBSD, called DAg-stream. The objective is not only to shift the bandwidth burden to end participating peers, but also to move the computation load for adapting video contents away from dedicated media-streaming/adaptation servers. It is an initiative to merge the adaptation operations and the P2P streaming basics to support the expansion of context-aware mobile P2P systems. DAg-stream organizes mobile and heterogeneous peers into overlays. For each video, a separate overlay is formed. No control message is exchanged among peers for overlay maintenance. We present a combination of infrastructure-centric and application end-point architecture. The infrastructure-centric architecture refers to a tree controller, named DAg-master, which is responsible for tree/overlay administering and maintenance. The application end-point architecture refers to video sharing, streaming and adaptation by the participating resourceful peers. The motivation for this work is based on the experiences and lessons learned so far about developing a video adaptation system for heterogeneous devices. In this article, we present our architecture and some experimental evaluations supporting the design concept for overlay video streaming and online adaptation.

Cloaking locations for anonymous location based services: a hybrid approach

An important privacy issue in Location Based Services is to hide a user’s identity while still provide quality location based services. Previous work has addressed the problem of locational -anonymity either based on centralized or decentralized schemes. However, a centralized scheme relies on an anonymizing server (AS) for location cloaking, which may become the performance bottleneck when there are large number of clients. More importantly, holding information in a centralized place is more vulnerable to malicious attacks. A decentralized scheme depends on peer communication to cloak locations and is more scalable. However, it may pose too much computation and communication overhead to the clients. The service fulfillment rate may also be unsatisfied especially when there are not enough peers nearby. This paper proposes a new hybrid framework called HiSC that balances the load between the AS and mobile clients. HiSC partitions the space into base cells and a mobile client claims a surrounding area consisting of base cells. The number of mobile clients in the surrounding cells is kept and updated at both client and AS sides. A mobile client can either request cloaking service from the centralized AS or use a peer-to-peer approach for spatial cloaking based on personalized privacy, response time, and service quality requirements. HiSC can elegantly distribute the work load between the AS and the mobile clients by tuning one system parameter base cell size and two client parameters - surrounding cell size and tolerance count. By integrating salient features of two schemes, HiSC successfully preserves query anonymity and provides more scalable and consistent service. Both the AS and the clients can enjoy much less work load. Additionally, we propose a simple yet effective random range shifting algorithm to prevent possible privacy leakage that would exist in the original P2P approach. Our experiments show that HiSC can elegantly balance the work load based on privacy requirements and client distribution. HiSC provides close to optimal service quality. Meanwhile, it reduces the response time by more than an order of magnitude from both the P2P scheme and the centralized scheme when anonymity level(value of ) or number of clients is large. It also reduces the update message cost of the AS by nearly 6 times and the peer searching message cost of the clients by more than an order of magnitude.