A GPU-oriented online recommendation algorithm for efficient processing of time-varying continuous data streams

Research on recommendation systems has gained a considerable amount of attention over the past decade as the number of online users and online contents continue to grow at an exponential rate. With the evolution of the social web, people generate and consume data in real time using online services such as Twitter, Facebook, and web news portals. With the rapidly growing online community, web-based retail systems and social media sites have to process several millions of user requests per day. Generating quality recommendations using this vast amount of data is itself a very challenging task. Nevertheless, opposed to the web-based retailers such as Amazon and Netflix, the above-mentioned social networking sites have to face an additional challenge when generating recommendations as their contents are very rapidly changing. Therefore, providing fresh information in the least amount of time is a major objective of such recommender systems. Although collaborative filtering is a widely used technique in recommendation systems, generating the recommendation model using this approach is a costly task, and often done offline. Hence, it is difficult to use collaborative filtering in the presence of dynamically changing contents, as such systems require frequent updates to the recommendation model to maintain the accuracy and the freshness of the recommendations. Parallel processing power of graphic processing units (GPUs) can be used to process large volumes of data with dynamically changing contents in real time, and accelerate the recommendation process for social media data streams. In this paper, we address the issue of rapidly changing contents, and propose a parallel on-the-fly collaborative filtering algorithm using GPUs to facilitate frequent updates to the recommendations model. We use a hybrid similarity calculation method by combining the item–item collaborative filtering with item category information and temporal information. The experimental results on real-world datasets show that the proposed algorithm outperformed several existing online CF algorithms in terms of accuracy, memory consumption, and runtime. It was also observed that the proposed algorithm scaled well with the data rate and the data volume, and generated recommendations in a timely manner.     

Adaptive listen for energy-efficient medium access control in wireless sensor networks

Wireless Sensor Networks (WSN) have nodes that are small in size and are powered by small batteries having very limited amount of energy. In most applications of WSN, the nodes in the network remain inactive for long periods of time, and intermittently they become active on sensing any change in the environment. The data sensed by the different nodes are sent to the sink node. In contrast to other infrastructure-based wireless networks, higher throughput, lower latency and per-node fairness in WSN are imperative, but their importance is subdued when compared to energy consumption. In this work, we have regarded the amount of energy consumption in the nodes to be of primary concern, while throughput and latency in the network to be secondary. We have proposed a protocol for energy-efficient adaptive listen for medium access control in WSN. Our protocol adaptively changes the slot-time, which is the time of each slot in the contention window. This correspondingly changes the cycle-time, which is the sum of the listen-time and the sleep-time of the sensors, while keeping the duty-cycle, which is the ratio between the listen-time and the cycle-time, constant. Using simulation experiments, we evaluated the performance of the proposed protocol, compared with the popular Sensor Medium Access Control (SMAC) (Ye et al. IEEE/ACM Trans Netw 12(3):493–506, 39) protocol. The results we obtained show a prominent decrease in the energy consumption at the nodes in the proposed protocol over the existing SMAC protocol, at the cost of decreasing the throughput and increasing the latency in the network. Although such an observation is not perfectly what is ideally desired, given the very limited amount of energy with which the nodes in a WSN operate, we advocate that increasing the energy efficiency of the nodes, thereby increasing the network lifetime in WSN, is a more important concern compared to throughput and latency. Additionally, similar observations relating energy efficiency, network lifetime, throughput and latency exist in many other existing protocols, including the popular SMAC protocol (Ye et al. IEEE/ACM Trans Netw 12(3):493–506, 39).     

Adaptive optimization of join trees for multi-join queries over sensor streams

Data processing applications for sensor streams have to deal with multiple continuous data streams with inputs arriving at highly variable and unpredictable rates from various sources. These applications perform various operations (e.g. filter, aggregate, join, etc.) on incoming data streams in real-time according to predefined queries or rules. Since the data rate and data distribution fluctuate over time, an appropriate join tree for processing join queries must be adaptively maintained in response to dynamic changes to prevent rapid degradation of the system performance. In this paper, we address the problem of finding an optimal join tree that maximizes throughput for sliding window based multi-join queries over continuous data streams and prove its NP-Hardness. We present a dynamic programming algorithm, OptDP, which produces the optimal tree but runs in an exponential time in the number of input streams. We then present a polynomial time greedy algorithm, XGreedyJoin. We tested these algorithms in ARES, an adaptively re-optimizing engine for stream queries, which we developed by extending Jess (Jess is a popular RETE-based, forward chaining rule engine written in java). For almost all instances, trees from XGreedyJoin perform close to the optimal trees from OptDP, and significantly better than common heuristics-based XJoin algorithms.     

CHESDA: continuous hybrid and energy-efficient secure data aggregation for WSN

The Journal of Supercomputing - Data aggregation is an effective mechanism to prolong lifetime in the wireless sensor networks by preventing extra data transmission. However, it may have some...     

Adaptive telecom churn prediction for concept-sensitive imbalance data streams

The Journal of Supercomputing - A key toward intelligent decision-making in industries lies in the ability to process and analyze vast quantities of business data. Concept drift and class imbalance...     

Adaptive infinite dropout for noisy and sparse data streams

Machine Learning - The ability to analyze data streams, which arrive sequentially and possibly infinitely, is increasingly vital in various online applications. However, data streams pose various...     

Adaptive scheduling for shared window joins over data streams

Recently a few Continuous Query systems have been developed to cope with applications involving continuous data streams. At the same time, numerous algorithms are proposed for better performance. A recent work on this subject was to define scheduling strategies on shared window joins over data streams from multiple query expressions. In these strategies, a tuple with the highest priority is selected to process from multiple candidates. However, the performance of these static strategies is deeply influenced when data are bursting, because the priority is determined only by static information, such as the query windows, arriving order, etc. In this paper, we propose a novel adaptive strategy where the priority of a tuple is integrated with realtime information. A thorough experimental evaluation has demonstrated that this new strategy can outperform the existing strategies.     

An energy-efficient protocol for data gathering and aggregation in wireless sensor networks

Data gathering is a major function of many applications in wireless sensor networks (WSNs). The most important issue in designing a data gathering algorithm is how to save energy of sensor nodes while meeting the requirement of applications/users such as sensing area coverage. In this paper, we propose a novel hierarchical clustering protocol (DEEG) for long-lived sensor network. DEEG achieves a good performance in terms of lifetime by minimizing energy consumption for in-network communications and balancing the energy load among all the nodes, the proposed protocol achieves a good performance in terms of network lifetime. DEEG can also handle the energy hetergenous capacities and guarantee that out-network communications always occur in the subregion with high energy reserved. Furthermore, it introduces a simple but efficient approach to cope with the area coverage problem. We evaluate the performance of the proposed protocol using a simple temperature sensing application. Simulation results show that our protocol significantly outperforms LEACH and PEGASIS in terms of network lifetime and the amount of data gathered.

Real-time data mining of non-stationary data streams from sensor networks

In real-world sensor networks, the monitored processes generating time-stamped data may change drastically over time. An online data-mining algorithm called OLIN (on-line information network) adapts itself automatically to the rate of concept drift in a non-stationary data stream by repeatedly constructing a classification model from every sliding window of training examples. In this paper, we introduce a new real-time data-mining algorithm called IOLIN (incremental on-line information network), which saves a significant amount of computational effort by updating an existing model as long as no major concept drift is detected. The proposed algorithm builds upon the oblivious decision-tree classification model called “information network” (IN) and it implements three different types of model updating operations. In the experiments with multi-year streams of traffic sensors data, no statistically significant difference between the accuracy of the incremental algorithm (IOLIN) vs. the regenerative one (OLIN) has been observed.     

A clustering approach for sampling data streams in sensor networks

The growing usage of embedded devices and sensors in our daily lives has been profoundly reshaping the way we interact with our environment and our peers. As more and more sensors will pervade our future cities, increasingly efficient infrastructures to collect, process and store massive amounts of data streams from a wide variety of sources will be required. Despite the different application-specific features and hardware platforms, sensor network applications share a common goal: periodically sample and store data collected from different sensors in a common persistent memory. In this article, we present a clustering approach for rapidly and efficiently computing the best sampling rate which minimizes the Sum of Square Error for each particular sensor in a network. In order to evaluate the efficiency of the proposed approach, we carried out experiments on real electric power consumption data streams provided by EDF (électricité de France).     

Exploiting punctuation semantics in continuous data streams

As most current query processing architectures are already pipelined, it seems logical to apply them to data streams. However, two classes of query operators are impractical for processing long or infinite data streams. Unbounded stateful operators maintain state with no upper bound in size and, so, run out of memory. Blocking operators read an entire input before emitting a single output and, so, might never produce a result. We believe that a priori knowledge of a data stream can permit the use of such operators in some cases. We discuss a kind of stream semantics called punctuated streams. Punctuations in a stream mark the end of substreams allowing us to view an infinite stream as a mixture of finite streams. We introduce three kinds of invariants to specify the proper behavior of operators in the presence of punctuation. Pass invariants define when results can be passed on. Keep invariants define what must be kept in local state to continue successful operation. Propagation invariants define when punctuation can be passed on. We report on our initial implementation and show a strategy for proving implementations of these invariants are faithful to their relational counterparts.     

传感器标定的统一数据处理方法

研究传感器静特性参数标定的实验数据处理方法。当被标定参数本身含有随机误差时,现有算法不能获得被标定参数均值的最优估计,基于信息融合理论提出了统一的参数最优估计算法,并通过数值仿真例子验证了其有效性。     

A simple, least-time, and energy-efficient routing protocol with one-level data aggregation for wireless sensor networks

The area of wireless sensor networks (WSN) is currently attractive in the research community area due to its applications in diverse fields such as defense security, civilian applications and medical research. Routing is a serious issue in WSN due to the use of computationally-constrained and resource-constrained micro-sensors. These constraints prohibit the deployment of traditional routing protocols designed for other ad hoc wireless networks. Any routing protocol designed for use in WSN should be reliable, energy-efficient and should increase the lifetime of the network. We propose a simple, least-time, energy-efficient routing protocol with one-level data aggregation that ensures increased life time for the network. The proposed protocol was compared with popular ad hoc and sensor network routing protocols, viz., AODV ( [35] and [12]), DSR (Johnson et al., 2001), DSDV (Perkins and Bhagwat, 1994), DD (Intanagonwiwat et al., 2000) and MCF (Ye et al., 2001). It was observed that the proposed protocol outperformed them in throughput, latency, average energy consumption and average network lifetime. The proposed protocol uses absolute time and node energy as the criteria for routing, this ensures reliability and congestion avoidance.     

Adaptive holistic scheduling for query processing in sensor networks

We observe two deficiencies of current query processing and scheduling techniques for sensor networks: (1) A query execution plan does not adapt to the hardware characteristics of sensing devices; and (2) the data communication schedule of each node is not adapted to the query runtime workload. Both cause time and energy waste in query processing in sensor networks. To address this problem, we propose an adaptive holistic scheduler, AHS, to run on each node in a wireless sensor network. AHS schedules both the query evaluation and the wireless communication operations, and is able to adapt the schedule to the runtime dynamics of these operations on each node. We have implemented AHS and tested it on real motes as well as in simulation. Our results show that AHS improves the performance of query processing in various dynamic settings.     

多源、多目标传感器数据处理方法探讨

多源、多目标传感器数据处理方法是当前研究多源、多目标微积分计算课题的一个重要的组成部分;有限集统计理论的核心是多源、多目标微积分计算.随着现代工业快速发展,对多传感器数据处理方法提出了更高的要求;怎样处理多源、多目标传感器数据是当前工程技术人员急需解决的问题.将单传感器有效组合进行拓展,以信任质量概念为基础,从数学形式上将多源、多目标的估计问题表现为单传感器、单目标问题;为解决多源多目标数据融合找到一种直观的处理方法.     

Accurate and energy-efficient boundary detection of continuous objects in duty-cycled wireless sensor networks

With the development and wide adoption of industrial wireless sensor networks to support various domain applications, the boundary detection of continuous objects has become an important research challenge, where improving the accuracy of boundary area while reducing the energy consumption are the first-class citizens to be considered. To address this research challenge, this article proposes a two-stage boundary face detection mechanism, where sensor nodes are duty-cycled and to be deployed in a dense fashion. When the occurrence of potential events are recognized using the initially activated sensor nodes, the boundary faces of continuous objects are constructed through adopting planarization algorithms. Thereafter, sensor nodes contained in certain boundary faces are examined, where their sensory data are estimated using spatial interpolation methods. Certain sensor nodes are selected to be woken up, only when their sensory data suggest that they should be more appropriate candidates of boundary sensor nodes. Consequently, the size of boundary faces is reduced, and this coarse-to-fine refinement procedure iterates, until all sensor nodes contained in the boundary faces have been examined. Experimental evaluation result shows that the boundary area can be refined significantly and be more precise, where the half of the initial boundary face area should be reduced in most situations.     

An energy-efficient,transport-controlled MAC protocol for wireless sensor networks

In wireless sensor networks (WSNs), one major cause of wasted energy is that the wireless network interface is always on to accept possible traffic. Many medium access control (MAC) protocols therefore adopted a periodic listen-and-sleep scheme to save energy, at sacrifice of end-to-end latency and throughput. Another cause is packet dropping due to network congestion, necessitating a lightweight transport protocol for WSNs. In this paper, we suggest a transport-controlled MAC protocol (TC-MAC) that combines the transport protocol into the MAC protocol with the aims of achieving high performance as well as energy efficiency in multi-hop forwarding. Although TC-MAC also works through a periodic listen-and-sleep scheme, it lowers end-to-end latency by reserving data forwarding schedules across multi-hop nodes during the listen period and by forwarding data during the sleep period, all while increasing throughput by piggybacking the subsequent data forwarding schedule on current data transmissions and forwarding data consecutively. In addition, TC-MAC gives a fairness-aware lightweight transport control mechanism based on benefits of using the MAC-layer information. The results show that TC-MAC performs as well as an 802.11-like MAC in end-to-end latency and throughput, and is more efficient than S-MAC in energy consumption, with the additional advantage of supporting fairness-aware congestion control.     

Application-driven,energy-efficient communication in wireless sensor networks

Several sensor network applications based on data diffusion and data management can determine the communication transfer rate between two sensors beforehand. In this framework, we consider the problem of energy efficient communication among nodes of a wireless sensor network and propose an application-driven approach that minimizes radio activity intervals and prolongs network lifetime. On the basis of possible communication delays we estimate packet arrival intervals at any intermediate hop of a fixed-rate data path. We study a generic strategy of radio activity minimization wherein each node maintains the radio switched on just in the expected packet arrival intervals and guarantees low communication latency. We define a probabilistic model that allows the evaluation of the packet loss probability that results from the reduced radio activity. The model can be used to optimally choose the radio activity intervals that achieve a certain probability of successful packet delivery for a specific radio activity strategy. Relying on the probabilistic model we also define a cost model that estimates the energy consumption of the proposed strategies, under specific settings. We propose three specific strategies and numerically evaluate the associated costs. We finally validate our work with a simulation made with TOSSIM (the Berkeley motes’ simulator). The simulation results confirm the validity of the approach and the accuracy of the analytic models.     

基于无线传感器网络的节能路由协议

针对LEACH协议应用在无线传感器网络当中存在的问题,提出了一种高效节能的LEACH方案,该方案能够提高能量利用率、延长网络生命周期.仿真结果表明,该方案是有效可行的.     

Reliable and energy-efficient data collection in sparse sensor networks with mobile elements

Sparse wireless sensor networks (WSNs) are emerging as an effective solution for a wide range of applications, especially for environmental monitoring. In many scenarios, a moderate number of sparsely deployed nodes can be sufficient to get the required information about the sensed phenomenon. To this end, special mobile elements, i.e. mobile data collectors (MDCs), can be used to get data sampled by sensor nodes. In this paper we present an analytical evaluation of the data collection performance in sparse WSNs with MDCs. Our main contribution is the definition of a flexible model which can derive the total energy consumption for each message correctly transferred by sensors to the MDC. The obtained energy expenditure for data transfer also accounts for the overhead due to the MDC detection when sensor nodes operate with a low duty cycle. The results show that a low duty cycle is convenient and allows a significant amount of correctly received messages, especially when the MDC moves with a low speed. When the MDC moves fast, depending on its mobility pattern, a low duty cycle may not always be the most energy efficient option.