Approximate dynamic programming for link scheduling in wireless mesh networks

In this paper a novel interference-based formulation and solution methodology for the problem of link scheduling in wireless mesh networks is proposed. Traditionally, this problem has been formulated as a deterministic integer program, which has been shown to be -hard. The proposed formulation is based on dynamic programming and allows greater flexibility since dynamic and stochastic components of the problem can be embedded into the optimization framework. By temporal decomposition we reduce the size of the integer program and using approximate dynamic programming (ADP) methods we tackle the curse of dimensionality. The numerical results reveal that the proposed algorithm outperforms well-known heuristics under different network topologies. Finally, the proposed ADP methodology can be used not only as an upper bound but also as a generic framework where different heuristics can be integrated.     

Distributed intelligence for multi-camera visual surveillance

Latest advances in hardware technology and state of the art of computer vision and artificial intelligence research can be employed to develop autonomous and distributed monitoring systems. The paper proposes a multi-agent architecture for the understanding of scene dynamics merging the information streamed by multiple cameras. A typical application would be the monitoring of a secure site, or any visual surveillance application deploying a network of cameras. Modular software (the agents) within such architecture controls the different components of the system and incrementally builds a model of the scene by merging the information gathered over extended periods of time. The role of distributed artificial intelligence composed of separate and autonomous modules is justified by the need for scalable designs capable of co-operating to infer an optimal interpretation of the scene. Decentralizing intelligence means creating more robust and reliable sources of interpretation, but also allows easy maintenance and updating of the system. Results are presented to support the choice of a distributed architecture, and to prove that scene interpretation can be incrementally and efficiently built by modular software.     

无线传感器网络动态重传算法

无线传感器网络路由协议通过点到点的重传来提高数据传输的可靠性,其重传机制没有考虑不同业务数据的可靠性需求差异,统一设定一个静态的最大重传次数。本文提出了一种动态重传算法,为每种业务分别根据其可靠性需求动态设定最大重传次数。对于较低可靠性需求的业务,相比于传统重传机制减少了重传次数。仿真表明动态重传算法能有效降低网络能耗。     

Rejection of non-meaningful activities for HMM-based activity recognition system

This paper presents a new test to distinguish between meaningful and non-meaningful HMM-modeled activity patterns in human activity recognition systems. Operating as a hypothesis test, alternative models are generated from available classes and the decision is based on a likelihood ratio test (LRT). The proposed test differs from traditional LRTs in two aspects. Firstly, the likelihood ratio, which is called pairwise likelihood ratio (PLR), is based on each pair of HMMs. Models for non-meaningful patterns are not required. Secondly, the distribution of the likelihood ratios, rather than a fixed threshold, is used as the measurement. Multiple measurements from multiple PLR tests are combined to improve the rejection accuracy. The advantage of the proposed test is that the establishment of such a test relies only on the meaningful samples.     

A framework for machine learning based on dynamic physical fields

Despite recent successes and advancements in artificial intelligence and machine learning, this domain remains under continuous challenge and guidance from phenomena and processes observed in natural world. Humans remain unsurpassed in their efficiency of dealing and learning from uncertain information coming in a variety of forms, whereas more and more robust learning and optimisation algorithms have their analytical engine built on the basis of some nature-inspired phenomena. Excellence of neural networks and kernel-based learning methods, an emergence of particle-, swarms-, and social behaviour-based optimisation methods are just few of many facts indicating a trend towards greater exploitation of nature inspired models and systems. This work intends to demonstrate how a simple concept of a physical field can be adopted to build a complete framework for supervised and unsupervised learning methodology. An inspiration for artificial learning has been found in the mechanics of physical fields found on both micro and macro scales. Exploiting the analogies between data and charged particles subjected to gravity, electrostatic and gas particle fields, a family of new algorithms has been developed and applied to classification, clustering and data condensation while properties of the field were further used in a unique visualisation of classification and classifier fusion models. The paper covers extensive pictorial examples and visual interpretations of the presented techniques along with some comparative testing over well-known real and artificial datasets.

Cost-sensitive learning with conditional Markov networks

There has been a recent, growing interest in classification and link prediction in structured domains. Methods such as conditional random fields and relational Markov networks support flexible mechanisms for modeling correlations due to the link structure. In addition, in many structured domains, there is an interesting structure in the risk or cost function associated with different misclassifications. There is a rich tradition of cost-sensitive learning applied to unstructured (IID) data. Here we propose a general framework which can capture correlations in the link structure and handle structured cost functions. We present two new cost-sensitive structured classifiers based on maximum entropy principles. The first determines the cost-sensitive classification by minimizing the expected cost of misclassification. The second directly determines the cost-sensitive classification without going through a probability estimation step. We contrast these approaches with an approach which employs a standard 0/1-loss structured classifier to estimate class conditional probabilities followed by minimization of the expected cost of misclassification and with a cost-sensitive IID classifier that does not utilize the correlations present in the link structure. We demonstrate the utility of our cost-sensitive structured classifiers with experiments on both synthetic and real-world data.     

An HMM-based call admission control policy for supporting QoS in wireless cellular networks

The multimedia services are getting to become the major trend in next-generation cellular networks. Call admission control (CAC) plays the key role for guaranteeing the quality of service (QoS) in cellular networks. The goal which keeps both the call dropping probability (CDP) and call blocking probability (CBP) below a certain level is more difficult owing to user indeterminate mobility. In this paper, the Hidden Markov Models (HMMs) concept which is suitable for solving a dynamic situation is introduced and applied to the call admission control policy. The prediction of user mobility can be modeled and resolved as the decoding problem of the HMMs. According to the prediction result, the proposed CAC method can reserve appropriate bandwidths for a handoff call beforehand. Thus, the call dropping probability can be kept below a lower level. Moreover, the call blocking probability is not sacrificed too much since the proposed method can reserve the suitable bandwidths in the appropriate cells but not reserve stationary ones which are always adopted by traditional CAC methods. Therefore, the proposed method not only can satisfy both CDP and CBP issues, but also improve the system utilization.     

A dynamic programming approach: Improving the performance of wireless networks

Traditional wireless networks focus on transparent data transmission where the data are processed at either the source or destination nodes. In contrast, the proposed approach aims at distributing data processing among the nodes in the network thus providing a higher processing capability than a single device. Moreover, energy consumption is balanced in the proposed scheme since the energy intensive processing will be distributed among the nodes. The performance of a wireless network is dependent on a number of factors including the available energy, energy–efficiency, data processing delay, transmission delay, routing decisions, security architecture etc. Typical existing distributed processing schemes have a fixed node or node type assigned to the processing at the design phase, for example a cluster head in wireless sensor networks aggregating the data. In contrast, the proposed approach aims to virtualize the processing, energy, and communication resources of the entire heterogeneous network and dynamically distribute processing steps along the communication path while optimizing performance. Moreover, the security of the communication is considered an important factor in the decision to either process or forward the data. Overall, the proposed scheme creates a wireless “computing cloud” where the processing tasks are dynamically assigned to the nodes using the Dynamic Programming (DP) methodology. The processing and transmission decisions are analytically derived from network models in order to optimize the utilization of the network resources including: available energy, processing capacity, security overhead, bandwidth etc. The proposed DP-based scheme is mathematically derived thus guaranteeing performance. Moreover, the scheme is verified through network simulations.     

Reinforcement learning of dynamic behavior by using recurrent neural networks

Reinforcement learning is a learning scheme for finding the optimal policy to control a system, based on a scalar signal representing a reward or a punishment. If the observation of the system by the controller is sufficiently rich to represent the internal state of the system, the controller can achieve the optimal policy simply by learning reactive behavior. However, if the state of the controlled system cannot be assessed completely using current sensory observations, the controller must learn a dynamic behavior to achieve the optimal policy. In this paper, we propose a dynamic controller scheme which utilizes memory to uncover hidden states by using information about past system outputs, and makes control decisions using memory. This scheme integrates Q-learning, as proposed by Watkins, and recurrent neural networks of several types. It performs favorably in simulations which involve a task with hidden states. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

A novel hybrid learning algorithm for full Bayesian approach of artificial neural networks

The Bayesian neural networks are useful tools to estimate the functional structure in the nonlinear systems. However, they suffer from some complicated problems such as controlling the model complexity, the training time, the efficient parameter estimation, the random walk, and the stuck in the local optima in the high-dimensional parameter cases. In this paper, to alleviate these mentioned problems, a novel hybrid Bayesian learning procedure is proposed. This approach is based on the full Bayesian learning, and integrates Markov chain Monte Carlo procedures with genetic algorithms and the fuzzy membership functions. In the application sections, to examine the performance of proposed approach, nonlinear time series and regression analysis are handled separately, and it is compared with the traditional training techniques in terms of their estimation and prediction abilities.     

Reinforcement learning for context awareness and intelligence in wireless networks: Review, new features and open issues

In wireless networks, context awareness and intelligence are capabilities that enable each host to observe, learn, and respond to its complex and dynamic operating environment in an efficient manner. These capabilities contrast with traditional approaches where each host adheres to a predefined set of rules, and responds accordingly. In recent years, context awareness and intelligence have gained tremendous popularity due to the substantial network-wide performance enhancement they have to offer. In this article, we advocate the use of reinforcement learning (RL) to achieve context awareness and intelligence. The RL approach has been applied in a variety of schemes such as routing, resource management and dynamic channel selection in wireless networks. Examples of wireless networks are mobile ad hoc networks, wireless sensor networks, cellular networks and cognitive radio networks. This article presents an overview of classical RL and three extensions, including events, rules and agent interaction and coordination, to wireless networks. We discuss how several wireless network schemes have been approached using RL to provide network performance enhancement, and also open issues associated with this approach. Throughout the paper, discussions are presented in a tutorial manner, and are related to existing work in order to establish a foundation for further research in this field, specifically, for the improvement of the RL approach in the context of wireless networking, for the improvement of the RL approach through the use of the extensions in existing schemes, as well as for the design and implementation of RL in new schemes.     

An improved method for voice pathology detection by means of a HMM-based feature space transformation

This paper presents new a feature transformation technique applied to improve the screening accuracy for the automatic detection of pathological voices. The statistical transformation is based on Hidden Markov Models, obtaining a transformation and classification stage simultaneously and adjusting the parameters of the model with a criterion that minimizes the classification error. The original feature vectors are built up using classic short-term noise parameters and mel-frequency cepstral coefficients. With respect to conventional approaches found in the literature of automatic detection of pathological voices, the proposed feature space transformation technique demonstrates a significant improvement of the performance with no addition of new features to the original input space. In view of the results, it is expected that this technique could provide good results in other areas such as speaker verification and/or identification.     

Clustering the wireless Ad Hoc networks: A distributed learning automata approach

In Ad Hoc networks, the performance is significantly degraded as the size of the network grows. The network clustering by which the nodes are hierarchically organized on the basis of the proximity relieves this performance degradation. Finding the weakly connected dominating set (WCDS) is a promising approach for clustering the wireless Ad Hoc networks. Finding the minimum WCDS in the unit disk graph is an NP-Hard problem, and a host of approximation algorithms has been proposed. In this article, we first proposed a centralized approximation algorithm called DLA-CC based on distributed learning automata (DLA) for finding a near optimal solution to the minimum WCDS problem. Then, we propose a DLA-based clustering algorithm called DLA-DC for clustering the wireless Ad Hoc networks. The proposed cluster formation algorithm is a distributed implementation of DLA-CC, in which the dominator nodes and their closed neighbors assume the role of the cluster-heads and cluster members, respectively. In this article, we compute the worst case running time and message complexity of the clustering algorithm for finding a near optimal cluster-head set. We argue that by a proper choice of the learning rate of the clustering algorithm, a trade-off between the running time and message complexity of algorithm with the cluster-head set size (clustering optimality) can be made. The simulation results show the superiority of the proposed algorithms over the existing methods.     

面向移动汇聚节点的无线传感器网络自适应路由方法

针对无线传感器网络面向移动汇聚节点的自适应路由问题,为实现路由过程中对节点能量以及计算、存储、通信资源的优化利用,并对数据传输时延和投递率等服务质量进行优化,提出一种基于强化学习的自适应路由方法,设计综合的奖赏函数以实现对能量、时延和投递率等多个指标的综合优化。从报文结构、路由初始化、路径选择等方面对路由协议进行详细设计,采用汇聚节点声明以及周期性洪泛机制加速收敛速度,从而支持汇聚节点的快速移动。理论分析表明基于强化学习的路由方法具备收敛快、协议开销低以及存储计算需求小等特点,能够适用于能量和资源受限的传感器节点。在仿真平台中通过性能评估和对比分析验证了所述自适应路由算法的可行性和优越性。     

Minimum classification error learning for sequential data in the wavelet domain

Wavelet analysis has found widespread use in signal processing and many classification tasks. Nevertheless, its use in dynamic pattern recognition have been much more restricted since most of wavelet models cannot handle variable length sequences properly. Recently, composite hidden Markov models which observe structured data in the wavelet domain were proposed to deal with this kind of sequences. In these models, hidden Markov trees account for local dynamics in a multiresolution framework, while standard hidden Markov models capture longer correlations in time. Despite these models have shown promising results in simple applications, only generative approaches have been used so far for parameter estimation. The goal of this work is to take a step forward in the development of dynamic pattern recognizers using wavelet features by introducing a new discriminative training method for this Markov models. The learning strategy relies on the minimum classification error approach and provides re-estimation formulas for fully non-tied models. Numerical experiments on phoneme recognition show important improvement over the recognition rate achieved by the same models trained using maximum likelihood estimation.     

Machine annotation and retrieval for digital imagery of historical materials

Annotating digital imagery of historical materials for the purpose of computer-based retrieval is a labor-intensive task for many historians and digital collection managers. We have explored the possibilities of automated annotation and retrieval of images from collections of art and cultural images. In this paper, we introduce the application of the ALIP (Automatic Linguistic Indexing of Pictures) system, developed at Penn State, to the problem of machine-assisted annotation of images of historical materials. The ALIP system learns the expertise of a human annotator on the basis of a small collection of annotated representative images. The learned knowledge about the domain-specific concepts is stored as a dictionary of statistical models in a computer-based knowledge base. When an un-annotated image is presented to ALIP, the system computes the statistical likelihood of the image resembling each of the learned statistical models and the best concept is selected to annotate the image. Experimental results, obtained using the Emperor image collection of the Chinese Memory Net project, are reported and discussed. The system has been trained using subsets of images and metadata from the Emperor collection. Finally, we introduce an integration of wavelet-based annotation and wavelet-based progressive displaying of very high resolution copyright-protected images. A preliminary version of this work has been presented at the DELOS-NSF Workshop on Multimedia in Digital Libraries, Crete, Greece, June 2003. The work was completed when Kurt Grieb and Ya Zhang were students of The Pennsylvania State University. James Z. Wang and Jia Li are also affiliated with Department of Computer Science and Engineering, The Pennsylvania State University. Yixin Chen is also with the Research Institute for Children, Children's Hospital, New Orleans.     

NeuralSens: A neural network based framework to allow dynamic adaptation in wireless sensor and actor networks

Wireless Sensor and Actor Networks (WSANs) constitute a new way of distributed computing and are steadily gaining importance due to the wide variety of applications that can be implemented with them. As a result they are increasingly present everywhere (industry, farm use, buildings, etc.). However, there are still many important areas in which the WSANs can be improved. One of the most important aspects is to give the sensor networks the capability of being wirelessly reprogrammed so that developers do not have to physically interact with the sensor nodes. Many proposals that deal with this issue have been proposed, but most of them are hardly dependent on the operating system and demand a high energy consumption, even if only a small change has been made in the code. In this work, we propose a new way of wirelessly reprogramming based on the concept of neural networks. Unlike most of the existing approaches, our proposal is independent of the operating system and allows small pieces of code to be reprogrammed with a low energy consumption. The architecture developed to achieve that is described and case studies are presented that show the use of our proposal by means of practical examples.     

Reinforcement learning for autonomous vehicle movements in wireless multimedia applications

We develop a Deep Reinforcement Learning (DeepRL)-based, multi-agent algorithm to efficiently control autonomous vehicles that are typically used within the context of Wireless Sensor Networks (WSNs), in order to boost application performance. As an application example, we consider wireless acoustic sensor networks where a group of speakers move inside a room. In a traditional setup, microphones cannot move autonomously and are, e.g., located at fixed positions. We claim that autonomously moving microphones improve the application performance. To control these movements, we compare simple greedy heuristics against a DeepRL solution and show that the latter achieves best application performance.As the range of audio applications is broad and each has its own (subjective) performance metric, we replace those application metrics by two immediately observable ones: First, quality of information (QoI), which is used to measure the quality of sensed data (e.g., audio signal strength). Second, quality of service (QoS), which is used to measure the network’s performance when forwarding data (e.g., delay). In this context, we propose two multi-agent solutions (where one agent controls one microphone) and show that they perform similarly to a single-agent solution (where one agent controls all microphones and has a global knowledge). Moreover, we show via simulations and theoretical analysis how other parameters such as the number of microphones and their speed impacts performance.     

A framework based on barycentric coordinates for localization in wireless sensor networks

Localization is a fundamental and vital problem in wireless sensor networks. This paper presents an optimizing framework for localization based on barycentric coordinates. The framework consists of two components. The first component retains the structure revealed by the distances between pairs of nodes; the second component constrains the boundary nodes to maintain the distance with their neighbor nodes. A hybrid localization algorithm is derived on top of the optimizing framework. A part of the computation is performed collaboratively by nodes, whereas the rest is executed on the sink node. Experimental results show that the proposed localization algorithm obtains lower location errors without higher communication costs.