Rollout sampling approximate policy iteration

Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions, which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.     

Design of output constraints for model-based non-square controllers using interval operability

A steady-state interval operability methodology is introduced here for multivariable non-square systems with fewer inputs than output variables to be used in the design of model-based constrained controllers (MPC, DMC). For such systems, set-point control is not possible for all the outputs and interval control is needed. The proposed iterative approach enables the selection of the needed interval constraints systematically, so that the tightest possible control is achieved without rendering the control problem infeasible. The application of this methodology to high-dimensional industrial problems characterizing processes of Air Products and Chemicals and DuPont shows that very significant reduction of the constrained region can be achieved from the steady-state point of view. Ratios of the initial to the calculated volume of the constrained regions examined range between 10⁴ and 10⁸.     

Ontology-based modeling of dynamic ubiquitous computing applications as evolving activity spheres

As the computer disappears in the environments surrounding our activities, the objects therein become augmented with sensors, actuators, processors, memories, wireless communication modules and they can receive, store, process and transmit information. In addition to objects, spaces also undergo a change towards becoming smart and eventually Ambient Intelligence (AmI) spaces. In order to model the way everyday activities are carried out within an AmI environment, we introduce the notion of “activity sphere”. In this paper, we are interested in the ontology-based representation of activity spheres from two different perspectives (as creators and as observers), as well as the modeling and control of the dynamic nature of activity spheres.     

Infinitesimal perturbation analysis of a queueing system with bursty traffic

We study aG/G/1 queueing system with a bursty arrival process. Based on a general model for such a bursty process, we derive infinitesimal perturbation analysis (IPA) derivative estimators of the mean system time with respect to various parameters of interest. The cases of both complete and partial state information are considered. To ensure unbiasedness and strong consistency of the estimators, different sample path representations are developed such that sample functions are continuous with respect to the particular parameter of interest. Some of these representations are applicable to a wider class of gradient estimation problems where sample path discontinuities arise. Simulation results are included to compare the convergence rates and variance properties of the different IPA estimators developed.     

A soft computing approach for privacy requirements engineering: The PriS framework

Soft computing continuously gains interest in many fields of academic and industrial domain; among the most notable characteristics for using soft computing methodological tools is the ability to handle with vague and imprecise data in decision making processes. Similar conditions are often encountered in requirements engineering. In this paper, we introduce the PriS approach, a security and privacy requirements engineering framework which aims at incorporating privacy requirements early in the system development process. Specifically, PriS provides a set of concepts for modelling privacy requirements in the organisation domain and a systematic way-of-working for translating these requirements into system models. The conceptual model of PriS uses a goal hierarchy structure. Every privacy requirement is either applied or not on every goal. To this end every privacy requirement is a variable that can take two values [0,1] on every goal meaning that the requirements constraints the goal (value 1) or not (value 0). Following this way of working PriS ends up suggesting a number of implementation techniques based on the privacy requirements constraining the respective goals. Taking into account that the mapping of privacy variables to a crisp set consisting of two values [0,1] is constraining, we extend also the PriS framework so as to be able to address the degree of participation of every privacy requirement towards achieving the generic goal of privacy. Therefore, we propose a fuzzification of privacy variables that maps the expression of the degree of participation of each privacy variable to the [0,1] interval. We also present a mathematical framework that allows the concurrent management of combined independent preferences towards the necessity of a privacy measure; among the advantages of the presented extended framework is the scalability of the approach in such a way that the results are not limited by the number of independent opinions or by the number of factors considered while reasoning for a specific selection of privacy measures.     

Adaptive Smooth Simulcast Protocol (ASSP) for Video Applications: Description and Performance Evaluation

In this paper, we present Adaptive Smooth Simulcast Protocol (ASSP) for simulcast transmission of multimedia data over best-effort networks. ASSP is a new multiple-rate protocol that implements a single rate TCP-friendly protocol as the underlying congestion control mechanism for each simulcast stream. The key attributes of ASSP are: (a) TCP-friendly behavior, (b) adaptive per-stream transmission rates, (c) adaptive scalability to large sets of receivers and (d) smooth transmission rates that are suitable for multimedia applications. We evaluate the performance of ASSP under an integrated simulation environment which combines the measurements of both network and video performance metrics. We also compare ASSP against other proposed solutions and the results demonstrate that the performance of ASSP is significantly better than the tested solutions. Finally, ASSP is a practical solution with very low implementation complexity for video transmission over best-effort networks.     

A repairable queueing model with two-phase service,start-up times and retrial customers

A repairable queueing model with a two-phase service in succession, provided by a single server, is investigated. Customers arrive in a single ordinary queue and after the completion of the first phase service, either proceed to the second phase or join a retrial box from where they retry, after a random amount of time and independently of the other customers in orbit, to find a position for service in the second phase. Moreover, the server is subject to breakdowns and repairs in both phases, while a start-up time is needed in order to start serving a retrial customer. When the server, upon a service or a repair completion finds no customers waiting to be served, he departs for a single vacation of an arbitrarily distributed length. The arrival process is assumed to be Poisson and all service and repair times are arbitrarily distributed. For such a system the stability conditions and steady state analysis are investigated. Numerical results are finally obtained and used to investigate system performance.     

On-line Optimal Control of a Class of Discrete Event Systems with Real-Time Constraints

We consider Discrete Event Systems (DES) involving tasks with real-time constraints and seek to control processing times so as to minimize a cost function subject to each task meeting its own constraint. It has been shown that the off-line version of this problem can be efficiently solved by the Critical Task Decomposition Algorithm (CTDA) (Mao et al., IEEE Trans Mobile Comput 6(6):678–688, 2007). In the on-line version, random task characteristics (e.g., arrival times) are not known in advance. To bypass this difficulty, worst-case analysis may be used. This, however, does not make use of probability distributions and results in an overly conservative solution. In this paper, we develop a new approach which does not rely on worst-case analysis but provides a “best solution in probability” efficiently obtained by estimating the probability distribution of sample-path-optimal solutions. We introduce a condition termed “non-singularity” under which the best solution in probability leads to the on-line optimal control. Numerical examples are included to illustrate our results and show substantial performance improvements over worst-case analysis.     

Providing advanced remote medical treatment services through pervasive environments

Advances on sensor technology, wireless environments and data mining introduce new possibilities in the healthcare sector, realizing the anytime-anywhere access to medical information. Towards this direction, integration of packet-switched networks and sensor devices can be effective in deploying assistive environments, such as home monitoring for elderly or patients. In this paper we describe a policy-based architecture that utilizes wireless sensor devices, advanced network topologies and software agents to enable remote monitoring of patients and elderly people; through the aforementioned technologies we achieve continuous monitoring of a patient’s condition and we can proceed when necessary with proper actions. We also present a software framework and network architecture that realizes the provision of remote medical services, in compliance with the imposed security and privacy requirements. A proof of concept prototype is also deployed, along with an evaluation of the overall architecture’s performance.     

An analytical model for multi-epidemic information dissemination

Contemporary distributed systems usually involve the spreading of information by means of ad-hoc dialogs between nodes (peers). This paradigm resembles the spreading of a virus in the biological perspective (epidemics). Such abstraction allows us to design and implement information dissemination schemes with increased efficiency. In addition, elementary information generated at a certain node can be further processed to obtain more specific, higher-level and more valuable information. Such information carries specific semantic value that can be further interpreted and exploited throughout the network. This is also reflected in the epidemical framework through the idea of virus transmutation which is a key component in our model. We establish an analytical framework for the study of a multi-epidemical information dissemination scheme in which diverse ‘transmuted epidemics’ are spread. We validate our analytical model through simulations. Key outcomes of this study include the assessment of the efficiency of the proposed scheme and the prediction of the characteristics of the spreading process (multi-epidemical prevalence and decay).