Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

Line patterning of graphite and the fabrication of cheap, inexpensive, “throw-away” sensors

A new simple method (“line patterning technique”) using only standard office equipment is described whereby clearly defined, electrically conducting patterns of graphite can be deposited on polymer (plastic) or paper substrates. The properties of the conductive patterns have been characterized by electrical conductivity and SEM measurements. Sensors were constructed by using interdigitated patterns of graphite deposited on plastic and paper, and coated with a thin film of conducting electronic polymer, e.g. polyaniline emeraldine salt.     

Advanced product planning: a comprehensive process for systemic definition of new product requirements

This paper reports results of research into the definition of requirements for new consumer products––specifically, electro-mechanical products. The research dealt with the derivation of design requirements that are demonstrably aligned with stakeholder needs. The paper describes a comprehensive process that can enable product development teams to deal with statements of product requirements, as originally collected through market research activities, in a systematic and traceable manner from the early, fuzzy front end, stages of the design process. The process described has been based on principles of systems engineering. A case study from its application and evaluation drawn from the power sector is described in this paper. The case study demonstrates how the process can significantly improve product quality planning practices through revision of captured product requirements, analysis of stakeholder requirements and derivation of design requirements. The paper discusses benefits and issues from the use of the process by product development teams, and identifies areas for further research. Finally, the conclusions drawn from the reported research are presented.

通过在线客户体验来推动销售

21世纪的消费者在消费体验方面更有自己的主张,直复营销也有了发展。尤其是那些已推出在线业务的企业,已有能力与个人进行互动。     

Integrating scientific modeling and supporting dynamic hazard management with a GeoAgent-based representation of human–environment interactions: A drought example in Central Pennsylvania, USA

Recent natural disasters indicate that modern technologies for environmental monitoring, modeling, and forecasting are not well integrated with cross-level social responses in many hazard-management systems. This research addresses this problem through a Java-based multi-agent prototype system, GeoAgent-based Knowledge System (GeoAgentKS). This system allows: (1) computer representation of institutional regulations and behavioral rules used by multiple social institutions and individuals in cross-level human–environment interactions, (2) integration of this representation with scientific modeling of dynamic hazard development, and (3) application of automated reasoning that suggests to users the appropriate actions for supporting cooperative social responses. This paper demonstrates the software architecture of GeoAgentKS and presents such an integrated approach by modeling the drought management processes in Central Pennsylvania, USA. The results show that it is possible to use GeoAgentKS to represent multilevel human–environment interactions and to use those interactions as input to decision making in hazard management.     

Business process management with the user requirements notation

A number of recent initiatives in both academia and industry have sought to achieve improvements in e-businesses through the utilization of Business Process Management (BPM) methodologies and tools. However there are still some inadequacies that need to be addressed when it comes to achieving alignment between business goals and business processes. The User Requirements Notation (URN), recently standardized by ITU-T, has some unique features and capabilities beyond what is available in other notations that can help address alignment issues. In this paper, a URN-based framework and its supporting toolset are introduced which provide business process monitoring and performance management capabilities integrated across the BPM lifecycle. The framework extends the URN notation with Key Performance Indicators (KPIs) and other concepts to measure and align processes and goals. An example process for controlling access to a healthcare data warehouse is used to illustrate and evaluate the framework. Early results indicate the feasibility of the approach.     

A taxonomy for and analysis of multi-person-display ecosystems

Interactive displays are increasingly being distributed in a broad spectrum of everyday life environments: they have very diverse form factors and portability characteristics, support a variety of interaction techniques, and can be used by a variable number of people. The coupling of multiple displays creates an interactive “ecosystem of displays”. Such an ecosystem is suitable for particular social contexts, which in turn generates novel settings for communication and performance and challenges in ownership. This paper aims at providing a design space that can inform the designers of such ecosystems. To this end, we provide a taxonomy that builds on the size of the ecosystem and on the degree of individual engagement as dimensions. We recognize areas where physical constraints imply certain kinds of social engagement, versus other areas where further work on interaction techniques for coupling displays can open new design spaces.     

Scheduling algorithms for fault-tolerance in hard-real-time systems

Many time-critical applications require predictable performance in the presence of failures. This paper considers a distributed system with independent periodic tasks which can checkpoint their state on some reliable medium in order to handle failures. The problem of preemptively scheduling a set of such tasks is discussed where every occurrence of a task has to be completely executed before the next occurrence of the same task can start. Efficient scheduling algorithms are proposed which yield sub-optimal schedules when there is provision for fault-tolerance. The performance of the solutions proposed is evaluated in terms of the number of processors and the cost of the checkpoints needed. Moreover, analytical studies are used to reveal interesting trade-offs associated with the scheduling algorithms.This work has been supported by grants from the Italian Ministero dell'Università e della Ricerca Scientifica e Tecnologica and the Consiglio Nazionale delle Ricerche-Progetto Finalizzato Sistemi Informatici e Calcolo Parallelo.     

Evolution of the cranial computed tomography scan in child abuse

Computed tomography (CT) scans obtained at the time of clinical presentation have occasionally been reported to be normal in children with history and findings of significant abusive head injury. We have retrospectively observed abnormalities in "normal" scans of some similar children. We have also seen abnormalities develop on serial scanning. To determine how frequently these situations occur, we reviewed charts of 34 children with a final diagnosis of child abuse who also had cranial CT scans performed. Their CT scans were retrospectively reviewed by a pediatric radiologist. Eleven (11/34) CT scans had initially been interpreted as normal. Four (4/11) of these had been reinterpreted during the hospitalization as abnormal, affecting medical (1) and legal (3) outcome. Repeat scanning in three of the remaining seven resulted in surgical drainage of a subdural effusion (1) and affected legal outcome (2). Four of the seven initial scans felt normal throughout the hospitalizations were judged abnormal on retrospective review. This evaluation was confirmed in the two rescanned. Initial CT interpretation most often failed to appreciate changes in parenchymal density and small amounts of falcine or cortical subdural blood. Subsequent scans also showed evolving effusions and infarcts. Changes were noted in 1 1/2 to 5 days. The CT scan frequently shows subtle changes in the immediate posttrauma period. If the child does not recover promptly, subsequent scans frequently result in significant changes in clinical and legal management.     

Occluded Video Instance Segmentation: A Benchmark

International Journal of Computer Vision - Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset...