Group online adaptive learning

Sharing information among multiple learning agents can accelerate learning. It could be particularly useful if learners operate in continuously changing environments, because a learner could benefit from previous experience of another learner to adapt to their new environment. Such group-adaptive learning has numerous applications, from predicting financial time-series, through content recommendation systems, to visual understanding for adaptive autonomous agents. Here we address the problem in the context of online adaptive learning. We formally define the learning settings of Group Online Adaptive Learning and derive an algorithm named Shared Online Adaptive Learning (SOAL) to address it. SOAL avoids explicitly modeling changes or their dynamics, and instead shares information continuously. The key idea is that learners share a common small pool of experts, which they can use in a weighted adaptive way. We define group adaptive regret and prove that SOAL maintains known bounds on the adaptive regret obtained for single adaptive learners. Furthermore, it quickly adapts when learning tasks are related to each other. We demonstrate the benefits of the approach for two domains: vision and text. First, in the visual domain, we study a visual navigation task where a robot learns to navigate based on outdoor video scenes. We show how navigation can improve when knowledge from other robots in related scenes is available. Second, in the text domain, we create a new dataset for the task of assigning submitted papers to relevant editors. This is, inherently, an adaptive learning task due to the dynamic nature of research fields evolving in time. We show how learning to assign editors improves when knowledge from other editors is available. Together, these results demonstrate the benefits for sharing information across learners in concurrently changing environments.

     

Congregation Formation in Multiagent Systems

We present congregating both as a metaphor for describing and modeling multiagent systems (MAS) and as a means for reducing coordination costs in large-scale MAS. When agents must search for other agents to interact with, congregations provide a way for agents to bias this search towards groups of agents that have tended to produce successful interactions in the past. This causes each agent's search problem to scale with the size of a congregation rather than the size of the population as a whole. In this paper, we present a formal model of a congregation and then apply Vidal and Durfee's CLRI framework [24] to the congregating problem. We apply congregating to the affinity group domain, and show that if agents are unable to describe congregations to each other, the problem of forming optimal congregations grows exponentially with the number of agents. The introduction of labelers provides a means of coordinating agent decisions, thereby reducing the problem's complexity. We then show how a structured label space can be exploited to simplify the labeler's decision problem and make the congregating problem linear in the number of labels. We then present experimental evidence demonstrating how congregating can be used to reduce agents' search costs, thereby allowing the system to scale up. We conclude with a comparison to other methods for coordinating multiagent behavior, particularly teams and coalitions.     

无人机群目标搜索的主动感知方法

为提升蚁群搜索算法在规模大的栅格环境中对未知目标的搜索效率,提出基于蚁群算法的主动感知搜索框架。该框架通过应用历史环境信息来选择无人机的运动方式,并由无人机运动方式和感知域信息得到新的环境信息,从而实现无人机群的智能自动化搜索功能。新方法计算出一种具有探索偏好的未搜索概率,可使无人机搜索时偏向未搜索程度高的栅格,以此来提高算法的搜索能力。同时,以未搜索概率和信息素作为运动方式决策的依据来建立一种新的运动方式选择机制。该机制不仅考虑了目标可能出现的区域,又可兼顾未知区域,从而可实现无目标先验信息条件下的搜索过程。仿真结果表明,此算法在规模大的栅格环境中,与现有算法相比具有更高的搜索效率,并且得到的目标分布信息将更加全面。     

Representation of time in digital calendars: An argument for a unified,continuous and multi-granular calendar view

Digital calendars have been heavily influenced by the design of the physical calendar and have invariably adopted their grid representation of days in the month. We argue that the alternative of a continuous list representation of successive days would offer several advantages such as faster calendar search, a more natural linear view of time, a scrollable and zoomable interface and better scalability for devices of different size. This alternative, linear calendar appears to be well suited to modern touch-centric platforms with their refined support for scrolling and zooming.We tested search performance and navigation with digital calendars in a comparison of grid and list representations by employing a remote, web-based method. On their personal computers, participants performed a series of search tasks in a fictitious calendar. The results show that calendar search is faster in list view when searching for dates, between month breaks and in the next month (with and without navigation). Searching for days is faster in grid view, however, highlighting days in list view eliminates this difference. The results indicate substantial promise for the list view digital calendar and we describe a high fidelity rendering of the user interface for a digital calendar with a list view.     

场景感知的分布式多智能体目标搜索方法

在视觉语义导航任务中,智能体通过视觉信息,寻找并导航到给定对象类别的目标处。然而,大部分现有的研究都是使用基于学习的框架来完成任务,这些研究在现实世界中应用的训练成本非常高,可移植性很低,并且它们只适用于单智能体,效率低下、容错能力差。为解决上述问题,本文提出一种基于场景感知的分布式多目标优化蒙特卡洛树搜索模型,该模型中多智能体实时在线规划并且不需要预先训练,利用场景感知先验知识结合观测信息实时对环境进行估计,并且利用改进的蒙特卡洛树搜索进行路径规划以此搜索目标。在Mtterport3D数据集中进行的实验表明,该模型在效率方面比单智能体有着显著的提高。     

PROBABILISTIC MODELS FOR FOCUSED WEB CRAWLING

A focused crawler is an efficient tool used to traverse the Web to gather documents on a specific topic. It can be used to build domain‐specific Web search portals and online personalized search tools. Focused crawlers can only use information obtained from previously crawled pages to estimate the relevance of a newly seen URL. Therefore, good performance depends on powerful modeling of context as well as the quality of the current observations. To address this challenge, we propose capturing sequential patterns along paths leading to targets based on probabilistic models. We model the process of crawling by a walk along an underlying chain of hidden states, defined by hop distance from target pages, from which the actual topics of the documents are observed. When a new document is seen, prediction amounts to estimating the distance of this document from a target. Within this framework, we propose two probabilistic models for focused crawling, Maximum Entropy Markov Model (MEMM) and Linear‐chain Conditional Random Field (CRF). With MEMM, we exploit multiple overlapping features, such as anchor text, to represent useful context and form a chain of local classifier models. With CRF, a form of undirected graphical models, we focus on obtaining global optimal solutions along the sequences by taking advantage not only of text content, but also of linkage relations. We conclude with an experimental validation and comparison with focused crawling based on Best‐First Search (BFS), Hidden Markov Model (HMM), and Context‐graph Search (CGS).     

Mobile robot navigation using grid line patterns via probabilistic measurement modeling

Mobile robots are generally equipped with proprioceptive motion sensors such as odometers and inertial sensors. These sensors are used for dead-reckoning navigation in an indoor environment where GPS is not available. However, this dead-reckoning scheme is susceptible to drift error in position and heading. This study proposes using grid line patterns which are often found on the surface of floors or ceilings in an indoor environment to obtain pose (i.e., position and orientation) fix information without additional external position information by artificial beacons or landmarks. The grid lines can provide relative pose information of a robot with respect to the grid structure and thus can be used to correct the pose estimation errors. However, grid line patterns are repetitive in nature, which leads to difficulties in estimating its configuration and structure using conventional Gaussian filtering that represent the system uncertainty using a unimodal function (e.g., Kalman filter). In this study, a probabilistic sensor model to deal with multiple hypotheses is employed and an online navigation filter is designed in the framework of particle filtering. To demonstrate the performance of the proposed approach, an experiment was performed in an indoor environment using a wheeled mobile robot, and the results are presented.     

一种基于移动Agent的网格资源发现方法

提出了一种基于移动Agent的非集中式网格资源发现方法,通过Agent共享服务请求环境中重叠部分的信息,减少整个信息索存储空间的数量,从而提高了查找效率。Agent并行处理查找任务,可扩展性提高了。同时提出了一种基于CE算法的资源查找算法ApRD,该算法是完全分布式的,由移动Agent来执行。实验结果表明该方法可获得较满意的资源定位性能,能适应网格资源的分布性、动态参与和良好的可扩展性。     

Cellular matrix model for parallel combinatorial optimization algorithms in Euclidean plane

We propose a parallel computation model, called cellular matrix model (CMM), to address large-size Euclidean graph matching problems in the plane. The parallel computation takes place by partitioning the plane into a regular grid of cells, each cell being affected to a single processor. Each processor operates on local data, starting from its cell location and extending its search to the neighborhood cells in a spiral search way. In order to deal with large-size problems, memory size and processor number are fixed as O(N), where N is the problem size. Then one key point is that closest point searching in the plane is performed in O(1) expected time for uniform or bounded distribution, for each processor independently. We define a generic loop that models the parallel projection between graphs and their matching, as executed by the many cells at a given level of computation granularity. To illustrate its efficacy and versatility, we apply the CMM, on GPU platforms, to two problems in image processing: superpixel segmentation and stereo matching energy minimization. Firstly, we propose an extended version of the well-known SLIC superpixel segmentation algorithm, which we call SPASM algorithm, by using a parallel 2D self-organizing map instead of k-means algorithm. Secondly, we investigate the idea of distributed variable neighborhood search, and propose a parallel search heuristic, called distributed local search (DLS), for global energy minimization of stereo matching problem. We evaluate the approach with regards to the state-of-the-art graph cut and belief propagation algorithms. For each problem, we argue that the parallel GPU implementation provides new competitive quality/time trade-offs, with substantial acceleration factors as the problem size increases.     

Domain Adaptation for Structured Regression

Discriminative regression models have proved effective for many vision applications (here we focus on 3D full-body and head pose estimation from image and depth data). However, dataset bias is common and is able to significantly degrade the performance of a trained model on target test sets. As we show, covariate shift, a form of unsupervised domain adaptation (USDA), can be used to address certain biases in this setting, but is unable to deal with more severe structural biases in the data. We propose an effective and efficient semi-supervised domain adaptation (SSDA) approach for addressing such more severe biases in the data. Proposed SSDA is a generalization of USDA, that is able to effectively leverage labeled data in the target domain when available. Our method amounts to projecting input features into a higher dimensional space (by construction well suited for domain adaptation) and estimating weights for the training samples based on the ratio of test and train marginals in that space. The resulting augmented weighted samples can then be used to learn a model of choice, alleviating the problems of bias in the data; as an example, we introduce SSDA twin Gaussian process regression (SSDA-TGP) model. With this model we also address the issue of data sharing, where we are able to leverage samples from certain activities (e.g., walking, jogging) to improve predictive performance on very different activities (e.g., boxing). In addition, we analyze the relationship between domain similarity and effectiveness of proposed USDA versus SSDA methods. Moreover, we propose a computationally efficient alternative to TGP (Bo and Sminchisescu 2010), and it’s variants, called the direct TGP. We show that our model outperforms a number of baselines, on two public datasets: HumanEva and ETH Face Pose Range Image Dataset. We can also achieve 8–15 times speedup in computation time, over the traditional formulation of TGP, using the proposed direct formulation, with little to no loss in performance.     

Planning approaches to constraint‐aware navigation in dynamic environments

Path planning is a fundamental problem in many areas, ranging from robotics and artificial intelligence to computer graphics and animation. Although there is extensive literature for computing optimal, collision‐free paths, there is relatively little work that explores the satisfaction of spatial constraints between objects and agents at the global navigation layer. This paper presents a planning framework that satisfies multiple spatial constraints imposed on the path. The type of constraints specified can include staying behind a building, walking along walls, or avoiding the line of sight of patrolling agents. We introduce two hybrid environment representations that balance computational efficiency and search space density to provide a minimal, yet sufficient, discretization of the search graph for constraint‐aware navigation. An extended anytime dynamic planner is used to compute constraint‐aware paths, while efficiently repairing solutions to account for varying dynamic constraints or an updating world model. We demonstrate the benefits of our method on challenging navigation problems in complex environments for dynamic agents using combinations of hard and soft, attracting and repelling constraints, defined by both static obstacles and moving obstacles. Copyright © 2014 John Wiley & Sons, Ltd.     

激光—相机系统语义栅格建图和路径规划

目的 SLAM（simultaneous localization and mapping）是移动机器人在未知环境进行探索、感知和导航的关键技术。激光SLAM测量精确，便于机器人导航和路径规划，但缺乏语义信息。而视觉SLAM的图像能提供丰富的语义信息，特征区分度更高，但其构建的地图不能直接用于路径规划和导航。为了实现移动机器人构建语义地图并在地图上进行路径规划，本文提出一种语义栅格建图方法。方法 建立可同步获取激光和语义数据的激光-相机系统，将采集的激光分割数据与目标检测算法获得的物体包围盒进行匹配，得到各物体对应的语义激光分割数据。将连续多帧语义激光分割数据同步融入占据栅格地图。对具有不同语义类别的栅格进行聚类，得到标注物体类别和轮廓的语义栅格地图。此外，针对语义栅格地图发布导航任务，利用路径搜索算法进行路径规划，并对其进行改进。结果 在实验室走廊和办公室分别进行了语义栅格建图的实验，并与原始栅格地图进行了比较。在语义栅格地图的基础上进行了路径规划，并采用了语义赋权算法对易移动物体的路径进行对比。结论 多种环境下的实验表明本文方法能获得与真实环境一致性较高、标注环境中物体类别和轮廓的语义栅格地图，且实验硬件结构简单、成本低、性能良好，适用于智能化机器人的导航和路径规划。     

栅格地图中多机器人协作搜索目标

目标搜索是多机器人领域的一个挑战.本文针对栅格地图中多机器人目标搜索算法进行研究.首先,利用Dempster-Shafer证据理论将声纳传感器获取的环境信息进行融合,构建搜索环境的栅格地图.然后,基于栅格地图建立生物启发神经网络用于表示动态的环境.在生物启发神经网络中,目标通过神经元的活性值全局的吸引机器人.同时,障碍物通过神经元活性值局部的排斥机器人,避免与其相撞.最后,机器人根据梯度递减原则自动的规划出搜索路径.仿真和实验结果显示本文提及的算法能够实现栅格地图中静态目标和动态目标的搜索.与其他搜索算法比较,本文所提及的目标搜索算法有更高的效率和适用性.     

基于模糊逻辑的机器人导航系统研究

由于未知环境下机器人导航容易出现死锁问题,设计了一种基于栅格的地图模型叫“数据栅格”,并在此基础上提出了一种基于行为的导航方法即“安全导航法”。数据栅格记录了周围环境中障碍物信息和机器人路径信息,安全导航法就是应用数据栅格技术来解决未知环境下机器人导航遇到的死锁问题。模糊逻辑用来设计和协调各种导航行为。仿真和实际环境的实验结果也证实了该方法的良好性能。     

The use of attention resources in navigation versus search

Personal information management research has consistently shown navigation preference over search. One possible explanation for this is that search requires more cognitive attention than navigation. We tested this hypothesis using the dual-task paradigm. We read a list of words to each of our 62 participants, asked them to navigate or search to a target file, and then compared the number of words recalled in each condition. Participants remembered significantly more words when retrieving by navigation than by search. The fact that they performed better at the secondary task when navigating indicates that it required less cognitive attention than search. Our results also cast doubt on the assumption that search is more efficient and easier to use than navigation: Search took nearly three times longer than navigation, was more vulnerable to mistakes and retrieval failures and was perceived as more difficult on a subjective evaluation. Our results also support the folk belief that women are better than men and that younger people are better than older ones, at multitasking.     

有领航者的多智能体系统的稳定性分析

本文研究了一类有领航者的多智能体系统在平面和高斯型源下的稳定性分析.该系统由领航者和跟随者两种智能体组成,只有领航者具有环境的信息;同时,吸引与排斥函数也被推广到具有某些性质的一类函数.证明了在满足适当假设条件时,在领航者智能体的导航作用下,多智能体系统的跟随者在没有环境信息的情况下也能准确地到达目标或远离目标.当多智能体系统在不同源的条件下聚集时有界区域的大小也被估计.给出数值仿真以证明理论结果的正确性.     

Efficient Exploration for Multi-Agent Reinforcement Learning via Transferable Successor Features

In multi-agent reinforcement learning (MARL), the behaviors of each agent can influence the learning of others, and the agents have to search in an exponentially enlarged joint-action space. Hence, it is challenging for the multi-agent teams to explore in the environment. Agents may achieve suboptimal policies and fail to solve some complex tasks. To improve the exploring efficiency as well as the performance of MARL tasks, in this paper, we propose a new approach by transferring the knowledge across tasks. Differently from the traditional MARL algorithms, we first assume that the reward functions can be computed by linear combinations of a shared feature function and a set of task-specific weights. Then, we define a set of basic MARL tasks in the source domain and pre-train them as the basic knowledge for further use. Finally, once the weights for target tasks are available, it will be easier to get a well-performed policy to explore in the target domain. Hence, the learning process of agents for target tasks is speeded up by taking full use of the basic knowledge that was learned previously. We evaluate the proposed algorithm on two challenging MARL tasks: cooperative box-pushing and non-monotonic predator-prey. The experiment results have demonstrated the improved performance compared with state-of-the-art MARL algorithms.      

Thinning-Based Topological Exploration Using Position Possibility of Topological Nodes

A grid map can be efficiently used in navigation, but this type of map requires a large amount of memory in proportion to the size of the environment. As an alternative, a topological map can be used to represent the environment in terms of discrete nodes with edges connecting them. It is usually constructed by Voronoi-like graphs, but in this paper the topological map is built based on the local grid map by using a thinning algorithm. This new approach can easily extract the topological information in real-time and be robustly applicable to the real environment, and this map can be autonomously built by exploration. The position possibility is defined to evaluate the quantitative reliability of the topological map and then a new exploration scheme based on the position possibility is proposed. From the position possibility information, the robot can determine whether or not it needs to visit a specific end node, which node will be the next target and how much of the environment has yet been explored. Various experiments showed that the proposed map-building and exploration methods can accurately build a local topological map in real-time and can guide a robot safely even in a dynamic environment.     

基于多智能体深度强化学习的协作导航应用

多机器人协作导航目前广泛应用于搜索救援、物流等领域, 协作策略与目标导航是多机器人协作导航面临的主要挑战. 为提高多个移动机器人在未知环境下的协作导航能力, 本文提出了一种新的分层控制协作导航(hierarchical control cooperative navigation, HCCN) 策略, 利用高层目标决策层和低层目标导航层, 为每个机器人分配一个目标点, 并通过全局路径规划和局部路径规划算法, 引导智能体无碰撞地到达分配的目标点. 通过Gazebo平台进行实验验证, 结果表明, 文中所提方法能够有效解决协作导航过程中的稀疏奖励问题, 训练速度至少可提高16.6%, 在不同环境场景下具有更好的鲁棒性, 以期为进一步研究多机器人协作导航提供理论指导, 应用至更多的真实场景中.