Recognition, prediction, and planning for assisted teleoperation of freeform tasks

The approach of inferring user’s intended task and optimizing low-level robot motions has promise for making robot teleoperation interfaces more intuitive and responsive. But most existing methods assume a finite set of candidate tasks, which limits a robot’s functionality. This paper proposes the notion of freeform tasks that encode an infinite number of possible goals (e.g., desired target positions) within a finite set of types (e.g., reach, orient, pick up). It also presents two technical contributions to help make freeform UIs possible. First, an intent predictor estimates the user’s desired task, and accepts freeform tasks that include both discrete types and continuous parameters. Second, a cooperative motion planner continuously updates the robot’s trajectories to achieve the inferred tasks by repeatedly solving optimal control problems. The planner is designed to respond interactively to changes in the indicated task, avoid collisions in cluttered environments, handle time-varying objective functions, and achieve high-quality motions using a hybrid of numerical and sampling-based techniques. The system is applied to the problem of controlling a 6D robot manipulator using 2D mouse input in the context of two tasks: static target reaching and dynamic trajectory tracking. Simulations suggest that it enables the robot to reach intended targets faster and to track intended trajectories more closely than comparable techniques.     

Decentralized Coordinated Tracking with Mixed Discrete–Continuous Decisions

In this paper, we study the problem of dynamically positioning a team of mobile robots for target tracking. We treat the coordination of mobile robots for target tracking as a joint team optimization to minimize uncertainty in target state estimates over a fixed horizon. The optimization is inherently a function of both the positioning of robots in continuous space and the assignment of robots to targets in discrete space. Thus, the robot team must make decisions over discrete and continuous variables. In contrast to methods that decouple target assignments and robot positioning, our approach avoids the strong assumption that a robot's utility for observing a target is independent of other robots’ observations. We formulate the optimization as a mixed integer nonlinear program and apply integer relaxation to develop an approximate solution in decentralized form. We demonstrate our coordinated multirobot tracking algorithm both in simulation and using a pair of mobile robotic sensor platforms to track moving pedestrians. Our results show that coupling target assignment and robot positioning realizes coordinated behaviors that are not possible with decoupled methods.     

Sensor Selection by GMB-REM in Real Robot Position Estimation

Modelling and reducing uncertainty are two essential problems with mobile robot localisation. Previously we developed a robot localisation system, namely, the Gaussian Mixture of Bayes with Regularised Expectation Maximisation (GMB-REM), using a single sensor. GMB-REM allows a robot"s position to be modelled as a probability distribution, and uses Bayes" theorem to reduce the uncertainty of its location. In this paper, a new system for performing sensor selection is introduced, namely an enhanced form of GMB-REM. Empirical results show the new system outperforms GMB-REM using sonar alone. More specifically, it is able to select between multiple sensors at each robot"s position, and further minimises the average robot localisation error.     

Autonomous robotic exploration using a utility function based on Rényi’s general theory of entropy

In this paper we present a novel information-theoretic utility function for selecting actions in a robot-based autonomous exploration task. The robot’s goal in an autonomous exploration task is to create a complete, high-quality map of an unknown environment as quickly as possible. This implicitly requires the robot to maintain an accurate estimate of its pose as it explores both unknown and previously observed terrain in order to correctly incorporate new information into the map. Our utility function simultaneously considers uncertainty in both the robot pose and the map in a novel way and is computed as the difference between the Shannon and the Rényi entropy of the current distribution over maps. Rényi’s entropy is a family of functions parameterized by a scalar, with Shannon’s entropy being the limit as this scalar approaches unity. We link the value of this scalar parameter to the predicted future uncertainty in the robot’s pose after taking an exploratory action. This effectively decreases the expected information gain of the action, with higher uncertainty in the robot’s pose leading to a smaller expected information gain. Our objective function allows the robot to automatically trade off between exploration and exploitation in a way that does not require manually tuning parameter values, a significant advantage over many competing methods that only use Shannon’s definition of entropy. We use simulated experiments to compare the performance of our proposed utility function to these state-of-the-art utility functions. We show that robots that use our proposed utility function generate maps with less uncertainty and fewer visible artifacts and that the robots have less uncertainty in their pose during exploration. Finally, we demonstrate that a real-world robot using our proposed utility function is able to successfully create a high-quality map of an indoor office environment.     

Efficient autonomous global localization for service robots using dual laser scanners and rotational motion

This study presents an alternative global localization scheme that uses dual laser scanners and the pure rotational motion of a mobile robot. The proposed method extracts the initial state of the robot’s surroundings to select robot pose candidates, and determines the sample distribution based on the given area map. Localization success is determined by calculating the similarity of the robot’s sensor state compared to that which would be expected at the estimated pose on the given map. In both simulations and experiments, the proposed method shows sufficient efficiency and speed to be considered robust to real-world conditions and applications.

     

基于场景图谱的室内移动机器人目标搜索

在移动机器人执行日常家庭任务时,首先需要其能够在环境中避开障碍物,自主地寻找到房间中的物体。针对移动机器人如何有效在室内环境下对目标物体进行搜索的问题,提出了一种基于场景图谱的室内移动机器人目标搜索,其框架结合了导航地图、语义地图和语义关系图谱。在导航地图的基础上建立了包含地标物体位置信息的语义地图,机器人可以轻松对地标物体进行寻找。对于动态的物体,机器人根据语义关系图中物体之间的并发关系,优先到关系强度比较高的地标物体旁寻找。通过物理实验展示了机器人在语义地图和语义关系图的帮助下可以实现在室内环境下有效地寻找到目标,并显著地减少了搜索的路径长度,证明了该方法的有效性。     

Intelligent and Efficient Strategy for Unstructured Environment Sensing Using Mobile Robot Agents

In field environments it is not usually possible to provide robots in advance with valid geometric models of its task and environment. The robot or robot teams need to create these models by scanning the environment with its sensors. Here, an information-based iterative algorithm to plan the robot's visual exploration strategy is proposed to enable it to most efficiently build 3D models of its environment and task. The method assumes mobile robot (or vehicle) with vision sensors mounted at a manipulator end-effector (eye-in-hand system). This algorithm efficiently repositions the systems' sensing agents using an information theoretic approach and fuses sensory information using physical models to yield a geometrically consistent environment map. This is achieved by utilizing a metric derived from Shannon's information theory to determine optimal sensing poses for the agent(s) mapping a highly unstructured environment. This map is then distributed among the agents using an information-based relevant data reduction scheme. This method is particularly well suited to unstructured environments, where sensor uncertainty is significant. Issues addressed include model-based multiple sensor data fusion, and uncertainty and vehicle suspension motion compensation. Simulation results show the effectiveness of this algorithm.     

Sensor planning for mobile robot localization using Bayesian network inference

We propose a new method of sensor planning for mobile robot localization using Bayesian network inference. Since we can model causal relations between situations of the robot's behavior and sensing events as nodes of a Bayesian network, we can use the inference of the network for dealing with uncertainty in sensor planning and thus derive appropriate sensing actions. In this system we employ a multi-layered-behavior architecture for navigation and localization. This architecture effectively combines mapping of local sensor information and the inference via a Bayesian network for sensor planning. The mobile robot recognizes the local sensor patterns for localization and navigation using a learned regression function. Since the environment may change during the navigation and the sensor capability has limitations in the real world, the mobile robot actively gathers sensor information to construct and reconstruct a Bayesian network, and then derives an appropriate sensing action which maximizes a utility function based on inference of the reconstructed network. The utility function takes into account belief of the localization and the sensing cost. We have conducted some simulation and real robot experiments to validate the sensor planning system.     

A mobile dual-arm manipulation robot system for stocking and disposing of items in a convenience store by using universal vacuum grippers for grasping items

ABSTRACT

A mobile dual-arm robot with universal vacuum grippers (UVG) for performing stocking and disposing tasks was developed. The robot grasps items in the tasks by using UVGs whose surface adapts to the various shapes of the items. A selection algorithm that determines whether the robot should use one or both manipulators to arrange an item was also developed. A ‘reachable-grasp pose’ is defined as a pose in which the robot’s UVG can easily place an item with a target attitude if it grasps the item. By using the selection algorithm, the robot re-grasps the item by adopting the reachable-grasp pose if the two manipulators do not collide when one is in the current grasp pose and the other is in the reachable-grasp pose. The robot system won the third prize of the Future Convenience Store Challenge 2018. In experiments on stocking and disposing tasks, the robot system achieved success rates of 100% for the stocking task and 80% for the disposing task. We believe that the results of this study will help researchers to develop a robot system for not only the stocking and disposing tasks but also other tasks in convenience stores (like customer interaction).     

RecAm: a collaborative context-aware framework for multimedia recommendations in an ambient intelligence environment

With an ever-increasing accessibility to different multimedia contents in real-time, it is difficult for users to identify the proper resources from such a vast number of choices. By utilizing the user’s context while consuming diverse multimedia contents, we can identify different personal preferences and settings. However, there is a need to reinforce the recommendation process in a systematic way, with context-adaptive information. The contributions of this paper are twofold. First, we propose a framework, called RecAm, which enables the collection of contextual information and the delivery of resulted recommendation by adapting the user’s environment using Ambient Intelligent (AmI) Interfaces. Second, we propose a recommendation model that establishes a bridge between the multimedia resources, user joint preferences, and the detected contextual information. Hence, we obtain a comprehensive view of the user’s context, as well as provide a personalized environment to deliver the feedback. We demonstrate the feasibility of RecAm with two prototypes applications that use contextual information for recommendations. The offline experiment conducted shows the improvement of delivering personalized recommendations based on the user’s context on two real-world datasets.     

On exploiting ‘negative’ sensor evidence for target tracking and sensor data fusion

In various applications of target tracking and sensor data fusion all available information related to the sensor systems used and the underlying scenario should be exploited for improving the tracking/fusion results. Besides the individual sensor measurements themselves, this in particular includes the use of more refined models for describing the sensor performance. By incorporating this type of background information into the processing chain, it is possible to exploit ‘negative’ sensor evidence. The notion of ‘negative’ sensor evidence covers the conclusions to be drawn from expected but actually missing sensor measurements for improving the position or velocity estimates of targets under track. Even a failed attempt to detect a target is a useful sensor output, which can be exploited by appropriate sensor models providing background information. The basic idea is illustrated by selected examples taken from more advanced tracking and sensor data fusion applications such as group target tracking, tracking with agile beam radar, ground moving target tracking, or tracking under jamming conditions.     

Global context-dependent recurrent neural network language model with sparse feature learning

Recurrent neural network language models (RNNLMs) are an important type of language model. In recent years, context-dependent RNNLMs are the most widely used ones as they apply additional information summarized from other sequences to access the larger context. However, when the sequences are mutually independent or randomly shuffled, these models cannot learn useful additional information, resulting in no larger context taken into account. In order to ensure that the model can obtain more contextual information in any case, a new language model is proposed in this paper. It can capture the global context just by the words within the current sequences, incorporating all the preceding and following words of target, without resorting to additional information summarized from other sequences. This model includes two main modules: a recurrent global context module used for extracting the global contextual information of the target and a sparse feature learning module that learns the sparse features of all the possible output words to distinguish the target word from others at the output layer. The proposed model was tested on three language modeling tasks. Experimental results show that it improves the perplexity of the model, speeds up the convergence of the network and learns better word embeddings compared with other language models.

     

基于5G通信技术的巡逻机器人定位误差自动补偿方法

以提升巡逻机器人执行巡逻任务能力为目的,提出基于5G通信技术的巡逻机器人定位误差自动补偿方法。该方法在巡逻机器人工作区域架设5G无线通信网络,并将AD7380型号位置传感器安装在巡逻机器人上,利用位置传感器获取巡逻机器人工作时的位置信息后,利用5G无线通信网络将其传输到用户PC端,得到巡逻机器人位置采样点数据；以该数据为基础,使用拉丁超立方采样方法描述巡逻机器人在其巡逻空间内的位置,得到巡逻机器人空间位置数据；再依据巡逻机器人空间位置数据,计算该机器人预设目标点误差矢量,并构建巡逻机器人定位误差补偿模型,利用该模型补偿巡逻机器人定位误差。实验结果表明：该方法可全面获取巡逻机器人在其巡逻空间内的位置信息,可有效且精准的对其定位误差进行自动补偿,使巡逻机器人定位误差保持在可允许范围内,应用效果较为显著。     

Improved Sensor Selection Technique by Integrating Sensor Fusion in Robot Position Estimation

Modelling and reducing uncertainty are two essential problems with mobile robot localisation. Previously we developed a robot localisation system, namely, the Gaussian Mixture of Bayes with Regularised Expectation Maximisation (GMB-REM), which introduced the sensor selection technique. GMB-REM allows a robot"s position to be modelled as a probability distribution and uses Bayes" theorem to reduce the uncertainty of its location. A new sensor selection technique incorporated with sensor fusion is introduced in this paper. Actually the new technique is realised by incorporating with the sensor fusion scheme. Empirical results show that the new system outperforms the previous GMB-REM with sensor selection alone. More specifically, we illustrate that the new technique is able to considerably constrain the error of a robot"s position.     

A robot learning from demonstration framework to perform force-based manipulation tasks

This paper proposes an end-to-end learning from demonstration framework for teaching force-based manipulation tasks to robots. The strengths of this work are manyfold. First, we deal with the problem of learning through force perceptions exclusively. Second, we propose to exploit haptic feedback both as a means for improving teacher demonstrations and as a human–robot interaction tool, establishing a bidirectional communication channel between the teacher and the robot, in contrast to the works using kinesthetic teaching. Third, we address the well-known what to imitate? problem from a different point of view, based on the mutual information between perceptions and actions. Lastly, the teacher’s demonstrations are encoded using a Hidden Markov Model, and the robot execution phase is developed by implementing a modified version of Gaussian Mixture Regression that uses implicit temporal information from the probabilistic model, needed when tackling tasks with ambiguous perceptions. Experimental results show that the robot is able to learn and reproduce two different manipulation tasks, with a performance comparable to the teacher’s one.     

基于分布式有限感知网络的多伯努利目标跟踪

针对感知范围受限的分布式传感网多目标跟踪问题, 在多伯努利滤波跟踪理论基础上提出分布式视场互补多伯努利关联算术平均融合跟踪方法. 首先, 通过视场互补扩大传感器感知范围, 其中, 局部公共区域只互补一次以降低计算成本. 其次, 每个传感器分别运行局部多伯努利滤波器, 并将滤波后验结果与相邻传感器进行泛洪通信使得每个传感器获取多个相邻传感器的后验信息. 随后, 通过距离划分进行多伯努利关联, 将对应于同一目标的伯努利分量关联到同一个子集中, 并对每个关联子集进行算术平均融合完成融合状态估计. 仿真实验表明, 所提方法在有限感知范围的分布式传感器网络中能有效地进行多目标跟踪.     

基于移动Sink的自组织视频传感网络目标跟踪算法

基于自组织视频传感网络的目标跟踪方法利用节点的分布式观测能力,实现目标的精确跟踪。在研究视频节点观测投射模型和通信模型的基础上,提出一种基于移动Sink的自组织视频传感网络目标跟踪算法MSTTA。该算法包括感知信息聚合和目标位置评估两个部分,利用节点分类机制周期性地更新网络拓扑以适应Sink位置的变化,根据目标运动状态预测目标位置的评估节点小组。仿真实验表明,MSTTA算法能够适应Sink移动带来的网络拓扑变化,具有较高的目标跟踪精度。     

Cooperative localization and evaluation of small-scaled spherical underwater robots

With the goal of supporting localization requirements of our spherical underwater robots, such as multi robot cooperation and intelligent biological surveillance, a cooperative localization system of multi robot was designed and implemented in this study. Given the restrictions presented by the underwater environment and the small-sized spherical robot, an time of flight camera and microelectro mechanical systems (MEMS) sensor information fusion algorithm using coordinate normalization transfer models were adopted to construct the proposed system. To handle the problem of short location distance, limited range under fixed view of camera in the underwater environment, a MEMS inertial sensor was used to obtain the attitude information of robot and expanding the range of underwater visual positioning, the transmission of positioning information could implement through the normalization of absolute coordinate, then the positioning distance increased and realized the localization of multi robot system. Given the environmental disturbances in practical underwater scenarios, the Kalman filter model was used to minimizing the systematic positioning error. Based on the theoretical analysis and calculation, we describe experiments in underwater to evaluate the performance of cooperative localization. The experimental results confirmed the validity of the multi robot cooperative localization system proposed in this paper, and the distance of cooperative localization system proposed in this paper is larger than the visual positioning system we have developed previously.

     

Towards modelling complex robot training tasks through system identification

Previous research has shown that sensor–motor tasks in mobile robotics applications can be modelled automatically, using NARMAX system identification, where the sensory perception of the robot is mapped to the desired motor commands using non-linear polynomial functions, resulting in a tight coupling between sensing and acting — the robot responds directly to the sensor stimuli without having internal states or memory.However, competences such as for instance sequences of actions, where actions depend on each other, require memory and thus a representation of state. In these cases a simple direct link between sensory perception and the motor commands may not be enough to accomplish the desired tasks. The contribution of this paper to knowledge is to show how fundamental, simple NARMAX models of behaviour can be used in a bootstrapping process to generate complex behaviours that were so far beyond reach.We argue that as the complexity of the task increases, it is important to estimate the current state of the robot and integrate this information into the system identification process. To achieve this we propose a novel method which relates distinctive locations in the environment to the state of the robot, using an unsupervised clustering algorithm. Once we estimate the current state of the robot accurately, we combine the state information with the perception of the robot through a bootstrapping method to generate more complex robot tasks: We obtain a polynomial model which models the complex task as a function of predefined low level sensor–motor controllers and raw sensory data.The proposed method has been used to teach Scitos G5 mobile robots a number of complex tasks, such as advanced obstacle avoidance, or complex route learning.