Live semantic data from building digital twins for robot navigation: Overview of data transfer methods

Increasing reliance on automation and robotization presents great opportunities to improve the management of construction sites as well as existing buildings. Crucial in the use of robots in a built environment is their capacity to locate themselves and navigate as autonomously as possible. Robots often rely on planar and 3D laser scanners for that purpose, and building information models (BIM) are seldom used, for a number of reasons, namely their unreliability, unavailability, and mismatch with localization algorithms used in robots. However, while BIM models are becoming increasingly reliable and more commonly available in more standard data formats (JSON, XML, RDF), they become more promising and reliable resources for localization and indoor navigation, in particular in the more static types of existing infrastructure (existing buildings). In this article, we specifically investigate to what extent and how such building data can be used for such robot navigation. Data flows are built from BIM model to local repository and further to the robot, making use of graph data models (RDF) and JSON data formats. The local repository can hereby be considered to be a digital twin of the real-world building. Navigation on the basis of a BIM model is tested in a real world environment (university building) using a standard robot navigation technology stack. We conclude that it is possible to rely on BIM data and we outline different data flows from BIM model to digital twin and to robot. Future work can focus on (1) making building data models more reliable and standard (modelling guidelines and robot world model), (2) improving the ways in which building features in the digital building model can be recognized in 3D point clouds observed by the robots, and (3) investigating possibilities to update the BIM model based on robot feedback.     

实现家庭服务机器人中文指令解析问题研究

为了实现家庭服务机器人在无人干预的情况下自主地执行中文指令中蕴涵的服务任务,提出一种基于回答集的中文指令任务规划方法,将组块标注和回答集编程(answer set programming,ASP)应用于家庭服务机器人任务规划。首先通过组块标注对中文指令进行预处理,然后根据转换规则将关键信息转换为谓词集,并将它转写成ASP规则。此外,给出中文服务指令处理的各个环节的实验结果,并结合实例展示从谓词集到机器人可以执行的动作序列的映射过程。最后,通过合并部分原子动作的方式对回答集进行改进,提高了求解效率,并在任务规划时加入了成本规划,确认求得最优动作序列,该方法对促进自然人-机器人交互技术的发展有重要的意义。     

Sensor planning for mobile robot localization using Bayesian network inference

We propose a new method of sensor planning for mobile robot localization using Bayesian network inference. Since we can model causal relations between situations of the robot's behavior and sensing events as nodes of a Bayesian network, we can use the inference of the network for dealing with uncertainty in sensor planning and thus derive appropriate sensing actions. In this system we employ a multi-layered-behavior architecture for navigation and localization. This architecture effectively combines mapping of local sensor information and the inference via a Bayesian network for sensor planning. The mobile robot recognizes the local sensor patterns for localization and navigation using a learned regression function. Since the environment may change during the navigation and the sensor capability has limitations in the real world, the mobile robot actively gathers sensor information to construct and reconstruct a Bayesian network, and then derives an appropriate sensing action which maximizes a utility function based on inference of the reconstructed network. The utility function takes into account belief of the localization and the sensing cost. We have conducted some simulation and real robot experiments to validate the sensor planning system.     

Robot introspection through learned hidden Markov models

In this paper we describe a machine learning approach for acquiring a model of a robot behaviour from raw sensor data. We are interested in automating the acquisition of behavioural models to provide a robot with an introspective capability. We assume that the behaviour of a robot in achieving a task can be modelled as a finite stochastic state transition system.Beginning with data recorded by a robot in the execution of a task, we use unsupervised learning techniques to estimate a hidden Markov model (HMM) that can be used both for predicting and explaining the behaviour of the robot in subsequent executions of the task. We demonstrate that it is feasible to automate the entire process of learning a high quality HMM from the data recorded by the robot during execution of its task.The learned HMM can be used both for monitoring and controlling the behaviour of the robot. The ultimate purpose of our work is to learn models for the full set of tasks associated with a given problem domain, and to integrate these models with a generative task planner. We want to show that these models can be used successfully in controlling the execution of a plan. However, this paper does not develop the planning and control aspects of our work, focussing instead on the learning methodology and the evaluation of a learned model. The essential property of the models we seek to construct is that the most probable trajectory through a model, given the observations made by the robot, accurately diagnoses, or explains, the behaviour that the robot actually performed when making these observations. In the work reported here we consider a navigation task. We explain the learning process, the experimental setup and the structure of the resulting learned behavioural models. We then evaluate the extent to which explanations proposed by the learned models accord with a human observer's interpretation of the behaviour exhibited by the robot in its execution of the task.     

Color learning and illumination invariance on mobile robots: A survey

Recent developments in sensor technology have made it feasible to use mobile robots in several fields, but robots still lack the ability to accurately sense the environment. A major challenge to the widespread deployment of mobile robots is the ability to function autonomously, learning useful models of environmental features, recognizing environmental changes, and adapting the learned models in response to such changes. This article focuses on such learning and adaptation in the context of color segmentation on mobile robots in the presence of illumination changes. The main contribution of this article is a survey of vision algorithms that are potentially applicable to color-based mobile robot vision. We therefore look at algorithms for color segmentation, color learning and illumination invariance on mobile robot platforms, including approaches that tackle just the underlying vision problems. Furthermore, we investigate how the inter-dependencies between these modules and high-level action planning can be exploited to achieve autonomous learning and adaptation. The goal is to determine the suitability of the state-of-the-art vision algorithms for mobile robot domains, and to identify the challenges that still need to be addressed to enable mobile robots to learn and adapt models for color, so as to operate autonomously in natural conditions.     

Integrating active localization into high-level robot control systems

High-level control systems are designed to enable mobile robots to successfully perform complex missions such as office delivery and survillance tasks. For that purpose they have to control, coordinate, and monitor different kinds of subtasks like navigation, manipulation, and perception. An important aspect of the effectiveness of high-level control systems is the ability to cope with failures that occur during the execution of such subtaks. In this paper we focus on the particular subtask of estimating the position of the robot and show how to achieve its robust integration into the high-level control system. The principle of this integration is to monitor the certainty of the position estimation and to autonomously relocalize the robot whenever the uncertainly grows too large. We present a localization approach which accurately and efficiently keeps track of the robot's position. Furthermore, it provides a measure for detecting localization failures and it is able to autonomously relocalize the robot in such situations. In addition to this, we introduce structured reactive plans, which can be interrupted by such active localization processes at any point in time and allow the robot to complete its mission afterwards. Our method has been implemented and shown to be robust in long-term experiments involving a typical office delivery scenario.     

Learning object deformation models for robot motion planning

In this paper, we address the problem of robot navigation in environments with deformable objects. The aim is to include the costs of object deformations when planning the robot’s motions and trade them off against the travel costs. We present our recently developed robotic system that is able to acquire deformation models of real objects. The robot determines the elasticity parameters by physical interaction with the object and by establishing a relation between the applied forces and the resulting surface deformations. The learned deformation models can then be used to perform physically realistic finite element simulations. This allows the planner to evaluate robot trajectories and to predict the costs of object deformations. Since finite element simulations are time-consuming, we furthermore present an approach to approximate object-specific deformation cost functions by means of Gaussian process regression. We present two real-world applications of our motion planner for a wheeled robot and a manipulation robot. As we demonstrate in real-world experiments, our system is able to estimate appropriate deformation parameters of real objects that can be used to predict future deformations. We show that our deformation cost approximation improves the efficiency of the planner by several orders of magnitude.     

Smooth task switching through behaviour competition

Navigation in large-scale environments is composed of different local tasks. To achieve smooth switching between these tasks and thus a continuous control signal, usually a precise map of the environment and an exact pose estimate of the robot are needed. Both are hard to fulfil for experiments in real-world settings. We present a system that shows how one can relax the need for accurate metric models of the environment while at the same time achieving smooth task switching. To facilitate this scheme the dynamical systems approach is used, which incorporates behaviour coordination through competition in a dynamic framework. Feature detectors use sonar data to provide means for local navigation. This ability combined with a simple topological map constitutes a complete navigation system for large-scale office environments. Experiments showed that a Scout robot using this scheme is able to successfully navigate through our whole institute. Through the use of the dynamic behaviour coordination, switching between the navigational tasks occurs in a smooth manner leading to continuous control of the platform.     

Robot formation motion planning using Fast Marching

This paper presents the application of the Voronoi Fast Marching (VFM) method to path planning of mobile formation robots. The VFM method uses the propagation of a wave (Fast Marching) operating on the world model to determine a motion plan over a viscosity map (similar to the refraction index in optics) extracted from the updated map model. The computational efficiency of the method allows the planner to operate at high rate sensor frequencies. This method allows us to maintain good response time and smooth and safe planned trajectories. The navigation function can be classified as a type of potential field, but it has no local minima, it is complete (it finds the solution path if it exists) and it has a complexity of order n(O(n)), where n is the number of cells in the environment map. The results presented in this paper show how the proposed method behaves with mobile robot formations and generates trajectories of good quality without problems of local minima when the formation encounters non-convex obstacles.     

Mobile robot navigation using motor schema and fuzzy context dependent behavior modulation

This paper presents a novel technique to autonomously select different motor schemas using fuzzy context dependant blending of robot behaviors for navigation. First, a set of motor schemas is formed as behaviors. Both strategic and reactive type schemas have been employed in order to facilitate both the aspects of global and local motion planning. While strategic schemas are formed using the prior knowledge of the environment, the reactive schemas are activated using current sensory data of the robot. For global path planning, a safe path is first created using a Voronoi diagram. For local planning, the Voronoi vertices are treated as immediate subgoals and are used to form schemas leading to achieve optimized traveled distance and goal oriented robot navigation. Two motor schemas are formed as reactive behaviors for obstacle avoidance. The unknown obstacles are modeled using the sensory data. The coordinated behavior is achieved while employing weighed vector summation of the schemas. The adaptation of weights are achieved through a fuzzy inference system where fuzzy rules are used to dynamically generate the weights during navigation. A novel approach is proposed for fuzzy context-dependent blending of schemas. Fuzzy rules are formed using two main criteria into account: the first criterion reasons out the context dependent activity of a schema for achieving goal and the second criterion reasons out cooperative activity of strategic schemas with high priority reactive schemas. Comprehensive results validate that the proposed technique eliminates the existing drawbacks of motor schema approaches available in literature and provides collision free goal oriented robot navigation.     

Hybrid mission planning with coalition formation

The increase in robotic capabilities and the number of such systems being used has resulted in opportunities for robots to work alongside humans in an increasing number of domains. The current robot control paradigm of one or multiple humans controlling a single robot is not scalable to domains that require large numbers of robots and is infeasible in communications constrained environments. Robots must autonomously plan how to accomplish missions composed of many tasks in complex and dynamic domains; however, mission planning with a large number of robots for such complex missions and domains is intractable. Coalition formation can manage planning problem complexity by allocating the best possible team of robots for each task. A limitation is that simply allocating the best possible team does not guarantee an executable plan can be formulated. However, coupling coalition formation with planning creates novel, domain-independent tools resulting in the best possible teams executing the best possible plans for robots acting in complex domains.     

Model-based learning for mobile robot navigation from the dynamicalsystems perspective

This paper discusses how a behavior-based robot can construct a "symbolic process" that accounts for its deliberative thinking processes using models of the environment. The paper focuses on two essential problems; one is the symbol grounding problem and the other is how the internal symbolic processes can be situated with respect to the behavioral contexts. We investigate these problems by applying a dynamical system's approach to the robot navigation learning problem. Our formulation, based on a forward modeling scheme using recurrent neural learning, shows that the robot is capable of learning grammatical structure hidden in the geometry of the workspace from the local sensory inputs through its navigational experiences. Furthermore, the robot is capable of generating diverse action plans to reach an arbitrary goal using the acquired forward model which incorporates chaotic dynamics. The essential claim is that the internal symbolic process, being embedded in the attractor, is grounded since it is self-organized solely through interaction with the physical world. It is also shown that structural stability arises in the interaction between the neural dynamics and the environmental dynamics, which accounts for the situatedness of the internal symbolic process, The experimental results using a mobile robot, equipped with a local sensor consisting of a laser range finder, verify our claims.     

Probabilistic Road Map sampling strategies for multi-robot motion planning

This paper presents a Probabilistic Road Map (PRM) motion planning algorithm to be queried within Dynamic Robot Networks—a multi-robot coordination platform for robots operating with limited sensing and inter-robot communication.

First, the Dynamic Robot Networks (DRN) coordination platform is introduced that facilitates centralized robot coordination across ad hoc networks, allowing safe navigation in dynamic, unknown environments. As robots move about their environment, they dynamically form communication networks. Within these networks, robots can share local sensing information and coordinate the actions of all robots in the network.

Second, a fast single-query Probabilistic Road Map (PRM) to be called within the DRN platform is presented that has been augmented with new sampling strategies. Traditional PRM strategies have shown success in searching large configuration spaces. Considered here is their application to on-line, centralized, multiple mobile robot planning problems. New sampling strategies that exploit the kinematics of non-holonomic mobile robots have been developed and implemented. First, an appropriate method of selecting milestones in a PRM is identified to enable fast coverage of the configuration space. Second, a new method of generating PRM milestones is described that decreases the planning time over traditional methods. Finally, a new endgame region for multi-robot PRMs is presented that increases the likelihood of finding solutions given difficult goal configurations.

Combining the DRN platform with these new sampling strategies, on-line centralized multi-robot planning is enabled. This allows robots to navigate safely in environments that are both dynamic and unknown. Simulations and real robot experiments are presented that demonstrate: (1) speed improvements accomplished by the sampling strategies, (2) centralized robot coordination across Dynamic Robot Networks, (3) on-the-fly motion planning to avoid moving and previously unknown obstacles and (4) autonomous robot navigation towards individual goal locations.     

一种基于操作条件反射原理的学习模型

针对认知机器人的自主学习问题,提出一种基于操作条件反射原理的学习模型(OCLM).该模型采用状态空间、操作行为空间、概率分布函数、仿生学习机制、系统熵等进行描述,给出状态的"负理想度"的概念,定义了取向函数的计算方法.运用模型对机器人避障导航问题进行仿真实验,并对参数设置进行了讨论.实验结果表明,基于OCLM模型的机器人能通过与环境的交互获得认知,成功避障到达目的地,具有一定的自学习能力,从而表明了模型的有效性.     

基于滚动窗口算法的机器人路径规划应用研究

研究了未知环境下,特别是动态环境下,移动机器人基于滚动窗口的路径规划避障策略。着重分析了如何利用探测的有限信息进行场景分析和场景预测的过程,阐述了如何在保证安全性的前提下机器人利用启发信息进行局部最优规划,结合窗口滚动和反馈机制实现机器人的全局规划。该算法以机器人为中心,具有很强的可操作性和实际应用价值。仿真结果证明了本算法的实时性和有效性。     

一种改进的未知动态环境下机器人混合路径规划方法

动态未知环境下的机器人路径规划是机器人导航领域的重要课题之一,采用传统的方法求解并不理想。针对这个问题,提出一种改进的机器人混合路径规划方法。首先利用改进的文化基因算法规划出较优的全局路径,指引机器人沿着全局路径行走,然后根据传感器探测到的局部环境信息,利用Morphin算法进行局部路径实时规划,使机器人有效地躲避动态障碍物。仿真实验表明,该算法在未知动态路径规划中具有良好的效果。     

基于语义建图的室内机器人实时场景分类

针对室内环境下的机器人场景识别问题,重点研究了场景分类策略的自主性、实时性和准确性,提出了一种语义建图方法.映射深度信息构建二维栅格地图,自主规划场景识别路径;基于卷积网络建立场景分类模型,实时识别脱离特定训练;利用贝叶斯框架融合先验知识,修正了错误分类并完成语义建图.实验结果表明:机器人能够进行全局自主探索,实时判断场景类别,并创建满足要求的语义地图.同时,实际路径规划中,机器人可以根据语义信息改善导航行为,验证了方法的可行性.     

Interactive Teaching for Vision-Based Mobile Robots: A Sensory-Motor Approach

For the last decade, we have been developing a vision-based architecture for mobile robot navigation. Using our bio-inspired model of navigation, robots can perform sensory-motor tasks in real time in unknown indoor as well as outdoor environments. We address here the problem of autonomous incremental learning of a sensory-motor task, demonstrated by an operator guiding a robot. The proposed system allows for semisupervision of task learning and is able to adapt the environmental partitioning to the complexity of the desired behavior. A real dialogue based on actions emerges from the interactive teaching. The interaction leads the robot to autonomously build a precise sensory-motor dynamics that approximates the behavior of the teacher. The usability of the system is highlighted by experiments on real robots, in both indoor and outdoor environments. Accuracy measures are also proposed in order to evaluate the learned behavior as compared to the expected behavioral attractor. These measures, used first in a real experiment and then in a simulated experiment, demonstrate how a real interaction between the teacher and the robot influences the learning process.     

Reinforcement based mobile robot navigation in dynamic environment

In this paper, a new approach is developed for solving the problem of mobile robot path planning in an unknown dynamic environment based on Q-learning. Q-learning algorithms have been used widely for solving real world problems, especially in robotics since it has been proved to give reliable and efficient solutions due to its simple and well developed theory. However, most of the researchers who tried to use Q-learning for solving the mobile robot navigation problem dealt with static environments; they avoided using it for dynamic environments because it is a more complex problem that has infinite number of states. This great number of states makes the training for the intelligent agent very difficult. In this paper, the Q-learning algorithm was applied for solving the mobile robot navigation in dynamic environment problem by limiting the number of states based on a new definition for the states space. This has the effect of reducing the size of the Q-table and hence, increasing the speed of the navigation algorithm. The conducted experimental simulation scenarios indicate the strength of the new proposed approach for mobile robot navigation in dynamic environment. The results show that the new approach has a high Hit rate and that the robot succeeded to reach its target in a collision free path in most cases which is the most desirable feature in any navigation algorithm.     

Towards robotic assistants in nursing homes: Challenges and results

This paper describes a mobile robotic assistant, developed to assist elderly individuals with mild cognitive and physical impairments, as well as support nurses in their daily activities. We present three software modules relevant to ensure successful human–robot interaction: an automated reminder system; a people tracking and detection system; and finally a high-level robot controller that performs planning under uncertainty by incorporating knowledge from low-level modules, and selecting appropriate courses of actions. During the course of experiments conducted in an assisted living facility, the robot successfully demonstrated that it could autonomously provide reminders and guidance for elderly residents.