Reinforcement learning in robotic applications: a comprehensive survey

In recent trends, artificial intelligence (AI) is used for the creation of complex automated control systems. Still, researchers are trying to make a completely autonomous system that resembles human beings. Researchers working in AI think that there is a strong connection present between the learning pattern of human and AI. They have analyzed that machine learning (ML) algorithms can effectively make self-learning systems. ML algorithms are a sub-field of AI in which reinforcement learning (RL) is the only available methodology that resembles the learning mechanism of the human brain. Therefore, RL must take a key role in the creation of autonomous robotic systems. In recent years, RL has been applied on many platforms of the robotic systems like an air-based, under-water, land-based, etc., and got a lot of success in solving complex tasks. In this paper, a brief overview of the application of reinforcement algorithms in robotic science is presented. This survey offered a comprehensive review based on segments as (1) development of RL (2) types of RL algorithm like; Actor-Critic, DeepRL, multi-agent RL and Human-centered algorithm (3) various applications of RL in robotics based on their usage platforms such as land-based, water-based and air-based, (4) RL algorithms/mechanism used in robotic applications. Finally, an open discussion is provided that potentially raises a range of future research directions in robotics. The objective of this survey is to present a guidance point for future research in a more meaningful direction.

     

人机智能技术及系统研究进展综述

人机智能系统是能够实现人机智能协作的机器人系统,近年来成为了机器人领域的研究热点,具有广泛的应用前景。针对人机智能系统技术和应用的国内外研究现状,从人机智能系统的关键技术和典型应用领域两方面进行了进展综述。重点综述了与传统机器人系统存在差异性的人机智能系统关键技术,从建模、交互、协同和优化4个方面的研究进展分别展开论述,对涉及的典型应用领域及典型人机智能系统进行总结,并对人机智能系统发展的挑战和未来研究方向进行了展望。     

Repeatability of Real World Training Experiments: A Case Study

We present a case study of reinforcement learning on a real robot that learns how to back up a trailer and discuss the lessons learned about the importance of proper experimental procedure and design. We identify areas of particular concern to the experimental robotics community at large. In particular, we address concerns pertinent to robotics simulation research, implementing learning algorithms on real robotic hardware, and the difficulties involved with transferring research between the two.     

复杂制造系统建模与优化研究现状及展望

近年来,复杂制造系统及其自动化、智能化和定制化等优势在汽车制造、芯片制造、机器人等领域得到了广泛关注,其建模与优化问题也已成为国内外的研究热点。本文首先介绍了复杂制造系统的研究现状和典型场景应用,比如设备管理、生产过程自动化和生产调度管理。紧接着汇总了常见的建模与优化方法以及所解决的实际工程问题,特别是深度学习、强化学习和合作博弈等方法在复杂制造系统建模与优化中的应用。最后,对复杂制造系统建模和优化问题进行了展望。     

Surgical robotics: Reviewing the past,analysing the present,imagining the future

This paper presents an overview of the surgical robotics field, highlighting significant milestones and grouping the various propositions into cohorts. The review does not aim to be exhaustive but rather to highlight how surgical robotics is acting as an enabling technology for minimally invasive surgery. As such, there is a focus on robotic surgical solutions which are commercially available; research efforts which have not gained regulatory approval or entered clinical use are mostly omitted. The practice of robotic surgery is currently largely dominated by the da Vinci system of Intuitive Surgical (Sunnyvale, CA, USA) but other commercial players have now entered the market with surgical robotic products or are appearing in the horizon with medium and long term propositions. Surgical robotics is currently a vibrant research topic and new research directions may lead to the development of very different robotic surgical devices in the future—small, special purpose, lower cost, possibly disposable robots rather than the current large, versatile and capital expensive systems. As the trend towards minimally invasive surgery (MIS) increases, surgery becomes more technically demanding for surgeons and more challenging for medical device technologists and it is clear that surgical robotics has now an established foothold in medicine as an enabling technology of MIS.     

An Anthropomorphic Robotic Platform for Progressive and Adaptive Sensorimotor Learning

In recent years, advances and improvements in engineering and robotics have in part been due to strengthened interactions with the biological sciences. Robots that mimic the complexity and adaptability of biological systems have become a central goal in research and development in robotics. Usually, such a collaboration is addressed to a 2-fold perspective of (i) setting up anthropomorphic platforms as test beds for studies in neuroscience and (ii) promoting new mechatronic and robotic technologies for the development of bio-inspired or humanoid high-performance robotic platforms. This paper provides a brief overview of recent studies on sensorimotor coordination in human motor control and proposes a novel paradigm of adaptive learning for sensorimotor control, based on a multi-network high-level control architecture. The proposed neurobiologically inspired model has been applied to a robotic platform, purposely designed to provide anthropomorphic solutions to neuroscientific requirements. The goal of this work is to use the bio-inspired robotic platform as a test bed for validating the proposed model of high-level sensorimotor control, with the aim of demonstrating adaptive and modular control based on acquired competences, with a higher degree of flexibility and generality than conventional robotic controllers, while preserving their robustness. To this purpose, a set of object-dependent, visually guided reach-and-grasp tasks and the associated training phases were first implemented in a multi-network control architecture in simulation. Subsequently, the offline learning realized in simulation was used to produce the input command of reach-and-grasp to the low-level position control of the robotic platform. Experimental trials demonstrated that the adaptive and modular high-level control allowed reaching and grasping of objects located at different positions and objects of variable size, shape and orientation. A future goal would be to address autonomous and progressive learning based on growing competences.     

Cognitive neuroscience and robotics: Advancements and future research directions

In recent years, brain-based technologies that capitalise on human abilities to facilitate human–system/robot interactions have been actively explored, especially in brain robotics. Brain–computer interfaces, as applications of this conception, have set a path to convert neural activities recorded by sensors from the human scalp via electroencephalography into valid commands for robot control and task execution. Thanks to the advancement of sensor technologies, non-invasive and invasive sensor headsets have been designed and developed to achieve stable recording of brainwave signals. However, robust and accurate extraction and interpretation of brain signals in brain robotics are critical to reliable task-oriented and opportunistic applications such as brainwave-controlled robotic interactions. In response to this need, pervasive technologies and advanced analytical approaches to translating and merging critical brain functions, behaviours, tasks, and environmental information have been a focus in brain-controlled robotic applications. These methods are composed of signal processing, feature extraction, representation of neural activities, command conversion and robot control. Artificial intelligence algorithms, especially deep learning, are used for the classification, recognition, and identification of patterns and intent underlying brainwaves as a form of electroencephalography. Within the context, this paper provides a comprehensive review of the past and the current status at the intersection of robotics, neuroscience, and artificial intelligence and highlights future research directions.     

Search and tracking algorithms for swarms of robots: A survey

Target search and tracking is a classical but difficult problem in many research domains, including computer vision, wireless sensor networks and robotics. We review the seminal works that addressed this problem in the area of swarm robotics, which is the application of swarm intelligence principles to the control of multi-robot systems. Robustness, scalability and flexibility, as well as distributed sensing, make swarm robotic systems well suited for the problem of target search and tracking in real-world applications. We classify the works we review according to the variations and aspects of the search and tracking problems they addressed. As this is a particularly application-driven research area, the adopted taxonomy makes this review serve as a quick reference guide to our readers in identifying related works and approaches according to their problem at hand. By no means is this an exhaustive review, but an overview for researchers who are new to the swarm robotics field, to help them easily start off their research.     

Survey on frontiers of language and robotics

ABSTRACT

The understanding and acquisition of a language in a real-world environment is an important task for future robotics services. Natural language processing and cognitive robotics have both been focusing on the problem for decades using machine learning. However, many problems remain unsolved despite significant progress in machine learning (such as deep learning and probabilistic generative models) during the past decade. The remaining problems have not been systematically surveyed and organized, as most of them are highly interdisciplinary challenges for language and robotics. This study conducts a survey on the frontier of the intersection of the research fields of language and robotics, ranging from logic probabilistic programming to designing a competition to evaluate language understanding systems. We focus on cognitive developmental robots that can learn a language from interaction with their environment and unsupervised learning methods that enable robots to learn a language without hand-crafted training data.     

蛇形机器人研究现况与进展

仿生技术与机器人技术的结合，使机器人从结构设计到运动模式的选择都有了 新的进展，这大大扩大了机器人的应用领域．本文阐述了仿蛇形机器人的应用背景和研究现 状，并展望了其未来的发展．