An instrumented glove for grasp specification invirtual-reality-based point-and-direct telerobotics

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping “flavors” has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as “put that there” cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest     

A practical operator support scheme and its application to safety-ensured object break using dual-arm machinery

A practical operator support system for disaster response work using construction machinery is applied to object-break operations using a dual-arm machine. An object-break operation, which separates an object into two different parts by applying massive pulling force, requires skillful force and trajectory controls. Operators typically have trouble visually perceiving the movement of the manipulator because the target object is strongly restrained. As a result, after the object has been separated into two pieces, the operator may continue performing lever operations because of unavoidable perception reaction time and the manipulator may collide with the surrounding environment. A control input cancellation system based on object-break identification is proposed to reduce the time difference between the object break and operation stoppage. A support system for disaster response situations must be safe and efficient and accommodate operators with various skill levels; thus, common physical phenomena related to the object breakage should be exploited. The system cancels control inputs to stop the movement of the manipulator when an object-break flag is detected, which is defined as a situation where the force applied to each joint of a manipulator decreases rapidly when a lever operation and load on the inside of the grapple are detected. Demolition experiments were conducted using a hydraulic dual-arm system. The results indicate that the displacement of the end-point after the object breaks is greatly decreased and the completion time of the task is reduced as well.     

无人机飞行控制系统模拟器设计

提出了一种基于工业控制计算机开发的某型无人机飞行控制系统模拟器设计;详细介绍了系统设计原理，给出了硬件构成方案、软件功能模块组成和程序流程图，最后分析了测试结果；该模拟器充分利用了当前计算机技术、自动控制技术、系统实时仿真技术和面向对象编程技术，具有功能强大、界面友好、操作灵活的特点。可用于无人机系统先期设计验证或后期测试维护；实际使用情况表明该系统性能良好，完全能够代替真实飞行控制系统进行实时仿真试验。     

基于Kinect的四旋翼无人机体感控制

针对无人机（ UAV）远程控制中操作复杂、专业性高的问题,基于Kinect传感器提出了一种无人机体感控制方案,并进行了验证。利用Kinect传感器提取操作者身体的骨骼节点数据,设计并识别操作者体势动作,进而生成对应的四旋翼飞行控制指令,通过无线数传模块传输控制指令,对无人机进行远程控制。实验结果表明：设计的识别算法可以准确地识别体势动作,对无人机进行实时的控制,控制方式直观简单、效果良好。     

无人机平台下的行人与车辆目标实时检测

在无人机图像中快速准确地检测行人和车辆是一项有意义但又极具挑战的任务,其广泛应用于军事侦察、交通管制以及偏远地区救援等任务中。然而,由于无人机属于小型移动设备,其内存和计算能力非常有限,使得如何保证其检测实时性一直是难题。针对SSD算法模型过大、运行内存占用量过高、很难在无人机设备上运行的问题,精心设计了轻量级的基准网络,通过削减原始网络的通道数目以及卷积数目来降低网络的参数量;针对无人机场景下目标小、场景复杂等问题,提出轻量级感受野模块来增强网络特征表示能力,并结合上下文信息来进一步提高小型目标的检测精度。实验结果表明,提出的方法在基于无人机的行人与车辆目标检测任务上有较高的准确性和实时性。     

A Unified Pilot Training and Control System for Underwater Robotic Vehicles (URV)

This paper introduces the work done to improve on a sophisticated Underwater Robotic Vehicle (URV) inspection and repair system for submerged structures. It is undertaken as part of a research programme grant to pursue research and development of technologies and systems for the advancement of knowledge and for possible commercial exploitation relevant to the oil and gas industry. In particular, the paper focuses on the development of a unified pilot training and controls system that incorporates an advance man–machine interface for improving operator dexterity. Few formalised training procedures exist for URV pilots. In spite of the high cost, most URV pilots receive their training on-the-job. Training simulators can be viewed as a viable solution to this problem. Some attention has been made to address this problem. Notably are efforts by Imetrix URV-Mentor system, which focuses on VE simulation and on-line tutoring. Simulators, however, represents additional costs and in some ways lacks the realism of working on the real system. In the R ² C the researchers proposed a novel simulator configuration. We have developed a dual-purpose topside control system configuration that can be used for training as well as for on-line operation of an actual URV. In the simulator configuration, the physical URV is replaced by a simulator module, which accepts actual commands from the control system and responds with a simulated URV status, using a dynamic model of the URV. The simulator module behaves much like the actual URV accepting commands and responds with status information. The advantage of such a system is perceived to be lower system cost as well as a more realistic testing and simulation of the relevant processes.     

融合目标检测和人体关键点检测的铁路司机行为识别

随着我国经济的快速发展,铁路运输在交通运输的地位愈为重要,在传统人工监管无力应对铁路司机安全监督的情况下,使用机器实现自动实时司机行为识别早已成为了一项极有意义的工作。为实现随车部署、实时进行铁路司机行为识别的目的,基于目标框检测算法实现目标检测和关键点检测的融合,搭建了一种可以同时检测司机人体关键点和手机的神经网络。经过网络运行输出人体姿态后,通过分析人体各关节角度和人体关键点与手机目标的位置关系等后处理对六类司机行为进行了分类识别,并通过TensorRT框架对模型进行了模型推理速度的加速和体积上的压缩。实验表明,该模型在嵌入式设备TX2上推理速度为25ms,可以达到较好检测效果下实时运行的目标。实现了实时进行铁路司机行为识别的目的。     

Development of an Intelligent Wheelchair Using Computer Graphics Animation and Simulation

A new robot simulator JC-1 is used as a control software development tool in a project in progress where an intelligent wheelchair for a blind user is being developed. The intelligent wheelchair is planned to be able to fulfill simple symbolic commands like "follow wall" or "follow object" and using the JC-1 simulator an evaluation team which includes e.g. the user, a rehabilitation engineer and a software engineer, can check control algorithms and user interface routines before constructing a real wheelchair prototype. The JC-1 simulator models the environment using simplified boundary- representation where objects, robot sensors and actuators are presented as symbolic objects in the graphics data-base of the simulator. In the JC-1 simulator a robot controller under development controls the motion of the graphical model of the robot while simulator commands or other robot controllers can be used to control the movement of disturbing obstacles. Computer graphics animation and simulation help to find fundamental design errors at an early design stage and as this paper suggests, enable the user of the final product to take part in to the designing process of the robot controller. Benefits and difficulties of using computer graphics simulation in the wheelchair development process are discussed.     

SeaMaster: an ROV-manipulator system simulator

In the absence of fully autonomous robots, manipulators operated by a person at a distance are used to carry out tasks in remote and/or hostile environments. SeaMaster software upgrades the human-machine interface of telemanipulators by letting operators control the actual and simulated manipulators either separately or simultaneously. It includes a data-driven control mode. In contrast to direct teleoperation, where the operator directly controls the telemanipulator and uses only video displays for feedback, the computer offers a means of automatically generating joint commands and uncoupling undesired motions of the ROV. The paper describes how the SeaMaster project achieved these goals through computer simulation. The simulator was designed to study the complex motions of an underwater remotely operated vehicle-manipulator system     

A modular approach to learning manipulation strategies from human demonstration

Object manipulation is a challenging task for robotics, as the physics involved in object interaction is complex and hard to express analytically. Here we introduce a modular approach for learning a manipulation strategy from human demonstration. Firstly we record a human performing a task that requires an adaptive control strategy in different conditions, i.e. different task contexts. We then perform modular decomposition of the control strategy, using phases of the recorded actions to guide segmentation. Each module represents a part of the strategy, encoded as a pair of forward and inverse models. All modules contribute to the final control policy; their recommendations are integrated via a system of weighting based on their own estimated error in the current task context. We validate our approach by demonstrating it, both in a simulation for clarity, and on a real robot platform to demonstrate robustness and capacity to generalise. The robot task is opening bottle caps. We show that our approach can modularize an adaptive control strategy and generate appropriate motor commands for the robot to accomplish the complete task, even for novel bottles.     

以网络为中心的语义特征造型协同设计

在网络化同步协同设计环境中，如何实现用户系统之间的实时数据交换、维护特征语义的一致性，成为语义特征造型协同设计的关键问题。该文介绍了基于细胞元表示的语义特征模型，根据该造型技术的分层体系结构，基于对协同环境下特征操作的分析，采取了传输造型操作命令的策略，为解决特征语义一致性问题，提出了一种用操作特征管理器调度操作执行的方法，并给出了一个实例。     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

Optimal unsupervised motor learning for dimensionality reduction ofnonlinear control systems

In this paper, optimal unsupervised motor learning is defined to be a technique for finding the coordinate system of minimum dimensionality which can adequately describe a particular motor task. An explicit method is provided for learning a stable controller that translates commands within the new coordinate system into motor variables appropriate for plant control. The method makes use of previously described neural network algorithms including the generalized Hebbian algorithm, basis-function trees, and trajectory extension learning. Examples of applications to a real direct-drive two joint planar robot arm and a simulated three joint robot arm with visual sensing are given.     

An experimental study about the effect of interactions among functional factors in performance of telemanipulation systems

The complexity of telemanipulation systems has increased and thus there are a variety of elements and factors that affect the performance of these systems. An experimental study was done in this field to obtain information about the effect of several factors on the quality of tasks carried out by the system. Not only the effect of the factors has been considered but also the interactions among them. A taxonomy of functional factors was proposed to facilitate this study. The factors were divided into two principal groups: intrinsic and extrinsic factors. A factorial design was conducted with five factors: operator (with vs. without training), movement control (position vs. rate control), force feedback (kinesthetic vs. visual feedback), master bandwidth (high vs. low bandwidth) and task type (insertion vs. tracking movement). An open platform for experimentation with telerobotics systems (PLATERO) was developed to perform all the proposed experiments. Analyzed variables include completion time, SOSF (sum of squared forces), insertion forces and tracking error. Results show that there is a great deal of interaction between type of task and the other factors. This means that for each task there is a system configuration that obtains better performance. Another important finding shows that an expert operator is able to adapt to the different system configurations and obtain good results. However, a novice operator obtains better results with some factors than with others. Finally, in order to determine which system configuration obtains the highest task quality, the main task requirement must be defined, because the best system configuration depends on it.     

基于显著图的弱监督实时目标检测

深度卷积神经网络(Deep convolutional neural network,DCNN)在目标检测任务上使用目标的全标注来训练网络参数,其检测准确率也得到了大幅度的提升.然而,获取目标的边界框(Bounding-box)标注是一项耗时且代价高的工作.此外,目标检测的实时性是制约其实用性的另一个重要问题.为了克服这两个问题,本文提出一种基于图像级标注的弱监督实时目标检测方法.该方法分为三个子模块:1)首先应用分类网络和反向传递过程生成类别显著图,该显著图提供了目标在图像中的位置信息;2)根据类别显著图生成目标的伪标注(Pseudo-bounding-box);3)最后将伪标注看作真实标注并优化实时目标检测网络的参数.不同于其他弱监督目标检测方法,本文方法无需目标候选集合获取过程,并且对于测试图像仅通过网络的前向传递过程就可以获取检测结果,因此极大地加快了检测的速率(实时性).此外,该方法简单易用;针对未知类别的目标检测,只需要训练目标类别的分类网络和检测网络.因此本框架具有较强的泛化能力,为解决弱监督实时检测问题提供了新的研究思路.在PASCAL VOC 2007数据集上的实验表明:1)本文方法在检测的准确率上取得了较好的提升;2)实现了弱监督条件下的实时检测.     

Model-based,sensor-directed remediation of underground storage tanks

Sensor-rich, intelligent robots that function with respect to models of their environment have significant potential to reduce the time and cost for the cleanup of hazardous waste while increasing operator safety. Sandia National Laboratories is performing experimental investigations into the application of intelligent robot control technology to the problem of removing waste stored in tanks. This article describes the experimental environment employed at Sandia with particular attention to the computing and software control environment. Intelligent system control is achieved through the integration of extensive geometric and kinematic world models with real-time sensor-based control. All operator interactions with the system are through fully animated, graphical representations that validate all operator commands before execution to provide for safe operation. Sensing is used to add information to the robot system's world model and to allow sensor-based servo control during selected operations. The results of a first critical features test are reported and the potential for applying advanced intelligent control concepts to the removal of waste in storage tanks is discussed.     

Human-centered control scheme for delayed bilateral teleoperation of mobile robots

Teleoperation task performance strongly depends on how well the human operator’s commands are executed. In this paper, we propose a control scheme for delayed bilateral teleoperation of mobile robots that considers user’s commands execution in order to achieve a high-performance teleoperation system in some important aspects like time to complete the task, safety, and operator dependence. We describe some evaluation metrics that allow us to address these aspects and a quantitative metric is proposed and incorporated in the control scheme to compensate wrong commands. A force feedback is applied to the master at the local site as a haptic cue. In addition, the system stability is analyzed taking into consideration the master and remote robot dynamic models and the asymmetric time-varying delays of the communication channel. Multiple human-in-the-loop simulations were carried out and the results of the evaluation metrics were discussed. Additionally, we present experiments where a user teleoperates a mobile robot via the Internet connection between Argentina and Italy.     

DCS一、二层接口防IIC失效风险的研究

核电站全厂控制系统广泛采用数字化的分布式控制系统(DCS)。一层为控制柜及供电、网络等设备。二层是以计算机信息和控制系统(系统代码为IIC)为主、后备盘(BUP)为辅的人机接口设备。一层担负着核电站的实时控制,是完整、独立的控制系统,其自动控制、保护等功能不依赖于二层。为防止因IIC失效给电厂运行造成风险,提出了一、二层接口设计的改进方法:一层接收二层的脉冲指令,防止二层失效对一层现有状态造成影响;一层逻辑中能闭锁IIC指令,防止BUP控制模式下IIC发出无效指令;增加选择开关KC模块的输入参数,将一层的当前模式上送IIC显示,防止发出错误指令或造成操作员误判。该研究为核电或其他工业控制系统一、二层接口设计提供了借鉴和参考。     

Improving performance on object recognition for real-time on mobile devices

Augmented reality has been on the rise due to the proliferation of mobile devices. At the same time, object recognition has also come to the fore. In particular, many studies have focused on object recognition based on markerless matching. However, most of these studies have focused on desktop systems, which can have high performance in terms of CPU and memory, rather than investigating the use of mobile systems, which have been previously unable to provide high-performance object recognition based on markerless matching. In this paper, we propose a method that uses the OpenCV mobile library to improve real-time object recognition performance on mobile systems. First, we investigate the original object recognition algorithm to identify performance bottlenecks. Second, we optimize the algorithm by analyzing each module and applying appropriate code enhancements. Last, we change the operational structure of the algorithm to improve its performance, changing the execution frequency of the object recognition task from every frame to every four frames for real-time operation. During the three frames in which the original method is not executed, the object is instead recognized using the mobile devices accelerometer. We carry out experiments to reveal how much each aspect of our method improves the overall object recognition performance; overall, experimental performance improves by approximately 800 %, with a corresponding reduction of approximately 1 % in object recognition accuracy. Therefore, the proposed technique can be used to significantly improve the performance of object recognition based on markerless matching on mobile systems for real-time operation.