基于深度学习的巡检机器人避障轨迹自动控制系统设计

为了提高巡检机器人在复杂环境下的避障能力,使机器人能够安全地完成巡检任务,设计基于深度学习的巡检机器人避障轨迹自动控制系统。设计由CCD传感器、信号处理芯片等设备组成的工业智能视觉CCD相机,基于FPGA和USB2.0的视频采集卡传输采集数据,完成硬件部分的设计。在软件设计中,对采集的图像实施目标分割、双目目标匹配等预处理,通过对摄像头实施双目视觉标定获取障碍物空间位置三维信息,基于深度学习中的CRNN设计机器人自主避障规划网络模型,并设计模糊轨迹控制器,实现避障中的轨迹自动控制。系统测试结果表明,设计系统最终成功避开了三个动态障碍物,最大轨迹控制误差的最大值为1.45°,最小轨迹控制误差的最大值为0.62°,动态避障巡检速度始终在3.5m/s左右,表现出了精准而稳定的轨迹控制效果。     

Navigation system including associative memory

A new adaptive linear robot control system for a robot work cell that can visually track and intercept stationary and moving objects undergoing arbitrary motion anywhere along its predicted trajectory within the robot's workspace is presented in this paper. The proposed system was designed by integrating a stationary monocular CCD camera with off-the-shelf frame grabber and an industrial robot operation into a single application on the MATLAB platform. A combination of the model based object recognition technique and a learning vector quantization network is used for classifying stationary objects without overlapping. The optical flow technique and the MADALINE network are used for determining the target trajectory and generating the predicted robot trajectory based on visual servoing, respectively. The necessity of determining a model of the robot, camera, all the stationary and moving objects, and environment is eliminated. The location and image features of these objects need not be preprogrammed, marked and known before, and any change in a task is possible without changing the robot program. After the learning process on the robot, it is shown that the KUKA robot is capable of tracking and intercepting both stationary and moving objects at an optimal rendezvous point on the conveyor accurately in real-time.     

Design and implementation of a sensor-based wireless camera system for continuous monitoring in assistive environments

Camera-based surveillance system is an important tool for assistive environment to monitor those who may have physical or cognitive impairment. It is, however, expensive to deploy a wired surveillance system and difficult to continuously monitor a moving subject in a large facility where many cameras are deployed. In this paper, we first evaluate the performance of streaming camera images over wireless networks in both residential and office environments and present the quantitative results to show the feasibility of using wireless backbones for camera surveillance systems. We then propose sensor-integrated camera surveillance (SICS) to address the continuous monitoring problem. SICS uses wearable wireless sensors to locate moving subjects and automatically selects the camera covering the subject, allowing human operators to focus on only one screen to monitor an individual. SICS uses a self-organizing wireless mesh network to allow flexible deployment at reduced cost. An on-board image-processing algorithm is used to reduce the bandwidth consumption. Through empirical evaluation, we found that the automatic camera hand-off enabled by SICS was effective for continuous camera monitoring and a sophisticated wireless network management system is required to deploy the SICS in practice.     

面向无人机电力巡检的语义实体构建及航迹控制方法

航迹的合理控制是影响无人机（UAV）智能决策重要因素。考虑UAV巡检的局部观测性和任务环境的高空复杂性，以电力巡检领域知识为背景，提出面向UAV电力巡检的语义实体构建及航迹控制方法。首先，基于电力巡检领域的实体知识构建空间拓扑网络，并生成关于位置节点的语义航迹序列网络及其语义接口；然后，根据空间拓扑结构相似性度量的结果集，提出安全许可机制和基于强化学习的航迹控制策略，实现UAV电力巡检在统一的概念内涵和位置结构上的轨迹控制。实验结果表明：作为UAV巡检的实例，所提方法得到的最优策略能获得最大化的鲁棒性能；同时，该方法通过强化学习方法使目标网络的适应度稳定收敛且实体区域覆盖率高于95%，为UAV电力巡检任务决策提供了飞行依据。     

Reference Trajectory-Based Visual Predictive Control

Abstract

In this paper we present an image predictive controller for an eye-in-hand-type servoing architecture, composed of a 6-d.o.f. robot and a camera mounted on the gripper. A novel architecture for integrating reference trajectory and image prediction is proposed for use in predictive control of visual servoing systems. In the proposed method, a new predictor is developed based on the relation between the camera velocity and the time variation of the visual features given by the interaction matrix. The image-based predictor generates the future trajectories of a visual feature ensemble when past and future camera velocities are known. In addition, a reference trajectory is introduced to define the way how to reach the desired features over the prediction horizon starting from the current features. The advantages of the new architecture are the reference trajectory used for the first time in the sense of the predictive control and the predictor based on a local model. Simulations reveal the efficiency of the proposed architecture to control a 6-d.o.f. robot manipulator.     

面向无人机电力巡检的语义实体构建及航迹控制方法

航迹的合理控制是影响无人机（UAV）智能决策重要因素。考虑UAV巡检的局部观测性和任务环境的高空复杂性,以电力巡检领域知识为背景,提出面向UAV电力巡检的语义实体构建及航迹控制方法。首先,基于电力巡检领域的实体知识构建空间拓扑网络,并生成关于位置节点的语义航迹序列网络及其语义接口;然后,根据空间拓扑结构相似性度量的结果集,提出安全许可机制和基于强化学习的航迹控制策略,实现UAV电力巡检在统一的概念内涵和位置结构上的轨迹控制。实验结果表明：作为UAV巡检的实例,所提方法得到的最优策略能获得最大化的鲁棒性能;同时,该方法通过强化学习方法使目标网络的适应度稳定收敛且实体区域覆盖率高于95%,为UAV电力巡检任务决策提供了飞行依据。     

Towards affective camera control in games

Information about interactive virtual environments, such as games, is perceived by users through a virtual camera. While most interactive applications let users control the camera, in complex navigation tasks within 3D environments users often get frustrated with the interaction. In this paper, we propose inclusion of camera control as a vital component of affective adaptive interaction in games. We investigate the impact of camera viewpoints on psychophysiology of players through preference surveys collected from a test game. Data is collected from players of a 3D prey/predator game in which player experience is directly linked to camera settings. Computational models of discrete affective states of fun, challenge, boredom, frustration, excitement, anxiety and relaxation are built on biosignal (heart rate, blood volume pulse and skin conductance) features to predict the pairwise self-reported emotional preferences of the players. For this purpose, automatic feature selection and neuro-evolutionary preference learning are combined providing highly accurate affective models. The performance of the artificial neural network models on unseen data reveals accuracies of above 80% for the majority of discrete affective states examined. The generality of the obtained models is tested in different test-bed game environments and the use of the generated models for creating adaptive affect-driven camera control in games is discussed.     

一种水下运动物体三维轨迹视觉测量方法

针对海洋工程中采用的设备深海悬垂法安装过程,采用多摄像头视频运动分析方 法计算水下三维运动轨迹可用于指导海洋工程的结构安装和分析设备水下运动特征。水下视频 和图像的处理获取面临着诸多挑战,首先由于水下环境悬浮物和颗粒较多,光在水下发生了散 射,使水下图像发生了退化;其次水下视频运动分析遇到的一个主要障碍是光线的折射引起的 图像误差。由于光在水、玻璃、空气不同介质间发生折射,光路发生弯曲,陆地上的摄像机成 像模型在水中不再适用,需要提出新的水下摄像机成像模型。本文引入带光线折射的水下摄像 机成像模型,研究水下摄像机的内参数和外参数标定方法,利用固定布置的 3 个水下摄像机拍 摄的目标水下运动视频来计算水下目标的轨迹。该方法适用于水池环境下水下物体大范围运动, 可以得到较为精确的轨迹,并得到了实验验证。     

Prediction and congestion control algorithm for networked motion tracking

In this paper a novel prediction method and a communication protocol is proposed for distributed motion tracking systems, for example robot control system over the Internet based on on-line visual information. It is assumed that the trajectory generator part of the control system is connected to the low level controller through wide area network (WAN). In this case the variable network delay, packet losses, irregular packet arrival can severely influence the control characteristics (transient behavior and tracking performance) in a negative sense. The proposed prediction method is based on dynamic filters and it generates the trajectory on the control system side in the control periods when no new information on the time varying reference trajectory arrives through the network. The developed application level communication protocol is meant to keep the packet loss under a predefined limit even if the network bandwidth varies below the value required by the control application. Simulations and real-time experiments show that the prediction algorithm applied jointly with the proposed communication protocol can effectively compensate the effect of networked communication on control characteristics.     

Omnidirectional Image Stabilization for Visual Object Recognition

In this paper, we present a pipeline for camera pose and trajectory estimation, and image stabilization and rectification for dense as well as wide baseline omnidirectional images. The proposed pipeline transforms a set of images taken by a single hand-held camera to a set of stabilized and rectified images augmented by the computed camera 3D trajectory and a reconstruction of feature points facilitating visual object recognition. The paper generalizes previous works on camera trajectory estimation done on perspective images to omnidirectional images and introduces a new technique for omnidirectional image rectification that is suited for recognizing people and cars in images. The performance of the pipeline is demonstrated on real image sequences acquired in urban as well as natural environments.     

Robust neural network control system design for linear ultrasonic motor

Linear ultrasonic motor (LUSM) has much merit, such as high precision, fast control dynamics and large driving force, etc.; however, the dynamic characteristic of LUSM is nonlinear and the precise dynamic model of LUSM is difficult to obtain. To tackle this problem, this study presents a robust neural network control (RNNC) system for LUSM to track a reference trajectory with L ₂ robust tracking performance. The developed RNNC system is composed of a neural network controller and a robust controller. The neural network controller is the principal controller used to mimic an ideal controller and the robust controller is adopted to achieve L ₂ robust tracking performance. The developed RNNC system is then applied to control an LUSM. Experimental results show that the developed RNNC system can achieve favorable tracking performance with unknown of LUSM model.     

Visibility Enhancement using Autonomous Multicamera Controls with Situational Role Assignment for Teleoperated Work Machines

The aim of this study is to provide a machine operator with enhanced visibility and more adaptive visual information suited to the work situation, particularly advanced unmanned construction. Toward that end, we propose a method for autonomously controlling multiple environmental cameras. Situations in which the yaw, pitch, and zoom of cameras should be controlled are analyzed. Additionally, we define imaging objects, including the machine, manipulators, and end points; and imaging modes, including tracking, zoom, posture, and trajectory modes. To control each camera simply and effectively, four practical camera roles with different combinations of the imaging objects and modes were defined: overview machine, enlarge end point, posture‐manipulator, and trajectory‐manipulator. A real‐time role assignment system is described for assigning the four camera roles to four out of six cameras suitable for the work situation (e.g., reaching, grasping, transport, and releasing) on the basis of the assignment‐priority rules. To test this system, debris‐removal tasks were performed in a virtual reality simulation to compare performance among fixed camera, manual control camera, and autonomous control camera systems. The results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and erroneous contacts and simultaneously increasing the subjective usability and time efficiency.     

Measuring consistency of web page design and its effects on performance and satisfaction

Symptoms of upper extremity cumulative trauma disorders (CTDs) often include weakness, discomfort, pain, numbness and stiffness, which are generally assessed clinically by using static tests or isolated movements. Little is known about the dynamic, functional ability of the upper extremity in CTD, yet, more than impairment, performance variables may relate to disability. The objectives of this study were to determine whether a manual tracking task was sensitive to the presence of symptoms associated with CTD and whether tracking performance related to disability. Forty-five volunteers who had frequently experienced one or more symptoms consistent with upper extremity CTD for at least 1 year and 22 control subjects performed the manual tracking task. Using a hand-held stylus over a digitizing tablet, subjects tracked a target that moved pseudo-randomly and was displayed on a computer screen. The root mean square error of the linear difference between target and stylus positions provided a measure of overall performance accuracy. Quadrant specific performance was also calculated to determine whether the location of the target (hence hand and wrist position) influenced performance. Additionally, the symptomatic group completed the Disability of the Arm, Shoulder and Hand (DASH) questionnaire reflecting physical disability level. Performance accuracy was poorer in symptomatic subjects than controls (p<0.001) and was influenced by target location (p<0.0001). The overall performance was associated with physical disability (r = 0.54). The findings suggest that tracking performance is sensitive to the presence of CTD symptoms and related to disability level. Further validation is required to determine whether the performance measure is sensitive to disease progression or intervention-induced changes.     

基于任务函数方法的机器人视觉伺服特性研究

为实现在统一的理论框架下对机器人视觉伺服基础特性进行细致深入的研究,本文基于任务函数方法,建立了广义的视觉伺服系统模型.在此模型基础之上,重点研究了基于位置的视觉伺服(PBVS)与基于图像的视觉伺服(IBVS)方法在笛卡尔空间和图像空间的动态特性.仿真结果表明,在相同的比较框架结构下,PBVS方法同样对摄像机标定误差具有鲁棒性.二者虽然在动态系统的稳定性、收敛性方面相类似,但是在笛卡尔空间和图像空间的动态性能上却有很大的差别.对于PBvS方法,笛卡尔轨迹可以保证最短路径,但是对应的图像轨迹是不可控的,可能会发生逃离视线的问题;对于IBVS方法,图像空间虽然能保证最短路径,但是由于缺乏笛卡尔空间的直接控制,在处理大范围旋转伺服的情况时,会发生诸如摄像机退化的笛卡尔轨迹偏移现象.     

Real-Time Feature Matching in Image Sequences for Non-Structured Environments. Applications to Vehicle Guidance

This paper describes an integrated vehicle control system with visual feedback. A general-purpose, low-level feature matching method, able to work in real time without any strict assumptions on the environment structure or camera parameters, generates low-level matching results, which are used as source of data for applications like mobile object tracking, among others. A generalized predictive path-tracking control approach keeps the vehicle on the trajectory defined by the moving target. In the low-level matching process, block-based features (windows) are selected and tracked along a stream of monocular images; least residual square error and similarity between clusters of features are used as constraints to select the right matching pair between multiple candidates. Real-time performance is achieved through optimized algorithms and a parallel DSP-based multiprocessor system implementation. Object detection and tracking is motion-based, and does not require a predefined model of the target. The integrated control system has been tested on the ROMEO-3R experimental vehicle.     

基于双目立体视觉的船舶轨迹跟踪算法研究

双目立体视觉模型通过模拟人眼可以实现对目标距离的测量。为了获得水上船舶实时的运动状态,提出了一种基于双目立体视觉的船舶轨迹跟踪方法。首先,通过摄像机标定、线性空间点三维重建可以测得以相机为中心到船舶的距离,得到船舶的部分运动轨迹;其次,在双目立体视觉测距系统的基础上采用常速(Constant Velocity,CV)模型的方法对船舶运动建模;最后,对建立的船舶运动模型利用强跟踪卡尔曼滤波(Strong Tracking Kalman Filter,STKF)船舶轨迹跟踪的方法跟踪船舶的轨迹并估算目标船舶实时的运动状态。实验结果表明,基于双目立体视觉的船舶轨迹跟踪的方法能有效地跟踪船舶轨迹且适用于工程应用的需求。     

一种基于弹道曲线模型的脱靶量测试方法

针对传统的脱靶量测试设备体积庞大、不便移动且测试成本高等缺点,提出了一种基于弹道曲线模型的双目立体视觉的脱靶量测试方法。研究了摄像机的成像模型及坐标系变换,分析了摄像机标定的一般方法,构建了摄像机标定模型,并推导了标定参数的表达式。基于双目立体视觉测试技术设计了弹道曲线模型的弹丸脱靶量测量系统,其测量过程简单且标定结果稳定,可实现弹丸运动目标的快速检测,通过对运动目标的空间定位和轨迹拟合实现了脱靶量的测试。实验结果表明:弹道直线模型和曲线模型均能简便地求解飞行弹丸脱靶量,后者比前者更贴近实际弹丸飞行轨迹,可获得更高精度的脱靶量,弹道曲线模型的脱靶量平均绝对误差(MAE)比直线模型降低了大约一半。     

机载雷达组网多站测角被动跟踪轨迹优化研究

研究多观测器轨迹优化控制问题,由于多站测角被动跟踪系统运行存在误差,用机载雷达组网的可移动传感器采集信息,可对雷达载体轨迹优化进行研究,利用控制雷达载体的飞行轨迹可有效解决跟踪目标的弱观测性及估计器的稳定性。为了改善传统轨迹优化算法容易陷入早熟收敛和局部最小的问题,提出一种模拟退火(Simulated Annealing,SA)和粒子群优化(Particle Swarm Optimization,PSO)算法的混合优化方法(SA-PSO)。在给出了角度信息的适应度函数表达式基础上,结合模拟退火算法的局部搜索能力和粒子群优化算法的全局搜索能力,提高优化算法的收敛速度、精度以及全局搜索能力。实验证明,改进的混合算法对雷达载体轨迹优化有效,并减小对机动目标的被动跟踪误差。     

Transverse function control with prescribed performance guarantees for underactuated marine surface vehicles

This paper studies the problem of stabilizing reference trajectories (also called as the trajectory tracking problem) for underactuated marine vehicles under predefined tracking error constraints. The boundary functions of the predefined constraints are asymmetric and time‐varying. The time‐varying boundary functions allow us to quantify prescribed performance of tracking errors on both transient and steady‐state stages. To overcome difficulties raised by underactuation and nonzero off‐diagonal terms in the system matrices, we develop a novel transverse function control approach to introduce an additional control input in backstepping procedure. This approach provides practical stabilization of any smooth reference trajectory, whether this trajectory is feasible or not. By practical stabilization, we mean that the tracking errors of vehicle position and orientation converge to a small neighborhood of zero. With the introduction of an error transformation function, we construct an inverse‐hyperbolic‐tangent‐like barrier Lyapunov function to show practical stability of the closed‐loop systems with prescribed transient and steady‐state performances. To deal with unmodeled dynamic uncertainties and external disturbances, we employ neural network (NN) approximators to estimate uncertain dynamics and present disturbance observers to estimate unknown disturbances. Subsequently, we develop adaptive control, based on NN approximators and disturbance estimates, that guarantees the prescribed performance of tracking errors during the transient stage of on‐line NN weight adaptations and disturbance estimates. Simulation results show the performance of the proposed tracking control.     

基于深度学习的多旋翼无人机单目视觉目标定位追踪方法

针对无人机对目标的识别定位与跟踪,本文提出了一种基于深度学习的多旋翼无人机单目视觉目标识别跟踪方法,解决了传统的基于双目摄像机成本过高以及在复杂环境下识别准确率较低的问题。该方法基于深度学习卷积神经网络的目标检测算法,使用该算法对目标进行模型训练,将训练好的模型加载到搭载ROS的机载电脑。机载电脑外接单目摄像机,单目摄像头检测目标后,自动检测出目标在图像中的位置,通过采用一种基于坐标求差的优化算法进行目标位置准确获取,然后将目标位置信息转化为控制无人机飞行的期望速度和高度发送给飞控板,飞控板接收到机载电脑发送的跟踪指令,实现对目标物体的跟踪。试验结果验证了该方法可以很好的进行目标识别并实现目标追踪