An optimal feedback control framework for grasping objects with position uncertainty

As we move, the relative location between our hands and objects changes in uncertain ways due to noisy motor commands and imprecise and ambiguous sensory information. The impressive capabilities humans display for interacting and manipulating objects with position uncertainty suggest that our brain maintains representations of location uncertainty and builds compensation for uncertainty into its motor control strategies. Our previous work demonstrated that specific control strategies are used to compensate for location uncertainty. However, it is an open question whether compensation for position uncertainty in grasping is consistent with the stochastic optimal feedback control, mainly due to the difficulty of modeling natural tasks within this framework. In this study, we develop a stochastic optimal feedback control model to evaluate the optimality of human grasping strategies. We investigate the properties of the model through a series of simulation experiments and show that it explains key aspects of previously observed compensation strategies. It also provides a basis for individual differences in terms of differential control costs-the controller compensates only to the extent that performance benefits in terms of making stable grasps outweigh the additional control costs of compensation. These results suggest that stochastic optimal feedback control can be used to understand uncertainty compensation in complex natural tasks like grasping.     

Gaze Interaction With Vibrotactile Feedback: Review and Design Guidelines

Vibrotactile feedback is widely used in mobile devices because it provides a discreet and private feedback channel. Gaze-based interaction, on the other hand, is useful in various applications due to its unique capability to convey the focus of interest. Gaze input is naturally available as people typically look at things they operate, but feedback from eye movements is primarily visual. Gaze interaction and the use of vibrotactile feedback have been two parallel fields of human–computer interaction research with a limited connection. Our aim was to build this connection by studying the temporal and spatial mechanisms of supporting gaze input with vibrotactile feedback. The results of a series of experiments showed that the temporal distance between a gaze event and vibrotactile feedback should be less than 250 ms to ensure that the input and output are perceived as connected. The effectiveness of vibrotactile feedback was largely independent of the spatial body location of vibrotactile actuators. In comparison to other modalities, vibrotactile feedback performed equally to auditory and visual feedback. Vibrotactile feedback can be especially beneficial when other modalities are unavailable or difficult to perceive. Based on the findings, we present design guidelines for supporting gaze interaction with vibrotactile feedback.     

A HUMANLIKE GRASPING FORCE PLANNER FOR OBJECT MANIPULATION BY ROBOT MANIPULATORS

Recently robot manipulators have been expected to perform sophisticated tasks such as object manipulation, assembly tasks, or cooperative tasks with human workers. In order to realize these tasks with robot manipulators, it is important to understand the human strategy of object grasping and manipulation. In this study, we have examined how a human being decides the grasping force necessary to manipulate an unknown object in order to apply human object-grasping strategy for robotic systems. Experiments have been performed with several kinds of objects under several kinds of conditions to investigate how much grasping force human subjects generate. Adjustment strategy of human grasping force when the object is manipulated or in contact with an environment is also examined. Neural networks (the desired grasping force planner) that generate the humanlike desired grasping force are then designed for robotic systems. The effectiveness of the proposed desired grasping force planner is evaluated via experiments.     

Control of a 2 DoF robot using a Brain–Machine Interface

In this paper, a non-invasive spontaneous Brain–Machine Interface (BMI) is used to control the movement of a planar robot. To that end, two mental tasks are used to manage the visual interface that controls the robot. The robot used is a PupArm, a force-controlled planar robot designed by the nBio research group at the Miguel Hernández University of Elche (Spain). Two control strategies are compared: hierarchical and directional control. The experimental test (performed by four users) consists of reaching four targets. The errors and time used during the performance of the tests are compared in both control strategies (hierarchical and directional control). The advantages and disadvantages of each method are shown after the analysis of the results. The hierarchical control allows an accurate approaching to the goals but it is slower than using the directional control which, on the contrary, is less precise. The results show both strategies are useful to control this planar robot. In the future, by adding an extra device like a gripper, this BMI could be used in assistive applications such as grasping daily objects in a realistic environment. In order to compare the behavior of the system taking into account the opinion of the users, a NASA Tasks Load Index (TLX) questionnaire is filled out after two sessions are completed.     

A novel mixed reality assistive system to aid the visually and mobility impaired using a multimodal feedback system

Most of the people affected by vision impairments are over 50 years old, which amplifies cognitive impairments and the risk of falling. This paper proposes a mixed reality assistive system to aid the navigation of visually impaired people by using a Smart Walker, audio feedback to provide cues to be followed, and vibratory feedback to indicate obstacles. To validate this system, 24 volunteers participated of three tasks consisting in a combination of a control strategy (Free Roam, Steering-Assisted Navigation, and Virtual Trail) and a path. To evaluate the experiments, task performance metrics and three questionnaires (Quality of Experience, SUS, and NASA-TLX) were assessed. According to SUS, all control strategies scored above 70, indicating their acceptability. Analyzing and comparing the questionnaires results and the strategies used, the Steering-Assisted Navigation, in which the volunteers had to follow a path indicated only by the haptic feedback provided by the controller, was the most physically and mentally demanding. The other two strategies required less effort, physical, and mental demand. Despite requiring different cognitive loads for navigation, all control strategies combined with the multimodal sensory feedback guided the volunteers during the tasks, evidencing the feasibility of the system.     

Navigation by vibration: Effects of vibrotactile feedback on a navigation task

In the background of spatial orientation in a navigation task, this study investigated the effect of frequency, duration and amplitude of vibrotactile feedback when it provided primary information modality. Multiple levels of each parameter were designed for an experiment conducted with 18 participants. Their performance was evaluated via number of errors, task completion time, annoyance level, and user preference. Result showed that medium level of frequency and duration was more preferred and can produce better performance. However, optimal amplitude level varied by individuals and also interacted with frequency. The paper summarized a set of design guidelines, which could be used to the design of future user interface with vibrotactile feedback. The study should provide great empirical data and meaningful insight for the design of vibrotactile feedback for future applications.Relevance to industryThe paper evaluated the vibrotactile interfaces and summarized a set of design guidelines, which could help to speed up the commercialization and industrial application of vibrotactile user interface.     

多自由度仿生假手嵌入式控制系统及其抓取策略

为一种能够实现5 指独立动作以及具备人机交互能力的多自由度仿生假手设计了手部嵌入式控制系 统．该系统由传感器系统和运动控制系统构成,集成于假手机体内部,通过通信总线与上层控制器交换信息．传感 器系统包括3 种类型,共12 个传感器,可为假手自主抓取以及人机交互中的感觉反馈提供数据,运动控制系统用于 控制、驱动各手指动作．此外,本文以基于位置的阻抗控制为底层,以动作预构形为上层设计了分层控制策略．实 验表明,该嵌入式控制系统和分层控制策略使假手实现了自主抓取功能,提高了抓取的柔顺性、稳定性和适应性．     

基于多约束条件的机器人抓取策略学习方法

针对机器人在多类别物体不同任务下的抓取决策问题,提出基于多约束条件的抓取策略学习方法.该方法以抓取对象特征和抓取任务属性为机器人抓取策略约束,通过映射人类抓取习惯规划抓取模式,并采用物体方向包围盒(OBB)建立机器人抓取规则,建立多约束条件的抓取模型.利用深度径向基(DRBF)网络模型结合减聚类算法(SCM)实现抓取策略的学习,两种算法的结合旨在提高学习鲁棒性与精确性.搭建以Refiex 1型灵巧手和AUBO六自由度机械臂组成的实验平台,对多类别物体进行抓取实验.实验结果表明,所提出方法使机器人有效学习到对多物体不同任务的最优抓取策略,具有良好的抓取决策能力.     

基于虚拟力引导的人机协同目标抓取方法

针对力反馈遥操作中传统人工势场法无法适应于机械臂整体的避障以及在作业过程中操作者难以控制机械臂到达所需位姿的问题,提出了一种基于虚拟力引导的人机协同目标抓取方法。力反馈设备向操作者提供力觉交互。通过结合人工势场法和虚拟夹具,构建管道形虚拟力场,生成实时虚拟力引导,实现协助操作者完成从端机器人的整体避障任务并在完成避障后引导机器人返回预定义路径并趋近目标点。当进行抓取任务时,构建锥形虚拟力场,实现协助操作者操作机械臂到达目标位置和姿态。此外,提出了一种机器人运动限制方法以降低操作者的操作失误对抓取任务的影响。实验证明,该方法能有效提高目标抓取操作的成功率和操作效率。     

Vibrotactile feedback to aid blind users of mobile guides

In this work, we report on a solution for providing support to the blind using mobile museum guides by exploiting the haptic channel as a complement to the audio/vocal one. The overall goal is to improve the autonomy and social integration of blind visitors. We followed an iterative approach in which the proposed system went through various user evaluations and further refinements. The final solution includes vibrotactile feedback enhancement for orientation and obstacle avoidance obtained through the use of unobtrusive actuators applied to two of the user's fingers combined with an electronic compass and obstacle detector sensors connected wirelessly to the mobile guide. Our study indicates that vibrotactile feedback is particularly useful to provide frequent unobtrusive indications of useful dynamic information, such as the level of proximity of an obstacle or the distance from the right orientation.     

The Effect of Pairs in Program Design Tasks

Pair programming involves-two developers simultaneously collaborating with each other on the same programming task to design and code a solution. Algorithm design and its implementation are normally interwoven in that implementation often provides feedback to enhance the design. Previous controlled pair programming experiments did not explore the efficacy of pairs versus individuals in program design-related tasks separately from coding. Variations in programmer skills in a particular language or an integrated development environment and the understanding of programming instructions can mask the skill of subjects in program design-related tasks. Programming aptitude tests (PATs) have been shown to correlate with programming performance. PATs do not require understanding of programming instructions and do not require a skill in any specific computer language. Two controlled experiments were conducted, with full-time professional programmers being the subjects who worked on increasingly complex programming aptitude tasks related to problem solving and algorithmic design. In both experiments, pairs significantly outperformed individuals, providing evidence of the value of pairs in program design-related tasks.     

Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid

Hierarchical inverse dynamics based on cascades of quadratic programs have been proposed for the control of legged robots. They have important benefits but to the best of our knowledge have never been implemented on a torque controlled humanoid where model inaccuracies, sensor noise and real-time computation requirements can be problematic. Using a reformulation of existing algorithms, we propose a simplification of the problem that allows to achieve real-time control. Momentum-based control is integrated in the task hierarchy and a LQR design approach is used to compute the desired associated closed-loop behavior and improve performance. Extensive experiments on various balancing and tracking tasks show very robust performance in the face of unknown disturbances, even when the humanoid is standing on one foot. Our results demonstrate that hierarchical inverse dynamics together with momentum control can be efficiently used for feedback control under real robot conditions.     

MM-UAV: Mobile Manipulating Unmanned Aerial Vehicle

Given significant mobility advantages, UAVs have access to many locations that would be impossible for an unmanned ground vehicle to reach, but UAV research has historically focused on avoiding interactions with the environment. Recent advances in UAV size to payload and manipulator weight to payload ratios suggest the possibility of integration in the near future, opening the door to UAVs that can interact with their environment by manipulating objects. Therefore, we seek to investigate and develop the tools that will be necessary to perform manipulation tasks when this becomes a reality. We present our progress and results toward a design and physical system to emulate mobile manipulation by an unmanned aerial vehicle with dexterous arms and end effectors. To emulate the UAV, we utilize a six degree-of-freedom miniature gantry crane that provides the complete range of motion of a rotorcraft as well as ground truth information without the risk associated with free flight. Two four degree-of-freedom manipulators attached to the gantry system perform grasping tasks. Computer vision techniques and force feedback servoing provide target object and manipulator position feedback to the control hardware. To test and simulate our system, we leverage the OpenRAVE virtual environment and ROS software architecture. Because rotorcraft are inherently unstable, introduce ground effects, and experience changing flight dynamics under external loads, we seek to address the difficult task of maintaining a stable UAV platform while interacting with objects using multiple, dexterous arms. As a first step toward that goal, this paper describes the design of a system to emulate a flying, dexterous mobile manipulator.     

Spatialized Vibrotactile Feedback Improves Goal-Directed Movements in Cluttered Virtual Environments

Spatial awareness in virtual reality (VR) is a dominant research topic. It plays an essential role in the assessment of human operators’ behavior in simulated tasks, notably for the evaluation of the feasibility of manual maintenance tasks in cluttered industrial settings. In such contexts, it is decisive to evaluate the spatial and temporal correspondence between the operator’s movement kinematics and that of his/her virtual avatar in the virtual environment (VE). Often, in a cluttered VE, direct kinesthetic (force) feedback is limited or absent. We tested whether vibrotactile (cutaneous) feedback would increase visuo-proprioceptive consistency, spatial awareness, and thus the validity of VR studies, by augmenting the perception of the operator’s contact(s) with virtual objects. We present experimental results obtained using a head-mounted display (HMD) during a goal-directed task in a cluttered VE. Results suggest the contribution of spatialized vibrotactile feedback to visuo-proprioceptive consistency.     

Robotic grasping and manipulation through human visuomotor learning

A major goal of robotics research is to develop techniques that allow non-experts to teach robots dexterous skills. In this paper, we report our progress on the development of a framework which exploits human sensorimotor learning capability to address this aim. The idea is to place the human operator in the robot control loop where he/she can intuitively control the robot, and by practice, learn to perform the target task with the robot. Subsequently, by analyzing the robot control obtained by the human, it is possible to design a controller that allows the robot to autonomously perform the task. First, we introduce this framework with the ball-swapping task where a robot hand has to swap the position of the balls without dropping them, and present new analyses investigating the intrinsic dimension of the ball-swapping skill obtained through this framework. Then, we present new experiments toward obtaining an autonomous grasp controller on an anthropomorphic robot. In the experiments, the operator directly controls the (simulated) robot using visual feedback to achieve robust grasping with the robot. The data collected is then analyzed for inferring the grasping strategy discovered by the human operator. Finally, a method to generalize grasping actions using the collected data is presented, which allows the robot to autonomously generate grasping actions for different orientations of the target object.     

考虑邻域结构动态调整的多星应急调度算法

实际应用中的卫星调度方案往往受到外界因素的影响,例如电磁干扰、卫星失效、云层遮挡以及新任务的动态到达等,需要在短时间内对原始调度方案进行调整并生成新的调度方案,以保证卫星系统的稳定性.对此,考虑新任务到达情况下的卫星应急调度,建立多星协同应急调度的整数规划模型,并提出一种基于动态邻域结构的卫星应急调度算法(satellite emergency scheduling algorithm based on dynamic neighborhood, SESA-DN).设计多种类型的邻域结构以及动态的邻域选择策略,能够根据任务完成情况进行有效反馈,通过对应急任务的插入与替换生成卫星应急调度方案;同时,构建多星应急调度场景,通过大量仿真实验将SESA-DN算法与多种对比算法进行比较.实验结果表明, SESA-DN算法的稳定性优于对比算法,对于多星协同动态调度问题具有很好的适用性.     

Improving virtual reality navigation tasks using a haptic vest and upper body tracking

Haptic feedback is an important component of immersive virtual reality (VR) applications that is often suggested to complement visual information through the sense of touch. This paper investigates the use of a haptic vest in navigation tasks. The haptic vest produces a repulsive vibrotactile feedback from nearby static virtual obstacles that augments the user spatial awareness. The tasks require the user to perform complex movements in a 3D cluttered virtual environment, like avoiding obstacles while walking backwards and pulling a virtual object. The experimental setup consists of a room-scale environment. Our approach is the first study where a haptic vest is tracked in real time using a motion capture device so that proximity-based haptic feedback can be conveyed according to the actual movement of the upper body of the user.User study experiments have been conducted with and without haptic feedback in virtual environments involving both normal and limited visibility conditions. A quantitative evaluation was carried out by measuring task completion time and error (collision) rate. Multiple haptic rendering techniques have also been tested. Results show that under limited visibility conditions proximity-based haptic feedback generated by a wearable haptic vest can significantly reduce the number of collisions with obstacles in the virtual environment.     

基于3D目标跟踪算法的机器人手眼协调研究

为了适应环境的复杂性和多样性,增强机器人抓取任务的鲁棒性,本文从3D目标跟踪算法出发,提出了一种实现机器人手眼协调的新方法。该方法采用改进的基于区域的位姿追踪算法同时跟踪机械臂夹持器和目标物体的位姿,根据二者的相对位置关系引导机械臂运动。对基于区域的位姿跟踪算法,本文提出根据局部区域分割线构建分割模型并改进模型颜色似然的线性更新方式,使得算法能够准确跟踪机械臂夹持器与目标物体。基于ROS平台搭建了一套仿真实验环境,并分别在仿真环境和真实环境下验证了此手眼协调系统的有效性和鲁棒性。这种方式不仅不需要手眼标定,更接近于人类“Sensor-Actor”带反馈的闭环控制方式,同时赋予了机器人足够的灵活性来应对弹性的任务和多变的环境。     

Assisting people with visual impairments in aiming at a target on a large wall-mounted display

Large interactive displays have become ubiquitous in our everyday lives, but these displays are designed for the needs of sighted people. In this paper, we specifically address assisting people with visual impairments to aim at a target on a large wall-mounted display. We introduce a novel haptic device, which explores the use of vibrotactile feedback in blind user search strategies on a large wall-mounted display. Using mid-air gestures aided by vibrotactile feedback, we compared three target-aiming techniques: Random (baseline) and two novel techniques – Cruciform and Radial. The results of our two experiments show that visually impaired participants can find a target significantly faster with the Cruciform and Radial techniques than with the Random technique. In addition, they can retrieve information on a large display about twice as fast by augmenting speech feedback with haptic feedback in using the Radial technique. Although a large number of studies have been done on assistive interfaces for people who have visual impairments, very few studies have been done on large vertical display applications for them. In a broader sense, this work will be a stepping-stone for further research on interactive large public display technologies for users who are visually impaired.     

Dexterity and versatility in the pinching motion of robot fingers with multi-degrees of freedom

This paper focuses on dexterity and versatility in pinching a rectangular object by a pair of robot fingers based on sensory feedback. In the pinching motion of humans, it is possible to execute concurrent pinching and orientation control quickly and precisely by using only the thumb and index finger. However, it is not easy for robot fingers to perform such imposed tasks agilely and simultaneously. In the case of robotic grasping, to perform concurrently such plural tasks retards the convergence speed in the execution of the overall task. This means that in order to increase versatility by imposing additional tasks, dexterity in the execution of each task may deteriorate. In this paper it is shown that both dexterity and versatility in the execution of such imposed tasks can be enhanced remarkably, without any deterioration in dexterity in the execution of each task, by using a sensory feedback method based on the idea of role-sharing joint control which comes from observation of the functional role of each human finger joint.