Laws of attraction: from perceptual forces to conceptual similarity

Many of the pressing questions in information visualization deal with how exactly a user reads a collection of visual marks as information about relationships between entities. Previous research has suggested that people see parts of a visualization as objects, and may metaphorically interpret apparent physical relationships between these objects as suggestive of data relationships. We explored this hypothesis in detail in a series of user experiments. Inspired by the concept of implied dynamics in psychology, we first studied whether perceived gravity acting on a mark in a scatterplot can lead to errors in a participant's recall of the mark's position. The results of this study suggested that such position errors exist, but may be more strongly influenced by attraction between marks. We hypothesized that such apparent attraction may be influenced by elements used to suggest relationship between objects, such as connecting lines, grouping elements, and visual similarity. We further studied what visual elements are most likely to cause this attraction effect, and whether the elements that best predicted attraction errors were also those which suggested conceptual relationships most strongly. Our findings show a correlation between attraction errors and intuitions about relatedness, pointing towards a possible mechanism by which the perception of visual marks becomes an interpretation of data relationships.     

Embodied categorisation for vision-guided mobile robots

This paper outlines a philosophical and psycho-physiological basis for embodied perception, and develops a framework for conceptual embodiment of vision-guided robots. We argue that categorisation is important in all stages of robot vision. Further, that classical computer vision is unsuitable for this categorisation, however, through conceptual embodiment, active perception can be effective. We present a methodology for developing vision-guided robots that applies embodiment, explicitly and implicitly, in categorising visual data to facilitate efficient perception and action. Finally, we present systems developed using this methodology, and demonstrate that embodied categorisation can make algorithms more efficient and robust.     

Communicating emotions and mental states to robots in a real time parallel framework using Laban movement analysis

This paper presents a parallel real time framework for emotions and mental states extraction and recognition from video fragments of human movements. In the experimental setup human hands are tracked by evaluation of moving skin-colored objects. The tracking analysis demonstrates that acceleration and frequency characteristics of the traced objects are relevant for classification of the emotional expressiveness of human movements. The outcomes of the emotional and mental states recognition are cross-validated with the analysis of two independent certified movement analysts (CMA’s) who use the Laban movement analysis (LMA) method. We argue that LMA based computer analysis can serve as a common language for expressing and interpreting emotional movements between robots and humans, and in that way it resembles the common coding principle between action and perception by humans and primates that is embodied by the mirror neuron system. The solution is part of a larger project on interaction between a human and a humanoid robot with the aim of training social behavioral skills to autistic children with robots acting in a natural environment.     

Experiments on human pattern recognition: a hierarchical sign-system approach

Successive stages can be distinguished in the development of the human visual system's ability to use and recognize signs. The stages involve perception of parts of objects, of whole objects, of several objects, and of their interrelations. The system of signs described in this paper was developed through experimental investigations of visual perception in adults, children, and mentally ill or brain-damaged persons.     

How color break‐up occurs in the human‐visual system: The mechanism of the color break‐up phenomenon

Abstract— In the present set of experiments, we examined the mechanisms underlying color break‐up (CBU), a phenomenon observed when images produced with a color‐sequential projector are viewed. The perceived position of CBU was measured during fast eye movement, saccade with static and briefly flashed stimuli. Results showed that CBU did not simply correspond to the locus of the stimulus on the retina during saccades, because the width of the CBU perception was narrower than the distance of the eye movements. This effect was thought to be related to visual stability, which allows objects to be perceived as stationary even when the eyes move and the retinal image changes position. Visual stability is assumed to operate by compensating for the change in retinal image position using eye‐position signals; however, this compensation is imperfect during saccades. Thus, incomplete compensation results in a CBU perception that is of a narrower width than the amplitude of the saccade. In conclusion, CBU cannot be simulated with moving video cameras because it results largely from the mechanisms of visual perception. Large inter‐individual differences in perception of CBU were also found. This observation also supports the idea that CBU depends on the mechanism of perception.     

Predictive control architecture for real-time image moments based servoing of robot manipulators

This paper presents a real-time architecture for visual servoing of robot manipulators using nonlinear based predictive control. In order to increase the robustness of the control algorithm, image moments were chosen to be the visual features which describe the objects from the image. A visual predictive architecture is designed to solve tasks addressed to robot manipulators with an eye-in-hand configuration. An implementation of the proposed architecture is done so that the capabilities of a 6 d.o.f robot manipulator are extended. The results of different experiments conducted with two types of image moments based controllers (proportional and predictive with reference trajectory) are presented and discussed.     

An Exploration of Embodied Visual Exploration

International Journal of Computer Vision - Embodied computer vision considers perception for robots in novel, unstructured environments. Of particular importance is the embodied visual exploration...     

轮廓矩不变量及其在物体形状识别中的应用

为了有效地刻画物体的形状特征，在基于区域的Hu矩不变量的基础上，构造了一种基于物体轮廓曲线的新的矩不变量，即轮廓矩不变量。该不变量不仅独立于物体本身的颜色和灰度级，而且具有平移、旋转和尺度不变性，因此可将轮廓矩不变量应用于物体形状的识别，为了能快速地进行物体形状识别，还讨论了小波边缘检测和轮廓的获取问题及其算法。实验表明，基于这种轮廓矩的识别算法具有很好的识别率。     

Research on multi-camera information fusion method for intelligent perception

Multimedia Tools and Applications - In this paper, the Gaussian Mixture Model and Mean Shift algorithm are used to detect and track moving objects in the visual perception network composed of...     

Transferring knowledge across robots: A risk sensitive approach

One of the most impressive characteristics of human perception is its domain adaptation capability. Humans can recognize objects and places simply by transferring knowledge from their past experience. Inspired by that, current research in robotics is addressing a great challenge: building robots able to sense and interpret the surrounding world by reusing information previously collected, gathered by other robots or obtained from the web. But, how can a robot automatically understand what is useful among a large amount of information and perform knowledge transfer? In this paper we address the domain adaptation problem in the context of visual place recognition. We consider the scenario where a robot equipped with a monocular camera explores a new environment. In this situation traditional approaches based on supervised learning perform poorly, as no annotated data are provided in the new environment and the models learned from data collected in other places are inappropriate due to the large variability of visual information. To overcome these problems we introduce a novel transfer learning approach. With our algorithm the robot is given only some training data (annotated images collected in different environments by other robots) and is able to decide whether, and how much, this knowledge is useful in the current scenario. At the base of our approach there is a transfer risk measure which quantifies the similarity between the given and the new visual data. To improve the performance, we also extend our framework to take into account multiple visual cues. Our experiments on three publicly available datasets demonstrate the effectiveness of the proposed approach.     

Accumulation of object representations utilising interaction of robot action and perception

We introduce a robotic-vision system which is able to extract object representations autonomously utilising a tight interaction of visual perception and robotic action within a perception action cycle [Ecological Psychology 4 (1992) 121; Algebraic Frames for the Perception and Action Cycle, 1997, 1]. Controlled movement of the object grasped by the robot enables us to compute the transformations of entities which are used to represent aspects of objects and to find correspondences of entities within an image sequence.A general accumulation scheme allows to acquire robust information from partly missing information extracted from single frames of an image sequence. Here we use this scheme with a preprocessing stage in which 3D-line segments are extracted from stereo images. However, the accumulation scheme can be used with any kind of preprocessing as long as the entities used to represent objects can be brought to correspondence by certain equivalence relations such as ‘rigid body motion’.We show that an accumulated representation can be applied within a tracking algorithm. The accumulation scheme is an important module of a vision based robot system on which we are currently working. In this system, objects are planned to be represented by different visual and tactile entities. The object representations are going to be learned autonomously. We discuss the accumulation scheme in the context of this project.     

Image-based visual servoing using improved image moments in 6-DOF robot systems

This paper addresses the challenges of choosing proper image features for planar symmetric shape objects and designing visual servoing controller to enhance the tracking performance in image-based visual servoing (IBVS). Six image moments are chosen as the image features and the analytical image interaction matrix related to the image features are derived. A controller is designed to efficiently increase the robustness of the visual servoing system. Experimental results on a 6-DOF robot visual servoing system are provided to illustrate the effectiveness of the proposed method.     

一种新的人眼视觉运动认知特性实验分析方法

视觉感知过程源于对象自身的大范围拓扑性质,而场景对象发生运动状态变化时,其所在空间的大范围拓扑关系也将随之改变。为探索大范围拓扑性质和运动性质同时改变时人眼视觉的认知优先特性,基于虚拟现实技术及眼动跟踪技术,通过构建虚拟驾驶及眼动轨迹分析实验平台,结合拓扑复杂度和空间拓扑关系概念分别探讨二维和三维空间中人眼视知觉对物体运动的认知特性。在此基础上,采用响应面分析方法设计场景中自变量因子的随机实验数值,通过可视化方法对比多组认知场景的无感反馈眼动监测结果。实验结果表明,在Tobii眼动仪采集的545个眼动数据中,发生在运动物体上的数据有516个,相比于物体自身属性的大范围拓扑性质变化,人眼视知觉过程更优先识别物体运动的大范围空间拓扑关系变化。该运动认知优先特性为可靠人工智能驾驶中的视觉注视预测技术提供了新的研究思路,同时验证了所提实验分析方法的有效性。     

Animating Causal Overlays

Most approaches to representing causality, such as the common causal graph, require a separate and static view, but in many cases it is useful to add the dimension of causality to the context of an existing visualization. Building on research from perceptual psychology that shows the perception of causality is a low‐level visual event derived from certain types of motion, we are investigating how to add animated causal representations, called visual causal vectors, onto other visualizations. We refer to these as causal overlays. Our initial experimental results show this approach has great potential but that extra cues are needed to elicit the perception of causality when the motions are overlaid on other graphical objects. In this paper we describe the approach and report on a study that examined two issues of this technique: how to accurately convey the causal flow and how to represent the strength of the causal effect.     

视觉感知正反馈的显著性检测

目的 人类视觉系统性能远超当前机器视觉,模拟人类视觉机制改进当前算法是有效研究途径,为此提出一种视觉感知正反馈模型,通过循环迭代、重复叠加视觉刺激生成更符合人类感知的视觉显著性图。方法 首先用多种常规方法检测图像显著度,模拟人类视觉多通道特性,再组合这些显著图为综合显著图;利用显著度大的像素构建初始注视区。其次借助集成RVFL（随机向量功能网络）模拟人脑神经网络产生视觉刺激,对注视与非注视区内像素在线“随机采样—学习建模”,图像像素经模型分类获得新注视区。对新注视区与非注视区,可重复迭代进行“随机采样—学习建模—像素分类”;迭代中若注视区连续相同,则表明感知饱和,迭代终止。若将每次像素分类结果看做是一种视觉刺激,则多次视觉刺激输出叠加,可生成新的图像显著性图。最终的像素分类结果就是图像分割目标。结果 将本文算法与现有方法在标准图像数据库上进行对比评测,包括通过对6种算法在ECSSD、SED2和MSRA10K 3个图像数据库上的P-R曲线,F-measure值和平均绝对误差（MAE）值上进行定量分析,对6种模型生成的显著性图作定性比较。数据表明,本文算法在SED2和MSRA10K图象数据库中性能最好,在ECSSD图象数据库中稍低于BL（bootstrap learning）和RBD（robust background detection）算法。本文算法的显著图与人类视觉感知更接近。且算法的正反馈迭代过程一般可迅速饱和,并未显著增加算法负担。实验结果表明,本文方法可作为一种有效的后处理手段,显著提升常规显著性检测算法的性能。结论 提出了一种模拟人类视觉机制的数据驱动显著性检测算法,无需图像先验知识和事先的标记样本。面对多目标,背景复杂等情况,本文方法具有相对好的鲁棒性和适用性,并且能够较好解决现实环境中图像处理算法的通用性、可靠性和准确性问题。     

Support relation analysis and decision making for safe robotic manipulation tasks

In this article, we describe an approach to address the issue of automatically building and using high-level symbolic representations that capture physical interactions between objects in static configurations. Our work targets robotic manipulation systems where objects need to be safely removed from piles that come in random configurations. We assume that a 3D visual perception module exists so that objects in the piles can be completely or partially detected. Depending on the outcome of the perception, we divide the issue into two sub-issues: (1) all objects in the configuration are detected; (2) only a subset of objects are correctly detected. For the first case, we use notions from geometry and static equilibrium in classical mechanics to automatically analyze and extract act and support relations between pairs of objects. For the second case, we use machine learning techniques to estimate the probability of objects supporting each other. Having the support relations extracted, a decision making process is used to identify which object to remove from the configuration so that an expected minimum cost is optimized. The proposed methods have been extensively tested and validated on data sets generated in simulation and from real world configurations for the scenario of unloading goods from shipping containers.     

A VE framework to study visual perception and action

In the real world, vision operates in harmony with self-motion yielding the observer to unambiguous perception of the three-dimensional (3D) space. In laboratory conditions, because of technical difficulties, researchers studying 3D perception have often preferred to use the substitute of a stationary observer, somehow neglecting aspects of the action-perception cycle. Recent results in visual psychophysics have proved that self-motion and visual processes interact, leading the moving observer to interpret a 3D virtual scene differently from a stationary observer. In this paper we describe a virtual environment (VE) framework which presents very interesting characteristics for designing experiments in visual perception during action. These characteristics arise in a number of ways from the design of a unique motion capture device. First, its accuracy and the minimal latency in position measurement; second, its ease of use and the adaptability to different display interfaces. Such a VE framework enables the experimenter to recreate stimulation conditions characterised by a degree of sensory coherence typical of the real world. Moreover, because of its accuracy and flexibility, the same device can be used as a measurement tool to perform elementary but essential calibration procedures. The VE framework has been used to conduct two studies which compare the perception of 3D variables of the environment in moving and in stationary observers under monocular vision. The first study concerns the perception of absolute distance, i.e. the distance separating an object and the observer. The second study refers to the perception of the orientation of a surface both in the absence and presence of conflicts between static and dynamic visual cues. In the two cases, the VE framework has enabled the design of optimal experimental conditions, permitting light to be shed on the role of action in 3D visual perception.     

Stereoscopic human interfaces

This article focuses on the use of stereoscopic video interfaces for telerobotics. Topics concerning human visual perception, binocular image capturing, and stereoscopic devices are described. There is a wide variety of video interfaces for telerobotic systems [1]. Choosing the best video interface depends on the telerobotic application requirements. Simple monoscopic cameras are good enough for watching remote robot movements or for teleprogramming a sequence of commands. However, when operators seek precise robot guidance or wish to manipulate objects, a better perception of the remote environment must be achieved, for which more advanced visual interfaces are required. This implies a higher degree of telepresence, and, therefore, the most suitable visual interface has to be chosen. The aim of this article is to describe the two main aspects using stereoscopic interfaces: · the capture of binocular video images, according to the disparity limits in human perception · the proper selection of the visualization interface for stereoscopic images.     

A General Illumination Model for Molecular Visualization

Several visual representations have been developed over the years to visualize molecular structures, and to enable a better understanding of their underlying chemical processes. Today, the most frequently used atom‐based representations are the Space‐filling, the Solvent Excluded Surface, the Balls‐and‐Sticks, and the Licorice models. While each of these representations has its individual benefits, when applied to large‐scale models spatial arrangements can be difficult to interpret when employing current visualization techniques. In the past it has been shown that global illumination techniques improve the perception of molecular visualizations; unfortunately existing approaches are tailored towards a single visual representation. We propose a general illumination model for molecular visualization that is valid for different representations. With our illumination model, it becomes possible, for the first time, to achieve consistent illumination among all atom‐based molecular representations. The proposed model can be further evaluated in real‐time, as it employs an analytical solution to simulate diffuse light interactions between objects. To be able to derive such a solution for the rather complicated and diverse visual representations, we propose the use of regression analysis together with adapted parameter sampling strategies as well as shape parametrization guided sampling, which are applied to the geometric building blocks of the targeted visual representations. We will discuss the proposed sampling strategies, the derived illumination model, and demonstrate its capabilities when visualizing several dynamic molecules.