Geometric preprocessing and neurocomputing for pattern recognition and pose estimation

This paper shows the analysis and design of feed-forward neural networks using the coordinate-free system of Clifford or geometric algebra. It is shown that real-, complex- and quaternion-valued neural networks are simply particular cases of the geometric algebra multidimensional neural networks and that they can be generated using Support Multi-Vector Machines. Particularly, the generation of RBF for neurocomputing in geometric algebra is easier using the SMVM that allows to find the optimal parameters automatically. The use of SVM in the geometric algebra framework expands its sphere of applicability for multidimensional learning.

We introduce a novel method of geometric preprocessing utilizing hypercomplex or Clifford moments. This method is applied together with geometric MLPs for tasks of 2D pattern recognition. Interesting examples of non-linear problems like the grasping of an object along a non-linear curve and the 3D pose recognition show the effect of the use of adequate Clifford or geometric algebras that alleviate the training of neural networks and that of Support Multi-Vector Machines.     

Sketching up the world: in situ authoring for mobile Augmented Reality

We present a novel system allowing in situ content creation for mobile Augmented Reality in unprepared environments. This system targets smartphones and therefore allows a spontaneous authoring while in place. We describe two different scenarios, which are depending on the size of the working environment and consequently use different tracking techniques. A natural feature-based approach for planar targets is used for small working spaces whereas for larger working environments, such as in outdoor scenarios, a panoramic-based orientation tracking is deployed. Both are integrated into one system allowing the user to use the same interaction for creating the content applying a set of simple, yet powerful modeling functions for content creation. The resulting content for Augmented Reality can be shared with other users using a dedicated content server or kept in a private inventory for later use.     

Object recognition and pose estimation for industrial applications: A cascade system

The research work presented in this paper focuses on the development of a 3D object localization and recognition system to be used in robotics conveyor coating lines. These requirements were specified together with enterprises with small production series seeking a full robotic automation of their production line that is characterized by a wide range of products in simultaneous manufacturing. Their production process (for example heat or coating/painting treatments) limits the use of conventional identification systems attached to the object in hand. Furthermore, the mechanical structure of the conveyor introduces geometric inaccuracy in the object positioning. With the correct classification and localization of the object, the robot will be able to autonomously select the right program to execute and to perform coordinate system corrections. A cascade system performed with Support Vector Machine and the Perfect Match (point cloud geometric template matching) algorithms was developed for this purpose achieving 99.5% of accuracy. The entire recognition and pose estimation procedure is performed in a maximum time range of 3 s with standard off the shelf hardware. It is expected that this work contributes to the integration of industrial robots in highly dynamic and specialized production lines.     

Invariant object recognition and pose estimation with slow feature analysis

Primates are very good at recognizing objects independent of viewing angle or retinal position, and they outperform existing computer vision systems by far. But invariant object recognition is only one prerequisite for successful interaction with the environment. An animal also needs to assess an object's position and relative rotational angle. We propose here a model that is able to extract object identity, position, and rotation angles. We demonstrate the model behavior on complex three-dimensional objects under translation and rotation in depth on a homogeneous background. A similar model has previously been shown to extract hippocampal spatial codes from quasi-natural videos. The framework for mathematical analysis of this earlier application carries over to the scenario of invariant object recognition. Thus, the simulation results can be explained analytically even for the complex high-dimensional data we employed.     

增强现实中的跟踪技术

增强现实是一种正在发展的新技术,它将由计算机产生的虚拟场景或信息准确叠加到真实环境中,可以增强用户对真实世界的感知与交互能力;对用户视场及视点的跟踪是增强现实中实现虚、实场景配准的关键技术之一;通过对增强现实中跟踪技术性能要求的讨论,介绍了基于磁场、声学、惯性、光学传感等多种跟踪技术,分析了各种方法的跟踪性能与局限性,认为以基于视觉跟踪为主的混合跟踪技术将是增强现实系统的主流跟踪技术,重点论述了基于视觉的跟踪与基于视觉一惯性的混合跟踪技术及其有待解决的问题,并针对户外增强现实中的跟踪技术做了专门讨论。     

Object Calibration for Augmented Reality

Augmented reality involves the use of models and their associated renderings to supplement information in a real scene. In order for this information to be relevant or meaningful, the models must be positioned and displayed in such a way that they align with their corresponding real objects. For practical reasons this alignment cannot be known a priori, and cannot be hard-wired into a system. Instead a simple, reliable alignment or calibration process is performed so that computer models can be accurately registered with their real-life counterparts. We describe the design and implementation of such a process and we show how it can be used to create convincing interactions between real and virtual objects.     

Comprehensible Visualization for Augmented Reality

This article presents interactive visualizations to support the comprehension of spatial relationships between virtual and real world objects for augmented reality (AR) applications. To enhance the clarity of such relationships we discuss visualization techniques and their suitability for AR. We apply them on different AR applications with different goals, e.g. in X-Ray vision or in applications which draw a user's attention to an object of interest. We demonstrate how Focus and Context (F+C) visualizations are used to affect the user's perception of hidden or nearby objects by presenting contextual information in the area of augmentation. We discuss the organization and the possible sources of data for visualizations in augmented reality and present cascaded and multi level F+C visualizations to address complex, cluttered scenes that are inevitable in real environments. This article also shows filters and tools to interactively control the amount of augmentation. It compares the impact of real world context preserving to a pure virtual and uniform enhancement of these structures for augmentations of real world imagery. Finally this paper discusses the stylization of sparse object representations for AR to improve x-ray vision.     

增强现实技术的应用

随着计算机技术的发展,增强现实技术逐渐成为下一代人机接口技术发展的主要方向之一。增强现实技术使用户可以容易地进入人们本来不能进入的微观世界,以及人们不能直接进入的特殊环境,并为人们的研究工作构造理想的交互环境,比如观察分子立体结构、探索基因芯片本质的增强现实系统等。增强现实技术与计算机图形学、人机交互技术、网络多媒体技术相结合,广泛应用于医学、军事、工业、娱乐等方面。     

浅谈增强现实技术

近年来,增强现实因其具有虚实结合的特点,逐渐成为研究的热点。通过增强现实电视节目中的应用引出对增强现实概念、特点和关键技术的介绍,简述了国内外增强现实的发展现状,并对其应用作了简要说明,最后展望增强现实未来对人类生活的影响。     

In-Place Augmented Reality

In this paper, we present a vision-based approach for transmitting virtual models for Augmented Reality, which we name In-Place Augmented Reality (IPAR). A two-dimensional representation of the virtual models is embedded in a printed image. We apply computer vision techniques to interpret the printed image and extract the virtual models, which are then overlaid on the printed image. The main advantages of our approach are: (1) the image of the embedded virtual models and their behaviors are understandable to a human without using an AR system and (2) no database or network communication is required to retrieve the models. To demonstrate the technology and test its usability, we implemented several applications and performed a user evaluation. We discuss how the proposed technique can be used for the development of applications in different domains such as education, advertisement, and gaming.     

Robust pose estimation and recognition using non-gaussian modeling of appearance subspaces

We present an original appearance model that generalizes the usual Gaussian visual subspace model to non-Gaussian and nonparametric distributions. It can be useful for the modeling and recognition of images under difficult conditions such as large occlusions and cluttered backgrounds. Inference under the model is efficiently solved using the mean shift algorithm     

Image set-based face recognition using pose estimation with facial landmarks

Multimedia Tools and Applications - Face recognition (FR) based on image set is an important topic in computer vision. There are numerous approaches that apply pose estimation method for single...     

Augmented and Virtual Reality techniques for footwear

The use of 3D imaging techniques has been early adopted in the footwear industry. In particular, 3D imaging could be used to aid commerce and improve the quality and sales of shoes. Footwear customization is an added value aimed not only to improve product quality, but also consumer comfort. Moreover, customisation implies a new business model that avoids the competition of mass production coming from new manufacturers settled mainly in Asian countries.     

Robust pose estimation

Standard least-squares (LS) methods for pose estimation of objects are sensitive to outliers which can occur due to mismatches. Even a single mismatch can severely distort the estimated pose. This paper describes a least-median of squares (LMedS) approach to estimating pose using point matches. It is both robust (resistant to up to 50% outliers) and efficient (linear in the number of points). The basic algorithm is then extended to improve performance in the presence of two types of noise: 1) type I which perturbs all data values by small amounts (e.g., Gaussian) and 2) type II which can corrupt a few data values by large amounts     

The complex representation of algebraic curves and its simpleexploitation for pose estimation and invariant recognition

Representations are introduced for handling 2D algebraic curves (implicit polynomial curves) of arbitrary degree in the scope of computer vision applications. These representations permit fast, accurate pose-independent shape recognition under Euclidean transformations with a complete set of invariants, and fast accurate pose-estimation based on all the polynomial coefficients. The latter is accomplished by a centering of a polynomial based on its coefficients, followed by rotation estimation by decomposing polynomial coefficient space into a union of orthogonal subspaces for which rotations within two-dimensional subspaces or identity transformations within one-dimensional subspaces result from rotations in x, y measured-data space. Angles of these rotations in the two-dimensional coefficient subspaces are proportional to each other and are integer multiples of the rotation angle in the x, y data space. By recasting this approach in terms of a complex variable, i.e., x+iy=z, and complex polynomial-coefficients, further conceptual and computational simplification results. Application to shape-based indexing into databases is presented to illustrate the usefulness and the robustness of the complex representation of algebraic curves     

Gravitational pose estimation

Problem of relative pose estimation between a camera and rigid object, given an object model with feature points and image(s) with respective image points (hence known correspondence) has been extensively studied in the literature. We propose a “correspondenceless” method called gravitational pose estimation (GPE), which is inspired by classical mechanics. GPE can handle occlusion and uses only one image (i.e., perspective projection of the object). GPE creates a simulated gravitational field from the image and lets the object model move and rotate in that force field, starting from an initial pose. Experiments were carried out with both real and synthetic images. Results show that GPE is robust, consistent, and fast (runs in less than a minute). On the average (including up to 30% occlusion cases) it finds the orientation within 6° and the position within 17% of the object’s diameter. SoftPOSIT was so far the best correspondenceless method in the literature that works with a single image and point-based object model like GPE. However, SoftPOSIT’s convergence to a result is sensitive to the choice of initial pose. Even “random start SoftPOSIT,” which performs multiple runs of SoftPOSIT with different initial poses, can often fail. However, SoftPOSIT finds the pose with great precision when it is able to converge. We have also integrated GPE and SoftPOSIT into a single method called GPEsoftPOSIT, which finds the orientation within 3° and the position within 10% of the object’s diameter even under occlusion. In GPEsoftPOSIT, GPE finds a pose that is very close to the true pose, and then SoftPOSIT is used to enhance accuracy of the result. Unlike SoftPOSIT, GPE also has the ability to work with three points as well as planar object models.     

Opportunistic Tangible User Interfaces for Augmented Reality

Opportunistic Controls are a class of user interaction techniques that we have developed for augmented reality (AR) applications to support gesturing on, and receiving feedback from, otherwise unused affordances already present in the domain environment. By leveraging characteristics of these affordances to provide passive haptics that ease gesture input, Opportunistic Controls simplify gesture recognition, and provide tangible feedback to the user. In this approach, 3D widgets are tightly coupled with affordances to provide visual feedback and hints about the functionality of the control. For example, a set of buttons can be mapped to existing tactile features on domain objects. We describe examples of Opportunistic Controls that we have designed and implemented using optical marker tracking, combined with appearance-based gesture recognition. We present the results of two user studies. In the first, participants performed a simulated maintenance inspection of an aircraft engine using a set of virtual buttons implemented both as Opportunistic Controls and using simpler passive haptics. Opportunistic Controls allowed participants to complete their tasks significantly faster and were preferred over the baseline technique. In the second, participants proposed and demonstrated user interfaces incorporating Opportunistic Controls for two domains, allowing us to gain additional insights into how user interfaces featuring Opportunistic Controls might be designed.