Active Object Recognition Integrating Attention and Viewpoint Control

We present an active object recognition strategy which combines the use of an attention mechanism for focusing the search for a 3D object in a 2D image, with a viewpoint control strategy for disambiguating recovered object features. The attention mechanism consists of a probabilistic search through a hierarchy of predicted feature observations, taking objects into a set of regions classified according to the shapes of their bounding contours. We motivate the use of image regions as a focus-feature and compare their uncertainty in inferring objects with the uncertainty of more commonly used features such as lines or corners. If the features recovered during the attention phase do not provide a unique mapping to the 3D object being searched, the probabilistic feature hierarchy can be used to guide the camera to a new viewpoint from where the object can be disambiguated. The power of the underlying representation is its ability to unify these object recognition behaviors within a single framework. We present the approach in detail and evaluate its performance in the context of a project providing robotic aids for the disabled.     

基于特征点检测的三维模型最佳视点选择

针对三维模型最佳视点选择问题,提出了一种基表面特征点检测的视点选择算法。首先引入一种基于局部平均形心距离差的顶点显著性度量方法,认为三维网格模型表面某顶点的显著性是由该顶点与形心之间距离和该顶点邻域内的顶点与形心之间距离的平均差值来刻画,而不是该点所在位置的弯曲程度所决定;然后根据顶点的显著性大小进行三维模型表面特征点检测;最后针对视点球体上的每个候选视点分析该视点下可见特征点的几何分布和显著度大小计算视点质量,提取出最佳视点。实验结果验证了基于三维模型特征点的视点选择方法的有效性,在能够选择出质量较优的视点的同时保证算法的效率。     

Automatic sensor placement for model-based robot vision.

This paper presents a method for automatic sensor placement for model-based robot vision. In such a vision system, the sensor often needs to be moved from one pose to another around the object to observe all features of interest. This allows multiple three-dimensional (3-D) images to be taken from different vantage viewpoints. The task involves determination of the optimal sensor placements and a shortest path through these viewpoints. During the sensor planning, object features are resampled as individual points attached with surface normals. The optimal sensor placement graph is achieved by a genetic algorithm in which a min-max criterion is used for the evaluation. A shortest path is determined by Christofides algorithm. A Viewpoint Planner is developed to generate the sensor placement plan. It includes many functions, such as 3-D animation of the object geometry, sensor specification, initialization of the viewpoint number and their distribution, viewpoint evolution, shortest path computation, scene simulation of a specific viewpoint, parameter amendment. Experiments are also carried out on a real robot vision system to demonstrate the effectiveness of the proposed method.     

3D直接操作的自动视图控制

阐述通过自动寻找虚拟相机的有效位置以增强3D直接操作过程的技术.当前许多直接操作3D几何形体的尖端技术都强烈地依赖于视角,因此需要用户在操作过程中确定视点的坐标.在某些情形下,这个过程可以自动化.这意味着系统可以自动避免退化位置,而在退化情形,平移与旋转操作是难以实现的.系统还能选择视点和视角,使得被操作的物体可见,确保不被其他物体遮挡.     

An Investigation into the Use of Physical Modeling for the Prediction of Various Feature Types Visible from Different Viewpoints

Given that aspect graph and viewsphere-based object recognition systems provide a valid mechanism for 3D object recognition of man-made objects, this paper provides a flexible, automated, and general purpose technique for generating the view information for each viewpoint. An advantage of the work is that the technique is unaffected by object complexity because each step makes no assumptions about object shape. The only limitation is that the object can be described by a boundary representation. A second advantage is that the technique can include other feature types such as specularity. The reason for this is that raytracing techniques are used to simulate the physical process of image generation. Hence it is extendible to visible features resulting from effects due to lighting, surface texture, color, transparency, etc. The work described in this paper shows how occluding and nonoccluding edge-based features can be extracted using image processing techniques and then parametrized and also how regions of specularity can be predicted and described. The use of physical modeling enables situations to be simulated and predicted that are intractable for CAD-based methods (e.g., multiscale feature prediction). An advantage of the method is that the interface between the technique and the raytracing module is a rendered image. Should better physics-based image formation algorithms become available, then they could replace the raytracing module with little modification to the rest of the method.     

Automatic form feature recognition and 3D part reconstruction from 2D CAD data

In this paper, a method which can recognize form features and reconstruct 3D part from 2D CAD data automatically is proposed. First, we use the divide-and-conquer strategy to extract the vertex-edge data from each 2D engineering drawing of IGES format. Then, a set of production rules are developed to facilitate the form feature matching process. A new structure of form feature adjacency graph (FFAG) is devised to record the related attibutes of each form feature. Finally, to avoid the combinatorial subparts composition problem, a sweeping operation and volumetric intersection approach is used to rapidly reconstruct the remaining 3D objects. The last reconstructed 3E object is used as the base of the FFAG. All the recognized form features in the FFAG can be classified as depression or protrusion features on the 3D part base. The FFAG structure can be easily transformed into CSG/DSG structure, which is readily integrated with the downstream CAPP/CAM systems. A detailed example is provided to illustrate the feasibility and effectiveness of the proposed system.     

A camera planning method for the 3D reconstruction of a single object based on statistical deformation model

In order to provide sufficient information for the 3D reconstruction of a single, static and known type object, this paper proposes measure indexes to evaluate the information amount and a camera planning method to guide the search of the optimal camera positions. The planning method is based on Statistical Deformation Model (SDM) which is generated from Point Distribution Model (PDM) by the Principal Component Analysis (PCA) method. The position from which the individual factor coefficients are observed clearly and the principal shape features are identified distinctly is the best viewpoint. The observed times of landmark are used to exclude the redundancy among views. So, in an iteration process new SDM is repeatedly computed to pick out the optimal viewpoints until all the landmarks are observed sufficiently. Therefore abundant information with the minimum redundancy is provided by fewer cameras, and an experiment which layout cameras for foot reconstruction was demonstrated.     

Efficient 2D viewpoint combination for human action recognition

The ability to recognize human actions using a single viewpoint is affected by phenomena such as self-occlusions or occlusions by other objects. Incorporating multiple cameras can help overcome these issues. However, the question remains how to efficiently use information from all viewpoints to increase performance. Researchers have reconstructed a 3D model from multiple views to reduce dependency on viewpoint, but this 3D approach is often computationally expensive. Moreover, the quality of each view influences the overall model and the reconstruction is limited to volumes where the views overlap. In this paper, we propose a novel method to efficiently combine 2D data from different viewpoints. Spatio-temporal features are extracted from each viewpoint and then used in a bag-of-words framework to form histograms. Two different sizes of codebook are exploited. The similarity between the obtained histograms is represented via the Histogram Intersection kernel as well as the RBF kernel with \(\chi ^2\) distance. Lastly, we combine all the basic kernels generated by selection of different viewpoints, feature types, codebook sizes and kernel types. The final kernel is a linear combination of basic kernels that are properly weighted based on an optimization process. For higher accuracy, the sets of kernel weights are computed separately for each binary SVM classifier. Our method not only combines the information from multiple viewpoints efficiently, but also improves the performance by mapping features into various kernel spaces. The efficiency of the proposed method is demonstrated by testing on two commonly used multi-view human action datasets. Moreover several experiments indicate the efficacy of each part of the method on the overall performance.     

三维物体的形态图表达方法

三维物体的表达方法是计算机视觉中的关键问题之一，现有的各种三维物体表达方法虽然在各自的识别中得到应用，但都存在各自的局限性，用形态图表达三维物体是一种以视点为中心的表达方法，由于它列举了一个物体所有可能的“定性”形象，即它可使用最少的二维投影线图（特征视图）来表达一个完整的三维物体，因此使三维物体识别转化为2D与2D的匹配，该文首先定义了二维线图拓扑结构等价的判别准则，然后给出了构造透明物体形态图的方法，最后根据拓扑结构等价准则来得到不透明物体的形态图和特征图，并用圆锥与圆柱相交的实例进行了验证。     

《OpenGL三维图形程序设计》系列讲座 第五讲 曲线和曲面(下)

《ＯｐｅｎＧＬ三维图形程序设计》系列讲座第五讲曲线和曲面（下）３ＯｐｅｎＧＬ中的曲面３．１二维算子一个Ｂｅｚｉｅｒ曲面是一个二元向量值函数Ｓ（ｕ，ｖ）＝［Ｘ（ｕ，ｖ），Ｙ（ｕ，ｖ），Ｚ（ｕ，ｖ）］这里ｕ和ｖ都可以在其定义域内变化。在二维情况下，除了所...     

Category‐Specific Salient View Selection via Deep Convolutional Neural Networks

In this paper, we present a new framework to determine up front orientations and detect salient views of 3D models. The salient viewpoint to human preferences is the most informative projection with correct upright orientation. Our method utilizes two Convolutional Neural Network (CNN) architectures to encode category‐specific information learnt from a large number of 3D shapes and 2D images on the web. Using the first CNN model with 3D voxel data, we generate a CNN shape feature to decide natural upright orientation of 3D objects. Once a 3D model is upright‐aligned, the front projection and salient views are scored by category recognition using the second CNN model. The second CNN is trained over popular photo collections from internet users. In order to model comfortable viewing angles of 3D models, a category‐dependent prior is also learnt from the users. Our approach effectively combines category‐specific scores and classical evaluations to produce a data‐driven viewpoint saliency map. The best viewpoints from the method are quantitatively and qualitatively validated with more than 100 objects from 20 categories. Our thumbnail images of 3D models are the most favoured among those from different approaches.     

A comparative study of the robustness of two pose estimation techniques

The paper presents an analysis of the stability of pose estimation. Stability is defined as sensitivity of the pose parameters towards noise in image features used for estimating pose. The specific emphasis of the analysis is on determining {how the stability varies with viewpoint} relative to an object and to understand the relationships between object geometry, viewpoint, and pose stability. Two pose estimation techniques are investigated. One uses a numerical scheme for finding pose parameters; the other is based on closed form solutions. Both are “pose from trihedral vertices” techniques, which provide the rotation part of object pose based on orientations of three edge segments. The analysis is based on generalized sensitivity analysis propagating the uncertainty in edge segment orientations to the resulting effect on the pose parameters. It is shown that there is a precomputable, generic relationship between viewpoint and pose stability, and that there is a drastic difference in stability over the range of viewpoints. This viewpoint variation is shared by the two investigated techniques. Additionally, the paper offers an explicit way to determine the most robust viewpoints directly for any given vertex model. Experiments on real images show that the results of the work can be used to compute the variance in pose parameters for any given pose. For the predicted {instable} viewpoints the variance in pose parameters is on the order of 20 (degrees squared), whereas the variance for robust viewpoints is on the order of 0.05 (degrees squared), i.e., two orders of magnitude difference.     

Solving the recognition problem for six lines using the Dixon resultant

The “Six-line Problem” arises in computer vision and in the automated analysis of images. Given a three-dimensional (3D) object, one extracts geometric features (for example six lines) and then, via techniques from algebraic geometry and geometric invariant theory, produces a set of 3D invariants that represents that feature set. Suppose that later an object is encountered in an image (for example, a photograph taken by a camera modeled by standard perspective projection, i.e. a “pinhole” camera), and suppose further that six lines are extracted from the object appearing in the image. The problem is to decide if the object in the image is the original 3D object. To answer this question two-dimensional (2D) invariants are computed from the lines in the image. One can show that conditions for geometric consistency between the 3D object features and the 2D image features can be expressed as a set of polynomial equations in the combined set of two- and three-dimensional invariants. The object in the image is geometrically consistent with the original object if the set of equations has a solution. One well known method to attack such sets of equations is with resultants. Unfortunately, the size and complexity of this problem made it appear overwhelming until recently. This paper will describe a solution obtained using our own variant of the Cayley–Dixon–Kapur–Saxena–Yang resultant. There is reason to believe that the resultant technique we employ here may solve other complex polynomial systems.     

Efficiently computing and representing aspect graphs of polyhedralobjects

An efficient algorithm and a data structure for computing and representing the aspect graph of polyhedral objects under orthographic projection are presented. The aspect graph is an approach to representing 3-D objects by a set of 2-D views, for the purpose of object recognition. In this approach the viewpoint space is partitioned into regions such that in each region the qualitative structure of the line drawing does not change. The viewing data of an object is the partition of the viewpoint space together with a representative view in each region. The algorithm computes the viewing data for line drawings of polyhedral objects under orthographic projection     

Solving the orientation-duality problem for a circular feature inmotion

Circular features have been commonly used in numerous computer vision application areas for 3D pose estimation. However, although the 3D position of the circle's center can be uniquely identified, the solution process yields two different feasible orientations, of which only one is the true solution. In this paper, two methods are presented for the solution of the orientation-duality problem for circular features that are in motion. The first approach is applicable to those features moving on a 3D line with constant orientation or to those which are moving on a plane with general motion. The second approach relies on the existence of additional object features, such as points or lines, which are coplanar to the circular feature. In this case, the circular feature can undergo an arbitrary 3D motion. Experimental results verify the validity of the proposed methods     

The automatic construction of a view-independent relational modelfor 3-D object recognition

A view-independent relational model (VIRM) used in a vision system for recognizing known 3-D objects from single monochromatic images of unknown scenes is described. The system inspects a CAD model from a number of different viewpoints, and a statistical interference is applied to identify relatively view-independent relationships among component parts of the object. These relations are stored as a relational model of the object, which is represented in the form of a hypergraph. Three-dimensional components of the object, which can be associated with extended image features obtained by grouping of primitive 2-D features are represented as nodes of the hypergraph. Covisibility of model features is represented by means of hyperedges of the hypergraph, and the pairwise view-independent relations form procedural constraints associated with the hypergraph edges. During the recognition phase, the covisibility measures allow a best-first search of the graph for acceptable matches     

ClipFlip : Multi‐view Clipart Design

We present an assistive system for clipart design by providing visual scaffolds from the unseen viewpoints. Inspired by the artists' creation process, our system constructs the visual scaffold by first synthesizing the reference 3D shape of the input clipart and rendering it from the desired viewpoint. The critical challenge of constructing this visual scaffold is to generate a reference 3D shape that matches the user's expectations in terms of object sizing and positioning while preserving the geometric style of the input clipart. To address this challenge, we propose a user‐assisted curve extrusion method to obtain the reference 3D shape. We render the synthesized reference 3D shape with a consistent style into the visual scaffold. By following the generated visual scaffold, the users can efficiently design clipart with their desired viewpoints. The user study conducted by an intuitive user interface and our generated visual scaffold suggests that our system is especially useful for estimating the ratio and scale between object parts and can save on average 57% of drawing time.