Isolated 3D object recognition through next view planning

In many cases, a single view of an object may not contain sufficient features to recognize it unambiguously. This paper presents a new online recognition scheme based on next view planning for the identification of an isolated 3D object using simple features. The scheme uses a probabilistic reasoning framework for recognition and planning. Our knowledge representation scheme encodes feature based information about objects as well as the uncertainty in the recognition process. This is used both in the probability calculations as well as in planning the next view. Results clearly demonstrate the effectiveness of our strategy for a reasonably complex experimental set     

Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds

A system for the detection, segmentation and recognition of multi-class hand postures against complex natural backgrounds is presented. Visual attention, which is the cognitive process of selectively concentrating on a region of interest in the visual field, helps human to recognize objects in cluttered natural scenes. The proposed system utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhanced the edges and shapes within the skin colored regions. The color features used are the discretized chrominance components in HSI, YCbCr color spaces, and the similarity to skin map. The hand postures are classified using the shape and texture features, with a support vector machines classifier. A new 10 class complex background hand posture dataset namely NUS hand posture dataset-II is developed for testing the proposed algorithm (40 subjects, different ethnicities, various hand sizes, 2750 hand postures and 2000 background images). The algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds. The algorithm provided a recognition rate of 94.36 %. A comparison of the proposed algorithm with other existing methods evidences its better performance.     

基于支持向量机与细节层次的三维地形识别与检索

提出对相似3D物体识别与检索的算法．该算法首先使用细节层次模型对3D物体进行三角面片数量的约减，然后提取3D物体的特征．由于所提取的特征维数很大，因此独立成分分析被用来进行3D特征约减．基于约减后的特征，使用支持向量机进行识别与检索．将该算法用于3D丘陵与山地的地形识别中，取得了良好效果。     

Neural networks for the recognition and pose estimation of 3D objects from a single 2D perspective view

In this paper we present a neural network (NN) based system for recognition and pose estimation of 3D objects from a single 2D perspective view. We develop an appearance based neural approach for this task. First the object is represented in a feature vector derived by a principal component network. Then a NN classifier trained with Resilient backpropagation (Rprop) algorithm is applied to identify it. Next pose parameters are obtained by four NN estimators trained on the same feature vector. Performance on recognition and pose estimation for real images under occlusions are shown. Comparative studies with two other approaches are carried out.     

感知器残差网络和超限学习机融合的3D物体识别

目的 随着3D扫描技术和虚拟现实技术的发展,真实物体的3D识别方法已经成为研究的热点之一。针对现有基于深度学习的方法训练时间长,识别效果不理想等问题,提出了一种结合感知器残差网络和超限学习机（ELM）的3D物体识别方法。方法 以超限学习机的框架为基础,使用多层感知器残差网络学习3D物体的多视角投影特征,并利用提取的特征数据和已知的标签数据同时训练了ELM分类层、K最近邻（KNN）分类层和支持向量机（SVM）分类层识别3D物体。网络使用增加了多层感知器的卷积层替代传统的卷积层。卷积网络由改进的残差单元组成,包含多个卷积核个数恒定的并行残差通道,用于拟合不同数学形式的残差项函数。网络中半数卷积核参数和感知器参数以高斯分布随机产生,其余通过训练寻优得到。结果 提出的方法在普林斯顿3D模型数据集上达到了94.18%的准确率,在2D的NORB数据集上达到了97.46%的准确率。该算法在两个国际标准数据集中均取得了当前最好的效果。同时,使用超限学习机框架使得本文算法的训练时间比基于深度学习的方法减少了3个数量级。结论 本文提出了一种使用多视角图识别3D物体的方法,实验表明该方法比现有的ELM方法和深度学习等最新方法的识别率更高,抗干扰性更强,并且其调节参数少,收敛速度快。     

基于局部指纹曲面片的点云三维物体识别

提出一种新的基于局部描述符的点云物体识别算法。算法根据点云的位置信息提取出邻域以及曲率信息,进而得到形状索引信息。根据形状索引提取到特征点,在每个特征点根据样条拟合原理得到测地距离和矢量夹角分割曲面得到曲面片集。每个曲面片的等距测地线构成了曲面片指纹,通过矢量和半径的变化描述,可以把每个模型物体得到的曲面片集描述存入数据库。对于给定的一个物体,根据上面步骤同样得到其曲面片集描述,通过和数据库中模型物体曲面片集的比对,得到初始识别结果。对每对初始识别结果进行对应滤波后,通过最近点迭代方法得到最终的识别结果。最后通过具体的实验说明了算法的有效性和高效性。     

Strategies of Multi-view and Multi-matching for 3D Object Recognition

A multi-view representation scheme and a multi-matching strategy for 3D object recognition are described; 3D objects are represented in terms of their 2D appearances so that 2D techniques can be applied to 3D recognition. Appearances of objects in the representation scheme are further organized in a hierarchical manner so that the matching process can reduce its search space by examining only the optimal view at every level of the representation scheme. In our multi-matching strategy, the matching module is composed of four components: point matcher, string matcher, vector matcher, and chamfer matcher. Each matcher is associated with a termination rule so that impossible views can be rejected at the early stages of the matching process. Experimental results reveal that the proposed strategies are feasible for 3D object recognition.     

A framework for heading-guided recognition of human activity

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. Heading-guided recognition (HGR) is proposed as an efficient method for adaptive classification of activity. The HGR approach is demonstrated using “motion history images” that are then recognized via a mixture-of-Gaussians classifier. The system is tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. In addition, experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.     

Invariant object recognition in the visual system with novel views of 3D objects

To form view-invariant representations of objects, neurons in the inferior temporal cortex may associate together different views of an object, which tend to occur close together in time under natural viewing conditions. This can be achieved in neuronal network models of this process by using an associative learning rule with a short-term temporal memory trace. It is postulated that within a view, neurons learn representations that enable them to generalize within variations of that view. When three-dimensional (3D) objects are rotated within small angles (up to, e.g., 30 degrees), their surface features undergo geometric distortion due to the change of perspective. In this article, we show how trace learning could solve the problem of in-depth rotation-invariant object recognition by developing representations of the transforms that features undergo when they are on the surfaces of 3D objects. Moreover, we show that having learned how features on 3D objects transform geometrically as the object is rotated in depth, the network can correctly recognize novel 3D variations within a generic view of an object composed of a new combination of previously learned features. These results are demonstrated in simulations of a hierarchical network model (VisNet) of the visual system that show that it can develop representations useful for the recognition of 3D objects by forming perspective-invariant representations to allow generalization within a generic view.     

Indexing without invariants in 3D object recognition

We present a method of indexing 3D objects from single 2D images. The method does not rely on invariant features. This allows a richer set of shape information to be used in the recognition process. We also suggest the kd-tree as an alternative indexing data structure to the standard hash table. This makes hypothesis recovery more efficient in high-dimensional spaces, which are necessary to achieve specificity in large model databases. Search efficiency is maintained in these regimes by the use of best-bin first search. Neighbors recovered from the index are used to generate probability estimates, local within the feature space, which are then used to rank hypotheses for verification. On average, the ranking process greatly reduces the number of verifications required. Our approach is general in that it can be applied to any real-valued feature vector. In addition, it is straightforward to add to our index information from real images regarding the true probability distributions of the feature groupings used for indexing     

Real-time user independent hand gesture recognition from time-of-flight camera video using static and dynamic models

The use of hand gestures offers an alternative to the commonly used human computer interfaces, providing a more intuitive way of navigating among menus and multimedia applications. This paper presents a system for hand gesture recognition devoted to control windows applications. Starting from the images captured by a time-of-flight camera (a camera that produces images with an intensity level inversely proportional to the depth of the objects observed) the system performs hand segmentation as well as a low-level extraction of potentially relevant features which are related to the morphological representation of the hand silhouette. Classification based on these features discriminates between a set of possible static hand postures which results, combined with the estimated motion pattern of the hand, in the recognition of dynamic hand gestures. The whole system works in real-time, allowing practical interaction between user and application.     

主轴方法和矩方法相融合的三维物体归一化的识别

针对主轴方法和矩方法各自的特点,提出将这两种方法相融合的物体归一化和识别的新 思想.首先推导出主轴唯一性判别准则,有效地解决了物体取向的归一化问题,然后推导出 了对物体平移、取向和比例变化归一化的3-D不变矩.最后提出由二阶、三阶3-D不变矩组 成的判别向量和识别判据.对12个三维物体进行归一化和识别的实验结果验证了本文方法 的有效性.     

Angle densities and recognition of 3D objects

Recognition of 3D objects using computer vision is complicated by the fact that geometric features vary with view orientation. An important factor in designing recognition algorithms in such situations is understanding the variation of certain critical features such as angles. In this paper we derive the two dimensional joint density function of two angles in a scene given an isotropic view orientation and an orthographic projection. The analytic expression for the densities are useful in determining statistical decision rules to recognize surfaces and objects. Experiments to evaluate the usefulness of the proposed methods are reported     

American Sign Language word recognition with a sensory glove using artificial neural networks

An American Sign Language (ASL) recognition system is being developed using artificial neural networks (ANNs) to translate ASL words into English. The system uses a sensory glove called the Cyberglove™ and a Flock of Birds^® 3-D motion tracker to extract the gesture features. The data regarding finger joint angles obtained from strain gauges in the sensory glove define the hand shape, while the data from the tracker describe the trajectory of hand movements. The data from these devices are processed by a velocity network with noise reduction and feature extraction and by a word recognition network. Some global and local features are extracted for each ASL word. A neural network is used as a classifier of this feature vector. Our goal is to continuously recognize ASL signs using these devices in real time. We trained and tested the ANN model for 50 ASL words with a different number of samples for every word. The test results show that our feature vector extraction method and neural networks can be used successfully for isolated word recognition. This system is flexible and open for future extension.     

双视图三维卷积网络的工业装箱行为识别

目的 在自动化、智能化的现代生产制造过程中,行为识别技术扮演着越来越重要的角色,但实际生产制造环境的复杂性,使其成为一项具有挑战性的任务。目前,基于3D卷积网络结合光流的方法在行为识别方面表现出良好的性能,但还是不能很好地解决人体被遮挡的问题,而且光流的计算成本很高,无法在实时场景中应用。针对实际工业装箱场景中存在的人体被遮挡问题和光流计算成本问题,本文提出一种结合双视图3D卷积网络的装箱行为识别方法。方法 首先,通过使用堆叠的差分图像(residual frames, RF)作为模型的输入来更好地提取运动特征,替代实时场景中无法使用的光流。原始RGB图像和差分图像分别输入到两个并行的3D ResNeXt101中。其次,采用双视图结构来解决人体被遮挡的问题,将3D ResNeXt101优化为双视图模型,使用一个可学习权重的双视图池化层对不同角度的视图做特征融合,然后使用该双视图3D ResNeXt101模型进行行为识别。最后,为进一步提高检测结果的真负率(true negative rate, TNR),本文在模型中加入降噪自编码器和two-class支持向量机(support vec...     

Stereo-based palmprint recognition in various 3D postures

In order to increase performance in palmprint recognition systems, various devices are normally used to restrict the movement of the hand. These can cause problems, especially for those users with physical disabilities. They also cause significant hygiene problems in multi-user systems. Recently, studies on palmprint recognition systems have progressed towards the development of unconstrained, contactless and unrestricted background techniques. The most common problem encountered in these studies is the alignment arising from the free movement of the hand. Despite 3D hand-acquisition devices which offer extra recognition features to overcome this problem, the applicability of these devices is low because of their increased cost. In this study, a stereo camera was proposed. Although due to matching problems, it is difficult to achieve precise, distinct feature extraction in the unrestricted 3D environment used for palmprint recognition, the orientation of the hand in 3D space can be determined by obtaining depth information. In this study, the depth information was extracted by using the binocular stereo approach. First, the orientation of the hand was estimated by fitting a surface model associated with the eigenvectors of the depth information. Pose correction was then accomplished by establishing a relationship between the orientation and the images. The pose correction greatly relieved the perspective distortion that usually occurs within the various poses of the hands. Next, the region of interest was determined by performing segmentation on the corrected images using the Active Appearance Model (AAM). The palmprint features were then extracted via Gabor-based Kernel Fisher Discriminant Analysis. In order to demonstrate the performance of the proposed approach, a new dataset was compiled from stereo images within various scenarios collected from 138 different individuals. As a result of these experimental studies, the EER values, especially on the images captured from different hand orientations in 3D, were reduced from around 14–0.75%. With the help of this suggested approach, the palmprint recognition system was transformed into a more portable form by removing the closed-box mechanisms and equipment restricting movement of the hand. This system can automatically perform pose estimation, hand segmentation and recognition processes without any special intervention.     

基于时频融合特征的3D空间手写识别

研究基于3D加速度传感器的空间手写识别技术,提出一种基于时频融合特征的分类识别方法。从加速度数据中提取短时能量 (STE)特征及低频分量,经快速傅里叶变换后提取频域特征WPD+FFT,将时域特征STE和频域特征WPD+FFT进行特征融合,利用主成分分析法对其降维,采用支持向量机进行分类识别。实验结果表明,该方法能提高空间手写识别系统的识别率。     

基于模糊自适应共振理论映射算法的单样本三维人脸识别

针对传统的三维人脸识别分类算法大多需要多个样本进行训练,而在单训练样本的前提下识别性能会严重降低的问题,提出了基于模糊自适应共振理论映射(Fuzzy ARTMAP)的算法对三维人脸数据库进行分类识别。首先对三维人脸深度图像进行局部二值模式(LBP)统一模式算子的特征提取,再对LBP特征进行Log-Gabor小波变换,提取图像的频域特征向量作为训练的输入向量,最后将单样本训练向量集送入Fuzzy ARTMAP分类器进行训练识别。该算法在FRGC v2.0三维人脸数据库中的识别率可达到87.15%,分类器的训练时间为24.88s,单张待识别人脸样本与单张已注册的人脸匹配时间为0.0015s,一张新的人脸样本在数据库完成一次搜索匹配则需要1.08s。实验结果表明,所提方法在测试中的性能优于概率神经网络(PNN)和极限学习机神经网络(ELM),既能保证较高的识别率,又能拥有较短的训练时间,且时间增幅稳定,可控性强。