Hopfield Neural Network Based Stereo Matching Algorithm

In this paper, a neural network based optimization method is described in order to solve the problem of stereo matching for a set of primitives extracted from a stereoscopic pair of images. The neural network used is the 2D Hopfield network. The matching problem amounts to the minimization of an energy function involving specified stereoscopic constraints. This function reaches its minimum when these constraints are satisfied. The network converges to its stable state when the minimum is reached. In the initial step, the primitives to match are extracted from the stereoscopic pair of images. The primitives we use are specific points of interest. The feature extraction technique is the one developed by Moravec, and called the interest operator. Its output comprises mostly corners or feature points with high variance. The Hopfield network is represented as a N _l × N _r matrix of neurons, where N _l is the number of features in the left image and N _r the number of features in the right one. An update of the state of each neuron is done in order to perform the network evolution and then allowing it to settle down into a stable state. In the stable state, each neuron represents a possible match between a left candidate and a right one.     

基于分层学习的三维模型兴趣点提取算法

针对基于学习的三维模型兴趣点提取问题,提出一种兴趣点分层学习的全监督算法.提取三维模型表面所有顶点的特征向量后,将人工标注的兴趣点分为稀疏点和密集点,对于稀疏点使用整个三维模型进行神经网络训练,对于密集点则找出兴趣点分布密集的区域进行单独的神经网络训练;然后对2个神经网络进行特征匹配,得到一个用于三维模型兴趣点提取预测的分类器.测试时,提取新输入的三维模型上所有顶点的特征向量,将其输入到训练好的分类器中进行预测,应用改进的密度峰值聚类算法提取兴趣点.算法采用分层学习的策略,解决了传统算法在模型细节处难以准确提取密集兴趣点的问题.在SHREC’11数据集上的实验结果表明,与传统算法相比,该算法提取兴趣点的准确率更高,出现的遗漏点和错误点更少,对解决越来越精细的三维模型的兴趣点提取问题有较大帮助.     

Evaluating 2D and 3D geovisualisations for basic spatial assessment

This study investigates the use of 2D and 3D presentations of maps for the assessment of distances in a geographical context. Different types of 3D representations have been studied: A weak 3D visualisation that provides static monocular depth cues and a strong 3D visualisation that uses stereoscopic and kinetic depth cues. Two controlled experiments were conducted to test hypotheses regarding subjects’ efficiency in visually identifying the shortest distance among a set of market locations in a map. As a general result, we found that participants were able to correctly identify shortest distances when the difference to potential alternatives was sufficiently large, but performance decreased systematically when this difference decreased. Noticeable differences emerged for the investigated visualisation conditions. Participants in this study were equally efficient when using a weak 3D representation and a 2D representation. When the strong 3D visualisation was employed, they reported visual discomfort and tasks solved were significantly less correct. Presentations of intrinsic 2D content (maps) in 3D context did not, in this study, benefit from cues provided by a strong 3D visualisation and are adequately implemented using a weak 3D visualisation.     

基于网格形变的立体图像内容重组

目的 近年来,随着数字摄影技术的飞速发展,图像增强技术越来越受到重视。图像构图作为图像增强中影响美学的重要因素,一直都是研究的热点。为此,从立体图像布局调整出发,提出一种基于Delaunay网格形变的立体图像内容重组方法。方法 首先将待重组的一对立体图像记为源图像,将用于重组规则确定的一幅图像记为参考图像;然后对源图像需要调整的目标、特征线和其他区域进行取点操作,建立Delaunay网格。将源图像的左图与参考图像进行模板匹配操作,得到源图像与参考图像在结构布局上的对应关系;最后利用网格形变的特性,移动和缩放目标对象,并对立体图像的深度进行自适应调整。结果 针对目标对象的移动、缩放和特征线调整几方面进行优化。当只涉及目标对象的移动或特征线调整时,立体图像视差保持不变;当目标对象缩放时,立体图像中目标对象的视差按照缩放比例变化而背景视差保持不变。实验结果表明,重组后的立体图像构图与参考图像一致且深度能自适应调整。与最新方法比较,本文方法在目标对象分割精度和图像语义保持方面具有优势。结论 根据网格形变相关理论,构建图像质量、布局匹配和视差适应3种能量项,实现了立体图像的内容重组。与现有需要提取和粘贴目标对象的重组方法不同,本文方法对目标对象的分割精度要求不高,不需要图像修复和混合技术,重组后的立体图像没有伪影和语义错误出现。用户可以通过参考图像来引导立体图像的布局调整,达到期望的图像增强效果。     

An effective calibration procedure for correction of parallax unmatched image pairs

The stereo image pairs or two-view video sequences can be captured by two cameras at two horizontally different positions. If the stereo image pairs possess incompatible convergences of vertical parallax and horizontal parallax, human eyes generally cannot properly exhibit stereo visualisation. If the image pairs are taken from single camera or selected from a video sequence, perception of stereo visualisation will become even worse. The authors propose an effective calibration procedure to adjust the image pairs to achieve better stereo visualisation. The proposed calibration procedure contains six steps, including feature point extraction, bidirectional feature point matching, relative distance checking, image transformation, hole-filling and reshaping. Experimental results show that the proposed system can effectively adjust vertical and horizontal parallax such that the calibrated stereo image pairs can properly exhibit stereo scenes in stereoscopic display systems. Experimental results also reveal that the proposed method can achieve less vertical parallax than the existing rectification methods.     

基于改进Hopfield神经网络的图像特征点匹配算法

针对图像特征点匹配算法的运行时间呈指数增长的问题,提出了一种新的匹配算法NHop.该算法通过加入新的网络输入输出函数、点对间差异的度量和启发式选择目标点的方式,对传统的Hopfield神经网络进行了改进.新算法不仅解决了传统Hopfield神经网络运行时间长、能量函数易陷入局部极小点的问题,而且也有效地实现了图像特征点的匹配.实验结果表明,与传统的Hopfield神经网络相比,NHop算法的匹配速度更快、准确率更高,对于图像特征点的匹配效果更好.     

Parallel distributed detection of feature trajectories in multiplediscontinuous motion image sequences

Concerns the 3D interpretation of image sequences showing multiple objects in motion. Each object exhibits smooth motion except at certain time instants when a motion discontinuity may occur. The objects are assumed to contain point features which are detected as the images are acquired. Estimating feature trajectories in the first two frames amounts to feature matching. As more images are acquired, existing trajectories are extended. Both initial detection and extension of trajectories are done by enforcing pertinent constraints from among the following: similarity of the image plane arrangement of neighboring features, smoothness of the 3D motion and smoothness of the image plane motion. The constraints are incorporated into energy functions which are minimized using 2D Hopfield networks. Wrong matches that result from convergence to local minima are eliminated using a 1D Hopfield-like network. Experimental results on several image sequences are shown.     

From laser-scanned data to feature human model: a system based on fuzzy logic concept

This paper describes the development of a prototype system using fuzzy logic concept for constructing a feature human model, which is to be stored in a 3D digital human model database. In our approach, the feature human model is constructed by unorganized cloud points obtained from 3D laser scanners. Firstly, noisy points are removed, and the orientation of the human model is adjusted; secondly, a feature based mesh generation algorithm is applied on the cloud points to construct the mesh surface of a human model; lastly, semantic features of the human model are extracted from the mesh surface. Compared with earlier approach, our method strongly preserves the topology of a human model; more details can be constructed; and both the robustness and the efficiency of the algorithm are improved. At the end of the paper, in order to demonstrate the functionality of feature human models, potential applications are given.     

Extracting 3D model feature lines based on conditional random fields

We propose a 3D model feature line extraction method using templates for guidance. The 3D model is first projected into a depth map, and a set of candidate feature points are extracted. Then, a conditional random fields (CRF) model is established to match the sketch points and the candidate feature points. Using sketch strokes, the candidate feature points can then be connected to obtain the feature lines, and using a CRF-matching model, the 2D image shape similarity features and 3D model geometric features can be effectively integrated. Finally, a relational metric based on shape and topological similarity is proposed to evaluate the matching results, and an iterative matching process is applied to obtain the globally optimized model feature lines. Experimental results showed that the proposed method can extract sound 3D model feature lines which correspond to the initial sketch template.     

3维模型局部高度研究

提出一种新的3维模型的特征点检测算法。该算法可以作为其他许多3维模型处理技术的预处理操作(如模型简化、模型匹配、视点选择等)。与其他3维模型特征点检测算法相比,该算法具有两个特点： 1)引入一种新的显著性度量方法——“局部高度”,而不是传统的曲率。认为3维模型表面某点的视觉重要性(即显著性)是由它所在位置的凸起程度来刻画,而不是该点所在位置的弯曲程度所决定,因此,提出局部高度这种新的显著性度量方式。 2)基于局部高度,引入Mean Shift算法这种非参数化的概率密度估计方法来对3维模型表面的局部高度分布进行聚类分析,然后计算出3维模型的特征点。实验结果表明,该算法能够很好地捕捉视觉上显著的3维模型特征点,且在不同分辨率下均有稳定的表现。     

Object-aware deep feature extraction for feature matching

Feature extraction is a fundamental step in the feature matching task. A lot of studies are devoted to feature extraction. Recent researches propose to extract features by pre-trained neural networks, and the output is used for feature matching. However, the quality and the quantity of the features extracted by these methods are difficult to meet the requirements for the practical applications. In this article, we propose a two-stage object-aware-based feature matching method. Specifically, the proposed object-aware block predicts a weighted feature map through a mask predictor and a prefeature extractor, so that the subsequent feature extractor pays more attention to the key regions by using the weighted feature map. In addition, we introduce a state-of-the-art model estimation algorithm to align image pair as the input of the object-aware block. Furthermore, our method also employs an advanced outlier removal algorithm to further improve matching quality. Experimental results show that our object-aware-based feature matching method improves the performance of feature matching compared with several state-of-the-art methods.     

Artificial neural networks for 3-D motion analysis. I. Rigid motion

Proposes an approach applying artificial neural net techniques to 3D rigid motion analysis based on sequential multiple time frames. The approach consists of two phases: (1) matching between every two consecutive frames and (2) estimating motion parameters based on the correspondences established. Phase 1 specifies the matching constraints to ensure a stable and coherent feature correspondence establishment between two sequential time frames and configures a 2D Hopfield neural net to enforce these constraints. Phase 2 constructs a 3-layer net to estimate parameters through supervised learning. The method performs motion analysis based on sequential multiple time frames. It represents an effective way to achieve optimal matching between two frames using neural net techniques. The energy function of the Hopfield net is designed to reflect the matching constraints and the minimization of this function leads to the optimal feature correspondence establishment. The approach introduces the learning concept to motion estimation. The structure of the net provides the flexibility in estimating motion parameters based on information from multiple frames.     

基于单相机的全向深度图获取

为了搜寻移动机器人周围最大的可通行区域,采用全向立体视觉系统,提出获取可靠的致密三维深度图方法。视觉系统由1个普通相机和2个双曲面镜组成。当系统标定后,空间点的三维坐标可以通过匹配上下镜面的成像点计算得出。匹配方法分3步：最大FX匹配,特征匹配和歧义去除。定义合适的能量函数通过动态规划来实现剩余点的匹配。实验表明该系统精度高、具有实用价值。     

三维图匹配算法

目的 现有的图匹配算法大多应用于二维图像,对三维图像的特征点匹配存在匹配准确率低和计算速度慢等问题。为解决这些问题,本文将分解图匹配算法扩展应用在了三维图像上。方法 首先将需要匹配的两个三维图像的特征点作为图的节点集;再通过Delaunay三角剖分算法,将三维特征点相连,则相连得到的边就作为图的边集,从而建立有向图;然后,根据三维图像的特征点构建相应的三维有向图及其邻接矩阵;再根据有向图中的节点特征和边特征分别构建节点特征相似矩阵和边特征相似矩阵;最后根据这两个特征矩阵将节点匹配问题转化为求极值问题并求解。结果 实验表明,在手工选取特征点的情况下,本文算法对相同三维图像的特征点匹配有97.56%的平均准确率;对不同三维图像特征点匹配有76.39%的平均准确率;在三维图像有旋转的情况下,有90%以上的平均准确率;在特征点部分缺失的情况下,平均匹配准确率也能达到80%。在通过三维尺度不变特征变换（SIFT）算法得到特征点的情况下,本文算法对9个三维模型的特征点的平均匹配准确率为98.78%。结论 本文提出的基于图论的三维图像特征点匹配算法,经实验结果验证,可以取得较好的匹配效果。     

基于相对角分布聚类和支持向量机的3维人脸特征点匹配技术的研究

人脸特征点自动定位及对应点匹配是计算机视觉和模式识别领域一个非常热门的研究方向,应用领域包括图像配准、对象识别与跟踪、3维重建、立体匹配等。通过相对角直方图分布和K均值聚类确定脸部特征点的聚类点集,再利用几何信息提取聚类点集的特征,进而采用支持向量机分类最终从点集中分离出39个脸部特征点。实验结果表明,此混合提取方法比单纯使用RAC得到了更好的匹配准确率,在给定的距离阈值范围内,50%的特征点定位准确率达到了100%。     

基于三维数据的人脸表情识别

为了充分利用三维图像,在前人的二维图像研究的基础上,该文提出了一种新的基于三维数据的人脸表情识别方法。在此方法中,根据三维成像仪的特性,直接利用得到的三维数据进行分析处理,避免了传统的通过二维图像生成三维图像的特点创建了新的模版匹配方法,同时也结合了原始的二维图像进行特征点识别,然后综合分析特征点的性质完成人验表情识别,实验表明,该方法能够对六种人脸表情进行识别,有较高的识别率。     

改进的统计模型三维人脸特征点标定算法框架

自动三维人脸特征点标定是计算机视觉领域的研究热点,其广泛应用于人脸识别,人脸模型配准,表情识别,脸部动画等领域。通过对三维人脸样本统计建模,采用遗传算法对待匹配模型的生成数目进行参数优化,利用模型相似性匹配方法及其映射关系对三维人脸特征点进行自动标定。首先,对三维人脸数据预处理,然后对其统计建模并通过模型形变得到有映射关系的基准模型和待匹配模型。利用遗传算法对待匹配模型中的待匹配模型生成数目参数进行优化,生成与之对应的待匹配模型数;接着计算待测模型与待匹配模型的相似度。最后,利用模型相似度和模型映射关系,间接得到待测模型的特征点。实验结果表明,提出的算法是可行的,能够在一定程度上提高原有算法的效率。该算法可以自动标定三维人脸模型的特征点,当距离阈值为10像素时,39个三维人脸特征点定位的准确率都可以达到100%,并有效解决了传统方法中三维人脸模型平滑区域特征点精度不高的问题。