Real-time three-dimensional object recognition with multiple perspectives imaging

A novel system for recognizing three-dimensional (3D) objects by use of multiple perspectives imaging is proposed. A 3D object under incoherent illumination is projected into an array of two-dimensional (2D) elemental images by use of a microlens array. Each elemental 2D image corresponds to a different perspective of the 3D object. Multiple perspectives imaging based on integral photography has been used for 3D display. In this way, the whole set of 2D elemental images records 3D information about the input object. After an optical incoherent-to-coherent conversion, an optical processor is employed to perform the correlation between the input and the reference 3D objects. Use of micro-optics allows us to process the 3D information in real time and with a compact optical system. To the best of our knowledge this 3D processor is the first to apply the principle of integral photography to 3D image recognition. We present experimental results obtained with both a digital and an optical implementation of the system. We also show that the system can recognize a slightly out-of-plane rotated 3D object.     

Estimating three-dimensional vehicle motion in an outdoor scene using stereo image sequences

We propose a motion estimation system that uses stereo image pairs as the input data. To perform experimental work, we also obtain a sequence of outdoor stereo images taken by two metric cameras. The system consists of four main stages, which are (1) determination of point correspondences on the stereo images, (2) correction of distortions in image coordinates, (3) derivation of 3D point coordinates from 2D correspondences, and (4) estimation of motion parameters based on 3D point correspondences. For the first stage of the system, we use a four-way matching algorithm to obtain matched point on two stereo image pairs at two consecutive time instants (t_i and t_{i + 1}). Since the input data are stereo images taken by cameras, it has two types of distortions, which are (i) film distortion and (ii) lens distortion. These two distortions must be corrected before any process can be applied on the matched points. To accomplish this goal, we use (i) bilinear transform for film distortion correction and (ii) lens formulas for lens distortion correction. After correcting the distortions, the results are 2D coordinates of each matched point that can be used to derive 3D coordinates. However, due to data noise, the calculated 3D coordinates to not usually represent a consistent rigid structure that is suitable for motion estimation; therefore, we suggest a procedure to select good 3D point sets as the input for motion estimation. The procedure exploits two constraints, rigidity between different time instants and uniform point distribution across the object on the image. For the last stage, we use an algorithm to estimate the motion parameters. We also wish to know what is the effect of quantization error on the estimated results; therefore an error analysis based on quantization error is performed on the estimated motion parameters. In order to test our system, eight sets of stereo image pairs are extracted from an outdoor stereo image sequence and used as the input data. The experimental results indicate that the proposed system does provide reasonable estimated motion parameters.     

一种基于三维模型投影的人脸识别新方法

研究了使用三维人脸模型进行不同姿势下的人脸识别问题,提出了一种三维建模二维识别的人脸识别算法,首先使用该方法将三维模型向不同方向投影,进而将不同姿势的二维图像与不同方向的投影结果相匹配,进行人脸识别。研究了使用Minolta Vivid 910进行数据获取,创建三维模型的方法和过程。实验结果表明,在进行不同姿势的人脸识别时,该方法的识别速度快于三维可变形模型方法,识别率远优于使用二维正面图像作为模板的人脸识别方法。     

Rapid analyte recognition in a device based on optical sensors and the olfactory system

We report here the development of a new vapor sensing device that is designed as an array of optically based chemosensors providing input to a pattern recognition system incorporating artificial neural networks. Distributed sensors providing inputs to an integrative circuit is a principle derived from studies of the vertebrate olfactory system. In the present device, primary chemosensing input is provided by an array of fiber-optic sensors. The individual fiber sensors, which are broadly yet differentially responsive, were constructed by immobilizing molecules of the fluorescent indicator dye Nile Red in polymer matrices of varying polarity, hydrophobicity, pore size, elasticity, and swelling tendency, creating unique sensing regions that interact differently with vapor molecules. The fluorescent signals obtained from each fiber sensor in response to 2-s applications of different analyte vapors have unique temporal characteristics. Using signals from the fiber array as inputs, artificial neural networks were trained to identify both single analytes and binary mixtures, as well as relative concentrations. Networks trained with integrated response data from the array or with temporal data from a single fiber made numerous errors in analyte identification across concentrations. However, when trained with temporal information from the fiber array, networks using "name" or "characteristic" output codes performed well in identifying test analytes.     

Processing of 3D sheet metal components in STEP AP-203 format. Part I: feature recognition system

To automate planning activities in a computer integrated manufacturing environment, an integrated system of feature recognition and reasoning is essential. An attempt is made in the present work to develop such a system for 3D sheet metal components. Though certain part-modellers use feature-based methodology, they lack the information required for manufacturing and entire feature information is lost when converted to a neutral format such as STEP AP-203. The proposed feature recognition identifies manufacturing features in a generic manner, while feature reasoning gives the information required for manufacturing. Taking 3D model data in STEP AP-203 format as input to the feature recognition system, the central plane of the component is first generated. Further processing of faces is carried out and various features with similar manufacturing attributes are identified using a set of rules based on the topology, geometry and Boolean logic. Different types of manufacturing features such as cut, stretched, drawn and bent features as well as composite features are effectively identified irrespective of their shape. The system proposed here was tested with components taken from industry and examples available in the published literature. The proposed feature recognition system serves as input to the feature reasoning system dealt with in Part II of this work (Kannan, T.R. and Shunmugam, M.S., Processing of 3D Sheet metal components in STEP AP-203 format. Part II: feature reasoning system. Int. J. Prod. Res., 2009 (in press)).     

一种基于视图和支持向量机的三维物体识别方法

为提高三维物体识别系统性能并减少计算复杂性,本文提出了一种基于视图的方法.首先从三维物体的二维视图中提取颜色矩、纹理特征和仿射不变矩.颜色矩对于物体的大小和姿态不敏感且性能稳健.纹理特征可区别形状相似但外观不同的物体.仿射不变矩在物体发生仿射形变下具有不变性.本文将上述各种特征组合为23个分量的特征向量,送入支持向量机进行训练并识别.基于两种公开的三维物体数据库COIL-100和ALOI测试了本文方法性能.当每物体训练视角为36个(视角间隔10°)时,在两个数据库上的实验都达到了100%的识别率.进一步减少训练视角数量也达到较满意的识别性能,优于文献中的方法.     

Spherical nonlinear correlations for global invariant three-dimensional object recognition

We define a nonlinear filtering based on correlations on unit spheres to obtain both rotation- and scale-invariant three-dimensional (3D) object detection. Tridimensionality is expressed in terms of range images. The phase Fourier transform (PhFT) of a range image provides information about the orientations of the 3D object surfaces. When the object is sequentially rotated, the amplitudes of the different PhFTs form a unit radius sphere. On the other hand, a scale change is equivalent to a multiplication of the amplitude of the PhFT by a constant factor. The effect of both rotation and scale changes for 3D objects means a change in the intensity of the unit radius sphere. We define a 3D filtering based on nonlinear operations between spherical correlations to achieve both scale- and rotation-invariant 3D object recognition.     

Optimal Contrast Correction for ICA-Based Fusion of Multimodal Images

In this paper, the authors revisit the previously proposed Image Fusion framework, based on self-trained Independent Component Analysis (ICA) bases. In the original framework, equal importance was given to all input images in the reconstruction of the “fused” image's intensity. Even though this assumption is valid for all applications involving sensors of the same modality, it might not be optimal in the case of multiple modality inputs of different intensity range. The authors propose a method for estimating the optimal intensity range (contrast) of the fused image via optimization of an image fusion index. The proposed approach can be employed in a general fusion scenario including multiple sensors.      

Three-dimensional object detection under arbitrary lighting conditions

A novel method of 3D object recognition independent of lighting conditions is presented. The recognition model is based on a vector space representation using an orthonormal basis generated by the Lambertian reflectance functions obtained with distant light sources. Changing the lighting conditions corresponds to multiplying the elementary images by a constant factor and because of that, all possible lighting views will be elements that belong to that vector space. The recognition method proposed is based on the calculation of the angle between the vector associated with a certain illuminated 3D object and that subspace. We define the angle in terms of linear correlations to get shift and illumination-invariant detection.     

MRMR Based Feature Vector Design for Efficient Citrus Disease Detection

In recent times, the images and videos have emerged as one of the most important information source depicting the real time scenarios. Digital images nowadays serve as input for many applications and replacing the manual methods due to their capabilities of 3D scene representation in 2D plane. The capabilities of digital images along with utilization of machine learning methodologies are showing promising accuracies in many applications of prediction and pattern recognition. One of the application fields pertains to detection of diseases occurring in the plants, which are destroying the widespread fields. Traditionally the disease detection process was done by a domain expert using manual examination and laboratory tests. This is a tedious and time consuming process and does not suffice the accuracy levels. This creates a room for the research in developing automation based methods where the images captured through sensors and cameras will be used for detection of disease and control its spreading. The digital images captured from the field's forms the dataset which trains the machine learning models to predict the nature of the disease. The accuracy of these models is greatly affected by the amount of noise and ailments present in the input images, appropriate segmentation methodology, feature vector development and the choice of machine learning algorithm. To ensure the high rated performance of the designed system the research is moving in a direction to fine tune each and every stage separately considering their dependencies on subsequent stages. Therefore the most optimum solution can be obtained by considering the image processing methodologies for improving the quality of image and then applying statistical methods for feature extraction and selection. The training vector thus developed is capable of presenting the relationship between the feature values and the target class. In this article, a highly accurate system model for detecting the diseases occurring in citrus fruits using a hybrid feature development approach is proposed. The overall improvement in terms of accuracy is measured and depicted.     

Three-dimensional range data compression using computer graphics rendering pipeline

This paper presents the idea of naturally encoding three-dimensional (3D) range data into regular two-dimensional (2D) images utilizing computer graphics rendering pipeline. The computer graphics pipeline provides a means to sample 3D geometry data into regular 2D images, and also to retrieve the depth information for each sampled pixel. The depth information for each pixel is further encoded into red, green, and blue color channels of regular 2D images. The 2D images can further be compressed with existing 2D image compression techniques. By this novel means, 3D geometry data obtained by 3D range scanners can be instantaneously compressed into 2D images, providing a novel way of storing 3D range data into its 2D counterparts. We will present experimental results to verify the performance of this proposed technique.     

Vectorial signatures for invariant recognition of position,rotation and scale pattern recognition

We investigate the development and utilization of vectorial signatures obtained from the application of properties of the scale and Fourier transform for images recognition. The signatures were applied to different input scenes. In order to find a new invariant digital system two one-dimensional vectors were calculated to achieve different mathematical transformations in the target as well as in the input scene. To recognize a target, signatures were compared, calculating the Euclidean distance between the target and the input scene. These sets are generated from two sources: alphabetical letters in Arial type and digital images of seven different copepod species. Considering the great similarity between the copepod species used in this work and between the male and female of each species the results obtained are very good.     

Using BlazePose on Spatial Temporal Graph Convolutional Networks for Action Recognition

The ever-growing available visual data (i.e., uploaded videos and pictures by internet users) has attracted the research community's attention in the computer vision field. Therefore, finding efficient solutions to extract knowledge from these sources is imperative. Recently, the BlazePose system has been released for skeleton extraction from images oriented to mobile devices. With this skeleton graph representation in place, a Spatial-Temporal Graph Convolutional Network can be implemented to predict the action. We hypothesize that just by changing the skeleton input data for a different set of joints that offers more information about the action of interest, it is possible to increase the performance of the Spatial-Temporal Graph Convolutional Network for HAR tasks. Hence, in this study, we present the first implementation of the BlazePose skeleton topology upon this architecture for action recognition. Moreover, we propose the Enhanced-BlazePose topology that can achieve better results than its predecessor. Additionally, we propose different skeleton detection thresholds that can improve the accuracy performance even further. We reached a top-1 accuracy performance of 40.1% on the Kinetics dataset. For the NTU-RGB+D dataset, we achieved 87.59% and 92.1% accuracy for Cross-Subject and Cross-View evaluation criteria, respectively.     

基于特征融合进行活动识别的DCNN 5法

研究了输入是可穿戴传感器获得的多通道时间序列信号,输出是预定义的活动的活动识别模型,指出活动中的有效特征的提取目前多依赖于手工和浅层特征学习结构,不仅复杂而且会导致识别准确率下降;基于深度学习的卷积神经网络（ CNN）不是对时间序列信号进行手工特征提取,而是自动学习最优特征;目前使用卷积神经网络处理有限标签数据仍存在过拟合问题。因此提出了一种基于融合特征的系统性的特征学习方法用于活动识别,用ImageNet16对原始数据集进行预训练,将得到的数据与原始数据进行融合,并将融合数据和对应的标签送入有监督的深度卷积神经网络（ DCNN ）中,训练新的系统。在该系统中,特征学习和分类是相互加强的,它不仅能处理端到端的有限数据问题,也能使学习到的特征有更强的辨别力。与其他方法相比,该方法整体精度从87.0％提高到87.4％。     

3D face detection based on salient features extraction and skin colour detection using data mining

Face detection has an essential role in many applications. In this paper, we propose an efficient and robust method for face detection on a 3D point cloud represented by a weighted graph. This method classifies graph vertices as skin and non-skin regions based on a data mining predictive model. Then, the saliency degree of vertices is computed to identify the possible candidate face features. Finally, the matching between non-skin regions representing eyes, mouth and eyebrows and salient regions is done by detecting collisions between polytopes, representing these two regions. This method extracts faces from situations where pose variation and change of expressions can be found. The robustness is showed through different experimental results. Moreover, we study the stability of our method according to noise. Furthermore, we show that our method deals with 2D images.     

3D model-based pose invariant face recognition from multiple views

A 3D model-based pose invariant face recognition method that can recognise a human face from its multiple views is proposed. First, pose estimation and 3D face model adaptation are achieved by means of a three-layer linear iterative process. Frontal view face images are synthesised using the estimated 3D models and poses. Then the discriminant `waveletfaces' are extracted from these synthesised frontal view images. Finally, corresponding nearest feature space classifier is implemented. Experimental results show that the proposed method can recognise faces under variable poses with good accuracy     

Image-based rendering for photo-realistic visualization

Digital models of objects have long stood synonymous for 3D models with fixed textures. More recently, image-based rendering has made its way to offer an effective alternative. Rather than producing a 3D model, views are created by interpolating between images taken from many different viewpoints. The advantages are that the resulting visualizations look very realistic and that a wider range of objects can be dealt with. One can go a step further and follow a similar strategy for the illumination direction: for every reference viewpoint, now take multiple images, each with a different illumination. Again, images for novel illumination directions follow from interpolation. But there is an obvious catch in that many, many images are needed, unless at least a crude 3D shape model and preferably also a model for the surface reflectance are used to support the interpolation from more sparsely taken images. Even then, the number of images needed remains appreciable. Hence, we present a dome to efficiently capture such images.

Fortunately, it is possible to generate this crude 3D model, as well as the surface reflectance characteristics' directly from the images. We describe methods to achieve this. Also, in order to make real-time visualization possible, all critical steps of the visualization pipeline are programmed on off-the-shelf graphics hardware. The dome provides an easy to use and structured acquisition procedure. Yet, the applicability of the algorithms is not limited to structured input. The images could, for example be taken with a hand-held camera.     

Image-based rendering for photo-realistic visualization

Digital models of objects have long stood synonymous for 3D models with fixed textures. More recently, image-based rendering has made its way to offer an effective alternative. Rather than producing a 3D model, views are created by interpolating between images taken from many different viewpoints. The advantages are that the resulting visualizations look very realistic and that a wider range of objects can be dealt with. One can go a step further and follow a similar strategy for the illumination direction: for every reference viewpoint, now take multiple images, each with a different illumination. Again, images for novel illumination directions follow from interpolation. But there is an obvious catch in that many, many images are needed, unless at least a crude 3D shape model and preferably also a model for the surface reflectance are used to support the interpolation from more sparsely taken images. Even then, the number of images needed remains appreciable. Hence, we present a dome to efficiently capture such images.

Fortunately, it is possible to generate this crude 3D model, as well as the surface reflectance characteristics’ directly from the images. We describe methods to achieve this. Also, in order to make real-time visualization possible, all critical steps of the visualization pipeline are programmed on off-the-shelf graphics hardware. The dome provides an easy to use and structured acquisition procedure. Yet, the applicability of the algorithms is not limited to structured input. The images could, for example be taken with a hand-held camera.     

Efficient 3D point clouds classification for face detection using linear programming and data mining

Most of the applications related to security and biometric rely on skin region detection such as face detection, adult 3D objects filtering, and gesture recognition. In this paper, we propose a robust method for skin detection on 3D coloured point clouds. Then, we extend this method to solve the problem of 3D face detection. To do so, we construct a weighted graph from initial coloured 3D point clouds. Then, we present a linear programming algorithm using a predictive model based on a data mining approach to classify and label graph vertices as skin and non-skin regions. Moreover, we apply some refinement rules on skin regions to confirm the presence of a face. Furthermore, we demonstrate the robustness of our method by showing and analysing some experimental results. Finally, we show that our method deals with many data that can be represented by a weighted graph such as 2D images and 3D models.     

3D Point Cloud Analysis for Detection and Characterization of Defects on Airplane Exterior Surface

Three-dimensional surface defect inspection remains a challenging task. This paper describes a novel automatic vision-based inspection system that is capable of detecting and characterizing defects on an airplane exterior surface. By analyzing 3D data collected with a 3D scanner, our method aims to identify and extract the information about the undesired defects such as dents, protrusions or scratches based on local surface properties. Surface dents and protrusions are identified as the deviations from an ideal, smooth surface. Given an unorganized point cloud, we first smooth noisy data by using Moving Least Squares algorithm. The curvature and normal information are then estimated at every point in the input data. As a next step, Region Growing segmentation algorithm divides the point cloud into defective and non-defective regions using the local normal and curvature information. Further, the convex hull around each defective region is calculated in order to englobe the suspicious irregularity. Finally, we use our new technique to measure the dimension, depth, and orientation of the defects. We tested and validated our novel approach on real aircraft data obtained from an Airbus A320, for different types of defect. The accuracy of the system is evaluated by comparing the measurements of our approach with ground truth measurements obtained by a high-accuracy measuring device. The result shows that our work is robust, effective and promising for industrial applications.