Consistent labeling of tracked objects in multiple cameras with overlapping fields of view

We address the issue of tracking moving objects in an environment covered by multiple uncalibrated cameras with overlapping fields of view, typical of most surveillance setups. In such a scenario, it is essential to establish correspondence between tracks of the same object, seen in different cameras, to recover complete information about the object. We call this the problem of consistent labeling of objects when seen in multiple cameras. We employ a novel approach of finding the limits of field of view (FOV) of each camera as visible in the other cameras. We show that, if the FOV lines are known, it is possible to disambiguate between multiple possibilities for correspondence. We present a method to automatically recover these lines by observing motion in the environment, Furthermore, once these lines are initialized, the homography between the views can also be recovered. We present results on indoor and outdoor sequences containing persons and vehicles.     

面向物流分拣的多立体摄像头物体操作系统

为满足物流分拣的低成本和实时性要求,提出了基于多个立体摄像头的系统获取典型物体的完整立体信息的方法,并结合机械臂搭建了实验硬件平台。实验采用了2个微软Kinect摄像头在水平面上实现了约3 mm精度的物体定位,根据物体的立体信息建立立体模型,并计算了物体的取向、尺寸、含有的平面等多个可用于物体操作的立体特征,计算速率约为1 s/帧。根据这些信息,使用了机械臂成功进行了连续100次抓取。实验结果表明,这套方法和平台无需离线学习即可以实时提取多种尺寸和形状的物体的立体特征,机械臂可以基于此进行精度较高的物体操作。     

Dynamic self-organizing maps with controlled growth for knowledgediscovery

The growing self-organizing map (GSOM) algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and continue with finer clustering of the interesting clusters only. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and of smaller volume. Therefore, this method facilitates the analysis of even very large data sets.     

Fractal initialization for high-quality mapping with self-organizing maps

Initialization of self-organizing maps is typically based on random vectors within the given input space. The implicit problem with random initialization is the overlap (entanglement) of connections between neurons. In this paper, we present a new method of initialization based on a set of self-similar curves known as Hilbert curves. Hilbert curves can be scaled in network size for the number of neurons based on a simple recursive (fractal) technique, implicit in the properties of Hilbert curves. We have shown that when using Hilbert curve vector (HCV) initialization in both classical SOM algorithm and in a parallel-growing algorithm (ParaSOM), the neural network reaches better coverage and faster organization.     

多摄像机图像间的稳定切换控制

为获得连续动态的图像雅可比矩阵,分析了融合方式和传统图像直接切换方法的缺陷,提出了一种基于融合的多图像稳定切换算法,算法采用动态加权融合策略。在移动机器人位置未知和无标定多摄像机的情况下,仿真和实验结果表明:该算法比传统方法具有更高的适应能力,图像切换过程的稳定性得到了很大提高。     

Exploiting spectral and spatial information for the identification of hazelnut fields using self-organizing maps

Automatic land-cover identification using remote-sensing images is essential for agricultural management and monitoring, which is an ongoing challenge. For permanent crops, which are of great importance economically and environmentally, it becomes even more challenging mainly due to the varying statistics of orchards such as the existence of different orchard types, different crown sizes even for the same type, different distances between orchards among various fields and overlapping crowns. This challenge necessitates the utilization of both spectral values and spatial relations of pixels. To accurately determine the fields of permanent crops, hazelnuts in particular, a classification system with hybrid learning, which merges an image features map (IFM) and learning vector quantization (LVQ), is proposed in this study. IFM is a variant of a self-organizing map (an unsupervised neural learning paradigm successfully used in many applications of remote-sensing imagery), which exploits both spectral and spatial information without additional computation of texture. LVQ, however, is supervised learning for fine-tuning of class boundaries. Experimental results on finding hazelnut fields using multispectral QuickBird imagery indicate that the proposed method achieves acceptable accuracies and often produces more accurate extraction than the accuracies obtained based only on spectral or on spatial information.     

EGT for multiple view geometry and visual servoing: robotics vision with pinhole and panoramic cameras

The Epipolar Geometry Toolbox (EGT) for MATLAB is a software package targeted to research and education in computer vision and robotics visual servoing. It provides the user with a wide set of functions for designing multicamera systems for both pinhole and panoramic cameras. Functions include camera placement and visualization, computation, and estimation of epipolar geometry entities. The compatibility of EGT with the Robotics Toolbox enables users to address general vision-based control issues. Two applications of EGT to visual servoing tasks are examined in this article. Several epipolar geometry estimation algorithms have been implemented.     

Cooperative information maximization with Gaussian activation functions for self-organizing maps

In this paper, we propose a new information-theoretic method to produce explicit self-organizing maps (SOMs). Competition is realized by maximizing mutual information between input patterns and competitive units. Competitive unit outputs are computed by the Gaussian function of distance between input patterns and competitive units. A property of this Gaussian function is that, as distance becomes smaller, a neuron tends to fire strongly. Cooperation processes are realized by taking into account the firing rates of neighboring neurons. We applied our method to uniform distribution learning, chemical compound classification and road classification. Experimental results confirmed that cooperation processes could significantly increase information content in input patterns. When cooperative operations are not effective in increasing information, mutual information as well as entropy maximization is used to increase information. Experimental results showed that entropy maximization could be used to increase information and to obtain clearer SOMs, because competitive units are forced to be equally used on average.     

Hierarchical clustering of self-organizing maps for cloud classification

This paper presents a new method for segmenting multispectral satellite images. The proposed method is unsupervised and consists of two steps. During the first step the pixels of a learning set are summarized by a set of codebook vectors using a Probabilistic Self-Organizing Map (PSOM, Statistique et méthodes neuronales, Dunod, Paris, 1997). In a second step the codebook vectors of the map are clustered using Agglomerative Hierarchical Clustering (AHC, Pattern Recognition and Neural Networks, Cambridge University Press, Cambridge, 1996). Each pixel takes the label of its nearest codebook vector. A practical application to Meteosat images illustrates the relevance of our approach.     

Ranked centroid projection: a data visualization approach with self-organizing maps

The self-organizing map (SOM) is an efficient tool for visualizing high-dimensional data. In this paper, the clustering and visualization capabilities of the SOM, especially in the analysis of textual data, i.e., document collections, are reviewed and further developed. A novel clustering and visualization approach based on the SOM is proposed for the task of text mining. The proposed approach first transforms the document space into a multidimensional vector space by means of document encoding. Afterwards, a growing hierarchical SOM (GHSOM) is trained and used as a baseline structure to automatically produce maps with various levels of detail. Following the GHSOM training, the new projection method, namely the ranked centroid projection (RCP), is applied to project the input vectors to a hierarchy of 2-D output maps. The RCP is used as a data analysis tool as well as a direct interface to the data. In a set of simulations, the proposed approach is applied to an illustrative data set and two real-world scientific document collections to demonstrate its applicability.     

A patent quality analysis and classification system using self-organizing maps with support vector machine

A plethora of patents are approved by the patent officers each year and current patent systems face a solemn quandary of evaluating these patents’ qualities. Traditional researchers and analyzers have fixated on developing sundry patent quality indicators only, but these indicators do not have further prognosticating power on incipient patent applications or publications. Therefore, the data mining (DM) approaches are employed in this article to identify and to classify the new patent's quality in time. An automatic patent quality analysis and classification system, namely SOM-KPCA-SVM, is developed according to patent quality indicators and characteristics, respectively. First, the self-organizing map (SOM) approach is used to cluster patents published before into different quality groups according to the patent quality indicators and defines group quality type instead of via experts. The kernel principal component analysis (KPCA) approach is used to transform nonlinear feature space in order to improve classification performance. Finally, the support vector machine (SVM) is used to build up the patent quality classification model. The proposed SOM-KPCA-SVM is applied to classify patent quality automatically in patent data of the thin film solar cell. Experimental results show that our proposed system can capture the analysis effectively compared with traditional manpower approach.     

Radiance-based color calibration for image-based modeling with multiple cameras

Photo-consistency estimation is an important part for many image-based modeling techniques.This paper presents a novel radiance-based color calibration method to reduce the uncertainty of photo-consistency estimation across multiple cameras.The idea behind our method is to convert colors into a uniform radiometric color space in which multiple image data are corrected.Experimental results demonstrate that our method can achieve comparable color calibration effect without adjusting camera parameters and is more robust than other existing method.Additionally,we obtain an auto-determined threshold for photo-consistency check,which will lead to a better performance than existing photo-consistency based reconstruction algorithms.     

非线性系统的自组织多模型自适应逆控制

针对离散时间非线性系统,在分析基于自组织映射神经网络的多模型控制方法的基础上,提出了一种自组织多模型直接逆控制方法.并分析了控制误差的有界性.进一步,借鉴参数空间多模型方法的切换一自适应策略,在固定模型的基础上增加一个参数可调节的自适应逆模型.提高了稳态控制性能.仿真实例表明,对于变化较快的信号,自组织多模型直接逆控制器和自组织多模型自适应逆控制器都能进行有界跟踪,对于稳态信号,自组织多模型自适应逆控制器还能进行渐近跟踪.     

Automatic generation of rules for a system that uses a grammatical approach to syntactic analysis

The feasibility of creating a method for the automatic generation of rules for systems that use grammatical syntactic analysis is shown. The method for creating new rules for the “Krossleitor” machine translation system, which allows the disassembly of sentences that could not be parsed with the help of the initial rule base, is described.     

Coherent video generation for multiple hand-held cameras with dynamic foreground

Computational Visual Media - For many social events such as public performances, multiple hand-held cameras may capture the same event. This footage is often collected by amateur cinematographers...     

ParaGro: a learning algorithm for growing parallel self-organizing maps with any input/output dimensions

Self-organizing maps (SOM) have become popular for tasks in data visualization, pattern classification or natural language processing and can be seen as one of the major contemporary concepts for artificial neural networks. The general idea is to approximate a high dimensional and previously unknown input distribution by a lower dimensional neural network structure so that the topology of the input space is mapped closely. Not only is the general topology retained but the relative densities of the input space are reflected in the final output. Kohonen maps also have the property of neighbor influence. That is, when a neuron decides to move, it pulls all of its neighbors in the same direction modified by an elasticity factor. We present a SOM that processes the whole input in parallel and organizes itself over time. The main reason for parallel input processing lies in the fact that knowledge can be used to recognize parts of patterns in the input space that have already been learned. Thus, networks can be developed that do not reorganize their structure from scratch every time a new set of input vectors is presented, but rather adjust their internal architecture in accordance with previous mappings. One basic application could be a modeling of the whole–part relationship through layered architectures.

The presented neural network model implements growing parallel SOM structure for any input and any output dimension. The advantage of the proposed algorithm is in its property of processing the whole input space in one step. All nodes of the network compute their step simultaneously, and are, therefore, able to detect known patterns without reorganizing. The simulation results support the theoretical framework presented in the following sections.     

Reliability-based camera handoff for cooperative tracking with multiple pan-tilt cameras

We present a new handoff method for multiple pan-tilt cameras for mobile robot tracking in an indoor environment. Camera handoff is an important step to consistently maintain the visibility of a mobile robot with maximized object tracking accuracy. First, we propose a method to estimate the position of a mobile robot using single pan-tilt camera. Then, the concept of position reliability is defined to quantitatively evaluate the accuracy of position estimation and tracking ability of individual pan-tilt cameras. Position reliability is used to decide when to trigger handoff and who to response handoff in the proposed handoff algorithm. Experimental results demonstrate that four pan-tilt cameras can systematically track a mobile robot in an indoor environment using the proposed method.     

Structure from motion with wide circular field of view cameras

This paper presents a method for fully automatic and robust estimation of two-view geometry, autocalibration, and 3D metric reconstruction from point correspondences in images taken by cameras with wide circular field of view. We focus on cameras which have more than 180/spl deg/ field of view and for which the standard perspective camera model is not sufficient, e.g., the cameras equipped with circular fish-eye lenses Nikon FC-E8 (183/spl deg/), Sigma 8 mm-f4-EX (180/spl deg/), or with curved conical mirrors. We assume a circular field of view and axially symmetric image projection to autocalibrate the cameras. Many wide field of view cameras can still be modeled by the central projection followed by a nonlinear image mapping. Examples are the above-mentioned fish-eye lenses and properly assembled catadioptric cameras with conical mirrors. We show that epipolar geometry of these cameras can be estimated from a small number of correspondences by solving a polynomial eigenvalue problem. This allows the use of efficient RANSAC robust estimation to find the image projection model, the epipolar geometry, and the selection of true point correspondences from tentative correspondences contaminated by mismatches. Real catadioptric cameras are often slightly noncentral. We show that the proposed autocalibration with approximate central models is usually good enough to get correct point correspondences which can be used with accurate noncentral models in a bundle adjustment to obtain accurate 3D scene reconstruction. Noncentral camera models are dealt with and results are shown for catadioptric cameras with parabolic and spherical mirrors.     

Using self-organizing maps with complex network topologies and coalitions for time series prediction

A Self-organizing Map (SOM) is a competitive learning neural network architecture that make available a certain amount of classificatory neurons, which self-organize spatially based on input patterns. In this paper we explore the use of complex network topologies, like small-world, scale-free or random networks; for connecting the neurons within a SOM, and apply them for Time Series Prediction. We follow the VQTAM model for function prediction, and consider several benchmarks to evaluate the quality of the predictions. Afterwards, we introduce the CASOM algorithm (Coalitions and SOM) that uses coalitions to temporarily extend the neighborhood of a neuron, and to provide more accuracy in prediction problems than classical SOM. The results presented in this work suggest that the most regular the network topology is, the better results it provides in prediction. Besides, we have found that not updating all the neurons at the same time provides much better results. Finally, we describe how the use of coalitions can enhance the capability of SOM for Time Series Prediction.