Automated eye tracking system calibration using artificial neural networks

The electro-oculogram (EOG) continues to be widely used to record eye movements especially in clinical settings. However, an efficient and accurate means of converting these recordings into eye position is lacking. An artificial neural network (ANN) that maps two-dimensional (2D) eye movement recordings into 2D eye positions can enhance the utility of such recordings. Multi-layer perceptrons (MLPs) with non-linear activation functions and trained with back propagation proved to be capable of calibrating simulated EOG data to a mean accuracy of 0.33 degrees . Linear perceptrons (LPs) were only nearly half as accurate. For five subjects, the mean accuracy provided by the MLPs was 1.09 degrees of visual angle ( degrees ) for EOG data, and 0.98 degrees for an infrared eye tracker. MLPs enabled calibration of 2D saccadic EOG to an accuracy not significantly different from that obtained with the infrared tracker. Using initial weights trained on another person reduced MLP training time, reaching convergence in as little as 20 iterations.     

Camera calibration using identical objects

This paper describes a method for camera calibration using identical products. In this paper, we postulate an imaginative rigid motion between any two identical products, and the imaginative rigid motion could offer a pair of circular points. As is known, three pairs of projections of the circular points are needed to result in the closed-form solution for calibration. In our method, we obtain three pairs of projections of the circular points from only two images of three identical products, or three images of two identical products. When only two identical products are utilized, our method is almost the dual of the stereo calibration from rigid motions. A direct approach is taken here instead of the two-step process in stereo calibration. Furthermore, a better projective reconstruction could be performed from the estimation of the camera parameters to avoid the dominant projective-to-affine error in the stereo calibration. Finally, we conduct a nonlinear refinement based on the maximum likelihood estimation. The experimental results from synthetic data and real data prove our method convenient and robust to noise.     

双目摄像机定标的神经网络方法

研究基于反向传播神经网络的摄像机双目立体视觉定标新方法。传统方法基于三角测量原理技术,会带入成像畸变非线性误差,而这种新方法可以消除非线性因素的影响。该方法利用了BP网络良好的非线性映射能力以及学习、泛化能力,通过采用高精度样本数据训练BP网络,最终建立起立体视觉定标的网络模型。由于不需要考虑视觉模型误差、光学调整误差、广角畸变等因素对视觉检测系统测量精度的影响,因而能够有效地克服常规建模方法的不足,保证了检测系统具有较高的精度。     

A practical use of cellular neural networks: the stereo-vision problem as an optimisation

A variational way of deriving the relevant parameters of a cellular neural network (CNN) is introduced. The approach exploits the CNN spontaneous internal-energy decrease and is applicable when a given problem can be expressed in terms of an optimisation task. The presented approach is fully mathematical as compared with the typical heuristic search for the correct parameters in the literature on CNNs. This method is practically employed in recovering information on the three-dimensional structure of the environment, through the stereo vision problem. A CNN able to find the conjugate points in a stereogram is fully derived in the proposed framework. Results of computer simulations on several test cases are provided. Received: 1 August 1997 / Accepted: 29 September 1999     

3D reconstruction and face recognition using kernel-based ICA and neural networks

Kernel-based nonlinear characteristic extraction and classification algorithms are popular new research directions in machine learning. In this paper, we propose an improved photometric stereo scheme based on improved kernel-independent component analysis method to reconstruct 3D human faces. Next, we fetch the information of 3D faces for facial face recognition. For reconstruction, we obtain the correct normal vector’s sequence to form the surface, and use a method for enforcing integrability to reconstruct 3D objects. We test our algorithm on a number of real images captured from the Yale Face Database B, and use three kinds of methods to fetch characteristic values. Those methods are called contour-based, circle-based, and feature-based methods. Then, a three-layer, feed-forward neural network trained by a back-propagation algorithm is used to realize a classifier. All the experimental results were compared to those of the existing human face reconstruction and recognition approaches tested on the same images. The experimental results demonstrate that the proposed improved kernel independent component analysis (IKICA) method is efficient in reconstruction and face recognition applications.     

基于径向基神经网络的双目摄像机标定方法的研究

传统的摄像机标定方法需要建立复杂的数学模型,计算量大、实时性不好.针对双目摄像机标定问题,提出了一种基于径向基函数(RBF)神经网络的双目摄像机标定方法,利用该网络具有很强的自组织、自学习、自适应和较强的非线性映射能力,准确的建立了双目视觉中三维空间物点坐标和两个摄像机坐标间的关系,与传统的方法相比,该方法具有重建速度快,运算精度高等优点.仿真结果表明该方法是正确性和有效性.     

Analysis of Tsai calibration method using two- and three-dimensional calibration objects

Camera calibration is a fundamental process for both photogrammetric and computer vision. Since the arrival of the direct linear transformation method and its later revisions, new methods have been developed by several authors, such as: Tsai, Heikkilä and Zhang. Most of these have been based on the pinhole model, including distortion correction. Some of these methods, such as Tsai method, allow the use of two different techniques for determining calibration parameters: a non-coplanar calibration technique using three-dimensional (3D) calibration objects, and a coplanar technique that uses two-dimensional (2D) calibration objects. The calibration performed by observing a 3D calibration object has good accuracy, and produces very efficient results; however, the calibration object must be accurate enough and requires an elaborate configuration. In contrast, the use of 2D calibration objects yields less accurate results, is much more flexible, and does not require complex calibration objects that are costly to produce. This article compares these two different calibration procedures from the perspective of stereo measurement. Particular attention was focused on the accuracy of the calculated camera parameters, the reconstruction error in the computer image coordinates and in the world coordinate system and advanced image-processing techniques for subpixel detection during the comparison. The purpose of this work is to establish a basis and selection criteria for choosing one of these techniques for camera calibration, according to the accuracy required in each of the many applications using photogrammetric vision: robot calibration methods, trajectory generation algorithms, articulated measuring arm calibration, and photogrammetric systems.     

Maximum entropy signal reconstruction with neural networks

The implementation of the maximum entropy reconstruction algorithms by means of neural networks is discussed. It is shown that the solutions of the maximum entropy problem correspond to the steady states of the appropriate Hopfield net. The choice of network parameters is discussed, and existence of the maximum entropy solution is proved.     

Camera calibration using court models for real-time augmenting soccer scenes

In this paper, we present a procedure to estimate the position, orientation and focal length of a camera in a soccer field. These parameters are then used in real-time overlay of graphics on a soccer pitch. The method uses court model composed by arcs and lines. A means of automatically initializing the tracking process is also presented which uses Hough transform with a combination of a non-linear least squares optimization method. For the tracking of camera parameters, two cases arise: the center of the pitch and the 18 m area. A combination of automatic court model recognition with the Kanade-Lucas-Tomasi (KLT) algorithm is also used.     

Job-shop scheduling using neural networks

Job-shop scheduling cannot easily be analytically accomplished, so, it is done by computer simulation using heuristic priority rules. The SLACK rule for calculating the margins of jobs to their due-dates is effective in meeting the due-dates. However, the calculated margins are not precise because the actual margin is shortened due to conflicts with other jobs. The authors propose a method for estimating the margins by using a neural network. It is found that the method is effective for improving the average lateness to due-dates but not the maximum lateness. This paper proposes a method for adding a second neural network for judging the reliability of the estimated margins composed to the first one and for switching to the margins calculated by the SLACK rule when the reliability is low. The proposed method is verified by scheduling simulations to be effective in decreasing the maximum lateness to due-dates as much as the average lateness.     

Aeromagnetic compensation using neural networks

Airborne magnetic surveys in geophysical exploration can be subject to interference effects from the aircraft. Principal sources are the permanent magnetism of various parts of the aircraft, induction effects created by the earth's magnetic field and eddy-current fields produced by the aircraft's manoeuvres. Neural networks can model these effects as functions of roll, pitch, heading and their time derivatives, together with vertical acceleration, charging currents to the generator, etc., without assuming an explicit physical model. Separation of interference effects from background regional and diurnal fields can also be achieved in a satisfactory way.     

Dynamic modelling using neural networks

A neural network structure is presented that uses feedback of unmeasured system states to represent dynamic systems more efficiently than conventional feedforward and recurrent networks, leading to better predictions, reduced training requirement and more reliable extrapolation. The structure identifies the actual system states based on imperfect knowledge of the initial state, which is available in many practical systems, and is therefore applicable only to such systems. It also enables a natural integration of any available partial state-space model directly into the prediction scheme, to achieve further performance improvement. Simulation examples of three varied dynamic systems illustrate the various options and advantages offered by the state-feedback neural structure. Although the advantages of the proposed structure, compared with the conventional feedforward and recurrent networks, should hold for most practical dynamic systems, artificial systems can readily be created and real systems can surely be found for which one or more of these advantages would vanish or even get reversed. Caution is therefore recommended against interpreting the suggested advantages as strict theorems valid in all situations.     

Portfolio selection using neural networks

In this paper we apply a heuristic method based on artificial neural networks (NN) in order to trace out the efficient frontier associated to the portfolio selection problem. We consider a generalization of the standard Markowitz mean-variance model which includes cardinality and bounding constraints. These constraints ensure the investment in a given number of different assets and limit the amount of capital to be invested in each asset. We present some experimental results obtained with the NN heuristic and we compare them to those obtained with three previous heuristic methods. The portfolio selection problem is an instance from the family of quadratic programming problems when the standard Markowitz mean-variance model is considered. But if this model is generalized to include cardinality and bounding constraints, then the portfolio selection problem becomes a mixed quadratic and integer programming problem. When considering the latter model, there is not any exact algorithm able to solve the portfolio selection problem in an efficient way. The use of heuristic algorithms in this case is imperative. In the past some heuristic methods based mainly on evolutionary algorithms, tabu search and simulated annealing have been developed. The purpose of this paper is to consider a particular neural network (NN) model, the Hopfield network, which has been used to solve some other optimisation problems and apply it here to the portfolio selection problem, comparing the new results to those obtained with previous heuristic algorithms.     

Coordinated three-dimensional motion of the head and torso bydynamic neural networks

The problem of trajectory tracking control of a three dimensional (3D) model of the human upper torso and head is considered. The torso and the head are modeled as two rigid bodies connected at one point, and the Newton-Euler method is used to derive the nonlinear differential equations that govern the motion of the system. The two-link system is driven by six pairs of muscle like actuators that possess physiologically inspired alpha like and gamma like inputs, and spindle like and Golgi tendon organ like outputs. These outputs are utilized as reflex feedback for stability and stiffness control, in a long loop feedback for the purpose of estimating the state of the system (somesthesis), and as part of the input to the controller. Ideal delays of different duration are included in the feedforward and feedback paths of the system to emulate such delays encountered in physiological systems. Dynamical neural networks are trained to learn effective control of the desired maneuvers of the system. The feasibility of the controller is demonstrated by computer simulation of the successful execution of the desired maneuvers. This work demonstrates the capabilities of neural circuits in controlling highly nonlinear systems with multidelays in their feedforward and feedback paths. The ultimate long range goal of this research is toward understanding the working of the central nervous system in controlling movement. It is an interdisciplinary effort relying on mechanics, biomechanics, neuroscience, system theory, physiology and anatomy, and its short range relevance to rehabilitation must be noted.     

Camera calibration from surfaces of revolution

This paper addresses the problem of calibrating a pinhole camera from images of a surface of revolution. Camera calibration is the process of determining the intrinsic or internal parameters (i.e., aspect ratio, focal length, and principal point) of a camera, and it is important for both motion estimation and metric reconstruction of 3D models. In this paper, a novel and simple calibration technique is introduced, which is based on exploiting the symmetry of images of surfaces of revolution. Traditional techniques for camera calibration involve taking images of some precisely machined calibration pattern (such as a calibration grid). The use of surfaces of revolution, which are commonly found in daily life (e.g., bowls and vases), makes the process easier as a result of the reduced cost and increased accessibility of the calibration objects. In this paper, it is shown that two images of a surface of revolution will provide enough information for determining the aspect ratio, focal length, and principal point of a camera with fixed intrinsic parameters. The algorithms presented in this paper have been implemented and tested with both synthetic and real data. Experimental results show that the camera calibration method presented is both practical and accurate.     

Camera calibration from images of spheres

This paper introduces a novel approach for solving the problem of camera calibration from spheres. By exploiting the relationship between the dual images of spheres and the dual image of the absolute conic (IAC), it is shown that the common pole and polar with regard to the conic images of two spheres are also the pole and polar with regard to the IAC. This provides two constraints for estimating the IAC and, hence, allows a camera to be calibrated from an image of at least three spheres. Experimental results show the feasibility of the proposed approach     

Camera assisted calibration of SCARA arms

This paper presents some experimental results obtained by applying an approach of Lenz and Tsai (1989) to SCARA arm calibration. A new camera calibration technique, which is singular-free for this type of system, is employed. Several practical recommendations are given for robot users in general and SCARA arm users in particular     

Multi-class pattern classification using neural networks

Multi-class pattern classification has many applications including text document classification, speech recognition, object recognition, etc. Multi-class pattern classification using neural networks is not a trivial extension from two-class neural networks. This paper presents a comprehensive and competitive study in multi-class neural learning with focuses on issues including neural network architecture, encoding schemes, training methodology and training time complexity. Our study includes multi-class pattern classification using either a system of multiple neural networks or a single neural network, and modeling pattern classes using one-against-all, one-against-one, one-against-higher-order, and P-against-Q. We also discuss implementations of these approaches and analyze training time complexity associated with each approach. We evaluate six different neural network system architectures for multi-class pattern classification along the dimensions of imbalanced data, large number of pattern classes, large vs. small training data through experiments conducted on well-known benchmark data.     

Graph partitioning using annealed neural networks

A new algorithm, mean field annealing (MFA), is applied to the graph-partitioning problem. The MFA algorithm combines characteristics of the simulated-annealing algorithm and the Hopfield neural network. MFA exhibits the rapid convergence of the neural network while preserving the solution quality afforded by simulated annealing (SA). The rate of convergence of MFA on graph bipartitioning problems is 10-100 times that of SA, with nearly equal quality of solutions. A new modification to mean-field annealing is also presented which supports partitioning graphs into three or more bins, a problem which has previously shown resistance to solution by neural networks. The temperature-behavior of MFA during graph partitioning is analyzed approximately and shown to possess a critical temperature at which most of the optimization occurs. This temperature is analogous to the gain of the neurons in a neural network and can be used to tune such networks for better performance. The value of the repulsion penalty needed to force MFA (or a neural network) to divide a graph into equal-sized pieces is also estimated.