Algorithms for coplanar camera calibration

Abstract. Coplanar camera calibration is the process of determining the extrinsic and intrinsic camera parameters from a given set of image and world points, when the world points lie on a two-dimensional plane. Noncoplanar calibration, on the other hand, involves world points that do not lie on a plane. While optimal solutions for both the camera-calibration procedures can be obtained by solving a set of constrained nonlinear optimization problems, there are significant structural differences between the two formulations. We investigate the computational and algorithmic implications of such underlying differences, and provide a set of efficient algorithms that are specifically tailored for the coplanar case. More specifically, we offer the following: (1) four algorithms for coplanar calibration that use linear or iterative linear methods to solve the underlying nonlinear optimization problem, and produce sub-optimal solutions. These algorithms are motivated by their computational efficiency and are useful for real-time low-cost systems. (2) Two optimal solutions for coplanar calibration, including one novel nonlinear algorithm. A constraint for the optimal estimation of extrinsic parameters is also given. (3) A Lyapunov type convergence analysis for the new nonlinear algorithm. We test the validity and performance of the calibration procedures with both synthetic and real images. The results consistently show significant improvements over less complete camera models. Received: 30 September 1998 / Accepted: 12 January 2000     

Towards the recovery of extrinsic camera parameters from video records of sewer surveys

The aim of the work reported here is the recovery, from a single image taken inside a roughly cylindrical brick sewer pipe of diameter up to one meter, of the pose of the camera relative to the central axis of the pipe. It is shown that the vanishing point associated with the longitudinal mortar lines carries valuable information about the pose. A method for the automatic detection of this point is presented and used to analyse the camera rotations underlying a number of sewer survey videos. It is similarly shown how the angles between the images of the longitudinal lines can be used to recover information about camera pose. The techniques might form an active part of a more comprehensive image understanding system recovering the three-dimensional shape of a surveyed pipe from survey videos and/or be used as an experimental tool during the design of such a system. Received: 24 June 1997 / Accepted: 17 March 1998     

Dry camera calibration for underwater applications

Is it possible to calibrate a camera in the air and then use the calibration results to infer a new calibration corresponding to the embedding of the camera in another fluid (possibly water)? This problem is dealt within the paper. It is important to avoid direct underwater calibration, as it is much more inconvenient for experiments than the usual (air) calibration by human workers. Optical laws that must be considered when using underwater cameras are investigated. Both theoretical and experimental point of views are described, and it is shown that relationships can be found between results of air and water (or any other isotropic fluid in which the camera can be submerged) calibration. Received: 22 April 2000 / Accepted: 13 May 2002 Correspondence to: J.M. Lavest     

Range image segmentation using local approximation of scan lines with application to CAD model acquisition

Abstract. Automatic acquisition of CAD models from existing objects requires accurate extraction of geometric and topological information from the input data. This paper presents a range image segmentation method based on local approximation of scan lines. The method employs edge models that are capable of detecting noise pixels as well as position and orientation discontinuities of varying strengths. Region-based techniques are then used to achieve a complete segmentation. Finally, a geometric representation of the scene, in the form of a surface CAD model, is produced. Experimental results on a large number of real range images acquired by different range sensors demonstrate the efficiency and robustness of the method. Received: 1 August 2000 / Accepted: 23 January 2002 Correspondence to: I. Khalifa     

Empirical evaluation of a calibration chart detector

In this paper, we systematically assess the performance of an automatic calibration chart detector. Through simulation we establish the optimal set of control parameters and the rate of successful detection as a function of pose. We validate the simulation results on real images taken from a camera mounted on a robot arm. The results confirm the utility of such simulation studies. The feedback obtained suggested a number of modifications for the chart detection system, which led to a significant improvement in performance. In particular, the chart design was changed to accommodate wider range and better stability in detection. Received: 16 December 1999 / Accepted: 15 October 2000     

Building a color classification system for textured and hue homogeneous surfaces: system calibration and algorithm

We present a system for classifying the color aspect of textured surfaces having a nearly constant hue (such as wooden boards, textiles, wallpaper, etc.). The system is designed to compensate for small fluctuations (over time) of the light source and for inhomogeneous illumination conditions (shading correction). This is an important feature because even in industrial environments where the lighting conditions are controlled, a constant and homogeneous illumination cannot be guaranteed. Together with an appropriate camera calibration (which includes a periodic update), our approach offers a robust system which is able to “distinguish” (i.e., classify correctly) between surface classes which exhibit visually barely perceptible color variations. In particular, our approach is based on relative (not absolute) color measurements. In this paper, we outline the classification algorithm while focusing in detail on the camera calibration and a method for compensating for fluctuations of the light source. Received: 1 September 1998 / Accepted: 16 March 2000     

Registration of technical drawings and calibrated images for industrial augmented reality

Despite tremendous progress in 3D modelling technology, most sites in traditional industries do not have a computer model of their facilities at their disposal. In these industries, 2D technical drawings are typically the most commonly used documents. In many cases, a database of fully calibrated and oriented photogrammetric images of parts of the plant is also available. These images are often used for metric measurement and 3D as-built modelling. For planning revamps and maintenance, it is necessary to use industrial drawings as well as images and 3D models represented in a common “world” coordinate system. This paper proposes a method for full integration of technical drawings, calibrated images and as-built 3D models. A new algorithm is developed in order to use only a few correspondences between points on a technical drawing and multiple images to estimate a metric planar transformation between the drawing and the world coordinate system. The paper describes the mathematical relationship between this transformation and the set of homographies needed for merging the technical drawing with all the calibrated images. The method is implemented and fully integrated into an industrial software we developed for 3D as-built reconstruction. We present examples of a real application, in which the method is successfully applied to create an augmented reality representation of a waste water plant. Accepted: 13 August 2001     

How to reconcile fault-tolerant interval intersection with the Lipschitz condition

Summary. We present a new fault-tolerant intersection function , which satisfies the Lipschitz condition for the uniform metric and is optimal among all functions with this property. thus settles Lamport's question about such a function raised in [5]. Our comprehensive analysis reveals that has exactly the same worst-case performance as the optimal Marzullo function , which does not satisfy a Lipschitz condition. The utilized modelling approach in conjunction with a powerful hybrid fault model ensures compatibility of our results with any known application framework, including replicated sensors and clock synchronization. Received: September 1999 / Accepted: November 2000     

Location Models from the Perspective of Context-Aware Applications and Mobile Ad Hoc Networks

Location models are crucial for providing location-dependent data to context-aware applications. In this paper, we present two approaches for modeling location information taken from an infrastructure-based and an ad hoc network-based application scenario. From these approaches we derive requirements for a general location modeling language for ubiquitous computing. Correspondence to: M. Bauer, Fakult?t Informatik, Universit?t Stuttgart, Breitwiesenstr. 20-22, D-70565 Stuttgart, Germany. Email: mabauer@informatik.uni-stuttgart.de     

Extracting curved text lines using local linearity of the text line

In order to enhance the ability of document analysis systems, we need a text line extraction method which can handle not only straight text lines but also text lines in various shapes. This paper proposes a new method called Extended Linear Segment Linking (ELSL for short), which is able to extract text lines in arbitrary orientations and curved text lines. We also consider the existence of both horizontally and vertically printed text lines on the same page. The new method can produce text line candidates for multiple orientations. We verify the ability of the method by some experiments as well. Received December 21, 1998 / Revised version September 2, 1999     

Laziness pays! Using lazy synchronization mechanisms to improve non-blocking constructions

Summary. We present a simple and efficient wait-free implementation of Lazy Large Load-Linked/Store-Conditional (Lazy-LL/SC), which can be used to atomically modify a dynamically-determined set of shared variables in a lock-free manner. The semantics of Lazy-LL/SC is weaker than that of similar objects used by us previously to design lock-free and wait-free constructions, and as a result can be implemented more efficiently. However, we show that Lazy-LL/SC is strong enough to be used in existing non-blocking universal constructions and to build new ones. Received: December 2000 / Accepted: September 2001     

Multimedia network protocols: where are we?

Protocols for multimedia communication are needed to integrate into a single network services intended to satisfy the different requirements of multiple types of traffic. An essential prerequisite for designing these protocols is that the services to be offered by the network must be selected and specified in detail. We present the service models proposed, or being developed, by the Internet community, by the ATM community, and by the Tenet Group. We compare their common characteristics, which reveal the characteristics of the first integrated services networks are likely to offer. The services referred to in this paper are those at the network and transport layers, which support the services to be offered to the system's end users.     

Seeking the truth about ad hoc join costs

In this paper, we re-examine the results of prior work on methods for computing ad hoc joins. We develop a detailed cost model for predicting join algorithm performance, and we use the model to develop cost formulas for the major ad hoc join methods found in the relational database literature. We show that various pieces of “common wisdom” about join algorithm performance fail to hold up when analyzed carefully, and we use our detailed cost model to derive op timal buffer allocation schemes for each of the join methods examined here. We show that optimizing their buffer allocations can lead to large performance improvements, e.g., as much as a 400% improvement in some cases. We also validate our cost model's predictions by measuring an actual implementation of each join algorithm considered. The results of this work should be directly useful to implementors of relational query optimizers and query processing systems. Edited by M. Adiba. Received May 1993 / Accepted April 1996     

A Taxi Ride to Late Capitalism: Hypercapitalism, Imagination and Artificial Intelligence

Through analogy this paper draws attention to hypercapitalism, that is, the profitability of the processes of economic recirculation that are independent of a materialist reality. Since neither materialist ideology nor perception are any longer at stake in hypercapitalism this opens the way for other realities to be revisited. In particular, this paper suggests that this radical shift in the logic of the economy resonates with the values of the Mediaeval period. The paper concludes by suggesting that the study of human consciousness may benefit from a new relationship with reason, particularly as we consider such terms as artificial and intelligent. Correspondence and offprint requests to: Michael Punt, Editor in Chief, Leonardo Digital Reviews, 26 Bridge Street, Chepstow, Monmouthshire NP16 5EZ, UK. Email: mpunt@easynet.co.uk     

A stereo-vision system for support of planetary surface exploration

Abstract. In this paper, we present a system that was developed for the European Space Agency (ESA) for the support of planetary exploration. The system that is sent to the planetary surface consists of a rover and a lander. The lander contains a stereo head equipped with a pan-tilt mechanism. This vision system is used both for modeling the terrain and for localization of the rover. Both tasks are necessary for the navigation of the rover. Due to the stress that occurs during the flight, a recalibration of the stereo-vision system is required once it is deployed on the planet. Practical limitations make it unfeasible to use a known calibration pattern for this purpose; therefore, a new calibration procedure had to be developed that could work on images of the planetary environment. This automatic procedure recovers the relative orientation of the cameras and the pan and tilt axes, as well as the exterior orientation for all the images. The same images are subsequently used to reconstruct the 3-D structure of the terrain. For this purpose, a dense stereo-matching algorithm is used that (after rectification) computes a disparity map. Finally, all the disparity maps are merged into a single digital terrain model. In this paper, a simple and elegant procedure is proposed that achieves that goal. The fact that the same images can be used for both calibration and 3-D reconstruction is important, since, in general, the communication bandwidth is very limited. In addition to navigation and path planning, the 3-D model of the terrain is also used for virtual-reality simulations of the mission, wherein the model is texture mapped with the original images. The system has been implemented, and the first tests on the ESA planetary terrain testbed were successful.     

An Approach to Security Requirements Engineering for a High Assurance System*

Requirements specifications for high-assurance secure systems are rare in the open literature. This paper examines the development of a requirements document for a multilevel secure system that must meet stringent assurance and evaluation requirements. The system is designed to be secure, yet combines popular commercial components with specialised high-assurance ones. Functional and non-functional requirements pertinent to security are discussed. A multidimensional threat model is presented. The threat model accounts for the developmental and operational phases of system evolution and for each phase accounts for both physical and non-physical threats. We describe our team-based method for developing a requirements document and relate that process to techniques in requirements engineering. The system requirements document presented provides a calibration point for future security requirements engineering techniques intended to meet both functional and assurance goals. RID="*" ID="*"The views expressed in this paper are those of the authors and should not be construed to reflect those of their employers or the Department of Defense. This work was supported in part by the MSHN project of the DARPA/ITO Quorum programme and by the MYSEA project of the DARPA/ATO CHATS programme. Correspondence and offprint requests to: T. Levin, Department of Computer Science, Naval Postgraduate School, Monterey, CA 93943-5118, USA. Tel.: +1 831 656 2339; Fax: +1 831 656 2814; Email: levin@nps.navy.mil     

Shooting a smooth video with a shaky camera

Abstract. The image sequence in a video taken by a moving camera may suffer from irregular perturbations because of irregularities in the motion of the person or vehicle carrying the camera. We show how to use information in the image sequence to correct the effects of these irregularities so that the sequence is smoothed, i.e., is approximately the same as the sequence that would have been obtained if the motion of the camera had been smooth. Our method is based on the fact that the irregular motion is almost entirely rotational, and that the rotational image motion can be detected and corrected if a distant object, such as the horizon, is visible. Received: 14 February 2001 / Accepted: 11 February 2002 Correspondence to: A. Rosenfeld     

A vision system based on a laser range-finder applied to robotic fruit harvesting

This paper describes a laser-based computer vision system used for automatic fruit recognition. It is based on an infrared laser range-finder sensor that provides range and reflectance images and is designed to detect spherical objects in non-structured environments. Image analysis algorithms integrate both range and reflectance information to generate four characteristic primitives which give evidence of the existence of spherical objects. The output of this vision system includes 3D position, radius and surface reflectivity of each spherical object. It has been applied to the AGRIBOT orange harvesting robot, where it has obtained good fruit detection rates and unlikely false detections.