Algorithms for coplanar camera calibration

Abstract. Coplanar camera calibration is the process of determining the extrinsic and intrinsic camera parameters from a given set of image and world points, when the world points lie on a two-dimensional plane. Noncoplanar calibration, on the other hand, involves world points that do not lie on a plane. While optimal solutions for both the camera-calibration procedures can be obtained by solving a set of constrained nonlinear optimization problems, there are significant structural differences between the two formulations. We investigate the computational and algorithmic implications of such underlying differences, and provide a set of efficient algorithms that are specifically tailored for the coplanar case. More specifically, we offer the following: (1) four algorithms for coplanar calibration that use linear or iterative linear methods to solve the underlying nonlinear optimization problem, and produce sub-optimal solutions. These algorithms are motivated by their computational efficiency and are useful for real-time low-cost systems. (2) Two optimal solutions for coplanar calibration, including one novel nonlinear algorithm. A constraint for the optimal estimation of extrinsic parameters is also given. (3) A Lyapunov type convergence analysis for the new nonlinear algorithm. We test the validity and performance of the calibration procedures with both synthetic and real images. The results consistently show significant improvements over less complete camera models. Received: 30 September 1998 / Accepted: 12 January 2000     

Towards the recovery of extrinsic camera parameters from video records of sewer surveys

The aim of the work reported here is the recovery, from a single image taken inside a roughly cylindrical brick sewer pipe of diameter up to one meter, of the pose of the camera relative to the central axis of the pipe. It is shown that the vanishing point associated with the longitudinal mortar lines carries valuable information about the pose. A method for the automatic detection of this point is presented and used to analyse the camera rotations underlying a number of sewer survey videos. It is similarly shown how the angles between the images of the longitudinal lines can be used to recover information about camera pose. The techniques might form an active part of a more comprehensive image understanding system recovering the three-dimensional shape of a surveyed pipe from survey videos and/or be used as an experimental tool during the design of such a system. Received: 24 June 1997 / Accepted: 17 March 1998     

Dry camera calibration for underwater applications

Is it possible to calibrate a camera in the air and then use the calibration results to infer a new calibration corresponding to the embedding of the camera in another fluid (possibly water)? This problem is dealt within the paper. It is important to avoid direct underwater calibration, as it is much more inconvenient for experiments than the usual (air) calibration by human workers. Optical laws that must be considered when using underwater cameras are investigated. Both theoretical and experimental point of views are described, and it is shown that relationships can be found between results of air and water (or any other isotropic fluid in which the camera can be submerged) calibration. Received: 22 April 2000 / Accepted: 13 May 2002 Correspondence to: J.M. Lavest     

Range image segmentation using local approximation of scan lines with application to CAD model acquisition

Abstract. Automatic acquisition of CAD models from existing objects requires accurate extraction of geometric and topological information from the input data. This paper presents a range image segmentation method based on local approximation of scan lines. The method employs edge models that are capable of detecting noise pixels as well as position and orientation discontinuities of varying strengths. Region-based techniques are then used to achieve a complete segmentation. Finally, a geometric representation of the scene, in the form of a surface CAD model, is produced. Experimental results on a large number of real range images acquired by different range sensors demonstrate the efficiency and robustness of the method. Received: 1 August 2000 / Accepted: 23 January 2002 Correspondence to: I. Khalifa     

Empirical evaluation of a calibration chart detector

In this paper, we systematically assess the performance of an automatic calibration chart detector. Through simulation we establish the optimal set of control parameters and the rate of successful detection as a function of pose. We validate the simulation results on real images taken from a camera mounted on a robot arm. The results confirm the utility of such simulation studies. The feedback obtained suggested a number of modifications for the chart detection system, which led to a significant improvement in performance. In particular, the chart design was changed to accommodate wider range and better stability in detection. Received: 16 December 1999 / Accepted: 15 October 2000     

Building a color classification system for textured and hue homogeneous surfaces: system calibration and algorithm

We present a system for classifying the color aspect of textured surfaces having a nearly constant hue (such as wooden boards, textiles, wallpaper, etc.). The system is designed to compensate for small fluctuations (over time) of the light source and for inhomogeneous illumination conditions (shading correction). This is an important feature because even in industrial environments where the lighting conditions are controlled, a constant and homogeneous illumination cannot be guaranteed. Together with an appropriate camera calibration (which includes a periodic update), our approach offers a robust system which is able to “distinguish” (i.e., classify correctly) between surface classes which exhibit visually barely perceptible color variations. In particular, our approach is based on relative (not absolute) color measurements. In this paper, we outline the classification algorithm while focusing in detail on the camera calibration and a method for compensating for fluctuations of the light source. Received: 1 September 1998 / Accepted: 16 March 2000     

Registration of technical drawings and calibrated images for industrial augmented reality

Despite tremendous progress in 3D modelling technology, most sites in traditional industries do not have a computer model of their facilities at their disposal. In these industries, 2D technical drawings are typically the most commonly used documents. In many cases, a database of fully calibrated and oriented photogrammetric images of parts of the plant is also available. These images are often used for metric measurement and 3D as-built modelling. For planning revamps and maintenance, it is necessary to use industrial drawings as well as images and 3D models represented in a common “world” coordinate system. This paper proposes a method for full integration of technical drawings, calibrated images and as-built 3D models. A new algorithm is developed in order to use only a few correspondences between points on a technical drawing and multiple images to estimate a metric planar transformation between the drawing and the world coordinate system. The paper describes the mathematical relationship between this transformation and the set of homographies needed for merging the technical drawing with all the calibrated images. The method is implemented and fully integrated into an industrial software we developed for 3D as-built reconstruction. We present examples of a real application, in which the method is successfully applied to create an augmented reality representation of a waste water plant. Accepted: 13 August 2001     

Efficient extraction of primitives from line drawings composed of horizontal and vertical lines

The performance of the algorithms for the extraction of primitives for the interpretation of line drawings is usually affected by the degradation of the information contained in the document due to factors such as low print contrast, defocusing, skew, etc. In this paper, we are proposing two algorithms for the extraction of primitives with good performance under degradation. The application of the algorithms is restricted to line drawings composed of horizontal and vertical lines. The performance of the algorithms has been evaluated by using a protocol described in the literature. Received: 6 August 1996 / Accepted: 16 July 1997     

How to reconcile fault-tolerant interval intersection with the Lipschitz condition

Summary. We present a new fault-tolerant intersection function , which satisfies the Lipschitz condition for the uniform metric and is optimal among all functions with this property. thus settles Lamport's question about such a function raised in [5]. Our comprehensive analysis reveals that has exactly the same worst-case performance as the optimal Marzullo function , which does not satisfy a Lipschitz condition. The utilized modelling approach in conjunction with a powerful hybrid fault model ensures compatibility of our results with any known application framework, including replicated sensors and clock synchronization. Received: September 1999 / Accepted: November 2000     

Extracting curved text lines using local linearity of the text line

In order to enhance the ability of document analysis systems, we need a text line extraction method which can handle not only straight text lines but also text lines in various shapes. This paper proposes a new method called Extended Linear Segment Linking (ELSL for short), which is able to extract text lines in arbitrary orientations and curved text lines. We also consider the existence of both horizontally and vertically printed text lines on the same page. The new method can produce text line candidates for multiple orientations. We verify the ability of the method by some experiments as well. Received December 21, 1998 / Revised version September 2, 1999     

Location Models from the Perspective of Context-Aware Applications and Mobile Ad Hoc Networks

Location models are crucial for providing location-dependent data to context-aware applications. In this paper, we present two approaches for modeling location information taken from an infrastructure-based and an ad hoc network-based application scenario. From these approaches we derive requirements for a general location modeling language for ubiquitous computing. Correspondence to: M. Bauer, Fakult?t Informatik, Universit?t Stuttgart, Breitwiesenstr. 20-22, D-70565 Stuttgart, Germany. Email: mabauer@informatik.uni-stuttgart.de     

Extra-Ordinary Human–Machine Interaction: What can be Learned from People with Disabilities?

When specifying, designing and implementing technology for work, it is very important to consider the whole of the potential user group, and not to forget that this might include people with disabilities. Obvious reasons for this are demography, disabled people forming a significant and growing proportion of the workforce, and legal obligations caused by equal opportunities legislation. There are more subtle reasons, however, which link so-called ‘disabled people’ with the rest of the population. The paper notes that most people have some disabilities, sometimes temporarily due to accident, or permanently due to ageing and other processes. In addition it notes that even otherwise fully functioning people can be seriously handicapped by hostile environments, and a parallel is drawn between the human interface needs of disabled people operating in ordinary situations and able-bodied people operating in high workload and stress situations. The paper concludes by suggesting a research agenda for the field on the basis of these considerations.     

Seeking the truth about ad hoc join costs

In this paper, we re-examine the results of prior work on methods for computing ad hoc joins. We develop a detailed cost model for predicting join algorithm performance, and we use the model to develop cost formulas for the major ad hoc join methods found in the relational database literature. We show that various pieces of “common wisdom” about join algorithm performance fail to hold up when analyzed carefully, and we use our detailed cost model to derive op timal buffer allocation schemes for each of the join methods examined here. We show that optimizing their buffer allocations can lead to large performance improvements, e.g., as much as a 400% improvement in some cases. We also validate our cost model's predictions by measuring an actual implementation of each join algorithm considered. The results of this work should be directly useful to implementors of relational query optimizers and query processing systems. Edited by M. Adiba. Received May 1993 / Accepted April 1996     

A stereo-vision system for support of planetary surface exploration

Abstract. In this paper, we present a system that was developed for the European Space Agency (ESA) for the support of planetary exploration. The system that is sent to the planetary surface consists of a rover and a lander. The lander contains a stereo head equipped with a pan-tilt mechanism. This vision system is used both for modeling the terrain and for localization of the rover. Both tasks are necessary for the navigation of the rover. Due to the stress that occurs during the flight, a recalibration of the stereo-vision system is required once it is deployed on the planet. Practical limitations make it unfeasible to use a known calibration pattern for this purpose; therefore, a new calibration procedure had to be developed that could work on images of the planetary environment. This automatic procedure recovers the relative orientation of the cameras and the pan and tilt axes, as well as the exterior orientation for all the images. The same images are subsequently used to reconstruct the 3-D structure of the terrain. For this purpose, a dense stereo-matching algorithm is used that (after rectification) computes a disparity map. Finally, all the disparity maps are merged into a single digital terrain model. In this paper, a simple and elegant procedure is proposed that achieves that goal. The fact that the same images can be used for both calibration and 3-D reconstruction is important, since, in general, the communication bandwidth is very limited. In addition to navigation and path planning, the 3-D model of the terrain is also used for virtual-reality simulations of the mission, wherein the model is texture mapped with the original images. The system has been implemented, and the first tests on the ESA planetary terrain testbed were successful.     

A framework for designing and implementing the user interface of a geographic digital library

Geographic data are useful for a large set of applications, such as urban planning and environmental control. These data are, however, very expensive to acquire and maintain. Moreover, their use is often restricted due to a lack of dissemination mechanisms. Digital libraries are a good approach for increasing data availability and therefore reducing costs, since they provide efficient storage and access to large volumes of data. One major drawback to this approach is that it creates the necessity of providing facilities for a large and heterogeneous community of users to search and interact with these geographic libraries. We present a solution to this problem, based on a framework that allows the design and construction of customizable user interfaces for applications based on Geographic Digital Libraries (GDL). This framework relies on two main concepts: a geographic user interface architecture and a geographic digital library model. Received: 15 December 1997 / Revised: June 1999     

Laziness pays! Using lazy synchronization mechanisms to improve non-blocking constructions

Summary. We present a simple and efficient wait-free implementation of Lazy Large Load-Linked/Store-Conditional (Lazy-LL/SC), which can be used to atomically modify a dynamically-determined set of shared variables in a lock-free manner. The semantics of Lazy-LL/SC is weaker than that of similar objects used by us previously to design lock-free and wait-free constructions, and as a result can be implemented more efficiently. However, we show that Lazy-LL/SC is strong enough to be used in existing non-blocking universal constructions and to build new ones. Received: December 2000 / Accepted: September 2001     

Exploring agent support at the user interface in e-commerce applications

In this paper, we present some of the results from our ongoing research work in the area of ‘agent support’ for electronic commerce, particularly at the user interface level. Our goal is to provide intelligent agents to assist both the consumers and the vendors in an electronic shopping environment. Users with a wide variety of different needs are expected to use the electronic shopping application and their expectations about the interface could vary a lot. Traditional studies of user interface technology have shown the existence of a ‘gap’ between what the user interface actually lets the users do and the users’ expectations. Agent technology, in the form of personalized user interface agents, can help to narrow this gap. Such agents can be used to give a personalized service to the user by knowing the user’s preferences. By doing so, they can assist in the various stages of the users’ shopping process, provide tailored product recommendations by filtering information on behalf of their users and reduce the information overload. From a vendor’s perspective, a software sales agent could be used for price negotiation with the consumer. Such agents would give the flexibility offered by negotiation without the burden of having to provide human presence to an online store to handle such negotiations. Published online: 25 July 2001     

Shooting a smooth video with a shaky camera

Abstract. The image sequence in a video taken by a moving camera may suffer from irregular perturbations because of irregularities in the motion of the person or vehicle carrying the camera. We show how to use information in the image sequence to correct the effects of these irregularities so that the sequence is smoothed, i.e., is approximately the same as the sequence that would have been obtained if the motion of the camera had been smooth. Our method is based on the fact that the irregular motion is almost entirely rotational, and that the rotational image motion can be detected and corrected if a distant object, such as the horizon, is visible. Received: 14 February 2001 / Accepted: 11 February 2002 Correspondence to: A. Rosenfeld