Serial Combination of Multiple Experts: A Unified Evaluation

Multiple expert decision combination has received much attention in recent years. This is a multi-disciplinary branch of pattern recognition which has extensive applications in numerous fields including robotic vision, artificial intelligence, document processing, office automation, human-computer interfaces, data acquisition, storage and retrieval, etc. In recent years, this application area has been extended to forensic science, including the identification of individuals using measures depending on biometrics, security and other applications. In this paper, a generalised multi-expert multi-level decision combination strategy, the serial combination approach, has been investigated from the dual viewpoints of theoretical analysis and practical implementation. Different researchers have implicitly utilised various approaches based on this concept over the years in a wide spectrum of application domains, but a comprehensive, coherent and generalised presentation of this approach from both theoretical and implementation viewpoints has not been attempted. While presenting here a unified framework for serial multiple expert decision combination, it is shown that many multi-expert approaches reported in the literature can be easily represented within the proposed framework. Detailed theoretical and practical discussions of the various performance results with these combinations, analysis of the internal processing of this approach, a case study for testing the theoretical framework, issues relating to processing overheads associated with the implementation of this approach, general comments on its applicability to various task domains and the generality of the approach in terms of reevaluating previous research have also been incorporated. Received: 9 October 1998, Received in revised form: 5 March 1999, Accepted: 29 April 1999     

A stereovision system for a planetary rover: calibration, correlation, registration, and fusion

This paper describes a complete stereovision system, which was originally developed for planetary applications, but can be used for other applications such as object modeling. A new effective on-site calibration technique has been developed, which can make use of the information from the surrounding environment as well as the information from the calibration apparatus. A correlation-based stereo algorithm is used, which can produce sufficient dense range maps with an algorithmic structure for fast implementations. A technique based on iterative closest-point matching has been developed for registration of successive depth maps and computation of the displacements between successive positions. A statistical method based on the distance distribution is integrated into this registration technique, which allows us to deal with such important problems as outliers, occlusion, appearance, and disappearance. Finally, the registered maps are expressed in the same coordinate system and are fused, erroneous data are eliminated through consistency checking, and a global digital elevation map is built incrementally.     

Geometric fusion for a hand-held 3D sensor

Abstract. This article presents a geometric fusion algorithm developed for the reconstruction of 3D surface models from hand-held sensor data. Hand-held systems allow full 3D movement of the sensor to capture the shape of complex objects. Techniques previously developed for reconstruction from conventional 2.5D range image data cannot be applied to hand-held sensor data. A geometric fusion algorithm is introduced to integrate the measured 3D points from a hand-held sensor into a single continuous surface. The new geometric fusion algorithm is based on the normal-volume representation of a triangle, which enables incremental transformation of an arbitrary mesh into an implicit volumetric field function. This system is demonstrated for reconstruction of surface models from both hand-held sensor data and conventional 2.5D range images. Received: 30 August 1999 / Accepted: 21 January 2000     

Global methods for stroke segmentation

Two methods for stroke segmentation from a global point of view are presented and compared. One is based on thinning methods and the other is based on contour curve fitting. For both cases an input image is binarized. For the former, Hilditch's method is used, then crossing points are sought, around which a domain is constructed. Outside the domain, a set of line segments are identified. These lines are connected and approximated by cubic B-spline curves. Smoothly connected lines are selected as segmented curves. This method works well for a limited class of crossing lines, which are shown experimentally. In the latter, a contour line is approximated by cubic B-spline curve, along which curvature is measured. According to the extreme points of the curvature graph, the contour line is segmented, based on which the line segment is obtained. Experimental results are shown for some difficult cases. Received October 31, 1998 / Revised January 12, 1999     

Automatic document classification and indexing in high-volume applications

In this paper a system for analysis and automatic indexing of imaged documents for high-volume applications is described. This system, named STRETCH (STorage and RETrieval by Content of imaged documents), is based on an Archiving and Retrieval Engine, which overcomes the bottleneck of document profiling bypassing some limitations of existing pre-defined indexing schemes. The engine exploits a structured document representation and can activate appropriate methods to characterise and automatically index heterogeneous documents with variable layout. The originality of STRETCH lies principally in the possibility for unskilled users to define the indexes relevant to the document domains of their interest by simply presenting visual examples and applying reliable automatic information extraction methods (document classification, flexible reading strategies) to index the documents automatically, thus creating archives as desired. STRETCH offers ease of use and application programming and the ability to dynamically adapt to new types of documents. The system has been tested in two applications in particular, one concerning passive invoices and the other bank documents. In these applications, several classes of documents are involved. The indexing strategy first automatically classifies the document, thus avoiding pre-sorting, then locates and reads the information pertaining to the specific document class. Experimental results are encouraging overall; in particular, document classification results fulfill the requirements of high-volume application. Integration into production lines is under execution. Received March 30, 2000 / Revised June 26, 2001     

Automatic defect classification for semiconductor manufacturing

Visual defect inspection and classification are important parts of most manufacturing processes in the semiconductor and electronics industries. Defect classification provides relevant information to correct process problems, thereby enhancing the yield and quality of the product. This paper describes an automated defect classification (ADC) system that classifies defects on semiconductor chips at various manufacturing steps. The ADC system uses a golden template method for defect re-detection, and measures several features of the defect, such as size, shape, location and color. A rule-based system classifies the defects into pre-defined categories that are learnt from training samples. The system has been deployed in the IBM Burlington 16 M DRAM manufacturing line for more than a year. The system has examined over 100 000 defects, and has met the design criteria of over 80% classification rate and 80% classification accuracy. Issues involving system design tradeoff, implementation, performance, and deployment are closely examined.     

Exploiting early sorting and early partitioning for decision support query processing

Decision support queries typically involve several joins, a grouping with aggregation, and/or sorting of the result tuples. We propose two new classes of query evaluation algorithms that can be used to speed up the execution of such queries. The algorithms are based on (1) early sorting and (2) early partitioning– or a combination of both. The idea is to push the sorting and/or the partitioning to the leaves, i.e., the base relations, of the query evaluation plans (QEPs) and thereby avoid sorting or partitioning large intermediate results generated by the joins. Both early sorting and early partitioning are used in combination with hash-based algorithms for evaluating the join(s) and the grouping. To enable early sorting, the sort order generated at an early stage of the QEP is retained through an arbitrary number of so-called order-preserving hash joins. To make early partitioning applicable to a large class of decision support queries, we generalize the so-called hash teams proposed by Graefe et al. [GBC98]. Hash teams allow to perform several hash-based operations (join and grouping) on the same attribute in one pass without repartitioning intermediate results. Our generalization consists of indirectly partitioning the input data. Indirect partitioning means partitioning the input data on an attribute that is not directly needed for the next hash-based operation, and it involves the construction of bitmaps to approximate the partitioning for the attribute that is needed in the next hash-based operation. Our performance experiments show that such QEPs based on early sorting, early partitioning, or both in combination perform significantly better than conventional strategies for many common classes of decision support queries. Received April 4, 2000 / Accepted June 23, 2000     

Building a color classification system for textured and hue homogeneous surfaces: system calibration and algorithm

We present a system for classifying the color aspect of textured surfaces having a nearly constant hue (such as wooden boards, textiles, wallpaper, etc.). The system is designed to compensate for small fluctuations (over time) of the light source and for inhomogeneous illumination conditions (shading correction). This is an important feature because even in industrial environments where the lighting conditions are controlled, a constant and homogeneous illumination cannot be guaranteed. Together with an appropriate camera calibration (which includes a periodic update), our approach offers a robust system which is able to “distinguish” (i.e., classify correctly) between surface classes which exhibit visually barely perceptible color variations. In particular, our approach is based on relative (not absolute) color measurements. In this paper, we outline the classification algorithm while focusing in detail on the camera calibration and a method for compensating for fluctuations of the light source. Received: 1 September 1998 / Accepted: 16 March 2000     

Diacritical processing for unconstrained online handwriting recognition using a forward search

Out-of-order diacriticals introduce significant complexity to the design of an online handwriting recognizer, because they require some reordering of the time domain information. It is common in cursive writing to write the body of an `i' or `t' during the writing of the word, and then to return and dot or cross the letter once the word is complete. The difficulty arises because we have to look ahead, when scoring one of these letters, to find the mark occurring later in the writing stream that completes the letter. We should also remember that we have used this mark, so that we don't use it again for a different letter, and we should also penalize a word if there are some marks that look like diacriticals that are not used. One approach to this problem is to scan the writing some distance into the future to identify candidate diacriticals, remove them in a preprocessing step, and associate them with the matching letters earlier in the word. If done as a preliminary operation, this approach is error-prone: marks that are not diacriticals may be incorrectly identified and removed, and true diacriticals may be skipped. This paper describes a novel extension to a forward search algorithm that provides a natural mechanism for considering alternative treatments of potential diacriticals, to see whether it is better to treat a given mark as a diacritical or not, and directly compare the two outcomes by score. Received October 30, 1998 / Revised January 25, 1999     

A system for a new two-dimensional code: Secure 2D code

A new 2D code called Secure 2D code is designed in this paper, both encoder and decoder are also proposed. Secure 2D code can store any kind of data and provides high security. With regard to security, the input data is divided into two parts: general and secret. The general data is transformed into a 2D code pattern, then secret data is hidden in the 2D code pattern. To raise the reading speed and allow various reading environments, some features are added around the 2D code pattern boundary. As to the reliability, RS code is adopted to treat damaged patterns. Received: 9 September 1997 / Accepted: 2 March 1998     

Tracking the human arm using constraint fusion and multiple-cue localization

Abstract. The use of hand gestures provides an attractive means of interacting naturally with a computer-generated display. Using one or more video cameras, the hand movements can potentially be interpreted as meaningful gestures. One key problem in building such an interface without a restricted setup is the ability to localize and track the human arm robustly in video sequences. This paper proposes a multiple-cue localization scheme combined with a tracking framework to reliably track the dynamics of the human arm in unconstrained environments. The localization scheme integrates the multiple cues of motion, shape, and color for locating a set of key image features. Using constraint fusion, these features are tracked by a modified extended Kalman filter that exploits the articulated structure of the human arm. Moreover, an interaction scheme between tracking and localization is used for improving the estimation process while reducing the computational requirements. The performance of the localization/tracking framework is validated with the help of extensive experiments and simulations. These experiments include tracking with calibrated stereo camera and uncalibrated broadcast video. Received: 19 January 2001 / Accepted: 27 December 2001 Correspondence to: R. Sharma     

Estimations of previewed road curvatures and vehicular motion by a vision-based data fusion scheme

In this study, a new framework of vision-based estimation is developed using some data fusion schemes to obtain previewed road curvatures and vehicular motion states based on the scene viewed from an in-vehicle camera. The previewed curvatures are necessary for the guidance of an automatically steering vehicle, and the desired vehicular motion variables, including lateral deviation, heading angle, yaw rate, and sideslip angle, are also required for proper control of the vehicular lateral motion via steering. In this framework, physical relationships of previewed curvatures among consecutive images, motion variables in terms of image features searched at various levels in the image plane, and dynamic correlation among vehicular motion variables are derived as bases of data fusion to enhance the accuracy of estimation. The vision-based measurement errors are analyzed to determine the fusion gains based on the technique of a Kalman filter such that the measurements from the image plane and predictions of physical models can be properly integrated to obtain reliable estimations. Off-line experimental works using real road scenes are performed to verify the whole framework for image sensing.     

Symbol and character recognition: application to engineering drawings

In this paper, we consider the general problem of technical document interpretation, as applied to the documents of the French Telephonic Operator, France Télécom. More precisely, we focus the content of this paper on the computation of a new set of features allowing the classification of multioriented and multiscaled patterns. This set of invariants is based on the Fourier–Mellin Transform. The interests of this computation rely on the excellent classification rate obtained with this method and also on using this Fourier–Mellin transform within a “filtering mode”, with which we can solve the well known difficult problem of connected character recognition.     

Automatic text segmentation and text recognition for video indexing

Efficient indexing and retrieval of digital video is an important function of video databases. One powerful index for retrieval is the text appearing in them. It enables content-based browsing. We present our new methods for automatic segmentation of text in digital videos. The algorithms we propose make use of typical characteristics of text in videos in order to enable and enhance segmentation performance. The unique features of our approach are the tracking of characters and words over their complete duration of occurrence in a video and the integration of the multiple bitmaps of a character over time into a single bitmap. The output of the text segmentation step is then directly passed to a standard OCR software package in order to translate the segmented text into ASCII. Also, a straightforward indexing and retrieval scheme is introduced. It is used in the experiments to demonstrate that the proposed text segmentation algorithms together with existing text recognition algorithms are suitable for indexing and retrieval of relevant video sequences in and from a video database. Our experimental results are very encouraging and suggest that these algorithms can be used in video retrieval applications as well as to recognize higher level semantics in videos.     

Active rules for XML: A new paradigm for E-services

XML is rapidly becoming one of the most widely adopted technologies for information exchange and representation. As the use of XML becomes more widespread, we foresee the development of active XML rules, i.e., rules explicitly designed for the management of XML information. In particular, we argue that active rules for XML offer a natural paradigm for the rapid development of innovative e-services. In the paper, we show how active rules can be specified in the context of XSLT, a pattern-based language for publishing XML documents (promoted by the W3C) which is receiving strong commercial support, and Lorel, a query language for XML documents that is quite popular in the research world. We demonstrate, through simple examples of active rules for XSLT and Lorel, that active rules can be effective for the implementation of e-commerce services. We also discuss the various issues that need to be considered in adapting the notion of relational triggers to the XML context. Received: 30 October 2000 / Accepted: 19 December 2000 Published online: 27 April 2001     

Mathematical expression recognition: a survey

Abstract. Automatic recognition of mathematical expressions is one of the key vehicles in the drive towards transcribing documents in scientific and engineering disciplines into electronic form. This problem typically consists of two major stages, namely, symbol recognition and structural analysis. In this survey paper, we will review most of the existing work with respect to each of the two major stages of the recognition process. In particular, we try to put emphasis on the similarities and differences between systems. Moreover, some important issues in mathematical expression recognition will be addressed in depth. All these together serve to provide a clear overall picture of how this research area has been developed to date. Received February 22, 2000 / Revised June 12, 2000     

REFLICS: Real-time flow imaging and classification system

An accurate analysis of a large dynamic system like our oceans requires spatially fine and temporally matched data collection methods. Current methods to estimate fish stock size from pelagic (marine) fish egg abundance by using ships to take point samples of fish eggs have large margins of error due to spatial and temporal undersampling. The real-time flow imaging and classification system (REFLICS) enhances fish egg sampling by obtaining continuous, accurate information on fish egg abundance as the ship cruises along in the area of interest. REFLICS images the dynamic flow with a progressive-scan area camera (60 frames/s) and a synchronized strobe in backlighting configuration. Digitization and processing occur on a dual-processor Pentium II PC and a pipeline-based image-processing board. REFLICS uses a segmentation algorithm to locate fish-egg-like objects in the image and then a classifier to determine fish egg, species, and development stage (age). We present an integrated system design of REFLICS and performance results. REFLICS can perform in real time (60 Hz), classify fish eggs with low false negative rates on real data collected from a cruise, and work in harsh conditions aboard ships at sea. REFLICS enables cost-effective, real-time assessment of pelagic fish eggs for research and management. Received: 12 April 2000 / Accepted: 6 July 2000