Agent-based computer vision in a dynamic, real-time environment

For computer vision systems to operate in many real-world environments, processing must occur in real-time under dynamic conditions. An agent-based methodology offers an approach to increase flexibility and scalability to accommodate the demands of a real-time, dynamic environment. This paper presents an agent-based architecture that uses a utility optimization technique to guarantee that important vision tasks are fulfilled even under resource constraints. To ensure that the processing of vision tasks is both reliable and flexible, multiple behaviors are utilized to accomplish the vision application's requirements. A vision behavior consists of a grouping of vision algorithms and a set of service levels associated with these algorithms. Utility functions are adopted to evaluate the performance of all possible behaviors that can address the requirements of a vision application within resource constraints. The maximum overall utility corresponds to the optimal behavior. Two example systems using this model are presented to show the applicability of the architecture. Experimental results show that this agent-based architecture outperforms traditional non-agent-based approaches.     

General-purpose vision chip architecture for real-time machine vision

To solve the I/O bottleneck problem in existing vision systems and to realize versatile processing adaptive to various and changing environments, we propose a new vision chip architecture for applications such as robot vision. The chip has general-purpose processing elements (PEs) with each PE being directly connected to a photo detector (PD) and can implement various visual processing algorithms. We developed and simulated some sample programs for the chip and proved that they can be processed within 1 ms/frame, a rate that is high enough for high-speed visual feedback for robot control. Aiming to complete the chip, we are now developing test chips based on the architecture. The latest design has 8 x 8 PEs and PDs in an area 3.3 mm x 3.0 mm using a 0.8 μm CMOS process.     

RealNet: a neural network architecture for real-time systems scheduling

Real-time embedded systems are spreading to more and more new fields and their scope and complexity have grown dramatically in the last few years. Nowadays, real-time embedded computers or controllers can be found everywhere, both in very simple devices used in everyday life and in professional environments. Real-time embedded systems have to take into account robustness, safety and timeliness. The most-used schedulability analysis is the worst-case response time proposed by Joseph and Pandya (Comput J 29:390–395,1986). This test provides a bivaluated response (yes/no) indicating whether the processes will meet their corresponding deadlines or not. Nevertheless, sometimes the real-time designer might want to know, more exactly, the probability of the processes meeting their deadlines, in order to assess the risk of a failed scheduling depending on critical requirements of the processes. This paper presents RealNet, a neural network architecture that will generate schedules from timing requirements of a real-time system. The RealNet simulator will provide the designer, after iterating and averaging over some trials, an estimation of the probability that the system will not meet the deadlines. Moreover, the knowledge of the critical processes in these schedules will allow the designer to decide whether changes in the implementation are required.This revised version was published online in November 2004 with a correction to the accepted date.     

Intermediate-level vision tasks on a memory array architecture

With the fast advances in the area of computer vision and robotics there is a growing need for machines that can understand images at very high speed. A conventional von Neumann computer is not suitable for this purpose, because it takes a tremendous amount of time to solve most typical image analysis problems. Thus, it is now imperative to study computer vision in a parallel processing framework in order to reduce the processing time. In this paper we demonstrate the applicability of a simple memory array architecture to some intermediate-level computer vision tasks. This architecture, called theAccess Constrained Memory Array Architecture (ACMAA) has a linear array of processors which concurrently access distinct rows or columns of an array of memory modules. Because of its efficient local and global communication capabilities ACMAA is well suited for low-level as well as intermediate-level vision tasks. This paper presents algorithms for connected component labeling, determination of area, perimeter and moments of a labeled region, convex hull of a region, and Hough transform of an image. ACMAA is well suited to an efficient hardware implementation because it has a modular structure, simple interconnect and limited global control.     

Toward designing intelligent PDEs for computer vision: An optimal control approach

Many computer vision and image processing problems can be posed as solving partial differential equations (PDEs). However, designing a PDE system usually requires high mathematical skills and good insight into the problems. In this paper, we consider designing PDEs for various problems arising in computer vision and image processing in a lazy manner: learning PDEs from training data via an optimal control approach. We first propose a general intelligent PDE system which holds the basic translational and rotational invariance rule for most vision problems. By introducing a PDE-constrained optimal control framework, it is possible to use the training data resulting from multiple ways (ground truth, results from other methods, and manual results from humans) to learn PDEs for different computer vision tasks. The proposed optimal control based training framework aims at learning a PDE-based regressor to approximate the unknown (and usually nonlinear) mapping of different vision tasks. The experimental results show that the learnt PDEs can solve different vision problems reasonably well. In particular, we can obtain PDEs not only for problems that traditional PDEs work well but also for problems that PDE-based methods have never been tried before, due to the difficulty in describing those problems in a mathematical way.     

Parallel computer vision on Polymorphic Torus architecture

Polymorphic Torus is a novel interconnection network for SIMD massively parallel computers, able to support effectively both local and global communication. Thanks to this characteristic, Polymorphic Torus is highly suitable for computer vision applications, since vision involves local communication at the low-level stage and global communication at the intermediate- and high-level stages. In this paper we evaluate the performance of Polymorphic Torus in the computer vision domain. We consider a set of basic vision tasks, namely,convolution, histogramming, connected component labeling, Hough transform, extreme point identification, diameter computation, andvisibility, and show how they can take advantage of the Polymorphic Torus communication capabilities. For each basic vision task we propose a Polymorphic Torus parallel algorithm, give its computational complexity, and compare such a complexity with the complexity of the same task inmesh, tree, pyramid, and hypercube interconnection networks. In spite of the fact that Polymorphic Torus has the same wiring complexity as mesh, the comparison shows that in all of the vision tasks under examination it achieves complexity lower than or at most equal to hypercube, which is the most powerful among the interconnection networks considered.     

A real-time vision system using an integrated memory array processor prototype

This paper describes a real-time vision system (RVS) architecture and performance and its use of an integrated memory array processor (IMAP) prototype. This prototype integrates eight 8-bit processors and a 144-kbit SRAM on a single chip. The RVS was developed with 64 IMAP prototypes connected in series in a 512 processor-system configuration. A host workstation can access the memory on the IMAP prototypes directly through a random access port. Images are inputted and outputted at high speed through serial access ports. The RVS performance is shown in real-time road-image processing and in a neural network simulation, as well as in low-level image processing algorithms, such as filtering, histograms, discrete cosine transform (DCT), and rotation. The RVS image processing is shown to be much faster than the video rate.     

机器人足球视觉系统中的实时图像处理

视觉系统是整个机器人足球系统的重要组成部分。根据机器人足球视觉系统的特点,提出基于游程长度编码(RLE)的实时快速图像处理算法。算法使用RLE对图像进行压缩,并且在处理阶段高效识别出图像中目标的尺寸和位置。压缩阶段算法的时间复杂度与图像尺寸成线性关系,图像处理阶段算法的时间复杂度与图像中目标的个数和每个目标所占的扫描行数成线性关系。     

Functional programming on a dataflow architecture: Applications in real-time image processing

This paper presents a dataflow functional computer (DFFC) developed at the Etablissement Technique Central de l'Armement (ETCA) and dedicated to real-time image processing. Two types of data-driven processing elements, dedicated respectively to low-level and mid-level processings are integrated in a regular 3D array. The design of the DFFC relies on a close integration of the dataflow-architecture principles and the functional programming concept. An image processing algorithm, expressed with a syntax similar to that of functional programming (FP) is first converted into a dataflow graph. The nodes of this graph are real-time operators that can be implemented on the physical processors of the dataflow machine. This dataflow graph is then mapped directly onto the processor array. The programming environment includes a complete compilation stream from the FP specification to hardware implementation, along with a global operator database. Apart from being a research tool for real-time image processing, the DFFC may also be used to perform the automatic synthesis of autonomous vision automata from a high-level functional specification. An experimental system, including 1024 lowlevel custom dataflow processors and 12 T800 transputers, was built and can perform up to 50 billion operations/s. Several image processing algorithms were implemented on this system and run in real-time at digital video speed.     

Real-time colour recognition in symbolic programming for machine vision systems

It is impossible to collect more than a tiny proportion of all of the possible examples of a given hue to form a training set for a machine that learns to discriminate colours. In view of this, it is argued that colour generelization is essential. Three mechanisms for learning colours, as defined by a human being, are described. One of these is based upon an idea developed by A.P. Plummer and is implemented in a commercial device known as the intelligent camera. This implementation can learn the characteristics of coloured scenes presented to it and can segment a video image in real-time. This paper presents four procedures that allow the range of colours learned by such a system to be broadened so that recognition is made more reliable and less prone to generating noisy images that are difficult to analyse. Three of the procedures can be used to improve colour discrimination, while a fourth procedure is used when a single and general colour concept has to be learned. Several experiments were devised to demonstrate the effectiveness of colour generelization. These have shown that it is indeed possible to achieve reliable colour discrimination / recognition for such tasks as inspecting packaging and fruit. A practical system based upon the intelligent camera and controlled by software written in PROLOG has been developed by the authors and is being used in a study of methods for declarative programming of machine vision systems for industrial applications.     

MinGPU: a minimum GPU library for computer vision

In the field of computer vision, it is becoming increasingly popular to implement algorithms, in sections or in their entirety, on a graphics processing unit (GPU). This is due to the superior speed GPUs offer compared to CPUs. In this paper, we present a GPU library, MinGPU, which contains all of the necessary functions to convert an existing CPU code to GPU. We have created GPU implementations of several well known computer vision algorithms, including the homography transformation between two 3D views. We provide timing charts and show that our MinGPU implementation of homography transformations performs approximately 600 times faster than its C++ CPU implementation.

基于机器视觉的色差检测算法

针对金属印刷质量中的色差检测问题,采用机器视觉的技术对色差检测算法进行了研究。通过在金属印刷品的留白区域印刷色标,使用工业CCD相机采集金属印刷产品上的色标区域,使用数字图像处理技术提取色标。使用了基于HSV颜色空间的色差检测算法和基于CIELAB颜色空间的色差检测算法,分析了两种颜色空间下色差检测的实验结果,采用两种颜色空间检测算法相结合的方法,实现对色差合理有效的快速检测,同时能保证检测结果的准确性。     

Tracking vision system for real-time motion analysis

This paper describes a high-speed motion tracking system based on full-search correlation matching. Incorporating the Advanced Correlation Processor, which computes more than 500 motion vectors per frame, the system enables the detection and tracking of moving objects at a video rate. The system is implemented on a single board and can be mounted inside a PC. Thus, it is compact and economical, and is well suited for a wide variety of applications such as automatic surveillance, traffic monitoring and sports play analysis.     

A general dynamic vision architecture for UGV and UAV

The expectation-based 4D approach to dynamic machine vision exploiting integral spatiotemporal models of objects in the real world is discussed in the application domains of unmanned ground and air vehicles. The method has demonstrated superior performance over the last half decade in autonomous road vehicle guidance with three different vans and busses, with an AGV on the factory floor and with completely autonomous relative state estimation for a twin turboprop aircraft in the landing approach to a runway without any external support; in all application areas only a small set of conventional microcomputers was sufficient for realizing the system. This shows the computational efficiency of the method combining both conventional engineering type algorithms and artificial intelligence components in a well balanced way.The modularity of the approach is demonstrated in a simulation set-up serving both the ground- and the air vehicle applications. Expermental results in both areas are discussed.     

A model-based computer vision system for recognizing handwritten ZIP codes

This paper describes a recognition system for handwritten ZIP Codes currently under development at the Environmental Research Institute of Michigan (ERIM). Included within this system are techniques for preprocessing address block images, locating ZIP Codes, splitting touching characters, and identifying handwritten numerals. These techniques rely on mathematical morphology-based image processing and on hierarchical matching of object models to symbolic image representations. The image processing uses adaptive filtering, thresholding, and skeletonizing to create binary and state-labeled images. The matching process uses these images and extensively developed handwritten digit models to identify ZIP Codes. The end-to-end system has been tested on 500 randomly selected address block images. The system correctly recognized a large portion of the ZIP Codes in the test images (45.0%), and incorrectly classified a very low percentage of isolated handwritten digits (0.9%). Overall performance continues to be improved through incremental digit model refinement.This work was funded by the Office of Advanced Technology, United States Postal Service under contract 104230-86-H-0042.     

Detecting corn tassels using computer vision and support vector machines

An automated solution for maize detasseling is very important for maize growers who want to reduce production costs. Quality assurance of maize requires constantly monitoring production fields to ensure that only hybrid seed is produced. To achieve this cross-pollination, tassels of female plants have to be removed for ensuring all the pollen for producing the seed crop comes from the male rows. This removal process is called detasseling. Computer vision methods could help positioning the cutting locations of tassels to achieve a more precise detasseling process in a row. In this study, a computer vision algorithm was developed to detect cutting locations of corn tassels in natural outdoor maize canopy using conventional color images and computer vision with a minimum number of false positives. Proposed algorithm used color informations with a support vector classifier for image binarization. A number of morphological operations were implemented to determine potential tassel locations. Shape and texture features were used to reduce false positives. A hierarchical clustering method was utilized to merge multiple detections for the same tassel and to determine the final locations of tassels. Proposed algorithm performed with a correct detection rate of 81.6% for the test set. Detection of maize tassels in natural canopy images is a quite difficult task due to various backgrounds, different illuminations, occlusions, shadowed regions, and color similarities. The results of the study indicated that detecting cut location of corn tassels is feasible using regular color images.     

Intelligent vision system for dynamic environments

This article describes an intelligent vision system that absorbs useful information from its environment and draws useful conclusions. This system can give instructions to locate vacant seats that are currently occupied in a theater. The extraction of useful information without viewing or exposing the inside details of an environment through an active vision system is proposed. Reasoning-based conclusions are drawn for optimum searching. The effectiveness of the proposed method is demonstrated using an experiment. This work was presented in part at the 10th International Symposium on Artificial Life and Robotics, Oita, Japan, February 4–6, 2005     

Robot-vision architecture for real-time 6-DOF object localization

This paper presents a new robot-vision system architecture for real-time moving object localization. The 6-DOF (3 translation and 3 rotation) motion of the objects is detected and tracked accurately in clutter using a model-based approach without information of the objects’ initial positions. An object identification task and an object tracking task are combined under this architecture. The computational time-lag between the two tasks is absorbed by a large amount of frame memory. The tasks are implemented as independent software modules using stereo-vision-based methods which can deal with objects of various shapes with edges, including planar to smooth-curved objects, in cluttered environments. This architecture also leads to failure-recoverable object tracking, because the tracking processes can be automatically recovered, even if the moving objects are lost while tracking. Experimental results obtained with prototype systems demonstrate the effectiveness of the proposed architecture.     

基于计算机视觉的红枣形状识别方法研究

以红枣为研究对象,通过图像处理技术获得红枣的边界轮廓。用极半径函数来表示红枣的边界轮廓,将极半径函数用傅里叶级数展开,并使用傅里叶级数前15项系数来描述红枣的形状特征。然后分别使用欧氏距离法和不规则度判别法对红枣的形状进行了分类实验研究,实验结果表明:2种方法对正常枣的分类准确率都比较高;对于畸形枣,欧氏距离法的识别率仅为35%,而不规则度判别法的识别率可达90%。     

An architecture for handwritten text recognition systems

This paper presents an end-to-end system for reading handwritten page images. Five functional modules included in the system are introduced in this paper: (i) pre-processing, which concerns introducing an image representation for easy manipulation of large page images and image handling procedures using the image representation; (ii) line separation, concerning text line detection and extracting images of lines of text from a page image; (iii) word segmentation, which concerns locating word gaps and isolating words from a line of text image obtained efficiently and in an intelligent manner; (iv) word recognition, concerning handwritten word recognition algorithms; and (v) linguistic post-pro- cessing, which concerns the use of linguistic constraints to intelligently parse and recognize text. Key ideas employed in each functional module, which have been developed for dealing with the diversity of handwriting in its various aspects with a goal of system reliability and robustness, are described in this paper. Preliminary experiments show promising results in terms of speed and accuracy. Received October 30, 1998 / Revised January 15, 1999