A wavelet-VQ system for real-time video compression

This paper presents a scheme and its Field Programmable Gate Array (FPGA) implementation for a system based on combining the bi-dimensional discrete wavelet transformation (2D-DWT) and vector quantization (VQ) for image compression. The 2D-DWT works in a non-separable fashion using a parallel filter structure with distributed control to compute two resolution levels. The wavelet coefficients of the higher frequency sub-bands are vector quantized using multi-resolution codebook and those of the lower frequency sub-band at level two are scalar quantized and entropy encoded. VQ is carried out by self organizing feature map (SOFM) neural nets working at the recall phase. Codebooks are quickly generated off-line using the same nets functioning at the training phase. The complete system, including the 2D-DWT, the multi-resolution codebook VQ, and the statistical encoder, was implemented on a Xilinx Virtex 4 FPGA and is capable of performing real-time compression for digital video when dealing with grayscale 512 × 512 pixels images. It offers high compression quality (PSNR values around 35 dB) and acceptable compression rate values (0.62 bpp).

Real-time image segmentation based on a parallel and pipelined watershed algorithm

The watershed transformation is a popular image segmentation technique for gray scale images. This paper describes a real-time image segmentation based on a parallel and pipelined watershed algorithm which is designed for hardware implementation. In our algorithm: (1) pixels in a given image are repeatedly scanned from top-left to bottom-right, and then from bottom-right to top-left, in order to achieve high performance on a pipelined circuit by simplifying memory access sequences, (2) all steps in the algorithm are executed at the same time in the pipelined circuit, (3) the amount of data that are scanned is gradually reduced as the calculation progresses by memorizing which data are modified in the previous scan, and (4) N pixels can be processed in parallel. In our current implementation on an off-the-shelf field-programmable gate array board, up to four pixels can be processed in parallel. The performance for 512 × 512 pixel images is fast enough to be the first step in real-time applications.

An FPGA-based architecture for a local tone-mapping operator

This paper presents an FPGA-based architecture for local tone mapping of gray scale high dynamic range images. The architecture is described in VHDL and has been synthesized using Altera Quartus tools. It achieves an operating frequency consistent with a video rate of 60 frames per second for a frame of 1,024 × 768 pixels. The proposed architecture is a modification of the nine-scale Reinhard operator. Approximations to the original Reinhard operator ensure that the operator is amenable to implementation in hardware. A peak signal-to-noise ratio study shows that our fixed-point hardware approximation produces results similar to a floating-point original.

Reconfigurable computing: design methodology and hardware tasks scheduling for real-time image processing

Technology evolution makes possible the integration of heterogeneous components as programmable elements (processors), hardware dedicated blocks, hierarchical memories and buses. Furthermore, an optimized reconfigurable logic core embedded within a System-on-Chip will associate the performances of dedicated architecture and the flexibility of programmable ones. In order to increase performances, some of the applications are carried out in hardware, using dynamically reconfigurable logic, rather than software, using programmable elements. This approach offers a suitable hardware support to design malleable systems able to adapt themselves to a specific application. This article makes a synthesis of the Ardoise project. The first objective of Ardoise project was to design and to produce a dynamically reconfigurable platform based on commercial FPGAs. The concept of dynamically reconfigurable architecture depends partially on new design methodologies elaboration as well as on the programming environment. The platform architecture was designed to be suitable for real-time image processing. The article outlines mainly the Ardoise tools aspect: development environment and real-time management of the hardware tasks. The proposed methodology is based on a dynamic management of tasks according to an application scenario written using C++ language.

TRLE—an efficient data compression scheme for image composition of volume rendering on distributed memory multicomputers

Data compression is a well-known method to improve the image composition time of parallel volume rendering on distributed memory multicomputers. In this paper, we propose an efficient data compression scheme, the template run-length encoding (TRLE) scheme, for image composition. Given an image with 2n×2n pixels, in the TRLE scheme, the image is treated as n×n blocks and each block has 2×2 pixels. Since a pixel can be a blank or non-blank pixel, there 16 templates in a block. To compress an image, the TRLE scheme encodes an image block by block similar to the run-length encoding scheme. However, the TRLE scheme can filter out or use small space to encode blocks whose four pixels are blank pixels, that is, the TRLE scheme can encode a partial image according to the shape of non-blank pixels. To evaluate the performance of the TRLE scheme, we compare the proposed scheme with the BR, the RLE, and the BRLC schemes. Since a data compression scheme needs to cooperate with some data communication schemes, in the implementation, the binary-swap, the parallel-pipelined, and the rotate-tiling data communication schemes are used. By combining the four data compression schemes with the three data communication schemes, we have twelve image composition methods. These twelve methods are implemented on an IBM SP2 parallel machine. Four volume datasets are used as test samples. The data computation time and the data communication time are measured. The experimental results show that the TRLE data compression scheme with the rotate-tiling data communication scheme outperforms other eleven image composition methods for all test samples.

Model-based mapping of reconfigurable image registration on FPGA platforms

Image registration is a computationally intensive application in the medical imaging domain that places stringent requirements on performance and memory management efficiency. This paper develops techniques for mapping rigid image registration applications onto configurable hardware under real-time performance constraints. Building on the framework of homogeneous parameterized dataflow, which provides an effective formal model of design and analysis of hardware and software for signal processing applications, we develop novel methods for representing and exploring the hardware design space when mapping image registration algorithms onto configurable hardware. Our techniques result in an efficient framework for trading off performance and configurable hardware resource usage based on the constraints of a given application. Based on trends that we have observed when applying these techniques, we also present a novel architecture that enables dynamically-reconfigurable image registration. This proposed architecture has the ability to tune its parallel processing structure adaptively based on relevant characteristics of the input images.

An FPGA implementation of real-time K-means clustering for color images

K-means clustering is a very popular clustering technique, which is used in numerous applications. In the k-means clustering algorithm, each point in the dataset is assigned to the nearest cluster by calculating the distances from each point to the cluster centers. The computation of these distances is a very time-consuming task, particularly for large dataset and large number of clusters. In order to achieve high performance, we need to reduce the number of the distance calculations for each point efficiently. In this paper, we describe an FPGA implementation of k-means clustering for color images based on the filtering algorithm. In our implementation, when calculating the distances for each point, clusters which are apparently not closer to the point than other clusters are filtered out using kd-trees which are dynamically generated on the FPGA in each iteration of k-means clustering. The performance of our system for 512 × 512 and 640 × 480  pixel images (24-bit full color RGB) is more than 30 fps, and 20–30 fps for 756 × 512 pixel images in average when dividing to 256 clusters.

Document image binarization by two-stage block extraction and background intensity determination

This paper presents a novel approach to binarizing document images. All blocks with individual background intensity values in a document image are first extracted using a two-stage extraction procedure. Then, the intensity distribution of each block is calculated to determine the variation ranges of background intensity. For each extracted block, interior pixels whose intensity values fall within these ranges are regarded as background pixels. For those pixels outside all extracted blocks, Otsu’s global threshold method is applied to binarize them. To evaluate the developed system, 275 representative document images are collected to evaluate the binarization results by recognizing characters extracted from those binarized images. These binarized images are generated using the proposed and other existent approaches and fed into the same optical character recognition system to evaluate the practicability of each method. The proposed document binarization method obtains the highest recognition accuracy.

A Necessary and Sufficient Contractivity Condition for the Fractal Transform Operator

This paper revisits the concept of fractal image coding and the contractivity conditions of the fractal transform operator. All such existing conditions are only sufficient. This paper formulates a necessary and sufficient condition for the contractivity of the fractal transform operator associated to a fractal code. Furthermore, analytical results on the convergence of the fractal image decoding will be derived.

An image rectification scheme and its applications in RST invariant digital image watermarking

This paper presents an image rectification scheme that can be used by any image watermarking algorithm to provide robustness against rotation, scaling and translation (RST) transformations. Rotation and scaling transformations in the spatial domain result in cyclically translational shifts in the log-polar mapping (LPM) of the magnitude of the Fourier transform spectrum of an image. We cut a small block from the LPM domain as a matching template. A new filtering method is proposed to compute the cross-correlation between this template and the magnitude of the LPM of the image having undergone RST transformations to detect the rotation and scaling parameters. We employ the same strategy in the spatial domain to detect the translational parameters in the spatial domain. The scheme can also be used to detect image flipping. The cost of the templates is low and the templates can even be compressed. The detection accuracy for rotation, scaling and translation is also presented. We compare the matching results for the different filters and their performance by the three criteria: signal-to-noise ratio (SNR), peak-to-correlation energy (PCE), and Horner efficiency. We demonstrate that our phase-only filtering method is the only one that can be used in the LPM domain. We also introduce three applications for this rectification scheme and give their experimental results.

FPGA-based minutia matching for biometric fingerprint image database retrieval

In this paper, a parallel-matching processor architecture with early jump-out (EJO) control is proposed to carry out high-speed biometric fingerprint database retrieval. The processor performs the fingerprint retrieval by using minutia point matching. An EJO method is applied to the proposed architecture to speed up the large database retrieval. The processor is implemented on a Xilinx Virtex-E, and occupies 6,825 slices and runs at up to 65 MHz. The software/hardware co-simulation benchmark with a database of 10,000 fingerprints verifies that the matching speed can achieve the rate of up to 1.22 million fingerprints per second. EJO results in about a 22% gain in computing efficiency.

FPGA-based real-time 3D image preprocessing for image-guided medical interventions

Minimally invasive image-guided interventions (IGIs) are time and cost efficient, minimize unintended damage to healthy tissue, and lead to faster patient recovery. One emerging trend in IGI workflow is to use volumetric imaging modalities such as low-dose computed tomography (CT) and 3D ultrasound to provide real-time, accurate anatomical information intraoperatively. These intraoperative images, however, are often characterized by quantum (in low-dose CT) or speckle (in ultrasound) noise and must be enhanced prior to any advanced image processing. Anisotropic diffusion filtering and median filtering have been shown to be effective in enhancing and improving the visual quality of these images. However, achieving real-time performance, as required by IGIs, using software-only implementations is challenging because of the sheer size of the images and the arithmetic complexity of the filtering operations. We present a field-programmable gate array-based reconfigurable architecture for real-time preprocessing of intraoperative 3D images. The proposed architecture provides programmable kernels for 3D anisotropic diffusion filtering and 3D median filtering within the same framework. The implementation of this architecture using an Altera Stratix-II device achieved a voxel processing rate close to 200 MHz, which enables the use of these processing techniques in the IGI workflow prior to advanced operations such as segmentation, registration, and visualization.

Custom parallel caching schemes for hardware-accelerated image compression

In an effort to achieve lower bandwidth requirements, video compression algorithms have become increasingly complex. Consequently, the deployment of these algorithms on field programmable gate arrays (FPGAs) is becoming increasingly desirable, because of the computational parallelism on these platforms as well as the measure of flexibility afforded to designers. Typically, video data are stored in large and slow external memory arrays, but the impact of the memory access bottleneck may be reduced by buffering frequently used data in fast on-chip memories. The order of the memory accesses, resulting from many compression algorithms are dependent on the input data (Jain in Proceedings of the IEEE, pp. 349–389, 1981). These data-dependent memory accesses complicate the exploitation of data re-use, and subsequently reduce the extent to which an application may be accelerated. In this paper, we present a hybrid memory sub-system which is able to capture data re-use effectively in spite of data-dependent memory accesses. This memory sub-system is made up of a custom parallel cache and a scratchpad memory. Further, the framework is capable of exploiting 2D spatial locality, which is frequently exhibited in the access patterns of image processing applications. In a case study involving the quad-tree structured pulse code modulation (QSDPCM) application, the impact of data dependence on memory accesses is shown to be significant. In comparison with an implementation which only employs an SPM, performance improvements of up to 1.7× and 1.4× are observed through actual implementation on two modern FPGA platforms. These performance improvements are more pronounced for image sequences exhibiting greater inter-frame movements. In addition, reductions of on-chip memory resources by up to 3.2× are achievable using this framework. These results indicate that, on custom hardware platforms, there is substantial scope for improvement in the capture of data re-use when memory accesses are data dependent.

FPGA implementation of a real-time biologically inspired image enhancement algorithm

This paper presents an FPGA implementation of a novel image enhancement algorithm, which compensates for the under-/over-exposed image regions, caused by the limited dynamic range of contemporary standard dynamic range image sensors. The algorithm, which is motivated by the attributes of the shunting center-surround cells of the human visual system, is implemented in Altera Stratix II GX: EP2SGX130GF1508C5 FPGA device. The proposed implementation, which is synthesized in an FPGA technology, employs reconfigurable pipeline, structured memory management, and data reuse in spatial operations, to render in real-time the huge amount of input data that the video signal comprises. It also avoids the use of computationally intensive operations, achieving the required specifications in terms of flexibility, timing, performance and visual quality. The proposed implementation allows real-time processing of color images with sizes up to 2.5 Mpixels, at frame rate of 25 fps. As a result, the architectural solution described in this work offers a low-cost implementation for automatic exposure correction in real-time video systems.

Fast real-time onboard processing of hyperspectral imagery for detection and classification

Remotely sensed hyperspectral imagery has many important applications since its high-spectral resolution enables more accurate object detection and classification. To support immediate decision-making in critical circumstances, real-time onboard implementation is greatly desired. This paper investigates real-time implementation of several popular detection and classification algorithms for image data with different formats. An effective approach to speeding up real-time implementation is proposed by using a small portion of pixels in the evaluation of data statistics. An empirical rule of an appropriate percentage of pixels to be used is investigated, which results in reduced computational complexity and simplified hardware implementation. An overall system architecture is also provided.

Adaptive training of video sets for image recognition on mobile phones

We present an enhancement towards adaptive video training for PhoneGuide, a digital museum guidance system for ordinary camera-equipped mobile phones. It enables museum visitors to identify exhibits by capturing photos of them. In this article, a combined solution of object recognition and pervasive tracking is extended to a client–server-system for improving data acquisition and for supporting scale-invariant object recognition. A static as well as a dynamic training technique are presented that preprocess the collected object data differently and apply two types of neural networks (NN) for classification. Furthermore, the system enables a temporal adaptation for ensuring a continuous data acquisition to improve the recognition rate over time. A formal field experiment reveals current recognition rates and indicates the practicability of both methods under realistic conditions in a museum.

---