Fast Electronic Digital Image Stabilization for Off-Road Navigation

Image stabilization can be used as front-end system for many tasks that require dynamic image analysis, such as navigation and tracking of independently moving objects from a moving platform. We present a fast and robust electronic digital image stabilization system that can handle large image displacements based on a two-dimensional feature-based multi-resolution motion estimation technique. The method tracks a small set of features and estimates the movement of the camera between consecutive frames. Stabilization is achieved by combining all motion from a reference frame and warping the current frame back to the reference. The system has been implemented on parallel pipeline image processing hardware (a Datacube MaxVideo 200) connected to a SUN SPARCstation 20/612 via a VME bus adaptor. Experimental results using video sequences taken from a camera mounted on a vehicle moving on rough terrain show the robustness of the system while running at approximately 20 frames/s.     

Stereoscopic Scene Flow Computation for 3D Motion Understanding

Building upon recent developments in optical flow and stereo matching estimation, we propose a variational framework for the estimation of stereoscopic scene flow, i.e., the motion of points in the three-dimensional world from stereo image sequences. The proposed algorithm takes into account image pairs from two consecutive times and computes both depth and a 3D motion vector associated with each point in the image. In contrast to previous works, we partially decouple the depth estimation from the motion estimation, which has many practical advantages. The variational formulation is quite flexible and can handle both sparse or dense disparity maps. The proposed method is very efficient; with the depth map being computed on an FPGA, and the scene flow computed on the GPU, the proposed algorithm runs at frame rates of 20 frames per second on QVGA images (320×240 pixels). Furthermore, we present solutions to two important problems in scene flow estimation: violations of intensity consistency between input images, and the uncertainty measures for the scene flow result.     

Effect of the field of view on perceiving world-referenced image motion during concurrent head movements

We have previously shown that concurrent head movements impair head-referenced image motion perception when compensatory eye movements are suppressed (Li, Adelstein, & Ellis, 2009) [16]. In this paper, we examined the effect of the field of view on perceiving world-referenced image motion during concurrent head movements. Participants rated the motion magnitude of a horizontally oscillating checkerboard image presented on a large screen while making yaw or pitch head movements, or holding their heads still. As the image motion was world-referenced, head motion elicited compensatory eye movements from the vestibular-ocular reflex to maintain the gaze on the display. The checkerboard image had either a large (73°H × 73°V) or a small (25°H × 25°V) field of view (FOV). We found that perceptual sensitivity to world-referenced image motion was reduced by 20% during yaw and pitch head movements compared to the veridical levels when the head was still, and this reduction did not depend on the display FOV size. Reducing the display FOV from 73°H × 73°V to 25°H × 25°V caused an overall underestimation of image motion by 7% across the head movement and head still conditions. We conclude that observers have reduced perceptual sensitivity to world-referenced image motion during concurrent head movements independent of the FOV size. The findings are applicable in the design of virtual environment countermeasures to mitigate perception of spurious motion arising from head tracking system latency.     

Video frame interpolation via optical flow estimation with image inpainting

As we all know, video frame rate determines the quality of the video. The higher the frame rate, the smoother the movements in the picture, the clearer the information expressed, and the better the viewing experience for people. Video interpolation aims to increase the video frame rate by generating a new frame image using the relevant information between two consecutive frames, which is essential in the field of computer vision. The traditional motion compensation interpolation method will cause holes and overlaps in the reconstructed frame, and is easily affected by the quality of optical flow. Therefore, this paper proposes a video frame interpolation method via optical flow estimation with image inpainting. First, the optical flow between the input frames is estimated via combined local and global-total variation (CLG-TV) optical flow estimation model. Then, the intermediate frames are synthesized under the guidance of the optical flow. Finally, the nonlocal self-similarity between the video frames is used to solve the optimization problem, to fix the pixel loss area in the interpolated frame. Quantitative and qualitative experimental results show that this method can effectively improve the quality of optical flow estimation, generate realistic and smooth video frames, and effectively increase the video frame rate.     

A real-time global stereo-matching on FPGA

An improved global stereo matching algorithm is implemented on a single FPGA for real-time applications. Stereo matching is widely used in stereo vision systems, i.e. objects detection and autonomous vehicles. Global algorithms have much more accurate results than local algorithms, but global algorithms are not implemented on FPGA since they rely over high-end hardware resources. In this implementation the stereo pairs are divided into blocks, the hardware resources are reduced by processing one block once. The hardware implementation is based on a Xilinx Kintex 7 FPGA. Experiment results show that the proposed implementation has an accurate result for the Middlebury benchmarks and 30 frames per second (fps) @1920 × 1680 is achieved.     

Performance assessment and tuning for exchange of clinical documents cross healthcare enterprises

BackgroundTo integrate electronic health records (EHRs) from diverse document sources across healthcare providers, facilities, or medical institutions, the IHE XDS.b profile can be considered as one of the solutions. In this research, we have developed an EHR/OpenXDS system which adopted the OpenXDS, an open source software that complied with the IHE XDS.b profile, and which achieved the EHR interoperability.ObjectiveWe conducted performance testing to investigate the performance and limitations of this EHR/OpenXDS system.MethodologyThe performance testing was conducted for three use cases, EHR submission, query, and retrieval, based on the IHE XDS.b profile for EHR sharing. In addition, we also monitored the depletion of hardware resources (including the CPU usage, memory usage, and network usage) during the test cases execution to detect more details of the EHR/OpenXDS system's limitations.ResultsIn this EHR/OpenXDS system, the maximum affordable workload of the EHR submissions were 400 EHR submissions per hour, the DSA CPU usage was 20%, memory usage was 1380 MB, the network usages were 0.286 KB input and 7.58 KB output per minute; the DPA CPU usage was 1%, memory usage was 1770 MB, the network usages were 7.75 KB input and 1.54 KB output per minute; the DGA CPU usage was 24%, memory usage was 2130 MB, the network usages were 1.3 KB input and 0.174 KB output per minute. The maximum affordable workload of the EHR queries were 600 EHR queries per hour, the DCA CPU usage was 66%, the memory usage was 1660 MB, the network usages were 0.230 KB input and 0.251 KB output per minute; the DGA CPU usage was 1%, the memory usage was 1890 MB, the network usages were 0.273 KB input and 0.22 KB output per minute. The maximum affordable workload of the EHR retrievals were 2000 EHR retrievals, the DCA CPU usage was 79%, the memory usage was 1730 MB, the network usages were 19.55 KB input and 1.12 KB output per minute; the DPA CPU usage was 3.75%, the memory usage was 2310 MB, and the network usages were 0.956 KB input and 19.57 KB output per minute.Discussion and conclusionFrom the research results, we suggest that future implementers who deployed the EHR/OpenXDS system should consider the following aspects. First, to ensure how many service volumes would be provided in the environment and then to adjust the hardware resources. Second, the IHE XDS.b profile is adopted by the SOAP (Simple Object Access Protocol) web service, it might then move onto the Restful (representational state transfer) web service which is more efficient than the SOAP web service. Third, the concurrency process ability should be added in the OpenXDS source code to improve the hardware usage more efficiently while processing the ITI-42, ITI-18, and ITI-43 transactions. Four, this research suggests that the work should continue on adjusting the memory usage for the modules of the OpenXDS thereby using the memory resource more efficiently, e.g., the memory configuration of the JVM (Java Virtual Machine), Apache Tomcat, and Apache Axis2. Fifth, to consider if the hardware monitoring would be required in the implementing environment. These research results provided some test figures to refer to, and it also gave some tuning suggestions and future works to continue improving the performance of the OpenXDS.     

A visual system for real time detection of goal events during soccer matches

During soccer matches a number of doubtful situations arise that cannot be easily judged by the referee committee. An automatic visual system that checks objectively image sequences would prevent wrong interpretations due to perspective errors, occlusions, or high velocity of the events. In this work we present a real time visual system for goal detection. Four cameras with high frame rates (200 fps) are placed on the two sides of the goal lines. Four computers process the images acquired by the cameras detecting the ball position in real time; the processing result is sent to a central supervisor which evaluates the goal event probability and, when the goal is detected, forwards a warning signal to the referee that takes the final decision.     

Action recognition via bio-inspired features: The richness of center–surround interaction

Motion is a key feature for a wide class of computer vision approaches to recognize actions. In this article, we show how to define bio-inspired features for action recognition. To do so, we start from a well-established bio-inspired motion model of cortical areas V1 and MT. The primary visual cortex, designated as V1, is the first cortical area encountered in the visual stream processing and early responses of V1 cells consist in tiled sets of selective spatiotemporal filters. The second cortical area of interest in this article is area MT where MT cells pool incoming information from V1 according to the shape and characteristic of their receptive field. To go beyond the classical models and following the observations from Xiao et al. [61], we propose here to model different surround geometries for MT cells receptive fields. Then, we define the so-called bio-inspired features associated to an input video, based on the average activity of MT cells. Finally, we show how these features can be used in a standard classification method to perform action recognition. Results are given for the Weizmann and KTH databases. Interestingly, we show that the diversity of motion representation at the MT level (different surround geometries), is a major advantage for action recognition. On the Weizmann database, the inclusion of different MT surround geometries improved the recognition rate from 63.01 ± 2.07% up to 99.26 ± 1.66% in the best case. Similarly, on the KTH database, the recognition rate was significantly improved with the inclusion of MT different surround geometries (from 47.82 ± 2.71% up to 92.44 ± 0.01% in the best case). We also discussed the limitations of the current approach which are closely related to the input video duration. These promising results encourage us to further develop bio-inspired models incorporating other brain mechanisms and cortical areas in order to deal with more complex videos.     

A MEMS-based resonant-scanning lamellar grating Fourier transform micro-spectrometer with laser reference system

A lamellar grating Fourier transform infra-red (FTIR) micro-spectrometer is presented in which the device is electromagnetically actuated in resonant mode so as to achieve larger displacements with a lower driving voltage. By actuating at resonance, we can also have a design with a higher spring stiffness design such that the micro-spectrometer will have little influence from external perturbation. A data acquisition electronic system is designed such that the interferogram of the IR source can still be acquired at a fixed optical path distance (OPD) intervals. This is achieved by using a reference laser source. Working at a resonant frequency of 330 Hz, a 100 μm (bi-directional) displacement is achieved by the device with an input voltage of 2.2 V. A tunable laser source is used to demonstrate the system performance. The peak of the recorded spectra is very close to the actual wavelength of the IR, with a maximum difference of less than 5 nm.     

Real time contour tracking with a new edge detector

In this paper, a new system for real time contour tracking is presented. If a rough contour of the desired structure is available on the first image of a sequence, the system can automatically outline the contours on the subsequent images at video rate. The method we used is based on a new edge detector which was obtained by the generalization of the first order absolute central moment operator. The new algorithm proved to be very robust to noise and fast enough to be implemented in real time. The contour tracking procedure was implemented on an integrated software/hardware platform composed of a personal computer equipped with a digital signal processing board. The system can capture an analog video signal with a resolution of 512×512 pixels, 25 frames/s, process the data and display the results in real time. A graphical user interface is also available to interact with the system. Tests on images of the descending thoracic aorta and of a carotid, recorded by echocardiography, are reported. The cross-sectional area of the aorta and the diameter of the carotid were computed in real time and plotted on the user interface. The system proved to be a useful tool for the investigation of vascular mechanisms.     

Facial feature tracking for cursor control

This work is motivated by the goal of providing a non-contact means of controlling the mouse pointer on a computer system for people with motor difficulties using low-cost, widely available hardware. The required information is derived from video data captured using a web camera mounted below the computer's monitor. A colour filter is used to identify skin coloured regions. False positives are eliminated by optionally removing background regions and by applying statistical rules that reliably identify the largest skin-coloured region, which is assumed to be the user's face. The nostrils are then found using heuristic rules. The instantaneous location of the nostrils is compared with their at-rest location; any significant displacement is used to control the mouse pointer's movement. The system is able to process 18 frames per second at a resolution of 320 by 240 pixels, or 30 fps at 160 by 120 pixels using moderately powerful hardware (a 500 MHz Pentium III desktop computer).     

Adaptive Census Transform: A novel hardware-oriented stereovision algorithm

This paper presents a new hardware-oriented approach for the extraction of disparity maps from stereo images. The proposed method is based on the herein named Adaptive Census Transform that exploits adaptive support weights during the image transformation; the adaptively weighted sum of SADs is then used as the dissimilarity metric. Quality tests show that the proposed method reaches significantly better accuracy than alternative hardware-oriented approaches. To demonstrate the practical hardware feasibility, a specific architecture has been designed and its implementation has been carried out using a single FPGA chip. Such a VLSI implementation allows a frame rate up to 68 fps to be reached for 640 × 480 stereo images, using just 80,000 slices and 32 RAM blocks of a Virtex6 chip.     

基于光流法的定点监控虚拟预置位图像提取方法

大范围定点监控存在摄像头预置位不足以覆盖全部监控区域的问题，而依靠摄像头本身的三维定位功能抓取监控点图像速度太慢，且可能存在定位误差。针对上述问题，提出了一种用于土地违法与违章建筑大范围定点监控的虚拟预置位图像提取及配准方法。对前后两帧图像进行灰度转化和中值滤波去除噪声点，使用基于金字塔的Lucas-Kanade光流算法计算前一帧图像强角点的光流，通过计算仿射变换得到帧间运动矢量；对相邻帧间运动矢量进行积分，得到每一帧图像总的运动矢量；通过提取出运动矢量与虚拟预置位运动矢量最接近的那一帧，得到所需的新时相虚拟预置位图像。实验结果表明，本文算法可以更快速地提取出同一监控点不同时相的监控对比图像，图像重叠率优于95%，不论是重叠率还是图像质量都可以很好地满足日常监控的需求。     

Multi-class fault diagnosis of induction motor using Hilbert and Wavelet Transform

The information extraction capability of two widely used signal processing tools, Hilbert Transform (HT) and Wavelet Transform (WT), is investigated to develop a multi-class fault diagnosis scheme for induction motor using radial vibration signals. The vibration signals are associated with unique predominant frequency components and instantaneous amplitudes depending on the motor condition. Using good systematic and analytical approach this fault frequencies can be identified. However, some faults either electrical or mechanical in nature are associated with same or similar vibration frequencies leading to erroneous conclusions. Genetic Algorithm (GA) is proposed and used successfully to find the most relevant fault frequencies in radial (vertical) frame vibration signal which can be used to diagnose the induction motor faults very effectively even in the presence of noise. The information obtained by Continuous Wavelet Transform (CWT) was found to be highly redundant compared to HT and thus by selecting the most relevant features using GA, the fault classification accuracy has considerably improved especially for CWT. Almost similar fault frequencies were found using CWT + GA and HT + GA for radial vibration signal.     

An enhanced one-dimensional SPIHT algorithm and its implementation for TV systems

In general, to achieve high compression efficiency, a 2D image or a 2D block is used as the compression unit. However, 2D compression requires a large memory size and long latency when input data are received in a raster scan order that is common in existing TV systems. To address this problem, a 1D compression algorithm that uses a 1D block as the compression unit is proposed. 1D set partitioning in hierarchical trees (SPIHT) is an effective compression algorithm that fits the encoded bit length to the target bit length precisely. However, the 1D SPIHT can have low compression efficiency because 1D discrete wavelet transform (DWT) cannot make use of the redundancy in the vertical direction. This paper proposes two schemes for improving compression efficiency in the 1D SPIHT. First, a hybrid coding scheme that uses different coding algorithms for the low and high frequency bands is proposed. For the low-pass band, a differential pulse code modulation–variable length coding (DPCM–VLC) is adopted, whereas a 1D SPIHT is used for the high-pass band. Second, a scheme that determines the target bit length of each block by using spatial correlation with a minimal increase in complexity is proposed. Experimental results show that the proposed algorithm improves the average peak signal to noise ratio (PSNR) by 2.97 dB compared with the conventional 1D SPIHT algorithm. With the hardware implementation, the throughputs of both encoder and decoder designs are 6.15 Gbps, and gate counts of encoder and decoder designs are 42.8 K and 57.7 K, respectively.     

Motion segmentation and depth ordering using an occlusion detector

We present a novel method for motion segmentation and depth ordering from a video sequence in general motion. We first compute motion segmentation based on differential properties of the spatio-temporal domain, and scale-space integration. Given a motion boundary, we describe two algorithms to determine depth ordering from two- and three- frame sequences. An remarkable characteristic of our method is its ability compute depth ordering from only two frames. The segmentation and depth ordering algorithms are shown to give good results on 6 real sequences taken in general motion. We use synthetic data to show robustness to high levels of noise and illumination changes; we also include cases where no intensity edge exists at the location of the motion boundary, or when no parametric motion model can describe the data. Finally, we describe human experiments showing that people, like our algorithm, can compute depth ordering from only two frames, even when the boundary between the layers is not visible in a single frame.     

Parallel c-means algorithm for image segmentation on a reconfigurable mesh computer

In this paper, we propose a parallel algorithm for data classification, and its application for Magnetic Resonance Images (MRI) segmentation. The studied classification method is the well-known c-means method. The use of the parallel architecture in the classification domain is introduced in order to improve the complexities of the corresponding algorithms, so that they will be considered as a pre-processing procedure. The proposed algorithm is assigned to be implemented on a parallel machine, which is the reconfigurable mesh computer (RMC). The image of size (m × n) to be processed must be stored on the RMC of the same size, one pixel per processing element (PE).     

视频镜头边界检测整体方案的设计与实现

由于DC系数不需要完全解码就能从MPEG码流中容易地提取且采用DC系数恢复的图像在计算帧间特征值差时能克服小运动的影响,同时具有抗噪声干扰的特性,基于对DC图像序列的处理,设计并实现了针对切变和渐变的整体检测方案。实验结果表明该方案对这两种镜头边界均能取得良好的检测效果。