Image Blending in a High Frame Rate FPGA-based Multi-Camera System

Panoptic is a custom spherical light field camera used as a polydioptric system where imagers are distributed over a hemispherical surface, each having its own vision of the surroundings and a distinct focal plane. The spherical light field camera records light information from any direction around its center. This paper revises previously developed Nearest Neighbor and Linear blending techniques. Novel Gaussian blending and Restricted Gaussian blending techniques for vision reconstruction of a virtual observer located inside the spherical geometry are presented. These new blending techniques improve the output quality of the reconstructed image with respect to the ordinary stitching techniques and simpler image blending algorithms. A comparison of the developed blending algorithms is also given in this paper. A hardware architecture based on Field Programmable Gate Arrays (FPGA) enabling the real-time implementation of the blending algorithms is presented, along with the imaging results and resource utilization comparison. A recorded omnidirectional video is attached as a supplementary material.     

A robust omnidirectional vision sensor for soccer robots

How to make robot vision work robustly under varying lighting conditions and without the constraint of the current color-coded environment are two of the most challenging issues in the RoboCup community. In this paper, we present a robust omnidirectional vision sensor to deal with these issues for the RoboCup Middle Size League soccer robots, in which two novel algorithms are applied. The first one is a camera parameters auto-adjusting algorithm based on image entropy. The relationship between image entropy and camera parameters is verified by experiments, and camera parameters are optimized by maximizing image entropy to adapt the output of the omnidirectional vision to the varying illumination. The second one is a ball recognition method based on the omnidirectional vision without color classification. The conclusion is derived that the ball on the field can be imaged to be an ellipse approximately in our omnidirectional vision, and the arbitrary FIFA ball can be recognized by detecting the ellipse imaged by the ball. The experimental results show that a robust omnidirectional vision sensor can be realized by using the two algorithms mentioned above.     

基于智能相机的高速三维表面信息获取

为了大幅度提高三维信息获取速度,提出了一种基于智能相机和激光三角法的高速三维信息获取与表面重建的系统,介绍了该系统的硬件架构和信号处理。实验结果表明,采用数字信号处理器和并行处理方式,该系统获取三维表面信息速度可达5600点/s以上。     

大视场红外成像探测系统实时信号处理技术研究

探讨了一种红外成像探测系统实时信号处理算法、硬件结构等关键技术.针对大视场红外探测系统,提出了适用于360°全方位红外目标搜索的实时处理算法原理和流程,并以高性能的数字信号处理器和FPGA芯片以及CPC I总线技术为核心设计了处理能力强、高速数据传输、接口可靠稳定的实时信号处理系统.实验测试表明,该系统工作稳定、可靠,满足总体性能要求.     

微光双谱彩色夜视系统研究

针对三代像增强器缺少蓝光相应的不足，设计了以图像融合为核心的微光实时彩色夜视系统，研究了适用于本系统的图像预处理和图像融合算法，并在软件仿真的基础上，以TMS320DM642评估板为硬件平台进行实时处理，取得了良好效果．     

Design and implementation of an omnidirectional vision system for robot perception

To meet the demand of surrounding detection of a humanoid robot, we developed an omnidirectional vision system for robot perception (OVROP) with 5 Degrees of Freedom (DOFs). OVROP has a modular design and mainly consists of three parts: hardware, control architecture and visual processing part (omnidirectional vision and stereovision). As OVROP is equipped with universal hardware and software interfaces it can be applied to various types of robots. Our performance evaluation proves that OVROP can accurately detect and track an object with 360° field of view (FOV). Besides, undistorted omnidirectional perception of surroundings can be achieved through calibrations of both monocular and stereo cameras. Furthermore, our preliminary experimental results show that OVROP can perceive a desired object within 160 ms in most cases. As a result, OVROP can provide detailed information on surrounding environment for full-scope and real-time robot perception.     

Registration of real and CT-derived virtual bronchoscopic images toassist transbronchial biopsy

This paper describes research work motivated by an innovative medical application: computer-assisted transbronchial biopsy. This project involves the registration, with no external localization device, of a preoperative three-dimensional (3-D) computed tomography (CT) scan of the thoracic cavity (showing a tumor that requires a needle biopsy), and an intraoperative endoscopic two-dimensional (2-D) image sequence, in order to provide assistance in transbronchial puncture of the tumor. Because of the specific difficulties resulting from the data being processed, a multilevel strategy was introduced. For each analysis level, the relevant information to process and the corresponding algorithms were defined. This multilevel strategy, thus, provides the best possible accuracy. Original image processing methods were elaborated, dealing with segmentation, registration and 3-D reconstruction of the bronchoscopic images. In particular, these methods involve adapted mathematical morphology tools, a “daemon-based” registration algorithm, and a model-based shape-from-shading algorithm. This pilot study presents the application of these algorithms to recorded bronchoscopic video sequences for five patients. The preliminary results presented here demonstrate that it is possible to precisely localize the endoscopic camera within the CT data coordinate system. The computer can thus synthesize in near real-time the CT-derived virtual view that corresponds to the actual endoscopic view     

A Unified FPGA-Based System Architecture for 2-D Discrete Wavelet Transform

This paper presents a novel unified and programmable 2-D Discrete Wavelet Transform (DWT) system architecture, which was implemented using a Field Programmable Gate Array (FPGA)-based Nios II soft-core processor working in combination with custom hardware accelerators generated through high-level synthesis. The proposed system architecture, synthesized on an Altera DE3 Stratix III FPGA board, was developed through an iterative design space exploration methodology using Altera’s C2H compiler. Experimental results show that the proposed system architecture is capable of real-time video processing performance for grayscale image resolutions of up to 1920?×?1080 (1080p) when ran on the Altera DE3 board, and it outperforms the existing 2-D DWT architecture implementations known in literature by a considerable margin in terms of throughput. While the proposed 2-D DWT system architecture satisfies real-time performance constraints, it can also perform both forward and inverse DWT, support a number of popular DWT filters used for image and video compression and provide architecture programmability in terms of number of levels of decomposition as well as image width and height. Based from the design principles used to implement the proposed 2-D DWT system architecture, a system design guideline can be formulated for SOC designs which plan to incorporate dedicated 2-D DWT hardware acceleration.     

Circuit board inspection using a range camera

A high-speed, high-accuracy three-dimensional (3-D) range camera that possesses sufficient speed and accuracy to inspect circuit boards in real time is presented. The architecture of the system is briefly described, and the algorithms used to make quantitative presolder measurements, including the orientations and lengths of leads and the locations and shapes of holes, are examined in detail. Since the unit runs in real time and performs sophisticated image analysis functions, considerable emphasis in the algorithm development was placed on the ability to partition tasks between general-purpose computers and array processors     

Investigation of a Sequential Three-Dimensional Process for Back-Illuminated CMOS Image Sensors With Miniaturized Pixels

A new 3-D CMOS image sensor architecture is presented as a potential candidate for submicrometer pixels. To overcome the scaling challenge related to miniaturized pixel design rules, far beyond traditional 3-D stacking alignment capabilities, a sequential construction is applied. This paper gives a technical overview of this 3-D scheme and validates a part of its building blocks. As a consequence of a sequential process, the thermal budget is limited to ensure bottom device immunity. Subsequently, high-quality SOI film transfer above the first layer by direct bonding and etch back is demonstrated. Finally, the low-temperature processing of HfO₂/TiN fully depleted silicon-on-insulator readout transistors is detailed and evaluated from a low frequency noise point of view.     

红外与可见光图像实时配准融合系统

描述了一个自主研制的基于实时分布式多处理机的图像配准和融合系统的设计与实现方案。本系统是具有并行计算机体系结构的通用高速实时图像融合处理系统,选择VxWorks实时操作系统和VEM64x总线的软硬件平台,采用AD公司新型的TS101DSP处理器为核心,多DSP处理器分布并行进行处理,完成多源图像实时高速配准和融合需要进行的大量运算,CPLD芯片完成了采集控制以及多传感器视频同步。由于采用了基于高性能DSP的实时嵌入式系统和通用标准化总线结构设计,该系统可以灵活地应用多种配准和融合算法来实现可见光和红外双通道数字图像的高速实时融合处理,比较好地解决多尺度图像配准融合算法的大数据量计算处理与系统实时性要求之间的矛盾,为多传感器实时图像配准融合处理系统的研制奠定了良好的技术基础。     

Planar mirrors for image-based robot localization and 3-D reconstruction

Planar catadioptric vision sensors consist of a pinhole camera observing a scene being reflected on two (or more) planar mirrors. These systems have recently received an increasing attention because, unlike stereo cameras, can capture two views of the same scene without the need of hardware multi-camera synchronization and calibration. In this paper we explore the original scenario in which a robot manipulator, equipped with a pinhole camera on its end-effector, observes an unknown 3-D scene both directly and reflected through multiple mirrors. We present new multiple-view properties for this scenario and, based on these theoretical results, we present new image-based camera localization and new 3-D scene reconstruction algorithms. Extensive simulation and real-data experiments illustrate the theory and show the effectiveness of the proposed designs.     

Parallel image processing with the block data parallel architecture

Many digital signal and image processing algorithms can be speeded up by executing them in parallel on multiple processors. The speed of parallel execution is limited by the need for communication and synchronization between processors. In this paper, we present a paradigm for parallel processing that we call the block data flow paradigm (BDFP). The goal of this paradigm is to reduce interprocessor communication and relax the synchronization requirements for such applications. We present the block data parallel architecture which implements this paradigm, and we present methods for mapping algorithms onto this architecture. We illustrate this methodology for several applications including two-dimensional (2-D) digital filters, the 2-D discrete cosine transform, QR decomposition of a matrix and Cholesky factorization of a matrix. We analyze the resulting system performance for these applications with regard to speedup and efficiency as the number of processors increases. Our results demonstrate that the block data parallel architecture is a flexible, high-performance solution for numerous digital signal and image processing algorithms     

基于视觉引导的陶瓷瓶口定位机器人设计

为了将视觉系统送入器皿瓷内部采集内表面图像,设计了一种门式三自由度直角坐标的陶瓷瓶口定位机器人。介绍了机器人的功能体系和硬件构成,研究了基于视觉原理引导机器人定位运动的实现方法：由CMOS相机拍摄瓶口图像,经过处理后拟合出瓶口外环圆,求取圆心坐标。以该坐标为目标位置,引导机器人XY轴作水平运动,将Z轴定位到瓶口中心。实验结果表明,机器人能快速、准确地获取瓶口图像和定位瓶口中心,为机器视觉应用在器皿瓷内表面的缺陷检测奠定了研究基础。     

Cascadable one/two-dimensional digital convolver

An architecture and a design for a high-speed CMOS digital convolver which can be used for real-time one-dimensional (1-D) and two-dimensional (2-D) signal processing are presented. In the 2-D mode this device can be used to convolve 10-bit image data with a 3×3 or 2×5 2-D eight-bit-per-coefficient impulse response at 20 M samples/s throughput. In 1-D applications it can be used as a ten-tap finite-impulse response (FIR) filter. Devices can be cascaded to increase the order of the convolution reference in both dimensions     

Robot-Vision Signal Processing Primitives

Recent advances in vision algorithms and increases in computer performance have made new capabilities available in autonomous robotics for real-time applications. In general, computer-vision-based solutions to robotics problems employ a high-level system architecture that makes use of low-level processing blocks. While the high-level system architecture is still an active area of research, many of the underlying low-level processing blocks and their associated methods have begun to stabilize, yielding a set of operators that has been found useful in a wide variety of tasks. In this article, we will briefly review these operators, which we call robot-vision signal-processing (SP) primitives, and their associated classes of methods. Although our taxonomy is quite general and related to those used in image processing and pattern recognition, we focus on the specific use of these primitives and associated classes in robot vision for SP purposes before presenting a robot-vision example- the Stanley robot racing car.     

用ADSP21062高速信号处理系统实现机载SAR实时成像处理器的方位向处理

机载SAR实时成像处理器可以在载机飞行的同时获得高分辨率的SAR图像,对于实时监测、军事侦察等应用具有重要意义。实时成像处理器就是用高速数字信号处理系统来实时地实现SAR的成像算法。该文介绍SAR实时成像处理器方位向处理部分的研制,该部分采用了自行开发的、基于ADSP21062的高速信号处理系统,8片ADSP21062被安排在4个并行处理通道中,具有960MFLOPS的峰值处理速度,优化的软件设计保证了硬件资源的利用效率。仿真测试和外场实验证明了该系统的设计是成功的。该文对方位向处理部分的实现原理、硬件结构、软件设计进行了详细介绍。     

一种新型机器视觉系统及其图像矫正算法探索

数字成像和图像传输技术的新发展,为研制新一代机器视觉系统提供了基础条件.基于此,提出了一种由多个像素少、成本低的摄像头构成的机器视觉系统的结构模型,并重点探讨了它的成像方法和图像矫正算法.这种新型的机器视觉系统可应用于模式识别,能获取并处理不规则的图像,有利于解决传统机器视觉系统的成本高和图像冗余数据量大的问题.     

Efficient Hardware-Based Image Compression Schemes for Wireless Sensor Networks: A Survey

Multidimensional sensors, such as digital camera sensors in the visual sensor networks VSNs generate a huge amount of information compared with the scalar sensors in the wireless sensor networks WSNs. Processing and transmitting such data from low power sensor nodes is a challenging issue through their limited computational and restricted bandwidth requirements in a hardware constrained environment. Source coding can be used to reduce the size of vision data collected by the sensor nodes before sending it to its destination. With image compression, a more efficient method of processing and transmission can be obtained by removing the redundant information from the captured image raw data. In this paper, a survey of the main types of the conventional state of the art image compression standards such as JPEG and JPEG2000 is provided. A literature review of their advantages and shortcomings of the application of these algorithms in the VSN hardware environment is specified. Moreover, the main factors influencing the design of compression algorithms in the context of VSN are presented. The selected compression algorithm may have some hardware-oriented properties such as; simplicity in coding, low memory need, low computational load, and high-compression rate. In this survey paper, an energy efficient hardware based image compression is highly requested to counter the severe hardware constraints in the WSNs.     

基于双目视觉的受电弓碳滑板磨耗检测

将数字图像处理技术和机器视觉应用于受电弓碳滑板磨耗测量,是碳滑板磨耗测量的主要发展方向,文章提出了一种基于双目视觉的受电弓碳滑板磨耗检测方法。该算法首先对双目测量系统进行相机标定,得到双目相机的标定矩阵,然后对双目相机采集得到的碳滑板图像进行边缘提取,利用标定矩阵计算得到两个相机中碳滑板边缘点的对应关系,最后利用对应点计算出碳滑板边缘点的实际空间坐标,从而计算出碳滑板的厚度值。实验及数据处理结果表明：系统的测量误差较小,且此系统安装方便,受外界因素的影响较小。