An In-Camera Data Stream Processing System for Defect Detection in Web Inspection Tasks

One of the aims of industrial machine vision is to develop computer and electronic systems to replace human vision in quality control of industrial production. Traditionally these systems consist of a line scan camera, host computer, frame grabber and one or more dedicated processing boards. In this paper we discuss the development of a new integrated design environment, developed for real-time defect detection, that eliminates the need for an external frame grabber and other associated host computer peripheral systems. The processing board contains a reconfigurable field programmable gate array FPGA inside a DALSA CCD camera. The FPGA is directly connected to the video data-stream and outputs data to a low bandwidth output bus. The system is targeted for web inspection but has the potential for broader application areas. We describe and show test results of the in-camera prototype system board and discuss some of the algorithms currently simulated and implemented for web inspection applications.     

Data Buffering and Allocation in Mapping Generalized Template Matching on Reconfigurable Systems

Image processing algorithms for 2D digital filtering, morphologic operations, motion estimation, and template matching involve massively parallel computations that can benefit from using reconfigurable systems with massive field programmable gate array (FPGA) hardware resources. In addition, each algorithm can be considered a special case of a generalized template matching (GTM) operation. Application performance on reconfigurable computer systems is often limited by the bandwidth to host or off chip memory. This paper describes the GTM operation and characterizes the data allocation and buffering strategies for the GTM operation on reconfigurable computers. Several mechanisms that support different levels of parallelism are proposed and summarized in the paper. Finally, the implementation of an infrared automatic target recognition application on two commercial FPGA boards is used to demonstrate the various design options with different data allocation and buffering mechanisms and the pruning of the design space based on the FPGA area and memory constraints.     

一种多通道并行固态存储系统的设计与实现

鉴于高速数据采集系统对实时数据存储带宽和容量的要求,提出一种基于现场可编程门阵列（FPGA）的高速多通道并行固态存储系统。该系统以现场可编程门阵列器件XCV5LX110T为核心,选用大容量高速闪存芯片作为存储介质,通过采用并行总线拓宽技术和流水线缓冲技术,在FPGA片内搭建高速多通道并行存储硬件架构,从硬件角度提高系统的数据吞吐带宽。设计一种基于超级页的地址映射策略,并使用该策略对闪存转换层算法的请求处理机制进行并行加速优化,从软件角度提高系统的存储并行性。测试结果表明,该系统的最大存储速度达到73MB／s,其性能指标能满足高速实时数据存储的需求,证明多通道存储架构和FTL算法具有良好的并行性和可扩展性。     

A heterogeneous computing system for coupling 3D endomicroscopy with volume rendering in real-time image visualization

Modern microscopic volumetric imaging processes lack capturing flexibility and are inconvenient to operate. Additionally, the quality of acquired data could not be assessed immediately during imaging due to the lack of a coherent real-time visualization system. Thus, to eliminate the requisition of close user supervision while providing real-time 3D visualization alongside imaging, we propose and describe an innovative approach to integrate imaging and visualization into a single pipeline called an online incrementally accumulated rendering system. This system is composed of an electronic controller for progressive acquisition, a memory allocator for memory isolation, an efficient memory organization scheme, a compositing scheme to render accumulated datasets, and accumulative frame buffers for displaying non-conflicting outputs. We implement this design using a laser scanning confocal endomicroscope, interfaced with an FPGA prototyping board through a custom hardware circuit. Empirical results from practical implementations deployed in a cancer research center are presented in this paper.     

基于DSP/FPGA的嵌入式实时目标跟踪系统

提出了一套基于DSP／FPGA的协处理器结构用以实现实时目标跟踪的嵌入式视觉系统。系统由DSP作为主处理器进行全局控制，利用具有流水线并行处理结构的FPGA作为协处理器实时完成DSP分配的处理任务。系统由FPGA快速完成最初的运动估计的结果，DSP在此基础上进一步分析和校正，并将校正信息反馈给FPGA，实现快速而准确的跟踪。     

基于FPGA的双目立体视觉系统

立体视觉的目的之一就是为了获得周围场景的3维信息,其关键在于匹配算法。然而即便是使用目前先进的通用处理器,其计算致密视差图所需的时间仍无法满足高速自主导航的需求。为了解决这个问题,提出了一种基于现场可编程门阵列(FPGA)的双目立体视觉系统的设计方案,同时介绍了系统的硬件结构,并在讨论区域匹配的快速算法的基础上,提出了基于FPGA的像素序列和并行窗口算法框架,用以实现零均值像素灰度差平方和(ZSSD)的匹配算法。该算法是先将视频信号经解码芯片生成场景立体图像对,并由FPGA来完成立体图像对的几何校正和ZSSD匹配算法,然后将获得的致密视差图通过PC I总线发送至上位机。实践表明,该算法效果好、速度快,不仅具有较强的鲁棒性,并且硬件系统性能稳定、可靠。此外,该方案还适用于像素灰度差的绝对值和(SAD)和像素灰度差的平方和(SSD)等多种传统区域匹配算法的快速实现和实时处理。     

基于PCI的双通道高速数据采集累加系统

介绍了一种基于PCI的双通道高速数据采集累加系统。它能够在高速采集的同时通过FPGA实现实时累加，并且通过双通道A／D交替采样提高采样率。同时介绍了双通道A／D同步的一种简单实现方法，给出了一种基于FPGA的高性价比累加方案，分析了累加操作中数据对齐、缓存、合并以及多时钟域传输的难点，最后给出了完整的系统调试方案。该系统可以用于强噪声背景中的周期性微弱信号的提取。     

基于FPGA的运动目标远程监视系统设计

运动目标检测是智能安防系统的重要组成部分,为了满足安防系统远距离监视目标以及视频传输实时性等需求,设计了一种基于FPGA平台的运动目标远程监视系统;该系统以Xilinx公司的Artix7系列FPGA芯片为核心,通过OV5640摄像头实现视频图像的采集,将采集到的图像进行灰度化处理,并通过DDR3存储器缓存处理后的图像,采用帧间差分法运动目标检测技术实现对多个运动物体的检测与标记,将检测结果通过以太网的UDP协议传输到上位机实时显示;实验结果表明,在图像分辨率为640*480时,以太网UDP传输速度为133Mbit/s,视频图像帧率为26fps,大于人眼的可视帧率24fps,满足视频传输实时性的要求,同时该系统能够远距离、高效地检测与跟踪多个运动目标,相比于其他系统具有可远程实时检测、小型化、低功耗的特点,可进一步应用到智能安防系统中。     

Computer Vision on Mars

Increasing the level of spacecraft autonomy is essential for broadening the reach of solar system exploration. Computer vision has and will continue to play an important role in increasing autonomy of both spacecraft and Earth-based robotic vehicles. This article addresses progress on computer vision for planetary rovers and landers and has four main parts. First, we review major milestones in the development of computer vision for robotic vehicles over the last four decades. Since research on applications for Earth and space has often been closely intertwined, the review includes elements of both. Second, we summarize the design and performance of computer vision algorithms used on Mars in the NASA/JPL Mars Exploration Rover (MER) mission, which was a major step forward in the use of computer vision in space. These algorithms did stereo vision and visual odometry for rover navigation and feature tracking for horizontal velocity estimation for the landers. Third, we summarize ongoing research to improve vision systems for planetary rovers, which includes various aspects of noise reduction, FPGA implementation, and vision-based slip perception. Finally, we briefly survey other opportunities for computer vision to impact rovers, landers, and orbiters in future solar system exploration missions.     

A Cognitive Memory-Augmented Network for Visual Anomaly Detection

With the rapid development of automated visual analysis, visual analysis systems have become a popular research topic in the field of computer vision and automated analysis. Visual analysis systems can assist humans to detect anomalous events (e.g., fighting, walking alone on the grass, etc). In general, the existing methods for visual anomaly detection are usually based on an autoencoder architecture, i.e., reconstructing the current frame or predicting the future frame. Then, the reconstruction error is adopted as the evaluation metric to identify whether an input is abnormal or not. The flaws of the existing methods are that abnormal samples can also be reconstructed well. In this paper, inspired by the human memory ability, we propose a novel deep neural network (DNN) based model termed cognitive memory-augmented network (CMAN) for the visual anomaly detection problem. The proposed CMAN model assumes that the visual analysis system imitates humans to remember normal samples and then distinguishes abnormal events from the collected videos. Specifically, in the proposed CMAN model, we introduce a memory module that is able to simulate the memory capacity of humans and a density estimation network that can learn the data distribution. The reconstruction errors and the novelty scores are used to distinguish abnormal events from videos. In addition, we develop a two-step scheme to train the proposed model so that the proposed memory module and the density estimation network can cooperate to improve performance. Comprehensive experiments evaluated on various popular benchmarks show the superiority and effectiveness of the proposed CMAN model for visual anomaly detection comparing with the state-of-the-arts methods. The implementation code of our CMAN method can be accessed at https://github.com/CMAN-code/CMAN_pytorch.      

Computer vision and Mathematica

Computer vision algorithms are strongly based on advanced mathematical methods. Tools to efficiently develop new methods are increasingly based on computer algebra systems, such as Matlab, Maple and Mathematica. For rapid prototyping both symbolic as numerical capability is required. However, the application to large datasets such as images has always been frustated by the expensive use of memory or long computing times. Today such programs have improved substantially in numerical data handling, making them an ideal tool for rapid prototyping in computer vision. This paper gives a short variety of examples of the use of Mathematica in computer vision research.     

基于FPGA的弹载数据回读系统设计

惯性传感器在各种导弹武器中的广泛应用,使得采集存储系统成为弹体飞行参数测量中不可或缺的存储设备。基于采集存储系统数据读取的需要,设计了一种基于FPGA的弹载数据回读系统。本系统以FPGA为控制核心,FT245BL芯片为USB控制芯片,FPGA模拟的FIFO为数据缓存,从而实现数据的传输。试验验证表明,该回读系统能够很好地完成数据传输工作,且传输数据迅速、准确,无错帧与丢帧现象,具备一定的工程实用价值。     

基于PCI总线多路图像采集卡的研究

本文在设计了一种基于PCI总线的多路同步图像采集卡,该视频采集卡以FPGA为逻辑控制中心,采用SAA7111将四路视频信号分别转换为数字图像数据,经FIFO缓存后,由PCI总线接口芯片PCI9052将数据送入计算机,最后通过应用程序将图像显示出来。     

基于FPGA与DDR2 SDRAM的大容量异步FIFO缓存设计

为了满足高速实时数据采集系统对所采集海量数据进行缓存的要求,通过研究FIFO的基本工作原理,利用FPGA和DDR2 SDRAM设计了一种高速大容量异步FIFO。使用Xilinx提供的存储器接口生成器(MIG)实现FPGA与DDR2的存储器接口,并结合片上FIFO和相应的控制模块完成FIFO的基本框架结构。详细介绍了各个组成模块的功能和原理,并设计了专门的测试模块。     

基于内存交换的网闸系统的研究与实现

物理隔离网闸是一种应用级的安全隔离系统,主要用来解决网络安全所带来的问题。文章重点介绍了一种新的基于内存交换的物理隔离网闸系统的工作原理、主控FPGA芯片的架构以及设备驱动如何控制硬件操作等。主控FPGA芯片采用ATA协议中UDMA 5实现主机间的通信,采用独立控制逻辑设计,还支持CRC校验以保证板间数据可靠传输,CRC校验出错支持数据重传。数据传输率是网闸系统的一个重要技术指标,文章同时分析了数据包长对数据传输率的影响。     

基于PCI Express高速数据采集卡的接口设计

随着数据采集技术和大容量Flash存储器的快速发展,对数据传输速度的要求越来越高,针对这一问题,提出了一种基于PCI Express的高速数据采集的接口设计方案,并给出了该系统的详细设计,实现了将外部数据高速传入计算机的功能。在硬件实现部分,主要对FPGA协议模块、PCI Express接口模块、数据接收、发送模块进行了描述与设计。     

基于计算机视觉的烟叶自动分级系统硬件设计

介绍了一种基于计算机视觉烟叶自动分级系统的硬件组成,该系统由输送装置、称重系统、视觉系统、均匀照明室4部分组成。输送装置可将烟叶打散摊平,使烟叶保持一定的厚度呈现给CCD摄像头;称重系统采用电子皮带秤完成烟叶的在线实时称重,并将数据发送至计算机内存;视觉系统同时监视、采集烟叶图像信息送至计算机内存进行处理;照明系统为CCD摄像头均匀、恒定的光源,保证图像的清晰。实验证明了系统的可行性,其辅助机构与烟叶实时分级决策算法尚待进一步的研究。     

A novel hardware plane fitting implementation and applications for bionic vision

This paper presents a high-speed real-time plane fitting implementation on a field-programmable gate array (FPGA) platform. A novel hardware-based least squares algorithm fits planes to patches of points within a depth image captured using a Microsoft Kinect v2 sensor. The validity of a plane fit and the plane parameters are reported for each patch of 11 by 11 depth pixels. The high level of parallelism of operations in the algorithm has allowed for a fast, low-latency hardware implementation on an FPGA that is capable of processing depth data at a rate of 480 frames per second. A hybrid hardware–software end-to-end system integrates the hardware solution with the Kinect v2 sensor via a computer and PCI express communication link to a Terasic TR4 FPGA development board. We have also implemented two proof-of-concept object detection applications as future candidates for bionic vision systems. We show that our complete end-to-end system is capable of running at 60 frames per second. An analysis and characterisation of the Kinect v2 sensor errors has been performed in order to specify logic precision requirements, statistical testing of the validity of a plane fit, and achievable plane fitting angle resolution.     

Low power data processing system with self-reconfigurable architecture

In this paper, a low power data processing system with a self-reconfigurable architecture and USB interface is presented. A single FPGA performs all processing and controls the multiple configurations without any additional elements, such as microprocessor, host computer or additional FPGAs. This architecture allows high performance with very low power consumption, a comprehensive alternative to microprocessor or DSP systems. In addition, a hierarchical reconfiguration system is used to support a large number of different processing tasks without the power consumption penalty of a big local configuration memory. Due to its simplicity and low power, this data processing system is especially suitable for portable applications, reducing the disadvantage of FPGAs against ASICS in low power consumption applications [A. Amara, F. Amiel, T. Ea, FPGA vs. ASIC for low power applications, Microelectronics Journal 37 (8) (2006) 669–677].