A CMOS Imager With a Programmable Bit-Serial Column-Parallel SIMD/MIMD Processor

An imager with an integrated fully programmable bit-serial column-parallel processor is proposed to meet the demand for a compact and versatile system-on-imager chip for consumer applications. The on-imager processor is targeting a computationally intensive low-level image processing task. The processor is physically arranged as a densely packed 2-D processing element (PE) array at an imager column level. The digital processor has a multiple-instruction-multiple-data (MIMD) architecture configuring multiple column-parallel single-instruction-multiple-data (SIMD) processors. The prototype imager chip with 128 times 128 pixels and 4 times 128 PE array designed with 0.6-mum technology was fabricated, and its functionality was tested. The estimation of performance level of the proposed processor architecture with an advanced technology such as the 0.09-mum process technology shows that the proposed imager chip architecture has a potential of giga sum operations per second per square millimeter class processing performance.     

A Programmable SIMD Vision Chip for Real-Time Vision Applications

A programmable vision chip for real-time vision applications is presented. The chip architecture is a combination of a SIMD processing element array and row-parallel processors, which can perform pixel-parallel and row-parallel operations at high speed. It implements the mathematical morphology method to carry out low-level and mid-level image processing and sends out image features for high-level image processing without I/O bottleneck. The chip can perform many algorithms through software control. The simulated maximum frequency of the vision chip is 300 MHz with 16 times 16 pixels resolution. It achieves the rate of 1000 frames per second in real-time vision. A prototype chip with a 16 times 16 PE array is fabricated by the 0.18 standard CMOS process. It has a pixel size of 30 mum times 40 mum and 8.72 mum W power consumption with a 1.8 V power supply. Experiments including the mathematical morphology method and target tracking application demonstrated that the chip is fully functional and can be applied in real-time vision applications.     

A twin-channel (p and n) FT-CCD imager with cross-antiblooming

A new class of Frame-Transfer CCD image sensors is presented, which is based on the use of both electrons and holes as information carriers and has a novel cross-antiblooming structure for overexposure protection. The device consists of alternate columns of p- and n-channel CCD's, which form two separately operating p and n imagers. This concept is based on the use of the n channel as a channel isolator for the p channel and vice versa and has five advantages. First, the complete area of the image section is active because no light-insensitive channel stop area is required. Secondly, both generated carriers electrons and holes can be stored and transported simultaneously. Thirdly, in a typical four-phase clocking system the electron pixels and the hole pixels are separated by half a pixel pitch in both the vertical and horizontal directions, which improves the pixel-packing density and aliasing suppression. Fourthly, the pattern also forms a line-quincunx sampling grid, which offers many advantages for signal processing, especially as the p- and n-output signals are simultaneously available. Finally, this pixel configuration is also ideally suitable for realizing a progressive-scan imager and a color imager     

Vertically integrated sensors for advanced imaging applications

A thin film on ASIC (TFA) image sensor is fabricated depositing an amorphous silicon thin-film detector onto a CMOS ASIC. With regards to advanced imaging systems, TFA provides enhanced performance and more flexibility than conventional technologies. Extensive on-chip signal processing is feasible, as well as small pixels for high resolution imagers. Two new TFA imager prototypes have recently been fabricated. High-resolution image sensor (HIRISE II) with 1024×128 pixels is an active pixel sensor suited for digital photography. Local autoadaptiver sensor (LARS II) with 368×256 pixels splits the illumination information into two signals, thereby providing a dynamic range of more than 120 dB, as required by automotive applications. Both prototypes include correlated double sampling and double delta sampling for efficient suppression of fixed pattern noise     

A Neuromorphic Cortical-Layer Microchip for Spike-Based Event Processing Vision Systems

We present a neuromorphic cortical-layer processing microchip for address event representation (AER) spike-based processing systems. The microchip computes 2-D convolutions of video information represented in AER format in real time. AER, as opposed to conventional frame-based video representation, describes visual information as a sequence of events or spikes in a way similar to biological brains. This format allows for fast information identification and processing, without waiting to process complete image frames. The neuromorphic cortical-layer processing microchip presented in this paper computes convolutions of programmable kernels over the AER visual input information flow. It not only computes convolutions but also allows for a programmable forgetting rate, which in turn allows for a bio-inspired coincidence detection processing. Kernels are programmable and can be of arbitrary shape and arbitrary size of up to 32 times 32 pixels. The convolution processor operates on a pixel array of size 32 times 32, but can process an input space of up to 128 times 128 pixels. Larger pixel arrays can be directly processed by tiling arrays of chips. The chip receives and generates data in AER format, which is asynchronous and digital. However, its internal operation is based on analog low-current circuit techniques. The paper describes the architecture of the chip and circuits used for the pixels, including calibration techniques to overcome mismatch. Extensive experimental results are provided, describing pixel operation and calibration, convolution processing with and without forgetting, and high-speed recognition experiments like discriminating rotating propellers of different shape rotating at speeds of up to 5000 revolutions per second     

CMOS image sensor with mixed-signal processor array

We present a single-chip integration of a CMOS image sensor with an embedded flexible processing array and dedicated analog-to-digital converter. The processor array is designed to perform convolution and transformation algorithms with arbitrary kernels. It has been designed to carry out the multiplication of analog image data with given digital kernel coefficients and to add up the results. The processor array is an analog implementation of a highly parallel architecture which is scalable to any desired sensor resolution while preserving video-rate operation. A prototype implementation has been realized in a 0.6-/spl mu/m CMOS technology. Switched current technique has been applied to obtain compact and robust circuits. The prototype's sensor resolution is 64 /spl times/ 128 pixels. The processor array occupies a small chip area and consumes only a small percentage of the power (250 /spl mu/W) of the whole image sensor.     

An abuttable CCD imager for visible and X-ray focal plane arrays

A frame-transfer silicon charge-coupled-device (CCD) imager has been developed that can be closely abutted to other imagers on three sides of the imaging array. It is intended for use in multichip arrays. The device has 420×420 pixels in the imaging and frame-store regions and is constructed using a three-phase triple-polysilicon process. Particular emphasis has been placed on achieving low-noise charge detection for low-light-level imaging in the visible and maximum energy resolution for X-ray spectroscopic applications. Noise levels of 6 electrons at 1-MHz and less than 3 electrons at 100-kHz data rates have been achieved. Imagers have been fabricated on 1000-Ω cm material to maximize quantum efficiency and minimize split events in the soft X-ray regime     

Focal-Plane Spatially Oversampling CMOS Image Compression Sensor

Image compression algorithms employ computationally expensive spatial convolutional transforms. The CMOS image sensor performs spatially compressing image quantization on the focal plane yielding digital output at a rate proportional to the mere information rate of the video. A bank of column-parallel first-order incremental DeltaSigma-modulated analog-to-digital converters (ADCs) performs column-wise distributed focal-plane oversampling of up to eight adjacent pixels and concurrent weighted average quantization. Number of samples per pixel and switched-capacitor sampling sequence order set the amplitude and sign of the pixel coefficient, respectively. A simple digital delay and adder loop performs spatial accumulation over up to eight adjacent ADC outputs during readout. This amounts to computing a two-dimensional block matrix transform with up to 8times8-pixel programmable kernel in parallel for all columns. Noise shaping reduces power dissipation below that of a conventional digital imager while the need for a peripheral DSP is eliminated. A 128times128 active pixel array integrated with a bank of 128 DeltaSigma-modulated ADCs was fabricated in a 0.35-mum CMOS technology. The 3.1 mm times 1.9-mm prototype captures 8-bit digital video at 30 frames/s and yields 4 GMACS projected computational throughput when scaled to HDTV 1080i resolution in discrete cosine transform (DCT) compression     

Time multiplexed color image processing based on a CNN withcell-state outputs

A practical system approach for time-multiplexing cellular neural network (CNN) implementations suitable for processing large and complex images using small CNN arrays is presented. For real size applications, due to hardware limitations, it is impossible to have a one-on-one mapping between the CNN hardware cells and all the pixels in the image involved. This paper presents a practical solution by processing the input image, block by block, with the number of pixels in a block being the same as the number of CNN cells in the array. Furthermore, unlike other implementations in which the output is observed at the hard-limiting block, the very large scale integrated (VLSI) architecture hereby described monitors the outputs from the state node. While previous implementations are mostly suitable for black and white applications because of the thresholded outputs, our approach is especially suitable for applications in color (gray) image processing due to the analog nature of the state node. Experimental complementary metal-oxide-semiconductor (CMOS) chip results in good agreement with theoretical results are presented     

A field programmable analog array for CMOS continuous-time OTA-Cfilter applications

A programmable high-frequency operational transconductance amplifier (OTA) is proposed and analyzed. A general configurable analog block (CAB) is presented, which consists of the proposed programmable OTA, programmable capacitor and MOSFET switches. Using the CABs, the universal tunable and field programmable analog array (FPAA) can be constructed, which can realize many signal-processing functions, including filters. A tuning circuit is also discussed. The proposed OTA has been simulated and fabricated in CMOS technology. The results show that the OTA has the transconductance tunable/programmable in a wide range of 700 times and the -3-dB bandwidth larger than 20 MHz. A universal 5×8 CAB array has been fabricated. The chip has also been configured to realize OTA-C 60-kHz and 500-kHz bandpass filters based on ladder simulation and biquad cascade     

A CCD color signal separation IC for single-chip color imagers

A CCD color signal separation IC for solid-state imagers with color filter arrays is described. The device simplifies peripheral circuitry and enhances picture qualities such as resolution, color fidelity, and stability for single-chip color imaging systems, by incorporating CCD delay lines, sample-and-hold circuits, and dual clamp circuit. Also described is a new geometry color filter array which is required by the separation IC. Color image corresponding to horizontal resolution of 340 TV lines with no erroneous color has been obtained from the device when used with a 580 × 475 element CCD imager.     

可编程细胞神经网络硬件实现及应用研究

 本文提出一种模板可编程细胞神经网络的硬件实现方法,设计构成CNN的细胞体电路、A模板电路和B模板电路,组成CNN并进行在图像处理中的应用研究.仿真结果表明,所设计的硬件电路具有结构简单、功耗低、频率特性好、模板参数可编程等特点,可以方便地构成各种规模的CNN,在图像处理应用中具有一定的灵活性和通用性.     

A 1 K×1 K high dynamic range CMOS image sensor with on-chipprogrammable region-of-interest readout

An integrated 1024×1024 CMOS image sensor with programmable region-of-interest (ROI) readout and multiexposure technique has been developed and successfully tested. Size and position of the ROI is programmed based on multiples of a minimum readout kernel of 32×32 pixels. Since the dynamic range of the irradiance normally exceeds the electrical dynamic range of the imager that can be covered using a single integration time, a multiexposure technique has been implemented in the imager. Subsequent sensor images are acquired using different integration times and recomputed to form a single composite image. A newly developed algorithm performing the recomputation is presented. The chip has been realized in a 0.5-μm n-well standard CMOS process. The pixel pitch is 10 μm² and the total chip area is 164 mm ²     

A programmable focal-plane MIMD image processor chip

An 80×78 pixels vision chip for focal-plane image processing is presented. The chip employs a Multiple-Instruction-Multiple-Data (MIMD) architecture to provide five spatially processed images in parallel. The size, configuration, and coefficients of the spatial kernels are programmable. The chip's architecture allows the photoreceptor cells to be small and parked densely by performing all computations on the read-out, away from the array. The processing core uses digitally programmed current-mode analog computation. Operating at 9.6 K frames/s in 800-lux ambient light, the chip consumes 4 mW from a 2.5-V power supply. Performing 11×11 spatial convolutions, an equivalent computation (5.5 bit scale-accumulate) rate of 12.4 GOPS/mW is achieved using 22 mm² in a 1.2-μm CMOS process. The application of the chip to line-segment orientation detection is also presented     

星载遥感短波红外高速高光谱成像仪坏像元识别

为改善干涉成像短波红外高速高光谱成像仪的坏像元对复原光谱的影响,利用高光谱成像仪测试流程建立了坏像元识别模板,以提高坏像元识别效率。首先,按照高光谱成像仪测试流程设置增益模板和帧频模板并采集图像数据,依据正常像元增益响应设定合理判定阈值Th₁,识别不同增益下异常像元并记录对应坐标值;再依据正常像元帧频响应灰度值设定合理判定阈值Th₂,识别不同帧频下异常像元并记录坐标值。最后,对比增益模板和帧频模板判定的异常像元,融合确定坏像元。实验结果表明基于增益模板和帧频模板的识别方法在不增加设备研制测试成本的同时有效识别出短波红外高光谱成像仪探测器的坏像元,为可靠识别短波红外高光谱成像仪坏像元提供了一种低成本、高效可靠的新方法,提高了干涉成像高光谱成像仪光谱反演准确性。     

Fast, Accurate and Shift-Varying Line Projections for Iterative Reconstruction Using the GPU

List-mode processing provides an efficient way to deal with sparse projections in iterative image reconstruction for emission tomography. An issue often reported is the tremendous amount of computation required by such algorithm. Each recorded event requires several back- and forward line projections. We investigated the use of the programmable graphics processing unit (GPU) to accelerate the line-projection operations and implement fully-3D list-mode ordered-subsets expectation-maximization for positron emission tomography (PET). We designed a reconstruction approach that incorporates resolution kernels, which model the spatially-varying physical processes associated with photon emission, transport and detection. Our development is particularly suitable for applications where the projection data is sparse, such as high-resolution, dynamic, and time-of-flight PET reconstruction. The GPU approach runs more than 50 times faster than an equivalent CPU implementation while image quality and accuracy are virtually identical. This paper describes in details how the GPU can be used to accelerate the line projection operations, even when the lines-of-response have arbitrary endpoint locations and shift-varying resolution kernels are used. A quantitative evaluation is included to validate the correctness of this new approach.      

A Modular Multi-Chip Neuromorphic Architecture for Real-Time Visual Motion Processing

The extent of pixel-parallel focal plane image processing is limited by pixel area and imager fill factor. In this paper, we describe a novel multi-chip neuromorphic VLSI visual motion processing system which combines analog circuitry with an asynchronous digital interchip communications protocol to allow more complex pixel-parallel motion processing than is possible in the focal plane. This multi-chip system retains the primary advantages of focal plane neuromorphic image processors: low-power consumption, continuous-time operation, and small size. The two basic VLSI building blocks are a photosensitive sender chip which incorporates a 2D imager array and transmits the position of moving spatial edges, and a receiver chip which computes a 2D optical flow vector field from the edge information. The elementary two-chip motion processing system consisting of a single sender and receiver is first characterized. Subsequently, two three-chip motion processing systems are described. The first three-chip system uses two sender chips to compute the presence of motion only at a particular stereoscopic depth from the imagers. The second three-chip system uses two receivers to simultaneously compute a linear and polar topographic mapping of the image plane, resulting in information about image translation, rotation, and expansion. These three-chip systems demonstrate the modularity and flexibility of the multi-chip neuromorphic approach.     

A CCD color signal separation IC for single-chip color imagers

A CCD color signal separation IC for solid-state imagers with color filter arrays is described. The device simplifies peripheral circuitry and enhances picture qualities such as resolution, color fidelity, and stability for single-chip color imaging systems by incorporating CCD delay lines, sample-and-hold circuits, and dual clamp circuit. Also described is a color filter array utilizing Bayer geometry, which is used in the separation IC. Color images corresponding to a horizontal resolution of 340 TV lines with no erroneous color have been obtained from the device when used with a 580/spl times/475 element CCD imager.     

A multiresolution 100-GOPS 4-Gpixels/s programmable smart vision sensor for multisense imaging

This paper presents a multiresolution general-purpose high-speed machine vision sensor with on-chip image processing capabilities. The sensor comprises an innovative multiresolution sensing area, 1536 A/D converters, and a SIMD array of 1536 bit-serial processors with corresponding memory. The sensing area consists of an area part with 1536 /spl times/ 512 pixels, and a line-scan part with a set of rows with 3072 pixels each. The SIMD processor array can deliver more than 100 GOPS sustained and the on-chip pixel-analysing rate can be as high as 4Gpixels/s. The sensor is ideal for high-speed multisense imaging where, e.g., color, greyscale, internal material light scatter, and 3-D profiles are captured simultaneously. When running only 3-D laser triangulation, a data rate of more than 20 000 profiles/s can be achieved when delivering 1536 range values per profile with 8 bits of range resolution. Experimental results showing very good image characteristics and a good digital to analog noise isolation are presented.     

A log-polar image sensor fabricated in a standard 1.2-μm ASICCMOS process

A CMOS log-polar or foveated image sensor for use in mobile robotic and machine vision applications has been designed, fabricated, and tested. The sensor benefits from a high degree of integration, minimal power consumption, and ease of manufacture due to the use of a standard 1.2 μm ASIC CMOS process. The sensor is composed of two distinct CMOS imager arrays which together solve the problem of obtaining good image resolution over a wide field of view. With resolution sensing is accomplished with a 40×40 array of individual pixels each measuring 9.6 μm on a side. A wide field of view is provided by an array of 64×16 pixels arranged on a log-polar grid. The maximum measured dynamic range for the fabricated log-polar array is 46 dB, while the lowest observed fixed-pattern noise is 0.5% of saturation. Combined power consumption of both arrays is under 2 mW when operating from a single 5-V supply at a frame rate of 30 frames/s