A 10 000 fps CMOS Sensor With Massively Parallel Image Processing

A high-speed analog VLSI image acquisition and pre-processing system has been designed and fabricated in a 0.35 mum standard CMOS process. The chip features a massively parallel architecture enabling the computation of programmable low-level image processing in each pixel. Extraction of spatial gradients and convolutions such as Sobel or Laplacian filters are implemented on the circuit. For this purpose, each 35 mum times 35 mum pixel includes a photodiode, an amplifier, two storage capacitors, and an analog arithmetic unit based on a four-quadrant multiplier architecture. The retina provides address-event coded output on three asynchronous buses: one output dedicated to the gradient and the other two to the pixel values. A 64 times 64 pixel proof-of-concept chip was fabricated. A dedicated embedded platform including FPGA and ADCs has also been designed to evaluate the vision chip. Measured results show that the proposed sensor successfully captures raw images up to 10 000 frames per second and runs low-level image processing at a frame rate of 2000 to 5000 frames per second.     

A Real-Time Image-Feature-Extraction and Vector-Generation VLSI Employing Arrayed-Shift-Register Architecture

A feature-extraction and vector-generation VLSI has been developed for real-time image recognition. An arrayed-shift-register architecture has been employed in conjunction with a pipelined directional-edge-filtering circuitry. As a result, it has become possible to scan an image, pixel by pixel, with a 64 x 64-pixel recognition window and generate a 64-dimensional feature vector in every 64 clock cycles. In order to determine the threshold for edge-filtering operation adaptive to local luminance variation, a high-speed median circuit has been developed. A binary median search algorithm has been implemented using high-precision majority voting circuits working in the mixed-signal principle. A prototype chip was designed and fabricated in a 0.18-mum 5-metal CMOS technology. A high-speed feature vector generation in less than 9.7 ns/vector element has been experimentally demonstrated. It is possible to scan a VGA-size image at a rate of 6.1 frames/s, thus generating as many as 1.5 x 10⁶ feature vectors per second for recognition. This is more than 10³ times faster than software processing running on a 3-GHz general-purpose processor.     

A Programmable SIMD Vision Chip for Real-Time Vision Applications

A programmable vision chip for real-time vision applications is presented. The chip architecture is a combination of a SIMD processing element array and row-parallel processors, which can perform pixel-parallel and row-parallel operations at high speed. It implements the mathematical morphology method to carry out low-level and mid-level image processing and sends out image features for high-level image processing without I/O bottleneck. The chip can perform many algorithms through software control. The simulated maximum frequency of the vision chip is 300 MHz with 16 times 16 pixels resolution. It achieves the rate of 1000 frames per second in real-time vision. A prototype chip with a 16 times 16 PE array is fabricated by the 0.18 standard CMOS process. It has a pixel size of 30 mum times 40 mum and 8.72 mum W power consumption with a 1.8 V power supply. Experiments including the mathematical morphology method and target tracking application demonstrated that the chip is fully functional and can be applied in real-time vision applications.     

Adaptive-integration-time image sensor with real-time reconstruction function

A computational image sensor is proposed in which the pixel controls its integration time to light intensity. The integration time of each pixel is selected from among several lengths of integration time and the integration time is shortened if the pixel intensity becomes saturated. Although the integration time of each pixel varies, the pixel intensity is adjusted on the sensor in real time. The dynamic range of the pixel value output from the proposed sensor is greatly widened. A prototype of 64/spl times/48 pixels has been fabricated by using 2-poly 2-metal 0.8-/spl mu/m CMOS process. The proposed sensor has simple functions for the comparison of intermediate integration value and threshold to control the integration time and nonlinear image reconstruction. Because the maximum number of the comparison-reset operations during a frame is three, one of the four integration times can be selected pixel by pixel. The circuit and layout design of the prototype which has computational elements based on column parallel architecture are described and the fundamental functions have been verified. By the experiments, it has been verified that the sensor can achieve a wide dynamic range by adapting to light.     

A 64 $times$ 64-Pixel CMOS Test Chip for the Development of Large-Format Ultra-High-Speed Snapshot Imagers

A 64 $times$ 64-pixel test circuit was designed and fabricated in 0.18-$mu{hbox {m}}$ CMOS technology for investigating high-speed imaging with large-format imagers. Several features are integrated into the circuit architecture to achieve fast exposure times with low-skew and jitter for simultaneous pixel snapshots. These features include an H-tree clock distribution with local and global repeaters, single-edge trigger propagation, local exposure control, and current-steering sampling circuits. To evaluate the circuit performance, test structures are periodically located throughout the 64 $times$ 64-pixel device. Measured devices have exposure times that can be varied between 75 ps to 305 ps with skew times for all pixels less than $pm$ 3 ps and jitter that is less than $pm$1.2 ps rms. Other performance characteristics are a readout noise of approximately 115 e- rms and an upper dynamic range of 310,000 e-.      

In-pixel autoexposure CMOS APS

A CMOS active pixel sensor (APS) with in-pixel autoexposure and a wide dynamic-range linear output is described. The chip features a unique architecture enabling a customized number of additional bits per pixel per readout, with minimal effect on the sensor spatial or temporal resolution. By utilizing multiple readouts via real-time feedback, each pixel in the field of view can automatically set an independent exposure time, according to its illumination. A customized, large increase in the dynamic range can be achieved and a scene containing both bright and dark regions can be captured. A prototype of 64 /spl times/ 64 pixels has been fabricated using 1-poly 3-metal CMOS 0.5 /spl mu/m n-well process available through MOSIS. Power dissipation is 3.7 mW at V/sub DD/ = 5 V. The special functions have been verified experimentally, and an increase of 2 bits over the inherent dynamic range captured is shown.     

Arbitrated Time-to-First Spike CMOS Image Sensor With On-Chip Histogram Equalization

This paper presents a time-to-first spike (TFS) and address event representation (AER)-based CMOS vision sensor performing image capture and on-chip histogram equalization (HE). The pixel values are read-out using an asynchronous handshaking type of read-out, while the HE processing is carried out using simple and yet robust digital timer occupying a very small silicon area (0.1times0.6 mm²). Low-power operation (10 nA per pixel) is achieved since the pixels are only allowed to switch once per frame. Once the pixel is acknowledged, it is granted access to the bus and then forced into a stand-by mode until the next frame cycle starts again. Timing errors inherent in AER-type of imagers are reduced using a number of novel techniques such as fair and fast arbitration using toggled priority (TP), higher-radix, and pipelined arbitration. A verilog simulator was developed in order to simulate the effect of timing errors encountered in AER-based imagers. A prototype chip was implemented in AMIS 0.35 mum process with a silicon area of 3.1times3.2 mm². Successful operation of the prototype is illustrated through experimental measurements     

A 74.8 mW Soft-Output Detector IC for 8 8 Spatial-Multiplexing MIMO Communications

In this paper, VLSI implementation of a configurable, soft-output MIMO detector is presented. The proposed chip can support up to 8 $, times ,$8 64-QAM spatial multiplexing MIMO communications, which surpasses all reported MIMO detector ICs in antenna number and modulation order. Moreover, this chip provides configurable antenna number from 2$,times,$ 2 up to 8$,times,$ 8 and modulation order from QPSK to 64-QAM. Its outputs include bit-wise log likelihood ratios (LLRs) and a candidate list, making it compatible with powerful soft-input channel decoders and iterative decoding system. The MIMO detector adopts a novel sphere decoding algorithm with high decoding efficiency and superior error rate performance, called modified best-first with fast descent (MBF-FD). Moreover, a low-power pipelined quad-dual-heap (quad-DEAP) circuit for efficient node pool management and several circuit techniques are implemented in this chip. When this chip is configured as 4$, times ,$4 64-QAM and 8$, times ,$ 8 64-QAM soft-output MIMO detectors, it achieves average throughputs of 431.8 Mbps and 428.8 Mbps with only 58.2 mW and 74.8 mW respective power consumption and reaches 10$^{-5}$ coded bit error rate (BER) at signal-to-noise ratio (SNR) of 24.2 dB and 22.6 dB, respectively.      

A DPS array with programmable resolution and reconfigurable conversion time

A CMOS digital pixel sensor (DPS) with programmable resolution and reconfigurable conversion time is described. The chip features a unique architecture based on the pulse width modulation (PWM) technique and operates with either an 8-b or 4-b accuracy. The 8-b conversion mode is used for high-precision imaging while the 4-b conversion mode provides a shorter conversion time and a two times increase in spatial resolution. Two quantization schemes are studied, namely, the uniform and the nonuniform time-domain quantizers, which are referred to as UQ and NUQ, respectively. It is shown that the latter scheme not only permits to linearize the nonlinear response of the PWM sensor, but also allows to significantly speed up the conversion time, particularly for wide dynamic range and low coding resolutions. A prototype of 32/spl times/32/64/spl times/32 pixels has been fabricated using 1-poly, 5-metal CMOS 0.35-/spl mu/m n-well standard process. Power dissipation is 10 mW at V/sub DD/=3.3 V, dynamic range is 90 dB, while dark current was measured at 1 pA. The reconfiguration features of the chip have been verified experimentally.     

A Computational Digital Pixel Sensor Featuring Block-Readout Architecture for On-Chip Image Processing

In this paper, a computational digital pixel sensor (DPS) equipped with an on-chip image-processing capability has been developed. In order to resolve the interconnection bottleneck between the sensor array and on-chip processing units, a new block-readout architecture has been proposed and implemented on the chip. The data from the sensor array are read out in a form of a pixel block compatible to kernel image processing, and they are processed in parallel by on-chip processing units. Such an architecture has enabled us to carry out an efficient kernel processing using a linear array of single-instruction–multiple-data processing units. In order to demonstrate the advantage of such an architecture, a rank-order filtering circuit has been implemented on the chip as a case study of the on-chip image processing. In this paper, a binary-search rank-order filtering algorithm has been implemented in a simple circuitry. A proof-of-concept chip having an array of 64$times$48 pixels was designed and fabricated using a 0.35-${rm mu}hbox{m}$ CMOS technology, and the concept has been verified by the measurement of fabricated chips.      

The Retinomorphic Approach: Pixel-Parallel Adaptive Amplification, Filtering, and Quantization

I describe a vision system that uses neurobiologicalprinciples to perform all four major operations found in biologicalretinae: (1) continuous sensing for detection, (2) local automaticgain control for amplification, (3) spatiotemporal bandpass filteringfor preprocessing, and (4) adaptive sampling for quantization.All four operations are performed at the pixel level. The systemincludes a random-access time-division multiplexed communicationchannel that reads out asynchronous pulse trains from a 64×64 pixel array in the imager chip, and transmitsthem to corresponding locations on a second chip that has a 64×64 array of integrators. Both chips are fully functional.I compare and contrast the design principles of the retina withthe standard practice in imager design and analyze the circuitsused to amplify, filter, and quantize the visual signal, withemphasis on the performance trade-offs inherent in the circuittopologies used.     

CMOS active pixel sensor star tracker with regional electronicshutter

A 64×64 element CMOS active pixel sensor (APS) for star tracker applications is reported. The chip features an innovative regional electronic shutter through the use of an individual pixel reset architecture. Using the regional electronic shutter, each star in the field of view can have its own integration period. This way, simultaneous capture of bright stars with dim stars is accommodated, enabling a large increase in tracker capability. The chip achieves 80 dB dynamic range, 50 e-rms read noise, low dark current, and excellent electronic shutter linearity     

A general-purpose processor-per-pixel analog SIMD vision chip

A smart-sensor VLSI circuit suitable for focal-plane low-level image processing applications is presented. The architecture of the device is based on a fine-grain software-programmable SIMD processor array. Processing elements, integrated within each pixel of the imager, are implemented utilising a switched-current analog microprocessor concept. This allows the achievement of real-time image processing speeds with high efficiency in terms of silicon area and power dissipation. The prototype 21 /spl times/ 21 vision chip is fabricated in a 0.6 /spl mu/m CMOS technology and achieves a cell size of 98.6 /spl mu/m /spl times/ 98.6 /spl mu/m. It executes over 1.1 giga instructions per second (GIPS) while dissipating under 40 mW of power. The architecture, circuit design and experimental results are presented in this paper.     

用于实时目标检测的高速可编程视觉芯片

视觉芯片是一种高速、低功耗的智能视觉处理系统芯片,在生产生活中有广阔的应用前景。文中提出了一种新型的可编程视觉芯片架构,该架构的设计考虑了传统计算机视觉算法和卷积神经网络的运算特点,使其能够同时高效地支持这两类算法。该视觉芯片集成了可编程的多层次并行处理阵列、高速数据传输通路和系统控制模块,并采用65 nm标准CMOS工艺制程流片。测试结果表明:视觉芯片在200 MHz系统时钟下达到413GOPS的峰值运算性能,能够高效地完成包括完成人脸识别、目标检测等多种计算机视觉和人工智能算法。该视觉芯片在可编程度、运算性能以及能耗效率等方面都大大超越了其他视觉芯片。     

High-Performance InP-Based Photodetector in an Amplifier Layer Stack on Semi-Insulating Substrate

A waveguide photodetector (PD) based on semi-insulating (SI) indium phosphide (InP) was simulated, designed, and fabricated. The layer stack for this PD was optimized for use as an optical amplifier or laser and it can be combined with the passive components. By using an SI substrate and deep etching, a small, efficient, and high-speed PD was made, which allows for easy integration of source, detector, and passive optical components on a single chip. A 3-dB bandwidth of 35 GHz and 0.25 A/W external radio-frequency reponsivity is measured at 1.55-$mu$ m wavelength for a 1.5-$mu$ m-wide and 30-$mu$ m-long waveguide PD at $-$ 4-V bias voltage. The polarization dependence in the responsivity is less than 0.27 dB.      

Multiple integration method for a high signal-to-noise ratio readout integrated circuit

A multiple integration method is reported that greatly improves the signal-to-noise ratio (SNR) for applications with a high-resolution infrared (IR) focal plane array. The signal from each pixel is repeatedly sampled into an integration capacitor and then output and summed into an outside memory that continues for n read cycles during each period of a frame. This method increases the effective capacity of the charge integration and improves sensitivity. Because a low-noise function block and high-speed operation of the readout circuit is required, a new concept is proposed that enables the readout circuit to perform digitization by a voltage skimming method. The readout circuit was fabricated using a 0.6-/spl mu/m CMOS process for a 64/spl times/64 midwavelength IR HgCdTe detector array. The readout circuit effectively increases the charge storage capacity to 2.4/spl times/10/sup 8/ electrons and then provides a greatly improved SNR by a factor of approximately 3.     

Computational image sensor for on sensor compression

We propose a novel integration of image compression and sensing in order to enhance the performance of an image sensor. By integrating a compression function onto the sensor focal plane, the image signal to be read out from the sensor is significantly reduced and the pixel rate of the sensor ran consequently be increased. The potential applications of the proposed sensor are in high pixel-rate imaging, such as high frame-rate image sensing and high-resolution image sensing. The compression scheme we employ is a conditional replenishment, which detects and encodes moving areas. In this paper, we introduce two architectures for on-sensor compression; one is the pixel parallel approach and the other is the column parallel approach. We prototyped a VLSI chip of the proposed sensor based on the pixel parallel architecture. We show the design and describe the results of the experiments obtained by the prototype chip     

13-ns, 500-mW, 64-kbit ECL RAM using Hi-BiCMOS technology

The development is discussed for a 13-ns, 500-mW, 16K word/spl times/4-bit emitter-coupled logic (ECL) RAM using high-performance bipolar CMOS (Hi-BiCMOS) technology that combines a bipolar and a CMOS device on the same chip. The power dissipation of the RAM is about one half that of the conventional 64-kb bipolar ECL RAM. This high-speed, low-power RAM has been realized through a concept of a MOS-type memory cell, bipolar circuits, and a CMOS combination gate to allow for increased LSI integration.     

Fully boosted 64K dynamic RAM with automatic and self-refresh

A novel high-speed low-power 64K dynamic RAM with enough margin has been attained using a double polysilicon and 3-/spl mu/m process technologies. To obtain a low soft error rate below 1/spl times/10/SUP -6/ errors per device hour without sacrificing the high-speed and low-power operation, some novel approaches are proposed in the circuit and device designs. In particular, fully boosted circuits and the Hi-C cell structure with polysilicon bit line are designed to increase the margin of the single 5-V power supply 64K dynamic RAM. The fabricated device provides a typical access time of 90 ns and an operating power of 190 mW at 25/spl deg/C. Also, the design features of the automatic and self-refresh functions on the same chip are described.     

An APS with 2-D winner-take-all selection employing adaptive spatial filtering and false alarm reduction

An active pixel sensor (APS) with two-dimensional winner-take-all (WTA) detection is presented. This system-on-a-chip employs adaptive spatial filtering of the processed image, with bad pixel elimination and false alarm reduction in case of a missing object. The circuit has a unique ability of adaptive spatial filtering that allows removal of the background from the image, one stage before it is transferred to the WTA detection circuit. A test chip of a 64/sup */64 array has been implemented in 0.5-/spl mu/m CMOS technology, has a 49% fill factor, is operated by a 3.3-V supply, and dissipates 36 mW at video rate. System architecture and operation are discussed and measurements from the prototype chip are presented.