脉冲视觉研究进展

视频是视觉信息处理的基础概念,传统视频的帧率只有几十Hz,不能记录光的高速变化过程,成为限制机器视觉速度的天花板,其根本原因在于视频概念脱胎于胶片成像,未能发挥电子和数字技术的潜力。脉冲视觉模型通过感光器件捕获光子,累积能量达到约定阈值时产生脉冲,形成脉冲的时间越长,表明收到的光信号越弱,反之光信号越强,据此可估计任意时刻的光强,从而实现连续成像。采用普通器件,研制了比影视视频快千倍的超高速成像芯片和相机,进而基于脉冲神经网络实现了超高速目标检测、跟踪和识别,打破了机器视觉提速依赖算力线性增长的传统范式。本文从脉冲视觉模型表达视觉信息的生物学基础和物理原理出发,介绍了脉冲视觉原理的软件模拟器及其模拟真实世界光子传播的计算过程,描述了基于脉冲视觉原理的高灵敏光电传感器件及芯片的工作机理和结构设计、基于脉冲视觉的影像重建原理以及脉冲视觉信号与普通图像信号融合的计算摄像算法与计算摄像系统,介绍了基于脉冲神经网络的超高速运动目标检测、跟踪与识别,通过对比国际国内相关研究内容和发展现状,展望了脉冲视觉的发展与演进方向。脉冲视觉芯片和系统在工业(高铁、电力和轮机等不停机监测,智能制造高速监视等)、民用(高速相机、智能交通、辅助驾驶、司法取证和体育判罚等)以及国防(高速对抗)等领域都具有巨大应用潜力,是未来值得重点关注和研究的一个重要方向。

Advances in spike vision

Video is the conceptual base of visual information processing technology. The frame rate of traditional video is tens of Hertz, which is incapable to represent the Ultra-high speed change process of light. It also constrains to the speed of machine vision. The concept of video is originated from film imaging, which can''t unleash the full potential of electronic and digital technology. The spike vision model generates a spike when the photon energy captured by a photosensitive device reaches the predefined threshold. The longer the spike timing is, the weaker the received light signals are. The occurred light intensity can be inferred to achieve consistent imaging. Current research of spiking vision is focused on developing disruptive ultra-high speed imaging chips and cameras that are 1 000 times faster than videos. It is substituting traditional paradigm derived from machine vision speed linear growth of computing power. Based on the biological evidence and physical principle of spiking vision, our review analyzes the software simulator of spiking vision, the computational process of the photon propagation simulation, spiking vision based visual image reconstruction, and the detailed mechanism of photoelectric sensor structure and the chip design. Our review reveals the ultra-high speed target motion detection, and spiking neural networks based tracking and recognition. The evolution of spiking vision is forecasted. Future spiking vision based chip and system has multifaceted potentials to further harness the industrial tasks (e.g., the monitoring of high-speed train, power grid, turbine and intelligent manufacturing), civil applications (e.g.,high-speed camera, intelligent transportation, auxiliary driving, judicial forensics and sports arbitration), and national defense construction (e.g., high-speed confrontation campaigns).