CuInP2S6-Based Electronic/Optoelectronic Synapse for Artificial Visual System Application

With the development of information processing, various neuromorphic synaptic devices have been proposed, including novel devices mimicking multiple sensory systems in the biosome, in which vision is a vital source of information. Due to the pressing issue of high energy consumption and the ever-increasing complexity of practical application scenarios, there is an urgent need to investigate optoelectronic synapses with simple structure but multifunctional capabilities, thereby broadening their application scope. Here, remarkable performances in both electrical and optical operation modes are achieved in multilayer graphene/CuInP₂S₆/Au electronic/optoelectronic device. By modulating the electrical and optical pulses, both short-term and long-term memory can be emulated in the same device, while visual perception, processing, and memorizing functions are demonstrated in this single cell with relatively low energy consumption. In addition, light adaptive behavior can also be simulated through optical–electrical cooperative modulation in the device, further providing a novel and promising strategy for future applications in artificial visual systems.     

Broadband Optoelectronic Synapse Enables Compact Monolithic Neuromorphic Machine Vision for Information Processing

Traditional machine vision is suffering from redundant sensing data, bulky structures, and high energy consumption. Biological-inspired neuromorphic systems are promising for compact and energy-efficient machine vision. Multifunctional optoelectronics enabling multispectrum sensitivity for broadband image sensing, feature extraction, and neuromorphic computing are vital for machine visions. Here, an optoelectronic synapse is designed that enables image sensing, convolutional processing, and computing. Multiple synaptic plasticity triggered by photons can implement photonic computing and information transmission. Convolutional processing is realized by ultralow energy kernel generators fully controlled by photons. Meanwhile, the device shows the ability of conductance modulations under electronic stimulations that implement neuromorphic computing. For the first time, this two-terminal broadband optoelectronic synapse enables front-end retinomorphic image sensing, convolutional processing, and back-end neuromorphic computing. The integrated photonic information encryption, convolutional image preprocessing, and neuromorphic computing capabilities are promising for compact monolithic neuromorphic machine vision systems.     

Bidirectionally Photoresponsive Optoelectronic Transistors with Dual Photogates for All-Optical-Configured Neuromorphic Vision

Bio-inspired neuromorphic vision sensors, integrating optical sensing, and processing functions have attracted significant attention for developing future low-power and high-efficiency imaging systems. However, the compulsory electrical signal modulation to achieve inhibitory behaviors in most reported neuromorphic vision sensors results in additional hardware and computational latency. Herein, bidirectional photoresponsive optoelectronic synapses based on In₂O₃/Al₂O₃/Y6 phototransistors are achieved, realizing all-optical-configured synaptic weight updates enabled by dual photogates. The inhibitory and excitatory photoresponses originate from the photogating effects provided by trapped photogenerated electrons in Al₂O₃ under near-infrared light and the ionized oxygen vacancies in In₂O₃ under ultra-violet light, respectively. The bidirectional phototransistor illustrates outstanding optoelectronic synaptic characteristics with low nonlinearity and asymmetry, demonstrating high efficiencies in both preprocessing and postprocessing tasks, such as noise reduction, contrast enhancement, and pattern recognition. The proposed dual-photogate optoelectronic synapses provide effective strategies to construct high-efficiency neuromorphic vision sensors and in-sensor computing systems.     

All-Optically Controlled Memristor for Optoelectronic Neuromorphic Computing

Neuromorphic computing (NC) is a new generation of artificial intelligence. Memristors are promising candidates for NC owing to the feasibility of their ultrahigh-density 3D integration and their ultralow energy consumption. Compared to traditional electrical memristors, the emerging optoelectronic memristors are more attractive owing to their ability to combine the advantages of both photonics and electronics. However, the inability to reversibly tune the memconductance with light has severely restricted the development of optoelectronic NC. Here, an all-optically controlled (AOC) analog memristor is realized, with memconductance that is reversibly tunable over a continuous range by varying only the wavelength of the controlling light. The device is based on the relatively mature semiconductor material InGaZnO and a memconductance tuning mechanism of light-induced electron trapping and detrapping. It is found that the light-induced multiple memconductance states are nonvolatile. Furthermore, spike-timing-dependent plasticity learning can be mimicked in this AOC memristor, indicating its potential applications in AOC spiking neural networks for highly efficient optoelectronic NC.     

基于铁电材料的有机光电突触晶体管研究

模拟生物突触结构的设备是实现神经网络计算的可行方案之一,其中人工视网膜器件为机器视觉和图像识别的实现提供了有力支持。通过旋涂制备聚偏氟乙烯-三氟乙烯(P(VDF-TrFE))制备铁电栅层,热蒸发酞菁铜(CuPc)作为半导体层,探究该晶体管模拟突触功能的光电响应。实验结果表明,该光电晶体管在625 nm具有显著的光响应,其能够产生兴奋性突触后电流(EPSC)并实现短期可塑性到长期可塑性的转变以及高通滤波功能。利用剩余极化强度模拟了大脑学习过程中提前施加注意的行为。此外,以栅电压和光照作为独立输入逻辑信号,在单个晶体管中实现了“与”和“或”的布尔逻辑功能。上述结果表明,CuPc可以与铁电材料进行良好结合并制备出具有突触响应特点的光电晶体管,这为人工视网膜器件的开发提供了有机铁电器件的参考。     

Advanced Optoelectronic Devices for Neuromorphic Analog Based on Low-Dimensional Semiconductors

Neuromorphic systems can parallelize the perception and computation of information, making it possible to break through the von Neumann bottleneck. Neuromorphic engineering has been developed over a long period of time based on Hebbian learning rules. The optoelectronic neuromorphic analog device combines the advantages of electricity and optics, and can simulate the biological visual system, which has a very strong development potential. Low-dimensional materials play a very important role in the field of optoelectronic neuromorphic devices due to their flexible bandgap tuning mechanism and strong light-matter coupling efficiency. This review introduces the basic synaptic plasticity of neuromorphic devices. According to the different number of terminals, two-terminal neuromorphic memristors, three-terminal neuromorphic transistors and artificial visual system are introduced from the aspects of the action mechanism and device structure. Finally, the development prospect of optoelectronic neuromorphic analog devices based on low-dimensional materials is prospected.     

Retina-Inspired Artificial Synapses with Ultraviolet to Near-Infrared Broadband Responses for Energy-Efficient Neuromorphic Visual Systems

Neuromorphic visual system with image perception, memory, and preprocessing functions is expected to simulate basic features of the human retina. Organic optoelectronic synaptic transistors emulating biological synapses may be promising candidates for constructing neural morphological visual system. However, the sensing wavelength range of organic optoelectronic synaptic transistors usually limits their potential in artificial multispectral visual perception. Here, retina-inspired optoelectronic synaptic transistors that present broadband responses covering ultraviolet, visible, and near-infrared regions are demonstrated, which leverage the wide-range photoresponsive charge trapping layer and the heterostructure formed between PbS quantum dots and organic semiconductor. Simplified neuromorphic visual arrays are developed to simulate comprehensive image perception, memory, and preprocessing functions. Benefitting from the flexibility of the charge trapping and organic semiconductor layers, a flexible neuromorphic visual array can be fabricated, having an ultralow power consumption of 0.55 fJ per event under a low operating voltage of −0.01 V. More significantly, an accelerating image preprocessing effect can be observed in a wide wavelength range even beyond the perception range of the human visual system, due to the gate-adjustable synaptic plasticity. These devices are highly promising for implementing neuromorphic visual systems with broadband perception, increasing image processing efficiency, and promoting the development of artificial vision.     

Broadband Visual Adaption and Image Recognition in a Monolithic Neuromorphic Machine Vision System

Bio-inspired machine visions have caused wide attentions due to the higher time/power efficiencies over the conventional architectures. Although bio-mimic photo-sensors and neuromorphic computing have been individually demonstrated, a complete monolithic vision system has rarely been studied. Here, a neuromorphic machine vision system (NMVS) integrating front-end retinomorphic sensors and a back-end convolutional neural network (CNN) based on a single ferroelectric-semiconductor-transistor (FST) device structure is reported. As a photo-sensor, the FST shows a broadband (275–808 nm) retina-like light adaption function with a large dynamic range of 20.3 stops, and as a unit of the CNN, the FST's weight can be linearly programmed. In total, the NMVS has a high recognition accuracy of 93.0% on a broadband-dim-image classification task, which is 20% higher than that of an incomplete system without the retinomorphic sensors. Because of the monolithic unit, the NVMS shows high feasibility for integrated bio-inspired machine vision systems.     

Carbon Nanotube Optoelectronic Synapse Transistor Arrays with Ultra-Low Power Consumption for Stretchable Neuromorphic Vision Systems

High-performance stretchable optoelectronic synaptic transistor arrays are key units for constructing and mimicking simulated neuromorphic vision systems. In this study, ultra-low power consumption and low-operation-voltage stretchable all-carbon optoelectronic synaptic thin film transistors (TFTs) using sorted semiconducting single-walled carbon nanotubes (sc-SWCNTs) modified with CdSe/ZnS quantum dots as active layers on ionic liquid-based composite elastomer substrates are first reported. The resulting stretchable TFT devices show enhancement-mode characteristics with excellent electrical properties (such as the record on/off ratios up to 10⁵, negligible hysteresis, and small subthreshold swing), excellent mechanical tensile properties (such as the only 12.4% and 6.4% degradations of the carrier mobility after 20% vertical and horizontal strain stretching), and optoelectronic synaptic plasticity (for the recognition of Morse codes) with ultra-low power consumptions (15.38 aJ) at the operating voltage from −1 to 0.2 V. At the same time, the designed nonvolatile conductance of the stretchable SWCNT optoelectronic synapse thin film transistors (SSOSTFTs) stimulated by UV light and the bending angle are first used to simulate stretchable neuromorphic vision systems (including the functions of the crystalline lens and optic cone cells as bionic eyes) for detecting the atmospheric environment with a record accuracy of 95.1% as a bionic eye.     

一种基于动态贝叶斯网的视觉注意力识别方法

头部姿态估计是识别用户视觉注意力目标的主要依据.但在实际应用场合下,大范围头部姿态、低分辨率图像以及光照变化等因素使得可靠、准确的头部姿态估计难以实现.针对这些困难,提出一种基于动态贝叶斯网模型的视觉注意力目标识别方法.通过人脸图像与多个人脸姿态类别的相似度向量对头部姿态进行度量而不是显式的计算具体姿态值.模型融合多注意力目标、多用户位置、多摄像机图像等因素间的概率依赖关系并进行联合推理.智能厨房原型环境下的实验结果表明提出的模型是有效的.     

基于注意模型的视觉替代方法

现有的视觉替代方法均是在特定环境中,通过目标识别获得映射特征,缺少普遍的适用性.针对这一问题,提出了一种基于注意模型的视觉替代方法.利用人眼的视觉特性,提取图像的感兴趣区域,并根据听觉显示的基本原理,提出了将感兴趣区域的位置、尺寸和颜色映射为音符响度、音长和音调的PSC映射方法.实验结果表明:将图像中引起视觉注意的区域映射为电子音符,符合人类视觉认知过程,有利于盲人获得外部环境的重要信息,降低了盲人训练和学习的难度,并且听感良好,不会造成听觉疲劳.     

一种基于视觉注意机制的感知物体提取算法

针对图像中感知物体提取问题,借鉴认知学对视觉信息表达的研究成果,提出一种感知物体的数学定义,并由视觉注意计算模型得到的视觉注意显著图选择区域增长算法的种子区域。建立图像中感知物体的马尔可夫随机场模型,通过度量图像中感知物体的显著性、边缘和同质性分布情况,最小化能量函数提取出图像中的感知物体。对含有飞机的图像中感知物体的提取实验,验证了提出算法的有效性、以及在认知上的合理性。     

Tin Doping Induced High-Performance Solution-Processed Ga2O3 Photosensor toward Neuromorphic Visual System

Ga₂O₃ is an emerging wide-bandgap semiconductor with high deep ultraviolet absorption, tunable persistent photoconductivity, and excellent stability toward electric fields, making it a promising component for neuromorphic visual systems (NVSs). However, Ga₂O₃-based photosensors with high responsivity and long response decay times are required for efficient NVSs. A solution-processed doping strategy for fabrication of Ga₂O₃ is proposed with tin foil as a dopant source. Tin-doped Ga₂O₃ (Ga₂O₃:Sn) photosensors are obtained with ultrahigh responsivity and extremely long response decay times. These behaviors are attributed to substitutional tin and oxygen vacancies that modulate defect-related hole trapping. High-performance Ga₂O₃:Sn photosensors can mimic photonic synaptic behaviors and image pre-processing functions. NVSs based on a Ga₂O₃:Sn photonic synapse array perform pattern recognition with an accuracy of 97.3% under an unprecedented low-light pulse stimuli of 0.5 µW cm⁻². This work provides a low-cost solution-processed approach to ultrasensitive Ga₂O₃:Sn NVSs and will facilitate developments in artificial intelligence technology.     

基于视觉注意力模型的红外目标检测

根据视觉心理学的相关理论,提出一种基于灰度特征提取和视觉注意力模型的红外目标检测方法,并应用于复杂背景的红外目标检测.首先对输入图像进行采样,生成高斯金字塔,并用center-surround算子提取多尺度的视觉差异,通过对灰度特征图的归一化和线性融合获得综合的显著图,最后通过基于相似性和邻接性的阈值判断得到最终检测结果.该方法应用于多种地面目标的检测均取得较好效果,待检测的目标在显著图中得到明显的增强.试验结果进一步验证了算法具有很好的探测性能.     

基于视觉注意机制和水平集方法的红外海面目标检测与识别

针对传统红外目标检测与识别方法所存在 的问题,即其处理过程总是盲目地对全图进行耗时搜索,提出了一 种基于视觉注意机制和水平集方法的红外海面目标检测与识别方 法。首先,搜索原始图像中的显著性区域,并以获胜点的形式 表示它们。接着,基于所得到的显著性区域,自动初始化水平集函 数,并使演化过程朝着期望的目标轮廓方向挺进,直至演化过程到 达最终的平衡状态。最后,针对远距离(近距离)成像时的输入数 据,给出检测结果(基于不变矩和神经网络框架的识别结果)。对 真实红外海面目标进行的实验证实了本文方法的有效性。     

Strategy Toward Semiconducting Ti3C2Tx-MXene: Phenylsulfonic Acid Groups Modified Ti3C2Tx as Photosensitive Material for Flexible Visual Sensory-Neuromorphic System

The excellent electronic and electrochemical properties make 2D MXenes suitable candidates for sensors, batteries, and supercapacitors. However, the metallic-like behavior of MXenes hinders their potential for optoelectronic devices such as photodetectors. In this study, the band gap of metalloid Ti₃C₂T_x MXene is successfully opened to 1.53 eV with phenylsulfonic acid groups and realized a transistor and high-performance near-infrared photodetector array for a flexible vision sensory-neuromorphic system. The phenylsulfonic acid groups modified Ti₃C₂T_x MXene (S-Ti₃C₂T_x)-based flexible photodetector has a maximum responsivity of 8.50×10² A W⁻¹ and a detectivity of 3.69×10¹¹ Jones under 1064 nm laser irradiation. Moreover, the fabricated flexible vision sensory-neuromorphic system for image recognition realizes a high recognition rate >0.99, leading to great potential in the field of biological visual simulation and biomimetic eye. Besides conventional devices with Au as the conductive electrodes, all Ti₃C₂T_x MXene-based devices are also fabricated with S-Ti₃C₂T_x as the photosensitive material and unmodified Ti₃C₂T_x as the conductive electrodes, exhibiting comparable optoelectronic performances.     

Short-Wave Infrared Synaptic Phototransistor with Ambient Light Adaptability for Flexible Artificial Night Visual System

Flexible organic monitoring system that can work in short-wave infrared (SIR) region has great potential in autonomous driving, night driving safety, military encryption, biomedical imaging, and robot engineering. Especially, the development of infrared artificial vision system device that can autonomously improve the computing speed and adapt to different brightness ambient light is very important. However, it is a challenge for mimicking infrared visual adaptation because of the need for infrared absorbing materials and the need to control the concentration of carriers. In this study, inorganic quantum dot material is combined with organic materials to promote carrier separation and introduce interface defects that adjust the carrier concentration in transistors, which induce the synaptic behavior under SIR (1100 nm) light in darkness and the ability to adapt to white ambient light. Furthermore, the device array realizes the image recognition of SIR light at night with white ambient light of different brightness, exhibiting good self-adaptability, and strong anti-interference ability. These results demonstrate promising applications of the infrared synaptic phototransistors in adapted bionic optoelectronic devices.