基于多模态迭代及修正的文本识别算法

针对场景文本识别在长距离建模时容易产生信息丢失和对低分辨率文本图像表征能力较弱的问题,提出了一种基于多模态迭代及修正的文本识别算法。本文算法的视觉模型(vision model)是由CoTNet(contextual transformer networks for visual recognition)、动态卷积注意力模块(dynamic convolution attention module,DCAM)、EA-Encoder(external attention encoder)和位置注意力机制组合而成的。其中CoTNet可以有效起到缓解长距离建模产生的信息丢失问题；DCAM在增强表征能力、专注于重要特征的同时,将重要的特征传给EA-Encoder,进而提高CoTNet和EA-Encoder之间的联系；EA-Encoder可以学习整个数据集上最优区分度的特征,捕获最有语义信息的部分,进而增强表征能力。经过视觉模型后,再经过文本修正模块(text correction model)和融合模块(fusion model)得到最终的识别结果。实验数据显示,本文所提出的算法在多个公共场景文本数据集上表现良好,尤其是在不规则数据集ICDAR2015上准确率高达85.9%。

Text recognition algorithm based on multimodal iteration and correction

A text recognition algorithm based on multimodal iteration and correction is proposed to address the problems that scene text recognition is prone to information loss when modeling over long distances and weak characterization for low-resolution text images.The visual model of the algorithm in this paper is a combination of contextual transformer networks for visual recognition (CoTNet),a dynamic convolutional attention module (DCAM),an external attention encoder (EA-Encoder),and a positional attention mechanism.The CoTNet can effectively alleviate the information loss problem arising from long-distance modeling.The DCAM enhances representation by focusing on the essential features while passing the critical components to the EA-Encoder,improving the connection between CoTNet and EA-Encoder.EA-Encoder learns the best distinguishing features on the entire dataset,capturing the most semantic information parts and thus enhancing representation.After the visual model,the text correction and fusion modules obtain the final recognition results.According to the experimental data,the algorithm proposed in this paper performs well on several public scene text datasets,especially on the irregular dataset ICDAR2015 with an accuracy of 85.9%.