基于多特征融合的细粒度视频人物关系抽取

视频人物关系抽取是信息抽取问题中的重要任务,在视频描述、视频检索,以及人物搜索、公安监察等方面具有重要价值。由于视频数据的底层像素与高层关系语义之间存在巨大的鸿沟,现有方法很难准确地抽取人物关系。现有研究大多通过粗粒度地分析人物共现等因素来抽取人物关系,忽略了具有丰富语义的视频中的细粒度信息。为解决现有算法难以准确、完整地抽取视频人物关系的问题,文中提出了一种基于多特征融合的细粒度视频人物关系抽取方法。首先,为了准确识别视频人物实体,提出了一种基于多特征融合的人物实体识别模型;然后,提出了一种基于细粒度特征的人物关系识别模型,该模型不仅融合了视频中人物的时空特征,而且考虑了与人物相关的细粒度物体信息特征,从而建立更好的映射关系来准确识别人物关系。以电影视频数据和SRIV人物关系识别数据集为实验数据,实验结果验证了该模型的有效性和准确性,与现有同类模型相比,所提模型的人物实体识别F 1值提高了约14.4%,人物关系识别的准确率提高了约10.1%。     

Decision making with an interval-valued fuzzy preference relation and admissible orders

In this paper we analyze under which conditions we must use interval-valued fuzzy relations in decision making problems. We propose an algorithm to select the best alternative from a set of solutions which have been calculated with the nondominance algorithm using intervals and different linear orders among them. Based on the study made by Orlovsky in his work about nondominance, we study a characterization of weak transitive and 0.5-transitive interval-valued fuzzy relations, as well as the conditions under which transitivity is preserved by some operators on those relations. Next, we study the case of interval-valued reciprocal relations. In particular, we describe the preservation of reciprocity by different operators and analyze the transitivity properties for these interval-valued reciprocal relations. Finally, we propose to use, in the nondominance algorithm, linear interval orders generated by means of operators which preserve transitivity.     

Off-line handwritten Korean character recognition based on stroke extraction and representation

In this paper, we propose an off-line recognition method for handwritten Korean characters based on stroke extraction and representation. To recognize handwritten Korean characters, it is required to extract strokes and stroke sequence to describe an input of two-dimensional character as one-dimensional representation. We define 28 primitive strokes to represent characters and introduce 300 stroke separation rules to extract proper strokes from Korean characters. To find a stroke sequence, we use stroke code and stroke relationship between consecutive strokes. The input characters are recognized by using character recognition trees. The proposed method has been tested for the most frequently used 1000 characters by 400 different writers and showed recognition rate of 94.3%.     

基于模式的远监督关系抽取算法

远监督关系抽取算法能够自动将关系库中的关系与无标注的文本对齐,以进行文本中的关系抽取。目前提出的远监督关系抽取算法中,大多数是基于特征的。然而,此类算法在将实例转换为特征时,经常会出现关键信息不突出、数据集线性不可分等问题,影响关系抽取的效果。该文提出了一种基于模式的远监督关系抽取算法,其中引入了基于模式的向量,并使用了基于核的机器学习算法来克服上述问题。实验结果表明,该文提出的基于模式的远监督关系抽取算法,能够有效地提升远监督关系抽取的准确率。     

一种基于SVG的在线连续手写汉字多步分割方法*

基于可伸缩矢量图SVG的在线手写汉字是以SVG图像作为汉字图像格式、以SVG的path对象作为笔画的基本存储单元来对汉字进行显示和存储的,笔画的轮廓是以手写过程中记录的坐标值作为特征数值加以确定的。基于此种SVG手写汉字存储和表示形式,本文提出一种基于图论的在线连续手写汉字多步分割方法。该方法根据汉字笔画间的坐标位置关系对手写笔画序列构建无向图模型,并利用图的广度优先搜索将原笔画序列分割为互不连通的笔画部件,使偏旁部首分离较远、非粘连汉字得到正确分割;然后利用改进的tarjan算法对部件中的粘连字符进行分割,最后基于笔画部件间距,利用二分类迭代算法对间距进行分类,找出全局最佳分割位置,对过分割的部件进行重组合并。实验结果表明,该方法对于在线手写汉字的分割是有效可行的。     

Discovering similar Chinese characters in online handwriting with deep convolutional neural networks

A primary reason for performance degradation in unconstrained online handwritten Chinese character recognition is the subtle differences between similar characters. Various methods have been proposed in previous works to address the problem of generating similar characters. These methods are basically comprised of two components—similar character discovery and cascaded classifiers. The goal of similar character discovery is to make similar character pairs/sets cover as many misclassified samples as possible. It is observed that the confidence of convolutional neural network (CNN) is output by an end-to-end manner and it can be understood as one type of probability metric. In this paper, we propose an algorithm by leveraging CNN confidence for discovering similar character pairs/sets. Specifically, a deep CNN is applied to output the top ranked candidates and the corresponding confidence scores, followed by an accumulating and averaging procedure. We experimentally found that the number of similar character pairs for each class is diverse and the confusion degree of similar character pairs is varied. To address these problems, we propose an entropy- based similarity measurement to rank these similar character pairs/sets and reject those with low similarity. The experimental results indicate that by using 30,000 similar character pairs, our method achieves the hit rates of 98.44 and 98.05 % on CASIA-OLHWDB1.0 and CASIA-OLHWDB1.0–1.2 datasets, respectively, which are significantly higher than corresponding results produced by MQDF-based method (95.42 and 94.49 %). Furthermore, recognition of ten randomly selected similar character subsets with a two-stage classification scheme results in a relative error reduction of 30.11 % comparing with traditional single stage scheme, showing the potential usage of the proposed method.     

Analysis of stroke structures of handwritten Chinese characters

Most handwritten Chinese character recognition systems suffer from the variations in geometrical features for different writing styles. The stroke structures of different styles have proved to be more consistent than geometrical features. In an on-line recognition system, the stroke structure can be obtained according to the sequences of writing via a pen-based input device such as a tablet. But in an off-line recognition system, the input characters are scanned optically and saved as raster images, so the stroke structure information is not available. In this paper, we propose a method to extract strokes from an off-line handwritten Chinese character. We have developed four new techniques: 1) a new thinning algorithm based on Euclidean distance transformation and gradient oriented tracing, 2) a new line approximation method based on curvature segmentation, 3) artifact removal strategies based on geometrical analysis, and 4) stroke segmentation rules based on splitting, merging and directional analysis. Using these techniques, we can extract and trace the strokes in an off-line handwritten Chinese character accurately and efficiently.     

On-line handwritten Chinese character recognition via a fuzzy attribute representation

Chinese characters are constructed by strokes according to structural rules. Therefore, the geometric configurations of characters are important features for character recognition. In handwritten characters, stroke shapes and their spatial relations may vary to some extent. The attribute value of a structural identification is then a fuzzy quantity rather than a binary quantity. Recognizing these facts, we propose a fuzzy attribute representation (FAR) to describe the structural features of handwritten Chinese characters for an on-line Chinese character recognition (OLCCR) system. With a FAR. a fuzzy attribute graph for each handwritten character is created, and the character recognition process is thus transformed into a simple graph matching problem. This character representation and our proposed recognition method allow us to relax the constraints on stroke order and stroke connection. The graph model provides a generalized character representation that can easily incorporate newly added characters into an OLCCR system with an automatic learning capability. The fuzzy representation can describe the degree of structural deformation in handwritten characters. The character matching algorithm is designed to tolerate structural deformations to some extent. Therefore, even input characters with deformations can be recognized correctly once the reference dictionary of the recognition system has been trained using a few representative learning samples. Experimental results are provided to show the effectiveness of the proposed method.     

基于汉字连通分量的印刷图像版面分割方法

针对彩色印刷图像背景色彩丰富和汉字存在多个连通分量,连通域文字分割算法不能精确提取文字,提出基于汉字连通分量的彩色印刷图像版面分割方法。利用金字塔变换逆半调算法对图像进行预处理,通过颜色采样和均值偏移分割图像颜色,标记文字连通分量,根据汉字结构和连通分量特性重建汉字连通分量,分析文字连通分量连接关系确定文字排列方向实现文字分割。实验结果表明,该方法能够有效地重建汉字连通分量,在彩色印刷图像中实现对不同字体、字号、颜色的文字分割。     

基于蚁群算法的现代藏文字符轮廓提取技术研究

由于对字符提取骨架往往会失去受污损部位的重要信息,因此本文提出了一种基于蚁群算法的现代藏文字符轮廓提取算法,旨在用字符的轮廓线代替骨架线来表征字符。本算法用于印刷体藏文轮廓提取,取得了良好的效果,避免了传统细化算法造成的畸变,提高了轮廓提取的抗干扰能力,并且减小了计算量,加快了特征提取的速度。     

一种基于分块树的手写数学公式结构分析算法

本文源于一个手写数学公式识别系统,该系统实现了手写数学公式到文本公式的自动转化。文中提出了一种基于分块树的数学公式结构分析方法,该方法首先根据其内部结构特征将数学表达式分解为若干子模块,并采用树型结构对每个子模块内部字符之间的结构关系进行表示,最终形成整个表达式的树型表示。该方法定义了一系列的字符结构属性,将字符及属性值作为结构分析的结果,这些属性值再现了公式的结构特征,并很容易被系统的公式文本显示部分所利用。另外,该方法对传统的字符空间关系类型进行了简化,减小了识别误差,而引入的分块处理方式更加适合具有根式和分式等多层嵌套结构公式的处理,并且具有较强的可扩展性。     

A deep learning based bank card detection and recognition method in complex scenes

With the widespread use of mobile Internet, mobile payment has become a part of daily life, and bank card recognition in natural scenes has become a hot topic. Although printed character recognition has achieved remarkable success in recent years, bank card recognition is not limited to traditional printed character recognition. There are two types of bank cards: unembossed bank cards, such as most debit cards which usually use printed characters, and embossed bank cards, such as most credit cards which mainly use raised characters. Recognition of raised characters is very challenging due to its own characteristics, and there is a lack of fast and good methods to handle it. To better recognize raised characters, we propose an effective method based on deep learning to detect and recognize bank cards in complex natural scenes. The method can accurately recognize the card number characters on embossed and unembossed bank cards. First, to break the limitation that YOLOv3 algorithm is usually used for object detection, we propose a novel approach that enables YOLOv3 to be used not only for bank card detection and classification, but also for character recognition. The CANNYLINES algorithm is used for rectification and the Scharr operator is introduced to locate the card number region. The proposed method can satisfy bank card detection, classification and character recognition in complex natural scenes, such as complex backgrounds, distorted card surfaces, uneven illumination, and characters with the same or similar color to the background. To further improve the recognition accuracy, a printed character recognition model based on ResNet-32 is proposed for the unembossed bank cards. According to the color and morphological characteristics of embossed bank cards, raised character recognition model combining traditional morphological methods and LeNet-5 convolutional neural network is proposed for the embossed bank cards. The experimental results on the collected bank card dataset and bank card number dataset show that our proposed method can effectively detect and identify different types of bank cards. The accuracy of the detection and classification of bank cards reaches 100%. The accuracy of the raised characters recognition on the embossed bank card is 99.31%, and the accuracy of the printed characters recognition on the unembossed bank card reaches 100%.

     

Multiplicative consistency-based decision support system for incomplete linguistic preference relations

The experts may have difficulty in expressing all their preferences over alternatives or criteria, and produce the incomplete linguistic preference relation. Consistency plays an important role in estimating unknown values from an incomplete linguistic preference relation. Many methods have been developed to obtain a complete linguistic preference relation based on additive consistency, but some unreasonable values may be produced in the estimation process. To overcome this issue, we propose a new characterisation about multiplicative consistency of the linguistic preference relation, present an algorithm to estimate missing values from an incomplete linguistic preference relation, and establish a decision support system for aiding the experts to complete their linguistic preference relations in a more consistent way. Some examples are also given to illustrate the proposed methods.     

Scene word recognition from pieces to whole

Convolutional neural networks (CNNs) have had great success with regard to the object classification problem. For character classification, we found that training and testing using accurately segmented character regions with CNNs resulted in higher accuracy than when roughly segmented regions were used. Therefore, we expect to extract complete character regions from scene images. Text in natural scene images has an obvious contrast with its attachments. Many methods attempt to extract characters through different segmentation techniques. However, for blurred, occluded, and complex background cases, those methods may result in adjoined or over segmented characters. In this paper, we propose a scene word recognition model that integrates words from small pieces to entire after-cluster-based segmentation. The segmented connected components are classified as four types: background, individual character proposals, adjoined characters, and stroke proposals. Individual character proposals are directly inputted to a CNN that is trained using accurately segmented character images. The sliding window strategy is applied to adjoined character regions. Stroke proposals are considered as fragments of entire characters whose locations are estimated by a stroke spatial distribution system. Then, the estimated characters from adjoined characters and stroke proposals are classified by a CNN that is trained on roughly segmented character images. Finally, a lexicondriven integration method is performed to obtain the final word recognition results. Compared to other word recognition methods, our method achieves a comparable performance on Street View Text and the ICDAR 2003 and ICDAR 2013 benchmark databases. Moreover, our method can deal with recognizing text images of occlusion and improperly segmented text images.     

汉字字形的关系稳定原理

本文对汉字的字形描述进行了深入的研究, 并在此基拙土总结得出了汉字字形的关来稳定原理在汉字字形中, 笔划基元的方向、长度、位置等属性均是不穗定的, 而各笔划塞元之间的关来是稳定的。基元间关亲是反映字形本质的因素, 是汉字字形信巴的主体。关来穗定原理作为反应汉字字形本质的重要原理, 除了在研究汉宇字形方面有重要意义之外, 最重要的应用就是对汉字识别的研究提供方向性的指导。     

Prototype learning for structured pattern representation applied to on-line recognition of handwritten Japanese characters

This paper describes prototype learning for structured pattern representation with common subpatterns shared among multiple character prototypes for on-line recognition of handwritten Japanese characters. Prototype learning algorithms have not yet been shown to be useful for structured or hierarchical pattern representation. In this paper, we incorporate cost-free parallel translation to negate the location distributions of subpatterns when they are embedded in character patterns. Moreover, we introduce normalization into a prototype learning algorithm to extract true feature distributions in raw patterns to aggregate distributions of feature points to subpattern prototypes. We show that our proposed method significantly improves structured pattern representation for Japanese on-line character patterns.     

Emotion recognition and its application to computer agents with spontaneous interactive capabilities

In this paper, we first study the recognition of emotions involved in human speech. We propose an emotion recognition algorithm based on a neural network and also propose a method to collect a large speech database that contains emotions. We carried out emotion recognition experiments based on the neural network trained using this database. An emotion recognition rate of approximately 50% was obtained in a speaker-independent mode for eight emotion states.

We then tried to apply this emotion recognition algorithm to a computer agent that plays a character role in the interactive movie system we are developing. We propose to use emotion recognition as key technology for an architecture of the computer characters with both narrative-based and spontaneous interaction capabilities.