Efficient collection and automatic annotation of real-world object images by taking advantage of post-diminished multiple visual markers*

ABSTRACT

To collect a human-annotated dataset for training deep convolutional neural networks is a very time-consuming and laborious process. To reduce this burden, we previously proposed an automated annotation by placing one visual marker above the detection target object in the training phase. However, in this approach, occasionally the marker hides the object surface. To avoid this issue, we propose placing a pedestal with multiple markers at the bottom of the object. If we use multiple markers, the object can be annotated even when the object hides some of the markers. Besides that, the simple modification of placing the markers on the bottom allows the use of simple background masking to avoid the neural network learning the remaining markers in the training image as a feature of the object. Background masking can completely remove the markers during the training process. Experiments showed the proposed vision system using our automatic object annotation outperformed the vision system using manual annotation in terms of object detection, orientation estimation, and 2D position estimation while reducing the time required for dataset collection from 16.1 hours to 7.30 hours.     

整合神经网络置乱图像的动态自反馈混沌系统图像加密

目的 针对当前大多数数字图像加密算法多采用单一的混沌系统,且置乱方法基本只采用像素行列互换、Arnold变换、Baker变换、序列排序构造替换表等几类,提出一种新的整合神经网络置乱图像的动态自反馈混沌系统图像加密算法。方法 该算法通过1维Logistic混沌、chebyshev混沌和自定义m（x）运算构造了一种动态自反馈混沌系统,通过频数检测、序列分布图、平衡度分析、相关性分析、Lyapunov指数验证了系统的随机性,并对其序列进行了均匀化处理,通过序列均匀性证明、序列分布图、序列期望和方差验证了均匀化效果。该算法从混沌序列中随机选取输入值和参数输入神经网络,采用每组神经网络输出值构造置乱矩阵进行初次全局置乱,再从bit位进行二次置乱;采用两组与明文相关的秘钥序列进行像素值替代扩散,使得明文到密文经过中间密文变化,增强了算法的安全性。结果 通过计算机仿真和性能分析表明该加密算法体现了良好的密码学特征,从秘钥空间、秘钥敏感性、统计分析、信息熵、差分分析、相邻像素相关性分析各方面验证了其安全性,数据表明该算法秘钥空间达到了2²¹⁶,信息熵为7.998 3,水平、垂直、对角方向相邻像素相关系数分别为-0.000 381、0.000 607、-0.000 309,NPCR值介于（0.995~80.996 6）之间,UACI值介于（0.333~0.338）之间。结论 该算法可以实现良好的加密效果,在数据对比上优于超混沌系统图像加密、像素位置和bit位双重置乱加密等,可以被广泛应用在灰度图像加密中乃至扩展到彩色图像加密中,能够起到图像信息在网络传输、存储中的隐私保护作用。     

A neural network based multi-classifier system for gene identification in DNA sequences

The paper presents a neural network based multi-classifier system for the identification of Escherichia coli promoter sequences in strings of DNA. As each gene in DNA is preceded by a promoter sequence, the successful location of an E. coli promoter leads to the identification of the corresponding E. coli gene in the DNA sequence. A set of 324 known E. coli promoters and a set of 429 known non-promoter sequences were encoded using four different encoding methods. The encoded sequences were then used to train four different neural networks. The classification results of the four individual neural networks were then combined through an aggregation function, which used a variation of the logarithmic opinion pool method. The weights of this function were determined by a genetic algorithm. The multi-classifier system was then tested on 159 known promoter sequences and 171 non-promoter sequences not contained in the training set. The results obtained through this study proved that the same data set, when presented to neural networks in different forms, can provide slightly varying results. It also proves that when different opinions of more classifiers on the same input data are integrated within a multi-classifier system, we can obtain results that are better than the individual performances of the neural networks. The performances of our multi-classifier system outperform the results of other prediction systems for E. coli promoters developed so far.

Neural network algorithms for analyzing multidimensional time series for predicting events and their application to study of Sun-Earth relations

In the paper, an original neural network algorithm for analysis of time series is presented. This algorithm allows predicting the occurrence of a certain event and finding a time interval to which a phenomenon (a precursor or a cause of the event) belongs. The characteristics of the algorithm functioning are investigated applied to the study of the solar-terrestrial relationship. Yu. V. Orlov. Candidate in Physics and Mathematics. Researcher at the Institute of Nuclear Physics, Moscow State University. Scientific interests: neural networks, genetic algorithms, algorithms of pattern recognition and image analysis. Yu. S. Shugai. Researcher at the Institute of Nuclear Physics, Moscow State University. Scientific interests: neural networks, genetic algorithms, algorithms of pattern recognition and image analysis, algorithms of classification and prediction. I. G. Persiantsev. Professor, Doctor in Mathematics and Physics. Head of the Laboratory, Leading Researcher at the Institute of Nuclear Physics, Moscow State University Scientific interests: neural networks, genetic algorithms, algorithms of pattern recognition and image analysis, algorithms of classification and prediction, inverse problems. Laureate of the USSR State Prize. S. A. Dolenko. Candidate in Physics and Mathematics. Senior Researcher at the Institute of Nuclear Physics, Moscow State University. Scientific interests: neural networks, genetic algorithms, algorithms of pattern recognition and image analysis, algorithms of classification and prediction, inverse problems.     

Comparison of image processing algorithms and neural networks in machine vision inspection

Automated vision inspection has become a vital part of the computer integrated manufacturing systems. This paper compares the development and performance of two methodologies for a machine vision inspection system. The first method is developed through conventional image processing algorithms and the second method is based on the neural networks. A case study was conducted to benchmark these two methods. The results showed that the conventional image processing algorithms required less development time than the neural networks. A considerable amount of time was spend on training the neural networks. However, the neural networks performed better than the conventional image processing algorithms in terms of accuracy.     

Enhanced motion parameters estimation for an active vision system

In this paper, we present an enhanced approach for estimating 3D motion parameters from 2D motion vector fields. The proposed method achieves valuable reduction in computational time and shows high robustness against noise in the input data. The output of the algorithm is part in a multiobject segmentation approach implemented in an active vision system. Hence, the improvement in the motion parameters estimation process leads to speed-up in the overall segmentation process. The text was submitted by the authors in English. Mohamed Shafik obtained his B.Sc. in mechanical engineering at the University of Banha. In 2004 he earned an Information Technology Diploma in Mechatronics from the Information Technology Institute (ITI). In 2006 he obtained his M.Eng. in applied mechatronics at the University of Paderborn. Since 2006, he is a PhD student and a scientific assistant in the GET Lab. His research interests focus on robotic vision, neural networks, and mechatronic systems. Baerbel Mertsching studied electrical engineering and obtained her PhD at the University of Paderborn. Between 1994 and 2003, she was professor of computer science at the University of Hamburg. In 2003 she returned to the University of Paderborn where she is now professor of electrical engineering and director of the GET Lab. Her research interests focus on cognitive systems engineering, especially active vision systems, and microelectronics for image and speech processing. She has been a member of a variety of scientific councils and editorial boards and is author of more than 120 scientific publications.     

多目标粒子群优化PCNN参数的图像融合算法

目的 脉冲耦合神经网络（PCNN）在图像融合上往往因为参数设置问题而达不到最佳效果,为了提高图像融合的质量,提出了一种基于多目标粒子群优化PCNN参数的图像融合算法。方法 首先用多目标粒子群对PCNN模型参数进行优化得到最优PCNN参数模型,然后利用双复树小波（DTCWT）对图像多尺度分解,将高频分量通过优化好的PCNN模型进行高频融合,低频分量通过拉普拉斯分量绝对和（SML）来进行低频融合,最后通过DTCWT逆变换实现图像的融合。结果 分别与DTCWT,拉普拉斯金字塔变换（LP）以及非下采样Contourlet变换（NSCT）进行实验对比,融合图像Clock,Lab的融合结果在客观指标上的互信息（8.062 3,7.908 5）、图像的品质因数（0.716 2,0.714 2）和标准差（51.213,47.671）都优于其他方法,熵和其他方法差不多,融合结果能够获得更好的视觉效果以及较大的互信息值和边缘信息保留值。结论 该方法有较好融合图像的能力,可适用于计算机视觉、医学、遥感等领域。     

Interactive online learning

This paper describes a system for visual object recognition based on mobile augmented reality gear. The user can train the system to the recognition of objects online using advanced methods of interaction with mobile systems: Hand gestures and speech input control “virtual menus,” which are displayed as overlays within the camera image. Here we focus on the underlying neural recognition system, which implements the key requirement of an online trainable system—fast adaptation to novel object data. The neural three-stage architecture can be adapted in two modes: In a fast training mode (FT), only the last stage is adapted, whereas complete training (CT) rebuilds the system from scratch. Using FT, online acquired views can be added at once to the classifier, the system being operational after a delay of less than a second, though still with reduced classification performance. In parallel, a new classifier is trained (CT) and loaded to the system when ready. The text was submitted by the authors in English. Gunther Heidemann was born in 1966. He studied physics at the Universities of Karlsruhe and Münster and received his PhD (Eng.) from Bielefeld University in 1998. He is currently working within the collaborative research project “Hybrid Knowledge Representation” of the SFB 360 at Bielefeld University. His fields of research are mainly computer vision, robotics, neural networks, data mining, bonification, and hybrid systems. Holger Bekel was born in 1970. He received his BS degree from the University of Bielefeld, Germany, in 1997. In 2002 he received a diploma in Computer Science from the University of Bielefeld. He is currently pursuing a PhD program in Computer Science at the University of Bielefeld, working within the Neuroinformatics Group (AG Neuroinformatik) in the project VAMPIRE (Visual Active Memory Processes and Interactive Retrieval). His fields of research are active vision and data mining. Ingo Bax was born in 1976. He received a diploma in Computer Science from the University of Bielefeld in 2002. He is currently pursuing a PhD program in Computer Science at the Neuroinformatics Group of the University of Bielefeld, working within the VAMPIRE project. His fields of interest are cognitive computer vision and pattern recognition. Helge J. Ritter was born 1958. He studied physics and mathematics at the Universities of Bayreuth, Heidelberg and Munich. After a PhD in physics at Technical University of Munich in 1988, he visited the Laboratory of Computer Science at Helsinki University of Technology and the Beckman Institute for Advanced Science and Technology at the University of Illinois at Urbana-Champaign. Since 1990 he has headed the Neuroinformatics Group at the Faculty of Technology, Bielefeld University. His main interests are principles of neural computation and their application to building intelligent systems. In 1999, she was awarded the SEL Alcatel Research Prize, and in 2001, the Leibniz Prize of the German Research Foundation DFG.     

Dynamic recurrent neural networks for a hybrid intelligent decision support system for the metallurgical industry

Knowledge-based modeling and implementation of the various manufacturing processes represent an intensive research area. It is known that it is difficult to analyze the mechanisms of many industrial production processes and build dynamic models by employing classical methods for intelligent systems in manufacturing. This paper describes how to use dynamic recurrent neural networks to provide the model base of a hybrid intelligent system for the metallurgical industry with a quality control model. The hybrid system extracts the features of image sequences obtained through the vision detection subsystem and employs a dynamic recurrent neural network to assess and predict the product qualities to further coordinate the entire production process.     

边缘增强深层网络的图像超分辨率重建

目的 针对基于学习的图像超分辨率重建算法中存在边缘信息丢失、易产生视觉伪影等问题,提出一种基于边缘增强的深层网络模型用于图像的超分辨率重建。方法 本文算法首先利用预处理网络提取输入低分辨率图像的低级特征,然后将其分别输入到两路网络,其中一路网络通过卷积层级联的卷积网络得到高级特征,另一路网络通过卷积网络和与卷积网络成镜像结构的反卷积网络的级联实现图像边缘的重建。最后,利用支路连接将两路网络的结果进行融合,并将其结果通过一个卷积层从而得到最终重建的具有边缘增强效果的高分辨率图像。结果 以峰值信噪比（PSNR）和结构相似度（SSIM）作为评价指标来评价算法性能,在Set5、Set14和B100等常用测试集上放大3倍情况下进行实验,并且PSNR/SSIM指标分别取得了33.24 dB/0.9156、30.60 dB/0.852 1和28.45 dB/0.787 3的结果,相比其他方法有很大提升。结论 定量与定性的实验结果表明,基于边缘增强的深层网络的图像超分辨重建算法所重建的高分辨率图像不仅在重建图像边缘信息方面有较好的改善,同时也在客观评价和主观视觉上都有很大提高。     

机器视觉表面缺陷检测综述

目的 工业产品的表面缺陷对产品的美观度、舒适度和使用性能等带来不良影响,所以生产企业对产品的表面缺陷进行检测以便及时发现并加以控制。机器视觉的检测方法可以很大程度上克服人工检测方法的抽检率低、准确性不高、实时性差、效率低、劳动强度大等弊端,在现代工业中得到越来越广泛的研究和应用。方法 以机器视觉表面缺陷检测为研究对象,在广泛调研相关文献和发展成果的基础上,对基于机器视觉在表面缺陷检测领域的应用进行了综述。分析了典型机器视觉表面缺陷检测系统的工作原理和基本结构,阐述了表面缺陷视觉检测的研究现状、现有视觉软件和硬件平台,综述了机器视觉检测所涉及到的图像预处理算法、图像分割算法、图像特征提取及其选择算法、图像识别等相关理论和算法研究,并对每种主要方法的基本思想、特点和存在的局限性进行了总结,对未来可能的发展方向进行展望。结果 机器视觉表面缺陷检测系统中,图像处理和分析算法是重要内容,算法各有优缺点和其适应范围。如何提高算法的准确性、实时性和鲁棒性,一直是研究者们努力的方向。结论 机器视觉是对人类视觉的模拟,机器视觉表面检测涉及众多学科和理论,如何使检测进一步向自动化和智能化方向发展,还需要更深入的研究。     

多尺度双通道卷积神经网络下的刺绣模拟

目的 针对现有刺绣模拟算法中针线感不强、针线轨迹方向单一等问题,提出了一种基于多尺度双通道卷积神经网络的刺绣模拟算法。方法 1）搭建多尺度双通道网络,选取一幅刺绣艺术作品作为风格图像,将MSCOCO（microsoft common objects in context）数据集作为训练集,输入网络得到VGG（visual geometry group）网络损失和拉普拉斯损失;2）将总损失值传回到网络,通过梯度下降法更新网络参数,并且重复更新参数直到指定的训练次数完成网络训练;3）选取一幅目标图像作为刺绣模拟的内容图像,输入训练完成的网络,获得具有刺绣艺术风格的结果图像;4）使用掩模图像将得到的结果图像与绣布图像进行图像融合,即完成目标图像的刺绣模拟。结果 本文算法能产生明显的针线感和多方向的针线轨迹,增强了刺绣模拟绘制艺术作品的表现力。结论 本文将输入图像经过多尺度双通道卷积神经网络进行前向传播,并使用VGG19、VGG16和拉普拉斯模块作为损失网络进行刺绣模拟。实验结果表明,与现有卷积神经网络风格模拟算法对比,本文提出的网络能够学习到刺绣艺术风格图像的针线特征,得到的图像贴近真实刺绣艺术作品。     

结合视觉注意机制与递归神经网络的图像检索

目的 图像检索是计算机视觉的一项重要任务。图像检索的关键是图像的内容描述,复杂图像的内容描述很具有挑战性。传统的方法用固定长度的向量描述图像内容,为此提出一种变长序列描述模型,目的是丰富特征编码的信息表达能力,提高检索精度。方法 本文提出序列描述模型,用可变长度特征序列描述图像。序列描述模型首先用CNN(convolutional neural network)提取底层特征,然后用中间层LSTM(long short-term memory)产生局部特征的相关性表示,最后用视觉注意LSTM(attention LSTM)产生一组向量描述一幅图像。通过匈牙利算法计算图像之间的相似性完成图像检索任务。模型采用标签级别的triplet loss函数进行端对端的训练。结果 在MIRFLICKR-25K和NUS-WIDE数据集上进行图像检索实验,并和相关算法进行比较。相对于其他方法,本文模型检索精度提高了512个百分点。相对于定长的图像描述方式,本文模型在多标签数据集上能够显著改善检索效果。结论 本文提出了新的图像序列描述模型,可以显著改善检索效果,适用于多标签图像的检索任务。     

An Overview of Neural OCR Networks

Abstract

Because neural networks specialize in handling ambiguous data, they are especially suited for such applications as speech recognition and optical character recognition (OCR). OCR applications are usually ambiguous because their data is generated by an inconsistent factor—the individual. This article provides an overview of neural networks and describes how this technology can be integrated with OCR technology to create neural OCR networks that can significantly improve the process of optical character recognition.     

Tracking the activity of participants in a meeting

A vision system suitable for a smart meeting room able to analyse the activities of its occupants is described. Multiple people were tracked using a particle filter in which samples were iteratively re-weighted using an approximate likelihood in each frame. Trackers were automatically initialised and constrained using simple contextual knowledge of the room layout. Person–person occlusion was handled using multiple cameras. The method was evaluated on video sequences of a six person meeting. The tracker was demonstrated to outperform standard sampling importance re-sampling. All meeting participants were successfully tracked and their actions were recognised throughout the meeting scenarios tested.H. Nait Charif was funded by UK EPSRC Grant GR/R27419/01. Hammadi Nait Charif was born in Tinghir, Ouarzazat, Morocco on 25 December 1965. He received his Master of Engineering (Ingenieur d'Etat Diploma) in electrical engineering in 1990 and after a short-term job with the Ministry of Telecommunication, was appointed lecturer at Mohamed I University in 1991. He was a Monbusho visiting research fellow at Chiba University, Japan (1994–1995) where he received his PhD in 1998. He was an Assistant Professor and then an Associate Professor in electrical engineering at Mohamed I University (1998–2001). In 1999, he was a Fulbright Visiting Assistant Professor at Michigan State University. At the University of Dundee he has worked on the EPSRC project “Advanced Sensors for Supportive Environments for Elderly”. His research interests include image processing, computer vision and neural networks. Stephen McKenna is a Senior Lecturer at the University of Dundee. He graduated BSc (Hons) in Computer Science from the University of Edinburgh and PhD from the University of Dundee (1994). He has held post-doctoral research positions at Queen Mary, University of London and Tecnopolis Csata, Italy and has been a visiting researcher at BT Labs and George Mason University. Funders of his research include EPSRC, BBSRC and MRC. He has served on international program committees and is an Associate Editor of the journal Machine Vision and Applications. He co-authored the book “Dynamic Vision” and has published 75 articles on computer vision and pattern recognition. His research interests include the application of computer vision, imaging and machine learning to intelligent human–computer interaction, monitoring, surveillance, medicine and biology.     

Stereophotogrammetric 3D real-time machine vision

Signal processing algorithms often have to be modified significantly for implementation in hardware. Continuous real-time image processing at high speed is a particularly challenging task. In this paper a hardware-software codesign is applied to a stereophotogrammetric system. To calculate the depth map, an optimized algorithm is implemented as a hierarchical-parallel hardware solution. By subdividing distances to objects and selecting them sequentially, we can apply 3D scanning and ranging over large distances. We designed processor-based object clustering and tracking functions. We can detect objects utilizing density distributions of disparities in the depth map (disparity histogram). Motion parameters of detected objects are stabilized by Kalman filters. The text was submitted by the authors in English. Michael Tornow was born in Magdeburg, Germany, in 1977. He received his diploma engineer degree (Dipl.-Ing.) in electrical engineering at the University of Magdeburg, Germany, in 2002. He is currently working on a PhD thesis focusing on hardware adapted image processing and vision based driver assistance. Robert W. Kuhn received his diploma engineer degree (Dipl.-Ing.) in geodesy at the Technical University of Berlin, Germany, in 2000. His current work on a PhD thesis focuses on calibration and image processing. Jens Kaszubiak was born in Blankenburg, Germany, in 1977. He received his diploma engineer degree (Dipl.-Ing.) in electrical engineering at the University of Magdeburg, Germany, in 2002. His current research work focuses on vision-based driver assistance and hardware-software codesign. Bernd Michaelis was born in Magdeburg, Germany, in 1947. He received a Masters Degree in Electronic Engineering from the Technische Hochschule, Magdeburg, in 1971 and his first PhD in 1974. Between 1974 and 1980 he worked at the Technische Hochschule, Magdeburg, and was granted a second doctoral degree in 1980. In 1993 he became Professor of Technical Computer Science at the Otto-von-Guericke University, Magdeburg. His research work concentrates on the field of image processing, artificial neural networks, pattern recognition, processor architectures, and microcomputers. Professor Michaelis is the author of more than 150 papers. Gerald Krell was born in Magdeburg, Germany, in 1964. He earned his diploma in electrical engineering in 1990 and his doctorate in 1995 at Otto-von-Guericke University of Magdeburg. Since then he has been a research assistant. His primary research interest is focused on digital image processing and compression, electronic hardware development, and artificial neural networks.     

Classification of remotely-sensed image data using artificial neural networks

Abstract

Abstract. Artificial neural networks have been used recently for speech and character recognition. Their application for the classification of remotely-sensed images is reported in this Letter. Remotely sensed image data are usually large in size and spectral overlaps among classes of ground objects are common. This results in low convergence performance of the Back-Propagation Algorithm in a neural network classifier. A Blocked Back-Propagation (BB-P) algorithm was proposed arid described in this Letter. It improved convergence performance and classification accuracy.     

高分辨卫星图像卷积神经网络分类模型

目的 卫星图像往往目标、背景复杂而且带有噪声,因此使用人工选取的特征进行卫星图像的分类就变得十分困难。提出一种新的使用卷积神经网络进行卫星图像分类的方案。使用卷积神经网络可以提取卫星图像的高层特征,进而提高卫星图像分类的识别率。方法 首先,提出一个包含六类图像的新的卫星图像数据集来解决卷积神经网络的有标签训练样本不足的问题。其次,使用了一种直接训练卷积神经网络模型和3种预训练卷积神经网络模型来进行卫星图像分类。直接训练模型直接在文章提出的数据集上进行训练,预训练模型先在ILSVRC（the ImageNet large scale visual recognition challenge）-2012数据集上进行预训练,然后在提出的卫星图像数据集上进行微调训练。完成微调的模型用于卫星图像分类。结果 提出的微调预训练卷积神经网络深层模型具有最高的分类正确率。在提出的数据集上,深层卷积神经网络模型达到了99.50%的识别率。在数据集UC Merced Land Use上,深层卷积神经网络模型达到了96.44%的识别率。结论 本文提出的数据集具有一般性和代表性,使用的深层卷积神经网络模型具有很强的特征提取能力和分类能力,且是一种端到端的分类模型,不需要堆叠其他模型或分类器。在高分辨卫星图像的分类上,本文模型和对比模型相比取得了更有说服力的结果。     

Forecasting foreign exchange rates using recurrent neural networks

This article proposes the use of recurrent neural networks in order to forecast foreign exchange rates. Artificial neural networks have proven to be efficient and profitable in fore casting financial time series. In particular, recurrent networks in which activity patterns pass through the network more than once before they generate an output pattern can learn ex tremely complex temporal sequences. Three recurrent architectures are compared in terms of prediction accuracy of futures forecast for Deutsche mark currency. A trading strategy is then devised and optimized. The profitability of the trading strategy taking into account trans action costs is shown for the different architectures. The methods described here which have obtained promising results in real time trading are applicable to other markets.     

SAR图像目标识别的卷积神经网模型

目的 合成孔径雷达图像目标识别可以有效提高合成孔径雷达数据的利用效率。针对合成孔径雷达图像目标识别滤波处理耗时长、识别精度不高的问题,本文提出一种卷积神经网络模型应用于合成孔径雷达图像目标识别。方法 首先,针对合成孔径雷达图像特点设计特征提取部分的网络结构;其次,代价函数中引入L2范数提高模型的抗噪性能和泛化性;再次,全连接层使用Dropout减小网络的运算量并提高泛化性;最后研究了滤波对于网络模型的收敛速度和准确率的影响。结果 实验使用美国运动和静止目标获取与识别数据库,10类目标识别的实验结果表明改进后的卷积神经网络整体识别率（包含变体）由93.76%提升至98.10%。通过设置4组对比实验说明网络结构的改进和优化的有效性。卷积神经网络噪声抑制实验验证了卷积神经网络的特征提取过程对于SAR图像相干斑噪声有抑制作用,可以省去耗时的滤波处理。结论 本文提出的卷积神经网络模型提高了网络的准确率、泛化性,无需耗时的滤波处理,是一种合成孔径雷达图像目标识别的有效方法。