基于视觉特征的非侵入式用户界面输入项识别

识别应用界面中的输入项是界面测试数据自动生成的基础。现有界面输入项信息获取依赖被测对象所在系统提供的信息，无法用于非侵入式测试场合。本文从应用界面的视觉图像出发，提出一种基于视觉特征的非侵入式用户界面输入项识别方法。该方法从外部摄像头等途径获得的应用界面图片出发，利用直线检测和轮廓检测算法得到候选输入项列表，基于视觉特征，利用支持向量机（SVM）模型来对候选输入项进行判别，能够在不依赖被测目标底层系统的情况下获取输入项信息。实验结果表明，本文方法能够基本准确地识别界面图片中的输入项，具有一定的有效性。

Non-intrusive GUI Input Item Recognition Based on Visual Feature

Recognizing input items in an application interface is a basis of automatic GUI test data generation. How to get information of input items in existing interface depends on the information provided by the system of the object under test. They cannot be used under non-intrusive scenarios. This paper proposes a non-intrusive GUI input item recognition method based on visual features. The method detects input items in a GUI interface from GUI images obtained from sources like external cameras. It first uses line detection and contour detection algorithms to obtain a list of candidate input items, then uses support vector machine (SVM) to determine which of these candidates are real input items based on visual features. The method does not rely on the underlying system of the target under test. Experimental results show that it can recognize the input items in a GUI interface with a good accuracy and hence is of effectiveness.