一种改进的基于抽象语法树的软件源代码比对算法

在软件同源性检测方法中,基于抽象语法树的比对方法能够有效地检测出基于代码全文拷贝、修改变量名、调整代码顺序等的抄袭手段,被广泛用于抄袭检测工具中。但基于抽象语法树的比对方法对于修改变量类型和添加无意义变量的抄袭手段束手无策。针对这种情况,提出了一种基于抽象语法树的改进思想,该思想通过剪去语法树中影响判断的叶子节点的手段来还原检测原文抄袭,能够达到有效检测修改变量类型和添加无意义变量等抄袭的目的。     

Detecting plagiarized mobile apps using API birthmarks

This paper addresses the problem of detecting plagiarized mobile apps. Plagiarism is the practice of building mobile apps by reusing code from other apps without the consent of the corresponding app developers. Recent studies on third-party app markets have suggested that plagiarized apps are an important vehicle for malware delivery on mobile phones. Malware authors repackage official versions of apps with malicious functionality, and distribute them for free via these third-party app markets. An effective technique to detect app plagiarism can therefore help identify malicious apps. Code plagiarism has long been a problem and a number of code similarity detectors have been developed over the years to detect plagiarism. In this paper we show that obfuscation techniques can be used to easily defeat similarity detectors that rely solely on statically scanning the code of an app. We propose a dynamic technique to detect plagiarized apps that works by observing the interaction of an app with the underlying mobile platform via its API invocations. We propose API birthmarks to characterize unique app behaviors, and develop a robust plagiarism detection tool using API birthmarks.     

基于多特征值的源代码相似性检测技术

在软件开发的过程中,开发人员通过复制粘贴式的开发方式或者模块化的开发方式来完成需求是十分常见的,这两种开发方式可以提高开发效率,但同时会导致软件系统中出现大量的相同代码或者相似代码,大量的相似代码会给软件维护等方面带来很大的困难,这也是最常见的重构对象。源代码相似性度量是指利用一定的检测方法分析程序源代码间的相似程度。该技术被应用于代码抄袭检测、代码克隆检测、软件知识产权保护、代码复用等多个领域。为了提高代码相似性度量的准确性,提出了一种基于多特征值的源代码相似性检测技术。构建了源代码注释、型构、代码文本语句与结构中特征提取的方法,并给出了源代码相似度检测的度量模型。通过与权威的代码相似检测系统Moss进行对比实验,结果表明该方法可以更准确地检测出相似代码。     

一种基于静态词法树的程序相似性检测方法*

传统的程序相似性检测工具并不能有效地检测出一些常见的高级词法、语义理解变换的抄袭方式。首先归纳了学生常用的三类抄袭手段,然后给出了基于词法树的程序相似性检测方法。以C语言为例,总结了生成词法树的结构体,并对程序的词法树进行主数据流、结构控制流和时序流分析后得出结构体依赖图;使用形式化的图同型方法来判断代码是否相似,还给出了一个聚类方法以获得彼此相似的程序子集。通过与JPlag、BuaaSim系统针对一组典型的抄袭样本集进行评测结果对比,本方法具有更好的检测效果。     

Efficient plagiarism detection for large code repositories

Unauthorized re‐use of code by students is a widespread problem in academic institutions, and raises liability issues for industry. Manual plagiarism detection is time‐consuming, and current effective plagiarism detection approaches cannot be easily scaled to very large code repositories. While there are practical text‐based plagiarism detection systems capable of working with large collections, this is not the case for code‐based plagiarism detection. In this paper, we propose techniques for detecting plagiarism in program code using text similarity measures and local alignment. Through detailed empirical evaluation on small and large collections of programs, we show that our approach is highly scalable while maintaining similar levels of effectiveness to that of the popular JPlag and MOSS systems. Copyright © 2006 John Wiley & Sons, Ltd.     

程序源代码剽窃检测串匹配算法的研究

检测剽窃的源代码,在验证学生的作业中很重要。代码的相似度度量是剽窃检测的关键技术。首先本文介绍了源代码剽窃检测技术在国内外的发展情况,然后提出了一种改进的基于Karp-Rabin和GST算法的程序源代码相似度测量方法。     

基于多种技术的混合式程序代码抄袭检测方法

在分析现有程序代码抄袭检测系统的特点及局限性的基础上,提出一种综合文本分析、结构度量和属性计数技术的混合式程序抄袭检测方法。应用文档指纹技术和Winnowing算法计算程序的文本相似度;将程序代码表示成动态控制结构树（Dynamic Control Structure tree,DCS）,运用Winnowing算法计算DCS树相似度,从而得到结构相似度;收集并统计程序中的每个变量信息,应用变量相似度算法分析变量信息节点获取变量相似度;分别赋予文本相似度、结构相似度和变量相似度一个权值,计算得到总体的代码相似度。实验结果表明,所提出的方法能够有效检测出各种抄袭行为。针对不同的抄袭门槛值,使用该方法的检测结果准确度和查全率高于JPLAG系统。特别对于结构简单的程序组,此方法和JPLAG系统检测结果的平均准确度分别为82.5%和69.5%,说明所提的方法更加有效。     

Semantic Role Labeling of Chinese Nominal Predicates with Dependency-Driven Constituent Parse Tree Structure

This paper explores a tree kernel based method for semantic role labeling （SRL） of Chinese nominal predicates via a convolution tree kernel. In particular, a new parse tree representation structure, called dependency-driven constituent parse tree （D-CPT）, is proposed to combine the advantages of both constituent and dependence parse trees. This is achieved by directly representing various kinds of dependency relations in a CPT-style structure, which employs dependency relation types instead of phrase labels in CPT （Constituent Parse Tree）. In this way, D-CPT not only keeps the dependency relationship information in the dependency parse tree （DPT） structure but also retains the basic hierarchical structure of CPT style. Moreover, several schemes are designed to extract various kinds of necessary information, such as the shortest path between the nominal predicate and the argument candidate, the support verb of the nominal predicate and the head argument modified by the argument candidate, from D-CPT. This largely reduces the noisy information inherent in D-CPT. Finally, a convolution tree kernel is employed to compute the similarity between two parse trees. Besides, we also implement a feature-based method based on D-CPT. Evaluation on Chinese NomBank corpus shows that our tree kernel based method on D-CPT performs significantly better than other tree kernel-based ones and achieves comparable performance with the state-of-the-art feature-based ones. This indicates the effectiveness of the novel D-CPT structure in representing various kinds of dependency relations in a CPT-style structure and our tree kernel based method in exploring the novel D-CPT structure. This also illustrates that the kernel-based methods are competitive and they are complementary with the feature- based methods on SRL.     

Detection of idea plagiarism using syntax–Semantic concept extractions with genetic algorithm

Plagiarism is increasingly becoming a major issue in the academic and educational domains. Automated and effective plagiarism detection systems are direly required to curtail this information breach, especially in tackling idea plagiarism. The proposed approach is aimed to detect such plagiarism cases, where the idea of a third party is adopted and presented intelligently so that at the surface level, plagiarism cannot be unmasked. The reported work aims to explore syntax-semantic concept extractions with genetic algorithm in detecting cases of idea plagiarism. The work mainly focuses on idea plagiarism where the source ideas are plagiarized and represented in a summarized form. Plagiarism detection is employed at both the document and passage levels by exploiting the document concepts at various structural levels. Initially, the idea embedded within the given source document is captured using sentence level concept extraction with genetic algorithm. Document level detection is facilitated with word-level concepts where syntactic information is extracted and the non-plagiarized documents are pruned. A combined similarity metric that utilizes the semantic level concept extraction is then employed for passage level detection. The proposed approach is tested on PAN13-14¹plagiarism corpus for summary obfuscation data, which represents a challenging case of idea plagiarism. The performance of the current approach and its variations are evaluated both at the document and passage levels, using information retrieval and PAN plagiarism measures respectively. The results are also compared against six top ranked plagiarism detection systems submitted as a part of PAN13-14 competition. The results obtained are found to exhibit significant improvement over the compared systems and hence reflects the potency of the proposed syntax-semantic based concept extractions in detecting idea plagiarism.     

Code analyzer for an online course management system

The online course management system (OCMS) assists online instruction in various aspects, including testing, course discussion, assignment submission, and assignment grading. This paper proposes a plagiarism detection system whose design is integrated with an OCMS. Online assignment submission is prone to easy plagiarism, which can seriously influence the quality of learning. In the past, plagiarism was detected manually, making it very time-consuming. This research thus focuses on developing a system involving code standardization, textual analysis, structural analysis, and variable analysis for evaluating and comparing programming codes. An agent system serves as a daemon to analyze the program codes for OCMS. For textual analysis, the Fingerprinting Algorithm was used for text comparison. Structurally, a formal algebraic expression and a dynamic control structure tree (DCS Tree) were utilized to rebuild and evaluate the program structure. For variables, not only the relevant information for each variable was recorded, but also the programming structure was analyzed where the variables are positioned. By applying a similarity measuring method, a similarity value was produced for each program in the three aspects mentioned above. This research implements an Online Detection Plagiarism System (ODPS) providing a web-based user interface. This system can be applied independently for assignment analysis of Java programs. After three comparison experiments with other researches, the results demonstrated the ODPS has many advantages and good performance. Meanwhile, a combined approach is proven that it is better than a single approach for source codes of various styles.     

Viewing functions as token sequences to highlight similarities in source code

The detection of similarities in source code has applications not only in software re-engineering (to eliminate redundancies) but also in software plagiarism detection. This later can be a challenging problem since more or less extensive edits may have been performed on the original copy: insertion or removal of useless chunks of code, rewriting of expressions, transposition of code, inlining and outlining of functions, etc. In this paper, we propose a new similarity detection technique not only based on token sequence matching but also on the factorization of the function call graphs. The factorization process merges shared chunks (factors) of codes to cope, in particular, with inlining and outlining. The resulting call graph offers a view of the similarities with their nesting relations. It is useful to infer metrics quantifying similarity at a function level.     

中文语义角色标注的树核空间研究

使用基于树核函数的方法来进行语义角色标注,有效的树核空间的设计是影响系统性能的关键。探索树核空间在中文语义角色标注上的应用,考虑到同一谓词的各论元间的相互影响,提出多论元-谓词特征(AAPF)空间,并在此基础上提出了三种受平面特征启发的树核空间设计方法。基于中文PropBank语料的实验表明,加入一些重要平面特征信息的树核空间,性能有了明显的提高,分类精确率由90.96%提高到92.54%。最后使用复合核将特征启发的树核与特征向量结合起来,精确率达到95.21%,性能高于同类系统。     

代码相似性检测在程序设计教学中的应用

代码剽窃是程序设计课程中经常出现的一种作弊行为,检测剽窃的源代码、验证学生程序作业的原创性在教学中很重要。程序代码的相似度度量是剽窃检测的关键技术。本文首先对现有程序代码相似性检测技术进行研究,然后改进Halstead算法,提出了基于统计学方法程序代码相似性检测算法,最后对算法的有效性进行了实验分析。     

面向代码相似性检测的相似哈希改进方法

代码相似性检测（Code Similarity Detection）是软件工程领域的基本任务之一，其在剽窃检测、许可证违反检测、软件复用分析以及漏洞发现等方向均有重要作用.随着软件开源化的普及以及开源代码量的高速增长，开源代码在各个领域的应用日益频繁，给传统的代码相似性检测方法带来了新的挑战.现有的一些基于词法、语法、语义的检测方法存在算法较为复杂，对解析工具有依赖性，消耗资源高，可移植性差，候选对比项数量较多等问题，在大规模代码库上有一定局限性.基于相似哈希（simhash）指纹的代码相似性检测算法将代码降维至一个指纹，能够在数据集规模较大的情况下实现快速相似文件检索，并通过海明距离阈值控制匹配结果的相似度范围.通过实验对现有的基于代码行粒度的相似哈希算法进行验证，发现其在大规模数据集下存在行覆盖问题，即高频行特征对低频行特征的覆盖现象，导致结果精确度较低.受TF-IDF算法思想启发，针对上述问题创新性地提出了分语言行筛选优化方法，通过各种语言的行筛选器对代码文件行序列进行筛选，从而消除高频出现但语义信息包含较少的行对结果的影响.对改进前后方法进行一系列对比实验，结果表明改进后的方法在海明距离阈值为0至8的情况下均能够实现高精确度的相似文件对检索，阈值为8时在两个数据集下的精确度较改进前的方法分别提升了98.6%和52.2%.在本文建立的130万个开源项目，386486112个项目文件的大规模代码库上进行实验，验证了本文的方法能够快速检测出待测文件的相似文件结果，平均单个文件检测时间为0.43s，并取得了97%以上的检测精度.     

一种基于BP神经网络的代码相似性检测方法

如何有效地检测程序设计课程作业中的抄袭现象是一个重要的问题。传统的抄袭检测方法主要利用代码的属性或结构信息来度量代码之间的相似性。给出了一种基于误差反向传播(BP算法)多层前向神经网络的代码抄袭检测方法。提取程序之间的7种比较特征作为神经网络的输入,经过网络计算后得出程序的相似值,并将该值与抄袭决策阈值相比较以判定存在抄袭现象的程序集。实验结果表明,本方法具有很好的检测效果。     

基于XML的C代码抄袭检测算法

主要研究一种基于XML文本的C代码抄袭检测算法。将C程序源代码转化为XML文本,提取XML文本各行标记的属性,计算XML文本行的相似值,得到程序对的相似度。并对抄袭检测算法实行分类测试：按十二种抄袭手段分类;按复杂语义和复杂数据结构分类。测试结果对比验证了抄袭检测算法的有效性。     

基于树核函数的中文语义角色分类研究

该文探索了基于树核函数的中文语义角色分类,重点研究如何获取有效的结构化信息特征。在最小句法树结构的基础上,根据语义角色分类的特点,进一步定义了三种不同的句法结构,并使用复合核将基于树核和基于特征的方法结合。在中文PropBank语料上的结果表明,基于树核函数的方法在中文语义角色分类任务中能够取得较好的结果,精确率达到91.79%。同时,与基于特征方法的结合,基于树核函数的方法能够进一步提高前者性能,精确率达到94.28%,优于同类系统。     

AADL构件到RTLinux平台C代码的转换方法研究*

为了提高嵌入式软件开发的自动化程度,代码自动生成是一种值得采用的有效方法.在研究体系结构分析与设计语言(AADL)和RTLinux(real-time Linux)平台C代码的特性的基础上,提出了AADL构件到RTLinux平台C代码的转换规则;然后用一个实例实现了代码自动生成,从而验证了转换规则的有效性.     

基于结构比对的软件同源综合检测工具的设计与实现

文章对基于文本、Token和抽象语法树的同源性检测技术进行探讨,详细介绍了关于抽象语法树的同源性检测技术。同时,在对实际应用大量研究的基础上,文章着重介绍了源代码同源性检测系统的架构设计,以及引擎比对、比对结果分析和比对结果输出等主要功能模块,并对开发的系统进行了系统测试和分析,验证了算法的可行性。     

Kernel-based semantic relation detection and classification via enriched parse tree structure

This paper proposes a tree kernel method of semantic relation detection and classification(RDC) between named entities.It resolves two critical problems in previous tree kernel methods of RDC.First,a new tree kernel is presented to better capture the inherent structural information in a parse tree by enabling the standard convolution tree kernel with context-sensitiveness and approximate matching of sub-trees.Second,an enriched parse tree structure is proposed to well derive necessary structural informat...