面向漏洞检测模型的强化学习式对抗攻击方法

基于深度学习的代码漏洞检测模型因其检测效率高和精度准的优势,逐步成为检测软件漏洞的重要方法,并在代码托管平台Github的代码审计服务中发挥重要作用.然而,深度神经网络已被证明容易受到对抗攻击的干扰,这导致基于深度学习的漏洞检测模型存在遭受攻击,降低检测准确率的风险.因此,构建针对漏洞检测模型的对抗攻击,不仅可以发掘此类模型的安全缺陷,而且有助于评估模型的鲁棒性,进而通过相应的方法提升模型性能.但现有的面向漏洞检测模型的对抗攻击方法,依赖于通用的代码转换工具,并未提出针对性的代码扰动操作和决策算法,因此难以生成有效的对抗样本,且对抗样本的合法性依赖于人工检查.针对上述问题,提出了一种面向漏洞检测模型的强化学习式对抗攻击方法.本方法首先设计了一系列语义约束且漏洞保留的代码扰动操作作为扰动集合;其次,将具备漏洞的代码样本作为输入,利用强化学习模型选取具体的扰动操作序列.最后,根据代码样本的语法树节点类型寻找扰动的潜在位置,进行代码转换,从而生成对抗样本.本文基于SARD和NVD构建了两个实验数据集共14,278个代码样本并以此训练了四个具备不同特点的漏洞检测模型作为攻击目标.针对每个目标模型,训练了一个强化学习网络进行对抗攻击.结果显示,本文的攻击方法导致模型的召回率降低了74.34%,攻击成功率达到96.71%,相较基线方法,攻击成功率平均提升了68.76%.实验证明了当前的漏洞检测模型存在被攻击的风险,需要进一步研究提升模型的鲁棒性.     

深度学习在代码表征中的应用综述

代码表征是对代码数值化的一种技术,把代码映射为一组连续的实值向量,提取隐藏在代码内部的属性,辅助程序员生成或分析代码,是代码克隆、代码推荐、代码剽窃等软件工程任务的核心技术和研究热点。研究者们对代码表征方面进行了一系列研究,根据源代码抽取信息的方式,分为基于文本的表征、基于语法的表征、基于语义的表征和基于功能的表征;根据表征粒度的大小,分为基于词汇的表征、基于语句的表征、基于函数的表征等不同等级;根据表征方法的不同,分为基于统计的模型、基于自然语言的模型和基于深度学习的模型。对近几年基于深度学习的代码表征研究进展进行了综述,并从表征粒度、表征层次、表征模型、应用场景等方面对现有工作进行了概括、比较和分析。对基于深度学习的代码表征的未来发展趋势进行分析和展望。     

软件源代码中的代码克隆现象及其检测方法

如果软件源程序中的一个代码段和同一程序中的另一个代码段在结构或语义上类似,这些代码段就成了代码克隆.概述代码克隆存在的各种形式,分析代码克隆产生的原因,并在概括了代码克隆检测的一般过程以后进一步阐述两类代码克隆检测方法:基于语义抽象树的检测方法和基于Token序列的检测方法.     

基于深度置信网络的Android恶意软件检测

针对采用重打包和代码混淆技术的Android恶意软件检测准确率低的问题,提出了一种基于深度置信网络的Android恶意软件检测算法。通过自动化提取Android应用软件的特征,构建对应的特征向量,训练基于深度置信网络的深度学习模型,实现了一种新的基于深度置信网络的Android恶意软件检测算法。实验结果表明,基于深度置信网络的深度学习模型可以更好地表征Android恶意软件,其检测效果也明显优于传统的机器学习模型。     

基于深度学习的计算机软件安全性能检测方法

由于计算机软件的大范围开发及开发者编写的代码逐渐简单化,导致软件存在漏洞,影响其安全性能,故提出基于深度学习的计算机软件安全性能检测方法。提取计算机软件代码数据,预处理代码数据作为模型输入参数,基于深度学习构建一个软件安全漏洞检测模型。实验结果表明,该方法的检测漏报率为0.04%、误报率为2.71%、检测时间为254 s,具有良好的软件安全漏洞检测能力。     

基于深度学习的表面缺陷检测方法综述

近年来,基于深度学习的表面缺陷检测技术广泛应用在各种工业场景中.本文对近年来基于深度学习的表面缺陷检测方法进行了梳理,根据数据标签的不同将其分为全监督学习模型方法、无监督学习模型方法和其他方法三大类,并对各种典型方法进一步细分归类和对比分析,总结了每种方法的优缺点和应用场景.本文探讨了表面缺陷检测中三个关键问题,介绍了...     

代码克隆检测研究进展

代码克隆（code clone）,是指存在于代码库中两个及以上相同或者相似的源代码片段.代码克隆相关问题是软件工程领域研究的重要课题.代码克隆是软件开发中的常见现象,它能够提高效率,产生一定的正面效益.但是研究表明,代码克隆也会对软件系统的开发、维护产生负面的影响,包括降低软件稳定性,造成代码库冗余和软件缺陷传播等.代码克隆检测技术旨在寻找检测代码克隆的自动化方法,从而用较低成本减少代码克隆的负面效应.研究者们在代码克隆检测方面获得了一系列的检测技术成果,根据这些技术利用源代码信息的程度不同,可以将它们分为基于文本、词汇、语法、语义4个层次.现有的检测技术针对文本相似的克隆取得了有效的检测结果,但同时也面临着更高抽象层次克隆的挑战,亟待更先进的理论、技术来解决.着重从源代码表征方式角度入手,对近年来代码克隆检测研究进展进行了梳理和总结.主要内容包括：（1）根据源代码表征方式阐述并归类了现有的克隆检测方法;（2）总结了模型评估中使用的实验验证方法与性能评估指标;（3）从科学性、实用性和技术难点这3个方面归纳总结了代码克隆研究的关键问题,围绕数据标注、表征方法、模型构建和工程实践4个方面,阐述了问题的可能解决思路和研究的未来发展趋势.     

基于字符串的代码克隆检测方法的分析

克隆代码是指在软件源程序中存在的相同或相似的代码片段。克隆代码在很多软件工程中,例如程序理解,代码质量分析,剽窃检测,漏洞查找和病毒检测,都需要通过找出语义或语法上相似的代码片段来实现。目前常用的检测方法有四种:基于文本(text-based)的检测,基于字符序列(token-based)的检测,基于语法树(tree-based)的检测和基于关系图(PDG-based)的检测。基于字符序列的克隆检测首先对源程序进行预处理转换,再经过匹配算法得到克隆检测结果。克隆代码的检测是软件分析的一个重要的部分。     

基于字符串的代码克隆检测方法的分析

克隆代码是指在软件源程序中存在的相同或相似的代码片段。克隆代码在很多软件工程中,例如程序理解,代码质量分析,剽窃检测,漏洞查找和病毒检测,都需要通过找出语义或语法上相似的代码片段来实现,目前常用的检测方法有四种：基于文本（text—based）检测,基于字符序列（token-based）的检测,基于语法树（tree-based）的检测和基于关系图（PDG—based）的检测。基于字符序列的克隆检测首先对源程序进行预处理转换,再经过匹配算法得到克隆检测结果：克隆代码的检测是软件分析的一个重要的部分。     

一种基于特征矩阵的软件脆弱性代码克隆检测方法

提出了一种基于特征矩阵的软件代码克隆检测方法.在此基础上,实现了针对多类脆弱性的检测模型.基于对脆弱代码的语法和语义特征分析,从语法分析树抽取特定的关键节点类型描述不同的脆弱性类型,将4种基本克隆类型细化拓展到更多类,通过遍历代码片段对应的语法分析树中关键节点的数量,构造对应的特征矩阵.从公开漏洞数据库中抽取部分实例作为基本知识库,通过对代码进行基于多种克隆类型的聚类计算,达到了从被测软件代码中检测脆弱代码的目的.与基于单一特征向量的检测方法相比,对脆弱性特征的描述更加精确,更具有针对性,并且弥补了形式化检测方法在脆弱性类型覆盖能力上的不足.在对android-kernel代码的测试中发现了9个脆弱性.对不同规模软件代码的测试结果表明,该方法的时间开销和被测代码规模成线性关系.     

基于软件漏洞的克隆代码稳定性评估

针对克隆代码与非克隆代码产生"漏洞"倾向性的问题进行了研究,基于"漏洞"对不同类型克隆和非克隆代码进行了比较分析。首先提取软件系统中具有漏洞的代码,并使用克隆检测工具检测出软件的克隆代码;其次分别提取能够产生"漏洞"的克隆和非克隆代码,并分别计算不同克隆类型和非克隆的BOC漏洞密度和LOC漏洞密度;最后对type-1、pure type-2、pure-type3的克隆和非克隆漏洞密度进行了对比分析,并对代码中产生的"漏洞"类型进行分类分析,使用曼—惠特尼检验(WMM)验证了结果的有效性。实验结果表明type-1类型的克隆更容易产生"漏洞",pure type-3类型的克隆引入漏洞的几率相对较小。研究还得出在克隆和非克隆代码中分别存在出现频率较高的"漏洞"集合,增加了对克隆特性的理解,帮助软件设计和开发人员减少代码克隆对软件造成的负面影响。     

Effective approaches to combining lexical and syntactical information for code summarization

Natural language summaries of source codes are important during software development and maintenance. Recently, deep learning based models have achieved good performance on the task of automatic code summarization, which encode token sequence or abstract syntax tree (AST) of code with neural networks. However, there has been little work on the efficient combination of lexical and syntactical information of code for better summarization quality. In this paper, we propose two general and effective approaches to leveraging both types of information: a convolutional neural network that aims to better extract vector representation of AST node for downstream models; and a Switch Network that learns an adaptive weight vector to combine different code representations for summary generation. We integrate these approaches into a comprehensive code summarization model, which includes a sequential encoder for token sequence of code and a tree based encoder for its AST. We evaluate our model on a large Java dataset. The experimental results show that our model outperforms several state-of-the-art models on various metrics, and the proposed approaches contribute a lot to the improvements.     

SeByte: Scalable clone and similarity search for bytecode

While source code clone detection is a well-established research area, finding similar code fragments in binary and other intermediate code representations has been not yet that widely studied. In this paper, we introduce SeByte, a bytecode clone detection and search model that applies semantic-enabled token matching. It is developed based on the idea of relaxation on the code fingerprints. This approach separates the input content based on the types of tokens into different dimensions, with each dimension representing the input content from a specific point of view. Following this approach, SeByte compares each dimension separately and independently which we refer to as multi-dimensional comparison in our research. As the similarity search function we use a well-known measure that supports our multi-dimensional comparison heuristic, the Jaccard similarity coefficient. Our preliminary study shows that SeByte can detect clones that are missed by existing approaches due to the differences in the input data and the search algorithm. We then further exploit the model to build a scalable bytecode clone search engine. This extension meets the requirements of a classical search engine including the ranking of result sets. Our evaluation with a large dataset of 500,000 compiled Java classes, which we extracted from the six most recent versions of the Eclipse IDE, showed that our SeByte search is not only scalable but also capable of providing a reliable ranking.     

Local outlier factor and stronger one class classifier based hierarchical model for detection of attacks in network intrusion detection dataset

Identification of attacks by a network intrusion detection system (NIDS) is an important task. In signature or rule based detection, the previously encountered attacks are modeled, and signatures/rules are extracted. These rules are used to detect such attacks in future, but in anomaly or outlier detection system, the normal network traffic is modeled. Any deviation from the normal model is deemed to be an outlier/ attack. Data mining and machine learning techniques are widely used in offline NIDS. Unsupervised and supervised learning techniques differ the way NIDS dataset is treated. The characteristic features of unsupervised and supervised learning are finding patterns in data, detecting outliers, and determining a learned function for input features, generalizing the data instances respectively. The intuition is that if these two techniques are combined, better performance may be obtained. Hence, in this paper the advantages of unsupervised and supervised techniques are inherited in the proposed hierarchical model and devised into three stages to detect attacks in NIDS dataset. NIDS dataset is clustered using Dirichlet process (DP) clustering based on the underlying data distribution. Iteratively on each cluster, local denser areas are identified using local outlier factor (LOF) which in turn is discretized into four bins of separation based on LOF score. Further, in each bin the normal data instances are modeled using one class classifier (OCC). A combination of Density Estimation method, Reconstruction method, and Boundary methods are used for OCC model. A product rule combination of the threemethods takes into consideration the strengths of each method in building a stronger OCC model. Any deviation from this model is considered as an attack. Experiments are conducted on KDD CUP’99 and SSENet-2011 datasets. The results show that the proposed model is able to identify attacks with higher detection rate and low false alarms.     

基于Token编辑距离检测克隆代码

针对当前Type-3克隆代码检测工具较少、效率偏低等问题,提出了一种基于Token的能有效检测Type-3克隆代码的检测方法。该方法同时能有效检测Type-1和Type-2克隆代码。首先将源代码Token化得到特定代码粒度的Token串,其次将所有Token串的定长子串进行映射,在对映射信息进行查询的基础上,利用编辑距离算法确定克隆对,然后通过并查集算法快速构建克隆群,最终反馈克隆代码信息。实现了原型工具FClones,利用基于代码突变的框架对工具进行了评价,并与领域内较优秀的两款工具NiCad及SimCad进行了对比。实验结果表明,FClones在检测三类克隆代码时查全率均不低于95%,查准率均不低于98%,能更好地检测Type-3克隆代码。     

Learning dataset representation for automatic machine learning algorithm selection

The algorithm selection problem is defined as identifying the best-performing machine learning (ML) algorithm for a given combination of dataset, task, and evaluation measure. The human expertise required to evaluate the increasing number of ML algorithms available has resulted in the need to automate the algorithm selection task. Various approaches have emerged to handle the automatic algorithm selection challenge, including meta-learning. Meta-learning is a popular approach that leverages accumulated experience for future learning and typically involves dataset characterization. Existing meta-learning methods often represent a dataset using predefined features and thus cannot be generalized across different ML tasks, or alternatively, learn a dataset’s representation in a supervised manner and therefore are unable to deal with unsupervised tasks. In this study, we propose a novel learning-based task-agnostic method for producing dataset representations. Then, we introduce TRIO, a meta-learning approach, that utilizes the proposed dataset representations to accurately recommend top-performing algorithms for previously unseen datasets. TRIO first learns graphical representations for the datasets, using four tools to learn the latent interactions among dataset instances and then utilizes a graph convolutional neural network technique to extract embedding representations from the graphs obtained. We extensively evaluate the effectiveness of our approach on 337 datasets and 195 ML algorithms, demonstrating that TRIO significantly outperforms state-of-the-art methods for algorithm selection for both supervised (classification and regression) and unsupervised (clustering) tasks.

     

基于数据增强和弱监督对抗训练的中文事件检测

当前的事件检测模型严重依赖于人工标注的数据,在标注数据规模有限的情况下,事件检测任务中基于完全监督方法的深度学习模型经常会出现过拟合的问题,而基于弱监督学习的使用自动标注数据代替耗时的人工标注数据的方法又常常依赖于复杂的预定义规则。为了解决上述问题,就中文事件检测任务提出了一种基于BERT的混合文本对抗训练（BMAD）方法。所提方法基于数据增强和对抗学习设定了弱监督学习场景,并采用跨度抽取模型来完成事件检测任务。首先,为改善数据不足的问题,采用回译、Mix-Text等数据增强方法来增强数据并为事件检测任务创建弱监督学习场景;然后,使用一种对抗训练机制进行噪声学习,力求最大限度地生成近似真实样本的生成样本,并最终提高整个模型的鲁棒性。在广泛使用的真实数据集自动文档抽取（ACE）2005上进行实验,结果表明相较于NPN、TLNN、HCBNN等算法,所提方法在F1分数上获取了至少0.84个百分点的提升。     

克隆代码分析方法研究

针对已有克隆代码检测工具只输出克隆组形式的检测结果,而无法分析克隆代码对软件质量的影响问题,提出危害软件质量的关键克隆代码的识别方法。首先,定义了克隆代码的统一表示形式,使之可以分析各种克隆检测工具的检测结果;接下来,解析源程序和克隆检测结果,识别标识符命名不一致性潜在缺陷;然后,定义了克隆关联图,在此基础上检测跨越多个实现不同功能的文件、危害软件可维护性的克隆代码;最后,对检测结果进行可视化统计分析。本文的克隆代码分析工具被应用于分析开源代码httpd,检测出了1组标识符命名不一致的克隆代码和44组危害软件可维护性的关键克隆类,实验结果表明,本文方法可以有效辅助软件开发和维护人员分析、维护克隆代码。     

面向网络社交媒体的少样本新冠谣言检测

在社交媒体上发布和传播有关新冠的谣言对民生、经济、社会等都产生了严重影响,因此通过机器学习和人工智能技术开展新冠谣言检测具有重要的研究价值和社会意义.现有谣言检测研究,一般假定进行建模和预测的事件已有充足的有标签数据,但对于新冠这类突发事件,由于可训练样本较少,所以此类模型存在局限性.该文聚焦少样本谣言检测问题,旨在使...     

A systematic literature review on the use of machine learning in code clone research

Context:Research related to code clones includes detection of clones in software systems, analysis, visualization and management of clones. Detection of semantic clones and management of clones have attracted use of machine learning techniques in code clone related research.Objective:The aim of this study is to report the extent of machine learning usage in code clone related research areas.Method:The paper uses a systematic review method to report the use of machine learning in research related to code clones. The study considers a comprehensive set of 57 articles published in leading conferences, workshops and journals.Results:Code clone related research using machine learning techniques is classified into different categories. Machine learning and deep learning algorithms used in the code clone research are reported. The datasets, features used to train machine learning models and metrics used to evaluate machine learning algorithms are reported. The comparative results of various machine learning algorithms presented in primary studies are reported.Conclusion:The research will help to identify the status of using machine learning in different code clone related research areas. We identify the need of more empirical studies to assess the benefits of machine learning in code clone research and give recommendations for future research.