Enlargement of vulnerable web applications for testing

There are two main kinds of vulnerable web applications, usual applications developed with a specific aim and applications which are vulnerable by design. On one hand, the usual applications are those that are used everywhere and on a daily basis, and where vulnerabilities are detected, and often mended, such as online banking systems, newspaper sites, or any other Web site. On the other hand, vulnerable by design web applications are developed for proper evaluation of web vulnerability scanners and for training in detecting web vulnerabilities. The main drawback of vulnerable by design web applications is that they used to include just a short set of well-known types of vulnerabilities, usually from famous classifications like the OWASP Top Ten. They do not include most of the types of web vulnerabilities. In this paper, an analysis and assessment of vulnerable web applications is conducted in order to select the applications that include the larger set of types of vulnerabilities. Then those applications are enlarged with more types of web vulnerabilities that vulnerable web applications do not include. Lastly, the new vulnerable web applications have been analyzed to check whether web vulnerability scanners are able to detect the new added vulnerabilities, those vulnerabilities that vulnerable by design web applications do not include. The results show that the tools are not very successful in detecting those vulnerabilities, less than well-known vulnerabilities.     

An empirical comparison of commercial and open-source web vulnerability scanners

Web vulnerability scanners (WVSs) are tools that can detect security vulnerabilities in web services. Although both commercial and open-source WVSs exist, their vulnerability detection capability and performance vary. In this article, we report on a comparative study to determine the vulnerability detection capabilities of eight WVSs (both open and commercial) using two vulnerable web applications: WebGoat and Damn vulnerable web application. The eight WVSs studied were: Acunetix; HP WebInspect; IBM AppScan; OWASP ZAP; Skipfish; Arachni; Vega; and Iron WASP. The performance was evaluated using multiple evaluation metrics: precision; recall; Youden index; OWASP web benchmark evaluation; and the web application security scanner evaluation criteria. The experimental results show that, while the commercial scanners are effective in detecting security vulnerabilities, some open-source scanners (such as ZAP and Skipfish) can also be effective. In summary, this study recommends improving the vulnerability detection capabilities of both the open-source and commercial scanners to enhance code coverage and the detection rate, and to reduce the number of false-positives.     

一种基于污点分析的文件型软件漏洞发现方法

基于黑盒测试思想的Fuzzing是漏洞分析的主要方法,但效率较低且不能分析未知格式.基于污点分析,提出一种针对文件型软件的漏洞发现新方法.利用污点分析寻找输入中能导致执行流到达脆弱点的字节,再改变这些字节产生新输入;同时根据污点信息产生特征码.利用插桩工具实现了原型系统,对三个真实漏洞进行了测试.实验结果表明该方法能有效发现漏洞,生成的测试用例远小于Fuzzing,且不依赖输入格式;特征码的误报率漏报率均较低.     

A threat model‐based approach to security testing

Software security issues have been a major concern in the cyberspace community, so a great deal of research on security testing has been performed, and various security testing techniques have been developed. Threat modeling provides a systematic way to identify threats that might compromise security, and it has been a well‐accepted practice by the industry, but test case generation from threat models has not been addressed yet. Thus, in this paper, we propose a threat model‐based security testing approach that automatically generates security test sequences from threat trees and transforms them into executable tests. The security testing approach we consider consists of three activities in large: building threat models with threat trees; generating security test sequences from threat trees; and creating executable test cases by considering valid and invalid inputs. To support our approach, we implemented security test generation techniques, and we also conducted an empirical study to assess the effectiveness of our approach. The results of our study show that our threat tree‐based approach is effective in exposing vulnerabilities. Copyright © 2012 John Wiley & Sons, Ltd.     

An empirical investigation into open source web applications’ implementation vulnerabilities

Current web applications have many inherent vulnerabilities; in fact, in 2008, over 63% of all documented vulnerabilities are for web applications. While many approaches have been proposed to address various web application vulnerability issues, there has not been a study to investigate whether these vulnerabilities share any common properties. In this paper, we use an approach similar to the Goal-Question-Metric approach to empirically investigate four questions regarding open source web applications vulnerabilities: What proportion of security vulnerabilities in web applications can be considered as implementation vulnerabilities? Are these vulnerabilities the result of interactions between web applications and external systems? What is the proportion of vulnerable lines of code within a web application? Are implementation vulnerabilities caused by implicit or explicit data flows? The results from the investigation show that implementation vulnerabilities dominate. They are caused through interactions between web applications and external systems. Furthermore, these vulnerabilities only contain explicit data flows, and are limited to relatively small sections of the source code.     

模拟攻击测试方式的漏洞检测系统的设计与实现

扫描方式的漏洞检测工具往往因为无法得到目标系统的准确信息而无法准确判断目标系统的安全状况,而模拟攻击测试方法可以准确判断目标系统是否存在测试的漏洞。大部分新漏洞发布的同时也会发布相应的测试程序,但是测试程序参数的复杂多样造成了集成的困难,把参数分为DR(运行时决定的类型)、DL(运行时查表决定的类型)和DV(默认值参数)三种类型,利用XML在数据结构描述方面的灵活性解决了这个问题。介绍了一个利用XML描述测试程序接口参数的模拟攻击测试方式的漏洞检测系统。     

SymFuzz:一种复杂路径条件下的漏洞检测技术

当前漏洞检测技术可以实现对小规模程序的快速检测,但对大型或路径条件复杂的程序进行检测时其效率低下。为实现复杂路径条件下的漏洞快速检测,文中提出了一种复杂路径条件下的漏洞检测技术SymFuzz。SymFuzz将导向式模糊测试技术与选择符号执行技术相结合,通过导向式模糊测试技术对程序路径进行过滤,利用选择符号执行技术对可能触发漏洞的路径进行求解。该技术首先通过静态分析获取程序漏洞信息;然后使用导向式模糊测试技术,快速生成可以覆盖漏洞函数的测试用例;最后对漏洞函数内可以触发漏洞的路径进行符号执行,生成触发程序漏洞的测试用例。文中基于AFL与S2E等开源项目实现了SymFuzz的原型系统。实验结果表明,SymFuzz与现有的模糊测试技术相比,在复杂路径条件下的漏洞检测效果提高显著。     

基于动态数据生成缺陷的XSS漏洞挖掘技术

XSS漏洞普遍存在于当前Web应用中,而且危害极其严重。随着Web2．0的到来,Web应用日趋大型化和复杂化,进一步为web漏洞的滋生提供了温床。针对大型web应用中复杂的数据组织结构,文章提出一种基于动态数据生成缺陷的XSS漏洞挖掘方法,能快速、高效地挖掘出大型Web应用中存在的XSS漏洞。同时,利用这一挖掘方法对web应用中存在的HTTPResponseSplitting漏洞、URLRedirection漏洞进行挖掘分析,都取得了非常显著的效果。     

Robustness testing for software components

Component-based development allows one to build software from existing components and promises to improve software reuse and reduce costs. For critical applications, the user of a component must ensure that it fits the requirements of the application. To achieve this, testing is a well-suited means when the source code of the components is not available. Robustness testing is a testing methodology to detect the vulnerabilities of a component under unexpected inputs or in a stressful environment. As components may fail differently in different states, we use a state machine based approach to robustness testing. First, a set of paths is generated to cover transitions of the state machine, and it is used by the test cases to bring the component into a specific control state. Second, method calls with invalid inputs are fed to the component in different states to test the robustness. By traversing the paths, the test cases cover more states and transitions compared to stateless API testing. We apply our approach to several components, including open source software, and compare our results with existing approaches.     

SimFuzz: Test case similarity directed deep fuzzing

Fuzzing is widely used to detect software vulnerabilities. Blackbox fuzzing does not require program source code. It mutates well-formed inputs to produce new ones. However, these new inputs usually do not exercise deep program semantics since the possibility that they can satisfy the conditions of a deep program state is low. As a result, blackbox fuzzing is often limited to identify vulnerabilities in input validation components of a program. Domain knowledge such as input specifications can be used to mitigate these limitations. However, it is often expensive to obtain such knowledge in practice. Whitebox fuzzing employs heavy analysis techniques, i.e., dynamic symbolic execution, to systematically generate test inputs and explore as many paths as possible. It is powerful to explore new program branches so as to identify more vulnerabilities. However, it has fundamental challenges such as unsolvable constraints and is difficult to scale to large programs due to path explosion. This paper proposes a novel fuzzing approach that aims to produce test inputs to explore deep program semantics effectively and efficiently. The fuzzing process comprises two stages. At the first stage, a traditional blackbox fuzzing approach is applied for test data generation. This process is guided by a novel test case similarity metric. At the second stage, a subset of the test inputs generated at the first stage is selected based on the test case similarity metric. Then, combination testing is applied on these selected test inputs to further generate new inputs. As a result, less redundant test inputs, i.e., inputs that just explore shallow program paths, are created at the first stage, and more distinct test inputs, i.e., inputs that explore deep program paths, are produced at the second stage. A prototype tool SimFuzz is developed and evaluated on real programs, and the experimental results are promising.     

Using combinatorial testing to build navigation graphs for dynamic web applications

Modelling a software system is often a challenging prerequisite to automatic test case generation. Modelling the navigation structure of a dynamic web application is particularly challenging because of the presence of a large number of pages that are created dynamically and the difficulty of reaching a dynamic page unless a set of appropriate input values are provided for the parameters. To address the first challenge, some form of abstraction is required to enable scalable modelling. For the second challenge, techniques are required to select appropriate input values for parameters and systematically combine them to reach new pages. This paper presents a combinatorial approach in building a navigation graph for dynamic web applications. The navigation graph can then be used to automatically generate test sequences for testing web applications. The novelty of our approach is twofold. First, we use an abstraction scheme to control the page explosion problem, where pages that are likely to have the same navigation behaviour are grouped together and are represented as a single node in the navigation graph. Second, assuming that values of individual parameters are supplied manually or generated from other techniques, we combine parameter values such that well‐defined combinatorial coverage of input parameter values is achieved. Using combinatorial coverage can significantly reduce the number of requests that have to be submitted while still achieving effective coverage of the navigation structure. We implement our combinatorial approach in a tool, Tansuo, and apply the tool on seven open‐source web applications. We evaluate the effectiveness of Tansuo's exploration process guided by t‐way coverage, for t = 1,2,3, with respect to code coverage, and find that the navigation structure exploration by Tansuo, in general, results in high code coverage (more than 80% statement coverage for most of our subject applications when dead code is removed). We compare Tansuo's effectiveness with two other navigation graph tools and find that Tansuo is more effective. Our empirical results indicate that using pairwise coverage in Tansuo results in the efficient generation of navigation graphs and effective exploration of dynamic web applications. Copyright © 2016 John Wiley & Sons, Ltd.     

Software assurance by bounded exhaustive testing

Bounded exhaustive testing (BET) is a verification technique in which software is automatically tested for all valid inputs up to specified size bounds. A particularly interesting case of BET arises in the context of systems that take structurally complex inputs. Early research suggests that the BET approach can reveal faults in small systems with inputs of low structural complexity, but its potential utility for larger systems with more complex input structures remains unclear. We set out to test its utility on one such system. We used Alloy and TestEra to generate inputs to test the Galileo dynamic fault tree analysis tool, for which we already had both a formal specification of the input space and a test oracle. An initial attempt to generate inputs using a straightforward translation of our specification to Alloy did not work well. The generator failed to generate inputs to meaningful bounds. We developed an approach in which we factored the specification, used TestEra to generate abstract inputs based on one factor, and passed the results through a postprocessor that reincorporated information from the second factor. Using this technique, we were able to generate test inputs to meaningful bounds, and the inputs revealed nontrivial faults in the Galileo implementation, our specification, and our oracle. Our results suggest that BET, combined with specification abstraction and factoring techniques, could become a valuable addition to our verification toolkit and that further investigation is warranted.     

Hunter:一种指令集体系结构无关的二进制级动态测试用例生成技术

动态测试用例生成技术是一类新兴的软件测试技术。由于使用该类技术无需任何人工干预,也无需验证人员具备任何专业知识,同时该类技术能够无误地发现程序错误,越来越多的研究者采用该技术查找预发布的二进制级软件错误。然而,已有的该类技术及其实现系统不具有可重定向性,只能处理面向某种特定指令集体系结构(ISA)的二进制代码,进行测试用例的生成与查错。本文提出了一种全新的指令集体系结构无关的二进制级动态测试用例生成技术,以及实现该技术的系统Hunter。与已有的动态测试用例生成技术不同,Hunter具有极强的可重定向性,可对任何指令集体系结构的二进制代码进行查错,定向地为其生成指向不同执行路径的测试用例。Hunter定义了一套元指令集体系结构(MetaISA),将在二进制代码执行过程中收集到的所有执行信息映射为MetaISA,并对生成的MetaISA序列进行符号化执行、约束收集、约束求解以及测试用例生成,从而使整个过程与ISA无关。我们实现了Hunter,将其重定向至32位x86、PowerPC和Sparc ISA,并使用该系统为6个含有已知错误的测试程序查错。实验结果表明,由于MetaISA的引入,只需很小的开销,Hunter系统即可容易且有效地重定向至不同的ISA,并且Hunter能够有效地发现面向32位x86、PowerPC和Sparc ISA编写的二进制应用中隐藏极深的错误。     

Predicting Cyber Risks through National Vulnerability Database

ABSTRACT

Software vulnerabilities are the major cause of cyber security problems. The National Vulnerability Database (NVD) is a public data source that maintains standardized information about reported software vulnerabilities. Since its inception in 1997, NVD has published information about more than 43,000 software vulnerabilities affecting more than 17,000 software applications. This information is potentially valuable in understanding trends and patterns in software vulnerabilities so that one can better manage the security of computer systems that are pestered by the ubiquitous software security flaws. In particular, one would like to be able to predict the likelihood that a piece of software contains a yet-to-be-discovered vulnerability, which must be taken into account in security management due to the increasing trend in zero-day attacks. We conducted an empirical study on applying data-mining techniques on NVD data with the objective of predicting the time to next vulnerability for a given software application. We experimented with various features constructed using the information available in NVD and applied various machine learning algorithms to examine the predictive power of the data. Our results show that the data in NVD generally have poor prediction capability, with the exception of a few vendors and software applications. We suggest possible reasons for why the NVD data have not produced a reasonable prediction model for time to next vulnerability with our current approach, and suggest alternative ways in which the data in NVD can be used for the purpose of risk estimation.     

Assessment Of Vulnerability Scanners

Securing information over the Internet can be facilitated by a multitude of security technologies. Technologies such as intrusion detection systems, anti-virus software, firewalls and crypto devices have all contributed significantly to the security of information. This article focuses on vulnerability scanners (VSs). A VS has a vulnerability database containing hundreds of known vulnerabilities, which it scans for. VSs do not scan for the same type of vulnerabilities since the vulnerability databases for each VS differ extensively. In addition, there is an overlap of vulnerabilities between the vulnerability databases of various VSs. The concept of harmonised vulnerability categories is introduced in this paper. Harmonised vulnerability categories consider the entire scope of known vulnerabilities across various VSs in a bid to act as a mediator in assessing the vulnerabilities that VSs scan for. Harmonised vulnerability categories, thus, are used to do an objective assessment of the vulnerability database of a VS.