<Emphasis Type="Italic">RAMBUTANS</Emphasis>: automatic AOP-specific test generation tool

Aspect-oriented programming (AOP) is a programmatic methodology to handle better modularized code by separating crosscutting concerns from the traditional abstraction boundaries. Automated testing, as one of the most demanding needs of the software development to reduce both human effort and costs, is a delicate issue in testing aspect-oriented programs. Prior studies in the automated test generation for aspect-oriented programs have been very limited with respect to the need for both adequate tool support and capability concerning effectiveness and efficiency. This paper describes a new AOP-specific tool for testing aspect-oriented programs, called RAMBUTANS. The RAMBUTANS tool uses a directed random testing technique that is especially well suited for generating tests for aspectual features in AspectJ. The directed random aspect of the tool is parameterized by associating weights to aspects, advice, methods, and classes by controlling object and joint point creations during the test generation process. We present a comprehensive empirical evaluation of our tool against the current AOP test generation approaches on three industrial aspect-oriented projects. The results of the experimental and statistical tests showed that RAMBUTANS tool produces test suites that have higher fault-detection capability and efficiency for AspectJ-like programs.     

Empirical evaluation of the fault detection effectiveness and test effort efficiency of the automated AOP testing approaches

Context

Testing process is a time-consuming, expensive, and labor-intensive activity in any software setting including aspect-oriented programming (AOP). To reduce the testing costs, human effort, and to achieve the improvements in both quality and productivity of AOP, it is desirable to automate testing of aspect-oriented programs as much as possible.

Objective

In recent past years, a lot of research effort has been devoted to testing aspect-oriented programs but less effort has been dedicated to the automated AOP testing. This denotes that the current research on automated AOP testing is not sufficient and is still in a stage of infancy. In order to advance the state of the research in this area and to provide testers of AOP-based projects with a comparison basis, a detailed evaluation of the current automated AOP testing approaches in a thorough and experimental manner is required. Thus, the objective of this paper is to provide such evaluation of the current approaches.

Method

In this paper, we carry out an empirical study based on mutation analysis to examine four (namely Wrasp, Aspectra, Raspect, and EAT) existing automated AOP testing approaches, particularly their underlying test input generation and selection strategies, with regard to fault detection effectiveness. In addition, the approaches are compared in terms of required effort in detecting faults as part of efficiency evaluation.

Results

The experimental results and comparison provided insights into the effectiveness and efficiency of automated AOP testing with their respective strengths and weaknesses. Results showed that EAT is more effective than the other automated AOP testing approaches but not significant for all approaches. EAT was found to be significantly better than Wrasp at 95% confidence level (i.e. p < 0.05), but not significantly better than Aspectra or Raspect. Concerning the test effort efficiency, Wrasp was significantly (p < 0.05) efficient with requiring the lowest amount of test effort compared to the other approaches. Whereas, EAT showed to be not very efficient by recording the highest amount of test effort.

Conclusion

This implies that EAT can currently be the most effective automated AOP testing approach but perhaps less efficient. More generally, search-based testing (as underlying strategy of EAT approach) might achieve better effectiveness but at the cost of greater test effort compared to random testing (as underlying strategy of other approaches).     

面向方面的程序设计综述

现在大多数的软件项目都选择OOP，但OOP技术也有其局限性，它对某些需求并不能很好地进行描述。本文介绍了一种全新的程序设计方法一面向方面的程序设计方法，这种程序设计方法可以有效地描述OOP方法不能描述的横切关注点；介绍了面向方面程序设计的基本思想以及面向方面程序设计语言AspectJ的基本概念及面向方面软件开发方法。     

An approach and tool for measurement of state variable based data-flow test coverage for aspect-oriented programs

ContextData-flow testing approaches have been used for procedural and object-oriented programs, and shown to be effective in detecting faults. However, few such approaches have been evaluated for aspect-oriented programs. In such programs, data-flow interactions can occur between base classes and aspects, which can affect the behavior of both. Faults resulting from such interactions are hard to detect unless the interactions are specifically targeted during testing.ObjectiveThis paper presents an approach and tool implementation for measuring data-flow coverage based on state variables defined in base classes or aspects in AspectJ programs. The paper also reports on an empirical study that compares the cost and effectiveness of data-flow test criteria that are based on state variables with two control-flow criteria.MethodEffectiveness of the criteria was evaluated for various fault types. Cost-effectiveness of test suites that cover all state variable definition-use associations (DUAs) was evaluated for three coverage levels: 100%, 90%, and 80%.ResultsThe effort needed to obtain a test case that achieves data-flow coverage is higher than the effort needed to obtain a test case that covers a block or a branch in an advised class. Covering certain data flow associations requires more effort than for other types of data flow associations. The data-flow test criteria based on state variables of a base-class are in general more effective than control-flow criteria.ConclusionsOverall, it is cost-effective to obtain test suites at the 90% coverage level of data-flow criteria.     

面向方面软件重构等价性形式化证明方法

软件重构在不改变程序行为的情况下通过对代码进行小的改进以提升设计,使之更容易理解和维护,面向方面的程序设计是软件开发的新技术,为了有效实施面向方面的软件重构,需要开发者识别面向方面程序的转化规则。然而,由于使用的AOP语言没有形式化的语义定义,难以确认转化和重构的程序运行行为。本文对MCI操作语义的面向方面的扩展使之支持程序的方面特征的描述,定义了两个程序的观测等价,讨论了AspectJ的形式语义模型的建立,在MCI的语义下形式化地精确证明了Add Before-executing编程规则的观测等价性,其基本原理和方法可以适用于其他规则的证明,通过上述工作提出了面向方面重构的程序和它的面向对象程序原型等价性的证明方法。     

On generating mutants for AspectJ programs

ContextMutation analysis has been widely used in research studies to evaluate the effectiveness of test suites and testing techniques. Faulty versions (i.e., mutants) of a program are generated such that each mutant contains one seeded fault. The mutation score provides a measure of effectiveness.ObjectiveWe study three problems with the use of mutation analysis for testing AspectJ programs:

• •The manual identification and removal of equivalent mutants is difficult and time consuming. We calculate the percentage of equivalent mutants generated for benchmark AspectJ programs using available mutation tools.
• •The generated mutants need to cover the various fault types described in the literature on fault models for AspectJ programs. We measure the distribution of the mutants generated using available mutation tools with respect to the AspectJ fault types.
• •We measure the difficulty of killing the generated mutants.

We propose the use of simple analysis of the subject programs to prevent the generation of some equivalent mutants.MethodWe revised existing AspectJ fault models and presented a fault model that removes the problems in existing fault models, such as overlapping between fault types and missing fault types. We also defined three new fault types that occur due to incorrect data-flow interactions occurring in AspectJ programs. We used three mutation tools: AjMutator, Proteum/AJ, and MuJava on three AspectJ programs. To measure the difficulty of killing the mutants created using a mutation operator, we compared the average number of the mutants killed by 10 test suites that satisfy block coverage criterion.ResultsA high percentage of the mutants are equivalent. The mutation tools do not cover all the fault types. Only 4 out of 27 operators generated mutants that were easy to kill.ConclusionsOur analysis approach removed about 80% of the equivalent mutants. Higher order mutation is needed to cover all the fault types.     

Tool-Supported Refactoring of Existing Object-Oriented Code into Aspects

Aspect-oriented programming (AOP) provides mechanisms for the separation of crosscutting concerns - functionalities scattered through the system and tangled with the base code. Existing systems are a natural testbed for the AOP approach since they often contain several crosscutting concerns which could not be modularized using traditional programming constructs. This paper presents an automated approach to the problem of migrating systems developed according to the object-oriented programming (OOP) paradigm into aspect-oriented programming (AOP). A simple set of six refactorings has been defined to transform OOP to AOP and has been implemented in the AOP-migrator tool, an Eclipse plug-in. A set of enabling transformations from OOP to OOP complement the initial set of refactorings. The paper presents the results of four case studies, which use the approach to migrate selected crosscutting concerns from medium-sized Java programs (in the range of 10K to 40K lines of code) into equivalent programs in AspectJ. The case study results show the feasibility of the migration and indicate the importance of the enabling transformations as a preprocessing step     

Source code size estimation approaches for object-oriented systems from UML class diagrams: A comparative study

BackgroundSource code size in terms of SLOC (source lines of code) is the input of many parametric software effort estimation models. However, it is unavailable at the early phase of software development.ObjectiveWe investigate the accuracy of early SLOC estimation approaches for an object-oriented system using the information collected from its UML class diagram available at the early software development phase.MethodWe use different modeling techniques to build the prediction models for investigating the accuracy of six types of metrics to estimate SLOC. The used techniques include linear models, non-linear models, rule/tree-based models, and instance-based models. The investigated metrics are class diagram metrics, predictive object points, object-oriented project size metric, fast&&serious class points, objective class points, and object-oriented function points.ResultsBased on 100 open-source Java systems, we find that the prediction model built using object-oriented project size metric and ordinary least square regression with a logarithmic transformation achieves the highest accuracy (mean MMRE = 0.19 and mean Pred(25) = 0.74).ConclusionWe should use object-oriented project size metric and ordinary least square regression with a logarithmic transformation to build a simple, accurate, and comprehensible SLOC estimation model.     

Execution levels for aspect-oriented programming: Design,semantics, implementations and applications

In aspect-oriented programming (AOP) languages, advice evaluation is usually considered as part of the base program evaluation. This is also the case for certain pointcuts, such as if pointcuts in AspectJ, or simply all pointcuts in higher-order aspect languages like AspectScheme. While viewing aspects as part of base level computation clearly distinguishes AOP from reflection, it also comes at a price: because aspects observe base level computation, evaluating pointcuts and advice at the base level can trigger infinite regression. To avoid these pitfalls, aspect languages propose ad-hoc mechanisms, which increase the complexity for programmers while being insufficient in many cases. After shedding light on the many facets of the issue, this paper proposes to clarify the situation by introducing levels of execution in the programming language, thereby allowing aspects to observe and run at specific, possibly different, levels. We adopt a defensive default that avoids infinite regression, and gives advanced programmers the means to override this default using level-shifting operators. We then study execution levels both in practice and in theory. First, we study the relevance of the issues addressed by execution levels in existing aspect-oriented programs. We then formalize the semantics of execution levels and prove that the default semantics is indeed free of a certain form of infinite regression, which we call aspect loops. Finally, we report on existing implementations of execution levels for aspect-oriented extensions of Scheme, JavaScript and Java, discussing their implementation techniques and current applications.     

面向方面的程序设计:概念、实现与未来

面向对象程序设计（OOP）的出现极大地提高了软件的可读性、可维护性和可复用性,但OOP并没有彻底解决软件的维护和复用问题,本质上是因为OOP难以模块化处理软件系统的横切关注点,以至于业务逻辑代码和其它代码（如日志记录、安全验证等）相互“纠缠”在一起,使得程序难以维护和复用。面向方面的程序设计（AOP）由此诞生,它最主要的设计原则是模块化系统的横切关注点,将业务代码与其它代码解耦,在更高的抽象和分解层次上,进一步提高了软件的可维护性、可复用性和可扩展性。首先深度探索了AOP的起源,然后阐述了AOP的本质,接着介绍了AOP的基本概念并以最为流行的实现语言AspectJ为例,剖析了AOP的实现机制,最后讨论了AOP在软件开发、测试、分析、度量和重构等领域的研究状况。     

Goal-oriented dynamic test generation

ContextMemory safety errors such as buffer overflow vulnerabilities are one of the most serious classes of security threats. Detecting and removing such security errors are important tasks of software testing for improving the quality and reliability of software in practice.ObjectiveThis paper presents a goal-oriented testing approach for effectively and efficiently exploring security vulnerability errors. A goal is a potential safety violation and the testing approach is to automatically generate test inputs to uncover the violation.MethodWe use type inference analysis to diagnose potential safety violations and dynamic symbolic execution to perform test input generation. A major challenge facing dynamic symbolic execution in such application is the combinatorial explosion of the path space. To address this fundamental scalability issue, we employ data dependence analysis to identify a root cause leading to the execution of the goal and propose a path exploration algorithm to guide dynamic symbolic execution for effectively discovering the goal.ResultsTo evaluate the effectiveness of our proposed approach, we conducted experiments against 23 buffer overflow vulnerabilities. We observed a significant improvement of our proposed algorithm over two widely adopted search algorithms. Specifically, our algorithm discovered security vulnerability errors within a matter of a few seconds, whereas the two baseline algorithms failed even after 30 min of testing on a number of test subjects.ConclusionThe experimental results highlight the potential of utilizing data dependence analysis to address the combinatorial path space explosion issue faced by dynamic symbolic execution for effective security testing.     

New AspectJ Pointcuts for Integer Overflow and Underflow Detection

ABSTRACT

Aspect-oriented Programming (AOP) appears to be a promising paradigm for software security hardening. Using AOP, security experts can be responsible for coding security properties, and developers can concentrate on the basic functionality of the program. AspectJ extends the Java programming language to implement crosscutting concerns modularly in general. In this paper, we have extended AspectJ with new pointcuts in order to detect integer overflows and underflows in Java. Integer overflows and underflows in Java occur silently without throwing an exception. A malicious user can exploit them to produce a security breach. Hence, we implement new pointcuts: addition, multiplication, and subtraction that allow to write advices around integer arithmetic operations to detect integer overflow and underflow and consequently prevent considerable number of security breaches.     

基于AOP的面向对象程序的单元测试

单元测试被视为横切关注点,在软件测试过程中很重要。而面向方面编程(AOP)的主要目标就是分离横切关注点,因此,单元测试非常适合用AOP来解决。该文提出了在对面向对象程序进行单元测试时遇到的问题,并用AOP方法加以解决,同时比较了传统方法和AOP方法进行单元测试的优缺点。     

Empirical evaluation of the effects of mixed project data on learning defect predictors

ContextDefect prediction research mostly focus on optimizing the performance of models that are constructed for isolated projects (i.e. within project (WP)) through retrospective analyses. On the other hand, recent studies try to utilize data across projects (i.e. cross project (CP)) for building defect prediction models for new projects. There are no cases where the combination of within and cross (i.e. mixed) project data are used together.ObjectiveOur goal is to investigate the merits of using mixed project data for binary defect prediction. Specifically, we want to check whether it is feasible, in terms of defect detection performance, to use data from other projects for the cases (i) when there is an existing within project history and (ii) when there are limited within project data.MethodWe use data from 73 versions of 41 projects that are publicly available. We simulate the two above-mentioned cases, and compare the performances of naive Bayes classifiers by using within project data vs. mixed project data.ResultsFor the first case, we find that the performance of mixed project predictors significantly improves over full within project predictors (p-value < 0.001), however the effect size is small (Hedges′ g = 0.25). For the second case, we found that mixed project predictors are comparable to full within project predictors, using only 10% of available within project data (p-value = 0.002, g = 0.17).ConclusionWe conclude that the extra effort associated with collecting data from other projects is not feasible in terms of practical performance improvement when there is already an established within project defect predictor using full project history. However, when there is limited project history, e.g. early phases of development, mixed project predictions are justifiable as they perform as good as full within project models.     

Hands-free administration of subjective workload scales: Acceptability in a surgical training environment

IntroductionSubjective workload measures are usually administered in a visual–manual format, either electronically or by paper and pencil. However, vocal responses to spoken queries may sometimes be preferable, for example when experimental manipulations require continuous manual responding or when participants have certain sensory/motor impairments. In the present study, we evaluated the acceptability of the hands-free administration of two subjective workload questionnaires – the NASA Task Load Index (NASA-TLX) and the Multiple Resources Questionnaire (MRQ) – in a surgical training environment where manual responding is often constrained.MethodSixty-four undergraduates performed fifteen 90-s trials of laparoscopic training tasks (five replications of 3 tasks – cannulation, ring transfer, and rope manipulation). Half of the participants provided workload ratings using a traditional paper-and-pencil version of the NASA-TLX and MRQ; the remainder used a vocal (hands-free) version of the questionnaires. A follow-up experiment extended the evaluation of the hands-free version to actual medical students in a Minimally Invasive Surgery (MIS) training facility.ResultsThe NASA-TLX was scored in 2 ways – (1) the traditional procedure using participant-specific weights to combine its 6 subscales, and (2) a simplified procedure – the NASA Raw Task Load Index (NASA-RTLX) – using the unweighted mean of the subscale scores. Comparison of the scores obtained from the hands-free and written administration conditions yielded coefficients of equivalence of r = 0.85 (NASA-TLX) and r = 0.81 (NASA-RTLX). Equivalence estimates for the individual subscales ranged from r = 0.78 (“mental demand”) to r = 0.31 (“effort”). Both administration formats and scoring methods were equally sensitive to task and repetition effects. For the MRQ, the coefficient of equivalence for the hands-free and written versions was r = 0.96 when tested on undergraduates. However, the sensitivity of the hands-free MRQ to task demands (η_partial² = 0.138) was substantially less than that for the written version (η_partial² = 0.252). This potential shortcoming of the hands-free MRQ did not seem to generalize to medical students who showed robust task effects when using the hands-free MRQ (η_partial² = 0.396). A detailed analysis of the MRQ subscales also revealed differences that may be attributable to a “spillover” effect in which participants’ judgments about the demands of completing the questionnaires contaminated their judgments about the primary surgical training tasks.ConclusionVocal versions of the NASA-TLX are acceptable alternatives to standard written formats when researchers wish to obtain global workload estimates. However, care should be used when interpreting the individual subscales if the object is to make comparisons between studies or conditions that use different administration modalities. For the MRQ, the vocal version was less sensitive to experimental manipulations than its written counterpart; however, when medical students rather than undergraduates used the vocal version, the instrument’s sensitivity increased well beyond that obtained with any other combination of administration modality and instrument in this study. Thus, the vocal version of the MRQ may be an acceptable workload assessment technique for selected populations, and it may even be a suitable substitute for the NASA-TLX.     

Can aspect-oriented programming lead to more reliable software?

Aspect-oriented programming (AOP) is a novel topic in the software engineering and languages communities. AOP appears to have the potential to significantly improve the reliability of programs, particularly by modularizing error-handling policies and allowing for easier maintenance and better reuse. In this article, we introduce AspectJ, the first AOP language, and demonstrate how you can use it to construct more reliable software     

Aspect-orientation is a rewarding investment into future code changes – As long as the aspects hardly change

ContextAspect-Oriented Programming (AOP) is often described as a technique which improves the resulting software’s modularity. However, previous experiments seem to indicate that AOP is a technique which potentially increases the development or maintenance time. A possible reason why previous experiments were not able to show such a benefit is that those experiments did not consider situations where AOP has its strength: situations where aspects change.ObjectivesOur objective is to analyze whether initial higher development times caused by aspect-oriented programming can be compensated by frequent changes on the aspect code.MethodThis study is an empirical study with 15 subjects based on a within-subject design (repeated measurement) using two randomized groups. The measurement is development time until programming tasks completion. Additionally, an exploratory study is performed which analyzed (a) in separation out- and under-performing subjects with respect to development time, (b) the possible influence of the lines of code, (c) the possible influence of the number of advice and pointcuts, and finally (d) the possible impact of if-statements in the aspect-oriented solutions.ResultsThe experiment shows that frequent changes in the crosscutting code which do not change the aspect’s underlying structure compensate an initial higher development time for those concerns. But it also shows that changes, which do alter the concern’s structure again, result in higher development times when using AOP. Furthermore, the paper shows that the results are similar for out- and under-performing subjects in the experiment. Finally it shows that if changes are required in an aspect’s structure, subjects tend to perform such structural changes in a non-uniform way.ConclusionAOP is a potential rewarding investment into future code changes – but it has risks.     

Why software repositories are not used for defect-insertion circumstance analysis more often: A case study

ContextRoot-cause analysis is a data-driven technique for developing software process improvements in mature software organizations. The search for individual process correlates of high defect densities, which we call defect insertion circumstance analysis (DICA), is potentially both effective and cost-efficient as one approach to be used when attempting a general defect root cause analysis. In DICA, data from existing repositories (version archive, bug tracker) is evaluated largely automatically in order to determine conditions (such as the people, roles, components, or time-periods involved) that correlate with higher-than-normal defect insertion frequencies. Nevertheless, no reports of industrial use of DICA have been published.ObjectiveDetermine the reasons why DICA is not used more often by practitioners.MethodWe use a single-case, typical-case, revelatory-type case study to evaluate in parallel the importance of six plausible reasons (R1–R6). The case is based on 11 years of repository data from a small but mature software company building a product in the high-end content management system domain and describes a four person-months effort to make use of these data.ResultsWhile DICA required non-negligible effort (R3) and some degree of inventiveness (R2), the most relevant roadblock was insufficient reliability of the results (R6) combined with the difficulty of assessing this reliability (R5). We identify three difficulties that led to this outcome.ConclusionCurrent repository mining methods are too immature for successful DICA. Gradual improvements are unlikely to help; different principles of operation will be required. Even with such different techniques, issues with input data quality may continue to make good results difficult-to-have.     

Assessing creative problem-solving with automated text grading

The work aims to improve the assessment of creative problem-solving in science education by employing language technologies and computational–statistical machine learning methods to grade students’ natural language responses automatically. To evaluate constructs like creative problem-solving with validity, open-ended questions that elicit students’ constructed responses are beneficial. But the high cost required in manually grading constructed responses could become an obstacle in applying open-ended questions. In this study, automated grading schemes have been developed and evaluated in the context of secondary Earth science education. Empirical evaluations revealed that the automated grading schemes may reliably identify domain concepts embedded in students’ natural language responses with satisfactory inter-coder agreement against human coding in two sub-tasks of the test (Cohen’s Kappa = .65–.72). And when a single holistic score was computed for each student, machine-generated scores achieved high inter-rater reliability against human grading (Pearson’s r = .92). The reliable performance in automatic concept identification and numeric grading demonstrates the potential of using automated grading to support the use of open-ended questions in science assessments and enable new technologies for science learning.