Automated clustering to support the reflexion method

A significant aspect in applying the Reflexion Method is the mapping of components found in the source code onto the conceptual components defined in the hypothesized architecture. To date, this mapping is established manually, which requires a lot of work for large software systems. In this paper, we present a new approach, in which clustering techniques are applied to support the user in the mapping activity. The result is a semi-automated mapping technique that accommodates the automatic clustering of the source model with the user’s hypothesized knowledge about the system’s architecture.This paper describes three case studies in which the semi-automated mapping technique, called HuGMe, has been applied successfully to extend a partial map of real-world software applications. In addition, the results of another case study from an earlier publication are summarized, which lead to comparable results. We evaluated the extended versions of two automatic software clustering techniques, namely, MQAttract and CountAttract, with oracle mappings. We closely study the influence of the degree of completeness of the existing mapping and other controlling variables of the technique to make reliable suggestions.Both clustering techniques were able to achieve a mapping quality where more than 90% of the automatic mapping decisions turned out to be correct. Moreover, the experiments indicate that the attraction function (CountAttract′) based on local coupling and cohesion is more suitable for semi-automated mapping than the approach MQAttract′ based on a global assessment of coupling and cohesion.     

Semi-automated architectural abstraction specifications for supporting software evolution

In this paper we present an approach for supporting the semi-automated architectural abstraction of architectural models throughout the software life-cycle. It addresses the problem that the design and implementation of a software system often drift apart as software systems evolve, leading to architectural knowledge evaporation. Our approach provides concepts and tool support for the semi-automatic abstraction of architecture component and connector views from implemented systems and keeping the abstracted architecture models up-to-date during software evolution. In particular, we propose architecture abstraction concepts that are supported through a domain-specific language (DSL). Our main focus is on providing architectural abstraction specifications in the DSL that only need to be changed, if the architecture changes, but can tolerate non-architectural changes in the underlying source code. Once the software architect has defined an architectural abstraction in the DSL, we can automatically generate architectural component views from the source code using model-driven development (MDD) techniques and check whether architectural design constraints are fulfilled by these models. Our approach supports the automatic generation of traceability links between source code elements and architectural abstractions using MDD techniques to enable software architects to easily link between components and the source code elements that realize them. It enables software architects to compare different versions of the generated architectural component view with each other. We evaluate our research results by studying the evolution of architectural abstractions in different consecutive versions of five open source systems and by analyzing the performance of our approach in these cases.     

On the automatic modularization of software systems using the Bunch tool

Since modern software systems are large and complex, appropriate abstractions of their structure are needed to make them more understandable and, thus, easier to maintain. Software clustering techniques are useful to support the creation of these abstractions by producing architectural-level views of a system's structure directly from its source code. This paper examines the Bunch clustering system which, unlike other software clustering tools, uses search techniques to perform clustering. Bunch produces a subsystem decomposition by partitioning a graph of the entities (e.g., classes) and relations (e.g., function calls) in the source code. Bunch uses a fitness function to evaluate the quality of graph partitions and uses search algorithms to find a satisfactory solution. This paper presents a case study to demonstrate how Bunch can be used to create views of the structure of significant software systems. This paper also outlines research to evaluate the software clustering results produced by Bunch.     

Software reflexion models: bridging the gap between design andimplementation

The artifacts constituting a software system often drift apart over time. We have developed the software reflexion model technique to help engineers perform various software engineering tasks by exploiting, rather than removing, the drift between design and implementation. More specifically, the technique helps an engineer compare artifacts by summarizing where one artifact (such as a design) is consistent with and inconsistent with another artifact (such as source). The technique can be applied to help a software engineer evolve a structural mental model of a system to the point that it is “good enough” to be used for reasoning about a task at hand. The software reflexion model technique has been applied to support a variety of tasks, including design conformance, change assessment, and an experimental reengineering of the million-lines-of-code Microsoft Excel product. We provide a formal characterization of the reflexion model technique, discuss practical aspects of the approach, relate experiences of applying the approach and tools, and place the technique into the context of related work     

The evolution support environment system

The evolution support environment (ESE) system, which provides a framework for capturing and making available semantic information about software components of an evolving software system, is described. The goal in the design of the ESE system was to provide integrated support for management of software architecture configuration, life-cycle configuration, and version control. Software architecture configuration management allows tracking of interconnections among software components that make up a system. Life-cycle management allows traceability among specifications, design, code, and test cases during software development. Adding version control allows specific versions of software objects and their associated objects, such as specifications and test cases, to be retrieved. The authors' experience with the use of the system is discussed     

一种实用的自动代码划分算法

在编译器中进行自动代码划分然后采用代码重叠技术,是内存容量较小的处理器发挥性能的关键技术之一。随着异构多核处理器的出现,这类处理器通常作为协处理器。本文提出了一种实用的自动代码划分算法,并面向Cell处理器在GNU工具链中进行了实现。该算法首先根据调用频率发现调用热点,然后采用简化的聚类方法进行热点指导的分区聚集,最后进行容量指导的分区合并。实验结果表明,算法复杂度较低并能取得很好的效果。     

Structural Epochs in the Complexity of Software over Time

A case study using a new complexity measurement framework called Structure 101 tracked the structural complexity of three open source software products through their different releases. The analysis found that, as these software products evolved, a large proportion of structural complexity in early releases at the application-code level progressively migrated to higher-level design and architectural elements in subsequent releases, or vice-versa. This pattern repeated itself throughout the evolution of the software product. Refactoring efforts successfully reduced complexity at lower levels, but shifted the complexity to higher levels in the design hierarchy. Conversely, design restructuring at higher levels shifted complexity to lower levels. If this trend holds true for other software products, then mere code refactoring might not be enough to effectively managing structural complexity. Periodic major restructuring of software applications at the design or architectural level could be necessary.     

Reengineering component-based software systems with Archimetrix

Many software development, planning, or analysis tasks require an up-to-date software architecture documentation. However, this documentation is often outdated, unavailable, or at least not available as a formal model which analysis tools could use. Reverse engineering methods try to fill this gap. However, as they process the system’s source code, they are easily misled by design deficiencies (e.g., violations of component encapsulation) which leaked into the code during the system’s evolution. Despite the high impact of design deficiencies on the quality of the resulting software architecture models, none of the surveyed related works is able to cope with them during the reverse engineering process. Therefore, we have developed the Archimetrix approach which semiautomatically recovers the system’s concrete architecture in a formal model while simultaneously detecting and removing design deficiencies. We have validated Archimetrix on a case study system and two implementation variants of the CoCoME benchmark system. Results show that the removal of relevant design deficiencies leads to an architecture model which more closely matches the system’s conceptual architecture.     

On the impact of software evolution on software clustering

The evolution of a software project is a rich data source for analyzing and improving the software development process. Recently, several research groups have tried to cluster source code artifacts based on information about how the code of a software system evolves. The results of these evolutionary approaches seem promising, but a direct comparison to traditional software clustering approaches based on structural code dependencies is still missing. To fill this gap, we conducted several clustering experiments with an established software clustering tool comparing and combining the evolutionary and the structural approach. These experiments show that the evolutionary approach could produce meaningful clustering results. While the traditional approach provides better results because of a more reliable data density of the structural data, the combination of both approaches is able to improve the overall clustering quality. A review of related studies shows that this approach of combining dependency information is also successful in other software engineering applications.     

Weighing lexical information for software clustering in the context of architecture recovery

In this paper, we present a software clustering approach that leverages the information conveyed by the zone in which each lexeme appears in the classes of object oriented systems. We define six zones in the source code: Class Name, Attribute Name, Method Name, Parameter Name, Comment, and Source Code Statement. These zones may convey information with different levels of relevance, and so their contribution should be differently weighed according to the software system under study. To this aim, we define a probabilistic model of the lexemes distribution whose parameters are automatically estimated by the Expectation-Maximization algorithm. The weights of the zones are then exploited to compute similarities among source code classes, which are then grouped by a k-Medoid clustering algorithm. To assess the validity of our solution in the software architecture recovery field, we applied our approach to 19 software systems from different application domains. We observed that the use of our probabilistic model and the defined zones improves the quality of clustering results so that they are close to a theoretical upper bound we have proved.     

Source transformation in software engineering using the TXL transformation system

Many tasks in software engineering can be characterized as source to source transformations. Design recovery, software restructuring, forward engineering, language translation, platform migration, and code reuse can all be understood as transformations from one source text to another. The tree transformation language, TXL, is a programming language and rapid prototyping system specifically designed to support rule-based source to source transformation. Originally conceived as a tool for exploring programming language dialects, TXL has evolved into a general purpose source transformation system that has proven well suited to a wide range of software maintenance and reengineering tasks, including the design recovery, analysis and automated reprogramming of billions of lines of commercial Cobol, PL/I, and RPG code for the Year 2000. In this paper, we introduce the basic features of modern TXL and its use in a range of software engineering applications, with an emphasis on how each task can be achieved by source transformation.     

嵌入式系统软硬件自动划分方法研究

软硬件划分一直是嵌入式系统软硬件协同设计中的难点,如果离开具体系统,单纯的软硬件划分,其性能很难评估。本文提出基于系统体系结构,应用遗传算法来进行多目标优化的软硬件自动划分方法。在具体设计中,使用数据流图对系统建模,采用邻接表进行个体编码,定义交叉、变异操作,同时引入小生境技术,保持解的多样性。该方法为嵌入式系统软硬件自动划分提供一种新思路。     

Using fold-in and fold-out in the architecture recovery of software systems

In this paper we present an approach to automate the architecture recovery process of software systems. The approach is built on information retrieval and clustering techniques, and, in particular, uses Latent Semantic Indexing (LSI) to get similarities among software entities (e.g., programs or classes) and the k-means clustering algorithm to form groups of software entities that implement similar functionality. In order to improve computational time in the context of the software evolution and then reduce energy waste, the architecture recovery process can be also applied by using fold-in and fold-out mechanisms that, respectively, add and remove software entities to the LSI representation of the understudy software system. The approach has been implemented in a prototype of a supporting software system as an Eclipse plug-in. Finally, to assess the approach and the plug-in, we have conducted an empirical investigation on five open source software systems implemented using the programming languages Java and C/C++. In the investigation special emphasis has been also given to the effect of using the fold-in and fold-out mechanisms.     

Generation and validation of traces between requirements and architecture based on formal trace semantics

The size and complexity of software systems make integration of the new/modified requirements to the software system costly and time consuming. The impact of requirements changes on other requirements, design elements and source code should be traced to determine parts of the software to be changed. Considerable research has been devoted to relating requirements and design artifacts with source code. Less attention has been paid to relating requirements (R) with architecture (A) by using well-defined semantics of traces. Traces between R&A might be manually assigned. This is time-consuming, and error prone. Traces might be incomplete and invalid. In this paper, we present an approach for automatic trace generation and validation of traces between requirements (R) and architecture (A). Requirements relations and architecture verification techniques are used. A trace metamodel is defined with commonly used trace types between R&A. We use the semantics of traces and requirements relations for generating and validating traces with a tool support. The tool provides the following: (1) generation and validation of traces by using requirements relations and/or verification of architecture, (2) generation and validation of requirements relations by using traces. The tool is based on model transformation in ATL and term-rewriting logic in Maude.     

基于代码审计技术的OpenSSL脆弱性分析

本文讨论应用代码审计技术,分析OpenSSL源代码,进行脆弱性分析,并作出针对性修补建议.在进行源码级分析时,主要采用数据流分析技术,动态污点分析技术,定理证明等.各类代码审计技术由于都主要采用形式化手段分析软件构架的安全需求,通常都对某种特定场景有较好效果,但实用性较差.在审计linux,xen等大型成熟软件项目时,存在效率低下,误报率高等缺陷,甚至可能根本无法挖掘出有效漏洞.为此通过采用搭配使用各种不同代码审计技术,同时使用一种新的安全属性定义手法,从底层角度定义安全属性,以提升其对软件安全需求描述的准确度,避免其审计缺陷.在保留代码审计技术自动化程度高的优点同时提升其审计效率以及降低误报率,深层次发掘代码脆弱性.     

Evolving Object-Oriented Designs with Refactorings

Refactorings are behavior-preserving program transformations that automate design evolution in object-oriented applications. Three kinds of design evolution are: schema transformations, design pattern microarchitectures, and the hot-spot-driven-approach. This research shows that all three are automatable with refactorings. A comprehensive list of refactorings for design evolution is provided and an analysis of supported schema transformations, design patterns, and hot-spot meta patterns is presented. Further, we evaluate whether refactoring technology can be transferred to the mainstream by restructuring non-trivial C++ applications. The applications that we examine were evolved manually by software engineers. We show that an equivalent evolution could be reproduced significantly faster and cheaper by applying a handful of general-purpose refactorings. In one application, over 14K lines of code were transformed automatically that otherwise would have been coded by hand. Our experiments identify benefits, limitations, and topics of further research related to the transfer of refactoring technology to a production environment.     

Design‐code traceability for object‐oriented systems

Traceability is a key issue to ensure consistency among software artifacts of subsequent phases of the development cycle. However, few works have so far addressed the theme of tracing object oriented (OO) design into its implementation and evolving it. This paper presents an approach to checking the compliance of OO design with respect to source code and support its evolution. The process works on design artifacts expressed in the OMT (Object Modeling Technique) notation and accepts C++ source code. It recovers an “as is” design from the code, compares the recovered design with the actual design and helps the user to deal with inconsistencies. The recovery process exploits the edit distance computation and the maximum match algorithm to determine traceability links between design and code. The output is a similarity measure associated to design‐code class pairs, which can be classified as matched and unmatched by means of a maximum likelihood threshold. A graphic display of the design with different green levels associated to different levels of match and red for the unmatched classes is provided as a support to update the design and improve its traceability to the code.     

程序重构预处理在提高软件模型检测效率中的应用

针对软件模型检测目前很难处理大型程序的问题,提出用程序重构技术对待检的源代码进行预处理,以提高模型检测算法的效率.程序重构将大型程序分解成语义一致的小型过程的集合,由于模型检测算法中过程总结边可单独计算,而且在程序中对某过程的调用可能有多次,这种预处理可以避免状态空间的重复搜索,从而降低模型检测算法在空间和时间上的开销.根据表达程序性质的线性时序逻辑LTL公式的构成,给出了程序重构预处理前后程序语义相等的充分条件;并给定程序和性质公式,用blast作为程序模型检测实验工具,比较程序重构预处理前后blast的运行结果.理论分析和部分程序上的实验表明:程序重构预处理能降低大型程序的模型检测开销,并满足软件模型检测的安全性要求.