Quality of manual data collection in Java software: an empirical investigation

Data collection, both automatic and manual, lies at the heart of all empirical studies. The quality of data collected from software informs decisions on maintenance, testing and wider issues such as the need for system re-engineering. While of the two types stated, automatic data collection is preferable, there are numerous occasions when manual data collection is unavoidable. Yet, very little evidence exists to assess the error-proneness of the latter. Herein, we investigate the extent to which manual data collection for Java software compared with its automatic counterpart for the same data. We investigate three hypotheses relating to the difference between automated and manual data collection. Five Java systems were used to support our investigation. Results showed that, as expected, manual data collection was error-prone, but nowhere near the extent we had initially envisaged. Key indicators of mistakes in manual data collection were found to be poor developer coding style, poor adherence to sound OO coding principles, and the existence of relatively large classes in some systems. Some interesting results were found relating to the collection of public class features and the types of error made during manual data collection. The study thus offers an insight into some of the typical problems associated with collecting data manually; more significantly, it highlights the problems that poorly written systems have on the quality of visually extracted data.     

Two topographic maps for data visualisation

We review a new form of self-organizing map which is based on a nonlinear projection of latent points into data space, identical to that performed in the Generative Topographic Mapping (GTM) [Bishop et al. (1997) Neurl Comput 10(1): 215–234]. But whereas the GTM is an extension of a mixture of experts, our new model is an extension of a product of experts [Hinton (2000) Technical report GCNU TR 2000-004, Gatsby Computational Neuroscience Unit, University College, London]. We show visualisation results on some real data sets and compare with the GTM. We then introduce a second mapping based on harmonic averages and show that it too creates a topographic mapping of the data. We compare these mappings on real and artificial data sets. Responsible editor: Soumen Chakrabarti.     

Environment as a first class abstraction in multiagent systems

The current practice in multiagent systems typically associates the environment with resources that are external to agents and their communication infrastructure. Advanced uses of the environment include infrastructures for indirect coordination, such as digital pheromones, or support for governed interaction in electronic institutions. Yet, in general, the notion of environment is not well defined. Functionalities of the environment are often dealt with implicitly or in an ad hoc manner. This is not only poor engineering practice, it also hinders engineers to exploit the full potential of the environment in multiagent systems. In this paper, we put forward the environment as an explicit part of multiagent systems.We give a definition stating that the environment in a multiagent system is a first-class abstraction with dual roles: (1) the environment provides the surrounding conditions for agents to exist, which implies that the environment is an essential part of every multiagent system, and (2) the environment provides an exploitable design abstraction for building multiagent system applications. We discuss the responsibilities of such an environment in multiagent systems and we present a reference model for the environment that can serve as a basis for environment engineering. To illustrate the power of the environment as a design abstraction, we show how the environment is successfully exploited in a real world application. Considering the environment as a first-class abstraction in multiagent systems opens up new horizons for research and development in multiagent systems.     

Network and device-level impacts: performance and reliability of active I/O storage systems

Distributed active storage architectures are designed to offload user-level processing to the peripheral from the host servers. In this paper, we report preliminary investigation on performance and fault recovery designs, as impacted by emerging storage interconnect protocols and state-of-the-art storage devices. Empirical results obtained using validated device-level and interconnect data demonstrate the significance of the said parameters on the overall system performance and reliability.     

On the effectiveness of early life cycle defect prediction with Bayesian Nets

Standard practice in building models in software engineering normally involves three steps: collecting domain knowledge (previous results, expert knowledge); building a skeleton of the model based on step 1 including as yet unknown parameters; estimating the model parameters using historical data. Our experience shows that it is extremely difficult to obtain reliable data of the required granularity, or of the required volume with which we could later generalize our conclusions. Therefore, in searching for a method for building a model we cannot consider methods requiring large volumes of data. This paper discusses an experiment to develop a causal model (Bayesian net) for predicting the number of residual defects that are likely to be found during independent testing or operational usage. The approach supports (1) and (2), does not require (3), yet still makes accurate defect predictions (an R ² of 0.93 between predicted and actual defects). Since our method does not require detailed domain knowledge it can be applied very early in the process life cycle. The model incorporates a set of quantitative and qualitative factors describing a project and its development process, which are inputs to the model. The model variables, as well as the relationships between them, were identified as part of a major collaborative project. A dataset, elicited from 31 completed software projects in the consumer electronics industry, was gathered using a questionnaire distributed to managers of recent projects. We used this dataset to validate the model by analyzing several popular evaluation measures (R ², measures based on the relative error and Pred). The validation results also confirm the need for using the qualitative factors in the model. The dataset may be of interest to other researchers evaluating models with similar aims. Based on some typical scenarios we demonstrate how the model can be used for better decision support in operational environments. We also performed sensitivity analysis in which we identified the most influential variables on the number of residual defects. This showed that the project size, scale of distributed communication and the project complexity cause the most of variation in number of defects in our model. We make both the dataset and causal model available for research use.     

Design of nonlinear controllers for active vehicle suspension with state constraints

Design of active vehicle suspension has tradeoffs between three main performance metrics (passenger comfort, suspension deflection and road holding ability). It has been known that each performance can be achieved by H _∞ controller and they can be gathered by LPV (Linear Parameter Varying) method. However, because the suspension deflection limit was not explicitly considered, this limit may be exceeded. In this paper, the authors propose a “reference shaping“ based method in order to improve the control performance. In this approach, a “virtual reference” signal is imposed to the system such that the suspension deflection is kept small. The effectiveness of the approach is examined by numerical simulations. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Modelling asymmetry in the peripheral auditory system of the lizard

The ears of lizards are highly directional. The directionality is generated by strong acoustical coupling of the eardrums. A simple lumped-parameter model of the ear followed by binaural comparisons has been shown to perform successful phonotaxis in robot implementations. However, such a model will produce localization errors in the form of response bias if the ears are asymmetrical. We have evaluated how large errors are generated by asymmetry using simulations of the ear model in Mathematica 5.2. The study shows that the effect of asymmetry is minimal around the most directional frequency of the ear, but that biases reduce the useful bandwidth of localization. This work was presented in part at the First European Workshop on Artificial Life and Robotics, Vienna, Austria, July 12–13, 2007     

2D artificial life system using network-type assembly-like language: influence of change in environment with costs of instructions

To unlock the full potential of evolution in digital media, a Tierra-like system using network-type assembly-like language has been constructed. In the system, like Avida, digital creatures, self-replicating programs, live in a discrete 2D torus space and an interaction between creatures is restricted locally. Bearing a genetic network in mind, network structure is introduced. In the previous works, it is shown that the possibility that the network-type model has more potential of evolution than a linear-type model like Avida in a simple environment. In this paper, to study the potential of evolution more precisely, we model the effects of environment other than creatures. As one of the simplest models of such environment, the cost of execution, i.e. executing time, is introduced. The difficult environment to live in costs high to execute instructions, the easy environment does low. In computer experiments, we have investigated the influence of change in environment by analyzing the process of evolution and diversity of the system. Experimental results show that the network-type system keeps more stable diversity than the linear-type system does, even when the environment changes drastically. This indicates the possibility that the network-type system has more potential of evolution than the linear-type system has. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008