Experimental models for validating technology

Experimentation helps determine the effectiveness of proposed theories and methods. However, computer science has not developed a concise taxonomy of methods for demonstrating the validity of new techniques. Experimentation is a crucial part of attribute evaluation and can help determine whether methods used in accordance with some theory during product development will result in software being as effective as necessary. By looking at multiple examples of technology validation, the authors develop a taxonomy for software engineering experimentation that describes twelve different experimental approaches     

A method for testing and validating executable statechart models

Software and Systems Modeling - Statecharts constitute an executable language for modelling event-based reactive systems. The essential complexity of statechart models solicits the need for...     

A goal-oriented,ontology-based methodology to support the design of AAL environments

One of the most critical issues in Ambient Assisted Living (AAL) is the design of systems that can evolve to meet the requirements of individuals as their needs and health conditions change. Although much work has been done on home and building automation systems for AAL, often referred to as assistive domotics, there is in fact still a substantial lack of solutions capable to support system designers in the early stage of development of such assistive systems. To this aim, the work contributes to the research on design of assistive domotic systems by presenting an ontology-driven methodology aimed to guide the development process. The novel contributions of the paper include the goal-oriented approach of the methodology, which involves the elicitation and analysis of AAL requirements and their formal representation in an ontology, where high-level goals are described in terms of subgoals and tasks, that are then linked to corresponding measures and devices. Moreover, logic-based reasoning enables more advanced functionalities useful at design time. We present a validation of the methodology showing typical use cases both related to the development from scratch of a domotic system with assistive capabilities starting from a set of high-level user requirements and the redesign of existing implementations according to changed requirements.     

A methodology for creating and validating psychological stories for conveying and measuring psychological traits

Personality impacts all areas of our lives; it governs who we are and how we react to life’s challenges. Personalized systems that adapt to end users should take into account the user’s personality to perform well. Several methodologies (e.g. User-as-Wizard, indirect studies) that use personality adaptation require first for personality to be conveyed to the participant; this has few validated approaches. Furthermore, measuring personality is often time consuming, prone to response bias (e.g. using questionnaires) or data intensive (e.g. using behaviour or text mining). This paper presents a methodology for creating and validating stories to convey psychological traits and for using such stories with a personality slider scale to measure these traits. We present the validation of the scale and evaluate its reliability. To evidence the validity of the methodology, we outline studies where the stories and scale have been effectively applied (in recommender systems, intelligent tutoring systems, and persuasive systems).

     

A dynamical trajectory-based methodology for systematically computing multiple optimal solutions of general nonlinear programming problems

In this paper, a novel dynamical trajectory-based methodology is developed for systematically computing multiple local optimal solutions of general nonlinear programming problems with disconnected feasible components satisfying nonlinear equality/inequality constraints. The proposed methodology, deterministic in nature, exploits trajectories of two different nonlinear dynamical systems to find multiple local optimal solutions. The methodology consists of two phases: Phase I starts from an arbitrary (infeasible) initial point and finds systematically multiple or all the disconnected feasible components; Phase II finds an adjacent local optimal solution from a local optimum via a decomposition point, thereby systematically locating multiple local optimal solutions which lie within each feasible component found in Phase I. By alternating between these two phases, the methodology locates multiple or all the local optimal solutions which lie in all the disconnected feasible components. A theoretical foundation for the proposed methodology is also developed. The methodology is illustrated with a numerical example with promising results.     

An update to experimental models for validating computer technology

In 1998 a survey was published on the extent to which software engineering papers validate the claims made in those papers. The survey looked at publications in 1985, 1990 and 1995. This current paper updates that survey with data from 2000 to 2005. The basic conclusion is that the situation is improving. One earlier complaint that access to data repositories was difficult is becoming less prevalent and the percentage of papers including validation is increasing.     

A Bayesian segmentation methodology for parametric image models

Region-based image segmentation methods require some criterion for determining when to merge regions. This paper presents a novel approach by introducing a Bayesian probability of homogeneity in a general statistical context. The authors' approach does not require parameter estimation and is therefore particularly beneficial for cases in which estimation-based methods are most prone to error: when little information is contained in some of the regions and, therefore, parameter estimates are unreliable. The authors apply this formulation to three distinct parametric model families that have been used in past segmentation schemes: implicit polynomial surfaces, parametric polynomial surfaces, and Gaussian Markov random fields. The authors present results on a variety of real range and intensity images     

Deriving tabular event-based specifications from goal-oriented requirements models

Goal-oriented methods are increasingly popular for elaborating software requirements. They offer systematic support for incrementally building intentional, structural and operational models of the software and its environment. They also provide various techniques for early analysis, notably, to manage conflicting goals or to anticipate abnormal environment behaviours that prevent goals from being achieved. On the other hand, tabular event-based methods are well-established for specifying operational requirements for control software. They provide sophisticated techniques and tools for late analysis of software behaviour models through simulation, model checking or table exhaustiveness checks. The paper proposes to take the best out of these two worlds to engineer requirements for control software. It presents a technique for deriving event-based specifications, written in the SCR tabular language, from operational specifications built according to the KAOS goal-oriented method. The technique consists of a series of transformation steps each of which resolves semantic, structural or syntactic differences between the KAOS source language and the SCR target language. Some of these steps need human intervention and illustrate the kind of semantic subtleties that need to be taken into account when integrating multiple formalisms. As a result of our technique SCR specifiers may use upstream goal-based processes à la KAOS for the incremental elaboration, early analysis, organization and documentation of their tables, while KAOS modelers may use downstream tables à la SCR for later analysis of the behaviour models derived from goal specifications.     

A goal-oriented trust model for virtual organization creation

Virtual organization (VO) is a main organizational paradigm for enterprises to collaborate in the rapidly changing environment. Advances in information and communication technologies are accelerating collaboration among enterprises in different domains as well as unknown enterprises. In this environment, an enterprise faces more chances to cooperate with other enterprises through the internet. Trust is a key concept in many kinds of interactions, allowing an enterprise to act under uncertainty and with the risk of negative consequences. To minimize the operating risk and guarantee the success of a VO, trust evaluation of potential partners should be considered during the partner selection process. However, trust evaluation methods in the literature have limitations in applying them to small and medium enterprises because they are based on evaluation of human experts. In this paper, a goal-oriented trust model is proposed to evaluate the trust values of enterprises. In the context, trust is the probability that a trustee satisfies the goals when it completes assigned tasks. This paper introduces the concept of a goal-oriented trust model with the core constructs of a goal, a project, a task, and a capability. It further addresses an example of applying the proposed model to the partner selection process of a project-based VO creation.     

Parameter estimation for continuous-time models—A survey

The paper reviews the progress of research on parameter estimation for continuous-time models of dynamic systems over the period 1958–1980. Major developments are considered in historical order and within a classification system which conforms as closely as possible to that which has arisen naturally over the past two decades. While every attempt is made to consider progress in other related scientific disciplines, such as econometrics, the major accent is on research in the control and systems field.     

A computer-based goal-oriented productivity index

This paper presents a Weighted Multi-Factor Productivity Index. This index monitors the performance of an organization and identifies productivity problems. Key resource factors are defined and a weighting technique, with both subjective and objective attributes, is presented. This productivity index is dynamic and responsive to the changing needs of an organization and is a goal oriented measuring tool.     

A methodology for building neural networks models from empirical engineering data

Neural networks (NN) are general tools for modeling functional relationships in engineering. They are used to model the behavior of products and the properties of processes. Nevertheless, their use is often ad hoc. This paper provides a sound basis for using NN as tools for modeling functional relationships implicit in empirical engineering data. First, a clear definition of a modeling task is given, followed by reviewing the theoretical modeling capabilities of NN and NN model estimation. Subsequently, a procedure for using NN in engineering practice is described and illustrated with an example of modeling marine propeller behavior. Particular attention is devoted to better estimation of model quality, insight into the influence of measurement errors on model quality, and the use of advanced methods such as stacked generalization and ensemble modeling to further improve model quality. Using a new method of ensemble of SG(k-NN), one could improve the quality of models even if they are close to being optimal.     

A methodology for control-relevant nonlinear system identification using restricted complexity models

A broadly-applicable, control-relevant system identification methodology for nonlinear restricted complexity models (RCMs) is presented. Control design based on RCMs often leads to controllers which are easy to interpret and implement in real-time. A control-relevant identification method is developed to minimize the degradation in closed-loop performance as a result of RCM approximation error. A two-stage identification procedure is presented. First, a nonlinear ARX model is estimated from plant data using an orthogonal least squares algorithm; a Volterra series model is then generated from the nonlinear ARX model. In the second stage, a RCM with the desired structure is estimated from the Volterra series model through a model reduction algorithm that takes into account closed-loop performance requirements. The effectiveness of the proposed method is illustrated using two chemical reactor examples.     

A survey of models of degradation for control applications

The analysis of equipment degradation has traditionally developed in two main directions. One approach, of great interest for control system design, has been to consider that degradation causes fundamental changes to the behaviour of a system. Another approach, used in optimal maintenance planning and production scheduling, considers degradation as a separate process that affects performance but does not necessarily change the behaviour. This article provides a review of mathematical models of degradation that will facilitate the integration of degradation modelling into control and optimisation schemes. To this end, a new unified classification is proposed. It takes into account the influence of degradation on the behaviour of the system, as well as the factors influencing degradation. Understanding these mutual influences will enable improved optimization, design and operation of control systems. The flexibility of the proposed classification is demonstrated in an industrial application to a multi-product batch scheduling process.     

A chi-square statistic for validating simulation-generated responses

A chi-square statistic is developed for validating simulation models. This statistic is used to test the hypothesis that the variance between the simulation generated output and actual output is less than some tolerable limit. The assumptions and the steps required to compute the statistic are presented. A real-world example is presented to illustrate the methodology and to demonstrate its usefulness. The results are shown to be consistent with previous statistical validation.     

The role of computer graphics in validating simulation models

Fault finding in a model that is only tentative can present major difficulties. Trying to detect an unknown number of unspecified errors by perusing tables of numbers is difficult and time–consuming with no guarantee of success. For simulation programs graphical techniques can be a useful means of detecting faults in coding and logic. The non-specific nature of visual monitoring makes use of the ability of the human brain to recognize spatial relationships and detect deviations from expected behaviour.     

An empirical approach toward the resolution of conflicts in goal-oriented models

One significant problem requirements engineers have to cope with is the management of unclear requirements, ambiguities, and conflicts that may arise between stakeholders. Such issues may be desirable since they may allow for further elicitation of requirements that would have been missed otherwise. Goal models capture the objectives and other intentions of different stakeholders, together with their relationships. They can be used to refine unclear requirements and to detect conflicts and ambiguities early during model validation. However, resolving such ambiguities and conflicts is key for the successful implementation of the goal models. In this paper, we propose a novel approach to validate models in the Goal-oriented Requirement Language and resolve conflicts between the perspectives of intervening stakeholders (and especially between stakeholders of a given group). Our approach is based on a statistical analysis of empirical data that we collect from surveys designed for each group of stakeholders. We apply concept analysis in order to fix goal-model artifacts that are subject to conflict. We illustrate our approach using a case study of a goal model describing the involvement of undergraduate students in university research activities.     

A family of simulation criteria to guide DEVS models validation rigorously,systematically and semi-automatically

The most common method to validate a DEVS model against the requirements is to simulate it several times under different conditions, with some simulation tool. The behavior of the model is compared with what the system is supposed to do. The number of different scenarios to simulate is usually infinite, therefore, selecting them becomes a crucial task. This selection, actually, is made following the experience or intuition of an engineer. Here we present a family of criteria to conduct DEVS model simulations in a disciplined way and covering the most significant simulations to increase the confidence on the model. This is achieved by analyzing the mathematical representation of the DEVS model and, thus, part of the validation process can be automatized.     

A survey of browsing models for content based image retrieval

The problem of content based image retrieval (CBIR) has traditionally been investigated within a framework that emphasises the explicit formulation of a query: users initiate an automated search for relevant images by submitting an image or draw a sketch that exemplifies their information need. Often, relevance feedback is incorporated as a post-retrieval step for optimising the way evidence from different visual features is combined. While this sustained methodological focus has helped CBIR to mature, it has also brought out its limitations more clearly: There is often little support for exploratory search and scaling to very large collections is problematic. Moreover, the assumption that users are always able to formulate an appropriate query is questionable. An effective, albeit much less studied, method of accessing image collections based on visual content is that of browsing. The aim of this survey paper is to provide a structured overview of the different models that have been explored over the last one to two decades, to highlight the particular challenges of the browsing approach and to focus attention on a few interesting issues that warrant more intense research.