Software reengineering for complex systems

Reengineering complex software is a difficult task that requires the expenditure of significant resources. To reduce associated costs and improve resultant systems, automated techniques have been developed that can be used throughout all phases of the reengineering process. Three systems are described that have been used successfully for reengineering at The Johns Hopkins University Applied Physics Laboratory. The first system automatically transforms software written in CMS-2 to software written in Ada. The second system supports database design and automated query formulation, and employs conceptual-level intermediate representations for reverse and forward engineering as well as reengineering. The final system utilizes automated techniques to reengineer critical functions for performance enhancement. The systems described are evolving, and will provide a basis for future reengineering efforts.     

Philosophies for engineering computer-based systems

A sound problem-relevant philosophy is the key to achieving successful implementation of complex computer-based systems and software engineering methods and tools naturally flow from this foundation. The importance of a philosophy is illustrated by considering the development of Simula. Several philosophy-relevant factors are discussed and experiences where philosophies have played a central role in realizing complex real-time applications are described. These experiences relate to telecommunication software, automatic train control and power dispatching     

Iterative reengineering of legacy systems

During its life, a legacy system is subjected to many maintenance activities, which cause degradation of the quality of the system: When this degradation exceeds a critical threshold, the legacy system needs to be reengineered. In order to preserve the asset represented by the legacy system, the familiarity with it gained by the system's maintainers and users, and the continuity of execution of current operations during the reengineering process, the system needs to be reengineered gradually. Moreover, each program needs to be reengineered within a short period of time. The paper proposes a reengineering process model, which is applied to an in-use legacy system to confirm that the process satisfies previous requirements and to measure its effectiveness. The reengineered system replaced the legacy one to the satisfaction of all the stakeholders; the reengineering process also had a satisfactory impact on the quality of the system. Finally, this paper contributes to validate the cause-effect relationship between the reengineering process and overcoming the aging symptoms of a software system.     

Linda implementations in Java for concurrent systems

This paper surveys a number of the implementations of Linda that are available in Java. It provides some discussion of their strengths and weaknesses, and presents the results from benchmarking experiments using a network of commodity workstations. Some extensions to the original Linda programming model are also presented and discussed, together with examples of their application to parallel processing problems. Copyright © 2004 John Wiley & Sons, Ltd.     

A comparison of parallelism in interface designs for computer-based learning environments

In this paper we discuss an experiment that was carried out with a prototype, designed in conformity with the concept of parallelism and the Parallel Instruction theory (the PI theory). We designed this prototype with five different interfaces, and ran an empirical study in which 18 participants completed an abstract task. The five basic designs were based on hypotheses of the PI theory that for solving tasks on screens all task relevant information must be in view on a computer monitor, as clearly as possible. The condition with two parallel frames and the condition with one long web page appeared to be the best design for this type of task, better than window versions that we normally use for our computer simulations on the web. We do not only describe the results of the abstract task in the five conditions, but we also discuss the results from the perspective of concrete, realistic tasks with computer simulations. The interface with two parallel frames is the best solution here, but also the interface with long web pages (‘virtual parallelism’) is a great favourite in practice when doing realistic tasks.     

Pilot implementations of reliable systems

This paper describes the pilot implementation of a modifiable reliable system. On the basis of this work, we have been led to conclusions on necessary hardware and software features for supporting a reliable system, techniques for building such systems, and the scope of the results to be expected from a pilot implementation.     

Examining the designs of computer-based assessment and its impact on student engagement,satisfaction, and pass rates

Many researchers who study the impact of computer-based assessment (CBA) focus on the affordances or complexities of CBA approaches in comparison to traditional assessment methods. This study examines how CBA approaches were configured within and between modules, and the impact of assessment design on students' engagement, satisfaction, and pass rates. The analysis was conducted using a combination of longitudinal visualisations, correlational analysis, and fixed-effect models on 74 undergraduate modules and their 72,377 students. Our findings indicate that educators designed very different assessment strategies, which significantly influenced student engagement as measured by time spent in the virtual learning environment (VLE). Weekly analyses indicated that assessment activities were balanced with other learning activities, which suggests that educators tended to aim for a consistent workload when designing assessment strategies. Since most of the assessments were computer-based, students spent more time on the VLE during assessment weeks. By controlling for heterogeneity within and between modules, learning design could explain up to 69% of the variability in students' time spent on the VLE. Furthermore, assessment activities were significantly related to pass rates, but no clear relation with satisfaction was found. Our findings highlight the importance of CBA and learning design to how students learn online.     

Measuring instant emotions based on facial expressions during computer-based assessment

Emotions are very important during learning and assessment procedures. However, measuring emotions is a very demanding task. Several tools have been developed and used for this purpose. In this paper, the efficiency of the FaceReader during a computer-based assessment (CBA) was evaluated. Instant measurements of the FaceReader were compared with the researchers’ estimations regarding students’ emotions. The observations took place in a properly designed room in real time. Statistical analysis showed that there are some differences between FaceReader’s and researchers’ estimations regarding Disgusted and Angry emotions. Results showed that FaceReader is capable of measuring emotions with an efficacy of over 87% during a CBA and that it could be successfully integrated into a computer-aided learning system for the purpose of emotion recognition. Moreover, this study provides useful results for the emotional states of students during CBA and learning procedures. This is actually the first time that student’s instant emotions were measured during a CBA, based on their facial expressions. Results showed that most of the time students were experiencing Neutral, Angry, and Sad emotions. Furthermore, gender analysis highlights differences between genders’ instant emotions.     

Planning the reengineering of legacy systems

How can you know if reengineering is cost-effective? If it is preferable to new development? Or to maintaining the status quo? The author proposes a way to quantify the costs and prove the benefits of reengineering over other alternatives and offers some advice on contracting a reengineering project     

Metrics for quality and concurrency in object-based systems

This paper defines metrics for object-based systems. A hierarchical, abstract representation of object-based software is presented, and is used to define programming-language-independent metrics formulas. Novel techniques are given for measuring information hiding, cohesion, encapsulation, coupling, object-orientedness, maintainability, and potential concurrency among and within objects. The metrics are useful for guiding design optimization. Additionally, the concurrency metrics are useful for guiding object distribution for execution on concurrent hardware platforms. A toolset that computes several of the metrics is also described.     

A taxonomy for computer-based assessment of problem solving

Computer-based assessment of problem solving is motivated by the need for educational assessments that are valid and efficient. Based on a recent revision of Bloom's taxonomy (Anderson et al., 2001, A taxonomy for learning, teaching, and assessing: a revision of Bloom's taxonomy of educational objectives. New York: Longman), assessment items should require applying a particular cognitive process to a particular type of knowledge. There are 19 types of cognitive processes that can be classified into six major categories: remember, understand, apply, analyze, evaluate, and create. There are four major categories of knowledge: factual, conceptual, procedural, and metacognitive. Examples of computer-based assessments of problem-solving are provided based on the evaluation of the cognitive consequences of children's participation in an after-school computer club.     

Metrics for evaluating human information interaction systems

Society today has a wealth of information available due to information technology. The challenge facing researchers working in information access is how to help users easily locate the information needed. Evaluation methodologies and metrics are important tools to assess progress in human information interaction (HII). To properly evaluate these systems, evaluations need to consider the performance of the various components, the usability of the system, and the impact of the system on the end user. Current usability metrics are adequate for evaluating the efficiency, effectiveness, and user satisfaction of such systems. Performance measures for new intelligent technologies will have to be developed. Regardless of how well the systems are and how usable the systems are, it is critical that impact measures are developed. For HII systems to be useful, we need to assess how well information analysts work with the systems. This evaluation needs to go beyond technical performance metrics and usability metrics. What are the metrics for evaluating utility? This paper describes research efforts focused on developing metrics for the intelligence community that measure the impact of new software to facilitate information interaction.     

Systems engineering of computer-based systems

This report from the State of Practice Working Group of the IEEE Computer Society Task Force on Engineering of Computer-Based Systems (ECBS) advocates a new discipline at the systems engineering level. The report addresses the need for an ECBS discipline, identifies current practice and needed research, and suggests improvements that are achievable today. It addresses problem areas: the ECBS process itself, requirements definition, design, interfaces, management, process automation, and documentation. The report concludes that an improved ECBS discipline is necessary for improving processes and for fostering research and training     

Safety and reliability in computer-based systems

Increasing use of computer-based systems in industry necessitates the high reliability of such systems. The assessment of safety and reliability thus becomes crucial. The article describes the work of the UK National Centre of Systems Reliability, which provides a consultancy and information service on system reliability. The Alvey Directorate is also working closely with a group of researchers from various industrial bodies to set up a basis for independent software certification and warranty.     

Experimental validation strategies for heterogeneous computer-based assessment items

Computer-based assessments open up new possibilities to measure constructs in authentic settings. They are especially promising to measure 21st century skills, as for instance information and communication technologies (ICT) skills. Items tapping such constructs may be diverse regarding design principles and content and thus form a heterogeneous item set. Existing validation approaches, as the construct representation approach by Embretson (1983), however, require homogenous item sets in the sense that a particular task characteristic can be applied to all items. To apply this validation rational also for heterogeneous item sets, two experimental approaches are proposed based on the idea to create variants of items by systematically manipulating task characteristics. The change-approach investigates whether the manipulation affects construct-related demands and the eliminate-approach whether the test score represents the targeted skill dimension. Both approaches were applied within an empirical study (N = 983) using heterogeneous items from an ICT skills test. The results show how changes of ICT-specific task characteristics influenced item difficulty without changing the represented construct. Additionally, eliminating the intended skill dimension led to easier items and changed the construct partly. Overall, the suggested experimental approaches provide a useful validation tool for 21st century skills assessed by heterogeneous items.     

A closed-loop assessment framework for integrated product and process development implementations

An assessment framework for identifying and measuring the impacts of Integrated Product and Process Development (IPPD) implementations is presented. This framework considers the four major areas of commercial enterprise; the external environment, business strategy, IPPD, and implementation, which together form a closed-loop relationship. The holistic nature of the assessment framework make it a useful educational tool for understanding the influence of activities undertaken to re-engineer the various business processes and how this impacts the customer.     

Metrics for weighted transition systems: Axiomatization and complexity

Simulation distances are essentially approximations of simulation which provide a measure of the extent by which behaviors in systems are inequivalent. In this paper, we consider the general quantitative model of weighted transition systems, where transitions are labeled with elements of a finite metric space. We study the so-called point-wise and accumulating simulation distances which provide extensions to the well-known Boolean notion of simulation on labeled transition systems.We introduce weighted process algebras for finite and regular behavior and offer sound and (approximate) complete inference systems for the proposed simulation distances. We also settle the algorithmic complexity of computing the simulation distances.     

Effects of feedback in a computer-based assessment for learning

The effects of written feedback in a computer-based assessment for learning on students’ learning outcomes were investigated in an experiment at a Higher Education institute in the Netherlands. Students were randomly assigned to three groups, and were subjected to an assessment for learning with different kinds of feedback. These are immediate knowledge of correct response (KCR) + elaborated feedback (EF), delayed KCR + EF, and delayed knowledge of results (KR). A summative assessment was used as a post-test. No significant effect was found for the feedback condition on student achievement on the post-test. Results suggest that students paid more attention to immediate than to delayed feedback. Furthermore, the time spent reading feedback was positively influenced by students’ attitude and motivation. Students perceived immediate KCR + EF feedback to be more useful for learning than KR. Students also had a more positive attitude towards feedback in a CBA when they received KCR + EF rather than KR only.