Speech intelligibility measured with shortened versions of Callsign Acquisition Test (CAT)

The Callsign Acquisition Test (CAT) is a new speech intelligibility test developed by the Human Research and Engineering Directorate of the U.S. Army Research Laboratory (ARL-HRED). CAT uses the phonetic alphabet and digit stimuli combined together to form 126 test items.

Objective

The purpose of this study was to assess the reliability of data collected with shorter versions of CAT.

Design

A total of 5 shorter versions of the original list (CAT-120, CAT-60, CAT-40, CAT-30, and CAT-24) were formed and evaluated using 19 participants. Each of the subsets of CAT was presented in pink noise at signal-to-noise ratios (SNRs) of −6 dB and −9 dB.

Results

Results showed that shortened CAT lists have the capability of providing the same predictive power as the full CAT with good test-retest reliability.

Conclusions

Under the experimental conditions of this study, any of the shorter versions of the CAT can be utilized in place of the full version to reduce testing times with no effect on predictive power.     

Discriminative effect of user influence and user responsibility on information system development processes and project management

Context

User participation in information system (IS) development has received much research attention. However, prior empirical research regarding the effect of user participation on IS success is inconclusive. This might be because previous studies overlook the effect of the particular components of user participation and other possible mediating factors.

Objective

The objective of this study is to empirically examine how user influence and user responsibility affect IS project performance. We inspect whether user influence and user responsibility improve the quality of the IS development process and in turn leads to project success, or if they have a direct positive influence on project success.

Method

We conducted a survey of 151 IS project managers in order to understand the impact of user influence and user responsibility on IS project performance. Regression analysis was conducted to assess the relationship among user influence, user responsibility, organizational technology learning, project control, user-developer interaction, and IS project management performance.

Results

This study shows that user responsibility and user influence have a positive effect on project performance through the promotion of IS development processes as mediators, including organizational technology learning, project control, and user-IS interaction.

Conclusion

Our results suggest that user responsibility and user influence respectively play an important role in indirectly and directly impacting project management performance. Results of the analysis imply that organizations and project managers should use both user participation and user influence to improve processes performance, and in turn, increase project success.     

A method for assessing confidence in requirements analysis

Context

During development managers, analysts and designers often need to know whether enough requirements analysis work has been done and whether or not it is safe to proceed to the design stage.

Objective

This paper describes a new, simple and practical method for assessing our confidence in a set of requirements.

Method

We identified four confidence factors and used a goal oriented framework with a simple ordinal scale to develop a method for assessing confidence. We illustrate the method and show how it has been applied to a real systems development project.

Results

We show how assessing confidence in the requirements could have revealed problems in this project earlier and so saved both time and money.

Conclusion

Our meta-level assessment of requirements provides a practical and pragmatic method that can prove useful to managers, analysts and designers who need to know when sufficient requirements analysis has been performed.     

An ant colony optimization algorithm to improve software quality prediction models: Case of class stability

Context

Assessing software quality at the early stages of the design and development process is very difficult since most of the software quality characteristics are not directly measurable. Nonetheless, they can be derived from other measurable attributes. For this purpose, software quality prediction models have been extensively used. However, building accurate prediction models is hard due to the lack of data in the domain of software engineering. As a result, the prediction models built on one data set show a significant deterioration of their accuracy when they are used to classify new, unseen data.

Objective

The objective of this paper is to present an approach that optimizes the accuracy of software quality predictive models when used to classify new data.

Method

This paper presents an adaptive approach that takes already built predictive models and adapts them (one at a time) to new data. We use an ant colony optimization algorithm in the adaptation process. The approach is validated on stability of classes in object-oriented software systems and can easily be used for any other software quality characteristic. It can also be easily extended to work with software quality predictive problems involving more than two classification labels.

Results

Results show that our approach out-performs the machine learning algorithm C4.5 as well as random guessing. It also preserves the expressiveness of the models which provide not only the classification label but also guidelines to attain it.

Conclusion

Our approach is an adaptive one that can be seen as taking predictive models that have already been built from common domain data and adapting them to context-specific data. This is suitable for the domain of software quality since the data is very scarce and hence predictive models built from one data set is hard to generalize and reuse on new data.     

The agile requirements refinery: Applying SCRUM principles to software product management

Context

Although agile software development methods such as SCRUM and DSDM are gaining popularity, the consequences of applying agile principles to software product management have received little attention until now.

Objective

In this paper, this gap is filled by the introduction of a method for the application of SCRUM principles to software product management.

Method

A case study research approach is employed to describe and evaluate this method.

Results

This has resulted in the ‘agile requirements refinery’, an extension to the SCRUM process that enables product managers to cope with complex requirements in an agile development environment. A case study is presented to illustrate how agile methods can be applied to software product management.

Conclusions

The experiences of the case study company are provided as a set of lessons learned that will help others to apply agile principles to their software product management process.     

Software product line testing - A systematic mapping study

Context

Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped.

Objective

This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research.

Method

A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type.

Results

A majority of the papers are of proposal research types (64%). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority.

Conclusions

More validation and evaluation research is needed to provide a better foundation for SPL testing.     

DELIVER! - An educational game for teaching Earned Value Management in computing courses

Context

To meet the growing need for education in Software Project Management, educational games have been introduced as a beneficial instructional strategy. However, there are no low-cost board games openly available to teach Earned Value Management (EVM) in computing programs.

Objective

This paper presents an educational board game to reinforce and teach the application of EVM concepts in the context of undergraduate computing programs complementing expository lessons on EVM basics.

Method

The game has been developed based on project management fundamentals and teaching experience in this area. So far, it has been applied in two project management courses in undergraduate computing programs at the Federal University of Santa Catarina. We evaluated motivation, user experience and the game’s contribution to learning through case studies on Kirkpatrick’s level one based on the perception of the students.

Results

First results of the evaluation of the game indicate a perceived potential of the game to contribute to the learning of EVM concepts and their application. The results also point out a very positive effect of the game on social interaction, engagement, immersion, attention and relevance to the course objectives.

Conclusion

We conclude that the game DELIVER! can contribute to the learning of the EVM on the cognitive levels of remembering, understanding and application. The illustration of the application of EVM through the game can motivate its usefulness. The game has proven to be an engaging instructional strategy, keeping students on the task and attentive. In this respect, the game offers a possibility to complement traditional instructional strategies for teaching EVM. In order to further generalize and to strengthen the validity of the results, it is important to obtain further evaluations.     

Design guidelines for software processes knowledge repository development

Context

Staff turnover in organizations is an important issue that should be taken into account mainly for two reasons:

1.

Employees carry an organization’s knowledge in their heads and take it with them wherever they go

2.

Knowledge accessibility is limited to the amount of knowledge employees want to share

Objective

The aim of this work is to provide a set of guidelines to develop knowledge-based Process Asset Libraries (PAL) to store software engineering best practices, implemented as a wiki.

Method

Fieldwork was carried out in a 2-year training course in agile development. This was validated in two phases (with and without PAL), which were subdivided into two stages: Training and Project.

Results

The study demonstrates that, on the one hand, the learning process can be facilitated using PAL to transfer software process knowledge, and on the other hand, products were developed by junior software engineers with a greater degree of independence.

Conclusion

PAL, as a knowledge repository, helps software engineers to learn about development processes and improves the use of agile processes.     

IT Service Management Process Improvement based on ISO/IEC 15504: A systematic review

Context

In recent years, many software companies have considered Software Process Improvement (SPI) as essential for successful software development. These companies have also shown special interest in IT Service Management (ITSM). SPI standards have evolved to incorporate ITSM best practices.

Objective

This paper presents a systematic literature review of ITSM Process Improvement initiatives based on the ISO/IEC 15504 standard for process assessment and improvement.

Method

A systematic literature review based on the guidelines proposed by Kitchenham and the review protocol template developed by Biolchini et al. is performed.

Results

Twenty-eight relevant studies related to ITSM Process Improvement have been found. From the analysis of these studies, nine different ITSM Process Improvement initiatives have been detected. Seven of these initiatives use ISO/IEC 15504 conformant process assessment methods.

Conclusion

During the last decade, in order to satisfy the on-going demand of mature software development companies for assessing and improving ITSM processes, different models which use the measurement framework of ISO/IEC 15504 have been developed. However, it is still necessary to define a method with the necessary guidelines to implement both software development processes and ITSM processes reducing the amount of effort, especially because some processes of both categories are overlapped.     

Data warehouse testing: A prototype-based methodology

Context

Testing is an essential part of the development life-cycle of any software product. While most phases of data warehouse design have received considerable attention in the literature, not much has been written about data warehouse testing.

Objective

In this paper we propose a comprehensive approach to testing data warehouse systems. Its main features are earliness with respect to the life-cycle, modularity, tight coupling with design, scalability, and measurability through proper metrics.

Method

We introduce a number of specific testing activities, we classify them in terms of what is tested and how it is tested, and we show how they can be framed within a prototype-based methodology. We apply our approach to a real case study for a large retail company.

Results

The case study we faced, based on an iterative prototype-based medium-size project, confirmed the validity of our approach. In particular, the main benefits were obtained in terms of project transparency, coordination of the development team, and organization of design activities.

Conclusion

Though some general-purpose testing techniques can be applied to data warehouse projects, the effectiveness of testing can be largely improved by applying specifically-devised techniques and metrics.     

Assessing PSP effect in training disciplined software development: A Plan-Track-Review model

Context

In training disciplined software development, the PSP is said to result in such effect as increased estimation accuracy, better software quality, earlier defect detection, and improved productivity. But a systematic mechanism that can be easily adopted to assess and interpret PSP effect is scarce within the existing literature.

Objective

The purpose of this study is to explore the possibility of devising a feasible assessment model that ties up critical software engineering values with the pertinent PSP metrics.

Method

A systematic review of the literature was conducted to establish such an assessment model (we called a Plan-Track-Review model). Both mean and median approaches along with a set of simplified procedures were used to assess the commonly accepted PSP training effects. A set of statistical analyses further followed to increase understanding of the relationships among the PSP metrics and to help interpret the application results.

Results

Based on the results of this study, PSP training effect on the controllability, manageability, and reliability of a software engineer is quite positive and largely consistent with the literature. However, its effect on one’s predictability on project in general (and on project size in particular) is not implied as said in the literature. As for one’s overall project efficiency, our results show a moderate improvement. Our initial finding also suggests that a prior stage PSP effect could have an impact on later stage training outcomes.

Conclusion

It is concluded that this Plan-Track-Review model with the associated framework can be used to assess PSP effect regarding a disciplined software development. The generated summary report serves to provide useful feedback for both PSP instructors and students based on internal as well as external standards.     

A systematic mapping study of software product lines testing

Context

In software development, Testing is an important mechanism both to identify defects and assure that completed products work as specified. This is a common practice in single-system development, and continues to hold in Software Product Lines (SPL). Even though extensive research has been done in the SPL Testing field, it is necessary to assess the current state of research and practice, in order to provide practitioners with evidence that enable fostering its further development.

Objective

This paper focuses on Testing in SPL and has the following goals: investigate state-of-the-art testing practices, synthesize available evidence, and identify gaps between required techniques and existing approaches, available in the literature.

Method

A systematic mapping study was conducted with a set of nine research questions, in which 120 studies, dated from 1993 to 2009, were evaluated.

Results

Although several aspects regarding testing have been covered by single-system development approaches, many cannot be directly applied in the SPL context due to specific issues. In addition, particular aspects regarding SPL are not covered by the existing SPL approaches, and when the aspects are covered, the literature just gives brief overviews. This scenario indicates that additional investigation, empirical and practical, should be performed.

Conclusion

The results can help to understand the needs in SPL Testing, by identifying points that still require additional investigation, since important aspects regarding particular points of software product lines have not been addressed yet.     

Components meet aspects: Assessing design stability of a software product line

Context

It is important for Product Line Architectures (PLA) to remain stable accommodating evolutionary changes of stakeholder’s requirements. Otherwise, architectural modifications may have to be propagated to products of a product line, thereby increasing maintenance costs. A key challenge is that several features are likely to exert a crosscutting impact on the PLA decomposition, thereby making it more difficult to preserve its stability in the presence of changes. Some researchers claim that the use of aspects can ameliorate instabilities caused by changes in crosscutting features. Hence, it is important to understand which aspect-oriented (AO) and non-aspect-oriented techniques better cope with PLA stability through evolution.

Objective

This paper evaluates the positive and negative change impact of component and aspect based design on PLAs. The objective of the evaluation is to assess how aspects and components promote PLA stability in the presence of various types of evolutionary change. To support a broader analysis, we also evaluate the PLA stability of a hybrid approach (i.e. combined use of aspects and components) against the isolated use of component-based, OO, and AO approaches.

Method

An quantitative and qualitative analysis of PLA stability which involved four different implementations of a PLA: (i) an OO implementation, (ii) an AO implementation, (iii) a component-based implementation, and (iv) a hybrid implementation where both components and aspects are employed. Each implementation has eight releases and they are functionally equivalent. We used conventional metrics suites for change impact and modularity to measure the architecture stability evaluation of the 4 implementations.

Results

The combination of aspects and components promotes superior PLA resilience than the other PLAs in most of the circumstances.

Conclusion

It is concluded that the combination of aspects and components supports the design of high cohesive and loosely coupled PLAs. It also contributes to improve modularity by untangling feature implementation.     

Increased variation of the response index of nociception during noxious stimulation in patients during general anaesthesia

Objective

Analgesia is an important part of general anaesthesia, but no direct indicators of nociceptive-anti-nociceptive balance have been validated in detail. The Response Index of Nociception (RN) is a multiparameter approach which combines photoplethysmographic waveform (PPG), State Entropy (SE), Response Entropy (RE), and heart rate variability (HRV). We aimed at evaluating RN during general anaesthesia; especially we wanted to compare pre- and post-index values of certain noxious stimuli to the average index values. Our assumption was that RN could be a useful indicator of nociceptive-anti-nociceptive balance during the surgery.

Methods

Sixty women undergoing gynaecological or breast surgery participated in the study. All patients had elective surgery and anaesthesia was maintained with propofol-remifentanil target controlled infusion. Neuromuscular blocking agent rocuronium was used at the beginning of the surgery. Electrocardiography (ECG), photoplethysmography (PPG) and electroencelophalography (EEG) were registered and extracted off-line. An index, reflecting amplitude and frequency of occurrence of abrupt increases (“peaks”) in the RN was evaluated during surgery in general and around occurrences of predefined noxious stimuli in particular.

Results

Fifty-four patients were eligible for analysis. Patient movement was associated with increased index values, both before and after the event. Post-event values of the index for intubation and skin incision were higher than its intra-surgery baseline, while pre-event values remained unchanged.

Conclusion

Changes in RN can be used to detect noxious stimuli during surgery. RN also predicted movement in our patients under propofol-remifentanil anaesthesia.     

Research synthesis in software engineering: A tertiary study

Context

Comparing and contrasting evidence from multiple studies is necessary to build knowledge and reach conclusions about the empirical support for a phenomenon. Therefore, research synthesis is at the center of the scientific enterprise in the software engineering discipline.

Objective

The objective of this article is to contribute to a better understanding of the challenges in synthesizing software engineering research and their implications for the progress of research and practice.

Method

A tertiary study of journal articles and full proceedings papers from the inception of evidence-based software engineering was performed to assess the types and methods of research synthesis in systematic reviews in software engineering.

Results

As many as half of the 49 reviews included in the study did not contain any synthesis. Of the studies that did contain synthesis, two thirds performed a narrative or a thematic synthesis. Only a few studies adequately demonstrated a robust, academic approach to research synthesis.

Conclusion

We concluded that, despite the focus on systematic reviews, there is limited attention paid to research synthesis in software engineering. This trend needs to change and a repertoire of synthesis methods needs to be an integral part of systematic reviews to increase their significance and utility for research and practice.     

Problems in the interplay of development and IT operations in system development projects: A Delphi study of Norwegian IT experts

Context

The assumption of the presented work is that the ability of system developers and IT operations personnel to cooperate effectively in system development projects has great impact on the quality of the final system solution, as well as on the service level of its subsequent operation.

Objective

The present research explores the interplay of system development and IT operations and the challenges they are meeting. We are especially interested in identifying problems encountered between these two parties in system development projects.

Method

We identify and rank problems by using a ranking-type Delphi study. We involved 42 Norwegian IT experts and split them into three expert panels: system developers, IT operations personnel and system owners. We then guided them through the three phases of the Delphi method - brainstorming, reduction and ranking.

Results

A comprehensive list of 66 problems, organized into seven groups, is compiled. Through a selection and ranking procedure, the panels found the following to be the six most serious problems in the interplay of system development and IT operations: (1) IT operations not being involved in the requirements specification; (2) poor communication and information flow; (3) unsatisfactory test environments; (4) lack of knowledge transfer; (5) systems being put into production before they are complete; and (6) operational routines not being established prior to deployment.

Conclusion

The sheer amount and variety of problems mentioned and the respondents’ explanations confirm that this interplay needs attention; the parties agree that they do not cooperate effectively in development projects. The results imply that IT operations should be regarded as an important stakeholder throughout several systems development activities, especially requirements analysis, testing and deployment. Moreover, such involvement should be facilitated by an increased focus on enhancing cooperation and communication.     

IDD Info: A software to manage surveillance data of Iodine Deficiency Disorders

Objectives

IDD info, a new software for managing survey data of Iodine Deficiency Disorders (IDD), is presented in this paper. IDD Info aims to create IDD project databases, process, analyze various national or regional surveillance data and form final report. It has series measures of choosing database from existing ones, revising it, choosing indicators from pool to establish database and adding indicators to pool. It also provides simple tools to scan one database and compare two databases, to set IDD standard parameters, to analyze data by single indicator and multi-indicators, and finally to form typeset report with content customized.

Methods

IDD Info was developed using Chinese national IDD surveillance data of 2005. Its validity was evaluated by comparing with survey report given by China CDC.

Results

The IDD Info is a professional analysis tool, which succeeds in speeding IDD data analysis up to about 14.28% with respect to standard reference routines. It consequently enhances analysis performance and user compliance.

Conclusions

IDD Info is a practical and accurate means of managing the multifarious IDD surveillance data that can be widely used by non-statisticians in national and regional IDD surveillance.     

A unit test approach for database schema evolution

Context

The constant changes in today’s business requirements demand continuous database revisions. Hence, database structures, not unlike software applications, deteriorate during their lifespan and thus require refactoring in order to achieve a longer life span. Although unit tests support changes to application programs and refactoring, there is currently a lack of testing strategies for database schema evolution.

Objective

This work examines the challenges for database schema evolution and explores the possibility of using various testing strategies to assist with schema evolution. Specifically, the work proposes a novel unit test approach for the application code that accesses databases with the objective of proactively evaluating the code against the altered database.

Method

The approach was validated through the implementation of a testing framework in conjunction with a sample application and a relatively simple database schema. Although the database schema in this study was simple, it was nevertheless able to demonstrate the advantages of the proposed approach.

Results

After changes in the database schema, the proposed approach found all SELECT statements as well as the majority of other statements requiring modifications in the application code. Due to its efficiency with SELECT statements, the proposed approach is expected to be more successful with database warehouse applications where SELECT statements are dominant.

Conclusion

The unit test approach that accesses databases has proven to be successful in evaluating the application code against the evolved database. In particular, the approach is simple and straightforward to implement, which makes it easily adoptable in practice.