A Systematic Mapping Study of HCI Practice Research

ABSTRACT

Human–computer interaction (HCI) practice has emerged as a research domain in the HCI field and is growing. The need to transfer HCI practices to the industry began significantly with the works of Nielsen on usability engineering. To date, methods and techniques for designing, evaluating, and implementing interactive systems for human use have continued to emerge. It is, therefore, justified to conduct a systematic mapping study to determine the landscape of HCI practice research. A Systematic Mapping Study method was used to map 142 studies according to research type, topic, and contribution. These were then analyzed to determine an overview of HCI practice research. The objective was to analyze studies on HCI practice and present prominent issues that characterize the HCI practice research landscape. Second, to identify pressing challenges regarding HCI practices in software/systems development companies. The results show that HCI practice research has steadily increased since 2012. The majority of the studies explored focused on evaluation research that largely contributed to the evaluation methods or processes. Most of the studies were on design tools and techniques, design methods and contexts, design work and organizational culture, and collaboration and team communication. Interviews, case studies, and survey methods have been prominently used as research methods. HCI techniques are mostly used during the initial phase of development and during evaluation. HCI practice challenges in companies are mostly process-related and on performance of usability and user experience activities. The major challenge seems to be to find a way to collect and incorporate user feedback in a timely manner, especially in agile processes. There are areas identified in this study as needing more research.     

Agile knowledge management: a survey of Indian perceptions

This paper is a pivot step to find out the level of KM adoption of Indian software organizations which are using agile practices for software development. After reviewing the literature it was found although many practitioners believe that KM practices are embodied in agile methodologies but it is not documented efficiently. Survey was used to take perception of agile organizations. KM practices are grouped into seven factors and respondents are taken from Indian software industry. Organizations are divided into different demographics: size, core area, type. Different statistical tests are used to conclude outputs of the questionnaire. It is found that Indian software industry working with agile practices lacks in providing any formal head for KM positioning. Learning and sharing through discussion forms is the most used practices among all respondents of organizations, whereas least accepted practice is dependent upon documents for transfer of knowledge. Indian organizations also lack in appointing a formal head who can provide guidelines for agile practice.     

Influence, discretion and time available: a case study of HCI practice in software development

In the field of human-computer interaction, reports of the involvement of its practitioners in system development projects are rarely available for general scrutiny. The paper draws upon the experience of an HCI team at work within a large collaborative software development project. This experience of four years of HCI practice suggests three key, interdependent, factors that are central to the effectiveness of HCI input. The factors are influence, discretion and time available, and are discussed in the context of other report, of the role of HCI practitioners in the field. A number of issues are identified about the nature and scope of HCI in practice. The experience reported is relevant to software development in general, particularly where there are several groups working, sometimes in different sites, towards a unified outcome.     

Evaluating performance in the development of software-intensive products

ContextOrganizational performance measurements in software product development have received a lot of attention in the literature. Still, there is a general discontent regarding the way performance is evaluated in practice, with few studies really focusing on why this is the case. In this paper research focusing on the context of developing software-intensive products in large established multi-national organizations is reported on.ObjectiveThe purpose of this research is to investigate performance measurement practices related to software product development activities. More specifically, focus is on exploring how managers engaged in software product development activities perceive and evaluate performance in large organizations from a managerial perspective.MethodThe research approach pursued in this research consist of exploratory multiple case studies. Data is collected mainly through 54 interviews in five case studies in large international organizations developing software-intensive products in Sweden. Focused group interviews with senior managers from eight companies have also been used in the data collection.ResultsThe results of this research indicate that managers within software product development in general are dissatisfied with their current way of evaluating performance. Performance measurements and the perception of performance are today focused on cost, time, and quality, i.e. what is easily measurable and not necessarily what is important. The dimensions of value creation and learning are missing. Moreover, measurements tend to be result oriented, rather than process oriented, making it difficult to integrate these measurements in the management practices.ConclusionManagers that are dissatisfied with their performance measurement system and want to improve the current situation should not start by focusing on the current measurements directly; instead they should focus on how the organization perceives performance and how important performance criteria are being developed. By developing relevant performance criteria the first step in developing an effective performance measurement system is made. Moreover, it is concluded that manager’s perception of performance is affected by the currently used measurements, hence limiting the scope of the performance criteria. Thus, a change in the way managers perceive performance is necessary before there can be any changes in the way performance is evaluated.     

Guest Editors' Introduction: Software Testing Practices in Industry

Four papers and a roundtable discussion shed light on the current state of software testing practices. Case studies from industry experience address topics including unit-testing practices, agile testing, and automating software testing. Although many of these approaches show promise, software testing is still one of the more neglected practices within the software development life cycle. Suggestions for improvements in industry and academia are offered.     

Transitioning international software engineering standards to academia: Analyzing the results of the adoption of ISO/IEC 29110 in four Mexican universities

Software standards, targeted for the software industry, were developed to contribute to the development of quality products within budget and schedule, by optimizing efforts and resources. For small companies, the largest percentage of software companies in Mexico, they are fundamental for their growth and survival. However, academic programs do not always match industry requirements. In previous studies, the curricula in Computer Science and Informatics, and Software Engineering, of 4 Mexican universities, were compared with two software industry standards: the MoProSoft standard, a Mexican standard designed for organizations having up to 50 people and the Basic profile of the ISO/IEC 29110 developed specifically for organizations having up to 25 people. The analysis of the academic programs showed a better coverage of ISO/IEC 29110 than MoProSoft. In this paper, these two standards are mapped to understand the results of the analysis in detail and provide recommendations regarding academic programs. The analysis provides an evidence that the processes of the Basic profile of ISO/IEC 29110 are better covered by the universities curricula because the processes provides the minimal set of practices to be performed while a project is executed from the beginning until the delivery of a software. In addition, this mapping presents a clear differentiation between these two standards that might help Software Development Centers to understand where to start in the implementation of one of them.     

A framework for creating hybrid-open source software communities

Abstract The open source software (OSS) model is a fundamentally new and revolutionary way to develop software. The success of the OSS model is also setting the stage for a structural change in the software industry; it is beginning to transform software industry from manufacturing to a service industry. Despite the success of the OSS model, for‐profit organizations are having difficulty building a business model around the open source paradigm. Whereas there are some isolated empirical studies, little rigorous research has been done on how traditional organizations can implement and benefit from OSS practices. This research explores how organizations can foster an environment similar to OSS to manage their software development efforts to reap its numerous advantages. Drawing on organizational theory, we develop a framework that guides the creation and management of a hybrid‐OSS community within an organization. We discuss the implications of this framework and suggest areas for future research.     

Achieving traceability in large scale continuous integration and delivery deployment,usage and validation of the eiffel framework

The importance of traceability in software development has long been recognized, not only for reasons of legality and certification, but also to enable the development itself. At the same time, organizations are known to struggle to live up to traceability requirements, and there is an identified lack of studies on traceability practices in the industry, not least in the area of tooling and infrastructure. This paper presents, investigates and discusses Eiffel, an industry developed solution designed to provide real time traceability in continuous integration and delivery. The traceability needs of industry professionals are also investigated through interviews, providing context to that solution. It is then validated through further interviews, a comparison with previous traceability methods and a review of literature. It is found to address the identified traceability needs and found in some cases to reduce traceability data acquisition times from days to minutes, while at the same time alternatives offering comparable functionality are lacking. In this work, traceability is shown not only to be an important concern to engineers, but also regarded as a prerequisite to successful large scale continuous integration and delivery. At the same time, promising developments in technical infrastructure are documented and clear differences in traceability mindset between separate industry projects is revealed.     

Understanding agile software development practices using shared mental models theory

ContextAgile software development is an alternative software development methodology that originated from practice to encourage collaboration between developers and users, to leverage rapid development cycles, and to respond to changes in a dynamic environment. Although agile practices are widely used in organizations, academics call for more theoretical research to understand the value of agile software development methodologies.ObjectiveThis study uses shared mental models theory as a lens to examine practices from agile software methodologies to understand how agile practices enable software development teams to work together to complete tasks and work together effectively as a team.MethodA conceptual analysis of specific agile practices was conducted using the lens of shared mental models theory. Three agile practices from Xtreme Programming and Scrum are examined in detail, system metaphor, stand-up meeting, and on-site customer, using shared mental models theory.ResultsExamining agile practices using shared mental models theory elucidates how agile practices improve collaboration during the software development process. The results explain how agile practices contribute toward a shared understanding and enhanced collaboration within the software development team.ConclusionsThis conceptual analysis demonstrates the value of agile practices in developing shared mental models (i.e. shared understanding) among developers and customers in software development teams. Some agile practices are useful in developing a shared understanding about the tasks to be completed, while other agile practices create shared mental models about team processes and team interactions. To elicit the desired outcomes of agile software development methods, software development teams should consider whether or not agile practices are used in a manner that enhances the team’s shared understanding. Using three specific agile practices as examples, this research demonstrates how theory, such as shared mental models theory, can enhance our understanding regarding how agile practices are useful in enhancing collaboration in the workplace.     

Requirements reuse and requirement patterns: a state of the practice survey

Requirements engineering is a discipline with numerous challenges to overcome. One of these challenges is the implementation of requirements reuse approaches. Although several theoretical proposals exist, little is known about the practices that are currently adopted in industry. Our goal is to contribute to the investigation of the state of the practice in the reuse of requirements, eliciting current practices from practitioners, and their opinions whenever appropriate. Besides reuse in general, we focus on requirement patterns as a particular strategy to reuse. We conducted an exploratory survey based on an online questionnaire. We received 71 responses from requirements engineers with industrial experience in the field, which were analyzed in order to derive observations. Although we found that a high majority of respondents declared some level of reuse in their projects (in particular, non-functional requirements were identified as the most similar and recurrent among projects), it is true that only a minority of them declared such reuse as a regular practice. Larger IT organizations and IT organizations with well-established software processes and methods present higher levels of reuse. Ignorance of reuse techniques and processes is the main reason preventing wider adoption. From the different existing reuse techniques, the simplest ones based on textual copy and subsequent tailoring of former requirements are the most adopted techniques. However, participants who apply reuse more often tend to use more elaborate techniques. Opinions of respondents about the use of requirement patterns show that they can be expected to mitigate problems related to the quality of the resulting requirements, such as lack of uniformity, inconsistency, or ambiguity. The main reasons behind the lack of adoption of requirement patterns by practitioners (in spite of the increasing research approaches proposed in the community) are related to the lack of a well-defined reuse method and involvement of requirement engineers. The results of our paper are interesting for practitioners since we highlight relevant observations from the survey participants’ experiences when deciding to implement requirements reuse practices. We also suggest future lines of research based on the needs pointed out in the results.     

Extracting and analyzing time-series HCI data from screen-captured task videos

Recent years have witnessed the increasing emphasis on human aspects in software engineering research and practices. Our survey of existing studies on human aspects in software engineering shows that screen-captured videos have been widely used to record developers’ behavior and study software engineering practices. The screen-captured videos provide direct information about which software tools the developers interact with and which content they access or generate during the task. Such Human-Computer Interaction (HCI) data can help researchers and practitioners understand and improve software engineering practices from human perspective. However, extracting time-series HCI data from screen-captured task videos requires manual transcribing and coding of videos, which is tedious and error-prone. In this paper we report a formative study to understand the challenges in manually transcribing screen-captured videos into time-series HCI data. We then present a computer-vision based video scraping technique to automatically extract time-series HCI data from screen-captured videos. We also present a case study of our scvRipper tool that implements the video scraping technique using 29-hours of task videos of 20 developers in two development tasks. The case study not only evaluates the runtime performance and robustness of the tool, but also performs a detailed quantitative analysis of the tool’s ability to extract time-series HCI data from screen-captured task videos. We also study the developer’s micro-level behavior patterns in software development from the quantitative analysis.     

A Representational Framework for Scenarios of System Use

Scenarios are becoming widely used in three areas of system development: software engineering, human–computer interaction (HCI), and organisational process design. There are many reasons to use scenarios during system design. The one usually advanced in support of the practice is to aid the processes of validating the developers’ understanding of the customers’ or users’ work practices, organisational goals and structures, and system requirements. All three areas identified above deal with these processes, and not surprisingly this has given rise to a profusion of scenario-based practices and representations. Yet there has been little analysis of why scenarios should be useful, let alone whether they are. Only by having such a framework for understanding what scenarios are, and what they are for, can we begin to evaluate different scenario approaches in specific development contexts. This paper is a contribution toward such a framework. We lay out a space of representational possibilities for scenarios and enumerate a set of values or criteria that are important for different uses of scenarios. We then summarise several salient representations drawn from the software engineering, HCI, and organisational process design communities to clarify how these representational choices contribute to or detract from the goals of the respective practices. Finally, we discuss how scenario representations from one area of design may be useful in others, and we discuss the relationship between these representations and other significant early-design and requirements engineering practices.     

A methodology for the selection of requirements engineering techniques

The complexity of software projects as well as the multidisciplinary nature of requirements engineering (RE) requires developers to carefully select RE techniques and practices during software development. Nevertheless, the selection of RE techniques is usually based on personal preference or existing company practice rather than on characteristics of the project at hand. Furthermore, there is a lack of guidance on which techniques are suitable for a certain project context. So far, only a limited amount of research has been done regarding the selection of RE techniques based on the attributes of the project under development. The few approaches that currently exist for the selection of RE techniques provide only little guidance for the actual selection process. We believe that the evaluation of RE techniques in the context of an application domain and a specific project is of great importance. This paper describes a Methodology for Requirements Engineering Techniques Selection (MRETS) as an approach that helps requirements engineers select suitable RE techniques for the project at hand. The MRETS has three aspects: Firstly, it aids requirements engineers in establishing a link between the attributes of the project and the attributes of RE techniques. Secondly, based on the evaluation schema proposed in our research, MRETS provides an opportunity to analyze RE techniques in detail using clustering. Thirdly, the objective function used in our approach provides an effective decision support mechanism for the selection of RE techniques. This paper makes contributions to RE techniques analysis, the application of RE techniques in practice, RE research, and software engineering in general. The application of the proposed methodology to an industrial project provides preliminary information on the effectiveness of MRETS for the selection of RE techniques.     

Comparing practices for reuse in integration‐oriented software product lines and large open source software projects

This article compares the organization and practices for software reuse in integration‐oriented software product lines (SPLs) and open source software projects. The main observation is that both approaches are successful regarding large variability and reuse, but differ widely in their practices and organization. To capture practices in large open source projects, we describe an open compositional model, which reflects their more decentralized organization of software development. We capture key practices and organizational forms for this and validate these by comparing four case studies of this model. Two of these studies are based on published SPL case studies, for the other two we analyze the practices in two large and successful open source projects based on their published developer documentation. Our analysis highlights key differences between the practices in the two open source organizations and the more integrational practices used in the other two cases. Finally, we discuss which practices are successful in which environment and how the current practices can move towards more open, widely scoped and distributed software development. Copyright © 2010 John Wiley & Sons, Ltd.     

Product line engineering, the state of the practice

Software product lines can be very powerful for ensuring quality, economic efficiency, and manageability of software system families. SPLs are relevant to large industrial enterprises that want to better manage their software-intensive systems' development. They can also provide small start-ups with unique and striking business models. However, SPL technology can be difficult to implement and maintain. A work group comprising five organizations has investigated and compared SPL practices. The results detail the state of SPL practice.     

IwC Special Issue “Feminism and HCI: New Perspectives” Special Issue Editors’ Introduction

As a word and as a set of theories and practices, feminism is a poorly understood concept. However, feminist perspectives have a lot in common with user- and value-centered design processes such as those espoused within the field of Human Computer Interaction. Examples include consideration of alternative viewpoints, considerations of agency (who get to say/do what and under what circumstances) and the development of reflective and reflexive methods for understanding how, when, where and why people do what they do. In the “Feminism and HCI: New Perspectives” special issue, we have invited researchers and practitioners to reflect on the ways in which feminist thinking, theory, and practice can and does have an impact on the field of Human Computer Interaction. This introductory editorial offers more background to our view that there is great value to understanding the actual and potential impact of feminist thinking on HCI, followed by a précis of each paper. We close with some observations regarding common themes, points of contention and possibilities for future work.     

Current state and potential of variability management practices in software-intensive SMEs: Results from a regional industrial survey

ContextMore and more, small and medium-sized enterprises (SMEs) are using software to augment the functionality of their products and offerings. Variability management of software is becoming an interesting topic for SMEs with expanding portfolios and increasingly complex product structures. While the use of software product lines to resolve high variability is well known in larger organizations, there is less known about the practices in SMEs.ObjectiveThis paper presents results from a survey of software developing SMEs. The purpose of the paper is to provide a snapshot of the current awareness and practices of variability modeling in organizations that are developing software with the constraints present in SMEs.MethodA survey with questions regarding the variability practices was distributed to software developing organizations in a region of Sweden that has many SMEs. The response rate was 13% and 25 responses are used in this analysis.ResultsWe find that, although there are SMEs that develop implicit software product lines and have relatively sophisticated variability structures for their solution space, the structures of the problem space and the product space have room for improvement.ConclusionsThe answers in the survey indicate that SMEs are in situations where they can benefit from more structured variability management, but the awareness need to be raised. Even though the problem space similarity is high, there is little reuse and traceability activities performed. The existence of SMEs with qualified variability management and product line practices indicates that small organizations are capable to apply such practices.     

In the 25 years since The Mythical Man-Month what have we learned about project management?

This paper discusses Brooks’ The Mythical Man-Month, a landmark work in the software project management field, and compares the software project management advice given there with practices employed some 25 years later. To find out the state of today's practice 20 experienced software developers were interviewed regarding their impressions of factors leading to success or failure of software development projects. Their observations are compared with the points raised by Brooks in his seminal work.     

Towards the development of classes of interaction: initial illustration with reference to off-load planning

In recent years, a number of difficulties in designing interactions between military personnel and their command and control systems have been identified. These difficulties have been attributed to a lack of carry forward between procurement projects. This paper suggests that lack of carry forward is an integral part of current 'case by case' approaches to HCI. Consequently, a fundamentally different approach to HCI is required. The approach suggested here is a class approach. A class approach to HCI makes classinstance relationships between knowledge representations explicit by organising knowledge representations into class hierarchies. Given such hierarchies, procurement projects may consider the relevance of existing knowledge by attempting to locate the problem at hand within the hierarchy. Thus, a class approach to HCI may encourage carry forward by providing: (a) the opportunity to develop multiple instances of classes of interaction by specialising and instantiating class knowledge representations for the instances at hand; (b) the opportunity to apply research knowledge at different levels of development-to the development of the class and the instance (not just the case); and (c) an additional means of reasoning about the completeness/selectivity of instance knowledge representationswith respect to relevant, super-ordinate representations. This paper presents an initial illustration of a class approach to HCI. It identifies some key characteristics of a class approach to HCI, and then presents research and development work which exhibits these characteristics. Such an illustration is required, because current understanding about the nature of HCI concerns, and the relationships between HCI knowledge, practices and problems is such that one may not assume that all desirable approaches to HCI are necessarily realisable. Successful initial illustration provides an additional, encouraging precedent for full development of the approach.