Academic methods for usability evaluation of serious games: a systematic review

Multimedia Tools and Applications - In the last years, there has been an increasing interest in the design of video games as a tool for education, training, health promotion, socialization, etc....     

Success factors for serious games to enhance learning: a systematic review

There is no doubt that an abundance of factors exists that makes learning with serious games successful. Research articles reporting on these factors, however, tend to focus on select serious game elements and do not combine all salient factors for successful learning with serious games. Addressing this gap is a necessity for the success of serious games and may even alleviate long-standing debates about pedagogy over enjoyment, how much realism is enough or whether artificial intelligence is worth the cost. This article examines existing academic literature from 2000 to 2015, extracting shared serious game success factors that have had an encouraging impact on gameful learning experiences. As such, we subsequently aim to withdraw the field from a perpetual spiral of does-my-game-work research toward more worthwhile why-does-my-game-not-work research. Qualitative content analysis through the constant comparison method (CCM) analyzed a total of 63 articles from a variety of recognized electronic libraries and databases. Through this analysis, we reveal five central serious game themes: backstory and production; realism; artificial intelligence and adaptivity; interaction; and feedback and debriefing, all of which require deliberate intertwining with pedagogical content to ensure successful learning. This review unravels each of the five themes into their constituent factors and consequently presents the factors as practical guidelines that serious games producers should strive to include in their game productions. Applying these recommendations whenever serious games are considered will provide a foundation for effective gameful learning experiences.     

Usability evaluation methods for the web: A systematic mapping study

Context

In recent years, many usability evaluation methods (UEMs) have been employed to evaluate Web applications. However, many of these applications still do not meet most customers’ usability expectations and many companies have folded as a result of not considering Web usability issues. No studies currently exist with regard to either the use of usability evaluation methods for the Web or the benefits they bring.

Objective

The objective of this paper is to summarize the current knowledge that is available as regards the usability evaluation methods (UEMs) that have been employed to evaluate Web applications over the last 14 years.

Method

A systematic mapping study was performed to assess the UEMs that have been used by researchers to evaluate Web applications and their relation to the Web development process. Systematic mapping studies are useful for categorizing and summarizing the existing information concerning a research question in an unbiased manner.

Results

The results show that around 39% of the papers reviewed reported the use of evaluation methods that had been specifically crafted for the Web. The results also show that the type of method most widely used was that of User Testing. The results identify several research gaps, such as the fact that around 90% of the studies applied evaluations during the implementation phase of the Web application development, which is the most costly phase in which to perform changes. A list of the UEMs that were found is also provided in order to guide novice usability practitioners.

Conclusions

From an initial set of 2703 papers, a total of 206 research papers were selected for the mapping study. The results obtained allowed us to reach conclusions concerning the state-of-the-art of UEMs for evaluating Web applications. This allowed us to identify several research gaps, which subsequently provided us with a framework in which new research activities can be more appropriately positioned, and from which useful information for novice usability practitioners can be extracted.     

A systematic literature review of empirical evidence on computer games and serious games

This paper examines the literature on computer games and serious games in regard to the potential positive impacts of gaming on users aged 14 years or above, especially with respect to learning, skill enhancement and engagement. Search terms identified 129 papers reporting empirical evidence about the impacts and outcomes of computer games and serious games with respect to learning and engagement and a multidimensional approach to categorizing games was developed. The findings revealed that playing computer games is linked to a range of perceptual, cognitive, behavioural, affective and motivational impacts and outcomes. The most frequently occurring outcomes and impacts were knowledge acquisition/content understanding and affective and motivational outcomes. The range of indicators and measures used in the included papers are discussed, together with methodological limitations and recommendations for further work in this area.     

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented.     

Semantic web technologies applied to software accessibility evaluation: a systematic literature review

Universal Access in the Information Society - Software accessibility is a current field of interest. Governments and organizations boost initiatives and legislations to assure universal access. On...     

严肃游戏中虚拟角色行为建模综述

严肃游戏是计算机游戏一个新的发展方向,可以提供形象互动的模拟教学环境,已经广泛应用于科学教育、康复医疗、应急管理、军事训练等领域。虚拟角色是严肃游戏中模拟具有生命特征的图形实体,行为可信的虚拟角色能够提升用户使用严肃游戏的体验感。严肃游戏中的图形渲染技术已经逐步成熟,而虚拟角色行为建模的研究尚在初级阶段。可信的虚拟角色必须能够具有感知、情绪和行为能力。本文分别从游戏剧情与行为、行为建模方法、行为学习和行为建模评价等4个方面来分析虚拟角色行为建模研究。分析了有限状态机和行为树的特点,讨论了虚拟角色的行为学习方法。指出了强化学习的关键要素,探讨了深度强化学习的应用途径。综合已有研究,归纳了虚拟角色行为框架,该框架主要包括感觉输入、知觉分析、行为决策和动作4大模块。从情感计算的融入、游戏剧情和场景设计、智能手机平台和多通道交互4个角度讨论需要进一步研究的问题。虚拟角色的行为建模需要综合地考虑游戏剧情、机器学习和人机交互技术,构建具有自主感知、情绪、行为、学习能力、多通道交互的虚拟角色能够极大地提升严肃游戏的感染力,更好地体现寓教于乐。     

Software engineering research for computer games: A systematic review

ContextCurrently, computer game development is one of the fastest growing industries in the worldwide economy. In addition to that, computer games are rapidly evolving in the sense that newer game versions arrive in a very short interval. Thus, software engineering techniques are needed for game development in order to achieve greater flexibility and maintainability, less cost and effort, better design, etc. In addition, games present several characteristics that differentiate their development from classical software development.ObjectiveThis study aims to assess the state of the art on research concerning software engineering for computer games and discuss possible important areas for future research.MethodWe employed a standard methodology for systematic literature reviews using four well known digital libraries.ResultsSoftware engineering for computer games is a research domain that has doubled its research activity during the last 5 years. The dominant research topic has proven to be requirements engineering, while topics such as software verification and maintenance have been neglected up to now.ConclusionThe results of the study suggest that software engineering for computer games is a field that embraces many techniques and methods from conventional software engineering and adapts them so as to fit the specific requirements of game development. In addition to that, the study proposes the employment of more elaborate empirical methods, i.e. controlled experiments and case studies, in game software engineering research, which, have not been extensively used up to now.     

A case study of user immersion-based systematic design for serious heritage games

Modern digital technologies support the preservation and transfer of cultural heritage information via devices and applications such as digital storage systems, electronic books and virtual museums. Advances in virtual and augmented reality, real-time computer graphics and computer games have made it possible to construct large virtual environments in which users may experience cultural heritage through a variety of interactions and immersions. Thus, an emerging problem is to implement an appropriate systematic design method for achieving various types of entertainment, learning and information transfer. This paper proposes two important design factors that impact on user immersion in serious heritage games: user interface space volume and subsystem sequence. The impact of the two factors on proposed systematic design methods was investigated through comparative studies by implementing a serious heritage game system on three different platforms.     

Usability risk level evaluation for physical user interface of mobile phone

The mobile phone has become an indispensable device in everyday life. However, many usability problems were generated by the multi-functionality of mobile phones. There was an important increase on the user interface (UI) design and usability. With the introduction of a variety of forms and keypads in mobile phones, the user's need of controllability, grip-stability, and usability were raised in the UI aspect. Therefore, in this study, we investigated the physical user interface (PUI) elements of mobile phones, and defined the degree of usability risk such as key level value, function level value, and grip level value. Also, this study proposed an evaluation framework to quantitatively measure the usability risk in the mobile phone PUI elements. For this, the mobile phone PUI elements were analyzed and classified by key type, function, and form-factor. Hence, in the case study was investigated and evaluated 133 mobile phones in the market by the proposed framework. As a result, the proposed evaluation framework enabled the measurement of the mobile phone usability risk level in the early stages of the concept design. This made possible the prediction of design problems related with the mobile phone PUI.     

Engagement in digital entertainment games: A systematic review

Since their introduction over 40 years ago, digital entertainment games have become one of the most popular leisure activities globally. While digital games clearly provide highly engaging activities, the nature of this engagement is not well understood. The current study aims to advance our understanding by reporting a systematic review of recent literature addressing engagement in computer games. The papers in the review comprise a sub-sample of papers relating to engagement in digital games that was selected from a broader literature search carried out on the outcomes and impacts of playing computer games. A diverse range of studies was identified that examined varied aspects of engagement in games including subjective experiences while playing games, the physiological concomitants of these experiences, motives for playing games, game usage and time spent playing games and the impact of playing on life satisfaction. A narrative review was carried out to capture these diverse aspects of engagement and to develop a more coherent understanding of engagement in computer games.     

Feature selection methods for text classification: a systematic literature review

Artificial Intelligence Review - Feature Selection (FS) methods alleviate key problems in classification procedures as they are used to improve classification accuracy, reduce data dimensionality,...     

OMR metrics and evaluation: a systematic review

Multimedia Tools and Applications - Music is rhythm, timbre, tones, intensity and performance. Conventional Western Music Notation (CWMN) is used to generate Music Scores in order to register music...     

A review of immersive virtual reality serious games to enhance learning and training

Multimedia Tools and Applications - The merger of game-based approaches and Virtual Reality (VR) environments that can enhance learning and training methodologies have a very promising future,...     

Studying the effects of computer serious games on people with intellectual disabilities or autism spectrum disorder: A systematic literature review

This study examines the available literature on the effects of serious games on people with intellectual disabilities or autism spectrum disorder. The studies were categorized based on the limitations in skills that these people address. Fifty‐four studies were selected, from different data sources. These studies address limitations in intellectual functioning and adaptive behaviour. The results showed that the majority of studies on the effects of serious games for people with intellectual disabilities or autism spectrum disorder had a positive impact. Also, most studies for people with autism aim to improve social and communicational skills, whereas conceptual and cognitive skills were mainly observed in studies for people with intellectual disabilities. Although this study covers serious games in all platforms or delivery systems, the overwhelming majority of the presented studies include computer serious games. Computer‐assisted learning through serious games is considered quite promising for people with intellectual disabilities or autism spectrum disorder.     

Usability of state public health department websites for communication during a pandemic: A heuristic evaluation

Study aims were to investigate how usable COVID-19 dedicated state public health websites in the US were, and whether case counts in different geographical regions in the US were related to website usability. 16 state websites representing the 2 highest and the 2 lowest case count states in each region were selected. Five experts used a heuristic evaluation procedure to independently rate all 16 websites on a severity scale of 0–4. Usability criteria published by the US Department of Health and Human Services and criteria on risk communication and data dashboards were used. Analyses involved cross tabulation of usability criteria with case counts, comparison of usability scores using Mood's median tests, test of differences in average usability scores using ANOVA and post-hoc tests, and identification of correlations between case counts and usability scores. Results from the Mood's median test showed that the median usability scores for the states were significantly different from each other at the 5% level of significance (df = 15, chi-square = 38.40; p = 0.001). ANOVA showed statistically significant differences between the mean usability scores for the states at the 5% level of significance (F = 6.33, p < 0.05). Although not statistically significant, results from a correlation analysis between case count and usability scores showed a negative correlation (r = -0.209, p = 0.437) indicating that the higher the case count, the better the usability score. Overall, the websites fared well on usability, but many websites were used as an information and data repository. These websites must communicate infection risk better.Relevance to industryThe study applies to public health agency websites that communicate essential information during a pandemic.     

Modeling the efficacy of persuasive strategies for different gamer types in serious games for health

Persuasive games for health are designed to alter human behavior or attitude using various Persuasive Technology (PT) strategies. Recent years have witnessed an increasing number of such games, which treat players as a monolithic group by adopting a one-size-fits-all design approach. Studies of gameplay motivation have shown that this is a bad approach because a motivational approach that works for one individual may actually demotivate behavior in others. In an attempt to resolve this weakness, we conducted a large-scale study on 1,108 gamers to examine the persuasiveness of ten PT strategies that are commonly employed in persuasive game design, and the receptiveness of seven gamer personalities (gamer types identified by BrianHex) to the ten PT strategies. We developed models showing the receptiveness of the gamer types to the PT strategies and created persuasive profiles, which are lists of strategies that can be employed to motivate behavior for each gamer type. We then explored the differences between the models and, based on the results, proposed two approaches for data-driven persuasive game design. The first is the one-size-fits-all approach that will motivate a majority of gamers, while not demotivating any player. The second is the personalized approach that will best persuade a particular type of gamer. We also compiled a list of the best and the worst strategies for each gamer type. Finally, to bridge the gap between game design and PT researchers, we map common game mechanics to the persuasive system design strategies.     

APRehab: a methodology for serious games design oriented to psychomotor rehabilitation in children with hearing impairments

Universal Access in the Information Society - In this work, we present APRehab, a methodology for the design of serious games for psychomotor rehabilitation of children with hearing impairments....     

Interoperability evaluation models: A systematic review

Interoperability is defined as the ability for two (or more) systems or components to exchange information and to use the information that has been exchanged. There is increasing demand for interoperability between individual software systems. Developing an interoperability evaluation model between software and information systems is difficult, and becoming an important challenge. An interoperability evaluation model allows knowing the degree of interoperability, and lead to the improvement of interoperability. This paper describes the existing interoperability evaluation models, and performs a comparative analysis among their findings to determine the similarities and differences in their philosophy and implementation. This analysis yields a set of recommendations for any party that is open to the idea of creating or improving an interoperability evaluation model.