An initial industrial evaluation of interactive search-based testing for embedded software |
| |
| Affiliation: | 1. Blekinge Institute of Technology, Karlskrona, Sweden;2. Chalmers and the University of Gothenburg, Gothenburg, Sweden;1. School of Computer Science and Technology, Xidian University, Xi’an, China;2. School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China;3. School of Software, Xidian University, Xi’an, China;1. School of Information Science & Engineering, Central South University, Changsha, Hunan 410083, China;2. Research Organization of Information and Systems, 4-3-13 Toranomon, Minato-ku, Tokyo 105-0001, Japan;3. The Institute of Statistical Mathematics, 10-3 Midori-cho, Tachikawa, Tokyo 190-8562, Japan;4. Hunan Province Higher Education Key Laboratory of Power System Safety Operation and Control, Changsha University of Science and Technology, Changsha, Hunan 410004, China;5. School of Business, Central South University, Changsha, Hunan 410083, China;6. Collaborative Innovation Center of Resource-Conserving & Environment-Friendly Society and Ecological Civilization, Changsha, Hunan 410083, China;1. Samsung Electronics, Republic of Korea;2. Intelligent Systems and Emotional Engineering (ISEE) Laboratory, Department of Mechatronics Engineering, Chungnam National University, Republic of Korea;1. International Islamic University, Islamabad, Pakistan;2. Gwangju Institute of Science and Technology, South Korea |
| |
| Abstract: | Search-based software testing promises the ability to generate and evaluate large numbers of test cases at minimal cost. From an industrial perspective, this could enable an increase in product quality without a matching increase in the time and effort required to do so.Search-based software testing, however, is a set of quite complex techniques and approaches that do not immediately translate into a process for use with most companies.For example, even if engineers receive the proper education and training in these new approaches, it can be hard to develop a general fitness function that covers all contingencies. Furthermore, in industrial practice, the knowledge and experience of domain specialists are often key for effective testing and thus for the overall quality of the final software system. But it is not clear how such domain expertise can be utilized in a search-based system.This paper presents an interactive search-based software testing (ISBST) system designed to operate in an industrial setting and with the explicit aim of requiring only limited expertise in software testing. It uses SBST to search for test cases for an industrial software module, while also allowing domain specialists to use their experience and intuition to interactively guide the search.In addition to presenting the system, this paper reports on an evaluation of the system in a company developing a framework for embedded software controllers. A sequence of workshops provided regular feedback and validation for the design and improvement of the ISBST system. Once developed, the ISBST system was evaluated by four electrical and system engineers from the company (the ‘domain specialists’ in this context) used the system to develop test cases for a commonly used controller module. As well as evaluating the utility of the ISBST system, the study generated interaction data that were used in subsequent laboratory experimentation to validate the underlying search-based algorithm in the presence of realistic, but repeatable, interactions.The results validate the importance that automated software testing tools in general, and search-based tools, in particular, can leverage input from domain specialists while generating tests. Furthermore, the evaluation highlighted benefits of using such an approach to explore areas that the current testing practices do not cover or cover insufficiently. |
| |
| Keywords: | Search-based software testing Interactive search-based software engineering User centered Embedded software Industrial experience |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
