基于相似性的CITCP强化学习奖励策略

在面向持续集成测试用例优先排序(continuous integration test case prioritization,CITCP)的强化学习方法中,智能体通过对测试用例实施奖励从而调整测试用例优先排序策略以适应后续集成测试,可以满足持续集成测试频繁迭代和快速反馈的需求.智能体通常只奖励执行失效测试用例,但实际工业程序持续集成测试具有集成高频繁但测试低失效的特点,对CITCP的实际应用提出新的挑战.测试低失效,即稀少的执行失效测试用例数量,会导致强化学习中奖励对象稀少,引发强化学习的稀疏奖励问题.本文研究一种强化学习奖励对象选择策略,在奖励执行失效测试用例的基础上,通过选择与执行失效测试用例相似的执行通过测试用例实施奖励,从而增加奖励对象,以解决奖励稀疏问题.研究具体包括,设计了一种测试用例历史执行信息序列和执行时间特征向量表示的相似性度量方法,并基于相似性度量选择与执行失效测试用例集相似的执行通过测试用例集实施奖励.在6个工业数据集上开展了实验研究,结果表明基于相似性的奖励对象选择策略通过增加有效奖励对象解决了稀疏奖励问题,并进一步提高了基于强化学习的持续集成测试用例优先排序质量.

Similarity-based Reward Strategy of Reinforcement Learning in CITCP

In the reinforcement learning method for the continuous integration test case prioritization (CITCP), the agent rewards the test cases to realize the adjustment of test case prioritization strategy, and thus they can meet the needs of frequent iteration and rapid feedback in continuous integration testing. The agent usually only rewards the failure test cases. However, in the actual industrial processes, the continuous integration testing features high-frequency integration and low-failure-rate tests, which poses a new challenge to the actual application of CITCP. Low-failure-rate tests can be understood as a sparse number of failure test cases, which can lead to the sparsity of reward objects in reinforcement learning and bring about the sparse reward problem. In this study, a reward object selection strategy is proposed to solve the sparse reward problem. With the failure test cases rewarded, passing test cases similar to failure test cases are selected to be rewarded, and thus the number of reward objects increases. Specifically, the similarity measure method for test cases is designed with the feature vector representation of historical execution information sequences and duration time. Then, the passing test cases similar to the failure test cases are selected to be rewarded through the similarity measure. The experiments are conducted in six industrial data sets, and the results show that the similarity-based reward object selection strategy can effectively solve the sparse reward problem by increasing the reward objects and further improve the quality of reinforcement learning-based CITCP.