BlackHorse: creating smart test cases from brittle recorded tests

Testing software that has a GUI is difficult. Manual testing is costly and error-prone, but recorded test cases frequently “break” due to changes in the GUI. Test cases intended to test business logic must therefore be converted to a less “brittle” form to lengthen their useful lifespan. In this paper, we describe BlackHorse, an approach to doing this that converts a recorded test case to Java code that bypasses the GUI. The approach was implemented within the testing environment of Research In Motion. We describe the design of the toolset and discuss lessons learned during the course of the project.